Are you sure?
This action might not be possible to undo. Are you sure you want to continue?
Course Organizers:
Jonas Gomes
Luiz Velho
Instituto de Matem´atica Pura e Aplicada, IMPA
Rio de Janeiro, Brazil
Course Notes – SIGGRAPH 99
Course Abstract
Most real world applications can be reduced to the problem of function
representation and reconstruction. These two problems are closely re
lated to synthesis and analysis of functions. The Fourier transform is the
classical tool used to solve them. More recently, wavelets have entered
the arena providing more robust and ﬂexible solutions to discretize and
reconstruct functions.
Starting from Fourier analysis, the course guides the audience to ac
quire an understanding of the basic ideas and techniques behind the
wavelets. We start by introducing the basic concepts of function spaces
and operators, both from the continuous and discrete viewpoints. We
introduce the Fourier and Window Fourier Transform, the classical tools
for function analysis in the frequency domain, and we use them as a guide
to arrive at the Wavelet transform. The fundamental aspects multires
olution representation and its importance to function discretization and
to the construction of wavelets is also discussed.
Emphasis will be given on ideas and intuition, avoiding the heavy
computations which are usually involved in the study of wavelets. The
attendees should have a basic knowledge of Linear Algebra, Calculus,
and some familiarity with Complex Analysis. Basic knowledge of signal
and image processing would be desirable.
These notes originated from a set of notes in Portuguese that we wrote
for a wavelet course on the Brazilian Mathematical Colloquium in 1997
at IMPA, Rio de Janeiro. We wish to thank Siome Goldenstein who
collaborated with us to produce the portuguese notes.
Course Timeline
1. Fundamentals of Fourier Analysis
Duration: 8:30 – 9:15 (45 minutes)
Speaker: Jonas Gomes
2. From TimeFrequency Localization to Wavelets
Duration 9:15 – 10:00 (45 minutes)
Speaker: Jonas Gomes
Coﬀee Break
Duration: 10:00 – 10:15 (15 minutes)
3. Filter Banks and Wavelets
Duration 10:15 – 11:00 (45 minutes)
Speaker: Luiz Velho
4. Wavelet Design
Duration 11:00 – 12:00 (60 minutes)
Speaker: Luiz Velho
Course Organizers
Jonas Gomes
IMPA  Instituto de Matematica Pura e Aplicada
Estrada Dona Castorina, 110
22460320, Rio de Janeiro, RJ, Brazil
Email: jonas@visgraf.impa.br
Short Biography. Jonas Gomes is a full professor at The Institute of Pure and Applied
Mathematics, IMPA, in Rio de Janeiro. He took a phd in Mathematics in 1984, and has
been working with computer graphics since then. He was the manager of the R&D team at
Globo Television Network from 1984 to 1988. He created the computer graphics research lab
at IMPA in 1989. This lab develops research and supports a graduate program in graphics.
He is the author of several books in graphics, and has published several papers. He has
already organized three SIGGRAPH courses: “Modeling in Graphics” in SIGGRAPH ’93,
and “warping and morphing of graphical objects” in SIGGRAPH ’95 and in SIGGRAPH
’97. His current research interests include the Mathematics of computer graphics, Modeling,
Visualization, Image Processing and Multimedia.
Luiz Velho
IMPA  Instituto de Matematica Pura e Aplicada
Estrada Dona Castorina, 110
22460320, Rio de Janeiro, RJ, Brazil
Email: lvelho@visgraf.impa.br
Short Biography. Luiz Velho is an Associate Researcher at IMPA  Instituto de Matematica
Pura e Aplicada. He received a BE in Industrial Design from ESDI  Universidade do Rio de
Janeiro in 1979, a MS in Computer Graphics from the Massachusetts Institute of Technology,
Media Laboratory in 1985, and a Ph.D. in Computer Science in 1994 from the University
of Toronto. His experience in computer graphics spans the ﬁelds of modeling, rendering,
imaging and animation. During 1982 he was a visiting researcher at the National Film Board
of Canada. From 1985 to 1987 he was a Systems Engineer at the Fantastic Animation Machine
in New York, where he developed the company’s 3D visualization system. From 1987 to 1991
he was a Principal Engineer at Globo TV Network in Brazil, where he created special eﬀects
and visual simulation systems. In 1994 he was a visiting professor at the Courant Institute of
Mathematical Sciences, New York University. He is the author of several books in graphics,
and has published several papers in this area. He has been a speaker in two SIGGRAPH
courses: “modeling in graphics” in SIGGRAPH ’93, and “warping and morphing of graphical
objects” in SIGGRAPH ’94. His current research interests include theoretical foundations of
computer graphics, physicallybased methods, wavelets, modeling with implicit objects and
volume visualization.
Contents
1 Introduction 1
1.1 Computational Mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Abstraction Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Relation Between the Abstraction Levels . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Functions and Computational Mathematics . . . . . . . . . . . . . . . . . . . . 6
1.3.1 Representation and Reconstruction of Functions . . . . . . . . . . . . . 6
1.3.2 Speciﬁcation of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 What is the Relation with Graphics? . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4.1 Description of Graphical Objects . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Where do Wavelets Fit? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5.1 Function Representation Using Wavelets . . . . . . . . . . . . . . . . . . 8
1.5.2 Multiresolution Representation . . . . . . . . . . . . . . . . . . . . . . . 9
1.6 About these Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.7 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Function Representation and Reconstruction 11
2.1 Representing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 The Representation Operator . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Basis Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Complete Orthonormal Representation . . . . . . . . . . . . . . . . . . . 13
2.3 Representation by Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Riesz Basis Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5 Representation by Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.6 Galerkin Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.7 Reconstruction, Point Sampling and Interpolation . . . . . . . . . . . . . . . . 17
2.7.1 Piecewise Constant Reconstruction . . . . . . . . . . . . . . . . . . . . . 18
2.7.2 Piecewise Linear Reconstruction . . . . . . . . . . . . . . . . . . . . . . 18
2.8 Multiresolution Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.9 Representation by Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.10 Redundancy in the Representation . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.11 Wavelets and Function Representation . . . . . . . . . . . . . . . . . . . . . . . 22
2.12 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
v
vi CONTENTS
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3 The Fourier Transform 25
3.1 Analyzing Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.1 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.2 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.3 Spatial and Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . 29
3.2 A Pause to Think . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.3 Frequency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.4 Fourier Transform and Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.5 Fourier Transform and Function Representation . . . . . . . . . . . . . . . . . . 34
3.5.1 Fourier Transform and Point Sampling . . . . . . . . . . . . . . . . . . . 35
3.5.2 The Theorem of ShannonWhittaker . . . . . . . . . . . . . . . . . . . . 36
3.6 Point Sampling and Representation by Projection . . . . . . . . . . . . . . . . . 37
3.7 Point Sampling and Representation Coeﬃcients . . . . . . . . . . . . . . . . . . 38
3.8 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4 Windowed Fourier Transform 43
4.1 A Walk in The Physical Universe . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2 The Windowed Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2.1 Invertibility of
˜
f(t, ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2.2 Image of the Windowed Fourier Transform . . . . . . . . . . . . . . . . 45
4.2.3 WFT and Function Representation . . . . . . . . . . . . . . . . . . . . . 46
4.3 Timefrequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3.1 The Uncertainty Principle . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.4 Atomic Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.5 WFT and Atomic Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.6 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5 The Wavelet Transform 57
5.1 The Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.1.1 Inverse of the Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . 58
5.1.2 Image of the Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . 59
5.2 Filtering and the Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . 60
5.3 The Discrete Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.3.1 Function Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.4 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
CONTENTS vii
6 Multiresolution Representation 69
6.1 The Concept of Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2 Scale Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.2.1 A Remark About Notation . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.2.2 Multiresolution Representation . . . . . . . . . . . . . . . . . . . . . . . 72
6.3 A Pause to Think . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
6.4 Multiresolution Representation and Wavelets . . . . . . . . . . . . . . . . . . . 75
6.5 A Pause... to See the Wavescape . . . . . . . . . . . . . . . . . . . . . . . . . . 78
6.6 Two Scale Relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.7 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7 The Fast Wavelet Transform 83
7.1 Multiresolution Representation and Recursion . . . . . . . . . . . . . . . . . . . 83
7.2 TwoScale Relations and Inner Products . . . . . . . . . . . . . . . . . . . . . . 85
7.3 Wavelet Decomposition and Reconstruction . . . . . . . . . . . . . . . . . . . . 86
7.3.1 Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.3.2 Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
7.4 The Fast Wavelet Transform Algorithm . . . . . . . . . . . . . . . . . . . . . . 89
7.4.1 Forward Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.4.2 Inverse Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.4.3 Complexity Analysis of the Algorithm . . . . . . . . . . . . . . . . . . . 91
7.5 Boundary Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.6 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
8 Filter Banks and Multiresolution 95
8.1 TwoChannel Filter Banks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
8.2 Filter Banks and Multiresolution Representation . . . . . . . . . . . . . . . . . 97
8.3 Discrete Multiresolution Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 99
8.3.1 Pause to Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
8.4 Reconstruction Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
8.5 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
8.6 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
9 Constructing Wavelets 107
9.1 Wavelets in the Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . 107
9.1.1 The Relations of
ˆ
φ with m
0
. . . . . . . . . . . . . . . . . . . . . . . . . 107
9.1.2 The Relations of
ˆ
ψ with m
1
. . . . . . . . . . . . . . . . . . . . . . . . . 108
9.1.3 Characterization of m
0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
9.1.4 Characterization of m
1
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
9.2 Orthonormalization Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
9.3 A Recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
viii CONTENTS
9.4 Piecewise Linear Multiresolution . . . . . . . . . . . . . . . . . . . . . . . . . . 113
9.5 Shannon Multiresolution Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 114
9.6 Where to Go Now? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
9.7 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
10 Wavelet Design 119
10.1 Synthesizing Wavelets from Filters . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.1.1 Conjugate Mirror Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.1.2 Conditions for m
0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
10.1.3 Strategy for Computing m
0
. . . . . . . . . . . . . . . . . . . . . . . . . 120
10.1.4 Analysis of P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
10.1.5 Factorization of P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.1.6 Example (Haar Wavelet) . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
10.2 Properties of Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
10.2.1 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
10.2.2 Support of φ and ψ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
10.2.3 Vanishing Moments and Polynomial Reproduction . . . . . . . . . . . . 127
10.2.4 Regularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
10.2.5 Symmetry or Linear Phase . . . . . . . . . . . . . . . . . . . . . . . . . 128
10.2.6 Other Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
10.3 Classes of Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
10.3.1 Orthogonal Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
10.3.2 Biorthogonal Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
10.4 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
11 Orthogonal Wavelets 133
11.1 The Polynomial P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
11.1.1 P as a Product Polynomial . . . . . . . . . . . . . . . . . . . . . . . . . 133
11.1.2 P and the Halfband Property . . . . . . . . . . . . . . . . . . . . . . . . 134
11.1.3 The Expression of P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
11.1.4 The Factorization of P . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
11.1.5 Analysis of P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
11.2 Examples of Orthogonal Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . 139
11.2.1 Daubechies Extremal Phase Wavelets . . . . . . . . . . . . . . . . . . . 140
11.2.2 Minimal Phase Orthogonal Wavelets . . . . . . . . . . . . . . . . . . . . 141
11.2.3 Coiﬂets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
11.3 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
CONTENTS ix
12 Biorthogonal Wavelets 147
12.1 Biorthogonal Multiresolution Analysis and Filters . . . . . . . . . . . . . . . . . 147
12.1.1 Biorthogonal Basis Functions . . . . . . . . . . . . . . . . . . . . . . . . 147
12.1.2 Biorthogonality and Filters . . . . . . . . . . . . . . . . . . . . . . . . . 149
12.1.3 Fast Biorthogonal Wavelet Transform . . . . . . . . . . . . . . . . . . . 150
12.2 Filter Design Framework for Biorthogonal Wavelets . . . . . . . . . . . . . . . . 151
12.2.1 Perfect Reconstruction Filter Banks . . . . . . . . . . . . . . . . . . . . 151
12.2.2 Conjugate Quadrature Filters . . . . . . . . . . . . . . . . . . . . . . . . 153
12.2.3 The Polynomial P and Wavelet Design . . . . . . . . . . . . . . . . . . . 154
12.2.4 Factorization of P for Biorthogonal Filters . . . . . . . . . . . . . . . . 154
12.3 Symmetric Biorthogonal Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . 160
12.3.1 BSpline Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
12.3.2 Wavelets with Closer Support Width . . . . . . . . . . . . . . . . . . . . 162
12.3.3 Biorthogonal Bases Closer to Orthogonal Bases . . . . . . . . . . . . . . 163
12.4 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
13 Directions and Guidelines 165
13.1 History and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
13.2 A Look Back . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
13.3 Extending the Basic Wavelet Framework . . . . . . . . . . . . . . . . . . . . . . 166
13.3.1 Studying Functions on other Domains . . . . . . . . . . . . . . . . . . . 167
13.3.2 Deﬁning other TimeFrequency Decompositions . . . . . . . . . . . . . . 167
13.3.3 Solving Mathematical Problems . . . . . . . . . . . . . . . . . . . . . . . 168
13.4 Applications of Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
13.5 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
A Systems and Filters 173
A.1 Systems and Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
A.1.1 Spatial Invariant Linear Systems . . . . . . . . . . . . . . . . . . . . . . 174
A.1.2 Other Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
A.2 Discretization of Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
A.2.1 Discrete Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
A.2.2 Discrete Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
A.3 Upsampling and Downsampling Operators . . . . . . . . . . . . . . . . . . . . . 182
A.4 Filter Banks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
A.5 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
B The Z Transform 185
B.1 The Z Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
B.1.1 Some Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
B.1.2 Transfer Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
x CONTENTS
B.1.3 The Variable z and Frequency . . . . . . . . . . . . . . . . . . . . . . . 189
B.2 Subsampling Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
B.2.1 Downsampling in the Frequency Domain . . . . . . . . . . . . . . . . . . 189
B.2.2 Upsampling in the Frequency Domain . . . . . . . . . . . . . . . . . . . 191
B.2.3 Upsampling after Downsampling . . . . . . . . . . . . . . . . . . . . . . 191
B.3 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
List of Figures
1.1 Abstraction levels for computational mathematics. . . . . . . . . . . . . . . . . 2
1.2 Spectral distribution function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1 Galerkin representation of a function f . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 (a) Graph of the synthesis function; (b) Reconstruction. . . . . . . . . . . . . . 18
2.3 (a) Graph of the synthesis function. (b) Reconstruction. . . . . . . . . . . . . . 19
2.4 Representation operator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1 Sine function with frequencies 2, 4 and 8. . . . . . . . . . . . . . . . . . . . . . 26
3.2 A signal and its Fourier transform. . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 Fourier transform of two impulses δ(t −0.05) e δ(t −0.42). . . . . . . . . . . . 31
3.4 Transfer function of an ideal lowpass ﬁlter. . . . . . . . . . . . . . . . . . . . . 33
3.5 Transfer function of an ideal highpass ﬁlter. . . . . . . . . . . . . . . . . . . . . 34
3.6 Transfer function of an ideal bandpass ﬁlter. . . . . . . . . . . . . . . . . . . . 34
3.7 Transfer function of an ideal bandstop ﬁlter. . . . . . . . . . . . . . . . . . . . 34
3.8 Sampling and the Fourier spectrum. . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1 Modulating function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2 Information cells. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.3 Atoms in the timefrequency domain. . . . . . . . . . . . . . . . . . . . . . . . . 48
4.4 (a) Point sampling. (b) Fourier sampling. . . . . . . . . . . . . . . . . . . . . . 49
4.5 Uniform lattice in timefrequency domain. . . . . . . . . . . . . . . . . . . . . . 50
4.6 Decompositions of the timefrequency domain using the DWFT. . . . . . . . . 51
4.7 Signal and its Fourier transform. . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.8 Hamming window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.9 Atomic decomposition using windowed Fourier transform. . . . . . . . . . . . . 53
4.10 (a) Quadratic chirp signal; (b) Ideal atomic decomposition of the signal. . . . . 54
4.11 Timefrequency decomposition of chirp signal using DWFT. . . . . . . . . . . . 54
5.1 Scales of a function: (a) s < 1; (b) s = 1; (c) s > 1. . . . . . . . . . . . . . . . . 58
5.2 Graph of a wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.3 Fourier transform of a wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.4 Graph of the sombrero wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.5 Signal f to be analyzed(Kaiser, 1994). . . . . . . . . . . . . . . . . . . . . . . . 63
5.6 The wavelet transform(Kaiser, 1994). . . . . . . . . . . . . . . . . . . . . . . . . 64
xi
xii LIST OF FIGURES
5.7 (a) Timescale lattice.(b) Timefrequency lattice. . . . . . . . . . . . . . . . . . 66
5.8 Timefrequency decomposition using wavelets. . . . . . . . . . . . . . . . . . . . 66
5.9 Haar wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.1 Lowpass ﬁlter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.2 Scaling of f by an scale factor of 2. . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.3 Spectrum of the scaling function. . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.4 Scale approximations of a function(Daubechies, 1992). . . . . . . . . . . . . . . 74
6.5 Frequency band between V
j
and V
j−1
. . . . . . . . . . . . . . . . . . . . . . . . 75
6.6 Frequency bands between V
j
and V
Jo−j
. . . . . . . . . . . . . . . . . . . . . . . 76
6.7 Image decomposition on the spaces V
j
e W
j
. . . . . . . . . . . . . . . . . . . . . 80
7.1 Wavelet decomposition of a function f. . . . . . . . . . . . . . . . . . . . . . . . 84
7.2 Wavelet reconstruction process of a function f . . . . . . . . . . . . . . . . . . 85
7.3 Boundary regions for convolution between ﬁnite sequences. . . . . . . . . . . . 91
7.4 Options for boundary computation (a) Extending with zeros; (b) Periodization;
(c) Reﬂection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
8.1 Diagram of a TwoChannel Analysis Filter Bank. . . . . . . . . . . . . . . . . . 95
8.2 TwoChannel Synthesis Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . 96
8.3 TwoChannel Analysis and Synthesis Filter Bank . . . . . . . . . . . . . . . . . 96
9.1 Double scale relation of the Haar scaling function. . . . . . . . . . . . . . . . . 111
9.2 Haar wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
9.3 Piecewise linear scaling function. . . . . . . . . . . . . . . . . . . . . . . . . . . 114
9.4 (a)Scaling function of the piecewise linear multiresolution; (b) Wavelet of the
piecewise linear multiresolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
9.5 Impulse response of the ideal reconstruction ﬁlter. . . . . . . . . . . . . . . . . 115
9.6 Band decomposition using Shannon multiresolution representation. . . . . . . . 116
9.7 Ideal bandpass ﬁlter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
9.8 Shannon wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
10.1 Halfband Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.2 Haar ﬁlter m
0
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
10.3 Magnitude of the frequency response of
ˆ
φ(ω) for the Haar function . . . . . . . 126
11.1 P . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
11.2 Limit curves for the roots of B
n
(y) and [Q
n
(z)[
2
(from (Strang & Shen, 1996)) 139
11.3 Roots of P(z) with n = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
11.4 Daubechies (D2) Scaling Funtion and Wavelet . . . . . . . . . . . . . . . . . . . 141
11.5 Fourier Transform of (D2) Scaling Funtion and Wavelet . . . . . . . . . . . . . 141
11.6 Least Assymmetric Scaling Funtion and Wavelet, with n = 4 . . . . . . . . . . 143
11.7 Coiﬄets of order 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
12.1 Biorthogonal Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
LIST OF FIGURES xiii
12.2 Biorthogonal box scaling functions (a) and wavelets (b). . . . . . . . . . . . . . 156
12.3 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . . . . . . . 156
12.4 Biorthogonal Hat scaling functions (a) and wavelets (b). . . . . . . . . . . . . . 157
12.5 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . . . . . . . 157
12.6 Biorthogonal quadratic Bspline scaling functions (a) and wavelets (b). . . . . . 158
12.7 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . . . . . . . 158
12.8 D2 Daubechies scaling functions (a) and wavelets (b). . . . . . . . . . . . . . . 159
12.9 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . . . . . . . 159
12.10Biorthogonal Bspline Wavelets of order 2 (from (Daubechies, 1992)). . . . . . . 161
12.11Biorthogonal Wavelets with Similar Support,(from (Daubechies, 1992)) . . . . . 162
12.12Biorthogonal Wavelets Close to Orthogonal,(from (Daubechies, 1992)) . . . . . 164
A.1 Basic System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
A.2 Delay Operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
A.3 Example of a Filter Bank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
B.1 Modulus of the frequency response of a ﬁlter. . . . . . . . . . . . . . . . . . . . 189
B.2 Downsampling Operator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
B.3 Upsampling Operator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
xiv LIST OF FIGURES
Chapter 1
Introduction
In this chapter we give a general overview of the area of Computational Mathematics and
computer graphics, introducing the concepts which motivate the study of wavelets.
1.1 Computational Mathematics
Mathematical modeling studies phenomena of the physical universe using mathematical con
cepts. These concepts allow us to abstract from the physical problems and use the mathe
matical tools to obtain a better understanding of the diﬀerent phenomena of our universe.
The advance of computing technology (both hardware and software) has enabled the use of
mathematical modeling to make simulations on the computer (synthetic simulations). These
simulations on the computer allow for a great ﬂexibility: We can advance in time, accelerate
processes, introduce local and global external factors, and change the diﬀerent parameters on
the mathematical models used. These conditions of simulation in general are very diﬃcult to
be attained in real simulations.
Computational Mathematics is the combination of mathematical modeling with simu
lations on the computer. Computational mathematics represents a return to the birth of
mathematics, where mathematical problems were directly related to the solutions of practical
problems of our everyday life.
In this context, computer graphics provides diﬀerent techniques to visualize the results
from the simulation. This visualization enables a direct interaction of the users with the
diﬀerent parameters of the simulation and also enables them to collect valuable qualitative
information about the simulation process.
1.1.1 Abstraction Levels
A useful way to organize the ideas discussed in the previous section, which will be used in the
book, consists in structuring the problem of computational mathematics using abstraction
levels. These abstraction levels encapsulate the diﬀerent phases of the simulation processing
giving us a better understanding of the whole process. We will use the paradigms of the
four universes. This paradigm uses four levels: Physical universe, Mathematical Universe,
Representation Universe and Implementation Universe (See Figure 1.1).
1
2 CHAPTER 1. INTRODUCTION
Figure 1.1: Abstraction levels for computational mathematics.
Level 1: Physical Universe. In this level the problems are originally posed and correctly
formulated. The elements of these abstraction levels consist of the diﬀerent phenomena from
the physical universe.
Level 2: Mathematical Universe. This level contains tools from diﬀerent areas of math
ematics (Analysis, Geometry, Diﬀerential Equations, Algebra etc.). The process of mathe
matical modeling consists in the association of elements from the physical universe (Level 1)
to elements of the mathematical universe.
Level 3: Representation Universe. To perform synthetic simulations the mathematical
models must be discretized and implemented on the computer. The representation universe
consists of the diﬀerent elements of discrete mathematics and the mathematical methods of
discretization. As part of this universe, we have virtual machines where we can describe
algorithms. The discretization of a problem consists in relating the mathematical model
associated to the problem in the mathematical universe (Level 2) to a discretized model in
the representation universe.
Level 4: Implementation Universe. This abstraction level is considered so as to allow
for a clear separation between the discretization problem in Level 3 and the problems inherent
to implementation. The elements of this universe consist in data structures and programming
languages with a well deﬁned syntax and semantics. The implementation of a problem con
sists in associating discrete structures from the representation universe to elements of the
implementation universe.
1.1. COMPUTATIONAL MATHEMATICS 3
This models of abstraction levels will be used along the book. These abstraction levels
split the problems into four major subproblems and favors an encapsulation of each of the
subproblems: At each level the problem is studied with a clear separation from its intrinsic
issues with issues related to the other abstraction levels.
We should point out that this abstraction paradigm constitutes in a characterization of
computational mathematics. Below will give some examples.
Example 1 (Measurements). Consider the problem of measuring objects in the physical
universe. For a given object we must associate to it a number which represents its length,
area or volume. In order to achieve this we must introduce a standard unit of measure which
is compared with the objects to provide a measurement.
From the point of view of the mathematical universe, to each measure we associate a real
number. Rational numbers correspond to commensurable objects, and irrational numbers
correspond to incommensurable ones.
To represent the measurements, we must look for a discretization of the real numbers. A
widely used choice is given by the ﬂoating point representation. Note that in this representa
tion the set of real numbers is discretized by a ﬁnite set of rational numbers. In particular, this
implies that in the representation universe we do not have the concept of incommensurable
measurements.
An implementation o the real numbers using ﬂoating point arithmetic can be done using
a standard IEEE speciﬁcation. A good reference for these topics is (Higham, 1996).
The above example, although simple constitutes the fundamental problem in the study of
computational mathematics. In particular, we should remark the loss of information when we
pass from the mathematical to the representation universe: incommensurable measures dis
appear. In general the step from the mathematical to the representation universe forces a loss
of information, and this is one of the most delicate problems we have to face in computational
mathematics.
Example 2 (Projectiles). Consider now the well known problem of studying the motion of
a small rigid body which is thrown into the space with a given initial velocity. The problem
consists in studying the trajectory of the body.
In the mathematical universe we will make a simpliﬁcation to model the problem. The
rigid body will be considered to be a particle, that is a point of mass. This abstraction allow
us to neglect the air resistance. Simple calculations well known from elementary physical
courses, allow us to solve the problem of determining the trajectory: a parabola
f(t) = at
2
+bt +c.
To visualize the trajectory in the computer we must obtain a discretization of the parabola.
This discretization can be obtained by taking a uniform partition 0 = t
0
< t
1
< < t
n
= t
of the time interval [0, t]. Using this partition the parabola is represented by a ﬁnite set of
points of the plane:
(t
0
, f(t
0
)), (t
1
, f(t
1
)), . . . , (t
n
, f(t
n
)).
These points constitute samples of the function the deﬁnes the parabola.
4 CHAPTER 1. INTRODUCTION
Finally, data structures to implement the samples are well known and can be easily im
plemented.
Note that in order to plot the parabola we must “reconstruct” it from the samples. In
terpolation methods must be used in the reconstruction process. Again, the representation
process by sampling looses information about the parabola. Depending on the interpolation
method used we will not be able to reconstruct the parabola exactly, but only an approxima
tion of it.
An important remark should be done. The solution to the problem in the above example
was initially attained in the mathematical universe it was an analytical solution providing
the equation of the trajectory. The discretization of the analytical solution was done in order
to visualize it. Another option would be discretizing the problem before solving it by using
discrete mathematical models to formulate the problem (Greenspan, 1981). Which is the best
strategy? Discretize the problem to obtain a solution or compute an analytical solution and
discretize it.
The answer to this problem is not simple and involves diﬀerent issues. Initially we should
remark that in several situations we do not have an analytic solution to the problem in the
mathematical universe. When this happens discretization a priori to determine a solution is of
fundamental importance. This is the case, for instance, in several problems which are modeled
using diﬀerential equations: The discretization allows us the use of numerical methods to solve
an equation where an analytical solution is diﬃcult or even impossible to be achieved.
When the problem has an analytical solution in the mathematical universe we have the
option to compute it before discretization. Nevertheless we should point out that the process
of discretization should not be decorrelated from the formulation of the problem in the physical
universe. In example 2 for instance the parameter of the parabola represents the time, and
we know that the horizontal motion is uniform. Therefore, a uniform discretization of time is
the most adequate to playback the animation of the particle motion.
1.2 Relation Between the Abstraction Levels
We should not forget the importance of establishing for each problem the relation between
the four universes in our abstraction paradigm described above. In its essence these relations
enable us to answer the question: How close a computational simulation is from reality? The
answer to this problem depends on the relation between the diﬀerent abstraction levels:
• Relation between the implementation of the problem, and its discretization;
• Relation between the discretization and the mathematical formulation of the problem;
• Relation between the mathematical model of the problem and the original problem in
the physical universe.
The relationship between the mathematical and the representation universe is of most
importance. We have seen that representation is a loss process, and a very important issue
consists in recovering the continuous model from the discrete representation. This step is
1.2. RELATION BETWEEN THE ABSTRACTION LEVELS 5
Figure 1.2: Spectral distribution function.
called reconstruction. Since the representation is a loss process, reconstruction in general is
not possible to be attained exactly. The example below will make this point clear.
Example 3 (Representation of Color). Color is produced by an electromagnetic radia
tion in the visible band of the spectrum. Therefore from the point of view of the mathematical
universe, a color is determined by its spectral distribution function. This function associates
to each wavelength λ the value of the associated energy (see Figure 1.2).
Therefore, the process if discretizing a color reduces to that of discretizing a function.
What is the most natural technique? A simple solution consists in using point sampling as
we did in the discretization of the parabola in example 2. In this case another question arises:
How many samples should we take?
Understanding this problem in its generality and devise adequate and robust solutions is
a diﬃcult task. In fact this is one topic to be covered in this book. In the case of color, we
can devise a simple solution going back to the physical universe. How does the human eye
processes color? From the theory of YoungHelmholtz the eye discretizes color using three
samples of the spectral distribution function: One sample of low frequency (Red), one sample
of medium frequency (Green) and another sample of high frequency (Blue)
This means that from the perceptual point of view it is enough to use three samples of the
spectral distribution function in order represent color. This means that the representation
of the color space (which is an inﬁnite dimensional function space) is given by the Euclidean
space R
3
. Note that this representation is completely justiﬁed from the point of view of
color perception on the physical universe. It is far away from being a good mathematical
representation.
Exact reconstruction of the original spectral color from the three RGB samples is in general
an impossible task. Nevertheless, in the way we posed the color sampling problem, we do not
need to guarantee an exact reconstruction: We should be able to provide a reconstruction
process in which the reconstructed color is perceptually equal to the color deﬁned by the
original spectral distribution function. This problem is called perceptual color reconstruction.
Perceptual reconstruction of color is the technology behind many color reconstruction devices
nowadays, such as color television: The TV set reconstructs colors which are only perceptually
equal to the original colors captured from the video camera.
6 CHAPTER 1. INTRODUCTION
1.3 Functions and Computational Mathematics
Functions play a distinguished role in computational mathematics. In fact they constitute the
basic tools of mathematical modeling: The relation between magnitudes of the physical world
are, in most cases, described by functions. Notice that the two examples of particle motion
and color reconstruction were reduced to the problem of representation and reconstruction of
functions.
Therefore, a highly relevant problem in computational mathematics turns out to be the
problem of function representation and reconstruction.
1.3.1 Representation and Reconstruction of Functions
The simplest method to represent a function is to use point sampling. In the onedimensional
case a real function of onevariable f : I ⊂ R → R is discretized by taking a partition t
1
<
t
2
< < t
n
of the domain interval I. The representation is given by the vector
f
n
= (f(t
1
), f(t
2
, ), . . . , f(t
n
)) ∈ R
n
.
In this way, the space of real functions deﬁned on the interval is represented by the Euclidean
space R
n
.
Is this a good representation? This question is directly related with the problem of in
formation loss in the representation of a function. A mathematical formulation of the above
question would be: Is point sampling an exact representation? In other words, is it possible
to reconstruct a function f from its representation vector? Note that in the case of point sam
pling a reconstruction technique consists of interpolating the points (t
i
, f(t
i
)), i = 1, . . . , n.
In general, this interpolation process gives only an approximation of the original function.
A ﬁnal word about function reconstruction is in order. Inside the computer any function
is in fact necessarily discretized. When we talk about a continuous function it means that we
are able to evaluate the function at any point of its domain. That is, we have a representation
of the function on the computer, along with a reconstruction technique that enables us to
evaluate the function at any point.
1.3.2 Speciﬁcation of Functions
In general when a function is speciﬁed on the computer the speciﬁcation technique results in
a representation of the function. The ﬁrst step towards doing computation with the function
consists in reconstructing it from its representation.
Therefore representation and reconstruction techniques are related with user interface
issues in a computational system.
1.4 What is the Relation with Graphics?
A broad view of computer graphics would lead us to say that it consists of the area that
manipulates graphical objects. That is, in computer graphics we are interested in creating,
processing, analyzing and visualizing graphical objects.
1.4. WHAT IS THE RELATION WITH GRAPHICS? 7
In order to fully understand this description we must give a precise meaning to the concept
of a graphical object. A simple, and eﬀective deﬁnition was given in (Gomes et al. , 1996): A
graphical object consists of a shape and its attributes. The shape deﬁnes the geometry and
topology of the object, and the attributes describes the diﬀerent properties associated to it,
such as color, material, velocity, density etc.
Mathematically, the shape is characterized as a subset U ⊂ R
n
, and the attributes are
encapsulated into a vector valued function
f : U ⊂ R
n
→R
p
.
The dimension of the set U is the dimension of the graphical object, and R
n
is the embedding
space. The object is denoted by O = (U, f).
This deﬁnition encompasses diﬀerent signals such as video, audio and image, and also
curves, surfaces and volumetric models. According the abstraction levels we have described
the main problems of computer graphics are: description, representation, reconstruction and
implementation of graphical objects. Notice that visualization of graphical objects, an im
portant part of the computer graphics applications, is in fact a reconstruction problem.
Example 4 (Image and Digital Image). The physical model of an image is well repre
sented by a photography. Analyzing a photography we observe a geometric support (the
paper where it is printed), and associated to each point of this support we have a color
information.
Therefore, from the mathematical point of view we can model an image as a function
f : U ⊂ R
2
→ (, where ( is a representation of the color space. Using the RGB color
representation of example 3 we have ( = R
3
.)
We can easily obtain a representation of an image using point sampling. We take partitions
∆X = ¦i∆x ; i ∈ Z, ∆x > 0¦ and ∆Y = ¦j∆y ; j ∈ Z, ∆y > 0¦ of the x and y axis of the
Euclidean plane, and we deﬁne a twodimensional lattice of the plane by taking the cartesian
product ∆X∆Y . This lattice is characterized by the vertices (i∆x, j∆y) which are denoted
simply by (i, j). The image f is represented by the matrix
f(i, j) = f(i∆x, j∆y).
1.4.1 Description of Graphical Objects
By description of a graphical object we mean the deﬁnition of its shape and attributes on the
mathematical universe.
There are basically three diﬀerent methods to describe the shape of a graphical object.
• Parametric description;
• Implicit description;
• Algorithmic description.
8 CHAPTER 1. INTRODUCTION
In the parametric description the object shape is deﬁned as the image of a function f : U ⊂
R
m
→R
n
. If U is an mdimensional subset of R
m
, the graphical object has dimension m.
In the implicit description the object shape U ⊂ R
m
is described as the zero set of a
function F : R
m
→R
k
. More precisely,
U = ¦x ∈ R
m
; F(x) = 0¦.
This set is denoted by F
−1
(0), and it is called the inverse image of 0 by the function f.
In the algorithmic description the shape is described by a function (either implicitly or
parametrically) which is deﬁned using an algorithm in some virtual machine (see (Blum,
1991)). Examples of functions deﬁned algorithmically are found in the description of frac
tals. Other common example in computational mathematics are the objects deﬁned using
diﬀerential equations, where we must use an algorithm to solve the equation.
An important remark is in order. The attributes of a graphical object are described by a
function. Also as discussed above, in general, the shape of the object is described by functions.
Therefore, both the shape and attributes of a graphical object are described by functions. The
study of graphical objects reduces to the problem of function description, representation and
reconstruction.
1.5 Where do Wavelets Fit?
In order to understand the role of the wavelets in the scenario of computational mathematics,
even without understanding what a wavelet is, we must remember that our major concern is
the description, representation and reconstruction of functions.
The diﬀerent uses of wavelets in computational mathematics, and in particular in computer
graphics, are related with two facts:
• Representation and reconstruction of functions;
• Multiresolution representation.
1.5.1 Function Representation Using Wavelets
An eﬃcient process to represent a function consists of decomposing it into simpler functions.
In other words, we must devise a collection ¦g
λ
¦ of functions λ ∈ Ω, where the parameter
space Ω is discrete, in such a way that every function f can be expressed by a sum
f =
¸
λ
c
λ
g
λ
. (1.1)
In this case, the coeﬃcient sequence (c
λ
) constitute the representation of the function
f, f → (f
λ
)
λ∈Ω
. The function f is reconstructed from its representation coeﬃcients using
equation (1.1). Wavelets constitute a powerful tool to construct families g
λ
, and compute the
representation coeﬃcients c
λ
.
1.6. ABOUT THESE NOTES 9
1.5.2 Multiresolution Representation
Our perception of the physical world occurs in diﬀerent scales. In order to recognize a house
it is not necessary to have details about its roof, doors or windows. The identiﬁcation is done
using a macroscopic scale of the house. Diﬀerent information about the house can be attained
by using ﬁner scales, where the measurements of the details has the same order of magnitude
of the scale used.
This multiple scale approach is the desired one when we study functions. A good repre
sentation of a function should enable us to devise representations in diﬀerent scales. This is
achieved by using wavelets.
1.6 About these Notes
These notes describe a guided journey to the wonderland of the wavelets. The starting point
of our journey is the kingdom of Fourier analysis. The notes have an introductory ﬂavor. We
have tried to use mathematics as a language which has the adequate semantics to describe the
wavescape along the trip. Therefore, in order to make the text easier to read we decided to
relax with the mathematical syntax. To cite an example, it is very common to refer to some
“space of functions” without making the concept precise. Inner products and norms on these
spaces will be used. Also, we will refer to linear operators and other operations on function
spaces relaxing about several hyphotesis that would be necessary to conﬁrm the validation of
some assertions.
There are several books that cover these syntax issues in detail on the literature. We have
tried to write a logﬁle of our journey which contains somehow the details without going deep
into the mathematical rigor. We are more interested into the semantics of the diﬀerent facets
of wavelet theory, emphasizing the intuition over the mathematical rigor.
In spite of the huge amount of material about wavelets on the literature, these notes
covers the subject with a certain degree of originality on what concerns the organization of
the topics. From the point of view of the abstraction paradigm of the four universes, we will
cover the role of wavelets on function representation and reconstruction. We will also discuss
some implementation issues along the way.
These notes originated from our need to teach wavelets to students originated from both
mathematical and computer science courses.
1.7 Comments and References
The abstraction levels of the four universes described in this section is based on (Requicha,
1980). A generalization of them along the lines presented on this section was published in
(Gomes & Velho, 1995).
Many good books on wavelets have been published. They emphasize diﬀerent aspects of
the theory and applications. We will certainly use material from these sources, and we will
cite them along our journey.
10 BIBLIOGRAPHY
Bibliography
Blum, L. 1991. Lectures on a Theory of Computation and Complexity over the Reals. 18
Coloquio Brasileiro de Matematica – IMPA.
Gomes, Jonas, & Velho, Luiz. 1995. Abstraction Paradigms for Computer Graphics. The
Visual Computer, 11, 227–239.
Gomes, Jonas, Costa, Bruno, Darsa, Lucia, & Velho, Luiz. 1996. Graphical Objects. The
Visual Computer, 12, 269–282.
Greenspan, Donald. 1981. Computer Oriented Mathematical Physics. International Series in
Nonlinear Mathematics, methods and Applications. Pergamon Press, New York.
Higham, Nicholas J. 1996. Accuracy and Stability of Numerical Algorithms. SIAM Books,
Philadelphia.
Requicha, A. A. G. 1980. Representations for Rigid Solids: Theory Methods, and Systems.
ACM Computing Surveys, 12(December), 437–464.
Chapter 2
Function Representation and
Reconstruction
In the previous chapter we concluded that one of the major problems in computational math
ematics is related to function representation and reconstruction. In this chapter we will give
more details about these two problems in order to motivate the study of wavelets.
2.1 Representing Functions
Functions must be discretized so as to implement them on the computer. Also, when a
function is speciﬁed in the computer the input is a representation of the function. As an
example, numerical methods that solve diﬀerential equations (RungeKutta, ﬁnite diﬀerences,
ﬁnite elements, etc.) compute in fact a representation of the solution.
Associated with each representation technique we must have a reconstruction method.
These two operations enable us to move functions between the mathematical and the repre
sentation universes when necessary. As we will see in this chapter, the reconstruction methods
are directly related with the theory of function interpolation and approximation.
2.1.1 The Representation Operator
We denote by o a space of sequences (c
j
)
j∈Z
of numbers
1
. We admit that o has a norm which
allow us to compute distance between sequences of the space.
A representation of a space of functions T is an operator F : T → o into some space of
sequences. For a given function f ∈ T, its representation R(f) is a sequence
R(f) = (f
j
)
j∈Z
∈ o.
R is called the representation operator. In general it is natural to demand that R preserves
norms. That is
[[R(f)[[ = [[f[[,
1
By numbers we mean a real or complex number
11
12 CHAPTER 2. FUNCTION REPRESENTATION AND RECONSTRUCTION
or that R satisﬁes some stability condition, such as
[[R(f)[[ ≤ C[[f[[.
When R is linear and continuous, we have a linear representation.
The most important examples of representations occurs when the space of functions T is
a subspace of the space L
2
(R), of square integrable functions (ﬁnite energy),
L
2
(R) = ¦f : R →R ;
R
[f(t)[
2
dt < ∞¦,
and the representation space o is the space
2
of the square summable sequences,
2
=
(x
i
)
i∈Z
, ; x
i
∈ C, and
+∞
¸
i=−∞
[x
i
[
2
< ∞
¸
.
When the representation operator is invertible, we can reconstruct f from its representa
tion sequence: f = R
−1
((f
i
)
i∈Z
). In this case, we have an exact representation, also called
ideal representation. A method to compute the inverse operator gives us the reconstruction
equation. We should remark that in general invertibility is a very strong requirement for a
representation operator. In fact weaker conditions such as invertibility on the left suﬃces to
obtain exact representations.
In case the representation is not exact we should look for other techniques to compute
approximate reconstruction of the original function. There are several representation/recons
truction methods on the literature. We will review some of these methods in this chapter.
2.2 Basis Representation
A natural technique to obtain a representation of a space of functions consists in constructing
basis of the space. A set B = ¦e
j
; j ∈ Z¦ is a basis of a function space T if the vectors e
j
are linearly independent, and for each f ∈ T there exists a sequence (α
j
)
j∈Z
of numbers such
that
f =
¸
j∈Z
α
j
e
j
. (2.1)
The above equality means the convergence of partial sums of the series in the norm of the
space T.
lim
n→∞
[[f −
n
¸
j=−n
α
j
e
j
[[ = 0.
We deﬁne the representation operator by
R(f) = (α
j
)
j∈Z
.
Equation (2.1) reconstructs the function f from the representation sequence (α
j
).
2.3. REPRESENTATION BY FRAMES 13
We must impose additional hypothesis on the representation basis in order to guarantee
unicity of the representation. A particular case of great importance (in fact the only one we
will be using in these notes) occurs when the space of functions has an inner product and it
is complete in the norm induced by this inner product. These spaces, called Hilbert spaces,
possess special basis as we will describe below.
2.2.1 Complete Orthonormal Representation
A collection of functions ¦ϕ
n
; n ∈ Z¦ on a separable Hilbert space H is a complete orthonormal
set if the conditions below are satisﬁed:
1. Orthogonality: 'ϕ
m
, ϕ
n
` = 0 if n = m;
2. Normalization: [[ϕ
n
[[ = 1 for each n ∈ Z;
3. Completeness: For all f ∈ H, N > 0, and any ε > 0,
[[ f −
N
¸
k=−N
'f, ϕ
k
`ϕ
k
[[ < ε.
The third condition says that linear combinations of functions from the set can be used to
approximate arbitrary functions from the space. Complete orthonormal sets are also called
orthonormal basis of the Hilbert spaces.
An orthonormal basis ¦ϕ
j
¦ deﬁne a representation operator R: H →
2
, R(f) = (f
j
) =
('f, ϕ
j
`). which is invertible. Therefore, the representation is exact. The reconstruction of
the original signal is given by
f =
+∞
¸
k=−∞
'f, ϕ
k
`ϕ
k
.
It is easy to see that the orthogonality condition implies that the elements ϕ
n
are linearly
independent. This implies in particular that the representation sequence ('f, ϕ
k
)` is uniquely
determined by f. This representation preserves the norm. That is
[[R(f)[[
2
=
¸
k∈Z
'f, ϕ
k
`
2
= [[f[[
2
. (2.2)
This expression is called Plancherel equation.
2.3 Representation by Frames
The basic idea of representing functions on a basis, consists in decomposing it using a countable
set of simpler functions.
The existence of a complete orthonormal set, and its construction is in general a very
diﬃcult task. On the other hand orthonormal representations are too much restrictive and
rigid. Therefore, it is important to obtain collections of functions ¦ϕ
n
; n ∈ Z¦ which do not
14 CHAPTER 2. FUNCTION REPRESENTATION AND RECONSTRUCTION
constitute necessarily an orthonormal set or are not linearly independent, but can be used to
deﬁne a representation operator. One such collection is constituted by the frames.
A collection of functions ¦ϕ
n
; n ∈ Z¦ is a frame if there exists constants A and B satisfying
0 < A ≤ B < +∞, such that for all f ∈ H, we have
A[[f[[
2
≤
∞
¸
n=−∞
['f, ϕ
n
`[
2
≤ B[[f[[
2
. (2.3)
The constants A and B are called frame bounds. When A = B we say that the frame is
tight. From the Plancherel formula (2.2) it follows that every orthonormal set is a tight frame
with A = B = 1. Nevertheless, there exists tight frames which are not orthonormal basis.
The following statement is true:
Theorem 1. If B = ¦ϕ
n
; n ∈ Z¦ is a tight frame with A = 1 e [[ϕ
n
[[ = 1, ∀n, then B is an
orthonormal basis.
If a frame is tight, it follows from (2.3) that
¸
j∈Z
['f, ϕ
j
`[
2
= A[[f[[
2
.
Using the polarization identity, we obtain
A'f, g` =
¸
j∈Z
'f, ϕ
j
`'ϕ
j
, g`,
that is,
f = A
−1
¸
j∈Z
'f, ϕ
j
`ϕ
j
.
The above expression (although deduced in the weak sense) shows how we can obtain
approximations of a function f using frames. In fact, it motivates us to deﬁne a representation
operator R analogous to what we did for orthonormal basis:
R(f) = (f
j
)
j∈Z
, where f
j
= 'f, ϕ
j
`.
We should remark that this operator in general is not invertible. Nevertheless it is possible
to reconstruct the signal from its representation R(f), as we will show below.
From equation (2.3) it follows that the operator R is bounded: [[Rf[[
2
≤ B[[f[[
2
. The
adjoint operator R
∗
de R is easy to obtain:
'R
∗
u, f` = 'u, Ru` =
¸
j∈Z
u
j
'f, ϕ
j
` =
¸
j∈Z
u
j
'ϕ
j
, f`,
Therefore
R
∗
u =
¸
j∈Z
u
j
ϕ
j
.
2.4. RIESZ BASIS REPRESENTATION 15
(It can be shown the convergence is also true on norm). By the deﬁnition of R we have
¸
j∈Z
['f, ϕ
j
`[
2
= [[Ff[[
2
= 'F
∗
Ff, f`.
On the other hand, since [[F
∗
[[ = [[F[[, we have [[F
∗
u[[ ≤ B
1/2
[[u[[. We conclude that
AI ≤ F
∗
F ≤ BI.
In particular, it follows that the operator F
∗
F : T →T is invertible.
The above results allow us to obtain an expression to reconstruct the function f from its
representation R(f). We will state the result without proof: Applying the operator (F
∗
F)
−1
to the elements ϕ
j
of the frame, we obtain a family of functions ˜ ϕ
j
,
˜ ϕ
j
= (F
∗
F)
−1
ϕ
j
.
This family constitutes a frame with bounds given by
B
−1
[[f[[
2
≤
¸
j∈Z
['f, ˜ ϕ
j
`[
2
≤ A
−1
[[f[[
2
.
Therefore we can associate to the frame ¦ ˜ ϕ
j
¦ a representation operator
˜
R: T →
2
(Z), deﬁned
in the usual way (
˜
Rf)
j
= 'f, ˜ ϕ
j
`. The frame ¦ ˜ ϕ
j
¦ is called reciprocal frame of the frame
¦(ϕ
j
)
j∈Z
¦. It is easy to see that the reciprocal frame of ¦ ˜ ϕ
j
¦ is the original frame ¦(ϕ
j
)¦.
The following identity can also be proved for all functions f ∈ T:
f =
¸
j∈Z
'f, ϕ
j
` ˜ ϕ
j
=
¸
j∈Z
'f, ˜ ϕ
j
`ϕ
j
. (2.4)
The ﬁrst equality gives us a reconstruction equation of the function f from its representation
sequence R(f) = ('f, ϕ
j
`)
l∈Z
. Note that the second equation in (2.4) gives a method to
describe a function f as a superposition of the elements of the original frame. Because of this
equation the reciprocal frame is also called dual frame.
2.4 Riesz Basis Representation
If a collection B = ¦ϕ
n
; n ∈ Z¦ is a frame, and the functions ϕ
j
are linearly independent, we
have a Riesz basis. Therefore if ¦e
n
¦ is a Riesz basis, for any f ∈ H we have
A[[f[[
2
≤
¸
n
['f, e
n
`[
2
≤ B[[f[[
2
.
For each function f ∈ L
2
(R), we deﬁne its representation
¸
k
'f, e
k
`e
k
,
16 CHAPTER 2. FUNCTION REPRESENTATION AND RECONSTRUCTION
on the Riesz basis ¦e
k
¦.
If L is a continuous linear operator with a continuous inverse, then L maps any orthonormal
basis of a Hilbert space onto a Riesz basis. Moreover, any Riesz basis can be obtained using
this method.
Also, it can be proved that from a Riesz basis ¦e
n
¦ of a Hilbert space H, we can construct
and orthonormal basis ¦˜ e
n
¦ of H. A proof of this fact can be found in (Daubechies, 1992),
page 139.
From the above remarks, we see that Riesz bases constitute the basis of a Hilbert space
that are closest to complete orthonormal sets.
2.5 Representation by Projection
For a given closed subspace V of a Hilbert Space T, the representation by projection consists
in taking a representation of a function f ∈ T as its unique orthogonal projection onto V (see
Figure 2.1).
If ¦ϕ
k
¦ is an orthonormal basis of V , than a representation of f is given by
R(f) = Proj
V
(f) =
¸
k
'f, ϕ
k
`ϕ
k
.
This representation is not exact, unless we have V = T. The best reconstruction we can have
consists in ﬁnding the unique vector of V which is closest to f. This vector can be computed
using optimization techniques.
An important particular case occurs when we have a family of subspaces ¦V
j
¦ such that
¸
V
j
= T, and the family of representations Proj
V
j
(f) converges to the function f. An
example occurs when each subspace V
j
has ﬁnite dimension, as described in the next section.
2.6 Galerkin Representation
The Galerkin method computes a representation R(f) = (f
j
), j = 0, 1, . . . , N − 1 in such a
way that there exists a reconstruction equation
f
N
(t) =
N−1
¸
k=0
a
N
(k)φ
N,k
(t),
which approximates the function f, in norm, when N →∞. That is,
[[f −f
N
[[ →0, if N →∞.
The representation f
N
is therefore computed in such a way to minimize the norm [[f −f
N
[[.
Geometrically, the functions
φ
N,0
, φ
N,1
, . . . , φ
N,N−1
generate a subspace V of the space of functions, and the representation f
N
is the orthogonal
projection of f onto V (see Figure 2.1).
2.7. RECONSTRUCTION, POINT SAMPLING AND INTERPOLATION 17
Figure 2.1: Galerkin representation of a function f
The coeﬃcients a
N
(k) are computed using a dual set ϕ
N,k
of the set φ
N,k
:
'φ
N,k
, ϕ
N,j
` = δ
i,j
, a
N
(k) = 'f, ϕ
N,k
`.
The functions ϕ
N,k
are called sampling functions, analysis functions, or representation func
tions. The functions φ
N,k
are called synthesis functions or reconstruction functions.
It is important to remark that the reconstruction functions are not uniquely determined.
Also, changing these functions imply in a change of the reconstructed function. If φ
N,k
is an
orthonormal set, then the synthesis functions may be used as analysis functions, that is,
a
N
(k) = 'f, φ
N,k
`.
The Galerkin representation is also known in the literature as representation by ﬁnite projec
tion.
Galerkin representation is very important in devising numerical methods because they use
ﬁnite dimensional representation spaces, which is very suitable for computer implementations.
This method is used in diﬀerent representations of functions by piecewise polynomial functions
in the theory of approximation.
2.7 Reconstruction, Point Sampling and Interpolation
The well known example of a Galerkin representation consists in taking a ﬁnite point sampling
of a function f : [a, b] →R. We take a partition a = t
0
< t
1
, , < t
n
= b and deﬁne
R(f) = (f(t
0
), . . . , f(t
n
)) ∈ R
n
.
In this case, the Dirac “functions” δ(t −t
k
), k = 0, 1, . . . , n constitute the sampling functions
f(t
k
) = 'f, δ(t −t
k
)` =
+∞
−∞
f(u)δ(t −t
k
)du.
We can choose several diﬀerent reconstruction functions from the above representation. We
will study some classical reconstruction basis below.
18 CHAPTER 2. FUNCTION REPRESENTATION AND RECONSTRUCTION
2.7.1 Piecewise Constant Reconstruction
In this reconstruction each synthesis function is given by the characteristic functions of the
partition intervals.
φ
N,k
= χ
[t
k
,t
k+1
]
=
0 if x < t
k
or t > t
k+1
1 if x ∈ [t
k
, t
k+1
]
(2.5)
The graph of this function is shown in Figure 2.2(a). Geometrically, the reconstructed function
is an approximation of the function f by a function which is constant in each interval of the
partition (see Figure 2.2(b)).
Figure 2.2: (a) Graph of the synthesis function; (b) Reconstruction.
In the literature of signal processing the synthesis function used here is called box function.
This method reconstructs the functions approximating it by a discontinuous functions. The
discontinuities in the reconstructed function introduce high frequencies in the reconstructed
signal (see Chapter 7 of (Gomes & Velho, 1997)).
2.7.2 Piecewise Linear Reconstruction
In this case the synthesis functions φ
N,k
are given by
φ
N,k
=
0 if x < t
k−1
or t > t
k+1
t−t
k−1
t
k
−t
k−1
if x ∈ [t
k−1
, t
k
]
t
k+1
−t
t
k+1
−t
k
if x ∈ [t
k
, t
k+1
]
(2.6)
The graph of this function is shown in Figure 2.3(a). Geometrically, the reconstructed function
is an approximation of the function f by a continuous function which is linear in each interval
of the partition used in the sampling process. Figure 2.3(b) shows, in dotted lines, the graph
of the original function, and the graph of the reconstructed function using a continuous line.
Higher Order Reconstruction
We could continue with these reconstruction methods using polynomial functions of higher
degree deﬁned on each interval of the sampling partition. The diﬀerentiability class of these
2.8. MULTIRESOLUTION REPRESENTATION 19
Figure 2.3: (a) Graph of the synthesis function. (b) Reconstruction.
functions on the whole interval can be controlled according with the degree of the polynomials.
The spaces of splines is the natural habitat for these reconstruction methods.
We should remark that in the Galerkin representation using point sampling the recon
struction methods consist in interpolating and approximating the original function from its
samples. A natural question would be:
Question 1. Is it possible to obtain exact reconstruction using the Galerkin representation
by point sampling?
Note that exact reconstruction in this representation has an interesting geometric meaning:
Find an interpolation method that is able to reconstruct the function f exactly from the set of
its samples f(t
i
). We will return to this question in the next chapter.
2.8 Multiresolution Representation
We perceive the world through a multiscale mechanism. First we use a coarse scale to recognize
the object, then we use ﬁner scales in order to discover its distinct properties in detail. As an
example, the identiﬁcation of a house can be done in a very coarse scale, but ﬁner scales are
necessary in order to observe details about the windows, doors, ﬂoor, and so forth.
Therefore, it is natural that we look for multiscale representation of functions. That is,
we are interested in obtaining a family of representations that could represent the function
at distinct scales. At the same time we need techniques that allow us to change between
representations on diﬀerent scales.
This can be achieved by using nested representation spaces. Consider a sequence of closed
subspaces ¦V
j
¦
j∈Z
such that
V
j+1
⊂ V
j
⊂ V
j−1
, ∀ ∈ Z.
And a family of representation operators P
j
: V →V
j
such that
[[v −P
j
(v)[[ ≤ C inf
u∈V
j
[[v −u[[, (2.7)
where C independs on j. The proximity condition of equation (2.7) is trivially satisﬁed if
V
j
is a closed subspace of a Hilbert space, and P
j
is the orthogonal projection onto V
j
, as
illustrated in Figure 2.4.
20 CHAPTER 2. FUNCTION REPRESENTATION AND RECONSTRUCTION
Figure 2.4: Representation operator.
Note that from equation (2.7) we have
[[P
j
(v)[[ ≤ c[[v[[ (2.8)
P
j
(v) = v, ∀v ∈ V
j
(i.e. P
2
j
= P
j
) (2.9)
Also we require a natural commutativity condition P
j
◦ P
j−1
= P
j
, which guarantees that the
diﬀerent representations match each other.
Intuitively each representation space V
j−1
contains more details (ﬁner scale) than the space
V
j
. This can be stated precisely by introducing the operator Q
j
: V →V
j−1
, deﬁned by
Q
j
(v) = P
j+1
(v) −P
j
(v).
If W
j
= Image(Q
j
) = Q
j
(V
j
), it follows that
V
j−1
= V
j
+W
j
. (2.10)
That is, Q
j
(v) is the detail we must add to the representation space V
j
to obtain V
j+1
. For
this reason, Q
j
is called a reﬁnement operator.
Iterating equation (2.10) we have the decomposition equation
V
J
= V
j
0
+ (W
j
0
+ +W
J+1
) , (2.11)
which says that a ﬁner representation can be obtained from a ﬁner one, by adding details.
Equation (2.11) can be rewritten using operators:
P
j
(v) = P
j
0
(v) +
j
0
¸
j=J+1
Q
j
(v). (2.12)
In order to be able to decompose any element v of V we impose some additional hypothesis
on the representation operators P
j
:
1. P
j
(v) →v if j →−∞;
2.9. REPRESENTATION BY DICTIONARIES 21
2. P
j
(v) →0 if j →+∞.
From the proximity condition of equation (2.7), these two conditions are equivalent to
1.
¸
j∈Z
V
j
is dense on V ;
2.
¸
j∈Z
= ¦0¦.
By taking the limit, j
0
→+∞ and j →−∞, in (2.12) we have
v =
¸
j∈Z
Q
j
(v), ∀v ∈ V.
That is, any vector v is the sum of all of its details in the diﬀerent representations.
In the particular case where the representation operators P
j
are orthogonal projections
over closed Hilbert spaces, the sum in equation (2.11), is in fact a direct sum, and the spaces
V
j
and W
j
are orthogonal.
Techniques to compute nested sequences of multiresolution representation are closed re
lated with wavelets.
2.9 Representation by Dictionaries
The problem of obtaining a representation of a function can be well understood using the
metaphor of natural language. When we write, we have an idea and we must represent it
using words of the language. If we have a rich vocabulary we will have a great conciseness
power to write the idea, on the contrary, we will have to use more words to express the same
idea. Based on this metaphor, S. Mallat (Mallat & Zhang, 1993) introduced in the literature
a representation method based on dictionary of functions.
A dictionary in a function space H is a family T = (g
λ
)
λ∈Γ
of vectors in H such that
[[g
λ
[[ = 1. This family is not necessarily countable. A representation of a function in a
dictionary T is a decomposition
f =
¸
λ∈Γ
α
γ
g
γ
,
such that (α
γ
) ∈
2
. The rationale behind the representation method consists in constructing
extensive dictionaries and devising optimal representation techniques that allow us to rep
resent a function using a minimum of words from the dictionary. Therefore, Representation
using dictionaries allows us the use of a great heterogeneity in the reconstruction functions,
which makes the representation/reconstruction process very ﬂexible.
Note that distinct functions use diﬀerent dictionary vectors in their representation which
makes the representation process nonlinear. In (Mallat, 1998) several dictionary systems
are described, as well as techniques to compute dictionary based representations. One basic
strategy to compute a representation for a function f is described below.
Let f
M
be the projection of f over the space generated by M vectors from the dictionary,
with index set I
M
:
f
M
=
¸
m∈I
M
'f, g
m
`g
m
.
22 CHAPTER 2. FUNCTION REPRESENTATION AND RECONSTRUCTION
The error e
M
in the approximation is the sum of the remaining coeﬃcients
e
M
= [[f −f
M
[[
2
=
¸
m/ ∈I
M
['f, g
m
`[
2
.
To minimize this error the indices in I
M
must correspond to the M vectors that have the
largest inner product amplitude ['f, g
m
`[. These are the vectors that have a better correlation
with f, that is, the vectors that best match the features of f. Certainly the error is smaller
than the error resulting from a linear approximation where the decomposition vectors of the
representation do not vary with f.
The above ideas lead to an algorithm to compute the representation. For details see
(Mallat & Zhang, 1993).
2.10 Redundancy in the Representation
The representation of a function is not unique in general. Besides nonunicity, we can have a
redundancy for a given representation. This occurs for instance in the representation using
frames. In fact, if the frame (ϕ
j
)
j∈Z
is an orthonormal basis, the representation operator
R: H →
2
, (Rf)
j
= 'f, ϕ
j
`
is an isometry, and the image of H is the whole space
2
. The reconstruction equation
f =
¸
j∈Z
'f, ϕ
j
` ˜ ϕ
j
computes each vector in such a way that there is no correlation between the coeﬃcients.
If the frame is not constituted by linearly independent vectors, there exists a redundancy
in the representation, which corresponds to a certain correlation between the elements of the
representation sequence. This redundancy can be used to obtain a certain robustness in the
representation/reconstruction process. An interesting discussion of the redundancy in frame
representation can be found in (Daubechies, 1992), page 97.
2.11 Wavelets and Function Representation
We can summarize what we have done up to this point in the statement that our main concern
is the study of representation and reconstruction of functions. Therefore our slogan at this
point should be “all the way to function representation and reconstruction techniques”.
In fact, we need to develop techniques that allow us to construct representation tools
(basis, dictionaries, frames, multiscale representations, etc.) which are ﬂexible, concise and
allows for robust reconstruction.
This leads us naturally to the study of function variation as an strategy to detect the
diﬀerent features of a given function, and take them into consideration when representing the
function. We will start our journey on this road from the next chapter on. It will take us
from the kingdom of Fourier to the wonderland of wavelets.
2.12. COMMENTS AND REFERENCES 23
2.12 Comments and References
Multiscale representation was introduced by S. Mallat in (Mallat, 1989) in the context of the
Hilbert spaces L
2
(R) of functions, and orthonormal projection. The more general introduction
we gave in Section 2.8 of this chapter was based on (Canuto & Tabacco, 1997). A detailed
description of Mallat’s work will be given later on.
A detailed study of representation and reconstruction using frames is found in (Daubechies,
1992), pages 5763, and also in Chapter V of (Mallat, 1998).
Bibliography
Canuto, Claudio, & Tabacco, Anita. 1997. Multilevel Decompositions of Functional Spaces.
The Journal of Fourier Analysis and Applications, 3(6).
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Gomes, J., & Velho, Luiz. 1997. Image Processing for Computer Graphics. SpringerVerlag.
Mallat, S. 1989. Multiresolution approximation and wavelets. Trans. Amer. Math. Soc., 315,
69–88.
Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press.
Mallat, S., & Zhang, A. 1993. Matching Pursuit with Timefrequency Dictionaries. Tech.
rept. Courant Institute of Mathematical Sciences.
Chapter 3
The Fourier Transform
In order to devise good representation techniques we must develop tools that enable us to
locate distinguished features of a function. The most traditional of these tools is the Fourier
Transform which we will study in this chapter. The study of Fourier transform, its strength
and limitations, is the starting point of our journey to the wavelets.
3.1 Analyzing Functions
To detect features of a function we must analyze it. This occurs in our everyday routine:
signals are analyzed and interpreted by our senses, a signal representation is grabbed from
this analysis and it is sent to our brain. This is the process used in our perception of colors
and sound.
Sound, Color, and other elements we interact with in our everyday life are characterized
by functions: To each point on the space, and to each instant of time the function produces
a certain output which we are able to detect. These functions are usually called signals.
The best way to analyze the features of a signal is by studying its frequencies. In an audio
signal, for example, the frequencies are responsible for what we are accustomed to identify as
an acute or grave sound. Also, the distinction from red to green is captured in the frequency
of the associated electromagnetic wave.
3.1.1 Fourier Series
In order to analyze the frequency content of a function we must initially answer the following
question: What is the frequency of a function? This is an easy task when the function is
periodic. In fact given the function f(t) = Asen(2πωt), A > 0, the parameter A measures the
amplitude (maximum and minimum values assumed by f), the parameter ω indicates how
many complete cycles of period exist on the interval [0, 1]. This number is directly connected
with the number of oscillations of the function in the unit of time, which is called the frequency
of the function. Figure 3.1 shows the graph of f for diﬀerent values of ω.
Consider a periodic function with period L > 0, that is, f(t + L) = f(t). We denote by
25
26 CHAPTER 3. THE FOURIER TRANSFORM
Figure 3.1: Sine function with frequencies 2, 4 and 8.
L
2
T
(R) the space of periodic functions of period T which are square integrable. That is
t
0
+T
t
0
[f(t)[
2
dt < ∞.
The theory of Fourier series says that f can be decomposed as
f(s) =
+∞
¸
j=−∞
a
j
e
i2πω
j
s
, a
j
∈ R, (3.1)
where ω
j
= j/T is a constant. This decomposition of a periodic function f is called the Fourier
series of f. It is well known that the family ¦e
i2πω
j
s
, j ∈ Z¦ is a complete orthonormal set
of the space L
2
T
(R). Therefore equation (3.1) is an orthogonal basis representation of the
function f.
In conclusion, the Fourier series shows that any periodic function can be decomposed as
an inﬁnite sum of periodic functions (sines and cosines). This decomposition makes it easy an
analysis of the frequencies present on the function f: There exists a fundamental frequency ω,
and all of the other frequencies are integer multiples ωj, j ∈ Z, of this fundamental frequency.
The coeﬃcient a
j
in the equation (3.1) of the Fourier series measures the amplitude of
the frequency component ω
j
on the function f. In particular, if a
j
= 0, this frequency is not
present in the function. This frequency amplitude a
j
is computed using the equation
a
j
=
L
0
f(u)e
i2πω
j
u
du, (3.2)
where L is the period of the function. Note that equation (3.1) completely characterizes the
function f by its frequencies. In other words, we have an exact representation of f
3.1.2 Fourier Transform
The above portrait of the function f worked perfectly well: We were able to obtain an exact
representation of the function f and this representation completely characterizes f by its
frequencies. The only drawback is the fact that f was a periodic function.
Is it possible to extend the above results for nonperiodic functions? In this case we do not
have a discrete spectrum of well deﬁned frequencies as in equation (3.1). In fact, every function
3.1. ANALYZING FUNCTIONS 27
f deﬁned by equation (3.1) is necessarily periodic. Nevertheless, we can use the Fourier series
representation as an inspiration to introduce the concept of frequency for arbitrary functions.
Take s = ω
j
in equation (3.2) which computes the amplitude of the frequency, and assume
that the variable s takes any value. We obtain
a(s) =
+∞
−∞
f(u)e
i2πsu
du. (3.3)
Notice that we have changed the notation from a
j
to a(s).
The operation f(u)e
i2πsu
in the integrand above is called modulation of the function f.
The exponential is called the modulating function. For each s, e
i2πsu
is a periodic function of
frequency s, s ∈ R. Therefore, for each s ∈ R equation (3.3) can be interpreted as an inﬁnite
weighted average of f using the modulating function as the weighting function.
The rationale behind the frequency computation in the modulation process can be ex
plained as follows: When f has oscillations of frequencies s, or close to s, these frequencies
result to be in resonance with the frequency s of the modulating function, therefore a(s) as
sumes nonzero values. On the contrary, when the oscillations of f and the frequencies of the
modulating function are completely distinct we have a cancellation eﬀect and the integral of
(3.3) is zero or close to zero. We conclude that a(s) measures the occurrence of the frequency
s on the function f. Since s varies in a continuum of numbers, it is interesting to interpret
(3.3) as describing a frequency density of the function f. When a(s) = 0 this means that
frequencies s occurs on f. The value of a(s) is a measure of the occurrence of the frequency
s on f.
We will change notation and denote a(s) by
ˆ
f(s) or by F(f)(s). Note that
ˆ
f is in fact an
operator that associates to the function f the function
ˆ
f = F(f), deﬁned by (3.3). Therefore
we have an operator deﬁned between two function spaces. What function spaces are these? It
is possible to show that the operator is well deﬁned for any function f satisfying the equation
[[f[[ =
R
[f(u)[
2
du < ∞. (3.4)
These functions are called square integrable functions or functions with ﬁnite energy. This
space is denoted by L
2
(R). Equation (3.4) deﬁnes a norm on L
2
(R), and it is easy to see that
this norm is induced by the inner product.
'f, g` =
R
f(u)g(u)du,
where g means the complex conjugate. It is well known that the space L
2
(R) with this inner
product is a Hilbert space, called the space of square integrable functions.
In sum, we have introduced an operator F =
ˆ
f : L
2
(R) →L
2
(R), deﬁned by
F(f)(s) =
ˆ
f(s) =
+∞
−∞
f(u)e
−i2πsu
du. (3.5)
This operator is called Fourier Transform. Note that we changed the signal of the exponent
when passing from (3.3) to (3.5) so as to keep the deﬁnition compatible with the deﬁnition of
the Fourier transform used in the literature.
28 CHAPTER 3. THE FOURIER TRANSFORM
Without going into too much details, we would like to clarify some mathematical points.
The Fourier transform is well deﬁned when the function f belongs to the space L
1
(R) of
integrable functions. In this case it is easy to see that
ˆ
f is bounded and continuous. In fact,
[
ˆ
f(w) −
ˆ
f(s)[ ≤
R
[f(t)[e
i2πwt
−e
i2πst
[ ≤
R
[f(t)dt
[w −s[.
The extension of f to the space L
2
(R) of square integrable functions is achieved by a limiting
process: Any function f ∈ L
2
(R) can be approximated by integrable functions of ﬁnite energy.
More precisely, for any f ∈ L
2
(R), there exists a sequence f
n
∈ L
1
(R) ∩ L
2
(R) such that
lim
n→∞
[[f
n
−f[[ = 0.
We deﬁne
ˆ
f as the limit of the sequence
ˆ
f
n
. Why is it important to deﬁne the Fourier
transform on L
2
(R), instead of L
1
(R)? We can give two main reasons for that:
• The Fourier transform of an integrable function is not necessarily integrable. Therefore
F is not an operator F : L
1
(R) →L
1
(R).
• The space L
2
(R) has a richer structure than the space L
1
(R) because of its inner product.
In fact it is a Hilbert space.
From the linearity of the integral it follows easily that the Fourier transform F : L
2
(R) →
L
2
(R) is linear. A fascinating result is that F is an isometry of L
2
. That is,
'f, g` = '
ˆ
f, ˆ g`, ∀f, g ∈ L
2
(R). (3.6)
This equation is known as Parseval identity. It follows easily from it that
[[f[[
2
= [[
ˆ
f[[
2
. (3.7)
This equation is known as Plancherel equation.
The inverse of the Fourier transform is given by
F
−1
(g)(t) =
+∞
−∞
g(s)e
i2πωt
ds. (3.8)
That is, F
−1
(
ˆ
f) = f. Therefore, we have
f(t) = F
−1
(
ˆ
f) =
+∞
−∞
ˆ
f(s)e
i2πst
ds. (3.9)
An important fact to emphasize is that we have two diﬀerent ways to “read” equation
(3.9) of the inverse transform:
1. It furnishes a method to obtain the function f from its Fourier Transform
ˆ
f.
2. It allows us to reconstruct f as a superposition of periodic functions, e
i2πst
, and the
coeﬃcients of each function in the superposition are given by the Fourier transform.
The second interpretation above shows that the equation of the inverse Fourier transform
is a nondiscrete analogous of equation (3.1) of the Fourier series.
3.2. A PAUSE TO THINK 29
3.1.3 Spatial and Frequency Domain
Analyzing from the mathematical point of view, the Fourier transform is an invertible linear
operator on L
2
(R). Nevertheless, in the applications we have a very interesting interpretation
of it. A function f : R →R, can be interpreted as associating to each value of t in the spatial
domain R some physical magnitude f(t). When we compute the Fourier transform of f, we
obtain another function
ˆ
f(s) deﬁned on L
2
(R). In this case, for each value of the parameter
s ∈ R the value
ˆ
f(s) represents the frequency density s in f. We interpret this by saying that
ˆ
f is a representation of the function f in the frequency domain. In summary, the Fourier
transform changes a function from the spatial to the frequency domain.
Since in our case the spatial domain has dimension 1 it is common to interpret the variable
t as time and call the spatial domain by the name of time domain.
Note that describing a function on the frequency domain allow us to obtain the frequency
contents of the function. The frequency contents is closely related with the features carried
by the function. As we have pointed out on the previous chapter, these features are important
elements to obtain good function representation.
3.2 A Pause to Think
Our purpose in this section is to have a better understanding of the potentialities of the Fourier
transform from the point of view of function representation and reconstruction. Equation (3.2)
provides us with information about the frequencies of a periodic function. For this reason, it
is called the analysis equation. Equation (3.1) allows us to obtain f from the coeﬃcients a
j
computed using the analysis equation. For this reason it is called a synthesis equation. From
the point of view of the previous chapter the analysis equation computes a representation of
the periodic function f by the sequence (a
j
), j ∈ Z. The synthesis equation provides us with
a reconstruction technique. Note that in this case the reconstruction is exact.
When the function is not periodic we know that equation (3.5), which deﬁnes the Fourier
transform, gives us an analysis of the frequencies of the function f. Equation (3.9) writes the
function f as a superposition of a “continuum” of periodic functions. This equation plays the
role of the reconstruction equation (3.1) of the function f (the analysis equation). Note that
the analysis and synthesis equation associated with the Fourier transform are not discrete as
in the case of the Fourier series, for periodic functions. Therefore we do not have a tool that
allow us to represent and reconstruct arbitrary functions as in the case of the Fourier series.
One important question can be posed now: How eﬀective is the Fourier transform analysis
of a function? A discussion of this question will be done in the next section.
3.3 Frequency Analysis
In this section we study some examples that will give a better insight into the analysis provided
by the Fourier transform.
Suppose that
ˆ
f(s
0
) = 0. From this we conclude that f has frequencies s
0
or close to s
0
.
In this case, the next step of analyzing f consists in determining the localization of these
30 CHAPTER 3. THE FOURIER TRANSFORM
frequencies on the spatial domain of f. This localization is of great importance in several
applications, and in particular in the problem of function representation. We can make an
analogy using a radar metaphor: the existence of a certain frequency detects the presence of
some object, and the localization of a frequency allows us to determine the object position.
In general localizing frequencies on the spatial domain using Fourier transform is im
possible. This happens because the modulating function (exponential) used to measure the
frequency density on the Fourier transform does not have compact support: the integral that
deﬁnes the transform extends to the whole line. Therefore the only information carried from
the fact that
ˆ
f(s) = 0 is that the frequency s, or frequencies close to s, are present on the
function f.
Example 5 (Signal with impulses). Consider the signal deﬁned by the function
f(t) = sin(2π516.12t) + sin(2π2967.74t) +δ(t −0.05) +δ(t −0.42).
It consists of a sum of two sines with frequencies 516.12Hz and 2967.74Hz, where we added
two impulses of order 3 at the two diﬀerent positions t
0
= 0.05s e t
1
= 0.42s. The graph
of this signal is shown on Figure 3.2(a). The graph of its Fourier Transform is shown on
Figure 3.2(b).
0 500 1000 1500 2000 2500 3000 3500 4000 4500
−2
−1
0
1
2
3
4
5
(a)
0 500 1000 1500 2000 2500
0
200
400
600
800
1000
1200
1400
1600
1800
2000
(b)
Figure 3.2: A signal and its Fourier transform.
Note that the Fourier transform detects well the two sine signals (the two spikes in Figure
(b)). How about the impulses? It is easy to see from the deﬁnition of the Fourier transform,
that translation of an impulse on the spatial domain introduces modulation by an exponential
on the frequency domain. Therefore the modulation introduced by the two impulses generate
a superposition of frequencies. These frequencies do not appear on Figure 3.2(b) because of
an scale problem. Figure 3.3 plots the graph in a diﬀerent scale so as to make it possible to
observe the superposed frequencies corresponding to the two impulses.
In conclusion the Fourier transform does not provide clear information about the two
impulses.
3.3. FREQUENCY ANALYSIS 31
0 500 1000 1500 2000 2500
0
1
2
3
4
5
6
Figure 3.3: Fourier transform of two impulses δ(t −0.05) e δ(t −0.42).
The diﬃculty in localizing frequencies on the spatial domain is one of the major weakness
of the Fourier transform in analyzing signals. Below we will give a classical example that
shows the ineﬃciency of the Fourier transform in representing functions.
Example 6 (Fourier analysis of the impulse signal). Consider the impulse “function”
Dirac δ. We have
+∞
−∞
f(t)δ(t)dt = f(0).
Taking f(t) = e
−i2πst
in this equation we have
ˆ
δ(s) =
+∞
−∞
δ(t)e
−i2πst
dt = 1.
This shows that the Fourier transform of the impulse δ is a constant function of value 1. Using
this result in the reconstruction equation (3.9), we obtain
δ(t) =
+∞
−∞
ˆ
δ(f)e
i2πst
ds =
+∞
−∞
e
i2πst
ds.
That is, exponentials of every frequency values s ∈ R must be combined in order to analyze
the impulse signal.
This fact has a very simple mathematical explanation: As the Fourier transform uses
modulating functions without compact support, and periodic, we must “sum” an inﬁnite
number of these functions so as to occur destructive interference. Nevertheless from the
physical point of view we do not ﬁnd a plausible explanation for the problem of representing
such a simple signal combining an inﬁnite number of periodic functions.
In general, if a function presents sudden changes (e.g. discontinuities), the high frequencies
relative to these changes are detected by the transform, but they inﬂuence the computation of
the Fourier transform along all of the domain because the modulating function does not have
compact support. The Fourier analysis is therefore more eﬃcient in the study of signals that
do not suﬀer sudden variations along the time. These signals are called stationary signals.
32 CHAPTER 3. THE FOURIER TRANSFORM
3.4 Fourier Transform and Filtering
Even though the classical theory of ﬁltering is developed for discrete signals, a good insight
is gained by studying ﬁltering of continuous signals. The relation of the ﬁltering theory with
the Fourier analysis is very important and will be exploited in many diﬀerent parts of the
book.
A ﬁlter is an operator L: L
2
(R) → L
2
(R), deﬁned on the space of signals with ﬁnite
energy. An important class of ﬁlters is given by the linear, time invariant ﬁlters. A ﬁlter is
linear if the operator L: L
2
(R) → L
2
(R), is linear. A ﬁlter is time invariant if a time delay
can be applied either before of after ﬁltering, with the same result. That is,
L(f(t −t
0
)) = (Lf)(t −t
0
).
Linear and space invariant ﬁlters are simple to study because they are completely determined
by their values on the impulse signal h = Lδ(t). Indeed, time invariance of L gives
Lδ(t −u) = h(t −u).
Therefore,
L(f(t)) = L
R
f(u)δ(t −u)du (3.10)
=
R
f(u)h(t −u) =
R
h(u)f(t −u)du. (3.11)
The last integral on the above equation is called the convolution product of h and f, and
it is denoted by h ∗ f(t). That is,
h ∗ f(t) =
R
h(u)f(t −u)du.
We conclude that ﬁltering signal f with a linear, time invariant ﬁlter L is equivalent to make
a convolution of f with the signal h = Lδ(t). The signal h is called the impulse response of
the ﬁlter. Sometimes h is called the kernel of the ﬁlter.
Applying the ﬁlter L to e
−i2πwt
yields
L(e
−i2πwt
) =
R
h(u)e
−i2πw(t−u)
du (3.12)
= e
−i2πwt
R
h(u)e
−i2πwu
du (3.13)
=
ˆ
h(w)e
−i2πwt
. (3.14)
This shows that each exponential e
−i2πwt
is an eigenvector of the ﬁlter L, and the corre
sponding eigenvalue is the value
ˆ
h(w) of Fourier transform of the impulse response function
h of L.
This result is of paramount importance and it is the link between ﬁlter theory and Fourier
analysis. In fact, as an immediate consequence it gives an insight in the action of the ﬁlter
on the frequencies of a signal f as we will show below.
3.4. FOURIER TRANSFORM AND FILTERING 33
From the equation of the inverse Fourier transform we have
f(t) =
R
ˆ
f(w)e
i2πwt
dw.
Applying L to f we get
L(f(t)) =
R
ˆ
f(w)L(e
i2πwt
)dw =
R
ˆ
f(w)
ˆ
h(w)e
i2πwt
dw. (3.15)
The above equation shows that the ﬁlter L modulates the sinusoidal components e
i2πwt
of
f, amplifying or attenuating them. Since Lf(t) = h ∗ f, equation (3.15) can be restated as
F(h ∗ f) = F(h)F(f), (3.16)
where F is the Fourier transform, and on the right we have a product of two functions. The
Fourier transform F(h) of the impulse response is called the transfer function of the ﬁlter.
Equation (3.16) will be used to give a better insight into the ﬁltering operation.
Linear and space invariant ﬁlters are classiﬁed according to the way they modify the
frequencies of the signal f. This classiﬁcation includes four basic types:
• Lowpass;
• Highpass;
• Bandpass
• Bandstop.
Lowpass Filters
This ﬁlter is characterized by the fact that they attenuate the high frequencies of the signal
without changing the low frequencies. From equation (3.16) it follows that the graph of the
transfer function of an ideal lowpass ﬁlter is shown in Figure 3.4.
Figure 3.4: Transfer function of an ideal lowpass ﬁlter.
Highpass Filter
This ﬁlter has a complementary behavior to that of the lowpass ﬁlter. It attenuates the low
frequencies and does not change the high frequencies. From equation (3.16) it follows that
the graph of a transfer function of an ideal highpass ﬁlter is shown in Figure 3.5.
34 CHAPTER 3. THE FOURIER TRANSFORM
Figure 3.5: Transfer function of an ideal highpass ﬁlter.
Bandpass Filter
This ﬁlter changes both low and high frequencies of the signal, but does not change frequencies
in some interval (band) of the spectrum. The graph of the transfer function of an ideal band
pass ﬁlter is shown in Figure 3.6. A lowpass ﬁlter is a bandpass ﬁlter for low frequencies.
Figure 3.6: Transfer function of an ideal bandpass ﬁlter.
Bandstop Filter
This is the complementary of a bandpass ﬁlter. This ﬁlter aﬀects frequencies on an interval
(band) of the spectrum. Frequencies outside this frequency band are not aﬀected. The graph
of the transfer function of an ideal bandstop ﬁlter is shown in Figure 3.7.
Figure 3.7: Transfer function of an ideal bandstop ﬁlter.
3.5 Fourier Transform and Function Representation
Is it possible to obtain representation and reconstruction techniques for a nonperiodic func
tion using the Fourier transform?
3.5. FOURIER TRANSFORM AND FUNCTION REPRESENTATION 35
Equation (3.9) which deﬁnes the inverse transform gives us a clue to look for an answer
to our question. In fact, this equation writes the function f as a superposition of periodical
functions, “modulated” by the Fourier transform.
Certainly there exists a redundancy in this “representation” of the function f by a con
tinuum of functions. We can eliminate this redundancy by taking only a discrete set of
frequencies s
j
= ω
0
j, ω
0
constant, j ∈ Z.
Unfortunately this discretization leads us to the Fourier series of the function f. In fact,
Fourier series are particular instances of Fourier transform for discrete signals. If
f(t) =
¸
n∈Z
f(n)δ(t −n),
then
ˆ
f(w) =
¸
n∈Z
f(n)e
−i2πwn
.
Therefore the method that seems natural to obtain function representations from the
Fourier transform is in fact inadequate. Nevertheless the Fourier theory, and its relationship
with ﬁlter theory, is of great importance in the study of diﬀerent representation methods.
This fact will be illustrated in the next section.
3.5.1 Fourier Transform and Point Sampling
In spite of all of the weakness of the Fourier transform that we have already discussed, it is a
powerful tool to analyze the problem of function representation. We will use it to understand
the problem of exact reconstruction from a representation by point sampling.
An important step in this direction is to understand the spectrum of a point sampled signal.
More precisely, suppose that we have a uniform sampling partition of the real numbers with
interval length ∆t. This length is called the sampling period. The discretized signal is given
by
f
d
(t) =
¸
k∈Z
f(k∆t)δ(t −k∆t).
The relation between the Fourier transform
ˆ
f of f and the Fourier transform
ˆ
f
d
of the dis
cretized signal f
d
is given by
ˆ
f
d
(s) =
1
∆t
¸
k∈Z
ˆ
f
s −
k
∆t
(3.17)
A clear interpretation of equation (3.17) is very important: Apart from the scaling factor
1/∆t, it says that the spectrum of the sampled signal f
d
is obtained from the spectrum
ˆ
f
of the continuous signal f by translating it by multiples of 1/∆t, and summing up all of the
translated spectra. This is illustrated in Figure 3.8: In (a) we show the graph of the function
f; in (b) we show the graph of the Fourier transform
ˆ
f; in (c) we show the graph of the
point sampling representation (f
i
) of f; in (d) we show the Fourier transform of the sampled
function f
d
.
36 CHAPTER 3. THE FOURIER TRANSFORM
Figure 3.8: Sampling and the Fourier spectrum.
It is important to remark that the translating distance varies inversely with the sampling
period: The sampling period is ∆t and the translating distance is 1/∆t. Therefore closer
samples produce more spaced translations and viceversa. Note that in particular, very high
frequencies are introduced in the sampling process by the inﬁnite translation of the spectrum.
Equation (3.17) is the key point to understand the reconstruction problem when we use
uniform point sampling to represent a function. This fact will be discussed in next section.
3.5.2 The Theorem of ShannonWhittaker
In this section we will use Fourier theory to answer a question posed on chapter 2: Is it
possible to obtain an exact representation of a function using point sampling?
We will see that the answer is positive if we restrict the function f and at the same time
impose conditions on the sampling rate. Initially we will demand the following conditions:
• The point sampling process is uniform. That is, the sampling intervals [t
k
, t
k+1
] have
the same length ∆t = t
k+1
−t
k
.
• The Fourier transform
ˆ
f of the function f, assumes zero values outside a bounded
interval [−Ω, Ω] of the frequency domain. We say that
ˆ
f has compact support.
Using the above restrictions, we have the classical:
Theorem 2 (Theorem of ShannonWhittaker). Consider a function f : R → R with
compact support, supp (
ˆ
f) ⊂ [−Ω, Ω], and a uniform partition t
i
, i ∈ Z of the real num
bers R such that 2∆t ≤ Ω. Then f can be reconstructed (exactly) from its point sampling
3.6. POINT SAMPLING AND REPRESENTATION BY PROJECTION 37
representation (f(t
i
))
i∈Z
. The reconstruction equation is given by
f(t) =
+∞
¸
k=−∞
f(
k
2Ω
)
sin π(2Ωt −k)
π(2Ωt −k)
. (3.18)
The inequality 2∆x ≤ Ω is called Nyquist limit. It says that we must take at least 2
samples for each complete cycle of maximum frequency occurring in the function.
The result of the theorem is very intuitive. The hypothesis of the theorem says:
1. The function f should not have very high frequencies, supp (
ˆ
f) ⊂ [−Ω, Ω];
2. We take samples of f suﬃciently close, such that we have at least 2 samples for a
complete cycle of maximum frequency (2∆t ≤ Ω);
The conclusion of the theorem is: f can be exactly reconstructed interpolating its samples
using equation (3.18).
A function f such that
ˆ
f has compact support is called a band limited function, because it
posses frequencies within a limited interval (band) of the frequency domain. It is interesting
to make a sketch of the proof of the theorem so that we can see the role of Fourier analysis
and ﬁltering.
Sketch of the Proof: Consider a band limited function f. The point sampling represen
tation transforms f into the sequence f(t
i
) which is null outside the points of the sampling
partition, and assumes the values of f on the points of the partition.
We have seen from equation (3.17) that the sampling process alters the frequency spectrum
of the function f, introducing very high frequencies by translating and summing up the original
spectrum. In order to reconstruct f we have to recover the original frequency spectrum from
the spectrum of the sampled function.
The translation distance of the spectrum of the original signal f depends on the sampling
period (the length of the intervals in the sampling lattice): The smaller the period (that means
more space between consecutive samples), the bigger will be the translation of the spectrum.
The Nyquist limit 2∆t ≤ Ω says that if we take the sampling period suﬃciently small, the
translated spectra will have disjoint domains. In this case, using an adequate lowpass ﬁlter
we can recover the original spectrum of the signal f. From this original spectrum we are able
to reconstruct the original signal using the inverse Fourier transform.
Details of this proof, including the computation of the reconstruction equation (3.18), can
be found in (Gomes & Velho, 1997).
3.6 Point Sampling and Representation by Projection
It can be shown that if we take
e
Ω
(t) =
sin π2Ωt
π2Ωt
,
38 CHAPTER 3. THE FOURIER TRANSFORM
the set ¦e
Ω
(t −k∆t)¦ is an orthogonal basis of the space
L
2
Ω
(R) = ¦f ∈ L
2
(R) ; supp(
ˆ
f) ⊂ [−Ω, Ω]¦.
Moreover, the reconstruction equation (3.18) is the projection of f on this basis. In sum, the
problem of point sampling and reconstruction is reduced to the problem of representation on
an orthogonal basis.
If a signal f is not bandlimited, that is, f / ∈ L
2
Ω
(R), and it is represented using point sam
pling with period ∆t ≤ Ω/2, and we reconstruct the sampled signal using the reconstruction
equation (3.18) of ShannonWhittaker, we obtain a function
˜
f ∈ L
2
(R) such that [[
˜
f −f[[ is
minimized. In fact,
˜
f is the orthogonal projection of f on L
2
Ω
(R).
3.7 Point Sampling and Representation Coeﬃcients
When we have an orthonormal basis ¦φ
j
¦ of the space L
2
(R), the representation of f in this
basis is given by
f =
¸
n
'f, φ
n
`φ
n
.
In this case we have the representation sequence
f →('f, φ
n
`)
n∈Z
.
Note that if φ
n
(x) = δ(x −n), then
'f, φ
n
` = 'f, δ(x −n)` = f(n).
That is, the elements of the representation sequence are samples of the function f. This fact
motivates us to pose the following question:
Question 2. What is the relation between the elements 'f, φ
n
` of the representation sequence
and the samples f(n) of the function f?
There is a simple answer for this question in a very particular case of great importance in
the study of wavelets. We will suppose that the functions φ
n
which constitute the orthonormal
basis are obtained from a single function φ by translations. More precisely,
φ
n
(x) = φ(x −n).
From Parseval equation, and the identity
ˆ
φ
n
(s) = F(φ(x −n)) = e
−i2πins
ˆ
φ(s),
we have
'φ
n
, f` = '
ˆ
φ
n
,
ˆ
f` =
+∞
−∞
e
2πins
ˆ
φ(s)
ˆ
f(s)ds = F(n),
3.8. COMMENTS AND REFERENCES 39
where the function F is given by its Fourier transform
ˆ
F(ω) =
ˆ
φ(ω)
ˆ
f(ω). (3.19)
Notice that
[
ˆ
F(s)[ ≤ [[
ˆ
φ[[ [[
ˆ
f[[ = [[φ[[ [[f[[,
therefore
ˆ
F(ω) is integrable, and this implies that F is continuous. This shows that the values
of the samples F(n) are well deﬁned.
If the function φ is a lowpass ﬁlter, equation (3.19) shows that F is obtained from the
original function f by a lowpass ﬁltering process, therefore the values of F(n) are close to
the values of the original function f, if it does not have great frequency variations.
For this reason, it is common to refer to the elements 'f, φ
n
` from the representation
sequence as samples of the function f, even when φ is not a lowpass ﬁlter. This fact is
resumed in the theorem below for future references:
Theorem 3. If ¦φ
n
= φ(x −n)¦ is an orthonormal basis of L
2
(R) then the terms 'f, φ
n
` of
the representation sequence of f on the basis ¦φ
n
¦ are obtained by ﬁltering f, F = f ∗ φ, and
sampling the resulting function F. That is,
'f, φ
n
` = 'f, φ(x −n)` = F(n)
3.8 Comments and References
There are several good books covering the theory of Fourier analysis. For a revision of the
classical theory we suggest the reader to consult (Weaver, 1989). This reference also covers
the discrete Fourier transform. A comprehensive reference for the discrete Fourier transform,
both from the conceptual and computational point of view, is found in (Briggs & Henson,
1995).
In practice we use ﬁnite signals when implementing the operations of sampling and recon
struction on the computer. Therefore, we need to study the Fourier transform of ﬁnite signals.
This Fourier transform is called Discrete Fourier Transform (DFT). For the reader interested
in details we recommend (Gomes & Velho, 1997) and (Briggs & Henson, 1995). The reference
(Gomes & Velho, 1997) brings a chapter with a review of signal theory, adequate for those
with some computer graphics background, it can be found on Chapter 1 of (Gomes & Velho,
1997).
From the computational viewpoint an important issue related with the Fourier transform is
the study of its computational complexity. That is, the study of the computational complexity
of the DFT when applied to a ﬁnite signal with N samples. There are diﬀerent ﬂavors of
computing with a Fast Fourier Transform, which reduces the computational complexity. For
those interested in these topics we recommend (Loan, 1996) and (Duhamel & Vetterli, 1990).
We have not stressed in this chapter a very important point when working with Fourier
analysis and ﬁltering: Since all of the theory studied here is linear, a natural representation
for them is to use matrices. This approach is important specially from the computational
point of view. Matrix notation is used all over in (Strang & Nguyen, 1996). We will use some
40 BIBLIOGRAPHY
matrix notation later on these notes. The reader should consult the Appendices to this course
notes where we introduce matrix notation.
The concepts of this chapter extend naturally to real functions of several variables. Of
particular importance in computer graphics is the case of functions f : R
2
→ R, which de
scribes an image, and the case f : R
3
→R which is related to the study of volumetric objects.
A good reference at an introductory level that covers twodimensional signal processing is
(Lim, 1990).
A detailed discussion of the diﬀerent problems arising from incorrect reconstruction of a
signal from a representation by point sampling, is found in Chapter 7 of (Gomes & Velho,
1997).
An analysis of the reconstruction problem when we use uniform point sampling represen
tation with a sampling rate superior to the Nyquist limit is found in page 20 of (Daubechies,
1992).
The problem of reconstructing a function from its samples called the attention of math
ematicians since the beginning of the century. For details about the history of this problem
we recommend (Butzer & Stens, 1992). A comprehensive discussion with diﬀerent versions of
the problem including solutions can be found on (Zayed, 1993).
The Theorem of ShannonWhittaker is a very important result in the theory of func
tion discretization. Nevertheless it presents certain deﬁciencies as a solution to the sam
pling/reconstruction problem. In particular, the hypothesis that
ˆ
f has compact support is
too much restrictive. Several research directions are raised from the Theorem of Shannon
Whittaker. We could mention the following ones:
• Look for versions of the theorem using a weaker hypothesis than that of compact support;
• Generalize the theorem for arbitrary domains of R
n
.
• Analyze what happens with the reconstruction when the sampling is done without obey
ing the Nyquist limit.
Bibliography
Briggs, W. L., & Henson, V. E. 1995. DFT: an owner’s manual for the discrete Fourier
transform. SIAM books, Philadelphia.
Butzer, P. L., & Stens, R. L. 1992. Sampling Theory for not necessarily bandlimited functions:
a historical overview. SIAM Reviews, 34(1), 40–53.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Duhamel, P., & Vetterli, M. 1990. Fast Fourier Transform: A tutorial review and a state of
the art. Signal Proc., 19(4), 259–299.
Gomes, J., & Velho, Luiz. 1997. Image Processing for Computer Graphics. SpringerVerlag.
Lim, Jae S. 1990. Two Dimensional Signal and Image Processing. New York: PrenticeHall.
BIBLIOGRAPHY 41
Loan, Van. 1996. Computational Frameworks for the Fast Fourier Transform. SIAM books,
Philadelphia.
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Weaver, J. 1989. Theory of Discrete and Continuous Fourier Transform. New York: John
Wiley & Sons.
Zayed, Ahmed. 1993. Advances in Shannon’s Sampling Theory. Boca Raton: CRC Press.
Chapter 4
Windowed Fourier Transform
In this chapter we will introduce a modiﬁcation in the deﬁnition of the Fourier transform in
order to obtain a transform with better localization properties in the timefrequency domain.
This transform will give us better results for the purposes of function representation.
4.1 A Walk in The Physical Universe
Our purpose is to obtain a transform that enables us to perform a local computation of
the frequency density. The inspiration for this transform is to analyze the audio analysis
performed by our auditory system. Consider for this an audio signal represented by a real
function f of one variable (time).
Real time analysis. The audio information we receive occurs simultaneously on time and
frequency. This means that the signal f is transformed by the auditory system in a signal
˜
f(t, ω) that depends on the time and the frequency.
Future sounds are not analyzed. This means that only values of f(t) for t ≤ t
1
can be
analyzed when computing the “transform”
˜
f(t, ω).
The auditory system has ﬁnite memory. That is, sounds that we have heard some
time ago do not inﬂuence the sounds that we hear in a certain instant of time. This means
that there exists a real number t
0
> 0 such that the computation of the “transform”
˜
f(t, ω)
depends only on the values of t on the interval [t −t
0
, t].
Mathematically, the last two properties show that the modulating function used to detect
frequencies in the computation of the “transform”
˜
f(t, ω) must have its values concentrated
in a neighborhood of t. We say that it is localized in time. D. Gabor, (Gabor, 1946), was the
ﬁrst to propose a transform with the above properties.
43
44 CHAPTER 4. WINDOWED FOURIER TRANSFORM
Figure 4.1: Modulating function.
4.2 The Windowed Fourier Transform
One method to obtain a localized modulating function consists in using an auxiliary function
g(u) to localize the modulating function e
−2iπωu
used in the Fourier transform in a certain
neighborhood of the time domain:
g
ω,t
(u) = g(u −t)e
−2πiωu
. (4.1)
This localization operator is illustrated in Figure 4.1. We take a function g(u), and for each
value of t ∈ R, we translate the origin to the point t and multiply the exponential by the
translated function g(u −t). If the function g is localized in the time domain, we obtain the
desired localization of the modulating function g(u −t)e
−2πiωu
.
From the above, the deﬁnition of our transform is given by
˜
f(ω, t) =
+∞
−∞
g(u −t)f(u)e
−2πiωu
du
=
+∞
−∞
g
t,ω
(u)f(u)du = 'g
t,ω
, f`. (4.2)
The transform f →
˜
f(t, ω) is called the windowed Fourier transform (WFT), or the short
time Fourier transform (STFT). Two important questions can be posed with respect to the
windowed Fourier transform
˜
f(t, ω):
Question 1. Is the transform
˜
f(t, ω) invertible?
Question 2. What is the image of the transform
˜
f(t, ω)?
The importance of these questions can be measured from the fact that the invertibility of
the Fourier transform is one of its strength. The underlying ideia of the importance of the
inverse transform is that if we are able to obtain a good representation using the transform,
its inverse will provide the reconstruction. The image of the transform is important because
it measures the scope of the representable functions.
We will discuss these two questions in the following sections.
4.2. THE WINDOWED FOURIER TRANSFORM 45
4.2.1 Invertibility of
˜
f(t, ω)
The problem of inverting the windowed Fourier transform consists in determining a function
f from its transformed function
˜
f(t, ω).
From (4.2) it follows that
˜
f(u, t) =
´
f
t
(u), where f
t
(u) = g(u−t)f(u). Applying the inverse
Fourier transform, we have
g(u −t)f(u) = f
t
(u) =
+∞
−∞
˜
f(ω, t)e
2πiωu
dω.
We can not divide by g(u −t) to get f(u), because the function g might as well assume zero
values. Multiplying both members of the equation g(u −t), and integrating in t we obtain
+∞
−∞
[g(u −t)[
2
f(u)dt =
+∞
−∞
+∞
−∞
e
−2πiωu
g(u −t)
˜
f(ω, t)dωdt.
That is,
f(u) =
1
[[g[[
2
ω,t
g(u −t)e
2πiωu
˜
f(ω, t)dω. (4.3)
As we did for the Fourier transform, equation (4.3) can be interpreted into two diﬀerent
ways:
1. It is an equation to compute f from its windowed Fourier transform
˜
f(ω, t);
2. It decomposes the function f as an inﬁnite superposition of localized waveforms
g
ω,t
(u) = g(u −t)e
2πiωu
.
These waveforms are called timefrequency atoms.
4.2.2 Image of the Windowed Fourier Transform
In this section we will discuss the second question posed before: What is the image space of
the windowed Fourier transform? We will only give some comments concerning the answer.
Given a function f its windowed Fourier transform
˜
f(ω, t) is a function of two variables.
It is possible to prove that if f ∈ L
2
(R), then
˜
f(ω, t)L
2
(R
2
). Also, it is possible to show that
the image of the transform
˜
f does not cover the whole space L
2
(R
2
). Therefore, the posed
question consists in characterizing the image set of the windowed Fourier transform. We will
not solve this problem here. A solution to it can be found in (Kaiser, 1994), page 56, or
Theorem 4.1 of (Mallat, 1998).
It is important to compare the result here with the analogous result for the Fourier trans
form: The Fourier transform is an isometry of the space L
2
(R). In particular its image is the
whole space.
46 CHAPTER 4. WINDOWED FOURIER TRANSFORM
4.2.3 WFT and Function Representation
Since our main focus is the problem of function representation, a natural question to be posed
now would be:
Question 3. Is it possible to obtain a function representation using the windowed Fourier
transform?
In fact since the windowed Fourier transform has good localization properties in the time
frequency domain, we should expect that the discretization of equation (4.3) would give good
discrete timefrequency atomic representations.
4.3 Timefrequency Domain
We have noticed that if f ∈ L
2
(R), then
˜
f(ω, t) ∈ L
2
(R
2
). Therefore the windowed Fourier
transform of a function is deﬁned on the domain (ω, t), called the timefrequency domain.
From the deﬁnition of the windowed Fourier transform (4.2) we know that if g is well
localized in time (i.e. g is small outside of a small time interval), then
˜
f is also well localized
in time. How about the frequency localization of
˜
f?
From equation (4.2) which deﬁnes the windowed Fourier transform, we have
˜
f(ω, t) = 'g
ω,t
, t` = 'ˆ g
ω,t
,
ˆ
f`,
where the second equation follows from Parseval’s identity. We conclude that if g has good
localization properties in the frequency domain (i.e. ˆ g is small outside an interval of frequency
ω), then the transform
˜
f is also localized in frequency.
Therefore the windowed Fourier transform enable us to analyze the function f in the time
frequency domain, in the sense that we have localized information both in time and frequency
domain. This results is completely in accordance with the problem we have discussed before:
Detect frequencies of the function f, and localize them on the time domain.
How precisely can we localize the information about f in the timefrequency domain? An
answer to this question is given below.
4.3.1 The Uncertainty Principle
From the previous section we could conclude that ﬁner and ﬁner analysis of a function f ∈
L
2
(R) could be obtained by using window functions g with very good localization properties
on the timefrequency domain (ω, t).
Unfortunately there is a limit to the localization precision in the timefrequency domain.
This limitation comes from a general principle that governs the timefrequency transforms.
This is the uncertainty principle which will be discussed now. In simply terms the statement
of this principle is: We can not obtain precise localization simultaneously in the time and
frequency domains. The intuition behind this principle is simple: To measure frequencies we
must observe the signal for some period of time. The more precision we need in the frequency
measurements the larger the time interval we have to observe.
4.3. TIMEFREQUENCY DOMAIN 47
Figure 4.2: Information cells.
In order to give a more quantitative statement of the uncertainty principle, we have to
deﬁne precisely the notion of “information localization” of a function. For this, we will suppose
that the norm L
2
(R) of the window function g is 1, that is, [[g[[
2
= 1. It follows from the
equation of Plancherel that [[ˆ g[[
2
= 1. We may consider g and ˆ g as probability distributions,
then the averages of g e ˆ g are computed by
t
0
=
+∞
−∞
t[g(t)[dt, and ω
0
=
+∞
−∞
ω[ˆ g(ω)[dω,
respectively. The size of the localization interval of g and ˆ g is given the standard deviation
T
2
=
+∞
−∞
(t −t
0
)
2
[g(t)[
2
dt,
and
Ω
2
=
+∞
−∞
(ω −ω
0
)
2
[ˆ g(ω)[
2
dω.
With the above deﬁnitions, the uncertainty principle states that
4πΩT ≥ 1.
Note that if g is well localized in frequency (Ω small) then T ≥ 1/4πΩ can not be small,
therefore g does not have good localization in time. The same reasoning applies for frequency
localization.
The localization of the signal in the timefrequency domain (ω, t) is represented geomet
rically by the rectangle of dimensions T Ω. This rectangle is called uncertainty window or
information cell of the transform. From the uncertainty principle, the area of this rectangle
is ≥ 1/4π (See Figure 4.2).
The uncertainty principle shows that do not exist coordinates in the timefrequency do
main. Or, to state this more precisely, the coordinates on the timefrequency domain are
not of great importance to the analysis of functions, unless for the fact that they are used to
measure the dimensions of the uncertainty windows.
48 CHAPTER 4. WINDOWED FOURIER TRANSFORM
Figure 4.3: Atoms in the timefrequency domain.
4.4 Atomic Decomposition
Our main goal is to provide better representations of a signal. The purpose of introducing
the windowed Fourier transform is to obtain representations in the timefrequency domain.
More precisely, given a function f we must obtain a decomposition
f =
¸
λ∈Ω
a
λ
g
λ
, (4.4)
where Ω is discrete, and the functions g
λ
have good localization properties in the time
frequency domain. The functions g
λ
are called timefrequency atoms, and the reconstruction
equation (4.4) is called atomic decomposition.
The atomic decomposition deﬁnes a representation operator
R(f) = (a
λ
)
λ∈Ω
∈
2
.
Each atom in this representation constitutes a basic element used to measure the frequency
density of the function in a small period of time. Each of these atoms is represented by
a rectangle whose sides indicate the measure of localization according to the uncertainty
principle. The degree of uncertainty in time and in frequency is indicated by the width and
height of the rectangle. The localization of the atom in the timefrequency domain is given
by the coordinates of the center of the rectangle, or by the coordinates of some of its vertices.
From the uncertainty principle the area of each rectangle is ≥ 1/4π.
We associate a gray color to each atom to indicate its energy in the decomposition. The
energy is directly related with the value of the coeﬃcients in the reconstruction equation. In
Figure 4.3 we depict some atoms. The associated signal to the atom on the left presents small
localization in frequencies and has small energy; the central atom has better localization of
frequencies (complete cycle) and therefore has more energy; the atom to the right has a good
frequency localization (several cycles are encompassed) and a high energy.
4.5. WFT AND ATOMIC DECOMPOSITION 49
Given a signal f represented by some ﬁnite atomic decomposition
f
N
(t) =
N−1
¸
k=0
a
N
(k)φ
N,k
(t),
this representation depicted by the corresponding rectangles of each atom φ
N,k
, and the
corresponding energy component a
N
(k)φ
N,k
.
In the case of representation by uniform sampling, we have an exact localization in time
and no localization in the frequency domain. The representation in the timefrequency domain
is illustrated in Figure 4.4(a), where the distribution between the vertical segments is given
by the sampling interval.
(a) (b)
Figure 4.4: (a) Point sampling. (b) Fourier sampling.
The atoms of the discrete Fourier transform
1, e
2πi
N
, . . . , e
2πi(N−1)s
N
constitute an orthonormal basis. We have N discrete values of frequency 0, s, 2s, . . . , (N−1)s.
These atoms have no time localization, therefore the atomic representation of a function using
this basis is localized in the frequency domain and has no time localization. This fact is
depicted in Figure 4.4(b).
4.5 WFT and Atomic Decomposition
The natural way to obtain atomic decompositions of the function f using the windowed Fourier
transform would be using the inverse transform (4.3) which writes f as a superposition of
indexed functions by the time frequency parameters. This equation gives us an indication that
by discretizing the time and frequency parameters we obtain representation/reconstruction
methods for functions f ∈ L
2
(R).
50 CHAPTER 4. WINDOWED FOURIER TRANSFORM
Figure 4.5: Uniform lattice in timefrequency domain.
In order to achieve this we should look for discrete versions of the windowed Fourier
transform. We ﬁx t
0
and ω
0
and we take discrete values of time t = nt
0
, and discrete values
of frequency ω = mω
0
, n, m ∈ Z. In the timefrequency domain we have the uniform lattice
∆
t
0
,ω
0
= ¦(mt
0
, nω
0
) ; m, n ∈ Z¦,
depicted in Figure 4.5
In this case we write the transform
˜
f in the form
˜
f
m,n
=
+∞
−∞
f(u)g(u −nt
0
)e
−2πimω
0
u
du, (4.5)
which is called discrete windowed Fourier transform (DWFT).
The discrete windowed Fourier transform uses a countable family of time frequency atoms
g
m,n
(u) = e
i2πmω
0
u
g(u −nt
0
).
If this family constitute a frame we can obtain a representation
f →('f, g
m,n
`)
m,n∈Z
,
of the function f. Moreover, f can be reconstructed from this representation using the
reciprocal frame ¯ g
m,n
, as described in Chapter 2:
f =
¸
m,n
'f, g
m,n
` ¯ g
m,n
.
Geometrically, this result means that when we position the information cells of the time
frequency atoms g
m,n
on the vertices of the lattice, we cover the whole timefrequency plane.
This fact is illustrated in Figure 4.6 for two distinct discretizations of the domain.
Now the crucial question to complement the above results is:
4.5. WFT AND ATOMIC DECOMPOSITION 51
Figure 4.6: Decompositions of the timefrequency domain using the DWFT.
Question 4. Does there exist frames g
m,n
with good localization properties in the time
frequency plane?
A detailed analysis of this question, with several related references, can be found in Chapter
4 of (Daubechies, 1992). We will give a brieﬁng of the results here:
1. If t
0
ω
0
> 2π, then frames g
m,n
do not exist.
2. If t
0
ω
0
= 2π, frames do exist but they do not have good localization properties in the
timefrequency domain. In fact, if we take
g(x) =
1 if x ∈ [0, 1]
0 if x < 0 ou x > 1,
then g
m,n
(x) is a basis. The same happens if we take g(x) = sen πx/πx.
3. If t
0
ω
0
< 2π, then there exists tight frames with good timefrequency localization prop
erties. A construction of such a tight frame is found in (Daubechies et al. , 1986).
From the point of view of timefrequency, the atomic decomposition of a function using the
discrete windowed Fourier transform, gives an uniform decomposition in rectangles, according
to the illustration on Figure 4.6. We will give some examples.
Example 7 (Sines with impulses). Consider the signal deﬁned by the function
f(t) = sin(2π516.12t) + sin(2π2967.74t) +δ(t −0.05) +δ(t −0.42).
This signal consists of a sum of two senoids with frequencies 516.12Hz e 2967.74Hz, with two
impulses of order 3 for time values of t
0
= 0.05s and t
1
= 0.42s. The graph of the signal is
shown in the image on the left of Figure 4.7. The graph of its Fourier transform is depicted
in the image on the right.
52 CHAPTER 4. WINDOWED FOURIER TRANSFORM
0 500 1000 1500 2000 2500 3000 3500 4000 4500
−2
−1
0
1
2
3
4
5
0 500 1000 1500 2000 2500
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Figure 4.7: Signal and its Fourier transform.
The analysis of this signal using the Fourier transform was done in Chapter 2. Our goal
here is to revisit this analysis using the windowed Fourier transform. For this, we use a
Hamming window with diﬀerent sizes. A Hamming window is deﬁned by a cosine curve
a +b cos t for convenient values of a and b. Its graph is shown in Figure 4.8.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 4.8: Hamming window.
The images (a), (b) and (c) of Figure 4.9 show the information cells of the atomic decom
position of the signal f using the discrete windowed Fourier transform. In the decomposition
shown in (a) we have used a Hamming window of width 32 (i.e. 32 samples) in the decompo
sition shown in (b) we used a Hamming window of width 64 (64 samples), and in (c) we have
used a Hamming window of width 256.
An analysis of the decompositions on the ﬁgure shows us the following:
• In (a) we see that two impulses were detected with a good localization in time. The
two sine waves also have been detected but the localization of their frequencies is not
good. Moreover several information cells show up which are not directly related with
the dominating frequencies of the signal.
• In (b) we have a better localization of the sine frequencies, but the information cells are
not able to distinguish the two impulses.
4.5. WFT AND ATOMIC DECOMPOSITION 53
Time
F
r
e
q
u
e
n
c
y
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
500
1000
1500
2000
2500
3000
3500
4000
(a)
Time
F
r
e
q
u
e
n
c
y
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
500
1000
1500
2000
2500
3000
3500
4000
(b)
Time
F
r
e
q
u
e
n
c
y
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
0
500
1000
1500
2000
2500
3000
3500
4000
(c)
Figure 4.9: Atomic decomposition using windowed Fourier transform.
• In (c) we have a good localization of the two impulses in time and good localization of
the sinusoidal frequencies. Also it should be noticed that most of the information cells
on the ﬁgure are related with the relevant frequencies of the signal. Nevertheless, it
should be remarked that if the two pulses were close together, we would not be able to
distinguish them.
The above example shows that making an adequate adjustment of the size of the window,
we can obtain satisfactory results in the analysis of the signal using the windowed Fourier
transform. The results are much better than those of the analysis using the Fourier transform
in the previous chapter.
Nevertheless we must take into consideration that the signal that we have used is very
simple: it contains only two distinguished frequencies of the sine waves, and the two impulses.
Now we will give an example of a signal which is very diﬃcult to be analyzed using the
windowed Fourier transform. The idea of constructing such an example is very simple: The
information cells of the windowed Fourier transform have constant width, thus if a signal has
frequencies which varies between diﬀerent orders of magnitude, it is very diﬃcult to obtain
an adequate width of a window that is able to detect all of the frequency values. We will give
an example illustrating this fact.
Example 8 (Quadratic chirp signal). Consider the signal deﬁned by the function
f(t) = sin
2
(t
2
).
The frequencies of this signal have a quadratic growth along the time. The graph of the
function f is shown in Figure 4.10(a). Since the signal has low frequencies close to the origin
t = 0 and they increase arbitrarily as the time increases, we conclude that a good atomic
decomposition of this signal should have timefrequency atoms as illustrated in Figure 4.10(b).
Figure 4.11 shows two atomic decompositions of the signal f using the windowed Fourier
transform. In both we have used Hamming windows of diﬀerent sizes. It should be noticed
54 CHAPTER 4. WINDOWED FOURIER TRANSFORM
0 500 1000 1500 2000 2500 3000 3500 4000 4500
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
(a)
Phase plane: CP Best Basis; Chirp Quadratico
Time
F
r
e
q
u
e
n
c
y
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(b)
Figure 4.10: (a) Quadratic chirp signal; (b) Ideal atomic decomposition of the signal.
that the relevant frequencies are detected and are correlated along a parabola, as we pre
dicted. Nevertheless several other information cells appear that are not related with relevant
frequencies of the signal. Moreover the information cells corresponding to the frequencies of
the signal do not posses good localization properties.
Time
F
r
e
q
u
e
n
c
y
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
500
1000
1500
2000
2500
3000
3500
4000
Time
F
r
e
q
u
e
n
c
y
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
500
1000
1500
2000
2500
3000
3500
4000
Figure 4.11: Timefrequency decomposition of chirp signal using DWFT.
The above example explicitizes the limitation of the windowed Fourier transform: It uses a
ﬁxed window size. In the next chapter we will introduce the wavelet transform in an attempt
to ﬁx this problem. The problem of representation and reconstruction of signals will be
revisited with this transform. As we will see, better atomic decompositions will be obtained.
4.6 Comments and References
The windowed Fourier transform was introduced in the literature by D. Gabor, (Gabor,
1946). D. Gabor used Gaussian windows, for this reason the windowed fourier transform with
gaussian windows is also called Gabor transforms on the literature.
BIBLIOGRAPHY 55
The reference (Kaiser, 1994) contains useful information with detailed proofs, the only
drawback is that it requires more familiarity of the reader with techniques from functional
analysis, and measure theory. In particular it generalizes the concept of frames for nondiscrete
frames, and along with the equations of the resolution of the identity provides elegant proofs
of several results.
The study of the windowed Fourier transform in (Daubechies, 1992) is quite complete.
Nevertheless frames for the windowed Fourier transform is discussed along with the problem
of wavelet frames. This could bring some diﬃculties for those with less experience.
Bibliography
Daubechies, I., Grossmann, A., & Meyer, Y. 1986. Painless nonorthogonal expansions. J.
Math. Phys., 1271–1283.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Gabor, D. 1946. Theory of Communication. J. of the IEEE, 429–457.
Kaiser, G. 1994. A Friendly Guide to Wavelets. Boston: Birkhauser.
Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press.
Chapter 5
The Wavelet Transform
In this chapter we will introduce the wavelet transform with the purpose of obtaining better
representation of functions using atomic decompositions in the timefrequency domain.
5.1 The Wavelet Transform
The windowed Fourier transform introduces an scale (the width of the window), and analyzes
the signal from the point of view of this scale. If the signal has important frequency details
outside of the scale, we will have problems in the signal analysis:
• If the signal details are much smaller then the width of the window, we will have a
problem similar to the one we faced with the Fourier transform: The details will be
detected but the transform will not localize them.
• If the signal details are larger than the width of the window they will not be detected
properly.
To solve this problem when we analyze a signal using the windowed Fourier transform, we
must deﬁne a transform which is independent of scale. This transform should not use a ﬁxed
scale, but the scale should vary.
The scale is deﬁned by the width of the modulation function. Therefore we must use a
modulation function which does not have a ﬁxed width. Moreover the function must have
good time localization. To achieve this we start from a function ψ(t) as a candidate of a
modulation function, and we obtain a family of functions from ψ by varying the scale: We ﬁx
p ≥ 0 and for all s ∈ R, s = 0, we deﬁne
ψ
s
(u) = [s[
−p
ψ(
u
s
) =
1
[s[
p
ψ(
u
s
). (5.1)
If ψ has width T (given as the standard deviation as explained in Chapter 3) then the width
of ψ
s
is sT. The modulation of the function ψ by the factor 1/[s[
2
, increases its amplitude
when the scale s decreases and viceversa. In terms of frequencies, we can state: For small
scales s, ψ
s
has high frequencies, and as s increases the frequency of ψ
s
decreases. This fact
is illustrated in Figure 5.1.
57
58 CHAPTER 5. THE WAVELET TRANSFORM
Figure 5.1: Scales of a function: (a) s < 1; (b) s = 1; (c) s > 1.
Analogous to what we did with the windowed Fourier transform of a function, we need to
localize each function ψ
s
in time. For this we deﬁne for each t ∈ R the function
ψ
s,t
(u) = ψ
s
(u −t) = [s[
−p
ψ(
u −t
s
) =
1
[s[
p
ψ(
u −t
s
). (5.2)
Note that if ψ ∈ L
2
(R), then ψ
s,t
∈ L
2
(R), and
[[ψ
s,t
[[
2
= [s[
1−2p
[[ψ[[
2
.
By taking p = 1/2, we have [[ψ
s,t
[[ = [[ψ[[.
Now we can deﬁne a transform on L
2
(R) in a similar way that we deﬁned the windowed
Fourier transform, using functions from the family ψ
s,t
as modulating functions. More pre
cisely, we have
˜
f(s, t) =
+∞
−∞
f(u)ψ
s,t
(u)du = 'ψ
s,t
, f`. (5.3)
This transform is known by the name of the wavelet transform.
As we did for the windowed Fourier transform, we can pose the following questions con
cerning the wavelet transform:
Question 1. Is the wavelet transform
˜
f(s, t) invertible?
Question 2. What is the image of the wavelet transform
˜
f(s, t)?
In the previous chapter we have explained the importance of these two questions for
function representation using timefrequency atoms.
5.1.1 Inverse of the Wavelet Transform
By deﬁnition we have
˜
f(s, t) = 'ψ
s,t
, f` = '
ˆ
ψ
s,t
,
ˆ
f`.
Moreover,
ˆ
ψ
s,t
(ω) = [s[
1−p
e
−2πiωt
ˆ
ψ(sω). (5.4)
5.1. THE WAVELET TRANSFORM 59
From this it follows that
˜
f(s, t) = [s[
1−p
+∞
infty
e
2πiωt
ˆ
ψ(s, ω)
ˆ
f(ω)dω (5.5)
= [s[
1−p
F
−1
ˆ
ψ(sω)
ˆ
f(ω)
, (5.6)
Where F indicated the Fourier transform.
Applying the Fourier transform to both sides of the equation we obtain
+∞
−∞
e
−2πiωt
˜
f(s, t)dt = [s[
1−p
ˆ
ψ(sω)
ˆ
f(ω). (5.7)
From the knowledge of
ˆ
f we can obtain f using the inverse transform. But we can not
simply divide the above equation by
ˆ
ψ, because it might have zero values. Multiplying both
sides of (5.7) by
ˆ
ψ(sω), and making some computations we obtain the result below:
Theorem 4. If ψ satisﬁes the condition
C =
+∞
−∞
[
ˆ
ψ(u)[
2
[u[
< ∞, (5.8)
then
f(u) =
1
C
R
2
[s[
2p−3
˜
f
s,t
(u)ψ
s,t
(u)dsdt. (5.9)
This theorem answers the ﬁrst question posed at the end of the previous section: The
wavelet transform is invertible and equation (5.9) reconstructs f from its wavelet transform.
As we did with the windowed Fourier transform, we can read equation (5.9) of the inverse
wavelet transform in two distinct ways:
1. The function f can be recovered from its wavelet transform;
2. The function f can be decomposed as a superposition of the timefrequency atoms
ψ
s,t
(u).
We have seen that the second interpretation is of great importance because, as in the case
of the windowed Fourier transform, it will lead us to obtain good representations by atomic
decompositions of the function f.
5.1.2 Image of the Wavelet Transform
In this section we will discuss the second question we asked before about the image of the
wavelet transform.
The wavelet transform, similarly with the windowed Fourier transform, takes a function
f ∈ L
2
(R) into a function
˜
f(s, t) of two variables. A natural question consist in computing
the image of the transform.
The interested reader should consult (Kaiser, 1994), page 69. Besides characterizing the
image space, this reference brings a proof that the wavelet transform deﬁnes an isometry over
its image. We will not go into details of the computation here.
60 CHAPTER 5. THE WAVELET TRANSFORM
5.2 Filtering and the Wavelet Transform
Equation (5.3) that deﬁnes the wavelet transform can be written as a convolution product
˜
f(s, t) = f ∗ ψ
s
(u),
where ψ
s
(u) is deﬁned in (5.1). Thus the wavelet transform is a linear spaceinvariant ﬁlter.
In this section we will discuss some properties of the wavelet ﬁlter.
The condition (5.8) that appears in the hypothesis of the Theorem 4 is called admissibility
condition. A function ψ that satisﬁes this condition is called a wavelet.
From the admissibility condition it follows that
lim
u→0
ˆ
ψ(u) = 0.
If
ˆ
ψ(u) is continuous, then
ˆ
ψ(0) = 0, that is,
+∞
−∞
ψ(u)du = 0.
Geometrically, this condition states that the graph of the function ψ must oscillate so as to
cancel positive and negative areas in order to have integral zero. Therefore the graph of ψ
has the form of a wave. In fact since ψ should have good time localization properties it has
a form of a “small wave” (see Figure 5.2). That is why ψ is named by wavelet.
Figure 5.2: Graph of a wavelet.
Another important conclusion can be drawn from the above computations. Since
ˆ
ψ(u) ∈
L
2
(R), then
lim
u→0
ˆ
ψ(u) = 0.
Along with the fact that
ˆ
ψ(0) = 0, we conclude that the graph of the Fourier transform
ˆ
ψ is
as depicted in Figure 5.3(a).
If
ˆ
ψ has a fast decay when u → 0 and u → ∞, then
ˆ
ψ(u) is small outside of a small
frequency band α ≤ [u[ ≤ β (see Figure 5.3(b)). It follows from equation (5.4) that
ˆ
ψ
s,t
≈ 0
outside of the frequency band
α
[s[
≤ [u[ ≤
β
[s[
.
5.2. FILTERING AND THE WAVELET TRANSFORM 61
(a)
(b)
Figure 5.3: Fourier transform of a wavelet.
Moreover, from equation (5.6) the wavelet transform
˜
f does not contain information about
f outside of this spectrum interval. In sum, the computations above show that “the wavelet
transform is a linear, time invariant bandpass ﬁlter”.
The next two examples are taken from (Kaiser, 1994).
Example 9 (Blur Derivative). Consider a function φ of class C
∞
, satisfying the conditions
φ ≥ 0;
R
φ(u)du = 1;
R
uφ(u)du = 0;
R
u
2
φ(u)du = 1.
That is, φ is a probability distribution with average 0 and variance (width) 1. Suppose that
lim
u→+∞
∂
n−1
φ
∂u
n−1
(u) = 0.
Deﬁning
ψ
n
(u) = (−1)
n
∂
n
φ
∂u
n
(u),
we have,
R
ψ
n
(u)du = 0.
62 CHAPTER 5. THE WAVELET TRANSFORM
That is, ψ
n
satisﬁes the admissibility condition (5.8). Therefore we can deﬁne a wavelet
transform
˜
f(s, t) =
R
ψ
n
s,t
(u)f(u)du, (5.10)
where
ψ
n
s,t
(u) =
1
s
ψ
n
(
u −t
s
).
(We are taking p = 1 in equation (5.2) that deﬁnes ψ
s,t
(u)). In an analogous way we deﬁne,
φ
s,t
(u) =
1
s
φ(
u −t
s
).
From the deﬁnition of ψ
n
we have that
ψ
−n
s,t
(u) = (−1)
n
s
−n
∂
n
φ
s,t
∂u
n
(u) = s
−n
∂
n
φ
s,t
∂t
n
(u). (5.11)
From equations (5.10) and (5.11) it follows that
˜
f(s, t) = s
−n
∂
n
∂t
n
R
φ
s,t
(u)f(u)du. (5.12)
The above integral is a convolution product of the function f with the function φ
s,t
, there
fore it represents a lowpass ﬁltering linear timeinvariant ﬁltering operation of the function
f, which is dependent of the scale s. We will denote this integral by f(s, t). Therefore we
have
˜
f(s, t) = s
−n
∂
n
f(s, t)
∂t
n
, (5.13)
that is, the wavelet transform of f is the nth time derivative of the average of the function
f on scale s. This derivative is known in the literature by the name of blur derivative.
We know that the nnth derivative of f measures the details of f in the scale of its
deﬁnition. Therefore, equation (5.13) shows that the wavelet transform
˜
f(s, t) gives the detail
of order n of the function f, in the scale s. Keeping this wavelet interpretation in mind is
useful, even when the wavelet does not come from a probability distribution.
Example 10 (The Sombrero Wavelet). We will use a particular case of the previous ex
ample to deﬁne a wavelet transform. Consider the Gaussian distribution
φ(u) =
1
√
2π
e
−u
2
/2
,
with average 0 and variance 1. The graph of this function is depicted in the image on the left
of Figure 5.4. Using the notation of the previous example, we have
ψ
1
(u) = −φ
(u) =
1
√
2π
ue
−u
2
/2
,
5.2. FILTERING AND THE WAVELET TRANSFORM 63
Figure 5.4: Graph of the sombrero wavelet.
and
ψ
2
(u) = φ
(u) =
1
√
2π
(u
2
−1)e
−u
2
/2
.
The function −ψ
2
is known as the “sombrero” function, because of the shape of its graph,
shown in the right of Figure 5.4.
From the previous example it follows that we can use the sombrero function to deﬁne a
wavelet transform. We will use this wavelet to illustrate the ﬂexibility of the wavelet transform
in analyzing frequencies of a signal. For this, consider the signal whose graph is shown in
Figure 5.5.
Figure 5.5: Signal f to be analyzed(Kaiser, 1994).
This signal has high frequencies localized in the neighborhood of t = 50, and t = 150.
From time t = 280, the signal has a chirp behavior: a continuum of increasing frequencies. In
this region the signal is deﬁned by the function
f(t) = cos(t
3
).
We know already that the windowed Fourier transform is not adequate to analyze signals
with this behavior. Figure 5.6 shows the graph of the signal and the graph of the wavelet
transform for 5 distinct values of the scale s (the scale decreases from top to bottom).
Note that the frequencies associated to the sudden change of the signal at time t = 50
and time t = 150 are detected by the wavelet transform. Moreover, as the scale s decreases
the high frequencies of the chirp signal cos(t
3
) are also detected.
64 CHAPTER 5. THE WAVELET TRANSFORM
Figure 5.6: The wavelet transform(Kaiser, 1994).
5.3 The Discrete Wavelet Transform
In the study of the windowed Fourier transform the timefrequency domain was discretized
using an uniform lattice
∆
(t
0
,ω
0
)
= ¦(mt
0
, nω
0
); m, n ∈ Z¦,
because of the constant width of the timefrequency atoms. The wavelet transform is deﬁned
on the timescale domain. A natural question is:
Question 3. How to discretize the timescale domain in such a way to obtain a discrete
wavelet transform?
We known that the scaling operation acts in a multiplicative way, that is, composing
two consecutive scalings is attained by multiplying each of the scale factors. Therefore the
discretization of the scaling factor is simple: We ﬁx an initial scale s
0
> 1, and we consider
the discrete scales
s
m
= s
m
0
, m ∈ Z.
Positive values of m produce scales larger than 1, and negative values of m produce scales
less than 1.
How to discretize the time? Initially we should observe that we must obtain a lattice
in the timescale domain in such a way that when we sample the wavelet transform
˜
f(s, t)
on this lattice, we are able to reconstruct the function f from the timescale atoms
˜
f
m,n
,
with minimum redundancy. As the wavelet width changes with the scale, we must correlate
the time with the scale discretization: As the scale increases the width of the wavelet also
increases, therefore we can take samples further apart in the time domain. On the other hand,
5.3. THE DISCRETE WAVELET TRANSFORM 65
when the width of the wavelet decreases with a reduction of the scale, we must increase the
frequency sampling.
To obtain the correct correlation between the scale and time discretization we observe
that an important property of the wavelet transform is: The wavelet transform is invariant
by change of scales. This statement means that if we make a change of scale in the function
f and simultaneously change the scale of the underlying space by the same scaling factor, the
wavelet transform does not change. More precisely, if we take
f
s
0
(t) = s
−1/2
0
f(
t
s
0
),
then
˜
f
s
0
(s
0
s, s
0
t) =
˜
f(s, t).
Invariance by changing of scale constitutes an essential property of the wavelet transform.
It is important that this property be preserved when we discretize the wavelet, so as to be
also valid for the discrete wavelet transform. In order to achieve this goal, when we pass from
the scale s
m
= s
m
0
to the scale s
m+1
= s
m+1
0
, we must also increment the time by the scaling
factor s
0
. In this way, we can choose a time t
0
and take the length of the sampling time
intervals as ∆t = s
m
0
t
0
. Therefore, for each scale s
m
0
the time discretization lattice is
t
m,n
= ns
m
0
t
0
, n ∈ Z.
Finally, the discretization lattice in the timescale domain is deﬁned by
∆
s
0
,t
0
= ¦(s
m
0
, ns
m
0
t
0
) ; m, n ∈ Z¦.
Example 11 (Dyadic Lattice). We will give a very important example of a wavelet dis
cretization using s
0
= 2 (dyadic lattice). We have
∆
2,t
0
= ¦(2
m
, n2
m
t
0
) ; m, n ∈ Z¦.
The vertices of this lattice are shown in Figure 5.7(a). This lattice is called hyperbolic lattice
because it is a uniform lattice in hyperbolic geometry (only the points are part of the lattice).
To obtain a timefrequency lattice, we must observe that the frequency is the inverse of
the scale. In this manner, for a given initial frequency ω
0
the lattice will be given by
∆
2ω
0
,t
0
= ¦(2
−m
ω
0
, n2
−m
t
0
) ; m, n ∈ Z¦.
The vertices of this lattice are shown in Figure 5.7(b).
66 CHAPTER 5. THE WAVELET TRANSFORM
(a) (b)
Figure 5.7: (a) Timescale lattice.(b) Timefrequency lattice.
5.3.1 Function Representation
From the point of view of atomic decomposition the timefrequency atoms deﬁne a tiling of
the timefrequency domain in rectangles as shown in Figure 5.8.
Figure 5.8: Timefrequency decomposition using wavelets.
The discretization of the wavelet transform
˜
f(s, t) = 'f, ψ
s,t
(u)` in the timescale lattice
is given by
˜
f
m,n
= 'f, ψ
m,n
(u)`,
5.3. THE DISCRETE WAVELET TRANSFORM 67
where
ψ
m,n
(u) = ψ
s
m
0
,nt
0
s
m
0
(u) (5.14)
= s
−m/2
0
ψ
u −nt
0
s
m
0
s
m
0
(5.15)
= s
−m/2
0
ψ(s
−m
0
u −nt
0
). (5.16)
In this context we can pose again the two questions which motivated the process of deﬁning
a discrete wavelet transform:
Question 4. Is the sequence 'f, ψ
m,n
`, m, n ∈ Z an exact representation of the function f?
Question 5. Is it possible to reconstruct f from the family of wavelet timefrequency atoms
ψ
m,n
?
A positive answer to these two questions would give us atomic decompositions of the
function f using a family ψ
m,n
. of discrete wavelets.
There are several directions we could take to answer the two questions above. Based on
the representation theory discussed in Chapter 2, two natural questions in this direction are:
Question 6. Is it possible to deﬁne a lattice such that the corresponding family ¦ψ
m,n
¦
constitutes an orthonormal basis of L
2
(R)?
Question 7. Is it possible to deﬁne lattices for which the family ¦ψ
m,n
¦ is a frame?
If we have orthonormal basis of wavelets or a frame, we know from Chapter 2 that the
answer to the two questions posed above are positive.
Chapter 3 of (Daubechies, 1992) brings a comprehensive discussion of frames of wavelets.
The explicit construction of some wavelet frames is given. In the Chapters to follow we will
discuss the construction of diﬀerent basis of wavelets.
Example 12 (Haar Basis). Consider the function
ψ(x) =
1 if x ∈ [0, 1/2)
−1 if x ∈ [1/2, 1)
0 if x < 0 ou x > 1.
The graph of f is shown in Figure 5.9. This function satisﬁes the admissibility condition (5.8).
It is possible to show that the set ψ
m,n
, where
ψ
m,n
(u) = 2
−m/2
ψ(2
−m
u −n), m, n ∈ Z,
constitutes an orthonormal basis of L
2
(R). Therefore we have an orthonormal basis of
wavelets. A direct, and long, proof of this fact is found in (Daubechies, 1992), Section 1.3.3.
The orthonormality of the set ψ
m,n
is easy to proof. The fact that the set generates the space
L
2
(R) is more complicated. This will follow as a consequence of the theory of multiresolution
analysis that we will study in next chapter.
68 BIBLIOGRAPHY
Figure 5.9: Haar wavelet.
5.4 Comments and References
There are several possibilities of extending the wavelet transform to functions of several vari
ables, i.e. L
2
(R
n
). The interested reader should consult (Daubechies, 1992), page 33, or
(Mallat, 1998).
The beautiful examples 9 and 10 of this chapter were taken from (Kaiser, 1994).
Bibliography
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Kaiser, G. 1994. A Friendly Guide to Wavelets. Boston: Birkhauser.
Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press.
Chapter 6
Multiresolution Representation
In the introductory chapter we stated that two properties of wavelets were responsible for their
applicability in the study of functions. One of these properties is the wavelet timefrequency
atoms we studied in the previous chapter. The second property is the relationship of wavelets
with multiresolution representation. This relationship will be exploited in two diﬀerent ways:
• From one side it allows us the use of wavelets to obtain multiresolution representations
of functions.
• On the other hand it will be used as a tool to construct wavelets.
6.1 The Concept of Scale
Our perception of the universe uses diﬀerent scales: Each category of observations is done in
a proper scale. This scale should be adequate to understand the diﬀerent details we need.
In a similar manner, when we need to represent an object, we try to use a scale where the
important details can be captured in the representation.
A clear and well known example of the use of scales occurs on maps. Using a small scale
we can observe only macroscopic details of the mapped regions. By changing the scale we can
observe or represent more details of the object being represented on the map.
Multiresolution representation is a mathematical model adequate to formalize the repre
sentation by scale in the physical universe. As we will see, this problem is intrinsically related
to the wavelets.
The idea of scale is intrinsically related with the problem of point sampling of a signal.
We call sampling frequency the number of samples in the unit of time. The length of the
sample interval is called the sampling period. When we sample a signal using a frequency
2
j
, we are ﬁxing a scale to represent the signal: Details (frequencies) of the signal that are
outside of the scale magnitude of the samples will be lost in the sampling process. On the
other hand, it is clear that all of the details of the signal captured in a certain scale will also
be well represented when we sample using a higher scale, 2
k
, k > m.
These facts are well translated mathematically by the sampling theorem of Shannon
Whittaker that relates the sampling frequency with the frequencies present on the signal.
69
70 CHAPTER 6. MULTIRESOLUTION REPRESENTATION
6.2 Scale Spaces
How to create a mathematical model to formalize the problem of scaling representation in
the physical universe? The relation between sampling and scaling discussed above shows us
the way. For a given integer number j, we create a subspace V
j
⊂ L
2
(R), constituted by the
functions in L
2
(R) whose details are well represented in the scale 2
j
. This means that these
functions are well represented when sampled using a sampling frequency of 2
j
.
The next step consists in creating a representation operator that is able to represent any
function f ∈ L
2
(R) in the scale 2
j
. A simple and eﬀective technique consists in using a
representation by orthogonal projection. This is the Galerkin representation we discussed
in Chapter 2. A simple and eﬀective way to compute this representation is to obtain an
orthonormal basis of V
j
. But at this point we will demand more than that to make things
easier: We will suppose that there exists a function φ ∈ L
2
(R) such that the family of functions
φ
j,k
(u) = 2
−j/2
φ(2
−j
u −k), j, k ∈ Z, (6.1)
is an orthonormal basis of V
j
.
Notice that we are using here a process similar to the one we used when we introduced
the wavelet transform: We deﬁne diﬀerent scales of φ producing the continuous family
φ
s
(u) =
1
[s[
1/2
φ(
u
s
).
The width of φ and φ
s
are related by
width(φ) = s width(φ
s
).
Thus, as the scale increases or decreases, the width of φ
s
does the same. Equation (6.1) is
obtained by discretizing the parameter s, taking s = 2
j
, j ∈ Z. Also, we have demanded that
the translated family
φ
j,k
= φ
2
j (u −k) = 2
−j/2
φ(2
−j
u −k)
is an orthonormal basis of V
j
. Note that when j decreases, the width of φ
j,k
also decreases,
and the scale is reﬁned. This means that more features of f are detected in its representation
on the space V
j
.
The representation of a function f ∈ L
2
(R) by orthogonal projection in V
j
is given by
Proj
V
j
(f) =
¸
k
'f, φ
j,k
`φ
j,k
.
We want the representation sequence ('f, φ
j,k
`) to contain samples of the function f in the
scale 2
j
. In order to attain this we know from Theorem 2 of Chapter 3 that the representation
sequence ('f, φ
j,k
`)
j,k∈Z
is constituted by the samples of a ﬁltered version of the signal f. More
precisely,
'f, φ
j,k
` = F(k),
where F is obtained from f by sampling with a ﬁlter of kernel φ
j,k
: F = f ∗ φ
j,k
. In order
that the elements of the representation sequence are close to the samples of f, the ﬁlter kernel
6.2. SCALE SPACES 71
φ
j,k
must deﬁne a lowpass ﬁlter. This can be attained by demanding that
ˆ
φ(0) = 1, because
ˆ
φ(ω) approaches 0 when ω →±∞. The graph of φ is depicted in Figure 6.1. With this choice
of φ, representing a function at scale 2
j
amounts to sample averages of f over neighborhoods
of width 2
j
.
The space V
j
is called space of scale 2
j
, or simply scale space.
Figure 6.1: Lowpass ﬁlter.
It is very important that we are able to change from a representation in a certain scale
to a representation on another scale. For this we must answer the question: How are the
diﬀerent scale spaces related?
Since the details of the signal which appear on scale 2
j
certainly must appear when we
represent the signal using a smaller scale 2
j−1
, we must have
V
j
⊂ V
j−1
. (6.2)
Given a function f ∈ L
2
(R), a natural requirement is
f ∈ V
j
if, and only if, f(2u) ∈ V
j−1
. (6.3)
In fact, the scaling of the variable of f by 2 reduces the width of f by the factor of 1/2
(see Figure 6.2). Therefore the details of f go to a ﬁner scale. Applying successively the
Figure 6.2: Scaling of f by an scale factor of 2.
condition in (6.3), we obtain
f ∈ V
j
if, and only if, f(2
j
u) ∈ V
0
.
72 CHAPTER 6. MULTIRESOLUTION REPRESENTATION
That is, all spaces are scaled version of the space V
0
. In particular, from the fact that φ
j,k
in
equation (6.1), is an orthonormal basis of V
j
, we conclude that
φ
0,k
(u) = φ(u −k)
is an orthonormal basis of the scale space V
0
.
The space L
2
(R), our universe of the space of functions, contains all of the possible scales.
This is reﬂected in the relation
¸
j∈Z
V
j
= L
2
(R).
On the other hand, we have
¸
j∈Z
V
j
= ¦0¦.
In eﬀect, this expression says that the null function is the only function that can be well
represented in every scale. In fact it should be observed that any constant function can be
represented in any scale, nevertheless the only constant function that belongs to L
2
(R) is the
null function.
6.2.1 A Remark About Notation
It is important here to make a remark about the index notation we use for the scale spaces,
because there is no uniformity on the literature. We use the notation of decreasing indices
V
1
⊂ V
0
⊂ V
−1
⊂ V
−2
.
From the discussion above, this notation is coherent with the variation of the scale when we
pass from one scale space to the other: As the indices decrease, the scale is reﬁned, and the
scale spaces get bigger.
If we use a notation with increasing indices
V
−1
⊂ V
0
⊂ V
1
,
which also appears on the literature, than the base φ
j,k
of the scale space V
j
should be
constituted by the functions
φ
j,k
(x) = 2
j/2
φ(2
j
x −k).
This is rather confusing because it is not in accordance with the notation used when we
discretized wavelets.
6.2.2 Multiresolution Representation
The scale spaces and their properties that we studied above deﬁne a multiresolution repre
sentation in L
2
(R). We will resume them into a deﬁnition to facilitate future references:
Deﬁnition 1 (Multiresolution Representation). We deﬁne a multiresolution represen
tation in L
2
(R) as a sequence of closed subspaces V
j
, j ∈ Z, of L
2
(R), satisfying the following
properties:
6.3. A PAUSE TO THINK 73
(M1) V
j
⊂ V
j−1
;
(M2) f ∈ V
j
if, and only if, f(2u) ∈ V
j−1
.
(M3)
¸
j∈Z
V
j
= ¦0¦.
(M4)
¸
j∈Z
V
j
= L
2
(R).
(M5) There exists a function φ ∈ V
0
such that the set ¦φ(u − k); k ∈ Z¦ is an orthonormal
basis of V
0
.
The function φ is called the scaling function of the multiresolution representation. Each of
the spaces V
j
is called scale spaces, or, more precisely, space of scale 2
j
.
Example 13 (Haar Multiresolution Analysis). Consider the function
φ(t) = χ
[0,1]
=
0 if x < 0 ou t ≥ 1
1 if x ∈ [0, 1)
It is easy to see that φ is a scale function of a multiresolution representation. In this case,
V
j
= ¦f ∈ L
2
(R); f[[2
j
k, 2
j
(k + 1)] = constant, k ∈ Z¦.
That is, the projection of a function f on the scale space V
j
is given by a function which is
constant on the intervals [2
j
k, 2
j
(k + 1)]. This is the Haar multiresolution representation.
We should notice that conditions (M1), . . . (M5), that deﬁne a multiresolution represen
tation are not independent. In fact it is possible to prove that condition (M3) follows from
(M1), (M2) e (M5). Moreover, condition (M5) can be replaced by the weaker condition that
the set ¦φ(u − k)¦ is a Riesz basis. Also, the reader might have noticed that we have not
imposed that the scale function φ satisﬁes the condition
ˆ
φ(0) = 1 (as we know, this condition
guarantees that φ is a lowpass ﬁlter). It can be proved that this lowpass ﬁlter condition
follows from (M4). For a proof of all of these facts we suggest consulting (Hernandez & Weiss,
1996) or (Daubechies, 1992). We will return to this problem with more details in next chapter
about construction of wavelets.
6.3 A Pause to Think
How to interpret geometrically the sequence of nested scale spaces in the deﬁnition of a
multiresolution representation?
In general, visualizing subspaces of some space of functions is not an easy task. Neverthe
less, in this case a very informative visualization of the nested sequence of scale space can be
obtained in the frequency domain.
Indeed, the orthogonal projection of a function f ∈ L
2
(R) in V
j
is obtained using a ﬁltering
operation of f with the diﬀerent kernels φ
j,k
, k ∈ Z which deﬁne lowpass ﬁlters. Indicating
the cutting frequency of these ﬁlters by α
j
(see Figure 6.3), we conclude that each space V
j
is constituted by functions whose frequencies are contained in the interval [−α
j
, α
j
], α
j
> 0.
74 CHAPTER 6. MULTIRESOLUTION REPRESENTATION
When we go from the space V
j
to the space V
j−1
we change from the scale 2
j
to a ﬁner
scale 2
j−1
. Therefore the frequency band increases to an interval [−α
j−1
, α
j−1
]. The graph
of the spectrum of φ
j−1,k
is the dotted curve in Figure 6.3. The scale space V
j−1
consists of
the set of all the functions whose spectrum is contained in [−α
j−1
, α
j+1
].
Figure 6.3: Spectrum of the scaling function.
For each space V
j
, with scale 2
j
, we have the representation operator R
j
: L
2
(R) → V
j
,
given by the orthogonal projection over V
j
R
j
(f) = Proj
V
j
(f) =
¸
k
'f, φ
j,k
`φ
j,k
.
From condition (M4) of the deﬁnition of a multiresolution representation, we have
lim
j→∞
R
j
(f) = f, (6.4)
that is, as the scale gets ﬁner we get a better representation of the function f. This is
illustrated in Figure 6.4 (from (Daubechies, 1992)) we show a function f, and its representation
on the spaces of scale V
0
e V
−1
of the Haar multiresolution representation.
Figure 6.4: Scale approximations of a function(Daubechies, 1992).
6.4. MULTIRESOLUTION REPRESENTATION AND WAVELETS 75
There is a diﬀerent, and very important way, to interpret equation (6.4) Consider the
graph representation of the space V
j
on Figure 6.5. We see that the space V
j−1
is obtained
from the space V
j
by adding all of the functions from L
2
(R) with frequencies in the band
[α
j
, α
j−1
] of the spectrum. We indicate this “detail space” by W
j
. It follows immediately
that W
j
is orthogonal to V
j
. Therefore we have
V
j−1
= V
j
⊕W
j
.
Figure 6.5: Frequency band between V
j
and V
j−1
.
The space W
j
contains the details of the signal in the scale V
j
. The above equation says
that a function represented on a ﬁner scale space V
j−1
is obtained from the representation on
a coarser scale space V
j
, by adding details. These details can be obtained using a bandpass
ﬁltering, whose passband is exactly the interval [α
j
, α
j−1
]. We have seen that the wavelets
constitute linear timeinvariant bandpass ﬁlters. Therefore it seems natural that there might
exist some relation between the detail spaces and the wavelets. We will discuss this relation
“with details” in next section.
6.4 Multiresolution Representation and Wavelets
We have proved that given two consecutive scale spaces V
j
⊂ V
j−1
, the orthogonal complement
W
j
of V
j
in V
j−1
could be obtained using a bandpass ﬁlter deﬁned on L
2
(R). In this section
we will show that this complementary space is in fact generated by an orthonormal basis of
wavelets.
For every j ∈ Z, we deﬁne W
j
as the orthogonal complement of V
j
in V
j−1
. We have
V
j−1
= V
j
⊕W
j
.
We remind that the best way to visualize the above equality is by observing the characteri
zation of these spaces on the frequency domain (Figure 6.5).
It is immediate to verify that W
j
is orthogonal to W
k
, if j = k. Therefore by ﬁxing J
0
∈ Z,
for every j < J
0
we have (see Figure 6.6)
V
j
= V
J
0
⊕
J
0
−j
k=0
W
J
0
−k
. (6.5)
76 CHAPTER 6. MULTIRESOLUTION REPRESENTATION
We should remark that because of the dyadic scales used in the discretization, the frequency
bands do not have uniform length, they are represented in the ﬁgure using logarithmic scale.
Figure 6.6: Frequency bands between V
j
and V
Jo−j
.
In sum, equation (6.5) says that the signals whose spectrum is in the frequency band of
V
j
, is the sum of the signals with frequency band in V
J
0
with those signals whose frequency
band are in W
J
0
, W
J
0
−1
, . . . , W
j
. All of the subspaces involved in this sum are orthogo
nals. If J
0
, k → ∞, if follows from conditions (M3) and (M4) that deﬁne a multiresolution
representation that
L
2
(R) =
j∈Z
W
j
,
that is, we obtain a decomposition of L
2
(R) as a sum of orthogonal subspaces.
We have seen that the projection of a function f in each subspace W
j
could be obtained
using a bandpass ﬁlter. In fact, this ﬁltering process can be computed by projecting f on an
orthogonal basis of wavelets. This fact is a consequence of the theorem below:
Theorem 5. For each j ∈ Z there exists an orthonormal basis of wavelets ¦ψ
j,k
, k ∈ Z¦ of
the space W
j
.
We will sketch the proof of the theorem because it has a constructive nature which will
provide us with a recipe to construct orthonormal basis of wavelets.
Basis of W
0
. Initially we observe that the spaces W
j
inherit the scaling properties of the
scale spaces V
j
. In particular,
f(u) ∈ W
j
if, and only if, f(2
j
u) ∈ W
0
. (6.6)
For this reason, it suﬃces to show that there exists a wavelet ψ ∈ W
0
such that the set
¦ψ(u −k)¦ is an orthonormal basis of W
0
. In fact, in this case, it follows from (6.6) that the
set
¦ψ
j,k
(u) = 2
−j/2
ψ(2
−j
u −k)¦
is an orthonormal basis of W
j
.
6.4. MULTIRESOLUTION REPRESENTATION AND WAVELETS 77
Lowpass ﬁlter and scaling function. Since φ ∈ V
0
⊂ V
−1
, and also φ
−1,k
is an orthonor
mal basis of V
−1
, we have
φ =
¸
k
h
k
φ
−1,k
, (6.7)
where
h
k
= 'φ, φ
−1,k
`, and
¸
k∈Z
[[h
k
[[
2
= 1.
Substituting φ
−1,k
(u) =
√
2φ(2u −k) in (6.7 we obtain
φ(x) =
√
2
¸
k
h
k
φ(2x −k).) (6.8)
Applying the Fourier transform to both sides of this equation, we have
ˆ
φ(ξ) = m
o
(
ξ
2
)
ˆ
φ(
ξ
2
), (6.9)
where
m
0
(ξ) =
1
√
2
¸
k
h
k
e
−ikξ
.
Note in equation (6.9) that
ˆ
φ(
ξ
2
) there exists a frequency band which has twice the size of
the frequency band of φ(ξ). Therefore, it follows from (6.8) that the function m
0
is a lowpass
ﬁlter. The function m
0
is called de lowpass ﬁlter of the scaling function φ. It is not diﬃcult
to see that m
0
is periodic with period 2π.
Characterization of W
0
. Now we need to characterize the space W
0
. Given f ∈ W
0
, since
V
−1
= V
0
⊕W
0
, we conclude that f ∈ V
−1
and f is orthogonal to V
0
. Therefore
f =
¸
n
f
n
φ
−1,n
, (6.10)
where
f
n
= 'f, φ
−1,n
`.
Computations similar to the ones we did to obtain the lowpass ﬁlter m
0
of the scaling
function, give us the equation
ˆ
f(ξ) = m
f
(
ξ
2
)
ˆ
φ(
ξ
2
), (6.11)
where
m
f
(ξ) =
1
√
2
¸
n
f
n
e
−inξ
.
After some computations, we can rewrite the equation (6.11) in the form
ˆ
f(ξ) = e
iξ
2
m
0
ξ
2
+π
ν(ξ)
ˆ
φ(
ξ
2
), (6.12)
where ν is a periodic function of period 2π.
78 CHAPTER 6. MULTIRESOLUTION REPRESENTATION
Choosing the Wavelet. Equation (6.12) characterizes the functions from W
0
using the
Fourier transform, up to a periodic function ν. A natural choice is to deﬁne a wavelet ψ ∈ W
0
such that
ˆ
ψ(ξ) = e
−iξ
2
m
0
ξ
2
+π
ˆ
φ(
ξ
2
). (6.13)
Taking this choice, from equation (6.12), it follows that
ˆ
f(ξ) =
¸
k
ν
k
e
−ikξ
ˆ
ψ(ξ),
and applying the inverse Fourier transform, we have
f(x) =
¸
k
ν
k
ψ(x −k).
We need to show that deﬁning ψ by the equation (6.13), ψ
0,k
is indeed an orthonormal
basis of W
0
. We will not give this proof here.
Details of the above proof can be found on (Daubechies, 1992) or (Hernandez & Weiss,
1996).
6.5 A Pause... to See the Wavescape
If V
j
is the scale space 2
j
we have V
j−1
= V
j
⊕ W
j
. We know that W
j
has an orthonormal
basis of wavelets ¦ψ
j,k
, k ∈ Z¦, therefore if R
j
is the representation operator on the scale space
V
j
, we have, for all f ∈ L
2
(R),
R
j−1
(f) = R
j
(f) +
¸
k∈Z
'f, ψ
j,k
`ψ
j,k
. (6.14)
The second term of the sum represents the orthogonal projection of the signal f on the
space W
j
and it will be denoted by Proj
W
j
(f). The terms of this representation sequence are
obtained using the discrete wavelet transform.
We know that the wavelet transform is a bandpass ﬁltering operation, ant the scale spaces
allow us to represent a function f in diﬀerent resolutions. When we obtain a representation of
f in a certain scale 2
j
, we are loosing details of the signal with respect with its representation
in the scale 2
j−1
. The lost details are computed by the orthogonal projection on the space
W
j
, that is,
Proj
W
j
(f) =
¸
k∈Z
'f, ψ
j,k
`ψ
j,k
, (6.15)
which is a representation of the signal f in the basis of wavelets of the space W
j
.
6.6. TWO SCALE RELATION 79
It is useful to interpret the decomposition V
j−1
= V
j
⊕W
j
in the language of ﬁlters. The
representation of a signal f in the scale V
j
,
R
j
(f) =
¸
k∈Z
'f, φ
j,k
`φ
j,k
,
is equivalent to ﬁlter the signal using the lowpass ﬁlter deﬁned by the scaling function φ.
The representation of the details of f in the space W
j
, equation (6.15) is obtained by ﬁltering
f with the bandpass ﬁlter deﬁned by the wavelet transform associated with ψ.
From the relation V
j−1
= V
j
⊕W
j
, we are able to write
R
j−1
(f) = R
j
(f) + Proj
W
j
(f)
R
j−2
(f) = R
j−1
(f) + Proj
W
j−1
(f)
.
.
.
Note that each line of the equation above represents a lowpass ﬁltering and a bandpass
ﬁltering of the signal. Iterating this equation for R
j−2
, . . . , R
j−J
0
, summing up both sides
and performing the proper cancellations, we obtain
R
j−J
0
(f) = R
j
(f) + Proj
W
j−1
(f) + Proj
W
j−J
0
(f). (6.16)
The projection R
j
(f) represents a version of low resolution (blurred version) of the signal,
obtained using successive lowpass ﬁltering with the ﬁlters φ
j
, φ
j−1
, . . . , φ
J
0
−j
. The terms
Proj
W
j−1
(f), . . . , Proj
W
j−J
0
(f) represent the details of the signal lost in each lowpass ﬁlter
ing. These details are obtained by ﬁltering the signal using the wavelets ψ
j
, ψ
j−1
, . . . , ψ
J
0
−j
.
Equation (6.16) states that the original signal f can be reconstructed exactly from the low
resolution signal, summing up the lost details.
Figure 6.7 shows an image (scale V
0
), the projection on the spaces V
−2
e V
−4
(on the left)
and the projections on the spaces W
−2
e W
−4
(on the right).
6.6 Two Scale Relation
In this section we will revisit some equations we obtained in the computations of this chapter
in order to distinguish them for future references.
Consider an scaling function φ associated to some multiresolution representation. Then
φ ∈ V
0
⊂ V
−1
and φ
−1,n
is an orthonormal basis of V
−1
. Therefore
φ =
¸
k∈Z
h
k
φ
−1,k
, (6.17)
with h
k
= 'φ, φ
−1,k
`. This equation can be written in the form
φ(x) =
√
2
¸
k∈Z
h
k
φ(2x −k). (6.18)
80 CHAPTER 6. MULTIRESOLUTION REPRESENTATION
Figure 6.7: Image decomposition on the spaces V
j
e W
j
.
Similarly, given a wavelet ψ associated with a multiresolution representation ψ ∈ V
0
, since
V
−1
= V
0
⊕W
0
, we have that ψ ∈ V
−1
, and ψ is orthogonal to V
0
, therefore
ψ =
¸
k∈Z
g
k
φ
−1,k
, (6.19)
or,
ψ(x) =
√
2
¸
k∈Z
g
k
φ(2x −k). (6.20)
Equations (6.17) and (6.19) (or equivalently (6.18) and (6.20)), are called twoscale re
lations, or scaling relations of the scaling function and the wavelet respectively. In several
important cases, the sum that deﬁnes the twoscale relations is ﬁnite:
φ(x) =
√
2
N
¸
k=0
g
k
φ(2x −k).
It is not diﬃcult to see that when this is the case, the support of the scaling function φ is
contained in the interval [0, N].
6.7. COMMENTS AND REFERENCES 81
Also, note that if φ is a solution of the equation deﬁned by the twoscale relation, then
λφ, λ ∈ R, is also a solution. In this way, to have uniqueness of the solution we must impose
some kind of normalization (e.g. ψ(0) = 1).
A priori, it is possible to construct a multiresolution representation and the associated
wavelet starting from an adequate choice of the function φ. This choice can be done using
the two scale relation (6.18). In a similar manner, the two scale equation (6.20) can be used
to obtain the associated wavelet.
6.7 Comments and References
The concept of multiresolution representation and its relation to wavelets was developed by
S. Mallat (Mallat, 1989). In the literature it carries diﬀerent names: multiscale analysis or
multiscale approximation. We have opted for multiresolution representation because it ﬁts
better to the emphasis we have been given on function representation.
The material covered in this chapter can be found on (Hernandez & Weiss, 1996). Never
theless the notation of the indices in the scale space diﬀers from the one used here.
For an exposition of the topics in this chapter using the language of operators in function
spaces the reader should consult (Kaiser, 1994). The approach is algebraically very clear and
clean, nevertheless a lot of geometric insight is lost.
Bibliography
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Hernandez, E., & Weiss, G. 1996. A First Course on Wavelets. Boca Raton: CRC Press.
Kaiser, G. 1994. A Friendly Guide to Wavelets. Boston: Birkhauser.
Mallat, S. 1989. Multiresolution approximation and wavelets. Trans. Amer. Math. Soc., 315,
69–88.
Chapter 7
The Fast Wavelet Transform
The Fast Wavelet Transform (FWT) algorithm, is the basic tool for computation with wavelets.
The forward transform converts a signal representation from the time (spatial) domain to its
representation in the wavelet basis. Conversely, the inverse transform reconstructs the signal
from its wavelet representation back to the time (spatial) domain. These two operations need
to be performed for analysis and synthesis of every signal that is processed in wavelet appli
cations. For this reason, it is crucial that the Wavelet Transform can be implemented very
eﬃciently.
In this chapter we will see that recursion constitutes the fundamental principle behind
wavelet calculations. We will start with a revision of the multiresolution analysis to show
how it naturally leads to recursion. Based on these concepts, we will derive the elementary
recursive structures which form the building blocks of the fast wavelet transform. Finally,
we will present the algorithms for the decomposition and reconstruction of discrete one
dimensional signals using compactly supported orthogonal wavelets.
7.1 Multiresolution Representation and Recursion
The eﬃcient computation of the wavelet transform exploits the properties of a multiresolution
analysis. In the previous chapters, we have seen that a multiresolution analysis is formed by
a ladder of nested subspaces
V
1
⊂ V
0
⊂ V
−1
where all V
j
are scaled versions of the central subspace V
0
.
From the above structure, we can deﬁne a collection of “diﬀerence” subspaces W
j
, as the
orthogonal complement of each V
j
in V
j−1
. That is,
V
j
= V
j+1
⊕W
j+1
As a consequence, we have a wavelet decomposition of L
2
(R) into mutually orthogonal sub
spaces W
j
L
2
(R) =
j∈Z
W
j
83
84 CHAPTER 7. THE FAST WAVELET TRANSFORM
Therefore, any square integrable function f ∈ L
2
(R) can be decomposed as the sum of its
projection on the wavelet subspaces
f =
¸
j∈Z
Proj
W
j
(f)
where Proj
W
j
(f) is the projection of f onto W
j
.
From V
j
= V
j+1
⊕W
j+1
, it follows that any function f
j
∈ V
j
can be expressed as
f
j
= Proj
V
j+1
(f) + Proj
W
j+1
(f).
This fact gives us the main recursive relation to build a representation of a function using the
wavelet decomposition.
If we denote the projections of f onto V
j
and W
j
respectively by f
j
= Proj
V
j
(f) and
o
j
= Proj
W
j
(f), we can write
f
j
= f
j+1
+ o
j+1
. .. .
f
j+2
+o
j+2
Applying this relation recursively we arrive at the wavelet representation
f
j
= f
j+N
+o
j+N
+ +o
j+2
+o
j+1
where a function f
j
in some V
j
is decomposed into its projections on the wavelet spaces
W
j+1
. . . W
j+N
, and a residual given by its projection onto the scale space V
j+N
. This recursive
process can be illustrated by the diagram in Figure 7.1.
f
j
→ f
j+1
→ f
j+2
→ → f
j+N
` ` ` `
o
j+1
o
g+1
o
j+N
Figure 7.1: Wavelet decomposition of a function f.
We assumed above that the process starts with a function f
j
which already belongs to
some scale subspace V
j
. This is not a restriction because we can take the initial j arbitrarily
small (i.e. a ﬁne scale). In practice, we work with functions that have some natural scale
associated with them.
The wavelet decomposition gives an analysis of a function in terms of its projections onto
the subspaces W
j
. Note that, since by construction W
j
⊥ W
l
if j = l and V
j
⊥ W
j
, this
decomposition of a function is unique once the spaces V
j
and W
j
are selected.
It is also desirable to reconstruct a function from its wavelet representation using a recur
sive process similar to the decomposition in Figure 7.1. It turns out that, since W
j
⊂ V
j−1
7.2. TWOSCALE RELATIONS AND INNER PRODUCTS 85
and V
j
⊂ V
j−1
, the original function can be obtained from the projections, and the wavelet
reconstruction is essentially the reverse of the decomposition, as illustrated in Figure 7.2.
The reconstruction gives a mechanism for the synthesis of functions from the wavelet
representation.
f
j+N
→ f
j+N−1
→ → f
j+1
→ f
j
o
j+N
o
j+N−1
o
j+1
Figure 7.2: Wavelet reconstruction process of a function f
To implement the wavelet decomposition and reconstruction we need to compute the
projections onto the spaces V
j
and W
j
. We know that the set of functions ¦φ
j,n
; n ∈ Z¦ and
¦ψ
j,n
; n ∈ Z¦, deﬁned as
φ
j,n
(x) = 2
−j/2
φ(2
−j
x −n) (7.1)
ψ
j,n
(x) = 2
−j/2
ψ(2
−j
x −n), (7.2)
are respectively orthonormal basis of V
j
and W
j
. Therefore, the projections operators Proj
V
j
and Proj
W
j
are given by inner products with the elements of these bases
Proj
V
j
(f) =
¸
n
'f, φ
j,n
`φ
j,n
=
¸
n
f(x)φ
j,n
(x)dx
φ
j,n
(7.3)
Proj
W
j
(f) =
¸
n
'f, ψ
j,n
`ψ
j,n
=
¸
n
f(x)ψ
j,n
(x)dx
ψ
j,n
(7.4)
The problem now is how compute the projection operators Proj
V
j
and Proj
W
j
eﬃciently.
In fact, we would like to avoid altogether computing the integrals explicitly. To ﬁnd a solu
tion we take advantage of the fact that the recursive decomposition/reconstruction processes
requires only projections between consecutive subspaces of the multiresolution ladder. For
that purpose we will rely on the two–scale relations.
7.2 TwoScale Relations and Inner Products
We have seen before that the interdependencies between two consecutive subspaces in a mul
tiresolution analysis are formulated by the equations below, called twoscale relations
φ(x) =
¸
k
h
k
φ
−1,k
(x) (7.5)
ψ(x) =
¸
k
g
k
φ
−1,k
(x) (7.6)
86 CHAPTER 7. THE FAST WAVELET TRANSFORM
Using these two relations, we can express the basis functions of the scale and wavelet
spaces, V
j
and W
j
, at level j in terms of the basis functions of the subsequent scale space
V
j−1
, at ﬁner level j −1. This is possible because, since V
j−1
= V
j
⊕W
j
, both V
j
⊂ V
j−1
and
W
j
⊂ V
j−1
.
Substituting (7.1) into (7.5), we have
φ
j,k
(x) = 2
−j/2
φ(2
−j
x −k)
= 2
−j/2
¸
n
h
n
2
1/2
φ(2
−j+1
x −2k −n)
=
¸
n
h
n
φ
j−1,2k+n
(x)
=
¸
n
h
n−2k
φ
j−1,n
(x) (7.7)
Similarly, substituting (7.2) into (7.6), we have
ψ
j,k
(x) = 2
−j/2
ψ(2
−j
x −k)
= 2
−j/2
¸
n
g
n
2
1/2
φ(2
−j+1
x −2k −n)
=
¸
n
g
n−2k
φ
j−1,n
(x) (7.8)
Now, we need to ﬁnd a way to use the sequences (h
n
)
n∈Z
and (g
n
)
n∈Z
to help us compute
recursively the inner products 'f, φ
j,k
`, and 'f, ψ
j,k
`. This can be easily done by inserting the
expressions obtained for φ
j,k
and ψ
j,k
into the the inner products.
'f, φ
j,k
` = 'f,
¸
n
h
n−2k
φ
j−1,n
` =
¸
n
h
n−2k
'f, φ
j−1,n
` (7.9)
'f, ψ
j,k
` = 'f,
¸
n
g
n−2k
φ
j−1,n
` =
¸
n
g
n−2k
'f, φ
j−1,n
` (7.10)
7.3 Wavelet Decomposition and Reconstruction
Using the two–scale relations, we showed how to relate the coeﬃcients of the representation
of a function in one scale 2
j−1
, with the coeﬃcients of its representation in the next coarse
scale 2
j
and with coeﬃcients of its representation in the complementary wavelet space. It
is remarkable, that from the inner products of the function f with the basis of V
j−1
, we
are able to obtain the inner products of f with the basis of V
j
and W
j
, without computing
explicitly the integrals! This is the crucial result for the development of the recursive wavelet
decomposition and reconstruction method described in this section.
7.3.1 Decomposition
The wavelet decomposition process starts with the representation of a function f in the space
V
0
. There is no loss of generality here because, by changing the units, we can always take
j = 0 as the label of the initial scale.
7.3. WAVELET DECOMPOSITION AND RECONSTRUCTION 87
We are given the function f = Proj
V
0
(f), represented by the coeﬃcients (c
k
) of its repre
sentation sequence in the scale space V
0
. That is
Proj
V
0
(f) =
¸
k
['f, φ
0,k
`φ
0,k
(x)] =
¸
k
c
0,k
φ
0,k
(7.11)
In case we only have uniform samples f(k), k ∈ Z of the function, the coeﬃcients (c
k
)
can be computed from the samples by a convolution operation. This fact is well explained in
Section 3.7 of Chapter 3 (see Theorem 2).
The goal of the decomposition is to take the initial coeﬃcient sequence (c
0
k
)
k∈Z
, and
transform it into the coeﬃcients of the wavelet representation of the function. The process
will be done by applying recursively the following decomposition rule
Proj
V
j
(f) = Proj
V
j+1
(f) + Proj
W
j+1
(f). (7.12)
In this way, the process begins with f
0
∈ V
0
= V
1
⊕ W
1
, and in the ﬁrst step, f
0
is
decomposed into f
1
+ o
1
, where f
1
= Proj
V
1
(f) and o
1
= Proj
W
1
(f). The recursion acts on
f
j
, decomposing it into f
j+1
+ o
j+1
, for j = 0, . . . N. The components o
j
are set apart. In
the end we obtain the wavelet representation of f, consisting of the residual scale component
f
N
and the wavelet components o
1
, . . . o
N
.
The core of the decomposition process splits the sequence (c
j
k
) of scale coeﬃcients associ
ated with f
j
, into two sequences (c
j+1
k
) and (d
j+1
k
), of scale and wavelet coeﬃcients associated,
respectively with f
j+1
and o
j+1
.
We can view this process as a basis transformation where we make the following basis
change (φ
j,k
)
k∈Z
→ (φ
j+1,k
, ψ
j+1,k
)
k∈Z
. Note that both sets form a basis of the space V
j
.
Equations (7.9) and (7.10) give the formulas to make the transformation on the coeﬃcients
of the bases:
c
j+1
k
=
¸
n
h
n−2k
c
j
n
(7.13)
d
j+1
k
=
¸
n
g
n−2k
c
j
n
(7.14)
with the notation a = (a
−n
)
n∈Z
.
Note that we are computing the coeﬃcients (c
j+1
k
) and (d
j+1
k
) by discrete convolutions,
respectively, with the sequences (h
n
) and (g
n
). Note also, that we are retaining only the even
coeﬃcients for the next step of recursion (because of the factor 2k in the indices). This is a
decimation operation.
In summary, if we start with a sequence (c
0
n
), containing n = 2
J
coeﬃcients, it will
be decomposed into the sequences (d
1
n/2
), (d
2
n/4
), . . . (d
J
n/2
J
), and (c
J
n/2
J
). Note that the
decomposition process outputs a wavelet representation with the same number of coeﬃcients
of the input representation.
Another important comment is that, up to now, we implicitly assumed doubly inﬁnite
coeﬃcient sequences. In practice, we work with ﬁnite representations, and therefore it is
necessary to deal with boundary conditions. This issue will be discussed in more detail later.
88 CHAPTER 7. THE FAST WAVELET TRANSFORM
7.3.2 Reconstruction
The reconstruction process generates the coeﬃcients of the scale representation from the
coeﬃcients of the wavelet representation. We would like to have an exact reconstruction,
such that the output of the reconstruction is equal to the input of the decomposition. This is
possible because we have just made an orthogonal basis transformation.
In order to bootstrap the recursive relations for the reconstruction process, we recall
that one step of the decomposition takes a function representation f
j−1
and splits into the
components f
j
and o
j
.
f
j−1
(x) = f
j
(x) +o
j
(x)
=
¸
k
c
j
k
φ
j,k
(x) +
¸
k
d
j
k
ψ
j,k
(x) (7.15)
We need to recover the coeﬃcients (c
j−1
n
) from (c
j
) and (d
j
)
c
j−1
n
= 'f
j−1
, φ
j−1,n
` (7.16)
Substituting (7.15) into (7.16), we obtain
c
j−1
n
= '
¸
k
c
j
k
φ
j,k
+
¸
k
d
j
k
ψ
j,k
, φ
j−1,n
` (7.17)
=
¸
k
c
j
k
'φ
j,k
, φ
j−1,n
` +
¸
k
d
j
k
'ψ
j,k
, φ
j−1,n
` (7.18)
Because both φ
0
∈ V
−1
and ψ
0
∈ V
−1
, they can be represented as a linear combination of
the basis ¦φ
−1,n
; n ∈ Z¦. Therefore φ
0
=
¸
n
'φ
0
, φ
−1,n
`φ
−1,n
and ψ
0
=
¸
n
' ψ
0
, φ
−1,n
`φ
−1,n
.
Since this representation is unique, using the two scale relations (7.5) and (7.6), we know that
h
n
= 'φ
0
, φ
−1,n
` (7.19)
g
n
= 'ψ
0
, φ
−1,n
` (7.20)
The above results provide a reconstruction formula for the coeﬃcients c
j−1
n
from the coef
ﬁcient sequences of the decomposition at level j.
c
j−1
n
=
¸
k
h
n−2k
c
j
k
+
¸
k
g
n−2k
d
j
k
=
¸
k
h
n−2k
c
j
k
+g
n−2k
d
j
k
(7.21)
The reconstruction process builds the ﬁnal representation (c
0
n
), from bottom up. At each step,
it combines the sequences (c
j
n
) and (d
j
n
) to recover the intermediate (c
j−1
n
), from j = J, . . . , 1.
7.4. THE FAST WAVELET TRANSFORM ALGORITHM 89
7.4 The Fast Wavelet Transform Algorithm
The fast wavelet transform (FWT) algorithm is a straightforward implementation of the
method described in the previous section. It consists of the recursive application of equations
(7.13) and (7.14) for the forward transform, and of equation (7.21) for the inverse transform.
In this section we present the pseudocode, in Clike notation, of an implementation of
the FWT algorithm. The code was structured for clarity and simple comprehension.
7.4.1 Forward Transform
The input of the algorithm is an array v, with 2
m+1
elements, containing the coeﬃcient
sequence to be transformed, and the number of levels m. It uses the global arrays containing the
twoscale sequences h and g. There are also global variables associated with these sequences:
their number of elements hn and gn; and their oﬀset values ho and go (i.e. the origins h
0
and g
0
of the sequences (h
n
) and (g
n
)). The main procedure wavelet fwd xform executes the
iteration of the basic wavelet decomposition.
wavelet_fwd_xform(v, m, h, g)
{
for (j = m; j >= 0; j)
wavelet_decomp(v, pow(2,j+1));
}
The procedure wavelet decomp performs the decomposition for just one level, splitting
the array v0 of size 2
j+1
, into two arrays v and w with sizes 2
j
. The result is accumulated
into the input array v, such that in the end of the decomposition the array v is partitioned
into [vN  wN  ...  w2  w1], with sizes respectively 1, 1, . . . , 2
m
, 2
m−1
.
wavelet_decomp(v, n)
{
zero (w, 0, n);
for (l = 0; l < n/2; l++) {
i = (2*l + ho) % n;
for (k = 0; k < hn; k++) {
w[l] += v[i] * h[k];
i = (i+1) % n;
}
i = (2*l + go) % n;
m = l + n/2;
for (k = 0; k < gn; k++) {
w[m] += v[i] * g[k];
i = (i+1) % n;
}
}
copy (w, v, n/2);
}
90 CHAPTER 7. THE FAST WAVELET TRANSFORM
The procedure uses a local array w that must have, at least, the same size of v. It calls
two auxiliary procedures, zero that ﬁlls and array with zeros, and copy that copies one array
to another.
7.4.2 Inverse Transform
The inverse transform takes as input an array containing the wavelet representation, in the
format produced by wavelet fwd xform, and converts it into a scale representation.
The procedure wavelet inv xform executes the iteration of the basic reconstruction step.
wavelet_inv_xform(v, m)
{
for (j = 0; j <= m; j++)
wavelet_reconst(v, pow(2, j+1));
}
The procedure wavelet reconst performs the reconstruction combining the components
vj and wj of the input array to reconstruct vj1. It replaces [vj wj...] with [vj1...].
Note that the number of elements of vj and wj is 1/2 of the number of elements of vj1,
therefore they use the same space in the array.
wavelet_reconst(w, n)
{
zero(v, 0, n);
for (k = 0; k < n; k++) {
i = floor((kho)/2) % (n/2);
m = (k  h.o) % 2;
for (l = m; l < hn; l += 2) {
v[k] += w[i] * h[l];
i = (i1) % (n/2);
}
i = floor ((kgo)/2) % (n/2);
m = (k  go) % 2;
for (l = m; l < gn; l += 2) {
v[k] += w[i + n/2] * g[l];
i = (i1) % (n/2);
}
}
copy(v, w, n);
}
7.5. BOUNDARY CONDITIONS 91
7.4.3 Complexity Analysis of the Algorithm
The computational performance of the algorithm is very important. Let’s determine what is
the computational complexity of the fast wavelet transform.
The computation of each coeﬃcient is a convolution operation with the twoscale se
quences. Assuming that these sequences have n coeﬃcients, then the convolution requires n
multiplications and n −1 additions.
In order to make the decomposition of a coeﬃcient sequence at level j, from V
j
into V
j+1
and W
j+1
, we have to compute 2
j
new coeﬃcients: 2
j+1
for the two components f
j+1
and
o
j+1
. Since each coeﬃcient requires 2n−1 operations, we have a total of 2
j
(2n−1) operations
for onelevel transformation.
The full decomposition process is applied for j log
2
(m) levels. Therefore, we have
O = 2
j
(2n −1) + 2
j+1
(2n −1) + + 2(2n −1)
factoring out (2n −1) and noting that m = 2
j
, we obtain:
O(m(2n −1)[1 + 2
−1
+ 2
−2
+ + 2
−j+1
])
O(m(2n −1)
1−2
−j
1−2
−1
)
O(mn)
The above analysis leads us to the following conclusions:
• The complexity is linear with respect to the size of the input sequence;
• The size of the twoscale sequences have a direct relation with the algorithm complexity.
7.5 Boundary Conditions
Since in practice we work with ﬁnite sequences, it is necessary take special care with the
computation near the beginning and the end of the sequences (boundaries).
In order to compute the coeﬃcients in the boundary regions, we have to perform a discrete
convolution with the twoscale sequences, and therefore, we may need coeﬃcients that lie
beyond the boundaries of the sequence. Note that, for this reason, the boundary region is
determined by size of the twoscale sequences. This situation is illustrated in Figure 7.3.
Figure 7.3: Boundary regions for convolution between ﬁnite sequences.
92 CHAPTER 7. THE FAST WAVELET TRANSFORM
There are some techniques to deal with boundary conditions:
• Extending the sequence with zeros (see Figure 7.4 (a));
• Periodization by translation of the sequence with x(N +i) ≡ x(i) (ﬁgure7.4 (b));
• Periodization by reﬂection of the sequence with x(N+i) ≡ x(N−i+1) e x(−i) ≡ x(i−1)
(Figure 7.4 (c));
• Use basis functions adapted to the interval (we are going to discuss this option later).
(a)
(b)
(c)
Figure 7.4: Options for boundary computation (a) Extending with zeros; (b) Periodization;
(c) Reﬂection.
In the implementation of the fast wavelet transform algorithm presented in Section 7.4,
we deal with the boundary problem by a simple periodization of the sequence. This is accom
plished using the coeﬃcient with indices i % m.
7.6. COMMENTS AND REFERENCES 93
7.6 Comments and References
The fast wavelet transform algorithm was introduced by Stephane Mallat (Mallat, 1989).
One of the ﬁrst references on the computational implementation of the algorithm appeared
in (Press et al. , 1996).
The code for the fast wavelet transform algorithm presented in this chapter was based
in the pseudocode from (Jawerth & Sweldens, 1994). This algorithm was implemented in
(BourgesS venier, 1994).
The book (Wickerhauser, 1994) describes a complete system for computation with wavelets,
including the fast wavelet transform.
Bibliography
BourgesS venier, Mikael. 1994. R alisation d’une biblioth que C de fonctions ondelettes. Tech.
rept. IRISA – INRIA.
Jawerth, B., & Sweldens, Wim. 1994. An overview of wavelet based multiresolution analyses.
SIAM Rev., 36(3), 377–412.
Mallat, S. 1989. Multifrequency Channel Decomposition of Images and Wavelet Models. IEEE
Transaction on ASSP, 37, 2091–2110.
Press, William H., Teukolsky, Saul A., & Vetterling, William T. 1996. Numerical Recipes :
The Art of Scientiﬁc Computing. Cambridge Univ Press. Chap. 13, pages 591–606.
Wickerhauser, Mladen Victor. 1994. Adapted Wavelet Analysis from Theory to Software.
Wellesley, MA: A. K. Peters.
Chapter 8
Filter Banks and Multiresolution
The goal of this chapter is to translate the theory of Multiresolution Representation to the
language of Signal Processing.
Therefore, this chapter takes an important step in the change from the mathematical
universe (continuous domain) to the representation universe (discrete domain), in the route
to implementation.
The reader not familiar with signal processing will ﬁnd basic concepts of linear systems
and ﬁlters in Appendices A and B.
8.1 TwoChannel Filter Banks
We are going to study in greater detail a particular case of a type of ﬁlter bank that will be
very important to understand the multiresolution representation in the discrete domain.
Consider a lowpass ﬁlter L and a highpass ﬁlter H. We deﬁne an analysis ﬁlter bank S
using those two ﬁlters together with downsampling operators as shown in Figure 8.1.
( 2)
( 2) H
L
z
z
1n
0n
y
0n
1n
y
x
n
Figure 8.1: Diagram of a TwoChannel Analysis Filter Bank.
Let’s study the operations in the above system: the input signal (x
n
) is processed by the
ﬁlter L in order to obtain its low frequency components (y
0n
), and also by the ﬁlter H to obtain
its high frequency components (y
1n
). After this ﬁrst level of processing the ﬁltered signals (y
0n
)
and (y
1n
) constitute together a representation of the original signal (x
n
), but with twice as
much samples. In order to reduce the size of these two signals to the size of the original signal,
we should discard terms. This can be achieved by performing a downsampling operation (see
95
96 CHAPTER 8. FILTER BANKS AND MULTIRESOLUTION
Appendix A). The output of the analysis bank, S(x
n
), is therefore a representation of the
input signal in terms of its low and high frequency components, with the same size of the
original signal. A natural question now is:
Question 1. Can we reconstruct the original signal (x
n
) from the representation S(x
n
) pro
duced by the analysis bank?
Because the downsampling operator ↓ 2 is not invertible, the answer to this question in
neither immediate nor obvious. An attempt of a solution is to deﬁne a synthesis ﬁlter bank
˜
S as shown in Figure 8.2.
( 2)
( 2)
x
n
z
1n
z
0n L
H
1
1
Figure 8.2: TwoChannel Synthesis Filter Bank
More precisely, we apply an upsampling operator to the components of the two chan
nels, lowpass and highpass, and then use a pair of ﬁlters L
1
e H
1
such that these processed
components when combined reconstruct the original signal.
The combination of these two ﬁlter banks S e
˜
S, forming a composite analysis and synthesis
ﬁlter bank
˜
S ◦ S, is shown in the diagram of Figure 8.3
( 2)
( 2)
( 2)
( 2)
x[n] x[n]
H
L L
H
1
1
Figure 8.3: TwoChannel Analysis and Synthesis Filter Bank
The ﬁlter bank S is called analysis bank because it produces a representation of the signal
in term of its frequencies. The ﬁlter bank
˜
S is called synthesis bank because it reconstruct
the signal from its representation. When the output of the composite ﬁlter bank
˜
S ◦ S is the
same as the input signal, that is
(ˆ x
n
) =
˜
SS(x
n
) = (x
n
),
we say that the ﬁlter bank has the property of perfect reconstruction. Usually, we have
perfect reconstruction, but with a delay of the signal, which is not a problem because we can
compensate that with a signal advance within the system.
8.2. FILTER BANKS AND MULTIRESOLUTION REPRESENTATION 97
Matrix Representation
Suppose that the ﬁlters L e H are deﬁned as the convolution operators
L(x
n
) =
3
¸
k=0
a(k)x(n −k)
and
H(x
n
) =
3
¸
k=0
b(k)x(n −k).
Then,the analysis bank is given by the matrix
S =
¸
↓ L
−−
↓ H
¸
,
or, using the results of applying the operator ↓ 2 to a matrix,
S =
¸
¸
¸
a(3) a(2) a(1) a(0)
a(3) a(2) a(1) a(0)
b(3) b(2) b(1) b(0)
b(3) b(2) b(1) b(0)
¸
.
It is clear that the perfect reconstruction property is related with the invertibility of the
matrix S. In fact, if S is invertible, we can make the synthesis bank as
˜
S = S
−1
, and we
have exact reconstruction. An important particular case occurs when the linear system S is
orthogonal, in this case the matrix of S is an orthogonal matrix. This system has the perfect
reconstruction property and the synthesis ﬁlter, the inverse matrix
˜
S is determined by the
transpose of the analysis matrix S:
˜
S = S
T
.
Question 2. Why we have discussed twochannel ﬁlter banks with perfect reconstruction?
The answer to this question is easy: Given a multiresolution analysis, we know that the
associated scaling function is a lowpass ﬁlter and the corresponding wavelet is a bandpass
ﬁlter. We are going to show that, in the discrete domain, the multiresolution analysis deﬁnes
a ﬁlter bank similar to the twochannel ﬁlter bank
˜
S ◦ S, which has the property of perfect
reconstruction. Furthermore, we will describe an algorithm that implements all the operations
of the ﬁlter bank (analysis and synthesis) in linear time, proportional to the number of samples
of the input signal (note that this is equivalent to the algorithm described in last chapter).
8.2 Filter Banks and Multiresolution Representation
In the mathematical universe (continuous domain) is easy to convince ourselves that a mul
tiresolution analysis deﬁnes a ﬁlter bank. This was already done earlier, but for the sake of
reviewing the concepts and notation, we will repeat here:
98 CHAPTER 8. FILTER BANKS AND MULTIRESOLUTION
If φ is the scaling function associated with the multiresolution analysis
⊂ V
1
⊂ V
0
⊂ V
−1
⊂
then V
j−1
= V
j
⊕W
j
, where W
j
is the complementary (or wavelet) space. Given f ∈ L
2
(R),
we have that
Proj
V
j−1
(f) = Proj
V
j
(f) + Proj
W
j
(f).
Also, Proj
V
j
(f) represents the lowpass component of the function f, and Proj
W
j
(f) represents
the bandpass component of the function f both deﬁned by the wavelet transform. Intuitively
the lowpass component gives a representation of f in the scale 2
j
, and the bandpass gives the
details that are lost when f is represented in this scale
Using the notation
L
j
(f) = Proj
V
j
(f), e H
j
(f) = Proj
W
j
(f),
we have
f = L
j
(f) +H
j
(f).
Applying successively this decomposition to the component L
j
(f) of low frequencies of the
signal, we arrive at
f = L
k
j
(f) +H
k
(f) +H
k−1
(f) + +H
j
(f). (8.1)
In summary, the signal f is decomposed into a low frequency component (i.e. in a scale
that cannot capture details), L
k
j
(f), together with high frequency components, H
n
(f), n =
k, k −1, . . . , j, which contain the details lost when going to a low resolution representation of
f.
The equation (8.1) states that we can recover the function f exactly from the low reso
lution component by adding the high resolution components, properly. Therefore, he have a
(continuous) twochannel ﬁlter bank, with perfect reconstruction.
The decomposition operations of the analysis ﬁlter bank are illustrated by the diagram
below
L
j
(f) L
2
j
(f) L
k
j
(f)
f
H
j
(f)
H
j+1
(f)
H
k
(f)
The reconstruction operations of the synthesis ﬁlter bank are illustrated by the diagram
below
L
k
(f)
L
k+1
(f)
L
j
(f)
f
H
k
(f) H
k+1
(f) H
j
(f)
8.3. DISCRETE MULTIRESOLUTION ANALYSIS 99
Our objective now is to study the above ﬁlter bank, that corresponds to a multiresolution
analysis, in the discrete domain.
8.3 Discrete Multiresolution Analysis
All ingredients for the discretization of a multiresolution analysis, and to revisit the associated
ﬁlter bank in the discrete domain, are already of our knowledge.
If ψ is the wavelet associated with a multiresolution analysis, we have
ψ
j,k
(x) = 2
−j
2
ψ(2
−j
x −k). (8.2)
We also have the twoscale relation that gives ψ as
ψ(x) =
¸
n
g
n
φ
−1,n
(x) =
√
2
¸
n
g
n
φ(2x −n),
where φ is the corresponding scaling function, and g
n
= 'ψ, φ
−1,n
`. Using this relation in
(8.2), we have
ψ
j,k
(x) = 2
−j
2
¸
n
g
n
2
1/2
φ(2
−j+1
x −2k −n)
=
¸
n
g
n
¸
n
g
n
2
−j+1
2
φ(2
−j+1
x −(2k +n))
=
¸
n
g
n
φ
j−1,2k+n
(x)
Making a change in the indices, we can rewrite the above equation in the form
ψ
j,k
(x) =
¸
n
g
n−2k
φ
j−1,n
. (8.3)
Then, it follows that
'f, ψ
j,k
(x)` =
¸
n
g
n−2k
'f, ψ
j−1,n
`. (8.4)
We can write the expression above using the ﬁlter operators so that the reader can more
easily identify it with a ﬁlter bank. Indeed, it is immediate for the reader to verify that
equation (8.4) above is equivalent to
'f, ψ
j,k
(x)` = (↓ 2)[('f, ψ
j−1,n
`) ∗ g
−n
]. (8.5)
The twoscale relation for the function φ of the multiresolution analysis is given by
φ =
¸
n
h
n
φ
−1,n
.
100 CHAPTER 8. FILTER BANKS AND MULTIRESOLUTION
Computations similar to the ones we did from the twoscale relation for the wavelet, give
φ
j,k
(x) =
¸
n
h
n−2k
φ
j−1,n
, (8.6)
From that, it follows
'f, φ
j,k
(x)` =
¸
n
h
n−2k
'f, φ
j−1,n
`. (8.7)
As before, we can write this expression using the convolution and downsampling operators:
'f, φ
j,k
(x)` = (↓ 2)[('f, φ
j−1,n
`) ∗ h
−n
] (8.8)
8.3.1 Pause to Review
The equations (8.4) and (8.7) given the decomposition and reconstruction formulas of the
ﬁlter bank associated with a multiresolution analysis. We are going to study these equations
in more detail.
Without loss of generality, we can assume that the function f is deﬁned initially in some
scale as f
0
. This scale is associated with some sampling frequency in which f can be rep
resented by a sequence of samples. We suppose that f
0
= Proj
V
0
(f), that is f
0
is the
discretization of f in the scale space V
0
. Because V
0
= V
1
+W
1
, we have that
f
0
= f
1
+o
1
.
Furthermore,
f
0
=
¸
n
c
0
n
φ
0,n
;
f
1
=
¸
n
c
1
n
φ
1,n
;
o
1
=
¸
n
d
1
n
φ
1,n
.
From equation (8.7) and (8.4), we have
c
1
k
=
¸
n
h
n−2k
c
0
n
;
d
1
k
=
¸
n
g
n−2k
c
0
n
.
These equations allow us to obtain a representation of f in the ﬁner scale V
−1
from the initial
representation sequence (c
0
n
)
n∈Z
, in the scale V
0
.
From a linear algebra point of view, we are just making a change of basis: From the basis
¦φ
0,n
¦
n∈Z
of the space V
0
to the basis ¦φ
1,n
, ψ
1,n
¦
n∈Z
.
8.3. DISCRETE MULTIRESOLUTION ANALYSIS 101
Indicating by L the matrix of the operator in (8.5) and by H the matrix of the operator
in (8.8), we can write
(c
1
n
) = L(c
0
n
) e (d
1
n
) = H(c
0
n
).
Similarly,
f
1
∈ V
1
= V
−2
⊕W
−2
and, therefore
f
1
= f
2
+o
2
, f
2
∈ V
−2
, o
2
∈ W
−2
.
Thus,
f
2
=
¸
n
c
2
n
φ
2,n
, o
2
=
¸
n
d
2
n
ψ
2,n
,
with
(c
2
n
) = L(c
1
n
), e (d
2
n
) = H(c
1
n
).
We have an analysis ﬁlter bank that is given by the diagram shown below
(d
1
n
) (d
2
n
) (d
k
n
)
H
H
H
(c
0
n
)
L
(c
1
n
)
L
(c
2
n
)
L
(c
k
n
)
The above ﬁlter bank is a very particular case of a ﬁlter bank called pyramid bank. In
fact, the pyramid structure is the best form to represent the type of ﬁlter bank in a diagram.
For example, consider a initial signal given by a sequence of samples with 8 elements, that is,
f
0
= (c
0
0
, c
0
1
, c
0
2
, . . . , c
0
7
).
The successive analyses of f
0
by the ﬁlter bank are represented in the inverted pyramid,
shown in the diagram below:
c
0
0
c
0
1
c
0
2
c
0
3
c
0
4
c
0
5
c
0
6
c
0
7
`
`
`
`
`¬
»

`
`
`
`
`¬
»

`
`
`
`
`¬
»

`
`
`
`
`¬
»

d
1
0
d
1
1
d
1
2
d
1
3
c
1
0
c
1
1
c
1
2
c
1
3
`
`
`
`
`
`
`
`·
.

`
`
`
`
`
`
`
`·
.

d
2
0
d
2
1
c
2
0
c
2
1
`
`
`
`
`
`
`
.
.
d
3
0
c
3
0
102 CHAPTER 8. FILTER BANKS AND MULTIRESOLUTION
In the ﬁrst level (base of the pyramid) we have the initial sequence of samples of f. In
the second level we obtain the coeﬃcients (c
1
n
) of the lower resolution scale and the detail
coeﬃcients (d
1
n
). We continue with the operation of the ﬁlter bank, going through each level
of the pyramid until we reach a sequence with only one element (c
3
0
) (which gives the average
of the signal).
In matrix notation, the ﬁlter is represented by the structure
S =
¸
¸
¸
L
k
[
H
k
[
−− − −
[ I
¸
¸
¸
¸
L
1
[
H
1
[
−− − −
[ I
¸
¸
¸
¸
L
−
H
¸
(8.9)
The order of each block
¸
L
k
−−
H
k
¸
with the lowpass ﬁlter L
k
and highpass ﬁlter H
k
is half of the order of the preceding block
¸
L
k−1
−−
H
k−1
¸
with ﬁlters L
k−1
e H
k−1
. Accordingly, the order of the block with the identity matrix doubles
its size in each matrix. This product of matrices is the matrix representation of the ﬁlter bank
of the multiresolution analysis. When this matrix is applied to the initial vector (c
0
n
) it pro
duces the complete multiresolution decomposition. The vector resulting from this operation
S((c
0
n
)) is
(c
J
0
, d
J
0
, d
J
0
−1
, . . . , d
0
), (8.10)
where
d
k
= (d
k
0
, d
k
1
, . . . , d
k
m
), k = J
0
, J
0
−1, . . . , 0.
Example 14 (Haar Multiresolution Analysis). The twoscale relations of the Haar mul
tiresolution analysis are
φ(t) = φ(2t) +φ(2t −1)
ψ(t) = ψ(2t) −ψ(2t −1)
For a signal represented by four samples, the corresponding ﬁlter bank matrix in the ﬁrst
level has order 4 and is given by
L
G
=
¸
¸
¸
1 1
1 1
1 −1
1 −1
¸
.
8.4. RECONSTRUCTION BANK 103
To make this matrix an orthogonal operator, we have to multiply its elements by r = 1/
√
2,
obtaining the matrix
¸
¸
¸
r r
r r
r −r
r −r
¸
.
In the next level of scale the matrix is given by
r r
r −r
.
Therefore the matrix of the ﬁlter bank is
¸
¸
¸
r r
r −r
1
1
¸
¸
¸
¸
r r
r r
r −r
r −r
¸
.
8.4 Reconstruction Bank
Since the scaling function and wavelet are orthonormal bases, the operators involved into the
analysis ﬁlter bank are also orthonormal. Therefore, the inverse operation for the synthesis
ﬁlter bank is given by the adjoint matrix (i.e. the transpose of the conjugate). Indeed, the
ﬁlter bank gives perfect reconstruction.
For completeness, we are going to derive the reconstruction expressions. We have
f
j−1
= f
j
+d
j
=
¸
k
c
j
k
φ
jk
+
¸
k
d
j
k
ψ
j,k
.
Therefore
c
j−1
n
= 'f
j−1
, φ
j−1,n
`
=
¸
k
c
j
k
'φ
j,k
, φ
j−1,n
` +
¸
k
d
j
k
'ψ
j,k
, φ
j−1,n
`
=
¸
k
[h
n−2k
c
j
k
+g
n−2k
d
j
k
] (8.11)
where we have used the equation (8.3) and (8.7) in the last line.
The equation (8.11) is the synthesis equation (reconstruction) of the ﬁlter bank associated
with the multiresolution: it allows us to obtain the representation sequence (c
j−1
n
) of the
function f in a ﬁner scale, from the sequences (c
j
n
) and (d
j
n
) in a lower resolution scale and
its complement.
104 CHAPTER 8. FILTER BANKS AND MULTIRESOLUTION
The reconstruction diagram is shown below
(c
J
0
−j
n
)
(c
J
0
−j+1
n
)
(c
J
0
−1
n
)
(c
J
0
n
)
(d
J
0
−j
n
) (d
J
0
−j+1
n
) (d
J
0
−1
n
)
In summary, using a representation of the signal f at a low resolution C
J
0
−j
and the details
corresponding to the information diﬀerences at intermediate resolutions (d
J
0
−j
n
), (d
J
0
−j−1
n
),
. . . , (d
J
0
−1
n
), we reconstruct exactly the representation c
J
0
of the signal at the scale 2
J
0
.
8.5 Computational Complexity
When we start with a function at some scale 2
j
we have j levels in the pyramid, which
correspond to j submatrices in the matrix that deﬁnes the ﬁlter bank of the multiresolution
analysis. If the associated lowpass and highpass ﬁlters have a kernel with T elements, we have
T nonzero entries in each line of the matrix. Therefore, we have a product of TL nonzero
elements in the computation of the ﬁrst level of the ﬁlter bank, where L = 2
j
, is the number
of samples of the representation sequence of the input signal.
In the second level we have TL/2 products, in the third level TL/4 products, and so on.
The number of products is reduced by one half in each level. Therefore, the total number of
product to compute the analysis operation of the ﬁlter bank is given by
TL
1 +
1
2
+
1
4
+ +
1
2
j−1
< 2TL.
We conclude that the computational complexity in the decomposition computation of a
multiresolution analysis is linear with the length of the input signal.
In terms of matrices, the reconstruction is given by the conjugate transpose of the matrix S
in (8.9), because the analysis operator is orthogonal. It follows trivially that the computational
complexity of the reconstruction ﬁlter bank is the same as the decomposition bank: i.e. is
also linear with the length of the input signal
8.6 Comments and References
We have shown in this chapter that the multiresolution analysis deﬁnes a ﬁlter bank with
perfect reconstruction. The ﬁlter bank has a pyramidal structure, and the decomposition and
reconstruction operations can be computed in linear time relative to the length of the input
signal.
The study of ﬁlter banks with perfect reconstruction started in the beginning of the 80’s,
and advanced independently of the studies in the area of wavelets, which intensiﬁed in the
middle of the 80’s (the Haar wavelet is known since the beginning of this century, but it was
not associated with the context of wavelets).
BIBLIOGRAPHY 105
We already mentioned in the previous chapter that the concept of multiresolution analysis
was introduced by Stephane Mallat. In fact, Mallat also discovered the relation of ﬁlter banks
and multiresolution analysis as we described in this chapter. He also developed the recursive
algorithm to implement these operations, as we described in the previous chapter. This
algorithm is known in the literature by various names: Mallat algorithm, pyramid algorithm
or fast wavelet transform.
The multiresolution analysis is covered in various books (Daubechies, 1992), (Vetterli &
Kovacevic, 1995), (Hernandez & Weiss, 1996) e (Strang & Nguyen, 1996). This last reference
(Strang & Nguyen, 1996) is one of the few which adopts a matrix notation.
Bibliography
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Hernandez, E., & Weiss, G. 1996. A First Course on Wavelets. Boca Raton: CRC Press.
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Vetterli, Martin, & Kovacevic, Jelena. 1995. Wavelets and Subband Coding. Englewood Cliﬀs,
New Jersey: Prentice Hall PTR.
Chapter 9
Constructing Wavelets
The proof of the existence of a wavelet associated with a multiresolution representation de
scribed in Chapter 6 has a constructive ﬂavor. In this chapter we will go over details of this
proof with the purpose of obtaining a recipe to construct wavelets.
9.1 Wavelets in the Frequency Domain
We start by analyzing the dilation equation, and looking at the bases functions φ and ψ in
the frequency domain.
9.1.1 The Relations of
ˆ
φ with m
0
Taking the Fourier transform of the twoscale equation
φ(x) =
√
2
¸
n
h
n
φ(2x −n) (9.1)
gives
ˆ
φ(ω) =
1
√
2
¸
n
h
n
e
−inω/2
ˆ
φ(ω/2) (9.2)
Equation (9.2) can be rewritten as
ˆ
φ(ω) = m
0
(ω/2)
ˆ
φ(ω/2) (9.3)
with
m
0
(ω) =
1
√
2
¸
n
h
n
e
−inω
(9.4)
The function m
0
is 2π–periodic, and m
0
∈ L
2
([0, 2π]), because
¸
n∈Z
[h
n
[
2
< ∞.
We also know that, by deﬁnition,
∞
−∞
φ(x)dx = 1
107
108 CHAPTER 9. CONSTRUCTING WAVELETS
hence,
ˆ
φ(0) = 1, and therefore
m
0
(0) = 1 (9.5)
Applying equation (9.3) recursively for w/2, w/4, . . . , we get
ˆ
φ(ω) = m
0
(ω/2)m
0
(ω/4)
ˆ
φ(ω/4) . . . ,
and arrive at the inﬁnite product formula
ˆ
φ(ω) =
1
√
2π
∞
¸
j=1
m
0
(2
−j
ω). (9.6)
A very important point is to show that this product converges to a function in L
2
(R). Details
of this proof can be found in (Daubechies, 1992).
We can see that
ˆ
φ is completely characterized by m
0
, as φ is completely characterized
by the sequence (h
n
), Note also that knowing m
0
gives us (h
n
). This is the ﬁrst important
connection between wavelets in the spatial and frequency domains.
9.1.2 The Relations of
ˆ
ψ with m
1
Similarly, if we express the two scale relation for the wavelet function ψ in the frequency
domain,
ψ(x) =
√
2
¸
n
g
n
ψ(2x −n) (9.7)
we get
ˆ
ψ(ω) =
1
√
2
¸
n
g
n
e
−inω/2
ˆ
φ(ω/2) (9.8)
or
ˆ
ψ(ω) = m
1
(ω/2)
ˆ
φ(ω/2) (9.9)
with
m
1
(ω) =
1
√
2
¸
n
g
n
e
−inω
(9.10)
where m
1
is also 2πperiodic.
Note that,
ˆ
ψ is deﬁned in terms of
ˆ
φ, through m
1
, in the same way ψ is deﬁned in terms
of φ through (g
n
) in the spatial domain.
9.1.3 Characterization of m
0
In order to deﬁne the properties of m
0
, we use the fact that φ(u − k), the integer translates
of φ form an orthonormal basis of V
0
. This imposes some restrictions on m
0
.
∞
−∞
φ(x)φ(x −k)dx =
∞
−∞
[
ˆ
φ(ξ)[
2
e
ikξ
dξ = δ
k,0
(9.11)
=
2π
0
e
ikξ
¸
l∈Z
[
ˆ
φ(ξ + 2πl)[
2
dξ = δ
k,0
(9.12)
9.1. WAVELETS IN THE FREQUENCY DOMAIN 109
The above equation implies that
¸
l
[
ˆ
φ(ξ + 2πl)[
2
=
1
2π
(9.13)
Substituting equation (9.3) in (9.13), with ω = ξ/2, we have
¸
l
[m
0
(ω +πl)[
2
[
ˆ
φ(ω +πl)[
2
=
1
2π
(9.14)
We can split the sum into terms with even and odd l, and since m
0
is 2πperiodic we have
[m
0
(ω)[
2
¸
l
[
ˆ
φ(ω + 2lπ)[
2
+[m
0
(ω +π)[
2
¸
l
[
ˆ
φ(ω + (2l + 1)π)[
2
=
1
2π
(9.15)
substituting (9.13), and simplifying, we obtain
[m
0
(ω)[
2
+[m
0
(ω +π)[
2
= 1 (9.16)
This is the ﬁrst important condition characterizing m
0
, via orthonormality of φ.
If we put together equation (9.5) with (9.13) , we conclude that
m
0
(π) = 0 (9.17)
This gives us a hint that m
0
is of the form (i.e. factorizes as)
m
0
(ω) =
1 +e
iω
2
m
Q(ω) (9.18)
with m ≥ 1, and where Q is a 2πperiodic function. (Observe that e
iπ
= −1. So, when ω = π
the ﬁrst term vanishes, and the product has to vanish.) We impose Q(0) = 1, to ensure that
m(0) = 1, and also Q(π) = 0, so that the multiplicity of the root of m
0
at π is not increased
by Q.
9.1.4 Characterization of m
1
Now, to link m
1
with m
0
we use the orthogonality between φ and ψ. More precisely, the
constraint that W
0
⊥ V
0
implies that ψ ⊥ φ
0,k
and
∞
−∞
ˆ
ψ(ω)
ˆ
φ(ω)e
ikω
dω = 0 (9.19)
or, in terms of the Fourier series
2π
0
e
ikω
¸
l
ˆ
ψ(ω + 2πl)
ˆ
φ(ω + 2πl)dω = 0 (9.20)
110 CHAPTER 9. CONSTRUCTING WAVELETS
hence
¸
l
ˆ
ψ(ω + 2πl)
ˆ
φ(ω + 2πl) = 0 (9.21)
for all ω ∈ R.
Substituting in the above equation the expressions (9.3) and (9.9) of
ˆ
φ and
ˆ
ψ in terms of,
respectively, m
0
and m
1
, we obtain after regrouping the sums for odd and even l
m
1
(ω)m
0
(ω) +m
1
(ω +π)m
0
(ω +π) = 0 (9.22)
This is the second important condition that caracterizes m
0
and m
1
.
We also know that m
0
(w) and m
0
(w +π) cannot be zero simultaneously because of (9.16),
therefore m
1
can be written using m
0
and a function λ
m
1
(ω) = λ(ω)m
0
(ω +π) (9.23)
such that λ satisﬁes
λ(ω) +λ(ω +π) = 0 (9.24)
The simplest choice is λ(ω) = e
iw
, which gives m
1
satisfying the above equation
m
1
(ω) = e
−iω
m
0
(ω +π) (9.25)
Note that m
1
is deﬁned in terms of m
0
, as expected. This also gives
ˆ
ψ in terms of
ˆ
φ
ˆ
ψ(ω) = e
iω/2
m
0
(ω/2 +π)
ˆ
φ(ω/2) (9.26)
From the above relations, we can construct an orthogonal wavelet from a scaling function φ,
using (9.25) and choosing the coeﬃcients ¦g
n
¦ as
g
n
= (−1)
n
h
−n+1
, (9.27)
that is,
ψ(x) =
√
2
¸
n
(−1)
n
h
−n+1
φ(2x −n) (9.28)
We conclude that, since m
1
is trivially deﬁned from m
0
, all we need to construct orthogonal
scale and wavelet bases, is to ﬁnd a function m
0
satisfying (9.16) and (9.22), or, equivalently,
ﬁnd the coeﬃcients (h
n
) of the representation sequence of m
0
.
Example 15 (Haar Wavelet). The scaling function of the Haar multiresolution represen
tation is given by
φ(x) =
1 if x ∈ [0, 1)
0 otherwise.
9.1. WAVELETS IN THE FREQUENCY DOMAIN 111
From the two scaling relation
φ(t) =
√
2
¸
k
c
k
φ(2t −k),
we have
c
n
=
√
2
+∞
−∞
φ(t)φ(2t −n)dt.
It is easy to see that only φ(2t) and φ(2t − 1) are not disjoint from φ(t), therefore only
the coeﬃcients of c
0
and c
1
are nonnull (see Figure 9.1). An easy computation shows that
c
0
= c
1
=
√
2
2
.
Therefore the two scale equation can be written as
φ(x) = φ(2x) +φ(2x −1),
This equation is illustrated in Figure 9.1: In (a) we have the graph of φ and in (b) we have
the sum of the functions φ(2x) and φ(2x −1).
(a) (b)
Figure 9.1: Double scale relation of the Haar scaling function.
The two scale relation (9.28) for the wavelet in this case is given by
ψ(x) = φ(2x) −φ(2x −1),
which gives the Haar wavelet already introduced before (See Figure 9.2).
Unfortunately the Haar example does not give a good view of the reality. The above
reasoning is correct and promising but taking this trail to get to the wavelets involves a lot
of work.
112 CHAPTER 9. CONSTRUCTING WAVELETS
Figure 9.2: Haar wavelet.
9.2 Orthonormalization Method
Given a function φ(u) satisfying the two scale relation (6.18), it may happen that the trans
lated functions φ(x −n) do not constitute an orthonormal basis. We can overcome this fact.
In fact, we have stated before that the orthonormality condition (M5) in the deﬁnition of
a multiresolution representation could be relaxed with the weaker condition that φ(x−n) is a
Riesz basis. This fact is the essence of the orthonormalization process stated in the Theorem
below:
Theorem 6. If ¦φ(u − k¦, k ∈ Z, is a Riesz basis of the space V
0
, and we deﬁne a function
φ
#
by
ˆ
φ
#
(ξ) =
1
√
2π
ˆ
φ(ξ)
¸
k
[
ˆ
φ(ξ + 2πk)[
2
, (9.29)
then φ
#
(u −k), k ∈ Z, is an orthonormal basis of V
0
.
The proof of the theorem can be found in (Daubechies, 1992), page 139.
In spite of using a weaker condition on (M5), the task of showing that a set of functions
is a Riesz basis is not an easy one, in general. Moreover there exists the other conditions
which must be satisﬁed in order to have a multiresolution representation. This problem can
be solved using the result from
Theorem 7. If φ ∈ L
2
(R) satisﬁes the two scale relation
φ(x) =
¸
k
c
k
φ(2x −k),
with
¸
k
[c
k
[
2
< ∞,
9.3. A RECIPE 113
and
0 < α <
¸
k
[
ˆ
φ(ξ + 2πk)[
2
≤ β < ∞, (9.30)
then φ deﬁnes a multiresolution representation.
9.3 A Recipe
In this section we will summarize the above results to construct a multiresolution representa
tion, along with the associated wavelet.
Step 1. We start from a function φ which deﬁnes the kernel of a lowpass ﬁlter. A suﬃcient
condition for this is that φ satisﬁes
R
φ(u)du = 0,
and also that φ and
ˆ
φ have a good decay at inﬁnity.
Step 2. We verify if the function φ of the previous step satisﬁes the two scale relation (6.18),
and the condition (9.30).
Step 3. If we have a function satisfying the two previous steps, but φ(u − k) is not an
orthonormal basis, then we can obtain an orthonormal basis from φ, according to Theorem 6.
Step 4. The wavelet associated to the multiresolution representation can be computed using
equation (6.13) from previous chapter.
9.4 Piecewise Linear Multiresolution
The Haar multiresolution representation allows us to obtain successive approximations of a
function using functions that are constant by parts. Now we will use the recipe from the
previous section to obtain a multiresolution representation whose scale space approximate a
function f ∈ L
2
(R) using piecewise linear functions.
We take as a candidate for the scaling function φ the function
φ(x) =
1 −[x[ if x ∈ [0, 1]
0 if x < 1 or x > 1.
The graph of φ is shown in Figure 9.3(a).
Certainly the function φ satisﬁes all of the conditions in Step 1 of our recipe. Moreover,
it is easy to see that φ also satisﬁes the two scaling relation
φ(x) =
1
2
φ(2x + 1) +φ(2x) +
1
2
φ(2x −1). (9.31)
114 CHAPTER 9. CONSTRUCTING WAVELETS
(a) (b)
Figure 9.3: Piecewise linear scaling function.
This relation is illustrated in Figure 9.3(b).
The Fourier transform of φ is given by
ˆ
φ(ω) =
1
√
2π
sin(ω/2)
ω/2
2
. (9.32)
An immediate calculus shows that
2π
¸
k∈Z
[
ˆ
φ(ω + 2πk[
2
=
2
3
+
1
3
cos ω =
1
3
(1 + cos
2
(
w
2
)). (9.33)
Therefore the condition (9.30) is satisﬁed. From Theorem 7, φ deﬁnes a multiresolution
representation.
Nevertheless, it is easy to verify that the family φ(u − k), k ∈ Z, is not orthonormal.
Therefore we must apply the orthonormalization process (Theorem 6) to obtain a scaling
function that deﬁnes a multiresolution orthonormal basis.
Substituting (9.32) and (9.33) in (9.29), we have
ˆ
φ
#
(ξ) =
3
2π
4 sen
2
(
ξ
2
)
ξ
2
1 + 2 cos
2
(
ξ
2
1/2
,
that is the Fourier transform of the scaling function we are looking for. The graph of
ˆ
φ
#
is
shown in Figure 9.4(a). The graph of the associated wavelet ψ(x) is shown in Figure 9.4(b).
9.5 Shannon Multiresolution Analysis
From classical Fourier analysis we know the lowpass ideal ﬁlter φ: R → R, whose transfer
function is given by
ˆ
φ(ω) = χ
[−π,π]
=
1 if x ∈ [−π, π]
0 if x < −π or x > π.
9.5. SHANNON MULTIRESOLUTION ANALYSIS 115
(a) (b)
Figure 9.4: (a)Scaling function of the piecewise linear multiresolution; (b) Wavelet of the
piecewise linear multiresolution.
The impulse response of the ﬁlter is given by the function
φ(x) = sinc(x) =
sin(πx)
πx
.
The graph of this function is shown in Figure 9.5.
6 4 2 2 4 6
0.4
0.2
0.2
0.4
0.6
0.8
1
Figure 9.5: Impulse response of the ideal reconstruction ﬁlter.
It is natural to expect that the function φ deﬁnes a multiresolution representation. This
is in fact true, but we will not give the details here. It is called Shannon multiresolution
representation. Shannon multiresolution representation divides the frequency domain into
bands according to the illustration shown in Figure 9.6.
The wavelet ψ associated to Shannon multiresolution representation is the ideal bandpass
ﬁlter, deﬁned by
ˆ
ψ = χ
I
, onde I = [−2π, −π) ∪ (π, 2π].
The graph of this ﬁlter is shown in Figure 9.7.
116 CHAPTER 9. CONSTRUCTING WAVELETS
Figure 9.6: Band decomposition using Shannon multiresolution representation.
Figure 9.7: Ideal bandpass ﬁlter.
This wavelet is called Shannon wavelet. It has an analytic expression given by
ψ(x) = −2
sen(2πx) + cos(2πx)
π(2x + 1)
.
The graph is shown in Figure 9.8.
6 4 2 2 4 6
1
0.5
0.5
1
Figure 9.8: Shannon wavelet.
9.6. WHERE TO GO NOW? 117
9.6 Where to Go Now?
Up to this point we have seen how to obtain atomic decompositions using wavelets, we have
studied the role the wavelets in the multiresolution representation, we have shown the ex
istence of orthonormal basis of wavelets and we have described a method to construct such
basis using multiresolution representation.
Certainly, in applications we need to have at our disposal a great abundance of basis
of wavelets. Therefore the natural path to follow should point in the direction of devising
methods to construct basis or frames of wavelets.
In order to amplify the range of possible applications, we should impose additional condi
tions to the wavelets we construct. Among many conditions we could mention:
• regularity;
• symmetry;
• compact support;
• orthogonality with polynomials of degree ≤ n.
The construction of wavelets satisfying some of the above properties, as well as a discussion
about the importance of these properties in diﬀerent types of applications is of fundamental
importance. In Chapter 10, we will present a framework for generating wavelets using a ﬁlter
design methodology.
9.7 Comments and References
In this chapter we have described how wavelets can be constructed. Devising robust techniques
to construct wavelets is a very important research topic. The reader should realize that each
application demands bases of wavelets with diﬀerent properties. We will cover this in the
chapters to follow.
The Haar wavelet and the wavelet associated with the piecewise linear multiresolution
representation are part of a family of wavelets called spline wavelets. This family is related
to a multiresolution representation that approximates a function f ∈ L
2
(R) using piecewise
polynomials of degree ≤ n. These types of wavelets are studied in (Chui, 1992).
Bibliography
Chui, C. K. 1992. An introduction to wavelets. Academic Press.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Chapter 10
Wavelet Design
We have seen in the previous chapters that orthogonal wavelets can be deﬁned from a mul
tiresolution analysis. This framework provides the basic relations to construct wavelets, and
to compute the fast wavelet transform.
As we discussed in Chapter 9, the scaling and wavelet functions, φ and ψ, are com
pletely characterized by the twoscale sequences (h
n
) and (g
n
), respectively. Therefore, a
direct method to generate new wavelets consists in ﬁnding φ and ψ that satisfy the twoscale
relations, and whose integer translates form orthonormal bases of V
0
and W
0
.
This simple approach can be used to produce examples of wavelets, but it presents two
main diﬃculties: ﬁrst, it does not give a systematic way to ﬁnd the functions φ and ψ; second,
it relies on an orthonormalization step which produces inﬁnitely supported bases.
It is clear that we need a more eﬀective and ﬂexible method to design wavelets. In this
chapter, we present an approach to generate wavelets based on a frequency domain analysis.
We will exploit the fact that the twoscale sequences are the coeﬃcients of a twochannel
discrete ﬁlter bank. Consequently, we can use the ﬁlter design methodology for creating
wavelets.
10.1 Synthesizing Wavelets from Filters
In this section we review the restrictions on the ﬁlter function m
0
, with the purpose of
discovering an expression for it that can be computed.
10.1.1 Conjugate Mirror Filters
The functions m
0
and m
1
can be interpreted as the discrete Fourier transform of a pair of
discrete ﬁlters H = (h
n
) and G = (g
n
), respectively, as we discussed in the previous chapter.
The function m
0
is a low pass ﬁlter for the interval [0, π/2], and the function m
1
is a band
pass ﬁlter for the interval [π/2, π].
From these observations, and from the deﬁnition of φ and ψ in the frequency domain,
we conclude that the main part of the energy of
ˆ
φ and
ˆ
ψ is concentrated respectively in the
intervals [0, π] and [π, 2π].
119
120 CHAPTER 10. WAVELET DESIGN
The fact that m
0
(0) = m
1
(π) and m
1
(0) = m
0
(π), together with the relation m
1
(ω)m
0
(ω)+
m
1
(ω+π)m
0
(ω +π) = 0 make H and G a pair of ﬁlters that are complementary. These ﬁlters
are called conjugate mirror ﬁlters, because their frequency responses are mirror images with
respect to the middle frequency π/2 (also known as the quadrature frequency).
The above means that the wavelet transform essentially decomposes the frequency space
into dyadic blocks [2
j
π, 2
j+1
π] with j ∈ Z.
Note that is possible to construct m
0
and m
1
that are quadrature mirror ﬁlters, but do
not correspond to any functions φ and ψ in L
2
(R) as deﬁned by a multiresolution analysis. In
order to guarantee that inﬁnite product formula (9.6) will converge to the Fourier transform
of a valid φ, the function m
0
must satisfy [m
0
(ω)[
2
+ [m
0
(ω + π)[
2
= 1, the orthogonality
condition for φ( −k). In addition, we have to impose extra conditions on m
0
to ensure that
¸
l
[
ˆ
φ(ω + 2πl[
2
= 1/2π. (See (Daubechies, 1992) for the technical details.)
10.1.2 Conditions for m
0
First, let’s revisit the two main conditions, (9.16) and (9.22), that the function m
0
must
satisfy to generate orthogonal scaling functions and wavelets. They can be summarized as:
Condition 1: m
0
(ω)m
0
(ω) +m
0
(ω +π)m
0
(ω +π) = 1
Condition 2: m
1
(ω)m
0
(ω) +m
1
(ω +π)m
0
(ω +π) = 0
where, in the ﬁrst equation we expanded [m
0
(ω)[
2
= m
0
(ω)m
0
(ω).
Notice that these two condition can be written in matrix form as
m
0
(ω) m
0
(ω +π)
m
1
(ω) m
1
(ω +π)
m
0
(ω)
m
0
(ω +π)
=
1
0
or Mx = y, where M is called the modulation matrix, in the ﬁlter bank theory.
M =
m
0
(ω) m
0
(ω +π)
m
1
(ω) m
1
(ω +π)
We will return to this matrix later, in the more general context of biorthogonal wavelets and
perfect reconstruction ﬁlters.
For orthogonal wavelets, we were able to transform condition 2, into
m
1
(ω) = e
−iω
m
0
(ω +π),
using the quadrature mirror ﬁlter construction (see equation (9.25) of previous Chapter).
Therefore, we only need to determine m
0
(ω) such that Condition 1 is satisﬁed.
The goal now, is to go from Condition 1 to a formula for m
0
.
10.1.3 Strategy for Computing m
0
What we would really like at this point is to obtain a closed form expression for m
0
. Unfor
tunately, this would not be possible. There are no simple formulas for the coeﬃcients (h
n
) of
m
0
. But, we will be able to compute (h
n
) numerically.
10.1. SYNTHESIZING WAVELETS FROM FILTERS 121
Instead of working directly with m
0
, we will consider [m
0
[
2
. We deﬁne the function P(ω)
as the product m
0
(ω)m
0
(ω)
P(ω) =
N
¸
l=0
h
l
e
ilω
N
¸
n=0
h
n
e
−inω
=
N
¸
k=−N
a
k
e
−ikω
(10.1)
Our strategy is to decompose the problem of ﬁnding m
0
satisfying Condition 1, into two
simpler subproblems:
1. Find a function P(ω) that satisfy P(ω) +P(ω +π) = 1;
2. From P obtain m
0
, factoring P(ω) into m
0
(ω)m
0
(ω)
It turns out that it is better to work with P than with m
0
, because using P, the analysis
of Condition 1 is much simpler. This strategy makes possible to ﬁnd an explicit expression for
P(ω), and therefore, the formulas to compute its coeﬃcients (a
k
). From that, we can obtain
m
0
through the spectral factorization of P (i.e. taking a “square root”).
We will see, in the next two chapters, how to derive a closed form expression for P(ω),
respectively in the context of orthogonal scaling functions, where P(ω) = m
0
(ω)m
0
(ω), and
in the more general context of biorthogonal scaling functions, where P(ω) = m
0
(ω) m
0
(ω)
(i.e., the functions m
0
and m
0
can be diﬀerent).
In the rest of this chapter, we will assume that P is known, and we will concentrate on
the details of how to get m
0
from P.
10.1.4 Analysis of P
First, to simplify the exposition, we will make a change of notation. We express re
iω
= z,
with r = [z[, to convert from the ω notation to the z notation. Let P(z) restricted to the unit
circle r = 1 denote P(ω). So, equation (10.1) becomes
P(z) =
N
¸
k=−N
a
k
z
k
(10.2)
There are many things we can say about P, even without knowing the speciﬁc expression
of P.
• P is a Laurent polynomial of degree N, with both positive and negative exponents. This
is apparent in equation (10.2).
• P(z) ≤ 0, is real and nonnegative, because P(z) = [m
0
(z)[
2
(in the orthogonal case).
• P has real coeﬃcients a
k
, k = −N, . . . , N, because m
0
has real coeﬃcients.
• P is symmetric, with a
k
= a
−k
, because m
0
(z) multiplies its conjugate m
0
(z).
122 CHAPTER 10. WAVELET DESIGN
Taking into consideration also Condition 1, we can say more about P. Using the fact that
e
iw+π/2
= −e
iw
, we write Condition 1 in the z notation as:
P(z) +P(−z) = 1 (10.3)
The terms with odd coeﬃcients cancel in equation (10.3) (and there is nothing we can
say about them). But, we can conclude that, except for the constant term, all other even
coeﬃcients of P have to be zero to make (10.3) true.
a
2m
=
1/2 if m = 0
0 if m = 0
(10.4)
In the ﬁlter theory, the function P which satisfy (10.3) is called a halfband ﬁlter. When it
comes from P(z) = [m
0
(z)[
2
, it is the autocorrelation ﬁlter of m
0
. It gives the power spectral
response, and m
0
is the spectral factor. Figure 10.1 shows a graph of P.
P(ω) P(ω +π)
0.1 0.2 0.3 0.4 0.5
0.2
0.4
0.6
0.8
1
Figure 10.1: Halfband Filter
10.1.5 Factorization of P
The function P(z) is a Laurent polynomial of degree N. Since it is symmetric there are N +1
independent coeﬃcients in P. In the next two chapters, we will derive a formula to compute
these coeﬃcients. Assuming that the coeﬃcients of P are known, the remaining problem is
to factor P to obtain m
0
.
A regular polynomial of degree d can be represented in two forms:
B(z) =
d
¸
n=0
b
n
z
n
=
d
¸
i=1
(z −z
i
)
where b
n
are the coeﬃcients of B, and z
i
are the roots of B.
The factorization of the polynomial P can be done by ﬁnding the zeros (roots) of P. There
are several methods for this purpose. One of the most eﬀective of these methods consists in
10.1. SYNTHESIZING WAVELETS FROM FILTERS 123
computing the eigenvalues the companion matrix. The roots of a regular polynomial B of
degree d are the eigenvalues of the d d companion matrix
B =
¸
¸
¸
¸
¸
¸
−b
d−1
b
d
−b
d−2
b
d
−b
1
b
d
−b
0
b
d
1 0 . . . 0 0
0 1 . . . 0 0
.
.
.
.
.
.
.
.
.
0 0 1 0
¸
(10.5)
where B(z) = det(B−zI) is the characteristic polynomial of B.
In order to apply the factorization method, we need ﬁrst to convert the Laurent polynomial
P(z) to the equivalent polynomial z
N
P(z) in regular form.
P(z) =
N
¸
i=0
(z −z
i
)
N
¸
j=0
(z
−1
−z
j
)
z
N
P(z) =
1
2
a
N
2N
¸
i=0
(z −z
i
)
with a
N
= 0.
This polynomial can be factored using any numerical package that implements the eigen
value method discussed above.
The factorization will give us the roots of P. Our knowledge about P will help us to
discriminate between the diﬀerent types of roots:
i. Since P has real coeﬃcients, the complex roots will occur in pairs that are conjugate
(i.e. if z
k
= x + iy is a root, then z
k
= x −iy is also a root). This ensures that, in the
orthogonal case, we will be able to compute the “square root” of m
0
.
ii. The coeﬃcients of P are also symmetric. With real symmetric coeﬃcients a
k
, we have
P(z) = P(1/z). If z
i
is a root, then 1/z
i
is also a root. Thus, when z
i
is inside the unit
circle, 1/z
i
is outside (roots with [z
i
[ = 1, on the unit circle have even multiplicity).
In summary, P has 2N roots. The complex roots come in quadruplets z
j
, z
−1
j
, z
j
, z
−1
j
, and
the real roots come in duplets r
l
, r
−1
l
. Therefore, regrouping these types of roots, we have
z
N
P(z) =
1
2
a
N
J
¸
j=1
(z −z
j
)(z −
1
z
j
)(z −z
j
)(z −
1
z
j
)
K
¸
k=1
(z −z
k
)
2
(z −z
k
)
2
L
¸
l=1
(z −r
l
)(z −
1
r
l
)
Once we obtain the 2N roots of z
N
P(z), it is necessary to separate then, assigning factors
M to the polynomial m
0
(z), and the remaining 2N −M factors to the polynomial m
0
(z), (or
124 CHAPTER 10. WAVELET DESIGN
˜ m
0
(z)). In the orthogonal case, one factor is the transpose of the other, but in the biorthogonal
case, this restriction is not imposed, and the dual scaling functions can be diﬀerent. Note
that, the polynomial m
0
(z) with degree N is not unique even for orthogonal scaling functions.
In the next two chapters we will discuss the rules for separating the roots of P, and eﬀects of
diﬀerent choices.
The function m
0
is completely determined after we assign a set of roots from P to it. But,
we still don’t have the coeﬃcients (h
n
) of m
0
, since it is in product form. Given the roots z
k
of the degree N polynomial
m
0
(z) = c
N
¸
k=1
(z −z
k
) (10.6)
the coeﬃcients h
n
can be computed with an iterated convolution. Then, we normalize the
coeﬃcients multiplying them by a scaling constant.
The coeﬃcients (h
n
) give the ﬁlter m
0
, and from them we get the coeﬃcients (g
n
) of the
ﬁlter m
1
, using (9.27).
Now, let’s summarize the steps required to generate a scaling and wavelet functions using
a ﬁlter design methodology:
1. Choose a Polynomial P satisfying Condition 1. Generate its coeﬃcients a
k
using an
explicit formula (see Chapters 11 and 12).
2. Find the roots z
k
of P by a factorization method.
3. Separate the roots z
k
into two sets. Assign one set of roots to the polynomial factor m
0
and the other set of roots to m
0
(or to m
0
).
4. Compute the coeﬃcients h
n
of m
0
from its roots, using iterated convolution.
5. Obtain the coeﬃcients g
n
of m
1
from h
n
.
10.1.6 Example (Haar Wavelet)
A concrete example will illustrate the wavelet design methodology.
We choose the following degree 1 polynomial:
P(z) =
1
4
z
−1
+
1
2
+
1
4
z
This is the lowest degree polynomial which satisﬁes Condition 1
P(z) +P(−z) = 1
We convert P to regular form, multiplying by z
zP(z) =
1
4
1 + 2z +z
2
10.1. SYNTHESIZING WAVELETS FROM FILTERS 125
This polynomial has one root at z = −1 with multiplicity 2. Therefore P is factored into
P(z) =
1
2
z
−1
(z + 1)
1
2
(z + 1)
=
1 +z
−1
2
1 +z
2
The lowpass ﬁlter is
m
0
(ω) =
1
2
+
1
2
e
−iω
=
1
√
2
1
√
2
+
1
√
2
e
−iω
with coeﬃcients h
0
= h
1
= 1/
√
2 which gives the Haar scaling function
φ(x) =
√
2
1
√
2
φ(2x) +
1
√
2
φ(2x −1)
= φ(2x) +φ(2x −1)
Figure 10.2 shows a plot of m
0
(ω).
0.1 0.2 0.3 0.4 0.5
0.2
0.4
0.6
0.8
1
Figure 10.2: Haar ﬁlter m
0
Notice that, since
m
0
(ω) =
1 +e
−iω
2
=
e
−iω/2
+e
iω/2
2
e
−iω/2
= cos(ω/2)e
−iω/2
the scaling function in frequency domain
ˆ
φ is the sinc function
ˆ
φ(ω) =
1
√
2π
∞
¸
j=1
cos(2
−j
ω/2)e
−i2
−j
ω/2
=
1
√
2π
e
−iω/2
sin(ω/2)
ω/2
which is the Fourier transform of the box function φ(x) = 1 for x in the interval [0, 1] (this
is the Haar scaling function, as expected). A derivation of this formula can be found in
(Daubechies, 1992), pg. 211. Figure 10.3 shows a plot of [
ˆ
φ(ω)[.
126 CHAPTER 10. WAVELET DESIGN
7.5 5 2.5 2.5 5 7.5
0.2
0.4
0.6
0.8
1
Figure 10.3: Magnitude of the frequency response of
ˆ
φ(ω) for the Haar function
The coeﬃcients of the highpass ﬁlter m
1
are given by g
n
= (−1)
n
h
1−n
g
0
=
1
√
2
, g
1
= −
1
√
2
The function
ψ(x) =
√
2
1
√
2
φ(2x) −
1
√
2
φ(2x −1)
= φ(2x) −φ(2x −1)
is the Haar wavelet.
10.2 Properties of Wavelets
In the last section we have seen how to synthesize wavelets in the frequency domain. The
procedure was based on the factorization of a polynomial P(ω). In this section we will
investigate how the properties of the scaling and wavelet functions relate to characteristics of
P. This connection will reveal the “design variables” that we can control to generate wavelets
with desired properties.
10.2.1 Orthogonality
The multiresolution analysis, by deﬁnition, leads to orthogonal scaling functions and wavelets.
In this way, the functions ¦φ
j,n
; n ∈ Z¦, and ¦ψ
j,n
; n ∈ Z¦ are, respectively, orthonormal basis
of the subspaces V
j
and W
j
, for all j ∈ Z. The subspaces W
j
are mutually orthogonal, and
the projection operators produce optimal approximations in the L
2
(R) sense.
Orthogonality is very a desirable property, but it imposes severe restrictions on the can
didate scaling and wavelet functions. We will see in Chapter 12 that orthogonality can be re
placed by biorthogonality, which is less restrictive, giving us more freedom to design wavelets.
10.2. PROPERTIES OF WAVELETS 127
10.2.2 Support of φ and ψ
In this chapter we have been concerned only with compactly supported basis functions.
The support width of the scaling and wavelet functions is determined, respectively, by the
length of the coeﬃcient sequences (h
n
) and (g
n
). More precisely, if h
n
= 0 for n < 0, n > N,
then the support of φ is the interval [0, N]. When ψ is deﬁned as in (9.28), then the support
of ψ is the interval [−M, M + 1], where M = (N −1)/2.
The length of the coeﬃcient sequences (h
n
) and (g
n
) is given by the degree of the polyno
mial m
0
(z). Consequently, the degree of the product polynomial P(z), determines the support
width of the associated basis functions.
10.2.3 Vanishing Moments and Polynomial Reproduction
Two related properties of a multiresolution analysis are: the vanishing moments of the wavelet
functions; and the ability of the scaling functions to reproduce polynomials. These are prob
ably the most important properties in the design of wavelets.
Given a mother wavelet function ψ(x) with p vanishing moments, that is
x
ψ(x)dx = 0 (10.7)
for = 0, 1, . . . , p −1. Then, the following equivalent properties are veriﬁed:
• The wavelets ψ(x −k) are orthogonal to the polynomials 1, x, . . . , x
p−1
.
• The combination of scaling functions φ(x − k) can reproduce exactly the polynomials
1, x, . . . , x
p−1
.
These properties are important in the rate of convergence of wavelet approximations of
smooth functions and in singularity detection using wavelets.
Vanishing moments are also a necessary (but not suﬃcient) condition for a wavelet to be
p −1 times continuously diﬀerentiable, i.e ψ ∈ C
p−1
.
In order to make the connection between the number of vanishing moments of a wavelet
and the condition imposed on the ﬁlter function m
1
(ω), we note that equation (10.7) implies
that
d
dω
ˆ
ψ
ω=0
= 0 (10.8)
for < p. (This can be veriﬁed expanding ψ in a Taylor series.) But, we know from (9.26)
that
ˆ
ψ(ω) = e
−iω/2
m
0
(ω/2 +π)
ˆ
φ(ω/2), and also from (9.5) that
ˆ
φ(0) = 0, this means that
m
0
is p −1 times diﬀerentiable in ω = π. Furthermore
d
dω
m
0
ω=π
= 0 (10.9)
for < p.
128 CHAPTER 10. WAVELET DESIGN
As a consequence of the above, m
0
must have a zero of order p at ω = π. Therefore, m
0
is of the form
m
0
(ω) =
1 +e
iω
2
p
Q(ω) (10.10)
with m
0
∈ C
p−1
and Q ∈ C
p−1
.
We already knew that a wavelet should integrate to zero, i.e. it should have at least one
vanishing moment. We also knew that m
0
should have at least one zero at π, i.e. it should
be of the form (10.10) with p >= 1.
The zero at π from m
0
produces factor (
1+z
2
)
2p
in the polynomial P.
10.2.4 Regularity
Smooth basis functions are desired in applications where derivatives are involved. Smoothness
also corresponds to better frequency localization of the ﬁlters.
The local regularity of a function at a point x
0
can be studied using the notion of Lipschitz
continuity. A function f(x) is C
α
at x
0
, with α = n+β, n ∈ N and 0 < β < 1, if f is n times
continuously diﬀerentiable at x
0
and its nth derivative is Lipschitz continuous with exponent
β.
[f(x)
(n)
−f(x
0
)
(n)
[ ≤ C[x −x
0
[
β
(10.11)
For global regularity, the above must hold for all x
0
.
Typically, compactly supported scaling functions and wavelets are more regular in some
points than in others. They also have noninteger Lipschitz exponents.
The regularity of scaling functions and wavelets is diﬃcult to determine. The techniques
for this investigation involve either the estimation of the decay of
ˆ
φ in the frequency domain,
or the estimation of the convergence rate of the recursive construction of φ in the spatial
domain. The ﬁrst class of techniques is more suitable for a global analysis, while the second
class of techniques is better for a local analysis.
The regularity of φ and ψ is related with the number of vanishing moments of ψ, and
consequently is related with the multiplicity p of the zeros at π of m
0
. The regularity is at
most p − 1 if we consider integer exponents, and at most p − 1/2 if we consider fractional
exponents. We remark that in many cases, the regularity is much smaller than p −1 because
of the inﬂuence of the factor Q.
Another important observation is that, for a ﬁxed support width of φ, the choice of fac
torizations of P that leads to maximum regularity is diﬀerent from the choice with maximum
number of vanishing moments for ψ.
10.2.5 Symmetry or Linear Phase
Symmetry is important in many applications of wavelets, such as image processing and com
puter vision. This property can be exploited in the quantization of images for compression,
and in the boundary processing of ﬁnite signals using symmetric extension (i.e. periodicization
by reﬂection of the data).
10.2. PROPERTIES OF WAVELETS 129
The symmetry of the basis functions implies in symmetric ﬁlters. It is easy to see from
(9.1), that φ will be symmetric only if the coeﬃcients (h
n
) are symmetric.
Symmetric ﬁlters are called linear phase ﬁlters. This is because ﬁlters with symmetric
coeﬃcients have a linear phase response.
The phase angle (or argument) of a complex number z = a + bi is the angle between the
vector (a, b) and the horizontal axis. The tangent of this angle is the ratio of the imaginary
and real parts of z:
tan(ω) =
·(z)
'(z)
=
b
a
. (10.12)
The same deﬁnition extends to a complex function m(z). This notion becomes more evident if
we use the polar representation of a complex number z = [z[e
iθ(ω)
, where [z[ is the magnitude
of z and θ(ω) is the phase angle of z.
A ﬁlter function m(ω) =
¸
k
a
k
e
ikω
is said to have linear phase if θ(ω) is of the form Kω
where K is a constant (i.e. it is linear in ω). This means that m(ω) can be written as
m(ω) = ce
iKω
M(ω) (10.13)
where c is a complex constant, and M is a real valued function of ω, not necessarily 2πperiodic.
Linear phase ﬁlters have symmetric or antisymmetric coeﬃcients around the central coeﬃcient
N/2. Therefore, h
k
= h
N−k
for symmetric ﬁlters and h
k
= −h
N−k
for antisymmetric ﬁlters.
Note that, when ﬁlter coeﬃcients a
k
are symmetric, or antisymmetric, the phase is linear
because terms with same coeﬃcients can be grouped. For example, if the ﬁlter is symmetric
and has an even number of coeﬃcients
m(ω) =
N/2
¸
k=0
h
k
(e
−ikω
+e
−i(N−k)ω
) = e
−i
N
2
ω
¸
N/2
¸
k=0
h
k
(e
ikω
+e
−ikω
)
¸
= e
−i
N
2
ω
¸
N/2
¸
k=0
2h
k
cos(kω)
¸
.
We can see that in this case the phase is N/2ω. Similar expressions hold for the case of
symmetric ﬁlters with odd number of coeﬃcients, as well as for antisymmetric ﬁlters.
We can conclude from the above discussion that, in order to obtain symmetric (anti
symmetric) scaling functions and wavelets, it is necessary to factor the polynomial P into
symmetric polynomials m
0
(and m
0
). Therefore, in the factorization of P the roots z
j
and
z
−1
j
must stay together. We remark that, although P is a symmetric polynomial, it obviously
can be factorized into two polynomials that are not symmetric.
In the next chapter, we will demonstrate that symmetry and orthogonality are incompati
ble properties, except for the case of Haar wavelets. This is because orthogonal ﬁlters require
that the roots z
j
and z
−1
j
are separated and assigned one to m
0
and the other to m
0
.
Nonetheless, despite of the orthogonality restriction, it is possible to design ﬁlters that
are close to linear phase and result in basis functions that are least assymmetric. This can
be accomplished by selecting from each pair of roots z, z
−1
, the one which contributes the
least to nonlinearity of the ﬁlter phase. For this purpose, we have to compute the nonlinear
contribution of the factors (z −z
k
) and (z −z
−1
k
) to the total phase of m
0
.
130 CHAPTER 10. WAVELET DESIGN
10.2.6 Other Properties
There are several other properties that are relevant to applications, but are only achievable
by some types of wavelets. Below we discuss a few of them.
• Analytic Form: The analytic expression for the scaling function and wavelet is, in
general, not available. These functions are deﬁned indirectly through the ﬁlter coeﬃ
cients (h) and (g). Nonetheless, the deﬁnition of the scaling function and wavelet in
analytic form is very useful in many applications. One important example of wavelet
basis with closed form analytic expression is the family of BSpline wavelets.
• Interpolation: The interpolation property makes it trivial to ﬁnd the values of the
scaling coeﬃcients from samples of the function. When the scaling function φ(k) = δ
k
,
for k ∈ Z, the coeﬃcients c
j
k
=< f, φ
j
> of the projection Proj
V
j
(f) of f on V
j
, are just
the values f(2
j
k) of f sampled on a grid with spacing 2
−j
.
• Rational Coeﬃcients: Filter coeﬃcients that are rational numbers make computer
implementation more eﬃcient and precise. It is even better if the coeﬃcients are dyadic
rational numbers. In this case, division by a power of two corresponds to shifting bits.
10.3 Classes of Wavelets
From our discussion of the various properties of wavelets, we can conclude that it is not
possible to achieve all of them at the same time. The reason is that diﬀerent properties may
be incompatible with each other, and imply in conﬂicting requirements on the ﬁlter function
m
0
. For example, on one hand, the support width is proportional to the number of ﬁlter
coeﬃcients and consequently small support requires a lowdegree m
0
. On the other hand,
the number of vanishing moments depends on the multiplicity of the zero at π of m
0
, and
consequently a large number of vanishing moments requires a high degree m
0
.
To make things even more complicated, there is a great interdependency among the dif
ferent requirements. For instance, m
0
is of the form 1/2(1 −e
iω
)
n
Q(ω). While the ﬁrst factor
controls the vanishing moments, the second factor is necessary to guarantee orthogonality. If
we increase the degree of the ﬁrst, we also have to increase the degree of the second.
Naturally, as in any design process, there is always a tradeoﬀ between diﬀerent properties
relative to their importance in a given application.
In general terms, there are two main classes of wavelets that combine some of the above
properties in the best way.
10.3.1 Orthogonal Wavelets
Orthogonal wavelets are the best choice in some cases. Unfortunately, except for the Haar
case, orthogonal wavelets cannot be symmetric.
It is possible to design orthogonal wavelets with more or less assymmetry.
Other properties that are important for orthogonal wavelets are: support width, number
of vanishing moments, and regularity.
10.4. COMMENTS AND REFERENCES 131
In Chapter 11, we will discuss how to generate compactly supported wavelets, and how to
control the various properties.
10.3.2 Biorthogonal Wavelets
Biorthogonal wavelets have most of the qualities of orthogonal wavelets, with the advantage
of being more ﬂexible.
There are many more biorthogonal wavelets than orthogonal ones. For these reason, they
make possible a variety of design options and constitute class of wavelets most used in practical
applications.
Biorthogonal wavelets can have symmetry. The are associated with perfect analysis /
reconstruction ﬁlter banks.
In Chapter 12 we will discuss how to generate biorthogonal wavelets.
10.4 Comments and References
The framework for wavelet design in the frequency domain has been developed since the
early days of wavelet research by Yves Meyer, (Meyer, 1990), and Stephane Mallat, (Mallat,
1989), among others. The initial methods investigated could only produce orthogonal wavelets
with inﬁnite support. The major breakthrough was made by Ingrid Daubechies in 1988,
(Daubechies, 1988).
Gilbert Strang presents in his book, (Strang & Nguyen, 1996), a complete and detailed
framework for generating wavelets using the ﬁlter design methodology.
Carl Taswell developed a computational procedure to generate wavelet ﬁlters using the
factorization of the polynomial P, (Taswell, 1997).
Bibliography
Daubechies, I. 1988. Orthonormal Bases of Compactly Supported Wavelets. Comm. Pure
Applied Math., XLI(41), 909–996.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Mallat, S. 1989. Multiresolution approximation and wavelets. Trans. Amer. Math. Soc., 315,
69–88.
Meyer, Y. 1990. Construction de bases orthonorm´ees d’ondelettes. Colloq. Math., 60/61,
141–149. bib.
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Taswell, Carl. 1997 (Dec.). The Systematized Collection of Wavelet Filters Computable by
Spectral Factorization of the Daubechies Polynomial. Tech. rept. Computational Tool
smiths, www.toolsmiths.com.
Chapter 11
Orthogonal Wavelets
In this chapter we will investigate the construction and design of compactly supported or
thogonal wavelets. We will derive a closed form expression for the polynomial P, introduced
in the previous chapter, and we will show how to factor P in order to generate orthogonal
wavelets with diﬀerent properties.
11.1 The Polynomial P
We saw in the previous chapter that the trigonometric polynomial P(z), with z = e
iω
is
the key element in the construction of compactly supported wavelets. There, we described a
method to synthesize wavelets by factoring P as a product of two polynomials. But, in order
to apply this method, we need ﬁrst an explicit expression for P, i.e. a formula to compute
the coeﬃcients a
k
of P(e
iω
)
¸
n
k=−n
a
k
e
ikω
.
To ﬁnd a closed form expression for P we recall the two conditions that P must satisfy in
the context of orthogonal ﬁlter banks.
• P is a reciprocal product polynomial
P(e
iω
) = [m
0
(e
iω
)[
2
(11.1)
where m
0
has a zero of order n at ω = π (or e
iω
= −1).
• P is a halfband ﬁlter
P(e
iω
) +P(e
iω+π
) = 1 (11.2)
We need to ﬁnd a polynomial P of the form (11.1), such that equation (11.2) is true. We
will address these two conditions one at time, and we will combine them afterwards to obtain
an expression for P
11.1.1 P as a Product Polynomial
The polynomial P(z) = [m
0
(z)[
2
is generated from m
0
, which in turn, is of the form
m
0
(e
iω
) =
1 +e
iω
2
n
Q
n
(e
iω
) (11.3)
133
134 CHAPTER 11. ORTHOGONAL WAVELETS
The factor (1 +e
iω
)
n
is a consequence of the fact that m
0
has a zero of order n at π.
We get an expression for P by substituting equation (11.3) into (11.1)
P(e
iω
) = [m
0
(e
iω
)[
2
(11.4)
= m
0
(e
iω
) m
0
(e
iω
) (11.5)
= m
0
(e
iω
) m
0
(e
−iω
) (11.6)
=
1 +e
iω
2
n
1 +e
−iω
2
n
[Q
n
(e
iω
)[
2
(11.7)
Since [1 +e
−iω
[
2
/2 = (1 + cos ω)/2, the condition of n zeros at π appears in P as
P(ω) =
1 + cos(ω)
2
n
[Q
n
(e
iω
)[
2
(11.8)
The factor [Q
n
(e
iω
)[
2
in P gives extra degrees of freedom to satisfy the halfband property
expressed in equation (11.2).
A question comes up at this point. Is it possible to factor every polynomial P(e
iω
) ≥ 0
into a product [m
0
(e
iω
)[
2
? The answer is yes. This is guaranted by the Riesz Lemma.
Lemma 11.1 (Riesz). If A(ω) ≥ 0 is a trigonometric polynomial of degree M, invariant
under the substitution ω →−ω; A has to be of the form
A(ω) =
M
¸
m=0
a
m
cos mω
with a
m
∈ R. Then there exists a polynomial B of order M
B(ω) =
M
¸
m=0
b
m
e
imω
with b
m
∈ R, such that [B(ω)[
2
= A(ω)
A proof of this lemma can be found in (Daubechies, 1992), page 172.
The above result guarantees that a factorization of P using the method described in the
previous chapter is possible. In fact, the proof is constructive and follows the steps necessary
for the spectral factorization. Note that, P is a polynomial in cos ω as enunciated by the
lemma.
11.1.2 P and the Halfband Property
We derived above a partial expression for P. We found that P has to be decomposed into
two factors P(ω) = p
1
(ω)q
1
(ω), where p
1
(ω) = (1 + cos ω)
n
/2 provides the zeros at π and q
1
is chosen such that the halfband property is veriﬁed.
To relate P and the halfband ﬁlter property in equation (11.2), we use the following
theorem:
11.1. THE POLYNOMIAL P 135
Theorem 1 (Bezout). If p
1
and p
2
are two polynomials with no common zeros, then there
exist unique polynomials q
1
and q
2
such that
p
1
(y)q
1
(y) +p2(y)q
2
(y) = 1 (11.9)
where p1, p
2
, q
1
, q
2
, are of degree n
1
, n
2
, n
2
−1, n
1
−1, respectively.
A proof of this theorem can be found (Daubechies, 1992), page 169.
For reasons that will become clear later, let’s apply Bezout’s theorem to the particular
case where p
1
(y) = (1 − y)
N
and p
2
(y) = y
N
. The theorem says that there exists unique
polynomials q
1
, q
2
of degree ≤ N −1, such that
(1 −y)
N
q
1
(y) +y
N
q
2
(y) = 1 (11.10)
When we substitute y for 1 −y, equation (11.10) becomes
(1 −y)
N
q
2
(1 −y) +y
N
q
1
(1 −y) = 1 (11.11)
Because q
1
and q
2
are unique, necessarily q
1
(y) = q
2
(1 −y), and also q
2
(y) = q
1
(1 −y).
This allows us to ﬁnd an explicit formula for q
1
.
q
1
(y) = (1 −y)
−N
[1 −y
N
q
1
(1 −y)]
= (1 −y)
−N
−y
N
R(y)
=
N−1
¸
k=0
N +k −1
k
y
k
−y
N
R(y)
(11.12)
where we expanded the ﬁrst N terms of the Taylor series for (1 −y)
−N
. Since the degree of
(q
1
) ≤ N −1, q
1
is equal to its Taylor expansion truncated after N terms. Therefore, we can
drop y
N
R(y) and make q
1
(y) = (1 −y)
−N
. This gives a closed form expression for q
1
, which
is the unique lowest degree solution solution for (11.10).
(1 −y)
N
B
n
(y) +y
N
B
n
(1 −y) = 1 (11.13)
where B
n
(y) is the binomial series for (1 −y)
−N
, truncated after N terms
B
n
(y) =
N−1
¸
k=0
N +k −1
k
y
k
(11.14)
Higher degree solutions for (11.10) also exist with q
1
(y) = B
n
(y) −y
N
R(y), provided that
R(y) satisﬁes R(y) −R(1 −y) = 0.
11.1.3 The Expression of P
Now we put together the two results above to obtain an expression for P. So far we know the
following:
136 CHAPTER 11. ORTHOGONAL WAVELETS
i. P is of the form P(ω) =
1+cos(ω)
2
n
[Q
n
(e
iω
)[
2
ii. A polynomial of the form P(y) = (1 −y)
n
B
n
(y) has the halfband property.
To relate (i) and (ii) we make a change of variables
e
iω
+e
−iω
2
= cos(ω) = 1 −2y
which gives
y =
1 −cos(ω)
2
and 1 −y =
1 + cos(ω)
2
Therefore, we can write P as
P(ω) =
1 + cos(ω)
2
n n−1
¸
k=0
n +k −1
k
1 −cos(ω)
2
k
(11.15)
It is clear that B
n
is the solution for [Q
n
[
2
which makes P satisfy the halfband property.
Making another change of variables we obtain an expression for P in terms of the complex
variable z = e
iω
.
1 + cos(ω)
2
=
1 +e
iω
2
1 +e
−iω
2
=
1 +z
2
1 +z
−1
2
1 −cos(ω)
2
=
1 −e
iω
2
1 −e
−iω
2
=
1 −z
2
1 −z
−1
2
Substituting this into equation (11.15) we arrive at the closed form expression for P in
the zdomain
P(z) =
1 +z
2
n
1 +z
−1
2
n n−1
¸
k=0
n +k −1
k
1 −z
2
k
1 −z
−1
2
k
(11.16)
Note that to produce P in the form of a Laurent polynomial P(z) =
¸
n
k=−n
a
k
z
k
, it is
necessary to expand the factors in each term, and then collect the resulting terms with same
exponent. This would be required to factorize P(z) directly, but we will see that it is more
eﬃcient to compute the roots of P(y) = (1 − y)
n
B
n
(y), and make the change of variables
afterwards.
11.1.4 The Factorization of P
The two forms of the polynomial P(z) and P(y) are related by the change of variables
(z +z
−1
)/2 = 1 −2y (11.17)
11.1. THE POLYNOMIAL P 137
We write this as a composition of conformal maps (f ◦ g)(z) = y, where f is the Joukowski
transformation (Markushevich, 1977),
x = f(z) =
z +z
−1
2
z = f
−1
(x) = x ±
x
2
−1
and g is an aﬃne transformation
y = g(x) = (1 −x)/2
x = g
−1
(y) = 1 −2y
The expressions for the change of variables are
y = g(f(z)) = (1 −(z +z
−1
)/2)/2 (11.18)
z = f
−1
(g
−1
(y)) = 1 −2y ±
(1 −2y)
2
−1 (11.19)
Notice that this change of variables in (11.19) associates two values of z for each value of
y. Equation (11.18) is, in fact, a quadratic equation for z. It is clear from (11.17) that one z
is inside the unit circle and the other 1/z is outside.
The polynomial P(y) = (1 − y)
n
B
n
(y), from subsection 11.1.3, has degree 2n − 1, but
because the change of variables, it will result in a polynomial P(z) of degree 4n − 2. Thus,
P(z) will have 4n − 2 roots. From these, 2n roots come from the ﬁrst factor (1 − y)
n
, and
correspond the the multiple root at z = −1. The remaining 2n −2 complex roots come from
the binomial factor B
n
(y).
The polynomial [Q
n
(e
iω
)[
2
is a reciprocal polynomial with real coeﬃcients. Thus, its
roots are in reciprocal and complex conjugate pairs. The change of variables z = f
−1
(g
−1
(y))
yields a doublyvalued solution with reciprocal pairs ¦z, z
−1
¦. The polynomial [Q
n
(z)[
2
, ex
pressed in regular form, can be factored by regrouping these pairs as complex quadruplets
¦z, z
−1
, ¯ z, ¯ z
−1
¦, and real duplets ¦r, r
−1
¦
[Q
n
(z)[
2
=
K
¸
i=1
U(z; z
i
)
L
¸
j=1
V (z; r
j
) (11.20)
U(z; z
i
) = (z −z
i
)(z −z
−1
i
)(z − ¯ z
i
)(z − ¯ z
−1
i
) (11.21)
U(z; r
j
) = (z −r
j
)(z −r
−1
j
) (11.22)
where K = (n −1)/2 and L = (n −1) mod 2.
The factored polynomial z
2n−1
P(z), in regular form, is obtained by including the multiple
zeros at z = −1.
z
2n−1
P(z) = (z + 1)
2n
K
¸
i=1
U(z; z
i
)
L
¸
j=1
V (z; r
j
) (11.23)
The ﬁlter function m
0
(z) can be computed from this factorization by noting that
P(z) = [m
0
(e
iω
)[
2
= m
0
(e
iω
)m
0
(e
iω
) = m
0
(e
iω
)m
0
(e
−iω
) = m
0
(z)m
0
(z
−1
).
138 CHAPTER 11. ORTHOGONAL WAVELETS
m
0
(z) = c
K
¸
i=1
(z −z
i
)(z − ¯ z
i
)
L
¸
j=1
(z −r
j
) (11.24)
where c is a constant, and z
i
and r
j
are selected from the elements of each reciprocal pair
¦z, z
−1
¦.
In consequence of the special structure of P, it is more eﬃcient to compute the 2n−1 roots
of P(y) ﬁrst, and using a change of variables, generate the 4n −2 roots of P(z). The roots of
m
0
are then selected from those roots. Therefore, the spectral factorization of m
0
is done in
two steps:
1. Find the n − 1 roots of B
n
(y), and make a change of variables to produce the 2n − 2
roots of [Q
n
(z)[
2
. From these, select appropriately n −1 roots and assign them to m
0
.
2. Include the n roots at z = −1 from the factor (
1+z
2
)
n
of (
1+z
2
)
n
(
1+z
−1
2
)
n
, and assign
them also to m
0
.
The orthogonal ﬁlter function m
0
(z) will have these 2n −1 roots.
11.1.5 Analysis of P
The family of orthogonal ﬁlters generated from the polynomial P(y) = (1−y)
n
B
n
(y) is called
maxﬂat in the ﬁlter design literature. This is because the factor (1 − y)
n
give a maximum
number of zeros at y = 1 (or z = −1). Consequently, the ﬁlter response of the ﬁlter is
maximally ﬂat at ω = π.
In fact, the polynomial P of degree 2n−1 is determined from n conditions at y = 1 and at
y = 0. P(y) is the unique polynomial with 2n coeﬃcients which satisﬁes these n conditions at
the endpoints, i.e. P(y) and its ﬁrst n −1 derivatives are zero at y = 0 and at y = 1, except
that P(0) = 1.
Figure 11.1 shows a graph of P(y) for n = 2 and 4. Observe that P decreases monotonically
from P(0) = 1 to P(1) = 0. The slope of P at y =
1
2
is proportional to
√
n. As n increases,
P becomes simultaneously steeper at the midpoint and ﬂatter at both endpoints.
0.2 0.4 0.6 0.8 1.0
0.2
0.4
0.6
0.8
1
Figure 11.1: P
11.2. EXAMPLES OF ORTHOGONAL WAVELETS 139
This property is very important because it means that P (and its “square root” m
0
) as a
ﬁlter will preserve low frequencies (near ω = 0) and will cut oﬀ high frequencies (near ω = π).
Gilbert Strang studied the asymptotic behavior of the zeros of P(y) as n →∞. In (Strang
& Shen, 1996), there is an insightful analysis of the factors B
n
(y) in P(y) = (1 − y)
n
B
n
(y),
and [Q
n
(z)[
2
in P(z) = [(
1+z
2
)
n
[
2
[Q
n
(z)[
2
. We recall that P(y) and P(z) are related by the
change of variables (1 − 2y) = (z + z
−1
)/2. Also, it is shown that, as n → ∞, the roots of
B
n
(y) approach the limiting curve [4y(1−y)[ = 1, whereas the roots of [Q
n
(z)[
2
approach the
curve [z −z
−1
[ = 2.
Figure 11.2 shows a plot of the graphs ot these two curves in the complex plane. Note
that, because the change of variables yields a doubly valued solution, the curve in y actually
corresponds to two curve segments [z + 1[ =
√
2 and [z − 1[ =
√
2. These curves meet at
z = ±i, which correspond to y = 1/2.
−1 −0.5 0 0.5 1 1.5 2
−1
−0.5
0
0.5
1
−0.6 −0.4 −0.2 0 0.2 0.4 0.6
−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
(a) (b)
Figure 11.2: Limit curves for the roots of B
n
(y) and [Q
n
(z)[
2
(from (Strang & Shen, 1996))
11.2 Examples of Orthogonal Wavelets
In this section we show how to generate, from the polynomial P, particular families of or
thogonal wavelets.
We have seen in the previous section that the factorization of P(z) produces roots that
can be grouped as quadruplets of complex zeros (z −z
i
)(z −z
−1
i
)(z −¯ z
i
)(z −¯ z
−1
i
) and duplets
of real zeros (z − r
j
)(z − r
−1
j
). In the spectral factorization of P(z) = [m
0
(z)[
2
, we have to
choose from each pair of reciprocal roots ¦z
k
, z
−1
k
¦ of P, either z
k
or z
−1
k
as a root of m
0
.
Also, if we require that m
0
is a polynomial with real coeﬃcients, complex conjugate pairs
must stay together. Therefore, in this case, we can select either ¯ z
k
with z
k
or ¯ z
−1
k
with z
−1
k
.
Even with these restrictions for separating the roots of P(z), we have many choices to
taylor m
0
according to design constraints. Typically, for P of degree m, there are 2
m
4
diﬀerent
choices for m
0
. Let’s now take a look at the most important ones.
140 CHAPTER 11. ORTHOGONAL WAVELETS
11.2.1 Daubechies Extremal Phase Wavelets
The simplest systematic choice for designing orthogonal wavelets produces the so called ex
tremal phase wavelets. This corresponds to selecting always the roots inside (or outside) the
unit circle, with [z[ < 1 (or [z[ > 1), from each reciprocal pair of roots ¦z,
1
z
¦ of P.
The resulting orthogonal wavelets will have the minimum (or maximum) phase among
all compactly supported wavelets of degree m. This corresponds to scaling functions and
wavelets that are the most assymmetric basis functions. We remark also that the minimum
and maximum phase are complementary choices. They lead to m
0
and the complex conjugate
of m
0
. Thus, the basis functions are mirror images of each other for these two choices.
Example 16 (Daubechies Wavelets of Order 2). In this case n = 2. Therefore P(y) =
(1 −y)
2
B
2
(y), with B
2
(y) = 1 + 2y.
P(y) = (1 −y)
2
(1 + 2y) (11.25)
The change of variables y →z gives
P(z) =
1 +z
2
2
1 +z
−1
2
2
1
2
−z + 4 −z
−1
(11.26)
The factor B
2
(y) has one root at y = −1/2. Equation (11.17) for the change of variables
gives the roots of [Q
2
(z)[
2
at z = 2 ±
√
3. The factor (1 − y)
2
will produce the four zeros at
z = −1 of P. The roots of P(z) include all these zeros, and are (−1, −1, −1, −1, 2−
√
3, 2+
√
3).
The choice of the root for Q
2
(z) inside the unit circle z = 2 −
√
3 leads to the minimum
phase m
0
with roots (−1, −1, 2 −
√
3) Figure 11.3 shows the roots of P and m
0
.
4
Figure 11.3: Roots of P(z) with n = 2.
The ﬁlter function m
0
is then
m
0
(z) =
1 +z
2
2
Q
2
(z) (11.27)
= c(1 +z
−1
)
2
(1 −(2 −
√
3)z
−1
) (11.28)
=
1
4
√
2
(1 −
√
3) + (3 +
√
3)z
−1
+ (3 −
√
3)z
−2
+ (1 −
√
3)z
−3
(11.29)
= h
0
+h
1
z
−1
+h
2
z
−2
+h
3
z
−3
(11.30)
11.2. EXAMPLES OF ORTHOGONAL WAVELETS 141
The coeﬃcients ¦h
n
¦ of m
0
are approximately 0.4830, 0.8365, 0.2241, −0.1294.
Figure 11.4 shows a plot of the Daubechies scaling function φ
D,2
and wavelet ψ
D,2
. Figure
11.5 shows the graphs of the Fourier Transform of the Daubechies scaling function φ
D,2
and
wavelet ψ
D,2
.
0.5 1 1.5 2 2.5 3
0.25
0.25
0.5
0.75
1
1.25
0.5 1 1.5 2 2.5 3
1
0.5
0.5
1
1.5
φ
D,2
ψ
D,2
i
Figure 11.4: Daubechies (D2) Scaling Funtion and Wavelet
7.5 5 2.5 2.5 5 7.5
0.2
0.4
0.6
0.8
1
7.5 5 2.5 2.5 5 7.5
0.2
0.4
0.6
0.8
1
ˆ
φ
D,2
ˆ
ψ
D,2
Figure 11.5: Fourier Transform of (D2) Scaling Funtion and Wavelet
11.2.2 Minimal Phase Orthogonal Wavelets
The choice of m
0
having all roots with absolute value less (or greater) than one leads to
wavelets with a very marked assymmetry.
Symmetry and orthogonality are conﬂicting requirements for compactly supported wavelets.
It can be shown that the Haar basis constitute the only scaling functions and wavelets with
compact support which are orthogonal.
We are not going to demonstrate this result, but it is intuitive to understand, from the
requirements on ﬁlter m
0
, why orthogonality and symmetry are incompatible. Recall from
the rules for separating the roots of P(z) that:
• for orthogonal ﬁlters, z and z
−1
must go separately; and
142 CHAPTER 11. ORTHOGONAL WAVELETS
• for symmetric ﬁlters z and z
−1
myst stay together.
The above facts imply that a symmetric orthogonal FIR ﬁlter m
0
(z) can only have two
nonzero coeﬃcients. In this case,
P(z) = (
1 +z
2
)(
1 +z
−1
2
),
which corresponds to the Haar wavelets with
m
0
(z) = (1 +z
−1
)/
√
2.
Nonetheless, it is possible to construct compactly supported orthogonal wavelets that are
less assymmetric than the ones in the previous subsection.
In order to generate the least assymmetric basis functions, we have to select the roots of
P such that m
0
is as close as possible to linear phase. Therefore, we need to estimate the
contribution of each root to the phase nonlinearity of m
0
.
Since m
0
is of the form
m
0
(ω) =
1 +e
−iω
2
n
¸
(e
−iω
−z
)(e
−iω
− ¯ z
)
¸
k
(e
−iω
−r
k
) (11.31)
where z
, ¯ z
, r
k
are the roots of m
0
and we have substituted z = e
−iω
in equation (11.23). The
phase of m
0
can be computed from the phase of each z
, ¯ z
and r
k
.
Since z
= r
e
−iα
and ¯ z
= r
e
iα
(e
−iω
−r
e
−iα
)(e
−iω
−r
e
iα
) = e
−iω
(e
−iω
−2r
cos α
+r
2
e
iω
) (11.32)
their phase contribution tanω = ·z/'z is
arctan
(r
2
−1) sin ω
(r
2
+ 1) cos ω −2r
cos α
(11.33)
Similarly, since
(e
−iω
−r
k
) = e
−iω/2
(e
−iω/2
−r
k
e
iω/2
) (11.34)
the phase contribution of r
k
is
arctan
(r
k
+ 1)
(r
k
−1)
tan
ω
2
(11.35)
To ﬁnd the polynomial m
0
of degree n with the smallest phase nonlinearity it is necessary
to compare all possible combination of roots. In practice, there are 2
n
2
−1
choices. For n = 2
or 3, there is eﬀectively only one set of basis φ
D,n
, ψ
D,n
. For n ≥ 4, we have to evaluate the
total phase nonlinearity of m
0
for all 2
n
2
−1
solutions.
11.3. COMMENTS AND REFERENCES 143
Figure 11.6 shows the graph of the least assymmetric φ and ψ for n = 4.
1
4 6 2
1
1
2
4 2
φ ψ
Figure 11.6: Least Assymmetric Scaling Funtion and Wavelet, with n = 4
11.2.3 Coiﬂets
Up to this point we have assumed that m
0
is the unique lowest degree polynomial which
satisfy (11.2), i.e. [m
0
(y)[
2
= (1 − y)
n
B
n
(y). We can use a higher degree [m
0
(y)[
2
= (1 −
y)
n
[B
n
(y) −y
n
R(y)] to provide more freedom in the design o orthogonal wavelets. The price
to pay is a wider support of the basis functions for a given number of vanishing moments.
The minimum support width 2n−1 for ψ with n vanishing moments is achieved when R ≡ 0.
Daubechies resorted to such higher degree solutions to construct a family of wavelets
named “coiﬄets”, because R. Coiﬀman requested them motivated by the research in (Beylkin
et al. , 1991). This basis functions are more regular and symmetric than the ones presented
in the previous subsections. More precisely, the scaling functions and wavelets are designed
so that:
φ(x)dx = 1,
x
φ(x)dx = 0 (11.36)
for = 1, . . . , L −1 and
x
ψ(x)dx = 0 (11.37)
for = 0, . . . , L −1.
In other words, both ψ and φ have n vanishing moments, except that the scaling function
integrates to one. This implies that < f, φ
−J ,k
>≈ 2
J/2
f(2
−J
k), making very simple to obtain the ﬁne
scale coeﬃcients < f, φ
−J ,k
> from samples of f.
Figure 11.7 shows a plot of the coiﬄets of order 4
11.3 Comments and References
The main result for the construction of compactly supported orthogonal wavelets is due to
Ingrid Daubechies. She formulated the expression for the halfband polynomial P based on
144 BIBLIOGRAPHY
1
2
1 2 1 3
2
2
1
3 2 1 1
1
2
φ ψ
Figure 11.7: Coiﬄets of order 4
the multresolution analysis conditions in the frequency domain and also in the restriction of
ﬁnite length ﬁlters, (Daubechies, 1988). Based on this formulation, she discovered the family
of extremal phase compactly supported orthogonal wavelets, which are knwon as Daubechies
Wavelets. In her book, (Daubechies, 1992), she gives a good overview of this methodology,
and also presents examples of several other families of compactly supported wavelets.
Carl Taswell in,(Taswell, 1995; Taswell, 1997), presents algorithms for the generation of
the Daubechies orthogonal wavelets and for the computation of their regularity.
The book by Gilbert Strang and Truong Nguyen gives a comprehensive treatment of the
framework for generating wavelets from a ﬁlter bank viewpoint, (Strang & Nguyen, 1996).
Bibliography
Beylkin, G., Coifman, R., & Rokhlin, V. 1991. Fast wavelet transforms and numerical algo
rithms. Comm. in Pure and Applied Math., 44, 141–183.
Daubechies, I. 1988. Orthonormal Bases of Compactly Supported Wavelets. Comm. Pure
Applied Math., XLI(41), 909–996.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Markushevich, A. I. 1977. Theory of Functions of a Complex Variable. Chelsea Publishing
Co.
Strang, G., & Shen, J. 1996. The zeros of the Daubechies polynomials. Proc. Amer. Math.
Soc.).
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Taswell, Carl. 1995 (Aug.). Algorithms for the Generation of Daubechies Orthogonal Least
Asymmetric Wavelets and the Computation of Their Holder Regularity. Tech. rept. Sci
entiﬁc Computing and Computational Mathematics, Stanford University, Stanford, CA.
BIBLIOGRAPHY 145
Taswell, Carl. 1997. Computational Algorithms for Daubechies LeastAsymmetric, Symmetric,
and MostSymmetric Wavelets. Miller Freeman. Pages 1834–1838.
Chapter 12
Biorthogonal Wavelets
We have seen in the previous chapters that orthogonality is a very strong constraint for the
construction of wavelets. This restricts signiﬁcantly the design choices of wavelet basis. For
example, we showed in Chapter 11 that the Haar wavelet is the only orthogonal basis which
is symmetric and has compact support.
A practical solution, that allows more ﬂexibility on the choice of wavelet functions with
desirable properties, is to replace orthogonality by a biorthogonality condition.
In this chapter we will introduce biorthogonal wavelet basis, will discuss its relations with
perfect reconstruction ﬁlter banks and will present a framework for the design of biorthogonal
wavelets.
12.1 Biorthogonal Multiresolution Analysis and Filters
12.1.1 Biorthogonal Basis Functions
Biorthogonal wavelets constitute a generalization of orthogonal wavelets. Under this frame
work, instead of a single orthogonal basis, a pair of dual biorthogonal basis functions is em
ployed: One for the analysis step and the other for the synthesis step, i.e. we have reciprocal
frames as deﬁned in Chapter 2.
Recall that, in the context of orthogonal multiresolution analysis, we have deﬁned the
projection operators onto the subspaces V
j
and W
j
, respectively:
Proj
V
j
(f) =
¸
k
analysis
. .. .
'f, φ
j,k
` φ
j,k
. .. .
synthesis
, and Proj
W
j
(f) =
¸
k
analysis
. .. .
'f, ψ
j,k
` ψ
j,k
. .. .
synthesis
,
where the functions φ and ψ perform a double duty, i.e. they are used for:
• analysis: compute the coeﬃcients of the representation of f in terms of the basis φ and
ψ of the spaces V
j
and W
j
, respectively c
j
k
= 'f, φ
j,k
` and d
j
k
= 'f, ψ
j,k
`;
• synthesis: reconstruct the projection of f onto V
j
and W
j
, from the coeﬃcients of the
representation, respectively Proj
V
j
(f) =
¸
k
c
j
k
φ
j,k
and Proj
W
j
(f) =
¸
k
d
j
k
ψ
j,k
.
147
148 CHAPTER 12. BIORTHOGONAL WAVELETS
The more general framework of biorthogonal multiresolution analysis employ similar pro
jection operators
P
j
f =
¸
k
'f, φ
j,k
`
¯
φ
j,k
, and Q
j
f =
¸
k
'f, ψ
j,k
`
¯
ψ
j,k
,
where the pair of functions φ,
˜
φ and ψ,
˜
ψ are used to share the workload: one function of the
pair act as the analyzing function, while the other act as the reconstruction function.
The functions φ and ψ are called, respectively, primal scaling function and wavelet. The
functions
¯
φ and
¯
ψ are called, respectively, dual scaling function and wavelet. The fact the
the roles of these functions can be interchanged is called duality principle. Although other
conventions are possible, here we will assume that the primal functions are used for analysis,
while the dual functions are used for synthesis.
In terms of a multiresolution analysis, this scheme leads to a family of biorthogonal scaling
functions and wavelets that are dual basis of the approximating and detail spaces.
More precisely, we deﬁne a pair of scaling functions φ
j
and
¯
φ
j
that are, respectively, Riesz
basis of the subspaces V
j
and
¯
V
j
. Similarly we deﬁne a pair of wavelet functions ψ
j
and
¯
ψ
j
that are, respectively, Riesz basis of the subspaces W
j
and
W
j
.
These functions generate dual multiresolution analysis ladders
⊂ V
1
⊂ V
0
⊂ V
−1
⊂
⊂
¯
V
1
⊂
¯
V
0
⊂
¯
V
−1
⊂
where V
0
= Span¦φ
0,k
[k ∈ Z¦ and
¯
V
0
= Span¦
¯
φ
0,k
[k ∈ Z¦. The spaces W
j
and
W
j
generated
by ψ
j,k
and
¯
ψ
j,k
are respectively the complements of V
j
in V
j−1
and of
¯
V
j
in
¯
V
j−1
. In other
words, V
j−1
= V
j
+ W
j
and
¯
V
j−1
=
¯
V
j
+
W
j
. The intersection of these spaces is null, i.e.
V
j
∩W
j
= ¦∅¦ and
¯
V
j
∩
W
j
= ¦∅¦, but the spaces V
j
, W
j
, and also
¯
V
j
,
W
j
, are not orthogonal,
in general.
In order to compensate for the lack of orthogonality within the approximating and detail
spaces, we impose instead a biorthogonality relation between the primal and dual multireso
lution ladders, such that
V
j
⊥
W
j
and
¯
V
j
⊥ W
j
(12.1)
and, consequently,
W
j
⊥
W
l
(12.2)
for j = l.
The two multiresolution hierarchies and their sequences of complement spaces ﬁt together
according to an intertwining pattern.
The above biorthogonality condition implies that the basis of these spaces must relate as
'
¯
φ(x), ψ(x −k)` =
¯
φ(x)ψ(x −k)dx = 0 (12.3)
'
¯
ψ(x), φ(x −k)` =
¯
ψ(x)φ(x −k)dx = 0 (12.4)
12.1. BIORTHOGONAL MULTIRESOLUTION ANALYSIS AND FILTERS 149
and
'
¯
φ(x), φ(x −k)` =
¯
φ(x)φ(x −k)dx = δ
k
(12.5)
'
¯
ψ(x), ψ(x −k)` =
¯
ψ(x)ψ(x −k)dx = δ
k
(12.6)
Which can be extended to the multiresolution analysis by a scaling argument resulting in
'
¯
φ
j,k
, φ
j,m
` = δ
k,m
, j, k, m ∈ Z (12.7)
'
¯
ψ
j,k
, ψ
l,m
` = δ
j,l
δ
k,m
, j, k, l, m ∈ Z (12.8)
12.1.2 Biorthogonality and Filters
The two pairs of scaling function and wavelet, φ, ψ, and
¯
φ,
¯
ψ, are deﬁned recursively by the
two pairs of ﬁlters m
0
, m
1
, and m
0
, m
1
.
In the frequency domain these relations are
´
φ(ω) = m
0
(ω/2)
´
φ(ω/2),
´
ψ(ω) = m
1
(ω/2)
´
φ(ω/2)
´
¯
φ(ω) = ´ m
0
(ω/2)
´
¯
φ(ω/2),
´
¯
ψ(ω) = ´ m
1
(ω/2)
´
¯
φ(ω/2)
where
m
0
(ω) =
1
√
2
¸
k
h
k
e
−ikω
, m
1
(ω) =
1
√
2
¸
k
g
k
e
−ikω
m
0
(ω) =
1
√
2
¸
k
¯
h
k
e
−ikω
, m
1
(ω) =
1
√
2
¸
k
¯ g
k
e
−ikω
By computing the Fourier Transform of the inner products in equations (12.3) to (12.6),
and using the same argument of Chapter 10 for the characterization of m
0
and m
1
, we can
see that the biorthogonality condition in the frequency domain is equivalent to
¸
k
´
¯
φ(ω +k2π)
´
φ(ω +k2π) = 1
¸
k
´
¯
ψ(ω +k2π)
´
ψ(ω +k2π) = 1
¸
k
´
¯
ψ(ω +k2π)
´
φ(ω +k2π) = 0
¸
k
´
¯
φ(ω +k2π)
´
ψ(ω +k2π) = 0
for all ω ∈ R.
150 CHAPTER 12. BIORTHOGONAL WAVELETS
This means that the ﬁlters m
0
, m
1
and their duals m
0
, m
1
have to satisfy
m
0
(ω)m
0
(ω) + m
0
(ω +π)m
0
(ω +π) = 1 (12.9)
m
1
(ω)m
1
(ω) + m
1
(ω +π)m
1
(ω +π) = 1 (12.10)
m
1
(ω)m
0
(ω) + m
1
(ω +π)m
0
(ω +π) = 0 (12.11)
m
0
(ω)m
1
(ω) + m
0
(ω +π)m
1
(ω +π) = 0 (12.12)
The set of equations above can be written in matrix form as
m
0
(ω) m
0
(ω +π)
m
1
(ω) m
1
(ω +π)
m
0
(ω) m
1
(ω)
m
0
(ω +π) m
1
(ω +π)
=
1 0
0 1
or
M(ω)M
T
(ω) = I (12.13)
where M is the modulation matrix introduced in Chapter 10
M(ω) =
¸
m
0
(ω) m
0
(ω +π)
m
1
(ω) m
1
(ω +π)
(12.14)
and I is the identity matrix.
12.1.3 Fast Biorthogonal Wavelet Transform
Because φ and ψ deﬁne a multiresolution analysis, we have that
φ(x) =
¸
k
h
k
φ(2x −k) and ψ(x) =
¸
k
g
k
φ(2x −k) (12.15)
Similarly,
¯
φ and
¯
ψ, also deﬁne a multiresolution analysis, and therefore
¯
φ(x) =
¸
k
¯
h
k
¯
φ(2x −k) and
¯
ψ(x) =
¸
k
¯ g
k
¯
φ(2x −k) (12.16)
We can derive the coeﬃcients of the ﬁlters ¯ m
0
and ¯ m
1
by combining the above two
equations with the biorthogonality relations '
¯
φ
j,k
, φ
j,m
` = δ
k,m
and '
¯
ψ
j,k
, ψ
l,m
` = δ
j,l
δ
k,m
.
This gives us
¯
h
k−2l
= '
¯
φ(x −l), φ(2x −k)` (12.17)
¯ g
k−2l
= '
¯
ψ(x −l), φ(2x −k)` (12.18)
and also
h
k−2l
= 'φ(x −l),
¯
φ(2x −k)` (12.19)
g
k−2l
= 'ψ(x −l),
¯
φ(2x −k)` (12.20)
12.2. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 151
By writing φ(2x − k) ∈ V
−1
in terms of the bases of V
0
and W
0
, we get the twoscale
relation
φ(2x −k) =
¸
l
¯
h
k−2l
φ(x −l) +
¸
l
¯ g
k−2l
ψ(x −l) (12.21)
and, since primary and dual functions are interchangeable, we also have
¯
φ(2x −k) =
¸
l
h
k−2l
¯
φ(x −l) +
¸
l
g
k−2l
¯
ψ(x −l) (12.22)
The fast biorthogonal wavelet transform uses the above decomposition/reconstruction
relation. The algorithm employs the two pairs of primary and dual ﬁlters and except for this
diﬀerence, it is essentially similar to the orthogonal case, presented in Chapter 7.
The pair of ﬁlters ¯ m
0
, ¯ m
1
is employed in the decomposition step and the pair of ﬁlters
m
0
, m
1
in the reconstruction step.
In the decomposition step we employ a discrete convolution with the ﬁlter coeﬃcients (h
k
)
and (g
k
) of the ﬁlters m
0
and m
1
.
c
j+1
n
=
¸
k
h
k−2n
c
j
k
(12.23)
and
d
j+1
n
=
¸
k
g
k−2n
c
j
k
(12.24)
Conversely, in the reconstruction step, we employ a discrete convolution with the ﬁlter
coeﬃcients, (
¯
h
k
) and (¯ g
k
), of the ﬁlters m
0
and m
1
c
j−1
l
=
¸
n
¯
h
l−2n
c
j
n
+
¸
n
¯ g
l−2n
c
j
n
(12.25)
Remember that, as we already noted, the roles of these two ﬁlter banks can be inter
changed.
12.2 Filter Design Framework for Biorthogonal Wavelets
12.2.1 Perfect Reconstruction Filter Banks
The ﬁlters m
0
, m
0
, m
1
, m
1
, deﬁne a twochannel ﬁlter bank, where m
0
, m
1
, are respectively
the lowpass and highpass ﬁlters used in the analysis step and, m
0
, m
1
, are respectively the
lowpass and highpass ﬁlters used in the synthesis step, as shown in Figure 12.1. Once again,
we note that the role of these ﬁlters can be interchanged,
We would like this twochannel ﬁlter bank to have the property of perfect reconstruction.
This means that a signal can be exactly reconstructed by the synthesis ﬁlters from the co
eﬃcients of the representation constructed by the analysis ﬁlters. These conditions can be
derived by following a a discrete signal, (x
n
), through the ﬁlter bank.
152 CHAPTER 12. BIORTHOGONAL WAVELETS
m
0
↓ 2
c
j
↑ 2
m
0
c
j−1
<
ˆ c
j−1
m
1
↓ 2
d
j
↑ 2
m
1
Figure 12.1: Biorthogonal Filter Bank
The output of the lowpass channel can be written in a compact notation using the z
notation (see Appendix B).
We will do this in three stages: First, we perform a discrete convolution of analysis lowpass
ﬁlter with the signal, whose ztransform is m
0
(z)x(z). This is followed by downsampling and
upsampling, which in the zdomain is 1/2 [m
0
(z)x(z) + m
0
(−z)x(−z)]. Finally, we perform
a discrete convolution of the resulting coeﬃcient sequence with the synthesis ﬁlter, whose
ztransform is
1
2
m
0
(z)[m
0
(z)x(z) +m
0
(−z)x(−z)] (12.26)
The output of the highpass channel is obtained in a similar manner
1
2
m
1
(z)[m
1
(z)x(z) +m
1
(−z)x(−z)] (12.27)
The ﬁlter bank combines the outputs of the lowpass and highpass channel by adding these
two expressions, which gives
´ x(z) =
1
2
[ m
0
(z)m
0
(z) + m
1
(z)m
1
(z)] x(z) +
1
2
[ m
0
(z)m
0
(−z) + m
1
(z)m
1
(−z)] x(−z)
where we rearranged the expression to separate the term involving x(z) from the terms in
volving x(−z).
Note that the downsampling/upsampling operators introduced aliasing in the signal, man
ifested by the appearance of terms x(−z), as well as x(z). By substituting, z = e
−iω
and
−z = e
−iω+π
, it is apparent that the frequencies w + π appear as aliases of frequencies w in
the signal (see Appendix A for details).
For perfect reconstruction we must have ´ x(z) = x(z), which implies in two conditions:
12.2. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 153
• alias cancellation: the aliasing components caused by subsampling in the lowpass
channel and in the highpass channel cancel each other.
m
0
(z)m
0
(−z) + m
1
(z)m
1
(−z) = 0
or, making z = e
−iω
m
0
(ω)m
0
(ω +π) + m
1
(ω)m
1
(ω +π) = 0 (12.28)
• no distortion: the signal is reconstructed without loss or gain of energy.
m
0
(z)m
0
(z) + m
1
(z)m
1
(z) = 2
or
m
0
(ω)m
0
(ω) + m
1
(ω)m
1
(ω) = 2 (12.29)
12.2.2 Conjugate Quadrature Filters
We have four ﬁlters to design, m
0
, m
1
, m
0
, m
1
. These ﬁlters must satisfy the conditions for
perfect reconstruction. An interesting option is to investigate how we can determine some of
the ﬁlters from the others, such that conditions (12.28) and (12.29) are automatically satisﬁed.
First, we use a conjugate quadrature scheme as done in Chapter 10, to deﬁne the highpass
ﬁlters in terms of the lowpass ﬁlters:
m
1
(ω) = e
−iω
m
0
(ω +π) (12.30)
m
1
(ω) = e
−iω
m
0
(ω +π) (12.31)
This option takes care of the alias cancelation, and guarantees that equation (12.28) is satis
ﬁed, as it can seen by substituting (12.30), (12.31), into (12.28).
Next, to express the nodistortion condition in terms of only m
0
and m
0
, we rewrite
equation (12.29) using (12.30), (12.31).
m
0
(ω)m
0
(ω) + m
0
(ω +π)m
0
(ω +π) = 2 (12.32)
Note that, when we normalize the ﬁlters (multiplying their coeﬃcients by 1/
√
2), equation
(12.32) is exactly (12.9), the ﬁrst condition for biorthogonality of the ﬁlters m
0
, m
0
, m
1
,
m
1
. The other conditions (12.10), (12.12) and (12.11), can be derived from the perfect
reconstruction conditions (12.28) and (12.29), using the conjugate quadrature relations (12.30)
and (12.31). This demonstrates that biorthogonal wavelets are associated with a subband
ﬁltering scheme with exact reconstruction.
154 CHAPTER 12. BIORTHOGONAL WAVELETS
12.2.3 The Polynomial P and Wavelet Design
Now, we deﬁne the product ﬁlter P(ω) = m
0
(ω)m
0
(ω), and equation (12.32) becomes
P(ω) +P(ω +π) = 2 (12.33)
To design the ﬁlters m
0
, m
0
, m
1
, m
1
that generate biorthogonal wavelets, we use the
following procedure:
1. Choose a polynomial P satisfying equation (12.33)
2. Factor P into m
0
m
0
3. Use (12.30) and (12.31) to obtain m
1
and m
1
.
Observe that this procedure is very similar to the one described in Chapter 10. In fact,
we are using the same ﬁlter design framework to create wavelet basis. The main diﬀerence is
that now, the analysis ﬁlters can be diﬀerent from the synthesis ﬁlters, because we have two
biorthogonal multiresolution hierarchies. This gives us more freedom to design wavelets with
desired properties.
As we have seen in Chapter 10, in order to satisfy the perfect reconstruction condition,
the polynomial P(ω) =
¸
N
k=−N
= a
k
e
ikω
, must be a halfband ﬁlter, i.e. the terms with index
k even must have coeﬃcient a
k
= 0, except for a
0
= 1, such that the terms with index k odd,
which have coeﬃcients a
k
= 0, will cancel in equation (12.33), because e
i2k(ω+π)
= −e
i2kω
.
The only remaining term will be a
0
= 1, resulting in P(ω) +P(ω+π) = 2 (or when the ﬁlters
are normalized a
0
= 1/2 and P(ω) +P(ω +π) = 1).
Also, because both φ and
¯
φ deﬁne multiresolution analyses, the polynomials m
0
(ω) and
m
0
(ω) must have, at least one zero at π (see Chapter 11 for details). Therefore, P(ω) =
m
0
(ω)m
0
(ω) has to be of the form
P(ω) =
1 +e
iω
n
2
B(e
iω
) (12.34)
The advantage of biorthogonal wavelets over orthogonal wavelets is that we can factorize P(ω)
in two diﬀerent polynomials m
0
, m
0
instead as the “square” of a single polynomial m
0
.
12.2.4 Factorization of P for Biorthogonal Filters
Let us see one example of the diﬀerent design choices for the factorization of P(ω) into m
0
(ω)
and m
0
(ω), from (Strang & Nguyen, 1996).
We will design the polynomial P such that it will have four zeros at π
P(ω) = [(1 +e
−iω
)
2
[
2
B(e
iω
)
= (1 +e
−iω
)
2
(1 +e
iω
)
2
B(e
−iω
)
Making z = e
−iω
P(z) = (1 +z
−1
)
2
(1 +z)
2
B(z)
= (z
−2
+ 4z
−1
+ 6 + 4z +z
2
)B(z)
= z
2
(1 +z
−1
)
4
B(z)
12.2. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 155
We have to choose B so that even powers are eliminated from P, making it a halfband
polynomial. The solution is B(z) = −z
−1
+ 4 −z
P(z) = (z
−2
+ 4z
−1
+ 6 + 4z +z
2
)(−z
−1
+ 4 −z)
= −z
−3
+ 9z
−1
+ 16 + 9z −z
3
We normalize it to make a
0
= 1
P(z) =
1
16
(−z
−3
+ 9z
−1
+ 16 + 9z −z
3
) (12.35)
Note that P satisfy the perfect reconstruction conditions and also have four zeros at −1.
We have several choices to factorize P(z) into m
0
(z)m
0
(z). The polynomial P has six
roots ¦−1, −1, −1, −1, 2 −
√
3, 2 +
√
3¦. The four roots at z = −1 come from (1 +z
−1
)
4
, and
the other two roots at c = 2 −
√
3 and 1/c = 2 +
√
3 come from B(z).
Each of the polynomials m
0
(z) and m
0
(z) will have, at least, one root at z = −1. There
fore, we can select the roots for the ﬁlter m
0
(or m
0
) in one of the following ways, — the
other ﬁlter m
0
(or m
0
) will have the remaining roots:
i. m
0
of degree 1: (1 +z
−1
)
ii. m
0
of degree 2: (1 +z
−1
)
2
, or (1 +z
−1
) (c −z
−1
), or (1 +z
−1
) (1/c −z
−1
)
iii. m
0
of degree 3: (1 +z
−1
)
3
, or (1 +z
−1
)
2
(c −z
−1
)
We have six diﬀerent choices for the factorization of P. For each choice the roots are
separated in two sets, and we can select which set of roots go to the analysis ﬁlter m
0
and to
the synthesis ﬁlter m
0
.
In option (i), one factor is (1 + z
−1
), which is the ﬁlter for a box function. The other
factor is a degree 5 polynomial
1
16
(−1 +z
−1
+8z
−2
+8z
−3
+z
−4
−z
−5
). The ﬁlter bank will
have ﬁlters with 2 and 6 coeﬃcients for analysis and synthesis respectively.
In option (ii), the ﬁrst possibility is one factor equal to
1
2
(1 + 2z
−1
+ z
−2
), which is the
ﬁlter for the hat function, and the other factor equal to
1
8
(−1 + 2z
−1
+ 6z
−2
+ 2z
−3
− z
−4
).
This choice corresponds to the order 2 Bspline biorthogonal wavelet, that will be discussed
in the next section. It is best to use the hat scaling function for synthesis, resulting in a 5/3
ﬁlter bank. The other possibilities are not so interesting.
In option (iii), we have two diﬀerent possibilities, both very important. The ﬁrst possibility
is a factorization as
1
8
(1+3z
−1
+3z
−2
+z
−3
), and
1
2
(−1+3z
−1
+3z
−2
−z
−3
). Notice that this
choice results in a pair of linear phase ﬁlters which correspond to symmetric scaling functions
and antisymmetric wavelets. The second possibility is a factorization with m
0
(z) = m
0
(z) =
1
4
√
2
[(1+
√
3) +(3+
√
3)z
−1
+(3−
√
3)z
−2
+(1−
√
3)z
−3
]. This choice gives us the Daubechies
orthogonal wavelet of order 2, that we presented in the previous chapter. Notice that the
orthogonal ﬁlters have extremal phase, which correspond to the most assymmetric scaling
functions and wavelets.
156 CHAPTER 12. BIORTHOGONAL WAVELETS
Figure 12.2 shows the graph of the biorthogonal scaling functions and wavelets derived in
case (i), and Figure 12.3 plots the frequency response of the corresponding ﬁlters.
functions
0 1 2 3 4 5
−1
−0.5
0
0.5
1
0 1 2 3 4 5
−1
−0.5
0
0.5
1
0 1 2 3 4 5
−1
−0.5
0
0.5
1
0 1 2 3 4 5
−1
−0.5
0
0.5
1
(a) (b)
Figure 12.2: Biorthogonal box scaling functions (a) and wavelets (b).
filters
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
(a) (b)
Figure 12.3: Frequency response of lowpass (a) and highpass (b) ﬁlters.
12.2. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 157
Figure 12.4 shows the graph of the biorthogonal scaling functions and wavelets derived in
case (ii), and Figure 12.5 plots the frequency response of the corresponding ﬁlters.
functions
0 1 2 3 4 5
−1.5
−1
−0.5
0
0.5
1
0 1 2 3 4 5
−1.5
−1
−0.5
0
0.5
1
0 1 2 3 4 5
−6
−4
−2
0
2
4
0 1 2 3 4 5
−6
−4
−2
0
2
4
(a) (b)
Figure 12.4: Biorthogonal Hat scaling functions (a) and wavelets (b).
filters
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
1.2
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
1.2
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
(a) (b)
Figure 12.5: Frequency response of lowpass (a) and highpass (b) ﬁlters.
158 CHAPTER 12. BIORTHOGONAL WAVELETS
Figure 12.6 shows the graph of the biorthogonal scaling functions and wavelets derived
in option (a) of case (iii), and Figure 12.7 plots the frequency response of the corresponding
ﬁlters.
functions
0 0.5 1 1.5 2 2.5 3
−1
−0.5
0
0.5
1
0 0.5 1 1.5 2 2.5 3
−1
−0.5
0
0.5
1
0 0.5 1 1.5 2 2.5 3
−200
−100
0
100
200
0 0.5 1 1.5 2 2.5 3
−200
−100
0
100
200
(a) (b)
Figure 12.6: Biorthogonal quadratic Bspline scaling functions (a) and wavelets (b).
filters
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3
0
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
(a) (b)
Figure 12.7: Frequency response of lowpass (a) and highpass (b) ﬁlters.
12.2. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 159
Figure 12.8 shows the graph of the orthogonal D2 Daubechies scaling functions and
wavelets derived in option (b) of case (iii), and Figure 12.9 plots the frequency response
of the corresponding ﬁlters.
functions
0 0.5 1 1.5 2 2.5 3
−1.5
−1
−0.5
0
0.5
1
0 0.5 1 1.5 2 2.5 3
−1.5
−1
−0.5
0
0.5
1
0 0.5 1 1.5 2 2.5 3
−1.5
−1
−0.5
0
0.5
1
0 0.5 1 1.5 2 2.5 3
−1.5
−1
−0.5
0
0.5
1
(a) (b)
Figure 12.8: D2 Daubechies scaling functions (a) and wavelets (b).
filters
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3
0
0.2
0.4
0.6
0.8
1
(a) (b)
Figure 12.9: Frequency response of lowpass (a) and highpass (b) ﬁlters.
160 CHAPTER 12. BIORTHOGONAL WAVELETS
12.3 Symmetric Biorthogonal Wavelets
One of the advantages of biorthogonal over orthogonal wavelets is that we can have symmetry.
In this section we present some families of symmetric biorthogonal wavelets, developed by
Ingrid Daubechies (Daubechies, 1992), using diﬀerent factorizations of the polynomial P.
For symmetric ﬁlters we have to factorize P such that the roots z
i
and z
−1
i
must stay
together when we split the roots between m
0
and m
0
. Therefore, we have
z
N
P(z) = a
N
N
¸
i=1
(z −z
i
)(z −
1
z
i
) (12.36)
where each pair of roots, ¦z
i
, 1/z
i
¦, must be assigned either to m
0
or to m
0
.
12.3.1 BSpline Wavelets
BSplines of order N are piecewise polynomials that come from the convolution of N box
functions. In terms of the twoscale relations, they can be constructed from a ﬁlter function
m
0
(z) of the form
m
0
(z) =
1 +z
−1
2
N
(12.37)
Each factor, (
1+z
−1
2
), corresponds to a convolution with a box function.
To construct biorthogonal Bspline wavelets we can split the polynomial P introduced in
the previous chapter as:
m
0
(z) =
1 +z
−1
2
M
(12.38)
where
M = 2
¯
is even or
M = 2
¯
+ 1 is odd, and
m
0
(z) =
1 +z
−1
2
M +
−σ
¸
m=0
+
¯
−σ +m
m
1 −z
2
m
1 −z
−1
2
m
(12.39)
where M = 2 is even, or M = 2 + 1 is odd, and σ = 0 if M is odd, or σ = 1 if M is even.
Note that this formulas give explicit expressions for m
0
and m
0
. We have, thus a family of
biorthogonal Bspline wavelets, in which the synthesis scaling function
¯
φ is a Bspline of order
M. For a ﬁxed
M, there is an inﬁnity of choices for M, resulting in biorthogonal functions
with increasing regularity and support width. Also, we remark that m
0
depends only on
M,
while m
0
depends on both M and
M.
In Figure 12.10 we show one example of a biorthogonal Bspline wavelets and scaling
functions of order 2 (linear spline),
M = 2, with M = 2 and 4.
This family of wavelets has two main advantages: First, there is a closed form expression
for the Bspline scaling functions
¯
φ and wavelets
¯
ψ; Second, the coeﬃcients for all ﬁlters m
0
,
12.3. SYMMETRIC BIORTHOGONAL WAVELETS 161
Figure 12.10: Biorthogonal Bspline Wavelets of order 2 (from (Daubechies, 1992)).
m
1
, m
0
, m
1
, are dyadic rationals, which makes them suitable for fast and exact computer
implementations.
The major disadvantage of biorthogonal Bspline wavelets is that the analysis and synthesis
162 CHAPTER 12. BIORTHOGONAL WAVELETS
functions have very diﬀerent support widths, as can be seen in Figure 12.10. This may or
may not be a problem in some applications.
12.3.2 Wavelets with Closer Support Width
It is possible to construct wavelets with closer support width by choosing an appropriate
factorization of the polynomial P. The goal is to ﬁnd ﬁlter functions m
0
and m
0
, that have
both linear phase and similar degree.
We determine all roots of P, real zeros r
i
and pairs of complex conjugate zeros ¦z
j
, z
j
¦
P(z) = c
¸
i=1
(z −r
i
)
¸
j=1
(z −z
j
)(z −z
j
) (12.40)
For a ﬁxed degree N = +
¯
, we have a limited number of factorizations. Therefore, we
can generate all the diﬀerent combinations of roots, such that r
i
, ¦z
j
, z
j
¦ are assigned either
to m
0
or to m
0
, and select the option +
¯
, that makes the lengths of m
0
, and m
0
as close as
possible.
Figure 12.11 shows an example of symmetric biorthogonal wavelets designed using this
method.
Figure 12.11: Biorthogonal Wavelets with Similar Support,(from (Daubechies, 1992))
12.4. COMMENTS AND REFERENCES 163
12.3.3 Biorthogonal Bases Closer to Orthogonal Bases
Motivated by a suggestion of M. Barlaud, Ingrid Daubechies developed a family of biorthogo
nal wavelet bases that are close to an orthogonal basis. Barlaud tried to construct biorthogonal
wavelets using the Laplacian pyramid ﬁlter, designed by P. Burt (Burt & Adelson, 1983), as
either m
0
or m
0
. These experiments lead to the discovery that the Burt ﬁlter is very close to
an orthonormal wavelet ﬁlter.
The particular construction for the Burt ﬁlter, was then generalized by Daubechies to a
family of biorthogonal wavelets that are close to orthogonal. She used the extended formula
for the polynomial P, including the factor R(z
N
), as discussed in the previous chapter.
The ﬁlter functions for this family are deﬁned as
m
0
(z) = S(z) +aR(z), and m
0
(z) = S(z) +bR(z) (12.41)
where the S, R are given below, and the constants a, b are computed by an optimization
procedure to guarantee the biorthogonality of the ﬁlters.
S(z) =
1 +z
2
2K
1 +z
−1
2
2K K−1
¸
k=0
K −1 +k
k
1 −z
2
2k
1 −z
−1
2
2k
(12.42)
R(z) =
1 −z
−2
4
2K
1 −z
2
4
2K
(12.43)
Note that, the ﬁlters m
0
and m
0
, diﬀer only by the constants a and b, therefore they are
very close to an orthogonal ﬁlter.
Figure 12.12 shows a plot of a wavelet constructed using this procedure.
12.4 Comments and References
The ﬁrst examples of biorthogonal wavelets were developed independently by (Cohen et al. ,
1992) and (Vetterli & Herley, 1992).
It is possible to construct biorthogonal wavelets such that the primal and dual scaling and
wavelet bases generate a single orthogonal multiresolution analysis. In this case, the scaling
function and wavelets are called semiorthogonal. The name “prewavelets” is also employed
to designate this type of wavelets. An extensive treatment of this subject can be found in
(Chui, 1992).
Bibliography
Burt, P. J., & Adelson, E. H. 1983. The Laplacian Pyramid as a Compact Image Code. IEEE
Trans. Commun., 31(4), 532–540.
Chui, C. K. 1992. An introduction to wavelets. Academic Press.
164 BIBLIOGRAPHY
Figure 12.12: Biorthogonal Wavelets Close to Orthogonal,(from (Daubechies, 1992))
Cohen, A., Daubechies, I., & Feauveau, J. C. 1992. Biorthogonal Bases of Compactly Sup
ported Wavelets. Comm. Pure Applied Math.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Vetterli, M., & Herley, C. 1992. Wavelets and ﬁlter banks: theory and design. IEEE Trans.
Acoust. Speech Signal Process., 40(9).
Chapter 13
Directions and Guidelines
In this concluding chapter we will review the material covered in the notes, and give some
directions for further studies on the subject of wavelets.
13.1 History and Motivation
The ﬁeld of wavelets, despite being relatively recent, is vast and is developing very rapidly.
This is true both in relation to theoretical aspects of wavelets, as well as to applications of
wavelets.
Wavelets are a product of ground work from many areas, ranging from pure mathematics
and physics to engineering and signal processing. Independent research in these areas pur
sued similar goals using diﬀerent approaches. The objective was to develop tools to describe
functions in time and frequency simultaneously. The separate lines of investigation reached
a mature point, and in the beginning of the 1980’s, the conﬂuence of this interdisciplinary
sources, was formalized originating the theory of wavelets. The subsequent uniﬁcation of the
ﬁeld was a key factor to make wavelets popular in applied mathematics, and also to give a
signiﬁcant impulse to new research.
Today, wavelets are well established. The basic theory is completely developed with suc
cessful applications in a large number of areas. Nonetheless, research is perhaps even more
active than before, with new results appearing from a growing scientiﬁc community. Also, the
application base is consolidating and expanding to new areas, with new experimental systems
and commercial products being released.
Considering the facts mentioned above, it would not be possible to cover the whole ﬁeld
of wavelets in a single text. Indeed, there are currently more than 20 textbooks, entirely
dedicated to the various aspects of wavelets. Not to mention, the enormous number of con
ference proceedings and special issues of journals devoted to wavelets. Most of these texts are
for experts, or for people with basic knowledge of wavelets. Even some of the introductory
books cover a lot of material with a fast pace, assuming maturity and dedication from the
reader. For this reason, we believe that there is still a need for a conceptual textbook about
the fundamentals of wavelets.
165
166 CHAPTER 13. DIRECTIONS AND GUIDELINES
13.2 A Look Back
In these course notes, we tried to give a comprehensive, but accessible, introduction to the ﬁeld
of wavelets. Our goal was to follow a route from Fourier Analysis to wavelets, showing how
the traditional tools for function representation in frequency domain evolved to the new tools
for joint timefrequency descriptions. This approach combined the concepts from function
analysis with the intuition from signal processing.
It is instrumental to take a look back and review the main topics that we covered in these
notes. They include:
• motivation and schemes for representation of functions;
• the Fourier transform as a tool for frequency analysis;
• the Windowed Fourier transform and the search for timefrequency localization;
• the continuous Wavelet transform as an adapted timefrequency function decomposition;
• the multiresolution representation as a tool to discretize and reconstruct functions;
• discrete wavelet bases and a ﬁlter bank methodology for design and computation;
• and ﬁnally we described the main families of wavelet bases.
As we stated earlier, these items constitute only the fundamental concepts behind wavelets.
In order to go from this basic level to more advanced topics and to practical applications, the
reader can continue the learning process in various directions. Below, we will discuss some of
the important aspects of wavelets not covered in these notes, and will indicate some of the
possible options for further studies.
In these notes, we restricted ourselves to functions deﬁned on the whole real line, which
correspond to signals with inﬁnite duration. This is the simplest case, and certainly makes
the theory more accessible.
We also discussed only the basic schemes for timefrequency decomposition of a function.
In particular, we have emphasized descriptions using nonredundant wavelet bases. This is
perhaps the most important representation in practical applications.
Lastly, we didn’t consider any concrete application of wavelet, besides a few simple example
scattered throughout the notes.
13.3 Extending the Basic Wavelet Framework
The basic wavelet framework presented in these notes can be further developed in three main
directions:
1. extending wavelets to general domains;
2. deﬁning generalized representations using timefrequency decompositions, and
3. applying the wavelet framework to solve problems in applied mathematics.
We now brieﬂy discuss what has been done in these three areas.
13.3. EXTENDING THE BASIC WAVELET FRAMEWORK 167
13.3.1 Studying Functions on other Domains
The ﬁrst generalization is to study wavelets other domains diﬀerent than the real line.
It is natural to deﬁne wavelets in ndimensional Euclidean spaces. The path to follow
consists in extending the 1D results that we have obtained to dimension n > 1. That is, to
study wavelet transforms and multiresolution analyses on the space L
2
(R
n
). We have several
options in this direction. Some concepts and results extend naturally. There are two ways
to construct wavelets in R
n
. The ﬁrst way is based on a tensor product of onedimensional
wavelets. This extension is straightforward, and leads to a separable wavelet transform which
can be implemented eﬃciently by multipass 1D methods. (Daubechies, 1992). The second
way is through a multidimensional multiresolution hierarchy generated by a single multivariate
scaling function in R
n
(St¨ ockler, 1992).
Another important extension is to deﬁne wavelets on compact subsets of the Euclidean
space R
n
. A particular case concerns wavelets that are adapted to the unit interval I = [0, 1].
For this purpose, we have to construct a multiresolution of L
2
(I), which is not generated
from a single function anymore, but includes functions that adapt to the boundaries of the
interval (Cohen et al. , 1993) This construction extends naturally to rectangular regions of
higher dimensions.
The ﬁnal relevant extension should point us in the direction of deﬁning wavelets and
multiresolution analysis on arbitrary manifolds. Note that the methods we have described
in these notes make strong use of the Euclidean space structure, because of the translations
used to obtain the wavelet basis. For this extension, we clearly need a methodology diﬀerent
from the Fourier Analysis. The Lifting scheme (Sweldens, 1995) and the similar construction
in (Dahmen & Schneider, 1998) are promising approaches in this direction.
13.3.2 Deﬁning other TimeFrequency Decompositions
The second extension is to formulate function representations based on other timefrequency
decomposition schemes.
Redundant wavelets bases are required for some operations. In particular, dyadic wavelets
give a translation invariant representation that can be used for characterization of singularities
and edge detection (Mallat & Hwang, 1992; Mallat & Zhong, 1992). Also, the steerable
pyramid provides a orientation tuning mechanism (Simoncelli & Freeman, 1995).
Another generalization consists of decompositions based on diﬀerent tilings of the time
frequency plane. Some examples are the wavelet packet basis (Wickerhauser, 1994), and the
local cosine basis, also known as Malvar wavelets (Malvar, 1990). Some of these representa
tions rely on a redundant dictionary of timefrequency atoms that can be used to construct
descriptions adapted to individual functions. Best basis selection (Coifman & Wickerhauser,
1992), and matching pursuit (Mallat & Zhang, 1993) are some of the optimization methods
used to generate these representations. Similar techniques have been reported in (Chen &
Donoho, 1994) and (Chen & Donoho, 1995).
A last generalization is a decomposition using a basis composed of several diﬀerent func
tions, known as multiwavelets (Malvar, 1990; Strang & Strela, 1994).
168 CHAPTER 13. DIRECTIONS AND GUIDELINES
13.3.3 Solving Mathematical Problems
The third direction is to study how wavelets can be applied in the solution of mathematical
problems. This means that we will use wavelets to represent operators and to exploit the
properties of this representation in order to derive more eﬃcient computational schemes.
Wavelets have been used to solve integral equations (Beylkin et al. , 1991), diﬀerential
equations (Bacry et al. , 1992), and optimization problems. We remark that this last scheme
has strong connections with multigrid methods.
13.4 Applications of Wavelets
Wavelets have been applied in may diﬀerent areas. Here we will give an overview only of the
applications in Computer Graphics and related areas.
We list below some of the main problems associated with the representation and process
ing of: geometric models, images, animation video sound and multimedia, that have been
successfully solved using wavelets.
• Signal and Image Processing
– Data compression
– Progressive Transmission
– Noise reduction
– Filtering
• Vision
– Edge detection
– Texture Analysis
– Feature Classiﬁcation
• Visualization and Graphics
– Radiosity
– Volume Rendering
– Paint Systems
• Geometric Modeling
– Wavelets on surfaces and Variable resolution meshes
– Mulitscale Editing
– Optimization of geometry
– Synthesis of fractal surfaces
13.5. COMMENTS AND REFERENCES 169
• Animation
– Timespace constraints
– Motion learning
• Other Areas
– Medical Imaging
– Geology
– GIS and Cartography
– Music
– Speech synthesis and recognition
– Databases for images and video.
A good review of wavelet application in computer graphics can be found in (Schroder,
1996) and (Stollnitz et al. , 1995a; Stollnitz et al. , 1995b). A book dedicated entirely to this
subject is (Stollnitz et al. , 1996).
Other sources of reference are the previous SIGGRAPH courses dedicated to wavelets
(Schroder & Sweldens, 1995; Fournier, 1994).
13.5 Comments and References
As a last remark, we mention that the Internet is a valuable source of information on current
research and application of wavelets. From the many excellent websites, and other online
resources, we would like to single out one: “The Wavelet Digest”, an electronic newsletter
moderated by Win Sweldens (http://www.wavelet.org/).
Bibliography
Bacry, Emmanual, Mallat, Stephane, & Papanicolaou, George. 1992. A wavelet based space–
time adaptive numerical method for partial diﬀerential equations. Mathematical Mod
elling and Numerical Analysis, 26, 793–834.
Beylkin, G., Coifman, R., & Rokhlin, V. 1991. Fast Wavelet Transforms and Numerical
Algorithms I. Communications on Pure and Applied Mathematics, XLIV, 141–183.
transformation en ondelettes.
Chen, Shaobing, & Donoho, David L. 1994. Basis Pursuit. Tech. rept. Stanford University.
Chen, Shaobing, & Donoho, David L. 1995. Atomic Decomposition by Basis Pursuit. Tech.
rept. Stanford University.
Cohen, A., Daubechies, I., & Vial, P. 1993. Wavelets on the interval and fast wavelet trans
forms. Applied and Computational Harmonic Analysis, 1(1), 54–81.
170 BIBLIOGRAPHY
Coifman, Ronald R., & Wickerhauser, Mladen Victor. 1992. EntropyBased Algorithms for
Best Basis Selection. IEEE Transactions on Information Theory, 38(2), 713–718.
Dahmen, Wolfgang, & Schneider, Reinhold. 1998. Wavelets on Manifolds I: Construction and
Domain Decomposition. Tech. rept. Institut fur Geometrie und Praktische Mathematik.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Fournier, Alain. 1994. Wavelets and their applica
tions in computer graphics. SIGGRAPH Course Notes,
http://www.cs.ubc.ca/nest/imager/contributions/bobl/wvlt/download/notes.ps.Z.saveme.
Mallat, S., & Zhang, A. 1993. Matching Pursuit with Timefrequency Dictionaries. Tech.
rept. Courant Institute of Mathematical Sciences.
Mallat, S., & Zhong, S. 1992. Characterization of signals from multiscale edges. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 14(7), 710–732.
Mallat, S. G., & Hwang, W. L. 1992. Singularity detection and processing with wavelets.
IEEE Transactions on Information Theory, 38(2), 617–643.
Malvar, H. S. 1990. Modulated QMF Filter Banks with Perfect Reconstruction. Electronics
Letters, 26(June), 906–907.
Schroder, Peter. 1996. Wavelets in Computer Graphics. Pages 615–625 of: Proceedings of the
IEEE, vol. 84.
Schroder, Peter, & Sweldens, Wim. 1995. Wavelets in computer graphics. SIGGRAPH Course
Notes, http://www.multires.caltech.edu/teaching/courses/waveletcourse/.
Simoncelli, Eero P., & Freeman, William T. 1995 (Oct.). The Steerable Pyramid: A Flexible
Architecture for MultiScale Derivative Computation. Pages 444–447 of: International
Conference on Image Processing, vol. 3.
St¨ ockler, J. 1992. Multivariate wavelets. Pages 325–356 of: Chui, C. K. (ed), Wavelets: A
Tutorial in Theory and Applications. Academic Press.
Stollnitz, Eric J., DeRose, Tony D., & Salesin, David H. 1995a. Wavelets for Computer
Graphics: a Primer, Part 1. IEEE Computer Graphics and Applications, 15(3), 76–84.
Stollnitz, Eric J., DeRose, Tony D., & Salesin, David H. 1995b. Wavelets for computer
graphics: a primer, part 2. IEEE Computer Graphics and Applications, 15(4), 75–85.
Stollnitz, Eric J., DeRose, Tony D., & Salesin, David H. 1996. Wavelets for Computer Graph
ics: Theory and Applications. San Francisco, CA: Morgann Kaufmann.
Strang, Gilbert, & Strela, Vasily. 1994. Orthogonal multiwavelets with vanishing moments.
Optical Engineering, 33(7), 2104–2107.
BIBLIOGRAPHY 171
Sweldens, W. 1995. The lifting scheme: A construction of second generation wavelets. Tech.
rept. 1995:6. Department of Mathematics, University of South Carolina.
Wickerhauser, Mladen Victor. 1994. Adapted Wavelet Analysis from Theory to Software.
Wellesley, MA: A. K. Peters.
Appendix A
Systems and Filters
In this appendix we will review the basic concepts of Signal Processing to introduce the
terminology and notation.
A.1 Systems and Filters
The word “system” has a very generic meaning in engineering. A system may change the
input signal in diﬀerent ways, producing an output signal.
A good example of a system is the human sound perception. It capture sounds, classiﬁes
and interprets them, in a way that we are not only able to understand the meaning, but we are
also capable to identify the source of the sound. This complex task is accomplished by a long
chain of processes. Among other things, the input sound is analyzed, separating its various
frequency components using a set of ﬁlters with diﬀerent sensibilities (this is performed by
the cochea, the basilar membrane, and other elements of the auditory system).
The term “ﬁlter” is employed to designate certain types of systems that alter only some
frequencies of an input signal. The name has exactly this meaning: it implies that a selection
is taking place and some frequency bands are altered, either attenuated or emphasized.
In the mathematical universe, a signal is modeled by a function. Therefore, we can model
a system by a transformation S : T
1
→ T
2
between two function spaces, as illustrated in
Figure A.1.
f −→ System −→S(f) = g
Figure A.1: Basic System.
The systems constitute the mathematical model to study the various operations involving
processing of signals in the physical universe. We can give some examples:
• The human eye is a system that process electromagnetic signals in the visible range of
the spectrum;
173
174 APPENDIX A. SYSTEMS AND FILTERS
• A television camera has as input an electromagnetic signal (as the human eye), and its
output is a video signal.
A.1.1 Spatial Invariant Linear Systems
In general, a system is characterized by some of its properties. Below, we describe the most
relevant properties for our study:
Linearity
A system S is linear if:
1. S(f +g) = S(f) +S(g)
2. S(αf) = αS(f)
where f and g are two input functions (signals) and λ ∈ R. In particular, S(0) = 0.
If a system S is linear and a signal f is represented by an atomic decomposition, as for
example:
f =
∞
¸
j=−∞
'f, ϕ
j
`ϕ
j
Its output S(f) can be written as:
S(f) =
∞
¸
j=−∞
'f, ϕ
j
`S(ϕ
j
)
That is, it is suﬃcient to known the output of the system for the basic decomposition atoms
to predict the processing of any signal in this representation.
Spatial Invariance
A system S is spatial invariant if it is possible to perform shifts in the signal, before or after
its processing by the systems, and yet obtain the same result. That is,
S(f(t −t
0
)) = S(f)(t −t
0
).
This class of system is generally called “time invariant” when the functions are of type f :
R → R, i.e. they have only one variable which usually represents the time evolution. When
we work in higher dimensions, this time analogy is not valid anymore (even though time may
still be one of the variables).
Given t
0
∈ R , the operator R : T
1
→T
2
deﬁned by
R(f) = f(t −t
0
)
is called delay operator by t
0
, or translation operator by t
0
. Geometrically it makes a transla
tion of t
0
units to the right in the graph of f, which is equivalent to a delay of t
0
time units
in the signal (see Figure A.2).
A.1. SYSTEMS AND FILTERS 175
Figure A.2: Delay Operation.
A system S is linear and spatial invariant if a delay in the input is equal to the same delay
in the output
SR
t
0
= R
t
0
S
Impulse Response
In this subsection we will characterize the spatial invariant linear systems.
The impulse response, h of a system S is the image h = L(δ) of the signal Dirac δ by the
system.
If f is an arbitrary signal, we have:
f(x) =
∞
−∞
f(t)δ(x −t)dt
If the system is spatial invariant, it follows from the above equation that:
S(f(x)) =
∞
−∞
f(t)S(δ(x −t))dt
=
∞
−∞
f(t)h(x −t)dt
The integral above is indicated by f ∗ h, and is called convolution product. It is easy to
verify that f ∗ h = h ∗ f. This result is summarized in the theorem below:
Theorem 8. A spatial invariant linear system S is completely determined by its impulse
response h = S(δ). More precisely, S(f) = h ∗ f for any signal f.
The impulse response h gives a complete characterization of the system and, for this
reason, it is called ﬁlter kernel.
A.1.2 Other Characteristics
For the moment, we will concentrate our attention only on these two properties mentioned
above: linearity and time invariance. Nonetheless, there are many other important charac
teristics that deserve to be mentioned:
176 APPENDIX A. SYSTEMS AND FILTERS
Finite Impulse Response
A system S has ﬁnite impulse response, (FIR), if its impulse response has compact support,
supp(S(δ)) ⊂ [−L, L], L < ∞.
The name originates from the fact that in the discrete domain, the impulse response of these
ﬁlters is represented by a ﬁnite sequence.
Causality
Causality is a property which says that the output of the system depends only of previous
facts in the input. That is, the system does not have any knowledge of the future.
A good example of a system in this class is the human ear, because sounds are processed
strictly in chronological order.
Stability
Many stability conditions can be deﬁned for a system, each of them related to the needs of
diﬀerent applications. A common stability condition is to enforce continuity of the operator
which deﬁnes the system:
[[S(f) ≤ c[[f[[
In other words, if the input signal has ﬁnite energy, the same occurs with the output signal.
Taking the norm of sup: [[f[[ = sup(f), this condition reduces to:
sup(S(f)) ≤ c sup(f)
A system which satisfy this condition is called in the literature BIBO system (“Bounded Input
– Bounded Output”).
A.2 Discretization of Systems
We have to work with discrete systems to be able to implement the model of continuous
systems in the computer. Therefore, we need to resort to methods to represent systems,
which ultimately are methods to represent functions. Discrete systems operate with discrete
signals.
A.2.1 Discrete Signals
As we have seen, various phenomena in the physical universe clearly have a discrete nature,
such as turning a light on or oﬀ. Other phenomenon, may even be conceived as continuous,
but they are observed (measured) at discrete instants of time, for example, the variation of
the tides.
Independently of a system being intrinsically continuous or discrete, in actual computer
implementations we need to work with discrete representations of the system.
A.2. DISCRETIZATION OF SYSTEMS 177
When we represent a signal f we obtain a discretization that associates to the signal a
sequence of “samples” (f
n
), n ∈ Z. We will use the notation f[n], f(n) or even f
n
to indicate
the nth “sample” of the signal. A sequence of samples of the signal will be indicated by (f
n
).
Sometimes, we will write (f
n
)
n∈Z
to emphasize that we are referring to a sequence.
The representation of a ﬁnite energy signal is deﬁned by an operator R: T →
2
, of the
space of signals into the space
2
of the square summable sequences: R(f) = (f
n
). The most
common form of representation of a signal is given by its projection in a closed subspace
V ⊂ T. If ¦ϕ
j
¦ is an orthonormal base of this space, then
R(f) =
¸
k
'f, ϕ
j
`ϕ
j
,
Therefore, the samples of the signal are given by f(j) = 'f, ϕ
j
`. Nonetheless, the reader could
imagine that a sampling sequence is represented by uniform sampling the signal with rate ∆t:
f(n) = f(n∆t) (we have seen that this is a good approximation for most cases of interest).
When the signal f is represented by a sequence (f
n
), the description of the data elements
which represent the physical measurements of the signal, is done through the integer variable
n. It is important to have a memory about the representation method employed, specially to
make possible the reconstruction of the signal. In the case of point sampling, for example, it
is important to know the sampling rate ∆t.
If the representation sequence is ﬁnite, he have in fact a representation vector
(f
n
) = (f(i), f(i + 1), . . . , f(j)).
Otherwise, the representation sequence is an inﬁnite vector
(f
n
) = (. . . , f(−2), f(−1), f(0), f(1), f(2), . . . )
A.2.2 Discrete Systems
Given a system S : T
1
→T
2
, we can take the representation R
1
: T
1
→
2
e R
2
: T
2
→
2
, to
produce the following diagram,
T
1
S
−−−→ T
2
R
1
R
2
2
S
−−−→
2
where S is the representation of the system S, that is, the discretized system S.
If S is linear and T
1
, T
2
have a ﬁnite dimensional representation, that is, R(T
1
) = R
m
and R(T
2
) = R
n
, then S : R
m
→ R
n
is a linear transformation, and can be represented by a
matrix of order n m:
¸
¸
¸
¸
a
11
a
1m
a
21
a
2m
.
.
.
.
.
.
.
.
.
a
n1
a
nm
¸
178 APPENDIX A. SYSTEMS AND FILTERS
In case the representation of the spaces T
1
, T
2
is not ﬁnite dimensional, we have S:
2
→
2
. It is useful to represent these operators by inﬁnite matrices
¸
¸
¸
¸
¸
¸
¸
.
.
.
y(−1)
y(0)
y(1)
.
.
.
¸
=
¸
¸
¸
¸
¸
¸
¸
.
.
.
a
−1,−1
a
−1,0
a
−1,1
a
0,−1
a
0,0
a
0,1
a
1,−1
a
1,0
a
1,1
.
.
.
¸
.
¸
¸
¸
¸
¸
¸
¸
.
.
.
x(−1)
x(0)
x(1)
.
.
.
¸
If the system S is linear and spatially invariant, we know that,
S(f) = h ∗ f
where h is the kernel of the ﬁlter, h = S(δ). In the discrete domain h = (h
n
) and f = (f
n
),
hence:
S(f
n
)(n) = ((h
n
) ∗ (f
n
))(n)
=
+∞
¸
k=−∞
h(k)f(n −k),
(A.1)
which is the direct expression for the convolution product.
Taking (f
n
) = (δ
n
), where (δ
n
) is the discrete version of the Dirac delta:
(δ
n
)(k) =
1 if k = 0;
0 if k = 0.
then
S(δ
n
) = h ∗ δ
n
= h,
As it was expected.
The reader should observe that equation (A.1) which deﬁnes the output of a discrete linear
system is a ﬁnite diﬀerence linear equation:
y(n) =
+∞
¸
k=−∞
h(k)f(n −k),
where S(f
n
) = (y
n
).
Because S is linear, we know that S is given by a matrix (possibly inﬁnite). Now, we
will obtain the matrix S using equation (A.1) below. For this, we need some linear algebra
notation.
Indicating by R the unit delay operator: that is R(f
n
) = (f
n−1
) we have that
f(n −k) = R
k
(f
n
)
A.2. DISCRETIZATION OF SYSTEMS 179
in other words, f(n − k) is obtained from the signal (f
n
) applying k delay operations. In
particular, if k = 0, we have
f(n −0) = f(n) = R
0
(f
n
) = I(f
n
)
where I is the identity operator. In this way, equation (A.1) can be written in operator form:
S(f
n
) =
+∞
¸
k=−∞
h(k)R
k
(f
n
) (A.2)
Given a matrix (a
ij
), which can be inﬁnite, the elements a
jj
constitute the main diagonal.
¸
¸
¸
¸
¸
¸
¸
.
.
.
a
0,0
a
1,1
a
2,2
.
.
.
¸
For each d ∈ Z, d > 0, the elements (a
j+d,j
) are the elements of the dth lower diagonal.
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
0
0 0
• 0 0
0 • 0 0
0 • 0 0
.
.
.
¸
Similarly, the elements (a
j,j+d
) are the elements of the dth upper diagonal.
The identity operator is represented by the identity matrix
I =
a
j,j
= 1
a
i,j
= 0 if i = j.
whose elements in the main diagonal are equal to 1, and the other elements are zero.
The unit delay operator
R(f
n
) = f(n −1)
is represented by the matrix I
−1
whose ﬁrst lower diagonal consists of ones and the other
elements are zero:
I
−1
=
a
j+1,j
= 1
a
i,j
= 0 if i = j + 1.
180 APPENDIX A. SYSTEMS AND FILTERS
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
0
1 0
0 1 0
0 1 0
.
.
.
¸
(A.3)
More generally, the matrix of the operator R
k
, shifted by k units is:
I
−k
=
a
j+k,j
= 1
a
i,j
= 0 if i = j +k.
that is, all elements outside the kth lower diagonal are zero, and the elements of this diagonal
are equal to one.
Similar results hold for the operation
f(n) −→f(n +k)
of time advance, considering the upper diagonals.
Observing equation A.2 we see that the matrix of S is a matrix whose main, lower and
upper diagonals are constant. More precisely, the kth diagonal is constituted by the elements
h(k).
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
h(1) h(0) h(−1)
h(1) h(0) h(−1)
h(1) h(0) h(−1)
h(1) h(0) h(−1)
.
.
.
¸
Observe that in the rows of this matrix we have the vector of the system impulse response,
translated by one unit to the right from one line to the next. Because the output (y
n
) will
be the result of the inner product of the nth line of the matrix by the input vector (x
n
), we
can either shift the input (x
n
) before the multiplication, or shift the output (y
n
) after the
multiplication: the result will be the same. This was expected, because the system represented
by this matrix is linear and spatially invariant.
It is easy for the reader to verify that if the system is causal, all upper diagonals are zero
(because the system cannot use advances of the signal in time). Therefore, the matrix is lower
triangular.
Example 17. Consider the ﬁlter deﬁned by the diﬀerence equation
y(n) = 3x(n −1) + 2x(n −2) +x(n −3)
Then, we can rewrite this diﬀerence equation in the form of a convolution product
y(n) =
¸
k
h(k)x(n −k)
A.2. DISCRETIZATION OF SYSTEMS 181
where,
h(k) =
0 if k ≤ 0,
3 if k = 1,
2 if k = 2,
1 if k = 3,
0 if k ≥ 4.
or, alternatively (h
n
) = (. . . , 0, 0, 3, 2, 1, 0, 0, . . . ).
The matrix of this ﬁlter is given by
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
ˆ x(3)
ˆ x(4)
ˆ x(5)
ˆ x(6)
ˆ x(7)
.
.
.
¸
=
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
0
3 0
2 3 0
1 2 3 0
0 1 2 3 0
0 1 2 3 0
.
.
.
¸
.
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
x(0)
x(1)
x(2)
x(3)
x(4)
x(5)
.
.
.
¸
Note that the ﬁlter is causal
Question 1. What would happen if the coeﬃcients of the diﬀerence equation were not con
stant?
The equation would be described as,
y
n
= +h
0
(n)x
n
+h
1
(n)x
n−1
+h
2
(n)x
In this case the ﬁlter kernel (h
n
), is a sequence which varies with the index (time), therefore,
it is necessary now to represent it as:
(h
n
) = (. . . , h
−1
(n), h
0
(n), h
1
(n), . . . )
The convolution product cannot be used to characterize the operation and its “general
ization” is
(y
n
) =
¸
k
h
k
(n)x
n−k
Finally, the matrix representation does not have constant diagonals:
¸
¸
¸
¸
¸
¸
¸
.
.
.
y
−1
y
0
y
1
.
.
.
¸
=
¸
¸
¸
¸
¸
¸
¸
.
.
.
h
1
(−1) h
0
(−1) h
−1
(−1)
h
2
(0) h
1
(0) h
0
(0)
h
3
(1) h
2
(1) h
1
(1)
.
.
.
¸
.
¸
¸
¸
¸
¸
¸
¸
.
.
.
x
−1
x
0
x
1
.
.
.
¸
182 APPENDIX A. SYSTEMS AND FILTERS
A.3 Upsampling and Downsampling Operators
Besides the delay operator, two other operators are important in the study of discrete ﬁlters:
they are the downsampling and upsampling operators.
The downsampling operator of order q, ↓ q:
2
→
2
, is deﬁned as
(↓ q)(u
n
) = (u
nq
).
That is, the operator discards all terms of a sequence, except the terms which are multiple of
q. This operator is also known in the literature as the decimation operator of order q.
Here, we will only study the case where q = 2. In this case the operator discards alternating
terms in the sequence, retaining only the terms with even index:
(↓ 2)(u
n
) = (u
2n
).
The matrix of this operator is given by
¸
¸
¸
¸
¸
1
0 0 1
0 0 1
0
¸
.
This matrix is obtained from the identity matrix I, by including a column of zeros in alterna
tion. Or equivalently, we can shift the lines of the matrix one unit to the right in alternation.
It is immediate to verify that the downsampling operator is not invertible. However, it
has a inverse to the right, which is the upsampling operator:
(↑ 2)(. . . , x
−1
, x
0
, x
1
, . . . ) = (. . . , x
−1
, 0, x
0
, 0, x
1
, 0, . . . ).
That is,
(↑ 2)(u
n
)(k) =
u(k) if n = 2k
0 if n=2k+1.
The upsampling operator simply intercalates zeros in between the elements of the represen
tation sequence.
1
It is easy for the reader to verify that the upsampling operator ↑ 2 is the inverse to the
right downsampling operator ↓ 2, that is (↓ 2)(↑ 2) = I. However, (↑ 2)(↓ 2) = I, as we can
see below:
(↑ 2)(↓ 2)
¸
¸
¸
¸
¸
¸
¸
.
.
.
u
−1
u
0
u
1
.
.
.
¸
=
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
¸
.
.
.
u
−2
0
u
0
0
u
2
0
.
.
.
¸
.
1
we can deﬁne the upsampling operator of order q, intercalating q zeros.
A.4. FILTER BANKS 183
In terms of matrices, it is easy to see that the matrix of the operator ↑ 2 is obtained by
intercalating rows of zeros in the identity matrix
↑ 2 =
¸
¸
¸
¸
¸
¸
¸
1 0
0 0
1 0 0
0 0
1 0
¸
.
This is equivalent to translate the columns of the identity matrix by one unit to the bottom.
It is immediate to verify that the matrix of the upsampling operator is the transpose of
the matrix of the downsampling operator, and viceversa. The relation between the operators
of downsampling and upsampling can be stated using matrices:
(↓ 2)(↑ 2) = I,
and
(↑ 2)(↓ 2) =
¸
¸
¸
¸
¸
¸
¸
¸
1
0
1
0
1
.
.
.
¸
.
Once more, we observe that the upsampling operator is the inverse to the right, but is not
an inverse to the left. At this point we can argue that the nature of the mathematical
objects is against us: as we will see, the most important operation is to recover a signal after
downsampling, and not the other way around.
A.4 Filter Banks
A ﬁlter bank is a system composed of several ﬁlters, together with delay, upsampling and
downsampling operators.
A ﬁlter bank with two ﬁlters, downsampling, and upsampling is illustrated in FigureA.3.
x H 2 2 G x
^
Figure A.3: Example of a Filter Bank
184 BIBLIOGRAPHY
A.5 Comments and References
A basic reference for signal processing is (Oppenhein & Willsky, 1983). It includes the whole
theory of signals and systems, from the continuous to the discrete. Another good source, but
somewhat older, discussing only continuous systems is (Lathi, 1974).
For a book more applied to the project and analysis of ﬁlters, the reader can use (Antoniou,
1993).
The literature of signal processing has several sources for the reader interested in systems
with multirate ﬁlter banks, one of them is (Vaidyanathan, 1993).
Bibliography
Antoniou, Andreas. 1993. Digital Filters. Second edition edn. McGrawHill.
Lathi, Bhagwandas P. 1974. Signals, Systems, and Control. Harper & Row.
Oppenhein, Alan V., & Willsky, Alan S. 1983. Signal and Systems. PrenticeHall inc.
Vaidyanathan, P. P. 1993. Multirate Systems and Filter Banks. Englewood Cliﬀs, New Jersey:
Prentice Hall PTR.
Appendix B
The Z Transform
The behavior of ﬁlters in the frequency domain gives a good intuition of some of its char
acteristics, which are diﬃcult to grasp by looking only at the ﬁlter coeﬃcients in the time
domain. We have seen that the Fourier transform is a tool used to study a signal in the
frequency domain. In this appendix, we will introduce a similar, but more general transform,
the Ztransform, which in fact contains the Fourier transform.
B.1 The Z Transform
Given a linear, time invariant, discrete system S with impulse response (h
n
), and an input
signal (x
n
), the output is then (y
n
) = (h
n
) ∗ (x
n
). However, let’s choose an input signal with
a special structure:
x(n) = z
n
, z ∈ Z
in this case, the output signal is then:
(y
n
) = (h
n
) ∗ (x
n
) =
+∞
¸
k=−∞
h(k)x(n −k)
=
+∞
¸
k=−∞
h(k)z
n−k
= z
n
+∞
¸
k=−∞
h(k)z
−k
that is, the value of the output signal y(n) is obtained by multiplying x(n) by a constant This
constant, however, varies with the value of z, which was ﬁxed for each input signal:
y(n) = H(z)z
n
where,
H(z) =
+∞
¸
k=−∞
h(k)z
−k
We should remark that if an arbitrary signal is decomposed as a direct sum of basic atoms,
it would be suﬃcient to know the eﬀect of the system over its atoms to be able to predict the
185
186 APPENDIX B. THE Z TRANSFORM
result of the system on any input signal. For this reason, we have chosen our input signal as
a sum of diﬀerent complex exponentials:
x
(n) =
¸
k
a
k
z
n
k
and, applying it to the system S, we can use its linearity property to obtain:
y
(n) =
¸
k
a
k
H(z
k
)z
n
k
Using a linear algebra terminology, the complex exponentials z
n
k
, are eigenvector functions
of S, and H(z
k
) are their respective eigenvalues.
Exploiting the above intuition, we will introduce the Z transform.
Given a discrete signal (x
n
), its Z transform is given by:
X(z) = Z¦x(n)¦ =
∞
¸
k=−∞
z
−k
x(k) (B.1)
Together with X(z) it is necessary to know the corresponding Convergence Region, that
is, the regions of the complex plane in which the variable z is deﬁned, such that this transform
converges.
Remark 1. Based on this transform, making z = e
i2πw
, we obtain the Fourier transform for
discrete aperiodic signals (DTFT – “Discrete Time Fourier Transform”).
B.1.1 Some Properties
Several important properties of the Z transform will help in the manipulation of discrete
systems. We will see that these properties contribute to the intuitive view alluded to in the
beginning of this Appendix.
• Linearity;
Z¦x
1
(n) +x
2
(n)¦ = Z¦x
1
(n)¦ +Z¦x
2
(n)¦
• Time Shift;
Z¦x(n −l)¦ =
∞
¸
k=−∞
z
−k
x(k −l) =
∞
¸
k=−∞
z
−k+l
x(k)
= z
l
Z¦x(n)¦
• Time Reversal;
x(n) ↔X(z) ←→ x(−n) ↔X(z
−1
)
This property, as the next one, is easily veriﬁed, just by applying the deﬁnition of the
transform.
B.1. THE Z TRANSFORM 187
• Convolution in Time is Multiplication in Frequency;
Z¦x
1
(n) ∗ x
2
(n)¦ = X
1
(z)X
2
(z)
Remark 2. The function delta, δ, exhibits an special behavior. We have seen when it was
the input of a system, the output was the impulse response function, which characterizes
the ﬁlter. This has two main reasons: First, the convolution product in the time domain is
equivalent to multiplication in frequency; Second, the representation of the delta function in
the frequency domain is:
Z¦δ(n)¦ = 1
This means that the function delta δ(n) is the neutral element of the convolution operation
in time and of the multiplication operation in frequency.
B.1.2 Transfer Function
The impulse response h(t) of a ﬁlter has a correspondent in the frequency domain, which is
called Transfer Function.
Given a system S, its impulse response h = S(δ) completely deﬁnes the ﬁltering operation
S(f) = h ∗ f.
What would be the interpretation of the fact above in the frequency domain, using the Z
Transform?
Suppose that S(x
n
) = (y
n
), that is,
(y
n
) = (h
n
) ∗ (x
n
).
Going to the frequency domain, and using the properties of the Z transform, we have:
Y (z) = Z¦(h
n
) ∗ (x
n
)¦ = Z¦(h
n
)¦.Z¦(x
n
)¦ = H(z)X(z)
We isolate H(z) = Z¦(h
n
)¦, and call this expression Transfer Function:
H(z) =
Y (z)
X(z)
Again, it is necessary to be careful with the convergence of this expression, and the division
operation takes care that the zeros of X(z) are outside the convergence region.
The transfer function H(z) gives exactly information on how the input frequencies (x
n
)
will be altered in order to produce the output (y
n
). This connection was one of the goals of
our intuitive view. A linear, time invariant, discrete system will react in a predictable manner
according to the complex exponentials applied to its input. Its output will be given as the
same input exponential, but with a diﬀerent “amplitude”.
Considering that a ﬁlter is exactly the system that makes this type of transformation,
then the Z transform is a good way to describe ﬁlters. What would be the disadvantages of
using (h
n
)?
188 APPENDIX B. THE Z TRANSFORM
One is immediate: the variable z is not discrete, it is also deﬁned in the entire complex
plane (in the Convergence Region). To store it in the computer it is necessary to do some
kind of “sampling”. The other option would be the use of a symbolic machinery, but the
manipulation would be more diﬃcult and less eﬃcient.
For the tasks of ﬁlter design and evaluation, the importance of the Z transform is undis
putable. To obtain the transfer function H(z) from the linear diﬀerence equation with constant
coeﬃcients, i.e. the convolution product, is a direct operation: the coeﬃcients of the equation
are exactly the coeﬃcients of a polynomial in z:
Z¦y(n)¦ = Z¦
¸
k
b
k
x(n −k)¦
Y (z) = (
¸
k
b
k
z
−k
)X(z)
and then,
H(z) =
¸
k
b
k
z
−k
In a more general form, we can have:
Z¦
N
¸
k=0
a
k
y(n −k)¦ = Z¦
¸
k
b
k
x(n −k)¦
(
N
¸
k=0
a
k
z
−
k)Y (z) = (
¸
k
b
k
z
−k
)X(z)
in this case H(z) is a ratio of polynomials
H(z) =
¸
k
b
k
z
−k
N
¸
k=0
a
k
z
−
k
Example 18. Going back to the system of example 17, in the previous chapter, it will be
instructive to determine the system transfer function, and look at its frequency behavior
ˆ x(n) = 3x(n −1) + 2x(n −2) +x(n −3)
Z¦y(n)¦ = Z¦3x(n −1) + 2x(n −2) +x(n −3)¦
Exploiting the linearity and time shift properties, we have:
Y (z) = 3z
−1
X(z) + 2z
−2
X(z) +z
−3
X(z)
and factoring out X(z),
H(z) =
Y (z)
X(z)
= 3z
−1
+ 2z
−2
+z
−3
B.2. SUBSAMPLING OPERATIONS 189
In order to really visualize the behavior of a ﬁlter in the diﬀerent frequencies that compose
the input signal, it is suﬃcient to evaluate the transfer function H(z) for the values of z = e
iw
,
where w = [0, π]. In this last example [H(e
j
w)[ is shown in Figure B.1:
3 2 1 1 2 3
1
2
3
4
5
6
Figure B.1: Modulus of the frequency response of a ﬁlter.
B.1.3 The Variable z and Frequency
Writing the complex variable z in polar form, we obtain
z = αe
iw
, where 0 < w < π
The discrete domain has a limited interval of possible frequencies due to the sampling
process. Making a normalization, all these possible frequencies will be contained in the in
terval [0, π], where π has a direct relation with 1/2 of the sampling frequency used in the
discretization process.
Therefore, it makes sense to consider the function H(e
iw
) as a periodic function. Again,
H(e
iw
) is also the DTFT of the nonperiodic ﬁlter (h
n
) and every DTFT is periodic with a
period of 2π
Through the behavior of systems in the various frequency regions, it is possible also to
classify the ﬁlters into basic types, such as lowpass, highpass, bandpass, etc, as we have seen
in Chapter 3.
B.2 Subsampling Operations
In this section we will discuss the downsampling and upsampling operators in the frequency
domain. We will derive the expressions for these operators using the znotation.
B.2.1 Downsampling in the Frequency Domain
The downsampling operator, introduced in the previous chapter, takes a discrete sequence
(x
k
)
k∈Z
and removes all elements with odd indices, keeping only even index elements. There
190 APPENDIX B. THE Z TRANSFORM
fore, v = (↓ 2)x implies that v
k
= x
2k
.
In order to derive the expression for the downsampling operator in the frequency domain,
we will take a sequence (u
n
), which has all the elements with even index from an arbitrary
sequence (x
n
), and the elements with odd indices equal to zero
u
n
=
x
n
if n is even
0 if n is odd.
That is, u = (. . . , x
0
, 0, x
2
, . . . ). Clearly, (↓ 2)x = (↓ 2)u. So, let’s write the Fourier transform
of u
U(ω) =
¸
n even
x
n
e
−inω
=
1
2
¸
all n
x
n
e
−inω
+
1
2
¸
all n
x
n
e
−in(ω+π)
(B.2)
The expression for U was split into two terms, so that when they are added together only
the terms with even indices remain. This is because, if n = 2l is even, then e
−i(2l)(ω+π)
=
e
−i2lω+2lπ
= e
−i2lω
, preserving the even index elements. But, if n = 2l + 1 is odd, then
e
−i(2l+1)(ω+π)
= e
−i(2l+1)ω+(2l+1)π
= e
−i(2l+1)ω+π
= −e
−i2lω
, removing the odd index ele
ments.
Therefore, the expression of U(ω) in the frequency domain can be written in terms of
X(ω) as U(ω) =
1
2
[X(ω) +X(ω +π)].
Now, using U(ω) which contains only even index terms, we can write the formula for the
downsampling operator in frequency domain just by halving the frequencies V (ω) = U(ω/2).
V (ω) =
1
2
[X(
ω
2
) +X(
ω
2
+π)] (B.3)
The downsampling operation corresponds to a change in the sampling rate, as it is implied
by halving the frequencies ω →ω/2. This may cause aliasing, which is essentially a frequency
folding, manifested by the introduction of a new term in ω/2 +π. This process is illustrated
in Figure B.2.
X(w)
X(w)
X(w + )
π
Figure B.2: Downsampling Operator.
In the Zdomain, the downsampling operator is
V (z) =
1
2
[X(z
1/2
) +X(−z
1/2
)] (B.4)
B.2. SUBSAMPLING OPERATIONS 191
B.2.2 Upsampling in the Frequency Domain
The upsampling operator, also introduced in the previous chapter, takes a discrete sequence
(x
k
)
k∈Z
and an interleave zeros inbetween the sequence elements.
u = (↑ 2)v ⇔
u
2k
= v
k
u
2k+1
= 0
In the frequency domain, this operation has a simple expression. We only retain the terms
with even index n = 2k, because terms with odd index are zero, u
2k+1
= 0.
U(ω) =
¸
u
n
e
−inω
=
¸
u
2k
e
−i2kω
=
¸
v
k
e
−i2kω
(B.5)
So, u = (↑ 2)v is U(ω) = V (2ω), or U(z) = V (z
2
).
The upsampling operation causes imaging. A bandlimited function V (ω) with period 2π
is mapped into a function U(ω) = V (2ω) with period π. Compressed copies of the graph
V (2ω) appear in the spectrum of U(ω). This is illustrated in Figure B.3.
X(w)
U(w) = X(2w)
π
Figure B.3: Upsampling Operator.
B.2.3 Upsampling after Downsampling
In a ﬁlter bank it is common to apply both the downsampling and upsampling operations.
This is because, usually downsampling is part of the analysis bank, and upsampling is part
of the synthesis bank.
Let’s see how these two operations can be combined together:
v = (↓ 2)x and u = (↑ 2)v
this result is u = (↑ 2)(↓ 2)x, which in the frequency domain is
U(ω) =
1
2
[X(ω) +X(ω +π)] (B.6)
or in the Zdomain
U(z) =
1
2
[X(z) +X(−z)] (B.7)
192 BIBLIOGRAPHY
B.3 Comments and References
A good reference for the use of ﬁlter banks and their relations with wavelets is (Strang &
Nguyen, 1996). A more general book about multirate systems and ﬁlter banks is (Vaidyanathan,
1993).
Bibliography
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Vaidyanathan, P. P. 1993. Multirate Systems and Filter Banks. Englewood Cliﬀs, New Jersey:
Prentice Hall PTR.
BIBLIOGRAPHY 193
Bibliography
Antoniou, Andreas. 1993. Digital Filters. Second edition edn. McGrawHill.
Bacry, Emmanual, Mallat, Stephane, & Papanicolaou, George. 1992. A wavelet based space–
time adaptive numerical method for partial diﬀerential equations. Mathematical Mod
elling and Numerical Analysis, 26, 793–834.
Beylkin, G., Coifman, R., & Rokhlin, V. 1991a. Fast wavelet transforms and numerical
algorithms. Comm. in Pure and Applied Math., 44, 141–183.
Beylkin, G., Coifman, R., & Rokhlin, V. 1991b. Fast Wavelet Transforms and Numerical
Algorithms I. Communications on Pure and Applied Mathematics, XLIV, 141–183.
transformation en ondelettes.
Blum, L. 1991. Lectures on a Theory of Computation and Complexity over the Reals. 18
Coloquio Brasileiro de Matematica – IMPA.
BourgesS venier, Mikael. 1994. R alisation d’une biblioth que C de fonctions ondelettes. Tech.
rept. IRISA – INRIA.
Briggs, W. L., & Henson, V. E. 1995. DFT: an owner’s manual for the discrete Fourier
transform. SIAM books, Philadelphia.
Burt, P. J., & Adelson, E. H. 1983. The Laplacian Pyramid as a Compact Image Code. IEEE
Trans. Commun., 31(4), 532–540.
Butzer, P. L., & Stens, R. L. 1992. Sampling Theory for not necessarily bandlimited functions:
a historical overview. SIAM Reviews, 34(1), 40–53.
Canuto, Claudio, & Tabacco, Anita. 1997. Multilevel Decompositions of Functional Spaces.
The Journal of Fourier Analysis and Applications, 3(6).
Chen, Shaobing, & Donoho, David L. 1994. Basis Pursuit. Tech. rept. Stanford University.
Chen, Shaobing, & Donoho, David L. 1995. Atomic Decomposition by Basis Pursuit. Tech.
rept. Stanford University.
Chui, C. K. 1992. An introduction to wavelets. Academic Press.
Cohen, A., Daubechies, I., & Feauveau, J. C. 1992. Biorthogonal Bases of Compactly Sup
ported Wavelets. Comm. Pure Applied Math.
Cohen, A., Daubechies, I., & Vial, P. 1993. Wavelets on the interval and fast wavelet trans
forms. Applied and Computational Harmonic Analysis, 1(1), 54–81.
Coifman, Ronald R., & Wickerhauser, Mladen Victor. 1992. EntropyBased Algorithms for
Best Basis Selection. IEEE Transactions on Information Theory, 38(2), 713–718.
194 BIBLIOGRAPHY
Dahmen, Wolfgang, & Schneider, Reinhold. 1998. Wavelets on Manifolds I: Construction and
Domain Decomposition. Tech. rept. Institut fur Geometrie und Praktische Mathematik.
Daubechies, I. 1988. Orthonormal Bases of Compactly Supported Wavelets. Comm. Pure
Applied Math., XLI(41), 909–996.
Daubechies, I., Grossmann, A., & Meyer, Y. 1986. Painless nonorthogonal expansions. J.
Math. Phys., 1271–1283.
Daubechies, Ingrid. 1992. Ten Lectures on Wavelets. Philadelphia, PA: SIAM Books.
Duhamel, P., & Vetterli, M. 1990. Fast Fourier Transform: A tutorial review and a state of
the art. Signal Proc., 19(4), 259–299.
Fournier, Alain. 1994. Wavelets and their applica
tions in computer graphics. SIGGRAPH Course Notes,
http://www.cs.ubc.ca/nest/imager/contributions/bobl/wvlt/download/notes.ps.Z.saveme.
Gabor, D. 1946. Theory of Communication. J. of the IEEE, 429–457.
Gomes, J., & Velho, Luiz. 1997. Image Processing for Computer Graphics. SpringerVerlag.
Gomes, Jonas, & Velho, Luiz. 1995. Abstraction Paradigms for Computer Graphics. The
Visual Computer, 11, 227–239.
Gomes, Jonas, Costa, Bruno, Darsa, Lucia, & Velho, Luiz. 1996. Graphical Objects. The
Visual Computer, 12, 269–282.
Greenspan, Donald. 1981. Computer Oriented Mathematical Physics. International Series in
Nonlinear Mathematics, methods and Applications. Pergamon Press, New York.
Hernandez, E., & Weiss, G. 1996. A First Course on Wavelets. Boca Raton: CRC Press.
Higham, Nicholas J. 1996. Accuracy and Stability of Numerical Algorithms. SIAM Books,
Philadelphia.
Jawerth, B., & Sweldens, Wim. 1994. An overview of wavelet based multiresolution analyses.
SIAM Rev., 36(3), 377–412.
Kaiser, G. 1994. A Friendly Guide to Wavelets. Boston: Birkhauser.
Lathi, Bhagwandas P. 1974. Signals, Systems, and Control. Harper & Row.
Lim, Jae S. 1990. Two Dimensional Signal and Image Processing. New York: PrenticeHall.
Loan, Van. 1996. Computational Frameworks for the Fast Fourier Transform. SIAM books,
Philadelphia.
Mallat, S. 1989a. Multifrequency Channel Decomposition of Images and Wavelet Models.
IEEE Transaction on ASSP, 37, 2091–2110.
BIBLIOGRAPHY 195
Mallat, S. 1989b. Multiresolution approximation and wavelets. Trans. Amer. Math. Soc.,
315, 69–88.
Mallat, S. 1998. A Wavelet Tour of Signal Processing. Academic Press.
Mallat, S., & Zhang, A. 1993. Matching Pursuit with Timefrequency Dictionaries. Tech.
rept. Courant Institute of Mathematical Sciences.
Mallat, S., & Zhong, S. 1992. Characterization of signals from multiscale edges. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 14(7), 710–732.
Mallat, S. G., & Hwang, W. L. 1992. Singularity detection and processing with wavelets.
IEEE Transactions on Information Theory, 38(2), 617–643.
Malvar, H. S. 1990. Modulated QMF Filter Banks with Perfect Reconstruction. Electronics
Letters, 26(June), 906–907.
Markushevich, A. I. 1977. Theory of Functions of a Complex Variable. Chelsea Publishing
Co.
Meyer, Y. 1990. Construction de bases orthonorm´ees d’ondelettes. Colloq. Math., 60/61,
141–149. bib.
Oppenhein, Alan V., & Willsky, Alan S. 1983. Signal and Systems. PrenticeHall inc.
Press, William H., Teukolsky, Saul A., & Vetterling, William T. 1996. Numerical Recipes :
The Art of Scientiﬁc Computing. Cambridge Univ Press. Chap. 13, pages 591–606.
Requicha, A. A. G. 1980. Representations for Rigid Solids: Theory Methods, and Systems.
ACM Computing Surveys, 12(December), 437–464.
Schroder, Peter. 1996. Wavelets in Computer Graphics. Pages 615–625 of: Proceedings of the
IEEE, vol. 84.
Schroder, Peter, & Sweldens, Wim. 1995. Wavelets in computer graphics. SIGGRAPH Course
Notes, http://www.multires.caltech.edu/teaching/courses/waveletcourse/.
Simoncelli, Eero P., & Freeman, William T. 1995 (Oct.). The Steerable Pyramid: A Flexible
Architecture for MultiScale Derivative Computation. Pages 444–447 of: International
Conference on Image Processing, vol. 3.
St¨ ockler, J. 1992. Multivariate wavelets. Pages 325–356 of: Chui, C. K. (ed), Wavelets: A
Tutorial in Theory and Applications. Academic Press.
Stollnitz, Eric J., DeRose, Tony D., & Salesin, David H. 1995a. Wavelets for Computer
Graphics: a Primer, Part 1. IEEE Computer Graphics and Applications, 15(3), 76–84.
Stollnitz, Eric J., DeRose, Tony D., & Salesin, David H. 1995b. Wavelets for computer
graphics: a primer, part 2. IEEE Computer Graphics and Applications, 15(4), 75–85.
196 BIBLIOGRAPHY
Stollnitz, Eric J., DeRose, Tony D., & Salesin, David H. 1996. Wavelets for Computer Graph
ics: Theory and Applications. San Francisco, CA: Morgann Kaufmann.
Strang, G., & Shen, J. 1996. The zeros of the Daubechies polynomials. Proc. Amer. Math.
Soc.).
Strang, Gilbert, & Nguyen, Truong. 1996. Wavelets and Filter Banks. Wellesley, MA:
WellesleyCambridge Press.
Strang, Gilbert, & Strela, Vasily. 1994. Orthogonal multiwavelets with vanishing moments.
Optical Engineering, 33(7), 2104–2107.
Sweldens, W. 1995. The lifting scheme: A construction of second generation wavelets. Tech.
rept. 1995:6. Department of Mathematics, University of South Carolina.
Taswell, Carl. 1995 (Aug.). Algorithms for the Generation of Daubechies Orthogonal Least
Asymmetric Wavelets and the Computation of Their Holder Regularity. Tech. rept. Sci
entiﬁc Computing and Computational Mathematics, Stanford University, Stanford, CA.
Taswell, Carl. 1997a. Computational Algorithms for Daubechies LeastAsymmetric, Symmet
ric, and MostSymmetric Wavelets. Miller Freeman. Pages 1834–1838.
Taswell, Carl. 1997b (Dec.). The Systematized Collection of Wavelet Filters Computable by
Spectral Factorization of the Daubechies Polynomial. Tech. rept. Computational Tool
smiths, www.toolsmiths.com.
Vaidyanathan, P. P. 1993. Multirate Systems and Filter Banks. Englewood Cliﬀs, New Jersey:
Prentice Hall PTR.
Vetterli, M., & Herley, C. 1992. Wavelets and ﬁlter banks: theory and design. IEEE Trans.
Acoust. Speech Signal Process., 40(9).
Vetterli, Martin, & Kovacevic, Jelena. 1995. Wavelets and Subband Coding. Englewood Cliﬀs,
New Jersey: Prentice Hall PTR.
Weaver, J. 1989. Theory of Discrete and Continuous Fourier Transform. New York: John
Wiley & Sons.
Wickerhauser, Mladen Victor. 1994. Adapted Wavelet Analysis from Theory to Software.
Wellesley, MA: A. K. Peters.
Zayed, Ahmed. 1993. Advances in Shannon’s Sampling Theory. Boca Raton: CRC Press.
Course Abstract
Most real world applications can be reduced to the problem of function representation and reconstruction. These two problems are closely related to synthesis and analysis of functions. The Fourier transform is the classical tool used to solve them. More recently, wavelets have entered the arena providing more robust and ﬂexible solutions to discretize and reconstruct functions. Starting from Fourier analysis, the course guides the audience to acquire an understanding of the basic ideas and techniques behind the wavelets. We start by introducing the basic concepts of function spaces and operators, both from the continuous and discrete viewpoints. We introduce the Fourier and Window Fourier Transform, the classical tools for function analysis in the frequency domain, and we use them as a guide to arrive at the Wavelet transform. The fundamental aspects multiresolution representation and its importance to function discretization and to the construction of wavelets is also discussed. Emphasis will be given on ideas and intuition, avoiding the heavy computations which are usually involved in the study of wavelets. The attendees should have a basic knowledge of Linear Algebra, Calculus, and some familiarity with Complex Analysis. Basic knowledge of signal and image processing would be desirable. These notes originated from a set of notes in Portuguese that we wrote for a wavelet course on the Brazilian Mathematical Colloquium in 1997 at IMPA, Rio de Janeiro. We wish to thank Siome Goldenstein who collaborated with us to produce the portuguese notes.
Course Timeline
1. Fundamentals of Fourier Analysis Duration: 8:30 – 9:15 (45 minutes) Speaker: Jonas Gomes 2. From TimeFrequency Localization to Wavelets Duration 9:15 – 10:00 (45 minutes) Speaker: Jonas Gomes Coﬀee Break Duration: 10:00 – 10:15 (15 minutes)
3. Filter Banks and Wavelets Duration 10:15 – 11:00 (45 minutes) Speaker: Luiz Velho 4. Wavelet Design Duration 11:00 – 12:00 (60 minutes) Speaker: Luiz Velho
Course Organizers
Jonas Gomes
IMPA  Instituto de Matematica Pura e Aplicada Estrada Dona Castorina, 110 22460320, Rio de Janeiro, RJ, Brazil Email: jonas@visgraf.impa.br Short Biography. Jonas Gomes is a full professor at The Institute of Pure and Applied Mathematics, IMPA, in Rio de Janeiro. He took a phd in Mathematics in 1984, and has been working with computer graphics since then. He was the manager of the R&D team at Globo Television Network from 1984 to 1988. He created the computer graphics research lab at IMPA in 1989. This lab develops research and supports a graduate program in graphics. He is the author of several books in graphics, and has published several papers. He has already organized three SIGGRAPH courses: “Modeling in Graphics” in SIGGRAPH ’93, and “warping and morphing of graphical objects” in SIGGRAPH ’95 and in SIGGRAPH ’97. His current research interests include the Mathematics of computer graphics, Modeling, Visualization, Image Processing and Multimedia.
Luiz Velho
IMPA  Instituto de Matematica Pura e Aplicada Estrada Dona Castorina, 110 22460320, Rio de Janeiro, RJ, Brazil Email: lvelho@visgraf.impa.br Short Biography. Luiz Velho is an Associate Researcher at IMPA  Instituto de Matematica Pura e Aplicada. He received a BE in Industrial Design from ESDI  Universidade do Rio de Janeiro in 1979, a MS in Computer Graphics from the Massachusetts Institute of Technology, Media Laboratory in 1985, and a Ph.D. in Computer Science in 1994 from the University of Toronto. His experience in computer graphics spans the ﬁelds of modeling, rendering, imaging and animation. During 1982 he was a visiting researcher at the National Film Board of Canada. From 1985 to 1987 he was a Systems Engineer at the Fantastic Animation Machine in New York, where he developed the company’s 3D visualization system. From 1987 to 1991 he was a Principal Engineer at Globo TV Network in Brazil, where he created special eﬀects and visual simulation systems. In 1994 he was a visiting professor at the Courant Institute of Mathematical Sciences, New York University. He is the author of several books in graphics, and has published several papers in this area. He has been a speaker in two SIGGRAPH courses: “modeling in graphics” in SIGGRAPH ’93, and “warping and morphing of graphical objects” in SIGGRAPH ’94. His current research interests include theoretical foundations of computer graphics, physicallybased methods, wavelets, modeling with implicit objects and volume visualization.
3 Representation by Frames . . 8 . . . .2. .5. . . . . 6 . . . . . . . . 2. . . . . . . . . . . .5 Representation by Projection .3. . . . . . . . . . . .7 Comments and References . . . . . . . . . . . . . . .11 Wavelets and Function Representation . . 2. . . . . . . . . . . . . Point Sampling and Interpolation 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. . . . . . . . . . . . . . . . .4 Riesz Basis Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Multiresolution Representation . . . . v .5 Where do Wavelets Fit? . . . . . . . . . . . . . . . . . . . 6 . . . . .2 Multiresolution Representation . . . . . .3. . . . .7 Reconstruction. . . . . . . . . . .1 Computational Mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . . 6 . . . . . . . . . . . . . .10 Redundancy in the Representation . . . . .1. . 1. . . . . . . . . . . . . . . . . . . . . . . . . . . 9 . . . . . . . . 1. . . . . . . . . . . . . . . . . 1. . . . . . . . . . . . . . . . .5. 1 . . . . . . 2. . . 2 Function Representation and Reconstruction 2. . . . . . . . . . . . . . . . 1 . . . . . . . . . 2. . . . 2. . . . . . .1. . 2. . . . . . . . . . . . 9 . . . . . . . . . 2. . . 1. 4 . . .6 Galerkin Representation . . . . . . . . . . .7. . . . . . . 1. . . . 2. . . . . . . . . . . . . . . . . . .1 The Representation Operator . . .7. .9 Representation by Dictionaries . . . . . . . . 2. . . . 7 . . . . . .4 What is the Relation with Graphics? . . . . . . .1 Abstraction Levels . . . . 8 . 10 . . .2 Speciﬁcation of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Piecewise Constant Reconstruction . . . 6 . . . .4. . . . . . . . . . . . . . 1. . . .2 Relation Between the Abstraction Levels . . . . . . . 2. . . . . . . . . . . . . . . . . . . 1. . . . . .3 Functions and Computational Mathematics . . . . . . . . . . . . . . . 2. . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . 11 11 11 12 13 13 15 16 16 17 18 18 19 21 22 22 23 .1 Function Representation Using Wavelets . . . . . . . . . . . . . . . . .1 Representing Functions . . . . . . . . . . 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Contents 1 Introduction 1. . . . . . . . . . . . . .2 Basis Representation .2 Piecewise Linear Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.12 Comments and References .6 About these Notes . . . . . . . 2. . . . 1. .1 Description of Graphical Objects . . . . . . 1 . . . . . . . 1.1 Representation and Reconstruction of Functions 1. . . . . . . 2. . . . . . . . . . . . . . . . . . .1 Complete Orthonormal Representation . . . . . . . . . . . . . . . . . .
.4 Comments and References . .7 Point Sampling and Representation Coeﬃcients . ω) . . . . . . . . . . . . . . . . . . . . . 4. . . . . . . . . . . . .2 Fourier Transform . . . . . . . . . 4. . . . . . . . . . . . . . . . . . . . . 4 Windowed Fourier Transform 4. . . . .1. . . . . . . . . . . . . 23 3 The Fourier Transform 3. . .2. . . . . . . . 4. . . . . . . ˜ 4. . . . . . . . . . . . . . . . . . . 3. . . . . . 4. . . . . . . . . . . . .5. .1 Fourier Series . . . . . . . . .5 Fourier Transform and Function Representation . . . . . . . . . . . . . . . . . . 4.2 The Windowed Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . .2 The Theorem of ShannonWhittaker . . . . . . . . . . . . . . . . . . . . . . .2 Image of the Wavelet Transform 5. . . Bibliography . . . . . . . . . . . . . . . . . . . . . . .5 WFT and Atomic Decomposition . . . . . . . . . . . . . . 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Analyzing Functions .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . . . . . . .3 Timefrequency Domain . . .8 Comments and References . . . . . . . . . . . .1 Invertibility of f (t. . .1. . . . 3. . . . . 3. . . . . . . . . . . . . . . . . . . . .1 The Uncertainty Principle . . . . . . . . . . 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Point Sampling and Representation by Projection . . . . . . . 3. . . . . . . . . . . . . . .3 Frequency Analysis . . 3. . . .5. . . . . . . 5. . .vi CONTENTS Bibliography . . . . . . 5. . . . . . . . . . . . . . . . . . . . . .2 Filtering and the Wavelet Transform . . . . . . . . . . . . . . . . . 25 25 25 26 29 29 29 32 34 35 36 37 38 39 40 43 43 44 45 45 46 46 46 48 49 54 55 57 57 58 59 60 64 66 68 68 . . . . . . .1. . . . 3.4 Atomic Decomposition . . . . . . . . . . . . . . . . 3. . . . 5 The Wavelet Transform 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 The Discrete Wavelet Transform . . . . . . . . . . .1 The Wavelet Transform . . . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . .1. . .1 A Walk in The Physical Universe . . . . . . . . . . . . . . . . . . . . . . .2. . . . .2 Image of the Windowed Fourier Transform 4. . . . . . 4. . .2. . . . . . .4 Fourier Transform and Filtering . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . .1 Fourier Transform and Point Sampling . . . . . . . . . . . . . . . 4. . . .2 A Pause to Think . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Inverse of the Wavelet Transform 5. . . . 3. . . . . . . . 5. . . . . . . . . . . . . .3 Spatial and Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . .6 Comments and References . . . . . . . . . .3 WFT and Function Representation . 5. . . . . . .1 Function Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. . . . . .
. .2 Inverse Transform . . . . . . . . . . . . . . . . . . . 107 . . . . . . . . 107 . . 6. . . . .1 Wavelets in the Frequency Domain ˆ 9. 7. . . . . . . . . . . . . . . . . . .2 Filter Banks and Multiresolution Representation 8. . .2. . . . . . . . . . . . . 9. 8. . . . . . . . . . 6. . . . . . . . 6. . . . . . . . . . . . . . . . . . . . . 107 . . . . 7. . . . 7. . . . . . . . . . . . . . . .1 TwoChannel Filter Banks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. .4. 7 The 7. . . .4 Characterization of m1 . .4 Multiresolution Representation and Wavelets 6. . . . . 7. . . . . . .6 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 69 70 72 72 73 75 78 79 81 81 83 83 85 86 86 88 89 89 90 91 91 93 93 95 95 97 99 100 103 104 104 105 8 Filter Banks and Multiresolution 8. . . .3 Fast Wavelet Transform Multiresolution Representation and Recursion TwoScale Relations and Inner Products . . . . . . . 112 . . . . . . . . . . . .2 The Relations of ψ with m1 9. . . . .2 Scale Spaces . . . 9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . 6. . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Comments and References . . . 6. . to See the Wavescape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliography .1. . . . . . . . 8. . . . . . . . . . . . . .3 A Pause to Think .2 Multiresolution Representation . . . . . . . . . . 113 . .3. . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.3 Characterization of m0 . . .1. . . . . . . . . . . . . . . . . . Wavelet Decomposition and Reconstruction . . . . . . . . . .3 A Recipe .4 The Fast Wavelet Transform Algorithm . . . . . . . . . . . . . . . . . . . . . . .5 Computational Complexity . . . . . . . . . . . . . .1 Decomposition . . . 8. . . . .1 A Remark About Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8. . .2 Reconstruction . . . . . . . . 6. . . . .4 Reconstruction Bank . . . . . .. . . . . . . . . .4. . . . . . .1 Forward Transform . . . . . . . . . . . . . . . . .5 A Pause. . . . . . . . . . . . . . . .7 Comments and References .1. . . . . . . . . . 8. . . . . . . . . . . . . . . . . . . . . . . . .1 The Relations of φ with m0 ˆ 9. . . . . . . . . . . . . . . . . . 108 . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Boundary Conditions . . . . . . . . . . . 9. . . . . . . . . . . . . . . . . . . . .1 7. .1. . . . . . . . . . . . . . .1 Pause to Review . . . . . . . Bibliography . . . . . 7. . . . 109 . . . . . . . . .3 Discrete Multiresolution Analysis . . . . . . . . . . 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Two Scale Relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Complexity Analysis of the Algorithm 7. . . . . . . . . . . . . . . . . . . .2 7.3. . . . . . . . .CONTENTS vii 6 Multiresolution Representation 6. . . . . . . . . 9 Constructing Wavelets 9. . . .2 Orthonormalization Method .4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 The Concept of Scale . . . . . 108 . . . . . . . . . . . . . . . . .
5 Analysis of P . . . 11. . . . . . 10. . . . . . . . . . . . 11. . . .4 Analysis of P . . . . . . . . . . . . 11. . . . 130 . . . . . . . .1 The Polynomial P . .1.1. . Bibliography . . . . . 139 . . . . . . . .6 Where to Go Now? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .viii CONTENTS 9. .5 Symmetry or Linear Phase . . . . . 126 . . . . . . .1 P as a Product Polynomial .2. . . . . . . 113 114 117 117 117 10 Wavelet Design 10. . . . . . . . . . . . . . . . 120 . . . . .4 Regularity . . .1. . . . . . . . . . . . . . 119 . . . . . 119 . . . 133 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 . . . . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . 11. . . . . . . . . . . . . . . . . . . . . . . . . 135 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . .1 Conjugate Mirror Filters . . . 10. . . . . . . . . .3 Comments and References . . . . . . . . . . . . . . 131 . . . . . . . . . .3. . . . . . . 133 . . . . .1 Orthogonality . . . . . . . . 9. . . 10. . 128 . . . . . . . . 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Conditions for m0 . . .2 Properties of Wavelets . . . . .6 Example (Haar Wavelet) . . .1. . . . . . 119 . . . . . . . . . . . . . . . . . . . . . . . . 122 . .4 The Factorization of P . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . .2 Examples of Orthogonal Wavelets . . . .1. . 10. . . . . . . . . . . . . . . . . . . . . 9. . . . 126 . . . 121 . . . . . . . . . . . . . 141 . . .5 Factorization of P . . . . . . .3 Coiﬂets . . . . .6 Other Properties . . . . . . . . . . . . . . 144 . . . . . . . . . . . . .1. . . . . . . . . .2 P and the Halfband Property . . . . . . . . . . . . . 143 .1 Orthogonal Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliography . . . .1 Daubechies Extremal Phase Wavelets 11. . . . . 140 . .3 The Expression of P .2. . . . . . 128 . . . . . . . . . . . . . . 10.2. . 124 . . . . . . . 131 . . . . . . . . . .4 Comments and References . . . . . 127 . . . . . . . . . . .1. . . . . 11. . . . . . . . . . . . . . . . . 9. . . . . . . . . . 10. . 136 . . 10. .1 Synthesizing Wavelets from Filters . . . . . .3 Strategy for Computing m0 . .1. . . . 11. . . . . . . 11 Orthogonal Wavelets 11. . . . . . . . . . .2. . . . . . . . . . . 10. . . . . . . . . . . . .5 Shannon Multiresolution Analysis . 134 . . . . . .1. . . .1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . 120 . . .4 Piecewise Linear Multiresolution . . . . . 11. . . . 10. . . . . .2 Support of φ and ψ . 127 . . . . . . . . . 10. . . . 11. . . . . . . . . . .1. . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 . . 10. . . .2. . . 130 . . . . 131 133 .2. . . . 11. . . .2 Biorthogonal Wavelets . . . . . . . . . . . . . . . . 10. . . . . . . . . 138 . . . . .3 Classes of Wavelets . . . . . . . . . . .3 Vanishing Moments and Polynomial Reproduction 10. 10.2 Minimal Phase Orthogonal Wavelets . . .
13. . . . . . . . . . . . . . . . . . . . . . . 166 . . . . . . . .1 Some Properties . 184 . . . . . .1 Biorthogonal Multiresolution Analysis and Filters . . . . . . .1. . . . . . . . . . . . . . . . . 147 147 147 149 150 151 151 153 154 154 160 160 162 163 163 163 165 . . . . . . . . . . . . . . . 167 .2 Conjugate Quadrature Filters . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . 175 . . . . . . . .2 Biorthogonality and Filters . . . . . . . . . . 12. . . . . . . . . . . . . . . . . . . . . . .2 Wavelets with Closer Support Width . . . . . . 166 . . . 176 . . . . . . . .2 Deﬁning other TimeFrequency Decompositions 13. . 169 . . . .1 Perfect Reconstruction Filter Banks . . 177 . . . . . . . . . 187 . . . . . .1 History and Motivation . . . . A. . . . . . . . . . . . . . . . . . . . . . . 13. . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . .2. . . . . Bibliography . . . . . . . . . . . . . . . . .3 Symmetric Biorthogonal Wavelets . . A. . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . . . . . . . .4 Applications of Wavelets . . . . . . . . . . 167 .4 Filter Banks . . . . .1 Studying Functions on other Domains . . . . . . . . . . . . . . . . . . . . . A. . . . . . .3 The Polynomial P and Wavelet Design . . . . . . . . . . . . . . . . . . . . .1 Spatial Invariant Linear Systems . . . . . .3. . . .3 Extending the Basic Wavelet Framework . . . . . . .3 Fast Biorthogonal Wavelet Transform . . . . . . . . . 176 . . . . . . . . . . . . . . .1. . . . . . . . . . . . . . . . .3. . . . . . .1 BSpline Wavelets . . . . .2. . . . . . . 13 Directions and Guidelines 13. . . . .2 Filter Design Framework for Biorthogonal Wavelets . A. . . . . . .5 Comments and References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 . . . . . . . . . . . . . . . . . .1 Systems and Filters . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . 12. . . 165 . . . .4 Comments and References . . . . .5 Comments and References . 183 . . . . 168 . . . . . . . .CONTENTS ix 12 Biorthogonal Wavelets 12. . .1. . . . . . . . 12. . . . . . . . . . . . . . 12. . . . . . . . . . . 12. . . 13. . . . . . . . . . . . . . . . . .1. .4 Factorization of P for Biorthogonal Filters . .2 A Look Back . . . . . . .3 Biorthogonal Bases Closer to Orthogonal Bases 12. . A. . . . . . . . . . . . . A. .2. . . . .1 Biorthogonal Basis Functions . . . . . . . . . . . . . . . . . . . . .1 The Z Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . 174 . . . . . . . . . . . . . .2 Discretization of Systems . . . . Bibliography . . . . . . 12.1. . 13. . . . . . . . . . . . . . . .3 Solving Mathematical Problems . . . . . 13. . .1 Discrete Signals . 186 B. . . . . . . . . . . . . . . . . . .3. . . . . . .2 Transfer Function . . .3. . . . . . . . . . . . . . . . . . . 12.3 Upsampling and Downsampling Operators A. . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . 184 B The Z Transform 185 B. . . . . . . 185 B. . . . . . . . .2 Other Characteristics . A. 13. 182 .1. . . . . . . 169 173 . . . . . . . . . . . . . . . . . . A Systems and Filters A. . .2. . . 168 . .2 Discrete Systems . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.
. . . . . . . . . . . . . B. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B. . . 189 189 189 191 191 192 192 193 . . . . . . . . . . . . .2 Upsampling in the Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Upsampling after Downsampling . . . . . . .3 Comments and References . . . . .2 Subsampling Operations . . . . . . . . . . . . . .x CONTENTS B. . . . . . . . Bibliography . . . . . . . . . . . . . . . .2. . . . . .2. . . . . . . . . . . . . . . . . . . . . B. . . . . . .1 Downsampling in the Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . B. . . .2. . . . . . . . . . . . . .1. .3 The Variable z and Frequency . Bibliography . . . . . . . . . . . B. . . . . .
. . . . . .1 3. . . . . . . . . . . (a) Quadratic chirp signal. . . . . . . . . . Spectral distribution function. . . .4 4. . . . 4 and 8. .6 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . (a) Point sampling. . . .List of Figures 1.4 3. . . . . . .5 3. . . . . . . . . . . . . . . . . . . . . . . . . . . .2 5. . . . . . . . . . . . . .1 4. . . . . . . . . . . . Atomic decomposition using windowed Fourier transform. . . . Information cells. . . . . Representation operator. . . . . . . . . . . . . . . . . . Decompositions of the timefrequency domain using the DWFT. . . . . . . . . . . . .6 Abstraction levels for computational mathematics. . . . . . . . . . . Atoms in the timefrequency domain.7 3. . . . . . . . . . . . . . . . . . .1 2. . . . . . . . . 1994). . . . . .11 5. . . . . . . . . (a) Graph of the synthesis function. . . (b) Fourier sampling. . .2 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42). . . . . . . . . Fourier transform of a wavelet. . . . Modulating function. . . .4 3. . . . .5 5. . . . . . . . . . . . . . . . . . . . . . Scales of a function: (a) s < 1. . . . . xi . . . . . . . . . . . . (b) Reconstruction. . . . . . . . . . . . . . . . . .2 4. . . . . . . . . . .3 5. . . . . . . . . . . . . . . Transfer function of an ideal bandpass ﬁlter. . . .4 5. . . . . . . . . . . . . . . . . . . . . 2 5 17 18 19 20 26 30 31 33 34 34 34 36 44 47 48 49 50 51 52 52 53 54 54 58 60 61 63 63 64 Sine function with frequencies 2. . . . . . . . . . . . . . Transfer function of an ideal highpass ﬁlter. . . . . . . . . . . . . . . . . . . . . . . . (b) s = 1. .1 1. . (c) s > 1. . . . . . . . . . . . . . . . . (b) Reconstruction. . . .3 2. . . .10 4. . . . . .3 3. . . . . The wavelet transform(Kaiser. .2 2. . . . . . . . . . . . Uniform lattice in timefrequency domain. . . . . . Sampling and the Fourier spectrum. Graph of a wavelet. . . . . . . . . . . Hamming window. . . . Transfer function of an ideal lowpass ﬁlter. . . . . . . . . . . . . 1994).8 4. . .2 3. . . A signal and its Fourier transform. . . . . . . Fourier transform of two impulses δ(t − 0. Galerkin representation of a function f . . . . . . . . . . . . . . . . . . . . .1 5. . . . . . . . . . . (b) Ideal atomic decomposition of the signal. . . Timefrequency decomposition of chirp signal using DWFT. . . . . . . . . . . . . Signal and its Fourier transform. . . . . . . .8 4. . . . . . Transfer function of an ideal bandstop ﬁlter. . . . . . Signal f to be analyzed(Kaiser. . . . . . . . . . . .5 4. . . . . . . . . . .9 4. . Graph of the sombrero wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 4. . . . . . . . . . . .05) e δ(t − 0. . . . . . . . . . . . . . . . . .6 4. . . . . . . . . . . . . . . . .7 4. . . (a) Graph of the synthesis function. . . . . . . . . .
85 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 8. . . . . . . . . . . . . . . . . . . . . . . . 114 . . . . . . . . . . . . . .4 6. . .8 (a) Timescale lattice. . . . . Ideal bandpass ﬁlter. . Spectrum of the scaling function. 84 . . . .7 7. . . . . 112 . . . . . . . . . . . . (b) Periodization. . . . . . . 66 Timefrequency decomposition using wavelets. . Frequency bands between Vj and VJo −j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Magnitude of the frequency response of φ(ω) for the Haar function .3 11. .7 5. . . . . . . . . 125 ˆ 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Boundary regions for convolution between ﬁnite sequences. . . . . . .1 Halfband Filter . . . . . . . . . . . . . . . . . . . . . . . .8 5. . . . . . . . . . . . . . . . . . . . Shannon wavelet. . . .6 6.4 8. . . . . . . . . . . . . . . 71 71 74 74 75 76 80 Wavelet decomposition of a function f . . . . . . . . .3 9. . 1992). . . . . . . . . Shen. . . . . . . . . . . . . . . . . . . 96 TwoChannel Analysis and Synthesis Filter Bank . Scaling of f by an scale factor of 2. . . . . . . . . . . . . . . . . . . .4 11.6 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .(b) Timefrequency lattice. . . . . 95 TwoChannel Synthesis Filter Bank . . . . . . . . . . . . . .2 9.1 7. . . . .2 8. . . . . . . . . . . . . . . . . . . . .3 6. . . . . . . . . . . . . . . Options for boundary computation (a) Extending with zeros. . . . . . . . .2 6. . . . .3 7. . . . . . . Piecewise linear scaling function. . . . . . . . . . . . . . . . . . . . .5 11. . . . . Band decomposition using Shannon multiresolution representation. . . . . . . . . . . . . . . . . . . . . . (a)Scaling function of the piecewise linear multiresolution. . . . . . .5 6. . . . .9 6. . . .4 9. . . . . . . . Coiﬄets of order 4 . . . . . . . . . . . . . . . . . . . . 96 Double scale relation of the Haar scaling function. . . . . . . . . 122 10. . . . . . . . Image decomposition on the spaces Vj e Wj . . . . . . . . . . . . . 111 . . . . . Least Assymmetric Scaling Funtion and Wavelet. . . . . . . . . . . . . . . . . . Scale approximations of a function(Daubechies. . Haar wavelet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Haar ﬁlter m0 . . . . . . . . . . . . . . . . . . . . . . . . . Limit curves for the roots of Bn (y) and Qn (z)2 (from (Strang & Roots of P (z) with n = 2. . . . . . . . . . . . . . . . . . . . .1 11. . . . . . . . . . . . .7 P . . . . 92 Diagram of a TwoChannel Analysis Filter Bank. . . . . . . . . . . . . . . . . . . Frequency band between Vj and Vj−1 . . . .xii LIST OF FIGURES 5. . . . . .1 Biorthogonal Filter Bank . . . 138 139 140 141 141 143 144 12. . . . .5 9. Daubechies (D2) Scaling Funtion and Wavelet . Impulse response of the ideal reconstruction ﬁlter. . . . .6 9. . . . . . 91 . . . . (c) Reﬂection. . . . . . (b) Wavelet of the piecewise linear multiresolution. . . . . . .7 9. . 115 115 116 116 116 10. . . . . . 1996)) . . . .1 6. . . . . . 66 Haar wavelet. . . . . . . . . . . . . . . . Wavelet reconstruction process of a function f . . . 126 11. . 152 . . . . .1 9. . . .3 9. . . . . . . . . . . . . . . . . 68 Lowpass ﬁlter. . . . . . . . . . . . . . . . with n = 4 . . . . . . . . .2 7. . . . . . . . . . Fourier Transform of (D2) Scaling Funtion and Wavelet . . . . . . . . . .2 11. . . . . . . . .
. . . . . . . . . 183 B. . . . . . . . . .1 Basic System. . . . . . . 12. .7 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . 1992)). . . . . . . . . . . . . . . . . .LIST OF FIGURES xiii 12.6 Biorthogonal quadratic Bspline scaling functions (a) and wavelets (b). .2 Delay Operation. . . . .2 Biorthogonal box scaling functions (a) and wavelets (b). . . . . . . . . 12.(from (Daubechies. 189 B. . . . . . . . . . . . . . . 175 A. . . . . . . . . . . . . . . . . . . . . . . . . . 1992)) . . .(from (Daubechies. . . .11Biorthogonal Wavelets with Similar Support. . . .2 Downsampling Operator. . . . . . . . .4 Biorthogonal Hat scaling functions (a) and wavelets (b). . 173 A. . . . . . . . . . . . . . . . . . . . . . . . . 12. . . . 190 B.9 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . .3 Upsampling Operator. . . . . . . . 12. . 156 156 157 157 158 158 159 159 161 162 164 A. . . . . 12. . . . . . . . . . . . . .10Biorthogonal Bspline Wavelets of order 2 (from (Daubechies. . . . 12. . 12. . . . . . . 1992)) . . . . . . 191 . . . . 12.3 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Example of a Filter Bank . . . . . . . . . . . . . . . . . . . . . 12. . . . . . . . . . .8 D2 Daubechies scaling functions (a) and wavelets (b). . . . . . . . . . . . . . .5 Frequency response of lowpass (a) and highpass (b) ﬁlters. . . . . . . . . . . 12. . . . . . .12Biorthogonal Wavelets Close to Orthogonal. . . . . . .1 Modulus of the frequency response of a ﬁlter. . .
xiv LIST OF FIGURES .
Chapter 1
Introduction
In this chapter we give a general overview of the area of Computational Mathematics and computer graphics, introducing the concepts which motivate the study of wavelets.
1.1
Computational Mathematics
Mathematical modeling studies phenomena of the physical universe using mathematical concepts. These concepts allow us to abstract from the physical problems and use the mathematical tools to obtain a better understanding of the diﬀerent phenomena of our universe. The advance of computing technology (both hardware and software) has enabled the use of mathematical modeling to make simulations on the computer (synthetic simulations). These simulations on the computer allow for a great ﬂexibility: We can advance in time, accelerate processes, introduce local and global external factors, and change the diﬀerent parameters on the mathematical models used. These conditions of simulation in general are very diﬃcult to be attained in real simulations. Computational Mathematics is the combination of mathematical modeling with simulations on the computer. Computational mathematics represents a return to the birth of mathematics, where mathematical problems were directly related to the solutions of practical problems of our everyday life. In this context, computer graphics provides diﬀerent techniques to visualize the results from the simulation. This visualization enables a direct interaction of the users with the diﬀerent parameters of the simulation and also enables them to collect valuable qualitative information about the simulation process.
1.1.1
Abstraction Levels
A useful way to organize the ideas discussed in the previous section, which will be used in the book, consists in structuring the problem of computational mathematics using abstraction levels. These abstraction levels encapsulate the diﬀerent phases of the simulation processing giving us a better understanding of the whole process. We will use the paradigms of the four universes. This paradigm uses four levels: Physical universe, Mathematical Universe, Representation Universe and Implementation Universe (See Figure 1.1). 1
2
CHAPTER 1. INTRODUCTION
Figure 1.1: Abstraction levels for computational mathematics. Level 1: Physical Universe. In this level the problems are originally posed and correctly formulated. The elements of these abstraction levels consist of the diﬀerent phenomena from the physical universe. Level 2: Mathematical Universe. This level contains tools from diﬀerent areas of mathematics (Analysis, Geometry, Diﬀerential Equations, Algebra etc.). The process of mathematical modeling consists in the association of elements from the physical universe (Level 1) to elements of the mathematical universe. Level 3: Representation Universe. To perform synthetic simulations the mathematical models must be discretized and implemented on the computer. The representation universe consists of the diﬀerent elements of discrete mathematics and the mathematical methods of discretization. As part of this universe, we have virtual machines where we can describe algorithms. The discretization of a problem consists in relating the mathematical model associated to the problem in the mathematical universe (Level 2) to a discretized model in the representation universe. Level 4: Implementation Universe. This abstraction level is considered so as to allow for a clear separation between the discretization problem in Level 3 and the problems inherent to implementation. The elements of this universe consist in data structures and programming languages with a well deﬁned syntax and semantics. The implementation of a problem consists in associating discrete structures from the representation universe to elements of the implementation universe.
1.1. COMPUTATIONAL MATHEMATICS
3
This models of abstraction levels will be used along the book. These abstraction levels split the problems into four major subproblems and favors an encapsulation of each of the subproblems: At each level the problem is studied with a clear separation from its intrinsic issues with issues related to the other abstraction levels. We should point out that this abstraction paradigm constitutes in a characterization of computational mathematics. Below will give some examples. Example 1 (Measurements). Consider the problem of measuring objects in the physical universe. For a given object we must associate to it a number which represents its length, area or volume. In order to achieve this we must introduce a standard unit of measure which is compared with the objects to provide a measurement. From the point of view of the mathematical universe, to each measure we associate a real number. Rational numbers correspond to commensurable objects, and irrational numbers correspond to incommensurable ones. To represent the measurements, we must look for a discretization of the real numbers. A widely used choice is given by the ﬂoating point representation. Note that in this representation the set of real numbers is discretized by a ﬁnite set of rational numbers. In particular, this implies that in the representation universe we do not have the concept of incommensurable measurements. An implementation o the real numbers using ﬂoating point arithmetic can be done using a standard IEEE speciﬁcation. A good reference for these topics is (Higham, 1996). The above example, although simple constitutes the fundamental problem in the study of computational mathematics. In particular, we should remark the loss of information when we pass from the mathematical to the representation universe: incommensurable measures disappear. In general the step from the mathematical to the representation universe forces a loss of information, and this is one of the most delicate problems we have to face in computational mathematics. Example 2 (Projectiles). Consider now the well known problem of studying the motion of a small rigid body which is thrown into the space with a given initial velocity. The problem consists in studying the trajectory of the body. In the mathematical universe we will make a simpliﬁcation to model the problem. The rigid body will be considered to be a particle, that is a point of mass. This abstraction allow us to neglect the air resistance. Simple calculations well known from elementary physical courses, allow us to solve the problem of determining the trajectory: a parabola f (t) = at2 + bt + c. To visualize the trajectory in the computer we must obtain a discretization of the parabola. This discretization can be obtained by taking a uniform partition 0 = t0 < t1 < · · · < tn = t of the time interval [0, t]. Using this partition the parabola is represented by a ﬁnite set of points of the plane: (t0 , f (t0 )), (t1 , f (t1 )), . . . , (tn , f (tn )). These points constitute samples of the function the deﬁnes the parabola.
4
CHAPTER 1. INTRODUCTION
Finally, data structures to implement the samples are well known and can be easily implemented. Note that in order to plot the parabola we must “reconstruct” it from the samples. Interpolation methods must be used in the reconstruction process. Again, the representation process by sampling looses information about the parabola. Depending on the interpolation method used we will not be able to reconstruct the parabola exactly, but only an approximation of it. An important remark should be done. The solution to the problem in the above example was initially attained in the mathematical universe it was an analytical solution providing the equation of the trajectory. The discretization of the analytical solution was done in order to visualize it. Another option would be discretizing the problem before solving it by using discrete mathematical models to formulate the problem (Greenspan, 1981). Which is the best strategy? Discretize the problem to obtain a solution or compute an analytical solution and discretize it. The answer to this problem is not simple and involves diﬀerent issues. Initially we should remark that in several situations we do not have an analytic solution to the problem in the mathematical universe. When this happens discretization a priori to determine a solution is of fundamental importance. This is the case, for instance, in several problems which are modeled using diﬀerential equations: The discretization allows us the use of numerical methods to solve an equation where an analytical solution is diﬃcult or even impossible to be achieved. When the problem has an analytical solution in the mathematical universe we have the option to compute it before discretization. Nevertheless we should point out that the process of discretization should not be decorrelated from the formulation of the problem in the physical universe. In example 2 for instance the parameter of the parabola represents the time, and we know that the horizontal motion is uniform. Therefore, a uniform discretization of time is the most adequate to playback the animation of the particle motion.
1.2
Relation Between the Abstraction Levels
We should not forget the importance of establishing for each problem the relation between the four universes in our abstraction paradigm described above. In its essence these relations enable us to answer the question: How close a computational simulation is from reality? The answer to this problem depends on the relation between the diﬀerent abstraction levels: • Relation between the implementation of the problem, and its discretization; • Relation between the discretization and the mathematical formulation of the problem; • Relation between the mathematical model of the problem and the original problem in the physical universe. The relationship between the mathematical and the representation universe is of most importance. We have seen that representation is a loss process, and a very important issue consists in recovering the continuous model from the discrete representation. This step is
In fact this is one topic to be covered in this book. such as color television: The TV set reconstructs colors which are only perceptually equal to the original colors captured from the video camera. reconstruction in general is not possible to be attained exactly. we can devise a simple solution going back to the physical universe. In this case another question arises: How many samples should we take? Understanding this problem in its generality and devise adequate and robust solutions is a diﬃcult task. called reconstruction.2. How does the human eye processes color? From the theory of YoungHelmholtz the eye discretizes color using three samples of the spectral distribution function: One sample of low frequency (Red). Exact reconstruction of the original spectral color from the three RGB samples is in general an impossible task. Color is produced by an electromagnetic radiation in the visible band of the spectrum. Example 3 (Representation of Color). What is the most natural technique? A simple solution consists in using point sampling as we did in the discretization of the parabola in example 2.1. a color is determined by its spectral distribution function. the process if discretizing a color reduces to that of discretizing a function. RELATION BETWEEN THE ABSTRACTION LEVELS 5 Figure 1. This problem is called perceptual color reconstruction. Therefore from the point of view of the mathematical universe. we do not need to guarantee an exact reconstruction: We should be able to provide a reconstruction process in which the reconstructed color is perceptually equal to the color deﬁned by the original spectral distribution function. in the way we posed the color sampling problem. Since the representation is a loss process. This means that the representation of the color space (which is an inﬁnite dimensional function space) is given by the Euclidean space R3 . It is far away from being a good mathematical representation. This function associates to each wavelength λ the value of the associated energy (see Figure 1. In the case of color. Nevertheless. Therefore. Perceptual reconstruction of color is the technology behind many color reconstruction devices nowadays. one sample of medium frequency (Green) and another sample of high frequency (Blue) This means that from the perceptual point of view it is enough to use three samples of the spectral distribution function in order represent color.2: Spectral distribution function. .2). The example below will make this point clear. Note that this representation is completely justiﬁed from the point of view of color perception on the physical universe.
In fact they constitute the basic tools of mathematical modeling: The relation between magnitudes of the physical world are.6 CHAPTER 1. f (tn )) ∈ Rn . . is it possible to reconstruct a function f from its representation vector? Note that in the case of point sampling a reconstruction technique consists of interpolating the points (ti . INTRODUCTION 1. The ﬁrst step towards doing computation with the function consists in reconstructing it from its representation. . In the onedimensional case a real function of onevariable f : I ⊂ R → R is discretized by taking a partition t1 < t2 < · · · < tn of the domain interval I. That is. . In general. 1. When we talk about a continuous function it means that we are able to evaluate the function at any point of its domain. described by functions. analyzing and visualizing graphical objects. . along with a reconstruction technique that enables us to evaluate the function at any point. 1. Inside the computer any function is in fact necessarily discretized. this interpolation process gives only an approximation of the original function. A ﬁnal word about function reconstruction is in order. Therefore. That is. f (t2 . . we have a representation of the function on the computer. . In this way. The representation is given by the vector fn = (f (t1 ). . Notice that the two examples of particle motion and color reconstruction were reduced to the problem of representation and reconstruction of functions. in computer graphics we are interested in creating. in most cases. the space of real functions deﬁned on the interval is represented by the Euclidean space Rn .4 What is the Relation with Graphics? A broad view of computer graphics would lead us to say that it consists of the area that manipulates graphical objects. i = 1.3 Functions and Computational Mathematics Functions play a distinguished role in computational mathematics. 1. A mathematical formulation of the above question would be: Is point sampling an exact representation? In other words. ).3. n. Is this a good representation? This question is directly related with the problem of information loss in the representation of a function. processing. f (ti )).2 Speciﬁcation of Functions In general when a function is speciﬁed on the computer the speciﬁcation technique results in a representation of the function. . a highly relevant problem in computational mathematics turns out to be the problem of function representation and reconstruction.1 Representation and Reconstruction of Functions The simplest method to represent a function is to use point sampling.3. Therefore representation and reconstruction techniques are related with user interface issues in a computational system. .
Notice that visualization of graphical objects. Therefore. WHAT IS THE RELATION WITH GRAPHICS? 7 In order to fully understand this description we must give a precise meaning to the concept of a graphical object.1 Description of Graphical Objects By description of a graphical object we mean the deﬁnition of its shape and attributes on the mathematical universe. . j) = f (i∆x. reconstruction and implementation of graphical objects. and the attributes describes the diﬀerent properties associated to it. j∆y) which are denoted simply by (i. is in fact a reconstruction problem. and also curves. and associated to each point of this support we have a color information. 1996): A graphical object consists of a shape and its attributes. A simple. This deﬁnition encompasses diﬀerent signals such as video. and eﬀective deﬁnition was given in (Gomes et al. such as color.) We can easily obtain a representation of an image using point sampling. i ∈ Z. Using the RGB color representation of example 3 we have C = R3 . and we deﬁne a twodimensional lattice of the plane by taking the cartesian product ∆X × ∆Y . ∆y > 0} of the x and y axis of the Euclidean plane. material. The object is denoted by O = (U. where C is a representation of the color space. j ∈ Z. • Parametric description. Analyzing a photography we observe a geometric support (the paper where it is printed). j). There are basically three diﬀerent methods to describe the shape of a graphical object. The image f is represented by the matrix f (i. The shape deﬁnes the geometry and topology of the object. surfaces and volumetric models.4. Example 4 (Image and Digital Image). • Algorithmic description. The dimension of the set U is the dimension of the graphical object. an important part of the computer graphics applications. and Rn is the embedding space. This lattice is characterized by the vertices (i∆x. density etc.1. The physical model of an image is well represented by a photography. from the mathematical point of view we can model an image as a function f : U ⊂ R2 → C. f ). 1. We take partitions ∆X = {i∆x . velocity. ∆x > 0} and ∆Y = {j∆y . representation. • Implicit description. and the attributes are encapsulated into a vector valued function f : U ⊂ Rn → Rp . Mathematically. j∆y). . the shape is characterized as a subset U ⊂ Rn . According the abstraction levels we have described the main problems of computer graphics are: description.4. audio and image.
we must devise a collection {gλ } of functions λ ∈ Ω. (1.8 CHAPTER 1. U = {x ∈ Rm . In the implicit description the object shape U ⊂ Rm is described as the zero set of a function F : Rm → Rk . . Examples of functions deﬁned algorithmically are found in the description of fractals. f → (fλ )λ∈Ω . 1991)). F (x) = 0}. in such a way that every function f can be expressed by a sum f= λ cλ gλ . and in particular in computer graphics. • Multiresolution representation. The attributes of a graphical object are described by a function.1 Function Representation Using Wavelets An eﬃcient process to represent a function consists of decomposing it into simpler functions.1) In this case. are related with two facts: • Representation and reconstruction of functions. the coeﬃcient sequence (cλ ) constitute the representation of the function f . in general. More precisely. The diﬀerent uses of wavelets in computational mathematics.5. Therefore. The function f is reconstructed from its representation coeﬃcients using equation (1. Also as discussed above. the graphical object has dimension m. where we must use an algorithm to solve the equation. If U is an mdimensional subset of Rm . 1. representation and reconstruction of functions. In the algorithmic description the shape is described by a function (either implicitly or parametrically) which is deﬁned using an algorithm in some virtual machine (see (Blum. Wavelets constitute a powerful tool to construct families gλ . In other words.5 Where do Wavelets Fit? In order to understand the role of the wavelets in the scenario of computational mathematics. INTRODUCTION In the parametric description the object shape is deﬁned as the image of a function f : U ⊂ Rm → Rn . representation and reconstruction.1). This set is denoted by F −1 (0). The study of graphical objects reduces to the problem of function description. and it is called the inverse image of 0 by the function f . and compute the representation coeﬃcients cλ . Other common example in computational mathematics are the objects deﬁned using diﬀerential equations. An important remark is in order. 1. the shape of the object is described by functions. both the shape and attributes of a graphical object are described by functions. where the parameter space Ω is discrete. even without understanding what a wavelet is. we must remember that our major concern is the description.
1980). we will cover the role of wavelets on function representation and reconstruction. Diﬀerent information about the house can be attained by using ﬁner scales. doors or windows. The notes have an introductory ﬂavor. Inner products and norms on these spaces will be used. Also. In spite of the huge amount of material about wavelets on the literature. To cite an example.7 Comments and References The abstraction levels of the four universes described in this section is based on (Requicha. emphasizing the intuition over the mathematical rigor.2 Multiresolution Representation Our perception of the physical world occurs in diﬀerent scales. We are more interested into the semantics of the diﬀerent facets of wavelet theory. 1995). These notes originated from our need to teach wavelets to students originated from both mathematical and computer science courses. 1. ABOUT THESE NOTES 9 1. we will refer to linear operators and other operations on function spaces relaxing about several hyphotesis that would be necessary to conﬁrm the validation of some assertions. From the point of view of the abstraction paradigm of the four universes. We will certainly use material from these sources. in order to make the text easier to read we decided to relax with the mathematical syntax. . these notes covers the subject with a certain degree of originality on what concerns the organization of the topics. This multiple scale approach is the desired one when we study functions. The identiﬁcation is done using a macroscopic scale of the house. Many good books on wavelets have been published. We will also discuss some implementation issues along the way.5. where the measurements of the details has the same order of magnitude of the scale used. In order to recognize a house it is not necessary to have details about its roof. it is very common to refer to some “space of functions” without making the concept precise. Therefore. A good representation of a function should enable us to devise representations in diﬀerent scales.1. They emphasize diﬀerent aspects of the theory and applications. We have tried to write a logﬁle of our journey which contains somehow the details without going deep into the mathematical rigor. The starting point of our journey is the kingdom of Fourier analysis. 1. A generalization of them along the lines presented on this section was published in (Gomes & Velho. We have tried to use mathematics as a language which has the adequate semantics to describe the wavescape along the trip.6 About these Notes These notes describe a guided journey to the wonderland of the wavelets.6. This is achieved by using wavelets. and we will cite them along our journey. There are several books that cover these syntax issues in detail on the literature.
Nicholas J. New York. International Series in Nonlinear Mathematics. Accuracy and Stability of Numerical Algorithms. 1996. Lucia. 12(December). & Velho. A. G. Lectures on a Theory of Computation and Complexity over the Reals. Graphical Objects. Darsa. 1980. Donald. The Visual Computer. Jonas. 18 Coloquio Brasileiro de Matematica – IMPA. Higham. methods and Applications. 1996. Bruno. & Velho. ACM Computing Surveys. Luiz. 1991.10 BIBLIOGRAPHY Bibliography Blum. 11. Computer Oriented Mathematical Physics. Representations for Rigid Solids: Theory Methods. and Systems. 437–464. The Visual Computer. Greenspan. Gomes. SIAM Books. 227–239. L. Luiz. 1981. . A. Gomes. Abstraction Paradigms for Computer Graphics. Philadelphia. Pergamon Press. Jonas. Costa. Requicha. 12. 1995. 269–282.
We admit that S has a norm which allow us to compute distance between sequences of the space. its representation R(f ) is a sequence R(f ) = (fj )j∈Z ∈ S. numerical methods that solve diﬀerential equations (RungeKutta. ﬁnite diﬀerences. In general it is natural to demand that R preserves norms.) compute in fact a representation of the solution.1. 2.1 Representing Functions Functions must be discretized so as to implement them on the computer. That is R(f ) = f . 1 By numbers we mean a real or complex number 11 . R is called the representation operator. In this chapter we will give more details about these two problems in order to motivate the study of wavelets. These two operations enable us to move functions between the mathematical and the representation universes when necessary. 2. Also. when a function is speciﬁed in the computer the input is a representation of the function. As an example. etc. ﬁnite elements. For a given function f ∈ F. the reconstruction methods are directly related with the theory of function interpolation and approximation.Chapter 2 Function Representation and Reconstruction In the previous chapter we concluded that one of the major problems in computational mathematics is related to function representation and reconstruction. A representation of a space of functions F is an operator F : F → S into some space of sequences. As we will see in this chapter. Associated with each representation technique we must have a reconstruction method.1 The Representation Operator We denote by S a space of sequences (cj )j∈Z of numbers1 .
of the square summable sequences. and the representation space S is the space 2 2 R f (t)2 dt < ∞}. we have a linear representation. We will review some of these methods in this chapter. we have an exact representation. We deﬁne the representation operator by R(f ) = (αj )j∈Z . A set B = {ej . When R is linear and continuous. n n→∞ lim f − j=−n αj ej  = 0. +∞ = (xi )i∈Z . FUNCTION REPRESENTATION AND RECONSTRUCTION or that R satisﬁes some stability condition. of square integrable functions (ﬁnite energy). we can reconstruct f from its representation sequence: f = R−1 ((fi )i∈Z ).1) The above equality means the convergence of partial sums of the series in the norm of the space F. A method to compute the inverse operator gives us the reconstruction equation. (2. We should remark that in general invertibility is a very strong requirement for a representation operator. In case the representation is not exact we should look for other techniques to compute approximate reconstruction of the original function. j ∈ Z} is a basis of a function space F if the vectors ej are linearly independent. The most important examples of representations occurs when the space of functions F is a subspace of the space L2 (R). . In fact weaker conditions such as invertibility on the left suﬃces to obtain exact representations. xi ∈ C.12 CHAPTER 2. such as R(f ) ≤ Cf . Equation (2. In this case. 2. L2 (R) = {f : R → R . When the representation operator is invertible. and for each f ∈ F there exists a sequence (αj )j∈Z of numbers such that f= j∈Z αj ej . and i=−∞ xi 2 < ∞ .2 Basis Representation A natural technique to obtain a representation of a space of functions consists in constructing basis of the space. . There are several representation/reconstruction methods on the literature. also called ideal representation.1) reconstructs the function f from the representation sequence (αj ).
Therefore. REPRESENTATION BY FRAMES 13 We must impose additional hypothesis on the representation basis in order to guarantee unicity of the representation. called Hilbert spaces. This representation preserves the norm. ϕk ) is uniquely determined by f . (2. n ∈ Z} on a separable Hilbert space H is a complete orthonormal set if the conditions below are satisﬁed: 1. it is important to obtain collections of functions {ϕn . Orthogonality: ϕm .2) This expression is called Plancherel equation. That is R(f )2 = k∈Z f. 2. Completeness: For all f ∈ H. 2. An orthonormal basis {ϕj } deﬁne a representation operator R : H → 2 . Normalization: ϕn  = 1 for each n ∈ Z. On the other hand orthonormal representations are too much restrictive and rigid.2. R(f ) = (fj ) = ( f. A particular case of great importance (in fact the only one we will be using in these notes) occurs when the space of functions has an inner product and it is complete in the norm induced by this inner product. which is invertible.3. and its construction is in general a very diﬃcult task. 2.1 Complete Orthonormal Representation A collection of functions {ϕn . Complete orthonormal sets are also called orthonormal basis of the Hilbert spaces. ϕj ). The reconstruction of the original signal is given by +∞ f= k=−∞ f. N > 0. ϕk 2 = f 2 . The third condition says that linear combinations of functions from the set can be used to approximate arbitrary functions from the space. the representation is exact. and any ε > 0. ϕk ϕk  < ε.3 Representation by Frames The basic idea of representing functions on a basis. It is easy to see that the orthogonality condition implies that the elements ϕn are linearly independent. consists in decomposing it using a countable set of simpler functions. This implies in particular that the representation sequence ( f. ϕn = 0 if n = m. N  f − k=−N f. The existence of a complete orthonormal set. possess special basis as we will describe below. 3. n ∈ Z} which do not . ϕk ϕk . Therefore. These spaces.2.
n ∈ Z} is a tight frame with A = 1 e ϕn  = 1. ϕj ϕj . as we will show below. ϕj . Nevertheless. g . If a frame is tight. From equation (2. ϕj ϕj . it motivates us to deﬁne a representation operator R analogous to what we did for orthonormal basis: R(f ) = (fj )j∈Z .3) The constants A and B are called frame bounds. When A = B we say that the frame is tight.3) that  f. there exists tight frames which are not orthonormal basis. . where fj = f. If B = {ϕn . In fact. but can be used to deﬁne a representation operator.2) it follows that every orthonormal set is a tight frame with A = B = 1. Ru = j∈Z uj f. One such collection is constituted by the frames. g = j∈Z f. then B is an orthonormal basis. f = A−1 j∈Z f. We should remark that this operator in general is not invertible. we have Af 2 ≤ ∞ n=−∞  f. Therefore R∗ u = j∈Z u j ϕj . such that for all f ∈ H. ϕj 2 = Af 2 . The adjoint operator R∗ de R is easy to obtain: R∗ u.3) it follows that the operator R is bounded: Rf 2 ≤ Bf 2 . j∈Z Using the polarization identity. ∀n. n ∈ Z} is a frame if there exists constants A and B satisfying 0 < A ≤ B < +∞. (2. ϕj = j∈Z u j ϕj . Nevertheless it is possible to reconstruct the signal from its representation R(f ). we obtain A f. From the Plancherel formula (2. that is.14 CHAPTER 2. f . f = u. it follows from (2. The above expression (although deduced in the weak sense) shows how we can obtain approximations of a function f using frames. FUNCTION REPRESENTATION AND RECONSTRUCTION constitute necessarily an orthonormal set or are not linearly independent. ϕn 2 ≤ Bf 2 . A collection of functions {ϕn . The following statement is true: Theorem 1.
By the deﬁnition of R we have  f. 2. n ∈ Z} is a frame. en 2 ≤ Bf 2 . for any f ∈ H we have Af 2 ≤ n  f. k . and the functions ϕj are linearly independent. ϕj ϕj = ˜ j∈Z f. we deﬁne its representation f. The frame {ϕj } is called reciprocal frame of the frame ˜ ˜ {(ϕj )j∈Z }. Therefore if {en } is a Riesz basis. ˜ This family constitutes a frame with bounds given by B −1 f 2 ≤ j∈Z  f. RIESZ BASIS REPRESENTATION 15 (It can be shown the convergence is also true on norm). f . ˜ The following identity can also be proved for all functions f ∈ F: f= j∈Z f.2. ˜ ˜ Therefore we can associate to the frame {ϕj } a representation operator R : F → 2 (Z). Note that the second equation in (2. ϕj 2 ≤ A−1 f 2 . ϕj 2 = F f 2 = F ∗ F f. ˜ ϕj = (F ∗ F )−1 ϕj . ϕj ϕj . It is easy to see that the reciprocal frame of {ϕj } is the original frame {(ϕj )}. ˜ (2. j∈Z On the other hand. we have F ∗ u ≤ B 1/2 u. ϕj )l∈Z .4) gives a method to describe a function f as a superposition of the elements of the original frame. In particular. since F ∗  = F . we have a Riesz basis. We will state the result without proof: Applying the operator (F ∗ F )−1 to the elements ϕj of the frame. deﬁned ˜ ˜ in the usual way (Rf )j = f. The above results allow us to obtain an expression to reconstruct the function f from its representation R(f ). For each function f ∈ L2 (R). ek ek .4 Riesz Basis Representation If a collection B = {ϕn . We conclude that AI ≤ F ∗ F ≤ BI.4. it follows that the operator F ∗ F : F → F is invertible. we obtain a family of functions ϕj . ϕj . Because of this equation the reciprocal frame is also called dual frame.4) The ﬁrst equality gives us a reconstruction equation of the function f from its representation sequence R(f ) = ( f.
.0 . . than a representation of f is given by R(f ) = ProjV (f ) = k f. An example occurs when each subspace Vj has ﬁnite dimension. This representation is not exact. any Riesz basis can be obtained using this method. A proof of this fact can be found in (Daubechies. we can construct and orthonormal basis {˜n } of H. j = 0. ϕk ϕk . if N → ∞. . when N → ∞. we see that Riesz bases constitute the basis of a Hilbert space that are closest to complete orthonormal sets.1). then L maps any orthonormal basis of a Hilbert space onto a Riesz basis. as described in the next section. unless we have V = F. The representation fN is therefore computed in such a way to minimize the norm f − fN .k (t). φN. Also. 1.1). the representation by projection consists in taking a representation of a function f ∈ F as its unique orthogonal projection onto V (see Figure 2. That is. φN. . 1992). . If L is a continuous linear operator with a continuous inverse. FUNCTION REPRESENTATION AND RECONSTRUCTION on the Riesz basis {ek }. the functions φN. The best reconstruction we can have consists in ﬁnding the unique vector of V which is closest to f . . .6 Galerkin Representation The Galerkin method computes a representation R(f ) = (fj ).16 CHAPTER 2. 2. This vector can be computed using optimization techniques. An important particular case occurs when we have a family of subspaces {Vj } such that Vj = F. and the representation fN is the orthogonal projection of f onto V (see Figure 2. f − fN  → 0. N − 1 in such a way that there exists a reconstruction equation N −1 fN (t) = k=0 aN (k)φN. .N −1 generate a subspace V of the space of functions. Moreover. it can be proved that from a Riesz basis {en } of a Hilbert space H.1 .5 Representation by Projection For a given closed subspace V of a Hilbert Space F. which approximates the function f . in norm. From the above remarks. 2. and the family of representations ProjVj (f ) converges to the function f . e page 139. Geometrically. If {ϕk } is an orthonormal basis of V . .
. or representation functions. In this case. .2. The functions φN. If φN.1: Galerkin representation of a function f The coeﬃcients aN (k) are computed using a dual set ϕN. The Galerkin representation is also known in the literature as representation by ﬁnite projection. .7 Reconstruction. We can choose several diﬀerent reconstruction functions from the above representation. 2. φN.k .j . analysis functions. It is important to remark that the reconstruction functions are not uniquely determined. then the synthesis functions may be used as analysis functions. Also. · · · . which is very suitable for computer implementations.k : φN. n constitute the sampling functions f (tk ) = f.k . . We take a partition a = t0 < t1 . POINT SAMPLING AND INTERPOLATION 17 Figure 2. . Point Sampling and Interpolation The well known example of a Galerkin representation consists in taking a ﬁnite point sampling of a function f : [a. ϕN. k = 0. aN (k) = f. The functions ϕN. b] → R. ϕN. This method is used in diﬀerent representations of functions by piecewise polynomial functions in the theory of approximation. δ(t − tk ) = +∞ −∞ f (u)δ(t − tk )du.7. .j = δi. Galerkin representation is very important in devising numerical methods because they use ﬁnite dimensional representation spaces. . the Dirac “functions” δ(t − tk ). . f (tn )) ∈ Rn .k of the set φN.k . < tn = b and deﬁne R(f ) = (f (t0 ). RECONSTRUCTION. that is.k are called synthesis functions or reconstruction functions. 1. changing these functions imply in a change of the reconstructed function.k are called sampling functions. . We will study some classical reconstruction basis below. aN (k) = f.k is an orthonormal set.
tk+1 ] tk+1 −tk (2.2(b)).k = χ[tk . 2.2 Piecewise Linear Reconstruction In this case the synthesis functions φN. FUNCTION REPRESENTATION AND RECONSTRUCTION 2. Higher Order Reconstruction We could continue with these reconstruction methods using polynomial functions of higher degree deﬁned on each interval of the sampling partition.1 Piecewise Constant Reconstruction In this reconstruction each synthesis function is given by the characteristic functions of the partition intervals. tk ] t −t k+1 if x ∈ [tk . Geometrically. the graph of the original function.2(a).k = tk −tk−1 if x ∈ [tk−1 . the reconstructed function is an approximation of the function f by a continuous function which is linear in each interval of the partition used in the sampling process.6) The graph of this function is shown in Figure 2. in dotted lines. Geometrically. The discontinuities in the reconstructed function introduce high frequencies in the reconstructed signal (see Chapter 7 of (Gomes & Velho. tk+1 ] (2. φN. Figure 2.5) The graph of this function is shown in Figure 2. the reconstructed function is an approximation of the function f by a function which is constant in each interval of the partition (see Figure 2.tk+1 ] = 0 if x < tk or t > tk+1 1 if x ∈ [tk .3(b) shows. This method reconstructs the functions approximating it by a discontinuous functions. In the literature of signal processing the synthesis function used here is called box function. Figure 2.18 CHAPTER 2. and the graph of the reconstructed function using a continuous line. The diﬀerentiability class of these . (b) Reconstruction.7.k are given by 0 if x < tk−1 or t > tk+1 t−tk−1 φN. 1997)).2: (a) Graph of the synthesis function.7.3(a).
and so forth. Is it possible to obtain exact reconstruction using the Galerkin representation by point sampling? Note that exact reconstruction in this representation has an interesting geometric meaning: Find an interpolation method that is able to reconstruct the function f exactly from the set of its samples f (ti ). as illustrated in Figure 2. . and Pj is the orthogonal projection onto Vj . (b) Reconstruction. 2.7) is trivially satisﬁed if Vj is a closed subspace of a Hilbert space. The proximity condition of equation (2. but ﬁner scales are necessary in order to observe details about the windows.8 Multiresolution Representation We perceive the world through a multiscale mechanism. That is. We should remark that in the Galerkin representation using point sampling the reconstruction methods consist in interpolating and approximating the original function from its samples. we are interested in obtaining a family of representations that could represent the function at distinct scales. then we use ﬁner scales in order to discover its distinct properties in detail.8. We will return to this question in the next chapter. The spaces of splines is the natural habitat for these reconstruction methods. it is natural that we look for multiscale representation of functions. At the same time we need techniques that allow us to change between representations on diﬀerent scales.7) And a family of representation operators Pj : V → Vj such that v − Pj (v) ≤ C inf u∈Vj v − u. doors.4.3: (a) Graph of the synthesis function. the identiﬁcation of a house can be done in a very coarse scale.2. A natural question would be: Question 1. Consider a sequence of closed subspaces {Vj }j∈Z such that · · · Vj+1 ⊂ Vj ⊂ Vj−1 · · · . This can be achieved by using nested representation spaces. functions on the whole interval can be controlled according with the degree of the polynomials. Therefore. ﬂoor. (2. As an example. First we use a coarse scale to recognize the object. ∀ ∈ Z. MULTIRESOLUTION REPRESENTATION 19 Figure 2. where C independs on j.
by adding details. Note that from equation (2.8) (2.9) Also we require a natural commutativity condition Pj ◦ Pj−1 = Pj . which guarantees that the diﬀerent representations match each other. Equation (2. Iterating equation (2.10) we have the decomposition equation VJ = Vj0 + (Wj0 + · · · + WJ+1 ) . If Wj = Image(Qj ) = Qj (Vj ). (2. (2.12) In order to be able to decompose any element v of V we impose some additional hypothesis on the representation operators Pj : 1. deﬁned by Qj (v) = Pj+1 (v) − Pj (v). This can be stated precisely by introducing the operator Qj : V → Vj−1 .11) can be rewritten using operators: j0 Pj (v) = Pj0 (v) + j=J+1 Qj (v). .11) which says that a ﬁner representation can be obtained from a ﬁner one. (2. Pj2 = Pj ) (2. ∀v ∈ Vj (i.e. FUNCTION REPRESENTATION AND RECONSTRUCTION Figure 2. Pj (v) → v if j → −∞. Intuitively each representation space Vj−1 contains more details (ﬁner scale) than the space Vj .7) we have Pj (v) ≤ cv Pj (v) = v.10) That is.4: Representation operator. Qj is called a reﬁnement operator.20 CHAPTER 2. For this reason. Qj (v) is the detail we must add to the representation space Vj to obtain Vj+1 . it follows that Vj−1 = Vj + Wj .
∀v ∈ V.9. as well as techniques to compute dictionary based representations. j0 → +∞ and j → −∞. When we write. . 2.12) we have v= j∈Z Qj (v). j∈Z Vj j∈Z is dense on V . That is. Representation using dictionaries allows us the use of a great heterogeneity in the reconstruction functions. 1993) introduced in the literature a representation method based on dictionary of functions. From the proximity condition of equation (2. 2. with index set IM : fM = f.7).2.11). the sum in equation (2. on the contrary. S. these two conditions are equivalent to 1. One basic strategy to compute a representation for a function f is described below. f= λ∈Γ such that (αγ ) ∈ The rationale behind the representation method consists in constructing extensive dictionaries and devising optimal representation techniques that allow us to represent a function using a minimum of words from the dictionary. Therefore. = {0}. REPRESENTATION BY DICTIONARIES 21 2. Let fM be the projection of f over the space generated by M vectors from the dictionary. Note that distinct functions use diﬀerent dictionary vectors in their representation which makes the representation process nonlinear. in (2. A dictionary in a function space H is a family D = (gλ )λ∈Γ of vectors in H such that gλ  = 1. Mallat (Mallat & Zhang. In the particular case where the representation operators Pj are orthogonal projections over closed Hilbert spaces. 1998) several dictionary systems are described.9 Representation by Dictionaries The problem of obtaining a representation of a function can be well understood using the metaphor of natural language. A representation of a function in a dictionary D is a decomposition αγ gγ . and the spaces Vj and Wj are orthogonal. which makes the representation/reconstruction process very ﬂexible. This family is not necessarily countable. any vector v is the sum of all of its details in the diﬀerent representations. gm gm . If we have a rich vocabulary we will have a great conciseness power to write the idea. m∈IM 2. Based on this metaphor. In (Mallat. Techniques to compute nested sequences of multiresolution representation are closed related with wavelets. is in fact a direct sum. we will have to use more words to express the same idea. Pj (v) → 0 if j → +∞. we have an idea and we must represent it using words of the language. By taking the limit.
and take them into consideration when representing the function. 2. Certainly the error is smaller than the error resulting from a linear approximation where the decomposition vectors of the representation do not vary with f . the representation operator R: H → 2 . We will start our journey on this road from the next chapter on. multiscale representations. In fact. .11 Wavelets and Function Representation We can summarize what we have done up to this point in the statement that our main concern is the study of representation and reconstruction of functions. gm . frames. concise and allows for robust reconstruction. and the image of H is the whole space f= j∈Z The reconstruction equation f. 1993). Therefore our slogan at this point should be “all the way to function representation and reconstruction techniques”.22 CHAPTER 2. we need to develop techniques that allow us to construct representation tools (basis. In fact. if the frame (ϕj )j∈Z is an orthonormal basis. page 97. FUNCTION REPRESENTATION AND RECONSTRUCTION The error eM in the approximation is the sum of the remaining coeﬃcients eM = f − fM 2 = m∈IM /  f.) which are ﬂexible. 2. the vectors that best match the features of f . dictionaries. that is. Besides nonunicity. If the frame is not constituted by linearly independent vectors. This leads us naturally to the study of function variation as an strategy to detect the diﬀerent features of a given function. This redundancy can be used to obtain a certain robustness in the representation/reconstruction process. The above ideas lead to an algorithm to compute the representation. It will take us from the kingdom of Fourier to the wonderland of wavelets. ϕj ϕj ˜ computes each vector in such a way that there is no correlation between the coeﬃcients. This occurs for instance in the representation using frames. These are the vectors that have a better correlation with f . is an isometry.10 Redundancy in the Representation The representation of a function is not unique in general. gm 2 . etc. which corresponds to a certain correlation between the elements of the representation sequence. ϕj 2. there exists a redundancy in the representation. we can have a redundancy for a given representation. To minimize this error the indices in IM must correspond to the M vectors that have the largest inner product amplitude  f. (Rf )j = f. 1992). An interesting discussion of the redundancy in frame representation can be found in (Daubechies. For details see (Mallat & Zhang.
rept. . S. A.8 of this chapter was based on (Canuto & Tabacco. J. 1998). and orthonormal projection. Math. & Tabacco.2.. 1998. pages 5763. SpringerVerlag. 1989. Multiresolution approximation and wavelets. Soc. 69–88. A Wavelet Tour of Signal Processing. A detailed description of Mallat’s work will be given later on. 3(6). 1992). Mallat. Academic Press. & Zhang. 1989) in the context of the Hilbert spaces L2 (R) of functions. Claudio. Mallat.. 1993. Mallat in (Mallat.12 Comments and References Multiscale representation was introduced by S. Courant Institute of Mathematical Sciences.12. Multilevel Decompositions of Functional Spaces. 1997. Tech. Amer. Luiz. A detailed study of representation and reconstruction using frames is found in (Daubechies. Gomes. Ingrid. 1997. 1997). Mallat. The Journal of Fourier Analysis and Applications. Philadelphia. PA: SIAM Books. Matching Pursuit with Timefrequency Dictionaries. Image Processing for Computer Graphics.. 315. S. Bibliography Canuto. & Velho. and also in Chapter V of (Mallat. Daubechies. S. Trans. 1992. Ten Lectures on Wavelets. COMMENTS AND REFERENCES 23 2. The more general introduction we gave in Section 2. Anita.
.
1 Analyzing Functions To detect features of a function we must analyze it. In an audio signal. We denote by 25 . These functions are usually called signals. the distinction from red to green is captured in the frequency of the associated electromagnetic wave.1. The most traditional of these tools is the Fourier Transform which we will study in this chapter. The best way to analyze the features of a signal is by studying its frequencies. which is called the frequency of the function. for example. 3. This occurs in our everyday routine: signals are analyzed and interpreted by our senses. the parameter ω indicates how many complete cycles of period exist on the interval [0. is the starting point of our journey to the wavelets. 3. A > 0.Chapter 3 The Fourier Transform In order to devise good representation techniques we must develop tools that enable us to locate distinguished features of a function. a signal representation is grabbed from this analysis and it is sent to our brain. the parameter A measures the amplitude (maximum and minimum values assumed by f ). that is. The study of Fourier transform. f (t + L) = f (t). This is the process used in our perception of colors and sound. and to each instant of time the function produces a certain output which we are able to detect. Sound. Color. In fact given the function f (t) = A sen(2πωt). Also. Consider a periodic function with period L > 0.1 shows the graph of f for diﬀerent values of ω. its strength and limitations.1 Fourier Series In order to analyze the frequency content of a function we must initially answer the following question: What is the frequency of a function? This is an easy task when the function is periodic. and other elements we interact with in our everyday life are characterized by functions: To each point on the space. the frequencies are responsible for what we are accustomed to identify as an acute or grave sound. This number is directly connected with the number of oscillations of the function in the unit of time. 1]. Figure 3.
1) is an orthogonal basis representation of the T function f . of this fundamental frequency. That is T t0 +T t0 f (t)2 dt < ∞. This frequency amplitude aj is computed using the equation aj = L 0 f (u)ei2πωj u du.1) of the Fourier series measures the amplitude of the frequency component ωj on the function f . we have an exact representation of f 3. the Fourier series shows that any periodic function can be decomposed as an inﬁnite sum of periodic functions (sines and cosines).2) where L is the period of the function. In particular. this frequency is not present in the function. Note that equation (3. The theory of Fourier series says that f can be decomposed as +∞ f (s) = j=−∞ aj ei2πωj s . It is well known that the family {ei2πωj s . In other words.2 Fourier Transform The above portrait of the function f worked perfectly well: We were able to obtain an exact representation of the function f and this representation completely characterizes f by its frequencies. The coeﬃcient aj in the equation (3. In conclusion.26 CHAPTER 3. L2 (R) the space of periodic functions of period T which are square integrable. every function .1. and all of the other frequencies are integer multiples ωj. j ∈ Z. j ∈ Z} is a complete orthonormal set of the space L2 (R). if aj = 0.1: Sine function with frequencies 2. (3. Is it possible to extend the above results for nonperiodic functions? In this case we do not have a discrete spectrum of well deﬁned frequencies as in equation (3.1). (3. The only drawback is the fact that f was a periodic function. This decomposition of a periodic function f is called the Fourier series of f .1) completely characterizes the function f by its frequencies. 4 and 8. This decomposition makes it easy an analysis of the frequencies present on the function f : There exists a fundamental frequency ω.1) where ωj = j/T is a constant. Therefore equation (3. aj ∈ R. In fact. THE FOURIER TRANSFORM Figure 3.
The rationale behind the frequency computation in the modulation process can be explained as follows: When f has oscillations of frequencies s. ANALYZING FUNCTIONS 27 f deﬁned by equation (3. or close to s. g = R f (u)g(u)du. ˆ In sum. On the contrary. This space is denoted by L2 (R). Note that f is in fact an ˆ = F (f ).3) is zero or close to zero. Note that we changed the signal of the exponent when passing from (3. The exponential is called the modulating function. When a(s) = 0 this means that frequencies s occurs on f . Since s varies in a continuum of numbers. and it is easy to see that this norm is induced by the inner product.5) This operator is called Fourier Transform. s ∈ R. these frequencies result to be in resonance with the frequency s of the modulating function. Take s = ωj in equation (3. we can use the Fourier series representation as an inspiration to introduce the concept of frequency for arbitrary functions.3) can be interpreted as an inﬁnite weighted average of f using the modulating function as the weighting function. deﬁned by (3.3) as describing a frequency density of the function f . (3. Nevertheless. It is well known that the space L2 (R) with this inner product is a Hilbert space. when the oscillations of f and the frequencies of the modulating function are completely distinct we have a cancellation eﬀect and the integral of (3.4) These functions are called square integrable functions or functions with ﬁnite energy. where g means the complex conjugate.3) to (3. ei2πsu is a periodic function of frequency s. Equation (3. f. The operation f (u)ei2πsu in the integrand above is called modulation of the function f .4) deﬁnes a norm on L2 (R). Therefore operator that associates to the function f the function f we have an operator deﬁned between two function spaces. ˆ ˆ We will change notation and denote a(s) by f (s) or by F (f )(s). and assume that the variable s takes any value. therefore a(s) assumes nonzero values. (3.2) which computes the amplitude of the frequency. it is interesting to interpret (3. Therefore. for each s ∈ R equation (3. . What function spaces are these? It is possible to show that the operator is well deﬁned for any function f satisfying the equation f  = R f (u)2 du < ∞.3. We conclude that a(s) measures the occurrence of the frequency s on the function f .3) Notice that we have changed the notation from aj to a(s). For each s.5) so as to keep the deﬁnition compatible with the deﬁnition of the Fourier transform used in the literature.1.1) is necessarily periodic. (3. deﬁned by ˆ F (f )(s) = f (s) = +∞ −∞ f (u)e−i2πsu du. we have introduced an operator F = f : L2 (R) → L2 (R). called the space of square integrable functions. We obtain a(s) = +∞ −∞ f (u)ei2πsu du. The value of a(s) is a measure of the occurrence of the frequency s on f .3).
In fact.1) of the Fourier series. (3. there exists a sequence fn ∈ L1 (R) ∩ L2 (R) such that n→∞ lim fn − f  = 0. ˆ ˆ f (w) − f (s) ≤ R f (t)ei2πwt − ei2πst  ≤ R f (t)dt w − s. 2. instead of L1 (R)? We can give two main reasons for that: transform on L • The Fourier transform of an integrable function is not necessarily integrable. That is.6) This equation is known as Parseval identity. This equation is known as Plancherel equation. ˆ ˆ We deﬁne f as the limit of the sequence fn .7) g(s)ei2πωt ds. we have ˆ f (t) = F −1 (f ) = +∞ −∞ +∞ −∞ (3. g .28 CHAPTER 3. It follows easily from it that ˆ f 2 = f 2 . and the coeﬃcients of each function in the superposition are given by the Fourier transform. (3. In this case it is easy to see that f is bounded and continuous.9) An important fact to emphasize is that we have two diﬀerent ways to “read” equation (3. It allows us to reconstruct f as a superposition of periodic functions. Therefore.9) of the inverse transform: ˆ 1. for any f ∈ L2 (R). • The space L2 (R) has a richer structure than the space L1 (R) because of its inner product. ˆˆ f. THE FOURIER TRANSFORM Without going into too much details. g = f . The second interpretation above shows that the equation of the inverse Fourier transform is a nondiscrete analogous of equation (3. A fascinating result is that F is an isometry of L2 . The extension of f to the space L2 (R) of square integrable functions is achieved by a limiting process: Any function f ∈ L2 (R) can be approximated by integrable functions of ﬁnite energy. ∀f. Therefore F is not an operator F : L1 (R) → L1 (R). g ∈ L2 (R). we would like to clarify some mathematical points. (3. . Why is it important to deﬁne the Fourier 2 (R). L2 (R) From the linearity of the integral it follows easily that the Fourier transform F : L2 (R) → is linear. In fact it is a Hilbert space. It furnishes a method to obtain the function f from its Fourier Transform f .8) ˆ f (s)ei2πst ds. The Fourier transform is well deﬁned when the function f belongs to the space L1 (R) of ˆ integrable functions. The inverse of the Fourier transform is given by F −1 (g)(t) = ˆ That is. F −1 (f ) = f . More precisely. ei2πst .
3 Frequency Analysis In this section we study some examples that will give a better insight into the analysis provided by the Fourier transform. When we compute the Fourier transform of f . When the function is not periodic we know that equation (3. One important question can be posed now: How eﬀective is the Fourier transform analysis of a function? A discussion of this question will be done in the next section. Note that in this case the reconstruction is exact. From the point of view of the previous chapter the analysis equation computes a representation of the periodic function f by the sequence (aj ). Note that the analysis and synthesis equation associated with the Fourier transform are not discrete as in the case of the Fourier series. Nevertheless.2) provides us with information about the frequencies of a periodic function. For this reason. can be interpreted as associating to each value of t in the spatial domain R some physical magnitude f (t).1) of the function f (the analysis equation). these features are important elements to obtain good function representation. The synthesis equation provides us with a reconstruction technique. for each value of the parameter ˆ(s) represents the frequency density s in f .2. We interpret this by saying that s ∈ R the value f ˆ f is a representation of the function f in the frequency domain. Equation (3.9) writes the function f as a superposition of a “continuum” of periodic functions.1) allows us to obtain f from the coeﬃcients aj computed using the analysis equation. As we have pointed out on the previous chapter. it is called the analysis equation. Therefore we do not have a tool that allow us to represent and reconstruct arbitrary functions as in the case of the Fourier series. For this reason it is called a synthesis equation.3 Spatial and Frequency Domain Analyzing from the mathematical point of view. Since in our case the spatial domain has dimension 1 it is common to interpret the variable t as time and call the spatial domain by the name of time domain. In summary. The frequency contents is closely related with the features carried by the function. gives us an analysis of the frequencies of the function f . In this case. ˆ Suppose that f (s0 ) = 0. This equation plays the role of the reconstruction equation (3. Note that describing a function on the frequency domain allow us to obtain the frequency contents of the function.1. which deﬁnes the Fourier transform.3. the next step of analyzing f consists in determining the localization of these . the Fourier transform is an invertible linear operator on L2 (R). in the applications we have a very interesting interpretation of it. we ˆ obtain another function f (s) deﬁned on L2 (R). From this we conclude that f has frequencies s0 or close to s0 . 3. the Fourier transform changes a function from the spatial to the frequency domain. 3. Equation (3. In this case.5). Equation (3. A function f : R → R. for periodic functions. j ∈ Z. A PAUSE TO THINK 29 3.2 A Pause to Think Our purpose in this section is to have a better understanding of the potentialities of the Fourier transform from the point of view of function representation and reconstruction.
05s e t1 = 0. or frequencies close to s. and in particular in the problem of function representation.42). The graph of its Fourier Transform is shown on Figure 3. Note that the Fourier transform detects well the two sine signals (the two spikes in Figure (b)). These frequencies do not appear on Figure 3.74t) + δ(t − 0. In conclusion the Fourier transform does not provide clear information about the two impulses. Therefore the modulation introduced by the two impulses generate a superposition of frequencies. where we added two impulses of order 3 at the two diﬀerent positions t0 = 0.2(b).05) + δ(t − 0. and the localization of a frequency allows us to determine the object position. Example 5 (Signal with impulses).2(b) because of an scale problem. This happens because the modulating function (exponential) used to measure the frequency density on the Fourier transform does not have compact support: the integral that deﬁnes the transform extends to the whole line.42s. THE FOURIER TRANSFORM frequencies on the spatial domain of f . In general localizing frequencies on the spatial domain using Fourier transform is impossible.12t) + sin(2π2967. We can make an analogy using a radar metaphor: the existence of a certain frequency detects the presence of some object. are present on the function f . The graph of this signal is shown on Figure 3. It consists of a sum of two sines with frequencies 516.2(a). .74Hz.2: A signal and its Fourier transform.12Hz and 2967. Consider the signal deﬁned by the function f (t) = sin(2π516. Therefore the only information carried from ˆ the fact that f (s) = 0 is that the frequency s. How about the impulses? It is easy to see from the deﬁnition of the Fourier transform. This localization is of great importance in several applications. Figure 3. that translation of an impulse on the spatial domain introduces modulation by an exponential on the frequency domain.30 CHAPTER 3. 5 2000 1800 1600 4 3 1400 2 1200 1000 1 800 0 600 400 −1 200 −2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 0 0 500 1000 1500 2000 2500 (a) (b) Figure 3.3 plots the graph in a diﬀerent scale so as to make it possible to observe the superposed frequencies corresponding to the two impulses.
Nevertheless from the physical point of view we do not ﬁnd a plausible explanation for the problem of representing such a simple signal combining an inﬁnite number of periodic functions. but they inﬂuence the computation of the Fourier transform along all of the domain because the modulating function does not have compact support. This shows that the Fourier transform of the impulse δ is a constant function of value 1. Taking f (t) = e−i2πst in this equation we have ˆ δ(s) = +∞ −∞ δ(t)e−i2πst dt = 1. and periodic. .42). discontinuities).3: Fourier transform of two impulses δ(t − 0. These signals are called stationary signals. Using this result in the reconstruction equation (3. exponentials of every frequency values s ∈ R must be combined in order to analyze the impulse signal.g. We have +∞ −∞ f (t)δ(t)dt = f (0).05) e δ(t − 0.3. FREQUENCY ANALYSIS 31 6 5 4 3 2 1 0 0 500 1000 1500 2000 2500 Figure 3. we must “sum” an inﬁnite number of these functions so as to occur destructive interference. The diﬃculty in localizing frequencies on the spatial domain is one of the major weakness of the Fourier transform in analyzing signals. The Fourier analysis is therefore more eﬃcient in the study of signals that do not suﬀer sudden variations along the time.3. Consider the impulse “function” Dirac δ. In general. That is. Example 6 (Fourier analysis of the impulse signal). the high frequencies relative to these changes are detected by the transform. Below we will give a classical example that shows the ineﬃciency of the Fourier transform in representing functions. we obtain δ(t) = +∞ −∞ ˆ δ(f )ei2πst ds = +∞ −∞ ei2πst ds.9). if a function presents sudden changes (e. This fact has a very simple mathematical explanation: As the Fourier transform uses modulating functions without compact support.
This result is of paramount importance and it is the link between ﬁlter theory and Fourier analysis. a good insight is gained by studying ﬁltering of continuous signals. The signal h is called the impulse response of the ﬁlter.13) (3. deﬁned on the space of signals with ﬁnite energy. Applying the ﬁlter L to e−i2πwt yields L(e−i2πwt ) = R h(u)e−i2πw(t−u) du = e−i2πwt ˆ = h(w)e R −i2πwt (3. We conclude that ﬁltering signal f with a linear. A ﬁlter is an operator L : L2 (R) → L2 (R). Therefore.32 CHAPTER 3. The relation of the ﬁltering theory with the Fourier analysis is very important and will be exploited in many diﬀerent parts of the book. L(f (t − t0 )) = (Lf )(t − t0 ). with the same result. That is.10) h(u)f (t − u)du. An important class of ﬁlters is given by the linear. and it is denoted by h ∗ f (t).12) (3. A ﬁlter is time invariant if a time delay can be applied either before of after ﬁltering. Linear and space invariant ﬁlters are simple to study because they are completely determined by their values on the impulse signal h = Lδ(t). Sometimes h is called the kernel of the ﬁlter. That is. h ∗ f (t) = R h(u)f (t − u)du.14) h(u)e−i2πwu du . and the correˆ sponding eigenvalue is the value h(w) of Fourier transform of the impulse response function h of L. In fact.4 Fourier Transform and Filtering Even though the classical theory of ﬁltering is developed for discrete signals. A ﬁlter is linear if the operator L : L2 (R) → L2 (R). . time invariance of L gives Lδ(t − u) = h(t − u). This shows that each exponential e−i2πwt is an eigenvector of the ﬁlter L. as an immediate consequence it gives an insight in the action of the ﬁlter on the frequencies of a signal f as we will show below. L(f (t)) = L = R R f (u)δ(t − u)du R (3. THE FOURIER TRANSFORM 3. Indeed. (3.11) f (u)h(t − u) = The last integral on the above equation is called the convolution product of h and f . time invariant ﬁlter L is equivalent to make a convolution of f with the signal h = Lδ(t). time invariant ﬁlters. is linear.
Linear and space invariant ﬁlters are classiﬁed according to the way they modify the frequencies of the signal f . Since Lf (t) = h ∗ f .5. • Bandpass • Bandstop.4. Highpass Filter This ﬁlter has a complementary behavior to that of the lowpass ﬁlter. equation (3.4: Transfer function of an ideal lowpass ﬁlter.3. .4. and on the right we have a product of two functions. It attenuates the low frequencies and does not change the high frequencies. ˆ f (w)L(ei2πwt )dw = R ˆ ˆ f (w)h(w)ei2πwt dw.16) it follows that the graph of the transfer function of an ideal lowpass ﬁlter is shown in Figure 3. This classiﬁcation includes four basic types: • Lowpass. Figure 3. (3.16) where F is the Fourier transform.16) it follows that the graph of a transfer function of an ideal highpass ﬁlter is shown in Figure 3. • Highpass. FOURIER TRANSFORM AND FILTERING 33 From the equation of the inverse Fourier transform we have f (t) = Applying L to f we get L(f (t)) = R R ˆ f (w)ei2πwt dw. Lowpass Filters This ﬁlter is characterized by the fact that they attenuate the high frequencies of the signal without changing the low frequencies. amplifying or attenuating them.16) will be used to give a better insight into the ﬁltering operation.15) The above equation shows that the ﬁlter L modulates the sinusoidal components ei2πwt of f . From equation (3. (3. Equation (3. From equation (3.15) can be restated as F (h ∗ f ) = F (h)F (f ). The Fourier transform F (h) of the impulse response is called the transfer function of the ﬁlter.
Bandpass Filter This ﬁlter changes both low and high frequencies of the signal. The graph of the transfer function of an ideal bandstop ﬁlter is shown in Figure 3. 3. Figure 3. Bandstop Filter This is the complementary of a bandpass ﬁlter. The graph of the transfer function of an ideal bandpass ﬁlter is shown in Figure 3.7: Transfer function of an ideal bandstop ﬁlter. Figure 3. THE FOURIER TRANSFORM Figure 3.34 CHAPTER 3.5: Transfer function of an ideal highpass ﬁlter. but does not change frequencies in some interval (band) of the spectrum.6: Transfer function of an ideal bandpass ﬁlter. A lowpass ﬁlter is a bandpass ﬁlter for low frequencies. Frequencies outside this frequency band are not aﬀected.7.6. This ﬁlter aﬀects frequencies on an interval (band) of the spectrum.5 Fourier Transform and Function Representation Is it possible to obtain representation and reconstruction techniques for a nonperiodic function using the Fourier transform? .
In fact. in (c) we show the graph of the point sampling representation (fi ) of f . and its relationship with ﬁlter theory.5. k∈Z ˆ ˆ The relation between the Fourier transform f of f and the Fourier transform fd of the discretized signal fd is given by 1 ˆ fd (s) = ∆t k ˆ f s− ∆t (3.9) which deﬁnes the inverse transform gives us a clue to look for an answer to our question. Nevertheless the Fourier theory. “modulated” by the Fourier transform. in (b) we show the graph of the Fourier transform f . . In fact.3. Unfortunately this discretization leads us to the Fourier series of the function f . This is illustrated in Figure 3. j ∈ Z. is of great importance in the study of diﬀerent representation methods. suppose that we have a uniform sampling partition of the real numbers with interval length ∆t. it is a powerful tool to analyze the problem of function representation.17) k∈Z A clear interpretation of equation (3.1 Fourier Transform and Point Sampling In spite of all of the weakness of the Fourier transform that we have already discussed. We can eliminate this redundancy by taking only a discrete set of frequencies sj = ω0 j. This length is called the sampling period. this equation writes the function f as a superposition of periodical functions. FOURIER TRANSFORM AND FUNCTION REPRESENTATION 35 Equation (3. We will use it to understand the problem of exact reconstruction from a representation by point sampling. If f (t) = n∈Z f (n)δ(t − n). An important step in this direction is to understand the spectrum of a point sampled signal.5. Fourier series are particular instances of Fourier transform for discrete signals. in (d) we show the Fourier transform of the sampled function fd . ω0 constant. then ˆ f (w) = n∈Z f (n)e−i2πwn . Certainly there exists a redundancy in this “representation” of the function f by a continuum of functions.17) is very important: Apart from the scaling factor ˆ 1/∆t.8: In (a) we show the graph of the function ˆ f . Therefore the method that seems natural to obtain function representations from the Fourier transform is in fact inadequate. it says that the spectrum of the sampled signal fd is obtained from the spectrum f of the continuous signal f by translating it by multiples of 1/∆t. The discretized signal is given by fd (t) = f (k∆t)δ(t − k∆t). This fact will be illustrated in the next section. More precisely. and summing up all of the translated spectra. 3.
assumes zero values outside a bounded ˆ interval [−Ω. i ∈ Z of the real numbers R such that 2∆t ≤ Ω. ˆ • The Fourier transform f of the function f . That is. Note that in particular.17) is the key point to understand the reconstruction problem when we use uniform point sampling to represent a function. we have the classical: Theorem 2 (Theorem of ShannonWhittaker). It is important to remark that the translating distance varies inversely with the sampling period: The sampling period is ∆t and the translating distance is 1/∆t. This fact will be discussed in next section.8: Sampling and the Fourier spectrum. Initially we will demand the following conditions: • The point sampling process is uniform. Equation (3.5. Using the above restrictions. supp (f ) ⊂ [−Ω. Consider a function f : R → R with ˆ compact support. very high frequencies are introduced in the sampling process by the inﬁnite translation of the spectrum. THE FOURIER TRANSFORM Figure 3. Ω] of the frequency domain. tk+1 ] have the same length ∆t = tk+1 − tk . the sampling intervals [tk . and a uniform partition ti . Then f can be reconstructed (exactly) from its point sampling . 3. Therefore closer samples produce more spaced translations and viceversa.2 The Theorem of ShannonWhittaker In this section we will use Fourier theory to answer a question posed on chapter 2: Is it possible to obtain an exact representation of a function using point sampling? We will see that the answer is positive if we restrict the function f and at the same time impose conditions on the sampling rate. We say that f has compact support.36 CHAPTER 3. Ω].
17) that the sampling process alters the frequency spectrum of the function f .18). including the computation of the reconstruction equation (3.6 Point Sampling and Representation by Projection eΩ (t) = sin π2Ωt . ˆ A function f such that f has compact support is called a band limited function. 3. The function f should not have very high frequencies. The translation distance of the spectrum of the original signal f depends on the sampling period (the length of the intervals in the sampling lattice): The smaller the period (that means more space between consecutive samples).6. the bigger will be the translation of the spectrum.18). Ω]. the translated spectra will have disjoint domains. The reconstruction equation is given by +∞ f (t) = k=−∞ f( k sin π(2Ωt − k) ) . and assumes the values of f on the points of the partition. We take samples of f suﬃciently close. Details of this proof. introducing very high frequencies by translating and summing up the original spectrum.18) The inequality 2∆x ≤ Ω is called Nyquist limit. π2Ωt It can be shown that if we take . can be found in (Gomes & Velho. From this original spectrum we are able to reconstruct the original signal using the inverse Fourier transform. 2Ω π(2Ωt − k) (3. We have seen from equation (3. Sketch of the Proof: Consider a band limited function f . The conclusion of the theorem is: f can be exactly reconstructed interpolating its samples using equation (3. supp (f ) ⊂ [−Ω. POINT SAMPLING AND REPRESENTATION BY PROJECTION 37 representation (f (ti ))i∈Z . The result of the theorem is very intuitive. using an adequate lowpass ﬁlter we can recover the original spectrum of the signal f . such that we have at least 2 samples for a complete cycle of maximum frequency (2∆t ≤ Ω). because it posses frequencies within a limited interval (band) of the frequency domain. 2. In order to reconstruct f we have to recover the original frequency spectrum from the spectrum of the sampled function. In this case. It says that we must take at least 2 samples for each complete cycle of maximum frequency occurring in the function. It is interesting to make a sketch of the proof of the theorem so that we can see the role of Fourier analysis and ﬁltering. The Nyquist limit 2∆t ≤ Ω says that if we take the sampling period suﬃciently small. The hypothesis of the theorem says: ˆ 1. 1997). The point sampling representation transforms f into the sequence f (ti ) which is null outside the points of the sampling partition.3.
the representation of f in this basis is given by f. In sum. If a signal f is not bandlimited. δ(x − n) = f (n). that is. the problem of point sampling and reconstruction is reduced to the problem of representation on an orthogonal basis. . and it is represented using point sam/ Ω pling with period ∆t ≤ Ω/2. THE FOURIER TRANSFORM the set {eΩ (t − k∆t)} is an orthogonal basis of the space ˆ L2 (R) = {f ∈ L2 (R) . φn = f. In fact.18) is the projection of f on this basis. we have ˆ ˆ φn . Note that if φn (x) = δ(x − n). What is the relation between the elements f. This fact motivates us to pose the following question: Question 2. and the identity ˆ ˆ φn (s) = F (φ(x − n)) = e−i2πins φ(s). φn )n∈Z . minimized. f ∈ L2 (R). Ω]}. we obtain a function f ∈ L2 (R) such that f − f  is ˜ is the orthogonal projection of f on L2 (R). the reconstruction equation (3.38 CHAPTER 3. Ω Moreover. then f. the elements of the representation sequence are samples of the function f . f = φn . From Parseval equation.7 Point Sampling and Representation Coeﬃcients When we have an orthonormal basis {φj } of the space L2 (R). f Ω 3. We will suppose that the functions φn which constitute the orthonormal basis are obtained from a single function φ by translations.18) of ShannonWhittaker. φn (x) = φ(x − n). and we reconstruct the sampled signal using the reconstruction ˜ ˜ equation (3. f = +∞ −∞ ˆ ˆ e2πins φ(s)f (s)ds = F (n). φn of the representation sequence and the samples f (n) of the function f ? There is a simple answer for this question in a very particular case of great importance in the study of wavelets. More precisely. f= n In this case we have the representation sequence f → ( f. That is. supp(f ) ⊂ [−Ω. φn φn .
1995). This fact is resumed in the theorem below for future references: Theorem 3. and sampling the resulting function F . Notice that ˆ ˆ ˆ F (s) ≤ φ f  = φ f . We have not stressed in this chapter a very important point when working with Fourier analysis and ﬁltering: Since all of the theory studied here is linear.3.8. That is. 1995). both from the conceptual and computational point of view.8 Comments and References There are several good books covering the theory of Fourier analysis. φ(x − n) = F (n) 3. the study of the computational complexity of the DFT when applied to a ﬁnite signal with N samples. A comprehensive reference for the discrete Fourier transform. In practice we use ﬁnite signals when implementing the operations of sampling and reconstruction on the computer. it can be found on Chapter 1 of (Gomes & Velho. From the computational viewpoint an important issue related with the Fourier transform is the study of its computational complexity. is found in (Briggs & Henson. which reduces the computational complexity. For this reason. For those interested in these topics we recommend (Loan. The reference (Gomes & Velho. (3. This approach is important specially from the computational point of view. φn from the representation sequence as samples of the function f . 1990). This reference also covers the discrete Fourier transform. 1996). 1996) and (Duhamel & Vetterli. 1989). adequate for those with some computer graphics background. therefore the values of F (n) are close to the values of the original function f . φn of the representation sequence of f on the basis {φn } are obtained by ﬁltering f . we need to study the Fourier transform of ﬁnite signals. We will use some . Matrix notation is used all over in (Strang & Nguyen.19) shows that F is obtained from the original function f by a lowpass ﬁltering process. 1997) brings a chapter with a review of signal theory. it is common to refer to the elements f. f. if it does not have great frequency variations. That is. If the function φ is a lowpass ﬁlter. and this implies that F is continuous. There are diﬀerent ﬂavors of computing with a Fast Fourier Transform. 1997) and (Briggs & Henson. even when φ is not a lowpass ﬁlter. equation (3. φn = f. a natural representation for them is to use matrices. 1997). This Fourier transform is called Discrete Fourier Transform (DFT). If {φn = φ(x − n)} is an orthonormal basis of L2 (R) then the terms f. COMMENTS AND REFERENCES 39 where the function F is given by its Fourier transform ˆ ˆ ˆ F (ω) = φ(ω)f (ω). Therefore. For a revision of the classical theory we suggest the reader to consult (Weaver.19) ˆ therefore F (ω) is integrable. This shows that the values of the samples F (n) are well deﬁned. F = f ∗ φ. For the reader interested in details we recommend (Gomes & Velho.
PA: SIAM Books. A comprehensive discussion with diﬀerent versions of the problem including solutions can be found on (Zayed. Luiz. P.40 BIBLIOGRAPHY matrix notation later on these notes. L. 1990. Gomes. 259–299.. 1992. & Henson. 1992). Butzer. 1995. SpringerVerlag. W. and the case f : R3 → R which is related to the study of volumetric objects. Nevertheless it presents certain deﬁciencies as a solution to the samˆ pling/reconstruction problem. 1990). For details about the history of this problem we recommend (Butzer & Stens.. 1997). R. Of particular importance in computer graphics is the case of functions f : R2 → R. E. & Velho. P.. 40–53. which describes an image.. Several research directions are raised from the Theorem of ShannonWhittaker. Lim. An analysis of the reconstruction problem when we use uniform point sampling representation with a sampling rate superior to the Nyquist limit is found in page 20 of (Daubechies. Bibliography Briggs. The Theorem of ShannonWhittaker is a very important result in the theory of function discretization.. Duhamel. is found in Chapter 7 of (Gomes & Velho. 1992). Sampling Theory for not necessarily bandlimited functions: a historical overview. The concepts of this chapter extend naturally to real functions of several variables. Daubechies. DFT: an owner’s manual for the discrete Fourier transform. 19(4). Signal Proc. Jae S. Ten Lectures on Wavelets. A good reference at an introductory level that covers twodimensional signal processing is (Lim. M. Ingrid. We could mention the following ones: • Look for versions of the theorem using a weaker hypothesis than that of compact support. The reader should consult the Appendices to this course notes where we introduce matrix notation. The problem of reconstructing a function from its samples called the attention of mathematicians since the beginning of the century. Two Dimensional Signal and Image Processing. • Generalize the theorem for arbitrary domains of Rn . the hypothesis that f has compact support is too much restrictive. V. . 34(1). 1993). A detailed discussion of the diﬀerent problems arising from incorrect reconstruction of a signal from a representation by point sampling. 1990. • Analyze what happens with the reconstruction when the sampling is done without obeying the Nyquist limit. 1992. 1997. & Vetterli. J. & Stens. New York: PrenticeHall. Image Processing for Computer Graphics. L. L. Philadelphia. In particular. Philadelphia. SIAM books. SIAM Reviews. Fast Fourier Transform: A tutorial review and a state of the art.
SIAM books. 1996. Wellesley. Van. Advances in Shannon’s Sampling Theory. Wavelets and Filter Banks. & Nguyen. Ahmed. . 1993. Computational Frameworks for the Fast Fourier Transform. Zayed. New York: John Wiley & Sons. Boca Raton: CRC Press. Truong. Weaver. J.BIBLIOGRAPHY 41 Loan. 1996. Philadelphia. MA: WellesleyCambridge Press. Strang. 1989. Theory of Discrete and Continuous Fourier Transform. Gilbert.
.
ω). Mathematically. the last two properties show that the modulating function used to detect ˜ frequencies in the computation of the “transform” f (t. ω) depends only on the values of t on the interval [t − t0 . ω) that depends on the time and the frequency. This means that the signal f is transformed by the auditory system in a signal ˜ f (t. That is.Chapter 4 Windowed Fourier Transform In this chapter we will introduce a modiﬁcation in the deﬁnition of the Fourier transform in order to obtain a transform with better localization properties in the timefrequency domain. The inspiration for this transform is to analyze the audio analysis performed by our auditory system. Real time analysis. The audio information we receive occurs simultaneously on time and frequency. We say that it is localized in time. was the ﬁrst to propose a transform with the above properties. t]. 4. This transform will give us better results for the purposes of function representation. This means that only values of f (t) for t ≤ t1 can be ˜ analyzed when computing the “transform” f (t. 43 . sounds that we have heard some time ago do not inﬂuence the sounds that we hear in a certain instant of time. ω) must have its values concentrated in a neighborhood of t. Future sounds are not analyzed. The auditory system has ﬁnite memory. This means ˜ that there exists a real number t0 > 0 such that the computation of the “transform” f (t. D. Gabor. (Gabor. 1946). Consider for this an audio signal represented by a real function f of one variable (time).1 A Walk in The Physical Universe Our purpose is to obtain a transform that enables us to perform a local computation of the frequency density.
2 The Windowed Fourier Transform One method to obtain a localized modulating function consists in using an auxiliary function g(u) to localize the modulating function e−2iπωu used in the Fourier transform in a certain neighborhood of the time domain: gω. and for each value of t ∈ R.1) This localization operator is illustrated in Figure 4. ω) is called the windowed Fourier transform (WFT).44 CHAPTER 4.1. we obtain the desired localization of the modulating function g(u − t)e−2πiωu . Is the transform f (t. We take a function g(u). (4. f . ω) invertible? ˜ Question 2. ω)? The importance of these questions can be measured from the fact that the invertibility of the Fourier transform is one of its strength. If the function g is localized in the time domain. From the above.1: Modulating function. 4.ω (u)f (u)du = gt. The image of the transform is important because it measures the scope of the representable functions. The underlying ideia of the importance of the inverse transform is that if we are able to obtain a good representation using the transform. its inverse will provide the reconstruction. Two important questions can be posed with respect to the ˜ windowed Fourier transform f (t. We will discuss these two questions in the following sections. . or the short time Fourier transform (STFT). we translate the origin to the point t and multiply the exponential by the translated function g(u − t). t) = = +∞ −∞ +∞ −∞ g(u − t)f (u)e−2πiωu du gt. the deﬁnition of our transform is given by ˜ f (ω. (4. WINDOWED FOURIER TRANSFORM Figure 4.ω .t (u) = g(u − t)e−2πiωu . ω): ˜ Question 1. What is the image of the transform f (t.2) ˜ The transform f → f (t.
˜ Given a function f its windowed Fourier transform f (ω. then f (ω. 4. t)dωdt. It is important to compare the result here with the analogous result for the Fourier transform: The Fourier transform is an isometry of the space L2 (R). In particular its image is the whole space. . ˜(u.t As we did for the Fourier transform. (4. the posed the image of the transform f question consists in characterizing the image set of the windowed Fourier transform.t (u) = g(u − t)e2πiωu . These waveforms are called timefrequency atoms. t). A solution to it can be found in (Kaiser. and integrating in t we obtain +∞ −∞ g(u − t)2 f (u)dt = +∞ −∞ +∞ −∞ ˜ e−2πiωu g(u − t)f (ω. We will not solve this problem here.2 Image of the Windowed Fourier Transform In this section we will discuss the second question posed before: What is the image space of the windowed Fourier transform? We will only give some comments concerning the answer. where ft (u) = g(u−t)f (u). Also.3) can be interpreted into two diﬀerent ways: ˜ 1. we have g(u − t)f (u) = ft (u) = +∞ −∞ ˜ f (ω. or Theorem 4. page 56. THE WINDOWED FOURIER TRANSFORM 45 4. Applying the inverse From (4. equation (4.2. t)e2πiωu dω.3) ω. t) = ft (u).1 of (Mallat. Therefore. Multiplying both members of the equation g(u − t).2. t)dω. 1994). it is possible to show that ˜ It is possible to prove that if f ∈ L ˜ does not cover the whole space L2 (R2 ). ω) The problem of inverting the windowed Fourier transform consists in determining a function ˜ f from its transformed function f (t. t) is a function of two variables. 1998).1 ˜ Invertibility of f (t. We can not divide by g(u − t) to get f (u). ω).4.2. f (u) = 1 g2 ˜ g(u − t)e2πiωu f (ω. That is. It is an equation to compute f from its windowed Fourier transform f (ω. 2. 2 (R). t)L2 (R2 ). because the function g might as well assume zero values. It decomposes the function f as an inﬁnite superposition of localized waveforms gω.2) it follows that f Fourier transform.
3 WFT and Function Representation Since our main focus is the problem of function representation. we have ˜ ˆ f (ω. t) ∈ L2 (R2 ). and localize them on the time domain. How about the frequency localization of f From equation (4. . t) = gω. g is small outside of a small time interval). t). 4.e. The intuition behind this principle is simple: To measure frequencies we must observe the signal for some period of time. Therefore the windowed Fourier transform of a function is deﬁned on the domain (ω. t = gω.t .3 Timefrequency Domain ˜ We have noticed that if f ∈ L2 (R). called the timefrequency domain.e.2. We conclude that if g has good localization properties in the frequency domain (i.t . f .2) which deﬁnes the windowed Fourier transform. g is small outside an interval of frequency ˆ ˜ ω). then f is also well localized ˜? in time. ˆ where the second equation follows from Parseval’s identity. in the sense that we have localized information both in time and frequency domain. then f (ω. In simply terms the statement of this principle is: We can not obtain precise localization simultaneously in the time and frequency domains.46 CHAPTER 4.3) would give good discrete timefrequency atomic representations. t). Is it possible to obtain a function representation using the windowed Fourier transform? In fact since the windowed Fourier transform has good localization properties in the timefrequency domain. a natural question to be posed now would be: Question 3.2) we know that if g is well ˜ localized in time (i. This limitation comes from a general principle that governs the timefrequency transforms. we should expect that the discretization of equation (4. How precisely can we localize the information about f in the timefrequency domain? An answer to this question is given below.1 The Uncertainty Principle From the previous section we could conclude that ﬁner and ﬁner analysis of a function f ∈ L2 (R) could be obtained by using window functions g with very good localization properties on the timefrequency domain (ω. then the transform f is also localized in frequency. Unfortunately there is a limit to the localization precision in the timefrequency domain. The more precision we need in the frequency measurements the larger the time interval we have to observe. WINDOWED FOURIER TRANSFORM 4.3. 4. Therefore the windowed Fourier transform enable us to analyze the function f in the timefrequency domain. From the deﬁnition of the windowed Fourier transform (4. This results is completely in accordance with the problem we have discussed before: Detect frequencies of the function f . This is the uncertainty principle which will be discussed now.
The same reasoning applies for frequency localization. the coordinates on the timefrequency domain are not of great importance to the analysis of functions. This rectangle is called uncertainty window or information cell of the transform. g ˆ then the averages of g e g are computed by ˆ t0 = +∞ −∞ tg(t)dt. It follows from the equation of Plancherel that ˆ2 = 1. Note that if g is well localized in frequency (Ω small) then T ≥ 1/4πΩ can not be small. unless for the fact that they are used to measure the dimensions of the uncertainty windows. we will suppose that the norm L2 (R) of the window function g is 1. The localization of the signal in the timefrequency domain (ω. we have to deﬁne precisely the notion of “information localization” of a function. the uncertainty principle states that 4πΩT ≥ 1. The size of the localization interval of g and g is given the standard deviation ˆ T2 = and Ω2 = +∞ −∞ +∞ −∞ (t − t0 )2 g(t)2 dt. From the uncertainty principle. g respectively. therefore g does not have good localization in time. (ω − ω0 )2 ˆ(ω)2 dω. . g2 = 1.4. Or. t) is represented geometrically by the rectangle of dimensions T × Ω. TIMEFREQUENCY DOMAIN 47 Figure 4. and ω0 = +∞ −∞ ωˆ(ω)dω. We may consider g and g as probability distributions. For this. In order to give a more quantitative statement of the uncertainty principle. The uncertainty principle shows that do not exist coordinates in the timefrequency domain. g With the above deﬁnitions. that is.2: Information cells.2). to state this more precisely.3. the area of this rectangle is ≥ 1/4π (See Figure 4.
4) is called atomic decomposition. WINDOWED FOURIER TRANSFORM Figure 4.4) where Ω is discrete. the atom to the right has a good frequency localization (several cycles are encompassed) and a high energy. In Figure 4. The functions gλ are called timefrequency atoms. The associated signal to the atom on the left presents small localization in frequencies and has small energy. (4. given a function f we must obtain a decomposition f= λ∈Ω aλ gλ .4 Atomic Decomposition Our main goal is to provide better representations of a signal.48 CHAPTER 4.3 we depict some atoms. The degree of uncertainty in time and in frequency is indicated by the width and height of the rectangle.3: Atoms in the timefrequency domain. Each atom in this representation constitutes a basic element used to measure the frequency density of the function in a small period of time. We associate a gray color to each atom to indicate its energy in the decomposition. More precisely. The atomic decomposition deﬁnes a representation operator R(f ) = (aλ )λ∈Ω ∈ 2 . Each of these atoms is represented by a rectangle whose sides indicate the measure of localization according to the uncertainty principle. The purpose of introducing the windowed Fourier transform is to obtain representations in the timefrequency domain. 4. and the reconstruction equation (4. or by the coordinates of some of its vertices. The localization of the atom in the timefrequency domain is given by the coordinates of the center of the rectangle. The energy is directly related with the value of the coeﬃcients in the reconstruction equation. . the central atom has better localization of frequencies (complete cycle) and therefore has more energy. and the functions gλ have good localization properties in the timefrequency domain. From the uncertainty principle the area of each rectangle is ≥ 1/4π.
(N −1)s.3) which writes f as a superposition of indexed functions by the time frequency parameters. . . The atoms of the discrete Fourier transform 1. We have N discrete values of frequency 0. WFT AND ATOMIC DECOMPOSITION 49 Given a signal f represented by some ﬁnite atomic decomposition N −1 fN (t) = k=0 aN (k)φN.k ..4(a). where the distribution between the vertical segments is given by the sampling interval. This fact is depicted in Figure 4. therefore the atomic representation of a function using this basis is localized in the frequency domain and has no time localization.e 2πi(N −1)s N constitute an orthonormal basis.k . 2s.4(b). e 2πi N .5 WFT and Atomic Decomposition The natural way to obtain atomic decompositions of the function f using the windowed Fourier transform would be using the inverse transform (4. . (a) (b) Figure 4. and the corresponding energy component aN (k)φN. .4: (a) Point sampling.. we have an exact localization in time and no localization in the frequency domain. . s. this representation depicted by the corresponding rectangles of each atom φN.4.5.. (b) Fourier sampling.k (t). The representation in the timefrequency domain is illustrated in Figure 4. . This equation gives us an indication that by discretizing the time and frequency parameters we obtain representation/reconstruction methods for functions f ∈ L2 (R). These atoms have no time localization. 4. In the case of representation by uniform sampling.
and discrete values of frequency ω = mω0 .ω0 = {(mt0 .n . We ﬁx t0 and ω0 and we take discrete values of time t = nt0 . m ∈ Z.5: Uniform lattice in timefrequency domain. WINDOWED FOURIER TRANSFORM Figure 4. In the timefrequency domain we have the uniform lattice ∆t0 . n ∈ Z}. (4. we cover the whole timefrequency plane.n (u) = ei2πmω0 u g(u − nt0 ). This fact is illustrated in Figure 4. gm.5) which is called discrete windowed Fourier transform (DWFT).n f.n gm. Geometrically.n )m. of the function f . If this family constitute a frame we can obtain a representation f → ( f. this result means that when we position the information cells of the timefrequency atoms gm. n.n = +∞ −∞ f (u)g(u − nt0 )e−2πimω0 u du. The discrete windowed Fourier transform uses a countable family of time frequency atoms gm.6 for two distinct discretizations of the domain. Now the crucial question to complement the above results is: .5 ˜ In this case we write the transform f in the form ˜ fm. nω0 ) . In order to achieve this we should look for discrete versions of the windowed Fourier transform.n∈Z .50 CHAPTER 4.n .n on the vertices of the lattice. gm. as described in Chapter 2: f= m. m. f can be reconstructed from this representation using the reciprocal frame gm. depicted in Figure 4. Moreover.
WFT AND ATOMIC DECOMPOSITION 51 Figure 4. 2. with two impulses of order 3 for time values of t0 = 0. Does there exist frames gm. Consider the signal deﬁned by the function f (t) = sin(2π516. The graph of its Fourier transform is depicted in the image on the right.n do not exist. A construction of such a tight frame is found in (Daubechies et al. We will give a brieﬁng of the results here: 1. 1] 0 if x < 0 ou x > 1. If t0 ω0 > 2π. We will give some examples. The graph of the signal is shown in the image on the left of Figure 4. frames do exist but they do not have good localization properties in the timefrequency domain. 1992). then there exists tight frames with good timefrequency localization properties. If t0 ω0 < 2π.12Hz e 2967.74Hz. if we take g(x) = 1 if x ∈ [0. with several related references. .05) + δ(t − 0.42). according to the illustration on Figure 4.42s. From the point of view of timefrequency. In fact. Example 7 (Sines with impulses). can be found in Chapter 4 of (Daubechies.5.74t) + δ(t − 0. then frames gm. The same happens if we take g(x) = sen πx/πx.4. Question 4. 3.6. the atomic decomposition of a function using the discrete windowed Fourier transform.12t) + sin(2π2967. If t0 ω0 = 2π.6: Decompositions of the timefrequency domain using the DWFT.05s and t1 = 0. . then gm. 1986).n with good localization properties in the timefrequency plane? A detailed analysis of this question. gives an uniform decomposition in rectangles.n (x) is a basis. This signal consists of a sum of two senoids with frequencies 516.7.
5 0. 32 samples) in the decomposition shown in (b) we used a Hamming window of width 64 (64 samples).8: Hamming window.2 0.7 0. Moreover several information cells show up which are not directly related with the dominating frequencies of the signal. . we use a Hamming window with diﬀerent sizes.3 0.8 0.7: Signal and its Fourier transform.8.9 1 Figure 4. but the information cells are not able to distinguish the two impulses.2 0.e.9 0.3 0. 1 0.1 0 0 0.52 CHAPTER 4. For this. WINDOWED FOURIER TRANSFORM 5 2000 1800 1600 4 3 1400 2 1200 1000 1 800 0 600 400 −1 200 −2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 0 0 500 1000 1500 2000 2500 Figure 4.4 0.9 show the information cells of the atomic decomposition of the signal f using the discrete windowed Fourier transform.7 0.4 0. A Hamming window is deﬁned by a cosine curve a + b cos t for convenient values of a and b. The analysis of this signal using the Fourier transform was done in Chapter 2. (b) and (c) of Figure 4. The two sine waves also have been detected but the localization of their frequencies is not good.6 0. The images (a). and in (c) we have used a Hamming window of width 256.1 0. In the decomposition shown in (a) we have used a Hamming window of width 32 (i.8 0.5 0. Our goal here is to revisit this analysis using the windowed Fourier transform. An analysis of the decompositions on the ﬁgure shows us the following: • In (a) we see that two impulses were detected with a good localization in time.6 0. • In (b) we have a better localization of the sine frequencies. Its graph is shown in Figure 4.
we conclude that a good atomic decomposition of this signal should have timefrequency atoms as illustrated in Figure 4. we can obtain satisfactory results in the analysis of the signal using the windowed Fourier transform.45 0. Figure 4.1 0.5 0 0 0. and the two impulses. Since the signal has low frequencies close to the origin t = 0 and they increase arbitrarily as the time increases. thus if a signal has frequencies which varies between diﬀerent orders of magnitude.1 0.1 0.05 0.35 0.3 0. The above example shows that making an adequate adjustment of the size of the window.25 Time 0.4.35 0. Consider the signal deﬁned by the function f (t) = sin2 (t2 ).10(a). Example 8 (Quadratic chirp signal). We will give an example illustrating this fact. The graph of the function f is shown in Figure 4.4 0.05 0.45 0.45 (a) (b) (c) Figure 4. It should be noticed . we would not be able to distinguish them.2 0.4 0.15 0.3 0.10(b). it should be remarked that if the two pulses were close together.2 0. it is very diﬃcult to obtain an adequate width of a window that is able to detect all of the frequency values. Now we will give an example of a signal which is very diﬃcult to be analyzed using the windowed Fourier transform.5 0 0 0. Also it should be noticed that most of the information cells on the ﬁgure are related with the relevant frequencies of the signal.15 0.3 0. The results are much better than those of the analysis using the Fourier transform in the previous chapter.05 0.5.9: Atomic decomposition using windowed Fourier transform. Nevertheless.2 0.11 shows two atomic decompositions of the signal f using the windowed Fourier transform.25 Time 0.15 0. The idea of constructing such an example is very simple: The information cells of the windowed Fourier transform have constant width. The frequencies of this signal have a quadratic growth along the time. Nevertheless we must take into consideration that the signal that we have used is very simple: it contains only two distinguished frequencies of the sine waves.35 0. WFT AND ATOMIC DECOMPOSITION 53 4000 4000 4000 3500 3500 3500 3000 3000 3000 2500 Frequency Frequency 2500 2500 Frequency 2000 2000 2000 1500 1500 1500 1000 1000 1000 500 500 500 0 0 0. In both we have used Hamming windows of diﬀerent sizes.4 0. • In (c) we have a good localization of the two impulses in time and good localization of the sinusoidal frequencies.25 Time 0.
1 0 0 0.1 0.5 0 0 0. Moreover the information cells corresponding to the frequencies of the signal do not posses good localization properties.25 Time 0. D.2 0 −0. as we predicted.2 0.10: (a) Quadratic chirp signal. that the relevant frequencies are detected and are correlated along a parabola.2 0.7 0. 4.8 0. Chirp Quadratico 0.25 Time 0. As we will see.3 0. 1946).5 0.2 0. (b) Ideal atomic decomposition of the signal.8 −1 0 Frequency 0. Gabor.1 0.3 0.45 0.6 Comments and References The windowed Fourier transform was introduced in the literature by D.4 0. better atomic decompositions will be obtained.6 0.2 0.6 0.4 500 1000 1500 2000 2500 3000 3500 4000 4500 0.9 0.11: Timefrequency decomposition of chirp signal using DWFT. (Gabor.45 0.6 0.3 0.4 0.5 Time 0. Gabor used Gaussian windows.4 0. 4000 4000 3500 3500 3000 3000 2500 Frequency 2500 Frequency 2000 2000 1500 1500 1000 1000 500 500 0 0 0.05 0.8 0.6 −0. Nevertheless several other information cells appear that are not related with relevant frequencies of the signal. The problem of representation and reconstruction of signals will be revisited with this transform. WINDOWED FOURIER TRANSFORM 1 1 Phase plane: CP Best Basis.05 0.15 0.4 0. for this reason the windowed fourier transform with gaussian windows is also called Gabor transforms on the literature.9 1 (a) (b) Figure 4.54 CHAPTER 4.4 −0. .3 0. The above example explicitizes the limitation of the windowed Fourier transform: It uses a ﬁxed window size.7 0.1 0.35 0.15 0. In the next chapter we will introduce the wavelet transform in an attempt to ﬁx this problem.35 0.2 −0.8 0.5 Figure 4.
Grossmann. Y. of the IEEE. and measure theory. In particular it generalizes the concept of frames for nondiscrete frames. 1998. 1992.. Boston: Birkhauser. Kaiser. The study of the windowed Fourier transform in (Daubechies. A. Philadelphia. J. D. . J. Phys. A Friendly Guide to Wavelets. Painless nonorthogonal expansions. S. the only drawback is that it requires more familiarity of the reader with techniques from functional analysis. 1994. Ten Lectures on Wavelets. Math. Nevertheless frames for the windowed Fourier transform is discussed along with the problem of wavelet frames. 1946. Bibliography Daubechies. 1992) is quite complete. and along with the equations of the resolution of the identity provides elegant proofs of several results. Theory of Communication. & Meyer. A Wavelet Tour of Signal Processing.. 1986. Daubechies. Gabor. 1994) contains useful information with detailed proofs. Academic Press. G. Mallat. 429–457.BIBLIOGRAPHY 55 The reference (Kaiser. Ingrid. 1271–1283. PA: SIAM Books. I. This could bring some diﬃculties for those with less experience..
.
Therefore we must use a modulation function which does not have a ﬁxed width. To achieve this we start from a function ψ(t) as a candidate of a modulation function.1.Chapter 5 The Wavelet Transform In this chapter we will introduce the wavelet transform with the purpose of obtaining better representation of functions using atomic decompositions in the timefrequency domain. and we obtain a family of functions from ψ by varying the scale: We ﬁx p ≥ 0 and for all s ∈ R. we will have problems in the signal analysis: • If the signal details are much smaller then the width of the window. and analyzes the signal from the point of view of this scale. s = 0.1 The Wavelet Transform The windowed Fourier transform introduces an scale (the width of the window). • If the signal details are larger than the width of the window they will not be detected properly. The modulation of the function ψ by the factor 1/s2 . In terms of frequencies. and as s increases the frequency of ψs decreases. we must deﬁne a transform which is independent of scale. This fact is illustrated in Figure 5. s s s (5. Moreover the function must have good time localization. ψs has high frequencies. The scale is deﬁned by the width of the modulation function. we deﬁne 1 u u ψs (u) = s−p ψ( ) = p ψ( ). increases its amplitude when the scale s decreases and viceversa. 57 . To solve this problem when we analyze a signal using the windowed Fourier transform. This transform should not use a ﬁxed scale. we can state: For small scales s.1) If ψ has width T (given as the standard deviation as explained in Chapter 3) then the width of ψs is sT . we will have a problem similar to the one we faced with the Fourier transform: The details will be detected but the transform will not localize them. but the scale should vary. 5. If the signal has important frequency details outside of the scale.
t) invertible? ˜ Question 2.1. 5.t  = ψ.t (ω) = s1−p e−2πiωt ψ(sω). . Now we can deﬁne a transform on L2 (R) in a similar way that we deﬁned the windowed Fourier transform.t ∈ L2 (R). t) = ψs.t (u) = ψs (u − t) = s−p ψ( 1 u−t u−t ) = p ψ( ).4) By deﬁnition we have Moreover.t as modulating functions. By taking p = 1/2. and ψs. As we did for the windowed Fourier transform. (5. we can pose the following questions concerning the wavelet transform: ˜ Question 1.2) Note that if ψ ∈ L2 (R).58 CHAPTER 5. using functions from the family ψs.t . f = ψs. Is the wavelet transform f (s.t (u)du = ψs. More precisely. THE WAVELET TRANSFORM Figure 5. t)? In the previous chapter we have explained the importance of these two questions for function representation using timefrequency atoms. (5. ˆ ˆ ψs. What is the image of the wavelet transform f (s.1 Inverse of the Wavelet Transform ˜ ˆ ˆ f (s. we have ˜ f (s. f .t . (b) s = 1. For this we deﬁne for each t ∈ R the function ψs.3) This transform is known by the name of the wavelet transform. Analogous to what we did with the windowed Fourier transform of a function.1: Scales of a function: (a) s < 1. we need to localize each function ψs in time.t . s s s (5. f . t) = +∞ −∞ f (u)ψs. (c) s > 1. then ψs. we have ψs.t 2 = s1−2p ψ2 .
t (u)ψs. The wavelet transform. similarly with the windowed Fourier transform. A natural question consist in computing the image of the transform. But we can not ˆ simply divide the above equation by ψ. t) of two variables. Multiplying both ˆ sides of (5. takes a function ˜ f ∈ L2 (R) into a function f (s. and making some computations we obtain the result below: Theorem 4. because it might have zero values. The function f can be recovered from its wavelet transform. 5. it will lead us to obtain good representations by atomic decompositions of the function f .7) ˆ From the knowledge of f we can obtain f using the inverse transform. We have seen that the second interpretation is of great importance because. this reference brings a proof that the wavelet transform deﬁnes an isometry over its image.1. page 69. as in the case of the windowed Fourier transform. .9) This theorem answers the ﬁrst question posed at the end of the previous section: The wavelet transform is invertible and equation (5. The function f can be decomposed as a superposition of the timefrequency atoms ψs.9) of the inverse wavelet transform in two distinct ways: 1. t) = s1−p +∞ inf ty ˆ ˆ e2πiωt ψ(s.1. we can read equation (5.6) ˆ ˆ = s1−p F −1 ψ(sω)f (ω) .5. Applying the Fourier transform to both sides of the equation we obtain +∞ −∞ ˜ ˆ ˆ e−2πiωt f (s. (5. ω)f (ω)dω (5. If ψ satisﬁes the condition C= then f (u) = 1 C R2 +∞ −∞ ˆ ψ(u)2 < ∞. (5. As we did with the windowed Fourier transform.t (u).9) reconstructs f from its wavelet transform. Where F indicated the Fourier transform. 1994). t)dt = s1−p ψ(sω)f (ω). u (5. 2.8) ˜ s2p−3 fs.2 Image of the Wavelet Transform In this section we will discuss the second question we asked before about the image of the wavelet transform.t (u)dsdt. THE WAVELET TRANSFORM 59 From this it follows that ˜ f (s. The interested reader should consult (Kaiser.7) by ψ(sω). Besides characterizing the image space.5) (5. We will not go into details of the computation here.
It follows from equation (5. Since ψ(u) ∈ then ˆ lim ψ(u) = 0. THE WAVELET TRANSFORM 5.2 Filtering and the Wavelet Transform ˜ f (s.3(b)). this condition states that the graph of the function ψ must oscillate so as to cancel positive and negative areas in order to have integral zero. Figure 5.8) that appears in the hypothesis of the Theorem 4 is called admissibility condition.3) that deﬁnes the wavelet transform can be written as a convolution product where ψs (u) is deﬁned in (5.t ≈ 0 outside of the frequency band β α ≤ u ≤ .3(a). That is why ψ is named by wavelet. u→0 ˆ ˆ Along with the fact that ψ(0) = 0. A function ψ that satisﬁes this condition is called a wavelet. From the admissibility condition it follows that u→0 ˆ lim ψ(u) = 0. Therefore the graph of ψ has the form of a wave.2). s s . t) = f ∗ ψs (u).4) that ψs. The condition (5.60 CHAPTER 5. L2 (R).1). ˆ ˆ If ψ has a fast decay when u → 0 and u → ∞. Thus the wavelet transform is a linear spaceinvariant ﬁlter. Geometrically. ˆ ˆ If ψ(u) is continuous. then ψ(0) = 0. then ψ(u) is small outside of a small ˆ frequency band α ≤ u ≤ β (see Figure 5.2: Graph of a wavelet. that is. we conclude that the graph of the Fourier transform ψ is as depicted in Figure 5. In this section we will discuss some properties of the wavelet ﬁlter. +∞ −∞ ψ(u)du = 0. ˆ Another important conclusion can be drawn from the above computations. In fact since ψ should have good time localization properties it has a form of a “small wave” (see Figure 5. Equation (5.
Suppose that ∂ n−1 φ (u) = 0.5. u2 φ(u)du = 1. time invariant bandpass ﬁlter ”. R ∂nφ (u). from equation (5. The next two examples are taken from (Kaiser. Consider a function φ of class C ∞ . . ∂un ψ n (u)du = 0. Example 9 (Blur Derivative). 1994). That is. u→+∞ ∂un−1 lim Deﬁning ψ n (u) = (−1)n we have.6) the wavelet transform f does not contain information about f outside of this spectrum interval.3: Fourier transform of a wavelet. In sum. satisfying the conditions φ ≥ 0. uφ(u)du = 0. φ is a probability distribution with average 0 and variance (width) 1. the computations above show that “the wavelet transform is a linear.2. FILTERING AND THE WAVELET TRANSFORM 61 (a) (b) Figure 5. R R R φ(u)du = 1. ˜ Moreover.
13) that is. therefore it represents a lowpass ﬁltering linear timeinvariant ﬁltering operation of the function f . The graph of this function is depicted in the image on the left of Figure 5.t (u) = s−n (u).13) shows that the wavelet transform f (s. This derivative is known in the literature by the name of blur derivative. Therefore we can deﬁne a wavelet transform ˜ f (s. 2π with average 0 and variance 1. Therefore we have ∂ n f (s. We will denote this integral by f (s. We will use a particular case of the previous example to deﬁne a wavelet transform. we have 1 2 ψ 1 (u) = −φ (u) = √ ue−u /2 . Consider the Gaussian distribution 1 2 φ(u) = √ e−u /2 . We know that the nnth derivative of f measures the details of f in the scale of its ˜ deﬁnition.t (u)f (u)du. the wavelet transform of f is the nth time derivative of the average of the function f on scale s. t) = s−n ∂tn (5. φs.11) From equations (5.2) that deﬁnes ψs. t).t (u) = φ( s s From the deﬁnition of ψ n we have that −n ψs. ψ n satisﬁes the admissibility condition (5. which is dependent of the scale s. t) = s−n n ∂t φs.4.8). ∂un ∂tn (5. t) gives the detail of order n of the function f .t (u)f (u)du. f (s. In an analogous way we deﬁne. t) = where R n ψs. s s (We are taking p = 1 in equation (5. in the scale s. Example 10 (The Sombrero Wavelet).t (u)). Therefore.10) 1 u−t n ψs.10) and (5. Using the notation of the previous example.62 CHAPTER 5.t ∂ n φs. (5. 2π . even when the wavelet does not come from a probability distribution.11) it follows that ∂n ˜ f (s. (5.t . 1 u−t ). Keeping this wavelet interpretation in mind is useful. THE WAVELET TRANSFORM That is.t (u) = (−1)n s−n ∂ n φs.t (u) = ψ n ( ). t) ˜ .12) R The above integral is a convolution product of the function f with the function φs. equation (5.
Figure 5. because of the shape of its graph. as the scale s decreases the high frequencies of the chirp signal cos(t3 ) are also detected.5: Signal f to be analyzed(Kaiser. and 1 2 ψ 2 (u) = φ (u) = √ (u2 − 1)e−u /2 . Moreover. consider the signal whose graph is shown in Figure 5. Note that the frequencies associated to the sudden change of the signal at time t = 50 and time t = 150 are detected by the wavelet transform. From the previous example it follows that we can use the sombrero function to deﬁne a wavelet transform. From time t = 280.2. and t = 150. We know already that the windowed Fourier transform is not adequate to analyze signals with this behavior.4. Figure 5.5. This signal has high frequencies localized in the neighborhood of t = 50. 2π The function −ψ 2 is known as the “sombrero” function. FILTERING AND THE WAVELET TRANSFORM 63 Figure 5.5. . In this region the signal is deﬁned by the function f (t) = cos(t3 ). the signal has a chirp behavior: a continuum of increasing frequencies. shown in the right of Figure 5. For this.4: Graph of the sombrero wavelet. 1994).6 shows the graph of the signal and the graph of the wavelet transform for 5 distinct values of the scale s (the scale decreases from top to bottom). We will use this wavelet to illustrate the ﬂexibility of the wavelet transform in analyzing frequencies of a signal.
0 Positive values of m produce scales larger than 1. Therefore the discretization of the scaling factor is simple: We ﬁx an initial scale s0 > 1.64 CHAPTER 5. m. nω0 ). t) ˜ on this lattice.n . composing two consecutive scalings is attained by multiplying each of the scale factors. How to discretize the time? Initially we should observe that we must obtain a lattice ˜ in the timescale domain in such a way that when we sample the wavelet transform f (s. with minimum redundancy. As the wavelet width changes with the scale. m ∈ Z. . and negative values of m produce scales less than 1. we must correlate the time with the scale discretization: As the scale increases the width of the wavelet also increases. The wavelet transform is deﬁned on the timescale domain. 1994). and we consider the discrete scales sm = sm . On the other hand.ω0 ) = {(mt0 . because of the constant width of the timefrequency atoms.6: The wavelet transform(Kaiser. THE WAVELET TRANSFORM Figure 5. that is. we are able to reconstruct the function f from the timescale atoms fm. n ∈ Z}. therefore we can take samples further apart in the time domain.3 The Discrete Wavelet Transform In the study of the windowed Fourier transform the timefrequency domain was discretized using an uniform lattice ∆(t0 . 5. How to discretize the timescale domain in such a way to obtain a discrete wavelet transform? We known that the scaling operation acts in a multiplicative way. A natural question is: Question 3.
In this manner. m. To obtain the correct correlation between the scale and time discretization we observe that an important property of the wavelet transform is: The wavelet transform is invariant by change of scales. More precisely.5.7(b). we must observe that the frequency is the inverse of the scale. This statement means that if we make a change of scale in the function f and simultaneously change the scale of the underlying space by the same scaling factor. In order to achieve this goal.t0 = {(sm . THE DISCRETE WAVELET TRANSFORM 65 when the width of the wavelet decreases with a reduction of the scale. when we pass from the scale sm = sm to the scale sm+1 = sm+1 .7(a). n ∈ Z}. 0 0 Example 11 (Dyadic Lattice). we can choose a time t0 and take the length of the sampling time intervals as ∆t = sm t0 . for each scale sm the time discretization lattice is 0 0 tm. Therefore. nsm t0 ) .n = nsm t0 . . The vertices of this lattice are shown in Figure 5. the discretization lattice in the timescale domain is deﬁned by ∆s0 . The vertices of this lattice are shown in Figure 5. We have ∆2. n2m t0 ) . if we take fs0 (t) = s0 then ˜ ˜ fs0 (s0 s. In this way. n ∈ Z}. s0 Finally.3. Invariance by changing of scale constitutes an essential property of the wavelet transform. 0 n ∈ Z. It is important that this property be preserved when we discretize the wavelet. the wavelet transform does not change. m. t). n2−m t0 ) . for a given initial frequency ω0 the lattice will be given by ∆2ω0 . we must increase the frequency sampling. −1/2 f( t ). This lattice is called hyperbolic lattice because it is a uniform lattice in hyperbolic geometry (only the points are part of the lattice). s0 t) = f (s.t0 = {(2−m ω0 . we must also increment the time by the scaling 0 0 factor s0 . m. To obtain a timefrequency lattice. so as to be also valid for the discrete wavelet transform. We will give a very important example of a wavelet discretization using s0 = 2 (dyadic lattice). n ∈ Z}.t0 = {(2m .
˜ The discretization of the wavelet transform f (s. ψs. THE WAVELET TRANSFORM (a) (b) Figure 5.66 CHAPTER 5. ψm.8. .n = f.t (u) in the timescale lattice is given by ˜ fm. t) = f.7: (a) Timescale lattice. Figure 5. 5.(b) Timefrequency lattice.n (u) .8: Timefrequency decomposition using wavelets.3.1 Function Representation From the point of view of atomic decomposition the timefrequency atoms deﬁne a tiling of the timefrequency domain in rectangles as shown in Figure 5.
n } is a frame? If we have orthonormal basis of wavelets or a frame.n . A direct. Chapter 3 of (Daubechies. 1992) brings a comprehensive discussion of frames of wavelets. The graph of f is shown in Figure 5. where ψm. Based on the representation theory discussed in Chapter 2. This will follow as a consequence of the theory of multiresolution analysis that we will study in next chapter. The fact that the set generates the space L2 (R) is more complicated. In the Chapters to follow we will discuss the construction of diﬀerent basis of wavelets.3. .5.n (u) = 2−m/2 ψ(2−m u − n).n is easy to proof. of discrete wavelets. The explicit construction of some wavelet frames is given.n . n ∈ Z. The orthonormality of the set ψm. 1992). Therefore we have an orthonormal basis of wavelets. constitutes an orthonormal basis of L2 (R). Example 12 (Haar Basis). There are several directions we could take to answer the two questions above.n } constitutes an orthonormal basis of L2 (R)? Question 7. 1) if x < 0 ou x > 1.n . m. THE DISCRETE WAVELET TRANSFORM 67 where ψm. two natural questions in this direction are: Question 6. Consider the 1 ψ(x) = −1 0 function if x ∈ [0. This function satisﬁes the admissibility condition (5. we know from Chapter 2 that the answer to the two questions posed above are positive. Is it possible to deﬁne a lattice such that the corresponding family {ψm. n ∈ Z an exact representation of the function f ? Question 5. s0 In this context we can pose again the two questions which motivated the process of deﬁning a discrete wavelet transform: Question 4.8).3. 1/2) if x ∈ [1/2. It is possible to show that the set ψm.n ? A positive answer to these two questions would give us atomic decompositions of the function f using a family ψm. Is it possible to reconstruct f from the family of wavelet timefrequency atoms ψm.3.9.15) (5. ψm. proof of this fact is found in (Daubechies. m. Is it possible to deﬁne lattices for which the family {ψm.16) u − nt0 sm −m/2 0 s0 ψ sm 0 −m/2 −m ψ(s0 u − nt0 ).14) (5. Section 1. and long.nt0 sm (u) 0 0 = = (5.n (u) = ψsm . Is the sequence f.
The interested reader should consult (Daubechies. L2 (Rn ). A Wavelet Tour of Signal Processing. or (Mallat. Kaiser. Mallat. Academic Press. The beautiful examples 9 and 10 of this chapter were taken from (Kaiser. page 33. Ten Lectures on Wavelets. 1998. Ingrid. 5. 1994). 1994. S. PA: SIAM Books. 1992). 1998).e.9: Haar wavelet.68 BIBLIOGRAPHY Figure 5. . Bibliography Daubechies. 1992. G. Philadelphia. i. Boston: Birkhauser. A Friendly Guide to Wavelets.4 Comments and References There are several possibilities of extending the wavelet transform to functions of several variables.
As we will see. 69 . The length of the sample interval is called the sampling period. 6. The second property is the relationship of wavelets with multiresolution representation. In a similar manner. This scale should be adequate to understand the diﬀerent details we need. We call sampling frequency the number of samples in the unit of time. The idea of scale is intrinsically related with the problem of point sampling of a signal.Chapter 6 Multiresolution Representation In the introductory chapter we stated that two properties of wavelets were responsible for their applicability in the study of functions.1 The Concept of Scale Our perception of the universe uses diﬀerent scales: Each category of observations is done in a proper scale. One of these properties is the wavelet timefrequency atoms we studied in the previous chapter. it is clear that all of the details of the signal captured in a certain scale will also be well represented when we sample using a higher scale. By changing the scale we can observe or represent more details of the object being represented on the map. Using a small scale we can observe only macroscopic details of the mapped regions. • On the other hand it will be used as a tool to construct wavelets. we are ﬁxing a scale to represent the signal: Details (frequencies) of the signal that are outside of the scale magnitude of the samples will be lost in the sampling process. this problem is intrinsically related to the wavelets. we try to use a scale where the important details can be captured in the representation. A clear and well known example of the use of scales occurs on maps. k > m. when we need to represent an object. On the other hand. 2k . When we sample a signal using a frequency 2j . Multiresolution representation is a mathematical model adequate to formalize the representation by scale in the physical universe. These facts are well translated mathematically by the sampling theorem of ShannonWhittaker that relates the sampling frequency with the frequencies present on the signal. This relationship will be exploited in two diﬀerent ways: • From one side it allows us the use of wavelets to obtain multiresolution representations of functions.
In order that the elements of the representation sequence are close to the samples of f . and the scale is reﬁned.2 Scale Spaces How to create a mathematical model to formalize the problem of scaling representation in the physical universe? The relation between sampling and scaling discussed above shows us the way. 1/2 s s We want the representation sequence ( f.k φj. where F is obtained from f by sampling with a ﬁlter of kernel φj. MULTIRESOLUTION REPRESENTATION 6. constituted by the functions in L2 (R) whose details are well represented in the scale 2j . But at this point we will demand more than that to make things easier: We will suppose that there exists a function φ ∈ L2 (R) such that the family of functions φj. φj.k ) to contain samples of the function f in the scale 2j . Thus. φj. f.k )j. as the scale increases or decreases.k = F (k). For a given integer number j. Note that when j decreases.1) is an orthonormal basis of Vj .k (u) = 2−j/2 φ(2−j u − k). the width of φj. the ﬁlter kernel . This means that more features of f are detected in its representation on the space Vj . Notice that we are using here a process similar to the one we used when we introduced the wavelet transform: We deﬁne diﬀerent scales of φ producing the continuous family φs (u) = The width of φ and φs are related by width(φ) = s width(φs ). This is the Galerkin representation we discussed in Chapter 2. we have demanded that the translated family φj.k . k ∈ Z. A simple and eﬀective way to compute this representation is to obtain an orthonormal basis of Vj . φj. More precisely. j.70 CHAPTER 6. In order to attain this we know from Theorem 2 of Chapter 3 that the representation sequence ( f.k = φ2j (u − k) = 2−j/2 φ(2−j u − k) is an orthonormal basis of Vj . Equation (6. φj. k 1 u φ( ). Also.k also decreases. the width of φs does the same. A simple and eﬀective technique consists in using a representation by orthogonal projection.k : F = f ∗ φj. The representation of a function f ∈ L2 (R) by orthogonal projection in Vj is given by ProjVj (f ) = f. The next step consists in creating a representation operator that is able to represent any function f ∈ L2 (R) in the scale 2j . This means that these functions are well represented when sampled using a sampling frequency of 2j .k∈Z is constituted by the samples of a ﬁltered version of the signal f . (6. j ∈ Z. we create a subspace Vj ⊂ L2 (R).1) is obtained by discretizing the parameter s.k . taking s = 2j .
we must have Vj ⊂ Vj−1 . This can be attained by demanding that φ(0) = 1. With this choice of φ.1.2) In fact. or simply scale space.3) (6.k must deﬁne a lowpass ﬁlter. f (2u) ∈ Vj−1 . Applying successively the Figure 6.2.3). the scaling of the variable of f by 2 reduces the width of f by the factor of 1/2 (see Figure 6.1: Lowpass ﬁlter. f (2j u) ∈ V0 . The graph of φ is depicted in Figure 6.2: Scaling of f by an scale factor of 2. representing a function at scale 2j amounts to sample averages of f over neighborhoods of width 2j . Therefore the details of f go to a ﬁner scale. and only if. a natural requirement is f ∈ Vj if. . condition in (6. we obtain f ∈ Vj if. Given a function f ∈ L2 (R). The space Vj is called space of scale 2j . because ˆ φ(ω) approaches 0 when ω → ±∞. It is very important that we are able to change from a representation in a certain scale to a representation on another scale.6. SCALE SPACES 71 ˆ φj.2). For this we must answer the question: How are the diﬀerent scale spaces related? Since the details of the signal which appear on scale 2j certainly must appear when we represent the signal using a smaller scale 2j−1 . Figure 6. and only if. (6.
If we use a notation with increasing indices · · · V−1 ⊂ V0 ⊂ V1 · · · . and the scale spaces get bigger. our universe of the space of functions.k in equation (6. the scale is reﬁned. contains all of the possible scales. than the base φj. In fact it should be observed that any constant function can be represented in any scale. We deﬁne a multiresolution representation in L2 (R) as a sequence of closed subspaces Vj . of L2 (R).1 A Remark About Notation It is important here to make a remark about the index notation we use for the scale spaces. We will resume them into a deﬁnition to facilitate future references: Deﬁnition 1 (Multiresolution Representation). nevertheless the only constant function that belongs to L2 (R) is the null function. this notation is coherent with the variation of the scale when we pass from one scale space to the other: As the indices decrease.k of the scale space Vj should be constituted by the functions φj. 6. from the fact that φj.2. is an orthonormal basis of Vj .2. all spaces are scaled version of the space V0 . The space L2 (R). which also appears on the literature. From the discussion above. This is rather confusing because it is not in accordance with the notation used when we discretized wavelets. this expression says that the null function is the only function that can be well represented in every scale. We use the notation of decreasing indices · · · V1 ⊂ V0 ⊂ V−1 ⊂ V−2 · · · .k (u) = φ(u − k) is an orthonormal basis of the scale space V0 . This is reﬂected in the relation Vj = L2 (R). because there is no uniformity on the literature.2 Multiresolution Representation The scale spaces and their properties that we studied above deﬁne a multiresolution representation in L2 (R). we conclude that φ0. we have Vj = {0}.k (x) = 2j/2 φ(2j x − k). j∈Z On the other hand. j ∈ Z. j∈Z In eﬀect.72 CHAPTER 6. In particular. satisfying the following properties: . MULTIRESOLUTION REPRESENTATION That is. 6.1).
It can be proved that this lowpass ﬁlter condition follows from (M4).3 A Pause to Think How to interpret geometrically the sequence of nested scale spaces in the deﬁnition of a multiresolution representation? In general. 2j (k + 1)]. Indeed.1] = 0 if x < 0 ou t ≥ 1 1 if x ∈ [0. 6. This is the Haar multiresolution representation. f [2j k. αj > 0. αj ]. Each of the spaces Vj is called scale spaces. Indicating the cutting frequency of these ﬁlters by αj (see Figure 6.k . the orthogonal projection of a function f ∈ L2 (R) in Vj is obtained using a ﬁltering operation of f with the diﬀerent kernels φj. k ∈ Z}.3. the reader might have noticed that we have not ˆ imposed that the scale function φ satisﬁes the condition φ(0) = 1 (as we know. 1) It is easy to see that φ is a scale function of a multiresolution representation. (M2) f ∈ Vj if. this condition guarantees that φ is a lowpass ﬁlter). in this case a very informative visualization of the nested sequence of scale space can be obtained in the frequency domain. visualizing subspaces of some space of functions is not an easy task. that deﬁne a multiresolution representation are not independent.3). . Consider the function φ(t) = χ[0. Moreover. or. and only if. The function φ is called the scaling function of the multiresolution representation. Vj = {f ∈ L2 (R). A PAUSE TO THINK 73 (M1) Vj ⊂ Vj−1 . the projection of a function f on the scale space Vj is given by a function which is constant on the intervals [2j k. . For a proof of all of these facts we suggest consulting (Hernandez & Weiss. Nevertheless. space of scale 2j .6. condition (M5) can be replaced by the weaker condition that the set {φ(u − k)} is a Riesz basis. (M2) e (M5). (M5). In this case. (M5) There exists a function φ ∈ V0 such that the set {φ(u − k). we conclude that each space Vj is constituted by functions whose frequencies are contained in the interval [−αj . = L2 (R). Also. Example 13 (Haar Multiresolution Analysis). k ∈ Z} is an orthonormal basis of V0 . 2j (k + 1)] = constant. . That is. k ∈ Z which deﬁne lowpass ﬁlters. (M3) (M4) j∈Z Vj j∈Z Vj = {0}. . f (2u) ∈ Vj−1 . more precisely. 1996) or (Daubechies. We should notice that conditions (M1). 1992). We will return to this problem with more details in next chapter about construction of wavelets. In fact it is possible to prove that condition (M3) follows from (M1).
(6. Figure 6. Therefore the frequency band increases to an interval [−αj−1 . For each space Vj . we have the representation operator Rj : L2 (R) → Vj .k is the dotted curve in Figure 6. with scale 2j .3. This is illustrated in Figure 6.4 (from (Daubechies. given by the orthogonal projection over Vj Rj (f ) = ProjVj (f ) = f.4: Scale approximations of a function(Daubechies.4) that is. .74 CHAPTER 6. The graph of the spectrum of φj−1. 1992)) we show a function f . we have j→∞ lim Rj (f ) = f.k . MULTIRESOLUTION REPRESENTATION When we go from the space Vj to the space Vj−1 we change from the scale 2j to a ﬁner scale 2j−1 . The scale space Vj−1 consists of the set of all the functions whose spectrum is contained in [−αj−1 . as the scale gets ﬁner we get a better representation of the function f . φj. αj+1 ].k φj. Figure 6.3: Spectrum of the scaling function. 1992). and its representation on the spaces of scale V0 e V−1 of the Haar multiresolution representation. k From condition (M4) of the deﬁnition of a multiresolution representation. αj−1 ].
by adding details. if j = k. Therefore we have Vj−1 = Vj ⊕ Wj . αj−1 ] of the spectrum.5).4) Consider the graph representation of the space Vj on Figure 6. αj−1 ]. whose passband is exactly the interval [αj .4 Multiresolution Representation and Wavelets We have proved that given two consecutive scale spaces Vj ⊂ Vj−1 . In this section we will show that this complementary space is in fact generated by an orthonormal basis of wavelets. 6.4. We indicate this “detail space” by Wj . and very important way. We have seen that the wavelets constitute linear timeinvariant bandpass ﬁlters.5.6) J0 −j Vj = VJ0 ⊕ k=0 WJ0 −k . (6. We remind that the best way to visualize the above equality is by observing the characterization of these spaces on the frequency domain (Figure 6. The above equation says that a function represented on a ﬁner scale space Vj−1 is obtained from the representation on a coarser scale space Vj . We have Vj−1 = Vj ⊕ Wj . we deﬁne Wj as the orthogonal complement of Vj in Vj−1 . For every j ∈ Z. Therefore it seems natural that there might exist some relation between the detail spaces and the wavelets. Therefore by ﬁxing J0 ∈ Z. Figure 6. We will discuss this relation “with details” in next section. to interpret equation (6. It is immediate to verify that Wj is orthogonal to Wk .6. These details can be obtained using a bandpass ﬁltering.5: Frequency band between Vj and Vj−1 . The space Wj contains the details of the signal in the scale Vj . It follows immediately that Wj is orthogonal to Vj .5) . MULTIRESOLUTION REPRESENTATION AND WAVELETS 75 There is a diﬀerent. We see that the space Vj−1 is obtained from the space Vj by adding all of the functions from L2 (R) with frequencies in the band [αj . the orthogonal complement Wj of Vj in Vj−1 could be obtained using a bandpass ﬁlter deﬁned on L2 (R). for every j < J0 we have (see Figure 6.
equation (6. . Basis of W0 . . f (u) ∈ Wj if. we obtain a decomposition of L2 (R) as a sum of orthogonal subspaces. they are represented in the ﬁgure using logarithmic scale. . . Wj . Initially we observe that the spaces Wj inherit the scaling properties of the scale spaces Vj . We will sketch the proof of the theorem because it has a constructive nature which will provide us with a recipe to construct orthonormal basis of wavelets. Figure 6. MULTIRESOLUTION REPRESENTATION We should remark that because of the dyadic scales used in the discretization.k . L2 (R) = j∈Z that is. In fact. In particular. if follows from conditions (M3) and (M4) that deﬁne a multiresolution representation that Wj . For each j ∈ Z there exists an orthonormal basis of wavelets {ψj. All of the subspaces involved in this sum are orthogonals. We have seen that the projection of a function f in each subspace Wj could be obtained using a bandpass ﬁlter. this ﬁltering process can be computed by projecting f on an orthogonal basis of wavelets.6) that the set {ψj.6: Frequency bands between Vj and VJo −j . the frequency bands do not have uniform length.76 CHAPTER 6. in this case.6) For this reason. . it follows from (6. In sum. k ∈ Z} of the space Wj . (6. and only if.5) says that the signals whose spectrum is in the frequency band of Vj . is the sum of the signals with frequency band in VJ0 with those signals whose frequency band are in WJ0 . it suﬃces to show that there exists a wavelet ψ ∈ W0 such that the set {ψ(u − k)} is an orthonormal basis of W0 .k (u) = 2−j/2 ψ(2−j u − k)} is an orthonormal basis of Wj . If J0 . In fact. This fact is a consequence of the theorem below: Theorem 5. WJ0 −1 . f (2j u) ∈ W0 . k → ∞.
7) where hk = φ.n . k (6.k (u) = 2φ(2u − k) in (6. n (6. Given f ∈ W0 . φ−1. we have φ= Since φ ∈ V0 ⊂ V−1 .n . φ−1.) k (6. give us the equation ξ ˆ ξ ˆ f (ξ) = mf ( )φ( ). Substituting φ−1.9) that φ( 2 ) there exists a frequency band which has twice the size of the frequency band of φ(ξ).12) where ν is a periodic function of period 2π. we can rewrite the equation (6.k is an orthonorhk φ−1. 2 2 where 1 mf (ξ) = √ 2 fn e−inξ .9) ˆ ξ Note in equation (6. √ and k∈Z hk 2 = 1.6. Computations similar to the ones we did to obtain the lowpass ﬁlter m0 of the scaling function.8) that the function m0 is a lowpass ﬁlter. it follows from (6. The function m0 is called de lowpass ﬁlter of the scaling function φ. Therefore f= n fn φ−1. mal basis of V−1 .8) Applying the Fourier transform to both sides of this equation. It is not diﬃcult to see that m0 is periodic with period 2π.k .11) After some computations.4. we conclude that f ∈ V−1 and f is orthogonal to V0 . we have ξ ˆ ξ ˆ φ(ξ) = mo ( )φ( ).10) where fn = f. 2 2 where 1 m0 (ξ) = √ 2 hk e−ikξ . (6. and also φ−1. k (6. since V−1 = V0 ⊕ W0 . Characterization of W0 .7 we obtain √ φ(x) = 2 hk φ(2x − k). Therefore. MULTIRESOLUTION REPRESENTATION AND WAVELETS 77 Lowpass ﬁlter and scaling function.11) in the form iξ ˆ f (ξ) = e 2 m0 ξ ˆ ξ + π ν(ξ)φ( ). 2 2 (6. Now we need to characterize the space W0 .k . .
We know that Wj has an orthonormal basis of wavelets {ψj. 6. ProjWj (f ) = f.k is indeed an orthonormal basis of W0 . Rj−1 (f ) = Rj (f ) + k∈Z f. 1992) or (Hernandez & Weiss.12).12) characterizes the functions from W0 using the Fourier transform.k . ψ0. therefore if Rj is the representation operator on the scale space Vj . We know that the wavelet transform is a bandpass ﬁltering operation. k∈Z (6. k ∈ Z}. for all f ∈ L2 (R). We will not give this proof here.5 A Pause. 1996). Details of the above proof can be found on (Daubechies. We need to show that deﬁning ψ by the equation (6. we are loosing details of the signal with respect with its representation in the scale 2j−1 . 2 2 (6.k . from equation (6.15) which is a representation of the signal f in the basis of wavelets of the space Wj . The terms of this representation sequence are obtained using the discrete wavelet transform. ψj.78 CHAPTER 6.13). to See the Wavescape If Vj is the scale space 2j we have Vj−1 = Vj ⊕ Wj . The lost details are computed by the orthogonal projection on the space Wj .14) The second term of the sum represents the orthogonal projection of the signal f on the space Wj and it will be denoted by ProjWj (f ).. Equation (6.k . up to a periodic function ν. we have. it follows that ˆ f (ξ) = k νk e−ikξ ˆ ψ(ξ). . we have f (x) = k νk ψ(x − k). ψj.13) Taking this choice. When we obtain a representation of f in a certain scale 2j . MULTIRESOLUTION REPRESENTATION Choosing the Wavelet. ant the scale spaces allow us to represent a function f in diﬀerent resolutions.k ψj.. A natural choice is to deﬁne a wavelet ψ ∈ W0 such that −iξ ˆ ψ(ξ) = e 2 m0 ξ ˆ ξ + π φ( ).k ψj. (6. and applying the inverse Fourier transform. that is.
. . φj.k . . φj−1 . . 6. . 0 (6. Therefore φ= k∈Z hk φ−1. φ−1. . . is equivalent to ﬁlter the signal using the lowpass ﬁlter deﬁned by the scaling function φ. ProjWj−J (f ) represent the details of the signal lost in each lowpass ﬁlter0 ing. .k φj. Then φ ∈ V0 ⊂ V−1 and φ−1.15) is obtained by ﬁltering f with the bandpass ﬁlter deﬁned by the wavelet transform associated with ψ.6. (6. Rj−J0 . The representation of the details of f in the space Wj . From the relation Vj−1 = Vj ⊕ Wj . .18) . ψJ0 −j . The representation of a signal f in the scale Vj . TWO SCALE RELATION 79 It is useful to interpret the decomposition Vj−1 = Vj ⊕ Wj in the language of ﬁlters. . . the projection on the spaces V−2 e V−4 (on the left) and the projections on the spaces W−2 e W−4 (on the right). . These details are obtained by ﬁltering the signal using the wavelets ψj .6 Two Scale Relation In this section we will revisit some equations we obtained in the computations of this chapter in order to distinguish them for future references. obtained using successive lowpass ﬁltering with the ﬁlters φj .7 shows an image (scale V0 ). The terms ProjWj−1 (f ). summing up both sides and performing the proper cancellations.k . Note that each line of the equation above represents a lowpass ﬁltering and a bandpass ﬁltering of the signal.6.16) states that the original signal f can be reconstructed exactly from the low resolution signal. . . equation (6. . summing up the lost details. we obtain Rj−J0 (f ) = Rj (f ) + ProjWj−1 (f ) + · · · ProjWj−J (f ). . φJ0 −j .n is an orthonormal basis of V−1 .k . Figure 6. Rj (f ) = k∈Z f. we are able to write Rj−1 (f ) = Rj (f ) + ProjWj (f ) Rj−2 (f ) = Rj−1 (f ) + ProjWj−1 (f ) . . Equation (6. (6. ψj−1 .17) with hk = φ. Iterating this equation for Rj−2 .16) The projection Rj (f ) represents a version of low resolution (blurred version) of the signal. Consider an scaling function φ associated to some multiresolution representation. This equation can be written in the form φ(x) = √ 2 k∈Z hk φ(2x − k). .
the sum that deﬁnes the twoscale relations is ﬁnite: φ(x) = √ 2 N gk φ(2x − k). we have that ψ ∈ V−1 . (6.19) or.20) Equations (6.17) and (6. MULTIRESOLUTION REPRESENTATION Figure 6. (6.19) (or equivalently (6.7: Image decomposition on the spaces Vj e Wj . given a wavelet ψ associated with a multiresolution representation ψ ∈ V0 . the support of the scaling function φ is contained in the interval [0.20)). therefore ψ= k∈Z gk φ−1. N ].80 CHAPTER 6. k=0 It is not diﬃcult to see that when this is the case. .k . In several important cases.18) and (6. ψ(x) = √ 2 k∈Z gk φ(2x − k). since V−1 = V0 ⊕ W0 . or scaling relations of the scaling function and the wavelet respectively. and ψ is orthogonal to V0 . are called twoscale relations. Similarly.
PA: SIAM Books. Math. In the literature it carries diﬀerent names: multiscale analysis or multiscale approximation. Hernandez. Mallat. For an exposition of the topics in this chapter using the language of operators in function spaces the reader should consult (Kaiser. Ten Lectures on Wavelets. E. 1992. COMMENTS AND REFERENCES 81 Also. We have opted for multiresolution representation because it ﬁts better to the emphasis we have been given on function representation. 6. 1989). Boca Raton: CRC Press. & Weiss. 1994). 1996). S. The approach is algebraically very clear and clean. Mallat (Mallat. 315. Trans. to have uniqueness of the solution we must impose some kind of normalization (e.20) can be used to obtain the associated wavelet. Bibliography Daubechies. Kaiser. 1989. the two scale equation (6.g. G. then λφ. This choice can be done using the two scale relation (6. A Friendly Guide to Wavelets. A First Course on Wavelets. . 1996. 69–88. Amer. In this way.6. Ingrid. ψ(0) = 1). is also a solution.7 Comments and References The concept of multiresolution representation and its relation to wavelets was developed by S.18).. G. Multiresolution approximation and wavelets. Boston: Birkhauser.. In a similar manner. 1994. λ ∈ R. note that if φ is a solution of the equation deﬁned by the twoscale relation. Soc. A priori. Nevertheless the notation of the indices in the scale space diﬀers from the one used here. The material covered in this chapter can be found on (Hernandez & Weiss. nevertheless a lot of geometric insight is lost. Philadelphia.7. it is possible to construct a multiresolution representation and the associated wavelet starting from an adequate choice of the function φ.
.
we can deﬁne a collection of “diﬀerence” subspaces Wj . In the previous chapters. we will derive the elementary recursive structures which form the building blocks of the fast wavelet transform. That is. We will start with a revision of the multiresolution analysis to show how it naturally leads to recursion. From the above structure. the inverse transform reconstructs the signal from its wavelet representation back to the time (spatial) domain. In this chapter we will see that recursion constitutes the fundamental principle behind wavelet calculations. For this reason. 7. we have seen that a multiresolution analysis is formed by a ladder of nested subspaces · · · V1 ⊂ V0 ⊂ V−1 · · · where all Vj are scaled versions of the central subspace V0 . These two operations need to be performed for analysis and synthesis of every signal that is processed in wavelet applications. Finally. as the orthogonal complement of each Vj in Vj−1 . Conversely. we will present the algorithms for the decomposition and reconstruction of discrete onedimensional signals using compactly supported orthogonal wavelets. it is crucial that the Wavelet Transform can be implemented very eﬃciently. is the basic tool for computation with wavelets. we have a wavelet decomposition of L2 (R) into mutually orthogonal subspaces Wj L2 (R) = j∈Z Wj 83 . Vj = Vj+1 ⊕ Wj+1 As a consequence.Chapter 7 The Fast Wavelet Transform The Fast Wavelet Transform (FWT) algorithm.1 Multiresolution Representation and Recursion The eﬃcient computation of the wavelet transform exploits the properties of a multiresolution analysis. The forward transform converts a signal representation from the time (spatial) domain to its representation in the wavelet basis. Based on these concepts.
a ﬁne scale). we can write fj = fj+1 fj+2 + oj+2 Applying this relation recursively we arrive at the wavelet representation fj = fj+N + oj+N + · · · + oj+2 + oj+1 where a function fj in some Vj is decomposed into its projections on the wavelet spaces Wj+1 . it follows that any function fj ∈ Vj can be expressed as fj = ProjVj+1 (f ) + ProjWj+1 (f ). we work with functions that have some natural scale associated with them. If we denote the projections of f onto Vj and Wj respectively by fj = ProjVj (f ) and oj = ProjWj (f ). any square integrable function f ∈ L2 (R) can be decomposed as the sum of its projection on the wavelet subspaces f= j∈Z ProjWj (f ) where ProjWj (f ) is the projection of f onto Wj .1: Wavelet decomposition of a function f . We assumed above that the process starts with a function fj which already belongs to some scale subspace Vj . . This is not a restriction because we can take the initial j arbitrarily small (i. . since by construction Wj ⊥ Wl if j = l and Vj ⊥ Wj . + oj+1 fj → fj+1 → fj+2 → · · · oj+1 og+1 ··· → fj+N oj+N Figure 7. The wavelet decomposition gives an analysis of a function in terms of its projections onto the subspaces Wj .1. It is also desirable to reconstruct a function from its wavelet representation using a recursive process similar to the decomposition in Figure 7. Wj+N . Note that.e. This fact gives us the main recursive relation to build a representation of a function using the wavelet decomposition. this decomposition of a function is unique once the spaces Vj and Wj are selected. In practice. THE FAST WAVELET TRANSFORM Therefore.1.84 CHAPTER 7. It turns out that. From Vj = Vj+1 ⊕ Wj+1 . and a residual given by its projection onto the scale space Vj+N . since Wj ⊂ Vj−1 . This recursive process can be illustrated by the diagram in Figure 7.
deﬁned as φj. For that purpose we will rely on the two–scale relations.n (x) = 2−j/2 φ(2−j x − n) ψj. n ∈ Z} and {ψj.k (x) gk φ−1.n (x) = 2 −j/2 (7.2 TwoScale Relations and Inner Products We have seen before that the interdependencies between two consecutive subspaces in a multiresolution analysis are formulated by the equations below. the projections operators ProjVj and ProjWj are given by inner products with the elements of these bases ProjVj (f ) = ProjWj (f ) = f.2. The reconstruction gives a mechanism for the synthesis of functions from the wavelet representation. ψj.7. 7. fj+N oj+N → fj+N −1 → · · · oj+N −1 ··· → fj+1 → fj oj+1 Figure 7.n (x)dx φj.n . we would like to avoid altogether computing the integrals explicitly. In fact. To ﬁnd a solution we take advantage of the fact that the recursive decomposition/reconstruction processes requires only projections between consecutive subspaces of the multiresolution ladder.k (x) k (7. are respectively orthonormal basis of Vj and Wj .n φj. the original function can be obtained from the projections.n n (7. TWOSCALE RELATIONS AND INNER PRODUCTS 85 and Vj ⊂ Vj−1 . n ∈ Z}. φj.6) ψ(x) = . called twoscale relations φ(x) = k hk φ−1. Therefore.n = n n f (x)φj.4) f.n = n The problem now is how compute the projection operators ProjVj and ProjWj eﬃciently.n f (x)ψj. We know that the set of functions {φj.n (x)dx ψj.2. as illustrated in Figure 7.n .3) (7.n ψj.1) (7. and the wavelet reconstruction is essentially the reverse of the decomposition.5) (7.2: Wavelet reconstruction process of a function f To implement the wavelet decomposition and reconstruction we need to compute the projections onto the spaces Vj and Wj .2) ψ(2 −j x − n).
k .1) into (7. . Vj and Wj . we have ψj. φj. It is remarkable. we can express the basis functions of the scale and wavelet spaces.1 Decomposition The wavelet decomposition process starts with the representation of a function f in the space V0 . f.7) Similarly. at ﬁner level j − 1. both Vj ⊂ Vj−1 and Wj ⊂ Vj−1 .k and ψj.2k+n (x) hn−2k φj−1.3. Substituting (7.k (x) = 2−j/2 φ(2−j x − k) = 2−j/2 n hn 21/2 φ(2−j+1 x − 2k − n) = n hn φj−1. = 7.10) f. φj−1.k = = f.k (x) = 2−j/2 ψ(2−j x − k) = 2−j/2 n gn 21/2 φ(2−j+1 x − 2k − n) (7. φj.2) into (7. substituting (7. There is no loss of generality here because.n gn−2k f.k into the the inner products.6). we are able to obtain the inner products of f with the basis of Vj and Wj . by changing the units. that from the inner products of the function f with the basis of Vj−1 . This is possible because.n n (7.n (x) Now.k . This can be easily done by inserting the expressions obtained for φj. ψj. since Vj−1 = Vj ⊕ Wj .n n = n hn−2k f. we can always take j = 0 as the label of the initial scale. with the coeﬃcients of its representation in the next coarse scale 2j and with coeﬃcients of its representation in the complementary wavelet space. we showed how to relate the coeﬃcients of the representation of a function in one scale 2j−1 . we need to ﬁnd a way to use the sequences (hn )n∈Z and (gn )n∈Z to help us compute recursively the inner products f. and f. n hn−2k φj−1.k f.n gn−2k φj−1. ψj. THE FAST WAVELET TRANSFORM Using these two relations. at level j in terms of the basis functions of the subsequent scale space Vj−1 . φj−1.8) = n gn−2k φj−1.n (x) n = (7. 7. we have φj.5).3 Wavelet Decomposition and Reconstruction Using the two–scale relations.86 CHAPTER 7. without computing explicitly the integrals! This is the crucial result for the development of the recursive wavelet decomposition and reconstruction method described in this section.9) (7.
7 of Chapter 3 (see Theorem 2).k φ0. if we start with a sequence (c0 ). represented by the coeﬃcients (ck ) of its representation sequence in the scale space V0 . Note also. with the sequences (hn ) and (gn ). .k (x)] = k k c0. . and in the ﬁrst step. . k ∈ Z of the function. and (cJ J ). f 0 is decomposed into f 1 + o1 . k k respectively with f j+1 and oj+1 . The process will be done by applying recursively the following decomposition rule ProjVj (f ) = ProjVj+1 (f ) + ProjWj+1 (f ). consisting of the residual scale component f N and the wavelet components o1 . into two sequences (cj+1 ) and (dj+1 ).10) give the formulas to make the transformation on the coeﬃcients of the bases: cj+1 = k dj+1 = k n n hn−2k cj n gn−2k cj n (7. (d2 ). .13) (7. k k respectively. WAVELET DECOMPOSITION AND RECONSTRUCTION 87 We are given the function f = ProjV0 (f ). and therefore it is necessary to deal with boundary conditions. for j = 0. up to now. In the end we obtain the wavelet representation of f . That is ProjV0 (f ) = [ f.k )k∈Z → (φj+1. The recursion acts on f j . Another important comment is that.11) In case we only have uniform samples f (k). . (7. (dJ J ). where f 1 = ProjV1 (f ) and o1 = ProjW1 (f ). it will n be decomposed into the sequences (d1 ).k φ0. Note that we are computing the coeﬃcients (cj+1 ) and (dj+1 ) by discrete convolutions. The core of the decomposition process splits the sequence (cj ) of scale coeﬃcients associk ated with f j . we work with ﬁnite representations. that we are retaining only the even coeﬃcients for the next step of recursion (because of the factor 2k in the indices). Note that both sets form a basis of the space V j . of scale and wavelet coeﬃcients associated. We can view this process as a basis transformation where we make the following basis change (φj. N . oN . decomposing it into f j+1 + oj+1 . Note that the n/2 n/4 n/2 n/2 decomposition process outputs a wavelet representation with the same number of coeﬃcients of the input representation.12) In this way.9) and (7.3.k )k∈Z . The components oj are set apart. This is a decimation operation. .7. and k transform it into the coeﬃcients of the wavelet representation of the function. the coeﬃcients (ck ) can be computed from the samples by a convolution operation.k . containing n = 2J coeﬃcients. . In practice. . ψj+1.14) with the notation a = (a−n )n∈Z . . This issue will be discussed in more detail later. we implicitly assumed doubly inﬁnite coeﬃcient sequences. φ0.k (7. The goal of the decomposition is to take the initial coeﬃcient sequence (c0 )k∈Z . the process begins with f 0 ∈ V0 = V1 ⊕ W1 . In summary. This fact is well explained in Section 3. . Equations (7.
Since this representation is unique.88 CHAPTER 7.17) (7.k .k . φj−1. n n .5) and (7.15) into (7. .6).20) j−1 The above results provide a reconstruction formula for the coeﬃcients cn from the coefﬁcient sequences of the decomposition at level j.16) Substituting (7.k . . φ−1. At each step. φj−1.n .n . φj−1.19) (7. from j = J. we know that hn = gn = φ0 . n j−1 it combines the sequences (cj ) and (dj ) to recover the intermediate (cn ). We would like to have an exact reconstruction. This is possible because we have just made an orthogonal basis transformation.18) = k cj φj. using the two scale relations (7. k φj−1. we obtain j−1 cn = k cj φj. Therefore φ0 = n φ0 .n k (7.n φ−1. THE FAST WAVELET TRANSFORM 7. j−1 = cn k hn−2k cj + k hn−2k cj k k gn−2k dj k (7.k (x) + k k dj ψj. φ−1.15) j−1 We need to recover the coeﬃcients (cn ) from (cj ) and (dj ) j−1 cn = f j−1 . .n (7. we recall that one step of the decomposition takes a function representation f j−1 and splits into the components f j and oj .2 Reconstruction The reconstruction process generates the coeﬃcients of the scale representation from the coeﬃcients of the wavelet representation. .n (7.n ψ0 . from bottom up.k + k k dj ψj. φ−1. f j−1 (x) = f j (x) + oj (x) = k cj φj.21) = k + gn−2k dj k The reconstruction process builds the ﬁnal representation (c0 ). n ∈ Z}. 1.k (x) k (7. φ−1.3.n + k k Because both φ0 ∈ V−1 and ψ0 ∈ V−1 . In order to bootstrap the recursive relations for the reconstruction process.n dj ψj.16). such that the output of the reconstruction is equal to the input of the decomposition. they can be represented as a linear combination of the basis {φ−1.n and ψ0 = n ψ0 .n φ−1.
splitting the array v0 of size 2j+1 . The code was structured for clarity and simple comprehension. l < n/2. In this section we present the pseudocode. 7.4. 2m . wavelet_fwd_xform(v.. . j >= 0. wavelet_decomp(v. with sizes respectively 1.4 The Fast Wavelet Transform Algorithm The fast wavelet transform (FWT) algorithm is a straightforward implementation of the method described in the previous section. } The procedure wavelet decomp performs the decomposition for just one level.7.1 Forward Transform The input of the algorithm is an array v. for (k = 0. THE FAST WAVELET TRANSFORM ALGORITHM 89 7. for (l = 0. and of equation (7.  w2  w1].j+1)).13) and (7. It consists of the recursive application of equations (7. It uses the global arrays containing the twoscale sequences h and g. for (k = 0. The result is accumulated into the input array v. the origins h0 and g0 of the sequences (hn ) and (gn )). } } copy (w.4. 2m−1 . k++) { w[l] += v[i] * h[k].21) for the inverse transform. with 2m+1 elements. } . k < gn. into two arrays v and w with sizes 2j .. j) wavelet_decomp(v. n/2). of an implementation of the FWT algorithm. m = l + n/2. The main procedure wavelet fwd xform executes the iteration of the basic wavelet decomposition. 0. .e. v. h. g) { for (j = m. and their oﬀset values ho and go (i. There are also global variables associated with these sequences: their number of elements hn and gn. n). k < hn.14) for the forward transform. . k++) { w[m] += v[i] * g[k]. n) { zero (w. m. in Clike notation. pow(2. such that in the end of the decomposition the array v is partitioned into [vN  wN  . } i = (2*l + go) % n. and the number of levels m. 1. . containing the coeﬃcient sequence to be transformed. i = (i+1) % n. l++) { i = (2*l + ho) % n. i = (i+1) % n.
7. l < gn. k < n. therefore they use the same space in the array. i = (i1) % (n/2). l += 2) { v[k] += w[i + n/2] * g[l].go) % 2. THE FAST WAVELET TRANSFORM The procedure uses a local array w that must have. } i = floor ((kgo)/2) % (n/2). n). i = (i1) % (n/2). and copy that copies one array to another. The procedure wavelet inv xform executes the iteration of the basic reconstruction step. Note that the number of elements of vj and wj is 1/2 of the number of elements of vj1.4. j+1)). m = (k . pow(2. j++) wavelet_reconst(v.] with [vj1. k++) { i = floor((kho)/2) % (n/2). for (k = 0. 0..90 CHAPTER 7. for (l = m.o) % 2. m) { for (j = 0. n) { zero(v. } } copy(v. n). wavelet_reconst(w.. and converts it into a scale representation. w. l += 2) { v[k] += w[i] * h[l].2 Inverse Transform The inverse transform takes as input an array containing the wavelet representation. It replaces [vj wj. zero that ﬁlls and array with zeros. m = (k . } The procedure wavelet reconst performs the reconstruction combining the components vj and wj of the input array to reconstruct vj1. for (l = m.]. the same size of v. } . at least.h. l < hn... j <= m. wavelet_inv_xform(v. It calls two auxiliary procedures. in the format produced by wavelet fwd xform.
3 Complexity Analysis of the Algorithm The computational performance of the algorithm is very important. In order to make the decomposition of a coeﬃcient sequence at level j. from Vj into Vj+1 and Wj+1 . Note that. we have a total of 2j (2n − 1) operations for onelevel transformation. BOUNDARY CONDITIONS 91 7.5. and therefore.5 Boundary Conditions Since in practice we work with ﬁnite sequences.4. it is necessary take special care with the computation near the beginning and the end of the sequences (boundaries). we have O = 2j (2n − 1) + 2j+1 (2n − 1) + · · · + 2(2n − 1) factoring out (2n − 1) and noting that m = 2j . In order to compute the coeﬃcients in the boundary regions. then the convolution requires n multiplications and n − 1 additions. Figure 7. we may need coeﬃcients that lie beyond the boundaries of the sequence. the boundary region is determined by size of the twoscale sequences. we have to perform a discrete convolution with the twoscale sequences. Assuming that these sequences have n coeﬃcients. we have to compute 2j new coeﬃcients: 2j+1 for the two components f j+1 and oj+1 .3: Boundary regions for convolution between ﬁnite sequences.7. we obtain: O(m(2n − 1)[1 + 2−1 + 2−2 + · · · + 2−j+1 ]) 1−2−j O(m(2n − 1) 1−2−1 ) O(mn) The above analysis leads us to the following conclusions: • The complexity is linear with respect to the size of the input sequence. . The full decomposition process is applied for j log2 (m) levels. Therefore. for this reason. Since each coeﬃcient requires 2n − 1 operations. • The size of the twoscale sequences have a direct relation with the algorithm complexity. The computation of each coeﬃcient is a convolution operation with the twoscale sequences. 7. Let’s determine what is the computational complexity of the fast wavelet transform.3. This situation is illustrated in Figure 7.
4 (c)).92 CHAPTER 7.4. . we deal with the boundary problem by a simple periodization of the sequence. THE FAST WAVELET TRANSFORM There are some techniques to deal with boundary conditions: • Extending the sequence with zeros (see Figure 7. • Periodization by translation of the sequence with x(N + i) ≡ x(i) (ﬁgure7. • Periodization by reﬂection of the sequence with x(N +i) ≡ x(N −i+1) e x(−i) ≡ x(i−1) (Figure 7.4 (a)). (b) Periodization. (a) (b) (c) Figure 7.4 (b)). • Use basis functions adapted to the interval (we are going to discuss this option later). (c) Reﬂection.4: Options for boundary computation (a) Extending with zeros. In the implementation of the fast wavelet transform algorithm presented in Section 7. This is accomplished using the coeﬃcient with indices i % m.
COMMENTS AND REFERENCES 93 7. 1989). IEEE Transaction on ASSP. Wickerhauser. K. Teukolsky. 1994. Jawerth. An overview of wavelet based multiresolution analyses. rept. 1994. One of the ﬁrst references on the computational implementation of the algorithm appeared in (Press et al. 1994.. S. 1994). 2091–2110. Peters. Tech. Mallat. The book (Wickerhauser.7. William H. Chap. William T. Numerical Recipes : The Art of Scientiﬁc Computing. Bibliography BourgesS venier. 36(3). IRISA – INRIA. Mikael.6 Comments and References The fast wavelet transform algorithm was introduced by Stephane Mallat (Mallat. The code for the fast wavelet transform algorithm presented in this chapter was based in the pseudocode from (Jawerth & Sweldens. Adapted Wavelet Analysis from Theory to Software. 377–412. Wellesley. Mladen Victor. Multifrequency Channel Decomposition of Images and Wavelet Models. including the fast wavelet transform. B.. Wim.6. . . pages 591–606. 13. MA: A. 1989. This algorithm was implemented in (BourgesS venier. SIAM Rev.. & Vetterling. 1996). Cambridge Univ Press. Saul A. 1996. 1994). R alisation d’une biblioth que C de fonctions ondelettes. Press. 1994) describes a complete system for computation with wavelets. & Sweldens.. 37.
Chapter 8
Filter Banks and Multiresolution
The goal of this chapter is to translate the theory of Multiresolution Representation to the language of Signal Processing. Therefore, this chapter takes an important step in the change from the mathematical universe (continuous domain) to the representation universe (discrete domain), in the route to implementation. The reader not familiar with signal processing will ﬁnd basic concepts of linear systems and ﬁlters in Appendices A and B.
8.1
TwoChannel Filter Banks
We are going to study in greater detail a particular case of a type of ﬁlter bank that will be very important to understand the multiresolution representation in the discrete domain. Consider a lowpass ﬁlter L and a highpass ﬁlter H. We deﬁne an analysis ﬁlter bank S using those two ﬁlters together with downsampling operators as shown in Figure 8.1.
L y0n ( 2)
z0n
xn
y1n
H
( 2)
z1n
Figure 8.1: Diagram of a TwoChannel Analysis Filter Bank. Let’s study the operations in the above system: the input signal (xn ) is processed by the ﬁlter L in order to obtain its low frequency components (y0n ), and also by the ﬁlter H to obtain its high frequency components (y1n ). After this ﬁrst level of processing the ﬁltered signals (y0n ) and (y1n ) constitute together a representation of the original signal (xn ), but with twice as much samples. In order to reduce the size of these two signals to the size of the original signal, we should discard terms. This can be achieved by performing a downsampling operation (see 95
96
CHAPTER 8. FILTER BANKS AND MULTIRESOLUTION
Appendix A). The output of the analysis bank, S(xn ), is therefore a representation of the input signal in terms of its low and high frequency components, with the same size of the original signal. A natural question now is: Question 1. Can we reconstruct the original signal (xn ) from the representation S(xn ) produced by the analysis bank? Because the downsampling operator ↓ 2 is not invertible, the answer to this question in neither immediate nor obvious. An attempt of a solution is to deﬁne a synthesis ﬁlter bank ˜ S as shown in Figure 8.2.
z0n
( 2)
L1
xn z1n
( 2)
H1
Figure 8.2: TwoChannel Synthesis Filter Bank More precisely, we apply an upsampling operator to the components of the two channels, lowpass and highpass, and then use a pair of ﬁlters L1 e H1 such that these processed components when combined reconstruct the original signal. ˜ The combination of these two ﬁlter banks S e S, forming a composite analysis and synthesis ˜ ◦ S, is shown in the diagram of Figure 8.3 ﬁlter bank S
L x[n] ( 2) ( 2) L1 x[n]
H
( 2)
( 2)
H1
Figure 8.3: TwoChannel Analysis and Synthesis Filter Bank The ﬁlter bank S is called analysis bank because it produces a representation of the signal ˜ in term of its frequencies. The ﬁlter bank S is called synthesis bank because it reconstruct ˜ the signal from its representation. When the output of the composite ﬁlter bank S ◦ S is the same as the input signal, that is ˜ (ˆn ) = SS(xn ) = (xn ), x we say that the ﬁlter bank has the property of perfect reconstruction. Usually, we have perfect reconstruction, but with a delay of the signal, which is not a problem because we can compensate that with a signal advance within the system.
8.2. FILTER BANKS AND MULTIRESOLUTION REPRESENTATION
97
Matrix Representation Suppose that the ﬁlters L e H are deﬁned as the convolution operators
3
L(xn ) =
k=0
a(k)x(n − k)
and H(xn ) =
3
b(k)x(n − k).
k=0
Then,the analysis bank is given by the matrix
↓L S = −− , ↓H
or, using the results of applying the operator ↓ 2 to a matrix, a(3) a(2) a(1) a(0) a(3) a(2) a(1) a(0) . S= b(3) b(2) b(1) b(0) b(3) b(2) b(1) b(0) It is clear that the perfect reconstruction property is related with the invertibility of the ˜ matrix S. In fact, if S is invertible, we can make the synthesis bank as S = S −1 , and we have exact reconstruction. An important particular case occurs when the linear system S is orthogonal, in this case the matrix of S is an orthogonal matrix. This system has the perfect ˜ reconstruction property and the synthesis ﬁlter, the inverse matrix S is determined by the ˜ = ST. transpose of the analysis matrix S: S Question 2. Why we have discussed twochannel ﬁlter banks with perfect reconstruction? The answer to this question is easy: Given a multiresolution analysis, we know that the associated scaling function is a lowpass ﬁlter and the corresponding wavelet is a bandpass ﬁlter. We are going to show that, in the discrete domain, the multiresolution analysis deﬁnes ˜ a ﬁlter bank similar to the twochannel ﬁlter bank S ◦ S, which has the property of perfect reconstruction. Furthermore, we will describe an algorithm that implements all the operations of the ﬁlter bank (analysis and synthesis) in linear time, proportional to the number of samples of the input signal (note that this is equivalent to the algorithm described in last chapter).
8.2
Filter Banks and Multiresolution Representation
In the mathematical universe (continuous domain) is easy to convince ourselves that a multiresolution analysis deﬁnes a ﬁlter bank. This was already done earlier, but for the sake of reviewing the concepts and notation, we will repeat here:
which contain the details lost when going to a low resolution representation of f. with perfect reconstruction. Given f ∈ L2 (R). . f = Lj (f ) + Hj (f ). we have that ProjVj−1 (f ) = ProjVj (f ) + ProjWj (f ).e. Intuitively the lowpass component gives a representation of f in the scale 2j . properly. . we have e Hj (f ) = ProjWj (f ). . k − 1. Also. Applying successively this decomposition to the component Lj (f ) of low frequencies of the signal.1) states that we can recover the function f exactly from the low resolution component by adding the high resolution components. j. where Wj is the complementary (or wavelet) space. in a scale that cannot capture details). and the bandpass gives the details that are lost when f is represented in this scale Using the notation Lj (f ) = ProjVj (f ). Therefore. he have a (continuous) twochannel ﬁlter bank.98 CHAPTER 8. The decomposition operations of the analysis ﬁlter bank are illustrated by the diagram below Lj (f ) L2 (f ) j Lk (f ) j E Hk (f ) ··· E Hj (f ) E Hj+1 (f ) f The reconstruction operations of the synthesis ﬁlter bank are illustrated by the diagram below Lk (f ) E Lk+1 (f ) E Lj (f ) ··· E f Hk (f ) Hk+1 (f ) Hj (f ) . FILTER BANKS AND MULTIRESOLUTION If φ is the scaling function associated with the multiresolution analysis · · · ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ · · · then Vj−1 = Vj ⊕ Wj . Hn (f ). and ProjWj (f ) represents the bandpass component of the function f both deﬁned by the wavelet transform.1) In summary. ProjVj (f ) represents the lowpass component of the function f . . j (8. n = j k. Lk (f ). we arrive at f = Lk (f ) + Hk (f ) + Hk−1 (f ) + · · · + Hj (f ). The equation (8. together with high frequency components. the signal f is decomposed into a low frequency component (i.
and gn = ψ.3) Then.n . (8.k (x) = n gn−2k φj−1.2) We also have the twoscale relation that gives ψ as ψ(x) = n gn φ−1. ψj−1. we have ψj.n ) ∗ g−n ]. where φ is the corresponding scaling function.5) hn φ−1.4) above is equivalent to f.3 Discrete Multiresolution Analysis All ingredients for the discretization of a multiresolution analysis. . we can rewrite the above equation in the form ψj. (8.k (x) = (↓ 2)[( f. DISCRETE MULTIRESOLUTION ANALYSIS 99 Our objective now is to study the above ﬁlter bank. it follows that f. ψj. we have ψj. ψj−1.2k+n (x) Making a change in the indices. that corresponds to a multiresolution analysis. in the discrete domain. 8. (8. ψj.3.n . are already of our knowledge. φ−1.2).n .4) We can write the expression above using the ﬁlter operators so that the reader can more easily identify it with a ﬁlter bank.k (x) = n gn−2k f.k (x) = 2 = n −j 2 gn 21/2 φ(2−j+1 x − 2k − n) n gn n gn 2 −j+1 2 φ(2−j+1 x − (2k + n)) = n gn φj−1. it is immediate for the reader to verify that equation (8. Indeed.8.k (x) = 2 −j 2 ψ(2−j x − k). If ψ is the wavelet associated with a multiresolution analysis.n (x) = √ 2 n gn φ(2x − n).n . and to revisit the associated ﬁlter bank in the discrete domain. The twoscale relation for the function φ of the multiresolution analysis is given by φ= n (8. Using this relation in (8.
f0 = n c0 φ0. n c1 φ1.4) and (8. we have that f 0 = f 1 + o1 . we are just making a change of basis: From the basis {φ0.100 CHAPTER 8. we can assume that the function f is deﬁned initially in some scale as f 0 . (8.7) As before.n .k (x) = n hn−2k φj−1. φj−1. n f = n 1 o = n 1 From equation (8.k (x) = (↓ 2)[( f. n gn−2k c0 .7) and (8. we have c1 = k d1 k = n n hn−2k c0 . we can write this expression using the convolution and downsampling operators: f. Because V0 = V1 + W1 . Furthermore. n d1 φ1. (8.3.6) From that.7) given the decomposition and reconstruction formulas of the ﬁlter bank associated with a multiresolution analysis.n .n . in the scale V0 . .n . it follows f. FILTER BANKS AND MULTIRESOLUTION Computations similar to the ones we did from the twoscale relation for the wavelet. We are going to study these equations in more detail. Without loss of generality.n .n .n ) ∗ h−n ] (8.4).n }n∈Z . φj.k (x) = n hn−2k f. φj.1 Pause to Review The equations (8.8) 8. This scale is associated with some sampling frequency in which f can be represented by a sequence of samples. ψ1. that is f 0 is the discretization of f in the scale space V0 . n From a linear algebra point of view.n }n∈Z of the space V0 to the basis {φ1. give φj. n These equations allow us to obtain a representation of f in the ﬁner scale V−1 from the initial representation sequence (c0 )n∈Z . φj−1. We suppose that f 0 = ProjV0 (f ).
3. shown in the diagram below: c0 0 c0 1 d ©¡ e e 1 ¡ e d0 ¡ e ¡ ¡ e c0 2 c0 3 d ©¡ e e 1 ¡ e d1 ¡ e ¡ ¡ e c0 4 c0 5 d ©¡ e e 1 ¡ e d2 ¡ e ¡ ¡ e c0 6 c0 7 d ¡ © e e 1 ¡ e d3 ¡ e ¡ ¡ e c1 0 s 2 C d0 w G c1 1 c1 2 c2 0 q 3 d0 s c3 0 c2 1 A C s 2 C d1 w G c1 3 . we can write (c1 ) = L(c0 ) e (d1 ) = H(c0 ). d2 ψ2. c0 .8). . c0 . and. .n . n n n n Similarly.n . DISCRETE MULTIRESOLUTION ANALYSIS 101 Indicating by L the matrix of the operator in (8. In fact. . that is. the pyramid structure is the best form to represent the type of ﬁlter bank in a diagram. c0 ). n n e (d2 ) = H(c1 ). For example.8. f2 = n f 2 ∈ V−2 . therefore Thus. 0 1 2 7 The successive analyses of f 0 by the ﬁlter bank are represented in the inverted pyramid. consider a initial signal given by a sequence of samples with 8 elements. f 1 ∈ V1 = V−2 ⊕ W−2 f 1 = f 2 + o2 . o2 = n o2 ∈ W−2 . n with (c2 ) = L(c1 ). . n c2 φ2.5) and by H the matrix of the operator in (8. f 0 = (c0 . n n We have an analysis ﬁlter bank that is given by the diagram shown below (d1 ) n (d2 ) n (dk ) n ··· H H H (c0 ) n L E 1 (cn ) L E 2 (cn ) L E k (cn ) The above ﬁlter bank is a very particular case of a ﬁlter bank called pyramid bank.
In matrix notation. . . . FILTER BANKS AND MULTIRESOLUTION In the ﬁrst level (base of the pyramid) we have the initial sequence of samples of f . .102 CHAPTER 8. dJ0 −1 . When this matrix is applied to the initial vector (c0 ) it pron duces the complete multiresolution decomposition. . 0 1 m k = J0 .9) −− − − · · · −− − − −  I  I H The order of each block Lk −− Hk with the lowpass ﬁlter Lk and highpass ﬁlter Hk is half of the order of the preceding block Lk−1 −− Hk−1 with ﬁlters Lk−1 e Hk−1 . (8. the corresponding ﬁlter bank matrix in the ﬁrst level has order 4 and is given by 1 1 L 1 1 . The vector resulting from this operation S((c0 )) is n (cJ0 . dJ0 . . J0 − 1. Accordingly. dk . In the second level we obtain the coeﬃcients (c1 ) of the lower resolution scale and the detail n coeﬃcients (d1 ). going through each level n of the pyramid until we reach a sequence with only one element (c3 ) (which gives the average 0 of the signal). = 1 −1 G 1 −1 . . . . . . 0. . the order of the block with the identity matrix doubles its size in each matrix. We continue with the operation of the ﬁlter bank. This product of matrices is the matrix representation of the ﬁlter bank of the multiresolution analysis. d0 ). the ﬁlter is represented by the structure Lk  L1  L Hk  H1  S= (8.10) Example 14 (Haar Multiresolution Analysis). The twoscale relations of the Haar multiresolution analysis are φ(t) = φ(2t) + φ(2t − 1) ψ(t) = ψ(2t) − ψ(2t − 1) For a signal represented by four samples. where dk = (dk . dk ).
4.11) is the synthesis equation (reconstruction) of the ﬁlter bank associated j−1 with the multiresolution: it allows us to obtain the representation sequence (cn ) of the function f in a ﬁner scale. The equation (8.4 Reconstruction Bank Since the scaling function and wavelet are orthonormal bases. from the sequences (cj ) and (dj ) in a lower resolution scale and n n its complement.k . k Therefore j−1 cn = f j−1 .11) where we have used the equation (8. Indeed.8. φj−1.k .n k (8. RECONSTRUCTION BANK 103 √ To make this matrix an orthogonal operator.e. We have f j−1 = f j + dj = k cj φjk + k k dj ψj.3) and (8. the inverse operation for the synthesis ﬁlter bank is given by the adjoint matrix (i. Therefore.k . r −r 1 r −r r r 8. For completeness. obtaining the matrix r r r r . φj−1.n = k cj φj. the ﬁlter bank gives perfect reconstruction. r −r Therefore the matrix of the ﬁlter bank is r r r −r 1 r r . .7) in the last line. the transpose of the conjugate). the operators involved into the analysis ﬁlter bank are also orthonormal. φj−1. we are going to derive the reconstruction expressions. we have to multiply its elements by r = 1/ 2.n + k [hn−2k cj k + = k k j gn−2k dk ] dj ψj. r −r r −r In the next level of scale the matrix is given by r r .
If the associated lowpass and highpass ﬁlters have a kernel with T elements.9). where L = 2j . we have a product of T L nonzero elements in the computation of the ﬁrst level of the ﬁlter bank. The ﬁlter bank has a pyramidal structure. Therefore. We conclude that the computational complexity in the decomposition computation of a multiresolution analysis is linear with the length of the input signal. In the second level we have T L/2 products. and so on. FILTER BANKS AND MULTIRESOLUTION The reconstruction diagram is shown below J (cn0 −j ) E (cJ0 −j+1 ) n E J (cn0 −1 ) E (cJ0 ) n J (dn0 −j ) J (dn0 −j+1 ) ··· J (dn0 −1 ) In summary. is the number of samples of the representation sequence of the input signal. It follows trivially that the computational complexity of the reconstruction ﬁlter bank is the same as the decomposition bank: i. in the third level T L/4 products.5 Computational Complexity When we start with a function at some scale 2j we have j levels in the pyramid. because the analysis operator is orthogonal. which intensiﬁed in the middle of the 80’s (the Haar wavelet is known since the beginning of this century. the reconstruction is given by the conjugate transpose of the matrix S in (8. (dn0 −1 ). (dn0 −j−1 ). but it was not associated with the context of wavelets). The study of ﬁlter banks with perfect reconstruction started in the beginning of the 80’s.104 CHAPTER 8. we have T nonzero entries in each line of the matrix. J . and advanced independently of the studies in the area of wavelets. . In terms of matrices. which correspond to j submatrices in the matrix that deﬁnes the ﬁlter bank of the multiresolution analysis. and the decomposition and reconstruction operations can be computed in linear time relative to the length of the input signal. The number of products is reduced by one half in each level. Therefore. is also linear with the length of the input signal 8. . 8. we reconstruct exactly the representation cJ0 of the signal at the scale 2J0 .6 Comments and References We have shown in this chapter that the multiresolution analysis deﬁnes a ﬁlter bank with perfect reconstruction. the total number of product to compute the analysis operation of the ﬁlter bank is given by TL 1 + 1 1 1 + + · · · + j−1 2 4 2 < 2T L.e. . using a representation of the signal f at a low resolution C J0 −j and the details J J corresponding to the information diﬀerences at intermediate resolutions (dn0 −j ). .
This last reference (Strang & Nguyen. as we described in the previous chapter. PA: SIAM Books. & Weiss. Vetterli. & Nguyen. . 1996. G. Wavelets and Subband Coding. Englewood Cliﬀs. Wellesley. 1992). 1996. & Kovacevic. This algorithm is known in the literature by various names: Mallat algorithm. MA: WellesleyCambridge Press. 1996) is one of the few which adopts a matrix notation. (Vetterli & Kovacevic. Strang. (Hernandez & Weiss. Truong. 1992. Boca Raton: CRC Press. A First Course on Wavelets. 1996). Ingrid. Ten Lectures on Wavelets. 1996) e (Strang & Nguyen. pyramid algorithm or fast wavelet transform. The multiresolution analysis is covered in various books (Daubechies. 1995). 1995.. Wavelets and Filter Banks. Philadelphia. Jelena.BIBLIOGRAPHY 105 We already mentioned in the previous chapter that the concept of multiresolution analysis was introduced by Stephane Mallat. Hernandez. He also developed the recursive algorithm to implement these operations. Gilbert. E. Martin. New Jersey: Prentice Hall PTR. In fact. Mallat also discovered the relation of ﬁlter banks and multiresolution analysis as we described in this chapter. Bibliography Daubechies.
.
because We also know that.1 Wavelets in the Frequency Domain We start by analyzing the dilation equation. 2π]). and m0 ∈ L2 ([0.2) can be rewritten as ˆ ˆ φ(ω) = m0 (ω/2)φ(ω/2) with 1 m0 (ω) = √ 2 hn e−inω n 2 n∈Z hn  ˆ hn e−inω/2 φ(ω/2) n (9. In this chapter we will go over details of this proof with the purpose of obtaining a recipe to construct wavelets. 9.4) < ∞. 9. The function m0 is 2π–periodic.2) (9.3) (9.1 ˆ The Relations of φ with m0 (9.1) Taking the Fourier transform of the twoscale equation √ φ(x) = 2 hn φ(2x − n) n gives 1 ˆ φ(ω) = √ 2 Equation (9.1.Chapter 9 Constructing Wavelets The proof of the existence of a wavelet associated with a multiresolution representation described in Chapter 6 has a constructive ﬂavor. by deﬁnition. and looking at the bases functions φ and ψ in the frequency domain. ∞ −∞ φ(x)dx = 1 107 .
. ψ is deﬁned in terms of φ. √ ψ(x) = 2 gn ψ(2x − n) (9.7) n we get 1 ˆ ψ(ω) = √ 2 or ˆ ˆ ψ(ω) = m1 (ω/2)φ(ω/2) with 1 m1 (ω) = √ 2 gn e−inω n ˆ gn e−inω/2 φ(ω/2) n (9. Details of this proof can be found in (Daubechies. .6) A very important point is to show that this product converges to a function in L2 (R). 1992). This imposes some restrictions on m0 . 9. and arrive at the inﬁnite product formula 1 ˆ φ(ω) = √ 2π ∞ j=1 m0 (2−j ω).2 ˆ The Relations of ψ with m1 Similarly.5) ˆ ˆ Applying equation (9. φ(0) = 1. through m1 . . ˆ We can see that φ is completely characterized by m0 . (9.11) (9.1.9) (9. if we express the two scale relation for the wavelet function ψ in the frequency domain. we get φ(ω) = m0 (ω/2)m0 (ω/4)φ(ω/4) . CONSTRUCTING WAVELETS ˆ hence. .12) ˆ φ(ξ + 2πl)2 dξ . ˆ ˆ Note that. Note also that knowing m0 gives us (hn ). and therefore m0 (0) = 1 (9. This is the ﬁrst important connection between wavelets in the spatial and frequency domains.8) (9. w/4. . .3) recursively for w/2. we use the fact that φ(u − k). . ∞ −∞ φ(x)φ(x − k)dx = = ∞ −∞ 2π 0 ˆ φ(ξ)2 eikξ dξ eikξ l∈Z = δk. 9. the integer translates of φ form an orthonormal basis of V0 . as φ is completely characterized by the sequence (hn ).0 = δk.0 (9.10) where m1 is also 2πperiodic. in the same way ψ is deﬁned in terms of φ through (gn ) in the spatial domain.1.3 Characterization of m0 In order to deﬁne the properties of m0 .108 CHAPTER 9.
So.k and ∞ −∞ ˆ ˆ ψ(ω)φ(ω)eikω dω = 0 (9.3) in (9.1. 9.14) l We can split the sum into terms with even and odd l.4 Characterization of m1 Now.20) . and also Q(π) = 0. in terms of the Fourier series 2π 0 eikω l ˆ ˆ ψ(ω + 2πl)φ(ω + 2πl)dω = 0 (9.18) with m ≥ 1.16) (9. and simplifying.15) substituting (9. to link m1 with m0 we use the orthogonality between φ and ψ. we have 1 ˆ m0 (ω + πl)2 φ(ω + πl)2 = 2π (9. factorizes as) m0 (ω) = 1 + eiω 2 m (9.13). so that the multiplicity of the root of m0 at π is not increased by Q. the constraint that W0 ⊥ V0 implies that ψ ⊥ φ0. we conclude that m0 (π) = 0 This gives us a hint that m0 is of the form (i. via orthonormality of φ. we obtain m0 (ω)2 + m0 (ω + π)2 = 1 This is the ﬁrst important condition characterizing m0 .) We impose Q(0) = 1.5) with (9. and where Q is a 2πperiodic function. and the product has to vanish.13).13) .1. If we put together equation (9. and since m0 is 2πperiodic we have m0 (ω)2 l ˆ φ(ω + 2lπ)2 + m0 (ω + π)2 l ˆ φ(ω + (2l + 1)π)2 = 1 2π (9.19) or.9.e. (Observe that eiπ = −1. when ω = π the ﬁrst term vanishes. with ω = ξ/2.17) Q(ω) (9. More precisely.13) Substituting equation (9. to ensure that m(0) = 1. WAVELETS IN THE FREQUENCY DOMAIN 109 The above equation implies that ˆ φ(ξ + 2πl)2 = l 1 2π (9.
all we need to construct orthogonal scale and wavelet bases.16) and (9. m0 and m1 .25) and choosing the coeﬃcients {gn } as gn = (−1)n h−n+1 . . we obtain after regrouping the sums for odd and even l m1 (ω)m0 (ω) + m1 (ω + π)m0 (ω + π) = 0 (9. therefore m1 can be written using m0 and a function λ m1 (ω) = λ(ω)m0 (ω + π) such that λ satisﬁes λ(ω) + λ(ω + π) = 0 The simplest choice is λ(ω) = eiw .26) (9. ﬁnd the coeﬃcients (hn ) of the representation sequence of m0 .24) (9. we can construct an orthogonal wavelet from a scaling function φ.3) and (9.9) of φ and ψ in terms of. 1) φ(x) = 0 otherwise. that is. ˆ ˆ Substituting in the above equation the expressions (9.28) We conclude that.25) (9. which gives m1 satisfying the above equation m1 (ω) = e−iω m0 (ω + π) ˆ ˆ Note that m1 is deﬁned in terms of m0 .22) This is the second important condition that caracterizes m0 and m1 . using (9. equivalently. ψ(x) = √ 2 n (9.21) for all ω ∈ R. respectively. We also know that m0 (w) and m0 (w + π) cannot be zero simultaneously because of (9. The scaling function of the Haar multiresolution representation is given by 1 if x ∈ [0.23) From the above relations. This also gives ψ in terms of φ ˆ ˆ ψ(ω) = eiω/2 m0 (ω/2 + π)φ(ω/2) (9.27) (−1)n h−n+1 φ(2x − n) (9. CONSTRUCTING WAVELETS hence ˆ ˆ ψ(ω + 2πl)φ(ω + 2πl) = 0 l (9. is to ﬁnd a function m0 satisfying (9. since m1 is trivially deﬁned from m0 .22). Example 15 (Haar Wavelet).110 CHAPTER 9. as expected. or.16).
which gives the Haar wavelet already introduced before (See Figure 9.1.2). The two scale relation (9. . It is easy to see that only φ(2t) and φ(2t − 1) are not disjoint from φ(t).9. therefore only the coeﬃcients of c0 and c1 are nonnull (see Figure 9.1: In (a) we have the graph of φ and in (b) we have the sum of the functions φ(2x) and φ(2x − 1). The above reasoning is correct and promising but taking this trail to get to the wavelets involves a lot of work. An easy computation shows that √ 2 c0 = c1 = . 2 Therefore the two scale equation can be written as φ(x) = φ(2x) + φ(2x − 1).28) for the wavelet in this case is given by ψ(x) = φ(2x) − φ(2x − 1). This equation is illustrated in Figure 9.1). WAVELETS IN THE FREQUENCY DOMAIN 111 From the two scaling relation φ(t) = we have cn = √ 2 k ck φ(2t − k). √ 2 +∞ −∞ φ(t)φ(2t − n)dt. Unfortunately the Haar example does not give a good view of the reality. (a) (b) Figure 9.1: Double scale relation of the Haar scaling function.
CONSTRUCTING WAVELETS Figure 9. 1992). page 139.2 Orthonormalization Method Given a function φ(u) satisfying the two scale relation (6. k ∈ Z. with k ck 2 < ∞. In spite of using a weaker condition on (M5).18). it may happen that the translated functions φ(x − n) do not constitute an orthonormal basis. Moreover there exists the other conditions which must be satisﬁed in order to have a multiresolution representation. The proof of the theorem can be found in (Daubechies. we have stated before that the orthonormality condition (M5) in the deﬁnition of a multiresolution representation could be relaxed with the weaker condition that φ(x − n) is a Riesz basis. is a Riesz basis of the space V0 .29) then φ# (u − k). We can overcome this fact. If {φ(u − k}. If φ ∈ L2 (R) satisﬁes the two scale relation φ(x) = k ck φ(2x − k). is an orthonormal basis of V0 . This problem can be solved using the result from Theorem 7. In fact.2: Haar wavelet. k ∈ Z.112 CHAPTER 9. 9. in general. (9. and we deﬁne a function φ# by 1 ˆ φ# (ξ) = √ 2π ˆ φ(ξ) 2 ˆ k φ(ξ + 2πk) . the task of showing that a set of functions is a Riesz basis is not an easy one. This fact is the essence of the orthonormalization process stated in the Theorem below: Theorem 6. .
(9. A suﬃcient condition for this is that φ satisﬁes R φ(u)du = 0. along with the associated wavelet. according to Theorem 6. Step 3. and the condition (9. We take as a candidate for the scaling function φ the function φ(x) = 1 − x if x ∈ [0.30). Moreover. 9. Step 1.3(a). We start from a function φ which deﬁnes the kernel of a lowpass ﬁlter.18). Step 4. If we have a function satisfying the two previous steps. Now we will use the recipe from the previous section to obtain a multiresolution representation whose scale space approximate a function f ∈ L2 (R) using piecewise linear functions. The graph of φ is shown in Figure 9.31) . ˆ and also that φ and φ have a good decay at inﬁnity.3 A Recipe In this section we will summarize the above results to construct a multiresolution representation.4 Piecewise Linear Multiresolution The Haar multiresolution representation allows us to obtain successive approximations of a function using functions that are constant by parts. 9. We verify if the function φ of the previous step satisﬁes the two scale relation (6. A RECIPE 113 and 0<α< k ˆ φ(ξ + 2πk)2 ≤ β < ∞. then we can obtain an orthonormal basis from φ. 2 2 (9. but φ(u − k) is not an orthonormal basis. The wavelet associated to the multiresolution representation can be computed using equation (6. it is easy to see that φ also satisﬁes the two scaling relation 1 1 φ(x) = φ(2x + 1) + φ(2x) + φ(2x − 1). Step 2. Certainly the function φ satisﬁes all of the conditions in Step 1 of our recipe.9.30) then φ deﬁnes a multiresolution representation.3.13) from previous chapter. 1] 0 if x < 1 or x > 1.
whose transfer function is given by 1 if x ∈ [−π.32) (9. 9. The graph of φ# is shown in Figure 9.29). 3 3 3 2 k∈Z (9.30) is satisﬁed. CONSTRUCTING WAVELETS (a) (b) Figure 9. φ deﬁnes a multiresolution representation. we have ˆ φ# (ξ) = 3 2π ξ 4 sen2 ( 2 ) ξ 2 cos2 ( 2 1/2 ξ2 1+ . k ∈ Z.4(a). π] ˆ φ(ω) = χ[−π. Nevertheless.4(b). The graph of the associated wavelet ψ(x) is shown in Figure 9.33) in (9.3(b).5 Shannon Multiresolution Analysis From classical Fourier analysis we know the lowpass ideal ﬁlter φ : R → R. This relation is illustrated in Figure 9.33) Therefore the condition (9. From Theorem 7.114 CHAPTER 9. Therefore we must apply the orthonormalization process (Theorem 6) to obtain a scaling function that deﬁnes a multiresolution orthonormal basis. φ(ω) = √ ω/2 2π An immediate calculus shows that 1 2 1 w ˆ 2π φ(ω + 2πk2 = + cos ω = (1 + cos2 ( )).32) and (9. . Substituting (9. is not orthonormal.π] = 0 if x < −π or x > π.3: Piecewise linear scaling function. The Fourier transform of φ is given by 1 sin(ω/2) 2 ˆ . ˆ that is the Fourier transform of the scaling function we are looking for. it is easy to verify that the family φ(u − k).
4 2 4 6 sin(πx) .9. but we will not give the details here. This is in fact true.5.5.6 0. SHANNON MULTIRESOLUTION ANALYSIS 115 (a) (b) Figure 9. It is natural to expect that the function φ deﬁnes a multiresolution representation. 1 0.2 0. deﬁned by ˆ ψ = χI .2 6 4 2 0. −π) ∪ (π. .5: Impulse response of the ideal reconstruction ﬁlter. (b) Wavelet of the piecewise linear multiresolution.6. The graph of this ﬁlter is shown in Figure 9. The wavelet ψ associated to Shannon multiresolution representation is the ideal bandpass ﬁlter. Shannon multiresolution representation divides the frequency domain into bands according to the illustration shown in Figure 9. The impulse response of the ﬁlter is given by the function φ(x) = sinc(x) = The graph of this function is shown in Figure 9. It is called Shannon multiresolution representation.4: (a)Scaling function of the piecewise linear multiresolution. 2π].4 0. onde I = [−2π. πx Figure 9.7.8 0.
116 CHAPTER 9. It has an analytic expression given by ψ(x) = −2 The graph is shown in Figure 9.5 6 4 2 2 4 6 0.7: Ideal bandpass ﬁlter. CONSTRUCTING WAVELETS Figure 9. This wavelet is called Shannon wavelet. . π(2x + 1) 0. 1 sen(2πx) + cos(2πx) . Figure 9.5 1 Figure 9.8: Shannon wavelet.8.6: Band decomposition using Shannon multiresolution representation.
Philadelphia.6 Where to Go Now? Up to this point we have seen how to obtain atomic decompositions using wavelets. WHERE TO GO NOW? 117 9. The construction of wavelets satisfying some of the above properties. We will cover this in the chapters to follow. Bibliography Chui.9. In order to amplify the range of possible applications. Devising robust techniques to construct wavelets is a very important research topic. • symmetry. . K. The reader should realize that each application demands bases of wavelets with diﬀerent properties. PA: SIAM Books. Therefore the natural path to follow should point in the direction of devising methods to construct basis or frames of wavelets. The Haar wavelet and the wavelet associated with the piecewise linear multiresolution representation are part of a family of wavelets called spline wavelets. Ten Lectures on Wavelets. 1992.6. we will present a framework for generating wavelets using a ﬁlter design methodology. An introduction to wavelets.7 Comments and References In this chapter we have described how wavelets can be constructed. This family is related to a multiresolution representation that approximates a function f ∈ L2 (R) using piecewise polynomials of degree ≤ n. Academic Press. 1992. Ingrid. Daubechies. • orthogonality with polynomials of degree ≤ n. we should impose additional conditions to the wavelets we construct. 9. In Chapter 10. Certainly. we have studied the role the wavelets in the multiresolution representation. 1992). Among many conditions we could mention: • regularity. These types of wavelets are studied in (Chui. as well as a discussion about the importance of these properties in diﬀerent types of applications is of fundamental importance. • compact support. C. in applications we need to have at our disposal a great abundance of basis of wavelets. we have shown the existence of orthonormal basis of wavelets and we have described a method to construct such basis using multiresolution representation.
.
10. are completely characterized by the twoscale sequences (hn ) and (gn ). as we discussed in the previous chapter. and to compute the fast wavelet transform. ˆ ˆ we conclude that the main part of the energy of φ and ψ is concentrated respectively in the intervals [0. 2π].Chapter 10 Wavelet Design We have seen in the previous chapters that orthogonal wavelets can be deﬁned from a multiresolution analysis. we present an approach to generate wavelets based on a frequency domain analysis. In this chapter. we can use the ﬁlter design methodology for creating wavelets. 10. but it presents two main diﬃculties: ﬁrst. π/2].1 Synthesizing Wavelets from Filters In this section we review the restrictions on the ﬁlter function m0 . From these observations.1 Conjugate Mirror Filters The functions m0 and m1 can be interpreted as the discrete Fourier transform of a pair of discrete ﬁlters H = (hn ) and G = (gn ). it does not give a systematic way to ﬁnd the functions φ and ψ. a direct method to generate new wavelets consists in ﬁnding φ and ψ that satisfy the twoscale relations. it relies on an orthonormalization step which produces inﬁnitely supported bases. π]. and from the deﬁnition of φ and ψ in the frequency domain. φ and ψ. We will exploit the fact that the twoscale sequences are the coeﬃcients of a twochannel discrete ﬁlter bank. π] and [π. It is clear that we need a more eﬀective and ﬂexible method to design wavelets. second. respectively. The function m0 is a low pass ﬁlter for the interval [0. and whose integer translates form orthonormal bases of V0 and W0 . As we discussed in Chapter 9. respectively. the scaling and wavelet functions. Therefore. with the purpose of discovering an expression for it that can be computed.1. This framework provides the basic relations to construct wavelets. Consequently. This simple approach can be used to produce examples of wavelets. and the function m1 is a band pass ﬁlter for the interval [π/2. 119 .
They can be summarized as: Condition 1: m0 (ω)m0 (ω) + m0 (ω + π)m0 (ω + π) = 1 Condition 2: m1 (ω)m0 (ω) + m1 (ω + π)m0 (ω + π) = 0 where. 10. where M is called the modulation matrix. The above means that the wavelet transform essentially decomposes the frequency space into dyadic blocks [2j π. Note that is possible to construct m0 and m1 that are quadrature mirror ﬁlters. In addition.1.1. (See (Daubechies. 2j+1 π] with j ∈ Z. the function m0 must satisfy m0 (ω)2 + m0 (ω + π)2 = 1. Unfortunately.6) will converge to the Fourier transform of a valid φ. that the function m0 must satisfy to generate orthogonal scaling functions and wavelets. But. we only need to determine m0 (ω) such that Condition 1 is satisﬁed. M= m0 (ω) m0 (ω + π) m1 (ω) m1 (ω + π) We will return to this matrix later. There are no simple formulas for the coeﬃcients (hn ) of m0 . These ﬁlters are called conjugate mirror ﬁlters. this would not be possible. we have to impose extra conditions on m0 to ensure that 2 ˆ l φ(ω + 2πl = 1/2π. into m1 (ω) = e−iω m0 (ω + π). WAVELET DESIGN The fact that m0 (0) = m1 (π) and m1 (0) = m0 (π). because their frequency responses are mirror images with respect to the middle frequency π/2 (also known as the quadrature frequency). in the ﬁlter bank theory.22).2 Conditions for m0 First. 1992) for the technical details. In order to guarantee that inﬁnite product formula (9. The goal now. but do not correspond to any functions φ and ψ in L2 (R) as deﬁned by a multiresolution analysis. (9. we will be able to compute (hn ) numerically. in the more general context of biorthogonal wavelets and perfect reconstruction ﬁlters.) 10. using the quadrature mirror ﬁlter construction (see equation (9. let’s revisit the two main conditions. we were able to transform condition 2.3 Strategy for Computing m0 What we would really like at this point is to obtain a closed form expression for m0 . together with the relation m1 (ω)m0 (ω)+ m1 (ω +π)m0 (ω + π) = 0 make H and G a pair of ﬁlters that are complementary. in the ﬁrst equation we expanded m0 (ω)2 = m0 (ω)m0 (ω). Therefore.16) and (9.120 CHAPTER 10. For orthogonal wavelets. . is to go from Condition 1 to a formula for m0 . the orthogonality condition for φ(· − k). Notice that these two condition can be written in matrix form as m0 (ω) m0 (ω + π) m1 (ω) m1 (ω + π) m0 (ω) m0 (ω + π) = 1 0 or M x = y.25) of previous Chapter).
• P is symmetric. Let P (z) restricted to the unit circle r = 1 denote P (ω). is real and nonnegative. . we will make a change of notation.1) = k=−N ak e−ikω Our strategy is to decompose the problem of ﬁnding m0 satisfying Condition 1. In the rest of this chapter. the formulas to compute its coeﬃcients (ak ).1) becomes N P (z) = k=−N ak z k (10. and therefore. because using P . SYNTHESIZING WAVELETS FROM FILTERS 121 Instead of working directly with m0 . how to derive a closed form expression for P (ω). to simplify the exposition. • P is a Laurent polynomial of degree N . even without knowing the speciﬁc expression of P . We deﬁne the function P (ω) as the product m0 (ω)m0 (ω) N P (ω) = l=0 N hl e ilω N n=0 hn e−inω (10. with r = z. factoring P (ω) into m0 (ω)m0 (ω) It turns out that it is better to work with P than with m0 .4 Analysis of P First. equation (10. N . k = −N. the analysis of Condition 1 is much simpler. because m0 has real coeﬃcients. in the next two chapters. and in the more general context of biorthogonal scaling functions.2).e. .e.1. .. . • P has real coeﬃcients ak . we can obtain m0 through the spectral factorization of P (i.1. with both positive and negative exponents. because P (z) = m0 (z)2 (in the orthogonal case). This is apparent in equation (10. respectively in the context of orthogonal scaling functions. From that. taking a “square root”). We express reiω = z. We will see.2) There are many things we can say about P . because m0 (z) multiplies its conjugate m0 (z). 10. and we will concentrate on the details of how to get m0 from P . This strategy makes possible to ﬁnd an explicit expression for P (ω). into two simpler subproblems: 1. where P (ω) = m0 (ω)m0 (ω) (i. where P (ω) = m0 (ω)m0 (ω). to convert from the ω notation to the z notation. . we will assume that P is known. with ak = a−k . 2. we will consider m0 2 . Find a function P (ω) that satisfy P (ω) + P (ω + π) = 1. the functions m0 and m0 can be diﬀerent). So. • P (z) ≤ 0. From P obtain m0 .10.
WAVELET DESIGN Taking into consideration also Condition 1.6 0. There are several methods for this purpose.1.3) The terms with odd coeﬃcients cancel in equation (10. One of the most eﬀective of these methods consists in .1 0.5 Factorization of P The function P (z) is a Laurent polynomial of degree N .1: Halfband Filter 10. it is the autocorrelation ﬁlter of m0 .3) is called a halfband ﬁlter. except for the constant term. A regular polynomial of degree d can be represented in two forms: d B(z) = n=0 bn z n = d (z − zi ) i=1 where bn are the coeﬃcients of B. we will derive a formula to compute these coeﬃcients.3) (and there is nothing we can say about them).5 P (ω + π) Figure 10. a2m = 1/2 0 if m = 0 if m = 0 (10. Since it is symmetric there are N + 1 independent coeﬃcients in P . It gives the power spectral response. Assuming that the coeﬃcients of P are known. we can say more about P . In the next two chapters. The factorization of the polynomial P can be done by ﬁnding the zeros (roots) of P .122 CHAPTER 10. Using the fact that eiw+π/2 = −eiw . the function P which satisfy (10. When it comes from P (z) = m0 (z)2 .4) In the ﬁlter theory.4 0.8 0.2 0.4 0. we can conclude that. we write Condition 1 in the z notation as: P (z) + P (−z) = 1 (10. and zi are the roots of B. and m0 is the spectral factor. the remaining problem is to factor P to obtain m0 .3 0. Figure 10.1 shows a graph of P . P (ω) 1 0.2 0.3) true. all other even coeﬃcients of P have to be zero to make (10. But.
If zi is a root. and −1 the real roots come in duplets rl . 0 bd −bd−2 bd 0 1 0 ··· . rl . 0 where B(z) = det(B − zI) is the characteristic polynomial of B. . SYNTHESIZING WAVELETS FROM FILTERS 123 computing the eigenvalues the companion matrix. The roots of a regular polynomial B of degree d are the eigenvalues of the d × d companion matrix −bd−1 1 B= 0 .1. zj . regrouping these types of roots. we will be able to compute the “square root” of m0 .5) 0 0 . the complex roots will occur in pairs that are conjugate (i.. (or . This ensures that. N N P (z) = i=0 (z − zi ) j=0 2N (z −1 − zj ) z N P (z) = 1 aN 2 (z − zi ) i=0 with aN = 0. we have z N P (z) = 1 aN 2 K k=1 J (z − zj )(z − j=1 2 1 1 )(z − zj )(z − ) zj zj L (z − zk ) (z − zk ) 2 (z − rl )(z − l=1 1 ) rl Once we obtain the 2N roots of z N P (z). . zj . Our knowledge about P will help us to discriminate between the diﬀerent types of roots: i. we have P (z) = P (1/z). then 1/zi is also a root. . Therefore. . The complex roots come in quadruplets zj . ··· −b1 bd 0 0 1 −b0 bd (10..e. With real symmetric coeﬃcients ak .. assigning factors M to the polynomial m0 (z). The coeﬃcients of P are also symmetric. In order to apply the factorization method..10. it is necessary to separate then.. then zk = x − iy is also a root). . when zi is inside the unit circle. Since P has real coeﬃcients. 1/zi is outside (roots with zi  = 1. The factorization will give us the roots of P . if zk = x + iy is a root. Thus. zj . we need ﬁrst to convert the Laurent polynomial P (z) to the equivalent polynomial z N P (z) in regular form. ii. P has 2N roots. in the orthogonal case. and the remaining 2N − M factors to the polynomial m0 (z). on the unit circle have even multiplicity). This polynomial can be factored using any numerical package that implements the eigenvalue method discussed above. . . −1 −1 In summary.
124
CHAPTER 10. WAVELET DESIGN
m0 (z)). In the orthogonal case, one factor is the transpose of the other, but in the biorthogonal ˜ case, this restriction is not imposed, and the dual scaling functions can be diﬀerent. Note that, the polynomial m0 (z) with degree N is not unique even for orthogonal scaling functions. In the next two chapters we will discuss the rules for separating the roots of P , and eﬀects of diﬀerent choices. The function m0 is completely determined after we assign a set of roots from P to it. But, we still don’t have the coeﬃcients (hn ) of m0 , since it is in product form. Given the roots zk of the degree N polynomial
N
m0 (z) = c
k=1
(z − zk )
(10.6)
the coeﬃcients hn can be computed with an iterated convolution. Then, we normalize the coeﬃcients multiplying them by a scaling constant. The coeﬃcients (hn ) give the ﬁlter m0 , and from them we get the coeﬃcients (gn ) of the ﬁlter m1 , using (9.27). Now, let’s summarize the steps required to generate a scaling and wavelet functions using a ﬁlter design methodology: 1. Choose a Polynomial P satisfying Condition 1. Generate its coeﬃcients ak using an explicit formula (see Chapters 11 and 12). 2. Find the roots zk of P by a factorization method. 3. Separate the roots zk into two sets. Assign one set of roots to the polynomial factor m0 and the other set of roots to m0 (or to m0 ). 4. Compute the coeﬃcients hn of m0 from its roots, using iterated convolution. 5. Obtain the coeﬃcients gn of m1 from hn .
10.1.6
Example (Haar Wavelet)
A concrete example will illustrate the wavelet design methodology. We choose the following degree 1 polynomial: 1 1 1 P (z) = z −1 + + z 4 2 4 This is the lowest degree polynomial which satisﬁes Condition 1 P (z) + P (−z) = 1 We convert P to regular form, multiplying by z zP (z) = 1 1 + 2z + z 2 4
10.1. SYNTHESIZING WAVELETS FROM FILTERS
125
This polynomial has one root at z = −1 with multiplicity 2. Therefore P is factored into P (z) = = The lowpass ﬁlter is m0 (ω) = 1 1 −iω 1 =√ + e 2 2 2 1 1 √ + √ e−iω 2 2 1 1 −1 z (z + 1) (z + 1) 2 2 1 + z −1 1+z 2 2
√ with coeﬃcients h0 = h1 = 1/ 2 which gives the Haar scaling function φ(x) = √ 2 1 1 √ φ(2x) + √ φ(2x − 1) 2 2 = φ(2x) + φ(2x − 1)
Figure 10.2 shows a plot of m0 (ω).
1 0.8 0.6 0.4 0.2 0.1 0.2 0.3 0.4 0.5
Figure 10.2: Haar ﬁlter m0 Notice that, since m0 (ω) = 1 + e−iω = 2 e−iω/2 + eiω/2 2 e−iω/2 = cos(ω/2)e−iω/2
ˆ the scaling function in frequency domain φ is the sinc function 1 ˆ φ(ω) = √ 2π
∞ j=1
cos(2−j ω/2)e−i2
−j ω/2
1 sin(ω/2) = √ e−iω/2 ω/2 2π
which is the Fourier transform of the box function φ(x) = 1 for x in the interval [0, 1] (this is the Haar scaling function, as expected). A derivation of this formula can be found in ˆ (Daubechies, 1992), pg. 211. Figure 10.3 shows a plot of φ(ω).
126
CHAPTER 10. WAVELET DESIGN
1 0.8 0.6 0.4 0.2
7.5
5
2.5
2.5
5
7.5
ˆ Figure 10.3: Magnitude of the frequency response of φ(ω) for the Haar function The coeﬃcients of the highpass ﬁlter m1 are given by gn = (−1)n h1−n 1 g0 = √ , 2 The function ψ(x) = is the Haar wavelet. √ 2 1 1 √ φ(2x) − √ φ(2x − 1) 2 2 = φ(2x) − φ(2x − 1) 1 g1 = − √ 2
10.2
Properties of Wavelets
In the last section we have seen how to synthesize wavelets in the frequency domain. The procedure was based on the factorization of a polynomial P (ω). In this section we will investigate how the properties of the scaling and wavelet functions relate to characteristics of P . This connection will reveal the “design variables” that we can control to generate wavelets with desired properties.
10.2.1
Orthogonality
The multiresolution analysis, by deﬁnition, leads to orthogonal scaling functions and wavelets. In this way, the functions {φj,n ; n ∈ Z}, and {ψj,n ; n ∈ Z} are, respectively, orthonormal basis of the subspaces Vj and Wj , for all j ∈ Z. The subspaces Wj are mutually orthogonal, and the projection operators produce optimal approximations in the L2 (R) sense. Orthogonality is very a desirable property, but it imposes severe restrictions on the candidate scaling and wavelet functions. We will see in Chapter 12 that orthogonality can be replaced by biorthogonality, which is less restrictive, giving us more freedom to design wavelets.
10.2. PROPERTIES OF WAVELETS
127
10.2.2
Support of φ and ψ
In this chapter we have been concerned only with compactly supported basis functions. The support width of the scaling and wavelet functions is determined, respectively, by the length of the coeﬃcient sequences (hn ) and (gn ). More precisely, if hn = 0 for n < 0, n > N , then the support of φ is the interval [0, N ]. When ψ is deﬁned as in (9.28), then the support of ψ is the interval [−M, M + 1], where M = (N − 1)/2. The length of the coeﬃcient sequences (hn ) and (gn ) is given by the degree of the polynomial m0 (z). Consequently, the degree of the product polynomial P (z), determines the support width of the associated basis functions.
10.2.3
Vanishing Moments and Polynomial Reproduction
Two related properties of a multiresolution analysis are: the vanishing moments of the wavelet functions; and the ability of the scaling functions to reproduce polynomials. These are probably the most important properties in the design of wavelets. Given a mother wavelet function ψ(x) with p vanishing moments, that is x ψ(x)dx = 0 for = 0, 1, . . . , p − 1. Then, the following equivalent properties are veriﬁed: • The wavelets ψ(x − k) are orthogonal to the polynomials 1, x, . . . , xp−1 . • The combination of scaling functions φ(x − k) can reproduce exactly the polynomials 1, x, . . . , xp−1 . These properties are important in the rate of convergence of wavelet approximations of smooth functions and in singularity detection using wavelets. Vanishing moments are also a necessary (but not suﬃcient) condition for a wavelet to be p − 1 times continuously diﬀerentiable, i.e ψ ∈ C p−1 . In order to make the connection between the number of vanishing moments of a wavelet and the condition imposed on the ﬁlter function m1 (ω), we note that equation (10.7) implies that d ˆ ψ dω =0 (10.8) (10.7)
ω=0
for < p. (This can be veriﬁed expanding ψ in a Taylor series.) But, we know from (9.26) ˆ ˆ ˆ that ψ(ω) = e−iω/2 m0 (ω/2 + π)φ(ω/2), and also from (9.5) that φ(0) = 0, this means that m0 is p − 1 times diﬀerentiable in ω = π. Furthermore d m0 dω for < p. =0 (10.9)
ω=π
128
CHAPTER 10. WAVELET DESIGN
As a consequence of the above, m0 must have a zero of order p at ω = π. Therefore, m0 is of the form m0 (ω) = 1 + eiω 2
p
Q(ω)
(10.10)
with m0 ∈ C p−1 and Q ∈ C p−1 . We already knew that a wavelet should integrate to zero, i.e. it should have at least one vanishing moment. We also knew that m0 should have at least one zero at π, i.e. it should be of the form (10.10) with p >= 1. The zero at π from m0 produces factor ( 1+z )2p in the polynomial P . 2
10.2.4
Regularity
Smooth basis functions are desired in applications where derivatives are involved. Smoothness also corresponds to better frequency localization of the ﬁlters. The local regularity of a function at a point x0 can be studied using the notion of Lipschitz continuity. A function f (x) is C α at x0 , with α = n + β, n ∈ N and 0 < β < 1, if f is n times continuously diﬀerentiable at x0 and its nth derivative is Lipschitz continuous with exponent β. f (x)(n) − f (x0 )(n)  ≤ Cx − x0 β (10.11)
For global regularity, the above must hold for all x0 . Typically, compactly supported scaling functions and wavelets are more regular in some points than in others. They also have noninteger Lipschitz exponents. The regularity of scaling functions and wavelets is diﬃcult to determine. The techniques ˆ for this investigation involve either the estimation of the decay of φ in the frequency domain, or the estimation of the convergence rate of the recursive construction of φ in the spatial domain. The ﬁrst class of techniques is more suitable for a global analysis, while the second class of techniques is better for a local analysis. The regularity of φ and ψ is related with the number of vanishing moments of ψ, and consequently is related with the multiplicity p of the zeros at π of m0 . The regularity is at most p − 1 if we consider integer exponents, and at most p − 1/2 if we consider fractional exponents. We remark that in many cases, the regularity is much smaller than p − 1 because of the inﬂuence of the factor Q. Another important observation is that, for a ﬁxed support width of φ, the choice of factorizations of P that leads to maximum regularity is diﬀerent from the choice with maximum number of vanishing moments for ψ.
10.2.5
Symmetry or Linear Phase
Symmetry is important in many applications of wavelets, such as image processing and computer vision. This property can be exploited in the quantization of images for compression, and in the boundary processing of ﬁnite signals using symmetric extension (i.e. periodicization by reﬂection of the data).
1). it is linear in ω). b) and the horizontal axis. although P is a symmetric polynomial. For example. k=0 N/2 k=0 = e−i 2 ω N We can see that in this case the phase is N/2ω. the one which contributes the least to nonlinearity of the ﬁlter phase. We remark that. except for the case of Haar wavelets.13) where c is a complex constant. it obviously can be factorized into two polynomials that are not symmetric. This means that m(ω) can be written as m(ω) = ceiKω M (ω) (10. In the next chapter. This is because orthogonal ﬁlters require −1 that the roots zj and zj are separated and assigned one to m0 and the other to m0 . Note that. It is easy to see from (9. not necessarily 2πperiodic. (z) a (10. in the factorization of P the roots zj and −1 zj must stay together. that φ will be symmetric only if the coeﬃcients (hn ) are symmetric. where z is the magnitude of z and θ(ω) is the phase angle of z.e. z −1 . the phase is linear because terms with same coeﬃcients can be grouped. we have to compute the nonlinear −1 contribution of the factors (z − zk ) and (z − zk ) to the total phase of m0 .12) The same deﬁnition extends to a complex function m(z).2. Therefore. in order to obtain symmetric (antisymmetric) scaling functions and wavelets. it is possible to design ﬁlters that are close to linear phase and result in basis functions that are least assymmetric. it is necessary to factor the polynomial P into symmetric polynomials m0 (and m0 ).10. For this purpose. Therefore. This can be accomplished by selecting from each pair of roots z. when ﬁlter coeﬃcients ak are symmetric. despite of the orthogonality restriction. and M is a real valued function of ω. if the ﬁlter is symmetric and has an even number of coeﬃcients N/2 m(ω) = k=0 hk (e −ikω +e −i(N −k)ω ) = e −i N ω 2 N/2 hk (eikω + e−ikω ) 2hk cos(kω) . A ﬁlter function m(ω) = k ak eikω is said to have linear phase if θ(ω) is of the form Kω where K is a constant (i. we will demonstrate that symmetry and orthogonality are incompatible properties. The phase angle (or argument) of a complex number z = a + bi is the angle between the vector (a. Nonetheless. The tangent of this angle is the ratio of the imaginary and real parts of z: tan(ω) = (z) b = . This is because ﬁlters with symmetric coeﬃcients have a linear phase response. PROPERTIES OF WAVELETS 129 The symmetry of the basis functions implies in symmetric ﬁlters. Similar expressions hold for the case of symmetric ﬁlters with odd number of coeﬃcients. . as well as for antisymmetric ﬁlters. hk = hN −k for symmetric ﬁlters and hk = −hN −k for antisymmetric ﬁlters. or antisymmetric. Symmetric ﬁlters are called linear phase ﬁlters. This notion becomes more evident if we use the polar representation of a complex number z = zeiθ(ω) . Linear phase ﬁlters have symmetric or antisymmetric coeﬃcients around the central coeﬃcient N/2. We can conclude from the above discussion that.
130 CHAPTER 10. Naturally. In general terms. For example. One important example of wavelet basis with closed form analytic expression is the family of BSpline wavelets. there is always a tradeoﬀ between diﬀerent properties relative to their importance in a given application. When the scaling function φ(k) = δk . orthogonal wavelets cannot be symmetric.2. the number of vanishing moments depends on the multiplicity of the zero at π of m0 . as in any design process. the support width is proportional to the number of ﬁlter coeﬃcients and consequently small support requires a lowdegree m0 . To make things even more complicated. number of vanishing moments. for k ∈ Z. division by a power of two corresponds to shifting bits. and imply in conﬂicting requirements on the ﬁlter function m0 . Nonetheless. . For instance. Below we discuss a few of them. Unfortunately. and consequently a large number of vanishing moments requires a high degree m0 . we also have to increase the degree of the second. • Rational Coeﬃcients: Filter coeﬃcients that are rational numbers make computer implementation more eﬃcient and precise. • Interpolation: The interpolation property makes it trivial to ﬁnd the values of the scaling coeﬃcients from samples of the function. If we increase the degree of the ﬁrst. there are two main classes of wavelets that combine some of the above properties in the best way. and regularity. the coeﬃcients cj =< f. m0 is of the form 1/2(1 − eiω )n Q(ω). not available. The reason is that diﬀerent properties may be incompatible with each other.6 Other Properties There are several other properties that are relevant to applications. we can conclude that it is not possible to achieve all of them at the same time.3 Classes of Wavelets From our discussion of the various properties of wavelets. there is a great interdependency among the different requirements. 10. These functions are deﬁned indirectly through the ﬁlter coeﬃcients (h) and (g).1 Orthogonal Wavelets Orthogonal wavelets are the best choice in some cases. On the other hand. It is possible to design orthogonal wavelets with more or less assymmetry. Other properties that are important for orthogonal wavelets are: support width. • Analytic Form: The analytic expression for the scaling function and wavelet is. It is even better if the coeﬃcients are dyadic rational numbers. 10. the second factor is necessary to guarantee orthogonality. While the ﬁrst factor controls the vanishing moments.3. In this case. in general. WAVELET DESIGN 10. except for the Haar case. on one hand. but are only achievable by some types of wavelets. φj > of the projection ProjVj (f ) of f on Vj . are just k the values f (2j k) of f sampled on a grid with spacing 2−j . the deﬁnition of the scaling function and wavelet in analytic form is very useful in many applications.
MA: WellesleyCambridge Press. Gilbert Strang presents in his book. bib. Math. 1988). 315. Taswell. 69–88. e 141–149. 1989). Gilbert. Math.. Wellesley. Y. 1997 (Dec. 1992. www. Carl. Wavelets and Filter Banks.4 Comments and References The framework for wavelet design in the frequency domain has been developed since the early days of wavelet research by Yves Meyer. (Taswell. Trans. among others. Strang. a complete and detailed framework for generating wavelets using the ﬁlter design methodology. 10. Ingrid.10. For these reason.toolsmiths. I. we will discuss how to generate compactly supported wavelets. Comm. with the advantage of being more ﬂexible. Bibliography Daubechies. 1997). PA: SIAM Books. In Chapter 12 we will discuss how to generate biorthogonal wavelets. (Meyer. There are many more biorthogonal wavelets than orthogonal ones. Carl Taswell developed a computational procedure to generate wavelet ﬁlters using the factorization of the polynomial P . 1996. . Biorthogonal wavelets can have symmetry. Colloq. Truong. Orthonormal Bases of Compactly Supported Wavelets. Amer. Mallat. Soc. Tech.. 909–996.com.). The are associated with perfect analysis / reconstruction ﬁlter banks. Meyer. & Nguyen. 1990). S. 60/61. (Mallat. Philadelphia. Computational Toolsmiths. The initial methods investigated could only produce orthogonal wavelets with inﬁnite support. Pure Applied Math. 10. The major breakthrough was made by Ingrid Daubechies in 1988. and how to control the various properties. 1996).2 Biorthogonal Wavelets Biorthogonal wavelets have most of the qualities of orthogonal wavelets.. 1988. Multiresolution approximation and wavelets. 1989. The Systematized Collection of Wavelet Filters Computable by Spectral Factorization of the Daubechies Polynomial. they make possible a variety of design options and constitute class of wavelets most used in practical applications. Ten Lectures on Wavelets.3. rept. COMMENTS AND REFERENCES 131 In Chapter 11. Daubechies.4. Construction de bases orthonorm´es d’ondelettes. (Daubechies. XLI(41). and Stephane Mallat. 1990. (Strang & Nguyen.
.
1. which in turn. a formula to compute ikω . We will derive a closed form expression for the polynomial P .1) We need to ﬁnd a polynomial P of the form (11. such that equation (11. 11. • P is a halfband ﬁlter P (eiω ) + P (eiω+π ) = 1 (11.2) (11.e. with z = eiω is the key element in the construction of compactly supported wavelets. the coeﬃcients ak of P (eiω ) n k=−n ak e To ﬁnd a closed form expression for P we recall the two conditions that P must satisfy in the context of orthogonal ﬁlter banks.3) . and we will combine them afterwards to obtain an expression for P 11. • P is a reciprocal product polynomial P (eiω ) = m0 (eiω )2 where m0 has a zero of order n at ω = π (or eiω = −1). and we will show how to factor P in order to generate orthogonal wavelets with diﬀerent properties.1 P as a Product Polynomial 1 + eiω 2 133 n The polynomial P (z) = m0 (z)2 is generated from m0 . we described a method to synthesize wavelets by factoring P as a product of two polynomials. introduced in the previous chapter. There. we need ﬁrst an explicit expression for P . i.Chapter 11 Orthogonal Wavelets In this chapter we will investigate the construction and design of compactly supported orthogonal wavelets.1). in order to apply this method.1 The Polynomial P We saw in the previous chapter that the trigonometric polynomial P (z). But. is of the form m0 (e ) = iω Qn (eiω ) (11.2) is true. We will address these two conditions one at time.
5) (11. we use the following theorem: . Note that. We found that P has to be decomposed into two factors P (ω) = p1 (ω)q1 (ω). A question comes up at this point. To relate P and the halfband ﬁlter property in equation (11. such that B(ω)2 = A(ω) A proof of this lemma can be found in (Daubechies.7) = m0 (e ) m0 (e ) n 1 + e−iω 1 + eiω = 2 2 Since 1 + e−iω 2 /2 = (1 + cos ω)/2.6) Qn (eiω )2 (11. A has to be of the form M A(ω) = m=0 am cos mω with am ∈ R. Then there exists a polynomial B of order M M B(ω) = m=0 bm eimω with bm ∈ R.4) (eiω ) −iω n (11. Is it possible to factor every polynomial P (eiω ) ≥ 0 into a product m0 (eiω )2 ? The answer is yes. The above result guarantees that a factorization of P using the method described in the previous chapter is possible. 11.2 P and the Halfband Property We derived above a partial expression for P .2). the proof is constructive and follows the steps necessary for the spectral factorization. P is a polynomial in cos ω as enunciated by the lemma. the condition of n zeros at π appears in P as P (ω) = 1 + cos(ω) 2 n Qn (eiω )2 (11.3) into (11.1 (Riesz). Lemma 11. 1992).1) P (eiω ) = m0 (eiω )2 = m0 (e ) m0 iω iω (11. where p1 (ω) = (1 + cos ω)n /2 provides the zeros at π and q1 is chosen such that the halfband property is veriﬁed.2). ORTHOGONAL WAVELETS The factor (1 + eiω )n is a consequence of the fact that m0 has a zero of order n at π.134 CHAPTER 11. page 172. If A(ω) ≥ 0 is a trigonometric polynomial of degree M . In fact.8) The factor Qn (eiω )2 in P gives extra degrees of freedom to satisfy the halfband property expressed in equation (11. invariant under the substitution ω → −ω.1. This is guaranted by the Riesz Lemma. We get an expression for P by substituting equation (11.
n2 − 1. which is the unique lowest degree solution solution for (11. Therefore. q2 of degree ≤ N − 1. we can drop y N R(y) and make q1 (y) = (1 − y)−N . (1 − y)N Bn (y) + y N Bn (1 − y) = 1 where Bn (y) is the binomial series for (1 − y)−N .9) Because q1 and q2 are unique. q2 .1. and also q2 (y) = q1 (1 − y).11.3 The Expression of P Now we put together the two results above to obtain an expression for P . p2 .10) becomes (1 − y)N q2 (1 − y) + y N q1 (1 − y) = 1 (11. So far we know the following: . then there exist unique polynomials q1 and q2 such that p1 (y)q1 (y) + p2(y)q2 (y) = 1 where p1. respectively.11) (11. such that (1 − y)N q1 (y) + y N q2 (y) = 1 When we substitute y for 1 − y.13) Bn (y) = k=0 N +k−1 k y k (11. q1 .10). Since the degree of (q1 ) ≤ N − 1. 1992). let’s apply Bezout’s theorem to the particular case where p1 (y) = (1 − y)N and p2 (y) = y N . q1 is equal to its Taylor expansion truncated after N terms. provided that R(y) satisﬁes R(y) − R(1 − y) = 0.10) (11. truncated after N terms N −1 (11.1. The theorem says that there exists unique polynomials q1 . For reasons that will become clear later. q1 (y) = (1 − y)−N [1 − y N q1 (1 − y)] = (1 − y)−N − y N R(y) N −1 = k=0 N +k−1 k y − y N R(y) k (11. n1 − 1.10) also exist with q1 (y) = Bn (y) − y N R(y). n2 .12) where we expanded the ﬁrst N terms of the Taylor series for (1 − y)−N . A proof of this theorem can be found (Daubechies. equation (11. If p1 and p2 are two polynomials with no common zeros. This allows us to ﬁnd an explicit formula for q1 . are of degree n1 .14) Higher degree solutions for (11. This gives a closed form expression for q1 . 11. THE POLYNOMIAL P 135 Theorem 1 (Bezout). necessarily q1 (y) = q2 (1 − y). page 169.
11.15) It is clear that Bn is the solution for Qn 2 which makes P satisfy the halfband property. ORTHOGONAL WAVELETS i. and then collect the resulting terms with same exponent.16) k Note that to produce P in the form of a Laurent polynomial P (z) = n k=−n ak z . 1 + cos(ω) 2 1 − cos(ω) 2 = = 1 + eiω 2 1 − eiω 2 1 + e−iω 2 1 − e−iω 2 = = 1+z 2 1−z 2 1 + z −1 2 1 − z −1 2 Substituting this into equation (11. A polynomial of the form P (y) = (1 − y)n Bn (y) has the halfband property.15) we arrive at the closed form expression for P in the zdomain P (z) = 1+z 2 n 1 + z −1 2 n n−1 k=0 n+k−1 k 1−z 2 k 1 − z −1 2 k (11. but we will see that it is more eﬃcient to compute the roots of P (y) = (1 − y)n Bn (y).136 CHAPTER 11.4 The Factorization of P The two forms of the polynomial P (z) and P (y) are related by the change of variables (z + z −1 )/2 = 1 − 2y (11. This would be required to factorize P (z) directly.1. P is of the form P (ω) = 1+cos(ω) 2 n Qn (eiω )2 ii. To relate (i) and (ii) we make a change of variables eiω + e−iω = cos(ω) = 1 − 2y 2 which gives y= 1 − cos(ω) 2 and 1−y = 1 + cos(ω) 2 Therefore. and make the change of variables afterwards. it is necessary to expand the factors in each term. Making another change of variables we obtain an expression for P in terms of the complex variable z = eiω . we can write P as P (ω) = 1 + cos(ω) 2 n n−1 k=0 n+k−1 k 1 − cos(ω) 2 k (11.17) .
its roots are in reciprocal and complex conjugate pairs. The polynomial Qn (eiω )2 is a reciprocal polynomial with real coeﬃcients. Equation (11. a quadratic equation for z. r−1 } ¯ ¯ Qn (z)2 = K L U (z.19) associates two values of z for each value of y. The factored polynomial z 2n−1 P (z). The remaining 2n − 2 complex roots come from the binomial factor Bn (y). Thus. can be factored by regrouping these pairs as complex quadruplets {z.11. THE POLYNOMIAL P 137 We write this as a composition of conformal maps (f ◦ g)(z) = y. and real duplets {r. z 2n−1 P (z) = (z + 1)2n K L U (z. x = f (z) = z + z −1 2 −1 z = f (x) = x ± x2 − 1 and g is an aﬃne transformation y = g(x) = (1 − x)/2 x = g −1 (y) = 1 − 2y The expressions for the change of variables are y = g(f (z)) z = f −1 = (y)) (1 − (z + z −1 )/2)/2 = 1 − 2y ± (1 − 2y)2 −1 (11. from subsection 11.18) is.3. in fact. rj ) − zi )(z − zi ) ¯ ¯−1 (11. 2n roots come from the ﬁrst factor (1 − y)n . has degree 2n − 1. z −1 }. z −1 }. in regular form. From these. Thus.1. z −1 . The polynomial P (y) = (1 − y)n Bn (y). expressed in regular form. zi ) i=1 V (z.22) U (z. rj ) (11. rj ) = (z − j=1 −1 zi )(z − zi )(z −1 rj )(z − rj ) where K = (n − 1)/2 and L = (n − 1) mod 2.1. It is clear from (11. zi ) i=1 j=1 V (z.17) that one z is inside the unit circle and the other 1/z is outside.23) The ﬁlter function m0 (z) can be computed from this factorization by noting that P (z) = m0 (eiω )2 = m0 (eiω )m0 (eiω ) = m0 (eiω )m0 (e−iω ) = m0 (z)m0 (z −1 ). 1977). . but because the change of variables. where f is the Joukowski transformation (Markushevich. is obtained by including the multiple zeros at z = −1.21) (11. zi ) = (z − U (z.18) (11. and correspond the the multiple root at z = −1. z . The polynomial Qn (z)2 . The change of variables z = f −1 (g −1 (y)) yields a doublyvalued solution with reciprocal pairs {z. P (z) will have 4n − 2 roots.19) (g −1 Notice that this change of variables in (11.20) (11. it will result in a polynomial P (z) of degree 4n − 2.
4 0. 1 0.1. P (y) is the unique polynomial with 2n coeﬃcients which satisﬁes these n conditions at the endpoints. Find the n − 1 roots of Bn (y). and assign 11.138 CHAPTER 11.6 0. As n increases. except that P (0) = 1.2 0. −1 )n .1: P . 2 P becomes simultaneously steeper at the midpoint and ﬂatter at both endpoints.6 0. From these. In fact. and using a change of variables. generate the 4n − 2 roots of P (z).8 1. ORTHOGONAL WAVELETS K L m0 (z) = c i=1 (z − zi )(z − zi ) ¯ j=1 (z − rj ) (11.1 shows a graph of P (y) for n = 2 and 4. i. and make a change of variables to produce the 2n − 2 roots of Qn (z)2 . Therefore.5 Analysis of P The family of orthogonal ﬁlters generated from the polynomial P (y) = (1 − y)n Bn (y) is called maxﬂat in the ﬁlter design literature. The slope of P at y = 1 is proportional to n. The orthogonal ﬁlter function m0 (z) will have these 2n − 1 roots.24) where c is a constant.8 0. and zi and rj are selected from the elements of each reciprocal pair {z. In consequence of the special structure of P . z −1 }. the spectral factorization of m0 is done in two steps: 1. the polynomial P of degree 2n − 1 is determined from n conditions at y = 1 and at y = 0.4 0. Figure 11. the ﬁlter response of the ﬁlter is maximally ﬂat at ω = π. Observe that P decreases monotonically √ from P (0) = 1 to P (1) = 0.e. it is more eﬃcient to compute the 2n−1 roots of P (y) ﬁrst.2 0. select appropriately n − 1 roots and assign them to m0 . The roots of m0 are then selected from those roots. Include the n roots at z = −1 from the factor ( 1+z )n of ( 1+z )n ( 1+z 2 2 2 them also to m0 . P (y) and its ﬁrst n − 1 derivatives are zero at y = 0 and at y = 1.0 Figure 11. This is because the factor (1 − y)n give a maximum number of zeros at y = 1 (or z = −1). 2. Consequently.
4 −0. particular families of orthogonal wavelets. We recall that P (y) and P (z) are related by the 2 change of variables (1 − 2y) = (z + z −1 )/2.1 0 0 −0. These curves meet at z = ±i. Also. from the polynomial P .5 −0. in this case. We have seen in the previous section that the factorization of P (z) produces roots that −1 can be grouped as quadruplets of complex zeros (z − zi )(z − zi )(z − zi )(z − zi ) and duplets ¯ ¯−1 −1 ). for P of degree m.5 0 0.5 1 1. Gilbert Strang studied the asymptotic behavior of the zeros of P (y) as n → ∞.5 2 (a) (b) Figure 11. .6 −1 −0.2 0 0.2 Examples of Orthogonal Wavelets In this section we show how to generate. either zk or zk as a root of m0 . as n → ∞.1 −0.2. the curve in y actually √ √ corresponds to two curve segments z + 1 = 2 and z − 1 = 2.5 −0. ¯ ¯−1 Even with these restrictions for separating the roots of P (z).2 0. complex conjugate pairs −1 must stay together. we can select either zk with zk or zk with zk . EXAMPLES OF ORTHOGONAL WAVELETS 139 This property is very important because it means that P (and its “square root” m0 ) as a ﬁlter will preserve low frequencies (near ω = 0) and will cut oﬀ high frequencies (near ω = π).6 −0. the roots of Bn (y) approach the limiting curve 4y(1 − y) = 1.3 0. if we require that m0 is a polynomial with real coeﬃcients.5 0. Also. we have many choices to m taylor m0 according to design constraints. 0. there are 2 4 diﬀerent choices for m0 .2 shows a plot of the graphs ot these two curves in the complex plane. Typically. we have to of real zeros (z − rj )(z − rj 0 −1 −1 choose from each pair of reciprocal roots {zk . zk } of P . Let’s now take a look at the most important ones.4 −0. and Qn (z)2 in P (z) = ( 1+z )n 2 Qn (z)2 .2 0.4 1 0. because the change of variables yields a doubly valued solution. Note that. Therefore.4 0. which correspond to y = 1/2.5 0. 1996)) 11. there is an insightful analysis of the factors Bn (y) in P (y) = (1 − y)n Bn (y). whereas the roots of Qn (z)2 approach the curve z − z −1  = 2. In (Strang & Shen. it is shown that. In the spectral factorization of P (z) = m (z)2 .3 −1 −0. Figure 11.2 −0.11. 1996).2: Limit curves for the roots of Bn (y) and Qn (z)2 (from (Strang & Shen.
2+ 3).28) (11. Thus. z } of P . 2− 3. −1. P (y) = (1 − y)2 (1 + 2y) The change of variables y → z gives P (z) = 1+z 2 2 (11. −1. with B2 (y) = 1 + 2y. We remark also that the minimum and maximum phase are complementary choices.2. Example 16 (Daubechies Wavelets of Order 2).3: Roots of P (z) with n = 2. Therefore P (y) = (1 − y)2 B2 (y). The resulting orthogonal wavelets will have the minimum (or maximum) phase among all compactly supported wavelets of degree m.3 shows the roots of P and m0 . −1. from each reciprocal pair of roots {z.17) for the change of variables = gives the roots of Q2 (z)2 at z = 2 ± 3. The roots of P (z) include all these zeros. √ The choice of the root for Q2 (z) inside the unit circle z = 2 − 3 leads to the minimum √ phase m0 with roots (−1.1 Daubechies Extremal Phase Wavelets The simplest systematic choice for designing orthogonal wavelets produces the so called extremal phase wavelets.25) 1 + z −1 2 2 1 −z + 4 − z −1 2 (11. This corresponds to scaling functions and wavelets that are the most assymmetric basis functions.27) (11.30) √ = c(1 + z −1 )2 (1 − (2 − 3)z −1 ) √ √ √ √ 1 √ (1 − 3) + (3 + 3)z −1 + (3 − 3)z −2 + (1 − 3)z −3 = 4 2 = h0 + h1 z −1 + h2 z −2 + h3 z −3 . and are (−1.29) (11. The ﬁlter function m0 is then m0 (z) = 1+z 2 2 Q2 (z) (11. Equation (11. 2 − 3) Figure 11. the basis functions are mirror images of each other for these two choices. In this case n = 2. The factor (1 − y)2 will produce the four zeros at √ √ z = −1 of P .26) The factor B2 (y) has one root at y√ −1/2. −1. ORTHOGONAL WAVELETS 11.140 CHAPTER 11. 4 Figure 11. They lead to m0 and the complex conjugate of m0 . with z < 1 (or z > 1). This corresponds to selecting always the roots inside (or outside) the 1 unit circle.
25 1 0.5 3 1 φD.5 1 1.8365.25 0. Recall from the rules for separating the roots of P (z) that: • for orthogonal ﬁlters.2 and wavelet ψD.5 5 7.4830.4 0.2 ˆ ψD. Figure 11.2 and wavelet ψD.2 i Figure 11.11.5: Fourier Transform of (D2) Scaling Funtion and Wavelet 11. z and z −1 must go separately.6 0.25 1 1. from the requirements on ﬁlter m0 .5 shows the graphs of the Fourier Transform of the Daubechies scaling function φD. −0.4 shows a plot of the Daubechies scaling function φD. but it is intuitive to understand.5 1.5 0. We are not going to demonstrate this result. and .2241.2 ψD.5 ˆ φD. why orthogonality and symmetry are incompatible.2 Figure 11.5 5 7.5 0.5 7.8 0.2 .5 2.6 0. Symmetry and orthogonality are conﬂicting requirements for compactly supported wavelets.2 7.2 1 0.5 2. It can be shown that the Haar basis constitute the only scaling functions and wavelets with compact support which are orthogonal. 0.4: Daubechies (D2) Scaling Funtion and Wavelet 1 0.4 0. EXAMPLES OF ORTHOGONAL WAVELETS 141 The coeﬃcients {hn } of m0 are approximately 0.5 2 2.5 5 2.75 0.2.2 . Figure 11.8 0.5 1 0.1294.5 0.2. 0.5 3 0. 1.5 2 2.5 5 2.2 Minimal Phase Orthogonal Wavelets The choice of m0 having all roots with absolute value less (or greater) than one leads to wavelets with a very marked assymmetry.
Therefore. there are 2 2 −1 choices. The above facts imply that a symmetric orthogonal FIR ﬁlter m0 (z) can only have two nonzero coeﬃcients. For n ≥ 4.32) To ﬁnd the polynomial m0 of degree n with the smallest phase nonlinearity it is necessary n to compare all possible combination of roots.35) (11. In this case. ¯ −iα and z = r eiα ¯ Since z = r e (e−iω − r e−iα )(e−iω − r eiα ) = e−iω (e−iω − 2r cos α + r2 eiω ) their phase contribution tan ω = arctan Similarly. The ¯ phase of m0 can be computed from the phase of each z . rk are the roots of m0 and we have substituted z = e−iω in equation (11. P (z) = ( 1 + z 1 + z −1 )( ). there is eﬀectively only one set of basis φD.23).n . ψD. ORTHOGONAL WAVELETS • for symmetric ﬁlters z and z −1 myst stay together. .n . we have to select the roots of P such that m0 is as close as possible to linear phase. In practice. we have to evaluate the n total phase nonlinearity of m0 for all 2 2 −1 solutions.33) (11. z . it is possible to construct compactly supported orthogonal wavelets that are less assymmetric than the ones in the previous subsection. 2 2 which corresponds to the Haar wavelets with √ m0 (z) = (1 + z −1 )/ 2. Nonetheless. we need to estimate the contribution of each root to the phase nonlinearity of m0 .31) where z . Since m0 is of the form m0 (ω) = 1 + e−iω 2 n (e−iω − z )(e−iω − z ) ¯ k (e−iω − rk ) (11.34) z/ z is (r2 − 1) sin ω (r2 + 1) cos ω − 2r cos α (11. In order to generate the least assymmetric basis functions. since (e−iω − rk ) = e−iω/2 (e−iω/2 − rk eiω/2 ) the phase contribution of rk is arctan ω (rk + 1) tan (rk − 1) 2 (11.142 CHAPTER 11. For n = 2 or 3. z and rk .
She formulated the expression for the halfband polynomial P based on .3. COMMENTS AND REFERENCES 143 Figure 11. both ψ and φ have n vanishing moments.11. . m0 (y)2 = (1 − y)n Bn (y). for = 1. More precisely. with n = 4 11.k >≈ f. 1 1 2 2 4 6 2 1 4 φ ψ Figure 11. L − 1 and x ψ(x)dx = 0 for (11.37) x φ(x)dx = 0 (11.6: Least Assymmetric Scaling Funtion and Wavelet. The minimum support width 2n − 1 for ψ with n vanishing moments is achieved when R ≡ 0. . This basis functions are more regular and symmetric than the ones presented in the previous subsections.6 shows the graph of the least assymmetric φ and ψ for n = 4.7 shows a plot of the coiﬄets of order 4 11. L − 1.e. . We can use a higher degree m0 (y)2 = (1 − y)n [Bn (y) − y n R(y)] to provide more freedom in the design o orthogonal wavelets. 1991). except that the scaling function integrates to one. This implies that < 2J/2 −J −J k). . making very simple to obtain the ﬁne . Daubechies resorted to such higher degree solutions to construct a family of wavelets named “coiﬄets”. In other words. samples of f . .36) = 0. i.2).k > < f. Figure 11.3 Coiﬂets Up to this point we have assumed that m0 is the unique lowest degree polynomial which satisfy (11. Coiﬀman requested them motivated by the research in (Beylkin et al. . . The price to pay is a wider support of the basis functions for a given number of vanishing moments. . . the scaling functions and wavelets are designed so that: φ(x)dx = 1.3 Comments and References The main result for the construction of compactly supported orthogonal wavelets is due to Ingrid Daubechies. φf (2 scale coeﬃcients from φ−J . because R.2.
144 BIBLIOGRAPHY 2 2 1 1 2 1 1 1 2 3 2 1 1 2 3 φ ψ Figure 11. she gives a good overview of this methodology. 1992. The zeros of the Daubechies polynomials. Pure Applied Math. I. 1977.). Markushevich. CA.. A. G. which are knwon as Daubechies Wavelets. Daubechies. Stanford University. V. Ten Lectures on Wavelets. 909–996. 141–183. Taswell. 1996. 44. Taswell. Amer. Soc. XLI(41). 1991. & Rokhlin.(Taswell. Scientiﬁc Computing and Computational Mathematics. Theory of Functions of a Complex Variable. Bibliography Beylkin. presents algorithms for the generation of the Daubechies orthogonal wavelets and for the computation of their regularity. Based on this formulation. Comm. Carl.. Wellesley. (Strang & Nguyen. Gilbert. Carl Taswell in. (Daubechies. Comm. 1992). 1996. Proc. in Pure and Applied Math. Algorithms for the Generation of Daubechies Orthogonal Least Asymmetric Wavelets and the Computation of Their Holder Regularity. R. MA: WellesleyCambridge Press. 1988.. Math. J. 1996). Strang. Fast wavelet transforms and numerical algorithms. she discovered the family of extremal phase compactly supported orthogonal wavelets. Chelsea Publishing Co.. and also presents examples of several other families of compactly supported wavelets. Orthonormal Bases of Compactly Supported Wavelets. 1997). In her book. Coifman. rept. Truong. G. (Daubechies.). I. & Shen. 1995. 1995 (Aug. Daubechies. Stanford. & Nguyen.. 1988). Strang. The book by Gilbert Strang and Truong Nguyen gives a comprehensive treatment of the framework for generating wavelets from a ﬁlter bank viewpoint. . Philadelphia. Tech. Wavelets and Filter Banks.7: Coiﬄets of order 4 the multresolution analysis conditions in the frequency domain and also in the restriction of ﬁnite length ﬁlters. Ingrid. PA: SIAM Books.
1997.BIBLIOGRAPHY 145 Taswell. . Symmetric. Carl. Miller Freeman. Computational Algorithms for Daubechies LeastAsymmetric. and MostSymmetric Wavelets. Pages 1834–1838.
.
instead of a single orthogonal basis.k and ProjWj (f ) = k dj ψj. a pair of dual biorthogonal basis functions is employed: One for the analysis step and the other for the synthesis step. Under this framework.1. we have deﬁned the projection operators onto the subspaces Vj and Wj . For example.k φj. Recall that. in the context of orthogonal multiresolution analysis.e. will discuss its relations with perfect reconstruction ﬁlter banks and will present a framework for the design of biorthogonal wavelets. respectively: analysis ProjVj (f ) = f. ψj. In this chapter we will introduce biorthogonal wavelet basis. φj. This restricts signiﬁcantly the design choices of wavelet basis. we showed in Chapter 11 that the Haar wavelet is the only orthogonal basis which is symmetric and has compact support.e. A practical solution. that allows more ﬂexibility on the choice of wavelet functions with desirable properties. ψj. i.Chapter 12 Biorthogonal Wavelets We have seen in the previous chapters that orthogonality is a very strong constraint for the construction of wavelets. φj. respectively ProjVj (f ) = k cj φj. respectively cj = f.k ψj.k .k .k . k analysis and ProjWj (f ) = f.1 Biorthogonal Multiresolution Analysis and Filters Biorthogonal Basis Functions Biorthogonal wavelets constitute a generalization of orthogonal wavelets.k . they are used for: • analysis: compute the coeﬃcients of the representation of f in terms of the basis φ and ψ of the spaces Vj and Wj . i. k k 147 . 12. is to replace orthogonality by a biorthogonality condition.k and dj = f. k synthesis synthesis where the functions φ and ψ perform a double duty. we have reciprocal frames as deﬁned in Chapter 2. k k • synthesis: reconstruct the projection of f onto Vj and Wj .1 12. from the coeﬃcients of the representation.
The functions φ and ψ are called. The functions φ and ψ are called. More precisely. respectively. The fact the the roles of these functions can be interchanged is called duality principle.k .k φj. ψj. we impose instead a biorthogonality relation between the primal and dual multiresolution ladders. φj. consequently. These functions generate dual multiresolution analysis ladders · · · ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ · · · · · · ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ · · · where V0 = Span{φ0.1) for j = l. Riesz basis of the subspaces Wj and Wj . respectively. In order to compensate for the lack of orthogonality within the approximating and detail spaces. Riesz basis of the subspaces Vj and Vj . In other words.e. while the other act as the reconstruction function. primal scaling function and wavelet. Wj . ψ(x − k) = ψ(x). dual scaling function and wavelet.4) . Although other conventions are possible.3) (12. in general. ψ are used to share the workload: one function of the pair act as the analyzing function.k and ψj.k k ∈ Z}. Similarly we deﬁne a pair of wavelet functions ψj and ψj that are. BIORTHOGONAL WAVELETS The more general framework of biorthogonal multiresolution analysis employ similar projection operators Pj f = k f. The above biorthogonality condition implies that the basis of these spaces must relate as φ(x). φ and ψ. and also Vj . while the dual functions are used for synthesis. respectively. In terms of a multiresolution analysis. Wj ⊥ W l (12. we deﬁne a pair of scaling functions φj and φj that are. The two multiresolution hierarchies and their sequences of complement spaces ﬁt together according to an intertwining pattern. this scheme leads to a family of biorthogonal scaling functions and wavelets that are dual basis of the approximating and detail spaces.148 CHAPTER 12. Vj−1 = Vj + Wj and Vj−1 = Vj + Wj . Wj . The spaces Wj and Wj generated by ψj. ˜ ˜ where the pair of functions φ. here we will assume that the primal functions are used for analysis. i. but the spaces Vj .k k ∈ Z} and V0 = Span{φ0. Vj ∩ Wj = {∅} and Vj ∩ Wj = {∅}. The intersection of these spaces is null.2) and Vj ⊥ Wj (12.k . respectively.k ψj. such that Vj ⊥ Wj and. and Qj f = k f. are not orthogonal.k are respectively the complements of Vj in Vj−1 and of Vj in Vj−1 . φ(x − k) = φ(x)ψ(x − k)dx = 0 ψ(x)φ(x − k)dx = 0 (12.
k . m ∈ Z j.l δk. ψ(x − k) = φ(x)φ(x − k)dx = δk ψ(x)ψ(x − k)dx = δk (12. .m = δj. ψl.6) Which can be extended to the multiresolution analysis by a scaling argument resulting in φj.1. k.m = δk. m ∈ Z (12.m . k ψ(ω) = m1 (ω/2)φ(ω/2) ψ(ω) = m1 (ω/2)φ(ω/2) 1 m1 (ω) = √ 2 1 m1 (ω) = √ 2 gk e−ikω k hk e k −ikω . ψ.8) 12. φ(ω) = m0 (ω/2)φ(ω/2). ψj. BIORTHOGONAL MULTIRESOLUTION ANALYSIS AND FILTERS 149 and φ(x).3) to (12. φ(x − k) = ψ(x).m . m1 . ψ.7) (12.2 Biorthogonality and Filters The two pairs of scaling function and wavelet. we can see that the biorthogonality condition in the frequency domain is equivalent to φ(ω + k2π)φ(ω + k2π) = 1 k ψ(ω + k2π)ψ(ω + k2π) = 1 k ψ(ω + k2π)φ(ω + k2π) = 0 k φ(ω + k2π)ψ(ω + k2π) = 0 k for all ω ∈ R. l. and m0 . and φ. φ.5) (12.1. where 1 m0 (ω) = √ 2 1 m0 (ω) = √ 2 hk e−ikω .k . gk e−ikω k By computing the Fourier Transform of the inner products in equations (12. k.12. and using the same argument of Chapter 10 for the characterization of m0 and m1 . φj. m1 . j.6). In the frequency domain these relations are φ(ω) = m0 (ω/2)φ(ω/2). are deﬁned recursively by the two pairs of ﬁlters m0 .
15) Similarly.m . also deﬁne a multiresolution analysis.1. BIORTHOGONAL WAVELETS This means that the ﬁlters m0 .16) We can derive the coeﬃcients of the ﬁlters m0 and m1 by combining the above two equations with the biorthogonality relations φj. This gives us hk−2l = gk−2l = and also hk−2l = gk−2l = φ(x − l). φj.20) φ(x − l).m = δj.12) 12.17) (12.11) (12.150 CHAPTER 12.18) .k . φ(2x − k) (12.10) (12.3 Fast Biorthogonal Wavelet Transform Because φ and ψ deﬁne a multiresolution analysis. m1 have to satisfy m0 (ω)m0 (ω) + m0 (ω + π)m0 (ω + π) = 1 m1 (ω)m1 (ω) + m1 (ω + π)m1 (ω + π) = 1 m1 (ω)m0 (ω) + m1 (ω + π)m0 (ω + π) = 0 m0 (ω)m1 (ω) + m0 (ω + π)m1 (ω + π) = 0 The set of equations above can be written in matrix form as m0 (ω) m0 (ω + π) m1 (ω) m1 (ω + π) or M (ω)M T (ω) = I where M is the modulation matrix introduced in Chapter 10 M (ω) = and I is the identity matrix. and therefore φ(x) = k hk φ(2x − k) and ψ(x) = k gk φ(2x − k) (12.m and ψj.13) m0 (ω) m1 (ω) m0 (ω + π) m1 (ω + π) = 1 0 0 1 (12.14) (12. φ(2x − k) ψ(x − l).19) (12. φ(2x − k) ψ(x − l).m = δk. ψl.9) (12. φ(2x − k) (12. we have that φ(x) = k hk φ(2x − k) and ψ(x) = k gk φ(2x − k) (12. m0 (ω) m0 (ω + π) m1 (ω) m1 (ω + π) (12. m1 and their duals m0 .l δk. φ and ψ.k .
23) and dj+1 = n k gk−2n cj k (12. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 151 By writing φ(2x − k) ∈ V−1 in terms of the bases of V0 and W0 . In the decomposition step we employ a discrete convolution with the ﬁlter coeﬃcients (hk ) and (gk ) of the ﬁlters m0 and m1 .22) The fast biorthogonal wavelet transform uses the above decomposition/reconstruction relation. we get the twoscale relation φ(2x − k) = l hk−2l φ(x − l) + l gk−2l ψ(x − l) (12.2 12. where m0 . since primary and dual functions are interchangeable.1. presented in Chapter 7. we employ a discrete convolution with the ﬁlter coeﬃcients. We would like this twochannel ﬁlter bank to have the property of perfect reconstruction. m1 . m1 . m1 . The pair of ﬁlters m0 . m1 . are respectively the lowpass and highpass ﬁlters used in the synthesis step. through the ﬁlter bank.21) and. it is essentially similar to the orthogonal case. the roles of these two ﬁlter banks can be interchanged. we note that the role of these ﬁlters can be interchanged. are respectively the lowpass and highpass ﬁlters used in the analysis step and. Once again. . m0 .1 Filter Design Framework for Biorthogonal Wavelets Perfect Reconstruction Filter Banks The ﬁlters m0 . These conditions can be derived by following a a discrete signal. The algorithm employs the two pairs of primary and dual ﬁlters and except for this diﬀerence. m1 in the reconstruction step. of the ﬁlters m0 and m1 j−1 cl = n hl−2n cj + n n gl−2n cj n (12. 12. in the reconstruction step. cj+1 = n k hk−2n cj k (12. m0 . we also have φ(2x − k) = l hk−2l φ(x − l) + l gk−2l ψ(x − l) (12.25) Remember that. as we already noted. m1 is employed in the decomposition step and the pair of ﬁlters m0 .24) Conversely. (xn ). deﬁne a twochannel ﬁlter bank. (hk ) and (gk ). This means that a signal can be exactly reconstructed by the synthesis ﬁlters from the coeﬃcients of the representation constructed by the analysis ﬁlters.2.2.12. as shown in Figure 12.
152 CHAPTER 12. whose ztransform is m0 (z)x(z). Finally. which in the zdomain is 1/2 [m0 (z)x(z) + m0 (−z)x(−z)].1: Biorthogonal Filter Bank The output of the lowpass channel can be written in a compact notation using the znotation (see Appendix B). we perform a discrete convolution of analysis lowpass ﬁlter with the signal. manifested by the appearance of terms x(−z).27) (12. For perfect reconstruction we must have x(z) = x(z). z = e−iω and −z = e−iω+π . By substituting. Note that the downsampling/upsampling operators introduced aliasing in the signal. This is followed by downsampling and upsampling. we perform a discrete convolution of the resulting coeﬃcient sequence with the synthesis ﬁlter. which gives x(z) = 1 [m0 (z)m0 (z) + m1 (z)m1 (z)] x(z) + 2 1 [m0 (z)m0 (−z) + m1 (z)m1 (−z)] x(−z) 2 where we rearranged the expression to separate the term involving x(z) from the terms involving x(−z). as well as x(z). BIORTHOGONAL WAVELETS m0 cj−1 d d d E ↓2 E cj E ↑2 E m0 d d d < E cj−1 ˆ m1 E ↓2 E dj E ↑2 E m1 Figure 12. it is apparent that the frequencies w + π appear as aliases of frequencies w in the signal (see Appendix A for details). which implies in two conditions: .26) The ﬁlter bank combines the outputs of the lowpass and highpass channel by adding these two expressions. We will do this in three stages: First. whose ztransform is 1 m0 (z)[m0 (z)x(z) + m0 (−z)x(−z)] 2 The output of the highpass channel is obtained in a similar manner 1 m1 (z)[m1 (z)x(z) + m1 (−z)x(−z)] 2 (12.
31) This option takes care of the alias cancelation. An interesting option is to investigate how we can determine some of the ﬁlters from the others.10). into (12. m1 .31). m1 .2 Conjugate Quadrature Filters We have four ﬁlters to design.29) are automatically satisﬁed. m0 (z)m0 (z) + m1 (z)m1 (z) = 2 or m0 (ω)m0 (ω) + m1 (ω)m1 (ω) = 2 (12.28) 12. m1 . First. m1 . m0 . can be derived from the perfect reconstruction conditions (12.30).28).29) (12. m0 (z)m0 (−z) + m1 (z)m1 (−z) = 0 or. we rewrite equation (12.2. . to express the nodistortion condition in terms of only m0 and m0 . The other conditions (12. m0 . FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 153 • alias cancellation: the aliasing components caused by subsampling in the lowpass channel and in the highpass channel cancel each other.30). as it can seen by substituting (12.30) (12.28) is satisﬁed. (12.29). to deﬁne the highpass ﬁlters in terms of the lowpass ﬁlters: m1 (ω) = e−iω m0 (ω + π) m1 (ω) = e−iω m0 (ω + π) (12. we use a conjugate quadrature scheme as done in Chapter 10. m0 .12. the ﬁrst condition for biorthogonality of the ﬁlters m0 .28) and (12. (12. using the conjugate quadrature relations (12.28) and (12.31).32) √ Note that. Next.30) and (12. equation (12. such that conditions (12. when we normalize the ﬁlters (multiplying their coeﬃcients by 1/ 2). This demonstrates that biorthogonal wavelets are associated with a subband ﬁltering scheme with exact reconstruction. and guarantees that equation (12.29) using (12.9).32) is exactly (12. m0 (ω)m0 (ω) + m0 (ω + π)m0 (ω + π) = 2 (12.2.31). making z = e−iω m0 (ω)m0 (ω + π) + m1 (ω)m1 (ω + π) = 0 • no distortion: the signal is reconstructed without loss or gain of energy.11). These ﬁlters must satisfy the conditions for perfect reconstruction. (12.12) and (12.
The main diﬀerence is that now. This gives us more freedom to design wavelets with desired properties. we are using the same ﬁlter design framework to create wavelet basis.33) 2. resulting in P (ω) + P (ω + π) = 2 (or when the ﬁlters are normalized a0 = 1/2 and P (ω) + P (ω + π) = 1). m0 instead as the “square” of a single polynomial m0 . Also. P (ω) = m0 (ω)m0 (ω) has to be of the form P (ω) = 1 + eiω n 2 B(eiω ) (12. Factor P into m0 m0 3. in order to satisfy the perfect reconstruction condition.4 Factorization of P for Biorthogonal Filters Let us see one example of the diﬀerent design choices for the factorization of P (ω) into m0 (ω) and m0 (ω). m1 . because ei2k(ω+π) = −ei2kω . we deﬁne the product ﬁlter P (ω) = m0 (ω)m0 (ω).3 The Polynomial P and Wavelet Design P (ω) + P (ω + π) = 2 (12. the terms with index the polynomial P (ω) = N k=−N = ak e k even must have coeﬃcient ak = 0. from (Strang & Nguyen. We will design the polynomial P such that it will have four zeros at π P (ω) = (1 + e−iω )2 2 B(eiω ) = (1 + e−iω )2 (1 + eiω )2 B(e−iω ) Making z = e−iω P (z) = (1 + z −1 )2 (1 + z)2 B(z) = (z −2 + 4z −1 + 6 + 4z + z 2 )B(z) = z 2 (1 + z −1 )4 B(z) . ikω . BIORTHOGONAL WAVELETS 12. must be a halfband ﬁlter.33). As we have seen in Chapter 10. will cancel in equation (12. the polynomials m0 (ω) and m0 (ω) must have. In fact.34) The advantage of biorthogonal wavelets over orthogonal wavelets is that we can factorize P (ω) in two diﬀerent polynomials m0 .154 CHAPTER 12. i. the analysis ﬁlters can be diﬀerent from the synthesis ﬁlters. because we have two biorthogonal multiresolution hierarchies. because both φ and φ deﬁne multiresolution analyses. Therefore. which have coeﬃcients ak = 0. m0 .2. such that the terms with index k odd. Choose a polynomial P satisfying equation (12. we use the following procedure: 1. m1 that generate biorthogonal wavelets. and equation (12.33) Now. at least one zero at π (see Chapter 11 for details).30) and (12.32) becomes To design the ﬁlters m0 . The only remaining term will be a0 = 1.e.31) to obtain m1 and m1 . Use (12. 12. 1996). Observe that this procedure is very similar to the one described in Chapter 10.2. except for a0 = 1.
or (1 + z −1 ) (c − z −1 ). both very important. The ﬁrst possibility is a factorization as 1 (1 + 3z −1 + 3z −2 + z −3 ). and 1 (−1 + 3z −1 + 3z −2 − z −3 ). and √ the other two roots at c = 2 − 3 and 1/c = 2 + 3 come from B(z).2. that will be discussed in the next section. . 2 + 3}. making it a halfband polynomial. −1. Notice that this 8 2 choice results in a pair of linear phase ﬁlters which correspond to symmetric scaling functions and antisymmetric√ wavelets. Notice that the orthogonal ﬁlters have extremal phase. Each of the polynomials m0 (z) and m0 (z) will have. — the other ﬁlter m0 (or m0 ) will have the remaining roots: i. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 155 We have to choose B so that even powers are eliminated from P . 2 − 3. The polynomial P has six √ √ roots {−1. Therefore. one factor is (1 + z −1 ). which is the 2 ﬁlter for the hat function.12. It is best to use the hat scaling function for synthesis. In option (iii). m0 of degree 3: (1 + z −1 ) (1 + z −1 )2 . at least. or (1 + z −1 )3 . that we presented in the previous chapter. In option (ii). and we can select which set of roots go to the analysis ﬁlter m0 and to the synthesis ﬁlter m0 . The other possibilities are not so interesting. The other 1 factor is a degree 5 polynomial 16 (−1 + z −1 + 8z −2 + 8z −3 + z −4 − z −5 ). and the other factor equal to 1 (−1 + 2z −1 + 6z −2 + 2z −3 − z −4 ). resulting in a 5/3 ﬁlter bank. 8 This choice corresponds to the order 2 Bspline biorthogonal wavelet. which is the ﬁlter for a box function. The solution is B(z) = −z −1 + 4 − z P (z) = (z −2 + 4z −1 + 6 + 4z + z 2 )(−z −1 + 4 − z) = −z −3 + 9z −1 + 16 + 9z − z 3 We normalize it to make a0 = 1 P (z) = 1 (−z −3 + 9z −1 + 16 + 9z − z 3 ) 16 (12. For each choice the roots are separated in two sets. We have several choices to factorize P (z) into m0 (z)m0 (z). or (1 + z −1 )2 (c − z −1 ) (1 + z −1 ) (1/c − z −1 ) We have six diﬀerent choices for the factorization of P . The√ second possibility is a factorization with m0 (z) = m0 (z) = √ √ 1 √ [(1 + 3) + (3 + 3)z −1 + (3 − 3)z −2 + (1 − 3)z −3 ]. In option (i). −1. which correspond to the most assymmetric scaling functions and wavelets. The four √ roots at z = −1 come from (1 + z −1 )4 . m0 of degree 2: iii. we can select the roots for the ﬁlter m0 (or m0 ) in one of the following ways. This choice gives us the Daubechies 4 2 orthogonal wavelet of order 2.35) Note that P satisfy the perfect reconstruction conditions and also have four zeros at −1. one root at z = −1. The ﬁlter bank will have ﬁlters with 2 and 6 coeﬃcients for analysis and synthesis respectively. m0 of degree 1: ii. −1. the ﬁrst possibility is one factor equal to 1 (1 + 2z −1 + z −2 ). we have two diﬀerent possibilities.
8 0.2 0 0 1 0.5 0. .5 0 0 −0.5 2 2. BIORTHOGONAL WAVELETS Figure 12.2 0 0 0.6 0.5 −0.5 −1 0 1 2 3 4 5 −1 0 1 2 3 4 5 (a) (b) Figure 12.5 3 1 0.3: Frequency response of lowpass (a) and highpass (b) ﬁlters.4 0.5 3 (a) (b) Figure 12.2 shows the graph of the biorthogonal scaling functions and wavelets derived in case (i).6 0.8 0.5 0.5 1 1.5 −1 0 1 2 3 4 5 −1 0 1 2 3 4 5 1 1 0.156 CHAPTER 12.5 2 2.4 0.5 −0.4 0.2 0 0 0. and Figure 12.5 1 1.5 2 2.5 0 0 −0.8 0.6 0.2 0 0 0.5 1 1. functions 1 1 0.6 0.3 plots the frequency response of the corresponding ﬁlters. filters 1 0.5 2 2.5 3 0.8 0.5 3 1 0.4 0.5 1 1.2: Biorthogonal box scaling functions (a) and wavelets (b).
5 3 0.5 0 1 2 3 4 5 −1.4 0. filters 1.6 0.5 2 2.8 0.6 0.5 2 2.4: Biorthogonal Hat scaling functions (a) and wavelets (b).4 0. functions 1 1 0.5 0 0 −0.5: Frequency response of lowpass (a) and highpass (b) ﬁlters.6 0.2 0 0 0. .8 0.8 0.5 0 1 2 3 4 5 4 2 0 −2 −4 −6 0 1 2 3 4 5 4 2 0 −2 −4 −6 0 1 2 3 4 5 (a) (b) Figure 12.5 1 1.5 2 2.5 plots the frequency response of the corresponding ﬁlters.2 0 0 0.5 −1 −1 −1.8 0.5 0.4 0.5 1 1.5 3 0.5 3 (a) (b) Figure 12. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 157 Figure 12.2 0 0 1.4 shows the graph of the biorthogonal scaling functions and wavelets derived in case (ii).2.5 2 2.5 1 1.5 1 1.2 0 0 0. and Figure 12.6 0.2 1 1 0.12.5 −0.4 0.2 1 1 0.5 3 0.
5 3 0.8 1 0.5 1 0.5 (a) (b) Figure 12.5 1 1.5 3 −1 0 0. BIORTHOGONAL WAVELETS Figure 12.5 3 0 0 0.5 1 1. .8 1 0.5 0.5 2 2.7: Frequency response of lowpass (a) and highpass (b) ﬁlters.5 1 1.4 0.5 1 1.4 0.5 2 2.5 −0.2 0 0 1. functions 1 1 0.5 3 (a) (b) Figure 12.5 2 2.158 CHAPTER 12.5 2 2.5 2 2. filters 1.5 3 200 100 0 −100 −200 0 0.6 0.6 shows the graph of the biorthogonal scaling functions and wavelets derived in option (a) of case (iii).6: Biorthogonal quadratic Bspline scaling functions (a) and wavelets (b).2 0 0 0.6 0.5 3 0.5 1 0.7 plots the frequency response of the corresponding ﬁlters.5 2 2.5 1 1.5 0 0 −0.5 0.5 1 1.5 3 0 0 0.5 1 1.5 2 2. and Figure 12.5 −1 0 0.5 3 200 100 0 −100 −200 0 0.5 1 1.5 2 2.
5 3 (a) (b) Figure 12.9: Frequency response of lowpass (a) and highpass (b) ﬁlters.5 2 2.8 0.5 0 −0. functions 1 0.5 0 0.5 1 1. .5 1 1.5 3 (a) (b) Figure 12.5 −1 −1.6 0.5 0 0.5 2 2.5 3 1 0.6 0.5 3 1 0.5 3 1 0.8 0.5 0 −0.2 0 0 0.2 0 0 0.6 0.5 1 1.5 1 1.5 0 −0.5 3 1 0.8 0.9 plots the frequency response of the corresponding ﬁlters.5 0 0.2.5 2 2.4 0.6 0. and Figure 12.5 1 1.5 2 2.8 0.5 2 2.4 0.5 1 1.5 2 2. filters 1 0.5 1 1. FILTER DESIGN FRAMEWORK FOR BIORTHOGONAL WAVELETS 159 Figure 12.5 −1 −1.5 2 2.8 shows the graph of the orthogonal D2 Daubechies scaling functions and wavelets derived in option (b) of case (iii).4 0.2 0 0 1 0.8: D2 Daubechies scaling functions (a) and wavelets (b).2 0 0 0.5 3 1 0.5 −1 −1.5 −1 −1.5 0 −0.5 0 0.5 2 2.12.5 1 1.5 3 0.4 0.
39) where M = 2 is even.3 Symmetric Biorthogonal Wavelets One of the advantages of biorthogonal over orthogonal wavelets is that we can have symmetry.1 BSpline Wavelets BSplines of order N are piecewise polynomials that come from the convolution of N box functions. For a ﬁxed M .10 we show one example of a biorthogonal Bspline wavelets and scaling functions of order 2 (linear spline).3. must be assigned either to m0 or to m0 . ( 1+z ). BIORTHOGONAL WAVELETS 12. We have. there is a closed form expression for the Bspline scaling functions φ and wavelets ψ. corresponds to a convolution with a box function. M = 2. In this section we present some families of symmetric biorthogonal wavelets. 12. . 2 To construct biorthogonal Bspline wavelets we can split the polynomial P introduced in the previous chapter as: m0 (z) = 1 + z −1 2 M (12. {zi .160 CHAPTER 12. Also.36) where each pair of roots. using diﬀerent factorizations of the polynomial P . thus a family of biorthogonal Bspline wavelets. Second. −1 For symmetric ﬁlters we have to factorize P such that the roots zi and zi must stay together when we split the roots between m0 and m0 . 1/zi }. there is an inﬁnity of choices for M . with M = 2 and 4. Note that this formulas give explicit expressions for m0 and m0 . This family of wavelets has two main advantages: First. In terms of the twoscale relations. or M = 2 + 1 is odd. developed by Ingrid Daubechies (Daubechies. we remark that m0 depends only on M . 1992). resulting in biorthogonal functions with increasing regularity and support width. they can be constructed from a ﬁlter function m0 (z) of the form m0 (z) = −1 1 + z −1 2 N (12.37) Each factor. we have z N P (z) = aN N (z − zi )(z − i=1 1 ) zi (12. the coeﬃcients for all ﬁlters m0 . while m0 depends on both M and M . and m0 (z) = 1 + z −1 2 M + −σ m=0 + −σ+m m 1−z 2 m 1 − z −1 2 m (12. In Figure 12. in which the synthesis scaling function φ is a Bspline of order M . or σ = 1 if M is even. Therefore. and σ = 0 if M is odd.38) where M = 2 is even or M = 2 + 1 is odd.
SYMMETRIC BIORTHOGONAL WAVELETS 161 Figure 12.10: Biorthogonal Bspline Wavelets of order 2 (from (Daubechies. are dyadic rationals.3. The major disadvantage of biorthogonal Bspline wavelets is that the analysis and synthesis . m0 . 1992)). m1 . m1 . which makes them suitable for fast and exact computer implementations.12.
we can generate all the diﬀerent combinations of roots. This may or may not be a problem in some applications.2 Wavelets with Closer Support Width It is possible to construct wavelets with closer support width by choosing an appropriate factorization of the polynomial P .3. as can be seen in Figure 12. {zj . BIORTHOGONAL WAVELETS functions have very diﬀerent support widths.11 shows an example of symmetric biorthogonal wavelets designed using this method.(from (Daubechies.162 CHAPTER 12. Therefore.40) For a ﬁxed degree N = + . z j } are assigned either to m0 or to m0 . z j } P (z) = c i=1 (z − ri ) j=1 (z − zj )(z − z j ) (12. 1992)) . We determine all roots of P .11: Biorthogonal Wavelets with Similar Support. Figure 12. we have a limited number of factorizations. that have both linear phase and similar degree. 12.10. that makes the lengths of m0 . real zeros ri and pairs of complex conjugate zeros {zj . The goal is to ﬁnd ﬁlter functions m0 and m0 . and select the option + . such that ri . and m0 as close as possible. Figure 12.
1+z 2 2K S(z) = 1 + z −1 2 2K K−1 k=0 K −1+k k 2K 1−z 2 2K 2k 1 − z −1 2 2k (12. P. . These experiments lead to the discovery that the Burt ﬁlter is very close to an orthonormal wavelet ﬁlter. 532–540. An extensive treatment of this subject can be found in (Chui. C. the ﬁlters m0 and m0 . designed by P. An introduction to wavelets. was then generalized by Daubechies to a family of biorthogonal wavelets that are close to orthogonal. Chui. Barlaud tried to construct biorthogonal wavelets using the Laplacian pyramid ﬁlter. COMMENTS AND REFERENCES 163 12. including the factor R(z N ). and the constants a.3 Biorthogonal Bases Closer to Orthogonal Bases Motivated by a suggestion of M. The name “prewavelets” is also employed to designate this type of wavelets. and m0 (z) = S(z) + bR(z) (12. therefore they are very close to an orthogonal ﬁlter.41) where the S. Figure 12. 31(4). Ingrid Daubechies developed a family of biorthogonal wavelet bases that are close to an orthogonal basis.43) Note that. as discussed in the previous chapter.12 shows a plot of a wavelet constructed using this procedure.. Barlaud. & Adelson. the scaling function and wavelets are called semiorthogonal. 1983). K.. diﬀer only by the constants a and b. H. The ﬁlter functions for this family are deﬁned as m0 (z) = S(z) + aR(z). 1992) and (Vetterli & Herley. The Laplacian Pyramid as a Compact Image Code. IEEE Trans. 1992. Bibliography Burt. It is possible to construct biorthogonal wavelets such that the primal and dual scaling and wavelet bases generate a single orthogonal multiresolution analysis.12. Academic Press. Burt (Burt & Adelson.4 Comments and References The ﬁrst examples of biorthogonal wavelets were developed independently by (Cohen et al. 1983. R are given below. J.42) R(z) = 1 − z −2 4 1 − z2 4 (12. The particular construction for the Burt ﬁlter. 12. E. Commun. She used the extended formula for the polynomial P .4.3. 1992). . as either m0 or m0 . 1992). In this case. b are computed by an optimization procedure to guarantee the biorthogonality of the ﬁlters.
A. Pure Applied Math. Ingrid. Strang. Vetterli.. 1992)) Cohen. Wavelets and Filter Banks. Daubechies. Wavelets and ﬁlter banks: theory and design. Biorthogonal Bases of Compactly Supported Wavelets. 1992. M. PA: SIAM Books. 1996. Comm. Philadelphia. 1992.12: Biorthogonal Wavelets Close to Orthogonal. & Herley. 40(9). Truong.164 BIBLIOGRAPHY Figure 12.(from (Daubechies.. Daubechies.. Ten Lectures on Wavelets. Gilbert. C. I. Speech Signal Process. J. . & Feauveau. 1992. IEEE Trans. Wellesley.. & Nguyen. Acoust. C. MA: WellesleyCambridge Press.
the application base is consolidating and expanding to new areas. entirely dedicated to the various aspects of wavelets. Even some of the introductory books cover a lot of material with a fast pace. and in the beginning of the 1980’s. The objective was to develop tools to describe functions in time and frequency simultaneously. was formalized originating the theory of wavelets. ranging from pure mathematics and physics to engineering and signal processing. The separate lines of investigation reached a mature point. Independent research in these areas pursued similar goals using diﬀerent approaches. research is perhaps even more active than before. with new results appearing from a growing scientiﬁc community.1 History and Motivation The ﬁeld of wavelets. For this reason. there are currently more than 20 textbooks. Considering the facts mentioned above. and also to give a signiﬁcant impulse to new research. Indeed. is vast and is developing very rapidly. wavelets are well established. This is true both in relation to theoretical aspects of wavelets. the conﬂuence of this interdisciplinary sources. despite being relatively recent. or for people with basic knowledge of wavelets. we believe that there is still a need for a conceptual textbook about the fundamentals of wavelets. it would not be possible to cover the whole ﬁeld of wavelets in a single text. Wavelets are a product of ground work from many areas. the enormous number of conference proceedings and special issues of journals devoted to wavelets. The subsequent uniﬁcation of the ﬁeld was a key factor to make wavelets popular in applied mathematics. Today. and give some directions for further studies on the subject of wavelets. Nonetheless. 13. with new experimental systems and commercial products being released. Also. assuming maturity and dedication from the reader. The basic theory is completely developed with successful applications in a large number of areas. Most of these texts are for experts. as well as to applications of wavelets. Not to mention. 165 .Chapter 13 Directions and Guidelines In this concluding chapter we will review the material covered in the notes.
This approach combined the concepts from function analysis with the intuition from signal processing. We also discussed only the basic schemes for timefrequency decomposition of a function. In these notes. DIRECTIONS AND GUIDELINES 13. applying the wavelet framework to solve problems in applied mathematics. 2. Below. • the continuous Wavelet transform as an adapted timefrequency function decomposition. • the multiresolution representation as a tool to discretize and reconstruct functions. It is instrumental to take a look back and review the main topics that we covered in these notes. Lastly. • and ﬁnally we described the main families of wavelet bases. we tried to give a comprehensive. Our goal was to follow a route from Fourier Analysis to wavelets.2 A Look Back In these course notes. and will indicate some of the possible options for further studies. • the Fourier transform as a tool for frequency analysis. we didn’t consider any concrete application of wavelet. we restricted ourselves to functions deﬁned on the whole real line. but accessible. introduction to the ﬁeld of wavelets. the reader can continue the learning process in various directions. which correspond to signals with inﬁnite duration. we have emphasized descriptions using nonredundant wavelet bases. • discrete wavelet bases and a ﬁlter bank methodology for design and computation. these items constitute only the fundamental concepts behind wavelets. extending wavelets to general domains. This is the simplest case.166 CHAPTER 13. This is perhaps the most important representation in practical applications. we will discuss some of the important aspects of wavelets not covered in these notes. besides a few simple example scattered throughout the notes. . We now brieﬂy discuss what has been done in these three areas. and certainly makes the theory more accessible.3 Extending the Basic Wavelet Framework The basic wavelet framework presented in these notes can be further developed in three main directions: 1. • the Windowed Fourier transform and the search for timefrequency localization. As we stated earlier. They include: • motivation and schemes for representation of functions. deﬁning generalized representations using timefrequency decompositions. In particular. In order to go from this basic level to more advanced topics and to practical applications. 13. showing how the traditional tools for function representation in frequency domain evolved to the new tools for joint timefrequency descriptions. and 3.
to study wavelet transforms and multiresolution analyses on the space L2 (Rn ). and leads to a separable wavelet transform which can be implemented eﬃciently by multipass 1D methods. Some of these representations rely on a redundant dictionary of timefrequency atoms that can be used to construct descriptions adapted to individual functions. Also.3.13. Some concepts and results extend naturally. The second way is through a multidimensional multiresolution hierarchy generated by a single multivariate scaling function in Rn (St¨ckler. . 1992). 1992. we clearly need a methodology diﬀerent from the Fourier Analysis. also known as Malvar wavelets (Malvar. Some examples are the wavelet packet basis (Wickerhauser. The ﬁrst way is based on a tensor product of onedimensional wavelets. 1995). 1994) and (Chen & Donoho. That is. o Another important extension is to deﬁne wavelets on compact subsets of the Euclidean space Rn . EXTENDING THE BASIC WAVELET FRAMEWORK 167 13. There are two ways to construct wavelets in Rn . (Daubechies. 1998) are promising approaches in this direction. and the local cosine basis.3. Similar techniques have been reported in (Chen & Donoho. For this purpose. Another generalization consists of decompositions based on diﬀerent tilings of the timefrequency plane. Strang & Strela. 1992). 1994). . which is not generated from a single function anymore. It is natural to deﬁne wavelets in ndimensional Euclidean spaces.3. The ﬁnal relevant extension should point us in the direction of deﬁning wavelets and multiresolution analysis on arbitrary manifolds. 1993) This construction extends naturally to rectangular regions of higher dimensions. we have to construct a multiresolution of L2 (I). We have several options in this direction. 1992). 1992). In particular. A particular case concerns wavelets that are adapted to the unit interval I = [0. For this extension. Mallat & Zhong. Note that the methods we have described in these notes make strong use of the Euclidean space structure. The path to follow consists in extending the 1D results that we have obtained to dimension n > 1. 13. because of the translations used to obtain the wavelet basis. dyadic wavelets give a translation invariant representation that can be used for characterization of singularities and edge detection (Mallat & Hwang. 1995). Redundant wavelets bases are required for some operations. 1990).1 Studying Functions on other Domains The ﬁrst generalization is to study wavelets other domains diﬀerent than the real line. 1995) and the similar construction in (Dahmen & Schneider. 1993) are some of the optimization methods used to generate these representations. 1994). and matching pursuit (Mallat & Zhang. This extension is straightforward. known as multiwavelets (Malvar. Best basis selection (Coifman & Wickerhauser. the steerable pyramid provides a orientation tuning mechanism (Simoncelli & Freeman. 1]. A last generalization is a decomposition using a basis composed of several diﬀerent functions. 1990.2 Deﬁning other TimeFrequency Decompositions The second extension is to formulate function representations based on other timefrequency decomposition schemes. The Lifting scheme (Sweldens. but includes functions that adapt to the boundaries of the interval (Cohen et al.
animation video sound and multimedia.4 Applications of Wavelets Wavelets have been applied in may diﬀerent areas. Wavelets have been used to solve integral equations (Beylkin et al. diﬀerential equations (Bacry et al. This means that we will use wavelets to represent operators and to exploit the properties of this representation in order to derive more eﬃcient computational schemes. . and optimization problems. . We list below some of the main problems associated with the representation and processing of: geometric models.3 Solving Mathematical Problems The third direction is to study how wavelets can be applied in the solution of mathematical problems. We remark that this last scheme has strong connections with multigrid methods. • Signal and Image Processing – Data compression – Progressive Transmission – Noise reduction – Filtering • Vision – Edge detection – Texture Analysis – Feature Classiﬁcation • Visualization and Graphics – Radiosity – Volume Rendering – Paint Systems • Geometric Modeling – Wavelets on surfaces and Variable resolution meshes – Mulitscale Editing – Optimization of geometry – Synthesis of fractal surfaces . 1992).3.168 CHAPTER 13. images. DIRECTIONS AND GUIDELINES 13. that have been successfully solved using wavelets. 1991). 13. Here we will give an overview only of the applications in Computer Graphics and related areas.
1992. 1995. Stanford University. G. Shaobing.wavelet. Basis Pursuit. Fournier. & Papanicolaou. A. A wavelet based space– time adaptive numerical method for partial diﬀerential equations.. David L. we mention that the Internet is a valuable source of information on current research and application of wavelets. Chen. 1993. Mallat. 54–81.5. Applied and Computational Harmonic Analysis.5 Comments and References As a last remark. Cohen. 1996) and (Stollnitz et al. 1994. Stephane. A book dedicated entirely to this subject is (Stollnitz et al. David L. and other online resources. 1994). Emmanual. Shaobing. . COMMENTS AND REFERENCES 169 • Animation – Timespace constraints – Motion learning • Other Areas – Medical Imaging – Geology – GIS and Cartography – Music – Speech synthesis and recognition – Databases for images and video. A good review of wavelet application in computer graphics can be found in (Schroder.. 1991. . Stollnitz et al. an electronic newsletter moderated by Win Sweldens (http://www. rept.13. 1(1). Atomic Decomposition by Basis Pursuit. we would like to single out one: “The Wavelet Digest”. From the many excellent websites. Chen. Mathematical Modelling and Numerical Analysis. 26. 793–834. 1995. R. & Vial. Other sources of reference are the previous SIGGRAPH courses dedicated to wavelets (Schroder & Sweldens. 1995b). & Donoho. Beylkin. Communications on Pure and Applied Mathematics. Stanford University.. & Rokhlin. 1996). Wavelets on the interval and fast wavelet transforms. 141–183. Tech. Daubechies. Bibliography Bacry. . I. rept.. Tech. 13. .org/). 1995a. transformation en ondelettes. V. XLIV. Fast Wavelet Transforms and Numerical Algorithms I. George. Coifman. P. & Donoho.
15(4).170 BIBLIOGRAPHY Coifman. 1995a. Matching Pursuit with Timefrequency Dictionaries.edu/teaching/courses/waveletcourse/. 1990. Mallat.). The Steerable Pyramid: A Flexible Architecture for MultiScale Derivative Computation. 84. Modulated QMF Filter Banks with Perfect Reconstruction. SIGGRAPH Course Notes. 15(3). S. & Wickerhauser. SIGGRAPH Course Notes. A. 1992. (ed). San Francisco.ps. IEEE Transactions on Information Theory. rept.. 2104–2107. Wolfgang.Z. Academic Press. Mallat. Tony D. W. S. Orthogonal multiwavelets with vanishing moments. Tony D. 1996. Wavelets for Computer Graphics: a Primer. Institut fur Geometrie und Praktische Mathematik. 1992. 710–732..cs. Wavelets in computer graphics.multires. rept. & Salesin.. William T. Alain. 1995. Singularity detection and processing with wavelets. Fournier. & Hwang. Part 1. Schroder.caltech. http://www. Stollnitz. Electronics Letters. vol. 713–718. Malvar. 1992. . Wavelets in Computer Graphics. part 2. DeRose. & Freeman. Strang. St¨ckler. J. Simoncelli. Pages 615–625 of: Proceedings of the IEEE. & Zhang. Wim.. Mallat. 14(7). http://www. Wavelets for computer graphics: a primer. Pages 444–447 of: International Conference on Image Processing.saveme. Optical Engineering. K. DeRose. & Zhong. 38(2). 906–907.. Courant Institute of Mathematical Sciences. H. Tech.. Eero P. Reinhold. & Sweldens. L. Tony D. & Salesin. Gilbert. IEEE Computer Graphics and Applications.ubc. S. S. 1992. CA: Morgann Kaufmann. Wavelets for Computer Graphics: Theory and Applications.. Ingrid. David H. 1995 (Oct. 1992. 33(7). Pages 325–356 of: Chui. vol. 75–85. Dahmen. Philadelphia. Eric J. Mladen Victor. 1994. 1996. Stollnitz. Ten Lectures on Wavelets. 26(June). & Strela. IEEE Computer Graphics and Applications. Peter.. Daubechies. Eric J. PA: SIAM Books. DeRose. 617–643. 1998. Ronald R. G.ca/nest/imager/contributions/bobl/wvlt/download/notes. 76–84. S. Stollnitz. David H. Tech. Wavelets and their applications in computer graphics. & Schneider.. 38(2).. & Salesin. EntropyBased Algorithms for Best Basis Selection. Vasily. Characterization of signals from multiscale edges. Eric J. 1994. Wavelets: A o Tutorial in Theory and Applications. 1995b.. IEEE Transactions on Pattern Analysis and Machine Intelligence. Wavelets on Manifolds I: Construction and Domain Decomposition. C. Peter. IEEE Transactions on Information Theory. David H. 3. 1993. Multivariate wavelets. Schroder.
1995:6. The lifting scheme: A construction of second generation wavelets. Department of Mathematics. Wickerhauser. 1994. K. Tech. University of South Carolina. MA: A. Peters. Wellesley.BIBLIOGRAPHY 171 Sweldens. . Adapted Wavelet Analysis from Theory to Software. rept. W. Mladen Victor. 1995.
.
as illustrated in Figure A. a signal is modeled by a function. the basilar membrane. either attenuated or emphasized. A system may change the input signal in diﬀerent ways. We can give some examples: • The human eye is a system that process electromagnetic signals in the visible range of the spectrum. In the mathematical universe. It capture sounds. we can model a system by a transformation S : F1 → F2 between two function spaces. producing an output signal. but we are also capable to identify the source of the sound. A good example of a system is the human sound perception. 173 .1 Systems and Filters The word “system” has a very generic meaning in engineering. the input sound is analyzed. The systems constitute the mathematical model to study the various operations involving processing of signals in the physical universe. A.1.Appendix A Systems and Filters In this appendix we will review the basic concepts of Signal Processing to introduce the terminology and notation. and other elements of the auditory system). This complex task is accomplished by a long chain of processes. separating its various frequency components using a set of ﬁlters with diﬀerent sensibilities (this is performed by the cochea. The term “ﬁlter” is employed to designate certain types of systems that alter only some frequencies of an input signal. Among other things. Therefore. in a way that we are not only able to understand the meaning. The name has exactly this meaning: it implies that a selection is taking place and some frequency bands are altered.1: Basic System. f −→ System −→ S(f ) = g Figure A. classiﬁes and interprets them.
we describe the most relevant properties for our study: Linearity A system S is linear if: 1. they have only one variable which usually represents the time evolution. as for example: ∞ f= j=−∞ f. this time analogy is not valid anymore (even though time may still be one of the variables). a system is characterized by some of its properties. SYSTEMS AND FILTERS • A television camera has as input an electromagnetic signal (as the human eye). That is. If a system S is linear and a signal f is represented by an atomic decomposition. the operator R : F1 → F2 deﬁned by R(f ) = f (t − t0 ) is called delay operator by t0 . or translation operator by t0 . When we work in higher dimensions. and its output is a video signal. . Geometrically it makes a translation of t0 units to the right in the graph of f . i. This class of system is generally called “time invariant” when the functions are of type f : R → R.e. ϕj ϕj Its output S(f ) can be written as: ∞ S(f ) = j=−∞ f. S(f + g) = S(f ) + S(g) 2.1. S(f (t − t0 )) = S(f )(t − t0 ). Given t0 ∈ R . Below. which is equivalent to a delay of t0 time units in the signal (see Figure A. A.174 APPENDIX A. before or after its processing by the systems.2). and yet obtain the same result. ϕj S(ϕj ) That is. it is suﬃcient to known the output of the system for the basic decomposition atoms to predict the processing of any signal in this representation. In particular. S(αf ) = αS(f ) where f and g are two input functions (signals) and λ ∈ R. S(0) = 0. Spatial Invariance A system S is spatial invariant if it is possible to perform shifts in the signal.1 Spatial Invariant Linear Systems In general.
we will concentrate our attention only on these two properties mentioned above: linearity and time invariance. A.2: Delay Operation. It is easy to verify that f ∗ h = h ∗ f . for this reason. and is called convolution product. we have: f (x) = ∞ −∞ f (t)δ(x − t)dt If the system is spatial invariant.2 Other Characteristics For the moment. If f is an arbitrary signal. More precisely. This result is summarized in the theorem below: Theorem 8. The impulse response. A system S is linear and spatial invariant if a delay in the input is equal to the same delay in the output SRt0 = Rt0 S Impulse Response In this subsection we will characterize the spatial invariant linear systems. S(f ) = h ∗ f for any signal f . it is called ﬁlter kernel.A. A spatial invariant linear system S is completely determined by its impulse response h = S(δ).1. Nonetheless. h of a system S is the image h = L(δ) of the signal Dirac δ by the system. it follows from the above equation that: S(f (x)) = = ∞ −∞ ∞ −∞ f (t)S(δ(x − t))dt f (t)h(x − t)dt The integral above is indicated by f ∗ h. SYSTEMS AND FILTERS 175 Figure A.1. there are many other important characteristics that deserve to be mentioned: . The impulse response h gives a complete characterization of the system and.
A common stability condition is to enforce continuity of the operator which deﬁnes the system: S(f ) ≤ cf  In other words. That is. L < ∞. Therefore. the system does not have any knowledge of the future. if the input signal has ﬁnite energy.2 Discretization of Systems We have to work with discrete systems to be able to implement the model of continuous systems in the computer. because sounds are processed strictly in chronological order. SYSTEMS AND FILTERS Finite Impulse Response A system S has ﬁnite impulse response. L]. A. each of them related to the needs of diﬀerent applications. Independently of a system being intrinsically continuous or discrete. The name originates from the fact that in the discrete domain.2. various phenomena in the physical universe clearly have a discrete nature. Causality Causality is a property which says that the output of the system depends only of previous facts in the input. Discrete systems operate with discrete signals. . we need to resort to methods to represent systems. in actual computer implementations we need to work with discrete representations of the system. this condition reduces to: sup(S(f )) ≤ c sup(f ) A system which satisfy this condition is called in the literature BIBO system (“Bounded Input – Bounded Output”). if its impulse response has compact support. which ultimately are methods to represent functions. Other phenomenon. for example. (FIR). Taking the norm of sup: f  = sup(f ). Stability Many stability conditions can be deﬁned for a system. the same occurs with the output signal. A good example of a system in this class is the human ear. may even be conceived as continuous. the impulse response of these ﬁlters is represented by a ﬁnite sequence. but they are observed (measured) at discrete instants of time. such as turning a light on or oﬀ. the variation of the tides.176 APPENDIX A. A.1 Discrete Signals As we have seen. supp(S(δ)) ⊂ [−L.
R(F1 ) = Rm and R(F2 ) = Rn . When the signal f is represented by a sequence (fn ). .. then R(f ) = k f. . The representation of a ﬁnite energy signal is deﬁned by an operator R : F → 2 . the samples of the signal are given by f (j) = f. Therefore. the reader could imagine that a sampling sequence is represented by uniform sampling the signal with rate ∆t: f (n) = f (n∆t) (we have seen that this is a good approximation for most cases of interest). Nonetheless. . . the description of the data elements which represent the physical measurements of the signal. . it is important to know the sampling rate ∆t. We will use the notation f [n]. is done through the integer variable n. DISCRETIZATION OF SYSTEMS 177 When we represent a signal f we obtain a discretization that associates to the signal a sequence of “samples” (fn ). Sometimes.A. . f (2). f (−1). . we will write (fn )n∈Z to emphasize that we are referring to a sequence. he have in fact a representation vector (fn ) = (f (i). ϕj ϕj . an1 · · · anm . that is. that is. . then S : Rm → Rn is a linear transformation. . the representation sequence is an inﬁnite vector (fn ) = (. to 2 −− −→ S 2 where S is the representation of the system S. for example. A sequence of samples of the signal will be indicated by (fn ). ϕj . the discretized system S. f (n) or even fn to indicate the nth “sample” of the signal. . . Otherwise. . f (1). If the representation sequence is ﬁnite.2. f (i + 1). . . . The most common form of representation of a signal is given by its projection in a closed subspace V ⊂ F. If {ϕj } is an orthonormal base of this space. we can take the representation R1 : F1 → produce the following diagram. It is important to have a memory about the representation method employed. If S is linear and F1 . of the space of signals into the space 2 of the square summable sequences: R(f ) = (fn ). and can be represented by a matrix of order n × m: a11 · · · a1m a21 · · · a2m . S −→ F2 F1 −− R R 1 2 e R2 : F2 → 2. f (j)). . f (0). In the case of point sampling. n ∈ Z. F2 have a ﬁnite dimensional representation.2 Discrete Systems 2 Given a system S : F1 → F2 . specially to make possible the reconstruction of the signal. ) A. . f (−2).2.
−1 a0.1 . . we know that S is given by a matrix (possibly inﬁnite). 0 if k = 0. . If the system S is linear and spatially invariant.1) below. where (δn ) is the discrete version of the Dirac delta: (δn )(k) = then S(δn ) = h ∗ δn = h.0 a0. Indicating by R the unit delay operator: that is R(fn ) = (fn−1 ) we have that f (n − k) = Rk (fn ) . h = S(δ). .0 a−1. we have S : It is useful to represent these operators by inﬁnite matrices . As it was expected. In the discrete domain h = (hn ) and f = (fn ).1) which deﬁnes the output of a discrete linear system is a ﬁnite diﬀerence linear equation: +∞ 1 if k = 0. . . . SYSTEMS AND FILTERS 2. .1 x(1) y(1) a1. (A. we will obtain the matrix S using equation (A. S(f ) = h ∗ f 2 → where h is the kernel of the ﬁlter. where S(fn ) = (yn ). we need some linear algebra notation. x(0) y(0) = a0.−1 a1. y(n) = k=−∞ h(k)f (n − k). hence: S(fn )(n) = ((hn ) ∗ (fn ))(n) +∞ = k=−∞ h(k)f (n − k). x(−1) y(−1) a−1.−1 a−1. . In case the representation of the spaces F1 .. . we know that.. The reader should observe that equation (A.1) which is the direct expression for the convolution product. F2 is not ﬁnite dimensional. . .0 a1. . For this. Because S is linear.1 . Now.178 APPENDIX A. Taking (fn ) = (δn ). . .
we have f (n − 0) = f (n) = R0 (fn ) = I(fn ) where I is the identity operator. Similarly.2) Given a matrix (aij ). . if k = 0. a0.. the elements (aj+d. and the other elements are zero. 0 0 0 • 0 0 0 • 0 0 0 • 0 0 . In particular. ..j = 1 I−1 = ai. . whose elements in the main diagonal are equal to 1.2 .j = 0 if i = j + 1.A. which can be inﬁnite..j ) are the elements of the dth lower diagonal. DISCRETIZATION OF SYSTEMS 179 in other words. a2. f (n − k) is obtained from the signal (fn ) applying k delay operations. The unit delay operator R(fn ) = f (n − 1) is represented by the matrix I−1 whose ﬁrst lower diagonal consists of ones and the other elements are zero: aj+1.j = 0 if i = j.1 . .j+d ) are the elements of the dth upper diagonal.0 a1. the elements (aj. In this way.1) can be written in operator form: +∞ S(fn ) = k=−∞ h(k)Rk (fn ) (A.2. . equation (A.. the elements ajj constitute the main diagonal.j = 1 ai. d > 0. The identity operator is represented by the identity matrix I= aj. For each d ∈ Z. .
180 APPENDIX A. . considering the upper diagonals. we can either shift the input (xn ) before the multiplication... lower and upper diagonals are constant. (A. Example 17. Consider the ﬁlter deﬁned by the diﬀerence equation y(n) = 3x(n − 1) + 2x(n − 2) + x(n − 3) Then. or shift the output (yn ) after the multiplication: the result will be the same. 0 1 0 0 1 0 0 1 0 . that is. More generally. Similar results hold for the operation f (n) −→ f (n + k) of time advance.. the matrix is lower triangular. translated by one unit to the right from one line to the next. the matrix of the operator Rk .j = 0 if i = j + k. Observing equation A. . the kth diagonal is constituted by the elements h(k). we can rewrite this diﬀerence equation in the form of a convolution product y(n) = k h(k)x(n − k) .2 we see that the matrix of S is a matrix whose main. .3) . all elements outside the kth lower diagonal are zero. because the system represented by this matrix is linear and spatially invariant.j = 1 ai. . It is easy for the reader to verify that if the system is causal. · · · h(1) h(0) h(−1) ··· · · · h(1) h(0) h(−1) ··· ··· h(1) h(0) h(−1) ··· ··· h(1) h(0) h(−1) · · · . More precisely. Observe that in the rows of this matrix we have the vector of the system impulse response. and the elements of this diagonal are equal to one. SYSTEMS AND FILTERS .. shifted by k units is: I−k = aj+k. all upper diagonals are zero (because the system cannot use advances of the signal in time). This was expected. Because the output (yn ) will be the result of the inner product of the nth line of the matrix by the input vector (xn ). Therefore.
. = 2. is a sequence which varies with the index (time). .2. .A. . therefore. . . 0. . h−1 (n).. . . 0 x(3) x(1) ˆ 3 0 x(4) x(2) ˆ 2 3 0 . . What would happen if the coeﬃcients of the diﬀerence equation were not constant? The equation would be described as. . . . .. . x0 · · · x1 . 0 3 h(k) = 2 1 0 if if if if if k k k k k ≤ 0. h0 (n). the matrix representation does not have constant . DISCRETIZATION OF SYSTEMS 181 where. diagonals: . . . ) The convolution product cannot be used to characterize the operation and its “generalization” is (yn ) = hk (n)xn−k k Finally. yn = · · · + h0 (n)xn + h1 (n)xn−1 + h2 (n)x In this case the ﬁlter kernel (hn ). . . 3. x(0) . 0. . ≥ 4. . alternatively (hn ) = (. 2. . . or. The matrix of this ﬁlter is given by .. . ). x(5) = x(3) ˆ 1 2 3 0 x(6) x(4) ˆ 0 1 2 3 0 x(7) x(5) ˆ 0 1 2 3 0 . = 3. . . 0. = 1. y−1 · · · h1 (−1) h0 (−1) h−1 (−1) y0 = · · · h2 (0) h1 (0) h0 (0) y1 · · · h3 (1) h2 (1) h1 (1) . . . . . . h1 (n). . it is necessary now to represent it as: (hn ) = (. . . 1. Note that the ﬁlter is causal Question 1. . . · · · x−1 · · · .. 0. . .
that is (↓ 2)(↑ 2) = I. Here. . is deﬁned as (↓ q)(un ) = (unq ). . 1 we can deﬁne the upsampling operator of order q. ··· This matrix is obtained from the identity matrix I. In this case the operator discards alternating terms in the sequence. ↓ q : 2 → 2 . . . . . . . The upsampling operator simply intercalates zeros in between the elements of the representation sequence. That is. except the terms which are multiple of q. which is the upsampling operator: (↑ 2)(. x1 . . by including a column of zeros in alternation. as we can see below: . . x−1 . we can shift the lines of the matrix one unit to the right in alternation. SYSTEMS AND FILTERS A. The matrix of this operator is given by 1 0 0 1 0 0 1 0 . we will only study the case where q = 2. u−2 . . . 0. . . That is. . intercalating q zeros.182 APPENDIX A. 0 . retaining only the terms with even index: (↓ 2)(un ) = (u2n ). ) = (. x−1 . (↑ 2)(↓ 2) 0 u1 u2 . (↑ 2)(↓ 2) = I. 1 It is easy for the reader to verify that the upsampling operator ↑ 2 is the inverse to the right downsampling operator ↓ 2. . 0. x0 . (↑ 2)(un )(k) = u(k) if n = 2k 0 if n=2k+1. x1 . . However. . . x0 . ). However. . It is immediate to verify that the downsampling operator is not invertible. The downsampling operator of order q. 0. it has a inverse to the right. two other operators are important in the study of discrete ﬁlters: they are the downsampling and upsampling operators. 0 u−1 u 0 = u0 .3 Upsampling and Downsampling Operators Besides the delay operator. . Or equivalently. This operator is also known in the literature as the decimation operator of order q. the operator discards all terms of a sequence.
At this point we can argue that the nature of the mathematical objects is against us: as we will see..3. x H 2 2 G ^ x Figure A. and viceversa. it is easy to see that the matrix of the operator ↑ 2 is obtained by intercalating rows of zeros in the identity matrix 1 0 0 1 ↑2= 0 0 0 0 0 1 0 . and upsampling is illustrated in FigureA. A.3: Example of a Filter Bank . and not the other way around. Once more. the most important operation is to recover a signal after downsampling. FILTER BANKS 183 In terms of matrices. we observe that the upsampling operator is the inverse to the right.A. A ﬁlter bank with two ﬁlters. downsampling. and (↑ 2)(↓ 2) = 1 0 1 . 0 1 . It is immediate to verify that the matrix of the upsampling operator is the transpose of the matrix of the downsampling operator. together with delay. upsampling and downsampling operators.4 Filter Banks A ﬁlter bank is a system composed of several ﬁlters.4. but is not an inverse to the left. The relation between the operators of downsampling and upsampling can be stated using matrices: (↓ 2)(↑ 2) = I. . ··· This is equivalent to translate the columns of the identity matrix by one unit to the bottom.
Digital Filters. one of them is (Vaidyanathan. discussing only continuous systems is (Lathi. . Alan S. Vaidyanathan. PrenticeHall inc. 1993).184 BIBLIOGRAPHY A.5 Comments and References A basic reference for signal processing is (Oppenhein & Willsky. Systems. Englewood Cliﬀs. 1974). Andreas. Lathi. 1983). 1993. For a book more applied to the project and analysis of ﬁlters. Second edition edn. Multirate Systems and Filter Banks. Bhagwandas P. from the continuous to the discrete. 1993). Signals. Bibliography Antoniou. McGrawHill. the reader can use (Antoniou. 1993. Oppenhein. Another good source. Alan V. It includes the whole theory of signals and systems. Harper & Row. P. The literature of signal processing has several sources for the reader interested in systems with multirate ﬁlter banks. but somewhat older. New Jersey: Prentice Hall PTR. & Willsky. 1974. P. 1983.. Signal and Systems. and Control.
Appendix B The Z Transform The behavior of ﬁlters in the frequency domain gives a good intuition of some of its characteristics. the output signal is then: +∞ (yn ) = (hn ) ∗ (xn ) = k=−∞ +∞ h(k)x(n − k) h(k)z k=−∞ n−k = =z n +∞ k=−∞ h(k)z −k that is. the value of the output signal y(n) is obtained by multiplying x(n) by a constant This constant. the output is then (yn ) = (hn ) ∗ (xn ). let’s choose an input signal with a special structure: . which are diﬃcult to grasp by looking only at the ﬁlter coeﬃcients in the time domain. we will introduce a similar. and an input signal (xn ). B. H(z) = k=−∞ +∞ h(k)z −k We should remark that if an arbitrary signal is decomposed as a direct sum of basic atoms. We have seen that the Fourier transform is a tool used to study a signal in the frequency domain. time invariant. it would be suﬃcient to know the eﬀect of the system over its atoms to be able to predict the 185 . In this appendix. but more general transform. which was ﬁxed for each input signal: y(n) = H(z)z n where. however.z ∈ Z x(n) = z n in this case. However. which in fact contains the Fourier transform.1 The Z Transform Given a linear. discrete system S with impulse response (hn ). the Ztransform. varies with the value of z.
1 Some Properties Several important properties of the Z transform will help in the manipulation of discrete systems. we obtain the Fourier transform for discrete aperiodic signals (DTFT – “Discrete Time Fourier Transform”). we have chosen our input signal as a sum of diﬀerent complex exponentials: x (n) = k n ak zk and. just by applying the deﬁnition of the transform. we can use its linearity property to obtain: y (n) = k n Using a linear algebra terminology. • Linearity. ∞ Z{x1 (n) + x2 (n)} = Z{x1 (n)} + Z{x2 (n)} Z{x(n − l)} = k=−∞ l z −k ∞ x(k − l) = k=−∞ z −k+l x(k) = z Z{x(n)} • Time Reversal. Given a discrete signal (xn ).1. We will see that these properties contribute to the intuitive view alluded to in the beginning of this Appendix. and H(zk ) are their respective eigenvalues. Based on this transform. such that this transform converges. as the next one. its Z transform is given by: ∞ n ak H(zk )zk X(z) = Z{x(n)} = k=−∞ z −k x(k) (B. is easily veriﬁed. we will introduce the Z transform. Remark 1.1) Together with X(z) it is necessary to know the corresponding Convergence Region. B. applying it to the system S. the regions of the complex plane in which the variable z is deﬁned. making z = ei2πw . are eigenvector functions of S.186 APPENDIX B. the complex exponentials zk . For this reason. Exploiting the above intuition. THE Z TRANSFORM result of the system on any input signal. that is. x(−n) ↔ X(z −1 ) x(n) ↔ X(z) ←→ This property. • Time Shift. .
it is necessary to be careful with the convergence of this expression. and using the properties of the Z transform.2 Transfer Function The impulse response h(t) of a ﬁlter has a correspondent in the frequency domain. and the division operation takes care that the zeros of X(z) are outside the convergence region. but with a diﬀerent “amplitude”. the output was the impulse response function. We have seen when it was the input of a system. discrete system will react in a predictable manner according to the complex exponentials applied to its input. What would be the disadvantages of using (hn )? .B.Z{(xn )} = H(z)X(z) We isolate H(z) = Z{(hn )}. then the Z transform is a good way to describe ﬁlters. Considering that a ﬁlter is exactly the system that makes this type of transformation.1. Second. Z{x1 (n) ∗ x2 (n)} = X1 (z)X2 (z) Remark 2. Going to the frequency domain. we have: Y (z) = Z{(hn ) ∗ (xn )} = Z{(hn )}. THE Z TRANSFORM 187 • Convolution in Time is Multiplication in Frequency.1. A linear. (yn ) = (hn ) ∗ (xn ). and call this expression Transfer Function: H(z) = Y (z) X(z) Again. which is called Transfer Function. using the Z Transform? Suppose that S(xn ) = (yn ). Given a system S. the representation of the delta function in the frequency domain is: Z{δ(n)} = 1 This means that the function delta δ(n) is the neutral element of the convolution operation in time and of the multiplication operation in frequency. The transfer function H(z) gives exactly information on how the input frequencies (xn ) will be altered in order to produce the output (yn ). This connection was one of the goals of our intuitive view. This has two main reasons: First. The function delta. δ. the convolution product in the time domain is equivalent to multiplication in frequency. Its output will be given as the same input exponential. B. time invariant. What would be the interpretation of the fact above in the frequency domain. exhibits an special behavior. which characterizes the ﬁlter. that is. its impulse response h = S(δ) completely deﬁnes the ﬁltering operation S(f ) = h ∗ f.
THE Z TRANSFORM One is immediate: the variable z is not discrete. H(z) = Y (z) = 3z −1 + 2z −2 + z −3 X(z) . but the manipulation would be more diﬃcult and less eﬃcient.e.188 APPENDIX B. To obtain the transfer function H(z) from the linear diﬀerence equation with constant coeﬃcients. i. we can have: N Z{ k=0 N ak y(n − k)} = Z{ k bk x(n − k)} bk z −k )X(z) ( k=0 ak z − k)Y (z) = ( k in this case H(z) is a ratio of polynomials H(z) = k N k=0 bk z −k ak z − k Example 18. and look at its frequency behavior x(n) = 3x(n − 1) + 2x(n − 2) + x(n − 3) ˆ Z{y(n)} = Z{3x(n − 1) + 2x(n − 2) + x(n − 3)} Exploiting the linearity and time shift properties. is a direct operation: the coeﬃcients of the equation are exactly the coeﬃcients of a polynomial in z: Z{y(n)} = Z{ k bk x(n − k)} bk z −k )X(z) Y (z) = ( k and then. it will be instructive to determine the system transfer function. the importance of the Z transform is undisputable. the convolution product. H(z) = k bk z −k In a more general form. The other option would be the use of a symbolic machinery. we have: Y (z) = 3z −1 X(z) + 2z −2 X(z) + z −3 X(z) and factoring out X(z). in the previous chapter. it is also deﬁned in the entire complex plane (in the Convergence Region). To store it in the computer it is necessary to do some kind of “sampling”. For the tasks of ﬁlter design and evaluation. Going back to the system of example 17.
introduced in the previous chapter. bandpass. it is possible also to classify the ﬁlters into basic types. SUBSAMPLING OPERATIONS 189 In order to really visualize the behavior of a ﬁlter in the diﬀerent frequencies that compose the input signal.2. B. where w = [0.1: 6 5 4 3 2 1 3 2 1 1 2 3 Figure B. where π has a direct relation with 1/2 of the sampling frequency used in the discretization process. π]. it is suﬃcient to evaluate the transfer function H(z) for the values of z = eiw . We will derive the expressions for these operators using the znotation. In this last example H(ej w) is shown in Figure B.1: Modulus of the frequency response of a ﬁlter. keeping only even index elements. B.2. Again. takes a discrete sequence (xk )k∈Z and removes all elements with odd indices.1. it makes sense to consider the function H(eiw ) as a periodic function. highpass. π]. where 0 < w < π Writing the complex variable z in polar form.2 Subsampling Operations In this section we will discuss the downsampling and upsampling operators in the frequency domain. etc.1 Downsampling in the Frequency Domain The downsampling operator. we obtain The discrete domain has a limited interval of possible frequencies due to the sampling process. H(eiw ) is also the DTFT of the nonperiodic ﬁlter (hn ) and every DTFT is periodic with a period of 2π Through the behavior of systems in the various frequency regions. There . all these possible frequencies will be contained in the interval [0. as we have seen in Chapter 3. B.B. Making a normalization. Therefore. such as lowpass.3 The Variable z and Frequency z = αeiw .
3) 2 2 2 The downsampling operation corresponds to a change in the sampling rate. if n = 2l is even. . which is essentially a frequency folding. using U (ω) which contains only even index terms. . x0 . preserving the even index elements. (↓ 2)x = (↓ 2)u. then e−i(2l)(ω+π) = e−i2lω+2lπ = e−i2lω . Therefore. u = (. 1 ω ω V (ω) = [X( ) + X( + π)] (B. .2. .4) .190 APPENDIX B. which has all the elements with even index from an arbitrary sequence (xn ). That is. 0. removing the odd index elements. Clearly. x2 . we can write the formula for the downsampling operator in frequency domain just by halving the frequencies V (ω) = U (ω/2). In the Zdomain. so that when they are added together only the terms with even indices remain.2: Downsampling Operator. This may cause aliasing. then e−i(2l+1)(ω+π) = e−i(2l+1)ω+(2l+1)π = e−i(2l+1)ω+π = −e−i2lω . X(w) X(w + π) X(w) Figure B. But. v = (↓ 2)x implies that vk = x2k . let’s write the Fourier transform of u U (ω) = n even xn e−inω 1 2 xn e−inω + all n = 1 2 xn e−in(ω+π) all n (B. if n = 2l + 1 is odd. This is because. THE Z TRANSFORM fore. So. and the elements with odd indices equal to zero un = xn 0 if n is even if n is odd. . 2 Now.2) The expression for U was split into two terms. In order to derive the expression for the downsampling operator in the frequency domain. This process is illustrated in Figure B. we will take a sequence (un ). as it is implied by halving the frequencies ω → ω/2. manifested by the introduction of a new term in ω/2 + π. the expression of U (ω) in the frequency domain can be written in terms of X(ω) as U (ω) = 1 [X(ω) + X(ω + π)]. the downsampling operator is 1 V (z) = [X(z 1/2 ) + X(−z 1/2 )] 2 (B. . ).
X(w) π U(w) = X(2w) Figure B. We only retain the terms with even index n = 2k. because terms with odd index are zero.2.3: Upsampling Operator.3 Upsampling after Downsampling In a ﬁlter bank it is common to apply both the downsampling and upsampling operations. u = (↑ 2)v ⇔ u2k u2k+1 = vk =0 In the frequency domain.B. Let’s see how these two operations can be combined together: v = (↓ 2)x and u = (↑ 2)v this result is u = (↑ 2)(↓ 2)x.2 Upsampling in the Frequency Domain The upsampling operator.5) So. and upsampling is part of the synthesis bank.3. U (ω) = = un e−inω = vk e−i2kω u2k e−i2kω (B. which in the frequency domain is 1 U (ω) = [X(ω) + X(ω + π)] 2 or in the Zdomain 1 U (z) = [X(z) + X(−z)] 2 (B.2. also introduced in the previous chapter. or U (z) = V (z 2 ). this operation has a simple expression. This is illustrated in Figure B. u = (↑ 2)v is U (ω) = V (2ω).7) (B. SUBSAMPLING OPERATIONS 191 B.6) . B. Compressed copies of the graph V (2ω) appear in the spectrum of U (ω). usually downsampling is part of the analysis bank.2. This is because. takes a discrete sequence (xk )k∈Z and an interleave zeros inbetween the sequence elements. A bandlimited function V (ω) with period 2π is mapped into a function U (ω) = V (2ω) with period π. The upsampling operation causes imaging. u2k+1 = 0.
MA: WellesleyCambridge Press.192 BIBLIOGRAPHY B. P. New Jersey: Prentice Hall PTR. . & Nguyen. Englewood Cliﬀs. Wellesley. 1996).3 Comments and References A good reference for the use of ﬁlter banks and their relations with wavelets is (Strang & Nguyen. P. Vaidyanathan. 1993. 1996. Bibliography Strang. Wavelets and Filter Banks. Truong. 1993). A more general book about multirate systems and ﬁlter banks is (Vaidyanathan. Gilbert. Multirate Systems and Filter Banks.
Cohen. Chen. 1994. L. Mathematical Modelling and Numerical Analysis. V. rept. 44.. Burt. & Stens. 1991. 532–540. 34(1). Applied and Computational Harmonic Analysis. & Donoho. Tech. & Tabacco.BIBLIOGRAPHY 193 Bibliography Antoniou. Basis Pursuit. & Rokhlin. R. J. Chen. Blum.. 40–53. 1992. SIAM books. Cohen. Ronald R. Coifman. 31(4). Beylkin. Shaobing. R. 1993. EntropyBased Algorithms for Best Basis Selection. 1(1). & Henson. J. Bacry. A. Comm. Mallat. K. Emmanual. Wavelets on the interval and fast wavelet transforms. Comm. Lectures on a Theory of Computation and Complexity over the Reals. Pure Applied Math. 1995.. Coifman. E. McGrawHill. L. Butzer. R alisation d’une biblioth que C de fonctions ondelettes. Stanford University. 26. & Adelson. W. A.. Tech. C.. Canuto. I. 1983. 1994. Shaobing. 141–183. Commun. Daubechies. P. 38(2). A wavelet based space– time adaptive numerical method for partial diﬀerential equations. E.. C. & Vial. Multilevel Decompositions of Functional Spaces. Anita.. 793–834.. The Journal of Fourier Analysis and Applications. 1991a. rept. 18 Coloquio Brasileiro de Matematica – IMPA. David L. 713–718. IEEE Trans. XLIV... & Rokhlin. rept. 3(6). & Donoho. Fast Wavelet Transforms and Numerical Algorithms I. 1992. & Feauveau. Digital Filters. Philadelphia. I. P. Communications on Pure and Applied Mathematics. Chui. V. 54–81. An introduction to wavelets. Mladen Victor. 1993. Tech. Coifman. L. G. Second edition edn. Academic Press. 1992. V. Sampling Theory for not necessarily bandlimited functions: a historical overview. 141–183. DFT: an owner’s manual for the discrete Fourier transform. Daubechies.. Claudio. BourgesS venier. Stephane. 1991b. Briggs. The Laplacian Pyramid as a Compact Image Code. transformation en ondelettes.. David L. George. L.. Mikael. Stanford University. 1997. Atomic Decomposition by Basis Pursuit. 1995. P. H. Beylkin. Fast wavelet transforms and numerical algorithms. 1992. IEEE Transactions on Information Theory. G. . R. & Papanicolaou. IRISA – INRIA. & Wickerhauser. Biorthogonal Bases of Compactly Supported Wavelets. in Pure and Applied Math. 1992. Andreas. SIAM Reviews..
Systems. Math. 1996. Accuracy and Stability of Numerical Algorithms. 259–299.. Gomes. Signal Proc. J. Jonas. Ten Lectures on Wavelets. Graphical Objects. S. Greenspan. Boca Raton: CRC Press. Alain. Van.. Philadelphia. 2091–2110. 1994. J. & Sweldens.ca/nest/imager/contributions/bobl/wvlt/download/notes. Grossmann. An overview of wavelet based multiresolution analyses. Darsa. and Control. G. Hernandez. Painless nonorthogonal expansions. International Series in Nonlinear Mathematics. 1946. Wolfgang. Theory of Communication. Bhagwandas P. 1995. Loan. 37.Z. Lucia. Jawerth. Gomes. E. 909–996. 11. SIAM Books. 12. Lathi. Boston: Birkhauser. Ingrid. A First Course on Wavelets. Donald. Lim. Signals. rept. IEEE Transaction on ASSP. Gabor. I. 1974. Multifrequency Channel Decomposition of Images and Wavelet Models. http://www. J. Duhamel. Orthonormal Bases of Compactly Supported Wavelets. 1990.ps. 1986. Luiz. Wavelets on Manifolds I: Construction and Domain Decomposition. Jae S. 1990. New York. 1997. A.. SIAM Rev. Philadelphia. Kaiser. Abstraction Paradigms for Computer Graphics.cs. Gomes.. 1992. 1988. 1994. Costa. 1981. & Weiss. 1271–1283. Bruno. of the IEEE. & Velho. Wim.ubc. Y. Nicholas J. P. 1994. Pergamon Press. 1989a. The Visual Computer. D. Luiz.saveme. New York: PrenticeHall.. 1996. 1998.. Harper & Row.. G.194 BIBLIOGRAPHY Dahmen. Wavelets and their applications in computer graphics. Jonas. B. SIGGRAPH Course Notes. Computer Oriented Mathematical Physics.. Tech. Comm. Fournier. Daubechies. M. Mallat. Pure Applied Math. Philadelphia. . Image Processing for Computer Graphics. SpringerVerlag. Daubechies. Computational Frameworks for the Fast Fourier Transform. & Meyer. 269–282. methods and Applications. 227–239. Reinhold. & Velho.. 377–412. Luiz. 19(4). & Vetterli. Fast Fourier Transform: A tutorial review and a state of the art. A Friendly Guide to Wavelets. XLI(41). 1996.. & Schneider. 1996. I. Daubechies. Institut fur Geometrie und Praktische Mathematik. & Velho. Two Dimensional Signal and Image Processing. 36(3). 429–457. Phys. The Visual Computer. PA: SIAM Books. SIAM books. Higham.
1992. Mallat. Academic Press. 3. 15(3). 1992. David H.edu/teaching/courses/waveletcourse/. Wavelets for Computer Graphics: a Primer. C. e 141–149. & Freeman. 1993. 315. Peter. Tony D. L. Multivariate wavelets. Singularity detection and processing with wavelets.. Theory of Functions of a Complex Variable. rept.. Schroder. Cambridge Univ Press. 38(2). Courant Institute of Mathematical Sciences. 1996. Stollnitz. Schroder. 1990. Characterization of signals from multiscale edges. 84. & Salesin. A. Wavelets in Computer Graphics.. Oppenhein. 1995. Simoncelli. Chap. Part 1. Academic Press. 617–643. IEEE Transactions on Information Theory. (ed). Mallat. Meyer. .. ACM Computing Surveys. Alan V. Matching Pursuit with Timefrequency Dictionaries. pages 591–606. & Willsky. vol. Malvar. The Steerable Pyramid: A Flexible Architecture for MultiScale Derivative Computation. Wavelets in computer graphics. 13.. S. 14(7). Representations for Rigid Solids: Theory Methods. Wim. Math. Mallat.. SIGGRAPH Course Notes. & Vetterling. 69–88. Eero P. 75–85. Multiresolution approximation and wavelets.. S. William T. G. and Systems. Colloq. W. A. vol. 1992. Math.. Teukolsky. Wavelets for computer graphics: a primer. 1995b. & Salesin. Requicha.. 1989b. Construction de bases orthonorm´es d’ondelettes. 1983.multires. DeRose. Pages 444–447 of: International Conference on Image Processing. Press. 26(June). 76–84. Numerical Recipes : The Art of Scientiﬁc Computing. H. http://www.caltech. IEEE Transactions on Pattern Analysis and Machine Intelligence. S. William H. bib. Peter. 1995a. 1990. Stollnitz. Eric J. Alan S. David H. Amer. K. A. Wavelets: A o Tutorial in Theory and Applications. Mallat. S. Saul A. S.. Markushevich. & Zhong. Modulated QMF Filter Banks with Perfect Reconstruction. Y. 1995 (Oct. & Zhang. 1996. Eric J. A. 906–907. PrenticeHall inc. & Sweldens. part 2. A Wavelet Tour of Signal Processing. Pages 615–625 of: Proceedings of the IEEE. DeRose.. Tech. 1977.. I. IEEE Computer Graphics and Applications. William T. 710–732. S. 15(4). 12(December). Chelsea Publishing Co. G. 60/61.). 1980. J. IEEE Computer Graphics and Applications.BIBLIOGRAPHY 195 Mallat. & Hwang.. 1998. Tony D. Pages 325–356 of: Chui. Signal and Systems. St¨ckler. S. Trans. Electronics Letters. Soc. 437–464.
M. Wavelets for Computer Graphics: Theory and Applications. Orthogonal multiwavelets with vanishing moments.. Math. Tech. Scientiﬁc Computing and Computational Mathematics. 1996. Algorithms for the Generation of Daubechies Orthogonal Least Asymmetric Wavelets and the Computation of Their Holder Regularity. Advances in Shannon’s Sampling Theory. Computational Toolsmiths. Wavelets and Subband Coding. Englewood Cliﬀs. MA: WellesleyCambridge Press. Tony D. . & Salesin. 1993. Wickerhauser. Miller Freeman. 40(9). and MostSymmetric Wavelets. Wavelets and Filter Banks. Computational Algorithms for Daubechies LeastAsymmetric. IEEE Trans. New Jersey: Prentice Hall PTR. 1996.196 BIBLIOGRAPHY Stollnitz. Tech. Mladen Victor. Adapted Wavelet Analysis from Theory to Software. 2104–2107. 1995. University of South Carolina. C.. San Francisco. rept. Carl. Zayed. 1997a. Sweldens. rept. Vetterli. 1994. P. Martin. Taswell. 1995:6. Wellesley. Weaver. Taswell. & Kovacevic. CA: Morgann Kaufmann. www. Strang. Carl. Wellesley. Gilbert. J. G. David H. Multirate Systems and Filter Banks.com. MA: A. Ahmed. 1992. Strang.). Optical Engineering. Boca Raton: CRC Press. Symmetric. 33(7).. 1995. Englewood Cliﬀs. Vetterli.. Tech. Vaidyanathan. Amer. Wavelets and ﬁlter banks: theory and design. The lifting scheme: A construction of second generation wavelets. P. Soc. Stanford. 1989. CA. Proc. Taswell. 1994. The Systematized Collection of Wavelet Filters Computable by Spectral Factorization of the Daubechies Polynomial. & Shen. Eric J. DeRose. Jelena. & Nguyen. Pages 1834–1838. Carl. & Herley. Strang. 1996. & Strela. Truong.). Department of Mathematics. New York: John Wiley & Sons. The zeros of the Daubechies polynomials. Acoust. Theory of Discrete and Continuous Fourier Transform.). Stanford University. Vasily. Peters. 1993. Speech Signal Process. K. 1995 (Aug.. New Jersey: Prentice Hall PTR.toolsmiths. Gilbert. rept. J. W. 1997b (Dec.
This action might not be possible to undo. Are you sure you want to continue?
Use one of your book credits to continue reading from where you left off, or restart the preview.