www.GetPedia.

com
* The Ebook starts from the next page : Enjoy !

Signal Analysis: Wavelets, Filter Banks, Time-Frequency Transforms and Applications. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 ElectronicISBN 0-470-84183-4

Signal Analysis

Signal Analysis
Wavelets, Filter Banks, Time-Frequency Transforms Applications and

Alfred Mertins
University of Wollongong, Australia

JOHN WILEY & SONS Chichester . New York . Weinheim . Brisbane . Singapore . Toronto

@B.G. Teubner Stuttgart 1996, Mertins, Signaltheorie Translation arranged with the approval of the publisher Teubner Stuttgart, from the original B.G. German edition into English. English (revised edition) Copyright 1999 by John Wiley& Sons Ltd, @ Baffins Lane, Chichester, West Sussex P019 IUD, England

National 01243 719717 International (+44) 1243 119117
e-mail (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on http://www.wiley.co.uk or http://www.wiley.com

this All Rights Reserved. No part of publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency, 90 Tottenham Court Road, London, W1P 9HE, UK, without the permission in writing of the publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the publication.
& Neither the authors nor John Wiley Sons Ltd accept any responsibility or liability for loss or damage occasioned to any person or property through using the material, instructions, methods or ideas contained herein, or acting or refraining from acting as a result of such use. The authors and Publisher expressly disclaim all implied warranties, including merchantability of fitness for any particular purpose.

Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where John Wiley Sons is aware of a claim, the product names appear in initial capital & or capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration.

Other wiley Editorial Ofices
John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, USA Wiley-VCH Verlag GmbH, Pappelallee 3, D-69469 Weinheim, Germany Jacaranda Wiley Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons (Canada) Ltd, 22 Worcester Road, Rexdale, OntarioM9W 1L1, Canada

British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library. ISBN 0-471-98626-7 Produced from PostScript files supplied by the author. Printed and bound in Great Britain by Bookcraft (Bath) Ltd. This book is printed on acid-free paper responsibly manufactured from sustainable forestry, in which at least two trees are planted for each one used in paper production.

Contents
Spaces Signal 1 Signals and 1 1.1 Signal Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Energy and Power Signals . . . . . . . . . . . . . . . . . 1 2 1.1.2 Normed Spaces . . . . . . . . . . . . . . . . . . . . . . . Spaces . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.3 Metric Product Spaces . . . . . . . . . . . . . . . . . . . 4 1.1.4 Inner Density and Correlation . . . . . . . . . . . . . . . . . . 8 1.2 Energy 1.2.1 Continuous-Time Signals . . . . . . . . . . . . . . . . . 8 1.2.2 Discrete-Time Signals . . . . . . . . . . . . . . . . . . . 9 10 1.3 Random Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Properties of Random Variables . . . . . . . . . . . . . . 11 Processes ..................... 13 1.3.2 Random 1.3.3 Transmission of Stochastic Processes through Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . .

.

20 22 22 26 29 34 34 35 35 36 43

2 Integral Signal Representations 2.1 Integral Transforms . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 The Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . 2.3 TheHartley Transform . . . . . . . . . . . . . . . . . . . . . . . 2.4 The Hilbert Transform . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Some Properties of the Hilbert Transform . . . . . . . 2.5 Representation of Bandpass Signals . . . . . . . . . . . . . . . 2.5.1 Analytic Signal and Complex Envelope . . . . . . . . Bandpass Processes . . . . . . . . . . . . . 2.5.2 Stationary
V

. . . .

vi

Contents

3 Discrete Signal Representations 3.1 Introduction ............................. 3.2 Orthogonal Series Expansions . . . . . . . . . . . . . . . . . . 3.2.1 Calculation of Coefficients . . . . . . . . . . . . . . . . 3.2.2 Orthogonal Projection . . . . . . . . . . . . . . . . . . 3.2.3 The Gram-Schmidt Orthonormalization Procedure . . 3.2.4 Parseval’s Relation . . . . . . . . . . . . . . . . . . . . . 3.2.5 Complete Orthonormal Sets ............... 3.2.6 Examples of Complete Orthonormal Sets ....... 3.3 General Series Expansions . . . . . . . . . . . . . . . . . . . . . 3.3.1 Calculating the Representation . . . . . . . . . . . . . 3.3.2 Orthogonal Projection . . . . . . . . . . . . . . . . . . 3.3.3 Orthogonal Projection of n-Tuples . . . . . . . . . . . 3.4 Mathematical Tools . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 The &R Decomposition . . . . . . . . . . . . . . . . . 3.4.2 The Moore-Penrose Pseudoinverse . . . . . . . . . . . 3.4.3 The Nullspace . . . . . . . . . . . . . . . . . . . . . . . 3.4.4 The Householder Transform . . . . . . . . . . . . . . . 3.4.5 Givens Rotations . . . . . . . . . . . . . . . . . . . . . .
4 Examples of Discrete Transforms 4.1 The z-Transform . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The Discrete-Time Fourier Transform ............. 4.3 The Discrete Fourier Transform (DFT) . . . . . . . . . . . . . 4.4 The FastFourierTransform .................... 4.4.1 Radix-2 Decimation-in-Time FFT . . . . . . . . . . . 4.4.2 Radix-2 Decimation-in-Frequency FFT . . . . . . . . . 4.4.3 Radix-4 FFT . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4 Split-Radix FFT . . . . . . . . . . . . . . . . . . . . . . 4.4.5 Further FFT Algorithms . . . . . . . . . . . . . . . . . 4.5 Discrete Cosine Transforms . . . . . . . . . . . . . . . . . . . . 4.6 Discrete Sine Transforms . . . . . . . . . . . . . . . . . . . . . . 4.7 The DiscreteHartleyTransform . . . . . . . . . . . . . . . . . 4.8 TheHadamardand Walsh-Hadamard Transforms . . . . . .

. . . . . . . . .

47 47 49 49 50 51 51 52 53 56 57 60
62

64

. 64 . 66
68

. 69
73
75 75

. 80
82 85 . 85 . 88 90 91 . 92 93 96 . 97 . 100

.

5 Transforms and Filters for Stochastic Processes 101 5.1The Continuous-Time Karhunen-Loitve Transform . . . . . . . 101 5.2 The Discrete Karhunen-Loitve Transform . . . . . . . . . . . . 103 5.3 The KLT of Real-Valued AR(1) Processes . . . . . . . . . . . . 109 5.4 Whitening Transforms . . . . . . . . . . . . . . . . . . . . . . . 111 5.5 Linear Estimation ......................... 113

. .6 LinearOptimalFilters .. .. . 162 .. . . . . . . ..2. .. . . .. . . . . . . . .6. . ..2. . . . . . . . . 6. . . . .1 Basic Multirate Operations .. 5.6. . . . . 155 . 147 148 148 . . . .. . . . . .6.. 174 . . .. . . . . . ..7 Estimation of Autocorrelation Sequences and Power Spectral Densities . . . . . . . .3 General Perfect Reconstruction Two-Channel Filter Banks ..7.7 Linear-PhaseFilterBanks in LatticeStructure . .5.5. . .3 Paraunitary Filter Banks .. . . . . . .2. . .4. 164 .. 151 . .. .. . .. . . . . . .1. .. . . 6 Filter Banks 6... . .. . .. .... . . . . 6. 168 168 170 . . . . . 6. . . 5. . . . . ..1 Wiener Filters . . . . . .. . 6.4 Matrix Representations . . . . . . . . 5. ... .3 MinimumMean SquareErrorEstimation . . . . . . . . . . .. .. .4 Design of Critically Subsampled M-Channel FIR Filter Banks . . . . . . .. 186 . . . . .6. 164 .3 Tree-Structured Filter Banks . . .. ..... . . 6. . . . . .2 Two-ChannelFilterBanks .. . 6. . . . . .. . .. 5.. .. . . . .7. . 6. .. 175 179 .. . . . . . ... . . . .1 Least-Squares Estimation .. 149 150 . . . . . . . . . . . .... . . . .. . . . . 6. .. .. . . . ... .. .4 UniformM-Channel FilterBanks . 5. . 6. . . . . . . . ... ..5.. . . . . .2 The Best Linear Unbiased Estimator (BLUE) . . . . 6. . 166 .. .. . . . . . . . . . 6. 6. . . 5. . . 159 160 . .2 One-Step Linear Prediction . .. 6. .2 The Polyphase Representation .4. . . . . . ...1 Critically Subsampled Case ..Contents vii 113 114 116 124 124 127 130 5. . . . .6 Paraunitary Filter Banks in Lattice Structure . . .2 Non-Parametric Estimation of Power Spectral Densities 5. .1Estimation of Autocorrelation Sequences .7.2 Polyphase Decomposition . . . .. . . .2.. . . ..2 Paraunitary Case . .. .. 6.3OversampledCosine-ModulatedFilterBanks . . .7LappedOrthogonalTransforms 133 133 134 141 143 144 .. . 158 . .2. .. .3 Filter Design on the Basis of Finite Data Ensembles .. . . .2. . 6. . .. . . .. .. . . .1.. . . . 6. . . . .. . . . . 6. . . . 5. ... . . . . . . . .. . . 183 184 . . .4.2.. .5 DFT Filter Banks . 6. . .5 Paraunitary Two-Channel Filter Banks .. . .1 P R Condition . . . .. . . 6. . . . .. . 6.6.. . . ..2 Quadrature MirrorFilters . 6. . . . .6.. . . . . ... . . . . 144 . . . . . . .. 6. . .. . . .. .. . 5. .1Decimation andInterpolation ... . . . ... . . . .4. . . 5.6 Cosine-ModulatedFilterBanks . . .. . .. ... . .6. .. .. . . . . . .1 Input-Output Relations . . . . . . .. . . . . .. . . .4 Pseudo-QMF Banks . ... . . . . .2. . .. . .. .8 Lifting Structures . . . . . .. . 6. . . .. . 6.3 Parametric Methods in SpectralEstimation .

v1 11

...

Contents

6.8 SubbandCoding of Images . . . . . . . . . . . . . . . . . . . . 188 6.9 Processing of Finite-Length Signals . . . . . . . . . . . . . . . . 189 6.10 Transmultiplexers . . . . . . . . . . . . . . . . . . . . . . . . . . 195 7 Short-Time Fourier Analysis 7.1 Continuous-Time Signals . . . . . . . . . . . . . . . . . . . . . . 7.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.2 Time-Frequency Resolution . . . . . . . . . . . . . . 7.1.3 TheUncertainty Principle . . . . . . . . . . . . . . . 7.1.4 The Spectrogram . . . . . . . . . . . . . . . . . . . . . . 7.1.5 Reconstruction . . . . . . . . . . . . . . . . . . . . . . . 7.1.6 Reconstruction via Series Expansion . . . . . . . . . 7.2 Discrete-Time Signals . . . . . . . . . . . . . . . . . . . . . . . 7.3 Spectral Subtraction based on the STFT . . . . . . . . . . .
8 Wavelet

. . . .

196 196 196 . 198 . 200 201 202 . 204 205 . 207 210

sform

8.1The Continuous-Time Wavelet Transform . . . . . . . . . . . 8.2 Wavelets for Time-ScaleAnalysis . . . . . . . . . . . . . . . . 8.3IntegralandSemi-DiscreteReconstruction ........... 8.3.1 Integral Reconstruction ................. 8.3.2 Semi-Discrete Dyadic Wavelets . . . . . . . . . . . . . 8.4 Wavelet Series ........................... 8.4.1 Dyadic Sampling . . . . . . . . . . . . . . . . . . . . . . 8.4.2 Better FrequencyResolution - Decomposition of Octaves . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 The Discrete Wavelet Transform (DWT) . . . . . . . . . . . . 8.5.1 Multiresolution Analysis . . . . . . . . . . . . . . . . . 8.5.2Wavelet Analysis by MultirateFiltering . . . . . . . . 8.5.3 Wavelet Synthesis by Multirate Filtering . . . . . . . . 8.5.4 The RelationshipbetweenFiltersandWavelets .... .................... 8.6 WaveletsfromFilterBanks 8.6.1 General Procedure . . . . . . . . . . . . . . . . . . . . . 8.6.2 Requirements to be Metby the Coefficients . . . . . . 8.6.3 Partition of Unity . . . . . . . . . . . . . . . . . . . . . 8.6.4 The Norm of Constructed Scaling Functions and Wavelets . . . . . . . . . . . . . . . . . . . . . . . . 8.6.5 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.6 Regularity . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.7 WaveletswithFinite Support . . . . . . . . . . . . . . 8.7 Wavelet Families . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.1 Design of BiorthogonalLinear-PhaseWavelets ....

. 219

. 210 . 214 . 217 . 217
223 223 225

. 227

. 227
. 232 . 233

. 234 .
237 237 241 241

242 243 244 . 245 247 . 247

Contents

ix

8.7.2 The Orthonormal Daubechies Wavelets ........ 8.7.3 Coiflets . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 The Wavelet Transform of Discrete-Time Signals . . . . . . . 8.8.1 The A TrousAlgorithm . . . . . . . . . . . . . . . . . 8.8.2 The Relationship between the Mallat and A Trous Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.3 The Discrete-Time MorletWavelet . . . . . . . . . . . 8.9 DWT-Based Image Compression . . . . . . . . . . . . . . . . ..................... 8.10 Wavelet-Based Denoising 9 Non-Linear Time-Frequency Distributions 9.1The Ambiguity Function . . . . . . . . . . . . . . . . . . . . . . 9.2 The WignerDistribution . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Definition and Properties . . . . . . . . . . . . . . . . 9.2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.3 Cross-Terms and Cross WignerDistributions . . . . . 9.2.4 Linear Operations ..................... 9.3General Time-Frequency Distributions . . . . . . . . . . . . . ..... 9.3.1 Shift-Invariant Time-Frequency Distributions 9.3.2 Examples of Shift-Invariant Time-Frequency Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.3 Affine-Invariant Time-Frequency Distributions .... 9.3.4 Discrete-Time Calculation of Time-Frequency Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 The Wigner-Ville Spectrum . . . . . . . . . . . . . . . . . . . . Bibliography Index

. 252
253

. 255 . 256
259

. 260 . 261
263 265 265 269 . 269 274 . 275 279 . 280 . 281 283

. 289
290 292 299
311

Preface
A central goal in signal analysis is to extractinformation from signals that are related to real-world phenomena. Examplesare theanalysis of speech, images, and signals in medical or geophysical applications. One reason for analyzing such signals is to achieve better understanding of the underlying physical phenomena.Another is to find compactrepresentations of signals which allow compact storage or efficient transmission of signals through real-world environments. The methods of analyzing signals are wide spread and range from classical Fourier analysis to various types of linear time-frequency transforms and model-basedand non-linear approaches. This book concentrates on transforms, but also gives a brief introduction to linear estimation theory and related signal analysis methods. The text is self-contained for readers with some background in system theory and digital signal processing, as typically gained in undergraduate courses in electrical and computer engineering. The first five chapters of this book cover the classical concepts of signal representation, including integral and discrete transforms. Chapter 1 contains anintroduction to signals and signal spaces. It explains the basic tools forclassifying signals and describing theirproperties.Chapter 2 gives an introduction to integral signal representation.Examplesarethe Fourier, Hartley and Hilbert transforms. Chapter 3 discusses the concepts and tools for discrete signal representation. Examples of discrete transforms are given in Chapter 4. Some of the latter are studied comprehensively,while others are only briefly introduced, to a level required in the later chapters. Chapter 5 is dedicated to the processing of stochastic processes using discrete transforms and model-based approaches. It explains the Karhunen-Lobve transform and the whitening transform, gives an introduction to linear estimation theory and optimal filtering, and discusses methods of estimating autocorrelation sequences and power spectra. The final four chapters of this book are dedicated to transforms that provide time-frequency signal representations. In Chapter 6, multirate filter banks are considered. They form the discrete-time variant of time-frequency transforms. The chapter gives an introduction to the field and provides an overview of filter design methods. The classical method of time-frequency analysis is the short-time Fourier transform, which is discussed in Chapter 7. This transform was introduced by Gabor in 1946 and is used in many applications, especially in the form of spectrograms. The most prominent example of linear transforms with time-frequency localization is the wavelet transform. This transform attracts researchers from almost any field of science, because
xi

xii

Contents

it has many useful features: a time-frequency resolution that is matched to many real-world phenomena, a multiscale representation, and a very efficient implementationbasedonmultirate filter banks.Chapter 8 discusses the continuous wavelet transform, thediscrete wavelet transform, and thewavelet transform of discrete-time signals. Finally, Chapter 9is dedicated to quadratic time-frequency analysis tools like the Wigner distribution, the distributions of Cohen’s class, and the Wigner-Ville spectrum. Thehistory of this book is relatively long. Itstarted in 1992 when I produced the first lecturenotes for courses on signal theoryand linear time-frequency analysis at the Hamburg University of Technology, Germany. Parts of the material were included in a thesis (“Habilitationsschrift”) that I submitted in 1994. In 1996, the text was published as a textbook on Signal Theory in German. This book appeared a series on InformationTechnology, in edited by Prof. Norbert J. Fliege and published by B.G. Teubner, Stuttgart, Germany. It was Professor Fliege who encouraged me to write the book, and I would like to thank him for that and for his support throughout many years. The present book is mainly a translation of the original German. However, I have rearranged some parts, expanded some of the chapters, and shortened others in order to obtain a more homogeneous andself-contained text. During the various stages, from the first lecture notes,over the German manuscript to the present book, many people helped by proofreading and commenting on me the text. Marcus Benthin, Georg Dickmann, Frank Filbir, Sabine Hohmann, Martin Schonle, Frank Seide, Ursula Seifert, and Jens Wohlers read portions of the German manuscript. Their feedbacksignificantly enhanced the quality of the manuscript. My sister,IngeMertins-Obbelode, translated the text from German into English and also proofread the new material that was not included in the German book. Tanja Karp and Jorg Kliewer went through the chapters on filter banks and wavelets, respectively, in the English manuscript and made many helpful suggestions. Ian Burnett went through a complete draft of the present text and made many suggestions that helped to improve the presentation. I wouldlike to thank them all. Without their effort and enthusiasm this project would not have been realizable. Alfred Mertins Wollongong, December 1998

Signal Analysis: Wavelets, Filter Banks, Time-Frequency Transforms and Applications. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 ElectronicISBN 0-470-84183-4

Chapter 1

Signals and Signal Spaces
The goal of this chapter is to give a brief overview of methods for characterizing signals and for describing their properties. Wewill start with a discussion of signal spaces such as Hilbert spaces, normed and metric spaces. Then, the energy density and correlation function of deterministic signals will be discussed. The remainder of this chapter is dedicated to random signals, which are encountered in almost all areas of signal processing. Here, basic concepts such as stationarity, autocorrelation, and power spectral densitywill be discussed.

1.l
1.1.1

Signal Spaces
Energy and Power Signals

Let us consider a deterministic continuous-time signalz(t), which may be real or complex-valued. If the energy of the signal defined by

is finite, we call it an energy signal. If the energy is infinite, but the mean power

1

2

Chapter 1 . Signals and Signal Spaces

is finite, we call z ( t ) a power signal. Most signals encountered in technical applications belong to these two classes. A second important classification of signals is their assignmentto the signal spaces L,(a, b ) , where a and b are the interval limits within which the signal is considered. By L,(a, b) with 1 5 p < m we understand that class of signals z for which the integral

I”

lX(t)lP dt

to be evaluated in the Lebesgue sense is finite. If the interval limits a and b are expanded to infinity, we also write L p ( m )or LP@). According to this classification, energy signals defined on the real axis are elements of the space L2 (R).

1.1.2

Normed Spaces

When considering normed signal spaces, understand signals as vectorsthat we are elements of a linear vector spaceX . The norm of a vector X can somehow be understood as the length of X. The notation of the norm is 1 1 ~ 1 1 . Norms must satisfy the following three axioms, where a is an arbitrary real or complex-valued scalar, and 0 is the null vector:

Norms for Continuous-Time Signals. The most common norms for continuous-time signals are the L, norms: (1.6) For p

+ m, the norm (1.6) becomes

llxllL, = ess sup Iz(t)l.
astsb

For p = 2 we obtain the well-known Euclidean norm:

Thus, the signal energy according to (1.1) can also be expressed in the form
00

X

E

L2(IR).

(1.8)

For p = 2 we obtain Thus. z ) .n E Z can be expressed as: n=-cc 1.4): Thus. (1.14) is satisfied. For two vectors z = a . the energy of a discrete-time signal z ( n ) .1.X). the metric induced by the norm is the norm of the difference vector: Proof (norm + metric). which means that also (1.5) leads to 1 1 2 .1.3).c ) I d(a. They are normed as follows: (1. can be understood as the distance between y) X and y. Signal Spaces 3 Norms for Discrete-Time Signals.2 / 1 1 the validity of (1. time equivalent to the spaces L p ( a .12) (1. y) = 0 if and only if X = y. 0 .13) is also satisfied. g) = 12 .9) Ix(n)I.3 Metric Spaces A function that assigns a real number to two elements X and y of a non-empty set X is called a metric on X if it satisfies the following axioms: (i) d(x.Y) = d(Y.1. d(x.b and y = b . + The metric d(x. z ) I d(x.c ) . With a = -1. The spaces l p ( n ln2) are the discrete.Lnl (1. (1.zlI. y) d(y.c the following holds according to (1. 1 and (1. (ii) (iii) y) 2 0. d(x.b) + d(b.2 / 1 1 = 19 . becomes llzlleoo = sup.12) imme1 diately follows from (1.14) d(X. For d ( z . d(a. Here.13) (1. A normed space is also a metric space.9) For p + CO.b ) .

b ) .. Thenotation is ( X . . The normed spaces L.16) Accordingly.yi E (0.y 11.20) Here. 4 (2. for these spaces inner products can be stated.4 Inner Product Spaces The signal spaces most frequently considered are the spaces L 2 ( a .21) .B are scalars with a. if 1 1 2 .Y) = C K X k + Y k ) mod 21. An inner product assigns a complex number to two signals z ( t ) and y ( t ) . Signals and Signal Spaces An example is the Eucladean metric induced by the Euclidean norm: 1/2 I 4 t ) .l2dt] .z) = Q ( X . A space is complete if any Cauchy sequence of the elements of the space converges within the space.X. and l . (aa:+Py.4 Chapter 1 . k=l which states the number of positions where twobinarycode words X = [Q. (1.~) 2 0. n2). which means that they are normed linear spaces which are complete with regard to their metric d ( z . y). .19) (1.. 1. . l} differ (the space of the code words is not a linear vector space). . b ) and &(nl. . E Y L z ( a .yn] with xi. 2.z l. That is.] and y = [ y l . y) = 1 1 2 .. respectively. . 2 2 .18) X = 0. Z ) + P ( Y . An inner productmust satisfy the following axioms: (i) (4 (iii) k.. for n + 00 lies in the space. we also find metrics which are not associated with a norm. the following distancebetween stated: discrete-time signals canbe Nevertheless. E (1. are so-called Banachspaces. ~ ) if and only if =0 (1.Y. An example is the Hamming distance n d(X.1. a and . + 0 as n and m + m. ( 2 . while the limit of X. or z(n) and y ( n ) .Y>=( Y A * (1.@E Examples of inner products are ( . ..and 0 is the null vector. Note.y ~.

a scalar prefactor of the left argument may directly precede the inner product: (az. As can easily be verified. = a (2.(1.23). A superscript H .20). The general definition of inner products of discrete-time signals is where G is a real-valued.18) shows that changing the order leads to a conjugation of the result. Signal Spaces 5 and The inner product (1. as in (1.19) indicates. the order in which the vectors occur must be observed: (1. If we want a prefactor y) y). means transposition and complex conjug& tion. This means that GH = GT = G. The mathematical rules for inner products basically correspond to those for ordinary productsof scalars. lThe superscript T denotestransposition. positive definite weighting matrix. Hermitian. and all eigenvalues Xi of G must be larger than zero.26) meet conditions (1.1.25) and (1. However. A vector a H is also referred to as the Herrnitian of a.1. As equation (1.22) may also be written as where the vectors are understood as column vectors:' More general definitions of inner products include weighting functions or weighting matrices.18) . a 5 t 5 b. the inner products (1.Theelements of a and g mayberealor complex-valued. .If a vector is to be conjugated but not to be transposed. An inner product of two continuous-time signals z ( t ) and y ( t ) including weighting can be defined as where g ( t ) is a real weighting function with g ( t ) > 0. we write a * such that a H = [=*lT.

we conclude 112 + Y1I2 I 1 1 4 1 2 + 2 l l 4 l 0 IlYll + 11YIl2 = ( 1 1 4 + llYll)2* This shows that also (1. + will be considered.4) holds.18). which states Ib . The norm induced by the inner product is We will prove this in the following along with the Schwarz inequality.18) and (1. By defining an inner product we obtain a norm and also a metric. an inner product (2.5) is also proved. From (1.~) is always real: ( 2 . since (1. Signals and Signal Spaces of the right argument to precede the inner product.29) is given only if X and y are linearly dependent. 2 ) 1 /= 2 la1 l l z l l . The validity of the equality sign in the Schwarz inequality (1. Y >I I l 1 4 IlYll.20) it follows immediately that (1. ~ ) = !I&{(%. z)}. Proof of the Schwarz inequality. if one vector is a multiple of the other. Proof (inner product + n o m ) . (1. a z y= [ Thus.3) is satisfied.29) Equality in (1. it must be conjugated. we conclude from (1. Now the expression 112 l1 a2 (2.z) = la1 ]1/2 ( 2 . We have Assuming the Schwarz inequality is correct.19) llazll = ( a z .18) and (1. (1. For the norm of a z .19) lead to Due to (1.29) for linearly dependent vectors can easily be proved .6 Chapter 1 . that is.

On the basis of (1.21) and (1.into (1.28) and (1.4 (z a y .28) shows that the inner products given in (1.(1. let us remark that a linear space with an inner product which is complete with respect to the induced metric is called a Hilbert space.Y)+a(Y.10).Y).18) . X + +ay) (1. (1. Finally. . a E C.z+ay)+(ay. This also holds for the special a (assumption: y # 0) and we get The second and the fourth termcancel.~)+a*(~.1.1.7) and (1.32) with (1.20) we have 0 I = (G.22) lead to the norms (1. Signal Spaces 7 by substituting z = a y or y = a z .28).30) = = (z. for X = a y we have In order to prove the Schwarz inequality for linearly independent vectors. some vector z = z + a y will be considered.29) confirms the Schwarz inequality.29) and rearranging the expression obtained.~)+aa*(Y.32) Comparing (1. 0 Equation (1. For example.z+ay) (~. observing (1.

“. ~ The quantity Iz(t)I2in (1. see Section 2. We use the following notation = IX(w)I2.2 %{?fx(r)}.G) - ( G .34) where X(W)is the Fourier transform of ~ ( t ) .2)) = 2 1 1 2 1 1 2 . (1. we freely use the properties of the Fourier transform. Signals and Signal Spaces 1. This can be seen from + d(2.”. IX(w)I2 in (1. ) cc (1.35) The energy density spectrum S. .. For more detail on the Fourier transform and Parseval’s theorem.34) can be viewed as the distribution of energy with respect to frequency W.(w) t r.(W) l c c r f z ( ~ ) = e-jwT d r . ( .) 2) 2 = 2 1 1 2 1 1 2 . (1. = S__lz(t)l2 dt.4 (2. = - (1.2 % { ( G . 21n this section.33) E.”.(r) = J z * ( t )z(t + r ) dt = X * ( . ) The autocorrelationfunction is a measure indicating the similarity between an energy signal z(t) and its time-shifted variant z r ( t )= z ( t r ) . Therefore IX(w)I2 is called the energy density spectrum of z ( t ) .1): E. According to Parseval’s theorem.2.r )* X. we may also write (1.”. + ( G . EnergyDensityandCorrelation Continuous-Time Signals 00 Let us reconsider (1.38) With increasing correlation the distance decreases.(w) can also be regarded as the Fourier transform of the so-called autocorrelation function cc r.37) The correspondence is denoted as S.8 Chapter 1 .33) represents the distribution of signal energy withrespect to time t .”.(r). (1.2 121 .36) -cc We have S. . accordingly.2A2 = = 112 - 2 4 (2.

40) (1. Energy Density and Correlation 9 Similarly. (1.(W) = I-. As in the continuous-time case. Fcc r.0 0 and the corresponding cross energy density spectrum S.2.42) According to Parseval's relation for the discrete-time Fourier transform.1. where between the two signals z ( t ) and y T ( t ) = y(t 7).43) in The term IX(ejW)12 (1.) + 1.(ej") is the discrete-time Fourier transform of the autocorrelation sequence ?-:.2.(ejw)= IX(ejW)12. The signals z ( n ) may be real or complex-valued.43) is called the energy density spectrum of the discrete-time signal. We use the notation S.". + (1.(r) C j W Td r .Fy(. the cross correlation function r.2 Discrete-Time Signals All previous considerations are applicable to discrete-time signals z ( n )as well.(m) = c 00 z*(n)z(n m ) .41) may be viewed as a measure of the similarity are introduced.44) The energy density spectrum S.(r) = [ cc y(t + r ) z*(t)d t (1. . .". (1.45) 3See Section 4.". we may alternatively compute E. from X ( e j w ) : 3 (1.E.2 for more detail on the discrete-time Fourier transform.39) J . we start the discussion with the energy of the signal: 00 (1.E.

(1.(m)= n=-cc c cc y ( n + m ) z*(n).1for an illustration.(m) = G I T S F z ( e j w )ejwm dw. In speech. First of all. If one assigns a function iz(t)to each feature mi. because only random signals carry information. 1 " (1. The definition of the cross correlation sequence is r. they appear as disturbances in the transmission of signals.48) m=-m that is (1. Signals and Signal Spaces m=-cc c M 5 r. audio.49) 1.one distinguishes between randomvariables and random processes.E. and image coding.46) Note that the energy density may also be viewed as the product X ( z ) X ( z ) . . the patterns that are to distinguished are modeled as random be processes. where X ( z ) is the z-transform of z ( n ) . then the totality of all possible functions is called a stochastic process. evaluated on the unit circle ( z = e j w ) . the signals to be compressed are modeled as such. The features (or events) occur randomly.10 We have Chapter 1 .3 Random Signals Random signals are encountered in all areas of signal processing. For example.47) For the corresponding cross energy density spectrum the following holds: cc (1. In pattern recognition. Even the transmitted and consequently also the received signals in telecommunications are of random nature. The features occur randomly whereas the assignment mi + i z ( t )is deterministic. A function i z ( t )is called the realization of the stochasticprocess z ( t ) .E. Note that the featuresthemselves may also be non-numeric.See Figure 1. A random variable is obtained by assigning a real orcomplex number to each feature mi from a feature set M .

(a) = 1.(a) = P ( x a ) .).51) . F.(.(a2) for a1 5 a2.1. (1. the axioms of probability hold.(al)5 F. a+w lim F. which state that a+--00 lim F. Random Signals 11 t" 3 \ 1 (b) Figure 1.(a) and also by its probability density function (pdf) p. 1. Random variables (a) and random processes (b).1 Properties of RandomVariables The properties of a real random variable X are thoroughly characterized by its cumulative distribution function F. The distribution states the probability P with which the value of the random variable X is smaller than or equal to a given value a: F.3.3. (1.1.(a) = 0.50) Here.

22(tl. The properties of a random variable are often described moments m?) = E by its (1. (1.)}= Icc .lzl (52 I&) is a conditional probability density (density 2 2 provided x1 has taken on the value 51).(t1) ([l. can be calculated from the density as E {dxt. m ([l. &) of (1. (1. (1. two random variables 21 and 22 is given by PZ1.52) Since the distribution is a non-decreasing function. we obtain the pdf by differentiation: (1...58) For g(x) = x we obtain the mean value (first moment): mz =E{x}= lcct C Q Pz(5) dt. E {-} denotes the expected value (statistical average). where g ( x ) is an arbitrary function of the random variable x.59) For g(%) = we obtainthe average power(second moment): . Herein. An expected value E {g(z)}.12 Chapter 1 ..Q C g(<) P X ( 5 ) d t .t22) =pz.55) density of its The pdf of a complex random variable is defined as the joint real and imaginary part: Moments. t2) = p..57) {Ixl"} . (t1) p. (1.54) PZZ1X1(t221t1). The joint probability density p. Signals and Signal Spaces Given the distribution. (&). we have Joint Probability Density.. of where pz.. If the variables 2 1 and 22 are statistically independent of one another.54) reduces to P m .

< t. = S.61) (1.. Theautocorrelation function of a general random process is defined as a second-order moment: (1..x t z .Then a second set of random variables is taken from theprocess x ( t ) . According tothe momenttheorem of the Fourier transform (see Section 2.applying a timeshift T: xtl+T. . . +with x t k + r = x(tk T). otherwise we call the process non-stationary. it is the Fourier transform of the pdf.mx12 as cc d =E { Ix . xi.m. . 2 (1.. Random Signals 13 The variance (second central moment) is calculated with g(x) = Ix .I2 P. apart from the sign of the argument.63) which means that.. Z t. If the joint densities of both ~ sets are equal for all time shifts T and all n.. ifwe have + then we speak of a strictly stationary process. The properties of these random variables are 2 .2). ..1.an).3. ~ t . the moments of the random variable can also be computed from the characteristic function as (1.. .. .62) The following holds: U..2 Random Processes The startingpoint for the following considerations is a stochastic process x ( t ) .m. Autocorrelation and Autocovariance Functions of Non-Stationary Processes. .mXl2}= 2 2 -cc 15 .64) 1. .z t n (a1. The characteristic function of a random variable (1. characterized by their joint pdf p z t 1 . that is. Characteristic Function.3.. .(<) d5.a ~ .66) . n E Z. from which the randomvariables xtl . with xtk = x(tk) are takenat times tl < t z < .xtz+T..

Autocorrelation and Autocovariance Functions of Wide-Sense Stationary Processes. There areprocesses whose mean value is constant and whose autocorrelation function is a function of tl . even if they are nonstationary according to the above definition.e. but if the properties repeat periodically. In the following we assume wide-sense stationarity.67) where mtk denotes the expected value at time tk.t2. Chapter 1 . Such processes are referred to as “wide-sense stationary”. since for the expected Euclidean distance we have The autocovariance function of a random process is defined as (1.69) can be written as (1. {e} (1.) + T) and 2 2 = z * ( t ) .70) For 21 = z(t Tzz(.. Forwide-sense stationary processes the autocorrelation function (acf) depends only on the time shift between the respective times. Because of the stationarity we mustassume that the process realizations are not absolutely integrable. then we speak of a cyclo-stationary process. we shallcontinue by looking at complex-valued processes. Cyclo-Stationary Process. it is given by T.68) Wide-Sense Stationary Processes. so that the first and second moments are independent of the respective time. Since in the field of telecommunications one also encounters complex-valued processes when describing real bandpass processes in the complex baseband. If a process is non-stationary according to the definition stated above. mtk =E {Z(tk)} ‘ (1. Signals and Signal Spaces 22 Basically. the autocorrelation function indicates how similar the process is at times tl and t2.14 where z1 = z(t1) and = x*(tz).the expected value E =E {X1 z2} = .(T) = E { z * ( t()t z + T)} . and that their Fourier transforms do not exist. i.

76) with and rect(t) = 0. ) T is identical to the power spectral density given in (1. otherwise t X T ( W ) t z ( t ) rect(-).. for It 1 0. which states that the physically meaningful power spectral density given by (1..73) + PowerSpectralDensity. Taking (1.4 1 [x(t ) .(-r) = r i z (7).3.74) $ (1. The power spectral density.77) . describes the distribution of power with respect to frequency. we obtain = r Z Z ( 0= ) LJ 27r -CQ SZZ(w) dw.5 1. Random Signals 15 The maximum of the autocorrelation function is located at r = 0. where its value equals the mean square value: Furthermore we have r.]) ) . or power density spectrum. (1. .(r) = E { [ x * ( t.74).1. ( r )e-jwT d r (1.m.75) This definition is based on the Wiener-Khintchine theorem. S.(w) = ~ m r . It is defined as the Fourier transform of the autocorrelation function: C Q S. (1. we get the autocovariance function c.. When subtracting the mean prior computing theautocorrelationfunction.75) for T = 0.

80) (1. denoted as Szy( W ) .79) Discrete-Time Signals.83) m=-cc (1. For the autocorrelation sequence we have r x x ( m )= E {x*(n) ( n x The autocovariance sequence is defined as +m)}.78) The Fourier transform of rXy(7) is the cross power spectral density. we have the correspondence (1. (1. (1. The cross correlation between two wide-sense stationary randomprocesses z ( t )and y ( t ) is defined as Txy (7) = E {X* ( t ) Y (t + 7)} .16 Chapter 1 . become correlation and covariance sequences. Signals and Signal Spaces Cross Correlation and Cross Power Spectral Density. the correlation and covariance functions. The following definitions for discrete-time signals basically correspond to those for continuous-time signals. Thus.82) The discrete-time Fourier transform of the autocorrelation sequence is the power spectral density (Wiener-Khintchine theorem). We have M (1. however.84) .

Y(n + N y The terms x x H and gxH are dyadic products.1. ..89) = E{YXH}. Random Signals 17 The definition of the cross correlation sequence is m=--00 A cross covariance sequence can be defined as Correlation Matrices. of a stationary process z(n) has the following Toeplitz structure: . .IllT. . where X = [z(n). A u t o and cross correlation matrices are frequently required. (1. Rzy = E{xxH}. z(n + + NZ .1 ) I T ..91) .. y ( n+ l). . .90) . We use the following definitions R.. (1.3. z(n l). (1. For the sake of completeness it shall be noted that the autocorrelation matrix R. . Y = [ y ( n ) .

we get (1. Ergodic Processes. An exception to this rule is the ergodic process. For the autocorrelation function of an ergodic continuoustime process we have (1. (1. the property r.93) and thecross correlation matrix R. Accordingly. Usually. A wide-sense stationary continuous-time noise process x ( t ) is said to be white if its power spectral density is a constant: (1.80) by taking stationarity into consideration. = E { y X"} has thefollowing structure: Auto and cross-covuriunce matrices can be defined in an analog way by replacing the entries rzy(m)through czy(m).92) which is concluded from (1. (-4= c (4 .(i).f. the autocorrelation function is calculated according to (1. we have . Continuous-Time White Noise Process.. 7 (1.70) by taking the ensemble average. . Signals and Signal Spaces Here.95) where iz(t)is an arbitrary realization of the stochastic process.zy(-i) = ..97) S z z ( W ) = CJ 2 . If two processes x ( n ) and y(n) are pairwise stationary. where the ensemble average can be replaced by a temporal average. has been used.96) for discrete-time signals.18 Chapter 1 .

Discrete-Time White Noise Process.98) The autocorrelation function of the process is a Dirac impulse with 2: r z z ( 7 )= uz d(7). See Figure 1.2.1. cpi(t)X ( t ) dt are Gaussian random variables with E { a ? } = cT2 we call x ( t ) a Gaussian white noise process.3. A discrete-time white noise process has the power spectral density SZZ(&) = cTz 4Series expansions are discussed in detail in Chapter 3.2 for an illustration. Continuous-Time Gaussian White Noise Process.99) . However. A bandlimited white noise process is a whitenoise process whose power spectral density is constant within a certain frequency band and zero outside this band. vi Bandlimited White Noise Process. t -%lax umax 0 Figure 1. Random Signals 19 weight (1. a). (1. The basis satisfies If the coefficients of the series expansion given by a = i 1. a] via a series expansion4 with an arbitrary real-valued orthonormal basis cpi(t)for L2 (-a. Since the power of such a process is infinite it is not realizable. Bandlimited white noise process. the white noise process is a convenient model process which is often used for describing properties of real-world systems. We consider a realvalued wide-sense stationary stochastic process ~ ( tand try to represent it ) on the interval [-a.

Q ! z ( t + ~ .101) = TZZ(T) * h(.103) = /rZZ(. .3 Transmission of Stochastic Processes through Linear Systems Continuous-Time Processes. Signals and Signal Spaces = 2 f J dmo..100) 1.X ) h(X)dX ~ } (1.X) /h*(a)h(a X) dadX + .).101): SZY(W) = S Z Z ( W ) H ( w ) . (1.P ) } h*(a)h(P)dadP ) (1. (1.102) Calculating the autocorrelation function of the output signal is done as follows: = / / E { x * ( ~ . The cross correlation function between the input process ~ ( t ) the output and process y ( t )is given by - L c m E { ~ * (xt() + T .3.20 and the autocorrelationsequence TZdrn) Chapter 1 . We assume a linear time-invariant system with the impulse response h(t).which is excited by a stationary process ~ ( t ) . The cross power spectral density is obtained by taking the Fourier transform of (1.

Discrete-Time Processes. where a system with impulse response h(n) is excited by a process z ( n ) .109) As before. (1. we obtain the power spectral density of the output signal: Sy. The cross correlation sequence between input and output is %y(m) = r z z ( m ) * h(m).yielding the output process y(n). Random Signals 21 Thus. only the magnitude frequency response of H ( w ) canbedetermined from S Z Z ( W ) and S y y ( 4 .(w) = Szz(w) I H ( w ) I 2 . Theresults for continuous-time signals and systems can be directly applied to the discrete-time case. but we will returntothis topic in Section 5 of Chapter 2. We observe that the phase of H ( w ) has no influence on Syy(w).104).107) For the autocorrelation sequence and thepower spectral density at the output we get (1. Here we cease discussion of the transmission of stochastic processes through linear systems.(ej") = szz(eju) IH(eju)l" (1.3. Szy(ejw) Szz(ej") (1. (1. we obtain the following relationship: Taking the Fourier transform of (1. where we will study the representation of stationary bandpass processes by means of their complex envelope.105) Consequently...(ej").1. .108) s.106) The cross power spectral density becomes = H(ej"). the phase of H(ej'") has no influence on S.

Then we will study the Fourier.1) Analogous to the reciprocal basis in discrete signal representations (see Section 3. t )d t . (2. and Hilbert transforms. In the following. e) 22 S E S. . 2.t) may be found such that the density P(s) can be calculated in the form *(S) = S. ~ ( t ( s .1 Integral Transforms The basic idea of an integral representation is to describe a signal ~ ( t ) its via density $(S) with respect to an arbitrarykernel p(t. S) ds.buttherearemany other transforms of interest. Finally. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Chapter 2 Integral Signal Represent at ions The integral transform is one of the most important tools in signal theory. Hartley. S): $(S) p(t. Filter Banks. t E T.3) a reciproalkernel O(s.Signal Analysis: Wavelets. The best known example is the Fourier transform. we will focus on real bandpass processes and their representation by means of their complex envelope. W will first discuss the basic concepts of integral transforms. Time-Frequency Transforms and Applications.

T ) . (2. X E &(R).1. = in Equations (2. L 2 ( a ) cp(t.2) we obtain 2(s) S. Integral Transforms 23 Contrary to discrete representations. W E R. By this we mean a generalized function with the property ~(t) = L cc d(t . T) p(t.S ) and @(S. From (2.2) and (2. we obtain In order tostatethe condition for the validity of (2.For the Dirac impulse the correspondence d ( t ) t 1 is introduced ) so that (2._ . that is G.4) show that the kernel and the reciprocal kernel must satisfy S.8) By substituting (2. t ) dt .3) and (2.1).T ) ) . S) ds = d ( t . t ) be integrable with respect to t. thatis cc GCY(u) = l c c g a ( t ) .4) can be expressed as X(W) 1 X(W) the frequency domain. ( X dT. we do not demand that thekernels cp(t.3) in a relatively simple form the so-called Dirac impulse d(t) is required.-jut - dt e. 201 W 2 we find that it approximates the constant one for a + 0. (2.2.4) The Dirac impulse can be viewed as the limit of a family of functions g a ( t ) that has the following property for all signals ~ ( t ) continuous at the origin: An example is the Gaussian function Considering the Fourier transform of the Gaussian function.1) into (2. = e(s.a) d a e ( s .(w) M 1.

.10) Sij Equations (2. 2 .12) For signals ~ ( t )span {p(t. The discrete representation via series expansion.11) Transforms that contain a self-reciprocal kernel are also called unitary.8j)= the discrete case (see Chapter 3). for Self-Reciprocal Kernels.t ) d t (2.si). 2 . (2.. In order to explain this relationship. let us consider the discrete set pi(t) = p(t.2) yields *(S) = L Z ( t ) O ( s . . . can be regarded as a special case of the integral representation.10) shows that in the case of a discrete representation the density ?(S) concentrates on the values si: *(S) = CQi 1. i = 1 . t ) . They correspond to orthonormal basesin the discrete case and satisfy p(t.c) O(s.Si). .si). 11 The Discrete Representation as a Special Case. (2. (2.15) . Integral Signal Representations p(t.13) i i Insertion into (2. 3 . A special category is that of self-reciprocal kernels. . because they yield 151 = 1 1 ~ 1 1 .14) The comparison with (2.} we may write E (2. t ) d t = S(s . = e*(s. . IT (2. i = 1 . which is discussed in detail in the next chapter.24 which implies that r Chapter 2.10) correspond to the relationship (cpi.0).8) and (2. &(S .

t ) d r d t d s . E L2 ( T ) .17). t ) @ * ( S . t ) d t . Let the signals z ( t ) and y(t) besquareintegrable. (2. t ) ( ~ ( 7 ).2. z.17) = 6 ( t . ( X O(s.18) we conclude that 6) = (2.20) and (2.18) T z ( t ) Y * ( t ) dt are introduced. ( X y*(r)dr.18) yields (2. @(S. Substituting (2. From (2.1. t ) is a self-reciprocal kernel satisfying S.7). !A * (2.19) T Because of (2.21) is known as Parseval’s relation. For y ( t ) = z ( t ) we obtain (&. (2. Now the inner products (X7 U) = / (2.r) y * ( t ) O*(s.r ) d t d r = l ($7 (2.22) . d s S (2. Integral Transforms 25 Parseval’s Relation.G) = l x ( r )l y * ( t ) 6 ( t .19) becomes @.g)= ( x 7 x ) + 121 = l l x l l 11 (2.16) into (2.7) d s S.fj) = /// S T ) .21) Equation (2. (2. O(s.For the densities let y ?(S) = lz(t) = O(s.16) where O(s.20) ) .

and for the reciprocal kernel we have' O(W. More elaborate discussions can be found in [114.f ) = exp(j2xft). Here.-jut.25) for all t where z ( t ) is continuous.2 The Fourier Transform We assume a real or complex-valued. Most proofs are easily obtained from the definition of the Fourier transform itself. m). l A self-reciprocal kernel is obtained either in the form cp(t. The kernel used is 1 ' 2n cp(t. z ( t ) can be reconstructed from X ( w ) via the inverse Fourier transform 0 0 z(t) = 2n X ( w ) ejWtdw --oc) (2. (2. S = (-m.23) exists.-jut dt (2.w) = exp(jwt)/& integrating over frequency f . then 00 ~ ' ( t )j w td t C 4.For such signals the Fourier transform 0 0 X ( w ) = L m z ( t ). (2. not over W = 2xf: cp(t.26) t ) = . X is continuous. 3. or by . If the derivative z'(t) exists and if it is absolutely integrable. We will now briefly recall the most important properties of the Fourier transform.24) + m and W + -m we have X ( w ) + 0. (2.W ) = -eJWt. and f is the frequency in Hertz. The Fourier transform X ( w ) of a signal X E Ll(IR) has the following properties: 1. T = (-m. X E ~ o o ( I R ) with I I X lloo I 11~111.27) In thefollowing we will use the notationz ( t ) t X ( w ) in order to indicate ) a Fourier transform pair. continuous-time signal z ( t ) whichis absolutely integrable (zE Ll(IR)). 2. For W = j w X(W). Integral Signal 2. If X ( w ) is absolutely integrable.26 Representations Chapter 2. W = 2 nf . 221. m).

+ (2. ) (2. (2. ) X(-W).t o ) Accordingly. The correspondence for conjugate functions is z*(t)t X * ( . The Fourier Transform 27 Linearity. Then ) X ( t ) t 27rz(-w). For any real a . The generalization of (2. Let z ( t ) t X ( w ) be a Fourier transform pair.WO) ) 2 + 1X ( w - +WO).W ) .29) (2. For any real 2 WO.24) is . ) d" dw " (2.36) Convolution. (2.z ( t ) t (jw)" ) d" dt" X(w). A convolution in the time domain results in a multiplication in the frequency domain.30) Shifting.28) Symmetry. we have (2. we have 1 coswot z ( t ) t .33) Conjugation. (2. theFourier transform of real signals z ( t )= X* ( t )is symmetric: X * ( W ) = Derivatives.35) Accordingly.23) that az(t) Py(t) + t ) a X ( w ) PY(w). (-jt)" z ( t ) t . . For any real t o . we have z(t .34) Thus. t e-jwto ) X(w).31) e j w o t z ( t ) t X ( w . ) Scaling.X @ ) . (2.X ( w .2.WO). It directly follows from (2.2.32) Modulation.

41) with z ( t ) = y(t) we see thatthe calculated in the time and frequency domains: signal energy be can (2. Using the notation of inner products. inner products of two signals can be calculated in the time as well as the frequency domain.21) by using the fact that the scaled kernel (27r)-iejwt is self-reciprocal.39) and the nth derivative of X ( w ) at the origin are related as (2. Parseval’s relation may also be written as 1 ( 2 ’ 9 )= # ’ V .( X ’ X ) .42) From (2.1.28 Chapter 2.40) Parseval’s Relation. z(t) y(t) t ) 1 X(w) 27r * Y(w). For signals z ( t )and y ( t ) and theirFourier transforms X ( w ) and Y ( w ) . Integral Signal Representations Accordingly. = (2. In vector notation it can be written as 1 (2’2) . we have L cc ~ ( t ) dt = y*(t) 27r X Y wd w ) ( *( . According to Parseval’s relation.41) This property is easily obtained from (2.44) 27r .respectively. (2. d n = 0.43) This relationship is known as Parseval’s theorem. The nth moment of z ( t ) given by cc tn ~ ( t t). ) -W (2.2. (2. (2.38) Moments.

given by cas w t = cos w t wt. 2lr -m where both the signal x(t) and the transform XH(W) real-valued. The Haxtley Transform 29 2. The Relationship between the Hartley and Fourier Transforms. given by The Fourier transform may be written as cc X(w) = l c c x ( t ) e-jwt dt cc x(t) coswt dt . The forward and inverse Hartley transforms are given by m + XH(W) l m x ( t )dt = caswt and x(t) = I XH(W) ] caswt dw. However. Let us consider the even and odd parts of the Hartley transform.jX&(W) XH(W) (2.45) This kernel can be seen as a real-valued version of d w t = cos w t j sin wt.3.47) In the literature.50) +Xff(-W) 2 .X f f ( .3 The Hartley Transform In 1942 Hartley proposed a real-valued transform closely related to theFourier transform [67]. It maps a real-valued signal into a real-valued frequency function using only real arithmetic. the kernel of the Fourier transform.W ) 2 . The kernel of the Hartley transform is the so-called cosine-and-sine (cas) function. are (2. we use the non-symmetric form in order t o simplify the relationship between the Hartley and Fourier transforms.2.46) (2.j XH(W) . sin + (2.j = X & ( W ) . also finds a more symmetric version based on the selfone reciprocal kernel (27r-+ cas wt.

Inthe following we summarize the most important ones. Time Inversion. we have z(t . The Hartley transform can be written in terms of the Fourier transform as X&) = % { X ( w ) }.sinal . Integral Signal Representations %{X(W)} = X % w ) .W ) . (2. some propertiesare entirely different.54) with a = -1 we get z(-t) t ) Xff(-w). 0 cc z(t .54) Proof.t o ) t coswto X H ( W ) sinwto X H ( .30 Thus.52) Due to their close relationship the Hartley and Fourier transforms share manyproperties.56) Proof. (2. However.S { X ( w ) } . We may write L cas ( a yields (2.t o ) caswt dt = L z(J) cas ( W [ [ + t o ] )dJ. sinp Expanding the integralon the right-hand side using the property + p) = [cosa + sinal cosp + COS^ . Linearity. For any real t o .56). ) + (2. From (2. (2.51) S { X ( W ) } = -X&(w). (2.53) * Scaling. Chapter 2. It directly follows from the definition of the Hartley transform that a z ( t )+PY(t) a X H ( w )+ P Y H ( W ) . For any real a.55) Shifting. we have (2.

57) Proof.2. ( w ) + sin (y) x x&(w.58) Proof.X& 2 Derivatives. Using the property 1 cosa casP = . For the nth derivative of a signal x(t) the correspondence is d" ~ dtn z ( t ) t W" [cos ) (y) X H ( W . For any real WO. Let y ( t ) = x ( t ) . By writing jn as jn = cos( + j sin(?). 0 . based on (2. 1.sin ) (y) w ) XH(-W)].].3.48) and (2. this means yH(w) = w n [cos(?) + cos (y).(w) For the Hartley transform. Rearranging this expression.WO) ) 2 1 (2.49). we get g Y(w) = = W" y) [cos (y) j + [cos sin (y)~ ( ] W" + j wn (y){ X ( w ) } .WO) 1 + .sin (y){ X ( W ) } ] % S [cos (y){ X ( W )+ sin (y){ x ( w ) } ] S } % x&((w) -sin(?) x.P) 2 we get 1 + 5 cas ( a + P). we have 1 +2 +WO).X H ( W .cas ( a . The Haxtley Transform 31 Modulation. yields (2. z(t) cas ( [ W +wo]t)dt 1 2 . (2. 00 x(t) coswot caswt dt x(t) cas ( [ W = -X& - welt) dt + +WO).The Fourier transform is Y ( w ) = (jw)" x ( w ) . coswot z ( t ) t .58).

w) To derive the last line. we made use of the shift theorem.49) we finally get (2. Pro0f . The corre- spondence is The expression becomes less complex for signals with certain symmetries. We consider a convolution in time of two signals z ( t ) and y(t).then z(t) * y(t) XH(W) YH(-w).r ) caswt dt - L cc 1 1 caswt dt dr z ( r ) [ c o s w ~ Y ~ ( + sinwTYH(-w) ] dr.32 Representations Chapter 2. Integral Signal Convolution. In the Fourier domain. Using (2. then z ( t )* y(t) t XH(W) ) YH(w).59). If z ( t ) is odd. 0 Multiplication.48) and (2. if z ( t ) has even symmetry. The correspondence for a multiplication in time is Proof. The Hartley transforms are XH(W) and YH(w). we have .respectively. For example. cc [z(t) y ( t ) ] caswt dt = * cc z ( r )y(t -r)dr = Iccz(r) [ cc .

we have L cc x ( t ) y(t) dt = '1 27r X H ( W )Y H ( wdw. oneof the reasons t o compute the Fourier transform of a signal x ( t ) is t o derive the energy density S..63) - X$@) + X&+) 2 The phase can be written as (2. the signal energy can be calculated in the time andin the frequency domains: cc E.62) These properties are easily obtained from the results in Section 2.1 by using the fact that the kernel (27r-5 cas wt is self-reciprocal.". In practice.". 0 ) + Parseval's Relation.2. For signals x ( t ) and y(t) and their Hartley transforms X H ( W )and Y H ( w ) . The Haxtley Transform 33 For the Hartley transform this means X g w ) * Y i ( w ) . = I c c x z ( t )d t (2.X & ( w ) * Y i ( w ) X.(w) * Y i ( w )+ X g w ) * Y i ( w ) .(w) = IX(w)I2and the phase L X ( w ) . ) (2.3.60). Writing this expression in terms of X H ( W )and Y H ( wyields (2.64) .61) -cc Similarly. Energy Density and Phase.(4 = I W w I l Z +I~{X(w))lZ (2. In terms of the Hartley transform the energy density becomes S. respectively.

s ds 1 $ 03 (2.69) (2.65) we obtain the Hilbert transform. The Hilbert transform of a signal with non-zero mean has zero mean. It is h(s . because of @ ( O ) = k(0)= 0. Here. we see that.72) B(w) ( W ) X sgn(w) ~ ( w ) . the integration hasto be carried out according to the Cauchy principal value: cc The Fourier transforms of p(t) and i ( t ) are: @(W) = j sgn(w) with -j @ ( O ) = 0. Furthermore.71) (2. (2.70) B(w) = sgn(w) with B(0)= 0.67) dt.t . We observe that the spectrum of the Hilbert transform $(S) equals the spectrum of z ( t ) .t ) = p(t .S) = -1 ~ Choosing the kernel 7r(t .t ) throughout the following discussion. For the reciprocal kernel O(s .t ) = 1 ~ 7r(s .S ) .t ) we use the notation i ( s .4.except for the prefactor .34 Representations Chapter 2.66) With i(s) denoting the Hilbert transform of z ( t ) we obtain the following transform pair: -1 x ( t ) = 7r -cc ?(S).4 2. Integral Signal 2. . (2.1 The Hilbert Transform Definition p(t .67) isvalidonly for signals z ( t ) with zero mean value. the transform pair (2.j sgn(w). In the spectral domain we then have: X(W) = X(w) = @(W) X(w) = j sgn(w) X ( w ) = -j (2.S ) ’ (2.

5.74) We prove this by making use of Parseval’s relation: 27r(z. A real-valued signal z ( t )is orthogonal to its Hilbert transform 2 ( t ) : (X.75) C Q = j -cc IX(W)~~ sgn(w) dw = 0.B . 3.4. See Figure 2.2) = ( X ’ X ) - L J cc X ( w ) [ . provided that the signal has zero mean value.67) and (2. WO B ] where WO 2 B > 0.2. Representation of Bandpass Signals 35 2. Since the kernel of the Hilbert transform is self-reciprocal we have 2. 2. From (2. Example of a bandpass spectrum.2 Some Properties of the HilbertTransform 1.5 Representation of Bandpass Signals A bandpass signal is understood as a signal whose spectrum concentrates in a region f [ w o .1 for an example of a bandpass spectrum. .1.&)= 0. + ’ -0 0 IxBP(W>l * 00 0 Figure 2. (2.j sgn(w)]* X * ( w ) dw (2.70) we conclude that applying the Hilbert transform twice leads to a sign change of the signal.

for positive This means that the analytic frequencies only.36 Representations Chapter 2.2 illustratestheprocedure of obtaining the complex envelope.76) Here. that purpose. In orderto recover a real bandpass signal zBP(t) from its complex envelope xLp t ) .(t) e-jwot.5.1 Analytic SignalandComplexEnvelope The Hilbert transform allows us to transfer a real bandpass signal xBP(t) into a complex lowpass signal zLP(t).+. The reason for this naming convention is outlined below. we first form the so-called For analytic signal xkP( t ) . We observe that it is not necessary to realize an ideal Hilbert transform with system function B ( w ) = . x ~ ~ ( w = xBP(w) ) + j JiBP(w) = xBP(w) l 0 for for (2. signal hasspectralcomponents In a second step. is The Fourier transform of the analytic signal is 2 XBp(w) for W W W > 0.j sgn(w) in order to carry out this transform.1.as shown in Figure 2. Figure 2. = 0. 2BP(t) the Hilbert transform of xBP(t).77) < 0.78) Here. (2.we make use of the fact that ( for (2.80) . (2. is The signal xLP(t) called the complex envelope of the bandpass signal xBp(t). Integral Signal 2. the complex-valued analytic signal can be shifted into the baseband: ZLP(t) = xc.first introduced in [61]: xzp(t) = XBP(t) +j ZBP(t). the frequency WO is assumed to be the center frequency of the bandpass spectrum.

cos(uot v(t) tane(t) = -.79) we then conclude for the bandpass signal: ZBP(t) = IZLP(t)l + e(t)). and the is . Representation of Bandpass Signals 37 \ ' / I WO W I 00 W Figure 2. Producing the complex envelope of a real bandpass signal. \ / 2 1 2 ( t ) + 212 ( t ) .2.2.3). (2.81) wit h IZLP ( t )I = .83) We see that IxLP(t)l can be interpreted asthe envelope of the bandpass signal (see Figure 2.82) From (2.5. Accordingly. u(t) (2. zLP(t) called the complex envelope. is Another form of representing zBP(t) obtained by describing the complex envelope with polar coordinates: (2.

84) signal Thispropertyimmediatelyresultsfromthefact contains only positive frequencies. Bandpass signal and envelope. Here. The real part u ( t ) is referred to as the in-phase component. Equation (2.85) is considered. which will be discussed below.3. the real bandpass signal zBP(t) produced from zLp t ) is ( according to (2. In thereceiver.P ( t ) e-jwot. zLp t )is finally reconstructed as described ( above. Application in Communications.38 Chapter 2. For O ( t ) = 8 0 we have a pure amplitude modulation. Here. In communications we often start with a lowpass complex envelope zLP(t) wish to transmit it asa real bandpass and signal zBP(t). have to add the imaginary signal ju(t)sinwot to the we bandpass signal: z(p)(t) := u(t) [coswot + j sin wot] = u ( t ) ejwot. Integral Signal Representations Figure 2. and the imaginary part w ( t ) is called the quadrature component. In order to reconstruct u(t) from zBP(t). However.87) .83) shows that bandpass signals can in general be regarded as amplitude and phase modulated signals. analytic signal is called the pre-envelope.79).86) Through subsequent modulation we recover the original lowpass signal: u ( t )= . thatananalytic (2. It should be mentionedthat the spectrumof a complex envelopeis always limited to -WO at the lower bound: XLP(w) 0 for W < -WO. () (2. u(t)is a given real lowpass signal. one important requirementmustbe met. The real bandpass signal zBp(t) = u ( t ) coswot (2. (2.

Bandpass Filtering and Generating the Complex Envelope. under condition (2. that is. ( The complex envelope describes the bandpass signal unambiguously.88) the Hilbert transform of the bandpass signal is given by 2 ( t ) = u(t) sinwot. In practice. is to generate u(t)sinwot from u(t)coswot in the receiver.5.88) the required signal z(p)(t) equals the analytic signal ziP(t). (2. The problem.4. reverse.88) is met. under condition (2. is only can the possible if condition (2.88) is violated.88) As can easily be verified.89) Thus.4. which means that U(w) 0 for w < -WO. however. however. This is illustrated in Figure 2.This means out .2. Complex envelope for the case that condition (2.and the complex envelope zLp t )is identical to the given u(t). We now assume that u ( t ) ejwOt is analytic. zBP(t) always be reconstructed from zLP(t). Representation of Bandpass Signals 39 -W0 I WO W Figure 2. generating a complex envelope usually involves the task of filtering the real bandpass signal zBP(t) of a more broadband signal z ( t ) . (2.

We consider a signal y(t) = z ( t )* g @ ) . we have If we finally describe the analytic bandpass envelope of the real bandpass by means of the complex (2.5 for an illustration. (W) =X (W +WO) G P(W). and the realization effort is reduced. L (2.where z ( t ) . This requirement means that IG(w)I musthave even symmetryaround W O and the phaseresponse of G ( w ) must be anti-symmetric. the analyticsignal can be calculated as (2. Realization of Bandpass Filters by Means of EquivalentLowpass Filters. The analytic bandpassg+@) associated with g ( t ) has the system function G+(w)= G ( w ) [l j + B(w)].y(t).93) this leads to XL. The equivalent lowpass G L P ( w ) usually has a complex impulse response. Only if the symmetry condition GLP(u) GE.(-w) is satisfied. Integral Signal that zBP(t) z ( t ) * g ( t ) has to be computed. and g ( t ) are . See Figure 2. the result is = a real lowpass. In this case we also speak of a symmetric bandpass.91) For the complex envelope.where g ( t ) is the impulse response = of a real bandpass.90) Using the analytic bandpass.40 Representations Chapter 2.94) We find that XLP(w) is also obtained by modulating the real bandpass signal with e-jwot and by lowpass filtering the resulting signal. (2.

97) Y(w) = = X ( W )G ( w ) : X ..WO) G : . (2.(-W-WO) The last two terms vanish since G.G : . real-valued. +: X. (W .(-W .X.WO) G. the system function of the filter can be written as 1 1 G(w)= .. (W L .5.WO) .WO) and X.X.W O ) -WO).WO).WO) (-W . (W .Y. = 0 for W < -WO .G:.. (2.G. (2. For the spectrum we have 1 2 1 + .95) X (W) = ..*.W O ) ..WO) WO) G:.WO) GLP(w-wo).. The signal z ( t )is now described by means of its complex envelope with respect to an arbitrarypositive center frequency W O : z ( t )= ?J3{ZLP(t) e j w o t } . W (W) .(-w (-W (2.5.(-W (W . Generating the complex envelope of a real bandpass signal.98) +:XL. (W .(W -WO) + .2.WO). ( --W . . Y.99) = . Representation of Bandpass Signals 41 Lowpass Figure 2. 2 (2. (-W ... 2 2 For the spectrum of the output signal we have + (-W .WO) -WO) +$ X.96) Correspondingly... (W) = 0 for < -WO: Y(w) = a xw(~ +a X.

We have (X+. Integral Signal Representations This means that a real convolution in the bandpass domain can be replaced by a complex convolution in the lowpass domain: Y(t) = z(t)* g ( t ) + YLP ( t )= 5 Z L P ( t )* QLP (t). The group and phase delay of a system C(w) are defined as (2.42 Altogether this yields Chapter 2. (2.106) where C ( W ) = IC(W)I &+). Y) * (2. Y) + (%C) + j ( 2 . we get (2.105) and (2.73).107) . Group and Phase Delay.102) where z(t) and y(t) are real-valued. This prefactor did not appear in the combination of bandpass filtering and generating the complex envelope discussed above.101) Note that theprefactor 1 / 2 must be takeninto account. . we get for the real part %{(X+. Inner Products.104) For the implementationof correlation operations this means that correlations of deterministic bandpass signals can be computed in the bandpass domain as well as in the equivalent lowpass domain.Y+> = ( X . We consider the inner product of two analytic signals z+(t)= ~ ( t ) 2 ( t ) +j and y+(t) = y(t) +j c ( t ) . Y + > l = 2 (2. (2.103) If we describe ~ ( t ) y(t) by means of their complex envelope with respect and to the same center frequency.Y) + j (X. 1 (2. a real filter gLP(t)is obtained if G(w) is symmetric with respect to WO. As before.C) Observing (2.

113) Nowwe consider the transformed process 2 ( t ) . The system function of the associated analytic bandpass may be written as Because of B << W O .. (2.522 ( W ) . with T~ and T~ according to (2. let us assume that C ( W is a narrowband bandpass ) with B << W O .100) we get yLP ) (W M -~ 1 ~ ( w e-jwoTp(wo) Io) 2 e--jwTg(wo) XLP@). in the time domain provides a phase shift by which means that the narrowband system C(W) T ~ ( W O ) and a time delay by T~( W O ) .j W O T p ( W o ) e-jwTg(wO).106). 2. wide-sense stationary bandpassprocess z ( t ) .110) CLP(w) M IC(w0)l e .. we conclude from (1.2 StationaryBandpass Processes In communicationswe must assumethat noise interferes with bandpasssignals that are to be transmitted. 5 B/2. (T) = T. .2. (2. We assume a real-valued. The autocorrelation function of the process is given by T. For the power spectral density of the transformed process. CLp( W ) may be approximated as For the complex envelope CLP(w) = C.109) (2.. (-T) =E { ~ ( t ) + T)} ~ ( t .111) Hence.105) and (2. If we now look at the inputoutput relation (2. zero mean.(W + W O ) it follows that W (2.5. Therefore the question arises of which statistical properties the complex envelope of a stationary bandpass process has.105): 1 for W#O o for W=O . Representation of Bandpass Signals 43 In order to explain this.5.

119) = 2 Tzz(. &(T) Tz?(-T) Now we form the analytic process z+(t): X+@) = z ( t ) +j q t ) . Thus. (2.+.+(W) {.121) = u ( t )+ j w(t). for the cross correlation functions: TZ&(T) = = +ZZ(T). qt (2.T ) = -tzz(.120) Finally.117) = t z z ( .l (2. &% Hence. The power spectral density is S.44 Representations Chapter 2.l . the process 2 ( t ) has the same power spectral ) density. For the real part u(t)we have u(t) = %{[z(t) j $ ( t ) ]e-jwot} + = z ( t ) coswot+P(t) sin& = (2. (2.102): SZ&) SjrZ(W) = = fi(4 &%(W).122) + [z+(t) e-jwot + [X+ ( t ) ] *ejwot]. we consider the complex process zLP(t) derived from the analytic process zLp t ) = z+(t)e-jwot ( (2. + rjrjr (.).+ (T) = = E { [ z ( t+ j 2 ( t ) ] * [z(t+ T) + j ) Tzz (T) + T)]} +j Tzjr (. (2.) + 2 j P. = This means that the autocorrelation function of the analytic process is an analytic signal itself.116) fi*(W) (W). Integral Signal where i ( t ) t I?(w).115) we get according For the cross power spectral densities Szj:( W ) and to (1. .+. and consequently the same autocorrelation function. for W < 0.). as the process z(t): Tjrjr (T) = Tzz (7).(.j TB..118) For the autocorrelation function we have T. 272% ( W ) (2. SZZ((w) for W > 0.

(T) sin WOT In a similar way we obtain for the autocorrelation function the imaginary part of the complex envelope.2..l = --Tvu(.5.123) two complex exponential functions dependent on whose prefactors reduce to zero: t are included E{[Z+(t)]* [z+(t+T)]*}* E { z + ( t )z+(t+.127) From (2. Representation of Bandpass Signals 45 (2.r)) = = E { ( z ( t ) j q t ) ) (z(t + + T) + j 2(t + T))} (2. of The cross correlation functionbetween the real and theimaginary part is given by Tuv (.125) = T.123) In (2..(2.127) we conclude that the autocorrelation function of the complex envelope equals the modulated autocorrelation function of the .) (2.125) . (T) cos WOT + F.

129) We notice that the complex envelope is a wide-sense stationary process with specific properties: 0 The autocorrelation function of the real part equals that of the imaginary part.46 analytic signal: Chapter 2. (2. < 0.(T) = 0. It also means that the cross correlation between the real and imaginary part vanishes: TU.130) Hence. we have (W> = SzLPzLP ( .W ) * (2. we get for the power spectral density: 4 &. we see that the autocorrelation function of xLP(t)is real-valued. we have Tuv(O) e = rvu(0)= 0.131) . In the special case of a symmetric bandpass process. (2. The cross correlation function between the real and imaginary part is antisymmetric with respect to r. v r. Integral Signal Representations Correspondingly.(W + WO) for W for W + + WO WO > 0. In particular.

Time-Frequency Transfomzs and Applications. optimal discrete representations will be discussed in Chapter 5. If the signals that are to be transformed consist of a finite set of values. Filter Banks. We consider a real or complex-valued.1 Introduction X. Moreover. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 ElectronicISBN 0-470-84183-4 Chapter 3 Discrete Signal Representations In this chapter we discuss the fundamental concepts of discrete signal representations. 3. Such representations are also known as discrete transforms. series expansions. Examplesof widely used discrete transforms are given in the next chapter.Signal Analysis: Wavelets. Discrete signal representations are of crucial importance in signal processing. and they allow easy handling of continuous and discrete-time signals on digital signal processors. or block transforms. one also speaks of block transforms. The term “discrete” refers to the fact that the signals are representedby discrete values.whereas the signals themselves may be continuous-time. continuous or discrete-time signal assuming that z can be represented in the form n i=l 47 . They give a certain insight into the properties of signals.

The space X is called the sum of the subspacesX1 and X2. .>.p. p } and is an element of the linear subspace X2 = span {pm+l. referred to as the representation of X with respect to the basis . . i = 1. . . we call them a basis for X . If the decomposition of X E X 2 2 lDefinition of a linear subspace: let M be a non-empty set of elements of the vector space X . . X One often is interested in finding the best approximation of a given signal by a signal 2 which has the series expansion i=l This problem will be discussed in Sections 3. the notation .3 in greater detail. The vectors pi.. Hence. This means that all linear combinations of the elements of M must be elements of M . .48 Representations Chapter 3. . i = 1. For the present we will confine ourselves to discussing some general concepts of decomposing signal spaces. if M itself is a linear space.n can be arranged as a vector whichis {cpl. If they are linearly independent.p} For this.. Then M is a linear subspace of X . . p. will be used henceforth. .n may be linearly dependent or linearly independent of each other. .. .}. . .. . .. .2 and 3. We start by assuming a decomposition of X into where Signal x1 is an element of the linear subspace' X1 = span { p l .. X itself is a linear subspace. . >P. The coefficients ai.}. The signal space itself is the set of all vectors which can be represented by linear combination of { p l . . . . . . Discrete Signal The signal X is an element of the signal space X spanned by {pl..

10) we have X E X with X = span ( u 1 . If a space X is the direct sum of two subspaces X1 and X2 and x1 E X1 and xz E X2 are orthogonal to one another for all signals X E X . .8) A direct sum is obtained if the vectors that span X1 are linearly independent of the vectors that span X. then X is the orthogonal sum of the subspaces X1 and X. .Series3. By taking the inner product of (3.2.11) Here. u1.n. Because of (3. . (3.} are given is easily answered. j = 1.n spans a one-dimensional subspace. .2. Each vector ui. . Orthogonal 49 into X I E X1 and x z E X2 is unique. (3.{ 0 for i = j .13) .. i= 1 n (3. I (3.xz)= 0 V X E X .U j ) = sij. . . .10) where the vectors ui satisfy the orthonormality condition ( U i . . . . and X is the orthogonal sum of these subspaces.n and using (3. .and X is called the direct sum of X1 and X.9) 3. .. . . j = 1. that is if (x1.. For this we write X =X1 e x2..12) For all signals X in (3.1 Orthogonal Series Expansions Calculation of Coefficients X We consider a signal that can be represented in the form X = C"i ui..u2. un form an orthonormal basis for X . The notation for the direct sum is X = X1 ex2. . un}. u2. (3.2 we speak of a direct decomposition of X into the subspaces X1 and X Z .11) we obtain "j =(X&)....2 3... The question ofhow the coefficients ai can be calculated if X and the orthonormal basis ( u 1 . (3. U. i = 1. 6ij is the Kronecker symbol 1 '' . . otherwise.10) with u j .11).

we have the following relationship between the norms of X . which means that m 1 1 (3.z. Orthogonal projection.. Possibly.M M The subspace M A is orthogonal to M m . x X. : .1 gives an illustration.10) we assumed that X can be represented by means of n coefficients a1 .17) (3. Figure 3. X E+ Em . Discrete Signal Representations Figure 3. 3. non-orthogonal bases. 3The proof is given in Section 3. As can easily be verified. 1 = (z.f. ..3.1.18) z=f++. f ) = 11% - 21 = (z. l ) Because of + 6 we call D the orthogonal projection of z l3 onto M.16) i d This result has a simple geometrical interpretation in terms of an orthogonal projection.f is orthogonal to f (notation: + & . so that for practical applications we are interested in finding the best approximation m (3.50 Chapter 3.15) The solution to this problem is3 (3.a. which means that the signal space X is decomposed as follows: X=Mm$Mk with I (3. .f)z = min.X and + (3.u i ) .2.19) 1 1 4 1 2 = llf112 + ll+112.2 Orthogonal Projection In (3.2 for general. and + = z . Each basis vector u spans a subspace that is orthogonal to the i subspaces spanned by u j . j # i.a l . . . n is infinitely large. .14) i= 1 in the sense of d ( z .

.3. n } .. i scheme: (3.20) This method is known as the Gram-Schmidt procedure.3 The Gram-Schmidt Orthonormalization Procedure Given a basis {vi.23) .2.22) we have (3. = 1. . . we can construct an orthonormal i basis { i i = 1.. n } by using the following u. .21) and (3. 3. . = 1. A re-ordering of the vectors pi before the application of the Gram-Schmidt procedure results in a different basis.. .4 Parseval’s Relation Parseval’s relation states that the inner product of two vectors equals the inner product of their representations with respect to an orthonormal basis. .. It is easily seen that the result is not unique. Given n X =x a i= 1 iu i (3.2.n } for the space span {vi. . OrthogonalSeriesExpansions 51 3.2.

19) and (3. any signal z(t) E Lz(a.2.whereas the inner product of the signals may have P a different definition.23) llzll = l l a l l .b) can be approximated with arbitrary accuracy by means of an orthogonal projection $(t> = c n (2.21) into (3. b ) is complete. Thus.27) i= 1 where n is chosen sufficiently large and the basis vectors cpi(t)are taken from a complete orthonormal set. (3.25) For z = y we get from (3. The inner product of the signals may even involve a weighting matrix or weighting function.26) It is important to notice that the inner product of the representations is defined as ( a . Discrete Signal Representations (3. V (3.5 Complete Orthonormal Sets It can be shown thatthe space Lz(a. i )cpi(t).24) This is verified by substituting (3.52 with Chapter 3. ) = p H a . According to (3. 3.23) and by making use of the fact that the basis is orthogonal: (3.23) we have for the approximation error: .

2. vi) An orthonormalset is said tobe complete if noadditional non-zero orthogonal vector exists which can be added to the set. The weighting function is g ( t ) = 1. l]. Legendre Polynomials.28) we conclude n (3. Note that any finite interval can be mapped onto the interval T = [-l. The Legendre polynomials Pn(t). OrthogonalSeriesExpansions 53 From (3.31) (3.( t 2 . are defined as Pn(t)= -.29) then becomes the completeness relation cI cc . 1 . .29) is called the Bessel inequality. When an orthonormalset is complete. the approximationerrortends towards zero with n + CO.1)n l P (3.6 Examples of Complete Orthonormal Sets Fourier Series. exists. Theinterval considered is T = [-l.33) which form a complete orthonormalset.32) 3.3.35) . .=l (X.30) Here.(n . n (3..29) i= 1 (3.n = 0 . It ensures that the squared sum of the coefficients ( X . (3.b).1)t Pn-l(t) .34) 2"n! dtn and can alternatively be computed accordingto the recursion formula 1 Pn(t)= -[(2n .1) Pn-z(t)].The Bessel inequality (3. One of the best-known discrete transforms is the Fourier series expansion. The basis functions are the complex exponentials (3. Parseval's relation states (3. +l] .(Pi) l2 = 1lXIl2 v E Lz(a.2.

1. lt T2(t) = 2t2 .36) Chebyshev Polynomials. Discrete Signal Representations A set pn(t).2. The first four polynomials are (3. T3(t) = 4t3 . . l] with weighting function g ( t ) = 1 is obtained by (3. ( t ) for n > o (3.3t.38) To(t) = 1 7 T ( ) = t.39) we get a set whichis orthonormal on the function g ( t ) = (1 - interval [-l7 with weighting +l] Laguerre Polynomials. n = 0 .Tn-2(t). . 1 . 2 . The Laguerre polynomials ~ .. . ( t = et-(tne-t). n 2 0. -1 5 t 5 1. (3. which is orthonormal on the interval [-l. (3.54 The first four functions are Chapter 3..1. The Chebyshev polynomials are defined as Tn(t)= cos(n arccos t ). . Using the normalization ~ o ( t ) for n = o V n ( t )= m ~ .37) and can be computed according to the recursion Tn(t)= 2t Tn-l(t) .40) . ) dt dn n = 0.

(t) = (2n . (3. (3.+ W ) . = 0 . the polynomials $"(t). The first four basis vectors are An alternative is to generate the set . 2 . Hermite Polynomials. ..43) which is orthonormal with weight one on the interval [0. .1)2 Ln-2(t)._l(t) The normalization - ( n .(s).44) The Laplace transform is given by (3. . 1 . . m] with weightingfunction g ( t ) = e c t . The Hermite polynomials are defined as (3.45) Thus. a function fn(t) is obtainedfromanetworkwiththetransfer function F.t ) L. 2 . OrthogonalSeriesExpansions 55 can be calculated by means of the recursion L. (3.41) (Pn(t) -L. As will be shown below. . m]. .2.42) yields a setwhich is orthonormal on the interval [0..1) Hk-z(t).47) .-tP $n(t) = T L " ( t ) . let e-Pt fn(t) = $"(2Pt) = .3. can be generated by a network. . . For this. 1 . 2 . which is excited by an impulse. n = 0 . The network can be realized as a cascade of a first-order lowpass and n first-order allpass filters.1 .(t) = 1 n! n = 0 . (3.46) A recursive computation is possible in the form Hk(t) = 2t Hk-l(t) . 1 . 71.2(k . (3.

. .2.56 Representations Chapter 3.50) Further functions can be computed by means of the recursion forO<t<. 1 . in data transmission a transmitted signal may have the form z(t) = Cm d(m) s ( t . . k = 0 . . . 2 . 1 ... they are named according to their number of zero crossings.. Orthogonality is achieved by appropriate zero crossings.m T ). . If we now assume that z(t) is . the Hermite functions cpk(t) = (2' /c! Hb(t). . then we . . However.1.. (3.2 shows the first six Walsh functions.a given basis is often notorthonormal.. .1) for05t<i for < t 5 1 i (3. (3.Correspondingly.51) Figure 3. Discrete Signal With the weighting function g ( t ) = e P t 2 the polynomials q5k(t) = (2k Ic! &)p &(t). The functions (Pk(t) are also obtained by applying the Gram-Schmidt procedure to the basis {tb eCt2I2.3 General Series Expansions If possible.49) form an orthonormal basis with weight one.mT)dt = S.48) form an orthonormal basis for L2 (R). . . where d ( m ) is the data and s(t) is an impulse response that satisfies s(t)s(t . Walsh Functions.} [57]. (3. ..1) for <t 5 1 i m = 1. . 3. = 1 . O transmitted through a non-ideal channel with impulse response h(t). k = 0 . 2 . in practice. For example. The first two functions are given by cpo(t) = ( 1 for O 5 t 5 I . one would choose anorthonormal basis for signal representation.2-1 Ic (2b) cpm+l(t) = ( 2 t . Walsh functions take on the values 1 and -1. k = 0.

} linear set of equations and also via the so-called reciprocal basis.] T (3.52) with pj.p. signals 2 E X with X = span { p l .'pj)=~ai(cpi.1 Calculating the Representation In the following. We assume that the n vectors { p l . the representation a = [ a l .3. . so that the m } question arises of how to recover the data if r ( t ) and g ( t ) are given. p can be computed by solving a . . .n.cpj). .. The set of equations is obtained by multiplying (inner product) both sides of (3.m T ) . .52) i= 1 As will be shown. This new basis { g ( t .p} are linearly independent so ..} will be considered. (3. d ( m ) g ( t. . . 3... . . General Series Expansions 57 have a signal r ( t ) = C .a.54) In matrix notation this is +a=p (3.mT) with g ( t ) = s ( t ) * h @ ) the receiver on side.3. . E Z is no longer orthogonal..n: n (". that all X E X can be represented uniquely as (3. .. i=l j=L.53) with respect to a given basis {pl.55) . j = 1.3.. . ... .

.57) sij which is known as the biorthogonality condition.ej) = .ej) i=l 6ij - Oj. . Discrete Signal Representations Figure 3 3 Reciprocal basis (The basis is ' p 1 = [0. n } . . (3.55) can be solved. the correspond. = vk) vi)*it has (vk. l]T . i . Due to the property 9 = a H . (vi. The disadvantage of the method considered above is that for calculating the representation (y.n. . . ing reciprocal basis is &= [-0. we directly obtain the representation by forming inner products . . 1IT. Multiplying both sides of (3. n . .5. .ej)= Cai = ai. 1IT. (vi. j = 1 . of a new X we first have to calculate P before (3. when using the reciprocal basis. 3 . .58) which means that. .5. . OIT).58 Chapter 3.52) with n (x. Figure 3. j = 1 . . j = 1 .n leads to (3. &= [0.3 illustrates (3. which satisfies the condition (cpi.. Much more interesting is the computation of the representation a by means of the reciprocal basis {ei. 9 is known as the Grammian matrix.57) in the two-dimensional plane. . . i = 1 . 'pz= [2. 2 .

.64) In the last step..P i ) ( Oi (3. n are bases for X . n as well as 8 j . . one of the signals is represented by means of the basis {vl. For the inner product of two signals X = c n ( X . k=l j = l. (3. P} corresponding reciprocal basis {&..65) . also for general bases a relationship between the inner product of signals and their representations can be established. .... General Series Expansions 59 A vector X can be represented as (3.. For this. O k ) = Sik was used. n with the yet unknown coefficients k= yjk: n ej = E r j kpk. . k = 1 .. . . . the vectors 8 j .3.3.. . . n can be written as linear combinations of ( P ~ . . . Calculation of the Reciprocal Basis.( . 1.n. the property (pi. .60) and also as (3.61) Parseval’s relation holds onlyfor orthonormal bases. However.63) we get (X7Y) = (3.. Since pk. . . j = 1. . and a second signal by means of the .62) i= 1 and (3. . On}... j = 1.

.3..72) .70) The reciprocal basis is obtained from (3. For the signal spaces let X = span {vl.vn} .69) r=( 9 ~ ) ~ ~ .67) and 9T = equation (3. Discrete Signal Multiplying this equation with p i . . the best approximation in the sense of (3.71) is obtained for (3. . where Mm C X .66) Wit h (3. i = 1 .n. (3.70). As we will see. (3.. .} . . with m < n. and Mm = span {vl. so that (3.n and using (3. .2 Orthogonal Projection We consider the approximation of X E X by X E Mm. .65). v. 3.67)and (3.68) rGT= I . . (3.60 Representations Chapter 3. . . j = 1 . .66) can be written as (3.57) leads to i . .

77) X = Mm For the vectors we have I @ M:..2 ) .. m } is the reciprocal basis to that the reciprocal basis satisfies {vi. . Note i (3. .( x . . x .75) we conclude that r ) is orthogonal to all vectors in M. e (3.(a . = 1 . . . Because we obtain (X.76) r)=x-x is orthogonal to all 8 j . j = 1 .72).all2 1 2 = ll(z .3.80) .74) Hence. . m.g). .73) Mm = span {vl.v. This also means that X is decomposed into an orthogonal sum (3. . In order to show that 2 according to (3. i. . .72) is the best approximation to X .} . = & j . j = 0 . .(a . (3. From (3. . . General Series Expansions 61 where { e i .. j ) . (3.75) (3..2 ) . . Equation (3.2 . . m. (z 2) . .72) is the orthogonal projectionof X E X onto M.(a .78) x=2+q.2 ) .. .0) = 1 .0j)=0. Requiring only (pi. m . j = 1 . (z-2. ej ) = (X.e j ) with 2 according to (3. . . . m } . (3.: q 1% for all 5 E Mm. m is not sufficient for 8j to form ej) the reciprocal basis.75) shows that j = 1 .. .73) and (3..} = span {el.x-2) + (a-2. i= 1 ei)v i.3. (3. .2)112 = ((X . of (v. 1 .(9-2. 2EMm. . we consider the distance between X and an arbitrary vector 5 E Mm and perform some algebraic manipulations: d2(X.79) The approximation2 according to (3. . i = 1 .e j ) = S i j First we consider the expression ( 2 .a-k). . 5 . . qEMA.e.2) . X E X .2 ) ) = ( x . .2 .

. and 11412 remains.2 ) E M . .}. However.u. B = [bl.72) clearly yields the best approximation.X. . and we have a large number of methods at hand for solving the problem. = span { b l . . for n-tuples the projection can concisely be described with matrices. A relationship between the norms of X. .75).] T E C . With .. .theorthogonal matrix P as projection canbe (3. .3.3 Orthogonal Projection of n-Tuples The solutions to theprojection problem considered far hold for all vectors.87) X=Px. = 1 1 4 2 + llv1I2 (3.lT the approximation is given by n X m matrix (3. so including n-tuples.84) m X 1 vector (3. . .62 Representations Chapter 3.80) are zero. so that (3. and (3. . onto subspaces M . where m < n and bi E C. .85) x=Ba.] and a = [ul..b.b.79) the second and the third termin the last row are zero. .83) 3. Discrete Signal Because of (5. of course.& and v is obtained from Because of (3. the second and third terms in (3.86) described by a Hermitian (3. we consider the projection of X = [XI. Furthermore. In the following.. . such that The minimum is achieved for 5 = P ..

. To compute the reciprocal basis 0 = [e. the representation is calculated according to (3.. (3.56) and (3. For an inner product with a weighting matrix G .91) If B contains an orthonormal basis. equations (3. (3.59) as a = OH% [ B H B ] .3. a = OHGx= [ B H G B ] . (3.56) and (3. Note that the representation according to (3.65) give rT = @-l.] the relationships (3. (3.92) which is known as the normal equation. . = With (3. we obtain = B~GB.90) 2 = BIBHB]-lBHz. (3.89) o = B [BHBI-'.l B H G x . Observing that the inverse of a Hermitian matrix is Hermitian itself. General Expansions 63 Inner Product without Weighting.70).90) is the solution of the equation [B%] a = B H 2 .96) 6 = BIBHGB]-lBHGx.and the projection problem is simplified. we have B H B = I.95) (3.86) the orthogonal projection is (3.Series 3.l B H z . (3. (3. + Thus.65) are used. Inner Product with Weighting.70). BHB.94) (3. 0 = B [BHGB]-l.88) o For the reciprocal basis we then get BrT.93) 0 = B P . .e. which can be written as rT a = = = +-l. 3 . (3. .

If the Grammian matrix is poorly conditioned.97) can be transformed via z = Ha: V into the equivalent problem = (3. Note.100) a=a .4.4 3.98) HB (3.97) and (3. Splitting G into G = H H H can for instance be achieved by applying the Cholesky decomposition G = LLH or by a singular value decomposition. Thus. Both methods can be applied in all cases since G must be Hermitian and positive definite in order to be a valid weighting matrix. numerical problemsmayoccur. even if the vectors are linearly independent. because of finite-precision arithmetic.99) The indices of the norms in (3. 3.99) stand for the weighting matrices involved. The computation of the reciprocal basis involves the inversion of the Grammian matrix. Discrete Signal Representations Alternatively. G can be split intoa product G = H H H .Robustmethods of handling such cases are the QR decomposition and theMoore-Penrose pseudoinverse. A numerically robust solution of = min ! (3.1 Mathematical Tools The QR Decomposition The methods for solving the projection problem considered so far require an inversion of the Grammian matrix.64 Chapter 3.and the problem (3. However. a poorly conditioned Grammian matrix may lead to considerable errors. which will be discussed in the next section. The inversion does not pose a major problem so long asthe vectors that span the subspace in question are linearly independent. the projection problem with weighting can be transformed into one without weighting.

100) yields (3.3.104) because a multiplication with a unitary matrix does not change the norm of a vector.YII = . f = [yj. Substituting (3. and R has the following form: (3. In the following we will show how (3.100) can be solved via &R decomposition. be computed by using Householder reflections or Givens rotations.103) For (3.107) .102) The QR decomposition can. IIQHQRa QHzll= IIRa .4.] (3. Q is a unitary matrix. Mathematical Tools 65 is obtained by carrying out a QR decomposition of B: B=QR.101) (3.4 and 3.105) Yn With . llRa .4. Here. for instance.5. see Sections 3. Using the abbreviation y = Q H x . rmm Ym Ym+l (3.4.101) in (3.106) (3.QHzll la = a l (3.103) we can also write = min.we get Y1 .

B + B a ] H B + B a= 0.110) The solutions (3.(3.(3.113) . However. so that a is easily computed by using Gaussian elimination.114) (3.111) (3. (3. For the norm of the error we have: (3. This matrix is called the Moore-Penrose pseudoinverse [3]. (3.116).66 Representations Chapter 3.112) B [ B HB H-xl.113) .BB+xIH BB+x = 0 . that is.91). a = 5 = [BHBI-l BHX.116) BB+ = ( B B + ) ~ BB'B = B B+BB+ = B+.4.113) (3.115) (3.90) and (3. The expressions B + B and B B + describe orthogonal projections.116) we have [X .108) Note that X is an upper triangular matrix. A general solution to the projection problem is obtained by introducing a matrix B+ via the following four equations B+B = (B+B)H (3. since under conditions (3. Discrete Signal The norm reaches its minimum if a = a is the solution of X a=.2 The Moore-Penrose Pseudoinverse We consider the criterion (3. There is only one B+ that satisfies (3. (3.117) [a. B] can only be applied if [ B H B ] exists. an orthogonal projection can also be carried out if B contains linearly dependent vectors.%. . if the columns of B are linearly independent.109) 3.

4. We will return to this topic in the next section.113) .111) and (3.l . 5 = m .124) 2 Note that (3. Mathematical Tools 67 Assuming that B is an n k = n. so that (3. (3.(3. (3.112) can be replaced by a = B+x.3. B+ = BB B H ] .120) The non-zero values C T ~> 0. X has the following form: (3. They satisfy the pseudoinverse B+ is given by B+ = V X + U H . With C T ~ are called the singular values of B . we have X m matrix which either has rank k = m or B+ = [ B H B ] .119) < n.122) are satisfied.123) (3.l B H .116) with B+ according to (3. [H 5 = n. . (3.118) B+ can for instance be computed via the singular value decomposition B =UXVH. For m (3.110).122) It can easily be shown that the requirements (3. = BB+X. U and V are unitary.123) is not necessarily the only solution to the problem (3.

. Note. which means that N ( B )is spanned by m . These vectors can be chosen to form an orthonormal basis for the nullspace. (3.128) a =h+ Np. It is easily observed that the solution to (3. The pseudoinverse may be written as B+ = [ B H B ] + B". = H V (3. which is dual to a given set cpi(t).T linearly independent vectors. Discrete Signal Representations By taking the productsB H Band B B H we obtain equations for calculating the singular value decomposition. The Nullspace Ba=X. If T = m then N ( B ) is only the null vector. The nullspace of a matrix B consists of all vectors a such that B a = 0 .T .T ) whose column subspace is the nullspace of B then BN=O. where zi = B+X = B + x .127) Let us consider the problem where X = B B+x is the orthogonal projection of an arbitrary X onto the column subspace of B . Finding all solutions is intimately related to finding the nullspace of matrix B . Further methods of calculating the pseudoinverse are discussed in [3].let us assume that B is an n X m matrix that has rank T .@). see ++ 343 .126) This property can be applied to continuous functions.129) . thesquares of the singular values of B arethe eigenvalues of B H B and at the same time of BB".119) we have B H B = V ~ ~ U H U ~ V v [xHx] H .127) is then given by (3. (3. and a = B+$ = B+x is the unique solution to (3.70).110).127) and thus also to (3. The set of all solutions to (3.In order to describe N ( B ) . (3.65) . If we define a matrix N of size m X ( m .110). and with rT = instead of rT = +-l we can compute a set of functions e.125) B B =uxvHvxHuH U ~ = [xx "1 uH. With B according to (3. or we have an infinite number of solutions. (3. It is denoted by N ( B ) . Depending on B we either have a unique solution a . U contains theeigenvectors of B B H .(3. That is. If T < m then N ( B ) is of dimension m . Matrix V contains the orthonormal eigenvectors of B H B .Correspondingly.127) also is the solution to (3.68 Chapter 3.

Inorder to explain the basicidea of the Householder transforms we consider two vectors X .4. P z P ~ + + + (3. Mathematical Tools 69 In (3. because B N = 0 implies that N H B += 0. that is for a = 6.133) The Householder transform is given by H. and we look at the projection of X onto a one-dimensional subspace W = span {W}: P.T columns of V.x.. because this is the solution with minimum Euclidean norm.130) The second and third terms vanish.4. The matrix N that contains the basis for the nullspace is easily found from the singular value decomposition B =UXVH..134) .129) p is an arbitrary vector of length m .T matrix X.T . In order to see this. (3. .1.3. and thus of solving normal equations. we get the vector a of shortest length for p = 0. The QR decomposition is carried out step by step by reflecting vectors at hyperplanes.W E (En. (En is -wHx. WHW (3.x=w Here. In some applications it is useful to exploit the free design parameters in p in order to find a solution a that optimizes an additional criterion.x = X -2 P. .132) decomposed into the orthogonal sum (3. Thus. Then an orthonormal matrix is given by the of N last m . (3.131) Let B have rank T and let the T nonzero singular values be the elements [X]1.4 The Householder Transform Householder transforms allow a simple and numerically robust wayof performing QR decompositions. let us determine the squared norm of a: llallez 2 aHa = [B+x+ NpIH[B+x+ Np] = = x ~ ( B + ) ~ B + z~ N ~ B + Z ~ ( B + ) ~ N P N ~ N P . [X]T. However. in most cases one will use the solution 2i given by the pseudoinverse. 3.

} = -1.=I-for the Householder matrix H . Furthermore we have det{H.= H . we consider a vector X and try to find that vector W for which only the ith component of H.70 Chapter 3.135) the following property of Householdermatricescanbe concluded: H ~ H .4. (3. Discrete Signal Representations - =I W -Pwx W pw X Figure 3.136) = I.& W W H ] (3. is unitary and Hermitian. Hence H .132) we get X at H. Householder reflection.& W W H ] [I . according to (3. It is also known as Householder reflection. H . . 2 WHW W wH (3.138) . because it is the reflection of the hyperplane W I . With P .4.137) In order to make practical use of the Householder transform. where (3.x is non-zero. We use the following approach: w=x+aei. as depicted in Figure 3. = [I .135) From (3.

2% [x + a ei] w H x ei. . 1.142) In order to avoid W M 0 in the case of positive sign in (3.145) with A= an.bll L min v (3.. n >m (3. f ith element (3.3.141) where xi is the ith component of x. (3.140) we finally get (3.140) = = x . By substituting this solution into (3.x we get - z-2-WHO W (3.O. As can easily be verified. . ill E (IFm. .143) =X xi + .142) and obtain W X M Dei for some E R we choose the (3.llxll 1% I ei.4. 0 .141) is satisfied for (3.140) must vanish: 1-2WHX WHW =l-2 llxll + a*xi + axg + a*xi + 11x112 l1 a2 = 0. (1-2G) x-2a In order to achieve that only the ith component of H.x is non-zero. Mathematical Tools 71 e: = [o. .144) Applying the Householder transform to the QR Decomposition.. We consider the problem l l ~ .146) . .0] . . anm .. the expression in parentheses in (3. .139) For H. (3.

where only the upper right-hand triangular matrix of R is non-zero. wi is chosen as wi = zi+llzill ei.. If IJzilJ 0. If one of the values a::) becomes zero. we choose H 2 = I . anm (‘3) (3. the columns must be exchanged. rlrn * * * 0 0 r& an3 (3) .. This means that we have carried out the &R decomposition. . we choose z1 to be the first column of A (3. Discrete Signal Representations and try to tackle the problem by applying the &R decomposition.148) where w1 is chosen as (3. . First.w2wf 2 7 W2 W2 and obtain H2HlA= [ rii 0 r2 1 V22 r3 1 .151) After maximally m steps we get Hrn-**H2HlA=R.149) yields a matrix where only r11 is non-zero in the first column: Then. = .147) Multiplying A with the Householder matrix (3.72 Chapter 3. Note.

.159) in order to obtain X’ = [r.$) 1 (3.Xi+l. . G We now consider a vector X = [XI.153) we get 2’ = G X = r cos(a .Xj+l. G ~ =I. . .Note that G according to (3. (3.156) = sin(a) = Jm* J W X2 For the rotated vector we have x‘=Gx= [:l=[ 1. rotations constitute a further possibility of performing QR decompositions. . 58) is unitary.160) . X.152) and (3.155) COS(Q!) = X1 @Tg (3. . .3. . xj-1.154) ’ We observe that for $ = Q! a vector X’ is obtained whose second component is zero. We first consider the rotationof a real-valued vector X by an angle $ throughmultiplication of X with anorthonormal rotation matrix G . . . This special rotation matrix is G = [-S c with c = S “1 c (3. for complex-valued vectors we can apply the rotation matrix G=[:* :] (3. xj. . xi.0lT. Mathematical Tools 73 3.157) As can easily be verified. For (3.xi-1.4.158) (3.] T (3.4.$) r sin(a .5 Givens Rotations Besides Householder reflections. .

Discrete Signal Representations and want to achieve a vector 2 ' = [XI.. The rotation is applied to the elements xi and xj only.161) with r= Jm (3. . Xj-l. . . Xj+l. T.] T (3. . + (3. X i .X. . . We have (3. .163) with 4 i G= i . .74 Chapter 3..164) A QR decomposition of a matrixcan becarried out by repeated application of the rotations described above.162) by carrying out a rotation. ..O.Xi+l. .l .

and the discrete Hadamard and Walsh-Hadamard transform. namely the discrete-time Fourier transform. even if z ( n ) is real. the discrete sine transform. We start with the z-transform. thediscrete Fourier transform (DFT). Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 ElectronicISBN 0-470-84183-4 Chapter 4 Examples of Discrete Transforms In this chapter we discuss the most important fixed discrete transforms. Filter Banks. Then we discuss several variants of Fourier series expansions. z is complex. such as the discrete cosine transform.+Transform The z-transform of a discrete-time signal z ( n ) is defined as n=-CQ Note that the time index n is discrete.Signal Analysk: Wavelets.1 The . 4. Time-Frequency Transforms and Applications. 75 . Moreover. whichis a fundamental tool for describing the input/output relationships in linear time-invariant (LTI) systems. Theremainder of this chapter is dedicated to other discrete transforms that are of importance in digital signal processing.1) is equal to the discrete-time Fourier transform. Further note that for z = ejw the z-transform (4. and the Fourier fast transform (FFT). the discrete Hartley transform. whereas z is a continuous parameter.

4). Proof. Examples o f Discrete Transforms In general. convergence of the sum in (4. With z = r ej4 we have I n=-cc M n=-cc Thus.76 Chapter 4.4) The integration has to be carried out counter-clockwise on a closed contour C in the complex plane. The ROC can be determined by finding the values r for which c cc n=-cc Iz(n) < m. which encloses the origin and lies in the region of convergence of X (2).' d z . For most sequences we only have convergence in a certain region of the z-plane. called the region of Convergence (ROC). IX(z)I is finite if z(n)r-" is absolutely summable.1) depends on the sequence z ( n ) and the value z. Proof of (4. We multiply both sides of (4.1) with zk-' and integrate over a closed contour in a counter-clockwise manner: . f 32n (4. 0 The inverse z-transform is given by z(n) = 1c X ( z ) z " .

see e.4) can be found rational X ( z ) . (4.4.l + azz-’ .9) Convolution )W . the most important properties of the z-transform will be briefly recalled. X ( z )= a 0 a l z . [80. on partial fraction expansion. For more detail. ( W = az(n) +py(n) t ) V ( z )= a X ( z )+ P Y ( z ) . . The z-Transform 77 Invoking the Cauchy integral theorem finally yields (4.‘ + + + Methods based on the residue theorem. . ( = z ( n ) * y ( n ) t V ( z )= X ( z ) Y ( 2 ) .10) . and on a direct expansion of X ( z ) into a power series in z-l are known. . . otherwise. for that is for bo b1z-l bzz-’ . Linearity ) . The simplest example is the z-transform of the discrete impulse: S(n) = 1. ) (4.4).g. 0 Reconstruction formulaesimpler than (4. n = O 0. In the following. Proofs which follow directly from the definition equation of the z-transform are omitted. We have S(n) t ) n=-cc For a delayed discrete impulse it follows that c cc S(n)z-n = 1. 1131.1.

we have (4. (4.14) Derivatives (4.78 Chapter 4.n ( - no) c n o X(z). = n=-cc c 00 n z ( n )2-" 3: nx(n).(n) t ) # 0.11) . (4. This includes a = eJwsuch that (4. 0 0 0 0 k=-m m=--00 = X ( z )Y ( z ) . ) t ) X (. = z(n)*S(n-no)and using Scaling/Modulation. Examples o f Discrete Transforms Pro0f.12) X (3. . For any real or complex a an .15) Pro0f.13) Time Inversion X ( .) . ( W This result is obtained by expressing w(n)as ) the convolution property above. Shifting .

If z ( n ) t X ( z ) .*(ir)zk k = C n . (4.16) = (g n=-cc cc .then ) coswn ) . we have (4.17) X(z). ( X X*(+) t ) t ) X ( z ) . Proof.4. 2 .* (4.*(n) H X * ( z * ) . X(z) = [ .p ( i r ) z . ( X t ) 1 [X(ejwz) x(e-jWz)] 2 + (4.1. The z-Transform 79 Conjugation. (4. where X ( z ) = [ X ( z ) ] *l a l = l .(-n) t ) X ( z )= X(z-1). X ( z ) is derived from X ( z ) by complex conjugation on the unit circle. Given the correspondence ) .we have ) .18) (-n)z-" $ X*(-n). it follows that . I That is. ( X sinwn ) t ) j [X(ejwz) x(e-jWz)] .(n) [ P ] * Paraconjugation.20) (4.19) Multiplication with cos o n and sin on. Given the correspondence z ( n ) t X ( z ) . For real signals z(n).21) and .~ ]* 114=1 = C.

80

Chapter 4. Examples o f Discrete Transforms

This follows directly from (4.13) by expressing coswn and sinwn via Euler's formulae cos a = +[ej" + e-j"] and sin a = i [ e j a- e-j"]. Multiplication in the Time Domain. Let z ( n ) and y(n) be real-valued sequences. Then

~ ( n )~ ( n ) = y(n)

t )

V(Z) =

(4.22)

where C is a closed contour that lies within the region of convergence of both X ( z ) and Y ( k ) .

Proof. We insert (4.4) into

V ( z )=
This yields

c
00

.(n) y(n) . K n .

(4.23)

n=-m

Using the same arguments, we may write for complex-valued sequences
) . ( W

=) . ( X

y*(n)

t )

V ( 2 )= -

1

2Tl c

f X ( v ) Y *(5) dv. v' is defined as

(4.25)

4.2

The Discrete-Time Fourier Transform

The discrete-time Fourier transform of a sequence ) . ( X

X(ej') =

C
n=-m

00

e-jwn.

(4.26)

Due to the 2~-periodicity of the complex exponential, X(ej") is periodic: X ( e j w ) = X ( e j ( , + 2.rr)). If ) . ( X is obtained by regularsampling of a

4.2. The Discrete-Time Fourier Transform

81

continuous-time signal z c t ( t ) such that z ( n ) = z,t(nT), where T is the sampling period,W can be understood as the normalized frequency 27r f T . W =

(4.27)

Convolution (4.28)

Multiplication in the Time Domain
z(n)y(n) w
-

27r

1 X ( e j w )* Y ( e j w ) = 27r

S"

X ( e j ( w - V ) ) Y ( e j V ) d v(4.29) .

l 1 (-m,

Reconstruction. If the sequence z(n) is absolutely summable m)), it can be reconstructed from X ( e j w ) via

(X E

z(n) = 27r

S"
-r

X ( e j w ) ejwn dw.

(4.30)

The expression (4.30) is nothing but the inverse z-transform, evaluated on the unit circle. Parseval's Theorem. As in the case of continuous-time signals, the signal energy can be calculated in the time and frequency domains. If a signal z ( n ) is absolutely and square summable ( X E l 1 (-m, m) n &(-m, m)), then (4.31)

Note that the expression (4.26) may be understood as a series expansion of the 27r-periodic spectrum X ( e j " ) , where the values z(n) are thecoefficients of the series expansion.

82

Chapter 4. Examples o f Discrete Transforms

4.3

TheDiscrete FourierTransform (DFT)
N-l

The transform pair of the discrete Fourier transform (DFT) is defined as

X(k) =

c

.(n)w;k
(4.32)

n=O

5
.(n)
=

1 N-l
k=O

c

x(k)wink)

(4.33) Due to the periodicity of the basis functions, the DFT can be seen as the discrete-time Fourier transform of a periodic signal with period N .

(4.35)

the above relationships can also be expressed as

x=wx

t )x

1 =-WHX. N

(4.36)

We see that W is orthogonal, but not orthonormal. The DFT can be normalized as follows: a = +H x t ) x = 9 a , where 1 (4.37)

9 = -WH.

a

(4.38)

The columns of 9,

4.3. The Discrete Fourier Transform (DFT)

83

then form an orthonormal basis as usual. We now briefly recall the most important properties of the DFT. For an elaborate discussion of applications of the DFT in digital signal processing the reader is referred to [113]. Shifting. A circular time shift by p yields
x p ( n ) = ~ ( ( np ) mod N)

+

t
X,(m)
=

N-l

n=O
N-l

c
i=O

x((n

+ p ) mod N)WGm
(4.39)

=

W,""X(m).

Accordingly,

Wp) . ( X

N-l
t )

n=O

c

x ( ~ ) W $ ~= X~ ) m Ic) mod N ) . + ((

+

(4.40)

Multiplication Circular and Convolution. Fourier transform (IDFT) of

For the inverse discrete

we have

(4.41)

That is,a multiplication in the frequency domain meansa circular convolution

84
in the time domain. Accordingly,
N-l

Chapter 4. Examples o f Discrete Transforms

~ ( n ) m(n) t )

n=O

c

X l ( n )X z ( ( m - n )modN).

(4.42)

Complex Conjugation. Conjugation in thetime yields 2*(n) X * ( N - n) and
2* ( N
-

or frequencydomains

(4.43)
(4.44)

n)

X * (n).

Relationship between the DFT and the KLT. The DFT is related to the KLT due to the fact that it diagonalizes any circulant matrix

(4.45)

In order to show the diagonalization effect of the DFT, we consider a linear time-invariant system (FIR filter) with impulse responseh ( n ) ,0 5 n 5 N - 1 which is excited by the periodic signal WN"/fl. The output signal y(n) is given by

(4.46)

4.4. The Fast Fourier Transform

85

Comparing (4.47) with (4.45) and (4.37) yields the relationship

H(k)p, =Hp,,

k=O, 1,.. . , N - l .

(4.48)

Thus, the eigenvalues X k = H ( k ) of H are derived from the DFT of the first column of H . The vectors p, k = 0,1, . . . , N - 1 are the eigenvectors of H . , We have

aHHa= diag{H(O), H(1),. . . ,H ( N

-

l)}.

(4.49)

4.4

The Fast Fourier Transform

For a complex-valued input signal ~ ( nof length N the implementation of ) the DFT matrix W requires N 2 complex multiplications. The idea behind the fast Fourier transform (FFT) is to factorize W into a product of sparse matrices that altogether require a lower implementation cost than the direct DFT. Thus, the FFT is a fast implementation of the DFT rather than a different transform with different properties. Several concepts for the factorization of W have been proposed in the literature. We will mainly focus on the case where the DFT lengthis a power of two. In particular, we will discuss the radix-2 FFTs, the radix-4 FFT, and the split-radix FFT. Section 4.4.5 gives a brief overview of FFT algorithms for cases where N is not a power of two. Wewill only discuss the forward DFT. For the inverse DFT a similar algorithm can be found.

4.4.1

Radix-2 Decimation-in-Time

FFT

Let us consider an N-point DFT where N is a power of two, i.e. N = 2K for some K E N. The first step towards a fast implementation is to decompose the time signal ) . ( X into its even and odd numbered components

51) . . the overall complexity is $ N . In the last step the properties W z k = W.-). . 1 .1.!$2 and W ( 2 n + l ) k = W. instead of N 2 for the direct DFT. . It is easily verified that the decomposition of the DFT results in a reduction of the number of multiplications: the two DFTs require 2 ( N / 2 ) 2 multiplications.N .1..51) for k = 0. . .1 (4.72. . . .. 1 . . the same decomposition principle can be used for the smaller DFTs and the complexity can be further + . Due to the periodicity of the DFT thevalues X ( L ) for Ic = $. . $ . (2n+l)k %-l = c~(n)W$. Since N is considered to be a power of two... n=O + W. . we havedecomposed an N-point DFT into two $-point DFTs and some extra operations for combining the two DFT outputs. nO = - 1. The next step is to write (4. The prefactors W&. . . .54) Thus..1 illustrates the implementation of an N-point DFT via two $point DFTs.1. Thus. Figure 4. . which are used for the combination of the two DFTs.. with -- + 2 -1 (4. .. 2 (4. (4. 5 = O . are called twaddle factors. .(.1 as X ( S ) = U ( S ) WiV(Ic). 2 Ic = -. .. 5 = 0 . n=O 5 = 0 .52) T 1 V(k) = nO = -- c u(n)W$. .1. l .86 Chapter 4 . ..-) 2 + W&V(Ic. . Examples of Discrete Transforms The DFT can be written as N-l n=O %-l = n=O c ZL(n)W?k + 8-1 n O = c . ~ u ( ~ ) W $k ~ 0.. . N =. and the prefactors W& requireanother N multiplications.N .53) N 2 N 2 T 1 V(k) = co(n)W. .)W. were used.1 are given by N N X ( k ) = U(Ic .

-N 4 2 V ( k )= The decompositionprocedurecanbecontinueduntiltwo-point DFTs are reached. l .. .l. ... N N C(k-$+W&9(k--).56) 1 A(L-T)+Wj$'"B(k-N) L = 4 7 . To be explicit.k D@).C = w(2n) = + 2) 2(4n + 1) (4. The two structures in Figure 4. B + N k = O ..2 are equivalent..55) d(n) = v(2n+ 1) = 2(4n + 3 ) .-4 1 (4.. Observing that = W&kwe get for the $-point DFTs U ( k ) and V ( L ) U ( L )= and { A ( k ) W&k ( k ) .-N 4 2 1 (4.4.... 4 + k=O... It turns out that all stages of the FFT are composed of so-called butterfly graphs as shown in Figure 4. The Fast Fourier Transform 87 reduced.57) N N k = .". we decompose the sequences u ( n ) and w(n) into their even and odd numbered parts: a(n) = u(2n) = x(4n) b ( n ) = u(2n+ 1) = x(4n )( .4.2. I C(L) W.. .l . 4 ..

1in binary form. we split the input sequence into the first .The number of stages is log. + Thecomputationalcomplexity of theFFT is as follows. one can proceed to the next stage. n = 3 is represented by [011] when an 8-point FFT is considered. However.3. N additions. but the input values appear in permuted order. In order to derive this algorithm. only a memory of size N 1 is required for computing an N-point FFT.N . and since the 4-point DFTs involve multiplications with 1. 4.2. since the 2-point DFTs do not require multiplications. 4 6 ) has to be connected to input node 3. The complete flow graph for an 8-point FFT basedon the butterfly in Figure4. one represents the numbers 0 . After this has been done for all butterflies of a given processing stage. This is the case for all N . Equivalent butterfly graphs.4. This order can bederived from the natural one as follows.2(b) is depicted in Figure 4. Thus. Thus. For example. . N .j only.2(b) saves us one complex multiplication. This yields a total number of i N log. This yields [l101 in reversed order. Since the butterfly operations within each stage of the FFT are independent of one another. First. and the decimal equivalent is 6. N complex multiplications and N log. the output values appear in their natural order. Each stage of the FFT requires N/2 complex multiplications and N additions. Examples of Discrete Transforms (4 01) Figure 4. -l.j.2 Radix-2 Decimation-in-Frequency FFT A second variantof the radix-2 FFT is the decimation-in-frequency algorithm. ..88 Chapter 4. Then the order bits is reversed and of the decimal equivalent is taken. but theone in Figure 4. N . The order of the input values is known as the bit reversed order. . and . the computationof the FFT can be carried out in place. 1 . This means that the pair of output values of a butterfly is written over the input. the actual number of full complex multiplications is even lower than iNlog. As we see.

60) n=O X(2k + 1) = N-l c [U(. ( v +N/2) In (4. and second halves and write the DFT as N-l n=O N/2-l (4. ( X = z(n ) . For the even andodd X(2k)= and c [U(.58) we haveused the fact that W’ :2 numbered DFT points we get N-l = -1. Transform 89 1 -1 -1 Figure 4.4. (4.61) n=O Because of W F k = W$2. ).59) where U(. ) ] WZh (4. 2 (4.Fourier The Fast 4.3. the even numbered DFT points X ( 2 k ) turn out to be the DFT of the half-length sequence U(. N n = 0 .l .) + v(. Flow graph for an 8-point decimation-in-time FFT. ) The odd numbered + . .)] W$) .58) n=O Nf2-1 = n=O c [U(. 1 ) .) + (-l)”(.)] -v ( ..) W. .) W ( .) = ) . W Z k .

. Flow graph for an 8-point decimation-in-frequency FFT.w(n)] W. but where the output values appear in bit reversed order. + 4.4. Examples o f Discrete Transforms 42) d3) 44) 45) 46) 47) -1 -1 -1 Figure 4. Using the principle repeatedly resultsin an FFT algorithm where the input values appear in their natural order. as with the decimation-in-time algorithm.3 Radix-4 FFT The radix-4 decimation-in-frequency FFT is derived by writing the DFT as N-l X(k) = n=O c . The comparison of Figures 4. Figure 4. Thecomplexity is the same as for the decimationin-time FFT.k . DFT points X(2k 1) are the DFT of [u(n). Thus.4 shows the complete flow graph of the decimation-infrequency FFT for the case N = 8.3 and 4.(n)w.90 40) 41) Chapter 4. the N-point DFTis decomposed into two N/2point DFTs.4.4 shows that the two graphs can be viewed as transposedversions of one another.

For this.66) + in 4 + j [ z ( n -) +N .asplit-radixbutterfly can be drawn as shown in Figure 4. and the others require one complex multiplication per point. requires the lowest number of operations of all currently known FFT algorithms.5. ) +z(n + .e-)WG" 1 w$4. 4. whichis amixture of the radix-2 and radix-4 algorithm. One of these four DFTs requires no multiplication at all. (4.z(n N X ( . (4. we have replaced the computation of an N-point DFT by four N/4point DFTs. ) .. It is also easily programmed on a computer. because fewerfull complex multiplications occur. and it requires only half as many stages as a radix-2 one.4.Thus. the decomposition principle can be used repeatedly. and the radix4 approach isused to compute two length-(N/4) subsequences of the odd numbered frequencies. X(k) is split into the following three subsets: N/2-1 X(2k) = n=O X(4k + 1) N/4-1 = n=O c c[ c[ [ X ( .64) [) X ( .z(n N + -)] 2 X(4k + 3) N/4-1 = [) X ( .z(n n=O N + -)] 2 . Compared to the radix2 FFT this means 3 X (N/4) instead of N/2 multiplications for the twiddle factors. Thesplit-radix + + 34)1] W$'W$. the overall number of multiplications is lower for the radix-4 case. .65) and (4.z ( n + y)] (4. The radix-2 approach is used to compute the even numbered frequencies.64).z ( n $)] The terms [ are the natural pairs to the terms in (4. As with the previous approaches. However. . (4. It turns out that the splitradix approach requires less multiplications than a pure radix-2 or radix4 FFT.)l n7 W$. the radix-4 algorithm requires only N/4-point DFTs.63) Thus. Therefore.66) and [z(n ): .4. The FastFourier Transform 91 Splitting X(k) into four subsequences X(4k + m) yields 4 N z ( n + .4.4 Split-Radix FFT Thesplit-radix FFT [46].

1381. the approach can be iterated. . 1. The efficiency of the Good-Thomas FFT . The basic idea dates back to papers by Good [64] and Thomas [143]. Note that the radix-2 approach occurs as a special case where P = 2 and Q = N/2. where P and Q are integers.1 and m = 0. The inner sum in the second line of (4. . As can easily be verified. Butterfly for a split-radix FFT. 1421. + (4. plus the twiddle factors in the middle of the second line in (4. Q . and the outer sum a Q-point is DFT. 4. Examples o f Discrete Transforms 0 use for X(4k+l) 0 use for X(4k+3) Figure 4. 20.67) turns out to bea P-point DFT. The DFT can then be written as P-1 Q-l X ( k P + m) = cc (iQ+j)(kP+m) z(iQ j ) W. If P and/or Q are composite themselves. Thus. The best known one is the Cooley-Tukey F F T [31]. The algorithm has been further developed in [88. which requires that the DFT-length is a composite number N = P&.67) j=O i=O for k = 0. .92 Chapter 4.. conceptcanbe generalized to other radices [152]. 164. and special forms are available for real and real-symmetric data [45.5 Further FFT Algorithms There area number of algorithms available for the case where the DFT length is not necessarily a power of two. . . and the complexity can be further reduced..1.P .4. the N-point DFT is decomposed into P Q-point and Q P-point DFTs. If the DFT-length canbe factored into N = P Q where P and Q are relatively prime (have no common divisor other than 1) a powerful algorithm known as the Good-Thomas F F T can be used.67).5. the complexity islower than for the direct N-point DFT. 1 ..

. and the transform can be implemented as a true two-dimensional transform. and W. to the power-of-two approaches. 101. if not superior.k = W.g. $ . when designing an algorithm where the DFT is involved.1. Thus.'.68) n O = The sum on the right side can be identified as a circular convolution of the sequences x(n)Wp$. (4.5. The mapping is based on the Chinese remainder theorem [g].. c [x(n)W&] -(k-n)' W.71) .. that is X ( n ) = W..70) DCT-11: c:'(. +)T k .1. (4. see e. [ x(n)W& * W'$]. n =0. Discrete Cosine 93 results from the fact that for relatively prime P and Q the twiddle factors (they arealways present in the Cooley-Tukey FFT) can beavoided..) = &k y COS (k ( n + ) . .. (4. because for almost any length an appropriate FFT algorithm can be found. Powerful FFT algorithms are most often associated with signal lengths that are powers of two.. The input data can be arranged in a two-dimensional array. one should not be bound to certain block lengths. In order to give an idea of how this is done.69) Efficiency is achieved by implementing the circular convolution via fast convolution based on the FFT.. N . .)n = 0 .Transforms 4. 4. 1 .5 Discrete Cosine Transforms We distinguish the following four types of discrete cosine transforms (DCTs) [122]: DCT-I: c:(")=&% c o s ( k . FFTs where N is a prime number can be realized via circular convolution [120. N . (4. we follow the approach in [l01 and write the DFT as N-l X ( k )= c x(n)W... However. prime factor algorithms such as the Winograd FFT [l641 are often competitive. [117].

1 .l . .74) 1 otherwise.(4. (4. N . . . .72) cLV(n) = p cos ( ( 5 + t . which is given by lSee Section 5. . 5.70) ..3 for the definition of an AR process. .(4. let us consider the forward and inverse DCT-11: N-l and N-l Especially the DCT-I1 is of major importance in signal coding because it is close to the KLT for first-order autoregressive processes with a correlation coefficient that is close to one.70) . n = 0 . .' To illustrate this. Tj = (4. 1 .94 DCT-111: Chapter 4.73) The constants yj in (4... .lT. we consider the inverse of the correlation matrix of an A R ( 1 ) process.n = 0 .72) are given by for j = 0 or j = N .. (4.ck(l). k . Examples o f Discrete Transforms cL"(n) = DCT-IV: 6 N yn cos (Ic ( '2)"") . In order to point out clearly how (4. N . The coefficients ck(n) are the elements of the orthonormal basis vectors ck(n) = [ck(o).1. cN+ i ) " ) .73) are to be understood. .

Such a concentration of signal energy in a few coefficients is the key to efficient compression. the representation Y is quantized and coded. the two-dimensional cosine transform is used [79. Instead of the original X .( l .. Application in Image Coding. where X N ~ is = N N such a signal block and U is the DCT-I1 transform matrix whose columns contain the basis vectors of the DCT-11. From the quantized representation Y ’ = Q ( Y ) an approximation of the original is finally reconstructed.g. 108. Discrete CosineTransforms 95 (4. If we were to simply transmit the top left sub-image and neglect the others.This is the case for most images.. 157.78) We see that Q approaches R.5.: for p + 1.a) -a -a 1 -a - -a Q= .157. MPEG [79. This operation can be written as YNXN UT X N ~ U . Compared to the KLT. are equal to those of R. we already could achieve drastic compression. the two-dimensional signal is decomposed into non-overlapping blocks. Since the eigenvectors of R.6gives an example.: the DCT-I1 approaches the KLT for p + 1.Figure 4.4.6 we see that most of the energy of the transformedsignal is concentrated in the top left sub-image. First.77) with P = p / ( l + p’)). . Each of these blocks is then transformed separately. which explains why most image coding standards (e.108. The basis vectors of the DCT-I1 are the eigenvectors of tridiagonal symmetric matricesof the form . 561) are based on the DCT-11. the DCT-I1 has the advantage that fast implementations based on the FFT algorithm exist [122]. JPEG. 1 -a -a (1-a)- (4.56]. In most standards for transform coding of images.In Figure 4. This means that the DCT-I1 has good decorrelation properties when the process which is to be transformed has high correlation ( p + 1). while the reconstructedsignal would still be relatively close to the original.

80) DST-111: N .. (a) original. divided into N (b) transformed image after rearranging the pixels.l . 2 . X N blocks. Examples Discrete I Figure 4.82) The constants ^/j in (4. .l . n = 0 .. (4... n = 1 . (4.81) are for j = o or j = N . (4.1. 4. N . (4.. n = 0.81) DST-IV: siv(.79) DST-11: N .) = J" sin ( ( k + f ) (Nn + i ) a ) . . 1 .l .(4...83) 1 otherwise.6. .. k . Transform coding of images.l . N .96 Transforms of Chapter 4.6 DiscreteSineTransforms The discrete sine transforms (DSTs) are classified as follows [122]: DST-I: s : ( n ) = g sin($).. k ..79) .. N .. 1 . ^/j = (4.. n = 0 . N . k . . N * k .

The forward and inverse discrete Hartley transform pair is given by N-l XH(IC) = n=O 27rnlc C z ( n )cas N (4.84) . 14.) (4.For example. 1601 and Bracewell [13. the fast Hartley transform (FHT) can be computed via the FFT. the discrete Hartley transform (DHT). and vice versa. the forward and the inverse DST-I1 are given by N-l x. This results in fast algorithms that are closely related to the FFT. Thus.3 received little attention until as its discrete version. the DHT can be implemented efficiently through a factorization of the transform matrix. was introduced in the early 1980s by Wang [158. 151.7 The DiscreteHartleyTransform The Hartley transform discussed in Section 2. 14.'(~c) sf'cn) k=O yk+1 - gyx.IT) y k + l g C x(n) sin ( ( k + 1)Nn+ ( . 4. . the FFT can be implemented via the FHT [161.85) The DST-I1 is related to the KLT by the fact that the KLT for an AR(1) process with correlation coefficient y + -1 approaches the DST-11. in [l391 a split-radix approach for the FHT has been proposed. 159.7. The Discrete Hartley Transform 97 To be explicit.86) 1 N-l z(n) = C xH(L) k=O 21rnlc cas N 7 . n=O and N-l z(n) = C x.4.~(lc) k=O sin ( ( k + 1)Nn + ( t.and in fact. Like other discrete transforms such as the DFT or the DCT.'(~c) = = C z ( n )sf'cn) N-l n=O (4. the DST-I1 has good compaction properties processes with negative correlation for of adjacent samples. 1391.

k)th frequency are the same. As with the DFT. From (4. The relationships between the DHT and the DFT are easily derived.89) Theproperties of theDHT can easily be derived from the definition (4. Examples o f Discrete Transforms where cas 4 = cos 4 sin 4.n). we get + + ’ + L (4. where a real-valued signal is transformed into a complex spectrum. which means that the same computer program or hardware can be usedfor the forward and inverse transform. Note that apart from the prefactor 1/N the DHT is self-inverse.4 ) (4.87) 2 2 and the periodicity in N .98 Chapter 4. Time Inversion. The kth and the ( N . most of them are very similar to the properties of the Fourier transform.n) t X H ( N . the sequence X H ( S ) is periodic with period N .S ( X ( 5 ) ) . ) Shifting.88) where X ( k ) denotes the DFT. (4. The signal z ( t )is considered to be real-valued.86) we see that z ( N . This is not the case for the DFT. The DHT can be expressed in terms of the DFT as X H ( 5 ) = ?J?{X(k)).86).90) + p ) mod N ) . The basis function with the highest frequency then occurs for k = N / 2 . Using the fact that 1-j e j 4 = -cas 4 -cas ( . so that also the transform is real-valued. We may interpret the basis sequences cas (27rnklN)as sample values of the basis functions cas w k t with wk = 27rk/N. The proofs are essentially the same as for the continuous-time case and are omitted here. We will briefly discuss the most important ones. Like in the continuous-time case. A circular time shift by p yields z( (n (4.

. N . For example an expression ZH( k ) = XH(L)* YH .For both the FFT and the FHT the complexity is Nlog. The DHT is somehow conceptually simpler than the DFTif the input signal is real. The question of whether the DFT or the DHT should be used in an application very much depends on the application itself. The correspondence for products z(n)y(n)is where the convolutions have to be carried out in a circular manner. For the forward and inverse FFT of a real signal. so that only one software routine or hardware device is needed for the forward and inverse FHT. but all operations can be carried out with the FFT and the FHT with the same complexity. An advantage of the DHT is the fact that the DHT is self-inverse.4. and one fast transform can be used in order to implement the other transform in an efficientway. The Discrete Hartley Transform 99 Circular Convolution. fast algorithms exist for both transforms.92) Multiplication. As mentioned earlier.k ) means ( Remarks. two different routines or devices are required.7. The correspondence for a circular convolution of two time sequences ~ ( n ) y(n) is and (4.

where X denotes the signal. (4.8 The Hadamardand Walsh-Hadamard Transforms The basis vectors of the discrete Hadamard and the discrete Walsh-Hadamard transforms consist of the values fa. andH the transform matrix of the Hadamard transform.this yields an order of the basis vectors with respect to their spectral properties. Somehow. k E IN can be calculated recursively [133]: (4. Examples o f Discrete Transforms 4. The transform matrix of the 2x2-Hadamard transform is given by (4.100 Chapter 4. y the representation. 2There exist Hadamard matrices whose dimension is not a power of two.2.6. Both transforms are unitary.96) (4.just like the Walsh functions discussed in Section 3.Basically they differ only in the order of the basis vectors.95) = Hy. . We have y X = Hx.97) From this. all transform matrices H ( n )of size2 n = 2 k . H is symmetric and self-inverse: H T = H = H-l.98) The Walsh-Hadamardtransform is obtained by takingtheHadamard transform and rearranging the basis vectors according to the number of zero crossings [66].

Filter Banks. the KLT is a signal expansion with uncorrelated coefficients.1 The Continuous-Time Karhunen-Lo'eve Transform Among all linear transforms. The transform can be formulated for continuous-time and discrete-time processes. respectively. Consider a real-valued continuous-time random process z ( t ) . Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Chapter 5 Transforms and Filters for Stochastic Processes In this chapter. [l49 ]. These properties make it interesting for many signal processing applications such as coding and pattern recognition. we sketch the continuous-time case [81]. for continuous and discrete-time signals. the Karhunen-Lo bve transform (KLT) is the one which best approximates a stochastic process in the least squares sense. We start with transforms that have optimal approximation properties. 101 . we consider the optimal processing of random signals.Signal Analysis: Wavelets. Then we discuss the relationships between discrete transforms. and optimal linear filters.The discrete-time case will be discussed in the next section in greater detail. a < t < b. optimal linear estimators. Time-Frequency Transforms and Applications. 5. in the least-squares sense. Furthermore. In this section.

a weaker condition is formulated. Transforms Processes Filters and Stochastic for We may not assume that every sample function of the random process lies in Lz(a.b) and can be represented exactly via a series expansion.3) is satisfied if Comparing (5. = Pi) l b Z(t) Pi(t) dt (5. For this. we require that the coefficients 2 i = (z.m=limit in the mean[38]. U) = E { 4 t ) 4) . .4) We see that (5. This can be expressed as = ! xj &j. 2 . b ll.5) with the orthonormality relation realize that Sij = S cpi(t)c p j ( t ) dt. i = 1 .} has to be derived from the properties of the stochastic process.(t.. . Therefore.2) of the series expansion are uncorrelated.3) function T. .) (5.102 Chapter 5. is the autocorrelation * The kernel of the integralrepresentationin(5.i. we . . which states that we are looking for a series expansion that represents the stochastic process in the mean:’ N The “unknown” orthonormal basis {vi@).

9) (5. the process can be written x=ua. the solutions c p j ( t ) . 5. . M with finite M . The values X j .thistransform is an enormous help. 2 . The Discrete Karhunen-Lobe Transform 103 must hold in order to satisfy (5.] T (5. . Further b). can be translated into a zero-mean process X by x=z-m2. .. . if SJT. that is.1) only for i = 1 .. solveestimation orrecognition problems for vectors with uncorrelated components and then interpret the results for the continuous-time case. Thus.a. theoretically. Signals can be approximated by carrying out the summation in (5..b).(t.6) form the desired orthonormal basis.5). However. since any process 2: with mean m. . The restriction to zero-mean processes means no loss of generality. Inpractice. These functions are also called eigenfunctions of the integral operator in (5. as (5. j = 1 . . The mean approximation error produced thereby is the sumof those eigenvalues X j whose corresponding eigenfunctions are not used for the representation. we obtain an approximation with minimal mean square error if those eigenfunctions are used which correspond to the largest eigenvalues.10) .. that is. properties and particular solutions of the integral equation are for instance discussed in [149]. . 2 .2 The DiscreteKarhunen-LoheTransform We consider a real-valued zero-mean random process X = [?l. j = 1 .. . are the eigenvalues. solving anintegralequation represents a majorproblem.then d t du E the eigenfunctions form a complete orthonormal basis for L ~ ( u . U.(~.. where the representation a = [ a l . of theintegralequation (5.5.}.6). ... . . 2 .8) With an orthonormal basis U = ( ~ 1 . . We can describe stochastic processes by means of uncorrelated coefficients. without solving the integral equation. Thus.U ) is positive definite. . Therefore the continuous-time KLT is of minor interest with regard to practical applications.2. If the kernel ~. Xn XEIR.U)Z(~)Z(U) > 0 for all ~ ( t )La(a.

i . . j = 1 . u = Xjuj.n are solutions to the eigenvalue problem j j R. . n. . . . j = 1 . we derive the KLT by demanding uncorrelated coefficients: E {aiaj} = X j Sij. . . Tkansforms and Filters Processes Stochastic for a = uT X. . condition (5. is a covariance matrix.. j = 1 . we obtain the orthonormal basis of the Karhunen-LoBve transform.12) we obtain E {urx x T u j } = X j Wit h Sij. .12) becomes For complex-valued processes X E (En7 . . . . u =Xj j Si. . 3.12) The scalars X j . From (5.. . Only real eigenvalues X i exist.n.15) is satisfied if the vectors u . .. i. Thus. (5.9) and (5. .'} (5. (5. their eigenvectors are linearly independent and can be chosen to be orthogonal to one another. .n. for all eigenvalues we have Xi 2 0. . j = 1 . that is. . (5..n are unknown real numbers with X j 2 0.15) We observe that because of uTuj = S i j . i . . Eigenvectors that belong to different eigenvalues are orthogonal to one another.16) Since R. . the eigenvalue problem has the following properties: 1. = 1.14) UT R. = E this can be written as {. If multiple eigenvalues occur.104 is given by Chapter 5. (5. we see that n orthogonal eigenvectors always exist.11) As for the continuous-time case. . Complex-Valued Processes. j j = 1 . . equation (5.n. (5.X. 4. . .13) R. A covariance matrix is positive definite or positive semidefinite. 2. By normalizing the eigenvectors.

... which belong to the largest eigenvectors leads to a minimal error. . um. .] is unitary. . 9 Best Approximation Property of the KLT.. ..U. . . . . with the covariance matrix uj = X j u j .19) = 5 xi. From the uncorrelatedness of the complex coefficients we cannot conclude that their real andimaginary partsare also uncorrelated. (5. The Discrete Karhunen-Lobe Transform 105 This yields the eigenvalue problem R.n R... In order to show that the KLT indeed yields the smallest possible error among all orthonormal linear transforms.5. i=m+l It becomes obvious that an approximation with thoseeigenvectors u1. . The eigenvectors are orthogonal to one another such that U = [ u l . j = 1 .. j = 1.R.. the variances of the coefficients: E { Jail2} x i . we look at the maximization of C z l E { J a i l }under the condition J J u i=J1. . With ai = U ~ this means J Z . . (5. 2 X From (5. .12) we get for . Again.18) (5. = i = 1 . . . i. = E {zz"} . E {!J%{ai} { a j } }= 0. . We henceforth assume that the eigenvalues are sorted such that X 1 2 .n is not implied. .17) For the mean-square error of an approximation D= we obtain Cai i=l m ui.2. that is. the eigenvalues are real and non-negative. m < n.

. this means that the KLT leads to random variables with a minimal geometric mean of the variances. i Figure 5. = o2 I we have X1=X2= .21) which is nothing but the eigenvalue problem (5. is the covariance matrix of a white noise process with R. Minimal Geometric Mean Property of the KLT. .19) shows that a white noise process can be optimally approximated with any orthonormal basis. zZIT. The KLT of White Noise Processes. (5.1.1 gives a geometric interpretation of the properties of the KLT.. Since the KLT leads to a diagonal covariance matrix of the representation.. Equation (5. j = 1.. the KLT is not unique in this case. Thus.= X n = 0 2 . For the special case that R.106 Chapter 5. . Tkansforms and Filters for Stochastic Processes Figure 5. again. optimal properties in signal coding can be concluded [76]. For any positive definite matrix X = X i j . where y are Lagrange multipliers.16) with y = Xi. We see that u1 points towards the largest deviation from the center of gravity m.22) Equality is given if X is diagonal. i. . . Setting the gradient to zero yields i R X X U i = yiui. Contour lines of the pdf of a process z = [zl.. From this.n the following inequality holds [7]: (5.

: we obtain R. The goal is to discriminate different types of vehicle (car. In the following.27) Application Example.: = U K I U H .2. etc..28) The mean values can be estimated by . The realizations are denoted as izl. truck. (5. zero-mean processes are generated: (5.23) A = UHR. The signals considered in this example aretakenfrom inductive loops embedded in the pavement of a highway in order to measure the change of inductivity while vehicles pass over them. Assuming that all eigenvalues are larger than zero.29) . and amplitude) we obtain the measured signals shown in Figure 5. for R..25) Finally. A-1 is given by (5. N (5.. .U. We obtain (5. = U A U H .15) as [ A1 . which are typical examples of the two classes. The stochastic processes considered are z1 (car) and z2 (truck).24) R. 01. Observing U H = U-'.). The Discrete Karhunen-Lobe Transform 107 Relationships between Covariance Matrices. bus. In a first step.2. (5. . we will consider the two groups car and truck. After appropriate pre-processing (normalization of speed. In pattern recognition it is important to classify signals by means of a fewconcise features. In the following we will briefly list some relationships between covariance matrices. N . With A=E{aaH}= we can write (5..= 1 .5. length. i z ~ i.

p1 and p 2 . can be estimated as R.Original 0.2 '0 0.6 0. where i x l and ix2 are realizations of the zero-mean processes x1 and respectively.2 0. (b) two sample functions and their approximations.6 0. (a) typical signal contours. .8 1 1 0.2 0. Tkansforms and Filters for Stochastic Processes 1 0. The first ten eigenvalues computed from a training set are: X1 X2 X3 X5 X4 X6 X7 X8 X9 X10 968 2551 3139 We see that by using only a few eigenvectors a good approximation can be expected. . P1 a= 1 N ixl ixT P2 + -C N+1. Examples of sample functions.. Approximation 0.31) can be defined.8 1 0... . 1 a= N ix2 ix..Original ..4 0.8 1 t - t+ Figure 5. (5.6 c 0.6 0. The covariance matrix R. . = E { x x ~ M } - C N+1.4 .4 0..4 0.2. and - N (5.4 0.108 Chapter 5..8 t 0. .2 1 '0 .6 8 0. Approximation 0. .6 8 0.8 0.Figure 5.4 0.2 0..2 0 .32) 22.2 shows two signals andtheir . . To give an example.30) Observing the a priori probabilities of the two classes. a process 2 =PlZl+ P222 (5.

u3. 44. 5. ( X = W(. = E { z ( n ) }= 0 . (5.u2. ( X ) . ( X =c p i i=O W(. an AR(p) process ) . 167.5.) +p X ( .3. 581.38) For determining the variance of the process X ( . it shows how a high proportion of information about a process can be stored within a features. However. (5. . see [59.3 The KLT of Real-Valued AR(1) Processes An autoregressiwe process of order p (AR(p) process) is generated by exciting a recursive filter of order p with a zero-mean. From (5.1) (5.33) with the basis {ul.~4}. few For more details on classification algorithms and further transforms feature for extraction. The KLT of Real-Valued AR(1) Processes 109 approximations (5. (5.37) we use the properties (5. It is also known as a first-order Markow process.i).35) where W(.36) we obtain by recursion: ) . ) . stationary white noise process.34) Thus.) + C p ( i ) X.i). The AR(1) process with difference equation ) . . the optimality andusefulness of extracted featuresfor discrimination is highly dependent on the algorithm that is used to carry out the discrimination. In general. ( X is described by the difference equation = W(. the feature extraction method described in this example is not meant to be optimal for all applications.) is white noise. The filter has the system function H ( z )= 1- 1 i=l c p ( i ) z-i P V > P ( P ) # 0.36) is often used as a simple model. ( i=l . Thus. mw = E { w ( n ) }= 0 + m.

N . Tkansforms and Filters Processes Stochastic for ?-..1) shall be considered.(N .p2’ For the autocorrelation sequence we obtain i=O We see thatthe autocorrelationsequence is infinitely long.... The eigenvalues are 1 Xk = I 1.. the eigenvalues Xk. N . However. we get (5..110 and Chapter 5.. < 1. Supposing IpI i=O - U2 1.... . For real signals and even N .42) The eigenvectors of R.2 p cos(ak) + p2 ’ k=O.(m) = E {w(n)w(n + m)} = 02smo. = 1 . henceforth only the values rzz(-N l). Ic = 0. the covariance matrix of the AR(1) process is a Toeplitz matrix.1 and the eigenvectors were analytically derived by Ray and Driver [123]. It is given by + R. .p2 o2 (5.1 .43) .. form the basis of the KLT.. (5. T. Because of the stationarity of the input process...39) where SmO is the Kronecker delta.

N . .44) The components of the eigenvectors u k . . . For A and X we have (5. . The covariance matrix can be decomposed as follows (KLT): R.4. . (5. are mainly applied in signal detection and pattern recognition.1 are given by 5.5..48) is preserved. the coefficients of the representation should not only be uncorrelated (as for the KLT). they should also have the same variance. property (5. because they lead t o a convenient process representation with additive white noise. That is.1 denotes the real positive roots of tan(Nak) = - (1 .46) We wish t o find a linear transform T which yields an equivalent process ii=Tn wit h (5.47) E {iiiiH} = E { T n n H T H= TR.p’) sin(ak) cos(ak) .49) . } (5. 5 = 0 .2p p COS(Qk).4 Whitening Transforms In this section we are concerned with the problem of transforming a colored noise process into a white noise process. . Such transforms. known as whitening transforms. = U A U H = U X E H U H .TH = I . .48) We already see that the transform cannot be unique since by multiplying an already computed matrix T with an arbitrary unitary matrix. .. + (5. Whitening Transforms 111 where ak.N . Let n be a process with covariance matrix Rnn = E { n n H } # a21. k = 0 .

T ~= L . The whitening transform is (5. The whitening transforms transfer (5. (5.112 Chapter 5.48): (5..50) (5. we can apply the Cholesky decomposition R. (5. For the covariance matrix we again have (5.~ L L H L H .51) This can easily be verified by substituting (5.54) E {+inH)T R . = (5.56) where S is a known signal and n is an additive colored noise processes.'= I .55) In signal analysis.57) with F = Tr.50) into (5. . Tkansforms and Filters Processes Stochastic for Possible transforms are T = zP1UH or T =U T I U H . (5.53) T = L-l.58) ii = Tn. one often encounters signals of the form r=s+n.52) Alternatively. I = Ts.56) into an equivalent model F=I+k (5. where n is a white noise process of variance IS:1. where L is a lower triangular matrix. = L L H . = .

Wewill focus on the case where the estimators are linear. Linear Estimation 113 5. in general. (5.59) where r is our observation. (5. and n is a noise process.1 Least-Squares Estimation We consider the model r=Sa+n.5.5. (5.5 Linear Estimation Inestimationthe goal is to determine a set of parametersas precisely as possible from noisy observations. Therefore they are optimal only under the linearity constraint. a is the parameter vector in question. 5.61) If we assume zero-mean noise n . linear estimators constitutethe globally optimal solution as far as Gaussian processes are concerned [ 1491. The linear estimation approach is given by h(. as described in Chapter 3. Because of the additive noise.62) . The requirement to have an unbiased estimate can be written as E { u ( r ) l a }= a. only moments up to the second order are taken into account. the estimates for the parameters are computed as linear combinations of the observations. the estimates u ( r ) l aagain form a random process. matrix A must satisfy A S = I (5. Matrix S can be understood as a basis matrix that relates the parameters to the clean observation S a . However. This problem is closely related to the problem of computing the coefficients of a series expansion of a signal.60) where a is understood as an arbitrarynon-random parameter vector.5. that is. non-linear estimators with better properties may be found. Linear methods do not require precise knowledge of the noise statistics. and.) = A r .

we get a(r) = [SHGS]-lSHGr (5. as can easily be verified. However.66) is the correlation matrix of the noise. we speak of a least-squares estimator.AsS]-~S~R.A r.64) where an arbitrary weighting matrix G may be involved in the definition of the inner product that induces the norm in (5.63) A S a The generalized least-squares estimator is derived from the criterion = mm. (5.AS]-'SHR. If we choose G = I .5. we speak of a generalized least-squares estimator.. yields an unbiased estimatorwith minimal variance. which is known as the best linear unbiased estimator (BLUE). the requirement (5. For weighting matrices G # I .67) u ( r )= [s~R. This is seen from E{h(r)la} = E { A rla} = A E{rla} = AE{Sa+n} = (5. The estimate is given by (5.68) . (5. = E { n n H } (5. Here the observation r is considered as a single realization of the stochastic process r .64). choosing G = R.A. 5. Tkansforms and Filters Processes Stochastic for in order to ensure unbiased estimates.114 Chapter 5. where R. CY = &(r) ! .2 The Best Linear Unbiased Estimator (BLUE) As will be shown below.65) to have an unbiased estimator is satisfied for arbitrary weighting matrices.:. then is A = [SHR. The estimator.95). Making use of the fact that orthogonal projections yield a minimal approximation error. the approach leaves open the question of how a suitable G is found. Assuming that [SHGS]-l exists.65) according to (3.

= [SHRiAS]-'.69) and theoptimality AS = I we have of (5. R.69) Proof of (5.. observe that with h ( r )--la = = A S a+A n-a An. .a.73) will be considered. (5.. (5.ARn. Linear Estimation 115 The variances of the individual estimates can be found on the main diagonal of the covariance matrix of the error e = u ( r ).71) In order to see whether A according to (5.R.67). (5.75) For the covariance matrix of the error E(r) = C(r).70) Thus. = = AE { n n H } A H AR.72) with (5. First.5.74) DS=O (null matrix).AS]-'SHR. (5.AS]-' [SHR.76) = ARnnAH + ARnnDH+ DRnnAH+ DRnnDH.a we obtain = = AR.A-H [ A D]Rnn[A DIH + + (5.A~ = = [SHR. given by R.67) is optimal. an estimation (5. Because of A S = I this means (5.5..AS]-'.AS[SHR. The urlbiasedness constraint requires that As==..

83) Again.3 MinimumMeanSquareError Estimation The advantage of the linear estimators considered in the previous section is their unbiasedness. This can beseen by using the Cholesky decomposition R.l r ~ . The equivalent estimator then is .78) We see that Rc2 is the sum of two non-negative definite expressions so that minimal main diagonal elementsof Rgc are yielded for D = 0 and thus for A according to (5.AS]-l v 0 (5. (5.nDH)H DRn. can be interpreted as an implicit whitening of the noise. (5. Tkansforms and Filters for Stochastic Processes (AR.76) reduces to R 2 = ARn. If we dispensewith this property.84) .80) (5.68) reduces to S(r) = [ s~s]-~s~~.116 With Chapter 5. Wewill start the discussion on the assumptions E { r } = 0. the linear estimator is described by a matrix A: S(r) = A r . (5.HH .estimateswith smaller mean square error may be found.77) = o (5. (5.AH + DRnnDH.~ s . 2 (5. 3 = L .5.79) Otherwise the weighting with G = R. E { a } = 0.AH = = DRnnR$S[SHRiAS]-' = DSISHR. (5.fi = L-ln.67). 0 In the case of a white noise process n .U(?) = [ S ~ 1 .82) 5.. = LLH and and by rewriting the model as F=Sa+fi. where (5.81) F = L-'r. I The transformedprocess n is a white noise process..

With A according to (5..88) is dependenton A . observe that R.AR..86) can be extended by R.91) .: R. R: = E { r a H } . RTaRF:Ra.. R. i.84) into (5.U ] E{aaH}-E{UaH}-E{aUH}+E{UUH}. Linear Estimation 117 Here.. = R. In Chapter 3 we saw that approximations D of signals z are obtained with minimal error if the error D . for A = R. { (5... [AH. The matrix A which yields minimal main diagonal elements of the correlation matrix of the estimation error e = a . E { a r H }= A E { r r H }. we have a minimum of the diagonal elements of R. (5. . has positive diagonal elements. A similar relationship holds between parameter vectors a and their MMSE estimates.(5.85) Substituting (5.:.. R..85) yields R.5.e.. R.: R.RTaR..... R.86) R. The correlation matrix of the estimation error is then given by Re. (5.87) = E{mH}. .R. Since only the first term on the right-hand side of (5..89) = R a a ... = A R.RF:] R.AH + AR. = = E [ U .AH with (5. but the inner relationship between r and a need not be known however.+ Raa. R.RF:Ra.z is orthogonal to D. = [ A .U is called the minimum mean square error (MMSE) estimator. In order to find the optimal A .5. and be re-written as .90) Orthogonality Principle.88) R.] - Clearly.R...89) we have R..U ] [U . (5. Assuming the existence of R.:. r is somehow dependent on a . (5...R. R. = = E {aaH}.:RaT* (5..

] A" 0. is at least positive semidefinite. . The orthogonality principle states that we get an MMSE estimate if the error S(r). There are cases where the correlation matrix R.99) .96) In order to show the optimality of (5. . .H + AR.. (5.86): R.97) with A according to (5.. we derive from (5.94) E { [ S .91) can also be written as E { a r H } E {Sr"} . R:.R. becomes singular and the linear estimator cannot be written as A = R.. = R...H . we get a minimum of the diagonal elements of R. Using the properties of the pseudoinverse. nothing has been about possible said dependencies between a and the noise contained in r .a] SH { } = = = . Assuming that r=Sa+n. ...A = R. R.93) (5.D~. = which yields (5. which involves the replacement of the inverse by the pseudoinverse.118 Chapter 5.A +DR:.R:.96).97) and (5. Tkansforms Filters and Stochastic for Processes This means that the following orthogonality relations hold: E [S . and (5.AR. for D = 0. the estimator A=A+D (5.95) A more general solution.R. Singular Correlation Matrix... .a] r H }= 0.a is uncorrelated to all components of the input vector r used for computing S ( r ) . (5. (5.96) constitutes oneof the optimalsolutions.R. (5.R&..98) Since R:.. So far. (5.92) With A r = S the right part of (5.... is A = R.96) and an arbitrary matrix D is considered.. Additive Uncorrelated Noise. [AR.:. The relationship expressed in (5.R.94) is referred to as the orthogonality principle.

SH+ R. be except R..102) is advantageous in terms of computational cost..102).103) with [R. We partition A and a into (5. .90).AS]from the left and with [SR.5. can be immediately stated..S] from the left yields (5.106) + (5..ARar = R.: +SHR..101).105) ...89) becomes A = R.89). A can be written as + R.[R.l . uncorrelated noise process. we get from (5. ...-. The matrices to inverted in (5.SH Alternatively. + SHR. ] . (5. typically have a much smaller dimension than those in (5. Equivalent Estimation Problems. (5. For R.104) so that the following expression is finally obtained: Re. If the noise is white. + The equality of both sides is easily seen.101) and (5. .: 1 + SHRLAS].SH R .5. R...102).SR.SHRLA.' S H R i A S ] ..... S ] .~SHR. we have (5.100) and (5.]-1.101) A = [R.SH = SHRLA[SR. = [R.100) and A according to (5. + S H R .102) This is verified by equating (5. and by multiplying the obtained expression with [R. respectively: [R. = I. Linear Estimation 119 where n is an additive.102): Ree = Raa .: + SHRLAS]R..] from the right. (5. (5..103) Multiplying (5. and(5. (5..SH [SR.

112) describe estimations of a1 and a in the 2 models r = S l a l + nl.113) (5. Rnn + S1Ra1a1Sf.110) 2) = 3c .(5.AS1 R.102) can be written as where S = [SI.111) and (5. the covariance matrix R.:al). Tkansforms Filters and Stochastic for Processes (5.:az)-1 SfRiA] .117) .1 SyR.inz can be written as = [R.:nl and R.107) If we assume that the processes a l .1 2)-l (5.: have the form and A according to (5.: - (SyR.:nz = [R.120 such that Chapter 5.115) + R. (5..116) Equations (5.5'21.114) The inverses R. Applying the matrix equation € 3 €-l +E-132)-1BE-1 4 .: .RiiS2 + R.(5.&?€-l3 yields with Rn1n1 = Rnn = + SzRazazSf.1 3 2 ) .:] . (SfRiAS2 (5. a2 and n are independent of one another. and its inverse R.

118) (5. Using the shorthand (5. Nonzero-Mean Processes. One could imagine that the precision of linear estimations with respect to nonzero-mean processes r and a can be increased compared to thesolutions above if an additional term taking care the mean of values of the processes is considered.121) can be rewritten as: b =Mx. let us denote the mean of the parameters as =E{a}.123) . (5.119) Thus.5..120) The estimate is now written as iL=Ar+c%+c.122) (5. Then we get and Re.An exception is given if SFR$S2 = 0 . (5. b = h--. M = [c.A] (5. Linear Estimation 121 r = S2a2 with +n2 (5. In order to describe this more general case.121) b = a--.:. where c is yet unknown.5. each parameter to be estimated can be understood as noise in the estimation of the remaining parameters. = [SBRLASI+ and we observe that the second signal component Sza2 has no influence on the estimate.. which means that S1 and S2 are orthogonal to each other with respect to the weighting matrix R.

Tkansforms Filters and Stochastic for Processes The relationship between b and X is linear as usual.Fe"] = E { [T .131) Equation (5. (5. So far the parameter vector to be estimated was assumed to be non-random. .(5.b = E{[U - si] T"} = E{[a-si] [T-e]"}.e] [T .ee"] .[R..: through correlation matrices of the processes a and T . Unbiasedness for Random Parameter Vectors.s i ] [T (5.FP"] [R. .122) .125) with R.127) R.. From (5. b and R. so that the optimal M can be given according to (5. .e.: = 1 + e" e .110) we obtain R. . For the zero-mean processes the estimation problem can be solved as usual. .128) Using (5.si and r .122 Chapter 5. If we consider a to be a random process.122) and E { b } = 0 we derive (5.126) where F=E{r}.e]"} E { [T . (5. Subsequently the mean value si is added in order to obtain the final estimate U.131) can be interpreted as follows: the nonzero-mean processes a and r are first modified so as to become zero-mean processes a .ee"1-l e [R. (5. ..89): M = R..129) From (5.129) and [R..124) Now let us express R .e] [T .e]"} we finally conclude U = E { [a.130) .]' (5.l -e" [R. (5.bR$.e]"}-l [T - e] + a..FP"] -l -l (5. various other weaker definitions of unbiasedness are possible. writes 1 = [F FH RT.

(5. because it satisfied for any A as faras r and a are zero-mean.: = 0 is substituted into (5.. (5. .5. = h.h z .67)): A = [SHRiAS]-l SHRP1 nn (5. ak+l etc. the equivalences discussed above may be applied. A = [hl.This result is of special interest if no unbiased estimator can be stated for all parameters because of a singular matrix SHR.135) Then.AS. j # k. . .g. = E { a a H }is unknown. . ak-1.134) in which n contains the additive noise n and the signal component produced by all random parameters a j .138) In the previous discussion it became obvious that it is possible to obtain unbiased estimates of some of the parameters and to estimate the others withminimummeansquareerror. I f R. and we obtain the BLUE (cf.102). is A useful definition of unbiasedness in the case of random parameters is to consider one of the parameters contained in a (e. Starting with the model r = = Sa+n skak +n . (5.137) The Relationship between MMSE Estimation and the BLUE. . r (5.5.IH is an estimator which is unbiased in the sense of (5. as random variables: In order to obtain an estimator which is unbiased in the sense of (5. we can write the unbiased estimate as H 6. R.133). Linear Estimation 123 The straightforward requirement is meaningless. a k ) as non-random and to regard all other parameters a l . ..133). .

1 .1. where)( .i).. . (5. Tkansforms and Filters Processes Stochastic for 5.142).C is a channel and W.140) Thus.6 5.3. . By linear filtering of the noisy signal r ( n ) = z(n) w ( n ) we wish to make y(n) = r ( n ) * h(n) as similar as possible to a desired signal d ( n ) . An applicationexample is theestimation of data d(n) from a noisy observation r ( n ) = Cec(C) d ( n .94). Assuming a causal FIR filter h(n) of length p . p . The quality criterion used for designing the optimal causal linear filter h(n)is + The solution to this optimization problem can easily be stated by applying the orthogonalityprinciple.i) = TT&).6.1 Linear Optimal Wiener Filters Filters We consider the problem depictedin Figure 5.124 Chapter 5. we have P-1 y(n) = c i=O h(i)r ( n .143) The optimal filter is found by solving (5.(j .l ) w(n). (5. + . according to (5.142) with (5. . . By using the optimal filter h(n) designed according to (5. ( ) is noise.142) the data is recovered with minimal mean square error. j = 0. the following orthogonality condition mustbe satisfied by the optimal filter: For stationary processes r ( n ) and d ( n ) this yields the discrete form of the so-called Wiener-Hopf equation: P-1 c i=O h(i) r.

144) with 0 = E { ld(n12}.147) (5.3. Substituting the optimal solution (5. Designing linear optimal filters.142) into (5.1 (5.6.5.146) wit h (5.1 i=O i=O w. In matrix notation (5.145) Matrix Notation. Linear Optimal Filters 125 1 ' Noise \ . Variance. For the variance of the error we have w.144) : yields P-1 (5.148) and . I w(n) 44 Figure 5.142) is (5.

. Tkansforms and Filters Processes Stochastic for From (5. h(p . p . . ..151) Uncorrelated Noise. 1.(m + D).145) we obtain the following alternative expressions for the minimal variance: = C T ~ - r% h (5. (5. (5.. D value.1. h(l). (i) Filtering: d ( n ) = z(n). .l)] . i=O j = 0 .i) = r m ( j + D). . . . (5. Here the goal is to predict a future (iii) Prediction: d ( n ) = z(n D).(j+D). we have and rTd(m) = r. The following three cases..153) and from (5.151) we derive P-1 c ([rz2(j-i)+rww(j-i)] h i ) =r.156) . D > 0. (ii) Interpolation: d ( n ) = z(n + D). . < 0..146) and (5. p .150) Special Cases. 1 .155) hT = [h(O). + R w w l h = r z z ( D ) with (5.154) In matrix notation we get [R.126 Chapter 5. (5. If the noise W) ( . .l. . is uncorrelated to z(n). j = 0. + For the three cases mentioned above the Wiener-Hopf equation is P-1 i=O c h(i) rw(j . where the desired signal is a delayed version of the clean input signal ~ ( n are of special interest: )..

. The minimal variance is a:min = 0. One-step linear prediction.the error becomes e(.158) 1) Tzz(P - 1) r z z (0) For the correlation matrix R.. the corresponding definition holds.161) .. LPC. With + qn)=- C..1).2 One-Step Linear Prediction One-step linear predictorsare used in manyapplicationssuch as speech and image coding (DPCM.i).5.160) where p is the length of the FIR filter a ( n ) . rzz(--P r x x (-P + 1) + 2) ..) = ) .while no additive noise is assumed. and in feature extraction for speech recognition. they may be regarded as a special case of Wiener-Hopf filtering.) = -h(n . in spectral estimation.). ADPCM.4.159) 5.@)( n . Figure 5. i= 1 P (5. Note that the filter U(.6. .. ( X = z(n) -g(.4. w ( n ) = 0.i).) is related to the Wiener-Hopf filter h(n) as U(. z i= 1 P (5.. . Basically.) + C a ( i )z ( n . - T ~ ( D ) h (5.6. (5. We consider the system in Figure 5. ...A comparison with Figure5. Linear Optimal Filters 127 and rxx(0) r x x (1) rxx(P - Rxx = rzz(-l) r x x (0) .3 shows that the optimal predictor can be obtained from the Wiener-Hopf equations for the special case D = 1 with d ( n ) = z(n l).

We consider an autoregressive process of order p (AR(p) process). p introduced in (5. As outlined in Section 5.i ) .. ..34) are replaced by the coefficients . (5.167) The input-output relation of the recursive system may be expressed via the difference equation z(n) = W(. The system function of the recursive system is supposed to be2 U ( 2 )= 1 1 + icd . .128 Chapter 5.. According to (5. .i ) = r.. .162) which are known as the normal equations of linearprediction.(i) i=l P r z z ( j. Inmatrix notation they are that is R z z a = -rzz(1) (5. the coefficientsp ( i ) .3. .(j). (5.(i) z(n .168) i= 1 21n order to keep in linewith the notationused in the literature.a ( i ) . . Tkansforms and Filters Processes Stochastic for Minimizing the error with respectto thefilter coefficients yields the equations -C . () (5. . i = 1 . such a process is generated by exciting a stable recursive filter with a stationary white noise process W(. j = 1 . .164) (5. . 2 . .P.(l). . P # 0. .165) with aT = [.159) we get for the minimal variance: Autoregressive Processes and the Yule-Walker Equations.). .(p)].(i) P 7 z-i . p . . i = 1.) - c P . .

166) and (5. (5.(i) P (5. Linear Optimal Filters 129 For the autocorrelation sequence of the process z(n) we thus derive r z z ( m ) = E {z*(n)z(n m ) } = rzw(m) - + i=l c . . The cross correlation sequence r z w ( m )is r z w ( m ) = E {z*(n)w(n m ) } + = i=l c cc U*(i) rww(i m ) U26(i+77A) + (5. p by solving (5.m ) . m > 0.6.171) we finally get - i= 1 rzz(m) = 0 .(m c a(i)rzz(m P P . The equations (5.) (u(n)= 0 for n < 0)..174) . . Since U(.172) are known as the Yule-Walkerequations. i = 1. From (5.i).1) T z z ( P . where u(n)is the impulse response of the recursive filter.5.173) As can be inferred from (5. U * ( ..171) By combining (5. In matrix form they are Tzz(0) Tzz (1) Tzz(-l) Tzz (0) Tzz(-2) * * Tzz(--P) Tzz(-1) T z z ( P )T z z ( P .1) .i). m = 0. we derive is causal (5. .173).169) and (5.. .172) c (-m).170) = 0. .i). m < 0. Tzz(0) (5. By observing the power of the prediction error we can also determine the power of the input process. i= 1 c a ( i ) ?-..163).172) we have (5.169) r z z ( m. we obtain the coefficients a ( i ) .

3 Filter Design on the Basis of Finite Data Ensembles In the previous sections we assumed stationary processes and considered the correlation sequences to be known.In practice. . the prediction is error filter A ( z ) is the inverse system to the recursive filter U ( z ) t u(n). ) This also means that the output signal of the prediction error filter is a white noise process.130 Chapter 5. because the calculation of A ( z ) onlytakesintoaccountthesecond-order statistics.i). thecorresponding prediction error filter cannot have zeros outside the unit circle.Since a stable filter does not have poles outside the unitcircle of the z-plane.4 with the coefficients U(. Even if ) .we now describe the prediction error 2(2). P i= 1 . . The system function of the prediction error filter is P P (5. Inorder to determinethepredictor filter a(.) according to (5.176) i=l i=O In the special case that ~ ( n )an autoregressive process.) from measured data %(l). all parameters of an autoregressive process can be exactly determined from the parameters of a one-step linear predictor. Introducing the coefficient a(0) = 1. The output signal of the so-called prediction error filter is the signal e ( n ) in Figure 5. 5. Tkansforms and Filters Processes Stochastic for Thus. the whitening transform is carried out at least approximately.4. Minimum Phase Property of the Prediction Error Filter. we obtain a minimum phase prediction error filter. which do not contain any phase information.105). . Our investigation of autoregressive processes showed that the prediction error filter A ( z ) is inverse to the recursive filter U ( z ) . cf.however. ( is not truly autoregressive. Hence. (1.163). the prediction error filter performs a whitening transform and thus constitutes an alternative to the methods considered in Section 5. Prediction Error Filter. If )X . i=O a(0) =(5. linear predictors must be designed on the basis of a finite number of observations. e ( n ) is given by P e ( n ) = C a ( i ) z(n .6.175) 1.z ( N ) . . ( X is not an autoregressive process.

for instance. (5.5. Textbooks that cover this topic are. (5.179) Here. is a biased estimate of the true autocorrelation # the sequence r. ( ) .*(n) . The autocorrelation method following estimation of the autocorrelation sequence: l + p (m)= N is basedon the .p2c ’ (1) 2 (5. and X and 2 contain the input data. In the following. the properties of the predictor are dependent on the definition of X and X . which enables us to efficiently solve the equation . (A R.178) with +LtC’(m) z(N) .1) . (5. which means that E{+$tC’(m)} r z z ( m ) .. The criterion llell = I I Xa xll L min (5.p . The term X a describes the convolution of the data with the impulse response a ( n ) .. [84..177) where a contains the predictor coefficients.p + l ) z(N .(m).182) .(n + m). . N-lml n=l - c .. 1171. C ) = . ~ ( N .. z(N-p) X = 7 X = (5. 99.180) As can be seen. the correlation matrix kzc’ built from +k?(rn) has a Toeplitz structure. Linear Optimal Filters 131 via the following matrix equation: e=Xa+x. The autocorrelation method can also be viewed as the solution to the problem (5.. However. 471 or the Schur algorithm [130].6.. Thus.(l) . autocorrelationmethod yields a biased estimate of the parameters of an autoregressive process.181) by means of the Levinson-Durbin recursion [89. Autocorrelation Method.178) + leads to the following normal equation: XHXa=-XHx. two relevant methods will be discussed.

132 and Chapter 5.187) The equation to be solved is (C") R. a = h -+L. x ( N .i.190) Note that kg") not a Toeplitz matrix. . (5. The cowariance method takes into account the prediction errors in steady state only and yields an unbiased estimate of the autocorrelation matrix.p ) ::: X@) .188) is much more is complex than solving (5. so that solving (5. (5. (5.191) . In this case X and X are defined as X = and [ x ( N .1) . However.181) via the Levinson-Durbin recursion.")(I). the covariance method has the advantage of being unbiased.186) (5.185) Covariance Method. we have E{ RE"'} = R x x . (5.188) where rxx +")(l) =XHz.. ] (5.. Tkansforms and Filters for Stochastic Processes We have and ?Lf)(l) = X H x .

N . . since NC + C e N r.m ) r z z ( n+ m ) . as N n=-cc c lr.(m)} = r. . (5. we cannot expect good estimates for large m as long as N is finite.. we will discuss methods for estimating the autocorrelation sequence of random processes from given sample values z(n). (5. although consistency is given.196) Such an estimate is said to be consistent. The variance of the estimate can be approximated as [77] v that l o o v 4 e x (m11 Thus.7. Unbiased Estimate..(m). the variance tends to zero: (5.193) is known as the Bartlett window.n = 0 .(m)} = However. Estimation of Autocorrelation Sequences and Power Spectral Densities 133 5..197) . .1 Estimation of Autocorrelation Sequences and Power Spectral Densities Estimation of Autocorrelation Sequences In the following.194) the estimate is asymptotically unbiased.(m) = N-In-l " N n=O c x * ( n ) z(n + m).(n)12 + rZx(n . .193) lim E {?!. As can easily be verified.1. The triangular window occurs in (5. (5. An unbiased estimate of the autocorrelation sequence is given by N-In-l (5. (5.(m). because the bias increases as Iml + N .(m) is biased with mean E {?!. the estimate ?:.195) + m. We start the discussion with the estimate l ?!.192) whichis the same astheestimate ?iic)(m).usedin theautocorrelation method explained in the last section.7 5. However.5.7.

(5. ( with n = 0 .. because the variance increases for Iml + N. Tkansforms and Filters Processes Stochastic for The variance of the estimate can be approximated as [77] N (N .(m) is a consistent estimate.203) . i.2 Non-Parametric Estimation of PowerSpectral Densities In many real-world problems one is interested in knowledge about the power spectral density of the data to be processed. 5. it is a logical consequence to look at the Fourier transforms of these estimates.(n + m ) .so that we canconclude that the spectrum P. Since the power spectral density is the Fourier transform of the autocorrelation sequence. . order to be explicit."...e. .200) which means that ?. The Fourier transform of +!. only a finite set of observations)X .(n .lmD2 var[+:. .N-l is available. (5. (5. .(m).(m) is a biased estimateof the true autocorrelation r z z ( m ) . However. In let us recall that with wB(m) being the Bartlett window. 1 .(m) will be denoted as sequence We know that ?!.(eJW). problems arise for large m as long as N is finite.m) r. Typically.199) + As N + CO. We start with ?:.(n)1~ rZ.134 Chapter 5.. gives this N+cc lim var[?&(m)] + 0.(eJ") is a biased estimate of the true power spectral density S. and since we have methods for the estimation of the autocorrelation sequence.7. (m)]M c cc l~.

we have (5.204) where W B ( e j w ) is the Fourier transform of w ~ ( m ) by given (5.206) P.(ej")} is a smoothed version of the true power spectral density S...(m) e-jwm. E {P.(eJ") is to compute the Fourier transform of ) .206) In the form (5.(eJ") is known as the periodogram.(m).. Another wayof deriving an estimate of the power spectral density is to consider the Fourier transform of the estimate ?:..205) Thus.(ej") from X(ej"). We use the notation Q.7. A second way of computing P. we get (5.(eJW) for this type of estimate: N-l Q z z ( e j w )= c ?:.192) into (5.5.(ej").201) and rearranging the expression obtained. where smoothing is carried out with the Fourier transform of the Bartlett window. ( X first and to derive P. By inserting (5. Estimation of Autocorrelation Sequences and Power Spectral Densities 135 In the spectral domain. (5.207) m=-(N-l) ...

.!{+&(m)} e-jwm m=-(N-l) N-l m=-(N-l) M (5. (ej") 1 = S. The mean E { Q s s ( e j w ) } is a smoothed version of S..209) is its Fourier transform: (5.. For example. of the periodogram becomes which yields N+CC lim var [ P. ( m ) .. ( X with power spectral density S Z z ( e J W ) the variance . The behavior of the variance of the estimatesis different. The same holds for P.(m) and ?&(m)become unbiased.."..(ej"). The reason for this is the fact that only a finite number of taps of the autocorrelation sequence used in is the computation of Q s s ( e j w ) .136 The expected value is Chapter 5. where smoothingis carried out with the Fourier transform of the rectangular window.(ej").208) m=-m where w R ( m ) is the rectangular window w R ( m )= and WR(ej") { 1. Tkansforms and Filters Processes Stochastic for N-l E {Q.(ej"). S. - 1 (5. While the estimatesof the autocorrelation sequences are consistent. for Iml 5 N otherwise. (5..211) is straightforward and is omitted here.(eJW). so that both estimates of the power spectral density are asymptotically unbiased. the periodogram does not give a consistent estimate of proof of (5.212) The Thus..(ej").(ej")} = c . for a Gaussian process ) . 0.(ejw) and Q. those of the power spectral density are not.the quantity Q z z ( e j w ) is a biased estimate of S. As N + co both estimates ?:.210) is This means that although +&(m) an unbiased estimate of T. .

. ..1. it can be efficiently evaluated at a discrete set of frequencies by using the FFT.216) The Bartlett estimate then is (5. i = 0. This procedure is known as zero padding. . . K . (5. In many applications. because no powerful FFT algorithm is at hand for the given length.217) . Bartlett Method.7. X ( i ) (n)e-jwn n = 0 .214) with w k = 27rk/N'. Various methods have been proposed for achieving consistent estimates of the power spectral density. i = 0. The Bartlett method does this by decomposing the sequence ) . Moreover. the obtained number of samples of P z z ( e j w ) may be insufficient in order to draw a clear picture of the periodogram. .1. .1. Estimation of Autocorrelation Sequences and Power Spectral Densities 137 Use of the DFT or FFT forComputing the Periodogram. . Since the periodogram is computed from the Fourier transform of the finite data sequence. K . (5.214) is typically carried out via the FFT. DFT.1.5.213) with w k = 27rk/N. We obtain (5. .Given a length-N sequence ~ ( n we may consider a length-N ). . M . resulting in Pzz(eJWk) = 1 1 2 N-l x ( n ) e-jZr'lN I) (5. The evaluation of (5. .1. ( X into disjoint segments of smaller length and taking the ensemble average of the spectrum estimates derived from the smaller segments.215) we get the K periodograms (ejw) = M : : : 1C . With ~ ( ~ ) =n ) ( z(n + iM). These problemscanbe solvedby extending the sequence ~ ( nwith zeros to an ) arbitrary length N' 2 N . 1 . . the DFT length may be inconvenient for computation.

224) Thus. Denoting the window and its Fourier transform as w(m) and W ( e j w ) respectively. this means that (5.. (5.220) m=-(N-l) In the frequency domain.225) . ( ej w BT .(m) e--jwm. E { p . The argument is that windowing allows us to reduce the influence of the unreliable estimates of the autocorrelation sequence for large m.> N-l = C w ( m ) w B ( m )r. T ( e j w ) }= w(0) S.. be written as N-l P.138 Chapter 5. Tkansforms and Filters Processes Stochastic for with W B( e~ w ) J being the Fourier transform of the length-M Bartlett window.222) in order to ensure that PLT(ejw)is positive for all frequencies.(e B jw 11 = -1v a r [ ~ . (5. butthe bias increases accordingly andthespectrum resolution decreases. (ej w BT - C w(m) ?. in order to achieve an asymptotically unbiased estimate. For finite N ..219) Thus...221) The window W(. (5. the window should satisfy w ( 0 ) = 1. M .) should be chosen such that W(ejw) > o V W (5. Blackman-Tukey Method. ( X into K sets results in a reduced variance. ( e j ~ ) ]= K K sin (W M ) (5. the estimate can . K + 00.. Assuming a Gaussian process z(n). The expected value of PLT(ejw)is most easily expressed in the form E {P. as N .(m) e-jwm. Blackman and Tukey proposed windowing the estimated autocorrelation sequence prior the Fourier transform [8].223) m=-(N-l) Provided that w ( m ) is wide with respect to r z z ( m )and narrow with respect to W B ( ~ )the expected value can be approximated as .(ejw). the variance tends to zero and the estimate is consistent. the decomposition of ) . (5. . the variance becomes var[P.

In the Welch method [l621 the data is divided into overlapping blocks z(~)(. . i + n = 0 ..M m=O M 21r c 1" -T S&. i = 0 .1. .7. K .(ej"). Each block is windowedprior to computation of the periodogram.229) which means that the analysis is carried out with a energy.(ejw) dw. . 1 .230) The expected value becomes with - M -M . . . ) Welch Method. .1 (5.227) with D 5 M . For D = M we approach the decomposition in the Bartlett method. For D < M we have more segments than in the Bartlett method. 1 .) = ~ ( nD ) . Taking the average yields the final estimate window of normalized (5.l l In the spectral domain.1 w2(m) = . . (5. . resulting in K spectral estimates The factor a is chosen as 1 a = . M .M . Estimation of Autocorrelation Sequences and Power Spectral Densities 139 For a symmetric window w ( m ) = W(-m) the variance can be estimated as [8] This approximation is based on the assumption that W(ej") iswide with respect to W ~ ( e j " and narrow with respect to the variations of S. this can be rewritten as .5.

For a Gaussian process..")) becomes narrow with respect to SzZ(dv) and the expected value tends to This shows that the Welch method is asymptotically unbiased. Hanning Window.. w(n) = 0. In the following.5-0.0.. the expression reduces to W var[P. 0 n = 0 . which shows that the Welch method is consistent.l. .54 .237) For k + 00 the variance approaches zero. . (5.1 .5~0s .1 otherwise.238) otherwise..(e j w) I = -var[v$(ejw)1 1 K M -1 : ~ ( e j ~ ) .140 Chapter 5. the variance of the estimate is (5..240) .1.. w(n) = +0. SEW(ej(".236) If no overlap is considered (D = M ) . Various windows with different properties are known for the purpose of spectral estimation.234) With increasing N and M . Hamming Window. (5. N . N .l (5.08 cos . otherwise.. n = (). . n = 0.46 cos 0. ~ K (5. 1 .239) Blackman Window.. Tkansforms and Filters Processes Stochastic for (5.N . . w(n) = 0. a brief overview is given.

Recall that in Section 5. so that the estimated power spectral density may have a negative bias in the vicinity of large peaks in S. A comprehensive treatment of this subject would go far beyond the scope of this section.241) I The coefficients b(n) in (5. 5. (ej'")..5 shows the windows. Window functions.7.3 Parametric Methods in Spectral Estimation Parametric methods in spectral estimation have been the subject of intensive research. The spectra of the HanningandHamming window have relatively large negative side lobes. Figure 5.241) are the predictor coefficients determined from . The Blackman window is a compromise between the Bartlett and the Hanning/Hamming approaches.7.5. which also means that the bias due to the Bartlett window is strictly positive.6shows their magnitude frequency responses. and Figure 5.6. and many different methods have been proposed. We will consider the simplest case only.5. Estimation of Autocorrelation Sequences and Power Spectral Densities 141 14 Time Index Figure 5.whichis related to the Yule-Walker equations. The spectrum of the Bartlett window is positive for all frequencies. Hence the power spectral density may be estimated as (5.2 we showed that thecoefficients of a linear onestep predictor are identical to the parameters describing an autoregressive process.

5 -0.). . = f Z Z (0) process estimated (5. the power of the white input 3is 8. -0. and the prediction filter is always minimum phase. . .5 -0. Tkansforms and Filters Processes Stochastic for Hanning I " " " " ' l I .5 0 Normalized Frequency 0.5 0 . By combining both predictors one can obtain an improved estimation of the power spectral density compared to (5.241). just as when using the true correlation matrix R%%. the covariance method this is not the For case. . . . Finally. . . I I . 0. . l 0. . . . . the estimated autocorrelation matrix has a Toeplitz structure. .242) + (1) h. . .6. the observed data.5 0. . Magnitude frequency responses of common window functions. .174): 1 .5 -0. An example is the Burg method [19]. . . it shall beremarked that besides a forward prediction a backward prediction may also be carried out. .142 Chapter 5. . If we apply the autocorrelation method to the estimation of the predictor coefficients G(. 0 . . I I . and according to (5. . . 0 . . . .5 Normalized Frequency Normalized Frequency Figure 5. .5 Normalized Frequency Hamming I ' " ' ' ' ' ' ~ I I " " " Blackman " ~ I . .

Filter Banks. Time-Frequency Transforms and Applications. bandpass. they are also referred to as multirate systems. The blocks with arrows pointing downwards in Figure 6.1 shows an M-channel filter bank. In the case M = N we speak of critical subsampling. 143 . Thus. This operation serves t o reduce or eliminate redundancies in the M subband signals. The input signal is decomposed into M socalled subb and signalsby applying M analysis filters with different passbands.1 indicate downsampling (subsampling) by factor N.Signal Analysis: Wavelets. Perfect reconstruction means that the output signal is a copy of the input signal with no further distortion than a time shift and amplitude scaling. Alfred Mertins Copyright 0 1999 John Wiley& Sons Ltd print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Chapter 6 Filter Banks Filter banks are arrangements low pass. The reason for their popularity is the fact that they easily allow the extractionof spectral components of a signal while providing very efficient implementations. and the blocks with arrows pointing upwards indicate upsampling by N. T o give an example. Upsampling by N means the insertion of N . The upsamplers are followed by filters which replace the inserted zeros with meaningful values. each of the subband signals carries information on the input signal in a particular frequency band. because this is the maximum downsampling factor for which perfect reconstruction can be achieved.Figure6.1 consecutive zeros between the samples. Subsampling by N means that only every N t h sample is taken. Since most filter banks involve various sampling rates. and highpass filters used of for the spectral decomposition and composition of signals. They play an important role in many modern signal processing applications such as audio and image coding. This allows us to recover the original sampling rate.

) can be written as . ( W are zero. the synthesis filter impulse responsesgk (n)form of the basis. Filter Subbandsignals Synthesis filter bank I i Figure 6. 0 otherwise. ( and U(. and thetime-shifted variants gk:(n. From the mathematical point of view.1 6. N 2=0 .i N ) .1) After downsampling and upsampling by N the values w(nN) and u ( n N ) are still equal. ( W resultsfrom inserting zeros into ~ ( r n ) .1 Basic Multirate Operations Decimation and Interpolation Inthis section. N-l . Because of the different sampling rates we obtain the following relationship between Y ( z )and V ( z ) : Y ( P )= V ( z ) .144 Analysis filter bank Banks Chapter 6.1. we consider the configuration in Figure 6. we derive spectralinterpretations for the decimationand interpolation operations that occur in every multirate system. Using the correspondence e j 2 m h / N = 1 for n / N E Z. { the relationship between)W .2. The sequence ) .1. i E Z. The maindifference to theblock transforms is that thelengths of the filter impulse responses are usually larger than N so that the basis sequences overlap. M-channel filter bank. (6. a filter bank carries out a series expansion. while all other samples of ) . wherethe subbandsignals are thecoefficients. 6. For this.

Typical components of a filter bank. N-l XG(z)H(W&z)X(W&z).5): With (6. i=O The relationship between Y ( z )and V ( z )is concluded from (6.6) and V ( z ) = H ( z ) X ( z ) we have the following relationship between Y ( 2 ) and X ( z ) : N-l From (6. Basic MultirateOperations 145 Figure 6.7) we finally conclude X(z) = G(z) (zN) Y = l - . N--1 l - N CU(W&z).1) and (6. The z-transform is given by V(z) = ~ n=-cc N-l c cc w(n)zP cc = .2.6. N a=O . .1.1) and (6.

G . which does not lead to aliasing effects.4. Filter Banks -2. The reason why such filter banks allow perfect reconstruction lies in the fact that they can be designed in such a way that the aliasing components from all parallel branches compensate at the output..4. However.R ' u(ej0) 1 I -R R 2. The spectraof the signals occurring in Figure 6. This means that the products G(z)(H(W&z)X(W&z)) in (6. It is clear that z(n) can only be recovered from if no aliasing occurs.2 are illustrated in Figure 6.146 Chapter 6. This is shown in Figure 6... The general case with aliasing occurs when the spectra become overlapping. -2. . .2 (non-aliased case). Signal spectra for decimation and interpolationaccording structure in Figure 6.. -2n n I -R Figure 6.G -R I R 2. Signal spectra in the aliased case.3 for the case of a narrowband lowpass input signal z(n)..G * W Figure 6. where the shaded areas indicate the aliasing components that occur due to subsampling. h i(ej0) I R n 2n . y(m) the aliased case is the normal operation mode in multirate filter banks.8) are zero for i # 0..3. * W tothe ...

) .11) x. (6. as shownin Figure 6. ) This decomposition iscalled a polyphase decomposition. which are briefly discussed below. and the xi(rn) are the polyphase components of X ( .1.5. ( X components is given by M-l into it4 X(2)= where &(z) c e=o 2-e X&M). Several types of polyphase decompositions are known.2 Polyphase Decomposition Consider the decomposition of a sequence ) .1. X . (.6.5. A type-l polyphase decomposition of a sequence ) .l ) .12) . Type-2.10) Figure 6. ( ) = z(nit4 + it4 .1. 6. t ) ze(n) = z(nM + l ) . Basic MultirateOperations 147 Figure 6.M) 7 (6. Type-l polyphase decomposition for M = 3. Type-l. Interleaving all xi(rn) again yields the original X ( . ( X into sub-sequences xi(rn). (6.(2)t). The decomposition into type-2 polyphase components is given by M-l X ( 2 )= where c e=o z-(M-l-l) X.5 shows an example of a type-l decomposition.

(6.) + HI(z)Gl(z)]X(z) ++ [Ho(-z) Go(z) + H1(-z) Gl(z)] X(-z). The definitions for type-2 and type-3 components are analogous.) + Y1(z2)Gl(z)] . : Y1(z2) = X(z) . (6. 6. the only difference between a type-l and a type-2 decomposition lies in the indexing: X&) = XL-.13) Type-3. The signals are related as Y0(Z2) = [ H o b ) X(z) + Ho(-z) X(-z)l. The first term describes the transmission of the signal X ( z ) through the system.18) Combining these equations yields the input-output relation X(Z) = .14) X&) t ) z:e(n)= z(nM -e).-&). [Ho(z)Go(. (6.15) The relation to the type-lpolyphase components is Polyphase decompositions are frequently used for both signals and filters.2. while the second term describes the aliasing component at the output . (6. A type-3 decomposition reads M-l X(z) = where c l=O ze X&"). Filter Thus.148 Banks Chapter 6.6.2 6. (6.1 Two-Channel Filter Banks PR Condition Let us consider the two-channel filter bank in Figure 6.17) = [Yo(z2) Go(.[ H l ( Z ) X ( z ) + H1(-z) X(-z)l. In the latter case we use the notation H i k ( z ) for the lcth type-l polyphase component of filter H i ( z ) .

6.) Hl(Z) = Ho(-z) (6. of the filter bank. and the remaining filters are constructed as = Hob) Go(. . but amplitude distortions may be present. the amplitude distortions also vanish.20) If (6.21) G ~ ( z ) = -H~(z).19) is only approximately satisfied. Two-channel filter bank.6.2 QuadratureMirrorFilters Quadrature mirror filter banks (QMF banks) provide complete aliasing cancellation at the output.2.2. In QMF banks. Critically subsampled filter banks that allow perfect reconstruction are also known as biorthogonal filter banks.20) are satisfied. 6. Perfect reconstruction is givenif the outputsignal is nothing but a delayed version of the input signal. The following sections give a brief overview. Two-Channel Filter Banks 149 Figure 6. the transfer function for the signal component.19) and (6. That is. The principle was introduced by Esteban and Galand in [52]. H o ( z ) is chosen as a linear phase lowpass filter. G~ ) + (6. Several methods for satisfying these conditions either exactly or approximately can be found in the literature. If both (6. denoted as S ( z ) .20) is satisfied. the output signal contains no aliasing.must satisfy and the transfer function F ( z ) for the aliasing component must be zero: F ( z ) = Ho(-z) Go(z) H~(-z) ( z = 0. but condition (6.

19) yields Using the abbreviation T ( z )= Go(2) H o ( z ) . Inserting the above relationships into (6.2. 6. independent of the filter H o ( z ) .24) .G with symmetry around ~ / 2 . Filter Figure 6..the condition F ( z ) = 0 is structurally satisfied.150 Banks Chapter 6. One important property of the QMF banks is their efficient implementation dueto the modulated structure. . For the polyphase components this means that Hlo(z) = Hoo(z) and H l l ( z ) = -Hol(z).7. so that one only has to ensure that S ( z ) = H i ( z ) H:(-z) M 22-4.3 GeneralPerfect Reconstruction Two-Channel Filter Banks A method for the construction of PR filter banks is to choose (6. As can easily be verified.q = IHo(& + q QMF bank prototypes with good coding properties have instance been for designed by Johnston [78]. The name QMF is due to the mirror image property + IHl(. The resulting efficient polyphase realization is depicted in Figure 6.7.22) Is is easily verified that (6. (6.20) is satisfied. where the highpass and lowpass filters are related as H l ( z ) = Ho(-z). QMF bank in polyphase structure.

8. (6.5 - * u 0 -0.25) Note that i [ T ( z ) T ( . the system T ( z ) has to satisfy n=q n=q+21. The remaining filters are then chosen according to (6.2.4 Matrix Representations Matrix representations are a convenient and compact way of describing and characterizing filter banks.8.2. (a) linear-phase.5 11.5 - m t v . Examples of Nyquist filters T ( z ) .6. this becomes 22-4 = T ( z ) + (-1)"l T(-z).z ) ] is the z-transform of a sequence that only has non-zero odd taps. 6. Two-Channel Filter Banks 1. ' ' ' 0 2 4 6 8 10 14 12 16 18 -0. (b) shortoverall delay.5 ' ' ' ' ' ' 0 42 6 8 14 12 10 18 16 n(4 (b) Figure 6. which form the zeros of Ho(z) and Go(z). This design method is known as spectral factorization. In the following we will give a brief overview of the most important matrices and their relation to the analysis and synthesis filters.5 . Examples of impulse responses t(n)satisfying the first Nyquist condition are depicted in Figure 6.z ) ] is the z-transform of a sequence that only has non-zero even taps.3 ' 0 0 0 ' ' ' ' ' ' c 2 -70 0 0. .24) in order to yield a PR filter bank. while i [ T ( z ). *J ' ' ' 0. The arbitrary taps are the freedesign parameters. - t v 0. condition (6.25). Altogether we can saythat in order to satisfy (6. which may be chosen in order to achieve good filter properties. filters can easily be designed by choosing a filter T ( z )and factoring it intoHo(z) and Go(z).l#O arbitrary n = q 21 1.5 e 151 ~~~~~~~~~ ~~~~~~~~~ 1. This can be done by computing the roots of T ( z ) and dividing them into two groups. Thus. + + + e a (6.T ( .26) is known as the first Nyquist condition.26) In communications.

z ) . we introduce the vectors 1 (6.29) which contains the filters Ho(z) and H I ( z ) andtheirmodulated Ho(-z) and H l ( . The input-output relations of the two-channel filter bank may also be written in matrix form.For this.n) ~ n = C h l O ( k ) zo(m .33) .I) c k + C h l l ( k ) z1(m . We get versions Polyphase Representation of the Analysis Filter Bank. k (6. Let us consider the analysis filter bankin Figure 6.28) and the so-called modulation matrix or alias component (AC) matrax (6.L). The signals yo(m) and y1 (m) may be written as and y1(m) = C h 1 ( n ) x ( 2 m .9(a). Filter Modulation Matrix.152 Banks Chapter 6.27) (6.

as depicted in Figure 6. Two-Channel Filter Banks 153 1 Figure 6. Thelast rows of (6.7. hOl(k) = ho(2k = + 11. because theseequations are easily implemented anddonotproduce unneeded values.33) respectively. + 11.33) in the z-domain using matrix notation: 2/P(Z) = E ( z )% ( z ) . unlike in the QMF bank case. gives more insight into the properties of a filter bank. (b) polyphase realiza- tion. (6.32) and (6.9(a) is that only the required output values are computed.and (6.9(b). see Sections 6.2.32) and (6. it allows simple filter design. Thus.6.2. However.35) (6.34) (6. show thatthe complete analysis filter bank can be realized by operating solely with the polyphase components.32). It is convenient to describe (6.When looking at the first rows of (6. hlO(k) = h1(2k). where we used the following polyphase components: boo@) = how).36) .2. hll(k) SO(k) = 2(2k). (a) direct implementation.6 and 6. and leads to efficient implementations based on lattice structures. Analysis filter bank.1).33) this soundstrivial. 51(k) = 2(2k .9. the polyphase realization does not necessarily lead to computational savings compared t o a proper direct implementation of the analysis equations. The advantage of the polyphase realization compared to the direct implementation in Figure 6.

(Z’). we use the notation W-’ = .38) [ z-l] (6.39) Here. Polyphase Representation of the Synthesis Filter Bank. Q (6. The filters Go(z) and Gl(z) can be written in terms of their type-2 polyphase components as and Gl(z) = z-’G:O(Z’) + G:. Filter Matrix E ( z ) is called the polyphase matrix of the analysis filter bank.WH for the inverse.44) . it is related to the modulation matrix as follows: (6.37) with W = [ ’1 -1 1 1 and = ’ (6. We consider the synthesis filter bank in Figure 6.154 Banks Chapter 6.10(a).43) = 2mo samples is (6.10. As can easily be seen by inspection. W is understood as the 2x2-DFT matrix. (6.41) This gives rise to the following z-domain matrix representation: The corresponding polyphase realization is depicted in Figure 6. In view of the general M channel case. Perfect reconstruction up to an overall delay of Q = 2mo 1 samples is achieved if + R ( z ) E ( z )= 2-0 The PR condition for an even overall delay of I.

k ( z )= ( E ( z ) y . we have Hm(z)Rm(z) Rm(z)Hm(z) 2 I = = for the modulation matrices of paraunitary two-channel filter banks. From (6. where (6. Modulation Matrices.2.2.A similar property can be stated for polyphase matrices as follows: E-yz) = E ( z ) . L = 0. As can be seen from (6.48) Since E ( z ) is dependent on z .47).10.47) E ( z ) k ( z )= k ( z )E ( z )= I . (a) direct implementation.47) is said to be paraunitary.n ) .5 ParaunitaryTwo-Channel Filter Banks The inverse of a unitary matrixis given by the Hermitian transpose. ( E ( z ) )stands for transposing the matrix and simultaneously conjugating the elements: In the case of real-valued filter coefficients we have that B ( z ) = ET(zP1)and fiik(z) = Hik(z-l). such that 11 2 (6. and since the operation (6. realization. (6. a matrix E ( z ) satisfying (6.6. Two-Channel Filter Banks 155 (4 (b) Figure 6. such E ( z ) ET(z-1) = ET(z-1) E ( z ) = I .46) has to be carried out on the unit circle. (b) polyphase 6.37) and (6.49) Matched Filter Condition.46) (6.45) = 1. (6. Synthesis filter bank. ~ Analogous to ordinary matrices.l. and not at some arbitrary point in the z plane.50) .49) we may conclude that the analysis and synthesis filters in a paraunitary two-channel filter bank are related as G ~ ( z )f i k ( ~ ) = t ) gk(n) = h i ( . (6.

55) Here. Filter This means thatan analysis filter andits corresponding synthesis filter together yield a Nyquist filter (cf. (6. the required analysis and synthesis filters can be derived as (6.24)) whose impulse responseis equivalent to the autocorrelation sequence of the filters in question: (6. Power-Complementary Filters. From (6.52) + IHo(ej ( W + 4 ) 12. where the receiver input filter is matched to the output filter of the transmitter such that the overall result is the autocorrelation sequence of the filter. For constructing paraunitary filter banks we therefore have to find a Nyquist filter T ( z ) which can be factored into T ( 2 )= Ho(z) f i O ( 2 ) .This approach was introduced by Smith and Barnwell in [135]. A filter that satisfies this condition is said to be valid.53). Since T ( e J Whas symmetry ) around W = 7r/2 such a filter is also called a valid halfband filter.156 Banks Chapter 6. The reason for choosing this special input filter is that it yields a maximum signal-to-noise ratio if additive white noise interferes on the transmission channel. . (6. This is known as the matchedfiltercondition. Given Prototype.53) be powerWe observe thatthe filters Ho(ejW) and Ho(ej(w+")) must complementary to one another.51) Here we find parallels to data transmission. L is the number of coefficients of the prototype. (6.54) Note that a factorization is possible only if T(ej") is real and positive. Given an FIR prototype H ( z ) that satisfies condition (6. which for z = eJ" implies the requirement 2 = IHO(ej")l2 + (6.49) we conclude 2 = Ho(z)fio(z) Ho(-z)fio(-z).

We will show that paraunitary two-channel filter banks are non-linear phase with one exception. this yields the requirement 0 = h0(25)h:(O).jG(z)ejp/') = c2 z-~. . - (6.57) Non-Linear Phase Property. n = 25. Prototypes for paraunitary two-channel filter banks have even length. 2 2 2 2 (6. In other words. This means that the filter has to have even length. This is seen by formulating (6.2. . 5 # 0. = llgoIle.which for ho(0) # 0 can only be satisfied by ho(2k) = 0. = llglllez = 1. It is easily verified that all filters in a paraunitary filter bank have energy one: llhollez = llhllle. assume that two filters H ( z ) and G ( z ) are We power-complementary and linear-phase: c2 = H(z)fi(z) G(z)G(z) = eja z L ~ ( z ) . The following proof is based on Vaidyanathan [145].(n 2 4 .. (6. ..ho(25): se0 = c 2k n=O h0(n)h.61) in order to be both power-complementary and linear-phase. Filter Energies.59) Both factors on the left are FIR filters.58) (linear-phase property). + B(z) G ( z ) = ejp z L G ( z ) . power-complementary linear-phase filters cannot have more than two coefficients. . We conclude (H(z)ejal' + jG(z)ejp/') (H(z)ej"/' .6.52) the time domain in and assuming an FIR filter with coefficients ho(O). pER 1 (6. so that Adding and subtracting both equationsshows that H ( z ) and G ( z )must have the form (6.56) For C = k . Two-Channel Filter Banks 157 Number of Coefficients.

we obtain filters of length L = 2 N . k = (). Since this lattice structure leads to a paraunitary filter bank for arbitrary we can thus achieve perfect reconstruction even if the coefficients must be quantized due to finite precision. D‘(z)B:_. . k = 0. we excite the filter bank with zeuen(n) dn0 = and ~ . Provided cos. N .2. Given a k .. .& # 0. [147].11(a). Filter 6. d d ( n ) = dnl and observe the polyphase components of Ho(z) and H l ( z ) at the output. .This means that all = rotations areinverted and additional delay is introduced.6 Paraunitary Filter BanksinLatticeStructure Paraunitary filter banks can be efficiently implemented in a lattice structure [53].65) with N-l 1 (6.63) and D ( z ) is the delay matrix D= [. the matrices B k . .1. - 1.. ..11(b). In addition.158 Banks Chapter 6. . k = 0.l. The implementation is shown in Figure 6. we decompose the polyphase matrix E ( z ) as follows: (6.62) Here.66) kO = This basically allows us to reduce the total number of multiplications.64) It can be shown that such a decomposition is always possible [146].. For this. following (6. The polyphase matrix of the synthesis filter bankhasthe factorization: R(2)= BTD’(2)BT . we can also write (6. such that D ‘ ( z ) D ( z ) z P 1 1 . For this.N . The realization of the filter bank by means of the decomposed polyphase matrix is pictured in Figure 6. zlll] (6. this structure may be used for optimizing the filters.) = J D ( z ) J . . .N - 1 are rotation matrices: (6. .67) with D’(. ak..

2. we have much more design freedom than in the paraunitary case. -a1 -a0 m + (b) UN-l Y 1 (m) +- + Figure 6. A Nyquist filter with P = 6 zeros can for instance be factored into two linearphase filters each of which has three zeros. .68) It results in a linear-phase P R filter bank. D(2)LO with (6. (a) analysis. The realization of the filter bank with the decomposedpolyphase matrix is depicted in Figure 6. any factorization of a Nyquist filter into two linear-phase filters is possible.7 Linear-Phase Filter Banks in Lattice Structure Linear-phase PR two-channel filter banks can be designed and implemented in various ways. In addition.6. as will be discussed in the following..2. However.2. Two-Channel Filter Banks 159 Y(K om ) UN-1 Tm 7 .l D ( 2 ) L N .2 .6. The following factorization of E ( z ) is used [146]: E ( 2 ) = L N . (b) synthesis.12. Since the filters do not have to be power-complementary. involves the restriction that the number of coefficients must be even and equal for all filters. or into one filter with four and one filter with two zeros.11. the structure is suitable for optimizing filter banks with respect to given criteria while conditions such as linear-phase and PR are structurally guaranteed. Paraunitary filter bank in lattice structure. . 6. + .. As in the case of paraunitary filter banks in Section 6. The number of filter coefficients is L = 2(N 1) and thus even in any case. For example. realizing the filters in lattice structure. we can achieve P R if the coefficients must be quantized because of finite-precision arithmetic.

a 2 0 in the polyphase domain as shown in Figure 6.~ z-~”B(z’) z-~A(z’)B(z’). The overall delay has increased by 2a. b 2 0.12. the overall structure still gives PR. Linear-phase filter bank in latticestructure. Filter Banks Yo(m)-r-K -aN-2 .. + + Now the overall delay is 2a 2b 1 with a. the delay may be and unchanged ifwe have chosen a = b = 0. In fact.13(a). ( X with the filter and subsampling.13(a).Clearly.~ . where the overall delay has to be kept below a given threshold. 6. Note that. we consider the two-channel filter bank in Figure 6.~ ~ . we use a dual lifting step that allows us to construct a new (longer) filter HI (2) as H~(z) =z . 1411 for the design of biorthogonal wavelets. The structure obviously yields perfect reconstruction with a delay of one sample. Nowwe incorporate a system A ( z ) and a delay z . the new yo(rn) results from filtering ) .13(b). This technique allows us to design PR filter banks with high stopband attenuation and low overall delay.13(c). while the new subband signal yo(rn) is different from the one in Figure 6. + + . Such filters are for example very attractive for real-time communications systems.8 Lifting Structures Lifting structures have been suggested in [71.. (b) synthesis. In the next step in Figure 6. g -a0 -a0 x^(4 Y 1 (m) + @) Figure 6. In orderto explain the discrete-time filter bank concept behind lifting. although we may already have relatively long filters Ho(z) H l ( z ) . (a) analysis.2.160 Chapter 6.

However. which are very popular in image compression. Figure 6.149604398. the lifting steps can easily be chosen t o yield linear-phase filters. the filters constructed via lifting are non-linear phase. Two-channel filter banks in lifting structure.2. Moreover. The parameters are a = -1. p = -0. . Figure 6. 6 = 1.6. the total number of operations is lower than for the direct and polyphase implementation of the same filters. To give an example. due to the joint realization of Ho(z) H l ( z ) . Lifting implementation of the 9-7 filters from [5] according to [37]. 6 = 0.13. In general. Two-Channel Filter Banks 161 m (c) Figure 6.586134342. Both lattice and lifting structures are very attractive for the implementation of filter banks on digital signal processors.05298011854. because coefficient quantization does not affect the PR property.8829110762.14 shows the lifting implementation of the 9-7 filters from [ 5 ] .4435068522. y = 0.14.

Further structuresare easily found. . It is easily seen that both systems are equivalent. 6. In orderto describe the system functions of cascaded filters with sampling rate changes. An example of the frequency responses of non-ideal octave-band filter banks in tree structure is shown in Figure 6. The proof is based on the Euclidean algorithm [g]. A simple way of designing the required filters is to build cascades of two-channel filter banks.15 shows two examples. P R is easily obtained if the two-channel filter banks.3 Tree-Structured Filter Banks In most applications one needs a signal decomposition into more than two. which are used as the basic building blocks. An effect. Their system function is For the system B2(z2)we have With this result. Unfortunately. The decomposition of a given filter bank into lifting steps is not unique. so that many implementations for the same filter bank can be found. the system functions of arbitrary cascades of two-channel filter banks are easily obtained. is the occurrence of relatively large side lobes.and also signal-adaptive conceptshavebeen developed.17. we consider the two systems in Figure 6. Filter An important result is that any two-channel filter bank can be factored into a finite number of lifting steps [37]. where the treeis chosensuch that it is best matched to the problem. which results from the overlap of the lowpass and highpass frequencyresponses. Figure 6. provide PR. (a) a regular tree structure and (b) an octave-band tree structure.16.162 Banks Chapter 6. say M . frequency bands. one cannot say a priori which implementation will perform best if the coefficients have to be quantized to a given number of bits. In all cases.

3. Equivalent systems.16. . Tree-structuredfilterbanks. Tree-Structured Filter Banks 163 4 h U U -I &- Figure 6. (b) octave- B2 (4 Figure 6. (a) regular tree structure.15.6. band tree structure.

8 0. In orderto obtain general results for uniform M-channel filter banks.6 0. From equations (6. (a) two-channel filter bank.0 0.164 Chapter 6. where M is the number of subbands. 6.1 shows such a filter bank. we start by assuming N 5 M .2 n.1.o 2 0.4 0.4.o 0.8 m/Jc Figure 6.4 UniformM-Channel Filter Banks This section addresses uniform M-channel filter banks for which the sampling rate is reduced by N in all subbands. 0. Figure 6. and Figure 6.17.2 0.6 g 2 8 0. Filter Banks 1. Frequency responses in tree-structured filter banks.0 6.8) we obtain .1 Input-Output Relations We consider the multirate filter bank depicted in Figure 6.18 shows some frequencyresponses. - 1.7) and (6.4 g Fr. (b) octave-band filter bank.

) . .dM a Figure 6. .71) shows that X ( z ) = 2 .71) C i=o k=O M-] Equation (6.69) M-lN-l k=O X(Z) = C C Gk(z)Hk(W&z)X(W&z). . . Frequency responses of the analysis filters in a uniform M-channel filter bank. M N i=O and _I C - 1.70) In order to achieve perfect reconstruction. UniformM-Channel Filter Banks 165 W -a 0 @) 2. (6. k (z = 0.M . . We obtain the PR requirement by first changing the order of the summation in (6.6. and parameters N and M must be chosen. 0 < i < N .18. = . .1.suitable filters H k ( z ) and GI.~&o.70): X(Z) =- 1 N-l C X(W&Z) G~(z)H~(W&Z). .4.4 M-l k=O X(z) holds if the filters satisfy (6. (a) cosine-modulated filter bank. . (6.72) C Gk(z)Hk(W&z) N z . i=o (6. k = 0 . (b) DFT filter bank. 1 N-l Yj(z) = H k ( W h z k ) X(W&zk).1.

The implementation of such a filter bank is depicted in Figure 6.2 ThePolyphaseRepresentation In Section 6.166 Using the notation Chapter 6. . (6.g T ( z ) H Z ( z ) = 2-9 1 N [1. PR requires that . z.(z).78) E ( z )= (6.75) 1 HT ( z ) g.81) .19. . . . Analysis.4.77) 6. . . X ( Z W N ) . (6.. X ( z W p ) ] T .76) Thus.2 we explained the polyphase representationof two-channel filter banks. the input-output relations may also be written as X(z) = (6. (6.O] .0.. The generalization to M channels with subsampling by N is outlined below. Filter Banks H m ( z )= and z m ( z ) = [ X ( Z ) . The analysis filter bank is described by !h(.) where = E(z)Z P k ) .

The generalization to any arbitrary delay of Mqo r M . From (6.4. Synthesis may be described in a similar way: (6. Uniform Banks Filter 167 Z -1 Z-1 Figure 6.84) as and let us assume that all elements of E ( z ) are FIR.1 samples is + + + (6.83) Perfect Reconstruction. (6. Let us write (6.85). FIR solutions for both the analysis and synthesis filters of a P R filter bank require that the determinants of the polyphase matrices are just delays.82) we conclude thePR requirement R ( z ) E ( z ) = 2-40 I .1 [146]. . (6.1 samples. Thus. Filter bank in polyphase structure.We see that theelements of R ( z ) are also FIR if det{E(z)} is a monomial in z .86) FIR Filter Banks.78) and (6.19.85) where 0 < r < M . The same arguments hold for the more general P R condition (6.M-Channel 6.84) which results in an overall delay of Mqo M .82) with R ( z )= (6. Synthesis.

where x p ( z )is the polyphase vector of a finite-energy input signal and yp(z) = E ( z ) xp(z)is the vector of subband signals.6 and 6. M . the impulse responses hk(n-mM) andgk(n-mM). which can be chosen free in order to obtain some desired filter properties. . . .ol] 0 * (6.l . .M . form orthonormal bases: (6.168 Chapter 6. . . 5 = 0 . . Filter Banks 6. . a useful objective function has to be defined and the free parameters have to be found via non-linear optimization. This may be expressed as llypll = l l z p l l V x p with llxpll < CO..2. (6. k = 0 . . m E Z.4. respectively.92) The matrices A k . This also implies that hk(n) = g.89) 6.88) Especially in the critically subsampled case where N = M .7. To achieve this. we in consider the following factorization of E ( z ) : where = [ IM-l .(-n) t ) (6. . The elements of these matrices are the design parameters. one tries to minimize the stopband energy of the filters.4.3 Paraunitary Filter Banks The paraunitarycase is characterized by the fact that the sum the energies of of all subband signals is equal to the energy of the input signal.87) H I . k = 0. .4 Design of Critically Subsampled M-Channel FIR Filter Banks Analogous to the lattice structures introduced Sections 6. K are arbitrary non-singular matrices. It can easily be verified that filter banks (oversampled and critically sampled) are paraunitary if the following condition holds: k ( z )E ( z )= I .1.2. .1. ( z )= G k ( z ) . Typically.

. so that the implementation cost is somehow reduced.I'(Z)A. Paraunitary FIR Filter Banks based on Reflections. Here.1 Givens rotations performed successively on the elements Ic. . . Filter design can be carried out by defining an objective function and optimizing the rotation angles. = I ..only have t o belong to the ~ subset of all possible M X M unitary matrices which can be written asa sequence of M .'. to be unitary.95) Figure 6. + .. (6. that guarantees the existence of A i l is t o use triangular matriceswith ones on the main diagonal. M .4. . I V + z-1vI.1 (k) (6.97) VI. A simple parameterization for the matrices AI. L 1 for Ic = 0. the polyphase matrix is written as follows: E ( z ) = VK(Z)VK-l(Z) * V .sin 6 K . ( z ) U .v:. Uniform M-Channel 169 The corresponding synthesis polyphase matrix can be designed as R ( z )= AilI'(z)ATII'(z) . . * The matrices V I . Paraunitary FIR Filter Banks based on Rotations. 1- (k) 4K-l sin 4K-l cos 6 K . The matrices Ao.2. . (6. .1.sin 4r) cos 4r) . The inverses then are also triangular..s Filter 6. Paraunitary filter banks are easily derived from the above scheme by restricting thematrices AI.94) Clearly. both E ( z ) and R ( z ) contain FIR filters. not all matrices have to be fully parameterized rotation matrices in order to cover all possible unitary filter banks [41].96) The last matrix AK has to be a general rotation matrix.93) (6. . ( Z ) reflection-like matrices of the type are (6. A secondway of parameterizing paraunitary filter banks was proposed in [148]. Interestingly.. Examples are given in [146].= sin4p) 1 - -1 1 (k) COS .: ..cos&) AI.20 illustrates the implementation of the filter bank according to the above factorizations.98) . The overall transfer matrix is R ( z ) E ( z )= C K I .1 (k) - - . A K . (6.

the analysis and synthesis filters. and in most cases the same prototypes can be used for both sides. For this special class the reader is referred to [137]. M-channel filter bank with FIR filters.vk[l .4 for the implementation of Householder reflections: instead of multiplying an input vector z ( z ) with an entire matrix V k ( z ) . I I @) Figure 6. Modulated filter banks have the great advantage that only suitable prototypes must be found.170 Banks Chapter 6.97) directly leads to an efficient implementation. The parameterization (6.It is easily proven that V r ( ~ . Due to the modulated structure very efficient implementations are possible. (b) synthesis. so thatthematrices = can indeed be used for parameterization. not the complete set of analysis and synthesis filters. Hk:(z) and Gk:(z). Filter . One prototype is required for the analysis and one for the synthesis side. methods for designing linear-phase paraunitary filter banks have also been developed. which generally yield nonlinear phase filters.one computes z ( z ) . where vk: is an M X 1 vector with llvkll = vTv = 1.~ ) V k ( z )I . where all filters are derived from prototypes via modulation. The matrix U has to be a general unitary matrix.z). In DFT banks.K'] [vTz(z)] in order to obtain Vk:(z)z(. 6. (a) analysis.20.5 DFT Filter Banks DFT filter banks belong to the class of modulated filter banks. which is similar to the one discussed in Section 3.4. In addition to the above parameterizations. are .

which requires extremely low computation effort. followedby an IDFT (without pre-factor l / M ) . 861 (6. as shownin Figure 6.( ) 1. the same principle can be used.6. On the synthesis side. For critical subsampling. the subband signals can be computed by filtering the polyphase components of the input signal with the polyphase components of the prototype.102) This means thatthe polyphase components of P R FIR prototypes are restricted to length one.100) nO = nO = We now substitute n = i M + j . If oversampling by afactor p = E Z is considered. and rewrite (6. is depicted in Figure 6. The analysis equation is L-l %(m) = p: k.-l z ( m M .21. The complete analysis/synthesis system.103) . the PR condition is easily found to be (6. Thus. critically subsampled DFT filter banks with P R mainly reduce to the DFT.21.100) as Thus.5. and the filtering degenerates to pointwise scalar multiplication. let us consider the critically subsampled case. DFT Filter Banks 171 Inorder to explain the efficient implementation of DFT banks. the PR condition becomes [33. L = ML.n) (6.

oversampled DFT filter banks offermoredesignfreedom than cosine-modulated ones. Figure 6. this filter bank is modified in such a way that PR is achieved with FIR filters [55].M . Filter Clearly.102) is included in (6. The prototypes are typically designed to be lowpass filters.) and Q ( z ) with good filter properties. the extraction of the real and imaginary parts takes place in adjoining channels in reverse order. On the other hand. This is most easily seen from (6. provided the outputsignal is downscaled by the oversampling factor. which will be discussed in the next section.103) means an increased design freedom compared to (6. . Thus. (6. . extracting the real and imaginary parts. The key t o P Ris subsampling the filter output signals by M/2.103). L = 0 . [82].104) At this point it should be mentioned that all P R prototypes for M-channel cosine-modulated filter banks. also serve as PRprototypes for oversampled 2M-channel DFT filter banks.103) is not sufficient in the cosine-modulated case. A common design criterion is to minimize the stopband energy and the passband ripple: S a ! passband (IP(ej")l. satisfying only (6. (6.1)zdw + S p IP(ej")12dw L min. . .102). Compared to the simple DFT filter banks described above.172 Banks Chapter 6.22 shows the MDFTfilter bank introducedby Fliege. This freedom can be exploited in order to design FIR prototypes P(. if a filter bank provides P R in the critically subsampled case.22. (6. Thus. MDFT Filter Bank. it also provides P R in the oversampled case.103) for p = 2: In general. . As can be seen in Figure 6. and using them to compose the complex subband signals yk:(rn).1.

The quality of a filter bank is not only dependent on whether it reconstructs perfectly or not.21.. the desired output signal is formed by another temporal reversal. This is not a feasible strategy if we work with one-dimensionalsignals. Husmy and Ramstad proposed to construct the polyphase components of the prototype as first-order IIR allpass filters [74]: P ( ) i 2= m 1 + aiz-1’ 1 ai + z . this leads to a problem concerningstability: if the analysis filters arestable. but in image processing we a priori have finite-length signals so that this method can be applied nevertheless. DFT PolyphaseFilterBank with IIR Filters andPerfectReconstruction.5.l i = o . Unfortunately. Modified complex modulated filter bank with critical subsampling. for example in order to provide a maximal coding .the synthesis filters determined according to (6. The actual purpose of the filter bank is to separate different frequency bands. DFT Filter Banks 173 X^@) Figure 6. A 4 . We consider theDFT filter bank in Figure 6.6.. .105) Using the synthesis filters then ensures perfect reconstruction. .22. (6. Then.1 . This problem can be avoided by filtering the subband signals “backwards”using the stable analysis filters.106) are not.

83. L.. 94. The variable D denotes the overall delay of the analysissynthesis system. from an FIR prototype q(n) according to hk(lZ) = 2p(n)cos[G ( k + i ) (v. However. are derived from an FIR prototypep ( n ) and the synthesis filters g k ( n ) . and an overall delay of D = 2sM 2M .4 dB for the prototype P(. i = 0 . A suitable choice for q5k is given by r$k = (-1)"/4 [87. M . 68. . 100.) of an eight-channel filter bank [74]. 1031.95]. 127. m. Thestopbandattenuation of theprototype P ( z ) composed of IIR allpasses is determined by the parameters ai..Ic = 0.1. Generalizations to all filter lengths and delays are given in [68].4 .174 Banks Chapter 6.. L . . 681. . . the MPEG audio For standard [l71 is based on cosine-modulated filter banks.1. gk(n) = 2q(n)cos[$ (k+i) n=O. and also as biorthogonal filter banks allowing low reconstructiondelay [110. . The most common case within this framework is the onewhere the sameprototype isusedfor analysis and synthesis. Perfect reconstruction is easily achieved by choosing an appropriate prototype. + . we will consider biorthogonalcosine-modulated filter banks where the analysis filters hk(n). . In the following.1 samples. 121.m' E IN. = 2mM and L. . In view of the extremely low computational cost this is an astonishing value.. in order to demonstrate some of the design freedom. .. 100. ...1. For the sake of brevity. example. 121. Filter gain. 861.. . 129. . Husrrry and Ramstad statea stopband attenuation of 36.6 Cosine-Modulated Filter Banks Cosine-modulated filter banks are very populardue to their real-valued nature and their efficient implementation via polyphase structure and fast DCT [116.l .-1 . 87. The length of the analysis prototype is L.M . so that these are the design parameters.) +qh]. M . and the length of the synthesis prototype is L. 110. paraunitary filter banks [94. Cosine-modulated filter banks can be designed as pseudo QMF banks [127]. we start with a more general approach where different prototypes are used for analysis and synthesis. . . . 6. = 2m'M. so that the design of lowdelay banks is included here. Note that the delay can be chosen independently of the filter length. we confine ourselves to even M . Ic = 0 . . analysis and synthesis prototypes with lengths L. n=O. 129. 87.

". (6. . k = O . .j = 2cos [G( k + ).PM-l(-z2)] . .Q1(-z2). . .~. M . .P2M-l(-z2)] .Z ~ ) .110) [ P M ( . k = O .. Q M ( . . (6.j - p) . ] . 2 M .109) P o ( z 2 ) = diag [P0(-z2). .1 .1 .108) where [Tl]k. Note that the matrices P 0 ( z 2 ) and P1 ( z 2 ) contain upsampled and modulated versions of the polyphase filters.1. 2 M . Note that in the case of DFT filter banks.1 . .Pl(-z2). j = o . j = 0 . M . and (6. (6. P M + .Banks 6.1 Critically Subsampled Case In the critically subsampled case the analysis polyphase matrix can be written as [112.107) e=o 6.. 2 M .~ ~ ).l .".j = 2cos [G(L + ).. P 1 ( z 2 ) = diag (6. . j = o .4 4 .6.". (2M .".Q~(-z2)] .l . only M polyphase components were used to describe an M-channel filter bank. we first decompose them into 2M polyphase components. . . . Qo(z2)] TT.6.Filter Cosine-Modulated 175 In order to derive the conditions that must be met by the prototypes P ( z ) and Q(z) to yield perfect reconstruction.I ( . For the synthesis polyphase matrix we get R ( z ) = [z-lQ1(z2).113) Q1(z2) = diag [ Q ~ M . (-~~).111) where [T2]k. . We use the type-l decomposition given by m-l P~(z) = C p(2lM + j ) z-e . . . 681 (6.112) Qo(z2) = diag [QM-I(-z~). Q M + I ( . .Z ~ ) . and (6. (j - p) + 4 k ] . .~ ~ ) ..

The prototype can for instance be designed by using the quadratic-constrained least-squares (QCLS) approach. For this.118) ) The condition (6.120) . % ..119) in quadratic form and optimize the prototype using constrained numerical optimization. Thus. we write the PR conditions as V(z)U(z) = z-l(-z-z)' 2M I.117) is satisfied for Q k ( z ) = az-PPk(z) and Q ~ + k ( z = az-8 P ~ + k ( z with arbitrary a . which guarantees P R and also leads to a very efficient implementation of the filter bank. .119) The M/2equations in (6. . Another approach. . is to design the filters via lifting [129. (6. p.119) may be understood P R conditions on M/2 as non-subsampled two-channel filter banks. .the remaining condition is PZM-l-k(Z) Pk(Z) + P M + k ( Z ) PM-l--k(Z) =2 " ! z-' k=O. .Here.. whichwas proposed byNguyen [lll]. but the PR constraints can be satisfied with arbitrary accuracy.1. which suggests the use of the same proto) type for both analysis and synthesis.176 Banks Chapter 6. (6. Filter The perfect reconstruction conditions are obtained by setting (6..114) Using the property [87] this yields the conditions which have to be met for k = 0 . The approach does not inherently guarantee PR. (6.-2 M 1. with Q ( z ) = P ( z ) . 831. we write all constraints given by (6. The relationship between qo and S is qo = 2s + 1.

The filter design is as follows.121) It is easily verified that (6. this operation is called zero-delay lifting.123) Longer filters with the same delay are constructed by introducing matrices of the type A i l ( z ) A i ( z )= I (6. From the Note that Ui+l ( z ) and Vi+l( z )retain the structure new matrices thepolyphase componentsof the prototype are easily extracted.116).(z)U.z (-l)s-lQz~-l--k-~(-z ) ) (-1)Sz-1Qk+~(-z2) Qd-z2) 2 (6.~ .126) can be repeated until the filters contained in U i ( z )and V i ( z ) have the desired length. Cosine-Modulated Filter Banks 177 where z .120) includes (6. 2M (6. where the subscript 0 indicates that this is the 0th iteration. Since the overall delay remains constant.124) L in between the product (6.~ ( . A second possibility is to introduce matrices .121).(z) = ~ z-1 I. The operation (6.114) by using the properties of the cosine functions [83].117) and (6.6. We have V.120) can also be derived straightforwardly from (6.~ Q ~ M . We start with l.126) given in (6.6. but (6.

. T.. [83]: 2cos [$F ( k [TlIk. Filter Banks = Ci(Z)Ui(Z). The straightforward polyphase implementation of (6.128) This type of lifting is known as maximum-delay lifting. the output signals of the corresponding synthesis polyphase filters are added..j + +) ( j . Again. a more efficient structure can be obtained by exploiting the periodicies in the rectangular matrices 12 and ' T 2 and by replacing them with M X M cosine modulation matrices T 1 and -T --l T 2 = T .108) is depicted in Figure 6. M . and since (6. = T.178 and to construct the new filters as Ui+l(Z) Chapter 6. (6. This already suggests the joint implementation of pairs of two filters. . On the analysis side. we see that always those two systems are fed with the same input signal which are connected in (6. This structure is depicted in Figure 6. . PR is structurally guaranteed.130) Thus.' .1.. Ui+l(z) and Vi+l(z) have the structure given in (6.24.129) for k = 0. .121).120) is satisfied.23. all polyphase filtering operationscanbe carried out via the lifting scheme described above where four filters are realized jointly.M-l ]= (6.1 M 2 = 2~0~[$F(k+i)(M+j-g)+4kj. Also other lifting schemes can easily be found. filter optimization can becarried out by optimizing the lifting coefficients in an unrestricted way.116)... In the synthesis bank.. Thus.g) + q5k] . . j = 0. . Note that the following signals are needed as input signals for the cosine transform: (6. However.. The advantageof the above approach is that only one lifting step with one lifting coefficient ai or ci is needed in order to increase the length of two polyphase components of each prototype. Implementation Issues.

that is. which leads to the following constraints on the prototype: (6. The prototype has tosatisfy P (z)P/C(. The same prototype is required for both analysis and synthesis.n) = p ( n ) .l + h k + k (Z)PM+k = 1 z .131) 1. The prototype has to linear-phase. (a) analysis.1. (6. 3.6.132) .2 Paraunitary Case In the paraunitary case with critical subsampling we have B(z)E(z)= I M . Cosine-modulated filter bank with critical subsampling.Filter Cosine-Modulated 179 (b) Figure 6. p ( L . (b) synthesis. 6.6.23.Banks 6. be 2.

23 and implement two filters at a time via the lattice in Figure 6. Filter (-1)S-l YM-1 (m) - M-k M-l li L.24. the more efficient structure in Figure 6. . Onecan use the structure in Figure 6.25 and choosing the rotationangles so as to minimize an arbitraryobjective function. d and efficient Figure 6. where four filters are realized via a common lattice. For this method a good starting point is required. A generalization is given in [62]. the polyphase filters can be realized jointly. Cosine-modulated filter bank with critical subsampling implementation structure. (a) analysis. The filter design may for instance be carried out by parameterizing the polyphase components using the lattice structure shown in Figure 6. becausewe have to optimize angles in a cascade of lattices and the relationships between the angles and the impulse response are highly nonlinear. which typically is less sensitive to the starting point. This was shown in [95] for special filter lengths.24 can also be used. As in the biorthogonal case. Alternatively. However. the QCLS approach [l111can be used. (b) synthesis.180 Banks Chapter 6.25.

The frequency responses of the filters #3 and #S are depicted in Figure 6.135) . For filter length L = 2M and L = 4M closed form solutions for PR prototypes are known.132) reduces to Pk which means +PM+k = 1 . The special case L = 2M is known as the modulated lapped transform (MLT).Banks 6. The filter design is carried out iteratively.134) The case L = 4M is known as the extended lapped transform (ELT). (6. The design procedure is basedon a subspaceapproach that allowsus to perform linear combinations of P R prototype filters in such a way that the resulting filter is also a linear-phase P R prototype.m 2 [ 1 + -)-7 r 2 2M] (6. Because of symmetry. In this case the PR condition (6. The ELT was introduced by Malvar. Closed Form Solutions.26. only the first 16 coefficients are listed.6. Table 6. In order to give some design examples.Filter Cosine-Modulated 181 pk (z) / \ / A A \ In [l031 a method has been proposed that allows the design of discretecoefficient linear-phase prototypes for the paraunitary case. who suggested the following prototype [95]: p(n) = -GZ 1 1 cos (n +. (6.133) is P(n) = 1 m sin [ .133) An example of an impulse response that satisfies (6.1 shows impulse responses of 8-band prototypeswith integer coefficients and filter length L = 32. p 2 ( n )+ p 2 ( M + n) = 21 ~ ( n + -)- . while the PR property is guaranteed throughout the design process. which was introduced by Princen and Bradley [116].

Perfect reconstruction prototypes for 8-band filter banks with integer coefficients &(L . Filter Banks 0 0.4 0.1 0. (a) filter #3. Table 6.2 0.5 Normalized Frequency (b) Figure 6. (b) filter #6.26. Frequency responses of 8-channel prototypes from Table 3.1 0.2 0.4 0.182 Chapter 6. For comparison the frequency response of the ELT prototype is depicted with dotted lines.3 0.3 0.1.1.n ) = p ( n ) ) .5 Normalized Frequency (a) 0 0.1. r 7 8 10 11 12 13 14 1 #3 -1 -1 0 0 0 0 2 2 4 4 6 6 7 7 8 8 #S -2190 -1901 -1681 -426 497 2542 3802 6205 9678 13197 16359 19398 22631 24738 26394 27421 1 1 1 1 1 2 2 2 2 .

. Requiring ) ( . .Filter Cosine-Modulated 183 6.138) }. As we see. . .pe~+1(-z2’).1. . .140) for perfect reconstruction yields [86] (6. . Qe(z2’) = diag{Qelv+(lv-l)(-z2’). . &) ) ( .6. Q ~ N + I ( . .the polyphase (6. p . ’).142) for L = 0. these conditions offer increased design freedom for an increasing oversampling rate. . . (6.144) and the overall delay amounts to q=N-l+qt)N. The delay qp) is related to S as qp) = 2ps + 2p(6. This is further discussed in [86].N .6. .3 OversampledCosine-Modulated Filter Banks E Inthe oversampled case withoversampling by p = matrices may be written as Z. .141) and 9 + e l v ( z ) Q~+k+elv(z) P ~ + k + e l v ( zQk+elv(z) L 0 ) (6..-P (6.z ~ ’ )(6.143) . where solutions based on a nullspace approach are presented. .1.1.~ ) ( . = ..~ ~ Qelv(-z2’)}. (6.136) and with Pe(z2’) = diag (pelv(-z2’).139) The superscript ( p ) indicates the oversampling factor. l = 0. .P e l v + ( ~ .Banks 6.

we get the following constraints on the prototype P ( z ) [85. for p Q ( z ) such that > 1. . Anefficientdesign method was proposed in [166]. Designing a pseudo-QMF bank is done as follows [127].) and with Example. This demonstrates the increased design freedom in the oversampled case.Since the constraints on the prototype are less restrictive than in the PR case. but nearly perfect reconstruction. that is. We consider a 16-bandfilter bank with linear-phase prototype and an overall delay of 255 samples. Figure 6. one tries suppress in to the remaining aliasing components by using filters with very high stopband attenuation. One ensures that the aliasing components of adjacent channels compensate exactly. Through filter optimization the linear distortions are kept as small as possible.6.184 Banks Chapter 6. 6.28.27 shows a comparison of frequency responses for the critically sampled and the oversampled case. one no longer seeks perfect reconstruction. we still may choose different prototypes P(. as illustrated Figure 6.4 Pseudo-QMF Banks In pseudo-QMF banks. d p ) ( z ) E ( p ) ( z )= I N . the prototypes typically have a higher stopband attenuation than the P R ones. It turns out that the PR prototype for the oversampled filter bank has a much higher stopband attenuation. Furthermore. Filter If we restrict an oversampled cosine-modulated filter bank to be paraunitary. This requires power complementarity of frequency shifted versions of the prototype. 861: Interestingly.

2 0..... . . .. . . . (b) oversampling by p = 2. ... . . . . . .. . . . 0 ' HiM Figure 6.. . . .... Design of pseudo-QMF banks. . .4 I . ..... .... . .. . .28. .27. . . .. . . . . .. . .. . . . . . . .... .6 0.. . . . . . . . .. . . .. . . . ..6. . ..4 n/rC (b) 0. .. . . . .. . . .. . . . .2 0. . . (a) critical subsampling. .. ... ... . ...8 1 0 . . . ... . .. . .. . -20 -40 -60 - g -80 -100 -120 -140 -160 I 0 \:::: 0.. .. .. Cosine-Modulated Filter Banks 185 0 .... :.... . .. . . . Frequencyresponses of 16-channel prototypes.6. .. .... I I n/rC (4 0. . . . .. . . .6 0.. . ...8 1 Figure 6. ... . .. ..... . . .. .. . . . ..4 0 0.. . .

an overlap of one blockis considered. P1 = P PT = O M x M . Note that B is orthogonal if PO and P1 satisfy the above conditions. which better allow us to smooth out blocking artifacts in coding applications. biorthogonal lapped transforms have also been proposed [96.148) P.146) (6. which means that the basis functions are of length L = 2M when the number of channels is M. T ~ T = T T ~ = I it follows that P. Define A = POP: and verify that PlPT = I . Unlike block transforms. 97.A ] B . Filter 6.149) .29 illustrates the structure of the transform matrix of a lapped orthogonal transform. More recently. O P1 = [ I . 1441. 21.186 Banks Chapter 6. Figure 6. Thus. O Now let B = PO P I . We will first consider the constraints on the M X M submatrices POand P I . Moreover. From the condition of orthogonality. LOTs may also be seen as a special kind of critically subsampled paraunitary filter banks. 2M input samples are combined in order to form M transform coefficients. Typically. (6. + P =AB.Like in an M-channelfilter bank with length-2Mfilters. POP: and P I P : are orthogonalprojections onto two subspaces that are orthogonal to one another. Transform matrix of a lapped orthogonal transform.29.A. they have overlapping basis functions.147) (6.TPo+PTP1=PoPoT+PIPT=IMxM and (6. Figure 6.7 Lapped Orthogonal Transforms Lapped orthogonal transforms (LOTs)were introduced in [21] and have been further studied in [93. Longer transforms have been designed in [l181 and are called generalized lapped orthogonal transforms (GenLOTs).

Orthogonal Lapped 6.7.

Transforms

187

The most general way of constructing LOTS is to start with two matrices A and B , where A is a projection and B is orthogonal. The desired matrices P O and P1 are then foundfrom(6.149).Thismethod, however, does not automatically yield linear-phase filters, which are desired in many applications. In [98], a fast linear-phase LOT based on the DCT was presented, which will be briefly explained in the following. For this, let D e and Do be matrices that contain the rows of the transposed DCT-I1 matrix with even and odd symmetry, respectively. Then,

is a LOT matrix that already satisfies the above conditions. J is the counter identity matrix with entries Ji,k = & + - l , i = 0,1, . . . ,N - 1. In an expression of the form X J , it flips the columns of X from left to right. Due to the application of J in (6.150), the first M / 2 rows of Q(') have even and the last M / 2 rows have odd symmetry. A transform matrix with better properties (e.g. for coding) can be obtained by rotating the columns of Q(') such that

Q = 2 Q('),

(6.151)

where 2 is unitary. For the fast LOT, 2 is chosen to contain only three plane rotations, which help to improve the performance, but do not significantly increase the complexity. The matrix Q(o)already has a fast implementation based on the fast DCT. See Figure 6.30 for an illustration of the fast LOT. The angles proposed by Malvar are O1 = 0 . 1 3 ~ ~= 0 . 1 6 ~ ~ O3 = 0 . 1 3 ~ . O2 and
0

1
2 3 4 5 6 l

Figure 6.30. The fast lapped orthogonal transform for M = 8 based on the DCT

and three plane rotations.

188

Banks

Chapter 6. Filter

6.8

Subband Coding of Images

Two-dimensional filter banks for the decomposition of images can be realized as separable and non-separable filter banks. the sake of simplicity, we will For restrict ourselves to the separable case. Information on non-separable filter banks and the corresponding filter design methods is given in [l,1541. In separable filter banks, rows and columns of the input signal (image) the are filtered successively. The procedure is illustrated in Figure 6.31 for an octave-band decomposition based cascades of one-dimensional two-channel on filter banks. In Figure 6.32 an example of such an octave-band decomposition is given. Note that this decomposition schemeis also known as the discrete wavelet transform; see Chapter 8. In Figure 6.32(b) we observe that most information is contained in the lower subbands. Moreover, local highfrequency information is kept locally within the subbands. These properties make such filter banksvery attractive for image coding applications. In order to achievehighcompression ratios, one quantizes the decomposed image, either by scalar quantization, or using a technique known as embedded zerotree coding [131, 1281; see also Section8.9. The codewords describing the quantized values are usually further compressed in a lossless way by arithmetic or Huffman coding [76,63]. To demonstrate characteristics of subband coding the with octave-band filter banks, Figures 6.32(c) and (d) show coding results at different bit rates.

Fl*FI*Fl* Fl*F
LH HH
LH HH LH HH

vertical

...
Figure 6.31. Separable two-dimensional octave-band filter bank.

6.9. Processing of Finite-Length Signals

189

Figure 6.32. Examples of subband coding; (a) original image of size 512 X 512; (b) ten-band octave decomposition; (c) coding at 0.2 bits per pixel; (d) coding at 0.1 bits per pixel.

6.9

Processing of Finite-LengthSignals

The term “critical sampling”, used in the previous sections, was used under the assumption of infinitely long signals. This assumption is justified with sufficient accuracy for audioand speech coding. However, if we want to decompose an image by means of a critically subsampled filter bank, we see that thenumber of subband samplesis larger than thenumber of input values. Figure 6.33 gives an example. Ifwe simply truncate the number of subband samples to the number of input values - which would be desirable for coding - then PR is not possible any longer. Solutions to this problem that yield PR

190

Chapter 6. Filter Banks

Figure 6.33. Two-channel decomposition of a finite-length signal.

with a minimum number of subband samples are discussed in the following. Circular Convolution. Assuming thatthelength of the signal to be processed is a multiple of the number of channels, the problem mentioned above can be solved by circular convolution. In this method, the input signal is extended periodically prior to decomposition [165], which yields periodic subband signals of which only one period has to be stored or transmitted. Figures 6.34(a) and 6.34(c) give an illustration. Synthesis is performed by extendingthesubband signals according to theirsymmetry, filtering the extended signals, and extracting the required part of the output signal. A drawback of circular convolution is the occurrence of discontinuities at the signal boundaries, which may lead to annoying artifacts after reconstruction from quantized subband signals. Symmetric Reflection. In this method, the input signal is extended periodically by reflection at the boundaries as indicated in Figures 6.34(b) and 6.34(d), [136, 16, 23, 61. Again, we get periodic subband signals, but the period is twice as long as with circular convolution. However, only half a period of the subband signals is required if linear-phase filters are used, because they lead to symmetry in the subbands. By comparing Figures 6.34(a) and (b) (or 6.34(c) and (d))we see that symmetric reflection leads to smoother transitions at the boundaries than circular convolution does. Thus, when quantizing the subband signals, this has the effect of less severe boundary distortions. The exact procedure depends on the filter bank in use and on the signal

6.9. Processing ofSignals Finite-Length

191

(c) (4 Figure 6.34. Periodic extension of the input signal; (a) one-dimensional circular convolution; (b) one-dimensional symmetric reflection; (c) two-dimensional circular

convolution; (d) two-dimensional symmetric reflection.

length. Figure 6.35(a) shows a scheme suitable for the two-band decomposition of an even-length signal with linear-phase odd-length biorthogonal filters. The input signal is denoted as ZO, 2 1 , . . . ,27, and the filter impulse responses are {A,B , C, B , A } for the lowpass and {-a, b, - a } for the highpass. The upper row shows the extended input signal, where the given input samples are showninsolidboxes. The lowpass and highpasssubbandsamples, c, and d,, respectively, are computed by takingthe inner products of the impulse responses the displayed positions with the corresponding part of the in extended input signal. We see that only four different lowpass and highpass coefficients occur and have to be transmitted. A second scheme for the same filters which also allows the decomposition of even-length signals into lowpass and highpass components of half the length is depicted in Figure 6.35(b). In order to distinguish between both methods we say that the starting position in Figure 6.35(a) is even and the one in Figure 6.35(b) is odd, as indicated by the indices of the samples. Combinations of both schemes can be used to decompose odd-length signals. Moreover, these schemes can be used for the decomposition of 2-D objects with arbitrary shape. We will return to this topic at the end of this section. Schemesfor the decomposition of even-length signals with even-length

192
............ j............ X j X, ,

Banks

Chapter 6. Filter
X, X , , X XX , ,X ,

I X, I X,

X X, X X X X .......... ........ , , , , X, j X j X j , , ,

I I I I

.......... ........

..................
j.................. X jX j X , , ,

X,X,

............ X jX , , ............j

IAlB CIB A AIB C B A A BlClBlAl A BlClBlAl

IAlBlClB A IAlBlC B A A B C BlAl A BlClBlAl

m m
............
+.l..jd,.l c,

m
..................
:

: I d1 Cl I 4 I c2 I d2 I c3 I 4 I.~,.i.4 c o .:...z.:

..................

............

j..... . i . 4 . ~ . . l . l ~ o I ~ I I ~ l I ~ 2 I ~ 2 I ~ ; I ~ 3 I ~ 4 l . d ~ . i . c 2

(4
,........... .................. j............ X X, j X , , XI X X X X X6 X X j X j X : , ; , , , ................. I , , ,

I I

I I I I

.................. j.................. X j X j XI XI X X ; , , ;

X X , ,

X6 X X , ,

............. X j X j , , ..............

IAlBlBlA lAlBBIA A I BB l A l AlBlBlAl I a I b I -bl -a l a I bb l - a a l b -bl-al a Ib I-bl-al

Figure 6.35. Symmetricreflection for even-lengthsignals. (a) odd-length filters, segment starting at an evenposition; (b) odd-lengthfilters,segment starting at an odd position; ( c ) even-length filters, segment starting at an even position; (d)

even-length filters, segment starting at an odd position.

linear-phase filters are depicted in Figures 6.35(c) and (d). The filter impulse responses are {A,B , B , for the lowpass and {-a, -b, b, a } for the highpass. A} Note that a different type of reflection isused and that we haveother symmetries in the subbands. While the scheme in Figure 6.35(c) results in the same number of lowpass and highpass samples, the one in Figure 6.35(d) yields an extra lowpass value, while the corresponding highpass value is zero. However, the additionallowpass samples can be turned intohighpass values by subtracting them from the following lowpass value and storing the differences in the highpass band. In object based image coding, for instance MPEG-4 [log], it is required to carry out subband decompositions of arbitrarily shaped objects. Figure 6.36

(c) highpass filter. the scheme yields a decomposition where the interiorregion of an object is processed as if the objectwas of infinite size. (d) and (e) lowpass and highpass subbands after horizontal filtering.36(f)-(i) finally show the object shapes after the vertical decomposition of the signals in Figures 6. Moreover. Thus.(f) and (g) lowpass and highpass decompositions of the signal in (d).36(d) and (e) based on the same reflection scheme.9. respectively. (a) arbitrarily shaped object and horizontal extension with pixel values as indicated. The 2-D decomposition is carried out in such a way that the horizontal decomposition introduces minimal distortion for the next vertical one and vice versa. Figures 6. Processing of Finite-Length Signals 193 Figure 6. Shape adaptive image decomposition using symmetric reflection for odd-lengthtwo-channelfilterbanks. (h) and (i) lowpass and highpass decompositions of the signal in (e).36. and the extension for the first horizontal decomposition is found outside this region. Such schemes are often called shape adaptive wavelet transforms. . the actual object shape only influences the subband samples close to the boundaries. Thearbitrarilyshapedinput signal isshown in the marked region. (b) lowpass filter. Note that the overall number of subband samples is equal to the number of input pixels. shows a scheme which is suitable for thistask usingodd-length filters. Figures 6.6.36(d) and (e) show the shape of the lowpass and highpass band.

37(d)and (e) show the shapeof the lowpass and highpass band after horizontal filtering.36 for further explanation. Note that in this case. Methods for this task have been proposed in [70.37(d) indicate the extra lowpass samples. 27. 101. one can optimizethe boundary processing schemes in order to achieve better coding properties. If the faster growing of the lowpass band is unwanted the manipulation indicated in Figure 6.36. In addition to thedirect use of symmetric reflection. 69. 39.194 l 0 0 1 2 3 4 5 6 7 8 9 9 l 0 0 1 2 3 4 5 6 7 8 8 7 3 2 1 1 2 3 4 5 6 7 8 8 7 Chapter 6.37(e) are positions where the highpass samples are exactly zero.37.37. see the comments to Figure 6. The shaded regions in Figures 6. Shape adaptive image decomposition using symmetric reflection for even-length two-channel filter banks. The zero-marked fields in Figure 6. the lowpass band grows faster than the highpass band.35(d) can be applied. 401. A scheme for the decomposition of arbitrarily shaped 2-D objects with even-length filters is depicted in Figure 6. These include the two-band. Filter Banks Figure 6. and the paraunitary case with non-linear phase filters. the more general M-band. 102. Then the subbands obtainedwith even-length filters will have the same shape as the ones in Figure 6. . The brighter regions within the object in Figure 6.

.1.. perfect reconstruction of the input datawith a delay of m samples can be obtained when the following condition holds: 0 ti. (6. and additive noise need to be considered in the transmultiplexer design. Transmultiplexer filter bank. An elaborate discussion of this topic is beyond the scope of this section. k (m)= q i . the synthesis filter bank is applied first and the analysis filter bank is then used to recover the subband samples yk(m). This essentially means that any P R subband codingfilter bank yields a P R transmultiplexer if the overall delay is a multiple of M . The transmission from input i to output k is described by the impulse responses t i . .154) Using the notationof modulation matrices these conditions may be written PR as T ( z M )= H . ( z ) G . which may be understood as components of a TDM signal.S . Essentially. .k(rn) = dik. M . ( z ) = M z-. crosstalk between different channels. Practical problems with transmultiplexers mainly occur due to non-ideal transmission channels. At the output of the synthesis filter bank we have an FDM signal where each data stream yk(m) covers a different frequency band. .. (6.10 Transmult iplexers Transmultiplexers are systems that convert time-division multiplexed (TDM) signals into frequency-division multiplexed (FDM) signals and vice versa [151]. Transmultiplexers 195 6. Contrary to the subband coding filter banks considered so far.10. (6. k = 0.153) In the noise-free case.38.152) (6.OM I .38.155) where the overall transfer matrix depends onz M . i . Thismeans that intersymbol interference.1.6. these systems are filter banks as shown in Figure 6.. Figure 6. k ( m M ) .

which clearly assigns time to frequency.1 Continuous-Time Signals Definition The short-time Fouriertransform (STFT) is the classical method of timefrequency analysis. see Figure 7. We multiply z ( t ) . Time-Frequency Transforms and Applications. because it does not allow an assignment of spectralcomponents to time. Therefore one seeks other transforms which give insight into signal properties in a different way.Signal Analysis: Wavelets. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Chapter 7 Short-Time Fourier Analysis A fundamental problem in signal analysis is to find the spectral components containedin a measuredsignal z ( t ) and/or to provide information about the time intervals when certain frequencies occur. with an analysis window y*(t .which is to be analyzed. An example of what we are looking for is a sheet of music. or in other words. It involves both time and frequency and allows a time-frequency analysis. The short-time Fourier transform is such a transform. The classical Fourier analysis only partly solves the problem. The concept is very simple. a signal representation in the time-frequency plane.T) and then compute the Fourier 196 .1 7.Filter Banks. 7.1.1.

Typically. As we see from the analysis equation (7. As we will see later. one will choose a real-valued window. the following derivations will be given for the general complex-valued case.T) suppresses z ( t ) outside a certain region. .1). because Gabor introduced the short-time Fourier transform with this particular window [61].2. do not necessarily have this property.t o ) leads to a shift of the short-time Fourier transform by t o .1. Moreover. we speak of the Gabor transform. such as the discrete wavelet transform. Nevertheless. transform of the windowed signal: cc F ~ (W )..-jut -cc dt. . Short-timeFouriertransform. other transforms. Ifwe choose the Gaussian function to be the window. Shift Properties. The analysis window y*(t . Figure 7. a modulation z ( t ) + z ( t ) ejwot leads to a shift of the short-time Fourier transform by W O . and the Fourier transform yields a local spectrum.2 illustrates the application of the window. a time shift z ( t ) + z(t . which may be regarded as the impulse response of a lowpass.T) . Figure 7. T = J z ( t )y * ( t . Continuous-Time 197 I Figure 7 1 Time-frequency representation.Signals 7.

1.W ) e-j(v - From Parseval's relation in the form J -03 we conclude That is. W +WO + A. . W +WO +A. Short-Time Fourier Analysis 7.W ) gives information on a signal( t )and its spectrum z X ( w ) in the time-frequency window [7+t0 -At .7) respectively. Let us assume that y*(t .j ( v (7.r ) simultaneously leads to windowing in the spectral domain with the window r*(v.198 Chapter 7.r ) ejwt ) t r7.3. The form of the time-frequency window is independent of r and W . ( t:= ~ (.W ) .2 Time-Frequency Resolution Applying the shift and modulation principle of the Fourier transform we find the correspondence ~ ~ .]. (7.A. so that we obtain a uniform resolution in the time-frequency plane. . The position of the time-frequency window is determined by the parameters r and W .7) e. as indicated in Figure 7. . .W ) are concentrated in the time and frequency intervals and [W + W O .r ) and r*(v .Wk) := S__r(t. windowing in the time domain with y * ( t . Then Fz(r.r+to +At] X [W + W O -A.].2) - w)t dt = r ( v .

. The center t o and the radius A.m * z1 z2 . to poor frequency resolution. . . However. . giving a lower bound for the area the window.. Intime-frequencyanalysisoneintends to achieve both high timeand frequency resolution if possible.. . . a long time window yields poor time resolution. .7. In other words. the center WO and the radius A... . W 2 .11) The radius A.3. . .1. ~ t to-& to to+& t z (4 (b) Figure 7. of the time window y*(t) are defined analogous to the mean value and the standard deviation of a random variable: Accordingly... Continuous-TimeSignals A 199 O A .10) r * ( w ) are defined as (7. .. may be viewed as half of the bandwidth of the filter y*(-t). Correspondingly. . . the uncertainty principle applies. . Let us now havea closer look at the size and position of the time-frequency window.. Choosing a short time of window leads to good time resolution and. Time-frequency window of the short-time Fourier transform. On the other hand. Basic requirements for y*(t) tobe called a time window are y*(t)E L2(R) and t y*(t) L2(R). of the frequency window (7. . we demand that I'*(w) E L2(R) E and W F*( W ) E Lz(R) for I'*(w) being a frequency window. . . . one aims at a time-frequency window that is as small aspossible. but good frequency resolution. inevitably. .

Without loss of generality we mayassume J t Ir(t)12 = 0 and S W l r ( w ) I 2 dw = 0.11) we have For the left term in the numerator of (7. Short-Time Fourier Analysis 7.200 Chapter 7.13) J -cc with [ ( t )= t y ( t ) .12) m cc w2 lP(W)I2 dw = ~ m l F { r ’ (l2 ) } t dw = %T (7. With (7. we may write (7.18) .16) as t y ( t ) y’*(t) d t } = i/_”.1.13).12).14) and 111112 = 27r lly112 we get for (AtA.1 1 t 1 1 2 ll-Y114 Applying the Schwarz inequality yields 1 ll-Y‘1I2 (7.t g Ir(t)12 dt. (7.16) By making use of the relationship 8 { t y ( t )y’*(t)} = 5 t 1 d Ir(t)12 7 (7. the right term in the numerator of (7.15) (AtAJ2 2 &f I (t. we may write the integralin (7.whichis the square of the area in the time-frequency plane beingcovered by the window.Using the differentiation principle of the Fourier transform.17) which can easily be verified.-Y’)l l2 (7.(7.3 The Uncertainty Principle Let us consider the term (AtAw)2.9) and (7. With (7.12) may be written as L (7.14) llY‘ll2 where y’(t) = $ y ( t ) .-Y’) l2 2 &f lR{(t.)2 = . becausethese dt properties are easily achieved for any arbitrary window by applying a time shift and a modulation.

25) Hence. It shows that the size of a time-frequency windows cannot be made arbitrarily small and that a perfect time-frequency resolution cannot be achieved. 7 1 4 The Spectrogram .21) so that we may conclude that 2 that is AtAw 2 1 4’ 5’ 1 (7.23) isachieved only if y ( t ) is the Gaussian function. implies that (7.7. In other words.23) The relation (7. Continuous-TimeSignals 201 Partial integration yields The property lim t Iy(t)12= 0. we often use the so-called spectrogrum for display purposes or for further processing stages.23) is only given if t y ( t ) is a multiple of y’(t). 1tl-w (7.the general solution with a time-frequency window of minimum size is a modulated and time-shifted Gaussian..1. equality in (7. y(t) must satisfy the differential equation t d t )= c whose general solution is given by (7. In (7. This is the squared magnitude of the short-time Fourier transform: .24) (7.16) we see that equality in (7.23) is known as the uncertainty principle. If we relax the conditions on the center of the time-frequency window of y ( t ) .22) (7. Since the short-time Fourier transform is complex-valued in general.20) which immediately follows from t y ( t ) E La.

(a) test signal.5. the values S (7.28) .) is possible in the form W (7. Each striation correspondsa single pitch period. . Reconstruction A reconstruction of z(t) from FJ(T. The resonance frequencies are known as the formantfrequencies. 715 . The uncertainty of the STFT in both time and frequency can be seen by comparing the result in Figure 7.202 Chapter 7. to A high pitch is indicated by narrow spacing of the striations. (b) ideal time-frequency representation.4(b).5. Fricative or unvoiced sounds are shown as broadband noise. Figure 7. Resonances in the vocal tract in voiced speech show up as darker regions in the striations.W ) are represented by different shades of gray. Short-Time Fourier Analysis v v v v v v v U UYYYYYYYYIVYYYYYYYV v v v v v v v v t" I Figure 7.. We see three of them in the voiced section in Figure 7. A second example that shows the applicationin speech analysis is pictured in Figure 7.4(c) with the ideal time-frequency representation in Figure 7.4 gives an example of a spectrogram.4. (c) spectrogram. The regular vertical striations of varying density are due to the pitch in speech production. Example of a short-time Fourieranalysis.

6 ( t .r ) e-iwt‘ dt’ g ( t .T) 6 ( t .29) = / x ( t ‘ ) / y * ( t ‘ . (b) spectrogram.1) into (7.Signals 7.r ) g(t .t’) = L 00 y*(t’ . Spectrogram of a speech signal. so that an infinite number of windows g ( t ) can be found which satisfy (7.r ) ejwt d r dw ejw(t-t’) dw d r dt‘ (7.27) is of course that the complete short-time spectrum must be known and must be involved in the reconstruction. . We can verify this by substituting (7.t’) dr dt’.29) to be satisfied.30) must hold.28).28) is satisfied.T) 27r = /z(t’)/y*(t’ .T) g ( t . For (7.r ) 6 ( t .28)is not very tight. The disadvantage of (7. which is true if (7.27) and by rewriting the expression obtained: z(t) = L / / / z ( t ’ ) 27r y*(t’ . Therestriction (7. Continuous-Time @l t - Figure 7.1.5.T) g ( t . (a) signal.t ’ ) d r (7.

....... However. . .... as with Shannon’s sampling theorem.. Short-Time Fourier Analysis 7... (7...mT) y * ( t . . Ic E Z of the short-time Fourier I transform are nothing but the coefficients of a series expansion of x ( t ) . W . Thus.m. . . the signal representation is redundant. * TM.32) representsa linear set of equations for determining g ( t ) .1) represents aone-dimensional signal in the twodimensional plane. ..... This type of series expansion was introduced by Gabor [61] and is also called a Gabor expansion.... each of the synthesis functions covers adistinctarea of the timefrequency plane of fixed size and shape.. Perfect reconstruction according to (7. In (7. here. .... . 721 . ( m T . .32) is satisfied [72].. .... Sampling the short-time Fourier transform...mT - 2T e-) UA = de0 Vt (7.. ... .. For reconstruction purposes this redundancy can be exploited by using only certain regions or points of the time-frequency plane..... .... For a given window y ( t ) . . ... Reconstruction from discrete samples in the time-frequency plane is of special practical interest... ~ u A ) .204 Chapter 7.. ..1.... . . a minimal sampling rate must be guaranteed.. .. For this we usually choose a grid consisting of equidistant samples as shown in Figure 7...32) can be satisfied only for [35. . Reconstruction is given by 0 0 0 0 The sample values F ..... where de0 is the Kronecker delta. ....6......6 Reconstructionvia Series Expansion Since the transform (7..... 3 t - O/i Figure 7.. f .31) is possible if the condition - 2T w A m=-m c 00 g ( t .6. .... since (7. ...31) we observe that the set of functions used for signal reconstruction is built from time-shifted and modulated versions of the same prototype g@).......

2.37) (7. ( and 2Q) (7.Ic) = where c n ) .34) we must observethat the short-time spectrum a function of the is discrete parameter m and the continuous parameter W . Discrete-Time Signals 205 Unfortunately. 321 Fz(m. or A.j 2 ~ / M Synthesis.36) X ( m . Frequency W is normalized to thesampling frequency. signal reconstruction from discrete values of the spectrum can be carried out in the form g(. in practice one would consider only the discrete frequencies wk = 2nIc/M.1. that is for T W A = 27r. :. as in Section 7.k ) = F . As in (7. The analysis and synthesis windows are denoted as y* and g.34) n Here we assume that the sampling rate of the signal is higher (by the factor N E W) than the rate used for calculating the spectrum.39) k=O . (7. is infinite. 7. then either A. = ) (7.m N ) WGkn. It shows that it is impossible to construct an orthonormal short-time Fourier basis where the window is differentiable and has compact support.ejw) C x ( n )r*(n m ~e-jwn. ( X y*(n .31).7. in the following they aremeant to be discrete-time. 119. k = 0 . M -(7. Then the discrete values of the short-time spectrum can be given by X ( m .38) W M = e. and equal analysis and synthesis windows. for critical sampling.1) by a summation. .35) 1. In (7. .m N ) W E . . . . This relationship isknown as the Balian-Low theorem[36].Ic) g(. However.) = cc c c M-l m=-m X ( m . it is impossible to have both a goodtimeanda goodfrequency resolution.2 Discrete-Time Signals The short-time Fourier transform of a discrete-time signal x(n) is obtained by replacing the integration in (7. It is then givenby [4. If y ( t ) = g ( t ) is a window that allows perfect reconstruction with critical sampling. (7.

k ) = and c n ) .7 shows the realization of the shorttime Fourier transform by means of a filter bank.. .206 Chapter 7. signal analysis and synthesis can be carried out by means of equivalent bandpass filters. The windows g ( n ) and r(n) typically have a lowpass characteristic. which has beendefined as theFourier transform of a windowed signal. k ) W&? (7.36) as we see that the analysis can also be realized by filtering the sequence ) . Figure 7. with The analysis and synthesis equations (7. because then all PR conditions are satisfied for g(n) = dnO t G ( e J w ) 1 ) = and any arbitrary length-M analysis window ~ ( n ) $0) = l / M [4. As wewill see below. Alternatively.42) The synthesis equation (7. ( X r*(n - m) WE (7. can berealized with filter banks as well. the STFT may be understood as a DFT filter bank. The validity of ?(n)= z(n) provided y(0) = 1/M can easily be verified by combining these expressions. k = 0. Short-Time Fourier Analysis The reconstruction is especially easy for the case N = 1 (no subsampling).39) can be seen as filtering the short-time spectrum with subsequent modulation.40) M-l qn)= c X ( n . the reader is referred to Chapter 6. Regarding the design of windows allowing perfect reconstruction in the subsampled case. . The short-timeFourier transform. M .41) k=O This reconstruction method is known as spectral summation.39) then become X ( m .44) .1 (7. Realizations using Filter Banks. By rewriting (7. (7. The analysis equation (7.36) and (7. . ( X with the bandpass filters hk(lZ) = y*(-n) WGk". and by subsequent modulation.36) can be interpreted as filtering the modulated signals z(n)W& with a filter h(n) = r*(-n). 1191.

see Figure 7. which illustrates the close relationship between filter banks and short-time transforms. On the other hand. M - 1. Lowpass realization of the short-time Fourier transform.46) takes place. Several methods areavailable for reducing the effect of noise in a more or less optimal way. Spectral Subtraction based on the STFT 207 Figure 7.8. then filtering with the bandpass filters gk(n) = g ( n ) wi-kn. (7.7.45) m=-cc k=O shows that synthesis canbeachievedwithmodulated filters as well. in Chapter 5.5 optimal linear filters that yield a maximum signal-to-noise ratio were presented. However. We realize thattheshort-time Fourier transformbelongs to the class of modulated filter banks. The most efficient realization of the STFTis achieved when implementing it as a DFT polyphase filter bank as outlined in Chapter 6.3. Rewriting (7. it has been introduced as a transform. 7 3 Spectral Subtraction based on the STFT .39) as cc M-I --k(n-mN) (7. L = 0. For example. To accomplish this. linear methods are . first the short-time spectrum is modulated.7.. . . . Inmanyreal-wordsituationsoneencounters signals distorted by additive noise.

the denoised signal is given in the frequency domain as X ( w ) = IR(w)I L Y ( w ) .50) Inspectralsubtraction. the energy density may be written as (7. which is very well suited for the restoration of speech signals.208 Chapter 7. not necessarily the optimal ones. especially if a subjective signal quality with respect to human perception is of importance. while the phase is not attached. Spectral subtraction is a nonlinear method for noise reduction. Thus.51) . Bandpass realization of the short-time Fourier transform.48) + in the frequency domain. We start with the model where we assume that the additive noise process n(t) is statistically independent of the signal ~ ( t Assuming that the Fourier transform of y ( t ) exists. we ).oneonlytries to restorethemagnitude of the spectrum.E { I N ( w ) 1 2 ) * (7.8. (7. the least squares estimate for IX(w)I2 can be obtained as lX(412 = IY(w)I2 . Due to statistical independence between signal and noise. Short-Time Fourier Analysis Figure 7. have Y ( w )= X ( w ) N(w) (7.49) If we now assume that E { IN(w)12} is known.

the time dependence of the statistical properties of the signal and the noise process hasnotbeen considered. one replaces the above spectra by the short-time spectra computed by the STFT. Several methods for solving this problem and for keeping track of the time-varying noise have been proposed.one tries to keep track of the actual (time-varying) noise process. For more detail. So far. Spectral Subtraction based on the STFT 209 Keeping the noisy phase is motivated by the fact that the phase is of minor importance for speech quality.k ) . Therefore.k ) N ( m .52) + is where m is the time and k is the frequency index.k ) = X ( m .7.50) and (7. 51. but within intervals of about 20 msec. and the assumption of stationarity is valid on a short-time basis. Instead of subtracting an average noise spectrum E { I N ( w ) I 2 } . 60.51) are then replaced by l X ( m 7 k ) 1 2= IY(m. the signal properties do not change significantly. ) . this yields Y ( m . Assuming a discrete implementation.k)12 .p v ( G k ) l 2 (7.k ) l2 IN(m. Equations (7.54) where IN(m. (7.k) the STFT of Y (m).k)I2 > 0. k ) = I X ( m .k)I2 is the estimated noise spectrum. will be studied in Section 8. 50. k .k)l L Y ( m . note that a closely related technique. Y Since it cannot beassured that the short-time spectra satisfy I (m.3. Y(m. h .53) and X ( m . the reader is referred to [12. k h (7. V m . This can for instance be done by estimating the noise spectrum in the pauses of a speech signal. Finally. Speech signals are highly nonstationary. one has to introduce further modifications such as clipping. 491. known as wavelet-based denoising.10.

8. finds most of its applications in data analysis.1 The Continuous-Time Wavelt Transform The wavelet transform W. where it yields an affine invariant time-frequency representation. it has been applied to almost all technical fields including image compression.a) of a continuous-time signal x ( t ) is defined as Thus.[106]. This transform has excellent signal compaction properties for many classes of real-world signals while being computationally very efficient.the so-called wavelet.. Since then. various types of wavelet transforms have been developed. If we consider t)(t) to be a bandpass impulse response. and pattern recognition. Filter Banks. is the discrete wavelet transform(DWT). however. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Chapter 8 Wavelet Transform The wavelettransform was introduced at the beginning of the 1980s by Morlet et al. By varying the scaling 210 . and many other applications ha vebeen found. Therefore.@. Time-Frequency Transforms and Applications. the wavelet transform is computed as the inner product of x ( t ) and translated and scaled versions of a single function $ ( t ) . The most famous version.Signal Analysis: Wavelets. The continuous-time wavelet transform. denoising. then the wavelet analysis can be understood as a bandpass analysis. also called the integral wavelet transform (IWT). numerical integration. who used it to evaluate seismic data [l05 ].

where Q ( w ) denotes the Fourier transform of the wavelet.1. (small a) we have good time localization but poor frequency resolution. Obviously. $(t)must be a bandpass impulseresponse.3. both are naturally related to each other by the bandpass interpretation. On the other hand. That is. we have good frequencybut poor time resolution.1) can be seen as a convolution of z ( t ) with the time-reversed and scaled wavelet: The prefactor lal-1/2 is introduced in order to ensure that all scaled functions l ~ l .for low analysis frequencies. The Wavelet 211 parameter a the center frequency and the bandwidth of the bandpass are influenced. For the wavelet transform the condition that must be met in order to ensure perfect reconstruction is C. = dw < 00. The variation of b simply means a translation in time. wavelet analysis is often called a time-scale analysisrather than a a time-frequency analysis. the wavelet is analysis can be understood as a constant-& or octave analysis. Figure 8. it should be ensuredthat the signal can be perfectly reconstructed from its representation. When using a transform in order to get better insight into the properties of a signal. so that for a fixed a the transform (8.2) will be given in Section 8.2) the wavelet must satisfy Moreover. the time and frequency resolution of the wavelet transform depends on For high analysis frequencies a. While the STFT a constant bandwidth analysis. /a) R Since the analysis function $(t)is scaled and not modulated like the kernel of the STFT. Otherwise the representation may be completely or partly meaningless.~ / ~ $ * ( t with a E I have the same energy. a variation of the time delay b and/or of the scaling parameter a has no effect on the form of the transform kernel of the wavelet transform.Continuous-Time Transform 8.1 shows examples of the kernels of the STFT and the wavelet transform. However. . lQ(w)I must decrease rapidly for IwI + 0 and for IwI + 00. in order to satisfy (8. This condition is known as the admissibility condition for the wavelet $(t). the transform is named wavelet transform. The proof of (8. However. Since a bandpass impulse response looks like a small wave. As we can see.

5) With the correspondences X ( w ) t z ( t ) and Q ( w ) t $(t).212 Chapter 8. real wavelet) for high and low analysis frequencies...> (8. the integral wavelet transform introducedby equation (8. we obtain By making use of Parseval’s relation we finally get .and the time ) ) and frequency shift properties of the Fourier transform.1) can also be written as a) = ( X ’ ?h. the real part is shown) and the wavelet transform (bottom. WaveletTransform Figure 8 1 Comparison of the analysis kernels of the short-time Fourier transform . Calculation of the WaveletTransformfrom Using the abbreviation the Spectrum X ( w ) . (top.

The time window narrows when a becomes small. U ) undergoes no other modification. the wavelet representation Affine Invariance.7) statesthatthe wavelet transformcan also be calculated by means of an inverse Fourier transformfromthe windowed spectrum X ( w )Q*(aw). a a WO -+-l. W O ) and the radii A. and it becomes narrow when a becomes large.@.a) is scaled as well. z ( t / c ) ) . a short analysis window leads to good time resolution on the one hand. t o .11). a A. a long analysis window yields good frequencyresolution but poor time resolution. and A. except this. Time-Frequency Resolution.8) and (7. A t . buton the otherto poor frequencyresolution. This gives and (8. The center ( t o . thefrequency window becomes wide when a becomes small. A t ] X WO A. respectively. On the other hand.(b. A t . and it widens when a becomes large. of the window are calculated according to (7. W.}.10) (8. [---.a ) provides information on a signal ~ ( tand its spectrum ) X ( w ) in the time-frequency window [ b + a . + W O } and { a . This means that the wavelet transform W. For this reason we also speak of an .a .1) shows that if the signal is scaled ( z ( t )+ W. is independent of the parameters a and b.Continuous-Time Transform 8. it is determined only by the used wavelet $(t). t o + a .12) The area 4 A t A . The Wavelet 213 Equation (8.(b.2 illustrates thedifferent resolutions of the short-time Fourier transform and the wavelet transform.b + a . Inorder to describe the time-frequency resolution of the wavelet transform we consider the time-frequency window associated with the wavelet. As mentioned earlier. Figure 8. Accordingly.1. a (8.11) For the center and the radii of the scaled function @($) lalQ(aw) we have {ado. Equation (8. +A.

16) Hence. (8. = u~.a(W) =0 for w 0.13) Analytic wavelets have a certain property. In other words. Analytic wavelets are especially suitable for this purpose.214 0 Chapter 8.t o ) ) leads to a shift of the wavelet representation W z ( b .e. ~ * ( a w oej'ob. for an analytic wavelet: w z ( b .14) into (8. ~ ( tthe following holds: ) %. but W z ( b . (8.WO) + S(w + (S@ WO)] t ) x ( t ) = cos(w0t). which will be discussed briefly below. afine invariant transform.a w o )e-juob I.7) yields W&U) = 1 I 2 . a shift of the signal ( x ( t ) + x ( t . (8.14) Substituting X ( w ) according to (8.2. The spectrum is X ( w ) = 7r [S(w . For this consider the real signal z ( t ) = cos(w0t). for the Fourier transform of an analytic wavelet $ ~ b .la[. a ) = . the wavelet transform is translation invariant. [ q * ( a w o ) ejuob + ~ * ( .U ) undergoes no other modification.w0) + S(W + w 0 ) ) Q*(aw) ejwb dw (8. i.2 Wavelets for Time-ScaleAnalysis In time-scale signal analysis one aims at inferring certain signal properties from the wavelet transform in a convenient way. WaveletTransform W 0 2 Zl Z2 z Figure 8. Like an analytic signal. 8. Resolution of the short-time Fourier transform (left) and the wavelet transform (right)./~ -cc . ) 1 2 . Furthermore.15) + l a [ . a ) by t o . they contain only positive frequencies.

8. For this. respectively). Example. The complex wavelet most frequently used in signal analysis is the Morlet wavelet.16) depends on b. let us consider the Fourier transform of the wavelet. The magnitude of W. like spectrograms. Often W O 2 5/3 is taken to be sufficient [65]. The example considered below is supposed to give a visual impression of a wavelet analysis and illustrates the difference from a short-time Fourier analysis. a modulated Gaussian function: (8. (b.20) we get Q ( w ) 5 2. can be represented images in which intensity as is expressed by different shades of gray. does not vanish exactly: (8. which leads to Q ( w ) 5 10-5. so that the amplitude of z ( t ) can be seen independent of time. it contains .a) is independent of b. which. which is sufficient for most applications [132]. The Scalogram. We see that here analytic wavelets should be chosen in order to = visualize the distribution of the signal energy in relation to time and frequency (and scaling. the frequency of z ( t )can be inferred from the phase of W. The Morlet Wavelet. a ) .rrP (8. any horizontal line in the time-frequency plane can beconsidered. In order to show this. This means that the magnitude of W .2. However.2) only approximately.18) Note that the Morlet wavelet satisfies the admissibility condition (8.7 X 10-9 for W 5 0. A scalogram is thesquaredmagnitude transform: of the wavelet Scalograms. WaveletsAnalysis 215 for Time-Scale Since only the argument of the complex exponential in (8. 5 0.(b. The chosen test signal is a discrete-time signal. Figure 8. by choosing proper parameters WO and / in (8.18) 3 one can make the wavelet at least “practically” admissible. for W = 0.(b.19) By choosing WO 2 2. a ) directly shows the time-frequency distribution of signal energy.3 depicts scalograms for ~ ( t )d ( t ) .

216 Transform 8.8 the question of how the wavelet transform of a discrete-time signal can be calculated will be examined in more detail. Figures 8.4(d). This is not the case in the wavelet analysis represented in Figure 8.4(d).(a) real wavelet. but the localization of the impulses is poor. . 'In Section 8. which is well illustrated in Figure 8. Both the periodic components and the impulses are clearly visible. two periodic parts and two impulses.4(a). Wavelet Chapter Imaginary component (b) t - Figure 8 3 Scalogram of a delta impulse ( W s ( b . We see that for a very short analysis window the discrimination of the two periodic components is impossible whereas the impulses are quite visible.4(b) and 8. a ) = l$(b/a)I2).' An almost analytic. A long window facilitates good discrimination of the periodic component. The signal is depicted in Figure 8.4(c) show two corresponding spectrograms (short-time Fourier transforms) with Gaussian analysis windows. . (b) analytic wavelet.. sampled Morlet wavelet is used. is that it clearly indicates non-stationarities of the signal. Another property of the wavelet analysis.

Specifically. Examples of short-time Fourier and wavelet analyses. 8.3.4. (c) spectrogram (long window). (d) scalogram.1 Integral Reconstruction As will be shown.8. the inner product of two signals ~ ( t ) y(t) is related to and the inner product of their wavelet transforms as . (a) test signal. Integral and Semi-Discrete Reconstruction 217 log c L (4 t - Figure 8. two variants of continuous wavelet transforms will be considered. they onlydifferin the way reconstruction is handled. 8. (b) spectrogram (short window).3. we will look at integral reconstruction from the entire time-frequency plane and at a semi-discrete reconstruction.3 Integral and Semi-Discrete Reconstruction In this section.

218 with C, as in (8.2).

Chapter 8. WaveletTransform

Given the inner product (8.21), we obtain a synthesis equationby choosing
y t ( t ' ) = d(t' - t ) ,

(8.22)

because then the following relationship holds:
m

(X7Yt)=
J

f ( t ' ) d(t'
-m

- t ) dt' = z ( t ) .

(8.23)

Substituting (8.22) into (8.21) gives

From this we obtain the reconstruction formula

z ( t )=

'ScQ c,

/m

W z ( b 7 a )lal-; .JI t - b

(T)

da db

(8.24)

-cQ -cQ

Proof of (8.2) and (8.21). With
P,(W)

= X ( W ) !P*(wa)

(8.25)

equation (8.7) can be written as

W z ( b ,a ) = la13 27r
Using the correspondence P, ( W )
t )

-m

P,(w) ejwbdw.

(8.26)

p,(b) we obtain
(8.27)

Similarly, for the wavelet transform of y ( t ) we get
&,(W)

=Y(w) * ( w ~ ) Q

~a(b),

(8.28)

(8.29)

8.3. Integral and Semi-Discrete Reconstruction

219

Substituting (8.27) and (8.28) into the right term of (8.21) and rewriting the obtained expression by applying Parseval's relation yields

(8.30) By substituting W = vu we can show that the inner integral in the last line of (8.30) is a constant, which only depends on $(t):

da =
Hence (8.30) is

[ l

dw.
IWI

(8.31)

This completes the proof of (8.2) and (8.21).

0

8.3.2

Semi-Discrete Dyadic Wavelets

We speak of semi-discrete dyadic wavelets if every signal z ( t ) E Lz(IR,) can be reconstructed from semi-discrete values W , ( b , a m ) , where am, m E Z are dyadically arranged: a, = 2,. (8.33) That is, the wavelet transform is calculated solely along the lines W,(b, 2,):
cc

W,(t1,2~) 2 - t =

z ( t ) $*(2-,(t

- b))

dt.

(8.34)

220

Transform 8. Wavelet Chapter

The center frequencies of the scaled wavelets are (8.35) with
WO

according to (8.9). The radii of the frequency windows are
- = 2-m
am

A,

A,,

m E Z.

(8.36)

In order to ensure that neighboring frequency windows

and

do adjoin, we assume
WO

=3

Au.

(8.37)

This condition can easily be satisfied, because by modulating a given wavelet &(t)the center frequency can be varied freely. From (8.33), (8.35) and (8.37) we get for the center frequencies of the scaled wavelets:
wm = 3 * 2-m A,,

m E Z.

(8.38)

Synthesis. Consider the signal analysis and synthesis shown in Figure 8.5. Mathematically, we have the following synthesis approach using a dual (also dyadic) wavelet ( t ) :

4

~(t) =

m=-m

c
cc

cc

2-4m

W 3 C ( b , 2 m4(2-"(t - b ) ) db. )

(8.39)

In orderto express the required dual wavelet 4(t)by t)(t),(8.39) is rearranged

8.3. Integral and Semi-Discrete Reconstruction

221

as

~(t) =
=

m=--00

E E

2-im/-00 W,(b, 2m)4(2-m(t - b ) ) db
-cc

2-:m

(W“ ( . , 2 r n ) , 4 * ( 2 F ( t- . )))

m=--00

For the sum in the last row of (8.40)

m=-cc

c

q*(2mw) 6(2mw)

=1

(8.41)

m=-cc

If two positive constants A and B with 0 < A

5 B < cc exist such that
(8.43)

A5

m=-cc we achieve stability. Therefore, (8.43) is referred to as a stability condition. A wavelet +(t) which satisfies (8.43) is called a dyadic wavelet. Note that because of (8.42), for the dual dyadic wavelet, we have: 1
cc
m=-cc

c
cc

1Q(2mw)12 B 5

1

B-

(8.44)

Thus, for * ( W ) according to (8.42) we have stability, provided that (8.43) is satisfied. Note that the dual wavelet is not necessarily unique [25]. One may find other duals that also satisfy the stability condition.

222

Chapter 8. Wavelet Transform

2- T**(-t/z")

Wx(t,2")

2-3%~(t/~)

I

Figure 8.5. Octave-band analysis and synthesis filter bank.

Finally it will be shown that if condition (8.43) holds the admissibility condition (8.2) is also satisfied. Dividing (8.43) by W andintegratingthe obtained expression over the interval (1,2) yields:

Wit h (8.46) we obtain the following result for the center term in (8.45): (8.47) Thus (8.48) Dividing (8.43) by
-W

and integrating over (-1, -2) gives

A In2 5

Lcc

W

dw

5 B ln2.

(8.49)

Thus the admissibility condition (8.2) is satisfied in any case, and reconstruction according to (8.24) is also possible.

8.4. Wavelet Series

223

8.4
8.4.1

Wavelet Series
Dyadic Sampling

In this section, we consider the reconstruction from discrete values of the wavelet transform. The following dyadically arranged samplingpoints are used: a, = 2,, b,, = a, n T = 2,nT, (8.50) This yields the values W , (b,,, sampling grid. Using the abbreviation
a) ,

= W , (2,nT, 2,).

Figure 8.6 shows the

(8.51)
-

2 - f . $(2Trnt - n T ) ,

we may write the wavelet analysis as

The values {W, (2,nT, 2,), m, n E Z form the representation of z ( t ) with } respect to the wavelet $(t) and the chosen grid. Of course, we cannotassume that anyset lClmn(t),m, n E Z allows reconstruction of all signals z ( t ) E L2(R). For this a dual set t+&,,(t), n E Z m, must exist, and both sets must span L2(R). The dual setneed not necessarily be built from wavelets.However, we are only interested in the case where qmn(t) derived as is

t+&,,(t)= 2 - 7 *t+&(2-Y n T ) , -

m, n E

Z

(8.53)

from a dual wavelet t+&(t). If both sets $ m n ( t )and Gmn(t)with m, n E the space L2(R), any z ( t ) E L2(R) may be written as

Z span

(8.54)
Alternatively, we may write z ( t ) as

(8.55)

...... Then the functions lClrnn(t). that is.. A = B... ... . Z are linearly independent........... . n E m.. . relation........ ......... . It can be shown that the existence of a dual set and the completeness are guaranteed if the stability condition M M (8. . n E Z are highly redundant.. The question of whether a dual set Gmn(t) exists at all can be answered by checking two frame bounds' A and B ........ .. ... . the values W ........ n E Z are linearly dependent...... . is This is also true if the samples W ... we have the orthonormal wavelets...... which is known as the biorthogonality condition....58) Wavelets that satisfy (8. ...... (2"nT....56) is also satisfied with 0 < A 5 B < CO.. . If T is chosen very small (oversampling)... .. perfect reconstruction with Gmn(t) = lClrnn(t) possible. bounds are.. ... ... .... For a given wavelet $(t).... . The tighter theframe m..... .. In the case of a tight frame.... If (8....... .... .. (2"nT.... .... 2").. ...... if the functions qmn(t)..... They are self-reciprocal and satisfy 2The problem of calculating the frame bounds will be discussed at theend of this section in detail...... Wavelet Chapter t log a ~ WO ......*...."- .. ...n E Z are linearly dependent. . m=-2 m=-l m=O m= 1 m=2 b - Figure 8.... the smaller is the reconstruction error if the reconstruction is carried out according to If T is chosen just large enough that the samples W ..... .. . n E Z contain no redundancyat all (critical sampling)...... . n E Z form a basis for L2 (R).. and an infinite number of dual sets qrnn(t) exists. . 2") contain redundancy. holds: (8... and reconstruction is very easy. . .. . 2")...........6... m.. As a special case.. Then the following m..58) are called biorthogonalwavelets.. (2"nT. the functions $mn(t).... ..* . m... ...*.. ..... m. the functions tjrnn(t).... Dyadic sampling of t h e wavelet transform.. .. .. .. ....the possibility of perfect reconstruction is dependent on thesampling interval T ...... . . ... .......*.. .. .56) with the frame bounds 0 < A I B < CO is satisfied [35].... ....224 Transform 8.. .

Sampling the wavelet transform can be further generalized by choosing the sampling grid am = a p . (8.. = k = 0. .60) Figure 8.64) and 3By “ess inf” and “ess sup” we mean the essential infimum and supremum. Wavelet Series 225 the orthonormality condition ($mn... M (8. This corresponds to M =a $ .. $(h)(t) $(a. Thus. m.56) as derived by Daubechies [35].. We here consider the case where the same sampling rate is used for all M subbands of an octave. This corresponds to a nesting of M dyadic wavelet analyses with the scaled wavelets q!I@)(t) 2A q!I(2Xt).56) is a special form of Parseval’s relation.63) (8..$t)7 IC = 0 ~ 1 .61) > 1. (8.59) m. Orthonormal bases always have the same frame bounds (tight frame).1.$lk) = &m1 b n k . Rather. in that case.4.n.62) For this general case we will list the formulae for the frame bounds A and B in (8.2 Better Frequency Resolution Octaves - Decomposition of An octave-band analysis is often insufficient. = am n T .4. 8. 1. in the orthonormal case... the functions q!Imn(t). IC E Z. The conditions for the validity of the formulae are:3 cc (8. with an arbitrary a0 the wavelets b. .7 shows the sampling grid of an analysis with three voices per octave.n E Z (8. we would prefer to decompose every octave into M subbands in order to improve the frequency resolution by the factor M . - nested wavelet analyses with .8. n E Z can be used for both analysis and synthesis. M - 1.1. (8. m. because.

... Sampling of the wavelet transform with three voices per octave. .. ..... ... . . . ....... ..... ... with If (8. WaveletTransform log-. .........226 Chapter 8.63) (8.... ..70) we finally have the following estimates for A and B: (8.....72) . ....67) (8. . ... . . .. the frame bounds A and B can be estimated on the basis of the quantities ~ (8.69) Provided the sampling interval T is chosen such that (8..71) (8...7...62). ..65) are satisfied for all wavelets defined in (8.... . . b - Figure 8..68) (8.... ... ... . ......... ....... ..

T . 91. (8. } mE Z.58). The Discrete Wavelet Transform ( D W T ) 227 8.n).76) W.79) . We will mainly consider the representation (8. 2 " ) = (2. = span {?)(2-"t . q. 8.78) n=-m Every signal z ( t ) E L2(R) can be represented as 00 (8. L 2 ( R ) may be understood as the direct sum of subspaces L2(R) = . .77) Each subspace W.90. n E Z. f ( 2 " n ..n E Z .73) are bases for &(R)satisfying the biorthogonality condition (8. 361. the derivations will be given for the more general biorthogonal case. For the subband signals we obtain from (8. Note that T = 1 is chosen in order to simplify notation.(n) = ~ .) .5. Mallat and Daubechies for theorthonormal case [104.n). .74): (8. m.55) and write it as with d..5.75) Since a basis consists of linearly independent functions. 341. n EZ .. @ W-1 €B W @ W1 €B. m . (8. covers a certain frequency band.8. O with (8. This framework has mainly been developed by Meyer.5 The Discrete WaveletTransform (DWT) In this section the idea of multiresolution analysisand the efficient realization of the discrete wavelet transformbasedonmultirate filter banks will be addressed.=-cc . (8.n). Since biorthogonal wavelets formally fit into the same framework [153.1 Multiresolution Analysis In the following we assume that the sets ?)mn(t) = 2-f ?)(2-79 ?jmn(t) = 2-f ? j ( 2 .

. (8.85) $mn(t) 2-%#j(2-.(t) E V.(t) by projection of x ( t ) E L2(R) onto the subspaces V this sequence converges towards x ( t ) : .(n) $mn(t) (8.84) Thus. . (8.80) we derive the following properties: (i) We have a nested sequence of subspaces .. .n). c V. Wm+1: m E Z as the direct sum of Vm+l and (8. (8.(t) = x ( t ) . z(t) E L2(R). From (8.77).80) V = Vm+l CE Wrn+l. .+m lim x. and (8.87) . z.86) Orthonormal Wavelets.t = The function +(t)is called a scaling function.n). Wavelet Chapter Nowwe define the subspaces V .t . } (8.$ W-1 1 1 $ W €B W1 €B O 1 1 . . (8. .-l c .(t) E V are expressed as . n ( t )= 2-?~(2-"t-n). .76). m. . n E Z form an orthonormal basis for L z ( R ) . any signal may be approximated with arbitrary precision. . n EZ .+l c v c v. the subband signals z. Here we may assume that the subspaces V contain lowpass signals and that .82) we may assume that the subspaces V are spanned by scaled and time-shifted versions of a single function $(t): . V = span {+(2-. then L 2 ( R ) is decomposed into an orthogonal sum of subspaces: L 2 ( R ) = . If the functions ~ .81) (ii) Scaling of z ( t ) by the factor two ( x ( t )+ x ( 2 t ) )makes the scaled signal z(2t) an element of the next larger subspace and vice versa: (iii) If we form a sequence of functions x. . zrn(t)= with n=-m c 00 c. . . . the bandwidth of the signals contained in V reduces with increasing m. (8. . Because of the scaling property (8.83) Thus.228 Transform 8.

Since the successive lowpass filtering results in an increasing loss of detail information.)} and {d. contain the high-frequency components of zo(t).90) If we assume that one of the signals x.y2 ( t ) . .(t). so that the decomposition is a successive lowpass filtering accompanied by separating bandpass signals. .8.n). .5.89) form orthonormal bases for the spaces V. .5 for 0. z1 ( t ) . The DiscreteWavelet Transform ( D W T ) 229 In this case (8.is known. . the sequences {cm(.n E Z. . then the functions $mn(t) 2-?4(2-.. yz(t).t = . m E Z. (8.etc. .+1 (t) + Y+ (t).5 5 t < 1 otherwise. The Haar function is the simplest example of an orthonormal wavelet: 1 -1 0 for 0 5 t < 0.80) we derive x (t) = 2. m.1 (8. Example: Haar Wavelets. .90): The signals y1( t ) .80) becomes an orthogonal decomposition: (8. Signal Decomposition. and since these details are contained in y1 ( t ) .(n)} for m > 0 may also be derived directly according to the scheme In the next section we will discuss this very efficient method in greater detail.. From (8. Assuming a known sequence { c o ( n ) } .88) If we assume 11q511 = 1. for example zo(t). we also speak of a multiresolution analysis (MRA). this signal can be successively decomposed according to (8..

5 n. see Figure 8. The corresponding scaling function is 1. .n E Z and +(2-jt . for n 5 IwI 5 2n.t 3n 7 -t. (8. +W-1) .8.94) @(W) { 1 for 0 5 IwI 0 otherwise. The functions +(t .n ) . . for 0 5 t < 1 0. .91) W J ) = 0 otherwise.n ) .T . m.8. . (8. n E Z span V and the functions +( ft . {1 3: = Chapter 8. The Shannon wavelets are impulse responses of ideal bandpass filters: +(t) = In the frequency domain this is sin . .95) . >t . .n ) . W ) .n ) . . . Furthermore. n Example: Shannon Wavelets. >t Figure 8. n E Z span the subspace W O .230 ~ . . n E Z span VI. The orthogonality among the 0 basis functions + ( 2 . Haar wavelet and scaling function.92) the impulse The scaling function that belongs to the Shannon waveletis response of the ideal lowpass: (8. cos 2 (8. otherwise. .n). j 2 m is obvious.n).93) (8. WaveletTransform '"I . the functions and E Z span WI.n). . + ( i t . . . . m. n E Z and the orthogonalityof the functions tj(2-Y . the functions $(t .

For the inner product of translated versions of +(t)at thesame scale.9.85) can be understood as the sample values of the ideally lowpass-filtered signal.n E Z in (8.5.96) by using Parseval's relation. The orthogonality of translated wavelets at the same scale is shown using a similar derivation.m)+*(t . Figure 8. Subspaces of Shannon wavelets. we get 00 +(t.n) = - 2. A drawback of the Shannon waveletsis theirinfinite support and the .8. m. The orthogonality between different scales is easily seen.9 illustrates the decomposition of the signal space.rr S" -" @(w)@*(w)e-J'(m-n)wdw (8. The Shannon wavelets form an orthonormal basis for Lz(lR.). The DiscreteWavelet Transform ( D W T ) 231 l -2n I I 1 - 2n W 1 -n n Figure 8. because the spectra do not overlap. The coefficients c m ( n ) .

8. but unsatisfactory frequency resolution.n) $lt(t) + h1 ( 2 -~ n ) $lt(t). Altogether.n) = C h o ( 2 L .85) for m = 0. we observed the opposite behavior. E Z can 0 1 n be written as linear combinations of the basis functions for the spaces V and 1 W1. l.97) is knownas thedecomposition relation. With the coefficients h o ( 2 l .n) $(t .98) We now consider a known sequence { c o ( n ) } .)}. thefrequency resolution is perfect. We get We see that the sequences {crn+l(l)} and {d.n) $(t .n ) . and we substitute (8. e (8. They had perfect time.n E Z the approach is 4on(t) = C h o ( 2 l . For the Haarwavelets.for which the following notation is used as well: & ( 2 t .n ) E VO. On the other hand.+l(l)} occurwith half the sampling rate of {crn(.5. Wavelet Chapter poor time resolution due to the slow decay. the decomposition(8.97) Equation (8.232 Transform 8.l). .2 Wavelet Analysis by Multirate Filtering Because of V = V @ W1 the functions $on@) = $(t .l) + h1(2c .97) into (8.100) is equivalent to a two-channel filter bank analysis with the analysis filters h0 ( n ) and h1 (n).n) and hl(2l . i 4 e (8.

101) and (8.8.5. Analysis filter bank for computing the DWT. n E Z in the form or equivalently as (8. 8. respectively. and if we know the coefficients co(n). which allow us to express the functions $lo(t) = 2-1/2$(t/2) E V and$lo(t) = 2-1/2$(t/2) E W1 as 1 linear combinations of $on(t) = $(t .10.(t) is a sufficiently good approximation of ~ ( t ) . and thus the values of the wavelet transform using the discrete-time filter bank depicted in Figure 8.10.102). we are able to compute the coefficients cm+1(n). The DiscreteWavelet Transform ( D W T ) 233 U Figure 8. For time-shifted functions the two-scale relation is (8.5. This is the most efficient way of computing the DWT of a signal. m > 0.n ) E VO.3 Wavelet Synthesis by Multirate Filtering Let us consider two sequences gO(n) and g1(n). If we assume that q. &+l ( n ) .103) .102) Equations (8. are referred to as the two-scale relation.

WaveletTransform From (8. (8.78).97).103).11.4 The Relationship between Filters and Wavelets Let us consider the decomposition relation (8. 8. (8. and (8. The filter bank is shown in Figure 8.5.105) describes a discrete-time two-channel synthesis filter bank.234 Chapter 8. that is .90) we derive The sequences gO(n) and g1(n) may be understood as the impulse responses of discrete-time filters.85) and (8.

108) yields (8.108) (8.106) with &(t) and qle(t) yields (8.112) .109) (8.8.111) (8.5.101) into (8.110) Substituting (8. The Discrete Wavelet Transform ( D W T ) 235 Taking the inner product of (8.

.(8. and W.2. 0 = Gl(z) H o ( z ) G1(-z) H o ( .112) and (8.By z-transform of (8.118) GI(-z). Hl(2) = Gl(2).).51)and (8. .) Go(-z).21) n and 2 = Go(. to (8. $1. cf. .113) Hl(-z). 1) (8.109) becomes h o w . + O k ) = dnk. Substituting the two-scale relation (8. Go(-z).) G o ( z ) +Go(-..z ) .116) Thus equations (8. If the sets $mn(t) ~ m n ( tm. 2 = Gl(z) HI(z) + GI(-z) HI(-z).114) yields + Observing (+On. 2 = G l ( z ) G ~ ( z ) G~(-z) GI(-z). Wavelet Chapter The conditions (8. 4 .(%) t ) Ho(z) = G o ( z ) .103) into (8. Orthonormal Wavelets.89)are orthonormal bases for V. (8.113) become (8. we derive ho(n) = g.114) h 1 W . 0 = Go(z) Hl(z) + Go(-z) (8.n) = @On.112) arenothingbutthe PR conditions for critically subsampledtwo-channel filter banks.formulated in thetimedomain.236 Transform 8.117) n 0 = C g 1 ( n ) go*(.(%) h1(n) = g. m E Z.112) we obtain 2 = Go(z) Ho(z) + Go(-z) Ho(-z). Section 6. n E Z according and ).n) = @On. 0 = 0 = + Go(z) G ~ ( z+ Go(-z) ) Gl(z) G o ( z ) + GI(-z) (8.

119) In the following the Fourier transform of equation (8. The starting point for constructing scaling functions is the first part of the two-scale relation (8. realize the transform via a multirate filter bank. we may introduce the normalization (8. (8. which. .122) n equation (8. using 1 W -jwn.119) is required.n).n ) t . One chooses the coefficients of a PR two-channel filter bank in such a way that the wavelets and scaling functions associated with these filters have the desired properties.121) (8. ) 2 2 is (8.6. Scaling Function.@(-) e 2 .6. as derived in Chapter 6.6 8.Go(e3T) W Jz (8.119) and (8. (8. we obtain (8. Due to theproperties of the wavelet transform we were able to show the existence of sequences h0 ( n ) h1 ( n ) go ( n ) . Wavelets 237 These are nothing but the requirements banks.and g1 ( n ) .from 8.124) If wenow apply (8.1 Waveletsfrom Filter Banks General Procedure In the previous sections we assumed that the wavelets and scaling functions are given.123) Since the scaling function +(t)is supposed to be a lowpass impulse response.125) . When constructing wavelets and scaling functions one often adopts the reverse strategy. for paraunitary two-channel filter 8.102): +(t)= c n go(n) d5 .which allow us to .121) is @(W) 1 ‘ W = .123) K times.120) + ( 2 t .

k=2 Jz (8. l Jz because we have specified @(O) = 1. Summary of Construction Formulae.126). However.12 illustrates therecursive calculation of the scaling function $(t).129) It is obvious that a smooth $(t) results in a smooth .128) which approaches the scaling function for i + CO. 0 otherwise. the synthesis scaling function is related to the synthesis lowpass as @(W) = k=l n Jz O01 . the convergence of the productdoes not guaranteethat the obtained scaling function is smooth. Figure 8.130) For the synthesis wavelet we get from (8. When starting with 1 for 0 5 t < 1.Go(ejw/2k) (8.n).G1(ejwI2) 1 Jz l W . it converges to c m * + CO to a (8.130) !@(W) = .119) allows us to determine the scaling function recursively. Thus (8.Go(e ) . Wavelet Chapter Nowwe let K + 00. If the scaling function $(t) is known.126) @(W) = k=l .$(t)= &1(n) n Jz $(2t . $(t) can be calculated by using the second part of the two-scale relation (8.131) . respectively.238 Transform 8. Wavelet.n ) . According to (8.125) converges for K continuous function.102): .127) we obtain the piecewise constant functions xi(t) by means of the recursion %+l ( t )= -Jz c n g o ( n ) xi (2t .regardless of the coefficients g 1 ( n ) . Figures 8.129) and (8. (8.13 and 8. If the product in (8. so that all concerns regarding smoothnessare related to the lowpass go (n).$(t).14 show examples leading to smooth and fractal scaling functions. (8. (8.Go(ejw/2k).

Thus. f}).12. theymay are be given in the frequency domain as (8. (-n) in the same way as $(t)and ~ ( t ) related t o go(. ${a. The analysis scaling function $(t)and the wavelet G(t)are related to the time-reversed and complex conjugated analysis filters h:(-n) and h.) and g1(n).132) and .6.the first two steps of the recursion are shown (coefficients: {gO(n)} = 1. Waveletsfrom Filter Banks 239 1 Figure 8.8. Recursive calculation of the scaling function q5(t).

13.240 Chapter 8.Exceptions aretheHaarand Shannon wavelets. of the scaling function 4(t) (coefficients 2t -2 O M 0 P L 2 E 4 I 0 2 t- 4 Non-Linear Phase Property of Orthonormal Wavelets. Recursive calculation { g o ( n ) } = ${l 3 3 l}). WaveletTransform 0 2 4 1 0 2 4 I 0 0 0b l 2 4 0 E 0 2 t t -4 - Figure 8.orthonormal wavelets have non-linear phase in general. . This property is transferred directly to the scaling functions and wavelets constructedwiththesefilters.Thus. In Chapter 6 we haveshown that paraunitary two-channel filter banks have non-linear phase filters in general.

is written n=-m c M $(t .3 Partition of Unity In order to enforce a lowpass characteristic of Go(z).135).134) f i n . (8. (8. (8.139) In the time domain. (8.137) n This means that the highpass filters in the two-channel filter bank must have zero mean in order to allow the construction of wavelets.6. 8. (8.124).135). it is useful to require Go(-l) = 0 t ) C(-l)ngo(n) n = 0.n) d(2t). Wavelets from Filter Banks 241 8.138) As will be shown in the following.129) we obtain and with (8.6.124) and J $ ( t ) dt = 0 we conclude (8.and (8. But. The correct scaling for the lowpass can be foundby integrating (8.6.2 Requirements to be Metby the Coefficients We havealready shown that to constructbiorthogonalandorthonormal scaling functions and wavelets the coefficients of PR two-channel filter banks are required.:.119): 00 $(t)dt This yields = Lc g o ( n ) / O O$(2t .140) . (8. in order to satisfy (8. which is known as the partition of unity.138) result in @(27rk) = { l .n) = 1. (8.0 0 c n go(n) = h. the coefficients must be scaled appropriately.8. this property of the scaling function.135) By integrating equation (8.

4 . $oo) by using (8. 0 8.242 Transform 8. For Ic = 0 . (8.6. = @(O) = 1.@(27r) k=3: k=4: =o. @ ( 3 ~ )= 0. .124) directly leads to (8.101) yields . and it turns out that (8. . @ ( 6 ~ ) = 0 . We may proceed in a similar way for the negative indices.140) with &.143) 141 = 1. we obtain k = 0 : @(O) k = 1 : @(27r) = O . =o.142) @(87~) = 1 . @ ( 7 r ) k = 2 : @(47r) = 1 ..141) = { f ( ~ k )k even.4 The Norm of Constructed Scaling Functions and Wavelets When the coefficients gO(n) belong to aparaunitary filter bank.@(4~) = 0. 2 . 3 . 11 We realize this by forming the inner product of (8. k odd.144) @(0)=1 (400:400) Forming the inner product ($oo. 1 . (8.139) holds.(t) and by making use of orthogonality: (8. Wavelet Chapter (8.

and. Most signals to be compressed are of a lowpass nature and can be well approximated locally by low-order polynomials.6.1.Assuming Q(0) = a leads to 11q511 = 11+11 = a.40) of the Fourier transform. As we shall see. Thus. Nq. In the biorthogonal case the relationship between the norm of the coefficients and the norm of the scaling function is much more complicated. polynomial signals of order Nq . it is useful t o seek wavelets with good approximation properties for low-order polynomials. Therefore. that is 00 tk $(t)dt = 0 for k = 0. . by the coefficients of the scaling function. the inner product of an analysis wavelet q ( t )having NG vanishing moments with a signal Nq -1 x(t)= k=O c ak tk is zero. vanishing moments.5 Moments Multiresolution signal decompositions are often carried out in order t o compress signals. the approximation propertiesof a multiresolution decomposition are intimatelyrelated to the number of vanishing wavelet moments. the wavelet has Nq. all wavelet coefficients are zero. .146) for the norm of the wavelet $(t). The kth moment of a wavelet $ ( t ) is given by (8.149) Clearly. Wavelets 243 which shows that 11+.6. . that is. .147) Using the property (2. 8.from 8. . so that the compaction properties of such decompositions are of crucial importance. 1 1 =1 (8. .1 are solely represented by the lowpass component. if Q ( w ) has NQ zeros at W = 0. (8. 1 .148) Thus. the moments can also be expressed as (8. consequently.

1 . equivalently.1. Note that according to (6. For the kth derivative of (ej") = h1 ( n )e-jwn (8.133): NG is given by the number of zeros of Hl(ej") at W = 0. Assuming that Go(. Further note that S(l) = 1 because of (8. 8.14 we saw that different filters may have completely different convergence properties.152) From this expression we see that if H l ( z ) has Nq zeros at z = 1. (8.1 are solely represented by the lowpass component. .Daubechies derived a test that can check the regularity and thus the convergence of the product in (8. Pointwiseconvergence of the functions zi(t) defined in (8.244 Transform 8.128) .) has N zeros at z = -1.151) C n we get (8. Typically.22).) can be written as N 1 2-1 GO(. . which should possibly have several continuous derivatives. In order to see this.154) 7 + Note that N 2 1 because of (8. N @.) = fi ( S ( z ) . Go(. . or. the number of vanishing moments of the synthesis wavelet is equal to the number of zeros of the analysis lowpass at z = -1. let us recall equation (8. by the number of zeros of H l ( z ) at z = 1. Similarly. Wavelet Chapter The number of vanishing moments is easily controlled when constructing wavelets from filter banks. .13 and 8. The discrete-time filters also have vanishing moments and corresponding approximation properties for discrete-time polynomial signals. one prefers smooth functions r$(t).6.6 Regularity In Figures 8. then h l ( n ) has NG vanishing moments in the discrete-time sense: E n e hl(n) = o for IC = 0 .so that we may alternatively say that Nq is given by the number of zeros of Go(z) at z = -1.153) n This means that sampled polynomial signals of order Nq .125) [34]. ) (8.137).135). H1 ( z ) is a modulated version of the synthesis lowpass Go(z).

(8.155) cannotbe satisfied because S(l)= 1. which can be expressed follows: if a scaling function is m-times continuously as differentiable and mth derivative its is Holder continuous of order a . then the function $(t)will also have continuous derivatives. Since the convergence is unique. Given a continuous function $(t). then (8. .l] for any arbitrary zo(t).6.) L-l %+l ( t )= Jz n O = c g o ( n ) Xi(2t .g o ( L .119): $(t)= h g o ( l ) $(2t .129) will be continuous for any sequence g I ( n ) .)I 171a vt.8.e). Precisely. $(t)is m-times continuously differentiable if (8. .155).7 Wavelets with Finite Support and g 1 ( n ) are FIR filters.l]: If go(. Wavelets from Filter Banks 245 towards a continuous function zoo@) $(t) is guaranteed if = sup IS(ej')l 05w527r < (8.158) Then. . again. This.6. The proofis straightforward. then its regularityis T = m a [125]. (8. L . One merely has to consider the iteration (8..157) + + c 8. If N is larger than theminimum number that is required to satisfy (8. respectively. The Holder exponent a is the maximum a for which I~'"'(t) .1) while assuming that x i ( t ) is restricted to the interval [0.156) Regularity isonly associated with the lowpass filters gO(n) and ho(n). if Go(z) hasno zero at z = -1..159) c e .n)..128) with the L coefficients go(O). L . based on the two-scale relation is (8.1.155) Clearly. The fact that the support known can be exploited to calculate the values is of $(t)at the times t. = n2rn. (8. function I+!I(t) the according to (8. then the resulting scaling functions and wavelets have finite support [34]. Rioulintroducedtheconcept of Holder regularity. HSlder Regularity. all recursively constructed functions are restrictedto 0 5 2t -n 5 L . . xm(t) = $(t) is restricted to [0.

By writing (8. conditions (8.160) we realize that we obtaintheintermediate values at each iterationstep. e e $($l Cgo(t) $(g -l).164) Since the columns of M containeither all even coefficients go(2n)or all odd coefficients go(2n l). .161) (8. the sum of all elements of the columns of M is one.159) as 4($) = = -Jz -Jz C g o ( t ) $@.4. . . . .160).135) and (8. Recalling (8.163) [M]ij:= -Jz go(2i . With m = [ $ P ) .j ) .246 Transform 8.m according to (8. (8. + . We conclude from (8.162) m=M. .140) it becomesobvious that we obtain the initial values by determining the right eigenvector m of M which belongs to the eigenvalue 1. l] with eigenvalue one. The initial values required can be determined exploiting the fact that theinitial values by remain unchanged during the iteration (8. Note. where the L - 1X L - 1 matrix M is given by (8. However.135) and (8.160).138) that the sum of the even coefficients equals the sum of the odd coefficients: (8.138) guarantee the existence of a left eigenvector [l. Wavelet Chapter Let us assume that the initial values $(n) are known. so far we only know the values $(O) = 0 and $ ( L ) = 0. . Thus.l)]' we get (8. 1. . $ ( L .

2 (8.) have to be and linear-phase.7 Wavelet Families Various wavelet families are defined in the literature.113).169) In order to yield linear-phase wavelets. depending on whether the filter length is even or odd. We consider an overall delay of 7.6in order to allow the construction of continuous scaling functions and wavelets.8. we consider the design of linear-phase biorthogonal wavelets according to Cohen. 8.7. (8. We will outline these properties for the filter Ho(z) start with odd-length and filters. On the unit circle.167) 8.6. For further design methods the reader is referred to [36.which completes a perfect reconstruction pair. We will only consider a few of those constructed from filter banks. Daubechies and Feauveau [28]. we may write H. The second filter Go ( z ) . the filters need to satisfy the regularity condition as outlined in Section 8.168) ) Go ( ej'" + Ho(ej('" + "1) Go(ej('" + "1) = 2 e-j"J7. Furthermore. start the discussion We with the first equation in (8. that is Ho(z) + Ho(-z) Go(z) Go(-z) = 2 2 7 .171) .(eju) = Jz (cos El2' ~ ( c o s w ) .170) zeros at where the delay ~h is aninteger. 1541. Odd-length linear-phase filters satisfy H o ( e j " ) = e-jwrh H.7. this means H W 0 (8. Wavelet 247 and we see that the eigenvalue 1 exists.(COSW).1 Design of BiorthogonalLinear-PhaseWavelets In this section. When expressing the linear-phase property. which is the PR condition for two-channel filter banks without delay. both filter Ho(z) Go(. (8. Assuming that HA(cosw) has W = X . Odd-Length Filters. two types of symmetry have to be considered. has the same type of symmetry. The eigenvalue problem we have to solve is given by [E] (8.

169) yields (cos .k . we may write Ho(ejw) = e-jw(Th + 3) (cos :)2~+1 2 p cos W ( 1.1 can be found by . (8. so that M(cosw) := F(sin2w/2).178) (8. H o ( e j " ) = e--ju(Th + Even-Length Filters.X ) .X ) .cosw)/2 = sin2w/2.173) and according to the above considerations. W (8. one can show that this condition is satisfied by a unique polynomial F ( z ) with a degree of at most lc .) 2 k 2 with M(COS ) = P(COS W W ) &(COS W) W for Ho(ejw) and cos + (sin 2 W) -)2k M ( .H.1 [28]. (cos W ) . Based on this property. Go(ej") then h e-j"(Tg + f) w (cos -)2L+1 Q(cosw).X) = 1 = 1.180) Using Bezout's theorem.k F(1.X k (1.179) F(X) = (1 .172) where T = ~h +T ~ . (8. Substituting factorizations the G o ( e j w ) into (8. the polynomial F ( z ) of maximum degree lc . We get (cos .174) where it is again assumed that Ho(ejw) has l! zeros at has a factorization of the form G o ( e j w )= = 7r. (8. so that Go(ej"') = e-jWTg h (cos E)" 2 Q(cosw).cosw) = 1 (8.176) (8. 2 (8. This 1 if it expression will now be reformulated by rewriting M(cosw) as a polynomial in (1 . Symmetric even-length filters can be expressed as cos .X)k F ( X ) with X = sin2w/2.X).) 2 k F(sin2 w/2) 2 or equivalently. Hence.175) Filter Construction.248 Transform 8. (1. Wavelet Chapter G o ( e J W )has It is easily shown that the same type of factorization.177) and lc = l! 2 if the filter length is odd and lc = l+ is even. 2 W (8. W + + + (sin -W) 2 k cos^ w/2) 2 + X k F(1.

170) (8. filters can be found by factorizing a given F(sin2w/2) into P(cosw) and Q(cosw).8.7. Given P(cosw) and Q(cosw) one easily finds Ho(ej") and Go(ej") from (8.183) The corresponding analysis filter is Even-length filters are given by (8.182) where R ( z ) is an odd polynomial. Spline wavelets based on odd-lengthfilters are constructed by choosing R ( z ) 0 and (8.175).180) into a Taylor series where only the first k terms are needed. Spline Wavelets. Wavelet expanding the right-hand side of (8. Based on this expression. This gives (8.181) The general solution of higher degree can be written as k-l n O = k+n-l (8.185) and .

However.16 . they are knownfor their excellent coding performance in wavelet-based image compression [155. For example.03782845543778 -0.187) Any regrouping into two polynomials yields a PR filter pair.02384946495431 0.11062440401143 -0.183) is a B-spline centered around T ~ and the one constructed from Go(z) according .41809227351029 -0.41809227351029 0. This allows us to choose filters with equal or almost equal length. This means an implementation advantage for the spline filters in real-time applications.1 shows some examples of odd-length filters.02384946495431 -0. Examples. Linear-phase odd-length biorthogonal wavelet filters.250 Transform 8.1. the 9-7 filters have superior coding performance. 5-3 n 4.15 and 8.78848561689252 0. While the coefficients of the spline filters (5-3 and 9-3) are dyadic fractions.1. Wavelet Chapter The scaling function 4(t) constructed from Go(z) according to (8.06453888265083 ho 0.187) are not even rational. the lengthof & ( z ) is typically much higher than the length of Go(z).06453888265083 -0.xi) n ( z z.37740285554759 -0. the 9-7 filters havebeenfound this way [28]. Figures 8.go 4 .2?Ji{~j} z z i= 1 + Izjl). one groups the zeros of F ( z ) into real zeros and pairs of conjugate complex zeros and rewrites F ( z ) as I J j=1 &() = A 'X ~ ( . to (8.04068941758680 -0. those of the 9-7 filters constructed from (8. go -1 2 6 2 -1 1 2 1 9-7 9-3 1 16 ' h0 3 -6 -16 38 90 38 -16 -6 3 go -0. (8. 1341. Table 8. For illustration. In the spline case.11062440401143 0.85269867833384 0. Table 8. In order to design filters with almost equal length.03782845543778 6 8 . Filters with Almost Equal Length. h0 .04068941758680 0.37740285554759 0.185) is a B-spline centered around T~ + i.16 show the analysis and synthesis scaling functions and wavelets generated from the 9-3 and 9-7 filters in Table 8.

-1.o 0.5l ' ' ' ' ' ' ' 0 1 2 3 4 5 6 7 8 I Figure 8. .o 0. -1.7.o 0.8.0 -1. .5 0 1 2 3 4 5 6 7 8 ~ 0 1 2 3 4 5 6 7 3.5 2.0' 0 ' ' ' ' ' ' ' 1 2 3 4 5 6 7 8 Figure 8. .0 2.0 1 1 -1.0 251 1. Wavelet Families 2. .0 1.o -1.5 1. 2.0 1.5 1 8 0 -0.5 l: . .5 1 8 0 -0.o 1 2 3 4 5 6 7 -1. .5 0 -0.0 1.5 1.5 -1. .0 0 1 2 3 4 5 6 7 -1.5 I *(t) :l .o -2.5 -1. Scaling functions and wavelets constructed from the 9-3 filters. .5 -0. Scaling functions and wavelets constructed from the 9-7 filters.5 0 -_E 1. . .5 -1. .15.o 0 -1. .5 -1.16. .

191) + (8.192) can also be written as W (cos2 E)' F(sin2 w/2) (sin2 -1' w/2) = 1.193) 2 2 or equivalently as (1 . the following factorization of H0 (ejw)is considered: (8. The family of Daubechies wavelets is derived for R ( x ) 0 by spectral factorization of F ( z ) into F ( z ) = P ( x ) P ( x . 2 2 W (8. but wenow have to satisfy F(sin2 w/2) 2 0 V W. Wavelet Chapter 8. Daubechies proposed to choose where R(z) is an odd polynomialsuch that F ( x ) 2 0 for X E [0. because F(sin2 w / 2 ) = IP(eju)12. (8. (8.188) Because of orthonormality.2 The Orthonormal Daubechies Wavelets Daubechies designed family of orthonormal wavelets with a maximal number a of vanishingmoments for a given support [34].7. P(. For this. This is essentially the same condition that occurred in the biorthogonal case.c o s w )= 1.194) + cos^ + with X = sin2 w/2. ) Inserting (8.252 Transform 8.X) = 1 (8.190) into (8.192) Using the same arguments as in the last section. Inorder to control the regularity.) then .189) yields (cos2 -)k ~ ( c o s w ) (sin2 . l].) k ~ ( . thePR condition to be met by the prototypefilter with ~ ( c o s w = IP(ejw)12. the zeros of F ( x ) have to be computed and grouped into zeros inside and outside the unit circle.' ) .X)k F ( X ) X k F(1.

known as symmlets. The frequency responses are the same as in Figure 8.196) and L 00 tk$(t) dt = 0 for Ic = 0 . l .. for Ic = 0 0.17. 1 .196) and (8. This can be expressed as 00 1. We observe that the scaling functions and wavelets become smoother with increasing filter length. are the same as for the minimum phase case. however. for Ic = 0 0. is used for the wavelet and the scaling function.. For comparison.18. 8.. . The magnitude frequency responses..1 and =O (8. . the corresponding scaling functions and the frequencyresponses of the filters. . and thecorresponding scaling functions are depicted in Figure 8.8.l. perfect symmetry is impossible..3 Coiflets The orthonormal Daubechies wavelets have a maximum number of vanishing wavelet moments for a given support. Wavelet 253 contains all zeros inside the unit circle..7. The idea behind the Coiflet wavelets is to trade off some of the vanishing wavelet moments to the scaling function. Further note that the same parameter l..199) Condition (8. For filters & ( z ) with at least eight coefficients. (8.17 shows some Daubechies wavelets. Figure 8.. called the order of the coiflet.2 .. This factorization results in minimum phase scaling functions.1 . l .197) Note that the 0th moments of a scaling function is still fixed to one.198) means for the filter Ho(ejw) that .. Vanishing moments of the scaling function have not been considered.1 (8. l . more symmetric factorizations are also possible. . (8. some Daubechies waveletswith maximal symmetry..7. forIc=1. l . Recall that with a few exceptions (Haar and Shannonwavelets).197) are 1. 2 . The frequency domain formulations of (8.198) for Ic=O.1. f o r I c = 1 . .

254 Transform 8. Wavelet Chapter -. - . Frequency responses of the maximally symmetric Daubechies filters and the correspondingscalingfunctionsandwavelets (the indicesindicatefilter length. the frequency responses are equal to those in Figure 8.18.50 I -0 1 t L f- l l : -50 -40 -30 0 L oh Figure 8.17).0 -0 L-1 5 f- l -0.

(8.201) For even e.189) is satisfied. solutions to this problem can be formulated as [361 f(ej").t c k Z(k) ?)*(2-"/c . that is.8.n). Onlyif the impulse response h(t)of the prefilter is chosen such that xo(t) = x ( t ) * h ( t )E VO. co(n)$(t .199) it follows that HO(ej") can also be written in the form (8. It could be shown that sample values of the wavelet transform can be computed by means of a PR filter bank. successively computed from co(n).n) x ( t ) +*(2-"t dt. the values d m ( n ) were sample values of the wavelet transform of a continuous-time signal. (8. (2"n. The Wavelet Transform of Discrete-Time Signals 255 for some U(ej"). For the sequences d m ( n ) .188) (8. 2") = (X. obtain a "genuine" we wavelet analysis.203) .202) where f ( e j " ) has to be found such that (8. 2") = 2 .8 The WaveletTransformof Signals Discrete-Time In the previous sections we assumedcontinuous-time signals and wavelets throughout.204) . m > 0. 8.8.n) are known. provided the coefficients c ~ ( n ) for representing an approximation xo(t) = C.) - 2-t L C Q (8. This results in e/2 quadratic equations for C/2 unknowns [36].$". it is helpful to discretize the integral in (8. If we wish to apply the theory outlined above to "ordinary" discrete-time signals x ( n ) . From (8. A considerable problem is the generation of the discrete-time signal c g ( n ) because in digital signal processing the signals to be processed are usually obtained by filtering continuous-time signals with a standard anti-aliasing filter and sampling.203): w.(2"n. we had drn(n) = W .

~ 5n ) .1 The A Trous Algorithm A direct evaluation of (8.204). we use an interpolation filter as .n E Z are tobe regarded as samples of a given wavelet +(t)where the sampling interval is T = 1.2) = @. Wavelet Chapter Here. This algorithm has been proposed by Holschneider et al. 5. Here. In thisform the wavelet analysis is not translation invariant because a delayed input signal z ( n .256 Transform 8. but approximately. the so-called ci trous algorithm allows efficient evaluation with respect to computing effort. The relationship between the B trous and the Mallatalgorithm was derived by Shensa [132]. Translation Invariance.19 are equal to those according to (8.204).i 7)*(-n/2). [73] and Dutilleux [48]. we obtain shifted versions of the same wavelet coefficients.2rn) = 2 . 8. we have wz(2n. The basic idea of the B trous algorithm is to evaluate equation (8.l) leads to wz(2"(n - 2 . For this. for many applications such as pattern recognition or motion estimation inthe wavelet domain it is desirable to achieve translation invariance. this is computationally very expensive. the computation is as efficient as with the DWT.2 .207) With this filter the output values of the first stage of the filter bankin Figure 8.204) notexactly. We start with dyadic sampling according (8. (8. but when using the B trous algorithm outlined in the next section. We are mainly interested in dyadically arranged values according to (8. (8.204) and (8.205) Only if l is a multiple of 2rn. (8.T l ) . This problem can be solved by computing all values Xi+)7)*(2-Y w2(n.204).206) In general. 2m) = 2-? = 2-7 C kz ( k - l) 7)*(2-rnlc - - n) [n.t C+) 7)*(2-rn(lc k - n)). The impulse response to of the filter H l ( z ) is chosen to be h1(n) = 2 .y . However.2). the values $ ~ ( 2 . m > 0.8.(2n.206) is very costly if the values of the wavelet transform must be determined several octaves becausethe number for of filter coefficients roughly doubles from octave to octave.

8. while the values h i ( 2 n 1) are interpolated. Analysis filter bank. (8. which refers to the fact that an interpolation lowpass filter is used. (8. we have b2(n) M 2-1 $*(-n/4).l $ * ( . h i ( 2 n + 1) = 0 with the interpolation filter the values h i ( 2 n ) remainunchanged. In order to explain this approach in more detail let us take a look at the flow graphs shown in Figure 8. 4 the analysis lowpass H 0 ( ~ ) .208) If Ho(z) an interpolation filter.209) + Iteration of this approach yields 4The term “A trous” means “with gaps”.By convolving the upsampled impulseresponse h i ( 2 n ) = hl(n).This may for instance be a Lagrange halfband filter. B2 (4 Figure 8. Equivalent arrangements. which both have the transfer function H1(z2) [Ho(z) Ho(-z)].19. the even numbered values of the impulse response b2(n) t B ~ ( z ) equal to the even numbered samples of 2-l$* (-n/4).n / 4 ) at the odd positions. . The transfer function & ( z ) is + + B z ( z ) = Ho(z) H1(z2). The Wavelet Transform of Discrete-Time Signals 257 - Figure 8.8. Thus.208) can be interpreted as follows: first is we insert zeros into the impulse responseh1 (n).20. ) are The interpolated intermediate values are approximately the sample values of 2 . but in principle any interpolation filter will do. (8. Thus.20.

the scheme in Figure 8. In many cases the frequency resolution of a pure octave-band analysis is not sufficient.212) Thus. 2 m ) m W.(n. An improved resolution can be obtained by implementing M .Ic) C z(k)b.213) by means of the filters bm(n)t B. (2%. 2m). the values 271. 2m) = 2 . 2m) may be efficiently computed by use of the filter bank ~~ m ).(2mn.19 yields an approximate wavelet analysis. Oversampled Wavelet Series.2m) = 2 .t according to (8. WaveletTransform For the impulse responses b.(n k (8. Although the coefficients of critically sampled representations containall information onthe analyzed signal. (8. z ~ ~> 1. Z and )H ~ ( . can be realized in polyphase structure.214) While the direct evaluation of these formulae meanshigh computational cost.n ) ) k (8.210): . The number of operations that have to be carried out is very small so that such an evaluation is suitable for real-time applications also.t C z ( k )@*(2-m(Ic . in Figure 8.(z) ) &(n.21.258 Chapter 8. they suffer from the drawback that the analysis is not translation invariant. (2%.19 are given by G. The aim is now to compute an approximation of Wz(n. The filters H O ( .2m) computed with the filter bank in Figure 8.(n) t ) & ( z ) we get The values ul.

(2%.(2mlc.8. For the filters B. 2m).and the sample values of the wavelet $ ( t ) .2 for the continuoustime case. m > 0. we have the following correspondence between the impulse response of the highpass filter.(z) defined in (8. 2m) we may proceed as follows: we take an interpolation filter Ho(z) and compute a filter Go(z) such that . let us consider a PR twochannel filter bank.216) 2. (8. where Ho(z) is an interpolation filter and where H1( z ) satisfies Hl(1) = 0.8. P ) . The application to a discrete-time analysis based on the B trous algorithm is straightforward. (2%.which is iteratively determined from ho(n) and hl(n): h1(n) = 2-i $*(-n/2).8. 1 2m) 1 = W. can be interpreted as sample values of a continuous wavelet transform provided the demandfor regularity is met: wz(2%.2 The Relationship between the Mallat and A Trous Algorithms The discussion above has shown that the only formal difference between the filters usedin the Mallat and B trous algorithms liesin the fact that in the Mallat algorithm the impulse response of the filter H l ( z ) does not. Based on the filter bank we can construct the associated continuous-time scaling functions and wavelets. Then. all computed wavelet coefficientsW.215) bm(n)= 2-? and we derive m +*(-2-"n). However. (2%.4. In order to determine filters that yield perfect wavelet analyses of discretetime signals with 211. 8. Transform Wavelet Discrete-Time The of Signals 259 octave filter banks in parallel where each bank covers only an Mth part the of octaves. in general. Since Ho(z) is supposed to be an interpolation filter.210) we have (8.217) wavelet transform exactly This meansthat theB trous algorithm computes the if Ho(z) and Hl(z) belong to a PR two-channel filter bank while Ho(z) is an interpolation filter. For this. 2m) = W . (2%. h l ( n ) . both concepts can easily be reconciled. 2m). consist of sample values of the continuous-time wavelet. This concept has been discussed in Section 8. 2m) = wz(2%. (8. m > 0.

so that one can choose a wavelet which has the desired properties.-P2n2/2.181). The lengthof the analysis highpass is restricted to 63 coefficients. Figures8. at least approximately. 8. (8. Example.210). Inorder admissible and analytic wavelet we choose 243 WO to obtaina“practically” 7r/2. (8. the wavelet is sampled in such a way that hl(n) = b l ( n ) = ejwon .218) is just an underdetermined linear set of equations. From Ho(z) and Go(z) we can then calculate the filters H l ( z ) and G1 ( z ) according to equation (6.the wavelet. Wavelet Chapter Note that (8. the parameters are chosen such that and p wo<n-1/2/3 is also satisfied [132].The frequency responses of Ho(z) and H l ( z ) are pictured in Figure 8.219) where b l ( n ) is defined as in (8.22(c).22) and can construct thewavelet via iteration.22(b) show the respective scaling function.218) is not unique. In order to ensure that we achieve an analytic wavelet we have to demand that !@(eJW) O = for 7r < W 5 27r WO In order to guarantee this.220) In the discrete-time case a further problem arises due to the periodicity of the spectra. In orderto realize a wavelet analysis of discrete-time signals.3 The Discrete-Time MorletWavelet The Morlet wavelet was introduced in Section 8. andthe sample values 4(-n/2) = ho(n) and +(-n/2) = h l ( n ) .8. (8.2. Thesolution to (8. The overall delay of the analysis-synthesis system is chosen such that a linear-phase highpass is yielded.22(a) and8.260 Transform 8.221) . For the analysis lowpass we use a binary filter with 31 coefficients as given in (8.

DWT-Based Image Compression 261 -0.9. the subband samples are quantized and further compressed.4 -20 ' 0 2t (4 0. The strategy is as follows: the image is first decomposed into a set of subband signals by using a separable5 2-D filter bank. Example.6 -20 0 2t @) - 20 t1 0 0 (c) Qln * 1 Figure 8.8.8 - I 20 t S O S -0. 8. satisfy certain conditions such as regularity and vanishing moments.8. (a) scaling function 4(t)and the sample values4(-nT/2) = ho(n). however. Togive an example of the discrete wavelet transform of a 2-D signal. Then. .22.(c)frequency responses of the analysis filters.9 DWT-BasedImageCompression Image compression based onthe DWT is essentially equivalent to compression based on octave-band filter banks as outlined in Section 6. The filters. 5Non-separable 2-D wavelets and filter banks are not considered throughout this book. (b) wavelet +(t)and the samplevalues +(-nT/2) = hl(n).

One of the most interesting features of this coding technique is that the bitstream can be truncated at any position. The refinement process can be continued until one reaches a desired precision. then it is likely that the belonging children are also small. . This relationship can be exploited in order to encode the subband pixels in an efficient way. Figure 8. it will be coded as a so-called zerotree by using a single codeword.23(b) shows its 2-D wavelet transform. Starting with coarse quantization ensures that a high numberof zerotrees can beidentified at the beginning of the encoding process. The squares in Figure 8. 1281.23. (a) original. During the refinement process. the number of zerotrees successively decreases.23(a) shows an original image and Figure8. resulting in an almost optimal rate-distortion behavior for any bit rate.23(b) indicate spatial regions that belong to thesame region in Figure 8. The coding technique is known as embedded zerotree wavelet coding [131.23(b). Separable 2-D discrete wavelet transform. Thearrows indicate parent-child relationships. We will not study the embedded zerotree coding here in great detail.262 Chapter 8. An important observation can be made from Figure 8. Thus. then it is likely that there is also little high-frequency information in that region. (b) DWT. the quantization of the waveletcoefficientsis carried out successively. using a bitplane technique. WaveletTransform Figure 8. if a parent pixel is small. Most importantly.23(a). Overall one gets an embedded bitstream where the most important information (in terms of signal energy) is coded first. Whenever a tree of zeros (pixels quantized to zero with respect to a given threshold) is identified. but we will give a rough idea of how such a coder works. One starts with a coarse quantization and refines it in every subsequent step. which is true for most natural images: if there is little low-frequency information in a spatial region. when the quantization step size is successively reduced.

w ( n ) may be a Gaussian white noise process. 421.8. Two approaches known as hard and soft thresholding have been proposed for this purpose [43. They use the following non-linearities: y(n).One tries to remove the noise by applying a nonlinear operation to the wavelet representation of y(n). the wavelet coefficients are manipulated in order to remove the noise component.222) For example. The same problem has been addressed in Chapter 7. thus performing discrete wavelet transform. Y(n) > -E () E +-E. (b) soft thresholding. wavelet-based denoising is closely related to spectral subtraction.3 in the context of the STFT. First.10 Wavelet-Based Denoising y(n) = z ( n ) w ( n ) . which is statistically independent of z ( n ) . y(n) Iy(n)l --E (soft) I -E Figure 8. .24 illustrates both techniques. where it was solved via spectral subtraction..1 0. The denoising procedure is as follows. the signal y(n) is decomposed using an octave-band filter bank. a Then.. In fact. Wavelet-Based Denoising 263 8. The aim of denoising is to remove the noise w ( n ) from a signal + (8. (4 (b) Figure 8. Y(n) > -E y(n) y(n). Thresholding techniques. 0 7 < --E Iy(n)l (hard) (8. The maindifference between both approaches lies in the fact that the wavelets used for denoising are real-valued while the STFT is complex. (a) hard.224) Y n .223) I -E (8.24.

the signal @(n) reconstructed from the manipulated wavelet coefficientswill contain muchlessnoise than y(n) does.the problem is to choose E . If E is too small. Wavelet Chapter Basically. If it is too large. while w ( n ) is white noise. this holds true if x ( n ) is a lowpass signal. . For example. Thus. the signal will be distorted. the idea of thresholding is that ~ ( n ) be representedvia a few can wavelet coefficients. while the large coefficients due to z(n) are only slightly affected. because the amount of noiseis usually not known a priori. provided the threshold E ischosen appropriately. the noise will not be efficiently removed. Inpractice. while the noise has wideband characteristics and spreads out on all coefficients. The thresholding procedure then sets the small wavelet coefficients representing w ( n ) to zero.264 Transform 8.

9. Time-Shifted Signals. (9-1) 265 . (1. Contrary to spectrograms and scalograms. We start by looking well at time and frequency shifts separately. Filter Banks. In this chapter time-frequency distributions derived in a different manner will be considered. they allow extremelygood insight into signal properties within certain applications.38)): ) . Both distributions are the result of linear filtering and subsequent forming of the squared magnitude. Time-Frequency Transforms and Applications.(t) = z(t T ) is related to the autocorrelation function T ~ ~ ( THere the following holds (cf. Although these methods do not yield positive distributions in all cases. + d(&. Zl. Alfred Mertins Copyright 0 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Chapter 9 Non-Linear TirneFrequency Distributions In Chapters 7 and 8 twotime-frequencydistributions were discussed: the spectrogram and the scalogram. =211412 -2 w%T)}.1 The Ambiguity Function The goal of the following considerations is to describe the relationship between signals and their time as as frequency-shifted versions. 2.Signal Analysis: Wavelets.) between an energy signal z ( t ) and its time-shifted version z. The distanced ( z . their resolution is not restricted by the uncertainty principle.

' Comparing (9. in (9. F. z v ) = 2 llz112 . z() and pEz(v). We have = J-W z*(t) z(t) ejutdt W (9-5) = J-CC sEz(t) ejVtdt with sEz(t) = lz(t)I2. .z>}.(w) = IX(w)I2: In applications in which the signal z(t) is transmitted and the timeshift T is to be estimated from the received signal z(t T ) . In the frequency domain this means that theenergy density spectrum should be as constant as possible. That is.5) we have an inverse Fourier transform in which the usual prefactor 1/27r does not occur because we integrate over t .".ofz For the inner product (zv. thedistance between a signal z ( t ) and its frequency-shifted version z v ( t )= z ( t ) e j u t is of crucial importance. However.5) with (9.2 x{(~.2. it is important that z ( t ) and z(t T) are as dissimilar as possible for T # 0. If one wants to estimate such frequency shifts in orderto determine the velocity of a moving object. = z) L CC z*(t)z(t + . this would lead to other inconveniences in the remainder of this chapter. ~ ( T ) can also be understood as the inverse Fourier transform of the energy density spectrum S. This peculiarity could be avoided if Y was replaced by -v and (9.4) . 'In (9. + + Frequency-Shifted Signals. not over W . (9-2) As explained in Section 1. where sEz((t) can be viewed as the temporalenergy density. T .3) shows a certain resemblance of the formulae for .5) was interpreted as a forward Fourier transform. Frequency-shifted versions of a signal z ( t ) are often produced due to the Doppler effect. (9.266 Chapter 9. Non-Linear Time-Ekequency Distributions where TE .r)dt.4) we will henceforth use the abbreviation z) (v). ( T ) ~ = (zT. The distance is given by d ( z . the transmitted signal z(t) should have an autocorrelation functionthat is as Dirac-shaped as possible.

= S_.With the abbreviation for the so-called time-frequency autocorrelation function or ambiguity function’ we get Thus.11) ‘We find different definitions of this term in the literature.(u) can be seen as the autocorrelationfunction of X ( w ) .9. In non-abbreviated form (9.~)1’ [150]. (9.1.-) X * ( w 2 U U + -) ejwT dw. we frequency domain the 2 X ( w . This becomes even more obvious if pF. . ) 2 (9. centered around z ( t ) .z*(t . distance between both signals.. Let us consider the signals which are time and frequency shiftedversions of one another. Time and Frequency-Shifted Signals.7 ) z(t T) 2 W + IGut dt.8) is Azz(u. with the time frequency domains being exchanged. the real part of A z z ( v r) is related to the . The Ambiguity Function 267 however. in r) Via Parseval’s relation obtain anexpression for computing A.(u) is stated in the frequency domain: We see that pF.10) (U. Some authors also use it for the term IAZz(u.

14) Thus.T) d W o T A.268 Chapter 9. when designing an appropriate signalz(t)the expression . 3. 2.T) = e-. (v. The classical problem in radar is to find signals z(t) that allow estimation of timeandfrequencyshiftswith high precision. A time shift of the input signal leads to a modulation of the ambiguity function with respect to the frequency shift v: This relation can easily be derived from (9.16) This is directly derived from (9... Therefore. = (9.12) which satisfies 1 1 2 11 = 1. is the signal energy.? e-&u2 (9.T). A modulation and/or time shift of the signal z ( t ) leads to a modulation of the ambiguity function.11) by exploiting the fact that x ( w ) = e-jwtoX(w). Radar Uncertainty Principle. The ambiguity function has its maximum at the origin. 1. where E.. Using the correspondence we obtain A.(~. Non-Linear Time-Ekequency Distributions Example. the ambiguity function is a two-dimensional Gaussian function whose center is located at the origin of the r-v plane. We consider the Gaussian signal (9. Properties of the Ambiguity Function. A modulation of the input signal leadsto a modulation of the ambiguity function with respect to I-: z(t)= eJwotz(t) + AEZ(V. but the principal position in the r-v plane is not affected.10).

if we achieve that IA. Finally we want to remark that. we first look at the ambiguity function.2 9. . W z(t + f ) y * ( t . (9. 9.2.(O.) .. so-called cross ambiguity functions are defined: A?/Z(V.18) That is.. ) we obtain for v = 0 r the temporal autocorrelation function from which we derive the energy density spectrum by means of the Fourier transform: (9. 7) = 1.1 The Wigner Distribution Definition and Properties The Wigner distributionis a tool for time-frequency analysis.18) is also : referred to as the radar uncertainty principle.2.0)I2 = E . Y*(w ) .)I2 takes on theform of an impulse at the origin. (v.gner 9. ..21) W - l m A . For this reason. r ) e-iwT d r ..r)I2 r dv = IA. which contains information onthe possible resolution of a given z ( t ) in the r-v plane. Cross Ambiguity Function. The ideal of having an impulse located at the origin of the r-v plane cannot be realized since we have [l501 m IAzz(v. ( O .19) X ( W. d : (9.(O. which has gained more and more importance owing to many extraordinary characteristics.(v. In order to highlight the motivation for the definition of the Wigner distribution. The 269 is considered. analogous to the cross correlation.f ) ejyt dt (9. From A. . it necessarily has to grow in other regions of the r-v plane because of the limited maximal value IA. 0)12 = E . + eJw7 dw.

The transformwith respect to v yields the temporal autocorrelation function4 (9. E { C J ~ ~ ~ (would )be the autocorrelation ~.24) The time-frequency distribution W.T } function of the process.25) (9. 41f z ( t )was assumed to be a random process. (9.W ) as the two-dimensional Fourier transform of A.24)can also be viewed as performing two subsequent one-dimensional Fourier transforms with respect to r and v. Non-Linear Time-Ekequency Distributions On the other hand. 3Wigner used W z z ( t .23) These relationships suggest defining a two-dimensional time-frequency distribution W z z ( t . w ) for describing phenomena of quantum mechanics [163].. so that one also speaks of the Wigner-Ville distribution. 7 )for 7 = 0: The temporal energy density &(-L) is the Fourier transform of &(v): (9..(t.~ is known as the Wigner distribu- The two-dimensional Fourier transform in (9.g) X * ( w + 5).26) = X ( w .270 Chapter 9.(v. . get the autocorrelation function (v) of the spectrum we pFz X ( w ) from A z z ( v . Ville introduced it for signal analysis later [156].W ) = 2n -m -m Azz(v.r)e W) -jvt e -jwr d U dr. ti~n. 7): W22( t .

(U. the Wigner distribution cannot be interpreted pointwise as a distribution of energy because it can also take on negative values.9. in full: ) Figure 9.26).2. However. they are omitted. The function @.27) with & . . as a distribution because it is supposed to reflect the distribution of the signal energy in the time-frequency plane.w) according to (9. Since the proofs can be directly inferred from equation (9.Altogether we obtain (9.. TheWignerDistribution 271 I Temporal autocorrelation Wignerdistribution I Temporal autocorrelation Figure 9. W) is so to say the temporal autocorrelation function of X(W). W) We speak of W.. The most important of these properties will be briefly listed. ~according to (9. Apart from this restriction it has all the properties one would wish of a time-frequency distribution.. Relationship between ambiguity function and Wigner distribution.1 pictures the relationships mentioned above.25) and @. (t.1.(Y.28) by exploiting the characteristics of the Fourier transform. ( t .

. (9.w) = 0 for W W > w2.w) d t = I X ( W ) ~ ~ .. then the Wigner distribution is also restricted to this frequency region: X ( w ) = 0 for W < w1 and/or < w1 and/or W > w2 (9.. Integrating over time and frequency yields the signal energy: W W W.272 Chapter 9.w) 2n -m m m dw = lz(t)I2.32) 5 . Non-Linear Time-Ekequency Distributions Some Properties of the Wigner Distribution: 1.(t. (9..(t.w) = 0 for (9. 6.(t.31) 4. If a signal z ( t ) is non-zero in only a certain time interval.(t) E =L/ W.(t.28).33) t < tl and/or t > t z .30) 3. If X ( w ) is non-zero only in a certain frequency region.”. This property immediately follows from (9. (9.(w) = J -m W.29) 2...34) U W. By integrating over W we obtain the temporal energy density s.(t. By integrating over t we obtain the energy density spectrum S.. The Wigner distribution of an arbitrary signal z(t) is always real.w) dw d t = (9. then the Wigner distribution is also restricted to this time interval: z ( t ) = 0 for t < tl and/or t > t 2 U W.

26) and (9..27)): Z ( t ) = z(t .7-) =X*(t - -1 2 7- 4 t + 5' ) 7- (9.39) cf. Along the line t = 7-/2 we get 2(. A modulation of the signal leads to a frequency shift of the Wigner distribution (cf. ( .W O ) .t o . Similarly. The squared magnitude of the inner product of two signals z ( t ) and y(t) is given by the inner product of their Wigner distributions [107].7-) = X*(O) X.W ) . w.2. (9. ( t. (9.t o ) * + WEE(t.40) This means that any z(t) can be perfectly reconstructed from its Wigner distribution except for the prefactor z*(O). [H]: . ) with W respect to W we obtain the function +zz(t. ) = W. A simultaneous time shift and modulation lead to a time and frequency shift of the Wigner distribution: ~ ( t )z(t .41) Moyal's Formula for Auto-Wigner Distributions. w= ~ .27). W . (9. w ) = W z z ( t . Time scaling leads to ~ E s ( t . we obtain for the spectrum X * ( u ) = Qzz(-. (9. ) 7- (9. .36) 9.(t W .27)): Z ( t ) = z(t)ejwot W g g ( t .) = +zz(5.to. By an inverse Fourier transform of W zz( t.U) = X ( 0 ) X * @ ) . A time shift of the signal leads to a time shift of the Wigner distribution (cf.W O ) . The 273 7.to)ejwOt = 10. (9. (9.25) and (9.37) ) Signal Reconstruction.35) 8.gner 9. U 2 (9.

(9. W) is e-at' W..w) = 2~ [AI2S(W .(t.14). We consider the signal (9. a frequency modulated (FM) signal whose instantaneous frequency linearlychanges with time: x ( t )= A We obtain .45) with (9. . Signals with Positive Wigner Distribution.43) W.49) we get (9. (9.46) The Wigner distribution W.. Non-Linear Time-Ekequency Distributions 9.2 Examples Signals with Linear Time-Frequency Dependency. Gaussian Signal.16)).2.274 Chapter 9.48) Hence the Wignerdistribution of a modulatedGaussiansignal is a twodimensional Gaussian whose center is located at [to. (9. (t. (9.W O ] whereas the ambiguity function is a modulated two-dimensional Gaussiansignal whose center is located at the origin of the 7-v plane (cf. The Gaussian signal and the chirp are to be regarded as special cases.(t.50) with W. e-n 1 W'.49) have a positive Wigner distribution [30]. Only signals of the form (9.15) and (9.44) This means that the Wigner distribution of a linearly modulated FM signal shows the exact instantaneous frequency. The prime example for demonstrating the excellent properties of the Wigner distribution in timefrequency analysis is the so-called chirp signal. (9...pt).WO . 2 e-"(t W) = - to)' e-a 1 [W . (9. W) 2 0 V t .(t.j+Dt2 ejwOt.WO]'. For the Wigner distribution of z ( t ) according to (9. = 2 u) and for WZZ W ) we get (t.47) wZE((t.W .

.

. Moyal's Formula for Cross Wigner Distributions.~ ( w I ~ 2 ) ) .z(t. ) we get W W z z ( t . We consider the sum of two complex exponentials (9. y) Example.W) + 2 WW.58) For W.W ) = A: S(W +2A1Az -wI) + A$ S(W -wZ) COS((W~ W1)t) - S(W - + (9. The occurrence of cross-terms WVZ(t.3 shows W. (t.41 + W&. Non-Linear Time-Ekequency Distributions It canberegarded as a two-dimensionalFouriertransform of the cross ambiguity function AYz(v.w) andillustrates the influence of the cross-term 2 A 1 A 2 C O S ( ( W ~ W 1 ) t ) S(W . (9. As can easily be verified.. For the inner product of two cross Wigner distributions we have [l81 with ( X .276 Chapter 9. W (9. - + .54) We now consider a signal and the corresponding Wigner distribution = WZZ(t. (t.59) ~ 2 ) ) Figure 9.W). ( 4 W ) = W:.u) complicates the interpretation of the Wigner distribution of real-world signals.(t. ) .. z ( t ) and y ( t )we have W. = J z ( t )y * ( t ) dt. Size and location of the interference terms are discussed in the following examples. for arbitrary signals 7).56) We see that the Wigner distribution of the sum of two signals does not equal the sum of their respective Wigner distributions.

In this example the sum considered: of two modulated Gaussian signals is 4 t ) = 4 t ) + Y(t> with z ( t ) = .5 show examples of the Wigner distribution.61) and (9. TheWignerDistribution 277 Figure 9.4 and 9.9.-fa(t - (9.tl) (9. also shown in Figure 9. Wigner distribution of the sum of two sine waves.jw1(t . which results from the fact that the ambiguity function is a time-frequency autocorrelation function. The center of the signal term is located at the origin. The interference terms concentrate around . We see that the interference term lies between the two signal terms.60) .2. Example. and themodulation of the interference term takes place orthogonal to theline connecting the two signal terms.3. This is different for the ambiguity function.5.62) Figures 9.

t Signal (real part): distribution m Wigner l wz----j-* Ambiguity function (c) Figure 9.5. w1 = w2. (b) tl # t z . (a) tl = t z . Wigner distribution and ambiguity function of the sum of two modulated and time-shifted Gaussians (tl # t z . Wigner distribution of the sum of two modulated and time-shifted Gaussians. w1 # w z ) .4. . w1 # wz. Non-Linear Time-Ekequency Distributions t (a) t (b) Figure 9.278 Chapter 9.

a multiplication in the time domain is equivalent to a convolution of the Wigner distributions W.~) dt’. We consider the signal Z ( t ) = z(t) h(t).W ) 271 : hh(t. For Z(t) = x(t) * h(t) (9.63) For the Wigner distribution we get - L W The multiplication of $==(t. or equivalently. and $ h h ( t .2. multiplying X ( W )and H ( w ) . Convolving z ( t ) and h ( t ) . the concept of windowing is introduced.(t.W ) with respect to W .9.~) W’h(t -t ’. A practical problem one encounters when calculating the Wigner distribution of an arbitrary signal x ( t ) is that (9. Pseudo-Wigner Distribution.4 Linear Operations Multiplication in the Time Domain. r ) with respect to r can be replaced T) by a convolution in the frequency domain: WE ( t .W ) = - l W. TheWignerDistribution 279 9. Therefore.W ) with respect to t . (9. Convolution in the Time Domain.66) = I (9.28) canonlybeevaluated for a time-limited z(t). one usually does not apply a single window ..w) and W h h ( t .2.67) Wzz(t’.. leads to a convolution of the Wigner distributions W.. For this.w) and W h h ( t . ( t .(t. W W) That is.

other desired properties mayget lost. Therefore one speaks of a pseudo.This means that the pseudo-Wigner distribution is a ) smoothed version of the Wigner distribution. ) 7 2 2 (9. A remedy is the introductionof additional two-dimensional smoothing kernels. 9. ( t . depending on the smoothing kernel. ) * H ( w ) W W (9. Unfortunately.65). Using the notation M h ( r ) &(t.Wigner distribution [26].3 General Time-Frequency Distributions The previous section showed that the Wigner distribution is a perfect timefrequency analysis instrument as long as there a linear relationship between is instantaneous frequency and time.70) 1 with H ( w ) t h@). general signals. ) = 2?rWzz(t. but one centers h ( t ) around the respective time of analysis: M z*(t.. r) e-jwT d r it is obvious that the pseudo-Wignerdistributioncanbe W. which guarantee for instance that the time-frequency distribution is positive for all signals. Non-Linear Time-Ekequency Distributions h(t) to z ( t ) .280 Chapter 9. .as in (9.7) x ( t + -) h ( ~e-jwT dT.69) calculated from WArW)(t. To illustrate this. will consider several we shift-invariant and affine-invariant time-frequency distributions. the time-frequency distribution according to (9.W) as (9.68) corresponds only approximately to the Wigner distributionof the original signal. the Wigner distribution For takes on negative values as well and cannot be interpreted asa “true” density function.68) Of course.

Since the kernel g(v. The product M ( v . .r ) in (9.. Multiplying A z z ( v .r ) . r )with g(v. Depending on the application.74) is known as the generalized ambiguity function.(t. From (9. ) with the Fourier transform of the kernel: W with 1 G(t.73) we derive the Wigner distribution for g ( v .r ) X* (U .. one can choose a kernel that yields the required properties. TZZ(t.71) This class of distributions is also known as Cohen's class.73) can also be expressed as the convolution of W. (9.t . g(v. (t. we get T.73) This means that the time-frequency distributions of Cohen's class are computed as two-dimensional Fourier transforms of two-dimensionally windowed ambiguity functions. 2 2 (9.71). General Time-frequency Distributions 281 9. r ) AZZ(v.1 Shift-Invariant Time-Frequency Distributions Cohen introduced a general class of time-frequency distributions of the form ~ 9 1 r r .w) = 271 ss g ( u .W) =2T /// + By choosing g ( v . . For g ( v . ej'(u .3.) A&.r) eCjyt eCjWT du d r . T) all possible shift-invariant time-frequency distributions can be generated.) = g(v. r ) = I.-jut e-jWT dv dr.9. If we carry out the integrationover u in (9. That is. (9.r)in (9. r ) = h ( r ) we obtain the pseudo-Wignerdistribution.71) is independent of t and W .W ) = 2T // g(v.76) .-) ~ ( u-) e-JWTdv du dr. ) . all time-frequency distributions of Cohen's class are shift-invariant. (9.3.

that is is obtained if the kernel satisfies the condition g(Y. du.80) A real distribution. In general the purpose of the kernel g(v.73) gives a straightforward interpretation of Cohen’s class. Non-Linear Time-Ekequency Distributions That is. .77) the kernel must satisfy the condition g(u.T ) .T) = g*(-V.82) Finally it shall be noted that although (9. we first integrate over Y in (9.73) into (9.O) = 1.282 Chapter 9.T ) is to suppress the interference terms of the ambiguity function which are located far from the origin of the T-Y plane (see Figure 9. With (9. (9.78) We realize this by substituting (9.71). For example. this again leads to reduced interference terms in the time-frequency distribution T. dr.w). Depending on the type of kernel. if one wants to preserve the characteristic (9. Equation (9.(t.83) . of For this. the kernel must satisfy the condition in order to preserve the property (9.71) is more advantageous.W ) . some of the desired properties of the time-frequency distribution are preserved while others get lost. the implementation (9..75) shows that the reduction of the interference terms involves “smoothing” and thus results in a reduction of time-frequency resolution. (9. Correspondingly. all time-frequency distributions of Cohen’s class can be computed by means of a convolution of the Wigner distribution witha two-dimensional impulse response G ( t .5).77) and integrating over dw.

r) Fouricr transform T.t .84) Figure 9. Furthermore. the short-time Fourier transform is In expressed in the form CC Fz(t.85) be written as S.t’) .3.89) . we obtain T.6..6 shows the corresponding implementation. described in detail in Chapter An interest7. the energy density lXt(w)12 can be computed from the Wigner distribution W..(t. General Time-frequency Distributions 283 4 0 Convolution with r(4. ) according to (9. (9. (W) = l X t ( 4 I 2. with the abbreviation X&’) = X@’) h*(t.-jut‘ dt’. (9.r ) z*(u .w) = J-CC z(t‘) h*(t.9. Generation of a general time-frequency distribution of Cohen’s class.t’).. W) = // T(U . 9. The best known example of a shift-invariant time-frequency distribution is the spectrogram.2 Examples ofShift-InvariantTime-Frequency Distributions Spectrogram.31): W (9..(44 Figure 9.88) can (9. (t’.87) (9. (9.3.85) Then the spectrogram is Alternatively.. ing relationship between the spectrogram and the Wigner distribution can be established [26]. order to explain this.-) z(u 7- 2 7 + -) 2 ’ .-JUT du dr.

Separable Smoothing Kernels.G(t.35) and (9. r ) in (9. Using separable smoothing kernels dv. The kernel g(v. which becomes 1 Tzz(t. we finally obtain from (9. G ( w ) gz(t). =G 1( 4 Q2 (.2). the resolution in the time-frequency plane is restricted in such a way (uncertainty principle) that (9. Non-Linear Time-Ekequency Distributions Observing (9. = g1(t) W GI(w).80) cannot be satisfied.W). This becomes obvious in (9.W ) 21r 1 = .75)): Although the spectrogram has the properties (9.81) and (9.w) = 27F //Wxx(t'.w = 1 1 -W') dt' dw' (9.72). (9. Thus the spectrogram results from the convolution of Wzz(t.95) Time-frequency distributions which are generated by means of a convolution of a Wigner distribution with separable impulse responses can also be understood as temporally smoothed pseudo-Wigner distributions.84) we derive the following formula for the time-frequency distribution.90) 2 Wzz(t. (9.W ) (9. .l.65).75).83) and (9. which can be implemented efficiently: Txx(t1w)= / [/ Z*(U - 5)Z(U + 5)g l ( u 7 7 - t ) du (9.93) * [ G z ( w )* wzz(t. This becomes immediately obvious when we think of the spectrogram of a time-limited signal (see also Figure 9.W ) ] G ( ~ .73) is the ambiguity function of the impulse response h(t) (cf.89): Sx(t. (9. The window 1 g2(T) .92) means that smoothingalongthetimeandfrequency axis is carried out separately.w') W h h ( t . the spectrogram belongs to Cohen's class.W ) = .77) and (9. w )g 1 ( t ) G Z ( W ) .284 Chapter 9. ) with the W Wigner distribution of the impulse response h@).-jwT dT.t'.w)* * 27F Whh(t.Therefore.94) From (9.g1(t) 21r where * * wzz(t.

ff. General Time-frequency Distributions 285 gZ(T) in (9.102) ffp < 1.5).p E R.9. Temporal smoothingis achieved ) by filtering with g1 ( t ) .95) plays the role of h ( ~in (9. It is obvious that smoothing should be carried out towards the direction of the modulation the cross term (compare Figures9.. T ) = . T. (9.p 2 ~ 2 / 4 .. T i ~ a u s B )is teven more smoothed than a spectrogram.e .100) It can be shown that for arbitrary signals a positive distribution is obtained if [75] (9. An often usedsmoothing kernel (especially in speech analysis) is the Gaussian g ( v .68).W ) for aP 2 1 can be computedmuch more easily and more efficiently via a spectrogram.3. The choice of (Y and p is dependent on the signal in question. 2 ff. In order to give a hint.96) Thus we derive the distribution z*(u .U ) we have with g1(t) = . ~is equivalent to a spectrogram Gaussian ) with ( . ( t . (Gauss) Since T. we look at .97) For the two-dimensional impulse responseG ( t .e-a2u2/4 e .w 2 / p P (9. (9.-) z(u r 2 r + -) 2 du dT.p > 0.99) and 1 Gz(w)= . computing with the smoothed pseudo-Wigner distribution is interesting only for the case (Gauss) (9. Although the modulation may occur in any direction.~) window.4 of and 9.101) ffp 2 1. For crp > 1. ( t . e 1 $/a2 (9. consider a signal z ( t ) consisting of the sum of two modulated time-shifted Gaussians. For aP = 1.

.

.

and the corresponding time-frequency distribution has reduced interference terms as well. (9.W ) = e-r2a(U . (9. Non-Linear Time-Ekequency Distributions Examples of Time-Frequency Distributions of Cohen's Class.104) We see that g(v. This type of distribution is of enticing simplicity. Atlas and Marks [l681 suggested the kernel (9.106) This yields the distribution .71). Zhao.104) maybeunderstoodas a free parameter. = -W(47W.r ) = 1 are satisfied so that theChoi-Williams distribution has the properties (9. The Rihaczek distribution is defined as [l241 T. Rihaczek distribution. Choi.r ) A. and (9. For the Choi-Williams distribution the following product kernel is used [24]: g(v. (9.. In the literature we find many proposals of shift-invariant time-frequency distributions.Williarns Distribution.0) = 1 and g(0. A survey is presented in [72] for instance.r ) = g(v. the kernel concentrates around the origin of the r-v plane. The quantity g in (9. In the following. except for the r and the v axis.105) Zhao-Atlas-Marks Distribution. three examples will briefly be mentioned.83).80). Thus we get a generalized ambiguity function M ( v . If a small g is chosen.t12/T2~*(u -) ~ ( u -) + 7- r 2 2 ' e-J'JT du dr. From (9.288 Chapter 9.103) = x * ( t )X ( W ) ejwt. g > 0.w) = S x * ( t )x ( t + r ) e-jwT dr (9.77) and (9.84) we get T g W )( t .(f)(t. r ) with reduced interference terms. but it is not real-valued in general.(v.

108) can be computed from the Wigner distribution by means of an affine transform [54].112) we derive I W . (9.3.‘) b t’ - Wzz(t‘. there exist other timefrequency distributions besides the Wigner distribution which belong to the shift-invariant Cohen class as well as to theaffine class.(t.111) (9. Since (9.109) This can be understood as correlating the Wigner distribution with kernel K along the time axis. $ = 2n ~~ (-.3. An example of the affine class is the scalogram. for instance.3 Affine-Invariant Time-Frequency Distributions Affine smoothing is an alternative to regular smoothing of the Wigner distribution (Cohen’s class). By varying W the kernel is scaled.113) Thus.72) do not exclude eachother.108) and (9.110) with (9.112) where (9. that is. A time-frequency distribution that belongs to the affine class is invariant with respect to time shift and scaling: Any time-frequency distribution that satisfies (9. from (9. the squared magnitude of the wavelet transform of a signal: 2 (9. [126]: T.9.a. Scalogram. all time-frequency distributions that originate from a product kernel.W’) dt’ h‘.w’/w) Wzz(t‘. U )L / / W $ . These are..W) = 2n ‘ s sK(W(t‘ -~). (9. such as the Choi-Williams distribution.114) . ( ~ . General Time-frequency Distributions 289 9.w‘) dt’ h’.

3. However. - :U’) W . which. 9.limited by the uncertainty principle. Thus. (n. ( ~ ’ . discrete-time signals have a periodic spectrum. . Non-Linear Time-Ekequency Distributions The substitutions b = t and a = wo/w finally yield (9..290 Chapter 9. If the signal and thekernel are bandlimited and if the sampling rate is far above the Nyquist rate for both signal and kernel. We have the following property: while the signal z(n) has a spectrum X ( e j w ) t z(n) with period 2n. such as the Choi-Williams distribution. the test signal may be discrete-time right away. W. ’) The resolution of the scalogram is.115) = -1 W $ . can be written as W X* (t .ejw + ). (9.28). Discrete-Time Wigner Distribution [26].118) . (n. The discrete-time Wigner distribution is defined as (9.117) As we know.r‘) z ( t + r’) e-i2wr‘ dr’. using the substitution r’ = 3-12.116) m Here.4 Discrete-Time Calculation of Time-Frequency Distributions If we wish to calculate the Wigner distribution or some other time-frequency distribution on a computer we are forced to sample our signal and the transform kernel and to replace all integrals by sums. so that one could expect the Wignerdistribution of a discrete-time signal to have a periodic spectrum also.. (9.116) is the discrete version of equation (9. On the other hand. sampling the kernel already poses a problem.e j w > = W. just like that of the spectrogram. in some cases. $ I 2n (%(Pt ) . so that discrete-time definitions of time-frequency distributions are required in any case. W dt’ h’. equation (9. kEZ. the period of the discrete) time Wigner distribution is only n. we do not face a substantial problem.

for kernels that are not bandlimited.124) . . w I f (9.84) and (9. Part 11. = 2 k) m=-Me=-N cc M N p ( [ . k= IX(k)12 = IX($W”)12 ) n (9.122) is necessary in order to preserve the properties [l11 C T . A detailed discussion of the topic can be found in [26].123) The normalization C . . Basically we could imagine the term p(C. M . .119) and not with f a 2 2 f m a z . for the discrete-time Choi-Williams distribution we therefore use the matrix with N e-uk2/4m2. In order to avoid aliasing effects in the Wigner distribution. (9. ( F W ) ( n .. .121) Here we have already taken into account that in practical applications one would only consider discrete frequencies 2 ~ k / L where L is the DFT length.N .116). h=-N . (9. 7)in However. a general discrete-time time-frequency distribution of Cohen’s class is defined as Tzz(n. General Time-frequency Distributions 291 The reason for this is subsampling by the factor two with respect to T .120) Because of the different periodicity of X ( e j w ) and W z z ( n . For example. n = -N. Analogous to (9. General Discrete-Time Time-Frequency Distributions.3. p(n.9.m) in (9. m = -M. . m) 2 * ( C +n - m) 2 ( C + n + m) e-j47rkmlL. where X ( w ) = 0 for I > 27~ m a z . sampling causes a problem. m ) = 1 in (9. .121) to be a 2M + 1 X 2 N + 1 matrix which contains sample values of the function ~ ( u . e j w ) is not it possible to transfer all properties of the continuous-time Wigner distribution to the discrete-time Wigner distribution. one has take care to that the bandlimited signal z ( t ) is sampled with the rate fa 24 fmaz (9.84).. (9.

. t r ..But by forming the expected value it generally contains fewer negative values than the Wigner distribution of a single sample function. and the Wigner-Ville spectrum becomes the power spectral density: W w. + The properties of the Wigner-Ville spectrum are basically the same as those of the Wignerdistribution.128) if z ( t ) is stationary. which is the autocorrelation function r z z(t 5.(9.4 The Wigner-Ville Spectrum So far the signals analyzed have been regarded as deterministic.~ )r~ ( t + .5) of the process z ( t ) . In order to illustratethis. t .127) + * )} 2 2 This means that the temporal correlation function (t. such as power spectral density. Contrary to the previous considerations. For stationary processes the autocorrelation function only depends on r. r ) } = E { ~r ( t ..(w) = (9.) = E { 4 z z ( t . We may view the deterministicanalyses considered so far as referring to single sample functions of a stochastic process. Non-Linear Time-Ekequency Distributions and 9.r ) is replaced by its expected value. donot give anyinformationon thetemporal distribution of power or energy.w) = S. Stationary Processes. z ( t )is henceforth defined as a stochastic process. The Wigner-Ville spectrum is of special interest when analyzing nonstationary or cyclo-stationary processes because here the usual terms.292 Chapter 9..the WignerVille spectrum will be discussed for various processes in connection with the standard characterizations. In order to gain information on the stochastic process we define the so-called Wigner-Vdle spectrum as the expected value of the Wigner distribution: with ~ z z ( t r .(t.

For cyclo-stationary processes it is sufficient to integrate over one period T in order to derive the mean power density: (9. an average energy density spectrum can be derived from the Wigner-Ville spectrum as rca For the mean energy we then have E. The Wigner-ViUe Spectrum 293 Processes with Finite Energy. = E {lm 1SW lz(t)I2 d t } = --m 27r w z z ( t . i E Z. For non-stationary processes with infinite energy the power spectral density is not defined.133) Example.The process d ( i ) is assumed to be zero-mean and stationary. g ( t ) is the impulse response of a filter that is excited with statistically independent data d ( i ) . a mean power density is given by (9.134) a=-cc Here.9. (9. The signal z(t) can be viewed as the complex envelope of a real bandpass signal. Now we consider the autocorrelation function of the process z(t). We . w ) dw dt. we consider the signal z(t) = c W d ( i ) g ( t .131) -m Non-Stationary Processes with Infinite Energy. However.iT).4. (9. If we assume that the process z ( t ) has finite energy.132) Cyclo-Stationary Processes. As a simple example of a cyclo-stationary process.

137) is satisfied for arbitrary roll-off factors.”.r 5 l w ~ ~5/ 1 + r . and in general the process ~ ( t )not stationary. which are designed as follows. .~ j=-W W = c $ cc c W W 2=-W E { d * ( i ) d ( j ) } g*(t .. which can be chosen in the region 0 5 r 5 1.(1 .135) As (9. From (9.t+eT). one chooses the filter g ( t ) such that its autocorrelationfunction satisfies the first Nyquist condition: = {0 1 for m = 0. ) t = E { x * ( t ) x ( t+ T ) } = i = .(w) t r:(t) we take ) s.135) shows.”.iT +T). (9. is because the statistical properties repeatperiodically: rzz(t+T. r g ( t ) is a windowed version of the impulse response of the ideal lowpass. r for IwTl/n 2 1 +r.iT) g ( t . :[I 0 + c o s [ g ( w ~ / . 1 1 for IwTl/n 5 1-r.294 obtain r.136) r: (T) Typically. For r = 0 we get the ideal lowpass.138) Here. r is known as the roll-off factor.r ) ) ] ] r for 1 . condition (9. the autocorrelation function depends on t and T . otherwise.138) we derive (9. Nevertheless. it is cyclo-stationary.(w) = (9. Because of the equidistantzeros of the si-function.j T + T) g*(t . For P > 0 the energy density decreases in cosine form.iT) g ( t . (9. for r > 0. For the energy density S.(t Chapter 9. (9.t)=rzz(t+T+eT. Non-Linear Time-Ekequency Distributions + 7. eEZ.137) Commonly used filters are the so-called raised cosine filters.139) As we see.

9.. As can be seen in Figure 9. We observe that for large roll-off factors there are considerable fluctuations in power in the course of a period.( 4 ~ t / T ) ~ ] + + where g(0) = - T( l 1 +P(- = . The Wigner-ViUe Spectrum 295 With (9. and the process ~ ( t ) becomes wide-sense stationary. In orderto show this.r.143) Here the summation is to be performed over the complex exponentials only. In the limit. Thus. . the fluctuations of power decrease with vanishing roll-off factor.142) Figure 9.t ) is written as the inverse Fourier transform of a convolution of G * ( . Figure 9-10).133) these effects are not visible (cf.141) g(t) = nt [l . ~ 'jWkT h/$. by using (9.9.r)/T) (9. (t+.144) we achieve (9.Wtd W .140) the required impulse response g ( t ) can be derived from (9.138) by means of an inverse Fourier transform: (4rtlT) cos(nt(1 r)/T) sin(nt(1. (9. the ideal lowpass is approached (T = 0).4.9 shows three examples of autocorrelation functions with period T and the corresponding Wigner-Ville spectra. 4 (9. When stating the mean power density in the classical way according to (9.1).145) . the autocorrelation functionT.w ) and G ( w ) : e'j(W ~ W').

Integrating over W yields T. Non-Linear Time-Ekequency r= 1 Distributions 0 ‘0 r = 0.1 /l ‘ T ‘0 ‘T Figure 9.296 Chapter 9. Periodic autocorrelation functions andWigner-Ville spectra (raised cosine filter design with various roll-off factors T ) . (t + 7. ) = t (9.146) If G ( w ) is bandlimited to x / T . and the .5 A r = 0.. only the term for k = 0 remains.9.

(9.5).iT . Stationarity within a realizable framework can be obtained introducing by a delay of half a sampling period for the imaginary part of the signal.iT) + j9{d(i)} g ( t . The modified signal reads d ( t )= c W R{d(i)} g ( t .(t+ T 7. t)dt and (9. However. = so r=.T / 2 ) . An example of such a modulation scheme is the well-knownoffset phase shift keying.10. Mean autocorrelation functions mean power spectral density ( r = 0. The Wigner-ViUe Spectrum 297 Figure 9. if we consider realizable systems we must assume a cyclo-stationary process.4.9.149) .148) I=- W Assuming that (9.=.147) This shows that choosing g ( t ) to be the ideal lowpass with bandwidth 7r/T yields a Nyquistsystem in which z ( t ) is a wide-sense stationary process.

According to (9. + If we regard z'(t) as the complex envelope of a real bandpass process xBP(t). + + . t ) and rzrzr( t T . t ) would have to be identical and would have to be dependent only on 7. ) = t We see that only the term for k = 0 remains if G(w) is bandlimited to 2lr/T.150) for the autocorrelation function...t ) and themean autocorrelation function are identical. ( t T ..' (t + 7. T 2-) (9.146) this can be written as T. which is the case for the raised cosine filters.I (t+7-.then we cannot conclude from the wide-sense stationarity of z'(t) the stationarity of zBp(t): for this to be true.(cf.iT) g(t .152) Tzlzr(t 7 .298 Chapter 9. the Wigner-Ville spectrum equals the mean power spectral density. t ) = U d .'. Correspondingly. Section 2.I.. Z 1 g T Hence the autocorrelationfunction T.TE g ( T ) .i-) g ( t 2 %=-m + 7. Non-Linear Time-Ekequency Distributions we have TZtZ' (t + 7.iT + 7) %=-m l l o o T T g*(t-iT--)g(t-iT+T--) 2 2 %=-m o o T g*(t. the autocorrelation functions T.. The autocorrelation function then is (9. t ) = + - 2 c 50: c $ c 1 5ui " g*(t.5).

Regression and the Moore-Penrose Pseudoinverse. Albert. Algorithms for time-frequency signal analysis. Academic Press. Soc. [l41 R.27(2):113-120.A. pages 966-975. April 1992.N. Filter Banks. Riley. [4]J. Reading. 1960. M. IEEE Trans. Multiresolution Signal Decomposition. 1992. [6] H. Allen and L. 1992. [l11 B. 73:1832-1835. Image Processing. [8] R. Haddad.Feb 1992. The Measur ementof Power Spectra Dover.E. 1995. The discrete Hartley transform. Bracewell. J. Mathieu.N.J.Bibliography Signal Analysis: Wavelets. P. Image coding using wavelet transform. [l31 R. New York. SPIE. December 1983. 1(2):205-220. Biemond. IEEE. 1958.W. Longman. Alfred Mertins & Copyright 0 1999 John Wiley Sons Ltd Print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 299 Bibliography [l]A. Cheshire. Weber. Akansu and R. A unified approach to STFT analysis and synthesis. Daubechies.E. New York. New York.N. Antonini. Barlaud. Signal Processing.72:lOlO-1018. Speech. America. November 1977. Blackman and J. Introduction to Matrix Analysis. McGraw-Hill. Akansu and F. [g] R. 1970. 65:1558 1564. Bluestein. Wadas. Signal Processing.Blahut. IEEE Trans. [3] A. On lapped orthogonal transforms. April 1979. Tukey. Proceedings of the IEEE.B.H. [7] R. Bellman. Rabiner.B.R. and J. 1984. Bracewell. Barnard. VCIP. Efficient signal extension for subband/ wavelet decomposition of arbitrary lengthsignals. J. New York. Fast Algorithms for DigitalSignalProcessing Wesley . . 1972. Addison [l01L. volume 2094. 40(2):439-443. and I. Boashash and A. - [5] M. [l21 S. In Proc. Suppression of acoustic noise in speech using spectral subtraction. Acoust. Time-Frequency Transforms and Applications. Audio and Electroacoustics. Proc. November 1993. Boashash (ed.).N. The fast Hartley transform. MA. In Time-Frequency Signal Analysis. A linear filtering approach to the computation of the discrete Fourier transform. B. 18:451-455.. Opt. Academic Press. [2] A. Boll. IEEE Trans. IEEE Trans.

Nguyen. University Press. IEEE Trans. 37(2):189-193. IEEE Trans. C.F.G. and G. New York.K. 1992. [25] C. De Bruijn. Staelin. on Communications.1980. Acoust. The Hartley Transform.276-300. Selected Reprint Papers. [20] C. Munchen.H.A. Reflected boundary conditions for multirate filter banks. .37thMeeting Society of Exploration Geophysicists. Maximum entropy spectral analysis. 35: 217-250. Journal of the Audio Engineering Society. Nieuw Archief voor Wiskunde (3). UK.S.SignalProcessing. [l81 N. IEEE Press. Speech.N.276.C. 1981.Brislawn. Dehery. A Handbook of Fourier Theorems.N. D. Burg. 29:806-817. Bracewell. Time-Frequency and Time-Scale Analysis. In Proc. Conf. A theory of generalized functions. and V. Res. Acoust. 1980. Symp. [l61 J. 1987. Burrus and P. [26] T. June 1994. The wigner distribution . Encoding of images based on a lapped orthogonal transform. IEEE Int.parts 1-111. Circuits and Systems. An in-place in-order prime factorFFT algorithm. F. In Proc. IEEE Int. 1978. Faber.I. New York.300 Bibliography Press. Childers (ed.. An Introduction to Wavelets.M. 37(6):862-871.G.Stoll.D. Coffrey.J.a tool for time-frequency signal analysis . [l51 R. Eschenbacher. Oklahoma City. de Jager. and K. Oxford University Oxford. Cambridge [23] L.Johnston. Williams. pages 2. Speech. IEEE Trans. Philips J...G. In Proc.Brandenburg.M. pages2129-2132.M.). [24] H. 42(10):780-792. 1992. MPEG-1 audio: A generic standard for coding of high-quality digital audio. pages 307-310. T. Mecklenbrauker. 1985. Academic Press. New York.The ISO[l71 K.. [22] D. Int. [21] P. [27] M. Signal Processing. 1989. October 1967. Boundary compensated wavelet bases. Reprint in Modern Spectrum Analysis. London. Choi and W.G. Symmetric extension methods for M-channel PR LP FIR analysis/synthesis systems. Canada. Cassereau.C.P.2.273 . Signal Processing. D. Bradley. In Proc. Acoust. Speech. [l91 J.Chui. February 1989. Improved time-frequency representation on multicomponent signals using exponential kernels. with applications to Wigner distribution and Weyl correspondence.P.372-389. Champeney. Chen. Claasen and W. Chan. October 1994. Symp. XXI:205-280. andJ.

[31] J. Donoho. [36] I. 301 [28] A. [34] I. SIAM. [29] L. B.. Phys..on Inf . March 1992. [41] Z. pages 1296 .. volume 11. Cheshire. In TimeFrequency Signal Analysis. October 1988. In Proc. Doganata.B. 1983. Daubechies. USA. Speech. Math. 1965. Detroit. 1992. 19:297-301. Acoust. [30] L. Orthonormal bases of compactlysupported Comm. Root.Bibliography May 1997. 46(5):1245-1255. Boashash (ed. Signal Processing. Generalized phase-space distribution functions. J. McGrawHill. Multirate Digital Signal Processing..Optimalorthogonalboundary banks. Factoring wavelet transforms intolifting steps.Q. Crochiere and L.R. [42] D. Feauveau. Acoust. An algorithm for machine calculationof complex Fourier series. NJ. 1958. De-noising by soft-thresholding. IEEE Trans. Sweldens. 1998. pages 613 . Oversampled filter banks. on Inf Theory. Introduction: A primer on time-frequency analysis. San Francisco. New York. Rabiner. Random Signals and Noise. P.P. [33] G. Pure and Appl..1992. 1992. Appl. 1988. Daubechies.L. [32] R. IEEEInt. Englewood Cliffs. Speech. and T. Subband processing of finite length signals without borderdistortions. IEEE Trans. May 1995. 4(3):245-267. Cohen. Rao. Conf. Nguyen. Davenport and W. September 1990. [37] I. [38] W. Math. Acoust.L. May 1998. Fourier Anal. Cohen..). Daubechies. J. Comm. Conf. IEEE Trans. USA. of Math.C. Longman. [39] R.R. Daubechies. IEEE Trans. de Queiroz. 7~781-786. Comp. Cooley and J. Ten Lectures on Wavelets.. Signal Processing. Math.1966.In Proc. wavelets. CvetkoviC and M.E. Daubechies and W.616.L. filter [40] R.L. General synthesis procedures for FIR lossless transfer matrices.1299. for perfect reconstruction multirate filter bank applications. 36(10):1561-1574. [35] I. time-frequency localization and signal analysis. Cohen. de Queiroz and K. Speech. 41:909-996. and J. Vaidyanathan. 36(5):961-1005.Signal Processing.Biorthogonal bases of compactly supported wavelets. IEEEInt. SignalProcessing. XLV:485560. I. The wavelet transform. Prentice-Hall. volume IV. Vetterli. Tukey. Pure and Appl.

NM. Wiley. [51] Y. Statistical-model-based speech enhancement systems. 1984. via wavelet [44] R. Signal Processing. real. J. [49] Y. FFT algorithms. [45] P. August 1985.L. Albuquerque. MPEG Video . In Proc. and W. 1994.M. Springer. D. Speech.. 81:425-455. Dutilleux. 46:306-316. Signal Processing. Affine smoothing of the Wigner-Ville distribution. [56] C. Conf. Multirate Digital Signal Processing. Acoust. Ephraim.J. Pennebaker. Meerkotter. [54] P. Acoust. Chichester. and K. of quadrature mirrorfilters [52]D. J. Fettweis. Electron. 1989. 1994.E. [46] P. IEEE Int. 1986. IEEE Trans. Malah. Speech enhancement using a minimum mean-squarelog-spectralamplitudeestimator. Speech. 41(3):613-627. pages 191-195. Fliege. IEEE Trans. Ephraimand D. Durbin. 20:14-16. Ephraimand D. Speech. [50] Y. pages 289-304. April 1985. Duhamel. 33(2):443-445. In Wavelets: Time-Frequency Methods and Phase Space.. Efficient estimation of parameters in moving average models. Acoust. Speech. Malah. 33(4):893-901.. UK. Biometrica. Proceedings of the IEEE. [43] D. Signal Processing. Signal Processing. Acoust. 34:285-295.Application to split band voice coding schemes. John Wiley and Sons.. Esteban and C. [48] P. 1959.Implementation of the split-radix FFT algorithms for complex. [53] A. Acoust. Conf. LeGall.J. Rioul. An implementation of the algorithm B trous to compute the wavelet transform. New York. IEEE Duns. IEEE Trans. [47] J. IPTI. 32(6):1109-1121.December 1984. Biometrika.L. Mitchell. Duhamel and H.. Speech. pages 2455-2458. Ideal spatial adaptation shrinkage. Duda and P. [55] N. Pattern Classification and Scene Analysis. May 1995. New York. Reconstruction of signals after filtering and sampling rate reduction. April 1990. October 1992.302 Bibliography Theory.A. Split radix Letters. Nossek.B. 80(10):1526-1555. Donoho and I. Hart. 1973. In Proc.Galand. Hollmann. May 1977. Acoust. and real-symmetric data. Fogg.SignalProcessing. Speech enhancement using a minimum mean-square short-time error spectral amplitude estimator. Flandrin and 0. Signal Processing.O. Johnstone. Speech. IEEE Int..

December 1993. Acoust. NY. Karp. 1996. KovaEeviC. Francks.M. Good.G. Boundary filters for finite-length signals and time-varying filter banks. 1996. New York. [59] K.L Hartley. [60] S. Seattle. Vetterli. on Signal Processing. 1991. Transmission of Information b y Orthogonal Functions. March 1942. A formulation of modulated filter banks. Springer. Gabor. Signal Processing.S. [67] R.V. Kluwer Academic Publishers. [63] A. Xia. IEEE Trans. R. Fourier analysis. Furui Marcel Dekker.N. Kronland-Martinet. New York. Introduction to Statistical Pattern Recognition. Vector Quantization and Signal Compression.. In Proc. New York. Prentice-Hall. Theory of communication. 1991. IEEE Int. K. IEEE Trans. Fukunaga. 42(2):102-114. and T. pages 2-20. Fu. Wavelets and recursive filter banks. IPTI. Springer. Morlet. Electrical Engineers. Signal Theory. and M. Sondhi. Ramachandran.Bibliography Compression Standard. Circuits and Systems 11. 1989. Heller. Speech. Gray. Proc. 1969. 303 [57] L.Q.Herley and M. 30:144-150. [58] K.and M. Signal Processing. Vetterli. New York. A more symmetrical Fourier analysis applied to transmission problems. 93:429-439. 1972. Chapman & Hall. [66] H.. Reading and understanding continuouswavelet transforms. . Gersho and R. Journal of the Institutefor [62] X. Springer. Tilings of the time-frequency plane: construction of arbitrary orthogonal bases and fast tiling algorithms.E. J. New York. He. J. Englewood Cliffs. Soc. [61] D. [69]C. [65] A. T. Nguyen. Conf.Herley. 20:361-372. Academic Press. [68] P. Grossmann. IEEE Trans. Submitted to IEEE Trans. February 1995. The interaction algorithm and practical Royal Stat. Harmuth. 41(12):33413359. Advances in Speech Signal Processing.M. [64] I. New York. 1958. [71]C. pages 1465-1468.Herley. Boston. 1946. NJ. May 1998. August 1993. 1972. Digital Pattern Recognition. 41(8):2536-2556. of the Inst. [70]C. volume 3. and X. 1980. A new implementation of arbitrary-length cosine-modulated filter bank. Radio Eng. Z. Signal Processing. In Time-Frequency Methods and Phase Space. Series B. and J. G m .

Hlawatsch and G. Seattle. Mertins. [79] Joint photographic experts group. CA. pages 744-747.. Englewood Cliffs. J. R. Mdft filter banks with perfect reconstruction. ISO/IEC JTCl/SC2/WG8. and Ph. Boudreaux-Bartels. [80] E.S. Finland. Phys. [83] T. [77] G. Prentice-Hall. Design of paraunitary oversampled cosinemodulated filter banks. 1987. 1988.. Tchamitchian. February 1998. NJ. Speech. 1984. Janssen. Jenkins and D. [81] K.37 Math. IEEE Int. Wiley.M.I. Mertins. Modern Spectral Estimation. A. [84] S. IPTI. Helsinki.A. Schuller. 1964. May 1995. 14:243-252. NJ. Johnston. In Proc. San Francisco. pages 291-294. Signal Processing.J. Acoust. Speech. Sci. Recent trends in the design of biorthogonal modulated filter banks. 1989. New York. CCITT SGVIII. and G. [85] J. In Proc. [74] J.Karhunen. Noll. A.H. Spectral Analysis and its Applications.G. 1990. IEEE SignalProcessingMagazine.E. JPEG technical specification: ReviAugust sion (DRAFT). Ser. Karp and N. Conf.A filter family designed for use inquadrature mirrorfilter banks. DigitalCoding of Waveforms. [73] M. 1990. Acad. Holden Day. 1968. 9(2):21-67.M. Ann. 1980. Englewood Cliffs. Linear and quadratic timefrequency signal representations. Kay. Fenn. 1947. pages 286-297. SignalProcessing. IEEE Int. Tampere. 20:279-292. Acoust. Conf. Application of an efficient parallel IIR filter bank to imagesubband coding. April 1992. Holschneider.M. IEEEInt. Theory and Application of the z-Transform Method. Ramstad. Husmy and T. Kronland-Martinet.F. Jury. Watts. Circuits and Systems. New York. Positivity of time-frequency distribution functions. In Proc. Fliege. Springer. Signal Processing. Prentice-Hall. [82] T.D.In Proc. Workshop On Transforms and Filter Banks '98. Jayant and P. A real-time algorithm for signal analysis with the help of the wavelet transform. [75] A. Karp. Signal . Symp.uber lineare methoden in der wahrscheinlichkeitsrechnung.304 Bibliography [72] F. Morlet. [76]N. [78] J.. Kliewer and A. In Time-Frequency Methodsand Phase Space.I.

H. Mertins.S.D. Staelin. Parks. volume 3. 40:770-783. Malvar. Mallat. CA. [92] S. 1992. [97]H.P.Speech. IEEE Trans. 46(4):941-955. Anal.. . Acoust. IEEE Trans. Malvar. Multifrequency channel decomposition of images and wavelet models. Signal Processing.. N. 1988..TheLOT: Transform coding without blocking effects. Mallat. April 1992. 1977. Speech. 25:261-278.Lappedbiorthogonaltransforms for transform coding Proc. pages 2073-2076. San Diego. [91] S. Acoust.1988. [95] H. IEEE Trans. Kliewer and A. [89] Levinson. The Wiener RMS errorcriterion prediction. Symp. April 1997. 40. Acoust. 1947. Extended lapped transforms: Fast algorithms and applications. A Wavelet Tour of Signal Processing.S.. Phys. [86] J . Signal Processing. IEEE Trans. Staelin. 11(7):674-693. A prime factor FFT algorithm using high speed convolution. [93] H. 1997. J. [88] D. MA. April 1998. Signal Processing with Lapped Norwood. [96] H. IEEEInt. pages2421-2424.. Intell. in filter design and [go] S. November 1992.SignalProcessing. [87] R. Malvar. Germany. 37:2091-2110. Munchen. pages 781-784. Transforms.S. Signal Processing. Reduction of blocking effects in image coding with a lapped-orthogonal transform. In Conf. Mallat.Cosine-modulated FIR filter banks satisfying perfect reconstruction.Malvar and D. and D.H.Bibliography 305 Processing. Artech House. IEEE Trans. Conf.S. Math. IEEE Int.S. Speech. IEEE Trans. Speech. Munich. with reduced blocking and ringing artifacts. pages 835-838. Vaidyanathan. December 1989. Germany. and Mach. Acoust. In Proc. April 1997.S. IEEE Int. Oversampled cosine-modulated filter banks with low system delay. July 1989. volume 3.In Proc. Signal Processing. Kolba and T..SignalProcessing. IEEE Trans. Circuits and Systems. Speech. A theory for multiresolution signal decomposition: The wavelet representation. [94]H.. Signal Processing.SignalProcessing. Koilpillai and P. [98] Malvar H. Academic Press. Acoust. The LOT: A link between transform coding and multirate filterbanks. 25:281294. Malvar. on Putt.

In Proc. 1998. January 1994. 1987. Version 11. Detroit. Ecole [l051 J. Chicago. ISO/IEC JTCl/SC 29/WG 11. MN.J. Nguyen and P. NJ.P. 45:99-124. Generic Coding of Moving Pictures and Associated Audio.Q. [l111 T.306 37(4):553-559. and M. Speech. Springer.. Minneapolis. volume 1. Signal Processing. Signal Processing. October 1998. Bibliography Prentice- [loo] J. IEEE Trans. Sampling theory and wave propagation. New York. Se'minare EDP. G. 47:203-236. Morlet. IEEE Trans. Hall. ISO/IEC CD 13818-2.E. Meyer. Acoust. USA. Giard. pages 233-261. [l031 A. In Proc. IEEE Int. [l011 A. [l081 MPEG-2 Video Compression Standard. banks with linear-phase prototype filter. Geophysics. 46(10):2812-2818. 1982. Englewood Cliffs. Generic Coding of Moving Pictures and Associated Audio. fast algorithms and attractive new properties.228. Digital Spectral AnalysiswithApplications. Quantum mechanics asastatisticaltheory. pages 111. Arens. Subspace approach for the design of cosine-modulated filter IEEE Trans. [l091 MPEG-4 Video Verification Model. Mertins. Wave propagation and sampling theory. September 1995.. 1993. In Proc.Q. SP42:24-31. [l071 J. May 1995. Nayebi.T.L. Ondelettes et fonction splines. Smith. Signal Processing. December 1986. I. Issues in acousticsignal/imageprocessingandrecognition. 1983.. April 1993. [l021 A. IEEE Int. Barnwell 111. Heller. [l101 K. Speech. Biorthogonalcosine-modulated filter . Conf. Camb. and D. Perfect reconstruction modulated filter banks. Signal Processing. [99] S.Phil. Signal Processing.225-111.N. wavelets. Mertins. Digital filter bank design quadratic constraint formulation. Acoust. Mertins. Time-varying and support preservative filter banks: Design of optimal transition and boundary filters via SVD. [l041 Y. Optimized biorthogonal shape adaptive ICIP'98. Low delay FIR filter banks: Design and evaluation. [l061 J. T. April 1989. Nguyen. Polytechnique. Moyal. Fourgeau. [l121 T. In NATO ASI series.pages 1316-1319. Morlet. 43:2103-2108. Marple Jr. Mau. Conf. SOC. 1949. October 1998. Paris.

NJ. 1975. PrenticeHall. Academic Press. Digital Signal Processing. Proakis. Speech. [l231 W. IEEE.L. [l181 R. USA. 5:1107-1108. [l161 J. on Inf.SignalProcessing. Conf.H. IEEE Trans.Raoand P. Tanem.. 1992. 14(3):369-374. [l191 L. Englewood Cliffs. and K.G.P. Signal Processing. [l271 J. [l251 0. McGraw-Hill. Englewood Cliffs.A. IEEE Trans.R. 1990. [l241 A. de Queiroz.B.SignalProcessing. Flandrin.Bibliography banks. Papoulis. Speech. New York.V.. Princen and A. Rothweiler. IEEE Int. November 1970. [l151 A.Q. Theory. [l131 A. C. 1992. Simply regularity criteria for subdivision schemes.M. Toronto. Ramstad and J.Furtherdecomposition of the karhunen lokve series representation of a stationary randomprocess. 1978. Anal. 1968. pages1789-1792.W. New York. [l261 0. T. IEEEInt.L. October 1986. Speech. IEEE Trans. Atlanta. In Proc. Acoust. F. Acoust. Proc. [l201 C.R. Analysis / synthesis filter bank design based ontime domain aliasing cancellation. Ont. Time-scale energy distributions: A general class extending wavelet transforms. Signal Analysis. Macmillan. Prentice-Hall.R. IEEE Trans. Signal Processing. DiscreteCosineTransform. Discrete Fourier transform when the number of data samples is prime. Rabiner and R. SIAM J. Papoulis. [l221 K. IT-16:663-668. 307 Conf. Nikias. and its Applications. McGraw-Hill. [l211 T. Acoust. 1977. Rao. 23:1544-1576.a new subband coding . Math.July 1992. May 1996. Bradley.M. Schafer.. In Proc. 34:1153-1161. Rioul and P. Signal energy distribution in time and frequency. Rader. DigitalProcessing of Speech Signals. Ray and R. Oppenheim and R. Cosine modulatedanalysis-synthesis filter bank with critical sampling and perfect reconstruction. IEEE Trans.. on Inf. March 1996. TheFourierIntegral New York.P. Driver. New York. 1968. [l171 J. Rihaczek. The GenLOT: Generalized linear-phaselappedorthogonaltransform.W. Theory.. NJ. 44(3):497-507. and C. 1962.W.D. [l141 A. Schafer.Signal Processing. Rioul. Advanced Digital Signal Processing. May 1991. Yip. Rader. Ling. Nguyen. Canada. Polyphase quadrature filters .

P.M. 34:434-441. Signal Processing.J. Sorensen. 147:205-232. 40(10):2464-2482.Shum. A new framework for modulated perfect reconstruction filter banks. June 1996.L. pages 1446-1456. May 1993. Math. Signal Processing. Signal Processing. IEEE Trans. IEEE Trans.J.K. May 1996. On power series which are bounded in the interior of the unit circle. Circ.SignalProcessing. 41(12):3480-3496.andW. [l351 M. Schur. In Proc. On calculation the splitradix FFT. 1986. for Video Technology.1983. Exact reconstruction for treestructured subband coders. J. [l321 M. D. factorizations and applications.. IEEE Trans. 5(5):689-704. August 1996. IEEE Int.T. Burrus. [l291 G.Brown.Q. [l381 H.SmithandT. Signal Processing. Analysis/synthesis techniques for subband coding. Signal Processing. 34:152-156. Aug 1990.On computing the discrete Hartley transform. October 1992. [l341 E. Speech processingwithWalshIEEE Trans. pages 174-179. Linear phase paraunitary filter banks: Theory. June 1973. 41(12):3445-3462. Soman. Schuller and M. Acoust. Heidemann. The discrete wavelet transform: Wedding the B trous and Mallat algorithms. Shapiro.V. ..andM. Said and W.B. [l311 J. Heidemann. 1917.. IEEE Trans. Angew. C S . Speech. Signal Processing. Elliot. da Silva and M. A. 44(8):1941-1954.T. Speech. Embedded image coding using zerotrees of waveletcoefficients. Audio on and Electroacoustics. IEEE Trans. pages 1280-1283. [l281 A. Sorensen. Speech. IEEE Trans. M. Speech. Burrus.T. IEEE Trans. T.D.SignalProcessing.J. [l331 F. October 1985. 33:1231-1238. IEEE Trans. 6(3):243-250. Reine U. June 1986. Acoust. [l301 I. A new fast and efficient image codec based on set partitioning in hierarchical trees. and C S . [l361 M..T. Shensa.A. Speech.. Acoust. (in German). [l391 H.P. IEEE Trans. Ghanbari. Pearlman. Vaidyanathan. Hadamardtransforms.T. Barnwell 111. On the performance of linear phase wavelet transforms in low bit-rate image coding.J. and Syst. Conf. December 1993.A.308 Bibliography technique.L. IEEE Trans. Image Processing. Signal Processing.. and Nguyen. Eddins. Smith. [l371 A.P. Acoust. Acoust. Smith and S.Jones.

1971. [l461 P. Comput. Duhamel.Q. Seattle.Bibliography 309 [l401 G. Conf. WellesleyCambridge Press. Vetterli. Split radix algorithms for length p" dfts. 1996. Vaidyanathanand P. [l451 P.Q.D. and Modulation Theory. July 1989. 3(2):186-200. Detection. Estimation.. Thomas.Latticestructures design and robust implementation of two-band perfect reconstruction QMF banks.. MA. and T. Journal of Appl. 34:57-64. December 1985. Acoustics.Signal Processing. 1996. pages 1441-2424. May 1998.P. Conf. Acoust. 1988. 1993. and Comput. IEEE Trans. The lifting scheme: A custom design construction of biorthogonal wavelets. Speech. On power-complementary FIR filters. Perfect transmultiplexers. [l421C.P. [l491H. USA. 3623-95. Speech. Vaidyanathan. Applications of Digital Computers.. Englewood Cliffs. Estimation. and Modulation Theory.9. for optimal [l471 P. IEEE Trans. Tran.andSignalProcessing. pages 48. . NJ. 1989. Vetterli and P. IEEE Trans. [l481 P. StrangandT. SIAM J. The generalized lapped biorthogonal transform. New York. Z. 32:1308-1310. Nguyen. Doganata. Wiley. IEEE Int. Wiley. IEEE Int. Nguyen. Prentice-Hall. [l521 M. MA.4.L. 12:808-823. volume 3. Vaidyanathan.1-48. In Proc.P.P. Tokyo. [l501H. Acoust. Using a computer to solve problems in physics. Multirate Systems and Filter Banks. September 1992. In [l441 T. Wellesley. 1991. Speech. R. 1968. Speech. IEEE Trans. In Proc. and T. Circuits and Systems. Part III.L. Acoust. Signal Processing. Signal Processing. Signal Processing. Saramaki. Nguyen. 40(9):22072232.9. Detection. Sci. [l531 M. [l431 L.. Temperton. Harmonic Analysis. 37(7):1042-1056. Acoust. Herley. New York. Sweldens. 1963. [l511 M. Ginn.. de Queiroz.Van Trees. Self-sorting in-place fast Fourier transforms. IEEE Trans. T. Improved technique for design of perfect reconstruction FIR QMF banks with lossless polyphase matrices. Wavelets and Filter Banks.Van Trees. Hoang. Speech. Signal Processing.. [l411 W.Q. Part I. Speech. Acoust. Boston. Wavelets and filter banks:Theoryand design. Vaidyanathan. Vetterli and C.L. April 1986.

Welch. Audio and Electroacoustics. Belzer. Englewood Cliffs. Wang. and Comput. odic case. Subbandcoding of images. Math. [l661H. 1948.K. The use of the fast Fourier transformfor the estimation of power spectra: A method based on time averaging over short modified periodograms. Appl. Periodic and bounded case. Wang. 1981. 4(8):1053-1060. 38(7):1084-1091. and Comput. 32~175-199.Y. Speech. and A. Theorie et applications de la notion de signal analytique.-S.E. Winograd.. [l681 Y.15:70-73. August 1984. 9:53-73. Ville. Wigner. IEEE Trans. NJ. Acoust. Liao. IEEE Trans. Wang. Math. Appl. [l671 T. Harmonic analysis with a real frequency function-11. 34(5):1278-1288. KovaEeviC.. pages 61-74.W. Wallace. Math. 34:31-44. 44(7):1657-1668. Acoust.1967. Efficient iterative design method for cosine-modulated QMF banks.1978. Speech. Aperi- [l591 Z. [l601 Z. April 1991. Image Processing. 2 A . Calvert. IEEE Trans. Antoniou. Young and T. Math. On [l641 S. and R. August 1995. 40:749-759. Hall. Elsevier. Woods and S.. [l571 G. Wang. 1932.D. IEEE Trans.. Signal Processing.P. New York. Harmonic analysis with a real frequency function-I. Appl. Speech. B. Physical Review.. 9:245-255. Zhao. of IEEE Trans. [l651 J. O’Neil.. [l561 J. 1995. IEEE Trans. Cables et Transmission. and Comput. Xu. Villasenor. Communications of the ACM.SignalProcessing. 212303-816. . Wavelets andSubbandCoding. Lu. 1974.D. Signal Processing. 9:153-163. [l551 J. The use of cone-shaped kernels for generalized time-frequency representations non-stationary signals. and J. Vetterli and J. W.. July 1996. Acoust. Marks 11. the discrete Fourier transform. May 1986. L.310 Bibliography Prentice- [l541 M. Harmonic analysis with a real frequency function-111. On the quantum correction for thermodynamic equilibrium. [l581 Z. Wavelet filter evaluation for image compression. Signal Processing. Atlas. computing Camp. 1981. [l631 E.1990. The JPEG still picture compression standard. Fast algorithms for the discrete Wtransfomand for the discrete Fourier transform. Estimation and Pattern Recognition. sequence. [l621 P. Data [l611 Z. Classification. 1981.J.

289 wavelet transform. 36ff.. Filter Banks. 39 signal. 188 A trous algorithm. 265ff. 18 Autocovariance sequence for random processes. 256ff. 210 Bandwidth. Autocorrelation function for analytic processes. 205 Banach space. Alfred Mertins Copyright © 1999 John Wiley & Sons Ltd Print ISBN 0-471-98626-7 Electronic ISBN 0-470-84183-4 Index AC matrix (see Modulation matrix) Admissibility condition. 267 Autocorrelation matrix. 269-271 time-frequency. 128 relationship between KLT and DCT. 109. 214 Alias component (AC) matrix(see Modulation matrix) Aliasing. 94 relationship between KLT and DST. 137-139 Bartlett window. 43 Bandpass analysis and wavelet transform. 173 Ambiguity function. 16 Autoregressive process.Signal Analysis: Waveless. 9 for random processes. 293 stationary process. 141 Basis. 97 Balian-Low theorem. 134 for deterministic signals. 197. 14 for wide-sense stationary processes. 16 Autocovariance function for random processes. 214 AR( 1) process (see Autoregressive process) Arithmetic coding. 146 All-polesourcemodel (see Autoregressive process) allpass filters. 214 Analytic wavelet. 215. 133-135. 8 for ergodic processes. 131 Autocorrelation sequence estimation. 222 Affine invariance time-frequency distributions. 133. 18 Autocorrelation method. 19 Bandpass filtering. 211 Bartlett method. 53 311 . 211. spectrum. 269 generalized. 13 for wide-sensestationary processes. 14. 18 for random processes. 15 Autocovariance matrix. 149 Analysis window. 216 Analytic signal. 48 Bessel inequality. 17. 13. 35. 4 Bandlimited white noise process. 14 temporal. 281 Amplitude distortion. 138. 298 stationary processes. 44 for deterministic signals. Time-Frequency Transforms and Applications. cross.

123 Burg method. 274 Choi-Williams distribution. 207 DHT (see Discrete Hartley transform) Dirac impulse. 175 Index oversampled. 183 paraunitary. Correlation matrix. 58. 224. 132 Critical subsampling. 137. 292. 8285. 93-95 Discrete Fourier transform(DFT). 93. 297 Daubechies wavelets. 174 Biorthogonal lapped transforms. Cohen-Daubechies-Feauveau wavelets. 47 BLUE. 8ff. 18 Cross correlation sequence for deterministic signals. 172. 170ff. 64. 87 Cauchy integral theorem. 114. 134. 10 for random processes. 9 discrete-time signals.. 141 Blackman-Tukey method. 18 Cross covariance sequence for random processes. 247 Coiflets. 253 Complete orthonormal sets. 11 Cyclo-stationary process. 10 Cross power spectral density for continuous-time random processes. 19. 29 Cosine-modulated filter banks. 136. 172 polyphase. oversampled. 84 Circular convolution. 298 Conditional probability density. 53 Complex envelope. 14. 143 Cross ambiguity function. 174ff. biorthogonal. 112 Circulant matrix.312 Best linear unbiased estimator (BLUE). 16 Cross correlation matrix. 77 Cauchy-Schwarz inequality (see Schwarz inequality) Center frequency. 36ff. 186 Biorthogonal wavelets. 227 Bit reversed order. 293. 54 Chinese remainder theorem. 52 Completeness relation. 211 Characteristic function. 275 Cross-terms. 281ff. 288-291 Cholesky decomposition. 165. 97. 99. 232 Denoising. 17 Cross energy density spectrum continuous-time signals. 11 DFT (see Discrete Fourier transform) DFT filter banks. 187 Covariance method. 18 Cosine-and-sine function. 17. 165. 275 Cumulative distribution function. 293. 123 Bezout’s theorem. 139 Block transforms. 9 for random processes. 252 DCT (see Discrete cosine transform) Decimation. 144 Decomposition relation. 137 . 12 Consistency. 16 for discrete-time random processes. 140 Constant-Q analysis. 17. 224. 247 Biorthogonality condition. 263 Density function. 99. 208. 138. 13 Chebyshev polynomials. 49 Discrete cosine transform (DCT). 114. 142 Butterfly graph. 190 Cohen’s class. 179 Counter identity matrix. 149 cosine-modulated. 248 Bias (see Unbiasedness) Biorthogonal filter banks. 140. 23 Direct sum of subspaces. 17 Cross covariance matrix. 93 Chirp signal. 17 Cross Wigner distribution. 83. 133. 269 Cross correlation function for deterministic signals. 174 critically subsampled. 211 Correlation. 88 Blackman window.

2. DFT. 4 Expected value. 23. 134 of power spectral densities. 85-88. 174ff. 109 Fourier series expansion. 53 Fourier transform for continuous-time signals. 114-116 least-squares. 162 Euclidean metric. 92 Fast Fourier transform (FFT). 91 Fast Hartley transform (FHT). 170ff. 54. 90-93. 90 split-radix FFT. 58. 47ff. 1 Ergodic processes. 134-142 Euclidean algorithm. 224 ELT (see Extendedlappedtransform) Embedded zerotree coding. 81 Downsampling. Discrete-time Fourier transform. 137 313 Cooley-Tukey FFT. 164ff. 88 radix-2 decimation-in-time. 188 First-order Markov process. 85 radix-4 decimation-in-frequency. 286 Gabor expansion. 53 Discrete sine transform (DST). 199. 186 GenLOT (see Generalized lapped orthogonal transforms) Givens rotations. 9799 Discrete polynomial transforms Chebyshev polynomials. 204 Gabor transform. 113. 55 Legendre polynomials. 9. tree-structured. 19 Generalized orthogonal lapped transforms. 168 modulated. 73 Gram-Schmidt procedure. 144 cosine-modulated. 97. basic multirate operations. Discrete wavelet transform (DWT). 51 Grammian matrix. 227ff. 274 Gaussian signal. 92 in place computation. 4 Euclidean norm. two-channel paraunitary. 143ff. 181 Fast Fourier transform Good-Thomas FFT. 12 Extended lapped transform (ELT). 223. 65. 197 Gaussian function. 99 FFT (see Fast Fourier transform) FHT (see Fast Hartley transform) Filter banks. 144 Dual wavelet. 262 Energy density spectrum computation via Hartley transform. 223 DWT (see Discrete wavelet transform) Dyadic sampling. 33 for continuous-time signals. 10 Energy signal. 162 two-channel. 26ff. 114 minimum error mean square (MMSE). 224-226 Frequency resolution. 147ff. 116-123 of autocorrelationsequences. 99. 158. M-channel. 156 general FIR. 170ff. 201. two-dimensional. 188. 80. 18 Estimation. for discrete-time signals. 42 .Index Discrete Hartley transform (DHT). 97. 155ff. 81 Frame bounds. 268. 55 Laguerre polynomials. 213. 188. 64 Group delay. 96. 159. 197. 113 best linear unbiased. 274 Gaussian white noise process.215. 97 Discrete transforms. 8 for discrete-time signals. 54 Hermite polynomials.216. 143. 167 lattice structures. polyphase decomposition. 148ff. 80.133. FIR prototype. 88 radix-2 decimation-in-frequency.

113 Legendre polynomials. Instantaneous frequency. 215 discrete-time. 53 Levinson-Durbin recursion. 260 Moyal’s formula for auto-Wigner distributions. Modulated lapped transform. 95 Karhunen-LoBve transform (KLT). 49 Lagrange halfband filter. 70 Householder reflections. 168 Least-squares estimation. 244 Moments of a random variable. 124-132 Linear prediction. 12 JPEG. 34ff. 186ff. 156 Hamming distance. 181 Modulation matrix. 24 KLT (see Karhunen-LoBve transform) Kronecker symbol. 174. metric space. 178 MDFT filter bank. 140. 273 cross-Wigner distributions. 280 Integral transforms. Interference term. 109 Kernel of an integral transform. 177 Linear estimation (see Estimation) Linear optimal filters. 66-68 Morlet wavelet. 29ff. 141 Hanning window. 229. 84 for continuous-time processes. 245 Haar wavelet. 172 Metric. 12 Moore-Penrose pseudoinverse. 4ff. 64. 126-130 Linear subspace. 276 MPEG. Lattice structure for linear-phase filter banks. 154 Moments. Hartley transform. 144 Joint Probability Density. 9799 Hermite functions. 243. 160 maximum-delay. 48 Linear-phase wavelets. 131 Lifting. 22. 22ff. 170ff. 56 Hermite polynomials. first-order. 141 Hard thresholding. 274. 192 MRA (see Multiresolution analysis) . 277 Interpolation. 263 Hartley transform. 100 Halfband filter. 7 Hilbert transform. 140. 5 Hilbert space. 38 Inner product spaces. 174 Mallat algorithm. discrete (DHT). 152. 23 self-reciprocal. 95. 173 In-phase component.314 Holder Regularity. 55 Hermitian of a vector. 69-72 Huffman coding. 103-108 for real-valued AR( 1) processes. 4 Hamming window. 188 IIR filters. 240 Hadamard transform. 22 reciprocal. 247 LOT (see Lapped orthogonal transforms) Low-delay filter banks. 116-123 MLT (see Modulated lapped transform) MMSE estimation (see Minimum mean square error estimator) Modulated filter banks. 259 Markov process. 65. 178 zero-delay. 54 Lapped orthogonal transforms. 156 Maximum-delay lifting. 257 Index Laguerre polynomials. 109 Matched-filter condition. 158 for M-channel filter banks. 3 Minimum mean square error estimation (MMSE). Householder reflections. 101103 for discrete-time processes. 159 for paraunitary filter banks.

147ff. 11 Product kernel. 143. 225 Oversampled cosine-modulated filter banks. 38 Quadrature mirror filters. 128 of orthogonal projection. 166 two-channel filter bank. 148 . 13 stationary. 42 Pitch. 208 Non-stationary process. 38 Prediction. normed space. 149. 167 Periodogram. 279 Pseudoinverse (see Moore-Penrose pseudoinverse) QCLS (see Quadratic-constrained leastsquares) QMF (see Quadrature mirror filters) QR decomposition. 124-132 Orthogonal projection. 53 for the Fourier transform. 292 Norm. 90. 49. 14 Random variable. axioms of. 117 Orthonormal basis. 3 Normal equations of linear prediction. 165. 156 P R (see Perfect reconstruction) P R condition two-channel filter banks. 1Off. 202 Polyphase decomposition. 10 Reciprocal basis. 147 type-3. 237 Parseval’s relation for discrete representations. 85-88. 134-142 for continuous-time random processes. 150. 268. 225 Optimal filters. 269 Radix-2 FFT. 294 Octave-band analysis. 155. 16 Power-complementary filters. 91 Raised cosine filter. 90 Radix-4 FFT. 11 Probability. 25 Parseval’s theorem. 14 non-stationary. 151. 294 Random process. 289 Pseudo-QMF banks. cyclo-stationary. 176 Quadrature component. 168 two-channel. 211. 2 Power spectral density estimation of. 28. 154 Power signal. 184 Pseudo-Wigner distribution. Pre-envelope. 149 Radar uncertainty principle. 172 Paraconjugation. type-l. 68. 28 general. 51. 135-140 Phase delay. 130 Probability density function. 49 Orthogonal sum of subspaces. 126-130 Prediction error filter. 20 wide-sense stationary.Index Multiresolution analysis. 50. 73 Quadratic-constrained least-squares. 227 Noise reduction. 60 Orthogonal series expansions. 49 Orthonormal wavelets. 13 transmission through syslinear tems. 81 Partition of unity. 58. 15 for discrete-time random processes. 49 Orthogonality principle. 63 Nullspace. 224 Orthonormality. 147 type-2. 69 Nyquist condition. 79 Parameterestimation 315 Polyphase matrix M-channel filter bank. 241 pdf (see Probability density function) Perfect reconstruction. 148ff. 183 Oversampled DFT filter banks. 2. 59 (see Estimation) Paraunitary filter banks M-channel. 13. 64.

215. 199. 269 Time-frequency autocorrelation function. 226 Thresholding hard. 113 Signal spaces. 265. 48 Spectral estimation. 206 Spectrogram. 196ff. 244. 86. 194 Shape adaptive wavelet transform. 213 Time-scale analysis. 201. 237 Scalogram. 253 Synthesis window. 97 Stability condition. 95 Twiddle factors. 190 Symmlets. 23 Region of convergence (ROC). 280 Rihaczek. 211. 213 Time-frequency window. 198. 280 separable. 281ff. 289 Schur algorithm. 214 Toeplitz. 284 Soft thresholding. 56 orthogonal. 48 Symmetric bandpass. 151 Spectral subtraction. 198 Time-frequency resolution. 213. 214. 240 Shape adaptive image decomposition. 228. 206 reconstruction. 24 Series expansions. 204 spectral summation. 205 perfect reconstruction. 40 Symmetric reflection. 134-142 Spectral factorization. 188 Subsampling (see Downsampling) Subspace. 288 Time-frequency localization. 131 Schwarz inequality. 247. 91. 216 discrete-time signals. 207 Singular value decomposition. 263 Span. 6 Self-reciprocal kernels. 196. 289 Choi-Williams. 286 Time-frequency analysis. 269271 Terz analysis. 281ff. 204 realizations using filter banks. lff. general. 216 Time-frequency plane. 95 Translation invariance. 196. 47 general. 249 Split-radix FFT. 17 Transform coding. 288-291 Cohen’s class. affine-invariant. 92 Split-radix FHT. 225 Time resolution. 211. 202 reconstruction via series expansion. 294 Scaling function. 230. 67 Smoothing kernel.Index Reciprocal kernel. 202 Temporal autocorrelation function. 207 Spectral summation. Signal-to-noise ratio. 267 Time-frequency distributions. 288 Roll-off factor. 215. 265ff. 22. 48 Sum of subspaces. 193. 252 Representation. 245. 13 STFT (see Short-time Fourier transform) Stochastic process (see Random process) Subband coding. 263 Tight frame. 275. 194 Shift-invariance. 221. 283 Speech analysis. 49 Shannon wavelet. 224 Stationary process. 206 Signal estimation. 64. 233. 281 Short-time Fourier transform. 263 soft. 265. 48 Rihaczek distribution. 193. 237 . 256 Transmultiplexers. 76 Regularity. 92 Two-scale relation. Zhao-Atlas-Marks.. 198. 288 shift-invariant. 195 Tridiagonal matrix. 211. 202 Spline wavelets.

243. 279 Wigner-Ville spectrum. 272 pseudo-. 245 semi-discrete reconstruction. 15 Wigner distribution. 141 Hanning window. 224 construction formulae. 238 continuous-time signals. Wavelet series. 223ff. 241 regularity. analysis by multirate filtering. 256 Welch method. 19 continuous-time. Shannon wavelet. 227ff. 140 White noise process bandlimited. 140. 224. 130 Wide-sense stationary processes. 113 for random parameters. finite support. 260 orthonormal. 225 time-scale analysis. 230 317 stability condition. 141 Hamming window. 137 Zero-delay lifting. 279 Moyal’s formula. 65. 244. 105. 100 Wavelet families. 228. 141 Blackman window. 136. 252 denoising. 290 linear operations. 223 dyadic. 247 Mallat algorithm. 70. series expansion. 265 radar. 75. 269 Unitary matrix. 140. 263 discrete (DWT). linear phase. 77-80 . 244 Variance. 124 Wiener-Hopf equation. 224 synthesis by multirate filtering. 275 discrete-time. 269ff. 18 discrete-time. 223 Wavelet transform. 229 integral reconstruction. 128. 197 Windows Bartlett window. 219ff. dyadic sampling. 143. 288 %Transform. 245 frame bounds. Windowed signal. 56 Walsh-Hadamard transform. 273. 140 Uncertainty principle. 197. 258 partition of unity. 133.Index Unbiasedness for deterministic parameters. 233 tight frame. biorthogonal. 140. 236 oversampling. 232 B trous algorithm. 200. 111 Upsampling. 124 Wiener-Khintchine theorem. discrete-time signals. 138. 224. 224 Haar wavelet. 144 Vanishing moments. 268. 247ff. 210 critical sampling. 292ff. 67. 210ff. 19 Whitening transforms. cross. 259 Morlet wavelet. 139. 133-135. 276 properties. 112. 138. 256ff. 136 of spectral estimates. 73. 214 translation invariance. 177 Zhao-Atlas-Marks distribution. 13 Walsh functions. 111. 141 Yule-Walker equations. 223ff. 14 Wiener filters. 141 Zero padding. 224 Daubechies wavelets. 255 dual wavelet. 122 of autocorrelation estimates. 217ff. 217ff.

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.