# Presents

Robi Polikar Dept. of Electrical & Computer Engineering Rowan University

The Story of Wavelets
Technical Overview
But«We cannot do that with Fourier Transform«. Time - frequency representation and the STFT Continuous wavelet transform Multiresolution analysis and discrete wavelet transform (DWT)

Application Overview
Conventional Applications: Data compression, denoising, solution of PDEs, biomedical signal analysis. Unconventional applications Yes«We can do that with wavelets too«

Historical Overview
1807 ~ 1940s: The reign of the Fourier Transform 1940s ~ 1970s: STFT and Subband Coding 1980s & 1990s: The Wavelet Transform and MRA

What is a Transform and Why Do we Need One ?
Transform: A mathematical operation that takes a function or sequence and maps it into another one Transforms are good things because« The transform of a function may give additional /hidden information about the original function, which may not be available /obvious otherwise The transform of an equation may be easier to solve than the original equation (recall your fond memories of Laplace transforms in DFQs) The transform of a function/sequence may require less storage, hence provide data compression / reduction An operation may be easier to apply on the transformed function, rather than the original function (recall other fond memories on convolution).

December, 21, 1807

³An arbitrary function, continuous or with discontinuities, defined in a finite interval by an arbitrarily capricious graph can always be expressed as a sum of sinusoids´ J.B.J. Fourier Jean B. Joseph Fourier (1768-1830)

Complex Function ! § .

weight i y .

Simple Function i i Complex function representation through simple building blocks Basis functions Using only a few blocks Compressed representation Using sinusoids as building blocks Fourier transform Frequency domain representation of the function ([) ! ´ f (t)e  j[t dt 1 f (t) ! ´ ([)e j[td[ 2T .

How Does FT Work Anyway? Recall that FT uses complex exponentials (sinusoids) as building blocks. j[ t e ! cos .

[ t  j sin .

small / zero FT coefficient. the correlation at that frequency is low / zero. the sinusoid at that frequency is compared to the signal. the correlation is high large FT coefficients. If the signal consists of that frequency. .[ t For each frequency of complex exponential. ([ ) ´ f (t ) e  j[t dt f (t ) 1 2T ´ ([ )e j[t d[ If the signal does not have any spectral component at a frequency.

FT At Work x1 (t ) ! cos(2T  5  t ) x2 (t ) ! cos(2T  25  t ) x3 (t ) ! cos(2T  50  t ) .

FT At Work x1 (t ) F X 1 ([ ) x2 (t ) F X 2 ([ ) x3 (t ) F X 3 ([ ) .

FT At Work x4 (t ) ! cos(2T  5  t )  cos(2T  25  t )  cos(2T  50  t ) x 4 (t ) F X 4 ([ ) .

sampled at 10 MHz   1 f (t ) ! 2T ´ F ([ )  e g   ´   j[t dt j[t dt .5 MHz transducer.FT At Work Complex exponentials (sinusoids) as basis functions: g F ([ ) ! f (t )  e g g F An ultrasonic A-scan using 1.

Need some other ways to determine time localization of spectral components . not where in time they are located. Why? Stationary signals consist of spectral components that do not change in time all spectral components exist at all times no need to know any time information FT works well for stationary signals However.Stationary and Non-stationary Signals FT identifies all spectral components present in the signal. however it does not provide any information regarding the temporal (time) localization of these components. non-stationary signals consists of time varying spectral components How do we find out which spectral component appears when? FT only provides what spectral components exist .

Stationary and Non-stationary Signals Stationary signals¶ spectral characteristics do not change with time x4 (t ) ! cos(2T  5  t )  cos(2T  25  t )  cos(2T  50  t ) Non-stationary signals have time varying spectra x5 (t ) ! [ x1  x2  x3 ]  Concatenation .

Stationary vs. Non-Stationary X4( ) Perfect knowledge of what frequencies exist. but no information about where X5( ) these frequencies are located in time .

and then take the Fourier transform of each segment.T. which differed from each other by the selection of the windowing function .Shortcomings of the FT Sinusoids and exponentials Stretch into infinity in time. no time localization perfect spectral localization Global analysis does not allow analysis of non-stationary signals Need a local analysis scheme for a time-frequency representation (TFR) of nonstationary signals Windowed F. Instantaneous in frequency. narrow enough to be considered stationary. (STFT) : Segmenting the signal into narrow time intervals. Followed by other TFRs. Gabor 1946. or Short Time F.T.

6. save. each FT provides the spectral information of a separate time-slice of the signal. providing simultaneous time and frequency information . 5. Incrementally slide the window to the right Go to step 3. 3. 4. 2. we obtain a different FT Hence.Short Time Fourier Transform (STFT) 1. Choose a window function of finite length Place the window on top of the signal at t=0 Truncate the signal using this window Compute the FT of the truncated signal. until window reaches the end of the signal For each time location where the window is centered.

[ ) ! )A ´ ?x(t ) W (t  t d e t j [t dt STFT of signal x(t): Computed for each window centered at t=t¶ Windowing function Windowing function centered at t=t¶ .STFT Time parameter Frequency parameter Signal to be analyzed FT Kernel (basis function) [ d STFTx (t .

STFT t¶=t¶=-8 t¶=-2 t¶=- t¶=4 t¶=8 .

STFT at Work .

STFT At Work .

STFT At Work .

and then putting them together Time-Frequency Representation (TFR) Maps 1-D time domain signals to 2-D time-frequency signals Consecutive time intervals of the signal are obtained by truncating the signal using a sliding windowing function How to choose the windowing function? What shape? Rectangular.STFT STFT provides the time information by computing a different FTs for consecutive time intervals. Elliptic«? How wide?  Wider window require less time steps low time resolution  Also. Gaussian. window should be narrow enough to make sure that the portion of the signal falling within the window is stationary  Can we choose an arbitrarily narrow window«? .

but no frequency information Wide analysis window poor time resolution.[ )A ) t STFT then gives the time signal back.[ )A t Two extreme cases: W(t) infinitely long: W (t ) ! 1 STFT turns into FT. good frequency resolution Narrow analysis window good time resolution. . poor frequency resolution Once the window is chosen. the resolution is set for both time and frequency. but no time information W(t) infinitely short: W (t ) ! H (t ) [ STFTx (t d ) ! ´ ?x(t )  H (t  t d e  j[t dt ! x(t d e  j[t d .Selection of STFT Window [ STFTx (t d ) ! ´ ?x(t ) W (t  t d e  j[t dt . providing excellent frequency information (good frequency resolution). with a phase factor. Excellent time information (good time resolution).

Heisenberg Principle 1 (t  (f u 4T Time resolution: How well two spikes in time can be separated from each other in the transform domain Frequency resolution: How well two spectral components can be separated from each other in the transform domain Both time and frequency resolutions cannot be arbitrarily high!!! We cannot precisely know at what time instance a frequency component is located. We can only know what interval of frequencies are present in which time intervals .

The Wavelet Transform Overcomes the preset resolution problem of the STFT by using a variable length window Analysis windows of different lengths are used for different frequencies: Analysis of high frequencies Use narrower windows for better time resolution Analysis of low frequencies Use wider windows for better frequency resolution This works well. Heisenberg principle still holds!!! The function used to window the signal is called the wavelet . if the signal to be analyzed mainly consists of slowly varying characteristics with occasional short high frequency bursts.

Signal to be measure of time measure of frequency constant analyzed 1 ] ] CW x (X . Scale parameter. s ) ! =x (X . s) ! s t ] ´ x.The Wavelet Transform A normalization Translation parameter.

All kernels are obtained by translating (shifting) and/or scaling the mother wavelet Scale = 1/frequency .) The mother wavelet.t ¨ t X ¸ © ¹dt ª s º Continuous wavelet transform of the signal x(t) using the analysis wavelet ](.

s ) ! =x (X .WTx (X . s ) ! 1 ¨ t X ¸ x .

t  © ¹dt ´ ª s º s t ¡ ¡ WT at Work Low frequency (large High frequency (small scale) scale) .

WT at Work .

WT at Work .

WT at Work .

Matlab Demos on CWT .

Often this results in a very redundant representation How to discretize the continuous time-frequency plane. can be controlled by time and scale step sizes. The resolution of the time-frequency grid can be controlled (within Heisenberg¶s inequality). s) ! =] (X . s ) ! x.Discrete Wavelet Transform CWT computed by computers is really not CWT. so that the representation is non-redundant? Sample the time-frequency plane on a dyadic (octave) grid 1 ¨ t X ] ] CW x (X . it is a discretized version of the CWT.

t  © x ´ ª s s t ¸ ¹ dt º ] kn (t ) ! 2  k ] 2  k  n .

n  Z . k.

Discrete Wavelet Transform Dyadic sampling of the time ±frequency plane results in a very efficient algorithm for computing DWT: Subband coding using multiresolution analysis Dyadic sampling and multiresolution is achieved through a series of filtering and up/down sampling operations x[n] H N y[n] y[n ] ! x[n ] * h[n ] ! h[n ] * x[n ] ! § x[k ]  h[n  k ] k !1 N ! § h[k ]  x[n  k ] k !1 .

down- .Discrete Wavelet Transform Implementation x[n] ~ G ~ H ~ yhigh [k ] ! § x[n] g[ n  2 k ] n ~ § yhigh [k ]  g[n  2k ] k x[n] 2 2 2 G H + ~ G ~ H 2 2 2 2 G H + 2 ylo [k ] ! § x[n] h[ n  2 k ] ¢ § yhigh [k ]  g[n  2k ] k n Decomposition G H Reconstruction 2 Down-sampling 2 Up-sampling Half band high pass filter Half band low pass filter 2-level DWT decomposition. The decomposition can be continues as long as there are enough samples for down-sampling.

-T -T/2 T/2 T w g[n] 2 d3: Level 3 DWT Coeff. h[n] 2 Length: 64 B: 0 ~ T/8 Hz Length: 64 B: T/8 ~ T/4 Hz «a3«. a1 Length: 256 B: 0 ~ T/2 Hz -T/2 T/2 w |G(jw)| g[n] 2 h[n] 2 Length: 128 B: 0 ~ T /4 Hz Length: 128 B: T/4 ~ T/2 Hz a2 d2: Level 2 DWT Coeff.Demystified Length: 512 B: 0 ~ T |H(jw)| g[n] Length: 256 B: T/2 ~ T Hz h[n] 2 2 d1: Level 1 DWT Coeff.Level 3 approximation Coefficients .DWT .

coef. Highest freq.Implementation of DWT on MATLAB Choose wavelet and number of levels Hit Analyze button Load signal s=a5+d5+«+d1 Approx. at level 5 Level 1 coeff. (Wavedemo_signal1) .

.

.

Applications of Wavelets Compression De-noising Feature Extraction Discontinuity Detection Distribution Estimation Data analysis Biological data NDE data Financial data .

can be zeroed-out! .Compression DWT is commonly used for compression. since most DWT are very small.

Compression .

Compression .

Compression .ECG.

analyze the signal with appropriate wavelets Hit Denoise (Noisy Doppler) .Denoising Implementation in Matlab First.

Denoising Using Matlab Choose thresholding method Choose noise type Choose thrsholds Hit Denoise .

Denosing Using Matlab .

Discontinuity Detection (microdisc.mat) .

mat) .Discontinuity Detection with CWT (microdisc.

ECG. EMG. etc analysis MRI Nondestructive Evaluation Ultrasonic data analysis for nuclear power plant pipe inspections Eddy current analysis for gas pipeline inspections Numerical Solution of PDEs Study of Distant Universes Galaxies form hierarchical structures at different scales .Application Overview Data Compression Wavelet Shrinkage Denoising Source and Channel Coding Biomedical Engineering EEG.

Application Overview Wavelet Networks Real time learning of unknown functions Learning from sparse data Turbulence Analysis Analysis of turbulent flow of low viscosity fluids flowing at high speeds Topographic Data Analysis Analysis of geo-topographic data for reconnaissance / object identification Fractals Daubechies wavelets: Perfect fit for analyzing fractals Financial Analysis Time series analysis for stock market predictions .

Fourier: All periodic functions can be expressed as a weighted sum of trigonometric function Denied publication by Lagrange.History Repeats Itself« 1807. Cooley & Tukey: Fast Fourier Transform . J. Legendre and Laplace 1822: Fourier·s work is finally published « « 143 years « « 1965.B.

Morlet¶s (geophysical engineer) problem: Time .History Repeats Itself: Morlet¶s Story 1946.frequency analysis of signals with high frequency components for short time spans and low frequency components with long time spans STFT can do one or the other. or low frequency components using a wide window. oscillatory signals wavelets Criticism for lack of mathematical rigor !!! Early 1980s. but not both Solution: Use different windowing functions for sections of the signal with different frequency content Windows to be generated from dilation / compression of prototype small. but not both Late 1970s. Gabor: STFT analysis: high frequency components using a narrow window. Grossman (theoretical physicist): Formalize the transform and devise the inverse transformation First wavelet transform ! Rediscovery of Alberto Calderon¶s 1964 work on harmonic analysis .

Orthogonal wavelet basis functions with better time and frequency localization Rediscovery of J.O.1980s 1984. Yeves Meyer : Similarity between Morlet·s and Colderon·s work. Simplest known orthonormal wavelets . 1909 (!). Stromberg¶s 1980 work the same basis functions (also a harmonic analyst) Yet re-rediscovery of Alfred Haar¶s work on orthogonal basis functions. 1984 Redundancy in Morlet·s choice of basis functions 1985.

1988 Discrete wavelet transform Cascade algorithm for computing DWT .D. 1986 Orthonormal bases of compactly supported wavelets (Daubechies wavelets). dissertation. 1988 Liberty in the choice of basis functions at the expense of redundancy Stephane Mallat: Multiresolution analysis w/ Meyer.Transition to the Discrete Signal Analysis Ingrid Daubechies: Discretization of time and scale parameters of the wavelet transform Wavelet frames. 1986 Ph.

«However« Decomposition of a discrete into dyadic frequencies (MRA) . 1976 (!) . Esteban and Galand. Croisier. known to EEs under the name of ³Quadrature Mirror Filters´.

1988 Subband coding Multidimensional filter banks. 1992 . 1986 Perfect reconstruction of signals using FIR filter banks.Transition to the Discrete Signal Analysis Martin Vetterli & Jelena Kovacevic Wavelets and filter banks.

Super-wavelets Matching Pursuit. 1993 ~ 1999 .1990s Equivalence of QMF and MRA. Daubechies. Mallat. and Wickerhauser. J. Cohen. Albert Cohen. Coifman. 1993 Wavelet packets. Meyer. 1990 Compactly supported biorthogonal wavelets. 1996 Zero Tree Coding. Schapiro 1993 ~ 1999 Search for new wavelets with better time and frequency localization properties. Feauveau.

Mallat Using a library of basis functions for decomposition New MPEG standard .New & Noteworthy Zero crossing representation signal classification computer vision data compression denoising Super wavelet Linear combination of known basic wavelets Zero Tree Coding. Schapiro Matching Pursuit .