Dsplec1 Handout

Big Picture of Signal Processing
From Time-Frequency Distributions to Continuous

Wavelet Transforms
What is a signal?
Dr. Xiao-Ping Zhang Data numerical data
Time series, x(t), x[n]
CASPAL
Department of Electrical and Computer Engineering Location series? image I(m, n)
Ryerson University
xzhang@ee.ryerson.ca
September 14, 2017
Dr. X.-P. Zhang (CASPAL, Ryerson) DSP II September 14, 2017 1 / 52 Dr. X.-P. Zhang (CASPAL, Ryerson) DSP II September 14, 2017 2 / 52
How to process a signal Structure of a signal
Deterministic signal x(t) with uncertainty n(t)
r (t) = x(t) + n(t) Signal dynamics x(t) > y (t)

Structure of data
AWGN why Gaussian/normal? Transforms and filters
Stochastic process Matched filter to match a structure
Stationary Correlation structures
Wide-sense stationary
Secondary statistics
Time-Frequency Time-Frequency
Time s(t),RFrequency s().

s() = 1 s(t)exp(jt)dt.
2
Energy distributions
Why is ejt so special? Time density: |s(t)|2
|s(t)|2 t total energy at t neighbourhood
Frequency density: |s()|2
|s()|2 total energy at neighbourhood
Parseval Theorem:
Totel energy E = |s(t)|2 dt = |s()|2 d = 1 (Normalized)
R R
Time-Frequency Density Can We Find an Ideal TFD?

Marginal
Density P(t, ) such

R that:
Z
2
|S()| = P(t, )dt
Totel energy E = P(t, )dtd = 1.
2
Marginal RDensity:
Z
1
|s(t)|2 = R P(t, )d and =
s(t) exp(jt)dt
2
|s()|2 = P(t, )dt 1
Z Z
Instantaneous frequency: = s (t 0 )s(t) exp(j(t t 0 ))dt 0 dt
2
|t = P(t, )d = 0 (t).
R
Group Rdelay: Let = t t 0 , d = dt 0 , we have

td | = tP(t, )dt = 0 (). 1
Z Z
|S()| = 2
s (t )s(t)exp(j )d dt
Positive and real-valued distribution 2
Z
1
= R( )exp(j )d.
2
Can We Find an Ideal TFD? WVD
Wigner-Ville Distribution Marginal
Z Z Z
R( ) = s (t)s(t + )dt = s (t )s(t)dt P(t, )dt = |S()|2
Z
= s t s t+ dt
2 2 Z Z Z
1
P(t, )d = Rt ( ) exp(j )d d
Now define: 2
Z Z
1
1
Z = Rt ( ) exp(j )dd
P(t, ) = Rt ( ) exp(j )d. 2
2 1
Z
= Rt ( ) 2( )d
where 2

Rt ( ) = s t s t+ = Rt (0) = |s(t)|2 .
2 2

Note that Rt ( ) = Rt ( ) in this definition. Why? (answer real) This is Wigner-Ville Distribution (WVD).
WVD Generalization Why TFD? Examples

Cohens class
Ex 1.1: Sinusoid
Generalized time-dependent autocorrelation function:
s(t) = ej0 t , W (t, ) = ( 0 )
Z Z
1
Rt ( ) = exp(j(u t))(, )s u s u+ dud Ex 1.2: Impulse:
2 2 2

i.e. s(t) = 2(t t0 ), W (t, ) = (t t0 )
Z
1
Rt ( ) = r (u t, )s u s u+ du Ex 1.3: Pure chirp (linear FM):
2 2 2
where 2 /2+j
Z s(t) = ejt 0t
.
r (u, ) = exp(ju)(, )d.
what is the instantaneous frequency?
For symmetry, the kernel function must take:
W (t, ) = ( t 0 ).
(, ) = (, )
Great!
Why TFD? Examples WVD the tale/sum of two signals
Ex 1.4: Sinusoid with Gaussian amplitude (AM):
s(t) = s1 (t) + s2 (t)
1/4 t 2 /2+j0 t
s(t) = (/) e Rt ( ) =?
what it should be? envelope?

W (t, ) = W11 (t, ) + W22 (t, ) + W12 (t, ) + W21 (t, ),
1 2 2
W (t, ) = et (0 ) / where cross Wigner distribution:
Z
1 j
Ex 1.5: Chirp with Gaussian amplitude: W12 (t, ) = s1 t s2 t + e d.
2 2 2
2 /2+jt 2 /2+j
s(t) = (/)1/4 et 0t Or in terms of spectrum
Z
1
what is the instantaneous frequency? W12 (t, ) = S1 S2 + ejt d.
2 2 2
1 t 2 (t0 )2 / = W , therefore
W (t, ) = e . Note W12 21

Even better!! W (t, ) = W11 (t, ) + W22 (t, ) + 2<{W12 (t, )}.
WVD the tale/sum of two sinusoids WVD - Cosine

Ex 1.7
s(t) = 2 cos 0 t.
Ex 1.6
s(t) = A1 ej1 t + A2 ej2 t . W (t, ) = ( 0 ) + ( + 0 ) + 2() cos 20 t.
What should be its TFD? Were not happy!
The WVD is: Ex. 1.8: Sinusoid plus impulse

W (t, ) = |A1 |2 ( 1 ) + |A2 |2 ( 2 ) s(t) = ej0 t + 2(t t0 ).
1
+ 2A1 A2 ( (1 + 2 )) cos(2 1 )t.
2
There is a cross-term!! W (t, ) = ( 0 ) + (t t0 )
2
+ cos[2( 0 )(t t0 ) 0 t0 ].
2
This is messy!
WVD - More WVD Help!
Ex 1.9: Sum of two tones with amplitude modulation:

2 /2+j t
s(t) = A1 (1 /)1/4 e1 t 1
2 /2+j t
+ A2 (2 /)1/4 e2 t 2
.
Kernel: smoothing function.

A21 1 t 2 (1 )2 /1 A22 2 t 2 (2 )2 /2
W (t, ) = e + e No free lunch: lose sharpness!
s
A1 A2 2(1 2 )1/2

2t
+ 2 cos ( + 2 1 1 2 )
1 + 2 1 + 2

2 2 1 2
exp {1 2 t + [ (1 + 2 )] } .
1 + 2 2
More messy!!!
Also not positive.
WVD - Inverse WVD More properties

Z
1
W (t, ) = Rt ( ) exp(j )d.
2
where

Rt ( ) = s t s t+
2 2 Note not all 2-D function is WVD realizable.
Note that Rt ( ) = Rt ( ) in this definition. Why? (answer real) Preserve inner product?
Inverting: Z 2 Z Z
s1 (t)s (t)dt = 2

W1 (t, )W2 (t, )dtd.
Z

2
s t s t+ = W (t, ) exp(j )d.
2 2
Take t = /2, and = t, we get
Z
s(t) = k W (t/2, ) exp(jt)d,
where k = 1/s (0). Recovered up to a constant, phase ambiguity

caused by multiplication.
TFD - Kernels TFD - Kernel properties
Cohens class For marginal properties
Z Z Z
1 Z
1
Z Z Z
P(t, ) = exp(jt j + ju)(, ) P(t, )d = ( ) exp(j(t u))(, )
4 2 2

s u

2
s u+
2
dud d s u s u+ dud d
Z Z 2 2
1
= exp(j(t u))(, 0)|s(u)|2 ddu.
2
To make it equal to |s(t)|2 ,
Z
1
exp(j(t u))(, 0)d = (t u),
2
which means
(, 0) = 1.
P(t, )dt = |S()|2 ,
R
Similarly, if we want
(0, ) = 1.
TFD - Kernel properties (cont) Uncertainty Principle
Definitions: Z
2
If total energy is preserved, i.e., T = t2 = (t < t >)2 |s(t)|2 dt
Z Z
2
P(t, )ddt = 1, B = 2 = ( < >)2 |S()|2 d
we must have The uncertainty principle is

(0, 0) = 1, 1
TB .
which is called normalization condition. Note that it is weaker than the 2
marginal conditions. A more general uncertainty principle is:
For reality
(, ) = (, ). 1
q
TB 2 .
1 + 4Covt
2
Note: Covt =< t0 (t) > < t >< >.
Uncertainty Principle Uncertainty Principle
Proof Proof cont
Without loss of generality, assume the signal is zero mean time and Now use Schwarz inequality
2
zero mean frequency, i.e., we can normalize a signal as following
Z Z Z
2 2

|f (x)| dx |g(x)| dx f (x)g(x)dx ,
Snew (t) = ej<>(t+<t>) sold (t+ < t >).
we have Z 2
The bandwidth is thus t2 2
0

ts (t)s (t)dt .

Z Z
2 = 2
|S()| dt = 2
|s0 (t)|2 dt. Now note s(t) = A(t)ej(t) ,
1d 1
The duration is ts (t)s0 (t) = tA0 A + jt0 A2 = (tA2 ) A2 + jt0 (t)A2 .
2 dt 2
Z
t2 = t 2 |s(t)|2 dt.
Take integration, the first term is zero (why? perfect differential).
Therefore Assume signal is normalized, then
Z Z
2 2
t2 2 = t 2 |s(t)|2 dt |s0 (t)|2 dt.
Z
2 2
0
1 1 2
t ts (t)s (t)dt = + jCovt = + Covt .
2 4

Minimum uncertainty product signals Minimum uncertainty product signals:cont
Necessary condition for the equality to hold:
cts(t) = s0 (t).
We also have to have the covariance part to be zero to reach lower

bound 1/2. Write
In general with mean time and frequency, we have
ct 2 /2 (cr jci )t 2 /2
s(t) e =e . 2 /2+j<>t
s(t) = (/)1/4 e(t<t>)
We have
Z Z
2
Covt = tci t|s(t)|2 dt < >< t >= ci t 2 ecr t dt.
To make it zero, ci has to be zero and c must be a real number. Lets

take c = , we then have
2 /2
s(t) = (/)1/4 et .
Examples: Chirp Examples: Chirp cont - Calculation tricks
Note that in general:

Z Z
1d
Ex: Chirp < >= |S()| d = 2
s (t) s(t)dt
2 2
j dt
s(t) = (/)1/4 et /2+jt /2+j0 t .
r Z

Z
2
< t >= tet dt = 0. < > = 2
2 |S()|2 d

1d 2
r Z Z
2 1
< t 2 >= t 2 et dt = . = s (t) s(t)dt
2 j dt
d2
Z
1
Hence t2 =< t 2 > < t >2 = 2 . = s (t) 2 s(t)dt
dt
Z 2
d
= s(t) dt
dt

Examples: Chirp cont Examples: Chirp cont
For Chirp signal:
1d
Bandwidth in general: s(t) = [jt + t + 0 ]s(t).
j dt
Z
2 = ( < >)2 |S()|2 d Therefore,
Z
2
s (t)(jt + t + 0 )s(t)dt
Z
1d <> =
= s (t) < > s(t)dt
j dt r Z
2
(jt + t + 0 )et dt = 0 .
2
=
Z
1d
= < > s(t) dt
j dt
Also,
2 + 2
r Z
2
< > =2
|jt + t + 0 |2 et dt = + 02 ,
2
Uncertainty Principle The Short-Time Fourier Transform
Examples: Chirp cont Spectrogram
Question: how to observe the spectrum of a signal at a give time?

which gives Natural answer: use an observation time window h(t).
2 + 2
2 = .
2 st ( ) = s( )h( t).
Z Z Now define STFT as:

0 2 2 2
Covt =< t (t) >= t(t + 0 )|s(t)| dt = t |s(t)| dt = Z Z
2 1 j 1
St () = e st ( )d = ej s( )h( t)d.
2 2
Finally,
r r r The energy density spectrum at time t, or spectrogram is then defined
1 2 + 2 1 2 1
q
t = = 1+ 2 = 2 .
1 + 4Covt as: Z 2
2 2 2 2 2
1 j

PSP (t, ) = |St ()| =
e s( )h( t)d .
2
The Short-Time Fourier Transform The Short-Time Fourier Transform

SFTT Relationship to WVD
Alternatively, we may observe the time property of a spectrum at a

given frequency using an observation frequency window H(). Kernel function:
Short-Frequency Time Transform: Z
1 ju 1
1
Z (, ) = h u e h u + du.
s (t) =
0
ej t S( 0 )H( 0 )d 0 . 2 2
2
Relationship to WVD:
If H() = 1 h(t)ejt dt, then
R
2 Z 2
1 j

PSP (t, ) =
e s( )h( t)d
St () = ejt s (t). Z Z2
= Ws (t 0 , 0 )Wh (t 0 t, 0 )dt 0 d 0
Note the spectrogram
PSP (t, ) = |St ()|2 = |s (t)|2 .
The Spectrogram The Spectrogram
Marginals Total Energy
Time:
Z
P(t) = PSP (t, )d
Z
1 0
Z Z Z Z
= s( )h( t)s ( 0 )h ( 0 t)ej( ) d d 0 d ESP = PSP (t, )dtd = 2
|s(t)| dt |h(t)|2 dt
2
Z
= s( )h( t)s ( 0 )h ( 0 t)( 0 )d d 0
i.e., STFT preserves the energy (Parsevals identity).
Not difficult to prove STFT preserves the inner product, i.e.,
Z
= |s( )|2 |h( t)|2 d Z Z Z

s(t)f (t)dt = St ()Ft ()dtd
Frequency:
Z
P() = |S( 0 )|2 |H( 0 )|2 d 0
Not good!

Inverse Gabor Transforms
R
STFT is invertible. Let h(t)dt = 1, then
2
Z
s(t) = s(t)h(t )d Choose Gaussian window: h(t) = (/)1/4 et /2 .
Z Z Recall uncertainty: most compact joint time-frequency.
1 Gabor Transforms
= S ()ej dd
2
Admissible low pass window
Examples The tale/sum of two signals
Sinusoid: s(t) = ej0 t . Using Gabor transform, STFT is s(t) = s1 (t) + s2 (t)
Z
1

( 0 )2
1
St () = e j(0 )t
exp St () = ej st ( )d
()1/4 2 2 Z
1
= ej [s1 ( ) + s2 ( )]h( t)d
PSP (t, ) = |St ()|2 . 2
= S1,t () + S2,t ().
Impulse: s(t) = 2(t t0 ).
2 /2 Note the spectrogram is then:
St () = h(t0 t)ejt0 = (/)1/4 ejt0 e(tt0 )
PSP (t, ) = |St ()|2 = |S1,t () + S2,t ()|2
PSP (t, ) = |St ()|2 .
= |S1,t ()|2 + |S2,t ()|2 + S1,t ()S2,t () + S1,t ()S2,t ()
Recall uncertainty principle.
= |S1,t ()|2 + |S2,t ()|2 + 2<{S1,t ()S2,t ()}
Whats the difference from WVD?

The Short-Time Fourier Transform Continuous Wavelet Transform

Examples
CWT is defined as:
The sum of a sinusoid and impulse
t b
Z
a,b 1/2
Ws (a, b) =< s, >= |a| s(t) dt.
s(t) = ej0 t + 2(t t0 ). a
We have Using the preservation of inner product of Fourier transform, its

( )2
frequency form is
1 j(0 )t 20
St () = e e Z
()1/4 a,b
Ws (a, b) =< s, >= |a| 1/2
S() (a)ejb d.
2 /2
+(/)1/4 ejt0 e(tt0 )
The function (t) is called mother wavelet. Assume is normalized,
and i.e., |||| = 1. The affine transforms of the mother wavelet constitute a
(0 )2 2 set of wavelets:
PSP (t, ) = |St ()|2 = ()1/2 e + (/)1/2 e(tt0 )
2
1/2
a,b t b
2 2
+ e(0 ) /2(tt0 ) /2 cos[(t t0 ) 0 t]. |a| .
a a,bR
See Figs. 7.1 and 8.5 in Cohens book. Note that the scale a is related to frequency. The larger is a, the
How about a different window? smaller is frequency.
Continuous Wavelet Transform Continuous Wavelet Transform
Parsevals Theorem Parsevals Theorem:Proof
The mother wavelet has to satisfy admissible condition:

Z Proof.
d
C = 2 |()|2 < ,
Z Z
dadb
Z Ws (a, b)Wg (a, b) 2
a
(0) = 0 (t)dt = 0. Z Z Z
1/2 jb
= |a| S() (a)e d
The condition enables the preservation of inner product:
Z
0 jb 0 0 dadb
Z Z
dadb |a|1/2 0
G ( )(a )e d
2
s, g L (R), Ws (a, b)Wg (a, b) 2 = C < s, g > . a2
a Z Z
da
This would imply Parsevals theorem, i.e., energy conservation, = 2 S()G ()|(a)|2 d
Z Z Z |a|
2 2 1 dadb = C < s, g > .
s L (R), Etotal = |s(t)| dt = |Ws (a, b|2 2 .
C a
Scalogram:
PSC (a, b) = |Ws (a, b)|2 . Note its clear that C has to be finite.

Inverse Reproducing kernel
Note
Ws (a, b) = < s, a,b >

da0 db0
Z Z
= Ws (a0 , b0 )Wa,b (a0 , b0 ) 02
Take g(t) = (t ), the inverse is straightforward from the above a
da0 db0
Z Z
formula: Z Z = Ws (a0 , b0 )K (a, b; a0 , b0 ) 02 ,
1 dadb a
s(t) = Ws (a, b) a,b 2 .
C a
where reproducing kernel
0 0
K (a, b; a0 , b0 ) = < a,b , a0 ,b0 > =< a ,b , a,b > .
Not all 2D functions are CWT of a signal.

CWT is redundant.
Examples Relation to STFT
Impulse: s(t) = (t t0 ). Let (t) = h(t)ejc t , we have

t b 1/2 t0 b
Z
Ws (a, b) = |a| 1/2
(t t0 ) dt = |a| t b t b c
a

a =h ej a (tb) .
a a
Sinusoid: s(t) = ej0 t . Fourier transform:

c jb
j0 t t b = |a|1/2 2H(a c )ejb
Z
Ws (a, b) = |a|1/2
e dt = 2|a|1/2 (a0 )ejb0 |a|1/2 2H a( ) e
a a
The center scale is a = c / and thus /c = Q constant Q.
2 For sinusoid,
PSC (t, ) = |Ws (a, b)| .
c jb0
Note that ! Ws (a, b) = 2|a|1/2 (a0 )ejb0 = 2|a|1/2 H a(0 ) e
a a
(a0 ) = 1 c
0 The scale center is at: a = 0 and the bandwidth is proportional to 0 .
Continuous Wavelet Transform Choice of Wavelets

Graphs
Gabor wavelets, Morlet wavelets, etc...

Show graphs.

Dsplec1 Handout

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dsplec1 Handout

Uploaded by

Copyright:

Available Formats

Big Picture of Signal Processing

From Time-Frequency Distributions to Continuous

September 14, 2017

How to process a signal Structure of a signal

Deterministic signal x(t) with uncertainty n(t)

r (t) = x(t) + n(t) Signal dynamics x(t) > y (t)

Time s(t),RFrequency s().

Time-Frequency Density Can We Find an Ideal TFD?

Density P(t, ) such

Group Rdelay: Let = t t 0 , d = dt 0 , we have

WVD Generalization Why TFD? Examples

what it should be? envelope?

WVD the tale/sum of two sinusoids WVD - Cosine

Ex 1.9: Sum of two tones with amplitude modulation:

Kernel: smoothing function.

WVD - Inverse WVD More properties

where k = 1/s (0). Recovered up to a constant, phase ambiguity

TFD - Kernel properties (cont) Uncertainty Principle

we must have The uncertainty principle is

Uncertainty Principle Uncertainty Principle

Necessary condition for the equality to hold:

We also have to have the covariance part to be zero to reach lower

To make it zero, ci has to be zero and c must be a real number. Lets

Note that in general:

Uncertainty Principle Uncertainty Principle

For Chirp signal:

Question: how to observe the spectrum of a signal at a give time?

Z Z Now define STFT as:

The Short-Time Fourier Transform The Short-Time Fourier Transform

Alternatively, we may observe the time property of a spectrum at a

PSP (t, ) = |St ()|2 = |s (t)|2 .

The Short-Time Fourier Transform The Short-Time Fourier Transform

Whats the difference from WVD?

The Short-Time Fourier Transform Continuous Wavelet Transform

We have Using the preservation of inner product of Fourier transform, its

The mother wavelet has to satisfy admissible condition:

Continuous Wavelet Transform Continuous Wavelet Transform

Ws (a, b) = < s, a,b >

Not all 2D functions are CWT of a signal.

Impulse: s(t) = (t t0 ). Let (t) = h(t)ejc t , we have

Sinusoid: s(t) = ej0 t . Fourier transform:

Continuous Wavelet Transform Choice of Wavelets

Gabor wavelets, Morlet wavelets, etc...

You might also like