Brief Introduction of Advanced

Signal Processing

1. Short Time Fourier Transform and Wavelets

2. Wold Representation and Linear Predictive Coding
3. Weiner Filters
4. Adaptive Filters
5. Power Spectrum Estimation
6. Direction Estimation
Short Time Fourier Transform (STFT)
 Fourier Transform

Fourier Transform reveals which frequency components

are present in a function:

(inverse DFT)

where: (forward DFT)

 Examples

f1 (t ) cos(2 5 t )

f 2 (t ) cos(2 25 t )

f 3 (t ) cos(2 50 t )
 Examples (contd)

F1(u)

F2(u)

F3(u)
 Fourier Analysis Examples (contd)

f 4 (t ) cos(2 5 t )
cos(2 25 t )
cos(2 50 t )

F4(u)
 Limitations of Fourier Transform

1. Cannot not provide simultaneous time and frequency

localization.
 Fourier Analysis Examples (contd)

f 4 (t ) cos(2 5 t )
cos(2 25 t )
cos(2 50 t )

Provides excellent
localization in the F4(u)
frequency domain but
poor localization in the
time domain.
 Limitations of Fourier Transform (contd)

1. Cannot not provide simultaneous time and frequency

localization.

2. Not very useful for analyzing time-variant, non-

stationary signals.
 Stationary vs non-stationary signals

Stationary signals:
time-invariant spectra
f 4 (t )

Non-stationary
signals: time-varying
spectra
f 5 (t )
 Stationary vs non-stationary signals (contd)
Stationary signal:

f 4 (t )

Three
frequency
components,
present at all
times! F4(u)
 Stationary vs non-stationary signals (contd)

Non-stationary signal:

f 5 (t )

Three
frequency
components,
NOT present at
all times! F (u)5
 Stationary vs non-stationary signals (contd)

Non-stationary signal:

f 5 (t )

Perfect knowledge of what

frequencies exist, but no
information about where
these frequencies are
located in time!

F5(u)
 Short Time Fourier Transform (STFT)
Segment the signal into narrow time intervals (i.e., narrow
enough to be considered stationary) and take the FT of each
segment.
Each FT provides the spectral information of a separate
time-slice of the signal, providing simultaneous time and
frequency information.
 STFT - Steps
(1) Choose a window function of finite length
(2) Place the window on top of the signal at t=0
(3) Truncate the signal using this window
(4) Compute the FT of the truncated signal, save results.
(5) Incrementally slide the window to the right
(6) Go to step 3, until window reaches the end of the signal
 STFT - Definition

Time Frequency Signal to

parameter parameter be analyzed

STFT fu (t , u ) f (t ) W (t t ) e j 2 ut dt
2D function t

STFT of f(t): Windowing Centered at t=t

computed for each function
window centered at
t=t
 Example

f(t)

[0 300] ms 75 Hz sinusoid
[300 600] ms 50 Hz sinusoid
[600 800] ms 25 Hz sinusoid
[800 1000] ms 10 Hz sinusoid
 Example

STFTfu (t , u)

f(t)

W(t)
scaled: t/20
 Choosing Window W(t)

What shape should it have?

Rectangular, Gaussian, Elliptic

How wide should it be?

Window should be narrow enough to ensure that the portion
of the signal falling within the window is stationary.
But very narrow windows do not offer good localization
in the frequency domain.
 STFT Window Size
STFT fu (t , u ) f (t ) W (t t ) e j 2 ut dt
t

W(t) infinitely long: W (t ) 1

STFT turns into FT, providing
excellent frequency localization, but no time localization.

W(t) infinitely short: W (t ) (t )

results in the time signal
(with a phase factor), providing excellent time localization but
no frequency localization.

STFT fu (t , u ) f (t ) (t t ) e j 2 ut dt f (t ) e jut
t
 STFT Window Size (contd)

Wide window good frequency resolution, poor

time resolution.

Narrow window good time resolution, poor

frequency resolution.

Wavelets: use multiple window sizes.

 Example
different size windows

(four frequencies, non-stationary)

 Example (contd)

STFTfu (t , u)

STFTfu (t , u)

scaled: t/20
 Example (contd)

STFTfu (t , u)

STFTfu (t , u)

scaled: t/20
 Heisenberg (or Uncertainty) Principle

1
t f
4

Time resolution: How well two spikes Frequency resolution: How well
in time can be separated from each two spectral components can be
other in the frequency domain. separated from each other in the
time domain

t and f cannot be made arbitrarily small !

 Heisenberg (or Uncertainty) Principle

We cannot know the exact time-frequency

representation of a signal.
We can only know what interval of frequencies are
present in which time intervals.
 Wold Representation and its
applications

The speech samples constitute a random process

 Random Processes
Strictly Stationary Process

Weiner-
Kintchie
Widesense stationary process theorem
Wold representation

the sequence v(m) is called the

cepstrum of x(n)
 This Laurent series expansion is possible only for
those functions which includes unit circle of z-
plane in its analytic region (possess derivatives of
all orders)

Causal part of the sequence v(m)

defines H(Z) and non-causal part
defines H(Z-1)

Taylor series
expansion is
applicable in this
case

This Fourier series coefficients v(m)

are known as cepstral coefficients and
the sequence v(m) is called the
cepstrum of x(n)
Autoregressive [AR(p)]; all pole process

Moving Average [MA(p)]; all zero process

AR-MA ; Pole-zero process (system)

 For Autoregressive-
Moving Average (AR-
MA) process (system).
Non-Linearity is
involved due to bkbk+m
term in ARMA as well
as MA systems.

For Autoregressive (AR) system the

equation is linear, called Yule-Walker
equation which can be represented
in matrix form.

Optimum value of p is desired for satisfying contradicting requirements of estimation

error and computational complexity.
For finding the coefficients of pth order system involves autocorrelation of x(n) over p
to p and the variance of innovations process w(n) produced by 1/H(z).
 Linear Predictive Coders (LPC)
Noise source
(for unvoiced)

Vocal tract filter

Periodic Pulse Source
(for voiced)

Rx Parameters

Voiced sounds: Periodic air pulses pass through vibrating vocal

chords. (freq of periodic pulse train should be equal to pitch)
Unvoiced sounds: Force air through a constriction in vocal tract,
producing turbulence.

Two methods for voiced/ unvoiced decision

(i) Zero Crossing unvoiced signals have much faster zero crossing as
compared to voiced.
(ii) Power unvoiced signal has very low power as compared to voiced
Variants of LPC:
Single Pulse Excited (single pulse excitation is used at
Rx, produces audible distortion)

Multipulse excited (8 pulses per period are used, position

and amplitude of an pulse is adjusted to reduce the
distortion )

Code excited (a codebook of Guassian excitation signals

is maintained by both Tx and Rx. Besides other
parameters, The transmitter transmits the index of code
where it finds the best match)

Residual excited (single pulse LPC is used for decoding

at Tx itself and synthesized voice is generated. Now the
difference (residue) of above two signals is coded and
send with the original LPC codes)
 Weiner Filter

Depending upon the

reference signal s(n)
Weiner filter can
perform 3 operations,
viz.
s(n)=d(n) it is
filtering,
s(n) = d(n+D) it is
Prediction,
s(n)=d(n-D) it is
smoothing This set of M linear equations are called Weiner-Hopf equation
or the normal equations of the optimum filter i.e. Weiner Filter.
 Adaptive Filter
Forward Filter
Three aspects -
r(t) r(t-T) r(t-2T) r(t-nT)

Type
INPUT
r(t) T T T

(linear/nonlinear)
C0 C1 C2 Cn

+ +
Algorithm
ERROR

+ +
ek
TRAINING
(zero forcing/ LMS/ RLS)
+ -
SEQUENCE

DECISION OUTPUT
Type
DEVICE Zk
(transversal/ lattice)

The linear transversal equalisation (LTE) is one of the simplest forms of

equaliser.
The tap coefficients (C1 to Cn) are adapt to suit the current channel
conditions. Normally this adaptation is done on a training sequence.
In the presence of severe amplitude and phase distortion, the required
inverse filter tends to result in an unacceptable degree of noise amplification.
l Training algorithm selection:
Convergence speed
(no of iterations, size of training seq, required to converge close enough
to its optimum value)

Complexity
(number of operations required to make one iteration)

Misadjustment
(Difference between final values of mean square error averaged over
an ensemble to optimal mean square error)

Numerical Stability
(Effect on stability due to numerical inaccuracies caused by rounding
off noise and presentation of error in computer.)
The Algorithms:
Zero forcing
[It amplifies the noise which was attenuated by channel (as
zeros becomes poles in inverse system) thus not suitable for
wireless channels which have low S/N]

LMS (Least Mean Squares)

[mean square of error is minimized which maximizes the
signal to distortion ratio for a given equalizer length but the
convergence rate is slow. It wont work if time dispersion in
CH>delay through EQ]

RLS (Recursive Least Squares) algorithm

[fast but computationally expensive, as time averages with
some recursive principle is used instead of ensamble
averages. It becomes unstable if an echo is more powerful
than the present symbol.]
Decision Feedback Equalizer (nonlinear)
The equaliser output signal is the
Forward Filter sum of the outputs of the
INPUT
r(t+nT) r(t+[n-1]T) r(t+[n-2]T) r(t)
feedforward and feedback
T T T sections of the equaliser.
The forward section similar to the
Cn C C C
n-1 n-2 0 LTE
+ +
ERROR
Decisions made from the output
+ +
+ -
ek
TRAINING of the equaliser are now feed
+
SEQUENCE
back through a second filter.
-
DECISION
DEVICE
OUTPUT
Zk
If these decisions are correct, the
- - ISI caused by these symbols can
bm b2 b1
be cancelled without noise
enhancement
T T T
^
X k-m
^
X k-2 ^
X k-1 ^
Xk However, errors made in hard
Feedback Filter decisions are feedback through
the equaliser and can cause
error propagation
 Power spectrum estimation
Non-parametric methods
It involves the FFT evaluation of truncated data or its autocorrelation
function
Need long data for better frequency resolution
Spectral leakage
Unrealistic assumptions like zero auto correlation outside window and
periodicity of I/P data

Parametric methods
A few parameters are estimated from the observed data. A model for
the signal generation is formed. From the model and the estimated
parameters the power spectra is estimated
AR, MA or ARMA processes are built from x(n) and |H(f)|2 is evaluated
using the variance of innovation process involved.
Many other methods exists.
 Selection of AR model filter
AR model is preferred over MA and ARMA since it represents the
spectra with narrow peaks and it results in simple linear equations for
AR parameter evaluation

If the order of AR model is too low then highly smoothed spectrum and
if its too high then spurious low level peaks are introduced in the
estimated spectra.

Final prediction error (FEP) criterion, Alkaike information criterion

(AIC), criterion autoregressive transfer (CAT) etc techniques can be
used for finding the optimum value of the p.

An AR(p) process corrupted by Additive white noise

is equivalent to an ARMA(p,p) process for which the
Eigenanalysis approach is commonly used.