You are on page 1of 21

Jos e C. M. Bermudez
Department of Electrical Engineering Federal University of Santa Catarina Florian opolis SC Brazil

## ENSEEIHT - Toulouse May 2009

Linear Digital Filters Statistical Linear Filtering Optimal Linear Filtering Adaptive Filtering Adaptive Filter Applications The LMS Adaptive Algorithm

## Regular Linear Digital Filters

FIR or IIR. Most useful for frequency selective applications. Desired signal and interference are in dierent frequency bands. Example: To extract high frequency noise (> 20 kHz) from speech.

FIR IIR

FIR Filters

Simple to design Can implement linear phase Are always stable Require coecients than IIR lters

IIR Filters

Harder to design than FIR lters Introduce phase distortion May become unstable Require coecients than FIR lters

## Statistical Linear Filtering

Many applications more than just frequency band selection Signal and interference are frequently within the same freq. band

## Basic Operations in Linear Filtering

1. Filtering Extraction of information at a given time n by using data measured before and at time n. 2. Smoothing Diers from ltering in that data measured both before and after n can be used to extract information at time n. 3. Prediction Predicts how the quantity of interest will be at time n + N for some N > 0 by using data measured up to and including n. (The lter is linear if its output is a linear function of the observations applied to the input.)

## How Do We Do Linear Statistical Filtering?

We assume some statistical properties of the useful and unwanted signals are available Mean Autocorrelation Cross-correlations etc.

We dene some statistical criterion for the optimization of the lter performance

## Optimal Linear Filtering The Wiener Filter - The FIR Case

d(n) u(n) FIR y (n)+ + Linear Filter e(n)

u(n) and d(n) stationary and zero-mean d(n): information plus some interference y (n): estimate of d(n) given some observations of u(n) (n|Un ) wk u(n k) = d

y (n) =

## d(n) u(n) FIR y (n)+ + Linear Filter e(n)

Performance index: Mean-Square Error w = [w1 , . . . , wN ]T y (n) = uT (n)w J (w) = E {e2 (n)} Mean-Square Error u(n) = [u(n), . . . , u(n N + 1)]T

## The Wiener Weights

e(n) = d(n) uT (n)w e2 (n) = d2 (n) 2d(n)uT (n)w + wT u(n)uT (n)w
2 (n) 2pT (n)w + w T R(n)w J (w ) = d

## where: p(n) = E [d(n)u(n)] R(n) = E [u(n)uT (n)]

2 = E [d2 (n)] d Optimum Weight Vector J (w ) =0 w = wo such that w 2p + 2Rwo = 0

## : cross-corr. between d(n) and u(n) : autocorrelation matrix of u(n)

Rwo = p

(Normal Equations)

## 3500 3000 2500 2000 1500 1000 500 0 20

10

0 w2

10

20

20

10 w1

10

20

wo : Minimum of this surface Solution of the Normal Equations Computationally intensive even with ecient algorithms

## What is an Adaptive Filter Anyway?

In several practical real-time applications:

## The signal involved are nonstationary

Normal equations must be solved for each n!?! The algorithms used for the stationary case become inecient What about Kalman Filters?

Requires a dynamics model (state-space) for d(n) Computationally heavy for real-time

Signal statistics may be unknown, and there may be no time to estimate them

Computational complexity between 2 input samples limited by processor speed and by cost

Change their weights as new input samples arrive Weight updating is controlled by an adaptive algorithm Optimal solution is approached by improving performance a little bit at each iteration

Optimal solution is approximated after several iterations (iteration complexity convergence time)

Filter weights become random variables that converge to a region about the optimum weights

## Application Network Echo Cancellation

2wire 4wire 2wire

speaker A H

speech from A

speaker B

echo from A

H speech from B

speech from A

x(n)

Alg.

w(n)
_

h echo y (n)
+ +

y (n)
+

r (n)

e(n)

d(n)

speech from B

## Application Channel Equalization

Transmitted symbols
Channel + + _ + Adaptive Equalizer

noise

decision error

wo y (n) + x(n)
input signal

+ +

w(n)

## Adaptive Algorithm - Least Mean Squares (LMS)

Optimization Problem Determine w that minimizes
2 J (w )|w=w(n) = d (n) 2pT (n)w(n) + w(n)T R(n)w(n)

## Adaptive Algorithm - Least Mean Squares (LMS)

LMS Algorithm

Steepest descent needs p and R !? Instantaneous approximation of the gradient E [e2 (n)] e2 (n) = 2e(n)u(n) w(n) w(n)

## LMS Typical Convergence Behavior

Mean Weight Error Vector

E [v (n)] = E [w (n)] w o

0.15 0.13 0.11 0.09 0.07 0.05 0.03 0.01 0.01 0.03

E[V(n)]

## = 0, 001 = 0, 002 = 0, 004

2e3 4e3 6e3 8e3 10e3 12e3 14e3 16e3 18e3 20e3
iterations

## LMS Typical Convergence Behavior

MSE (dB) MeanSquare Error

30 10 10

= 0, 001 = 0, 002

30

= 0, 004
50 70 90 110 0 2e3 4e3 6e3 8e3 10e3 12e3 14e3 16e3 18e3 20e3 iterations