Chapter

9

Adaptive Filters
Introduction

The term adaptive filter implies changing the characteristic of a
filter in some automated fashion to obtain the best possible signal
quality in spite of changing signal/system conditions. Adaptive
filters are usually associated with the broader topic of statistical
signal processing. The operation of signal filtering by definition
implies extracting something desired from a signal containing
both desired and undesired components. With linear FIR and IIR
filters the filter output is obtained as a linear function of the
observation (signal applied) to the input. An optimum linear filter in the minimum mean square sense can be designed to extract
a signal from noise by minimizing the error signal formed by
subtracting the filtered signal from the desired signal. For noisy
signals with time varying statistics, this minimization process is
often done using an adaptive filter.
For statistically stationary inputs this solution is known as a
Wiener filter.1
+
d[n]
e [ n ] Error
Σ
Desired
Signal

x[n]

Observation

Signal

Wiener
Filter

y[ n]

MMSE
Estimate
of d[n]

1. Simon Haykin, Adaptive Filter Theory, fourth edition, Prentice Hall, 2002.
ECE 5655/4655 Real-Time DSP

9–1

Chapter 9 • Adaptive Filters

Wiener Filter
• An M tap discrete-time Wiener filter is of the form
y[n] =

M–1

∑ wm x [ n – m ]

(9.1)

m=0

where the w m are referred to as the filter weights
– Note: (9.1) tells us that the Wiener filter is just an M-tap
FIR filter
• The quality of the filtered or estimated signal y [ n ] is determined from the error sequence e [ n ] = d [ n ] – y [ n ]
• The weights w m , m = 0, 1, …, M – 1 are chosen such that
2

2

E{e [n]} = E{(d[n] – y[n]) }

(9.2)

is minimized, that is we obtain the minimum mean-squared
error (MMSE)
• The optimal weights are found by setting
2

E { e [ n ] } = 0, m = 0, 1, …, M – 1
∂ wm

(9.3)

• From the orthogonality principle1 we choose the weights
such that the error e [ n ] is orthogonal to the observations
(data), i.e.,
E { x [ n – k ] ( d [ n ] – y [ n ] ) } = 0, k = 0, 1, …, M – 1

(9.4)

1. A. Papoulis, Probability, Random Variables, and Stochastic Processes,
third edition, McGraw-Hill, 1991.
9–2

ECE 5655/4655 Real-Time DSP

= .6) for k = 0..7) where R xx is the M × M correlation matrix associated with x[ n] .5) or M–1 w m φ xx [ m – k ] = ∑ m=0 φ xd [ – k ] (9. φ xx [ 0 ] (9. ….. . φ xx [ M – 1 ] . .. . φ xx [ – M + 1 ] .8) w o is the optimum weight vector given by ECE 5655/4655 Real-Time DSP 9–3 .. R xx φ xx [ 0 ] . . .. 1. M – 1 are known as the Wiener-Hopf or normal equations – Note: φ xx [ k ] is the autocorrelation function of x [ n ] and φ xd [ k ] is the cross-correlation function between x [ n ] and d[n] • In matrix form we can write R xx w o = p xd (9.Wiener Filter – This results in a filter that is optimum in the sense of minimum mean-square error • The resulting system of equations M–1 w m E { x [ n – k ]x [ n – m ] } ∑ m=0 = E{x[n – k](d[n])} (9.

and crosscorrelation functions with time averages of the form N–1 1 φ xx [ k ] ≅ ---N x [ n + k ]x [ n ] ∑ n=0 1 φ xd [ k ] ≅ ---N N–1 x [ n + k ]d [ n ] ∑ n=0 (9. that is we replace the true statistical auto.Chapter 9 • Adaptive Filters w o = w o0 w o1 … w oM – 1 T (9.10) • The optimal weight vector is given by –1 w o = R xx p xd (9.9) and p xd is the cross-correlation vector given by p xd = φ xd [ 0 ] φ xd [ – 1 ] … φ xd [ 1 – M ] T (9.13) where N is the sample block size Adaptive Wiener Filter • In an adaptive Wiener filter the error signal e [ n ] is fed back to the filter weights to adjust them using a steepest-descent algorithm 9–4 ECE 5655/4655 Real-Time DSP .11) can be solved using sample statistics.11) • As a matter of practice (9.12) (9.

is convex cup (a bowl shape) Mean Square Error Optimum at [1. that due to the squared error criterion. the error e [ n ] can be viewed as an M dimensional error surface.1] w0 w1 Error Surface for M = 2 • The filter decorrelates the output error e [ n ] so that signals in common to both d [ n ] and x [ n ] in a correlation sense are cancelled ECE 5655/4655 Real-Time DSP 9–5 .Adaptive Wiener Filter • With respect to the weight vector w .

.14) pˆ xd = x [ n ]d [ n ] (9. ˆ xx = x [ n ]x T [ n ] R (9.Chapter 9 • Adaptive Filters • A block diagram of this adaptive Wiener (FIR) filter is shown below + d[n] Desired Signal x[n] Observation Σ e[ n] Error Signal y[ n] MMSE Estimate of d[n] Wiener Filter Adaptive Algorithm Adaptive Wiener FIlter Least-Mean-Square Adaptation • Ideally the optimal weight solution can be obtained by applying the steepest descent method to the error surface.g.15) where x[n] = x[n] x[n – 1] … x[n – M + 1] T (9. but since the true gradient cannot be determined.16) • A practical implementation involves estimating the gradient from the available data using the least-mean-square (LMS) algorithm 9–6 ECE 5655/4655 Real-Time DSP . e. which is simply the instantaneous estimate of R xx and p xd from the available data. we use a stochastic gradient.

19) • For algorithm stability.Adaptive Wiener Filter • The steps to the LMS algorithm. for each new sample at time n.21) k=0 • In theory. are: – Filter x [ n ] to produce: y[n] = M–1 ∑ T w ˆ [ n ]x [ n ] ˆ m [ n ]x [ n – m ] = w (9.18) – Update the weight vector using step-size parameter μ : w ˆ [n + 1] = w ˆ [ n ] + μx [ n ]e [ n ] (9. (9.17) m=0 – Form the estimation error: e[n] = d[n] – y[n] (9.22) where λ max is the maximum eigenvalue of R xx ECE 5655/4655 Real-Time DSP 9–7 .20) where M–1 tap-input power = ∑ 2 E{ x[n – k] } (9.20) is equivalent to saying 2 0 < μ < ----------λ max (9. the step-size μ must be chosen such that 2 0 < μ < -------------------------------------tap-input power (9.

Stearns. New Jersey. B.Chapter 9 • Adaptive Filters Adaptive Filter Variations1 • Prediction s[n] d[n] Delay x[n] Adaptive Filter - + Σ y[n] e[n] • System Identification s[n] d[n] Plant Adaptive Filter x[n] y[n] + Σ e[n] • Equalization s[n] Training Pattern Delay Plant/ Channel Adaptive Σ + Filter x [ n ] + Noise y[n] d[n] + Σ e[n] 1. 1985. 9–8 ECE 5655/4655 Real-Time DSP . Adaptive Signal Processing. Widrow and S. Prentice Hall.

Adaptive Line Enhancement • Interference Canceling d[n] Signal + Interference Interference’ x[n] Adaptive Filter - + Σ y[n] e[n] Adaptive Line Enhancement • A relative of the interference canceling scheme shown above. is the adaptive line enhancer (ALE) • Here we assume we have a narrowband signal (say a sinusoid) buried in broadband additive noise x [ n ] = NB [ n ] + BB [ n ] + Σ - z –Δ x[n – Δ] Adaptive Filter e[n] y[n] • The filter adapts in such a way that a narrow passband forms around the sinusoid frequency. thereby suppressing much of the noise and improving the signal-to-noise ratio (SNR) in y[ n] ECE 5655/4655 Real-Time DSP 9–9 .

wo. 4/27/98 % % n x x % Sinusoid SNR = (A^2/2)/noise_var = 0:N.N.Wo] = lms_ale(SNR.mu) % %*******LMS ALE Simulation************ % SNR = Sinusoid SNR in dB % N = Number of simulation samples % M = FIR Filter length % mu = LMS step-size % % n = Index vector % x = Noisy input % y = Filtered output % e = Error sequence % wo = Final value of weight vector % F = Frequency response axis vector % Wo = Frequency response of filter %************************************** % Mark Wickert.23) • The SNR is defined as 2 A SNR = ---------22σ w (9.x.wo. % Here A = 1. % actually length N+1 = 1*cos(2*pi*1/20*n).Wo] = lms_ale(SNR.F.y.Chapter 9 • Adaptive Filters Example: MATLAB ALE Simulation • A simple MATLAB simulation is constructed using a single sinusoid at normalized frequency f o = 1 ⁄ 20 plus additive white Gaussian noise x [ n ] = A cos [ 2πf o n ] + w [ n ] (9.24) function [n.x.M.mu) % [n.e.F.e. so delay x by one sample 9–10 ECE 5655/4655 Real-Time DSP . White Noise -> Delta = 1. Fo/Fs = 1/20 = x + sqrt(1/2/(10^(SNR/10)))*randn(1.M.N.N+1).y.

^2) subplot(211) plot(n.y) plot(F.M).N+1). % Update the weight vector wo = wo + 2*mu*e(k)*xdm.5]) subplot(212) plot(n. % Form the error sequence e(k) = x(k) .e.Adaptive Line Enhancement xd = filter([0 1].01/64). plot(n. % Initialize weight vector to zero wo = zeros(1.M). % Initialize output vector y to zero y = zeros(1.64.e.1).y. end %end loop on time index k % Create filter frequency response [Wo. % Initialize filter memory to zero z = zeros(1. % Initialize error vector e to zero e = zeros(1.x. • A simulation is run using 1000 samples.M-1).Wo] = lms_ale(10.x).F.5 1.01 ⁄ 64 » » » » » » » » [n. % Update vector used for correlation with e(k) xdm = [xd(k) xdm(1:M-1)].x(k).1. Wo = 20*log10(abs(Wo)).. M = 64.1.z] = filter(wo.1.Wo) ECE 5655/4655 Real-Time DSP 9–11 .F] = freqz(wo.512. and μ = 0.z). SNR = 10 dB.wo. for k=1:N+1 % Filter one sample at a time [y(k).1000.y(k).N+1). % Initialize a vector for holding xd of length M xdm = zeros(1.x) axis([600 1000 -1.

01/64 1.5 0 -0.5 -1 600 Index n 9–12 ECE 5655/4655 Real-Time DSP . mu = 0.Chapter 9 • Adaptive Filters Convergence Occurs in Here (~275 samples) ALE x[n] & y[n] for SNR = 10 dB.5 -1 -1.5 600 650 700 750 650 700 750 800 850 900 950 1000 800 850 900 950 1000 1 y[n] 0.5 0 -0.5 1 x[n] 0.

5 Normalized Frequency f • A C version of the above MATLAB code would be very similar except all of the vector operations would have to be replaced by for loops • TBD C6x Code Examples A Two Channel Input Signal + Signal or Signal + Noise Canceller • In this first example a modified version of Chassaing’s Adaptnoise. mu = 0.35 0.01/64 5 0 -5 Sinusoid Frequency -10 |Wo(f)|dB -15 -20 -25 -30 -35 -40 -45 0 0.05 0. Resp.2 0.4 0.1 0.C6x Code Examples ALE Freq. for SNR = 10 dB.25 0.3 0.45 0.15 0.c program is considered ECE 5655/4655 Real-Time DSP 9–13 .

Wright.h" // Function Prototypes long int rand_int(void). but not exactly the same • Floating point C code is shown below: // Welch. but still be able to manipulate the left and right channels independently. #define LEFT 0 #define RIGHT 1 9–14 ECE 5655/4655 Real-Time DSP .Chapter 9 • Adaptive Filters • In this program the following system is implemented d[n] Signal1 + Signal2 (or noise) Signal2 x[n] Adaptive Filter y[n] + Σ e[n] • Notice that signal2 is present at both inputs in the exact same form. The union allows the data to be accessed as a single entity when transferring to and from the serial port. // Real-time Digital Signal Processing. // // // // Data is received as 2 16-bit words (left/right) packed into one 32-bit word. 2011 // Modified by Mark Wickert February 2012 to include GPIO ISR start/stop postings /////////////////////////////////////////////////////////////////////// // Filename: Adaptive_ISRs.c // // Synopsis: Interrupt service routine for codec data transmit/receive // /////////////////////////////////////////////////////////////////////// #include "DSP_Config. & Morrow. in a real application the input x [ n ] would be similar to signal2 as seen in d [ n ] .

ReadCodecData. //input right chan ECE 5655/4655 Real-Time DSP 9–15 . float beta = 1E-11. // Write to GPIO J15. E=0. Int16 Channel[2]. //weights for adapt filter float delay[N]. //input left chan noise = (float) CodecDataIn.// get input data samples /* add your code starting here */ desired = (float) CodecDataIn.Channel[ RIGHT].//output type for slider float alpha = 10. //overall output short out_type = 1.C6x Code Examples volatile union { Uint32 UINT. desired=0.e.//rate of convergence #define N 64 //# of weights (coefficients) #define LEFT 0 //left channel #define RIGHT 1 //right channel float w[N]. CodecDataOut. beta *= alpha. //input buffer to adapt filter short output.UINT = ReadCodecData(). dplusn=0. pin 6. // so serial port is reset to recover CodecDataIn. WriteCodecData // // Notes: None /////////////////////////////////////////////////////////////////////// { /* add any local variables here */ WriteDigitalOutputs(1). } CodecDataIn. /* add any global variables here */ //#define beta 1E-10. halted DSP) return. float yn=0.//alpha from slider interrupt void Codec_ISR() /////////////////////////////////////////////////////////////////////// // Purpose: Codec interface interrupt service routine // // Input: None // // Returns: Nothing // // Calls: CheckForOverrun.//scale beta by slider alpha value if(CheckForOverrun())// overrun error occurred (i.Channel[ LEFT]. noise=0. begin ISR timing pulse short i.

//"error" signal=(d+n)-yn //output of adaptive filter for (i = N-1. return a.UINT). pin 6.Channel[ LEFT] = output. //update delay samples } if (out_type == 1) //if slider in position 1 { output = (short) E. // Write to GPIO J15. i++) { yn += (w[i] * delay[i]).//desired+noise } CodecDataOut. 2011 /////////////////////////////////////////////////////////////////////// // Filename: main. & Morrow. //error signal as overall output } else if (out_type == 2) { output = (short) dplusn. i--) //to update weights and delays { w[i] = w[i] + beta*E*delay[i]. a = (a*125) % 2796203. end ISR timing pulse } //White noise generator for filter noise testing long int rand_int(void) { static long int a = 100001. i < N.Chapter 9 • Adaptive Filters dplusn = desired + noise. } //to calculate output of adaptive FIR E = (desired + noise) . // Real-time Digital Signal Processing.yn. //desired+noise delay[0] = noise. //update weights delay[i] = delay[i-1]. /* end your code here */ WriteCodecData(CodecDataOut. } // Welch.Channel[RIGHT] = 0. Wright.// send output data to port WriteDigitalOutputs(0).c // 9–16 ECE 5655/4655 Real-Time DSP . i >= 0. //noise as input to adapt FIR for (i = 0.//overall output result CodecDataOut.

interrupts drive operation while(1) { .0.//input buffer to adapt filter void StartUp() { InitDigitalOutputs(). // main stalls here. } //init variables //init weights of adaptive FIR //init buffer for delay sampless } ECE 5655/4655 Real-Time DSP 9–17 . T < N.C6x Code Examples // Synopsis: Main program file for demonstration code // /////////////////////////////////////////////////////////////////////// #include "DSP_Config. for (T = 0.//weights for adapt filter extern float delay[N]. // Real-time Digital Signal Processing. & Morrow. } } // Welch.c // // Synopsis: Adaptive filter initialize // /////////////////////////////////////////////////////////////////////// #include "DSP_Config. T++) { w[T] = 0. delay[T] = 0.h" #define N 64 //# of weights (coefficients) extern float w[N]. int T = 0. Wright. 2011 /////////////////////////////////////////////////////////////////////// // Filename: StartUp.0. // initialize filter weights and delay states DSP_Init().h" int main() { // call StartUp for application specific code // defined in each application directory StartUp().

the filter history in delay[n] is also updated in this loop GEL Slider to toggle between dplusn and the error E GEL slider to set the value of coefficient beta between 1E-12 and 1E-10 • A second GEL slider controls which filter output goes to the left codec channel • This adaptive filter was tested with two sinusoid inputs. one at 1 kHz as the desired signal and one at 2.Chapter 9 • Adaptive Filters • The left channel input (desired) is added to the right channel input (noise) and forms the LMS input d [ n ] (dplusn) • The right channel input by itself forms the input x [ n ] (noise) • A direct form FIR filter is implemented using floats in the first for loop • In the second for loop the filter tap weights w[n] are updated using coefficient beta which can be adjusted using a GEL file.5 kHz the noise signal • Analog outputs were captured using the PC sound card and 9–18 ECE 5655/4655 Real-Time DSP .

0 kHz Sinusoid + 2.5 kHz Noise Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 0 500 1000 1500 2000 2500 Frequency 3000 3500 4000 Error Signal Output: Noise Removed Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 0 500 1000 ECE 5655/4655 Real-Time DSP 1500 2000 2500 Frequency 3000 3500 4000 9–19 .C6x Code Examples spectrum analyzed using MATLAB Desired 1.

24. 9–20 ECE 5655/4655 Real-Time DSP .c program is considered • In this program the following system is implemented x [ n ] = NB [ n ] + BB [ n ] + Σ - z –Δ x[n – Δ] Adaptive Filter e[n] y[n] • Floating point C code is shown below: //Adaptpredict_AIC23. #define DSK6713_AIC23_INPUT_MIC 0x0015 #define DSK6713_AIC23_INPUT_LINE 0x0011 Uint16 inputsource=DSK6713_AIC23_INPUT_LINE. // select input //*********************************************************** // For left and right channel processing // left and right help in packed 32 bit word #define LEFT 0 #define RIGHT 1 union {Uint32 combo.} AIC23_data.Chapter 9 • Adaptive Filters Adaptive Line Enhancer • In this first example a modified version of Chassaing’s Adaptpredict.96} Uint32 fs=DSK6713_AIC23_FREQ_8KHZ.c Adaptive predictor to cancel interference //using interrupts // Enhanced version of Chassaing program // by Mark Wickert Spring 2009 //********************************************************** #include "DSK6713_AIC23.44. // select input //Uint16 inputsource=DSK6713_AIC23_INPUT_MIC.48.32.1.h" //set sampling rate {8. short channel[2].16.

//scale beta by slider alpha value AIC23_data.//alpha from slider interrupt void c_int11() //interrupt service routine { short i.combo = input_sample().yn. //output of adapt FIR filter //(wb+noise)-out adapt FIR for (i = N-1.//yn=out adapt FIR. //external interference short output. //update buf corr wb } ECE 5655/4655 Real-Time DSP 9–21 . E = splusn[0] . error signal float wb_signal.//output type for slider float alpha = 10. float yn.//rate of convergence beta *= alpha.C6x Code Examples // Program globals #define N 64 //# of coefficients of adapt FIR #define Ns 10 //length splusn buffer float splusn[Ns]. //wb signal+interference delay[0] = splusn[3].1*((short) rand_int()). //buffer wideband signal+interference float w[N]. i < N. i--) { w[i] = w[i]+(beta*E*delay[i]). //init output of adapt FIR for (i = 0. E.//output to codec float beta = 1E-13. //buffer for weights of adapt FIR float delay[N]. //delayed input to adapt FIR yn = 0. //in l&r as 32-bit wb_signal = (float) AIC23_data. i++) yn += (w[i] * delay[i]).//intern noise splusn[0] = wb_signal + noise. i >= 0. //update buf delay samps if (i < Ns-1) { splusn[i+1] = splusn[i].channel[RIGHT].//wideband desired signal float noise.channel[LEFT]. //buffer for input to adapt FIR Uint32 out_type = 1. //desired on l //noise = (float) AIC23_data. //noise on r noise = 0.//update wgts of FIR delay[i+1] = delay[i].

//init weights of adaptive FIR delay[T] = 0. T++) //init variables { w[T] = 0. //init wideband+interference } } comm_intr().//WB signal } else if (out_type == 2) { output =((short)yn). T < N. //infinite loop } //16 bit word random number generator int rand_int(void) { static int a = 100001. //init buffer for delay samples if (T < Ns) { splusn[T] = 0. /* if (out_type == 1)//if slider in position 1 { output = ((short)E). //init DSK.0.0. for (T = 0.//NB signal }*/ output =((short)yn).//NB signal //output =((short)wb_signal). return. codec. } void main() //main function { int T = 0. 9–22 ECE 5655/4655 Real-Time DSP . output_sample(output). a = (a*125) % 2796203.Chapter 9 • Adaptive Filters } //out_type = 2. McBSP while(1).//output resultto left chan.//WB signal //output = ((short)wb_signal).

. so here Δ = 3 • A GEL file allows the left channel of the codec to output either the wideband signal at setting 1 or the narrowband signal at setting two • The adaptation coefficient is set to be 10 ECE 5655/4655 Real-Time DSP – 12 9–23 . e. here Ns = 10 • splusn[3] serves as x [ n – Δ ] . } • The inputs are assumed to be a wideband signal. noise. e.C6x Code Examples return a..g.g. a sinusoid • The inputs arrive at the left and right codec inputs and are summed in code as wb_signal + noise • An input delay buffer of Ns samples is created in the array splusn[]. and a narrowband signal.

8 kHz sinusoid is input from a function generator and wideband noise is input from another function generator. the frequency response of the converged filter weights w[n] is displayed in CCS • The corresponding spectra of y [ n ] and e [ n ] are captured via the PC sound card The Wideband Signal Output Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 9–24 0 500 1000 1500 2000 2500 Frequency 3000 3500 4000 ECE 5655/4655 Real-Time DSP .Chapter 9 • Adaptive Filters • A 1.

C6x Code Examples The Narrowband Signal Output Power Spectrum Magnitude (dB) 30 20 10 0 −10 −20 −30 −40 −50 −60 0 500 1000 ECE 5655/4655 Real-Time DSP 1500 2000 2500 Frequency 3000 3500 4000 9–25 .

Chapter 9 • Adaptive Filters 9–26 ECE 5655/4655 Real-Time DSP .