Final Year Project Progress Report

Frequency-Domain Adaptive Filtering

Myles Friel 01510401

Supervisor: Dr.Edward Jones

Abstract
The Final Year Project is an important part of the final year of the Electronic Engineering Degree Course in NUI, Galway. The project brief that I was assigned comes under the Signal Processing area of Electronics. Specifically it involves Adaptive Filter Design. Basically an adaptive filter is a filter that can mimic the impulse response of other filters or other unknown systems. An everyday example of adaptive filters is in the telephone system. Impedance mismatches causing echoes of a signal are a significant source of annoyance to the users of the system. The job of the adaptive filter here is to estimate the characteristics of the echo path, generating the echo and compensate for it. To do this the echo path is viewed as a filter with a given impulse response and the adaptive filter must mimic this response. Adaptive Filters are generally implemented in the time Domain and as part of the project I must implement this, Matlab would be the programming language of choice here to develop the algorithm and indeed all other algorithms. Adaptive Filters in the time domain generally work quite well so long as the effective impulse response of the echo path remains relatively short. When the impulse response of the unknown system becomes too long time domain implementation of adaptive filters becomes quite inefficient. However there exists an alternative solution and that is to implement the filters in the frequency domain. In this project we will investigate fixed frequency domain algorithm to set the scene for the adaptive frequency domain algorithm. Investigation of the so called crossover point, this is the point where the frequency domain implementation becomes more efficient than the time domain implementation. Remember there is high cost associated with frequency domain implementation and the impulse response must be sufficiently long to make it more efficient than the time domain. Once this algorithm is tried and tested in Matlab it will be translated to the C programming language as a precursor for real time implementation. So far the algorithm is tried and tested in Matlab and I am currently investigating the cross-over point and comparing the memory requirements of time domain and frequency domain implementation.

Table of Contents
Page Chapter 1 Section 1.1 Section 1.2 Chapter 2 Section 2.1: Section 2.2: Section 2.3: Chapter 3 Section 3.1: Section 3.1.1: Section 3.1.2: Section 3.2: Section 3.2.1: Section 3.2.2: Conclusion Reference and Bibliography Introduction Project Specification Project Time Frame The Least Mean Squared Algorithm The LMS Algorithm explained Coding The LMS Algorithm The Weakness of The LMS Algorithm Filtering in The Frequency Domain A Fixed Frequency Domain Algorithm The Fast Fourier Transform The Overlap-Save Method An Adaptive Frequency Domain Algorithm The Fast LMS Algorithm, step by step The First Input Block. 1 2 3 4 5 6 8 9 10 10 10 11 12 14 15 16

Chapter 1
Introduction.

page1

Chapter 1

Introduction

Section 1.1:

Project Specification

Adaptive filters are widely used in many situations where the characteristics of some filter or other are unknown. One application for these filters is quite an important part of today's telephone system, that is, compensating for the echo problem. The echo path can be viewed as a filter with a certain impulse response, and the task of the adaptive filter is to mimic this impulse response. Adaptive filter implementation is common in the time domain, however for an unknown system with a very long impulse response it becomes more efficient to implement the filter in the frequency domain. The objective of this project is to investigate the design and implementation of adaptive filter algorithms, with emphasis on their use in system identification problems. It will be necessary to develop simulations of adaptive filter algorithms in the time and frequency domains and compare them from the point of view of performance and implementation complexity (in particular to determine the cross over point where frequency-domain implementation becomes more efficient). It will then be necessary to translate the frequency-domain adaptive filter into the C programming language. A suitable laboratory test bench involving a real circuit should be constructed in order to demonstrate the functionality in near real-time. Further investigation in other applications scenarios may also be carried out, as well as some investigation of more efficient implementation using integer arithmetic. With the completion of the project we should have a fully operational adaptive filter based in frequency domain. This filter will be applicable where the length of the unknown system's impulse response is long enough for efficient implementation of frequency domain adaptive filtering. Time domain adaptive filtering will not be replaced entirely as it is far more efficient for systems with short impulse responses. A possible application for the adaptive filter in the frequency domain arises in acoustic echo cancellation for hands free telephony.

page2

Chapter 1

Introduction

Section 1.2:

Project Time Frame

Pass

Development of time-domain adaptive filter simulation focusing on Time allowed: 3 weeks system identification applications. Investigation of frequencyTarget Date: 20 October domain filter algorithms and simulation in Matlab. H2.2 Extending the fixed frequency-domain filter to an adaptive filter. Time allowed: 5-6 weeks Comparison of time and frequency-domain approaches with respect Target Date: 26 November to performance and implementation complexity. H2.1 Translating the frequency-domain adaptive filter to the C programming language, a precursor to real time implementation. H1 Developing a suitable system for real-time operation and modifying Time allowed: 2-3 weeks the C simulation to allow this real-time operation. A suitable test Target Date: 8 February bench with an analogue interface and an unknown system will demonstrate the adaptive filter in operation in the lab. H1(outstanding) Investigation of further applications, adaptive equalization and analysis of the algorithm behaviour when using integer arithmetic. Time allowed: 2-3 weeks Target Date: 1 March Time allowed: 2-3 weeks Target Date: 21 January

page3

Chapter 2
The Least Mean Square Algorithm.

page4

Chapter 2

The Least Mean Squared Algorithm

Section 2.1:
Updated value of tap-weight vector =

The LMS Algorithm explained

old value learningtapof tap-weight + [ rate x input x error signal ] vector parameter vector

The Least-Mean-Square algorithm in words:

The simplicity of the LMS algorithm and ease of implementation means that it is the best choice for many real-time systems. The implementation steps for the LMS algorithm 1. Use the Matlab filter function to find the desired response. Set each coefficient weight hi=0 ,i=1,2,3,... , N , For each sampling instant (k) carry out steps (2) to (4): 2. Load the current data sample k into the first position in the filter memory, clock it into the shift register and move all the previous entries along one position. Calculate the output of the adaptive filter by multiplying each element in the array of filter coefficients by the corresponding element in the array representing the shift register and all the results are summed to give the output corresponding to that data that was earlier clocked into the shift register. y k = hi x i
i=0 N 1

to zero.

3. Before the filter coefficients can be updated the error must be calculated simply find the difference between the desired response and the output of the adaptive filter. e k = y k d k 4. To update the filter coefficients multiply the error by , the step size parameter and then multiply the result by the contents of the shift register and add this result to the values of the input coefficients previously. hk 1=hk 2 e k x k There are also other LMS based algorithms which include The complex LMS, The block LMS algorithm and The Time sequenced LMS algorithm. These algorithms are described in more detail in [1] in the References section. page5

Chapter 2

The Least Mean Squared Algorithm

Section 2.2:

Coding the LMS Algorithm

A very important part of the code is the updating of the filter coefficients as would be typical for all adaptive filter algorithms. The step size parameter , is critical for the update and must be chosen accurately. Updating the filter coefficients is important because this is the part of the code that governs how well the filter will converge to the desired output. Another element that has a key role in this convergence is the number of filter coefficients. The effects of varying both the value of and the number of filter coefficients, N is demonstrated in the project by programs which vary these numbers and indicate the resulting effects by means of graphs, examples of which are shown in figures 2.1 and 2.2.

1.4

x 10

-7

Mean Square Error (Effect of varying mu)

1.2

0.8 Error 0.6 0.4 0.2 0 1

4 Values of mu

6 x 10

7
-3

Figure 2.1. The effect of varying mu.

page6

Chapter 2

The Least Mean Squared Algorithm

Mean Square Error (Effect of varying N) 0.09

0.08

0.07

0.06

0.05 Error 0.04 0.03 0.02 0.01 0

10

15

20

25 Values of N

30

35

40

45

50

Figure 2.2. The effect of varying the number of filter coefficients

Sample unknown system 0.05

0.04

0.03

0.02

Amplitude

0.01

-0.01

-0.02

-0.03

-0.04

10

15

20

25 30 Sample instants

35

40

45

50

Figure 2.3. The sample unknown system used as a reference page7

Chapter 2

The Least Mean Squared Algorithm

Close examination of figures 2.2 and 2.3 will reveal the relationship between the length of the impulse response of the unknown system and the number of filter coefficients necessary. Although the unknown system in figure 2.3 has fifty values taken at sample intervals it is clear that twenty five or even less is enough to describe the system. Therefore twenty five values will also suffice for the adaptive filter as is illustrated in figure 2.2.

Section 2.3:

The Weakness of the LMS Algorithm

The LMS Algorithm has one profound weakness, and it is that all the filter coefficients must be updated for every sample value taken in. This is mainly due to the fact that the algorithm lies in the time domain, leaving the algorithm at an obvious disadvantage when an impulse response is very long. The computational power required becomes too high for efficient use of the algorithm. Frequency domain adaptive filtering holds the key to the solution of the very long impulse response problem. In the next chapter we will first discuss some fixed frequency domain algorithms as a basis for the adaptive frequency domain algorithms.

page8

Chapter 3
Filtering in the Frequency Domain.

page9

Chapter 3

Filtering in the Frequency Domain

Section 3.1:
Section 3.1.1.

A Fixed Frequency Domain Algorithm The Fast Fourier Transform

The Fast Fourier Transform (FFT) is an algorithm used to compute the Discrete Fourier Transform (DFT) of a vector x. In the context of this project the vector x is likely to contain sampled values representing a signal in the time domain and when the Discrete Fourier Transform of a signal is calculated the resulting sequence is converted to the frequency domain.

Section 3.1.2.

The Overlap-Save Method

The Overlap-Add and the Overlap-Save are the two main fixed frequency domain algorithms. In this section we will discuss the Overlap-Save method since the Fast-LMS algorithm which is used for the adaptive frequency domain filter is based on it and also because of the fact that we can simply save the desired elements and not worry about adding the correct elements as would be involved with overlap-Add. The block diagram in figure 3.1. illustrates the overlap-save algorithm as implemented in the overlap-save m file included in the disc that comes with this report.

Input Block Length 2len

len Zeros + Impulse Response

2N point FFT

2N point FFT

Y = (xi_fft) x (hn_fft)

2N point IFFT

yOut + circular artifacts Figure 3.1. Block Diagram for the overlap-save algorithm.

page10

Chapter 3 The Overlap-Save Algorithm, step by step.

Filtering in the Frequency Domain

1. In the implementation of this algorithm that is included on the disc let N the length of the impulse response of the system. The length of the input is twice this at 2N. Len zeros are added to the left of the impulse response so that the result of the FFT will be the same length as that of the FFTs of the input sections. 2. N zeros are also added to the left and to the right of the input array to ensure that all segments of length N will occupy both halves of the input block of which the FFT will be calculated. 3. The FFT of the impulse response is calculated and stored in memory because it will remain unchanged. 4. Next the first block of the input is taken and the FFT of it is calculated. 5. The two FFTs are now multiplied, that is to say that each element in one of the arrays will be multiplied by the corresponding element in the other. This procedure corresponds to convolution in the time domain.[1] 6. The second half of this result is dumped1 for each convolution. The first half is added to an array as the output of the filter for the given input block. 7. The input block is updated applying 50% overlap.

This algorithm has been tried and tested in the overlap-save m file, which is included on the disc accompanying this report. It is a fine algorithm so long as the filter coefficients are known. In the next section we will consider an algorithm that will enable adaptive filters to operate in the frequency domain.
Note 1. This seemingly important data can be simply dumped because of the missing zeros in the input data which is generally added for the purposes of convolution. A general rule of convolution is that, if N2 is the length of the impulse response and N1 is the length of the input signal then N2-1 zeros should be added to the the input sequence and N1-1 zeros should be added to the impulse response allowing the correct linear convolution to be obtained. Note that zeros are added to the start of the impulse response in this case and that no corresponding zeros were added to each block of input data and therefore instead of N zeros preceding N data samples, what is had here is 2N data samples. With the 50% overlap intact the second half of each convolution sum can be dumped since it contains data which is a result of circular convolution.

Section 3.2.

An Adaptive Frequency Domain Algorithm

Some research into this topic quickly identified the Fast LMS Algorithm as the main frequency domain based adaptive algorithm and so it is on this algorithm that the frequency domain adaptive filter will be based for the project. A block diagram illustrating the flow of the algorithm is shown in figure 3.2. page11

Chapter 3 u(n)

FFT

U(k)

Y(k)

IFFT

Filtering in the Frequency Domain y(n) Save last Block

Delay X FFT Append Zero Block I

Gradient Constraint

Delete Last Block Conjugate IFFT Uh(k) X E(k) FFT Insert Zero Block e(n) d(n)

Fig 3.2. Block Diagram for the Fast LMS algorithm[1]

page12

Chapter 3

Filtering in the Frequency Domain

Section 3.2.1.

The Fast LMS Algorithm, step by step.

1. As in the Overlap-Save algorithm N is the length of the impulse response of the unknown system. Blocks of size 2N will be taken from the input at a time with 50% overlap as before. W will donate the filter coefficients which will be initialized to zero and updated after each block. 2. The desired output is obtained by using the Matlab Filter function. This function takes the impulse response of the unknown system and creates a filter based on and passes the input through this filter. The desired response of the adaptive filter is now known and the will use this to update it's coefficients correctly. 3. Similar to the Overlap-Save Algorithm we add N zeros to the start and to the end of the input array to ensure correct convolution results. 4. An input block of size 2N is taken, U the FFT of this block is calculated and it is stored in the main diagonal of a matrix for ease of access. U (the FFT of the input block) is now multiplied by W (the filter coefficients) the result is correspondent to convolution in the time domain, the inverse FFT is computed and the first half of the result is dumped because they correspond to circular convolution and the second half is temporarily saved in yn until it is added to the output array. 5. The error signal is computed next by means of simple subtraction to calculate the difference en = dn yn where dn is the corresponding section of the desired response. The error is brought into the frequency domain by adding N zeros to the start of en and by computing a 2N point FFT and the result is called Ek. Ek = FFT [zeros, en] 6. The conjugate of U is found by the Matlab function conj this is multiplied by Ek and the inverse FFT of the result is calculated. This time it is the second half of the result that is dropped due to circular convolution. 7. N zeros are now added to the end of what we are left with and F2_fft, the 2N point FFT of the resulting sequence is calculated and the result is multiplied by (the step size parameter) this is called W1 and is added to W and this process is how the update of the coefficients is conducted. W1 = F2_fft. mu W = W + W1 8. This newly updated W will now be used as the filter coefficients for the next block of input an error will exist however, as W is updated more often this error will diminish as is indicated in figure 3.3. which shows the convergence of the filter coefficients to the optimum. page13

Chapter 3
Convergence quality 1 Desired Adaptive Filter

Filtering in the Frequency Domain

0.8

0.6

0.4

0.2 Amplitude

-0.2

-0.4

-0.6

-0.8

-1

100

200

300

400 500 600 Sample Number

700

800

900

1000

Figure 3.3. The convergence of the adaptive filter coefficients Notice that the two both the desired response and the result that the adaptive filter produces map almost perfectly upon each other after only a few input blocks have been processed.

Section 3.3.2

The First Input Block

Notice also in figure 3.3. that the results for the first block are also quite good despite their being no previous values on which to base the filter coefficients. This occurs because of the special treatment given to the first block in this version of the algorithm. Initially the filter coefficients are all set to zero and thus the output for the first block would also be zero. To avoid this problem the the error associated with the first input block is calculated with yn still set at zero, which means that the error becomes the actual desired response itself, this error is then used to set the coefficients for the filter and these are used for the first input block instead of the zeros, the coefficients are updated after this as per usual in preparation for the next input block. page14

Conclusion
The project is progressing at a satisfactory rate. All areas of the project implemented to date have been discussed in this Report, including the relevant algorithms which have been explained in detail. I hope to start converting the Fast LMS Algorithm to the C Programming Language soon and hopefully it will be straight forward to verify it's functionality in a simulation environment.

page15

References
[1] Digital Signal Processing: A practical Approach 2nd edition E.Ifeachor and B.Jervis, Prentice Hall. [2] Adaptive Filter Theory, Simon Haykin

Bibliography
[1] www.music.miami.edu/programs/Mue/mue2003/research/jvandekieft/jvchapter2. [2] Signal Processing Algorithms in Matlab S.Stearns and R.David, Chapter 9 [3] IEEE Transactions on Signal Processing, Vol. 39,No. 10, October 1991.

page16