Final Year Project Progress Report

Frequency-Domain Adaptive Filtering

Myles Friel

01510401

Supervisor: Dr.Edward Jones

 Abstract

The Final Year Project is an important part of the final year of the Electronic Engineering Degree
Course in NUI, Galway. The project brief that I was assigned comes under the Signal Processing
area of Electronics. Specifically it involves Adaptive Filter Design. Basically an adaptive filter is a
filter that can mimic the impulse response of other filters or other unknown systems. An everyday
example of adaptive filters is in the telephone system. Impedance mismatches causing echoes of a
signal are a significant source of annoyance to the users of the system. The job of the adaptive filter
here is to estimate the characteristics of the echo path, generating the echo and compensate for it. To
do this the echo path is viewed as a filter with a given impulse response and the adaptive filter must
mimic this response.

Adaptive Filters are generally implemented in the time Domain and as part of the project I must
implement this, Matlab would be the programming language of choice here to develop the algorithm
and indeed all other algorithms. Adaptive Filters in the time domain generally work quite well so
long as the effective impulse response of the echo path remains relatively short. When the impulse
response of the unknown system becomes too long time domain implementation of adaptive filters
becomes quite inefficient. However there exists an alternative solution and that is to implement the
filters in the frequency domain. In this project we will investigate fixed frequency domain algorithm
to set the scene for the adaptive frequency domain algorithm. Investigation of the so called cross-
over point, this is the point where the frequency domain implementation becomes more efficient
than the time domain implementation. Remember there is high cost associated with frequency
domain implementation and the impulse response must be sufficiently long to make it more efficient
than the time domain. Once this algorithm is tried and tested in Matlab it will be translated to the C
programming language as a precursor for real time implementation.
So far the algorithm is tried and tested in Matlab and I am currently investigating the cross-over
point and comparing the memory requirements of time domain and frequency domain
implementation.
 Table of Contents
Page

Chapter 1 Introduction 1
Section 1.1 Project Specification 2
Section 1.2 Project Time Frame 3

Chapter 2 The Least Mean Squared Algorithm 4

Section 2.1: The LMS Algorithm explained 5
Section 2.2: Coding The LMS Algorithm 6
Section 2.3: The Weakness of The LMS Algorithm 8

Chapter 3 Filtering in The Frequency Domain 9

Section 3.1: A Fixed Frequency Domain Algorithm 10
Section 3.1.1: The Fast Fourier Transform 10
Section 3.1.2: The Overlap-Save Method 10
Section 3.2: An Adaptive Frequency Domain Algorithm 11
Section 3.2.1: The Fast LMS Algorithm, step by step 12
Section 3.2.2: The First Input Block. 14

Conclusion 15
Reference and Bibliography 16
Chapter 1

Introduction.

page1
Chapter 1 Introduction
Section 1.1: Project Specification

Adaptive filters are widely used in many situations where the characteristics of some filter or other
are unknown. One application for these filters is quite an important part of today's telephone system,
that is, compensating for the echo problem. The echo path can be viewed as a filter with a certain
impulse response, and the task of the adaptive filter is to “mimic” this impulse response.
Adaptive filter implementation is common in the time domain, however for an unknown
system with a very long impulse response it becomes more efficient to implement the filter in the
frequency domain.
The objective of this project is to investigate the design and implementation of adaptive
filter algorithms, with emphasis on their use in “system identification” problems. It will be
necessary to develop simulations of adaptive filter algorithms in the time and frequency domains
and compare them from the point of view of performance and implementation complexity (in
particular to determine the “cross over point” where frequency-domain implementation becomes
more efficient). It will then be necessary to translate the frequency-domain adaptive filter into the C
programming language. A suitable laboratory test bench involving a real circuit should be
constructed in order to demonstrate the functionality in near real-time. Further investigation in other
applications scenarios may also be carried out, as well as some investigation of more efficient
implementation using integer arithmetic.
With the completion of the project we should have a fully operational adaptive filter based in
frequency domain. This filter will be applicable where the length of the unknown system's impulse
response is long enough for efficient implementation of frequency domain adaptive filtering. Time
domain adaptive filtering will not be replaced entirely as it is far more efficient for systems with
short impulse responses. A possible application for the adaptive filter in the frequency domain
arises in acoustic echo cancellation for hands free telephony.

page2
Chapter 1 Introduction

Section 1.2: Project Time Frame

Pass
Development of time-domain adaptive filter simulation focusing on Time allowed: 3 weeks
system identification applications. Investigation of frequency-
Target Date: 20 October
domain filter algorithms and simulation in Matlab.
H2.2
Extending the fixed frequency-domain filter to an adaptive filter. Time allowed: 5-6 weeks
Comparison of time and frequency-domain approaches with respect
Target Date: 26 November
to performance and implementation complexity.
H2.1
Translating the frequency-domain adaptive filter to the C Time allowed: 2-3 weeks
programming language, a precursor to real time implementation.
Target Date: 21 January
H1
Developing a suitable system for real-time operation and modifying Time allowed: 2-3 weeks
the C simulation to allow this real-time operation. A suitable test
Target Date: 8 February
bench with an analogue interface and an “unknown system” will
demonstrate the adaptive filter in operation in the lab.
H1(outstanding)
Investigation of further applications, adaptive equalization and Time allowed: 2-3 weeks
analysis of the algorithm behaviour when using integer arithmetic.
Target Date: 1 March

page3
 Chapter 2

The Least Mean Square Algorithm.

page4
Chapter 2 The Least Mean Squared Algorithm

Section 2.1: The LMS Algorithm explained

The Least-Mean-Square algorithm in words:
Updated value old value learning- tap-
of tap-weight = of tap-weight + [ rate x input x error signal ]
vector vector parameter vector

The simplicity of the LMS algorithm and ease of implementation means that it is the best choice for
many real-time systems.

The implementation steps for the LMS algorithm

1. Use the Matlab “filter” function to find the desired response. Set each coefficient weight
hi=0 ,i=1,2,3,... , N , to zero.
For each sampling instant (k) carry out steps (2) to (4):
2. Load the current data sample k into the first position in the filter memory, clock it into the shift
register and move all the previous entries along one position. Calculate the output of the adaptive
filter by multiplying each element in the array of filter coefficients by the corresponding element in
the array representing the shift register and all the results are summed to give the output
corresponding to that data that was earlier clocked into the shift register.
N −1
y k = ∑ hi x i
i=0

3. Before the filter coefficients can be updated the error must be calculated simply find the
difference between the desired response and the output of the adaptive filter.
e k = y k −d k 
4. To update the filter coefficients multiply the error by µ, the step size parameter and then multiply
the result by the contents of the shift register and add this result to the values of the input
coefficients previously.
hk 1=hk 2  e k  x k 

There are also other LMS based algorithms which include

The complex LMS, The block LMS algorithm and The Time sequenced LMS algorithm.

These algorithms are described in more detail in [1] in the References section.
page5
Chapter 2 The Least Mean Squared Algorithm

Section 2.2: Coding the LMS Algorithm

A very important part of the code is the updating of the filter coefficients as would be typical for all
adaptive filter algorithms. The step size parameter µ, is critical for the update and must be chosen
accurately. Updating the filter coefficients is important because this is the part of the code that
governs how well the filter will converge to the desired output. Another element that has a key role
in this convergence is the number of filter coefficients.

The effects of varying both the value of µ and the number of filter coefficients, N is demonstrated
in the project by programs which vary these numbers and indicate the resulting effects by means of
graphs, examples of which are shown in figures 2.1 and 2.2.

-7 Mean Square Error (Effect of varying mu)

x 10
1.4

1.2

0.8
Error

0.6

0.4

0.2

0
1 2 3 4 5 6 7
Values of mu -3
x 10

Figure 2.1. The effect of varying mu.

page6
Chapter 2 The Least Mean Squared Algorithm

Mean Square Error (Effect of varying N)

0.09

0.08

0.07

0.06

0.05
Error

0.04

0.03

0.02

0.01

0
0 5 10 15 20 25 30 35 40 45 50
Values of N

Figure 2.2. The effect of varying the number of filter coefficients

Sample unknown system

0.05

0.04

0.03

0.02
Amplitude

0.01

-0.01

-0.02

-0.03

-0.04
0 5 10 15 20 25 30 35 40 45 50
Sample instants

Figure 2.3. The sample unknown system used as a reference

page7
Chapter 2 The Least Mean Squared Algorithm

Close examination of figures 2.2 and 2.3 will reveal the relationship between the length of the
impulse response of the unknown system and the number of filter coefficients necessary. Although
the unknown system in figure 2.3 has fifty values taken at sample intervals it is clear that twenty
five or even less is enough to describe the system. Therefore twenty five values will also suffice for
the adaptive filter as is illustrated in figure 2.2.

Section 2.3: The Weakness of the LMS Algorithm

The LMS Algorithm has one profound weakness, and it is that all the filter coefficients must be
updated for every sample value taken in. This is mainly due to the fact that the algorithm lies in the
time domain, leaving the algorithm at an obvious disadvantage when an impulse response is very
long. The computational power required becomes too high for efficient use of the algorithm.
Frequency domain adaptive filtering holds the key to the solution of the very long impulse response
problem.
In the next chapter we will first discuss some fixed frequency domain algorithms as a basis
for the adaptive frequency domain algorithms.

page8
 Chapter 3

Filtering in the Frequency Domain.

page9
Chapter 3 Filtering in the Frequency Domain

Section 3.1: A Fixed Frequency Domain Algorithm

Section 3.1.1. The Fast Fourier Transform

The Fast Fourier Transform (FFT) is an algorithm used to compute the Discrete Fourier Transform
(DFT) of a vector x. In the context of this project the vector x is likely to contain sampled values
representing a signal in the time domain and when the Discrete Fourier Transform of a signal is
calculated the resulting sequence is converted to the frequency domain.

Section 3.1.2. The Overlap-Save Method

The Overlap-Add and the Overlap-Save are the two main fixed frequency domain algorithms. In
this section we will discuss the Overlap-Save method since the Fast-LMS algorithm which is used
for the adaptive frequency domain filter is based on it and also because of the fact that we can
simply save the desired elements and not worry about adding the correct elements as would be
involved with overlap-Add.
The block diagram in figure 3.1. illustrates the overlap-save algorithm as implemented in the
overlap-save m file included in the disc that comes with this report.

Input Block Length 2len len Zeros + Impulse Response

2N point FFT 2N point FFT

Y = (xi_fft) x (hn_fft)

2N point IFFT

yOut + circular artifacts

Figure 3.1. Block Diagram for the overlap-save algorithm.

page10
Chapter 3 Filtering in the Frequency Domain
The Overlap-Save Algorithm, step by step.
1. In the implementation of this algorithm that is included on the disc let N the length of the
impulse response of the system. The length of the input is twice this at 2N.
Len zeros are added to the left of the impulse response so that the result of the FFT will be the
same length as that of the FFTs of the input sections.
2. N zeros are also added to the left and to the right of the input array to ensure that all segments
of length N will occupy both halves of the input block of which the FFT will be calculated.
3. The FFT of the impulse response is calculated and stored in memory because it will remain
unchanged.
4. Next the first block of the input is taken and the FFT of it is calculated.
5. The two FFTs are now multiplied, that is to say that each element in one of the arrays will be
multiplied by the corresponding element in the other. This procedure corresponds to
convolution in the time domain.[1]
6. The second half of this result is dumped1 for each convolution.
The first half is added to an array as the output of the filter for the given input block.
7. The input block is updated applying 50% overlap.

This algorithm has been tried and tested in the overlap-save “m file”, which is included on the disc
accompanying this report. It is a fine algorithm so long as the filter coefficients are known. In the
next section we will consider an algorithm that will enable adaptive filters to operate in the
frequency domain.

Note 1. This seemingly important data can be simply dumped because of the missing zeros in the input data which is
generally added for the purposes of convolution. A general rule of convolution is that, if N2 is the length of the impulse
response and N1 is the length of the input signal then N2-1 zeros should be added to the the input sequence and N1-1
zeros should be added to the impulse response allowing the correct linear convolution to be obtained. Note that zeros are
added to the start of the impulse response in this case and that no corresponding zeros were added to each block of
input data and therefore instead of N zeros preceding N data samples, what is had here is 2N data samples. With the
50% overlap intact the second half of each convolution sum can be dumped since it contains data which is a result of
circular convolution.

Section 3.2. An Adaptive Frequency Domain Algorithm

Some research into this topic quickly identified the Fast LMS Algorithm as the main frequency
domain based adaptive algorithm and so it is on this algorithm that the frequency domain adaptive
filter will be based for the project. A block diagram illustrating the flow of the algorithm is shown
in figure 3.2.
page11
Chapter 3 Filtering in the Frequency Domain
u(n) U(k) Y(k) y(n)
FFT X IFFT Save last Block

Delay
∑

αI
X

FFT Gradient
Constraint

Append
Zero Block

Delete
Last Block
Conjugate
IFFT

Uh(k) E(k) Insert e(n) d(n)

X FFT ∑
Zero Block
Fig 3.2. Block Diagram for the Fast LMS algorithm[1]

page12
Chapter 3 Filtering in the Frequency Domain

Section 3.2.1. The Fast LMS Algorithm, step by step.

1. As in the Overlap-Save algorithm N is the length of the impulse response of the unknown
system. Blocks of size 2N will be taken from the input at a time with 50% overlap as before. W
will donate the filter coefficients which will be initialized to zero and updated after each block.
2. The desired output is obtained by using the Matlab “Filter” function. This function takes the
impulse response of the unknown system and creates a filter based on and passes the input
through this filter. The desired response of the adaptive filter is now known and the will use this
to update it's coefficients correctly.
3. Similar to the Overlap-Save Algorithm we add N zeros to the start and to the end of the input
array to ensure correct convolution results.
4. An input block of size 2N is taken, U the FFT of this block is calculated and it is stored in the
main diagonal of a matrix for ease of access. U (the FFT of the input block) is now multiplied by
W (the filter coefficients) the result is correspondent to convolution in the time domain, the
inverse FFT is computed and the first half of the result is dumped because they correspond to
circular convolution and the second half is temporarily saved in yn until it is added to the output
array.
5. The error signal is computed next by means of simple subtraction to calculate the difference
en = dn – yn
where dn is the corresponding section of the desired response.
The error is brought into the frequency domain by adding N zeros to the start of en and by
computing a 2N point FFT and the result is called Ek.
Ek = FFT [zeros, en]
6. The conjugate of U is found by the Matlab function “conj” this is multiplied by Ek and the
inverse FFT of the result is calculated. This time it is the second half of the result that is dropped
due to circular convolution.
7. N zeros are now added to the end of what we are left with and F2_fft, the 2N point FFT of the
resulting sequence is calculated and the result is multiplied by μ (the step size parameter) this is
called W1 and is added to W and this process is how the update of the coefficients is conducted.

W1 = F2_fft. mu
W = W + W1
8. This newly updated W will now be used as the filter coefficients for the next block of input an
error will exist however, as W is updated more often this error will diminish as is indicated in
figure 3.3. which shows the convergence of the filter coefficients to the optimum.

page13
Chapter 3 Filtering in the Frequency Domain

Convergence quality
1
Desired
Adaptive Filter
0.8

0.6

0.4

0.2
Amplitude

-0.2

-0.4

-0.6

-0.8

-1
0 100 200 300 400 500 600 700 800 900 1000
Sample Number

Figure 3.3. The convergence of the adaptive filter coefficients

Notice that the two both the desired response and the result that the adaptive filter produces map
almost perfectly upon each other after only a few input blocks have been processed.

Section 3.3.2 The First Input Block

Notice also in figure 3.3. that the results for the first block are also quite good despite their being no
previous values on which to base the filter coefficients. This occurs because of the special treatment
given to the first block in this version of the algorithm. Initially the filter coefficients are all set to
zero and thus the output for the first block would also be zero. To avoid this problem the the error
associated with the first input block is calculated with yn still set at zero, which means that the error
becomes the actual desired response itself, this error is then used to set the coefficients for the filter
and these are used for the first input block instead of the zeros, the coefficients are updated after this
as per usual in preparation for the next input block.

page14
 Conclusion
The project is progressing at a satisfactory rate. All areas of the project implemented to date have
been discussed in this Report, including the relevant algorithms which have been explained in
detail. I hope to start converting the Fast LMS Algorithm to the C Programming Language soon and
hopefully it will be straight forward to verify it's functionality in a simulation environment.

page15
 References
[1] “Digital Signal Processing: A practical Approach”
2nd edition E.Ifeachor and B.Jervis, Prentice Hall.
[2] “Adaptive Filter Theory”, Simon Haykin

Bibliography
[1] www.music.miami.edu/programs/Mue/mue2003/research/jvandekieft/jvchapter2.
[2] “Signal Processing Algorithms in Matlab”
S.Stearns and R.David, Chapter 9
[3] IEEE Transactions on Signal Processing, Vol. 39,No. 10, October 1991.

page16