Wong & Lok: Theory of Digital Communications

4. ISI & Equalization

Chapter 4 Intersymbol Interference and Equalization
The all-pass assumption made in the AWGN (or non-dispersive) channel model is rarely practical. Due to the scarcity of the frequency spectrum, we usually filter the transmitted signal to limit its bandwidth so that efficient sharing of the frequency resource can be achieved. Moreover, many practical channels are bandpass and, in fact, they often respond differently to inputs with different frequency components, i.e., they are dispersive. We have to refine the simple AWGN (or non-dispersive) model to accurately represent this type of practical channels. One such commonly employed refinement is the dispersive channel model1:

ִص
where ٴص is the transmitted signal, with power spectral density Ƽ by the linear filter

Ù £ ´Øµ · Ҵص

(4.1)

´Øµ.

¾.

´Øµ is the impulse response of the channel, and Ҵص is AWGN

In essence, we model the dispersive characteristic of the channel

The simplest dispersive channel is the bandlimited channel for which the

channel impulse response

´Øµ is that of an ideal lowpass filter.

This lowpass filtering smears the

transmitted signal in time causing the effect of a symbol to spread to adjacent symbols when a sequence of symbols are transmitted. The resulting interference, intersymbol interference (ISI), degrades the error performance of the communication system. There are two major ways to mitigate the detrimental effect of ISI. The first method is to design bandlimited transmission pulses which minimize the the
1

For simplicity, all the signals considered in this chapter are real baseband signals. All the developments here can, of

course, be generalized to bandpass signals using either the real bandpass representation or the complex baseband representation.

4.1

Wong & Lok: Theory of Digital Communications

4. ISI & Equalization

effect of ISI. We will describe such a design for the simple case of bandlimited channels. The ISIfree pulses obtained are called the Nyquist pulses. The second method is to filter the received signal to cancel the ISI introduced by the channel impulse response. This approach is generally known as equalization.

4.1 Intersymbol Interference
To understand what ISI is, let us consider the transmission of a sequence of symbols with the basic waveform

ٴص. To send the Òth symbol Ò , we send Ò Ù´Ø   ÒÌ µ, where Ì is the symbol interval.
Ò Ù´Ø

Therefore, the transmitted signal is
Ò

  ÒÌ µ   ÒÌ µ · Ҵص

(4.2)

Based on the dispersive channel model, the received signal is given by

ִص
where Ú ´Øµ

Ò

Ò Ú ´Ø

(4.3)

Ù £ ´Øµ is the received waveform for a symbol. If a single symbol, say the symbol ¼ , Ú ´ Øµ and then sample the matched filter output at

is transmitted, the optimal demodulator is the one that employs the matched filter, i.e., we can pass the received signal through the matched filter Ú ´Øµ time Ø

¼ to obtain the decision statistic. When a sequence of symbols are transmitted, we can still
ÑÌ to obtain the decision statistic for the symbol ÞÑ
Ñ . At Ø

employ this matched filter to perform demodulation. A reasonable strategy is to sample the matched filter output at time Ø of the matched filter is

ÑÌ , the output

Ò Ñ

£ Ú´ÑÌ   ÒÌ µ · ÒÑ Ú ¾· Ò Ú £ Ú ´ÑÌ   ÒÌ µ · ÒÑ
ÒÚ Ò Ñ

(4.4)

where ÒÑ is a zero-mean Gaussian random variable with variance

Ƽ Ú ¾ ¾. The first term in (4.4)

is the desired signal contribution due to the symbol Ñ and the second term contains contributions from the other symbols. These unwanted contributions from other symbols are called intersymbol interference (ISI).

4.2

The effect of ISI and other noises can be observed on an oscilloscope displaying the output of the matched filter on the vertical input with horizontal sweep rate set at multiples of a display is called an eye diagram. and there is no ISI. Therefore. hence. ISI & Equalization 1. Unfortunately. Ú £ Ú ´Øµ are not timelimited and hence ISI is. Ú ´Øµ ¼ except for ¼ ¼ except for  Ì Ø Ì . rectangular pulse ÔÌ ´Øµ and binary signaling is employed. The effect of ISI is to cause a reduction in the eye opening by reducing the peak as well as causing ambiguity in the timing information.5 5 10 15 20 25 30 Figure 4. One common way to observe and measure (qualitatively) the effect of ISI is to look at the eye diagram of the received signal. Therefore.1: Eye diagram of a BPSK signal with no ISI Suppose Ú £ ڴص Ú ´Øµ were timelimited.5 0 −0. let us consider the basic waveform ٴص is the 4. for a bandlimited channel.5 1 0.e. in general.. ½ Ì . Ø As a result. respectively. the demodulation strategy above can be interpreted as matched filtering for each symbol. Then it is easy to see that ¼ for all Ò Ñ. a timelimited waveform is never bandlimited. Such For illustration.2. Ú ´Øµ and.Wong & Lok: Theory of Digital Communications 4. Ú £ Ú´ÑÌ   ÒÌ µ Ì . The eye diagrams for the cases where the channel is all-pass (no ISI) and lowpass (ISI present) are shown in Figures 4. i.3 .5 −1 −1. present.1 and 4.

we can set ½. first let ÜÆ ´Øµ ½ Ò  ½ Ü´ÒÌ µÆ ´Ø   ÒÌ µ 4. letting ܴص we can rewrite the decision statistic ÞÑ in (4. ISI & Equalization 1. It turns out that it is more illustrative to restate the Nyquist condition in frequency domain.5 0 −0. i. To do so.4) reviews that it is possible to have no ISI even if the Ú ´Øµ is bandlimited. without loss of generality.7) . The Nyquist condition in this form is not very helpful in the design of ISI-free pulses.4) as: Ú £Ú´Øµ.e.6) where is some constant and.5 1 0.4 (4.Wong & Lok: Theory of Digital Communications 4. ÞÑ Ñ Ü´¼µ · Ò Ñ Ò Ü´ÑÌ   ÒÌ µ · ÒÑ ¼ ¼ (4.2 Nyquist Pulses A careful observation on (4. the basic pulse shape ٴص and/or the channel is bandlimited.5) There is no ISI if the Nyquist condition is satisfied: Ü´ÒÌ µ ¼ Ò for Ò for (4.2: Eye diagram of a BPSK signal with ISI 4.. More precisely.5 5 10 15 20 25 30 Figure 4.5 −1 −1.

´ µ looks.Wong & Lok: Theory of Digital Communications 4.3.6) is equivalent to the condition ÜÆ ´Øµ Æ ´Øµ or Æ ´ µ ½ in frequency domain.e. i. copies of ´ µ can just touch Ò   Ì µ is flat if and only if Ï otherwise (4.. Now.5 Ø Ìµ (4. we get ½ Ò Ò  Ì (4.9) This is the equivalent Nyquist condition in frequency domain.10) ¯ ½ Suppose that the symbol rate is slower so that Ì È their neighbors. The Nyquist condition in (4. Æ´ µ ¾Ï : non-overlapping spectrum ½ ½ Ì Ò  ½ where ´ µ is the Fourier transform of ܴص. Then.11) . When the channel is bandlimited to Ï Hz. The folded spectrum ½  ½ ´ Ò ´ µ Ì for for ¼ ܴص The corresponding time domain function is the sinc pulse sinc´ 4.8)  ½ Ò  Ì Ì (4. the Nyquist condition cannot be satisfied and ISI is inevitable. the folded spectrum Ƚ  ½ ´   Ò There are gaps between copies of ´ µ.3: Case Ì Taking Fourier transform. the following implications: ´ µ ¼ for Ï . by employing (4. the Nyquist condition has ¯ ½ Suppose that the symbol rate is so high that Ì Ò Ì µ looks like the one in Figure 4.8). ISI & Equalization ½ Figure 4. No matter how ¾Ï . Then. ¾Ï . It says that the folded spectrum of ܴص has to be flat for not having ISI.

ܴص When « 2 « ¼.6 . the raised cosine spectrum gives a pulse that is much ´ µ × Ò´ Ø Ì µ Ó×´ «Ø Ì µ Ø Ì ½   « ¾ ؾ Ì ¾ (4. ܴص decays as ½ Ø¿ while for ¼.5): ´ µ where Ì Ì for ¼ ¼ « ¾ ½ · Ó× «Ì ´   ½ « µ ¾Ì ¼ for for ½ « ¾Ì ½·« ¾Ì ½ « ¾Ì ½·« ¾Ì (4. The Ì corresponding time domain function (Figure 4. is. Therefore. Hence. We note that for « ¼. the larger Ì has to be and the slower is the symbol rate. A truncated and delayed version is used as an approximation. the larger « 4.4. ܴص (sinc pulse) decays as ½ Ø. The folded spectrum of Ƚ ´ µ. Therefore. a widely used ISI-free spectrum is the raised-cosine spectrum (Figure 4. it is not physically ¾Ï above which ISI is unavoidable is known as the Nyquist rate. ½ realizable.6) is: ½ is called the roll-off factor.13) Another way to interpret this is that to fit into a channel bandlimited to Ï . When the symbol rate is below the Nyquist rate. An example is shown in Figure 4. Therefore. ISI & Equalization ½ Figure 4. The critical rate Ì ¯ ½ Suppose that the symbol rate is even slower so that Ì their neighbors.4: Case Ì ¾Ï : overlapping spectrum We note that the sinc pulse is not timelimited and is not causal. Then. we can design an ISI-free pulse shape Ò  ½ ¾Ï . it reduces to the sinc function. ½·« ¾Ì Ï .12) It determines the excess bandwidth2 beyond ¾½ .Wong & Lok: Theory of Digital Communications 4. copies of ´ µ overlap with Ò ´   Ì µ can be flat with many different choices which gives a flat folded spectrum.

4 −0.4 0.7 0.5 1 0 −1 −0.7 .5: Raised-cosine spectrum 1 0.6 0.4 −3 −2 −1 0 t/T 1 2 3 Figure 4.5 0.1 0.2 frequency/symbol rate 0.8 −0.4 0.9 0.6 −0.2 −0.2 0 1 0.6: Time domain function of the raised-cosine spectrum 4.6 0.8 0. ISI & Equalization Raised Cosine Spectrum 1 0.4 x(t) 0.8 1 Figure 4.2 α=0 0.5 α=0 −0.8 0.Wong & Lok: Theory of Digital Communications 4.3 0.2 0 0.6 X(f) 0.

δ µ ¾ À ´ µÍ ´ µ ¾ (4.14) to obtain the simple case of a ban- dlimited channel. we can employ (4.8 . Therefore. For a communication system system employing a linear modulation. Such a channel can be modeled by an LTI filter followed by an AWGN source as shown in Figure 4. respectively. we can simply also µ to be ´ µ.7: Dispersive channel model less sensitive to timing errors than the sinc pulse. Finally.Wong & Lok: Theory of Digital Communications 4. the resulting ISI-free pulse ٴص is called the square-root raised-cosine pulse. Then the Fourier transform the transfer function of the matched filter is ´ µ. such as BPSK. the whole system can be described the conceptual model 4. ´ µ where Í ´ µ. through a dispersive channel. and À ´ µ are the Fourier transform of ٴص. Just like all other bandlimited pulses..e. This is the dispersive channel model we describe before. truncation and delay is required for realization. and ´Øµ. recall that ܴص is the overall response of the transmitted pulse passing through the ban- dlimited channel and the receiving filter.14) Î ´ µ. Given that an ISI-free spectrum choose Í ´ Õ Õ ´ µ is chosen. Ú ´Øµ. ISI & Equalization HC ( f ) AWGN Figure 4. ܴص from the raised cosine spectrum is not timelimited. For example. Mathematically. such as telephone lines. if the raised-cosine spectrum is chosen. ISI is often introduced. not only are they bandlimited.7. i. suitable truncation and delay are required for physical realization. the channel does not introduce any distortion within its passband.3 Equalization For many physical channels. 4. but they also introduce distortions in their passbands. In general. Of course.

9. À ´ µ.15) is satisfied. Therefore. As for the previous method. 3 If the matched filter demodulation technique is employed.e. the dispersive channel ´ µ Á and ÀÌ ´ µ. we may choose to build a simpler (practical) filter ÀÊ ´ µ.e. we have to know the channel response À ´ µ in advance to construct the transmission and receiving filters. we should choose one that can be easily built. and choose the transmission filter so that (4.2 can be easily generalized to the above communication system. However.15). Ò  Ì Ì ÀÌ ´ µÀ ´ µÀÊ ´ µ. take samples at intervals Ì . and the receiving filter3 . and put a digital filter.. Letting condition for no ISI is that the folded spectrum ´ µ.9 . 4. is constant for all frequencies. An alternative method is to fix the transmission filter4 and choose the receiving filter shaping method is that it is in general difficult to construct the appropriate analog filters for ÀÌ ´ µ and ÀÊ ´ µ to satisfy the condition in (4. Moreover. the ½ Ò  ½ (4. This approach to remove ISI is usually known as equalization.. respectively. at the output to eliminate ISI analog filter as shown below in Figure 4.8. called equalizer. as well as to fit different channel conditions.15) One method to achieve the Nyquist condition is to fix the receiving filter to be the matched filter. set ÀÊ ´ µ £ ÀÌ ´ µÀ £ ´ µ. The main advantage of this approach is that a digital filter is easy to build and is easy to alter for different equalization schemes. i. notice that what we want eventually are the samples at intervals Ì at the receiver. 4 Of course. it is also difficult to build the appropriate ÀÊ ´ µ to eliminate ISI.2. ISI & Equalization Ik HT ( f ) HC ( f ) HR ( f ) Decision every T Impulses AWGN Figure 4. in which the sequence of information symbols is denoted by and ÀÊ ´ µ are the transfer functions of the transmission (pulse-shaping) filter. This is the Nyquist pulse design method described in Section 4.Wong & Lok: Theory of Digital Communications 4. The Nyquist condition for no ISI developed in Section 4. The major disadvantage of this pulse- ÀÊ ´ µ in practice.8: Continuous-time communication model over a dispersive channel in Figure 4. the receiving filter ÀÊ ´ µ is chosen to be the complex conjugate of ÀÌ ´ µÀ ´ µ. i.

in which the output sequence Á ¼ is given by Á¼ Á Á ¼· 4. The LTI filter. The following steps describe the translation process: ¯ Instead of considering AWGN being added before the receiving filter ÀÊ ´ µ.10 from the information source to the sampler as a digital filter.10.11. and get back a sample at the output of the sampler every Ì seconds. Since ÀÌ ´ µ. equivalent colored Gaussian noise being added after ÀÊ ´ µ due to AWGN. À ´ µ and ÀÊ ´ µ are LTI filters. we can represent the communication system in Figure 4.16) .3. The ¯ We input a bit or a symbol to the communication system every Ì seconds. The resulting model is shown in µ when we analyze the system.10: Equivalent communication system with colored Gaussian noise 4. Therefore.9 to an equivalent discrete-time model that is easier to work with. they can be combined and represented by an equivalent digital ½ .Wong & Lok: Theory of Digital Communications 4.10   ·Ò Á   ·Ò (4.1 Equivalent discrete-time model Our goal is to design the equalizer which can remove (or suppress) ISI. we can consider an equivalent colored noise is the output of ÀÊ ´ Figure 4.9: Communication system with equalizer Ik HT ( f ) HC ( f ) HR ( f ) Equalizer every T Decision Impulses colored Gaussian noise Figure 4. Denote its transfer function by À ´Þ µ and its impulse response by  ½ result is the discrete time-linear filter model shown in Figure 4. ISI & Equalization Ik HT ( f ) HC ( f ) HR ( f ) Equalizer every T Decision Impulses AWGN Figure 4. To do so. we translate the continuous-time communication system model in Figure 4.

12: Typical equalizer In general. The output sequence Á is given by Á Á   ·Ò Á ¼· Á   ·Ò (4. At the output of ÀÏ ´Þ µ. Notice that the noise sequence ¯ Usually. equivalently.11: Equivalent discrete-time communication system model with colored noise HW ( z ) EQZ Figure 4. the equalizer consists of two parts. ¯ Let ´Þµ À ´ÞµÀÏ ´Þµ. and can be determined a prior according to our choice of ÀÊ ´ µ.Wong & Lok: Theory of Digital Communications 4. Notice that ÀÏ ´Þ µ depends only on ÀÊ ´ µ. and Ò is an . the noise sequence is white.14. ISI is present. Therefore. we can consider the equivalent discrete-time model shown in Figure 4.12. in which Ò is an effect of AWGN sequence.11 ´Þµ. ISI & Equalization Ik H(z) Ik ’ Equalizer Decision nk Figure 4. The ÀÏ ´Þ µ is to “whiten” the noise sequence so that the noise samples are uncorrelated. The communication system from the information source to the output of the noise whitening filter can now be represented by the discrete-time white-noise linear filter model in Figure 4.17) where is the impulse response corresponding to the transfer function 4. Ò consists of samples of the colored Gaussian noise (AWGN filtered by ÀÊ ´ µ). and is not white in general.13. namely. a noise-whitening digital filter ÀÏ ´Þ µ and an equalizing circuit that equalizes the noise-whitened output as shown in Figure 4. Therefore. ¼ for some ¼.

We will work with this discrete-time model in all the following sections. ISI & Equalization Ik H(z) HW ( z ) ~ Ik EQZ Decision ~ nk Figure 4. This method is called zero-forcing equalization since the ISI component at the equalizer output is forced to zero. Then the output of the equalizer is given by Á Á  (4.14: Equivalent discrete-time white-noise linear filter model AWGN sequence.18) Ideally. This can be achieved by simply setting the transfer function À ´Þ µ ½ ´Þ µ.e. Á contains only contributions from the current symbol Á and the AWGN sequence with small variance.. The focus of our coming discussion is the design of this equalizer. the equalizing circuit (we simply call it the equalizer from now on) attempts to remove ISI from the output of ´Þµ. 4.2 Zero-forcing equalizer First. i. The simplest way to remove the ISI is to choose À ´Þ µ so that the output of the equalizer gives back the information sequence. Suppose that the equalizer is also an LTI filter with transfer function À ´Þ µ and corresponding impulse response . Á Á for all if noise is not present.Wong & Lok: Theory of Digital Communications 4. let us consider the use of a linear equalizer..3. ¯ Finally. i. we employ an LTI filter with transfer function À ´Þ µ as the equalizing circuit. 4.12 .13: Equivalent discrete-time communication system model with white noise Ik G(z) ~ Ik EQZ Decision ~ nk Figure 4.e.

the signal component at its output should be exactly Á .20) and the PSD of the colored Gaussian noise samples Ò in Figure 4. there may be a chance that the equalizing filter will greatly enhancing the noise power and hence the error performance of the resulting receiver will still be poor. let us evaluate the signal-to-noise ratio at the output of the zero-forcing equalizer when the transmission filter ÀÌ ´ fixed and the matched filter is used as the receiving filter. noise is always present.Wong & Lok: Theory of Digital Communications 4.11 is given by ¨Ò ¾ Ì ¬ Ƽ ½ ¬ ¬ ¾Ì Ò  ½ ¬ÀÌ (4. can be an infinite length sequence..21) Hence. the noise-whitening filter ÀÏ ´Þ µ can be chosen as ÀÏ ¾ Ì Õ À´ ¾ ½ ̵ (4.24) Since the zero-forcing filter simply inverts the effect of the channel on the original information symbols Á . If we model the Á as iid random variables with zero mean and unit variance.19) In this case. i. Suitable We note that the effect of the equalizing filter on the noise is neglected in the development of the zero-forcing equalizer above.14 is ¾ Ì Ò is simply Ƽ ¾.22) and then the PSD of the whitened-noise samples filter ´Þµ in Figure 4.e.23) Now. As a result. Although the ISI component is forced to zero. To see this. the overall digital ¾ Ì Õ À ¾ Ì À Ï À´ ¾ ̵ (4. then the PSD of the signal component is 4. µ is ÀÊ ´ µ £ ÀÌ ´ µÀ £ ´ µ (4. In reality. we choose the zero-forcing filter À ´Þµ as ´ ½ ¾ ̵ Õ À ¾ Ì À´ ¾ ½ ̵ (4. ISI & Equalization In general. it is easy to see that the digital filter À ´Þ µ is given by À ¾ Ì Ì ½ Ò ½  ½ ¬ ¬ ¬ ¬ ÀÌ Ò  Ì À Ò  Ì À Ò ¬¾   Ì ¬¬¬ Ò  Ì ¬¾ ¬ ¬ ¬ (4.13 ½ and hence the signal . the corresponding impulse response truncation and delay is applied to get an approximation.

Then MSE E E where Á Á     ÁÌ Ä  Ä Á  ¾ ½¾ ¿ (4.26) Without knowing the values of the information symbols Let us employ the FIR filter of order ¾Ä · ½ shown in Figure 4.3 MMSE equalizer The zero-forcing equalizer. It is based on the mean square error (MSE) criterion.28) (4. On the other hand. Defining the SNR as the ratio of the signal energy to the noise energy. we model each symbol Á as a random variable. If there is a certain region in the folded spectrum with very small magnitude. A different equalizer that takes noises into account is the minimum mean square error (MMSE) equalizer. We choose a linear equalizer À ´Þ µ to minimize the MSE between the original information symbols Á and the output of the equalizer Á : MSE E ¾ E ´Á   Á µ¾ (4.14 Á  Ä Ä Ì Ì (4. the PSD of the noise . Assume that the information sequence Á is WSS. Á beforehand. we have SNR Ƽ Ì ¾ ½ ¾Ì ½ ¾  ½ ¾Ì Ò  ½ ¬ ¬ ¬ ¬ ÀÌ Ò  Ì À Ò ¬¾  ½   Ì ¬¬¬  ½ (4.29) .3.27) Á Á ·Ä  Ä 4. 4. Hence the noise energy at the equalizer ¬ ½ Ì . although removes ISI. ¾¼ We note that a delay of Ä symbols is incurred at the output of the FIR filter. ISI & Equalization energy at the output of the equalizer is just  ½ ¾Ì component at the output of the equalizer is Ê ½ ¾Ì output is  ½ ¾Ì Ƽ ¾ ¬ ¬ ¬ À ¬ ¾ Ì ¬¾ ¬ ¬ Ƽ ¬ ¾ ¬ À ¬ ¾ Ì ¬¾ .Wong & Lok: Theory of Digital Communications Ê ½ ¾Ì 4.25) Notice that the SNR depends on the folded spectrum of the signal component at the input of the receiver.15 as the equalizer. may not give the best error performance for the communication system because it does not take into account noises in the system. then the SNR can be very poor.

z -1 h E.15   Ê µ.33) ÁÁ If Ê and are available. In our notation. First. we get  Ä ..Wong & Lok: Theory of Digital Communications 4. To decrease the MSE. we get E Á Rearranging. It can be shown that the signal-to-noise ratio at the output of the MMSE equalizer is better than that of the zero-forcing equalizer. the gradient is  ¾´ 4. Differentiating with respect to each Á   ÁÌ ¼ (4. then the MMSE equalizer can be found by solving the linear matrix equation (4. ISI & Equalization ~ I k z -1 z -1 .31) Ê E Á ÁÌ E (4. The linear MMSE equalizer can also be found iteratively. Ä .31). The gradient of the MSE with respect to gives the direction to change for the largest increase of the MSE. notice that the MSE is a quadratic function of .32) (4.L Σ ^ I k-L Figure 4. we .30) Ê where (4.-L hE..15: FIR filter as an MMSE equalizer We want to minimize MSE by suitable choices of and setting the result to zero. .

34) where is a small positive constant that controls the rate of convergence to the optimal solution. we can actually minimize the sum of the square errors. the transmitter can transmit In many applications. Suppose that the known sequence lasts for à symbols.37) This time.35) This is a stochastic steepest descent algorithm called the least mean square (LMS) algorithm. which is a statistical average.38) (4. the training sequence is known to the equalizer. However. The algorithm becomes: ´µ ´   ½µ · Á   ÁÌ ´   ½µ Á (4. the “data” sequence. respectively. in advance. This is the steepest descent algorithm: At the th step. Alternatively. With a training sequence. the vector ´ µ is updated as ´µ ´   ½µ ·   Ê ´   ½µ (4.36) Differentiating with respect to ´Ã µ and setting the result to zero. Then the sum of the square errors is given by à à ¾ à ½ ½ ´Á   Á µ¾ Á   ÁÌ ´Ã µ ¾ ´Ã µ à à (4.3. i. 4. we can replace Ê and at each step in the steepest descent algorithm by the rough estimates Á ÁÌ and Á Á .e. ISI & Equalization can update in the direction opposite to the gradient.Wong & Lok: Theory of Digital Communications 4.16 ÁÁ . Ê´Ã µ ½ ½ Á ÁÌ (4. Instead of minimizing the MSE.39) ´Ã µ 4..4 LS equalizer In the training period for the MMSE equalizer. we get Ê´Ã µ ´Ã µ (4. the receiver can estimate Ê and . we do not know Ê and a training sequence that is known a priori by the receiver. This is called the least squares (LS) criterion. with a training sequence.

. we need to have knowledge of   Þ µ ·   . Notice that Ê´Ã · ½µ Ê´Ã µ · Áà ·½ÁÌ ·½ à (4. Assuming Ê´Ã µ is non-singular. the RLS algorithm converges much faster than the steepest descent algorithm at the expense of more complex computation.3. However. ´ µ to be able to do so. This approach is called decision feedback equalization. In many cases. there is a more efficient approach. ´ Of course. it is possible to use previously decided symbols (output from the decision device) provided that we have made correct decisions on them. ISI & Equalization Suppose that we are given one more training symbol. i. actually.40) (4. However. 4. we get ´Ã µ · Áà ·½Áà ·½ Ê ½ ´Ã · ½µ Ê ½ ´Ã µ   ´Ã · ½µ Ê ½ ´Ã µÁà ·½ÁÌ ·½Ê ½ ´Ã µ Ã Ì Ê ½ ´Ã µÁ ½ · Áà ·½ à ·½ Ì ·½ ´Ã µ · Áà ·½   Áའ´Ã µ Ê ½´Ã µÁà ·½ ½ · ÁÌ ·½Ê ´Ã µÁà ·½ à (4.43) The procedure is called the recursive least squares (RLS) algorithm. we have to recalculate Ê´Ã and ´Ã ·½µ. If we had known the other symbols exactly. With decision feedback. ´Ã µ Ê ½ ´Ã µ ´Ã µ.42) (4. 4.17 .5 Decision feedback equalizer Recall from the equivalent discrete-time model in Figure 4.41) · ½µ ´Ã · ½µ Using the matrix inversion lemma5.Wong & Lok: Theory of Digital Communications 4.e. and solve the matrix equation all over again. we can think of the equalizer to contain two parts — a feedforward part and a  ½  ½  ½    ½  ½ ¡ ½  ½ 5 6 Assuming all the required invertibilities.45) In general. the equalizer would give Á Á   Á   (4. Apparently. we do not know all the symbols that are affecting the reception of the current symbol. an obvious approach to eliminate ISI would be to subtract their effects off6 .44) The current symbol we want to determine is Á .14 that Á Á ¼· Á   ·Ò (4.

Then Á ¼  Ä½ Á  · ľ ½ Á  (4. Again. significant improvement over linear equalizers can be obtained with the decision feedback equalizer.53) (4. The equations for optimal reduce to · E Á ÁÌ E Á ÁÌ · 4.56) ¼ . ISI & Equalization feedback part.47) where Á Á Á ·Ä½ Á ·Ä½  ½ Á  ½ Á  ¾  Ä½ ½ ¾  Ä½·½ Á Á  Ä¾ Ì (4.55) (4.Wong & Lok: Theory of Digital Communications 4. the identity matrix.52) Differentiating with respect to ´Á   ÁÌ Á ´Á   ÁÌ   ÁÌ   ÁÌ ¼ ¼ (4. We seek the filters and that minimize the MSE given by E ´Á   Á µ¾ and E Á E E ´Á   ÁÌ .18 EÁ Á (4.51) Further assume that the data symbols Á are zero-mean unit-variance iid random variables. i. Á Á . Suppose that the feedforward filter is of order Ľ · ½ and the feedback filter is of order ľ .46) can be found by minimizing the where Á are the decided symbols.. Consider a DFE with a feedforward filter of order Ľ ·½ and a feedback filter of order ľ .e.54) Notice that E Á Á and ¼ and E Á ÁÌ E Á ÁÌ Áľ ¢Ä¾ . i. In general. the filter coefficients MSE.e. Assume perfect decision feedback.49) ¼ ľ Ì Ì (4.50) (4. Then Á ÁÌ · ÁÌ Ì (4..48) (4. we get   ÁÌ µ µ µ¾ (4.

7 We assume the use of FIR filters for the MMSE. The resulting “optimal” receiver is referred to as the maximum likelihood sequence (MLS) receiver. Here.19 . and then make a decision on the current symbol based on the equalizer output. and decision-feedback equalizers. we can also solve for the feedforward and feedback filters using the steepest descent approach. it is intuitive that the optimal receiver should observe not only the segment of received signal concerning the desired symbol. Because of the fact that the effect of a symbol is spread to other symbols. we opt for another possibility. we are not strictly observing the whole received signal. In fact. Although this approach is simple and practical. this strategy is also employed in the equalization techniques described previously7. ISI & Equalization Solving these equations.58)  E Á ÁÌ Similar to the case of the MMSE equalizer. to minimize the average symbol error probability. which decides one transmitted symbol at a time. but some long segments of it. Instead of deciding a transmitted symbol at a time.Wong & Lok: Theory of Digital Communications 4. we can employ the ML principle to achieve our goal. we can send a training sequence to facilitate the estimation of them.57) (4. LS. In this way. but the whole received signal instead. If we do not know the expectations of the matrices above a priori. it turns out that all the equalization methods discussed in the previous section are not optimal. we aim at minimizing the probability of choosing the wrong sequence of symbols instead of the average symbol error probability. The development is rather involved and the optimal symbol-by-symbol detector is usually too complex to implement.4 Maximum Likelihood Sequence Receiver Whenever it is not practical to construct ISI-free transmission pulses. Using the whole received signal. 4. we can consider to decide the whole transmitted symbol sequence simultaneously from the received signal. we can employ the MAP principle to develop the optimal symbolby-symbol detector. In these cases. we can use an equalizer to eliminate (or reduce) ISI. With this sequence detection approach. 4. we have no idea whether it is optimal in terms of minimizing the average symbol error probability. As a matter of fact. we have E Á ÁÌ  EÁ  ½ ÁÌ E Á ÁÌ EÁ Á (4.

61) Defining ÝÒ Ö £ Ú ´ÒÌ µ and ÜÒ Ü´ÒÌ µ Ã Ò Ú £ Ú ´ÒÌ µ. 4. the MLS receiver has to check all the ¾Ã sequences to find the one 8 9 We cannot decide the future symbols until we have observed them. which is depicted in Figure 4. Ironically. The re- ´Ã · ½µÌ .16. there is no interference in this interpretation. This is the problem we have solved.5. we can employ the ML receiver to minimize the probability of making an error when determining the transmitted sequence.e. we can write the metric as ¼ Ò ÝÒ Ã´ õ ½  ¾ Ã Ò Ã ¼Ñ ¼ Ò Ñ ÜÒ Ñ (4. is ¼ Ò Ú ´Ø ִص Ã Ò   ÒÌ µ · Ҵص (4.3) with the above assumptions applied..3. we are equivalently considering an AWGN channel with Å Ã Ò Ò ¼ based on our observation ִص up to time ´Ã · Å -ary communication system over the ¾Ã exhausting all the possible transmitted bit sequences.62) Then the MLS receiver. As mentioned above. although our goal is to combat ISI. With this point of view. decides Ã Ö Ñ Ü Ã´ õ à (4. we try to decide the sequence of transmitted symbols ½µÌ 8. From the results in Section 2. i.59) as our desired signal9.63) In order to make the decision.60) up to time ´Ã · ½µÌ is Ã Ö Ö ¾ à ½ ÑÒ Ö´Øµ   Ø Ò Ú ´Ø   ÒÌ µ à  ½ Ò ¼ à ½ ½ ½ Ã Ú´Ø   ÒÌ µ ¾ Ø ÑÜ Ö´Øµ Ò Ú ´Ø   ÒÌ µ Ø   ¾  ½ Ò ¼ Ò Ã  ½ Ò ¼ (4.Wong & Lok: Theory of Digital Communications 4.59) This is simply a rewrite of (4. To do so. Each bit sequence is treated as an Å -ary “symbol”. Assuming all the bit sequences are equally probable. we treat the whole summation on the right hand side of (4. we know that the ML estimator of the bit sequence à à à  ½ ¼ (4.20 . ISI & Equalization Now. let us develop the MLS receiver. for Ø ½Ì Ò Ò ½ ¼ symbols per second and the transmission and channel filters are both causal. Suppose we transmit a sequence of information bits with rate ceived signal up to time ´Ã · ½µÌ .

From (4.67) Recall our goal is to maximize à ´ à µ.64) to simplify the maximization of the metric.64) We are going to make use of two important observations from (4.66) Ñ Ü Ã´ õ à 10 ÑÜ Ã  ½ ´×Ã Ñ ×Ü Ñà ´×à µ · Ô´Ýà à ×à àĠ½ à à à ×à µ (4. (4. However.21 . even when ÜÒ is not strictly of finite-support.64) can be rewritten as à  Ä ½ µ · Ô´Ýà à  ½ Ò Ã  Ä à ô õ à  ½ ´×à ¼ ×à µ ½ (4.e. When ÜÒ is of finite-support10. a significant simplification can be applied to the calculation of the maximum metric rendering the MLS receiver practical. Hence its complexity increases exponentially with time. This is clearly impractical and this is why the relatively simple equalization techniques described in the previous section are more popular in practice.66) where Ô´Ýà à ×à µ à Ýà   ½ Ò ÜÒ Ã   à ܼ ¾ à  Ä ½ µ · Ô´Ýà à (4.65) ×à à  Ä ½ . The first observation is that the updating part (the second term on the right hand side) depends only on ×à If we decompose à into à à à  ½ à  ¾ à  Ä (4. First. i.. the simplification here can still be applied approximately provided ÜÒ decays fast enough.Wong & Lok: Theory of Digital Communications 4. we can update à  ½ Ò Ã  Ä ô õ à  ½ ´ à  ½ µ · à Ýà   ½ Ò ÜÒ Ã   à ܼ ¾ ½ (4. 4.68) ×à µ This is generally not true for a channel with a bandlimited response unless some form of transmission pulse design is applied. ISI & Equalization @t=nT r(t) ~ v(t) Calculate metrics Select largest ^ bK Figure 4.16: Maximum likelihood sequence receiver that gives the largest metric. ÜÒ the metric in the following way: ¼ ¼ for Ò Ä.

ISI & Equalization where Ñà ´×à µ àĠ½ ÑÜ Ã  ½ ´×à à  Ä ½ µ à (4. all the receiver needs to know is the value of Ñà ´×à µ Ñà ´×à µ for each state ×à . The order of complexity of the maximization reduces dramatically from ¾Ã to ¾Ä . We need to calculate let each possible pattern of ×à  ½ system at the time instant Ø arrives at time Ø tant simplification in the maximization of à ´¡µ that we do not need to test all the ¾Ã patterns of à This contradicts with our original claim that à maximizes à ´¡µ. Hence. the receiver calculates à ´ à µ making use of (4.Wong & Lok: Theory of Digital Communications 4.66) that à ´ à µ Indeed. But àĠ½ àĠ¾ àĠ½ Ñ Ü Ã  ½´ àĠ½ ÑÜ Ã  ½ µ à  ¾ ´×à  ½ à  Ä ¾ µ · Ô´Ýà  ½ à  ½ à  ½ ×à  ½ µ ×à  ½µ (4. This observation reveals the impor- ¼ 11 à  Ä ½ µ. When the received sample Ýà is available at the output of the matched filter in Figure 4. this à  ¿ task can be efficiently accomplished by an iterative algorithm known as the Viterbi algorithm. which does not grow with time. where à à  Ä ½ must be the one that maximizes ¼ à  Ä ½ µ à ×à ¼ à  Ä ½ . we can use (4. the system changes state from ×à  ½ to ×à à  ½ à  ¾ à  Ä . There are altogether ¾Ä states. However. First.68). as long as we know the values of Ñà ´×à µ for all the ¾Ä patterns of ×à . × 4.16. suppose that there is another segment ¼Ã  Ä ½ such that à  ½ ´×à à ´ à µ.70) àĠ½ Ñ Ü Ñ Ü Ã  ¾´×à  ½ à  Ä ¾µ · Ô´Ýà  ½ àĠ¾ Ñ Ü Ñà  ½´×à  ½µ · Ô´Ýà  ½ à  ½ ×à  ½µ Ñà ´×à µ and the Therefore.22 . Ñà ´×à µ for each of the ¾Ä patterns of ×à .69) is the sequence that maximizes à ´¡µ.70) to update and store à  Ä ½ that maximizes it: à  Ä ½ ´×à µ 11 Ö Ñ Ü Ñà ´×à µ àĠ½ The important observation here is that Ô´Ýà à à µ does not depends on à  Ä ½ . When the new input bit à à  ¾ ÃÌ . Then it is clear from (4. then the segment à  ½ ´×à à  ½ ´×à The second equality above is due to our second important observation that if à ×à ¼ à  Ä ½ ¡µ. Fortunately. there is still a problem needed to be solved before we can benefit from this large complexity reduction. during the transition from state ×à  ½ to state ×à . ´Ã   à  Ä ½ represents a state of the communication ½µÌ .

71) with à  Ä ¾ ´×à  ½ µ as the sequence à  Ä ¾ that maximizes Ñà  ½ ´×à  ½ µ.17. then à  Ä ½ ´×à µ needed values of Ñà ´×à µ for the determination of à  ½ à  Ä ¾ ´×à  ½ µ à  Ä ¾ ´×à  ½ µ . à  Ä ½ ´×à µ .23 . We perform this update for all the ¾Ä states ×à at state transition to make the ½ ¼ Ö Ñ ×Ü Ñà ´×à µ · Ô´Ýà à à à ×à µ ÑÃÜ × Ã  Ä ½ ´×à µ (4.-1] [-1. Otherwise. Therefore.73) The update relations of the states are depicted by the trellis diagram.-1] SK-2 SK-1 SK bn = -1 bn = +1 Figure 4. à  Ä à  ¾ à  ¿ Ò ½ and ×¼¼  ½ à à  ½ à  ¾ à  ¿ à  Ä  ½ . 4. we know that the transition can only come from two previous states.17: Trellis diagram for the case of Ä ¾ à  Ä ¾ ´×à  ½ µ Ö Ñ Ü Ñà  ½´×à  ½µ · Ô´Ýà  ½ àĠ½ à  ½ ×à  ½ µ (4.+1] [+1. This update is usually performed in a “backward” manner. The other previous information can be discarded.72) If ׼à  ½ is the one chosen above.Wong & Lok: Theory of Digital Communications 4.+1] [-1. ISI & Equalization SK-3 [+1. For each state ×à namely ׼à  ½ à  ½ à  ¾ à  Ä . Ñà ´×à µ Ñ Ü Ñà  ½´×¼Ã  ½µ · Ô´Ýà  ½ ¼ Ó ×¼Ã  ½µ Ñà  ½ ´×¼¼  ½µ · Ô´Ýà  ½ à  ½ ×¼¼  ½µ à à (4. The trellis diagram for the case of Ä ¾ is given in Figure 4.

its symbol error performance is usually better than that of the equalization techniques. the Viterbi algorithm described above can be easily generalized to the cases of non-binary symbols. First. 4.Wong & Lok: Theory of Digital Communications 4.24 . ISI & Equalization Let us end our discussion on the MLS receiver by making two remarks. Second. although the MLS receiver minimizes only the sequence error probability.