Data Detection for Controlled ISI

*Symbol by symbol suboptimum detection

For the duobinary signal pulse

h(nT) = 1 for n=0,1 and zero otherwise.

The samples at the output of the receiving filter(demodulator) have the form

ym = Bm + vm = am + am-1 +vm

where { am} is the transmitted sequence of amplitudes and { v m } is a sequence of

additive gaussian noise samples.

If the noise contribution is ignored if am = 1 with equal probability then

Bm = { -2,0,2} with probab. 1/4,1/2,1/4.

If am-1 is the detected symbol from the (m-1)th signaling interval, its effect on B m
(received signal in the mth signaling interval) can be eliminated by subtraction, thus
allowing am to be detected.

* The major problem with this procedure is that errors arising from the additive noise
tend to propagate. For example if am-1 is in error its effect on Bm is not eliminated but, in
fact it is reinforced by the incorrect subtraction. Consequently the detection of Bm is also
likely to be in error.

*Error propagation can be avoided by precoding the data at the transmitter instead of
eliminating the controlled ISI by subtraction at the receiver. The precoding is performed
on the binary data sequence prior to modulation. From the data sequence {D n} of 1 s and
0 s that is to be transmitted, a new sequence {Pn} called the precoded sequence is
generated.

*For the duobinary signal, the precoded sequence is defined as

Pm = Dm  Pm-1 m=1,2,....

where  denotes modulo 2 addition. Then we set am = -1 if Pm = 0 and am=1 if Pm=1.

i.e. am = 2Pm - 1

Noise free samples at the output of the receiving filter are given by

90
 Bm = am + am-1
= (2Pm -1) + (2Pm-1 - 1)
= 2(Pm + Pm-1 -1)

Pm + Pm-1 = 1/2 Bm + 1

since Dm = Pm  Pm-1 it follows that

Dm = 1/2 Bm + 1 (mod 2)

Thus if Bm =  2 then Dm = 0 and if Bm = 0 then Dm = 1.

In the presence of additive noise the sampled outputs from the receiver are given by

ym = Bm + vm

This can be compared with two threshold set at +1 and -1. The data sequence {Dn} is
obtained according to the detection rule

1 (| ym | 1)
Dm = 
0 (| ym | 1)

E.g. Binary signaling with duobinary pulses ( no noise)

Sequences

Data Dm 1 0 1 1 0 1 0 0 1 0 1 1 1 0

Precoded Pm 0 1 1 0 1 1 0 0 0 1 1 0 1 0 0
= (Dm-Pm-1)mod 2

Transmitted am -1 1 1 -1 1 1 -1 -1 -1 1 1 -1 1 -1 -1=2Pm-1

Received Bm 0 2 0 0 2 0 -2 -2 0 2 0 0 0 -2=am+am-1

Decoded Dm 1 0 1 1 0 1 0 0 1 0 1 1 1 0 =(1/2* Bm+1)

mod 2

*The extension from binary PAM to multilevel PAM signaling using the duobinary
pulses is straightforward. In this case the M-level amplitude sequence {am} results in a
noise free sequence

Bm = am + am-1, m=1,2,....

91
which has 2M-1 possible equally spaced levels. The amplitude levels are determined
from the relation

am = 2Pm - (M-1)

where Pm = Dm  (M-1)Pm-1 (mod M)

the possible values of {Dm} are 0,1,2,....M-1.

In the absence of noise, the samples at the receiving filter may be expressed as

Bm = am + am-1 = 2[Pm + Pm-1- (M-1)]

Hence Pm + Pm-1 = 1/2 Bm + (M-1)

Since Dm = Pm + Pm-1 (mod M) it follows that

Dm = 1/2 Bm + (M-1) (mod M)

In the presence of noise, the received signal plus noise is quantized to the nearest of the
possible signal levels and the rule given above is used on the quantized values to recover
the data sequence.

E.g. Four level signaling with duobinary pulses ( no noise)

Sequences

Data Dm 1 0 2 3 0 2 1 0 3 1 1 2 1 0

Precoded Pm 0 1 3 3 0 0 2 3 1 2 3 2 0 1 3 =Dm-Pm-1mod 4

Transmitted am -3 -1 3 3 -3 -3 1 3 -1 1 3 1 -3 -1 3 =2Pm-3

Received Bm -4 2 6 0 -6 -2 4 2 0 4 4 -2 -4 2 =am-am-1

Decoded Dm 1 0 2 3 0 2 1 0 3 1 1 2 1 0 =1/2*Bm+3
mod 4

In the case of modified duobinary pulse , the controlled ISI is specified by the values
h(n/2w) = -1 for n=1, h(n/2W) = 1 for n=-1 and zero otherwise. Thus noise free sampled
output can be given as

Bm = am - am-2
Pm = Dm + Pm-2 (mod M)

92
It can be shown that

Dm = 1/2 Bm (mod M)

Therefore as shown above, the precoding of the data at the transmitter makes it possible
to detect the received data on a symbol by symbol basis without having to look at
previously detected symbols. Hence error propagation is avoided.

The symbol by symbol detection rule described above is not the optimum detection
scheme for PR signals due to the memory inherent in the received signal. Nevertheless,
symbol by symbol detection is easy to implement and is used in many practical
applications involving duobinary and modified duobinary pulse signals.

Signal Design for Channels with Distortion

* In the earlier analysis we assumed that the channel is ideal,i.e., |C(f)| = 1 for |f|  W

Here it is not ideal and the response C(f) is known.

The criterion is the maximization of the SNR at the output of the demodulation filter or
equivalently at the input to the detector.

For the signal component at the o/p of the demodulator we must satisfy

GT(f)C(f)GR(f) = Hd(f) e-j2fto , |f|  W

where Hd(f) is the desired frequency response of the cascade and t o is the necessary
time delay required to ensure the physical realizability of the modulation and
demodulation filters.

For combined frequency response of a raised cosine H rc(f) with zero ISI it can be shown
that when the additive noise at the input to the demodulator is WGN,

| Hrc ( f )|1/2
|GR(f)| = K1 , f W
|C( f )|1/2
| Hrc ( f )|1/2
|GT(f)| = K2 , f W K1,K2 are arbitrary scale factors.
1/2
|C( f )|

_and the phase characteristics satisfy T ( f )  c ( f )   R ( f )  2ft0

93
 Optimum Receiver for Channels with ISI and AWGN

* Objective is to derive the optimum demodulator and detector for digital transmission
through a non ideal bandlimited channel with additive Gaussian noise.

Maximum-Likelihood Sequence Estimation of Digital Sequences in the Presence of

Intersymbol Interference

Paper by G. David Forney , IEEE Trans. on Information Theory, pp. 363-378 Vol IT-18,
No.3, 1972

The simplest model if a digital communication system subject to ISI occurs in PAM
illustrated in Fig.1.

Noise n(t)

i/p x0, x1,x2.. Channel Signal s(t) r(t)=s(t)+n(t)

Filter +
h(t)

A sequence of real numbers xk drawn from a discrete alphabet passes through a linear
channel whose impulse response h(t) is longer than the symbol separation T, and the
filtered signal

s(t) =  xk h(t  kT )
k

is corrupted by white Gaussian noise n(t) to give a received signal

r(t) = s(t) + n(t)

Note: h(t) of finite duration (FIR) is considered.

Definitions:

We use the PAM model of Fig.1 The inputs x k are assumed to be equally spaced and
scaled and biased to be integers in the range 0  xk  m-1; they are assumed to start at
time k=0 and to continue until k=K, where K may be infinite.

94
With the input sequence we associate the formal power series in the delay operator D (
D-transform D  Z-1)

x(D)  x0 + x1D + x2D2 + ...... (1)

which will be referred to as the input sequence.

The channel is characterized by a finite impulse response h(t) of L symbol intervals;i.e.,

L is the smallest integer such that h(t) = 0 for t > LT.

The response h(t) is assumed to be square-integrable,


h
2
||h|| 
2
(t )dt   (2)

We define
h(k)(t)  h(t - kT) (3)

and use the inner-product notation in which


[a(t),b(t)]   a (t )b(t )dt (4)

Then the pulse autocorrelation coefficients of h(t) are

Rk-k’  [h(k)(t) , h(k’)(t) ]


=
 h(t  kT )h(t  k ' T )dt | k  k '| L  1
(5)


0 | k  k '| L

We define the pulse autocorrelation function of h(t) as


R(D)   Rk D k (6)
k 

where L - 1 is called the span of h(t).

The response h(t) may be regarded as a sequence of L chips hi(t) , 0  i  where hi(t) is
a function that is nonzero only over the interval [0,T); i.e.,

h(t) = h0(t) + h1(t - T) + .......+ h(t - T) (7)

95
Then it is natural to associate with h(t) the chip D-transform

h(D,t)   hi (t ) Di (8)
i 0
which is a polynomial in D of degree  with coefficients in the set of functions over
[0,T).

It is can be verified that (show this!!) the chip D-transform has the following properties.

1) The pulse autocorrelation function is given by

T
R(D) = [h(D,t) ,h(D ,t)] =  h( D, t )h( D 1 , t )dt
-1
(9)
0

2) A transversal filter is a filter with a response

g(t) =  gi (t  iT ) (10)

i
foe some set of coefficients gi. This response is not square integrable, but we assume that
the coefficients gi are square summable,  gi2   . We say a transversal filter g(t) is
i
characterized by g(D) if

g(D) =  gi Di (11)
i

The chip D-transform g(D,t) of a transversal filter is

g(D,t) = g(D)(t) (12)

3) The cascade of a square-integrable filter with response g(t) and a transversal filter
characterized by a square summable f(D) is a filter with square integrable response h(t),
chip D-transform

h(D,t) = f(D)g(D,t) (13)

and pulse autocorrelation function

Rhh(D) = f(D)f(D-1)Rgg(D) (14)

The signal s(t) is defined as

96
 K K
s(t)   xk h(t  kT )   x k h ( k ) (t ) (15)
k 0 k 0

The received signal r(t) is s(t) plus white Gaussian noise n(t).

* It is well known that in the detection of signals that are linear combinations of
some finite set of square-integrable basis functions h(k)(t), the output of a bank of
matched filters, one matched to each basis function, form a set of sufficient statistics
for estimating the coefficients.

Thus the K+1 quantities

ak = [ r(t) , h(k)(t) ]


=  r (t )h(t  kT )dt 0 k  K (16)


form a set of sufficient statistics for estimation of the x k, 0  k  K, when K is finite. But
these are simply the sampled outputs of a filter h(-t) matched to h(t).

Thus the following proposition can be stated.

Proposition1: When x(D) is finite, the sampled outputs a k [defined by (16)] of a filter
matched to h(t) form a set of sufficient statistics for estimation of the input sequence
x(D).

Note: This property does not depend on x(D) being finite.

Since the matched filter is linear and its sampled outputs can be used without loss of
optimality, any optimal linear receiver must be expressible as a linear combination of the
sampled matched filter outputs ak.

Corollary: For any criterion of optimality, the optimum linear receiver is expressible as
the cascade of a matched filter and a (possibly time-varying) transversal filter.

The whitened Matched Filter (WMF)

Define the matched-filter output sequence as

97
 K
a(D) =  ak D k (17)
k 0
Since

ak = [ r(t), h(k)(t) ]

=  xk ' [h ( k ') (t ), h ( k ) (t )]  [n(t ), h ( k ) (t )]

k'

=  xk ' Rk  k '  nk' (18)

k'
we have

a(D) = x(D)R(D) + n’(D) (19)

Here n’(D) is zero-mean colored Gaussian noise with autocorrelation function 2R(D),
since

nk' nk' '   dtd n(t )n( )h(t  kT )h(  k ' T ) =  Rk-k’
2
(20)

where 2 is the spectral density of noise n(t), so that E[n(t)n()] = 2t-).

Since R(D) is finite with 2 + 1 nonzero terms, it has 2 complex roots; further since
R(D) = R(D-1), the inverse -1 of any root  is also a root of R(D), so the roots break up
into  pairs. Then if f’(D) is any polynomial of degree  whose roots consist of one root
from each pair of roots of R(D), R(D) has the spectral factorization

R(D) = f’(D)f’(D-1) (21)

We can generalize (21) slightly by letting f(D) = Dnf’(D) for any integer delay n; then

R(D) = f(D)f(D-1) (22)

Now let n’(D) be zero mean white Gaussian noise with autocorrelation function 2; we
can represent the colored noise n’(D) by

n’(D) = n(D)f(D-1) (23)

since n’(D) then has the autocorrelation function 2f(D-1)f(D) =2R(D) and zero mean
Gaussian noise is entirely specified by its autocorrelation function. Consequently we may
write (19) as

98
 a(D) = x(D)f(D)f(D-1) + n(D)f(D-1) (24)

This suggests that we simply divide out the factor f(D-1) formally to obtain a sequence

z(D) = a(D)/f(D-1) = x(D)f(D) + n(D) (25)

in which the noise is white.

When f(D-1) has no roots on or inside the unit circle, the transversal filter characterized
by 1/f(D-1) is actually realizable in the sense that its coefficients are square summable.

Then the sequence z(D) of (25) can actually be obtained by sampling the outputs of the
cascade of a matched filter h(-t) with a transversal filter characterized by 1/(f(D-1) (with
whatever delay is required to assure causality).

* We call such a cascade a whitened matched filter.

Thus if the a filter w(t) represented by the chip D-transform

1
w(D,t)  h ( D, t ) (26)
f ( D)

its time reversal w(-t) can be used as a WMF in the sense that its sampled outputs


zk =  r (t ) w(t  kT )dt (27)


satisfy (25) with n(D) a white Gaussian noise sequence. It can be shown that the set of
functions w(t - kT) is orthonormal. [ Rww(D) = 1]

Finally the set of functions is a basis for the signal space since the signals s(t) have chip
D-transforms

s(D,t) = x(D)h(D,t)
= x(D)f(D)w(D,t) (28)

so that s(t) =  yk w(t  kT ) (29)

k

where the signal sequence y(D) is defined as

y(D)  x(D)f(D) (30)

99
We note that only K + + 1 of the yk are nonzero.

Theorem:

Let h(t) be finite with span and let f(D)f(D-1) be any spectral factorization of
Rhh(D). Then the filter whose chip D transform is w(D,t) = h(D,t)/f(D) has square
integrable impulse response w(t)and the sampled outputs zk of its time reverse


zk =  r (t ) w(t  kT )dt (31)

form a sequence

z(D) = x(D)f(D) + n(D) (32)

in which n(D) is a white Gaussian noise sequence with variance 2 and which is a set
of sufficient statistics for estimation of the input sequence x(D).

Discrete-Time Model

x(D)
D D D

f0 f1 f2 fv

y(D)

The signal sequence

y(D) = x(D)f(D) (33)

100
is the convolution of the input sequence x(D) with the finite impulse response f(D),
whose autocorrelation function is R(D) = f(D)f(D-1). Without loss of generality we
assume that f(D) is a polynomial of degree  with f0  0. The received sequence z(D) is
the sum of the signal sequence y(D) and a white Gaussian noise sequence n(D) with
autocorrelation function 2.

The output signal to noise ratio is defined to be

SNR  y2 / 2 = x2 ||f||2 /2 (34)

where x2 is the input variance [(m2 -1) /12] and


||f|| 
2
 f i2 = R0 (35)
i 0

is the energy in the impulse response f(D). (If f(D) is derived from a continuous time
response h(t), then ||f||2 = ||h||2 = R0.)

* It is crucial to observe that the signal sequence y(D) may be taken to be generated by a
finite state machine driven by an input sequence x(D). We may imagine a shift register of
 m-state memory elements containing the  most recent inputs, with yk formed as a
weighted sum of the shift register contents and the current input xk as shown in the
Figure.

Clearly the machine has m states, the state at any time being given by the  most recent
inputs:

sk  (xk-1,xk-2,.....,xk-) (36)

where by convention xk =0 for k < 0.

We define the state sequence s(D) as

s(D)  s0 + s1D + s2D2 + ..... (37)

where each state sk takes on values from an alphabet of m states Sj, 1  j  m

The maps from input sequences x(D) to state sequences s(D) and thence to signal
sequences y(D) are obviously one to one and hence invertible. In fact two successive
states uniquely determine an output

yk = y(sk , sk+1) (38)

101
i.e., given a transition from sk to sk+1, the corresponding output yk is determined.

An allowable state sequence s(D) or signal sequence y(D) is defined as one that could
result from an allowable input sequence.

Maximum-Likelihood Sequence Estimation (MLSE)

Maximum-likelihood sequence estimation is defined as the choice of that x(D) for which
the probability density p[z(D)|x(D)] is maximum.

Since we have permitted sequences to be semi-infinite, so that p[z(D)|x(D)] may be zero

for all x(D), some sort of limiting operation is implied.

As the maps from x(D) to s(D) and to y(D) are one to one, MLSE can equivalently be
defined as choosing from the allowable s(D) that which maximizes p[z(D)|s(D)] or from
the allowable y(D) that which maximizes p[z(D)|y(D)].

Here we consider the problem of estimating the state sequence of a finite state machine
from noisy observations.

To construct the recursive estimation algorithm known as the “Viterbi Algorithm”, we

first use the fact that the noise terms nk are independent.

Then the “log likelihood ratio” ln p[z(D)|s(D)] breaks up into a sum of independent
increments:

ln p[z(D)|s(D)] =  ln pn [ zk  y ( sk , sk 1 )] (39)
k

where pn(.) is the probability density function (pdf) of each noise term n k. For notational
convenience we define the partial sums

k 2 1
[s(D)]k1 k2
  ln pn [ zk  y ( sk , sk 1 )] , 0  k1 < k2 (40)
k  k1

Suppose for the moment that we knew that the state sk at time k was Sj. Then for any
allowable state sequence s(D) that starts with the known initial state s 0 = 0 and passes
through the state Sj at time k, the log likelihood would break up into two independent
parts:

[s(D)]0K = [s(D)]0k + [s(D)]kK (41)

102
Let sj^(D) be the allowable state sequence from time 0 to k that has maximum log
likelihood [s(D)]0k among all allowable state sequences starting with s0 = 0 and ending
with sk = Sj.

We call sj^(D) the survivor at time k corresponding to state Sj. Then we assert that sj^(D)
must be the initial segment of the ML state sequence s(D); for we can replace the initial
segment s’(D) of any allowable state sequence with greater log likelihood [s(D)]0K
unless

[s’(D)]0k = [s(D)]0k.

In fact we do not know the state sk at time k; but we do know that it must be one of the
finite number of states Sj, 1  j  mof the shift register of the Figure.

Consequently while we cannot make a final decision as to the identity of the initial
segment of the ML state sequence at time k, we know that the initial segment must be
among the m survivors sj^(D) ,1  j  mone for each stateSj.
Thus we need only store m sequences sj^(D) and their log likelihoods [s(D)]0k,
regardless of how large k becomes.
* To update the memory at time k+1 recursion proceeds as follows.

1) For each of the m allowable continuations sj’(D) to time k+1 of each of the m
survivors sj^(D) at time compute

[sj’(D)]0k+1 = [sj^(D)]0k + ln pn[zk - yk(Sj,Sj’)] (42)

This involves m+1 = mL additions.

2) For each of the states Sj, 1  j’  m compare the log likelihoods [sj’(D)]0k+1 of the m
continuations terminating in that state and select the largest as the corresponding
survivor. This involves m m-ary comparisons, or (m-1) m binary comparisons.

The search can be easily understood by the trellis diagram of the finite state
machine.
In principle the Viterbi algorithm can make a final decision on the initial state segment
up to time k- when and only when all survivors at time k have the same initial state
sequence segment upto time k-. The decoding delay  is unbounded but is generally
finite with probability 1.

In implementation one actually makes a final decision after some fixed delay , with 
chosen large enough that the degradation due to premature decisions is negligible
(approx. 5).

103