4.

Nyquist
4. Nyquist Criterion
Criterion for
for Distortionless
Distortionless
Baseband Binary
Baseband Binary Transmission
Transmission

Objective: To design hT (t ) and hd (t ) under the

following two conditions:

(a). There is no ISI at the sampling instants (Nyquist

criterion, this section ).

(b). A controlled amount of ISI is allowed (correlative

coding, next section)

@G. Gong 1
 Design of Bandlimited Signals for Zero
ISI - Nyquist criterion
Recall the output of the receiving filter, sampled at t = kT,
is given by
y (kT ) = bk + bn p (kT nT ) + no (kT )
n k

Thus, in time domain, a sufficient condition for p(t) such

that it is ISI free is
1 n =1
p ( nT ) = (1)
0 n 0

Question. What is the condition for P(f) in order for p(t) to

satisfy (1) (Nyquist, 1928)?

@G. Gong 2
Theorem. (Nyquist) A necessary and and sufficient condition for
p(t) to satisfy (1) is that the Fourier transform P(f) satisfies

n
P( f ) =T (2)
n
T

This is known as the Nyquist pulse-shaping criterion or

Nyquist condition for zero ISI.
Proof.

@G. Gong 3
Proof. When we sample p(t ) at On the other hand
t = kT , k = 0, 1, 2, L , P (f)= p (kT ) F ( (t kT ))
we have the following pulses k
p (t ) p (t ) (t kT ) ( p (kT ) is constant for t.)
k

= p (kT ) (t kT ) = p (kT ) exp( j 2 fkT )

k k

The Fourier transform of p (t ) = 1 ( from (1) ) ( 4)

is given by
From (3) and (4), ISI free
P ( f ) = F ( p (t ))
1 k
=F p (kT ) (t kT ) P( f ) =1
T k
T
k

1 k which gives the result in (2).

= P( f ) ( 3)
T k
T
 Investigate possible pulses which satisfy the
Nyquist criterion

Suppose that the channel has a bandwidth of W, then

H c ( f ) = 0 for | f |> W
Since P( f ) = H T ( f ) H c ( f ) H d ( f ) , we have

P( f ) = 0 for | f |> W

We write Z ( f ) = P( f n /T)
n

and distinguish the following three cases:

@G. Gong 5
 Z(f)

-1/T-W -1/T -1/T+W -W W 1/T-W 1/T 1/T+W f

Fig. 4.1 Z(f) for the case T < 1/(2W)
Z(f)

1
-1/T W= 1/T f
2T

Fig. 4.2 Z(f) for the case T = 1/(2W)

Z(f)

-1/T -W 1/T-W W 1/T f

-1/T+W
Fig. 4.3 Z(f) for the case T > 1/(2W)
 1 1
1. T< , or > 2W
(i.e., bit rate > 2W, impossible!) No
2W T
choices for P(f) such that Z(f) = 0.

1 1
2. T= , i.e., W= (the Nyquist rate)
2W 2T
In this case, if we choose
T | f| W f
P( f ) = i.e., P ( f ) = T rect
0 otherwise 2W
t
which results in p (t ) = sin c
T
This means that the smallest value of T for which the
transmission with zero ISI is possible is
1 1 This is called the ideal
T= ( R = 2W, bit rate )
2W T Nyquist channel.
 In other words,

Ideal Nyquist channel :

1
R = 2 Bo =
T
W = Bo ( R = Rb , T = Tb )

@G. Gong 8
 Disadvantages:

(a) an ideal LPF is not physically realizable.

(a) Note that

t 1
p (t ) = sin c
T |t |

Thus, the rate of convergence to zero is slow since the

tails of p(t) decay as 1/|t|.

Hence, a small mistiming error in sampling the output

of the matched filter at the demodulator results in an
infinite series of ISI components.

@G. Gong 9
 1 1
T> < 2W
3. For , i.e,
2W T, in this case, there exists
numerous choices for P(f) such that Z(f) = T. The
important one is so called the raised cosine spectrum.

The raised cosine frequency characteristic is given by

1
0 f < (1 " ) B0
2 B0
1 ! (| f | (1 " ) B0 )
P( f ) = 1 + cos (1 " ) B0 f < (1 + " ) B0
4 B0 2"B0
0 f (1 + " ) B0

R
where " # [0,1] is called the rolloff factor and B0 =
2
( i.e., B0 = 1 ) .
2T

@G. Gong 10
 1
2 B0

B0 B0 f
(1 + " ) B0 (1 " ) B0 (1 " ) B0 (1 + " ) B0

Z(f) = T by the following sum of three terms at any

interval of length 2Bo:

P( f ) + P( f 2 B0 ) + P( f + 2 B0 ) = T B0 f B0

P( f ) + P( f 2 B0 ) + P( f + 2 B0 ) = T B0 f 3B0

 The time response p(t), the inverse Fourier transform of
P(f), is given by

cos2 "B0t
p(t ) = sinc2 B0t (5)
1 16" B0 t
2 2 2

This function has much better convergence property than

the ideal Nyquist channel. The first factor in (5) is
associated with the ideal filter, and the second factor that
decreases as 1/|t|2 for large |t|. Thus

1
p (t )
t3

@G. Gong 12
@G. Gong 13
 Summary:

Nyquist Criteria
+$
1
P( f nR) = T , R =
n= $ T

-Bo Bo

Ideal Nyquist Channel

Raised Cosine Spectrum

@G. Gong 14
Example 1
The Fourier transform P(f) of the basis pulse p(t) employed in a
certain binary communication system is given by

6 f 6
10 1 if 10 f ( Hz ) 10 6
P( f ) = 10 6
0 otherwise

1. From the shape of P(f), explain whether this pulse satisfies the
Nyquist criterion for ISI free transmission.
2. Determine p(t) and verify your result in part 1.
3. If the pulse does satisfy the Nyquist criterion. What is the
transmission rate ( in bits/sec.) and what is the roll-off factor?

@G. Gong 15
Solution: 1. The Nyquist criterion is
$
Z( f ) = P( f nR ) = T , R = 1 / T
n= $

If we choose R = 10 6 Hz , then p(t) satisfies Nyquist criterion

for ISI free transmission, shown as Figure 2.
Z( f ) = T
P(f)
6 6
10 10

6 6 f (Hz)
10 0 10 106 10 6
= Rb = Rb
Figure 1
Figure 2

@G. Gong 16
2. We have p ( t ) = F 1 [ P ( f ) ] = sinc 2 (10 6 t )
Consequently, p(0) = 1, and p(nT ) = 0 where n = 1, 2, ....

Therefore, if the received signal is sampled

at t = 0, T , 2T ,..., the pulses p (t nT ) ( n = 0, 1, 2, ....)
will not interfere with each other, i.e. , ISI - free transmission.

p ( t ) = sinc 2 (10 6 t )

t
( s)
-3 -2 -1 0 3 T
1 2

@G. Gong 17
3. The relationship between the bandwidth and the roll-off factor is

1 R
W = (1 + " ) B0 f1 = (1 " ) B0 B0 = =
2T 2
where the transmission rate R = 106 (bits/s).
In this case, we have, f1 = 0 and B0 = 0.5 R = 0.5 10 6 Hz % " = 1

@G. Gong 18
 5. Correlative
5. Correlative Coding
Coding and
and Equalization
Equalization

A. Correlative Coding

For zero ISI, the symbol rate R = 1/T < 2W, the Nyquist rate.
We may relax the condition of zero ISI in order to achieve
R = 2W.

The schemes which allow a controlled amount of ISI to achieve the

symbol rate 2W are called correlative coding or partial response
signaling schemes.

@G. Gong 19
 The condition for zero ISI is
1, n=0
p1 (nT ) = (1)
0, n 0

Suppose that we design the band - limited signal p (t ) to

have controlled ISI at one time instant, i.e., to allow one
additional nonzero value in the samples {p ( nT )}
for example,
1, n = 0 and n = 1
p 2 ( nT ) = (2)
0, otherwise

@G. Gong 20
 p2 (t ) has a larger time duration t han p1 (t );
% P2 ( f ) = F [ p2 (t )] has a smaller bandwidth
on frequency domain than P1 ( f ) = F [ p1 (t )] ;
% Spectral efficiency is increased by using p2 (t ).

Note. The ISI we introdece by using p2(t) is deterministic or

controlled and, hence, its effect on signal detection at the
receiver can be removed, as discussed below.

@G. Gong 21
 (A) Duobinary Signaling (Class I partial response)

The prefix duo implies doubling of the transmission capacity

of a straight binary system. Figure 1 shows a duobinary
encoder.

1/T
{an }
{yn }
{y n } {b }
n
{a n }
{bn } Ideal channel duobinary Post-
pre-coder + HNyquist( f ) decoder coder
overall channel
Delay
T

Duobinary encoder

Figure 1. Block diagram of duobinary encoder

 Legend in Figure 1:
1). {an }, an # {0,1} with an and an+k being independent for k 0.
The sample(bit or symbol) duration is T .
2) {bn }, the NRZ polar, the output of the memoryless precoder
(or level converter)
+d if an = 1
bn =
d if an = 0.
3) The frequency response of the duobinary encode is given by
H I ( f ) = H1 ( f ) H Nyquist ( f )
which is a cascaded two filters, since we have
+ (t) h1(t) = (t) + (t T )
= H1( f )
Delay
T

Figure 2. Transfer function of delay operator

 4) The filter output y n can be represente d by the sum of
the present input pulse bn and its previous value bn 1 ,
y n = bn + bn 1 (3)

Note that here we consider noiseless channel and = 1

which is from the following formula:
y k = Y ( kT ) = bk + bn p (t nT ) + no ( kT ) (3)-a
n k

Notice that
2d if a n = a n 1 = 1
y n = bn + bn 1 = 0 if a n a n 1 (4)
2d if a n = a n 1 = 0

The effect of (3): {bn } , two level {d, -d}, uncorrelated,

% { y n } three level {-2d, 0, 2d}, correlated.
@G. Gong 24
4) On the frequency domain, the transfer function of the
duobinary encoder is:

H I ( f ) = H1 ( f ) H Nyquist ( f ) = H Nyquist ( f ) F[ (t ) + (t T )]
= H Nyquist ( f )[1 + exp( j 2 fT )]
= H Nyquist ( f ) exp( j fT )[exp( j fT ) + exp( j fT )]
= 2 H Nyquist ( f ) exp( j fT ) cos( fT )
Since an ideal Nyquist channel of bandwidth B0 = 1 / 2T has
transfer function
1, if f 1 / 2T
H Nyquist ( f ) =
0, if f > 1 / 2T
Then 2 cos( fT ) exp( j fT ) if f 1 / 2T
HI ( f ) =
0, if f > 1 / 2T .

@G. Gong 25
 HI ( f )
1
2.0 R= = bit rate
T

arg [ H I ( f )]
R/2 0 R/2 f
2
f
R/2 0 R/2

2
Figure 3. The frequency response of the duobinary encoder

@G. Gong 26
5) The impulse response corresponding to H I ( f ) consists of two
sinc (Nyquist) pulses that are time-displaced by T seconds with
respect to each other, which can be derived as follows.

hI (t ) = F 1[ H I (t ) ] = F 1[ H Nyquist (t ) ] ) F 1[ H1 (t ) ]
1 t
= sinc )[ (t ) + (t T )]
T T
1! t t T
= sinc + sinc
T T T

1 ! sin( t / T ) sin( (t T ) / T )
= +
T t /T (t T ) / T

sin( t / T ) sin( t / T ) T sin( t / T )

= =
t (t T ) t (T t )
 2
Note : The tails of hI (t ) decay as 1 / t , which is faster rate of decay tha n the
is 1 / t encountere d in the ideal Nyquist cahnnel. Therefore, the ISI due to
bit synchroniz ation error is reduced by the duobinary signaling.

hI (t )

1.0

-2T -T 0 T 2T 3T 4T t

Figure 4.

@G. Gong 28
(B). Decoding of the Duobinary Signaling
The original two - level sequence {bn } may be detected
from the duobinary- coded sequence {yn } using a feedback
decoder based on Eq. (3) . Specifically, let bn represent
the estimate of the original pulse bn as received by
the receiver at time t = nT . Then, subtracting the previous
estimate b from y , we get
n 1 n

bn = yn bn 1

yn + bn
+ Drawback: error
- Delay propagation
T

Decision feedback Figure 5

@G. Gong 29
Duobinary Scheme with Precoder
To uniquely determine the source bit in the kth signaling
interval, even if an error is made on the (k-1)th bit, the kth
source bit, we introduce the precoding:

bn* = an * bn* 1 (5) where * is modulo 2 operation.

{bn }
{an } {b }
*
n
1/T {y n }
Level Duobinary
* converter encoder

Delay
T

Precoder

Figure 6
 { }
The precoded sequence bn* is given by
bn* 1 , if an = 0
b =
*
n
bn* 1 , if an = 1. (6)
where the bar represents the complement of the symbol.
{}
The binary sequence bn* is applied to a level converter, producing
a corresponding two - level sequence {bn }, where bn = d with sample
duration T as before. Then
yn = bn + bn 1 (7)

From (4), we have Combining with (6),

2d if b * n = b * n 1 = 1
0 if an = 1
yn = 0 *
if b n b n 1 *
yn = (8)
2d if an = 0
2d if b * n = b * n 1 = 0

@G. Gong 31
 From (8) we deduce the following
decision rule for detecting the original binary
sequence {a n } from {y n }:
if y n < d , then symbol a n is 1
(A)
if y n > d , then symbol a n is 0
if y n = d , randomly guessing a n

@G. Gong 32
A useful feature of this detector is that no knowledge
of any input sample other than the present one
is required. Hence, error propagation cannot occur in the
detector.

{y n } { y } {a n }
n
Decision
Rectifier
device

threshold d

Figure 7

@G. Gong 33
 Summary: Correlative coding can achieve a
transmission rate of 2W symbols per second
by using the duobinary scheme together with
the precoding.

{an } {b }
*
{yn }
{y n } {b }
n
{a n }
n
LC
{bn } Ideal channel duobinary Post-
+ + HNyquist( f ) decoder coder
D D overall channel

Duobinary Scheme with Precoding

@G. Gong 34
 Example. Precoding with memory and duobinary coding.

Consider the binary data sequence 0010110. To proceed with the

precoding of this sequence, which involves feeding the precoder
output back to the input, we add and extra (initialization) bit to the
precoding output. This extra bit is chosen arbitrarily to be 1. Hence,
using (4), we find that the sequence {b*n} at the precoder output is as
shown in row 2 of the following table. The polar formart {bn} of the
sequence {b*n} is shown in row 3 of the table. Finally, using (7), we
find the duobinary encoder output has the amplitude levels given in
row 4 of the table. To detect the original binary sequence, we apply
the decision rule, given by (A), so, obtain the binary sequence given
in row 5 in the table.

The last row shows that, in absence of noise, the original binary
sequence is detected correctly.

@G. Gong 35
Example 1. Duobinary coding with precoding.
Binary
Sequence
{an } 0 0 1 0 1 1 0
Precoded
sequence 1 1 1 0 0 1 0 0
{b }
*
n

Two-level
sequence +d +d +d -d -d +d -d -d
{bn }
Duobinary
Encoder
output +2d +2d 0 -2d 0 0 -2d
{yn }
Detected
binary
sequence 0 0 1 0 1 1 0
{a n }

@G. Gong 36
 Example 2. Find the error probability of the duobinary signaling in
AWGN where the symbols are equally likely.

Solution. 0 1 0

From (3)-a and (8), we have

-2d -d 0 d 2d
no (kT ) if ak = 1
yk =
2d + no (kT ) if ak = 0
We write N = no(kT).
Since for {ak}, 0 and 1 are equally Consequently,
likely, the output levels 2d each N < d or N > d if ak = 0
occur with and the output level 0
occurs with prob. assuming no Thus, error occurs when
noise. If the thresholds are set at d,
errors occurs as follows: d < yk < d if ak = 0
If ak = 0, then yk < d or yk > d if ak = 1
(i) 2d + no (kT ) < d or no (kT ) < d
(ii) 2d + no (kT ) > d or no (kT ) > d
@G. Gong 37
 1 1
P(e) = P{ d < yk < d | 0} + [ P{ yk < d | 1} + P{ yk > d | 1}]
2 4
1 1
= [ P{N > d} + P{N < d}] + [ P{N < d} + P{N > d}]
2 4
3
= [ P{N < d} + P{N > d}]
4
3 d
= Q
2 N0 / 2

Remark. If the factor of 3/2 is ignored, the fraction of F = (4/ )2

amounts to a degradation in signal-to-noise ratio of 2.1 dB of
duobinary over direct binary. That is, to achieve the same error
probability, the transmission power for duobinary must be 2.1 dB
greater than that for direct binary, assuming ideal channel filtering
and AWGN. This is the sacrifice that paid for the smaller
bandwidth required by duobinary signaling. .
@G. Gong 38
5. Correlative Coding and Equalization (Cont.)
B. Eye Pattern
Eye pattern is an experimental tool to evaluate the
combined effect of receiver noise and ISI on overall system
performance in an operational environment.
It is defined as the synchronized superposition of all possible
realizations of the signal of interest (e.g. received signal,
receiver output) viewed within a particular signaling interval.

The eye pattern deserves its name from the fact that it
resembles the human eye for binary waves. The interior
region of the eye pattern is called the eye opening.

@G. Gong 39
Binary 0 0 1 0 1 1 1 0 0 0
data

t
T
Fig. 8 (a) Distorted binary wave with noisy, but no ISI

Fig. 8 (b)
t t
Eye pattern
Fig. 9 (b)
Eye pattern
T T
Binary 0 0 1 0 1 1 1 0 0 0
data

t
T
Fig 9. (a) Distorted binary wave with noisy and ISI
 Figure 10
@G. Gong 41
Remark 1. An eye pattern provides a great deal of useful information
about the performance of a data transmission system, as described
in Figure 10. Specifically, we may make the following statements:

1. The width of the eye opening defines the time interval over
which the received signal can be sampled without error from ISI.
It is apparent that the preferred time for sampling is the instant
of time at which the eye is open the widest.

2. The sensitivity of the system to timing errors is determined by the

rate of closure of the eye as the sampling time is varied.

3. The height of the eye opening, at a specified sampling time,

defines the noise margin of the system.

4. When the effect of ISI is severe , traces from the upper portion
of the eye pattern cross traces from the lower portion, with the result
that the eye is completely closed. In such a situation, it is impossible to
avoid errors due to the combined presence of ISI and noise in the system.
 Remark 2. In the case of an M-ary system, then eye
pattern contains (M - 1) eye openings stacked up
vertically one on the other, where M is the number
of discrete amplitude levels used to construct the
transmitted signal.

In a strictly linear system with truly random data, all these eye
openings would be identical. Figures 11 and 12 show the eye
diagrams for a baseband PAM transmission system using
M = 2 and M = 4 respectively, under the idealized conditions:
no channel nose and no bandwidth limitation (i.e., noiseless
and zero ISI), and Figures 13 show the eye diagrams with a
bandwidth limitation.

Note. For how to generate eye diagrams, see Handout 3.

@G. Gong 43
 Eye Diagram
Eye Diagram 1.5
1.5

1
1

0.5 0.5

Amplitude
Amplitude

0 0

0.5
0.5

1
1

1.5
2 1.5 1 0.5 0 0.5 1 1.5 2 1.5
Time 2 1.5 1 0.5 0 0.5 1 1.5 2
Time

Sample instance Sample instance

Figure 11. M = 2 Figure 12. M = 4
Noiseless and zero ISI
@G. Gong 44
 Eye Diagram Eye Diagram
2 1.5

1.5
1

0.5
0.5

Amplitude
Amplitude

0
0

0.5
0.5

1.5

1.5
2 2 1.5 1 0.5 0 0.5 1 1.5 2
2 1.5 1 0.5 0 0.5 1 1.5 2 Time
Time

Eye Diagram Eye Diagram

1.5 2

1.5
1

0.5
0.5

Amplitude
Amplitude

0 0

0.5

0.5

1
1.5

2
1.5 2 1.5 1 0.5 0 0.5 1 1.5 2
2 1.5 1 0.5 0 0.5 1 1.5 2 Time
Time

@G. Gong Figure 13. Band-width Limitation 45

 Equalization

In the preceding sections, we discussed that if a band-limited channel

Hc(f) is known, then it is possible to achieve ISI-free transmission by
using a suitable pair of Tx and Rx.

In practice we often encounter Channel equalizers: To compensate for the

channels whose frequency channel distortion, a linear filter with
response characteristics are adjustable parameters may be employed.
either unknown or change with The filter parameters are adjusted on the
time. The methodology to basis of measurements of the channel
overcome this problem is to characteristics. These adjustable filters are
employ channel equalizers. called channel equalizers or, simply,
equalizers. (Figure 14)

@G. Gong 46
p(kT+NT) p(kT+T) p(kT) p(kT-T) p(kT-NT)
Delay Delay Delay Delay
T T T T

w w w w0 w1 wN wN
N N +1
1 1

+
y(kT)
Figure 14

Effective Equalizer
Channel p(t) w(t)
N
Figure 15 w(t ) = wk (t kT )
k= N
Recall that the output of the overall filter may be sampled
periodically to produce the sequence

y k = bk + bn p k n + nok (1)
n k

where y k = y (kT ) p n = p(nT ) and nok = no (kT )

The middle term of the equation (1) represents the ISI.

In the practical system, it is reasonable to assume that the ISI

affects a finite number of symbols. Hence the ISI observed at
the output of the receiving filter may be viewed as being
generated by passing the data sequence though a linear filter.

@G. Gong 48
 Zero-forcing equalizer Note that any term in the sequence {gn }
is the weighted sum of consecutive 2N
Suppose that the equalizer is connected +1 terms of {pn } .
in cascade with the effective channel
(which consists of the Tx filter, To eliminate the ISI, according to the
physical channels and Rx filter), as Nyquist criterion for the distortionless
shown in Figure 15. transmission, we should satisfy
1 if k = 0
Let g(t) denote the impulse response of gk =
0 if k 0
the equalized systems, then
N From (8), we may force the conditions
g (t ) = p(t ) ) w(t ) = wn p(t nT )
1 for k = 0
n= N gk = (9)
0 for k = 1, 2, ..., N
At the time instance t = kT,
From (8) and (9), we obtain a set of
N
linear equations:
gk = wn pk n (8)
n= N N
1 for k = 0
wn pk = (10)
n
0 for k = 1, 2, ..., N
where pn = p(nT ) and g k = g (kT ) n= N
Equivalently, we have the following matrix form

! p0 ... p N +1 p N p N 1 ... p 2N !w N !0
: : : : : : :
pN 1 ... p0 p 1 p 2 ... p N 1 w 1 0
pN ... p1 p0 p 1 ... p N w0 = 1 (11)
p N +1 ... p2 p1 p0 ... p N +1 w1 0
: : : : : : :
p2 N ... p N +1 pN pN 1 ... p0 wN 0

A tapped-delay-line equalizer described by Eq. (10) or (11) is referred

to as a zero-forcing equalizer. Such an equalizer is optimum in the
sense that it minimizes the peak distortion (ISI).

@G. Gong 50
 In summary ,
(i) in presence of additive white Gaussian noise, a matched
filter is the optimum detector; and
(ii) in the presence of ISI , an equalizer is the desired
structure to mitigate ISI.

Intuitively, the optimum receiver should consist of a matched

filter and an equalizer in tandem, as shown in Figure 16.

x(t) r(t)
+ Matched filter equalizer

Front-end
n(t)
receiver
Figure 16
@G. Gong 51
 It should match to the Tx and the physical
Question: What should the front- channel: H ( f ) = H T ( f ) H C ( f ) (Fig. 17).
end of the receiver match to ?
matched n (t )
filter
j 2 ft 0
| H( f )|e
H ( f ) = H T ( f )H C ( f )
noise path
{an } {b n }
precoder HT ( f ) HC ( f ) | H( f )|e j 2 ft 0
+ equalizer

Tx filter physical
channel

Figure 17 Signal path

Signal path: the effective channel

Noise path: the filter noise must pass
has the impulse response:
through the equalizer which is not
equalizing the matched filter
% a further stretching of the pulse
% equalizing will enhance residual
% the matched filter accentuates ISI
additive noise.
% the equalizer needs to work harder.
 Two types of equalizers

Preset equalizers: On channels

Minimum mean-square error
whose frequency-response
(MSE) equalizer: :The tap weights
characteristics are unknown, but
are chosen to minimize MSE of all
time-invariant, we may measure the
the ISI terms plus the noise power
channel characteristics, adjust the
at the output of the equalizer.
parameters of the equalizer, and
once adjusted, the parameters
remain fixed during the
transmission of data. Remark. Most high-speed
telephone line modems use an MSE
weight criterion, because it is
Adaptive equalizers: update their superior to a zero-forcing criterion,
parameters on a periodic basis and it is more robust in the present
during the transmission of data. of noise and large ISI.

@G. Gong 53
 Remark on equalization of digital data transmission

The zero-forcing equations (10) or (11) in Sec. 4. 5 do not

account for the effect of noise. In addition, a finite-length
filter equalizer can minimize worst-case ISI only if the peak
distortion (i.e., the magnitude of the difference between the
channel output and desired signal ) is sufficiently small.

The sequence {wk} , in Figure 11 in Sec. 4. 5, can be chosen in a

way such that one can minimize the mean-square error (MSE) of
all the ISI terms plus the noise power at the output of the
equalizer. This is called minimum MSE equalizer. (Here, MSE is
defined as the expected value of the squared difference between
the desired data symbol and the estimated data symbol . )
 A Few Remarks on the Definition of bandwidth
and Relation between Channel Bandwidth and
Transmission Rate
The bandwidth dilemma

For all bandlimited spectra, the waveforms are not realizable, and
for all realizable waveforms, the absolute bandwidth is infinite.
The mathematical description of a real signal does not permit the
signal to be strictly duration limited and strictly bandlimited.

All bandwidth criteria have in common that attempt to specify a

measure of the width, W, of a nonegative real-valued psd defined
for all frequencies | f |< $ .

@G. Gong 55
 General shape of H X ( f ) = T sin c 2 ( f f c )T
psd

1
fc
fc 1
T fc +
T
(a)
(b)
(c)
(d)
(e) 35 dB

(e) 50 dB
@G. Gong 56
(a) Half-power bandwidth. This is (d) Fractional power containment
the interval between frequencies at bandwidth. Federal Communication
which H X ( f ) has dropped to half- Commission (FCC Rules and
power, or 3 dB below the peak value. Regulations Section 2.202). It states
that the occupied bandwidth is the
(b) Equivalent rectangular or noise band that exactly .5% of the signal
equivalent bandwidth. It is defined by power above the upper band limit and
WN = PX / H X ( f c ) , where PX is the total exactly 0.5% of the signal power
signal power over all frequencies. below the lower band limit. Thus 99%
of the signal power is inside the
(c ) Null-to-null bandwidth. It is occupied band.
defined as the width of the main
spectral lobe, where the most of the (e) Bounded power spectral density.
signal power is contained (the most Everywhere outside the specified band,
popular measure of bandwidth. ) H X ( f ) must have fallen at least to a
certain stated level below that found at
(f) Absolute bandwidth. This is the
the band center. Typical attenuation
interval between frequencies, outside
levels might be 35 or 50 dB.
of which the spectrum is zero. (Useful
abstraction. For all realizable
waveforms, this is infinite.)
 Example. Digital Telephone Circuits.
Compare the system bandwidth requirements for a terrestrial 3-kHz analog
telephone voice channel with that of a digital one. For the digital channel,
the voice is formatted as a PCM bit stream, where the sampling rate is
8000 samples/s and each voice sample is quantized to one of 256 levels.
The bit stream is then transmitted using a PAM waveform and received
with zero ISI.

Solution. The resulting of the sampling and quantization process yields PCM
words such that each word has one of L = 256 levels. If each sample were sent
as a 256-ary PAM pulse (symbol). Thus the required system bandwidth
without ISI for sending Rs symbols/s would be W Rs / 2 . Since each PCM
word is converted to 8 bits. Thus, the system bandwidth required using PCM is
1
WPCM (8 bits/symbol)(8000symbols/s) = 32 kHz.
2
Therefore, the PCM format, using 8-bit quantization and binary signaling with
binary PAM, requests at least eight times the bandwidth required for the
analog channel.
 A Note on Relation between Channel
bandwidth and transmission rate
Question:
In the ideal Nyquist channel, W = R/2 . How can it be possible for
the channel bandwidth W to be smaller than the transmission rate R?

Answer:
1) The channel bandwidth W (Hz) and the transmission rate
R (bit per second , or bps) are two different physical
quantities. In general, they are proportional to each other,
but it is NOT necessary for them to be equal.
2) Transmitter Physical Receiver
Filter channel filter
H T ( f ) H C ( f ) H R ( f )

Transmitted Received
signal signal

Effective channel P( f ) = H T ( f )H C ( f )H R ( f )
(assuming = 1)

2
The psd of the transmit ted signal HT ( f )
% the bandwidth of the transmi tted signal is the same
as the bandwidth of H T ( f )
 Let BT : bandwidth of H T ( f )
(= bandwidth of the transmitted signal)
BC : bandwidth of H C ( f )
BR : bandwidth of H R ( f )
The system should be designed in such a way that
BT = BC = BR
in order to achieve
1) high spectrum utilization efficiency
2) high transmitted power utilization efficiency
3) high transmission accuracy

3). For the discrete PAM signal formats, the signal bandwidth
(e.g., defined as the frequency interval which contains 99% of
the total power, Definition (d)) may not be equal to the
transmission rate 1/Tb.
@G. Gong 61
4) Consider a special case where

1, f <W
H T ( f ) = H C ( f ) = H R ( f ) = P( f ) =
0, f >W
The ideal Nyquist channel

We have the Fourier pair : 2Wsinc(2Wt ) + H T ( f )

Using 2Wsinc( 2Wt ) for symbol "1" and 2Wsinc(2Wt )

for symbol "0" , we can transmit a binary information
sequence at a rate R = 2W without ISI , as the symbol
interval can be as small as 1 / 2W under the constraint
of ISI - free transmission.
@G. Gong 62
Summary of Chapter 4 (Chapter 6 in the textbook)

1.PAM signals and power spectrum density of binary PAM signals

$
1 2
S X ( f ) = HT ( f ) RB (n) exp( j 2 nfT )
T n= $

2. ISI due to bandlimited channel

3. Nyquist criteria for ISI - free transmission
1, n=0
p(nT ) =
0, n 0
$
P( f nR ) = T R = 1/ T
n $

Ideal Nyquist Channel R = 2W

2W
Raised Cosine Spectrum R =
1+"
@G. Gong 63
4. Correlative coding and equalization

Duobinary signaling: achieving the maximum transmission rate

2W with zero ISI
i) Pre-coder with memory: an , bn , yn
encoding and decoding (error propagation)
ii) Pre-coder without memory: an , bn* , bn , yn
encoding and decoding (no error propagation)
iii) Psd of the duobinary PAM signals and error probability

Eye Patterns:

Equalization: to mitigate the effects of ISI, zero-forcing

equalizer.
{an } {b } *
{yn }
{y n } {b }
n
{a n }
n
CL
{bn } Ideal channel duobinary Post-
+ + HNyquist( f ) decoder coder
-
2D 2D overall channel

Modified Duobinary Scheme with Precoding