PRINCIPLES OF COMMUNICATION SYSTEMS Second Edition Herbert Taub Donald L. Schilling Professors of Electrical Engineering The City College of New York McGraw-Hill Publishing Company New York St Lous San Francisco Auckland Boqots Caracas Hamburg Liston London Madrid Mexico Milan ‘Montreal New Delhi Okioma City) Paris San Juan ‘Sho Paulo ‘Singapore Srdney Tokyo TorontoCHAPTER ONE SPECTRAL ANALYSIS INTRODUCTION Suppose that two people, separated by a considerable distance, wish 10 commu ricate with one another. I there i a pair of conducting wires extending from one Tocation to another, and if each place is equipped with # microphone and eat piece, the communication problem may be solved. The microphone, at one end of the wire communications channel, smpresses an electric signal voltage on the line, ‘which voltage is then received at the other end. The received signal. however, will have associated with it an erratic, random, unpredictable voltage wavelorm whieh is deseribed by the term nose. The origin of this noise will be discussed more flly in Chaps. 7 and 14, Here we need but to note that at the atomic level the universe is in a constant state of agitation, and that this agitation is the source of a very great deal of this noige, Hecause of the length of the wire link. the received message signal voltage will be greatly attenvated in comparison with its level at the transmitting end of the link. As a result the message signal voltage may not be very large in comparison with the noise voltage. and the message will be perceived with difficulty or possibly not at all. An amplifier at the receiving end will not solve the problem, since the amplifier will amplify signal and noise slike. As a matter of fat, as we shall see, the amplifier itself may well be @ source ‘of additional noise. 'A principal concern of communication theory and a matter which we discuss extensively in this book is precisely the study of methods to suppress, as far as possible, the effect of noise. We shall see that, for this purpose, may be better fot to transmit directly the original signal (the microphone output in our2 puNcIPLS OF COMMUNICATION SYSTEMS example). Instead, the original signal is used to generate a different signal wave form. which new signal waveform is then impressed on the line. This processing of the original signal to generate the transmitted signal is called encoding ot nodulation. At the receiving end an inverse process called decoding or demodu- Tation is requited to recover the original signe! ‘may well be that thee i a considerable expense in providing the wire com: munication link. We ate therefore, naturally led to inquire whether we may use the link more effectively by arranging forthe simultaneous transmission over the Tink of more than just a single waveform. It wins out that such multiple tran mission is indeed possible and may be accomplished in a number of ways. Such ‘Simultaneous multiple transmission is called multiplexing and is again @ principal trea of concern of communication theory and of this book. It isto be noted that ‘when wire communications links are employed, then, at least in principle feparate links may be used for individual messapes. When, however, the comme: fications medium is free space, as in radio communication from antenna to antenna, multiplexing i essential in summary then, communication theory addresses itself to the following questions: Given a communication channel, how do we arrange to transmit as many simultaneous signals as possible, and how do we devise to suppress the tllect of noise to the maximum extent possible? In this book, after a few mathe- tatical preliminaries, We shall address ourselves precisely to these questions, frst, to the matter of multiplexing, and thereafter tothe discussion of noise in commu 'A branch of mathematics which is of inestimable value in the study of com- munications systems is spectral analysis. Spectra) analysis concerns itself with the description of wavelorms in the jrequeney domain and with the correspondence between the Irequency-domain description and the time-domain description. It is assumed that the reader has some familiarity with spectral analysis, The presen tation in this chapter is intended as a review, and will serve further to allow 9 Compilation of results which we shall have occasion to use throughout the remainder ofthis text 1.1 FOURIER SERIES! [A periodic function of time oft) having a fundamental period T, ean be represent fd as an infinite sum of sinusoidal waveforms. This summation called a Fourie ‘series. may be written in several forms. One such form is the following. ot) = Ag+ z A, cos Te + ze sin aay ned [08 0 SPECTRAL ANALYSIS 3 while the coeficients 4, and B, are given by ‘Am alternative form for the Fourier series is wip Cu+ 5 Cscos 224 o,) as) whete Co, Cyvand @, are related t0 Ag, A,vand B by the equations Com Ae (1.166) C= JAE (11-66) wd é.2tn (1-60 “The Fourier series of @ periodic function is thus seen to consist of a summ tion of harmonies of a fundamental frequency fy = 1/Tq. The coefficients C, are called spectral amplitudes; that i, C, isthe amplitude of the spectral component 08 (2anfot @,) at frequency nfp- A typical amplitude spectrum of a periodic wavelorm is shown in Fig, I.t-1a, Here, at each harmonic frequency, a vertical Tine has been drawn having a length equal to the spectral amplitude associated with each harmonic frequency. Of course, such an amplitude spectrum, lackins the phase information, does not specify the waveform 1). tlrttiedy 4 3h a o Figure 11-1 (a) A onesided plot of spectral ampbiude of erode waveform, (6) Te eorespondinz ‘wosidd plo4 rnixciris oF conmunacaTion sysrins 1.2 EXPONENTIAL FORM OF THE FOURIER SERIES ‘The exponential form of the Fourier series finds extensive application in commu: nication theory. This form is given by a= 5 erm a2) where His given by vine a (12.2) “The coefficients ¥, have the property that ¥, and V-, are complex conjugates of ‘one another, that is, V, = V2,. These coefiients are related to the C's in Ea (11-5) by © (1239) is om (12.36) ‘The V's are the spectral amplitudes of the spectral components V,e%¥*, The amplitude spectrum ofthe Vs shown in Fig. 1.1-1b corresponds to the amplitude spectrum of the C,'s shown in Fig. -1-la, Observe that while Vo = Co, otherwise each spectral line in I.l-1a at frequency f is replaced by the 2 spectral lines in Li-1b, each of half amplitude, one at frequency f and one at frequency ~f. The amplitude spectrum in 1.1-1a is called a single-sided spectrum, while the spectrum in L.1-1b is called a two-sided spectrum, We shall find it more convenient t0 use the two-sided mplitude spectrum and shall consistently do so {rom this point on. 1.3 EXAMPLES OF FOURIER SERIES ‘A waveform in which we shall have occasion t0 have some speci interest is shown in Fig. 13-1a, The wavelorm consists of @ periodic sequence of impulses of Strength J. As a siier of convenience we have selected the ume feale £0 that an impulse occurs at t= 0. The impulse at ¢= 0 is wntten as I 6). Here d(r) is the delta function which has the property that Jt) = 0 except when 1 = O and further Pos i ‘The strength of an impulse is equal to the area under the impulse, Thus the strength of ) is 1, and the strength of! (0) ‘The periodic impulse train is writen w=1 Atk) (132) SPECTRAL ANALYSIS 5 reuse) HausTe) — [18 umTy) UBT) =e i om ah oe A a Figure 13.4 Examples of perio frtions. (a) A perio aia of implies. (6) pedi ain of pubes of erator» We then find, using Eqs. (1-2) and (1.13). mek fe wwaed 038 asa Vo= Ve te (13-7)6 pnmicins oF COMMUNICATION SYSTEMS Hence oft) may be writen in the forms 1 wal Yok = 7 38) [As a second example, let us find the Fourier series for the periodic train of pulses of amplitude A and duration tas shown in Fig. 13-1. We find (13.9) _2At sin (na Te) el, (1310) and 0 0 3a) Thus At, 2At & sin ne/Te) ... 2am 1) = ES OT, (13-120) sin net/TS) rear, (13.128) Suppose that in the waveform of Fig. 1.3-1b we reduce + while adjusting A so that Arisa constant, say At = I, We would expect that inthe limit, as t=» Othe Fourier series forthe pulse train in Eq, (13-12) should reduce to the series for the impulse train in Eq. (1.3). It is readily verified that such is indeed the case since ast0 sin nns/To)_ fe 03413) 14 THE SAMPLING FUNCTION ‘A function frequently encountered in spectral analysis is the sampling function ‘Sox) defined by [A closely related function is sine x defined by sinc x tion Sala) is plotted in Fig. 14-1. 1 is symmetrical about x = 0, and at x = 0 has SPECTRAL ANALYSIS 7 Figure 144 (0) The fncion Sb) Te spectl amplitudes Wf the 1wosided Fourie represen ation ofthe pulse tino Fig. Lebar = and Ty = the valve Sa(0) = 1, It oscillates with an amplitude that decreases with increasing x The function passes through zero at equally spaced intervals at values of x= tnx, where mis an integer other than zero. Aside from the peak at x = 0. the maxima and minima occur approximately midway between the 2er0s, ie, at x = “(n+ 3)n, where [sin x| = 1, This approximation is poorest for the minima Ria) (1183) ‘This result is rather intuitively obvious since we would surely expect that simi larity between o(t) and (+ x) be a maximum when t = 0, The student is guided through a more formal proof in Prob. 118-1 RG) = Rl) (1.184) Ths the suacorelton oncton san even ution of To prove Es, (1.184, sume atthe aust Ok moved inte ete ection fy aunt The the megrand in E181) mold become th and RC) mols Rose Hi) Si: were meton cen eed em" ys Shit inte at cin bane mo ete on te tle fhe. Ipa Ther Rit) = R(—1) ae Th tres characteris given in as, (1.1210 (1.14 re ears a cnt of Re) dene by E81) bt to fr Hh ened y ae 1163) tnd (163) or the pei case and he ower ese eee. athe Iter fcoue RO) = Ethene aire he ower 1.19 AUTOCORRELATION OF A PERIODIC WAVEFORM When the function is periodic, we may write E neon (119-1) iom ntgrl in the form of Ea (1.162), we have pny moe 2[™ (FE memnll § erm) gus OTs Jara 4 ‘and, using the corte ‘The order of integration and summation may be interchanged in Eq, (1.19-2) If we do so, we shall be left with a double summation over m and n of terms Jy, given by i = oie. a eee PO he “ (11930) items U8 + I) xim=m) (119.38) Since m and m are integers, we see from Eq, (1.19-38) that Iq, = 0 except when m= ~nor m+ n=O. To evaluate Jp in tis latter ease, we return to Eq, (1-19- 3a) and find WV, een (119-4)[32 PRINCIPLES OF COMMUNICATION SYSTEMS Finally, then. Ripe E vyvegetmete= SF pvyetre (4.1950) F cos 2am = 19-58) VP 608 2am > (119-56) = lh +2 We note from Eq, (1.19-Sb) that Rie) = R(—1) as anticipated, and we note as well that for this case ofa periodic wavelorm the correlation R(t) is also periodic with the same fundamental period Ty ‘We shall now relate the correlation function R(x) of a periodic waveform to its power spectral density. For this purpose we compute the Fourier transform of (), We find, using R(e) as in Eq (1.19-Sa), that SCE ate eH a (19.9 FIR) Interchanging the order of integration and summation vielé= sinen= Fine [emerson a as-n Using Ea, (111-22, we may writ Eg (119-7) 2» sima= 5 Kel s- 2 (119-8) ‘Comparing Eq, (1.19-8) with Eq, (18-6), we have the interesting result that for 3 periodic waveiorm Gif) = F TREN (119.9) and, of course. conversely Ri) = FOUN (119-10) Expressed in words, we have the following: The power spectral density and the correlation function f aperiodic waveform are a hourter transform pair. 1.20 AUTOCORRELATION OF NONPERIODIC WAVEFORM OF FINITE ENERGY For pulse-type waveforms of finite energy there isa relationship between the cor- relation function of Eq, (1.181) and the energy spectral density which corre: sponds to the relationship piven in Eg. (1.19.9) for the periodic waveform. This relationship is that the cortelation function R(t) and the energy spectral density are a Fourier translorm pai. This result is established as follows. SPECTRAL ANALYSIS 33, We use the convolution theorem. We combine Eqs. (1.12-1) and (112-3) for the case where the waveforms rj(2) and 0,() are the same waveforms, that is, 14 () = nt) = off, and get Foon aa)tt — 9) és (1201) Since V(—f) = V*f) = F[x(—], Eq, (120-1) may be writen FUN = FICK = | eens — ny de (1.202) ‘The integral in Eq, (1.20-2) isa function of t, and hence this equation expresses F TVW) as « function of tI we want to express F~*[V(/W*(f)} as fonction of r without changing the form of the function, we need but to intet- change rand ¢, We then have * worn f Aojat ~ 2) a (120.3) ‘The integral in Eg. (1.20-3 is precisely R(e), and thus FIR] = VIAN) = LY (120-4) which verifies that R(s) and the energy spectral density | V(/)|? are Fourier trans- form pairs 1.21 AUTOCORRELATION OF OTHER WAVEFORMS In the preceding sections we discussed the relationship between the autocorrela- tion function and power or energy specttal density of deterministic waveforms, We use the term “deterministic” to indicate that at least, in principe, it is pos sible to write a function which specifies the value of the tunction at all times. For such deterministic waveforms, the availability of an autocorrelation function is of ‘no particularly preat value. The avtocorrelation lunetion does not include within itself complete information about the function. Thus we note that the autocorrel ation function is related only to the amplitudes and not to the phases of the spe: tral components of the wavelorm. The wavelorm cannot be reconstructed from a knowledge of the autocorrelation functions. Any charactenstie of a deterministic waveform which may be calculated with the aid of the autocorrelation function may be calculated by direct means a least as conveniently ‘On the other hand, in the study of communication systems we encounter waveforms which are not deterministic but are instead random and unpredictable in nature. Such waveforms are discussed in Chap. 2. There we shall find that for such random waveiorms no explicit function of tume can be writen, The wave- forms must be described in statistical and probabilistic terms. It isin connection with such waveiorms that the concepts of correlation and autocorrelation find their true usefulness. Specifically, it turns out that even for such random wave-[4 PaINcIPLES OF CONOLUNICATION SYSTEMS forms, the autocorrelation funetion and power spectral density are 8 Fourier transform pair. The proof? that such is the case is formidable and will not be undertaken here 1.22 EXPANSIONS IN ORTHOGONAL FUNCTIONS Let us consider a set of functions 9 (x) g0%h -»» 040) °°» defined over the inter~ val x, lo Hi: Conse (nae topes 112 07 Expand and iterate erm by term, Show thst = 2180) RUN} 2 1.182 Determine am expeson forthe coneation function of square wae vig the wales | or (and a period 7.4 princiPits OF COMMUNICATION SYSTEMS 194 () Find the power petal density of squasewave vege by Fourie transforming he cr resto fenton, Use the eats of Prob 1182 18) Compare the answer fo (a) wh he spectral density obcained rom the Fourier sis (Prob 9.1 1982.1 = sin gs (a) Find Ris) (0) MG{/)~ (RL), fd Gs) diety and compare 1204, A wavelorm consis single pub of amplitude A extening fom 1 = (a) Fin the auoeorelation uncon Ref this waseor (8) Calla ne ney spel desy ofthis ule by evabaing Gf) = FRC) (o)Caleaat Gl) rectly by Paseva’s theorem and compare 1.44, Inerchang he nb) a5 0) 08 te waveforms of Fig. 1.-1e and wit his new labe- Ing apply the Grat- Sebi procedure fo nd expentone of the wavelorms i ems of orthonormal 200 1.242, Use the Gram-Schid procedure to expr the anctons in Fig PL2#-2 in verms of ortho vm components 0) sa so] 1242 Ase of seal lk = 12,34 en by 40 cs(erstd) osest ‘Ue the Gram- Schmidt procedure to fad an orthonormal vet functions wich the fonction 5) 1264, Verify Eq (1.251 262 (a) Show that he potsporesn theorem apis o sen functions. Thai show that i) nd 0) ate orthogonal then the sat othe length of) etned a in Ea (25-104) pls the ate ofthe length of xual the guar of he ng of he sum) + 0) (t) Les) ad 3) be the spe shown on Fig, P1262. Draw the ial 0) +540 Show ‘hat g)and tare otbopoea Show ha a6)+ 5? = INP + Ll) igwe P1282 1288. Very Eq (1.2513) SPECTRAL ANALYSIS 58 1284.) Refer to Fig 128.2 and (125-19) Ves tha the parameter in Bg, (1.25.3) she tangent ofthe ange between the anes the au, coordinate tem ad thew connate syste 1b) Expand the lonetion 0) ands) of Fig 121 i terms of orihonrma functions wt) and (whic ate the aes of coordinate stm which rotated 60 eunterlokone rom the cord ste system whose aes are (and wll he same re 1261. Apply the Gram-Schmid procedure to the eight signals of Ea. (1266) Vey that tno thonormal components ufc o alo Tepesenation of any ofthe signals Show thal aa coo. ‘ite sytem ofthese two orthonormal sgn, the geometric representation ofthe cet spas 3° shown in Fig. 126 1262 A set of sah is 5K)» VIP, in wt, f= ~V3P, sm eg A second set of sina 16s = VRP in tus = DP, gt +2) the two se ial ac have the same de Nepasabiy, what the abo PP. 126 Ges 00 seas swe) OStSTs ayo=—M) OsteT Stow that independent fxm ff) he distinguish of Bese is ea by /2E whee Elsie normaiues ener ofthe wart 1264, () Use the Gram Sehr prcrdre t expe the functions in Fig. L264 in ee of boreal components In apy the proce, invoie the fncvons nthe ore Sh ane 30) Pt the lanctons as poi a vont system n whch the sori ee ‘eshed in nit fhe orthonnmal (0) Repeat pat) ener tha the tion ae to ead th de 0 oh fh and 01 Pte acon (0) Show tht te pres of pars) a () the same datas bees ncn reins ou 500 sa sa 5| a 5) 4| 2 2| 2 2| 1 tts 1 ompapat ype T tT? ype 2} -2 -3 -3 -3 igre P64GiarreR TWO RANDOM VARIABLES AND PROCESSES ‘A waveform which can be expressed, at least in principle, as an explicit function of time xf is called a deterministic waveform. Such a waveform is determined for all times in that, il we select an arbitrary time ¢ = 1, there need be no uncertainty bout the value of xt) at that time. The wavelorms encountered in communica~ tion systems, on the other hand, are in many instances unpredictable. Consider, say, the very waveform itself whichis transmitted for the purpose of communica- tion, This waveform, which is called the signal, must, at least in part, be unpri dictable. If such were not the case, ie, if the signal were predictable, then sts transmission would be unnecessary, and the enlire communications system would serve no purpose, This point concerning the unpredictability of the signal is explored further in Chap. 13. Further, as noted briefly in Chap. 1, transmitted signals are invariably accompanied by noise which results from the ever-present apitation of the universe atthe atomic level These noise waveforms are also not predictable. Unpredictable wavelorms such as a signal voltage () or noise voltage ait) are examples of random processes. [Note that in writing symbols lke sft) and n(t) we do not imply that we can write explicit functions for these time functions.) While random processes are not predictable, neither are they completely unpredictable. Its generally possible to predict the foture perlormance of a random process with a certain probability of being correct. Accordingly, in this chapter we shall present some elemental ideas of probability theory and apply them to the description of random processes. We shall rather generally limit our discussion to the development of only those aspects of the subject which we shall have occasion to employ inthis text. [RANDOM VARIABLES AND PROCESSES 57 2.1 PROBABILITY! ‘The concept of probability occurs naturally when we contemplate the possible ‘outcomes of an experiment whose outcome is not always the same. Suppose that ‘one of the possible outcomes is called A and that when the experiment is repeat ‘ed N times the outcome A occurs N, times. The relative frequeney of occurrence of A is NIN, and this ratio N,/N is not predictable unless N is very large. For ‘example, et the experiment consist of the tossing of a die and let the outcome A ‘correspond to the appearance of, say, the number 3 on the die. Then in 6 tosses the nomber 3 may not appear at all, or it may appear 6 times, or any number of times in between. Thus with N = 6, N,/N may be 0 or 1/6, etc, up to NIN = 1 On the other hand, we know ftom experience that when an experiment, whose outcomes are determined by chance is repeated very many times, the relative fre- quency of a particular outcome approsches a fixed limit. Thus, if we were 10 toss 4 die very many times we would expect that Ny/N would turn out (0 be very close to 1/6, This limiting value of the relative frequency of occurrence is called the probability of outcome A, written P(A), s0 that im Ba en) In many cases the experiments needed to determine the probability of an ‘event are done more in thought than in practice. Suppose that we have 10 balls in a container, the balls being identical in every respect except that § are white and 2 are black, Let us ask about the probability that, in a single draw, we shall select a black ball. If we draw blindly, so that the color has no influence on the ‘outcome, we would surely judge that the probability of drawing the black ball is 2/10, We arrive at this conclusion on the basis that we have postulated that there is absolutely nothing which favors one ball over another. There are 10 possible ‘outcomes ofthe experiment, that is, any of the 10 alls may be drawn: ofthese 10 outcomes, 2 are favorable to our interest. The only reasonable outcome we can imagine is that, in very many drawings, 2 out of 10 will be black. Any other ‘outcome would immediately suggest that either the black or the white balls had been favored, These considerations lead to an alternative definition of the prob lity of occurrence of an event A, that s ‘number of possible favorable outcomes total number of possible equally likely ovicomes Its apparent from either definition, Eq. (21-1) oF (21-2) that the probability of occurrence of an event P isa positive number and that O'< P < 1. Ifan event 4s not possible, then P =O, while il an event is certain, P = | PA) A) 212) 22 MUTUALLY EXCLUSIVE EVENTS ‘Two possible outcomes of an experiment are defined as being mutually exclusiee if the occurrence of one outcome precludes the occurrence of the other. In this case, if the events are A, and A; with probabilities P(4,) and P(4,), then the‘94 pRrsciPLs OF COMMUNICATION SYSTEMS the shaded area in Fig, 222-2 measures the probability that m, was transmitted and read as my In Fig. 2.22-3 we represent the situation in which one of four messages m,, ‘mz,ms, and mg are sent yielding receiver responses 5, 5;, 55, and s4. The prob- ability density function fyi) ofthe noise (multiplied by 4) is shown in Fig. 222- 3a. Also shown are the functions Pim,fr— 5) for k= 1,2, 3, and 4, We have taken all the probabilities Pim) to be equal at Pim, = 4. The boundaries of the received response R at which decisions change are also shown and are marked 131 P2384 Pau "The sum of the two shaded areas shown, each of area A, equals the probabil ity P(E, m;) that m, is transmitted and that the message is erroneously read. Thus tne « Lies) Aide) Ars) Aye) Ly OM Figure 222.3 Probailitycemty when fur signals are tarsi (Probably density off) then oie alone i reeved. 8) Probability density hen one of fur sioals i ransmited and ‘esd in nove RANDOM VARIABLES AND PROCESSES. 96 PEE, m,) = 2A. The remaining area under 4fr ~ 53) i the probability that m, is transmitted and is correctly read. This probability is P(C, m,) = 4 — 2A. We find of course that PLE, ms) = P(E, m,). But it is most interesting to note that when wwe evaluate P(E, m,) or PIE, m,) there is only a single area A involved. Hence PLE, m,) = PLE, ma) = 4P(E, my) = 4PLE, ms). Thus, we arrive at the most inte esting result that given four equally likely messages, when we make boundary decisions which assure minimum average likelihood of error, the probability of ‘making an error is not the same for all messages. Of course, ths result applies lor any number of equally likely messages greater than two. “The probability that a message i read correctly is PIC) = PIC, my) + PLC, ma) + PIC, ms) + PIC, ma) A) +d -20)+G-28) 40-4) 68 222-4) Thus the probability ofan error PIE) =1— AC) = 64, 2225) 223 RANDOM PROCESSES ‘To determine the probabilities of the various possible outcomes of an experiment, it is necessary to repeat the experiment many times. Suppose then that we are interested in establishing the statistics associated with the tossing of a die. We ‘might proceed in either of wo ways, On one hand, we might use a single die and toss it repeatedly. Alternatively, we might toss simultaneously a very large number of dice. Intuitively, we Would expect that both methods would give the ‘same results, Thus, we would expect that a single die would yield a particular ‘outcome, on the average, of | time out of 6, Similarly, with many dice we would expect that 1/6 ofthe die tossed would vield a particular outcome ‘Analogously, let us consider s random process such as & noise waveform mie) ‘mentioned at the beginning of this chapter. To determine the statistics of the noise, we might make repeated measurements of the noise voltage output of a single noise source. or we might at least conceptually, make simultaneous mea surements of the output of a very large collection of statistically identical noise sources. Such a collection of sources is ealled an ensemble, and the individyal noise wavelorms are called sample functions. A statistical average may be detet- ‘mined from measurements made at some fixed time 1 = t, on all the sample func. tuons of the ensemble. Thus to determine, say, (, we Would, at =f, measure the voltages n(,) of each noise source, square and add the voltages, and divide by the (large) number of sources in the ensemble, The average so determined is the ensemble average of r(y) [Now nft,) is a random variable and will have associated with ita probability density function. The ensemble averages will be identical with the statistical aver ‘ages computed earlier in Secs. 2.11 and 2.12 and may be represented by the same