SIGNAL ANALYSIS ATHANASIOS PAPOULIS POLYTECHNIC INSTITUTE OF NEW YORK McGRAW-HILL BOOK COMPANY New York St. Louis San Francisco Auckland Bogoté Dusseldorf Johannesburg London Madrid Mexico Montreal New Delhi Panama Paris Sdo Paulo Singapore Sydney Tokyo ‘brontoSIGNAL ANALYSIS Copyright © 1977 by McGraw-Hill, Inc. All rights reserved. Printed in the United States of America. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. 7890 DODO 8987654 This book was set in Times New Roman. The editors were Peter D. Nalle and Michael Gardner; the cover was designed by Scott Chelius; the production supervisor was Leroy A. Young. The drawings were done by J & R Services, Inc. R. R. Donnelley & Sons Company was printer and binder. Library of Congress Cataloging in Publication Data Papoulis, Athanasios, date Signal analysis. Includes bibliographies and index. 1. Signal processing. I. Title. TK5102.5.P35 621.38043 76-54353, ISBN 0-07-048460-0CONTENTS Preface ix PART ONE SIGNALS, SYSTEMS, AND TRANSFORMS Chapter 1 Introduction 3 1-1 Discrete Signals and Systems 4 1-2 Analog Signals and Systems 12 1-3. Digital Simulation of Analog Systems 25 Chapter 2 Discrete Systems 30 2-1 zTransforms 31 2-2 Recursion Equations 40 2-3 Finite-Order Systems 43 Chapter 3 Fourier Analysis 56 3-1 Fourier Transforms = 57 3-2 Line Spectra and Fourier Series 66 3-3. From Fourier Integrals to Discrete Fourier Series 74 3-4 Discrete Fourier Series and Fast Fourier Transforms 79 Appendix 3-A The Mean-Value Theorems 92 Appendix 3-B Asymptotic Properties of Fourier Transforms 94 Appendix 3-C Singularity Functions 96 Chapter 4 Continuous Systems 101 4-1 Moment Expansion and Spectrum Analyzers 102 4-2 Filters 116 4-3 Finite-Order Systems 124 Appendix 4-A Maximum Response of Linear Systems 134vi CONTENTS. Chapter 5 Digital Processing of Analog Signals 139 5-1 Sampling and Interpolation 140 5-2 Mean-Square Approximations 146 5-3 Digital Simulation of Analog Systems — 154 5-4 Nonrecursive Filters 160 5-5 Filters 165 5-6 Recursive Frequency-Domain Filtering 173 PART TWO SELECTED TOPICS Chapter 6 Bandlimited Functions 183 6-1 Properties of Bandlimited Functions 184 6-2 Generalized Sampling 191 6-3 Bounds and Extreme Values of Bandlimited Functions — 196 6-4 The Prolate Spheroidal Functions 205 Appendix 6-A Extrema of Integrals of Trigonometric Polynomials and Bandlimited Functions 216 Chapter 7 Factorization, Windows, Hilbert Transforms 221 7-1 Analytic and Asymptotic Properties of Laplace Transforms = 222 7-2. The Factorization Problem —- 227 7-3 Windows and Extrapolation 234 7-4 Hilbert Transforms 251 Appendix 7-A The Paley-Wiener Theorem 258 Appendix 7-B Jordan’s Lemma —- 261 Chapter 8 Frequency Modulation, Uncertainty, Ambiguity 262 8-1 Frequency Modulation and the Method of Stationary Phase 263 8-2 Uncertainty Principle and Sophisticated Signals 273 8-3 Two-dimensional Transforms and Hankel Transforms 278 8-4 The Ambiguity Function 284 PART THREE DATA SMOOTHING AND SPECTRAL ESTIMATION Chapter 9 Stochastic Processes 299 9-1 Correlations and Spectra 300 9-2 Linear Systems with Stochastic Inputs 305 9-3 Spectral Analysis 312 9-4 Discrete Processes 319Chapter 10 Data Smoothing 324 10-1 Known Signals in Noise 325 10-2 Unknown Signals in Noise 329 10-3 Random Signals and Wiener Filters 336 10-4 Discrete Processes 344 Chapter I] Ergodicity, Correlation Estimators, Fourier Transforms 11-1 Ergodicity 352 11-2 Correlation Estimates 359 11-3. Fourier Transforms of Random Signals 363 Appendix 11-A Normal Processes and Cumulants 374 Chapter 12 Spectral Estimation 378 12-1 Sample Spectrum 379 12-2. Smoothed Spectrum 380 12-3. Theory 386 Problems 393 Solutions 406 Index 421 CONTENTS Vii 351PREFACE Educators are in general agreement that engineering students are best prepared for future challenges if they are instructed not only in their specialty but also in disciplines of broad scope. One such discipline, rich in theory and applications, is signal analysis. It is used in astronomy, oceanography, crystallography, bio- engineering, antenna design, communications technology, system theory, computer sciences, and many other fields. I have selected from this elegant subject several areas related to Fourier transforms and linear systems, and have sought to develop them with clarity, perspective, and economy. This book is the result. Most derivations follow rigorously from a small number of clearly defined concepts. However, in certain cases, I give only plausibility arguments relating the conclusions to familiar concepts. This occasional sacrifice of rigor for the sake of brevity is not uncommon, even in pure mathematics where, for example, Jordan’s curve theorem is usually accepted as self-evident and the axiom of choice is ignored. The material is divided into three self-contained but closely related parts. The level of difficulty of each part is fairly uniform, but the degree of elaboration varies depending on the importance or novelty of the topic. I have used in several instances the word elaborate in parenthesis to indicate that the explanation is brief. The book includes several applied topics, some of them new. However, I am primarily an educator, and I concentrated on what I thought I could do best— concepts of general interest rather than detailed coverage of special topics. During the preparation of the manuscript, I had the benefit of lengthy discussions with many students and colleagues. I am indebted in particular to Professors R. Haddad, A. E. Laemmel, and D. C. Youla. I also wish to express my appreciation to Professor G. Temes of U.C.L.A. for his critical comments and constructive suggestions. Athanasios PapoulisSIGNAL ANALYSISCHAPTER ONE INTRODUCTION In this chapter, we present briefly the basic concepts of the theory of discrete and continuous systems. In Sec. 1-1, we develop the discrete form of linearity, establish the equivalence between convolution and terminal characterization of linear systems, and introduce the z transform as system function.’ The discussion is repeated in Sec. 1-2 for continuous systems and Fourier transforms.* In Sec. 1-3, we explain the principle of digital simulation of analog systems. These topics are developed with greater elaboration in subsequent chapters. ' B. Gold and C. Rader, “ Digital Processing of Signals,” McGraw-Hill Book Company, New York, 1969. ? L. E. Franks, “Signal Theory,” Prentice-Hall, Inc., Englewood Cliffs, N.J., 1969.4 SIGNALS, SYSTEMS, AND TRANSFORMS 1-1 DISCRETE SIGNALS AND SYSTEMS The notation f[n] will mean a sequence of numbers, real or complex, defined for every integer n. The sequence f[n] will be called a discrete or digital signal, and the index n discrete time. The following special cases will be used often (Fig. 1-1): Step sequence Delta sequence _fl nz0 5 U=i9 pcg = ; nao We note that é[n — 3] equals 1 for n = 3, and 0 for n # 3. For any k, 1 n=k ain—K) =|) ia (1-1) In the above, it is understood that n is the discrete time and k is a constant parameter. From Eq. (1-1) it follows that an arbitrary sequence f[n] can be written as a weighted sum of delta sequences: f{n] = 2 STK] dfn — Kk] (1-2) This is illustrated in Fig. 1-2 Discrete Systems A discrete system is a rule for assigning to a sequence f[n] another sequence g[n]. Thus, a discrete system is a mapping (transformation) of the sequence f[n] into the sequence g[n]. We shall use the notation aln] = L{f[n]} for this mapping. The sequence f[n] will be called the input, and the sequence g[n] the output, or response (Fig. 1-3). In general, to determine the value of the output g[n] for a specific n, we must know the input f[n] for every n, past and future. However, as we see in the following illustrations, this is not always necessary. ue ain] tia Ul Figure 1-1INTRODUCTION 5 fla) = -%lnr i] + 36(n] + bl0— 1] 3 3 1 1 0 n 0 n 0 n 0 n Figure 1-2 Example 1-1 (a) g[n] =f?[n]. This system is nonlinear, and the present value g[v] of the output depends only on f[n] (memoryless system). (b) gln] = nf[n]. This is a linear, memoryless, time-varying system. (c) g[n] = 2f[n] + 3f[n — 1]. The present value g[n] depends on f[n] and the preceding value f[n — 1]. The system has finite memory In the systems (nonrecursive) of Example 1-1, g[n] was expressed directly in terms of f[n]. Example 1-2 alr] + 2g[n — 1] = f[n] In this example, to find g[n] we need to know not only f[n] but also gfn — 1]. Thus, gf] is obtained by solving a recursion equation. We have, in fact, an infinite set of equations, one for each n. As we shall show, under certain conditions (causality), these equations have a unigue solution; therefore, they define a system (recursive). The following simple systems are of particular interest: Delay element Multiplier gt] =ffn—- 1] ofr} = af [n] These systems will be represented by the block diagrams of Fig. 1-4. The letter a in the triangle representing the multiplier is its gain. The significance of the letter z~* (the system function) in the block representing the delay element will be discussed presently. We show later that an arbitrary linear system can be realized by a combina- tion of delay elements and multipliers. As an illustration, Fig. 1-5 gives the realization of the system g[n] = 2f[n] + 3f[n — 1]. Sn ell=L(01) Figure 1-36 SIGNALS, SYSTEMS, AND TRANSFORMS stn} {=f fla) Se fits afta, para larvae tle 0 oO 0 o Figure 1-4’ Linearity A system L is linear if Lia, flr] + 42 fll} = a LE Ln} + a LEfln}} (1-3) for any a,, a5, f,[n], and f,[n]. From the definition it follows that the response to af[n] equals ag[n]. Further- more, if g,[n] and g,[n] are the responses to f,[n] and f,[n], respectively, ‘then the response to f,[n] + f,[”] equals g,[»] + g2[n]. Time-invariance A system L is time-invariant if L{f[n— k]}} = g[n— k] - (4) for any k. In words: A shift of the input results in an equal shift of the output. Example 1-3 (a) The system g{n] =| f{n]| is nonlinear (explain) but time-invariant. () The system g[n] = nf[n] is linear, but time-varying, because the response to fn — k] equals nf{n — k], whereas g[n — k] = (n — k)f[n — Kk]. (c) The system g[n] = 2/[n] + 3/[n — 1] is linear and time-invariant. In the following, the term “linear system,” or simply “system,” will mean linear and time-invariant. The delta response We shall denote by h[{n] the response of a system to the delta sequence d[n] (Fig. 1-6): L{9[n}} = Afr] (1-5) a(n) = 2ftn] + 3fln — 1) Figure 1-5INTRODUCTION 7 L thi 0 a(n} AU] Figure 146 We note that the sequence A[n] is not necessarily zero for n <0. If h{n]=0 for n 0. By an easy induction, we find bn] = (2)"ULn] (1-9) In Chap. 2 we develop simpler methods for determining hn]. In Fig. 1-7 we show a block-diagram realization of the above system. We note that the two systems in (1-7) and (1-8) have the same delta response A{n]; hence, they are equivalent. That is, they yield the same response to the same© SIGNALS, SYSTEMS, AND TRANSFORMS hin} 0 ala] = etn - 11 tytn) Figure 1-7 input [see (1-11)]. The system in (1-7) is nonrecursive, but an infinite number of delay elements is needed for its realization. The system in Eq. (1-8) (recursive) is realized with one delay element only. Discrete convolution We shall express the response g[n] of a linear system to an arbitrary input f{n] in terms of h[{n] and f[n]. We note that the response to d[n —k] equals A{n — k] for any k (time- invariance): L{6[n — k]}} = h[n — k] (1-10) Hence, the response to f[k] d[n — k] equals f[k]h[n — k] (linearity). From the above and (1-2) it follows that alm] = L{f[n}} = 2 J HLon — A= sean — 4] = kere The last sum is the ae convolution of f[n] with h{n]. This operation will be denoted by f[n] * g[n]. As it is easy to see, it is commutative. We have, thus, reached the important conclusion that lr] = fn] » Afn) -5 SUkh[n = k} = . fln—kjh{k] (1-11) Example 1-7 Osn<4 otherwise Alm] = &)"'U[n]—f[n] = Ulm] - Ufn - 4] = Find gln] = fn] « h[n] for n= 2and n= 5. To find g[2], we multiply f[k] by h[2—k] and sum for all k. As we see from Fig. 1-8a, [2] = F(2)A{0] + SUIAL!] + f[O}A[2] = 1 +445 Similarly, 5] = S(3)AL2] + 12)H(3] + SCALA] + f[O]ALS] = (4)? + (4)? + (4)* + (4)?INTRODUCTION 9 | tk) | fk} 0 A 0 A | h{2 ~ ky ALS AT | fe wl 2 0 k @ (by Figure 1-8 We note that, if h[n] = 0 for n <0, then abr] = 3 steam a) = X sl — KH (1-12) ke=n@ If, also, f[n] = 0 for n < 0, then g[n] = 0 for n <0; for n > 0 it is given by alm] =X fUeALn = k= ffm = KALK] (1-13) k=0 k=0 In particular, 0] =F(O}R(0] oft) = FLO)AL 1] + FL1A[0} gl2] = FLOMM(2] + fL1AL1] + £(2)AL0] Example 1-8 f{n] = Ufn] - Ulm = 3] = Afr] Find g[n]. It is easy to see that g[n] = 0 for n < O and n> 4. ForO0 1 t>0 1 |t|aINTRODUCTION 13 Figure 1-12 The delta function 5(t) This important concept will be discussed in Appendix 3-C. We note here only that 5(t) can be viewed as the limit of a family of functions S{t) such that [ podar [> Lee at 00) (1-19) aed coe co for any function g(t) continuous at the origin. This interpretation of 6(t) leads to the identity J 8) ear = 90) (1-20) from which all formal properties of 6(t) can be deduced. Analog Systems An analog system is a rule for assigning to a function f(t) another function g(t). A system is thus a transformation mapping the input f(t) to the output or response: a(t) = LiF ()] Linearity A system L is linear if La, fil) + a, £0) = LAO] + 2 LEA (] (1-21) for any a,, a3, f,(t), and f,(t). Time-invariance A system L is time-invariant if L[f(t — to)] = a(t ~ to) (1-22) for any real t,. Example 1-11 (a) g(¢) = | f(0) (rectifier): nonlinear, time-invariant (b) g(t) = ef (0): linear, time-varying (c) g(t) =f(e — a) (delay line): linear, time-invariant Note: If a system is linear and f(t) = 0, then g(t) = 0, because then f(t) = 2f(t); hence, g(t) = 2g(t). If, however, f(t) =0 for t < ty only, it does not follow that (tg) = 0 because g(t)) depends on all values of f(t), past and future.14 SIGNALS, SYSTEMS, AND TRANSFORMS Causality We shall say that a function f(t) is causal if S(t)=0 for t<0 We shall say that a system is causal if a causal input yields a causal output. Thus, a causal system has the following property [see Eq. (1-22)]: If f(@)=0 for tty then = g(t) =Ofort 0 only and g(0) = 0 (zero initial conditions), then, setting f(r) = 0 for t < 0, we can assume that it holds for all t. The system so defined is real if « is a real number. In Fig. 1-13 we show the solution of (1-25) for two special forms of f(t) of unit area: 1 {au-e%) Oc ac fl = 1 eu a= paleo") > — ae It is of interest to observe that, if c is small, then 9, (t) = g2(t) = e-“U(t) for t>cINTRODUCTION 15 a ey a 20) fy 0 ¢ (a) () ° Figure 1-13 Thus, as c 0, the response reaches a limiting value e~*'U(c) that is the same for both inputs. This important observation leads to the concept of the impulse response. The impulse response To an arbitrary linear system L we apply a sequence of inputs £0 = L4(¢) (1-26) of unit area as in Eq. (1-19). The resulting response g,(t) is a signal that depends on the system L, the form f(t) of the input, and the scaling factor c. It can be shown [see (4-9)] that, as c + 0, g,(t) tends to a limit: LLL()] = Gl) + h(t) e+ 0 The limit h(t) depends on the system L but it is independent of the form of the input, as long as its area equals 1. Since f(t) tends to d(t) [see Eq. (1-19)] we shall say that A(t) is the response of L to the delta function 6(t): h() = L{(0)] (1-27) The function h(t) we shall call the impulse response of the system. From the above it follows that if the input to a system is a signal f(t) of arbitrary form but of sufficiently short duration’ relative to A(t), then the resulting response is approximately equal to Ah(t), where A is the area of f(t). Iff (t) takes significant values not near the origin but near the point tg as in Fig. 1-14, then (time-invariance) the approximate response will equal Ah(t — t9). Example 1-12 The equation g'(t) + a9(t) = f(t) defines a linear, causal system with impulse response e~*U(t). If f(t) is a triangle of area A = Be (as in Fig. 1-14) and ¢ < 1/a, then g(t) = Ah(t — to) = Bee" b> ty $e ‘The expression “f(e) is of short duration relative to h(e)” will mean loosely that f(t) is negligible for |r| > ¢ and h(t) is approximately constant in any interval of length 216 SIGNALS, SYSTEMS, AND TRANSFORMS (0) Beh(t ~ t) 7 Be eee fey sO +08) =f 0 fo t ° Figure 1-14 Convolution We shall show that the response g(t) of a linear system L to an arbitrary input f(t) is given by S(t)h(t — 2) de (1-28) Proof The function f(t) can be written as a sum of elementary functions f,(t) as in Fig. 1-15. Denoting by g,(t) the response of L to f,(t), we have (linearity) fM=LAQ a(t) = Dail) (1-29) § If At is sufficiently small, then the area of f,(t) equals f(c,) At. Hence, the resulting response is approximately f(t;) At h(t — t,) because f,(t) is concentrated near the point t;. With At > 0, we thus conclude that Laie) ~ LS (sme ~ 4) def” Fle\n(e— 2) de and (1-28) follows. The integral in Eq. (1-28) is the convolution of the input f (¢) with the impulse response h(t): alt) =e he) =f Seone—Dde= [fle oMe)de (1-30) If the system L is causal, then h(t)=0 for t<0 (1-31) £0) 1a) Ti ’ Figure 1-15INTRODUCTION 17 a(r) hty 7) fir) ht, = 9) 0 + Figure 1-16 because 6(t) = 0 for t < 0. In this case, (1-30) yields 1 © g(t) I Sf (t)h(t — 2) dt = J, f(t — dhl) de (1-32) If also f(t) = 0 for t < 0, then g(t) = 0 for t < 0; for t > 0 it is given by g(t) =| S(t)h(t — 2) de = J S(t = h(t) de (1-33) ° ° The evaluation of the convolution integral is often facilitated by a semi- graphical method: To find g(ty) for some t = ty, we form the function h(—7) and its displacement h(t, — t). The area of the product f(t)h(t) — t) yields g(t,). Example 1-13 We wish to find the convolution g(r) of the two functions f(t) and h(t) of Fig, 1-16. As we see, h(t) is causal and f(t) is zero outside the interval (a, b). From the figure it follows that g(t) = 0 for = t, b We note that, if f(¢) = 0 outside the interval (a,, b,) and A(t) = 0 outside the interval (a,, b,), then g(t) = 0 outside the interval (a, + a,b, + b). Example 1-14 If f(t) = U(t — 1) — U(t — 3) is a pulse as in Fig. 1-17, and h(t) = f(t), then g(¢) is a triangle. f(y hte) a(t) m= My Figure 1-1718 SIGNALS, SYSTEMS, AND TRANSFORMS, Output center of gravity We denote by "pdt om =| f(t) dt 1, = (1-34) ae i 7 the area, first moment, and center of gravity, respectively, of f(t). We define the corresponding quantities for h(t) and g(t) similarly. Theorem If g(t) = f(t) * A(t), then A,=AypA, y= "yp + (1-35) Proof Integrating both sides of Eq. (1-28) and changing the order of integrations on the right side, we have, with t — t = x, fl atede= J fee) "Me a)dede= J sear] hx) dx Hence, A, = A; A,- aaa [ tg(t) a= fo se) f th(t — 1) dt =f ref (x + t)h(x) dx dt = in f(t) dt la xh(x) dx +f af (2) af lie dx Thus, m, = A,;m, + Aymy, Dividing both sides by A, In Example 1-14, A, A, A,, we conclude that y, =, +n,. A, = 2,and A, = 2; also, 9, = 4,1, = 2, and y, = 2. The System Function Suppose that the input to a linear system is an exponential F(t) = 2%" As we see from (1-30), the resulting response is given by ay={- eo h(z) dr = eieot U 7! h(t) de (1-36) =~ -o Thus, g(t) is also an exponential multiplied by the value H(w) of the Fourier transform H(w) = { h(t)e~4 dt V(1-37) of h(t). Since this is true for every wy, we conclude that Liel) = H(w)el (1-38)INTRODUCTION 19 Ho) sha Figure 1-18 We shall call the factor H(w) the system function. It can be determined either from (1-37) or from (1-38). To determine H() from (1-38), we apply the input f(t) = e/". The function H(w) equals the coefficient of the resulting response H(w)e*. In general, H(c) is complex with amplitude A(w) and phase (w): H(o) = A(w)e™ (1-39) Example 1-15 (Smoothing) Consider a system whose impulse response h(t) is a pulse as in Fig. 1-18. From Eq, (1-37) it follows that the corresponding system function is given by H(o) = It is easy to see that 1 | = = 1-4 9) =F (0) * 52 Pall) = 5 (1-40) Example 1-16 From (1-37) it follows that if h(t) = e7*'U(0), for « > 0, then 7 1 =f ene Je dy = —— 1-41 H(o) J, ee goa (1-41) Example 1-17 (Differentiator) Consider a system whose output is the derivative /“(t) of the input. If f(t) = e, then a(t) = f'(¢) = joel Hence, H(w) = jo. Example 1-18 The differential equation g(t) +ag(t)=s(t) allt (1-42) specifies a causal system with input f(t) and output g(¢). As we see from (1-38), if f(t) = 2", then g(t) = H(w)el" Inserting into (1-42), we obtain JooH (w)e™ + all (wel = ei Hence, H(w) = ——20 SIGNALS, SYSTEMS, AND TRANSFORMS From (1-41) it follows that the inpulse response of the system equals e~*U(t). Therefore, the solution of (1-42) is given by g(t) = en* j f (t)e* de Real systems The system function of a linear system is, in general, an arbitrary function H(w) of the real variable (frequency) w. However, if h(t) is real, then H*(o) = { h(t)ei*" de = A(w)e7 4 [see Eq. (1-37)]. But the above integral equals H(—.); hence, H*(o)=H(-@) — A(-@) = Al) —g@(-0) = -e(w) _(1-43) Thus, the system function of a real system has even amplitude and odd phase; therefore, if it is known for w > 0, then it is known also for w <0. The response of an arbitrary system, real or complex, to elot 4 pm jot 2 H(- HU=0) S(t) = cos wt = Hl de equals g(t) = eft From (1-43) it follows that, if A(t) is real, then a(t) = Re [H(a)e*™] This follows also from Eq. (1-24): cos wt = Re ei Hence, g(t) = Re [H(w)e!] = Re [A(co) ei” eior] We thus conclude that L{cos wt} = A(w) cos [wt + o(o)] (1-44) Example 1-19 We wish to solve the equation g(t) + 2g(t) = cos 3¢ As we have seen, g(t) can be considered as the response of a system with input cos 3¢ and system function 1/(2 + joo). Since 1 1 H(3) = = LL g-iuant 372 (3) 24+ SB it follows from (1-44) that (= ses (3¢— tan JBINTRODUCTION 23 Kaede ° Figure 1-21 Example 1-20 An ideal low-pass filter is a system with constant amplitude A(w) and linear phase g(w) in the interval (—@,, @,): H( Palarjen #02 From (1-53) it follows that its impulse response is given by in w(t 27 10% phat dey = 0 as in Fig. 1-22. If the input is a step function f(t) = U(t), then the resulting output, denoted by a(r), is the step response of the system. As we see from (1-30), a=] We) de = dr=s+ sin @,(t I . -a B&G) 272, x edt=t0) ro sin x —d. Ix We note that, if f(t) = cos wyt, then g(t) = A(w,) cos w4(t — t,) [see Eq. (1-44)]. Therefore, if > w,, then g(t) = 0; if @) <@,, then g(t) = cos w(t ~ t4). The preceding theorem expresses an arbitrary function h(t) in terms of its Fourier transform H(«) defined by (1-37). The assumption that A(t) is the impulse response of a system was introduced merely to derive (1-53) from (1-48) in a simple way. Applying the theorem to the output g(t) of a system and its transform G(w) = F(w)H(), we obtain g(t) = zl, G(o)e! do = x/ 7 F(o)H(w)e dw (1-54) Periodic inputs It is well known that a periodic function with period T can be written as a sum of exponentials (see Fourier series, Sec. 3-2) at f= (1-55) Figure 1-2224 SIGNALS, SYSTEMS, AND TRANSFORMS 1.72 where 4,== { f(te7 2% dt (1-56) -T/2 From (1-38) it follows that the response of a linear system to a periodic input f(t) is given by (1-57) This formula is useful only if a small number of terms in the above sum are significant. Suppose, for example, that the system is a low-pass filter, as in Example 1-20. If @ < w, < 2a, then (1-57) yields g(t) = ag + a,el06-t0) 4 g_e~svottt0) If H(q) is not narrow band, or, equivalently, if the duration of h(t) is of the order of T (see Sec. 8-2), then g(t) is best determined from the convolution integral, Eq. (1-28). The Poisson sum formula We have used the notion of a system to prove the convolution theorem (1-47)and the inversion formula (1-53). Proceeding similarly, we shall prove the Poisson sum formula (1-59). This important identity will be used repeatedly. The impulse train ¥ a(t +n) (Fig. 1-23) is a periodic function with Fourier-series coefficients 7/2 ae 4] 5(t) eo de = b -T/2 This follows from Eqs. (1-20) and (1-56) and from the fact that, in the interval (—T/2, T/2), the above sum equals 6(t). Inserting into (1-55), we obtain oo 12. Y st + nT) == PY einoot (1-58) ns-@ ® 4 Bit +0) E yesnn Ei oO T LS ginwor z we =p E emo Figure 1-23INTRODUCTION 25 Consider an arbitrary function y(¢) with Fourier transform vo Y(w) = | y(te2" de -« Using (1-58), we shall show that 2 y erat) at Y ¥(nw,)emot (1-59) where T is an arbitrary constant and wo n/T. Proof We form a system (Fig. 1-23) with impulse response y(t) and system function Y(q). The response of this system to the left side of (1-58) equals the left side of (1-59), and the response to the right side of (1-58) equals the right side of (1-59). Hence, (1-59) is a consequence of (1-58). We note that the left side of (1-59) is a periodic function and its Fourier- series coefficients equal the sample values ¥(nw,)/T of Y()/T. 1-3 DIGITAL SIMULATION OF ANALOG SYSTEMS Consider an analog system H,(«) with input f(t) and output g(t). We wish to find a discrete system H(z) such that, if its input f[n] equals the sample values J (nT) off (t), then the resulting output g[n] equals the sample values g(nT) of g(t): If finJ=s(nT) then g[n] = g(nT) (1-60) If such a system exists, then we shall say that it is a digital simulator of H,(). As we shall see, (1-60) cannot, in general, be true for any f(t). The simulation is possible only if the class of inputs is restricted. Indeed, suppose first that f(t) is an exponential. As we know [see (1-36)], g(t) is, then, also an exponential (Fig. 1-24): S(t) = el" g(t) = H,(ao)e”"" (1-61) eloot H, (429 eo! H,(w) 7) a) 2 Ko) gla =g(n7) ° a flr} = find Hat) He) ala) choot Hobs ems Bove 1.0426 SIGNALS, SYSTEMS, AND TRANSFORMS The sample values f (nT) of f(t) form a geometric progression with ratio e/°°7 Sn] =f (nr) = enor Hence, the response g[n] of the discrete system H(z) is given by alr] = H(i? emoot (1-62) This follows from (1-14) with r = e/@T Since g(nT) = H,(w,)e"”, to satisfy (1-60) we must choose H(z) such that He") = Ho) (1-63) Consider next an arbitrary signal with Fourier transform F(w). From the inversion formula it follows that so=z] Flojedo g(t) =5-[ ” Fo)H,(a)e™ da (1-64) Since S{n] =f (nT) = Fall F(w)eT deo we conclude as in (1-62) that the Saal of H(z) is given by ale] = J Floynte*)em” de (1-65) But (see (1-64)] . g(nT) = x F(w)H,(w)e"°T dor (1-66) Hence, for (1-60) to be true, the integrals in (1-65) and (1-66) must be equal for every n. This is the case if H(e*") = H,(w) (1-67) for every w for which F(c) # 0. If, therefore, (1-60) holds for every f(t), (1-67) must hold for every w. However, this is not always possible because H(e“’) is a periodic function of « whereas, in general, H_,() is not. To satisfy the simula- tion condition (1-60), we must restrict the class of inputs. Simulation theorem If f(t) is bandlimited, i.e., if F(w)=0 for Jol>o c= (1-68) = T and H(z) is such that H(e*")=H,(w) for = jal o (Fig. 1-25). Denoting by h,(c) its impulse response, we have h,(t) = xf H,(w)e dw (1-72) and Eq. (1-71) yields h(n] = Th, (nT) (1-73) From the preceding discussion it follows that to determine the discrete simulator H(z) of the analog system H,(«), we proceed as follows: 1. We evaluate h[n] by sampling the impulse response h,(t) of the truncated system H,(w). 2. Or, we specify H(z) in terms of its values on the unit circle [see (1-69)].28 SIGNALS, SYSTEMS, AND TRANSFORMS, Example 1-21 We wish to simulate a differentiator, ic, an analog system with system function H,{o) = jo ot cos ot — sin ot Ly Since b= 57 | joe do 7 a (1-74) (1-73) yields h{n] = Th,(nT) = for n #0 and h{0] (-1" nT Note: As we have shown, if F(w) = 0 for |w| > a, then g(nT) = f'(nT). This leads to the identity f'(nT) = S (oO por —kT) (1-75) expressing the derivative ofa bandlimited function in terms ofits sample values. Example 1-22 We wish to simulate the ideal low-pass filter H,(o) = p,,() with cut-off frequency w, < o (Fig. 1-26). In this case, H,(w) = H,(w) because H,(w) = 0 for |w| > o. Hence, sin @,t sin, Tn h(t) = h,() = h[n] = Th,(nT) mi Comb filters We shall say that an analog system is a comb filter if its system function H,(q) is periodic: H,(@ + c) = H,(o) (1-76) Such a system can be simulated digitally for any input provided that T = 2n/c, thatis,o = c/2. Indeed, if H(z) is determined from Eq. (1-69), then H(e/*7) = H,,(«) for all w. H,lw) ~o “we Oe ow Figure 1-26INTRODUCTION 29 V(o= 1H(e!2T 1 Figure 1-27 Example 1-23 The equation 1 plato — alt — T)) + aa(t) = £10) (1-77) defines a system with input f(t) and output g(t). If f(t) =e, then g(t) = H, (we. Inserting into (1-77), we have z H,(w)e™ — z H, (ae? + aH (coe = el" Hence (Fig. 1-27), where and eee Ppresesnciasts "aT +l aT +1 Clearly, H,() is a comb filter with period 2n/T and its digital simulator is given by H(t) = 4) : “ {ont Example 1-24 If H,(c) = (1 + e747)? then H(z) = (1 + 27')?and h{n] = d[n] + 25[n — 1) + 5[n — 2]. In many applications, it is desirable to have a system whose system function H,,(@) meets certain design requirements. The system is realized either by a combination of various elements, lumped or distributed, or by a computer. The first approach involves network synthesis techniques; the second approach is based on the digital simulation theorem. In Sec. 5-3 we rederive this theorem and discuss various methods for approximating the discrete system H(z) with a system that can be realized with a finite number of delay elements.CHAPTER TWO DISCRETE SYSTEMS Continuing the discussion of Sec 1-1, we develop in this chapter the properties of discrete systems, stressing clarification of the main concepts rather than detailed coverage of specific applications.':* In Sec. 2-1, we give a self-contained and, for our purposes, complete presentation of the theory of z transforms. To avoid any reliance on the theory of complex variables, we derive the inversion formula in terms of Fourier series, thus demonstrating the connection between z transforms and Fourier series. In Sec. 2-2, we show that a system that can be realized with a finite number of delay elements and multipliers is equivalent to a recursion equation of finite order with constant coefficients. We discuss the solutions of such equations. In Sec. 2-3, we cover analysis and synthesis of finite-order systems with a brief discussion of the properties of their frequency response. The approximation problem is considered in Chap. 5. ‘A. V. Oppenheim and R. W. Schafer, “Digital Signal Processing,” Prentice-Hall, Inc., Englewood Clifis, N.J., 1975. 2 L.R. Rabiner and B. Gold, “Theory and Applications of Digital Signal Processing,” Prentice- Hall, Inc., Englewood Cliffs, N.J., 1975. 30DISCRETE SYSTEMS 31 2-1 z TRANSFORMS The series o F()= YO flnje" (2-1) nzm@ establishes a correspondence between the sequence f[n] and the function F(z). We shall use the notation fln}o F(z) for this correspondence. The function F(z) will be called the z transform of f[n]. Example 2-1 (a) If fn] = d[n — k], then F(z) = 27% (b) If f{n] = 3 d[n — 2] +2 d[n — 5] as in Fig, 2-1, then F(z) = 327? + 2275. The z transform F(z) is defined only for the values of z, real or complex, for which the series (2-1) converges. As is known from the theory of functions of complex variables, the region R of convergences of the Laurent series (2-1) is a ring r, <|z| + co and — oo, respectively. In this ring, F(z) is an analytic function of z, that is, it has derivatives of any order. The poles or any other singularities of F(z) are outside the region R. If f[n] = 0 for n <0, then r, = co because (2-1) has only negative powers of z. In this case, R = R, is the exterior |z| >r, of the circle of Fig. 2-2b. If f[n] = 0 for n> 0, then r, = 0 because (2-1) has only positive powers of z. In this case, R = R, is the interior |z| 1 of the unit circle (Fig. 2-3). j-ln<0 qin | 0 n20 i (6) f[n] = -U[-2- t= fla) Fz) = 32-2 + 22-5 2 0 2 5 m Figure 2-132. SIGNALS, SYSTEMS, AND TRANSFORMS Figure 2-2 The sum converges for |z| <1; hence, the region R, of existence of F(z) is the interior |z| <1 of the unit circle. Note: The z transforms of the above two sequences U[n] and —U[—n — 1] have the same algebraic expression z/(z — 1). However, the corresponding regions R, and R, of their existence are different. Example 2-3 (a) From the identity |z] >a it follows that Un] > — jz) > oo a°U[n] > z2|>a (b) Differentiating Eq. (2-2) with respect to a, we obtain 2 Zz nan 127" = — = & (z-a)* Hence, nat Ub) + |z|>a f{n)} = Uln} fin] = -Ul-n -1) Figure 2-3 (23)DISCRETE SYSTEMS 33 (c) Differentiating (2-2) m times with respect to a, we find Yo nln =A) (n= m+ Varo n=0 mlz Therefore, ( ve "et le] >a (2-4) m (=a Inversion of the z Transform We wish to find a sequence f[n] whose z transform equals a given function F(z). If F(z) is specified by a series expansion F(z) = y c{njz-" ze R” (2-5) n=—@ then f[n] is unique and it equals c[n]. Example 2-4 If F(z) = 3z + 427? — 527°, then f[—1] = 3, [2] =4, f[10] = —5, and f{n] = 0 otherwise. If F(z) is specified by an algebraic expression, its inverse is not, in general, unique. Indeed, as we have seen in Example 2-2, the function z/(z — 1) is the z transform of two different sequences. To have a unique inverse, we must know not only F(z), but also the region R of convergence of the series (2-1). If it is known that F(z) is the transform of a causal sequence, then, its region of con- vergence is the exterior of a circle containing all its poles. We shall discuss three methods for determining f[n] from F(z). 1. Series expansion We expand F(z) into a series as in (2-5). The unknown numbers f[n] equal the coefficients c[n] of this expansion. Since F(z) has a unique expansion in a ring R, its inverse f[n] is uniquely determined if R is known. Example 2-5 It is easy to see that iz is a at : ~22" oer es Parr delete seated deep En S| 3) The first expansion is valid for |z| > 2, and the second for |z| <2. Hence, -2y"§ nab W-peronss sel = hte =| mee or sda taf” *S? The inverse of F(z) is f,{n] if R is the region |z| > 2; the inverse of F(z) is fy{n] if R is the region 0 < |z| <2 (interior of the circle |z| = 2 excluding the origin).34 SIGNALS, SYSTEMS, AND TRANSFORMS 2. Tables If the function F(z) can be written as a sum F(2) = ¥ a; F (2) (2-6) where F,(z) are functions with known inverses f[n], then (uniqueness) fin) = La, flr] Example 2-6 From the identity z =2 p21 Pearcy we conclude that the inverse f[7] of the function z3/(z — 1) in the region R = R, is given by (see Example 2-2) S(n] = d[n + 2] + [nm + 1] + Ulm] = Uln + 2] The above method is of particular interest if F(z) is rational. Rational transforms A rational function is the ratio of two polynomials of z: F(2)=5 (2-7) or, equivalently, of z~!. We prefer to use polynomials of z. We can assume, if necessary removing a polynomial from F(z) by synthetic division, that the degree of N(z) does not exceed the degree of D(z). We shall first determine the causal inverse f[n] of F(z) using the pair z/'U[n]o Z— for the simple roots, and the pair (2-4) for the multiple roots. Suppose that all the roots z; of the denominator of F(z) are simple. As is well known, the proper fraction F(z)/z can be written as a sum F(z) _ x A, where aa F(2\(z - 2) z 1272 z z= 2; Hence, (2-8) Example 2-7 z+2 1 FO=sa aes TZ 23 F(z)_23 1 1/3 flr] ot + otal - (5) +t x *U[n) 12 Multiple roots can be handled similarly. We give only an illustration.DISCRETE SYSTEMS 35 Example 2-8 =F 2 4 (@-1F 2 12> 1 ‘As we see from (2-4), the inverse of z/(z — 1)? equals nU[n]. Hence, f(r] = [4G)" + 2n — 4]ULn] Noncausal inverse The rational function F(z) has a causal inverse f[n] given by Eq. (2-8) if the region of existence of F(z) is the exterior R, of the circle |z| =r, = max |z,| containing all poles z, of F(z). We shall now determine the inverse of F(z) assuming that R is an arbitrary ring as in Fig. 2-4. Example 2-9 (a) From the identity lz] 3, all poles are interior; hence, S(n] = 3 [nm] — G)"ULn] + § x 3°U[n] (b) In the ring $ < |z| <3, the pole p, = 4 is interior, and the pole s, =3 is exterior; 1 1 hence, S{n] = 4 o[n] — GYULn] - $x 3°U[-n~ 1] (c) In the region |z| <4, both poles are exterior; hence, f{n] = $ 6[n] + GUL —n — 1] — 5 x 3°UL-n- 1] 3. The inversion formula With z = e/7, it follows from (2-1) that Feit) = Stet Thus, on the unit circle, F(z) is a periodic function of with period 2n/T and Fourier-series coefficients f[n]. Hence, s(n) = x { Flee etT dono = (2-12) Note: For arbitrary sequences, the constant T has no particular significance and can be assumed to equal 1. In this case, w is the phase of z and o =x. If f[n] is obtained by sampling an analog signal as in Eq. (1-71), then T equals the sampling interval. Complex form If z = e/”T, then dz = jT eT dw = jTz dw Hence, (2-12) can be written as a contour integral fr] ii f Fee “dz (2-13)DISCRETE SYSTEMS 37 where C is the unit circle. Thus, to compute f[n], it suffices to evaluate the above integral. This can be done with the calculus of residues, which for rational functions involves the fraction expansion (2-8). Theorems 1, Shifting. If f[n]<> F(z), then for any integer m, f[n =m] oz F(z) (2-14) Proof Y slw= ment = sere = =F) n=—e@ kee Example 2-11 Since z3/(z — 1) = z*-z/(z — 1) and the inverse of 2/(z~ 1) equals U[n}, it follows from (2-14) with m= —2 that 3 Uln +o ; 7a 2. Convolution. If f,[n] + F, (z) and f,[n] — F(z), then E AUaln = HF 2)F (2) (2-15) Proof The above was derived in (1-18). It can also be proved by multiplying the series expansions of F,(z) and F,(z). We shall carry out the details for causal sequences: F(z) =f,[0] +f[z7! +--+ fn" + F(z) =f,[0] + fz"! + + A[nJe" + Fi(2)F2(z) = All Al0] + (ALN AL + AL AL)2"* + + (3 sagan K=O The right side is the series expansion of F,(z)F,(z); hence, its inverse is the coefficient of z~". Example 2-12 Using z transforms, we shall find the sum of the integers from 0 to n and the sum of their squares. = a Zz (a) Aled = Ek = (Ula) « Ube gt GaP But mest) Ufo (z-1)38 SIGNALS, SYSTEMS, AND TRANSFORMS. Hence (shifting), filn] = Fk? = | + Ufo 22 0) flo = 3 = PUA) + UE Ee because r?U[n] = [2 mn= 1), not eas + oF n z But (3) bel eae Hal Aled = (n +2) ie + im (nt a =1)_Qn+ iN + 1)n 3. Conjugate sequences. If f[n] + F(z), then st-aeor(5) (2-16) Proof 2 Stone Sse Notes: If the sequence f[n] is real, then F*(1/z*) = F(1/z). If F(z) converges in the ring r, < |z| Gage F X(z) and y[n]<> Y(z), then for any T = x/o, Y xbobtbo) = 5 J Xe TY) de (2-18) n= =o Proof From (2-15) and (2-12) it follows that SAMAK = J F(R eer do 19) k=-@ -o If f,[n] = x{n] and f,[—n] = y*[n], then F,(z) = X(z) and F,(z) = Y*(1/z*), and (2-18) follows from Eq. (2-19) with n= 0.DISCRETE SYSTEMS 39 Corollary From (2-18) it follows, with x[n] = y[n], that "| x(el#T) 2 deo (2-20) 5. Initial and final value. If the sequence f[n] is causal, then F(z) exists for |z| > r, and, as we see from (2-1), f[0] = F(co). The following is a generalized converse of the above: If F(z) exists for |z| > r, and if for some integer m, positive or negative, lim z"F(z) = A < 0, then f[m]=A and f[n]}=0 — forn 1. Hence, f[—2] = 1 and f[n] = 0 for n< —2. The above theorem relates the behavior of F(z) for large z to the “initial value” f[m] of f[n]. The following result relates the behavior of f[n] for large n to the radius r, of the region R,. If F(z) exists for |z| > r, and r, < 1, then S[n]>0 forn>c (2-22) This theorem is a consequence of the following property of Laurent series: If the series (2-1) converges for r, < |z| 0, then the unilateral transform of f[n — m] equals 2"Fi(z) + f[- Leo? + f[- Ze? + + fm] (2-25)40 SIGNALS, SYSTEMS, AND TRANSFORMS, Proof With n — m= k, we have Eifln— mje’ = S steer te = fom) to sane LEMS flkem# (2-26) and (2-25) results because the sum on the left is the unilateral transform of the sequence f[n — m] and the last sum on the right equals F(z). 2-2 RECURSION EQUATIONS A recursion equation of order m is a relationship between two sequences f[n] and g[n] of the form g{n] + a,g[n — 1] +++ +4, [n — m] = by f[n] +--+ +b, f[n— mJ] (2-27) We shall assume that, if f[n] = 0 for n < n,, then g[n] = 0 for n 0. (6) ff] = Un] — Un — 3] = an] + dfn — 1) + dfn — 2] alm] = f[n] + An] = (—2)"U fn) + (—2)" Ufa = 1] + (= 2)" 2U[n - 2] Hence, g[0] = 1 and g[1] = —1, and g[n] = 3(—2)" for n> 2. The solution (2-29) was obtained under the assumption that (2-27) was true for all n. Suppose now that (2-27) is true for n>0 only. To determine g[n] for n > 0, we need to know not only f[n] but also the values (initial conditions) aA-m of-m+ 1} of 1] (2-31) of g[n]. If these values and the corresponding values f[—m], ..., f[—1] of f[n] are zero, then the solution g[n] of (2-27) is given by (2-29). Indeed, setting g{n] = 0 and f[n] = 0 for n < 0, we conclude that (2-27) is true for all n; hence, (2-29) holds. Example 2-16 The sequence f[n] and g[n] in Fig. 2-5 satisfy the recursion equation alm] — 3af" - 1) = SE] (2-32) Figure 2-542. SIGNALS, SYSTEMS, AND TRANSFORMS, The switch S closes at n=0, and the input f{n] =5 for n>0. We wish to find g{n] under the assumption that g[—1]=0. We know that Eq. (2-32) holds for n > 0; setting f{n] = 5U[n] and g[n] = 0 for n <0, we can assume that it holds for all n. Hence, we can use (2-29): Fe) Sz®@ Sz, 102 S@)= 7-72" G-he-y 2-47 2-1 From the above we conclude that g[n] = —5(4)" + 10 for n> 0. If (2-27) holds for n = 0 only and the initial conditions are not zero, then we can no longer use (2-29). To determine g[n], we must apply unilateral z transforms and the modified form (2-25) of the shifting theorem. We illustrate with two examples. Example 2-17 ga{n] — Sofn— 1] + 69[n—2]=1 n20 gf-1]=3 g[-2])=2 We take unilateral transforms of both sides of the equation. Since the unilateral transform of f[n] = 1 equals 2/(z — 1), we conclude from (2-25) that Gy(z) ~ S{27-*Gy(z) + of — 1} + 6f2-7Gy(z) + 27 of 1] + of - 2} = 1827? z/| 1) 3284622) 1-527 + 627? Hence, The first fraction is due to the initial conditions, and the second to the input f[n]. Expanding into partial fractions, we find 1 9 G2) = gin] =5+8x2—-5x3" 20 92/2 3 Example 2-18 (a) Show that if g[n] + a,g[n — 1] + a, g[n — 2] = 0 for n > 0, then g[n] = Az," + Bz," where z, and z, are the roots of the equation z? + a,z + a, =0. Proof Taking unilateral transforms and expanding into partial fractions, we find _ral= Maz +42) a,2%f-2]_ Az Bz 1 2 +az+a, Zz G,(2) eae: The constants A and B can be expressed in terms of g{— 1] and g[—2]. (b) Show that if g[n] — 2ag[n — 1] + g[n — 2] =0 forn >0 and a> 1, then alr] =Ccoshan+Dsinhan where cosha=a