Chapter 2 1 SOLUTIONS TO CHAPTER 2 Background 2.1 The DFT of a sequence x(n) of length N may be expressed in matrix form as follows: X= Wx where x (0), x(1), x(N ~1)]" is a vector containing the signal values and X is a vector containing the DFT coefficients X(k), (a) Find the matrix W. (b) What properties does the matrix W have? (©) What is the inverse of W? Solution | oertsnerner seers : (a) The DPT of a sequence x(n) of length N is XQ) = Fo a(mpe dH = > a(nyvgt where Wy = e1¥, If we define wll = [Lk Ww... WHO] ‘then X(k) is the inner product X(8) = wih -x Arranging the DFT coeficients in a vector we have, X(0) wilx x) wilx ae le =Wx ‘X(N -1) wiix where wi 1 1 1 we Wy WR we w a Lowy) wa yg (b) The matrix W is symmetric and nonsingular. In addition, due to the orthogonality of the complex exponentials, wllema Slevten {Ns tel fer 0 3 kel i Soon th W la oeagnal (€) Due to the orthogonality of W, the inverse is woProblem Solutions 2.2 Prove or disprove each of the following statements, (a) The product of two upper triangular matrices is upper triangular. (b) The product of two Toeplitz matrices is Toe (6) The product of two centrosymmetrie matrices is centrosymmetrie. Solution (a) With an | Ho tan it follows that if A is upper triangular then aj) = 0 for all (: ‘Therefore, b is orthogonal to b+. The matrix PX is a projection matrix that projects a vector nto the space that is orthogonal to the space spanned by the columns of A.Chapter 2 7 2.5 Consider the problem of trying to model a sequence (x) as the sum of a constant plus a complex exponential of frequency wy, A(n) = eta 5 n=0,1,..,.N=1 where ¢ and a are unknown, We may express the problem of finding the values for ¢ and a as one of solving a set of overdetermined linear equations a 1 2(0) te |rey_| 0 fe n=) (a) Find the least squares solution for © and a. (b) If N is even and wy = 27k/N for some integer k, find the least squares solution for ¢ and a. Solution (a) Assuming that wy # 0,2n,..., the columns of the matrix a 1 ee A 1 cil How are linearly independent, and the least squares solution for ¢ and a is given by [<]-cray tame yo Sem ao Sem ow ‘Therefore, the invorse of (AMA) is (aay N $ (njeomeProblem Solutions which becomes 1 = eiNen HI NS ofa) — ae sane WSS xen ee SS sand ‘Therefore, we haveChapter 2 9 2.6 It is known that the sum of the squares of n from n = 1 to N~1 has a closed form expression of the following form V+ agN? + agN* Given that a third-order polynomial is uniquely determined in terms of the values of the polynomial at four distinct points, derive a closed form expression for this sum by setting up a set of linear equations and solving these equations for ao, a1,@2,a3. Compare your solution to that given in Table 2.3. Solution Assuming that ip + aN + azN? + a3N¥ ‘we may evaluate this sum for N = 1,2,8,4 and write down the following set of four equations in four unknowns 111 1][a% o 124 a||a 1 ee een la 5 1416 6}la} [1s Solving these equations for w, ay, and a, we fd rn 0 af_| v6 a | =| -1/2 as 1/8 which gives the following closed-form expression for the sum, Lyi nan — gNUN - DeN ~1)10 Problem Solutions 2.7 Show that a projection matrix P.4 has the following two properties, 1. It is idempotent, P3, = Pa 2. It is Hermitian. Solution geese erga Given a matrix A, the projection matrix Py is Pa=A(atay tal Therefore, PA = A(AMA)TAMA(AMA) 1AM A(AMA)AY = Py and it follows that P. is idempotent. Also, pit = [acatay at)” = alata l)tat Since AA" is Hermitian, then so is its inverse, (tay = (aay and PH = Aca" aya ‘Thus, Py is HermitianChapter 2 ul 2.8 Let A > 0 and B > 0 be positive definite matrices. Prove or disprove the following statements, (a) AP >0, (b) ArT > 0. () A+B>o. Solution (a) Let vy be an eigenvector and A the corresponding eigenvalue of A. Since Alva = A(Avi) = Ave then vy is an eigenvector of A® and 2} is the corresponding eigenvalue. IF A > 0, then Ax > 0. ‘Therefore, 3f > 0, an it follows that A? > 0, (b) If A > 0, then the eigenvalues of A are positive, Ay > 0. In addition, A~! exists and the eigenvalues of AF are Az. Since Ax > 0, it follows that Ag > 0 and, therefore, A~! > 0. (6) Let v #0 be an arbitrary vector. Then vA + B)y =v" Av4v"By IFA > Q and B > 0, thea vilAv>0 5 v"Bv>0 ‘Therefore, vil(A+ By >0 and it follows that (A +B) > 0.R Problem Solutions 2.9 (a) Prove that each eigenvector of a symmetric Toeplitz matrix is either symmetric or anti- symmetsic, ie, vk= 3p (b) What property can you state about the eigenvalues of a Hermitian Toeplitz mateix? Solaten (a) If A isa symmetric Toeplitz matrix, then wAI= where J isthe exchange matrix. If vs is an eigenvector of A with eigenvalue Ay, then Ave = ive and, using the identity above, we have STAIve = Avy Since J is unitary, J°I that 1, if we multiply both sides of this equation on the left by J, it follows Advi = Adv ‘Therefore, if vs is an eigenvector with eigenvalue Ac, then Ivy is also an eigenvector with the same eigenvalue. Consequently, if the eigenvalue y is distinct, then vx and Jv, must be equal ‘to within a constant, wedve However, since the exchange matrix reverses the order of the elements of the veetor vp, the only possible values for this constant are ¢= 1. Therefore, eee and the eigenvector Vp is either symmetric or anti-symmetric. Now let us consider the case in which the eigenvalue Ax is not distinet. We will assume that the multiplicity is two. ‘The following diseussion may be easily generalized to higher multiplicities. In this case, ve and Js span a two-dimensional space, and any twn linearly independent vectors in this space may be selected as the eigenvectors. Therefore, we may choose Ty ve bu and ag = Ve Ive as the two eigenvectors. Note that V4, is symmetric and Vp, is anti-symmetric, ‘This completes the proof (b) In the case of Hermitian Toeplitz matrices, the eigenvectors are either Hermitian or anti-Hermitian,Chapter 2 13 ab] [it] a bt ba Seite SS A (a) ‘The eigenvalues are the roots of the characteristic equation det(A — M1 (o-a)'-B=0 Expanding the quadratic in A we have P= 2ar+ (0? = =[A-(@+5][A-(a-9) =0 ‘Therefore, the eigenvalues are Ay =a +b and Xp = a~ 6. The eigenvectors, on the other hand, are solutions to the equation Ave = Acvn For the first eigenvector, vi, we have [3 2][tg]-e+o[ se] of] ‘Simarly, the elgenveetor vz Is found to be whieh gives vir = 212, oF (b) With av ba | the eigenvalues are the roots of the characteristic equation ddet(A ~ AI) = (0 2)? ~ by : 2? —Dad+ a? + |b)? = [A — (a+ [O))][A— (a — [0))] = 0 ‘Thus, Ar = a+ [6] and Az = a — |b)ul Problem Solutions ‘The eigenvector that has eigenvalue 2; is the solution to [oo )[m]=e+m [2] [ vit ] [ ip ] 6 hich gives oa = pours OF Similarly, for v2 we haveChapter 2 16 2.11 Establish Property 5 on p. 45. Solution Let B be an 1 > matrix with eigenvalues Ax and eigenvectors vi. With A=Bhal note that Ave = By tave Deva bave = (An +a)va ‘Therefore, A and B have the same eigenvectors, and the eigenvalues of A are A; +a.6 Problem Solutions 2.12 A necessaty and sufficient condition for a Hermitian matrix A to be positive definite is that there exists a non-singular matrix W such that, A=w'w (a) Prove this result (b) Find a factorization of the form A = W/W for the matrix [34] (a) TFA > 0, then A may be factored as follows A=vav" Solution ‘where A= diag{At,...,Aw} with 2;> 0. ‘Therefore, A may be factored as fllows Aanayne where AN? = diag{AY,...,AN?} >0. Thus, we may write A= (vale) (a¥2v") = (ave! (Wav! where W = A/V > 0 is nonsingular. Conversely, suppose that A may be factored as A=w'w where W is a nonsingular matrix. ‘Then W may be factored as follows w=vav" where A is a diagonal matrix and V is a unitary matrix. Thus, A= w"lw = (VAV#)" (VAV") = vatv! Since the diagonal terms of A? are positive, then A > 0. (b) ‘The eigenvalues of A are dy = 8 and Xp = 1, and the normalized eigenvectors ate ili] + vali] ‘Therefore,Chapter 2 a [23] (a) Find the eigenvalues and eigenvectors of A. 2.13 Consider the 2 x 2 matrix (b) Are the eigenvectors unique? Are they linearly independent? Are they orthogonal? (0) Diagonalize A, ie, find V and D such that viav =D where D is a diagonal matrix. Solution det(A — AI) =? +1=0 wh re.= 4. Theelgenetrcrsponing othe gene y= sts the equation [a] [3]=4[3] ‘hich implies that vp = jy. ‘Therefore, the normalized eigenvector is al] Pela val i] a= Bl -3 (b) The eigenvectors are unique, linearly independent, and orthogonal (wus v2) = viva =0. (c) With V the matrix of normalized eigenvectors, al; =] Aemisecieee: sche viav=D sme p-[3 5]1s Problem Solutions 2.14 Find the eigenvalues and eigenvectors of the matrix [2a Solution ‘The eigenvalues of a matrix A are the roots of the characteristic equation det(A~ AL) =0 For the given matrix, we have et A — A) Ta a(45) ch) = (-auu-ay+2 2 5A46=(4—3)(A—2) ‘Therefore, the eigenvalues are dy = 3 and dy = 2. The eigenvectors are found by solving the equations For Ay = 3 we have ‘The first equation is ‘Therefore, the eigenvector is Repeating for Mg = 2.we findChapter 2 19 2.15 Consider the following 3 3 symmetric matrix 1-1 0 A=[-1 2-1 Ct (a) Find the eigenvalues and eigenvectors of A. (b) Find the determinant of A. (c) Find the spectral decomposition of A. (@) What are the eigenvalues of A +I and how are the eigenvectors related to those of A? Solution (a) The eigenvalues are found from the roots of the characteristic equation, ot(A — A) =0 ‘The roots are 4 = 3,1,0. Given the eigenvalues, the eigenvectors are found by solving the ‘equations Av; = Avy; for i= 1,2,3. The eigenvectors (unnormalized) are ld o4 11 (b) The determinant is equal to the product of the eigenvalues, det A= TTA V= [rivavs (6) The spectral decomposition for A is A= Daw! where vj are the normatized eigenvectors of A. Since Ay =O, this decomposition becomes i 1 w]e 2 nal ‘| 0-1] A - ran to -t 2 4 -2]4g] 00 0 1201 “1001 (a) I the eigenvalues of A are A,, then the eigenvalues of A +I are A, +1, and the eigenvectors are the same. Therefore, the eigenvalues of A+ I ate = 4,2,1, A=20 Problem Solutions 2.16 Suppose that an n x 7 matrix A has eigenvalues Ay,...,2y and eigenvectors viy...4¥n (a) What are the eigenvalues and eigenvectors of A2? (b) What are the eigenvalues and eigenvectors of A“? Solution (a) With v; an eigenvector of A with eigenvalue 4, note that Aty, = A(Av,) = ACA) ‘Therefore, the eigenvectors of A® are the same as those for A, and the eigenvalues are 22, (b) Since Av = Avi then, assuming that A“! exists v= MAY, ‘Therefore, A=! has the same eigenvectors as A, and the eigenvalues are 1/2Chapter 2 a 2.17 Find a matrix whose eigenvalues are Ay = 1 and Ay = 4 with eigenvectors vy = [3,1]7 and v= Bal? Solution = From the given information, wo have aE}-(2] 0 [f° [2] et ‘Then we have and Subtracting these two equations gives Also, we have Therefore,2 Problem Solutions 2.18 Gerschgorin’s cirele theorem states that every eigenvalue of a matrix A lies in at least one of the circles Ci,...,C,y in the complex plane where C; has center at the diagonal entry a4 and its radius is ry = Sg lal 1. Prove this theorem by using the eigenvalue equation Ax = Xx to write Q~ aies = Dayry and then use the triangle inequality, IE a Dlava! im 2, Use this theorem to establish the bound on Ayjar given in Property 7. 3. The matrix 4 3 2 0 6 is said to be diagonally dominant since |ai| > 71. Use Gerschgorin’s circle theorem to show that this matris is nonsingular Solution 1, Let x = [x1,...,2]” be an eigenvector, and A the corresponding eigenvalue for the matrix A. Assume that 2y isthe largest component of x, Le, fn) > fay] for ally 2 &. With Ax =x, ie follows that Soeyey = das (-aa)es= Doysy * Therefore, Since |x| > [xy] for all j #4, then the ratios x,/x) are less than or equal to one, and 2 lies in the ith circle defined by Pa anl or Yas! A whereChapter 2 2, From Gerschgorin’s circle theorem, Since then Therefore, 23 each eigenvalue, A, there is an é sueh that Test la- Al = laisl $ 1A al Del Se Sey 3 Let A be a matrix that is diagonally dominant, lau > re Assume that one of the eigenvalues is zero (A is singular). From Gerschgorin’s circle theorem, we know that, for each eigenvalue, However, if Ax = 0, then Paul ee aygu Sin ba Sine then the power spectrum is P(e)Chapter 3 3 3.6 Find the autocorzelation sequence corresponding to each of the following power spectral den- sities. (2) Pele!) = 3-4 2eosw. 1 oo ae ©) FeO) = FS ccew 222 452 (c) Pelz) Solution (a) Expanding P.(©) in terms of eamplew mepanentials, Pe B+ 2eo8w pe 4 ei it follows that r.(0) = 8 and re(1) = re(—1) = 1 (b) Recall the DIPT pair =a ce ae] = ae) © 1 a8) = Barco Since 1 1s PAC) = rama * Te few it follows that ra(k) = 4(-9) (©) With ot 4 52-2 Beles ~ Gryeee) s lps ‘using the pair it follows thatuM Problem Solutions 3.7 Let x(n) be a zero mean WSS process with an Nx N autocorrelation matrix R. Determine whether each of the following statements are True or False (a) If the eigenvalues of Re are equal, 4 = Ay = --+ = Aw, then re(k) = 0 for k = 1.2)... (b) IFA, > 0 and Ay = 0 for k= 3,...,N, then re(k) = Aci Solution (a) This statement is True. To show this, the first step is to recall the Bordering Theorem which, for Toeplitz matrices, states that if Ry_1 is a p x p Toeplitz matrix with ordered eigenvalues Dr S Ag SoS Ape and if Ry is the (p+1} x(p-+1) Toeplitz matrix that is formed by adding one row and column to R,1, then the ordered cigenvalues of R, denoted by Ay < Az S--- < Apa are interlaced with those of Ry as follows EMS Ta SMS S Ip ApS Dp ‘What this implies is that if the eigenvalues of Ry are equal, X1 = Xo Jost, then the eigenvalues ofeach of the lower-order Toeplitz matrices mist also be equal ‘The next step isto note that for any 2 2 Toeplitz matrix n= [78 283] if the eigenvalues are equal then rz(1) =O. ‘This follows easily from the fact that the eigenvalues of Ry are dy = 12(0) + ra(1) and Ay = r5(0} — rg(1}- Therefore, A1 will be equal to Az if and only ifr.(1) = 0. This property may’ also be established for a 3 x'3 Toeplita matrix by equating the coefficients of the powers of \ in the equation t[Re 2] (A — Ao)? where the three eigenvalues of Rz are equal too. ‘We may now establish the result by induction, Assume that Ry» is a k x k Toeplitz matrix with equal eigenvalues, and that Rar = Toep{rs(0),0.0,...,0} We will now show that if Re is a (k-+ 1) x (k +1) Toeplitz matrix Re = Toer{72(0),0..-s0,re(&)} with equal eigenvalues, then rz(K) = 0. ‘The eigenvalues of Ry are the roots of the polynomial det(Ry — M1) =0 If the eigenvalues are to be equal, then det(Ry ~ AL) = (A Ap)" However, note that det(Ry — M) = [A — rs (0)]* + r2(p)[4— re(0)}?Chapter 3 35 where the sign depends on whether f is even or odd. ‘Therefor, re(0)]* e209) [A ro(O)]"? = (A - do)" if and only if Ay = re(0) and r9(K) = 0 28 was to be shown, (b) This statement is True. To show this, note that if Ay > O and Ay = 0 for k= 2,3,...,.M, then ‘the autocorrelation matri las the form, R. = Awivil where vs is the eigenvector associated with the nonzero eigenvalue Ay. Let vs(k) be the coeff cients of the eigenvector v, (1), e4(2), 5 ex” Then 2 oitey ee Raa] > | [iG we) aN) ]= (i) we") (N72) oscar Since Rs is Toeplitz, then the terms along the main diagonal must be equal, Ju (QF = fen)? = == In) ‘Therefore, the coefficients ux (k) must have the form, View In addition, the Toeplitz structure of Ry implies that the terms along the diagonal below the ‘main diagonal must be equal, ul) y(2)o7 (1) = v4 (809 (2) = = w(NJOF(W 1) ‘Therefore, (4 ~ du--1) must be a constant, 4 ~ On and, with (1) = Ye, it follows that ny = Va letra] Finally, note that since the first column of Ry contains the autocorrelations r,(k) for k= Odeon N=, then, (8) = nC + Loy () = VA etl] V7 end = 4 eho as was to be shown,36 Problem Solutions 3.8 Consider the random process a(n) = Acos(ns +6) + w(n) where w(n) is zero mean white Gaussian noise with a variance o3,. For each of the following ‘cases, find the autocorrelation sequence and if the process is WSS, find the power spectrum, (0) 4 is Gaussian random variable with zero mean and variance a and both w and ¢ are constants (b) ¢ is uniformly distributed over the interval [-r,7] and both A and w are constants. (6) w is @ random variable that is uniformly distributed over the interval fy — A, + A] and both A and ¢ are constants, Solution (a) When w and ¢ are constants, then ra(k,l) = E{2(k)z(0)} = B{ Acos(hw + 6) cos(lw +4) + o3,0(k — 0) ‘Vhs, rellel) = B(A?}con( +9) com(k +8) + 035(k ~1) = oh cos(kir+ 6) cos(lis+ @) + oB(k ~ 1) Note that since r(F,1) does not depend on the difference & ~ 1, then x(n) is not widesense Stationary, and the power spectrum is not dened for this proces (b) When A and w are constants and ¢ is a random variable that is uniformly distributed over the interval [-,, then the autocorrelation is ra(kyl) = Ef A*cos( his + 6) cos(lar + 6)} + o28(k 1) = PPB foos[(k + No + 26)} + LAE Lcos(k ~ Yu} + 025(k—) However, since 2 {cos(k-+ Du + 26]} = 0, then the autocorrelation fs 5A eoslk Ds + 08 (0) rel) = Therefore, r9(le1) depends on the difference (kk~1), and the process is WSS. The power spectrum, Pele) = 2) ae ta) +08 (c) As in parts (a) and (b), the autocorrelation of the process 2(n) is ralkyt) = B{x(k)o(0)} = B{Acas(hu + 6)A cos(tis+ 9)} + 025(k —1) In this case, however, w isa random variable, and the expectation of the product of the cosines Bf Acos(ku + 4)Acos(lw + ¢)} = A peos[(h— Dua] + $ os[(h-+ thu + 26]}Chapter 3 37 Since w is uniformly distributed over the interval [ay ~ A,uip + J, the expectation of the frst term is Bfcos[(k uJ} = AL cote Dulde 7 SEE Linlts- Dlow +A)] ~sin{(# -Dfeo - a] With @ a constant, the expectation of the second term is Ecos |(k + 1)u +24] mer [sine + 1am + 2-+26)] ~sinf(k +(e — A +26)] ‘Therefore, x(n) is not WSS. However, if is a random variable that is uniformly distributed over Une interval [=7,27, then this second expectation is zero, and the autocorrelation becomes relk) = 4 5 {ein[ Ce — Dhan +4} ~ sink ~~ )}} a = aaa einlte— a] cosl te ~] ‘and the process is WSS. With an autocorrelation sequence given by AP sink 20 ak ralf) 08 ky the DTFT pair sink 7h re(h) = Pe) it follows that the power spectrum of z(n) is nat nae W(k)rw(n — k) & where w(n) is zero mean white noise with variance 02, (a) Find the unit sample response of the filter that generates y(n) from w(7). (b) Find the autocorrelation of y(n). (c) Find the power spectrum of y(n). Solution (a) ‘The process y(n) is generated by filtering white noise with an FIR filter that has a system function given by He) =oaHe* ‘Thus, the unit sample response is oe (b) The autocorrelation sequence for y(n) is ecikl ry(k) = o25(K) » h(h) «A(—K) = 02S B(DHIKL +) ) The power spectrum of y(n) is Pyle) = 02 Bl) Ble)42 Problem Solutions 3.1 Suppose we are given a zero-mean process .r(n) with autocorrelation re(X) =10(3)" +3(3)""" +35)" (a) Find a filter which, when driven by unit vatiace white nok, wil yield a random proces vith this astocortlation (b) Find a stable and causal flter which, when excited by z(n), will produce zero mean unit variance, white noise, Solution (a) The power spectrum of a(n) is (1432-40 +32) ») ‘Therefore, if then the response ofthis filter to unit variance white noe will ben random process with the siven autocorrelations, (b) Consider the filter having a system function Boles Wa teh I Clearly this ter is stable and caussl. Furthermore, if we filter 2(n) with g(n) then the power spectrin ofthe fered signal wil be GQ) = lal>d Py(a) = G2)G(E™)P. 2) ‘Therefore, o(n) is the whitening filter that will profce unit varianee white noise fom x(n)Chapter 3 43 3.14 For each of the following, determine whether or not the random process is 1. Wide-sense stationary. 2, Mean ergodic. (a) 2(n) = A where A is a random variable with probability density function f..(a). (b) x(n) = Acos(nwy) where A is a Gaussian random variable with mean m4 and variance zm hy (©) 2(n) = Acos(nup+¢) where ¢ is a random variable that is uniformly distributed between =m and x. (a) a(n) = Acns(nim) + Rsin(ni) where A and Rare uncorrelated zero mean random variables with variance 0, (€) A Bemoulli process with Pr(x(n) = 1} =p and Pr{x(n) = —1} (1) y(n) = 2(n) ~ x(n 1) where 2(n) is the Bernoulli process defined in patt (6). Solution (a) Weare given a process x(n) = A, where 4 isa random variable with probability density function fala). To check for wide-sense stationarity we need to compute the mean and autocorrelation (of (1). The mean of this process is ma(n) = B{z(n)} = BLA} which is a constant. ‘The autocorrelation is ro(k il) = {atk \2()} = BLA?) Which is also a constant. Therefore, x(n) is WSS. To check for ergodicity in the mean, note that cl) = rll) — mi = BAP} — B°( 4} Which is a constant. Therefore, x(n) is erg ex(k) = 0, (b) With 2(n) = Acos(nuip), note that the mean of the process is in the mean only if the variance of A is zero, a(n) = B{2{n)} = E{ Acos(nan)} = B(A} cos(ne) = ma cos(nuso) Which depends on n. Thus, 2(n) is not WSS and, therefore, not ergodic in the mean, (c) For 2(n) = Acos(nay + 6) with a random variable that is uniformly distributed between — and sr, the mean of 2(n) is oe sma(n) = Bfx(n)} = B{Acos(ous +4)} =f cos(ray + 6)d6 =0M4 Problem Solutions which isa constant, For the autocorrelation we have ralkl) = B{Acos(huy + 4) A costly +6) $4? {cosh ~ Ham) + cos(k + ay + 26)} JA? cosh ~ Dury which is a function of (— 1). Therefore, a(n) is WSS. ‘To check for ergodicity in the mean, note that Ew = 125 ote rasa Le ikwn = FSS gem rpm 1 Soon) which goes to zero as N — oo, provided that wy = 0. Ian = 0; then 2(n) = Acosd and x(k) = A2/2. In this ease, 2(n) is not ergodic in the mean, ‘Therefore, 2(n) is ergodic in the mean only ifu #0. (@) ‘The mean of this process is B{2(n)} = { Since B{ A} = E{B} =0, then E{2(n)} = 0. For the autocorrelation we have A} cos(nian) + B{B}sin(nan) ra(k,2) E{x(k)2{@)} = B{[Acos(wok) + Bsin(wok)|[A cos(un!) + Bsin(wnl)}} B( A} cos{ hap) e0s(lup) + 2{.3"} sin(ay) sin(tun) +B {AB} [c0s( Fi) sin( lun) + sin kg) cos(le}] Since 4 and B are uncorrelated and have zero mean, then E{AB) =O and £{ A?) = B{ B®} = 7, Therefore, we have (kyl) = 02{cos(huip) c0s(lup) + sin( i) sin(lap)] = 0? eos(k — Dy Since the mean is a constant and the correlation function r(k,!) depends only the diference, k= [then a(n) isa wide-scnse stationary process. [As in part (c),2(n) is ergodic in the mean only if wy =O. (6) With x(n) a Bemoulli process with Pr{(n) = 1} = p and Pr{x(n) = —1} = 1 ~p, the mean of a(n) 8 s(n) = E{2(n)} = p—(1-p) = 2-1 fand the autocorrelation is B(z(R) skal rteo=vtetexO}={ study | EBLChapter 3 4s ‘Therefore, af pta-p=t kat ete {BE ER rrelll) = Apt — pb ~ 2) + (1 2p)? ‘which isa finetion of kL. Therefore, x(n) is WSS. With x(k) = ra(h) — m2 = 4p(1 — p(k) it follows that im 1 FD et) = Mpls -) 008 N00 ‘Phus, 2(n) 18 ergodic mn the mean, With y(n) = x(n) — x(n ~1) where 2(n) is n Bernoulli process, wide-sense stationarity may be casily checked using the direct approach taken in parts (a)-(e) of this problem. However, it is fasier to note that since y(n) is the response of a linear shift-invariant system to an input that is a WSS process, then y(n) will be WSS. For ergodicity in the mean, note that since ex(t) = 4p(0 ~ p)5(0) w then 6y(B) = (+ W(R) # (—B) = Ap = p)[25(K) ~ H(& — 2) ~ 3¢ +] and, cleanly, FD elt) 0.05 N0046 Problem Solutions 3.15 Determine which of the following correspond to a valid autocorrelation sequence for a WSS random process. For those that are not valid, state why not. For those that are valid, describe a way for generating a process with the given autocorrelation, {a) ra(k) = 6(k ~ 1) + 6(&-+1) (b) re(K) = 85(K) + 26( ~ 1) + 26(8-+ 1) (6) ra(k) = exp(ikn/4) “ft Ben @) n= { Rota Lien © ran-{ ie 0; ele () re) =2-# Solution (a) ‘This autocorrelation sequence is not valid since we must have r-(0) > jrs(1)) (b) This autocorrelation sequence is not valid since the pawer spectrurn is not nor-negative Pilot) = B42 + 2c" = 34 deosw (c) This autocorrelation sequence is valid, ane corresponds to a harmonic process. Given a random variable @ that is uniformly distributed between —x and 2, this process may be generated as follows, a(n) = hF+9) (@) This autocorrelation sequence is not valid since the power spectrum joy _ sin PA) = oe Is not non-negative. (e) This autocorrelation soquence is valid cinco r(B) ie qymmetric, and its discrete-time Fourier twansform is non-negative for all w. This process may be generated by filtering unit variance white noise with the FIR filter that has a unit sazmple response given by er Moet oe (f} The sequence is a sampled Gaussian pulse. The DTFT of r,(k) is an allased Gaussian, which is positive forall &. Since r,() is symmetric and P,(eM) > 0, this represents a valid autocorrelation sequence, This process may be generated by filtering white noise with a linear shift-invariant system that hhas a Gaussian shaped unit sample response.Chapter 3 a 3.16 Show that the cros+-corvelation, ray( ki), between two jointly wide-sense stationary processes x(n) and y(n) satisfies the following inequalities, (@) Ire) < [ro(r,(0)]'” (0) Wrey(¥)| < 3 fr(0) + r4(0)] Solution (9) Note that for any constant a, Bf [ax(n +b) ~ y(y]?} 2 0 Expanding the square we have ob {2%(n + 8} ~ 2aB{e(n-+ Auln)} + B{A(a)) 20 This Is a quadratic equation in a, and is non-negative. ‘Therefore, its discriminant must be non-positive, AE*{2(n + K)y(n)} < 4B {P%(n +k} Ef y(n) : F208) S ralOr(0) ‘Taking the square root, the result follows (0) To establish this inequality, mote that P {x(n + k) + y(n)|?} 20 Expanding the squae it follows that 14(0) + 2ray(h) +74 (0) 20 Therefore, trey(h) < $ [re(O) + ry(0)] and Iray(BL < $ fre(O) + r4(0)]48 Problem Solutions 3.17 Given a wide-sense stationary random process x(n), we would like to design a “linear predic- tor” that will predict the value of 2(n +1) using a linear combination of 2(n) and x(n ~ 1) ‘Thus, our predictor for 2(n + 1) is of the form 3(n + 1) = ax(n) + bx(n ~ 1) where a and } are constants. Assume that the process has zero mean F {a(n} =0 ‘and that we want to minimize the mean-square etror €= B{[e(n+1) ~ a(n + OP} (0) With r2(j4) the autocorrelation of x(n), determine the optimum predictor of a(x) by finding the values of a and b that minimize the mean-square error. (b) What is the minimum mean-square error of the predictor? Express your answer in terms of the autocorrelation r(k). (6) If x(n +1) is uncorrelated with (7), what form does your predictor take? (@) If2(n + 1) is uncorrelated with both 2(n) and x(n — 1), what form does your predictor take? Solution (2) The mean-square eror that we want to minimize i = 2 {le(n +1) - 34 IP} = Bfa%(n +1) ~2a(n4 1)8(n+ 1) + F%(n+ 1} Since the estimate of 2(n+1) is 3n-+1) = ax(n) +be(n = 1) then setting the derivative of € with respect to a and b equal to zero we have ee re % = 2p fain + tain ~1)} + ELR(n-+ Ipeln—1)} <0 Dividing by 2 and substituting for 2(n + 1) gives AF {2°(n)} + bE (2(n)e(n—1)} = E{a(n+ 1)e(n)} a (x(n)s(n ~1)} + BE {2°(n~1)} = B {x(n + 1}x(n — 1)} ee [28 28] [¢]-[28] eee [8] -aomaan[ "Moe a ]Chapter 3 49 (b) Por the minimum mean-square error we have € = Bf{le(n+1)- 2+ 1yP} = B{fe(n + 1) - Hn +1) fo(n +1) ~ 2( + HI} = B{lx( +1) ~ 2+ Dat +1)} ~ Efleln+1) ~ Ha H}z(n41)} Note that for values of a and b that minimize the mean-square error, the derivatives of ¢ with respect to a and b are equal to 2cto, which implies that the second term in the equation above is equal to zero. Therefore, the minimum mean-square esor is § = B{[xln+1)- 20+ DJa(n+ D} = Bf [a(n 1) ~ a2{n) — bala D]o(n+1)} © = ra(0)— urs(1) ~ tre(2) re(O)re(1) ~ ra(Dre(2) 72(0) — (0) re(0)r2(2) ~ *3(1) 7) = FF) = r4(0)— raft) Fe) (6) If x(n +1) and 2(n) are uncorrelated, sn r2(1) =0, and the values for @ and b become b= re(2)/re(0) In this cas, the linear predictor is ala) Feet) = a(n 1) (@) If2(x-41) is uncorrelated with both x(n) and x(n ~ 1), then the values for @ and b are tnd the linear predictor is ant) =0 which is equal to the expected value of 2(n+ 1), An +1) = Blain 1)50 Problem Solutions 3.18 ‘Due or False: If 2(n) is a WSS process and y(n) is the process that is formed by filtering x(n) with a stable, linear shift-invariant filter h(n), then B02 > ihn? 6} where o2 and o3 are the variances of the processes z(n) and y(n), respectively Solution —_____ Ifa WSS process 2(n) is Filtered with a filter that has a unit sample response h(n), then the autocor relation of the output process is rhe) = rele) # AU) # A(R) Assuming that 2(n) has zero-mean, then 0) and = 74 (0) ‘Therefore, the question is whether oF not the following relationship is true: ry(0) =r2(0) SO [h(n)P If this is true then o depends only on r=(0), and not on re(R) for k # 0. Cleaty, this isnot true unless r2(K) = 0 for [k| > 0, ie, if x(n) is white nose. ‘Therefore, the statement is FaleChapter 3 aL 3.19 Show that a sufficient condition for a wide-sense stationary process to be ergodic in the mean is that the autocovariance be absolutely summable, elk] <00 Solution Beginning with the variance of the sample mean, vataun)= hE (Mas not that i : ¥ let) k for some ky. Therefor, if we let to c=Yat) she, ve sli ae De a0 gin and ergodicity is established (b) This statement is true and may be shown as follows. An autoregressive process is formed by Sltering finite variance white noise w(r) with a stable, causal all-pole filter (n), x(n) = h(a) « (0) Assuming, without any loss in generality, that w(n) has zero mean, the covariance of a(n) is col) = 08, AUK) # A(R) where 02, < oo is the variance of w(n). The condition that h(n) is stable guarantees that c(h) is absolutely summable and, therefore, x(n) is ergodic in the meanChapter 3 53 8.21 Let x(n) be a real WSS Gaussian random process with autocovariance function ¢x(k). Show that 2(n) will be correlation ergodic if and only if i Aa t= Hint: Use the moment factoring theorem for real Gaussian random variables which states that Bfxrwarars} = Bferre}B {ava} + Efeiza}E(aa2s} + Efeize} Elexts) Solution We are given a WSS Gaussian random process, =(1), that iz ergodic in the mean, Por convenience, let us assume that the process has zero mean. In this problem we will be using the moment theorem for Gaussian random variables. This theorem states that if x(2n), x(n), 2(F), and 2({), are zero mean Jointly distributed Gaussian random variables then the fourth-order moment E{-r(m)(n)(K)a(0)} is Batmatnje(bie(D} = Bletrm}eln)pBaCH el) + Blelom)al) Cela} +4 (x(mm)a(} {a(n 2(8)} Now, for a fixed value of &, let vuln) = a(n + Be(n) Since Buln) it follows that 2(n) wil be cortelation ergodic re(h) mz SS ef + k2(n) = re(k) fi soe NS if and only if daw 00 =0 where cyl) = Efy(m + Duylm)} ~ E{y(m +) E{y(m)} = B{x(m +14 Balm + Nx(m + k)a(mn)} — 13(k) Using the moment theorem, we have ell) = Ela(n t+ a(n + DP ELolrn + K)a(m)} + Elo(m +1 + helm + B)}ELe(on + Diz(m)} + E(a(m +14 R)x(en) Palm + Bc(m +} — 208) = 10) + rok Drele-1) Note that we are assuming that y(n) is wide-sense stationary. However, it may be shown that since 2(n) is a WSS Gaussian process then itis also strict sense stationary and, therefore, y(n) is WSS.oA Problem Solutions ‘Therefore, 2(n) will be correlation ergodic if and only if jim, DEO +ratk-+ Drak) =0 (P3211) What we would like to establish isthe equivalence of Eg. (P3.21-1) and Ea. (P: if Bq. (P3.21-1) is true, then Eq. (P3.21-1) holds. We may see this by setting To establish the converse, we use the inequality 1). It is clear that, 0 in Eq, (P8.21-1). Drak + Ure(k =D) Sr3(e+l) + r2(E~D, Therefore, wa sD relk + Dralk 0] < AIS ta+0 +rt0-o) ‘where, in the last inequality, we used the fact that Ire(4)] < re(0) for all k, ‘Thus, as N — o0, the right-side goes to zero and we have established the equivalence of Eq, (P3.21-1) and Bq. (P3.21-1)Chapter 3 35 3.22 Exgodicity in the mean depends on the asymptotic behavior ofthe autocovariance of a process, cx(k). The asymptotic behavior of c(t), however, is related to the behavior of the power spectrum P,(e) at w = 0, Show that 2(n) is ergodic in the mean if and only if P.(e™) is continuous at the origin, w = 0. Hint: Express Eq. (3.65) a8 a limit, as NV — oo, of the convolution of ex(k) with a pulse mop {WN 5 OS nex with the convolution being evaluated at k Solution A necessary and sulicient condition for a process WSS process +(n) with autocovariance ¢,(k) to be ‘ergodic in the mean is Yelk) =0 ‘To show that (n) is ergodic in the mean if and only if P(e) is continous at the origin, let an (k) = alk) «py (h) where 1N 3 OsneN acor={ a rn Note that a aw(N) = 4S ea(h) ms Since sy(#) is the convolution of ex(K) with py(A), then Sy(el*) = Palel)Po(e) joy gnatw-nya sin Neo/2 eee eta aneh ‘Therefore pines, ee jaya —1yj2 80 New/2 ie, en(W) S(O" des Pelee die a Note that the term multiplying the power spectrum P;(e™!) inside the integral is bounded by one in ‘magnitude and, as N+ co, this term goes to zero for all w # 0, and for w = 0 this term is equal to one, Therefore, lim, sw) = [POY and it follows that sy(W) goes t02er0 a5 V+ oo if and only if P(e )lueor = Pol loco- P,(ei) is continuous at the origin. ~ Pele.sn0-]56 8.23 In Section 3 Problem Solutions 4 it was shown that for real-valued zero mean random variables the autocorre- Jation is bounded by one in magnitude lpxy| $1 Show that this bound also applies when x and y are complex random variables with nonzero mean, Determine what relationship must, hold between x and y if \pey| Soliton ee Without any loss of generality, we may assume zero mean for both z and y. For non-zero mean random variables, the following derivation is modified by replacing # with ~ m, and y with y — my. What ‘we sant to show is that jefe })’ sete? }etvey [Note that for any constant a, E{le~ayl?} 20 with equality if and only if x = ay with probability one. Expanding the square we have B{lx— ay?) = Bf *} ~ ak {a"y} ~a°E{2y"} + |a)*E{\yl} > 0 Now let Ber) UP) Ten (eh) — Ber EEL efor EE «yey REBEL) 9 Cancelling terms and simplifying this becomes etter) EEE 9 and the inequality follows,Chapter 3 87 3.24 Let P(e) be the power spectrum of a wide-sense stationary process (n) and let Xy be the eigenvalues of the M x M autocorrelation matrix R. $2¢96’s theorem states that if g(-) is a continuous function then gu) + 9 Qa) im we & [strated Using this theorem, show that ea ad afin face ea] — exp (3 [" n[ Pe) ] a Solution = ‘The determinant ofa air equal to the product of ks eigenaloes, dete = Ff ou Taare . (even = (ff) ‘Taking logarithms, we have =| In(@etR,)! = Fore Using Szegi’s theorem yields sdimtn(dere) = im Fp Yon gf inlet] ae and the result fellows.98 Problem Solutions 8.25 In some applications, the data collection process may be flawed so that there are either missing data values or outliers that should be discarded. Suppose that we are given V samples of a WSS process «(n) with one value, (np), missing. Let x be the vector containing the given sample values, [2(0), 2(1), ---.2(m9—1), (no +1), ---,20N))” (a) Let R, be the antocorrelation matrix for the vector x, Re = Bfxx"} Which of the following statements are true: 1. Ry is Toeplitz, 2. Re ip Hermitian. 3. Ry is positive semidefinit. (b) Given the autocorrelation matrix for x, is it possible to find the antocorrelation matrix for the vector [200), 20), ...,2()]” that does not have (ni) missing? If so, how would you find it? Ifnot, explain why not. Solution (@) The matrix is not Toeplitz. This may be shown easly by example. I'x = [x(0},2(2),2(3)], then : e(0)/? (0) (2) 8 | [38 re(2) 8 Bfxx"}=E] a(2)e*(0) |x(2)/*— x(2)2*(3) re(2) re(0) ra(1) 2(3)x°(0) x(3)2"(2) Jat)? re(8) re(1) re(0) which is clearly not Toeplitz. However, by definition, Ry is Hermitian, RY = Bfxx"?)" = Bfxel} = Re Finally, Ry is non-negative definite, which may be shown as follows. Let v be any non-zero vector. Then, VE Rev =v" (xx! v= Bfvllxx"'v} "Therefore, vRay = B{|v"x?} 20 and Ry 20. (b) There are several ways to find the autocorrelation matrix for the vector feo. 20), -...200)]" that does not have 2(0) missing. One way sms follows. Note thatthe ist column of iy that is formed from the vector that has (ng) mising isa follows, [r2(0),re()e->.72(n0 ~ 1), r2(mo + D)y-+ + re(N)] ‘Therefore, all that we need is the missing correlation re(7n9). Note, however, that this term is found in the second column of row (ng +2) (sce example in part (a) above). ‘Thus, given r(129) the Toeplitz matrix may then be formed.60 Problem Solutions 3.26 The power spectrum of a wide-sense stationary process 4(n) is P,(e™*) Find the whitening filter H(z) that produces unit variance white noise when the input is x(n). Solution Expanding the cosines in the expression for the power spectru 25 — 120 — evi or, in terms of z, the power spectrum becomes 25 — 12: Plz) (2)G(2"") where ‘Therefore, if then ula) = h(n) #2 ‘will be unit variance white noiseChapter 3 a 3.27 We have seen that the autocorrelation matrix of a WSS process is positive semidefinite, R.>0 ‘The spectral factorization theorem states that if P(e) is continuous then the power spec- trum may be factored as Pele) = of 1Q(e*)? where Q(e!) corresponds to a causal and stable filer. (a) Assuming that of 4 0 and Q(e*) is nonzero for all «, show that the autocorrelation matrix is positive definite. (b) Givon an example of a nontrivial procose for whieh Rez is not positive definite. Solution — (a) The pesitive definite property of Rs may be easily established with the help of the Eigenvalue Extremal Property which states that the eigenvalues of the autocorrelation matrix of a zero mean WSS random process are upper and lower bounded by the maxirsum and minimum values, respectively, of the power spectrum, tin P(e!) 0 and, therefore, O ce(Ax)"u(n) where the Ag’s are distinct complex numbers, (a) Show that the Padé approximation method can be used to determine the parameters ¢, and Ay for k= 1,2,...,L. Is the answer unique? (b) The first eight values of a signal x(n), which is known to be of the form given above 82, 16, 8, 12, 18, a8, B45, 198 95) Determine cy and Xx for k= 1,2,3 Solution (a) With x(n) = J) en(As)"u(n) the z-transform is ya - HOF Me BL = V2 x)= 3 Tall! t+ all) ‘which isa rational funetion of = of order (L~1) in the mumerator and order L in the denominator. Therefore, the Padé approximation may be used to find the polynomials A(z) and B(=) provided p> Land q > (L~1). The covfficients are the roots of the polynomial A(z) and the ‘cooficients cy may be found with a partial fraction expansion of X(2) (b) The Padé equations for the denominator cowffciante are (a) xa-1) x(q-p+1) ] [ a(t) aa+0) x(q+1) a) xq—p+2) || a(2) |__| xia+2) xa+p—1) 2(q+p—2) x(a) aly) x(q +p) 3, then we set p=3 and ¢=2. With (32, 16, 8, 12, 18, 33, 64.5, 128.25)” 8 16 32) [ a(t) 2 8 16] | a(2) ws} | aay the Padé equations become66 Problem Solutions ‘The solution for a = [1, a(t), 0(2), a(8))? is fl 1 aq) = | 18 ag) | =| -0.75 a3) 0375 For the numerator coefficients, the Padé eqnations are (0) 0 o ° 1 0) 20) 2) 0 o ag) || a) 22) 20) (0) 0 a) | = | oe) x) (9-1) xq-2) ata-v) J Lo) | Lo) Thus, so] feo oof ty] pe wa) [=| 16 32 0 0 | | A) | =| -32 02) 8 is 2 0] | a =40 ‘Thus, the model is 82-3221 — 402? H(z)Chapter 4 67 4.4 A consumer electronics device includes a DSP chip that contains a linear shift-invariant digital filter that is implemented in ROM. In order to perform some reverse engineering on the product, it is necessary to determine the system function of the filter. Therefore, the unit sample response is measured and it is determined that the first eight values are as listed in the following table. Having no knowlege of the order of the filter, it is assumed that H(=) contains two poles and (a) Based on this assumption, determine a candidate system function, H(2), for the filter (b) Based on the solution found in (a) and the given values for h(n), is it possible to deter- mine whether or not the hypothesis about the order of the system is correct? Explain. Solution (a) The Padé approximation may be used to find the system function of the filter. With p= the Padé equations are -1 0 0 4a) 2-1 0/f1 ay 3 2-1] | aa | =| a2) 2 3 2] | a@) 0 123 0 [3 3) [s8]--[2] a(l) fee eemaccesias (33 ]-3[4 3)[2]-[0] ‘b(0) -1 0 0 a ay [B)-[3 4 lle ]-[e] ‘Therefore, the system function is mit gets (2)Problem Solutions (b) We may check to see if the inverse z-transform of H(2) matches all of the given values of h(n) Alternatively, ifthe system function is correct, then e(a) should be equal to zero for n > 5, en) =a(n)«h(n)=0 5 nB5 Since (5) = h(5) + a(1)A(A) + a(2)K(3) = 8/5 40 then the hypothesis about the model order is not correct, ‘There must he more poles and/orChapter 4 69 4.5 The Padé approximation models a signal as the response of a filter to a unit sample input, 5(n). Suppose, however, that we would like to model a signal -r(n) as the step response of a filter as shown in the following figure u(n) (2) B(n) 088 In the following, assume that H(z) is a second-order filter having a system function of the form 20) +o) = 4@) = Tae Fale (a) Using the Padé approximation method with a unit step input, derive the set of equations ‘that must be solved so that (nm) = a(n) for n=0,1,..6,4 (b) If the first eight values of the signal, a(n), are x=[L, 0, 2, -1, 2,0, 1, 2" find 6(0), 6(2), 6(2), e(1), and a(2) Soledon Sener (a) What we would like to find are polynomials A(=) and B(s) so that B(z) 9) Fy = *O) TOS BE) = Ae)X() Multiplying both sides of the equation by 1 ~ 2~¥, we have @-2)4@)X(2) = Be) Expressing this equation in the time domain by combining (1 — 2?) with X(2) leads to the following set of linear equations that may be easily solved for the filter coefficients a() and (4) v@) 9 0 0) v0) 20) 0 1 a) 7@ za) 20) || aa) | =| 62 7@) 2@) za) | Lae) 0 a) 28) ¥Q) 0Problem Solutions where the sequence 2"(n) is defined as fallows 2 (n) = a(n) x(n 1) with x(n) = 0 for n <0. Alternatively, combining the term (1 ~ =~!) with A(z} leads to the following set of equations EQ) 0 0. 0, : (0) =) 20) 0 0 vit) ) 30) att) x) 0 | | OG) | = | 5) #3) 2) aft) (0) || ig) o 2() 2(3) #2) (1) o where the coefficients a/(k) are defined by af(k) = a(k) ~ afk 1) ‘with a(0) = 1 and a(k) = 0 for & <0 and & > 2 Yet another possibility is to express the following equation in the time domain A()X(=) = Since the term on the right is the convolution of @ step with the coellleients B(K) we have 20) 0 0 4(0) 20) 2) 0 1 4) +001) 2) 20) 2(0) | | @@) | =| 40) +00) +8) 23) 22) a(t) | | ata) J} (0) +002) +02) 214) 2(3) 22) (0) +0(1) +012) Any one of these seis of equations may be used to solve for the coefficients a(k) and B(R). (b) Using the frst approach derived in part (a), we first form the sequence 2'(n), (1-1, 2, -3, 3, -2, 1, 1” "The Pade equations are den ee: 410) a1 off 4 oa) 2-1 1] ] aa) | =| 6) -3 2-1] | aQy 0 3-3 2 0 From the last two equations we have Solving for a(1) and a(2) we find (3 )-[3 J[4]-[3]Chapter 4 Finally, solving for 6(0), b(1), and b(2) we have (0) 10 oa) f=] 14 0) 2-1 ‘Therefore, the model is 7R Problem Solutions 4.6 With a real-valued signal 2(n) known only for n = 0,1,...,N, the backwards covariance ‘method finds the coefficients of an all-pole model Ghat minimize the backward prediction error x = Lie oy? where » 65 (n) = x(n —p) 4D aplAzln +k - 7) & (a) Show that the coefficients ap(k) that minimize &> satisfy a set of normal equations of the form, Ry ry where Ay = [ap(1), ap(2), + apr) and find explicit expressions for the entries in Ry and ry, (b) Is the solution to the backwards covariance method the same as the solution to the covariance method? Why ar why not? (0) Consider a new error that is the sum of the forward and backward predietion errors ep= Sti FOP + les? } = where ¢5 (n) is the error defined above and &(n) is the forward prediction error used in the covariance method, efln) . ) +S oy(Baln—) ms Derive the normal equations for the coefficients that minimize this error. (This approach is known as the Modified Covariance Method.) (a) Consider the signal x( With p = 1 find the first-order all-pole model that minimizes &# and determine the minimum value of 62. For what values of is the model stable? What happens to the ‘model and the modeling error as N + 00? SolutionChapter 4 3 (a) With é; = wwe have 7 (n)l, setting the partial derivative of > with respect to y(t} equal to zero Ves niein+i-p)=0 Dividing by two, and substieuting for 6; () yields Sancmtnat-ne Sno [Esieos-maest-a]=o Tf we define - “ rally) = Sat + k—plaln tl =p) = (ut Bln then the normal equations become “ Lotirats Hre(00) TLD (b) No, the backwards covariance method does not give the same solution as the covariance method since the definitions for r2(k,0) are different in the two methods, Specifically, for the covariance method . Yatn-a(n—) (hd (©) As before, we diferentiate & with respeet to ap(l), and sot the result equal to zero, ace Frei = LAG dela ep lndnin t= Dividing by two, and substituting for e;(n) and ¢(n) we have 0 = Vleet) + x(n— pain +t- p) +L ap(8) Do [alo- Bala —) + a(n +k ~ pha(n +1 -p)] Defining rolls) = So x(n = t)a(n ~ 8) we have rath.) ralb = ba) +3 og [8 +r bp 8] =0 ‘Therefore, the normal equations are Deel) [re 8) + re — hp (LO) +rep— bp) 5 TAP4 Problem Solutions (a) With p= 1 we have ay = _Te(10)-# ra(0,1) 2re(1.0) a) = FE Aye F=(0,0) ~ Fels) + re) Since z(n) = 9" for m=0,1,..0,.N, then r2(0,0) = 72%) = rt) = Vn-1)= ra(t 0) = SDetne(—1) = "Therefore, . w= Note that for any value of 3, (14)? =14+428>0 ‘Tins, 146% -26 ‘and the model is stable forall values of f. For the modeling error, we have EP = ral 0,0) + rel.9) +30 ab) [Pe 0.4) + Pols ~ 8] Therefore, Pe(0,0) + ra(2,1) + a(t) ra(0,1) + re(1,0)] 1-9, (ie ee ee BO + sg] ~ Tee and the modelling error does not go to zero as N +00.Chapter 4 i} 4.7 Suppose that we would like to derive a rational model for an unknown system $' using the approach shown in the following figure x(n) u(r) en) 8 Ale) For a given input 2(n) the output of the eystom, y(n); ie obeorved. ‘The coefficients of the two FIR filters A(z) and B(2) that minimize the sum of the squares of the error signal e(n) are then to be determined, Assume that the sum is for all n > 0 as in Eq, (4.73). (a) Derive the normal equations that define the optimal solution for the coefficients of A(=) and B(2). (b) ‘The philosophy of this method is that if the error is small then B(z}/A(s) is a reasonable model for §. Suppose that $ is a linear shift-invariant system with a rational system function. Show that this method will identify the parameters of $ exactly assuming that the orders of the filters A(z) and B(z) are choson appropriately. Solution (a) Note that £(2) = ¥(2)A(2) — X(2)B(2), s0 e(n) = Salk}y(n — &) — $7 b(Hx(n— k) With . E=den) the Normal Equations are found by setting the derivatives of € with respect to a(k) and &(k) equal to 210, ae aoe = ° Vathun-9 - Doan - o} vin ~ k) {eau =) LUdatn = obec -k76 Problem Solutions Dividing by two, and rearranging the sums, we have suo {S00 = Duln nh yw {Ex Data »} and Sof vio pana} + Sno {Sone} Co I we define - royltd) = Soxln—duln—k) rth = Dovln=ytn—&) ra(kil) = Salm —Da(n—%) then these equations become Lalyry(k)— Lr) = 0; RAL. —LaWrelD+ Mek) = 05 K=O. 8 Assuming that the coefficients have been normalized so that a(0) Late) — oral) = =r4(6.0) 4 ra at Lary, + adr Waiting these in matrix form we obtain Ky -Hy ][ a [-m, “e]8] [a(1),0(2),..,a(p)], b” = [0(0), (2), ..-,0(Q)],#F = [re(2,0),r2(2,0),--.r2CP,0)], irsy(1s0)}72y(2,0)y---s729(9.0)). Also, Ry is a px p matrix with entries ry (fl), Ry is a (q°+ 1) x (q+ 1) matrix with entries ry(k,l), and Ry i a p x (q-+ 1) matrix with entries ell?) (b) Suppose ${2) = C(2)/D(z). Then ryelhiO) ¢ R= OA Bee) = Bo xta)- FE aaxte) and the error can be made oqual to zero if B= SEA) o BD De) AG)Chapter 4 7 4.8 Consider a signal, «(n), which is the unit sample response of a causal all-pole filter with system function L THONG OB NT Ta We observe a(n) over the interval 0 6). (b) For the autocorrelation method, the roots will always lie inside the unit circle and will vary with as NV gets large, the roots will move towards the minimum phase solution, with 05, 0.75, 0.5.78 Problem Solutions 4.9 Equation (4.129) may be used to reduce the amount of computation required to set-up the covariance normal equations. (a) Show that the elements along the main diagonal may be computed recursively beginning with r(1,1). (0) Show how the elements along the lower diagonals may be computed recursively beginning with r_(k, 1). How may the terms along the upper diagonals be obtained? () Determine how many multiplies and adds are necessary to set-up the covariance normal equations (do not forget the evaluation of the vector on the right-hand side) Solution esas Using the relationship between r(& + 1,141) and r4(K\0), re(k+ 1141) = re(hyl) — 2(N —a*(N — k) +a(p—1-Dz*(p—- 1-8) ‘we may evaluate the terms in the covariance normal equations recursively. (0) Bexinning wth r4(1,1), the elements along the main diagonal ofthe covariance normal equations nay be found recursively as follows ra(k-+ 1K +1) =re(h 8) ~ be — BIE + Jal —1- BYP "This requires two multiplications and two additions for each term. (b) Beginning with ra(k, 1), the elements along the lower diagonals may be computed recursively as follows ro(k+1,2) = re(k,1) —2(N~ 1)a*(N —k) + 2(p—2)e*(p—K) and elk 2,3) = rol + 1,2) ~2(N ~2)2"*(N or, in general, ra(k L442) = ra(RALLEL) ~a(N EA) kD t2lp—t-2)2"(p— k= 1- 1) ‘As with the terms along the main diagonal, each term on the lower diagonals requires two mul tiplications and two additions. Note that the upper diagonals may be found using the conjugate synumetry of the covariance normal equations. (c} The covariance normal equations require finding the elements of a p x p Hermitian matrix, As shown in paris (a) and (b), given the first column of the matrix, the remaining entries may bbe computed recursively. Given the elements in the first column, ra(K,1), each of the (p — i) terms along the diagonals, ie, re(K-+ 1) for = 1,...,p~k-~ 1, require 2 multiplications and 2 additions. ‘This requires 1)+2(p—3)x"{p—k-1) pe -1) ruliplications, and the saine umber of additions. In addition, tis necessary to evaluate the p terms in the rst column, relist) = Sol Deak) 2, lis requires p(N — p+ 1) multiplications and p(N — p) additions. Therefore, the total number of kpicttons isChapter 4 # mls, and the total number of additions is # adds. = p(W —p) + p1p-1) = Np—p (N p+ 1) +n(9=1) = Np 7980 Problem Solutions 4.10 We want to model a signal (n) using an all-pole model of the form 20) He) (0) 1+ For example, with p= 2 the model is 40) = ae z Derive the normal equations that define the coefficients ay(A) that minimize the Prony error &= Liew? = where : e(n) = a(n) + raplt)a(n = N) a and derive an expression for the minimum error. Solution ‘The equations for the coeficents a,(k), k= 1,...p, that minimize the error &, are found by setting the derivatives of & with respect £0 ap(k) equal to zero, Thus, assuming that z(n) is real, we have dE, > Bei - Eel t-m Dividing by two, and substituting for e(n), we have ss [x + Seytetn 1 | an-k=N)=0 Seu ses eek »] =-Sranaln— kN) Ie we define allyl) = > x(n ~ a(n ~ 8) & then it is easily shown that re(,t) depends only on the diflerence, kl, and we may write ali) = Soa(n)a(n~ 8) | ‘ius, the normal equations become Yo, (lre(k = = elk +)81 Chapeer 4 Finally, using the orthogonality conition Yeinjan-k-a) =0 ‘we have, for the minimum error, (Eshoin = Selo) [x9 + Daeta-t- »] = Setn)a(n) Therefore eo ys ini+ Seana 1-9] a | = r(0) + So ag(t)ra(t +N)82 Problem Solutions 4.11 Suppose that we would like to model a signal 2(n) as shown in the following figure. 5(n) a(n) ——4 A) where h(n) is an all-pole filter that has a system function of the form (0) H(2)= —, 9) _ 1+ Dag (hye mi Modify the Prony normal equations so that one can determine the coefficients ap(k) in H(z) from a sequence of signal values, (n) Solution ‘To minimize the Prony error where en) = 210) + asta — 2) we set the derivative of & with respect to ap() equal to zer0, “ao = e(n}x(a~2k) = 0 which gee ap(dz(n—20] 2028) =0 "Therefor, Septrel2k —2) = ra) where . ~ re(0k ~ 21) = $oa(n~ a(n 2k) For example, with p=2 the equations have the form (8) 2][28]-[28]Chapter 4 83 4.12 Suppose that we would like to model a signal z(n) that we believe to be quasiperiodic. Based on our observations of +(n) we estimate the autocorrelations through lag k = 10 to be ra(k) = [L0, 0.4, 0.4, 03, 0.2, 0.9, 04, 04, 0.2, 01, 0.7)” (a) In formulating an all-pole model to take into account the suspected periodicity let us consider a two-coefficiont, model of the form, aa) B+ a(lO)e Find the values for the coefficients «(5) and a(10) that minimize the all-pole modeling (b) Compare the error obtained with the model found in (a) to the error that is obtained with a model of the form (0) HG) Tra T+ ally (c) Now consider an all-pole model of the form (0) HO = Toye where both a(N) and N are considered to be model parameters. Find the value for a(N) and N that minimize the all-pole modeling error and evaluate the modeling error. Solution (a) With an all-pole model of the form we begin by defining the error that we waut to minimize. Let. e=Sem with fn) = xh) + af S}a(n — ) + 10}0(0 —10) ‘To find the coefficients a{5) and a(20) that minimize €, wo set the partial derivatives of € with respect to a(5) and a(10) equal to zera, 23 eln}ein —5) = 2 [x(n +alS)e(n—5) + a(10)a(n-— 10](n—5)Problem Solutions and Se e(ne(n — 10) YS [eln) + a(8)o(n ~ 5) + a(10)x(n —s0)}e(n — 10) = Dividing by two and rearranging we have Sx ~ 5) +00) Sox 1}2(n- 5) = — So alapeln-8) and = (0) rata S)a(n 10) +403 20-10) = ~ YE a(nja(n- 10) ‘These equations may be written as [ 58 20 | [ sa ] es [ By | where re(t) = Yo atn)a(n—&) Using the given autocorrelation, thse become [5 0) [e89 J--[] sing rf) ad) ed [28] [i Finally, for the modeling error, we have = r{0) + a(5)r(5) + a(10)re(10) = 0.1263 (b) With a model of the form (0) -- [28] oa oa the normal equations are [2 ra(1) ] on] [x8 7200) | 3] “a8 [23 ]=[22] Lata) J =| 277 J ‘Thus, Finally, the modeling error is €=r4(0) +alI)ra(1) + a(2)ra(2) = 1.2286Chapt () Using a modet of the form H(z) = 40) Tray the value for the coefficient a(V) that minimizes the mean-square exvor is (x) 0) aN) and the minimum mean-square error is given by £2(0) = 2) 7510) ‘Therefore, the mean-square error is the smallest when N = 5, which is the value of k for which irs(i)] is the largest, In other words, to minimize the error we want to find the value of NV for ‘which x(n) and (n+ N) have the highest correlation, {Er}eun86 Problem Solutions 4.13 We would like to build a predictor of digital waveforms. Such a system would form an estimate of a later sample (say ng samples later) by observing p consecutive data samples. ‘Thus we would set . (n+p) = aplR)xln — h) ei ‘The predictor coefficients ap(k) are to be chosen to minimize Sle(n + no) 5 a(n +0)? (a) Derive the equations that define the optimum set of coefficients p(X). (b) If ny = 0, how is your formulation of this problem different fivas Proay's auethud? Solution (a) We want to find the predictor coefficients ay(k) that minimize the linear prediction error = Deo? where (n) = 2(n ng) ~ #(0 + m0) To find these coefficients, diferentite & with respect to ap(k), and set the derivatives equal to 2210 a8 follows Since Bn) = Youtn -8 fen 2mm) _ iy py Bay(k) and we hse ab Levan Dividing by two, and substituting for e(n), we have Y fal m9) ~ Po apdetn Opeth Therefore, the normal equations are Sopra) = rath, ma)Chapter 4 87 where rolls) = oat —Dakn —h) (b) With ng = 0, these equations are the same as the all-pole normal equations using Prony’s method, except that the right-hand side does not have @ minus sign, Therefore, the solution differs in sign,88 Problem Solutions 4.14 You are told that it is always possible to determine whether or not a causal all-pole filter is stable from a finite number of values of its unit sample response. For example, if H(z) is a plivorder all-pole filter, given h(n) for n = 0,1,...,.V, then the stability of H(2) may be determined. If this is true, explain the procedure and list any conditions that must be placed on por N. If false, explain why it cannot be done Solution —___ is true that the stability of a causal all-pole iter can be determined from a finite number of values ofits unit sample response. Given a pth-ander all-pole filter, bo) TY ag(ae# (2) = the coefficients a(k) may be found using the Padé approximation for an all-pole model, (0) 0 o ay) wy nO) (OY 0 |] es@ |__| ae Ai) hp—2) (0) | | opin) ho) ‘Thus, given h(n) for m= 0,1,...,p, the coefficients may be found, and the roots of the polymonntal Leathe they lie inside the unit eieleChapter 4 89 4.15 Let H(z) be a first-order model for a real-valued signal z(n) = 3(n) + 4(n — 1), He aa ) and let . Exs =D le(n) — a(n? % be the error that is to be minimized. By setting the derivatives of Ex with respect to (0) and a(1) equal to zero, try to find an analytic solution for the values of 6(0) and a(1) that minimize Es. (This problem illustrates how difficult the direct method of signal modeling may be, even for a first-order model.) Solution ‘Wo are given a signal x(n of the all-pole filter The goal is to minimize the error SS thn) — ny? ‘Thus, the mode for x(n) will be Hn) = H(OVa(2}}%ulm) ‘To find the values for a(1) and (0) that minimize Ey, we begin by setting the derivative of Ex with respect to a{1) and (0) equal to zero, Thus, we have 84, Ball) ~ = 2 fo(a) — 2409] nbfoyt“"(4) = 0 Dividing by two, and substituting for x(n) and #(n) we have Sloe) —A(e] mOe"-HC) = HoyCa—alapo0))- (o(0)a"(2) nb(0Ja"-"(A) So ee = HfO)(1 ~ a(1)0(0)) — b(0)?a-*(1) 7? na**(1) + H(0)Fa() = Ho) - 40)2a-*() > na?™(Y)} a(1)a(0)? = 4O)- qr aaF Simplifying we have .a(1)6(0) = {1 — a2(2)]? (P4151)90 Problem Solutions Differentiating £5 with respect to B(0) we have as 80) L2lel) - a(n) =0 Again, dividing by two, and substituting for 2(n) and 2(n) we have Ylete) - so") [= 8(0)) + a(4y(4 ~ a(2)6(0)] ~ > 6(0)02"(1) = [1 = 6(0)) + a(1)(1 — a(1)0(0)] — B(O)a*(1), yaa b(0)a4(2) ~ fH raeDa enw) TORE) which may be simplified to (0) = 1+ a(t) ~ a?) — (a) Multiplying both sides of this equation by a(1), and using Eq. (P4.15-1), we have [= aay]? = a(ayfa + a(t) ~ 221) ~ 08(1)] 2af(t) + 0%) ~ 8a%(1) ~ a(t) + which may be factored as follows (a(t) +17(a(4) = (a(1) - 0.5) =0 Of these roots, clearly we want a(1) = 0.5, The value for 6(0) is, therefore, (0) = 1 +a(1) ~ a2(1) ~ a%(1) = 2 ‘Thus, our model for x(n) becomes ‘#(n) = §(0.5)"u(n) swith a oquared error of fis = (1- $F + 0- BP + Xo. 5)" = 259Chapter 4 91 4.16 We have a signal r(x) for which we would like to obtain an all-pole model of the form (0) TyaQe Using the autocorrelation method, find explicit formulas for 6(0), a(1), and a(2) in terms of r2(0), rx(1), and re(2). Solution ‘The autocorrelation normal equations are [28 281028] Solving for the covlficients we have [28 ]--aw=aml 8 28) )[28)] re(2) re(2) ] re(0)re(1) = ra(1)ra(2) aoa | 131) + re(O)re(2) and (0) = rs(0) + a(d)ra(t) + a02)re(2)92 Problem Solutions 4.17 Ifone is modeling a signal r(x) whose transform, X(z), contains zeros, then an all-pole model may be used to effectively model a zero with an infinite mumber of poles. In this problem, we look at how a zero is modeled with the autocorrelation method. Let x(n) = 6(n) — a(n ~ 1) where [al < 1 and ais real (a) Determine the pth-order all-pole model Ap(2) for 2(n) where p is an arbitrary positive integer, and find the value for the squared error £5, (b) For the all-pole model determined in part (a), what is the limit of A,(z) as p > co? What does & converge to as p+ 00? Justify your answers. (6) Repeat parts (a) and (b) for jaj > 1. Solution (0) With x(n) = 6(n) — a8{n ~ 1), note that the autocorrelation sequence is ro(k) = (1 + 02)5(K) — afte — 1+ 6k+ 0] ‘Therefore, the autocorrelation normal equations for a pth-order all-pole model are lta? -a 0 0 1 He -a 1+a? ay 0 ay(1) 0 ee o || a2 |-.}0 00. 0 14a? | | agin) 6 or, in matrix notation, ‘Thus, the pth-order model is aa where Ry'uy is the first column of the inverse of the (p-+1) x (p+1) autocorrelation matrix, R,. With Aj = detR,, note that ay may be written as Spt OAy-2 Aya Furthermore, since the fist coefficient of ay is equal to one, then we must have f= Mp Opa ‘Tins, Apt aby-2 =| otaysChapter 4 93 and, for the kth coefficient, Due to the tridiagonal Toeplita structure of R we may find a clased form expression for Ay as follows. First, note that for j =O and j = 1 we have dy = 1407 Ay = (lte?)*-at=1 40% +a! ‘We may show, by induction, that Asaltat tat yg att = Sat = Specifically, assume that this relation holds for A;-1, and let us verify that it must also hold for ‘Ay. Note that the determinant of Ry is (4 0A)dyatadet Ayr where -a 0 o lta? -a oO -a +a? oO 0 o o lta? is the (j— 1) x (j— 1) submatrix formed by deleting the second coluran and the fist row of Ry. Since det Aya = -aAs-2 ‘then we have the following expression for Ay, Ay = (1 407);-1 - a Aya "Therefore, Ay— (0402) Dat at Pat Sat as was to be shown, ‘Ths, for the coeficentap() we have alot) oo (paar) (b) IE we assume that Jal <1, then as p—+ 00, the term multiplying a in ap(k) goes to zero, and (b) = ‘Therefore, n the limit as p —+ 90, the all-pole model ison Problem Solutions {) Now, let us consider what happens when |al > 1. ‘The expression for ay(k) given in Eq. (P4.17-1) still holds, However, as p + 20 we have ea ano-H4) Bao Sor = lim et Spar m2 and, in the limit as p — 90, the allepole model is For the squared error, we have ‘Thus, for fa] <1 we have and, for fal > 1Chapter 4 95 4.18 Find a closed-form expression for the FIR least squares inverse of length V for each of the following systems. (a) Glz) = (b) Gt) (©) Ge) Solution (a) Since hyy(n) = 8(ne) ~ a(n. ~ 1) (b) To find the least squares inverse of Gl)=1-2 ‘We must solve the linear equations Ryhy = 9(0)m i Since ‘and (0) = 1, then these equations become 2-100 hy(O) 1 fei hy(1) 0 o-1 2 hy) |=] 0 The solution to these equations (see Example 44.5) is of the form y(n) 1 tne where ¢; and cz are constants that are determined by the boundary conditions at n = 0 and n=N-1, hy (0) —hy(1)=1 5; -hy(N ~2)+2hy(W 1) =0 Using the given form for hy(n), these boundary conditions become Ber) ~ (er +09 1 + (N= 2)ea] + 2fer + (N— A)ex)Problem Solutions Solving for es and ca we find 1 ae Nel eared ‘Therefore, for n= 0,1,...,N — 1, we have N-n y(n) ae and hy(n) = 0 for all other values of m. Again, to find the least squares inverse, we must solve the linear equations Ryhy = g(0)uy Note, however, that G(e) is an all-pass filter, late) ‘Therefore, glk) = of) + a(-K) = (8) and the least-squates inverse is y(n) = 6(n) Note that the least squares inverse is the same for all systems that are related by an all-pass filter.Chapter 4 97 4.19 An important application of least squares inverse filtering is deconvolution, which is concerned with the recovery of a signal d(n) that has been convolved with a filter g{n) x(n) = d(n) + 9(n) ‘The problem is to design a filter hy(n) that may be used to produce an estimate of a(n) from a(n), : ain) = x(n) + hy(n) One of the difficulties, however, is that noise in the observed signal may be amplified by the filter. For example, if we observe uln) = dln) « gfn) +vln) then the filtered observations become y(n) * y(n) = din) + v(n) # hy(n) where u(n) = u(n) « hy(n) is the filtered noise. One way to reduce this noise is to design a least squares inverse filter that minimizes = yam? + abthene) where e(n) = 4(n — no) — h(n) + 9(n) and \ > 0 is a parameter that is to be selected. Note that for large values of A, minimizing E will force a large reduction in the filtered noise at the expense of a decrease in resolution, i.e., larger e(n), whereas smaller values of 4 lead to higher resolution and Inrger noise, (a) Assume that v(m) is zero-mean white noise with a varianee 02. Show that Ef\u(n)?} = 93 bY ts of the filter hyy(n). (b) Derive the normal equations that result from minimizing the error where hy is a vector containing the coeff E=eMe+Aoz h¥hy where e = [e(0), e(1), ...]", and show that they may be written in the form (Ry + al)hy = Bry where a > 0 isa prewhitening parameter that depends upon the values of A, and gis the veetor on the right-side of Eq, (4.101).98 Problem Solutions Solution § {a) From Eq, (3.90) on p. 101, we have P{luln) 2) = WER hy Since 21 then Pf lu(n)P} = 02 bly. ‘as was to be shown, (b) The error that we want to minimize is es Slew? + AB{|u(n)/?} where en) = 6(n = no) = hn) + gn) a Ka B{\u(n)|?} = of b¥hy = 03 Yh? ‘To minimize the error, we set the derivative with respect to jy(k) equal to zera for kt = Odeo, (n)g" (a8) +2 08 hth) Substituting for e(n) we have =¥ [sin no} ~ hv lDgl— Dore —8) +2 08 vit) =0 =a" (ro B) + LO[> vl alr =D] o"(n =f) +9 oF hy(k) =0 Interchanging the order of summation yiel ~s'(r0 8) +S yl [Soaln—Dot(n = H)] +202 hth) <0 wat i rolk = 1) = Salm Dg" (ab) it follows that . YT hy(Org(k = 1) +2 08 hay(R) = 9" (no ~ 8)Chapter 4 Written in matrix form, this becomes (R, + al)hy = 8%, 99100 Problem Solutions 4.20 We are given a signal, x(n), that we want to model as the unit sample response of an all-pole filter. We have reason to believe that the signal is periodic and, consequently, the poles of the model should lie on the unit circle. ‘Thus, assuming a second-order model for the signal, the system function is constrained to have the form . (0) T¥a(t+ With [a(1)] < 2 this model produces a pair of poles on the unit circle at an angle of @ defined by H(2) 2eosd = ~a(1) (a) Using the autocorrelation method, derive the normal equations that define the value of (1) that minimizes the arror &=Lein) ca (b) Pind an expression for the minimum error, {&)}in Solution (®) The error that we want to minimize is = ew) where elnjz(n) + a(1)2(n = 1) + a(n 2) ‘To find the value of @(2) that- minimizes €,, we set the derivative of € with respect to a(1) equal to zero, a, way = 24 ase Since the partial of e(n) with respect to a(1) is x(n ~ 1), the normal equations are Yetmetn-1 =0 > ete) + atabe(n 1) + x(n —2)}e(n—1) =0 With . ~ ral = Salmon A) these become . a(1)r2(0) = -2re(1) ‘Therefore, a(t) = ~2ra(1)/re(0) (Note that |a(1)| <2)Chapter 4 (b) To find the minimum error, we have Evbain = Elem + a(t}a(n— 1) + a(n —2)]e(n) = Slew +4(n~2)]e(n) " Y [xe + x0n—2)]fel—) + ot0)e(u~ 1) 2l0-2] 2fre(0) + a(re(0) + ra(2)] 101102 Problem Solutions 4.21 Voiced speech may be modeled as the output of an all-pole filter driven by an impulse train Pna(n) = > 5(n — kno) where the time between pulses, no, is known as the pitch period. Suppose that we have a segment of voiced speech, and that we know the pitch period, np. We extract a subsequence, x(n), of length N= 2ng and model this signal as shown in the following figure Pro() 0) T+ EEa p(B * where the input, Pag(1), consists of two pulses, Pno(1) = 6(n) + 6(n ~ np) Find the normal equations that define coefficients ¢p(k) that minimize the error ya 5- ¥en) = where e(n) = apf) #260) ~ Hn) Pag (n) ‘and b(n) = 6(0)4(n). Solution ___ IE we define @,(0) then the error e(n) is etn) in(n) + 2(n) — b(n) * Daoln) = ¥> ap(Dar(n — £) ~ b(0)[5{n) + 5(n — no)] land the mean-square error that we want to minimize is Fem = [Eaox-9 Hos) Hos) Setting the derivative with respect to aj(K) equal to zero, we have 2 ee Beniy ~ 35 2| Syma Mosey— Hoste] If we define ang t reli) = 35 (nDa(n— 8)Chapter 4 103 then the normal equations become (recall that (0) = 3) regtreths 0) ~ HO.z(£) — H0)2(m — &) = —relk,0) + Assuming that z(n) = 0 for n < 0, with x = [2(nq—1),2(no~2),..-2(n0~p)| the normal equations ray be written in matrix form as follows Raa 10} Finally, differentiating with respect to b(0) we have sa = = Y2]¥ aglt)atn — 1) — 0(0)8(n) — b(0)6(n ~ n9)| [5(ra) + 6(n~ mol] Thus, (0) ~ 2(0) + > aplt)x(0m ~ 1) ~ 0) = (0) fr, in vector form, we have xTa—24(0) = Putting all of these together in matrix form yields [= f][ +o] .2(0) ~ 2{) eG ]104 Problem Solutions 4.22 You would like to design a linear predictor of a signal x(n) but, due to hardware constraints, are limited to a two-tap predictor. However, since delays can be tolerated in the predictor, you decide to design a predictor of the form (n) = a(1)2(n — Ni) + a(2)x(n — No) where Vy and Nz are positive integers, The goal is to find the values of a(1), @(2), Ni, and Np that minimize the mean-square error B{e%(n)} where en) = x(n) ~ 2(n) Assuming that 2(n) isa zero mean wide-sense stationary process, you estimate the autocorte- Tation of x(n) and find that the valios of re(k) for k = 0,1,...,7 are as given in the following table. For k > 7, it is determined that the autocorrelation is approximately zero Autocorrelation values: bes (Ossi Lisa et ered aeeO ean OheT 02 06 02 0. (a) Ifyou were to design an optimum predictor of the form #(n) = a(1)xr(n—1), what would be the mean-square error in your prediction of 2(nJ? What about for the predictor 3(n) = a(I)e(n — 3)? (b) Derive a general expression for the minimum mean-square error for a predictor of the form 2(n) = a(1)r(n~ Ni) with your expression given only in terms of autocorrelations ra(k). What value of Nj minimizes the mean-square error? (c) Find the values of a(1), a(2), Nj, and Ny in the two-coefficient linear predictor defined above that minimize the mean-square error B{e*(n)}. (a) Find the value for the mean-square prediction error for your predictor designed in part ©) Solution (a) With a prodictor of the form £(n) = a(t)2(n— 1), the value of a(2) that auinimizes the mean square errr i _ ral) (0) a(t) = fand the minimum error is (0) ~ r2(1) r2(0) Gs = re(0) + a(d)re() = For the given autocorrelations, this becomes Es = 0.99Chapter 4 {b) © 105 On the other hand, with a predictor of the form #(n) = a(1}x( — 3), the value of a(3) that minimizes the m a (0) re(0) and the minimum mean-square error is vs = 10) 09) =n) 2 = BOD For the given autocorrelations, this becomes Gos = 0.75 For a predictor of the form #(n) = a(1}2(n ~ Ny), the optimum value for a(t) is att) = 72080 (0) tnd the minimum mean-squaxe error is us = r+ ofp) = (0) = EOD =D) ‘Therefore, we want to pick the value of Ny that maximizes the absolute value of r_(N;). For the ssiven set of autocorrelations, the value we want to select is Ny = 5. With a linear predictor of the form Bn) = a1}n(n ~ Ny) + a(2)2(n ~ Ne) the normal equations that define the optimum set of predictor coefficients are found by differen- tiating the mean-square error with respect to a(t) and a(2) and setting the result equal to zero, Thus, with = 2B{e(n)a(n— M)} aa Bey ~ 2 lo)ein Ny) =0 wwe have Bf [2(0) ~ a(2)x(n~ N) = a(2)z(n — Na)]e(n—N)} = 0 B{ f2(n) ~ a(t)a(n ~ Ni) ~ a(2}x(n - Ny)Je(n— Ny} = 0 Therefore, at)ra(0) + a(2)re(Mi ~ Na) = rel Na) a(t)re(Na~ 4) +a(2)rc(0) = re) on, in matex form, [ction Mey [8 ] = [2883106 Problem Solutions Based on the given set of autocorrelations, the values for Ny and Nz that: minimize Gans = re(0) + a(tra(N1) ~ a(2)re(Na) My tnd the coeficient values are a(t) = ~05 a(2) = 06 (@) ‘The minimum mean-square error is Gaus = E{e(n)2(0)} = (0) — a(t)ra( Ns) ~ a(2)7a(Na) ‘which, for the given values of Ni, Nay a(1), and a(2), is €= 039Chapter 4 107 4.28 If r9(0) = 1, re(1) = 05, and rz(2) = 0.75, find the values of a(1), a(2), and 4(0) in the following AR(2) model for z(n), a(n) + a(1}2(n ~ 1) + a(2)e(n — 2) = b(0)w(n) where w(n) is unit variance white noise, Solution —___ ‘The equations we want to solve, Rea = —r,, are as follows Therefore, and108 Problem Solutions 4.24 Use the method of spectral factorization to find a moving average model of order 2 for a process whose autocorrelation sequence is ry = (3, 1.5, 17 Solution Given ry [8 1.5, 1] the power spectrum of x(n) is : Pa(z) =B415[8 +2] + [P+ which isa fourth-order polynomial, Using the method of spectral factorization, we must factor P,(=) in the form Ps(s) —o8[L +) s+ O(a} f+ BCa)= + 8(2)=4] Either by tafal and error, or using a polynomial factoring program or MATLAB, we find that the factorization is Pale) = Slt ht 24] [Lt 2422] ‘Therefore, a moving average model for x(n) is What should be learned from this problem is that spectral factorization is a computationally dificult solution to the MA modeling problem,Chapter 4 109) 4.25 Suppose that the frst five values in the autocorrelation sequence for the process x(n) are nr, =[B, 9/4, 9/8, 9/16, 9/32 (a) Use the modified Yule-Walker equation method to find an ARMA(1,1) model for x(n). (b) Are the given values in the autocorrelation sequence consistent with the model that you found in part (a)? Solution (a) As we did in Example 4.7.1, we would like to find an ARMA(1,1) model for 2(n) that has the given autocorrelation values. Since the Yule-Walker equations are r(0) re) ] x0) re(1) r2(0) =| (1) [2 8 ] Loto] [ ° then the modified Yule-Walker equations for a(1) are re(I)a{) = re(2) which gives ay (1) = ~re(2)/re(1) = —1/2. For the moving average coefficients, we begin by computing ¢y(0) and ¢y(1) using the Vale- Walker ‘equations as follows rx(0) re(1) 1 )_f a0) [8 re (0) | [ ay(1) | [ all) ] With the given vals fo (sng (2) = 1/2, wend (o8)-[¢ ][-te]-[ 58] [Ge], = Brg Multiplying by A¥(1/2" [Crt] ARG /s*) — OB + $27) (1 ‘Therefore, the eausal part of P,(2) is 1 2 we have [Po], = fcr] ataye")| = $43 Using the symmetry of Py(2), we have CaARA/2*) = By(2)BR(1/s* Performing a spectral factorization gives Pyle) = BEB*A/2") = Fa +-HU+2) so the ARMA(I,1) model is0 Problem Solutions (b) Yes. The model matches ra(k) for k= 0,1,2, and for & > 2 note that ra(k) = Bre( 1) which they do.112 Problem Solutions SOLUTIONS TO CHAPTER 5 The Levinson Recursion [1] Given the sequence of autocorrelation values, r2(0) ra(l) =08, r2(2)=05, re(3)=0.1 Find the reflection coefficients, I, the model parameters, a;(k), and the modeling errors, ¢;, for j = 1,2,3. Solution _ ‘This problem is a straightforward application of the Levinson-Durbia recursion, Far the sequence of ‘utocorrelations 7(0)=1, re(t)=08, r2(Q)=05, ro(8)=01 the reflection coefficient sequence is T=(-08, 03889, 0.4727)" ‘The model parameters a;(k), therefore, are 1 ° 1 p=) 2M | yp, | 9889 | _ | -0.9273 S= J ossea | +7) ran |= | -o.1364 0 1 ara Finally, for the model errors, we have 6 = r(0)=1 «(1-1 = 0.36 @ = a(1-13) =0.3056 @ = @(1-1}) =02373Chapter 5 113 [2] Let 4p1(2) be a polynomial of order (p — 1) of the form mt Ap-lz) =14 YF ap-a(h)e (P52) and lot Ty,T2,...,Dy-1 be the reflection coefficients that are generated by the Levinson- Durbin recursion, A’ pthvorder polynomial is formed via the Levinson update equation as follows Ap(2) = Ap-1(2) + Tp2PAS_s(1/2*) (@) IEP] <1 for the zeros of yl)? (6) Suppose Ay(2) may be factored as 1,...,p— Land if [Pp] = 1, what can be said about the location of » Ap(2) = T] 0 ~ ax27) isi ies the zergs of Ap(s) are at a1,a2y-.-yap. If Ay(2) is a polynomial with reflection coefficients F where Ty for G=1,2...,p—1 R= tt How are the zeros of Ap(2) related to those of Ay(2)? (€) If we consider Ap(z) in (P5.2-1) to be a continuous function of the reflection coefficient Tp, using your results derived in parts (a) and (b), deseribe how the zeros move in the plane as Ty is varied in a continuous fashion from some number, say e, to its reciprocal, Ve Solution (a) IE) <1 for § 1, then the zeros of Ay-1(2) are inside the unit eile, With IPp| = 1, let Pr, 7. We then have Ap(2) = Apei(z) + eM=°PAS_(1/2") Conjugating and replacing = with 1/2", AS(L/2") = Ap_a(I/z") +e 2PAy_1(2) ‘Multiplying both sides of this equation by es" gives eM PAL(L/2") = Ap-ale) +P AS, (1/2) which is equal to Ay(). ‘Thus, with Ap(z) = e27PAR(L/2")ut () © Problem Solutions it follows that if Ap(2) has a zero at 2 = 29, then Ap(2) must also have a zero at * = 1/25. Now, recall that if[P;| < 1 for j = 1,2,-..,p then the zeros of A,(2) must lie inside the unit citele, no matter hovr close [Uy] may be to one. Since the zeros of A,(2) move continously as Tis varied, the zeros of Ay(2), which lie in reciprocal pairs when T', = 1, must all be on the unit eiree. By dofinition, we have Ay(2) ate) Apealz) + p2PAp_ (1/2) Ap-ale) + (Q/T5)2-P45 (1/2") Multiplying both sides of the expression for 4y(=) by 1% yields TAs pAp-a(a) +27PAp 1/2") Conjugating and replacing = with 1/2” this becomes TpAG(A/2*) = VpAp-s(1/2") + 2p als) Multiplying both sides by =~” gives (1/2") = PT pAj. (1/2 + Apal2) and we see that the right-hand side is equal to Ay(=). Therefore, aye") Ags) = Tye PA and it follows that 45(1/2") is qual to zero when Ay(2) is equal to zero, In other words, the zeros of Ay(z) are reflected about the unit citle, so that n zero at 2 ~ 2p in Ag(2) beoomes sero at 2 =1/35 in (2) ‘As increases from Ty = cto Ty — 1, the zeros of Ap(2) move towards the unt circle. When Tr, =" all of the zeron le onthe unit circle, AST, increases beyond 1, the zetos move outside te unit eine and spproedh thelr miror image location as Cy 1/Chapter 5 5, [3] Let ap(k) be the set of filter coefficients corresponding to the reflection coefficients Py, for ba1,2,..4p. (a) Prove that if the reflection coefficients are modulated by (—1)* Be =(-1)'Te then the new set of filter coeffi ents p(k) are p(k) = (1S ap) (b) Can you generalize the result in part (a) to the ease in which f=a'Ty where a is a complex number with a] =1? What about if jal <1? ——— (a) We want to show that, if i, = (—1)*T,, then p(k) = (Lag () or, if we let Ay(s) and Ay(s) be the ztransforms of p(f) and ap), respectively, we want to Show that Ale) = Ant ‘We begin by noting that, for p = 1, we have Ag) =14M2 and Ay(2) = 1-12 t= ay(=2) ‘Therefore, let ws assume that Ap(z) = Ap-a(-zh and show that Ay(2) = Ay(~2). From the Levinson order update equation we have Ap-ala) +0 p= PAs (1/2) and Fy) = Ap-a(z)+ Bye? Ay_(1/2") = Ay ale) + (UP p= PAL (0/2) ‘Thus, a Ale) = Ap-a(—2) + Epl~2) PAG al and we have the desired result16 Problem Solutions (0) If, =P with ol = 1, then we may write oy, for some real number @. As in part (a), for p= 1 we have Ai) = 14027 and Asya 14 etryet = Ales) Therefore let ws ass that 4, we have Ay-1(e"!2). From the Levinson order update equation Age) = Apaale) +P p= 745 (U/e") and Fale) = Fpals)+ Py? Ape") = Fyale) + oP Ty2 9A (1/2") Tin, Ayl2) = Ay ales) +OMT 2 PAS (eH /2 and we have the desired result, = Aple*2) Ae) = Agle*2) If al < 1, then the eoefcients change in no predictable manner. Consider, for exemple, the case of a second-order model, 1 y(1+Ta) Pr 1 onary | oT TP. = oT and ovis real, thenChapter 5 uz [4] For the reflection coefficient sequence T, k=1,2, with |a| < 1, prove that the zeros of the polynomials p(k) lie on a circle of radius a for every p21. Solution —_ ‘We will show this by indnetion. For a first-order model, we have Ai(z)= 14027! Which has a root at 2 = —a. Now, assume that Pe =a and that Ay—1(2) has all of its roots on a. circle of radius a. If we anultiply the sequenee ap-1(k) by a~*, then the z-transform of the sequence tp-a(ets) Ay-a(2) is symmetric, 2DA, (2-4) = Ap-a(aa) =F Ay (az) Now, for Ay(2), we have Ap(2) Ip-i(2) +aP2-PAy (2) Ie we multiply the coeificients ap(K) by a, then the z-transform of this sequence becomes Ap(2) = Aplaz) = Ay-1(az) + a%(az)" Ay 1(a/2} Ay-a(az) +27 Ay-(a/2) Since A_sp(z) is symmetric, then so is Ap(2), and the roots of A,(a) either Iie on the unit circle or in reciprocal pairs, ie., if 2a isa root, then so is = 1/a. However, since Ay(z) has al of its roots inside the unit cirele (IP,| < 1), then the roots of 4y(z) must le on the unit circle. ‘Thus, it follows ‘thatthe roots of 4y(=) must le on a crcl of radius aus. Problem Solutions [5] Without factoring any polynomials, determine whether or not a linear shift-invariant filter with system function 140.827! - 0.927? +0.32- T-092* 0.82? 052 is minimum phase, ie., all the poles and zeros are inside the unit circle, H(z) Solution ‘The reflection coefficients corresponding to the numerator coefficients, b [1,0.8,-0.9,0.3]" is T= [-4.6522, -1.2527, 0.3)” Since [Py| > 1 and [['2 > 1, then the mmerator polynomial is not minimum phase (there is atleast one root outside the unit circle). However, since the reflection coefficients of the denominator polynomial are T = [-0.4545, 0.4667, ~0.5)". then the filter is stableChapter 5 19 [6] Consider the signal x(n) = 6(n) + 8(n — 1) Suppose that we observe x(n) for n = 0,1,...,., (a) Using the autocorrelation method, find the 2nd-order all-pole model for (7). (b) Suppose we want to find a p-pole model for x(n). Let Pj denote the jth reflection coefficient of the lattice filter implementation of the all-pole model. Find a recursion for the reflection coefficients that expresses [ in terms of Py and Tj-2 Solution (#) Por Ure signal (1) = 6(1) + 86(01 ~ 1), the autocorrelation sequence Is ve(K) = (14 02)0(4) + BAC — 3) + baU + 1) ‘Therefore, using the autocorrelation method, the second-order all-pole model for z() is found by solving the normal equations [28 3B 1/0] : io] re) re(0) | | ax(2) ] =~ | re) Ths, 14P 6] faa > Ps" ste }[28]--[0] and we have [23] 1 [| a |“Tree | oF | (0) The reflection coeficents are given by where ay sre +1) + Day(ireli i+ 1) However, since r(t) = 0 for {kl > 1 then y= a (dre(1) = OTs ‘Therefore, ‘and, since then120 Problem Solutions Thus, @ recursion for T's is given by TyaChapter 5 11 {7] Suppose we have a data sequence whose z-transform is of the form: G in) X() 14) Ss although the value of p is unknown. The coefficients of the model are computed using Levin- son's recursion. How can the value of p be determined by looking at the sequence of reflection coefficients T; for j =1,2,...2 Solution eeeriearan Since (7) may be modeled exactly with an all-pole model of order p, if the autocorrelation sequence for x(n) is known exactly. then Prony's method shonld yield an exaet made! Tanking at the reflection covfficients, we would discover that Ty = 0 for all j > p.122 Problem Solutions [8] Let re(k) be a complex: autocorrelation sequence given by r= [2, 0.5(1 +3), 0.53)" Use the Levinson-Durbin recursion to solve the autocorrelation normal equations for a second- order all-pole model Solution 1. First-order model: n= and 2. Second-order model: and ze 0 . is |e _| 2049) ee | eae at i T? 0 aeChapter 5 123, [9] Determine whether the following statements are Thue or False. (a) If r2(k) is an autocorrelation sequence with r2(k) = 0 for |k| > p, then T, = 0 for [kl > p. (b) Given an autocorrelation sequence, r=(k) for k = 0,...,p, ifthe (p+) x (p+1) Toeplitz matrix Ry = Toep{re(0),r=(1),---,7p(P)} is positive definite, then Pr tr = [re(0),-..,72(P),0,0, will always be a valid autocorrelation sequence, i.e, extending r-(k) with zeros is a. valid autocorrelation extension, (c) If rz(k) is periodic, then I; will be periodic with the same period. Solution (a) False: If Ty =0 for k > p, then this corresponds to an all-pole model which does not have a finite length autocorrelation sequence. (b) False: ‘The condition requited for rs(p-+ 1) to be a valid extension is given by Eq. (5.98) and is illustrated in Fig. 5.15. Clearly, unless the point ra(p + 1) =O lies within the shaded circle, then this is not a valid extension of the partial autocorrelation sequence. For example, suppose re(0) = re(1) = 1. In this case, there is only one valid extension (r(&) = 1 for all k > 1), ‘and the discrete-time Fourier transform of the sequence that is formed by extending this partial ‘autocorrelation sequence with zeros is not a valid power spectrum. (c) False: If r2(B) is periodic, then 2(n) is periodie (perfectly predictable) and the model will have all ofits poles on the unit circle (Ip = +1 for some value of p). For example, if r(k) = 1 forall k, then P= [1, 0, 0, J".wa Problem Solutions [10] In our discussions of the Levinson-Durbin recursion, we demonstrated the equivalence between the following three sets of parameters: © r2(0),re(1),.-ra(P) # p(1),0p(2),---+49(P),B(0) PL Pa Poe For each of the following signal transformations, determine which of these parameters change and, if possible, describe how. (a) ‘The signal x(n) is sealed by a constant C, a(n) = Cx(n) (b) The signal 2(n) is modulated by (~1)", x(n) = (-1)"2(n) Solution, (a) With x/(n) = Cx(n) the autocorrelation sequence is scaled by C2, ie., r4(k) = C®rg(k). For the all-pole model, the numerator 6(0) will change by a factor of C;, but the location of the poles will not change and, therefore, the reflection coefficients will not change. ‘This may be shown in ‘a number of ways, For example, note that the solution to the normal equations Ruts, will not be affected if all of the correlations are scaled by C?. (b) If x(n) is modulated by (—1)", then re(K) is modulated by (1). ‘This results in a modulation of the all-pole parameters by (—1)* (this may also be seen by looking at the normal equations) and, since P' = a;(3), then the reflection coefficients will be modulated by (=1)). Since ¢j = Ga1(L = 19); then ¢ Will be unchanged and, therefore, (0) will not be affected.Chapter 5 [11] The autocorrelation of a signal consisting of a random phase sinusoid in noise is ra(k) = Peos(hw) + 02,5(8) where wy is the frequency of the sinusoid, P is the power, and o3, is the variance of the noise, Suppose that we fit an AR(2) model to the data. (a) Find the coefficients ap = (1, ax(1), a2(2)]" of the AR(2) model as a function of wy, 02, and P. (b) Find the reflection coellicients, Ty and TP, corresponding to the AR(2) model (c) What are the limiting values of the AR(2) parameters and the reflection coefficients as a0? Solution (0) The coefficients of @ second-order model for a(n) ate found by solving the normal equations 72(0) re(1) Jf an(t) ] _f re) [ea rio [2 |=[2203] Given thot the autocorrelation sequence of a random phase sinusoid is a(R) = Peas( ha) + 26() then these equations become P+o% Peosup ][ a1) ] _[ Poosur [rowm bet |[20)]-[ rete ] Solving these equations for ap(1) and a3(2) we find [a |= Preosan(P +03) — P* cosut c08 2a (2) Peony [ 2 cosy + Peon 2uy(P +08) Pre (b) For the reflection coefficients, we have w N=) ~~ Peas and : Ty = (2) = Piceten= Peosdun(P+ 08) (Pe ok Peo ay [381-771 and, for the reflection coefficients, (6) If we let 0 — 0 then Ty =coswy and Py126 Problem Solutions [12] Given that r2(0) = 1 and that the first three reftection coefficients are Py = 0.5, P and Ty = 0.25, 05, (a) Find the associated autocorrelation sequence (1), r(2),r2(3). (b) Find the autocorrelation sequence r9(1),r(2),r2(8) for the ease in which the reflection coefficient Ty is a free variable, Le., solve for the autocorrelation values as a function of ts. (©) Repeat part (b) when both P2 and Ls are free parameters Solution (0) From the given rellection coefficient sequence we frst find the vectors a1, az, and ay. For ay me 7 »-[Al-[h] and for aa 1 ° 1 aos] oa(t) | +02] a(t) | =| 3/4 o 1 12 and fora, 1 ° 1 a(t) ox(2) |_| 7/8 aa) | *T° oat) | = | ais ° 7 yA Using the inverse Levinson-Durbin recursion, we may find the utocorelaion sequence. Fist, wwe have re(1) = Dirs(0) = -1/2 and a= rl 1H) = 3/4 Now, with maton = 8 and 41 = 72(2) ban(1)ro(1) solving for r(2) we have re(2) = 71 ~ an(1Jre(1) = 1/8 and eis e=al-1)=9/6 Finally, with a= Trea = -9/64 and 2 = ral3) + aa(1)rs(2) + a(2)re(1) solving for r,(3) we have (8) = 92 ~ 02(1)re(2) ~ 02(2)re(1) = 18/64Chapter 5 127 ‘Thus, the autocorrelation sequence is r=[l, -1/2, -1/8, 13/64)" (b) From part (a) we have ¥4(8) = ~€P 3 ~ an(t)re(2) ~ a2)ra(1) ‘Since e2 = 9/16 then 72(8) = fl ~ an(1)ra(2) ~ an(2)ra(1) = Blo + (c) With Ty a free parameter, the vector ay becomes, ~ [us] Also, ¢p = ex(1~ 13) = $(1 = 13), Therefore, re(3) = ~ $V a(1 ~T9) ~ 3(1 +Ta)ro(2) — Pare(d) However, note that (2) = -aP2 ~ ar(1)re(1) = "Thus, with r(1) = -1/2, we have 14(3) = a+} $s(1 12) — 3+ Ta) + Yt Hla128 Problem Solutions [13] Using the autocorrelation method, an all-pole model of the form hhas been found for a signal x(n). ‘The constant in the numerator has been chosen so that the autocorrelation of the signal matches the autocorrelation of the model, ie., re(h) = ra(k) where 1(b) is the autocorrelation of h(n). In addition, you know the following about the signal and the model: 1 r(0)—4 2. Ts =05 3.7; >and Py >0 4. 2(0)=1 6. ». €g = 11/16, where ¢ is the third-order modeling error. det(R2) = 11, where Ry = Toep{r's(0),r+(1),re(2)}- Find the values for the model parameters a(t), a(2), a(3), and 6(0) Solution We ave given @=e0-1) =1/16 Since Py = 1/2, we may solve for ep as follows, Now, with dot (Ra) = eoeree = 11 and &9 = re(0) = 4 we may solve for 1, oa oa PU We may now solve for the reflection coefficients, With =e -T3) tnd, since I) > 0, thenChapter 5 129 For the second reflection coefficient, we have e@=at-T) Thus, tnd, since T > 0, then Finally, we may generate the coefficients ay() using the step-up recursion, [RE Tees and 1 ° 1 _ | ae sis |_| a/a a=! se | +¥8l ane | =] sie 0 1 1/2 Therefore, Agla)= 14 get Bete fet and Wo) = va= Vii130 Problem Solutions [14] The first seven values of the unit sample response, h(n), of a 3rd-order linear shift-invariant filter dt bade t+ bQ)e 84 163) Tale + a(Q)e2 + ala He are given by ll, 1/4, 1/2, 1, 0, 0, 7/8)” Determine whether or not the filter is stable. If more information is needed, state what is required and make the necessary assumptions. Solution Since h(n) isthe unit somple response of a linear shift-invarlant filter having 3 poles and 3 zeros, given the first 7 values of h(n) the system function may be found using the Padé approximation, For the denominator polynomial coefficients we have ha) Atg-1) Aq-p+) ap(1) Ag+) hag+) ‘A(g) A(q—p+2) ap(2) |__| Ma+2) Natp—1) Wg-+p-2) na) ole) Map) With p and using the given values for h(a) we have 1/2 1/4] asta) o o 1 12} asf) |=} 0 oo 1 JL as) 78 (1) = Solving for as(k) we find a3) = 7/18 + ‘and, therefore, wwe have, for ay [2a ]- [803 }} [a8] Ty = aa(2) = 28/15 > 1 ‘Therefore, and the filter is unstable,Chapter 5 131 [15] ‘The extendibility problem in power spectrum estimation concerns the issue of whether or not a finite length sequence, ra(1)s7e(2)sr2(8),--- sel) may be extended (extrapolated) into a legitimate autocorrelation sequence in such a way that YS ralbye (P5.15.1) is a valid power spectrum. In other words, is it possible to find values of ra(k) for jk| > p such that P,(e) in (P5.15-1) is a non-negative real function of w? Peel) {a) Develop a procedure that uses Levinson’s recursion to test the extendibility of a sequence. (b) Use your procedure developed in (a) to determine the constraints ou a aud & Ual ate necessary and sufficient in order for the sequence re(O)=1, re(1) =a, r9(2)=6 to be an extendible sequence. (©) Assuming that the sequence in (b) is extendible, find two different legitimate extensions. Solution (a) In order for a sequence r4(0)sr4(I)y+--sFe(9) to be extendible, it is necessary and sufficient fr the corresponding reflection coeficient sequence to be bounded by one in magnitude, This moy be Justified ss follows, Suppose that r4(0),re(1),---+re(p) is extendible. Then the autocorelation Tnatrc Riot be postive defnite and, ase know, this imple thot [fy] < 1. Converse tuppowe that Iyl-<1 fr j= 1ensp, Then tho soquace wif Ty: Ghee me{ Ql 5S3 which corresponds to an all-pole power spectrum, represents a valid extension of the reflection coefficient sequence and a valid autocorrelation sequence extension, (b) With 12(0) = 1, ra(1) =, and r(2) = 6, we require that Ijsa 0 and find the all-pole models Ay(k) for all p > 1. Solution With an autocorrelation sequence ra{k) = 02 6(K) for the first reflection coefficient. we have, [Note that since ra(K) = 1 for k 2 1 then [reG + Varels--- sre] =14 DS ay(t) (0) “ From the Levinson-Durbin coefficient update equation we have 2 1 0 a,(1) aj-1(1) as —1) : |= +0; G1) 4516-0) aj-a() (9) 0 1 Since 7j is the sum of the coefficients a,(), and +1 is the sum of the coefficients aj_s(k), then y= wall thy) with it follows that ‘Therefore, (P5181) 1/(1+ 6) it follows that ‘and, in general, we have by induction, For the model, note that136 Problem Solutions and wLidelél feds] However, from Eq. (P5.18-1) we see that, in general, Tyr — TP = 25 and Tig =D) 4 Byas) (P5181) ‘Therefore, 1 a= | Te tT Our elaim, then, is that 1 Ty for all j. ‘To show this, assume that this holds for a.1, ie Tjaa yay Tye)” ‘Then 1 ° 1 Ta Tpal+ty) as +1 = Ba Nya Tad +ry) o 1 ry and, using Ba. (P5.18-1), the result follows.Chapter 5 137 [19] A pth-order all-pole model for a signal x(n) is parameterized by the p +1 parameters ey and ap(Ke). Since the reflection coefficients P for a stable model are bounded by one in magnitude, they are automatically scaled. In speech processing, therefore, there has been an interest in coding a speech waveform in terms of its reflection coefficient sequence. ‘The relationship between a reflection coefficient sequence and the spectrum, however, is not easily discernable, Consider, for example, the following three reflection coefficient sequences 3.Tp kyr Although the only difference between these reflection coefficient sequences is in terms of the sign of Ty, the power spectra are quite different. For each of these sequences, find the corresponding autocorrelation sequence, r(k), and power spectrum, P,(e), in closed form, Solution (a) Since the reflection coefficients only determine the autocorrelation sequence to within a scale factor, let os normalize ra) s0 that r4(0) —1. Then nfl) =—T= Note that since A Mee ie then pp ng yk? - (e+1). MRT RFT Since ¢» = re(0) = 1, solving this recursion for e, we have lke tee Now, let us find r(2), ra) = 1 ~ an(l)re(1) = -aPa +} =H) +4 ‘We will now show by induction that r,(&) = D for all k > 2. Assume that (2) = r¢(3) = +++ = a(R) = 0. Then b+? Ry re(k +1) ye ~ an (hre(1) = ~ebiga + $e = as was to be shown. Thus, we have afk) = 8(h) ~ 0.560% —1) — k+1) which has a power speetrum, Pele")138 Problem Solutions (b) Note that this reflection coefficient sequence corresponds to the one given in Problem 5.18 with o2 = 1. Therefore, ral) = 5(h) +1 which corresponds to a power spectrum Palel?) = 14 2x6(u) (c) Since these reflection coefficients are abtained from those in part (a) by multiplying by (1), then the autocorrelation sequence is (—1)* times the autocorrelation sequence found in part (a), re(k) which has a power spectrum 5K) + 0.54(k — 1) + 0.55¢k +1) Palo) = 1 4 conusChapter 5 Bo [20] Let 2(n) be a random process with autocorrelation sequence ro(k) = (0.2)! (a) Find the reflection coefficients Py and T, for a second-order predictor and draw the lattice filter network. (b) Suppose that uncorrelated! white noise with a variance of ¢2 = 0.1 is added to 2(n), y(n) = 2(n) + w(n) How do the reflection coefficients change? (c) Can you make any general statements about the effect on the reflection coefficients when white noise 18 added to a process? Solution (a) Since the autocorrelation sequence corresponds to an AR(1) process, the second-order prediction error filter is A(z) = 1~0.22~, and the reflection coefficients ate Fy = ~0.2 and Ty = 0. The conresponding lattice filter is shown below, Ht (b) IF white noise added to 2(n), then the autocorrelation sequence is ny(h) and the equations for the second-order predictor are [8 E81 Le) ra(A) + 0.18(K) The predictor, therefor, is a= [1, a(1), a(2)]” = [1, -0.1979, -0.0004]” ‘Thus, the fist reflection coeficient is smaller than in the no-noise case, and the second reflection coofficient is non-zero. (c) Generally, the addition of white tends to moves the poles closer to the origin,140 Problem Solutions [21] The reflection coefficients corresponding to a third-order all-pole model of a signal (n) are T,=025 T2,=050 T3=0.25 and the modeling error is given by es = (15/16)? (a) Find the direct form filter coefficients, a3(K), for this third-order model. (b) Find the autocorrelation values r,(1), r2(2), and r-(3) that led to this model (6) Ifa fourth-order model were found for 2(n), what value or values for re(4) result in the rminintam model error, ¢4? (a) Repeat part (c) and find the value or values for r-(4) that produce the maximum error, & Solution (a) Using the step-up recursion, we find that the transversal filter coefficients are fayifa w=(a]-[se] 1 ° 1 o=[te] | te]=[24] 3 1] Liz 1 0 1 3/8 12 12 a= |p w(B]-] a ° 1 “A (b) Using the inverse Levinson-Durbin recursion, we have o> (#8)? = re(0)G = IG a - ‘Therefore, r-(0) = 4/3 and (OEP Doaaliret2~ 1) = ~ealt)ra(t) ~e3(2)r2(0) = 8 DY aali}re(S - &) = ~as(1)re(2) ~ a5(2)re(1) ~ a3(3)r2(0)Chapter 5 11 (c) From the inson-Durbin recursion we have oP = relF +1) + Da y(i)re(G — 841) r+ 1) = Gl - Day(ra 441) With j we have fr r= (4) Late) = Now, recall that if [Py] = 1 then c =0. Therefore, the ervor is minima if re(d) = P's He) Te + ( #8) = ~co.sTanyr, + 0.72 (= 0.8789 + 0.3872 (2) Using the results of part (e}, wessee that if’, = 0 then ¢, takes on the maximum value. ‘Therefore wwe have (4) = 0.872uz Problem Solutions [22] The reflection coefficients for a two-pole model are Ty = 0.25 and [2 = 0.25 and the “modeling 0 (a) Uf r2(3) = 1 find the modeling error, ¢3, for a three-pole model. (b) If the signal values, x(n), are multiplied by 1/2, ie., y(n) = 0.52(n), find the reflection coefficients and the modeling error for a two-pole model of y(n). Solution (a) To find the error, ¢5, we use co) [Ja =r) We begin by finding re(0) as follows 2 = re{O)(L THY) = (0) ( ‘Therefore, r2(0)= (18)? ‘We now need to find the value for the third reflection coefficient, T's = 72/ea, where (8) + 0n(1)ra(2) + a2(2}re(1) First, we most find 7 as follows, The first and second order models are w=[alsLa] [i }(41- C4 Computing the autocorrelation values, we find re(1) = —Pire(0) (2) = —aa(t)rs(1) ~ an(2)ra(0) Finally, evaluating 2 we have -H- B= 92 = re(8) + an(1}re(2) + a2(2)re(1) Therefore, 1y=-w/a= 22 sont and 6s = ll 13) = 8.006Chapter 5 us. (b) IF the signal values 2(n) are scaled by any number, then the all-pole model will not change. Specifically, if y(a) = 0.52(n) then ry(F) = 0.25r,(k). Therefore, since the normal equations are then they are the same as the normal equations for 2(n) Ras -ry ‘Thus the model coefficients for 2(n) and y(n) must be the same aud, therefore, P' in both eases. However, since a =r [0-19 then the error will be reduoed by 1/4, Le, c= 9/4 = 2.25Ma Problem Solutions [23] You are given the following sequence of autocorrelation values rz= (10, -1, 0.1, -1]" (2) Use the Schur recursion to find the reflection coefficient sequence P',T',T (b) What is the final generator matrix, Gs, equal to? (©) Find the modeling error, ¢s Solution With the autocorrelation sequence Fe = [10, ,0.1, 1], the Schur recursion proceeds as follows 1. Step 1: Define the generator matrix Go 0 raft) rs) re(3) -[ ne a Go= | re(0) rofl) re) v2(8)] [10 -1 0a 1 2. Step 2: From the shifted matric Ae cn Go=[5 io “t ot] {follows that Py = 0.1 and eee el aa i] 3. Step 3: Forming the product ©,G, we find ie ea on al fo 0 0 099 0 eee leat 10 -1 01|=|0 99 -099 0 4, Step 4: From the shifted matrix ,-[9 9 9 -099 &=[0 9 on mom ‘we see that Py = 0 and 5. Step 5: Forming the product 6,G, we obtain a-06,-[22 8 22] 00 99 -099 6. Step 6: Finally, forming the shifted matrix Go, &,- [9 0 0 099 e=[3 50 99 wwe find that Py =0.1 andChapter 5 ‘Therefore, with Ge. = eG: wwe find, that the error isM6 Problem Solutions [2] Let r2(0),re(1),-..,72(p) be a set of autocorrelation values and let Ry be the corresponding (p +1) x (p +1) autocorrelation matrix. Show that cet Rp det 1-79 Rp [et Ra where Ty is the pth reflection coefficient. Solution Recall that Therefore, In addition, since then dot Ry Cars ae 2 = ety det Ryo [deta]? Thos,Chapter 5 ur [25] IF |f,) <1, derive a bound for the coefficients 6; in the split Levinson recursion. Solution ‘The definition of 5, given in terms of the reflection eoeffcients in Ea, (5.149), is a =F) Ty) 5 ‘Therefore, if [Py] < 1 for each j, then oct <2; O 0. Again your method should have less than 4’ multiplications and divisions at each step. (0) Write an expression for the error En, at the noth step of the recursion in terms of the coefficients 9(n) and the coefficients of the least squares inverse filter fi (1). (a) Write a MATLAB program that implements the Simpson sideways recursion. (e) How can this recursion may be used to find the inverse of a Toeplita matrix? Solution (a) If we let ag(k) be the fiter coefficients using « lag a then foChapter 5 149 and the optimum filter coefficients are found as follows. With 2 praia) =281 aE (el0)} = 2Ble(mpxln aH} =0 then 2f{ fren Sasa « a]sm—a-w} aq(0)ra(k =D) =rylk +041) In matrix form, these equations ace r4(0) ra) ral) f aa(0) re(a+ 1) ra(l) 70) re) |} aad) rela +2) : (P5261) relP) rel-2) v0) J Lact) J | rela+r+t) Rya =re(a) Finally, since the mean square error is cp = E (e(nx(n + 1) then w= affrnen -Eeone-nafeosa} = re(0)~ Y7aa(k)re(a+ k+ 1) (0) Let £ = (60,40), +40) be the solution to r2(0)—ra(t) rele) ] f £00) a(t) rat) 1200) no-v 1170] _ | ne tel6) rel) rao) J L 7) relp+)) ic, f= “ay. Note that this equation may be equivalently written as ro(0) rat) <= ret) fo i: ra(l) 70) rate) |} 40) | | 0 rn) me) eo-D |] JQ) |e] a (P5262) rel +3) ral?) ro) JL] Lo where 9 = F(0) + Hre(l-+ 1)150 Problom Solutions is « constant that is related to the modeling error ¢p. Due to the Toeplitz structure of Ry we inay rewrite (P5.26-2) as follows r4(0) ret) ral 41) ] | f00) ° rol) r(0) 0) |] se-n | | 0 - (P5263) ralo) rel0—1) re) 50) o rept} ‘ra(p) ¥2(0) 1 Pn Now, we begin the derivation ofthe recursion. Assume that we have the solution to nO rt 20) ] [ ) et) rll) re rato) | | n 1) m0) @-0] | wa) |_| ne) co re) re) re) | Loot) | L tee) (a=) and that we want to derive the solution to re(0) — re(1) i re(p) ay(0) re(2) re) re) rete) | | a n 0) 0) G1) } fae) |_| me ne rs(0) ralP-D) re) JL ae) | [reo+2 (oc=1). Ifwe augment the vector a with ero and multiply by the Toeplita matrix Ry then r=(0)—ra() ral) rel +1) ] [ a0) (1) ra) r4(0) = ral) re) | | wo) 2) 2 = i (P5.26-6) Fol) ral—1) --r(0) raft) | foto) || rato +2) r+) “ral9) rQ)—r9(0) ° fi where : f= Dorelo +1 — aol) Combining (P5.26-3) and (P5.26-6) we have 0(0) f0) re) aa) fo-0) 2) Roar tnfo: [be ato) 110) r+) 0 1 any ‘Thereforo, if we set n= [rep + 2) - Gly then aa(0) f) re(l) a(t) f=) 720) Tyas +n = utp) £0) rel) 0 1 re(p+2)Chapter 5 Finally, using (P5.26-2) we have (0) ft) a(t) Jp) Roa +n aan) FO) 0 1 where © is some constant. Therefore, let 151 1 ra(l) +e £10) 742) +o = (P5.26-7) flp-1) r(p+1) Lin) r(p+2) 1 = ~40(0) ~ nf) then the first coefficient in the sum of the three vectors above is zero and we may write 7=(0) rat) re) retpe] [0 rte rG) 20) re) re) | | a0) (2) re) rep) re} ret) | | xt) | | rele) relot}) 79) nO 120) J an F042) where we have defined 2x0) salt) Se) J00) eel eee) || ae feel 10 |e| se) 4) 0 1 t0) I-we eliminate the first equation from (PS. 26-7) and use the fect that the leading coefficient in the veetor multiplied by Ry & 2er0 We sce that a) isthe solution to (P5.26-5). ‘The extension of this approach to solve for i =0)1,....p~ Leet gti given ay is straight-forward. In particular, for aati) = aa(i+ 1) + Youn SP—t-1)+ torr SC) and where Yor Ps are coefficients that neod to be evaluated, asap) = Yass + SaaS) [relp +042) ~Eosl/Pp (0) ~ mfp) each value af a” and where Pp = rol0) + > fOrs(1+ 1) o=¥ ra(p +1 ~ Bag(k) are fixed constants that are independent of a:6.1 Design a two-pole lattice filter that has poles at rei and r Problem Solutions SOLUTIONS TO CHAPTER 6 Lattice Filters J? and draw a carefully Inbeled flowgraph of your filter. Ce A second-order filter with roots at = Al) ‘re* has a system function given by 1 = 2reasds") 4 r?2 A first-order lattice filter, on the other hand, has a system function given by Ay(z) = 1+ P27 ‘a second-order filter hus a system funetion [z3)-[3] "EE ]-[t>] Therefore, Ao(2) =1+0y(1-+Pa)27! + Taz and It follows that we want to pick and A flowgraph for this filter is shown below x(n) wn)Chaptor 6 153 6.2 Consider the all-pole filter HO) = my 0a F706 Draw the flowgraph for a lattice filter implementation of H(2) using (2) A Kelly-Lochbaum lattice filter. (b) A normalized lattice filter (6) A one-nuliplier lattice filter: For each structure, determine the number of multiplies, adds, and delays required to im- plement the filter and compare them to a direct-form realization of H(z). Based solely on computational considerations, which structure is the most efficient? Solution The first step in the implementation of this allcpole filter as a lattice filter isto compute the reflection coeflicents. Using the Levinson-Durbin recursion, we find T= [-0.8798, 0.8125, 0.6000) “The structures may now be drawn as follows. For the Kelly-Lochbaum lattice filter we have 16 non eset “o) Note that this filter requires 10 multiplications and 5 adds per output point, and it has 3 delays. For the normalized lattice lattice filter we have on oss ass 0) ao a ‘which has the same number of multiplies, adds, and delays as the Kelly-Lochbaun filter. Pinal, for the one-multiplier lattice filter we have154 Problem Solutions Unlike the previous two filters, the one-multiplier lattice filter only requires 3 multiplications, but 8 adds per output point. As with the other filters, it has & delays.Chapter 6 ast 6.3 Find the system function H(z) for the lattice filter given in the figure below. an) Solution ‘The frst step is to use the steprup recursion on the reflection coeiciont sequence P = [02, 06, ~O]" to find the all-pole models Ax(2), Aa(2), and Ay(2). Beginning with the first-order model, we have w=[1]=[e2] Next, for ag we have 1 02 06 Finally, for as we have 1 0 1 _ | 032 os |_| 026 ™=1 a6 [+h] os2 | =| 0568 0 1 Ol ‘Therefore, the denominator polynomial is Ag(z) =1+0.202"! + 0.5682"? 0.12% ‘To find the numerator Bs(2), we use Bq, (6.50), 8468) = Deaf - 1») Thus, we have (3) = ex(8)a9(0) = 0.1 b9(2) = ea(2)uq(0) + ca(3)an(1) = 0.5260 by(1) = ca(1)ax(0) + ep(2)a2(1) + e9(8)ag(2) = 0.5168 b5(0) = e3(0)a0(0) + es{t)ar(1) + e9(2)a2(2) + ep(S)aa(2) = 0.15 ‘Therefore, the system function is 0.15 + 0.51682"! + 0.52602 1.0000 + 0.3 (2)156 Problem Solutions 6.4 Sketch a lattice filter structure for each of the following system functions. 0 HO ae @) HG) = (©) He) = Solution (a) To implement the filter 2 Teor $0? using a lattice filter structure, we must frst find the reflection coefficients corresponding to the denominator polynomial. Using the step-down recursion, we have He 1 = [0.4698, 0.49)” Next, we must find the coefficients cy(k) that will produce the numerator polynomial Using the recursion eal8) = (8) — SO laa) we have el) = (0 and 4(0) = by (0) ~ ey(2)ay(1) = 2.4698 (note that ay(1) = Ts). ‘Therefore, the lattice filter structure for this system is as shown in the figure below “Chapter 6 157 (b) To find the lattice filter structure for ve frst use the step-down reeusion to find the reflection coeficients, which are T= (0.5, 0.75)" Next, wo use the recursion eqlk) = by(k) ~ Seal a)a3( - 8) to find the coefficients es(). ‘This recursion requires the first and second-order all-nole models, which are fh a(u]” = [1 05)” a = [1, aa(1), a2(2)]” = [1, 0875, 0.75)" ‘Thus, for the coefficients ¢2(&), we have 22) = ba(2) = 0.75 ex{1) = ba(1) ~ ea(2)an(2) = 0.65625 €2(0) = ba(0) ~ca(1)ax(1) ~ e2(2)ap(2) = 0.109375, ‘The lattice filter structure for this system is shown fn the figure below (©) Note that the system fumetion (2) = we same as the system function in part (b), then and the lattice filter is as shown in the figure Js an allpass filter. Since the denominator is the reflection coefficients are T = [0.5, 0.75 below ws)158 Problem SolutionsChapter 6 159 6.5 Determine the system function of the lattice filter shown in the figure below, a0) 1 or 12 Solution ‘This isa third-order latice filter with 3 poles and 3 zeros. Note that since the eoelficients in the upper branch are equal to one plus the coefficient in the adjacent down-going branch, and the coefficients in the lower branch are one minus the coefficient in the adjacent down-going branch, then the all-pole part of the lattice filter uses the Kelly-Lochbaumn lattice. Thus, the reflection coefficients are -02, -0.3, ol" Using the step-up recursion, we find and, 1 0 1 a=] -02 | +12] -02 |=] -o14 0 a 03 and, finally, 1 o 1 0.4 -03 |_| -0.26 -o3 | +l) ou ~0.356, ° 1 oa ‘Therefore, the denominator polynomial is Ag(e) = 1 ~ 0.262" ‘To find the numerator, Bg(z), we use Eq. (6.50), 4(k) = Sega ~ 8) [Note that this requires that we have Aj(2) for j = 2,3. ‘Thus, we have bs(8) = e3(8)as(0) = 0.2 bs(2) = es(2)a2(0) + es(3)as(1) = 0.448 b3(1) = es(1)ai(0) + e5(2)aa(1) + e9(8)as(2) = 0.1588 (0) = e5(0)a9(0) + e5(2)ax(1) + ey(2}aa(2) + e9(3)a9(3) = 0.23,160 Problem Solutions ‘Therefore, the system fumetion is 023-4 0.15882"! + 0.44827? + 0.22 026s? — 0.356? Ode (2)Chapter 6 161 6.6 Shown in the figure below is a split lattice filter. “0 aa vm) Tae wo) . (a) What is the order of the filter (number of poles and zeros)? (b) Does this filter have minimum phase? (c) Find the system function H(z) = Y(z)/X(z). (a) What is the transfer function V(z)/X(z) between x(n) and v(n)? (6) How would you modify this structure to add a zero in H(2) at 2 Solution (2) This filter is a secon-order FIR filter, Therefore, it has two zeros and no poles (b) ‘The filter parameters for this structure are as follows. Plast of all, since 24, = 1.6, then 4, =08 rT which gives Ty =02 Secondly, since f= (1-Ta)(1+0y) = 0.72 then r= 04 ‘Therefore, since [Py] < 1 and F2| <1, then the poles of the filter are inside the unit circle, ie, it has minimum phase, (c) With Ty = 0.2 and P2 = 04, we have [id] Therefore, system function H(z) = ¥(2)/X(2) is H(z) = 140.282" +0427 (a) The sequence v(m) is equal to v(m) = ea(n) =162 Problem Solutions () Since ul) = yl — 1) + ealn = 1) ~ L.5e5(n) if we were to add a zero to H(z), then the output would be 0) vuln) — u(r = 1} ea(n.—1) ~ 1 5ea(n) ‘Therefore, all that we need to do is remove the feeclback path at the end of the structure,Chapter 6 163 6.7 Trueor False: Let H(2) be the system function of a linear shift-invariant filter with coefficients p(k) and bo(k), and let Py and ¢q(k) be the coefficients in the lattice filter realization of H(2) IF ap(k) and by(K) are modified as follows fig(k) = (—1)ap(K) 5 by(&) = (—1)*Bp(K) then the coefficients in the lattice filter are modified in the same way, Le., E(k) = (1)keq(h) Solution ‘This statement is tue. As shown in Problom 5.3, given ap(k) and Ty, the reflection coefficients ‘corresponding to the coefficients p(k) = (~1)Sa,(k) are Ty = (—1)'T'g. Now, for the coefficients ‘q(b), note that they are defined recursively in ters ofthe coefficients by(k) and ap(k) as follows, call) = 48) — > eyGidakG ~ with ea = baa) ‘Therefore, if]y(a) = -1)%64(9) Furthermore, for each k fom 40 we have Hlk) = Belk) ~ S> BGG -*) = (-1yhbg(k) — > (1 e9(9)(-1) Ma G - &) smn = (-thbg(B) = > (-12)9(—1)Feg af (G — 4) jm = "ft - OY edafG-a)] = (-Lfeg(k)164 Problem Solutions 6.8 As shown in Fig, 6.2b, a lattice filter may be used to generate the forward and backward prediction errors, e}(n) and ¢(n), respectively (a) What is the relationship between the magnitudes of the discrete-time Fourier transforms of ef (n) and ¢5(n)? (b) Isit possible to design a realizable filter (causal and stable) that will produce the response e(n) to the input &(n)? If so, describe how and, if not, state why not. (6) Is it possible to design a realizable filter that will produce the response ef (n) to the input ¢5(n)? If s0, describe how and, if not, state why not. Solution {@) Recall thatthe backward prediction errr, ej (n) i elated to the forward prediction eror ef (n) by an allpass filter, BS (=) = Hop(=)ER (=) ‘Therefore, the DTFT magnitudes of e(n) and 5 (n) are equal, IBS (eo)? = [BE (e)P (b) Ifthe roots of Ay(2) are inside the unit circle, jag| < 1, then e; (n) may be generated from > (n) using the stable and causal allpass filter shown in the figure below, « tis not possible, in general, to find a causal and stable filter to produce a response, ef (n), to the input 6; (n). ‘The reason is that, if the roots of 4y(=) are inside the unit circle, then ¢; (n) will be related to ef(n) by an allpass filter that has p poles inside the unit circle and p 2et0s outside the unit circle. Therefore, the inverse of this allpass has p poles outside the unit circle land p zeros inside, and therefore cannot he causal and stableChapter 6 165 6.9 The all-pole lattice filter in Fig. 6.7b may be used to generate the all-pole approximation #(n) to a signal x(n). In this problem, we investigate another use for this filter, Suppose that (0) is initialized to one at time n= 0, and all of the remaining states are set equal to zero. Determine the ontpat of the all-poe filter for n = 1,2,...,.N. Hint: Consider the expression for »y given in Eq. (5.10) of Chapter 5 Solution Before we begin, let is see if we can gain some insight into this problem by computing a few output values, ‘The system that we are considering, for a third-order lattice, is shown in the figure below. nning with time n= 0, we ean easily generate the following sequence of outputs, yO) = 1 u@) = Tr =rs(0)/re(0) 2) = M=TA-T2) Now note that yQ) = VF-naa-Tp) ri) 2 “HO ~ ROTH MY 2G) _ 72) +Pire(t) _ re(2) AO) 0) Ba-rp n THO) 72(0) ‘Therefore, let us hypothesize that un) ‘We will show this by induction, Fizst, let us make « few observations 1. ‘The input acts as an initial condition in the delay line that propagates to the left 2. The last non-zero value in the delay line (at time n= 9) is tlhe ‘This is clear from the lattice filter structureProblem Solutions 166 3. Looking into the all-pole lattice filter, at time j, there is effectively an input, at time j +1, equal to ~Py41¢;/re(0) as illustrated in the figure below, Pie ene Now let us assume that y(j) = re(4)/ra(0)- Since y(j) is equal to a linear combination of outputs, re(j — #) multiplied by the filter coefficients, a,(2), then 2 Sy yind WO 2G - yoy uli+1) = +2) 720)Chapter 6 or 6.10 Consider the following modification to the Burg error N ae 2 ef = Xmen {leronf + lesen} where w,(n) is a window that is applied to the forward and backward prediction errors. (0) Derive an expression that defines the value for the reflection coefficient FY that minimizes the modified Burg error €’. (b) What conditions, ifany, are necessary in order to guarantee that the reflection coefficients are bounded by one in magnitude? Solution (a) Reproducing the derivation of the Burg recursion, we begin by setting the derivative of the error &= Som flere? + le? } at with respect to T} equal to zero as follows ae ~ Lemp (ho? + Ise") a DY ey (mle! wy} Substituting for ef(n) and e; (n) and solving for T) gives 2D wines ato) lepate—" = - Yun {l = (b) For the Burg recursion, the property that |[| <1 is based on the inequality 2l(a.b)| < fal? + [bl ‘We may use this inequality to place constraints on the window, ws(n), so that the reflection coefficients are bounded by one in magnitude. We do this as follows. If a = e7_,(n) and b= ef y(n—1), then efatD[efaale— DT | < leh lol? + feta — 1 uj(n) > 0, then we may multiply both sides ofthis inequality by wy(n), 2wsn)fefs(m)fef_a(n—0]"| < ws(m){ ef ale)? + lef al — 0/7}168 Problem Solutions Summing both sides from n = j to n-+.V we have efaln vp} 23° wyln)fet stm fefatn—n]"| s ss wy {leata) Since [oewstmef so let atm —2))"] < Do wylnyfef rtrlet atm = 1)]"| st 5 (recall that wj(n) > 0), then ASS vse sonlesor— v)"] < Souton flea lef aln FF and it follows that [P| < 1. Therefore, the constraint on the window is that it be non-negative, ay) 20Chapter 6 169 6.11 Consider the sequence of reflection coefficients PP, P/, and I" where P is the reflection coefficient as proposed by Itakura and I" is the reflection coefficient that is formed using the minimum method (see Section 6.5.3) (a) Establish the following relationship between these reffection coefficients: wp < PL < Ie (b) Are there any conditions under which all three reflection coefficients will be the same for all 3? (6) Let Pf be the set of reflection coefficients corresponding to the all-pole model that is derived from the modified covariance method. Ie it possible to upper or lower bound these coefficients in terms of 8, Tor T™? Solution (@) To establish this inequality, we begin ky showing that the sign of [} and [isthe same. With and (n) [eza(n—1)]* Lletey? note that the nomerators are the same, and the denominators are pestive, Therefore, Pan TTF have the same sign. Now note that since Tis the harmonie mean of P} and Pj, then iis aways true that "tS ITPL ‘Therefore, all that we need to establish is that rel 0 and TF > 0 (the proof is similar if both are negative). ‘Then the inequality that we wait to establish becomes O< Wy 40S ary rj However, the right-hand side is equal to (P} ~ 05), which is clearly non-negative, and the desired result follows Since P? is the harmonic mean of C} and Iy, then FP will be equal to P§!" (and also equal to TP™) if and only if T} = Ty. In this ease, it will also be true that P! = TJ. In general however, I} is not equal to I}. A special case in which they are equal is when modeling a random process. With The and — B{ef-sylez ata -1))"} Je eran ime) these rellection coefcients will be equal if the proces is wide-sense stationary. {As we sow in Example 6.5.4, the allpole model that is derived from the modified covariance method is not guaranteed to be stable. Therefore, the reflection coeticients T' may be greater than one in magnitude, and the only possible hound that may be val i ys ryeChapter 6 IL 6.12 In Sect, 6.6 it was shown that the backward prediction errors are orthogonal, i., g 3 taj aeronjonrp= {9 | 133 Establish the following orthogonality conditions: (@) Eef(n)az"(n-W}=0 5 LSkSE (b) EXey(n)a*(n—k)}=0 ; O 1 and, for n = 0,1, the filter coefficients are approximately tae same 1s those found in pact (a)Chapter 7 181182 Problem Solutions 7.6 Suppose that a process x(n) has been recorded but there is a missing gap over the interval Ni, Nal, Le, x(n) is unknown over this interval (a) Derive the optimum estimate of 2(N) using the data in the semi-infinite interval (—o0, Ny q. (b) Derive the optimum estimate of x(.N;) using the data in the semi-infinite interval [Np + 1,00) (c) Derive the optimum estimate of (Ni) that is formed by combining together the two estimates found in parts (a) and (b). (A) Generalize your result in part (e) to find the optimum estimate of x(x) at an arbitrary point m in the interval [N,.No]. Solution (a) This is the one-step causal linear prediction problem, ‘The optimum estimate of 2(Ns) is Bm) = Soa(etMs — where | Me = a = Oj and Q(2) isthe minimum phase factor of P.(2), Pelz) = 9§Q(2)Q(2") (0) Here we want to design the optimunn Biter for predicting backwards Np ~ Ny +1 samples. ‘The system function for the optimum causal (Np ~ Ny + 1)step forward predictor is = 1 fymemttgyy me) = ay |F ate)] ‘Therefore, the optimum estimate of (Nj) is Bali) = Y> hal )x( Na + 1+ &) (0) We are given two estimates of e(NN1), which we have ealled #(.N}) and (IN). Let the vatiance ofthese estimates be denoted by of and o3, respectively. We would now like fo find the optim ‘state of (Ni) using an estimate of the form BUN) = K2(Ni) + (1 ~ K)Ba(Ni) ‘where K is a constant that is to be determined. Note that this form for the estimate guarantees that 2(Ny) will be unbiased if 2(M,) and 2(N,) are unbiased. ‘To find the value of K that ‘minimizes the mean-square error, P{ [ety - 2())"}Chapter 7 183, we set the derivative of € with respect to K equal to zero and solve. With (M1) = (M4) ~ HM) = K[2(M) ~ BM] + = Kal) —200)] wo have He ae{el0) (fen) = #,(M1)] ~ [2(1) -#()}) } =0 Substituting for e((Vi) and taking the expected value we have, assuming that the estimation cervors €3(N1) = 2( Ni) ~ (Ni) and e9(N1) = 2(Ny) ~ 22(%i) are uncorrelated, Ko} ~(1- Kyo} =0 Solving for K’ we have "ara which leads to the following estimate for (Ni), 2M) = a BAUM) + BE RM) (a) For an arbitrary point No in the interval [Ni, Na], the result derived in part (c) produces the ‘optimum estimate of x(q) provided 1f,(2) is replaced with184 Problem Solutions 7.7 In this problem we consider the design of a causal IIR. Wiener filter for p-step prediction, F(t p) = SoMR)a(n — 8) (a) If e(n) is a real-valued random process with power spectral density Pa(z) = o8Q(2)Q(27" find the system function of the causal Wiener filter that minimizes the mean-square error § = Bllx(n +p) —2(n + p)P?} (b) If-2(n) is an AR(1) process with power spectrum find the causal p-step linear predictor and evaluate the mean-square error. (c) If x(n) is an MA(2) process that is generated by the difference equation 2(n) = Av(n) ~ 2v(n = 1) + of = 2) where o(n) is zero mean unit variance white noise, find the system function of the owo- step (p = 2) predictor and evaluate the mean-square error. (4) Repeat part (¢) for a three-step predictor Solution (3) ‘The optimum causal Wiener filter is eee rag) 0 = a5 [9 With d{n) = 2(n-+ 7), ral) = Folk +p) and Pao(2) = 2"Pal2) = 268Q12)Q(2") ‘Therefore, 1 me) = aylrae)],, (0) With the power spectrum of 2(n) given ty 1-a? BG) = Gao a)Chapter 7 185 wwe ee that of = Therefore, and the optinmun causal Wiener filter for estimating (n+p) is 1 HG o] wlax(n — &) (©) Compute the mean-square error E{ld(n) ~ d(n)?} and compare it to the mean-square error that results when h(n) = 6(n), i.e., with no filtering of x(n). Solution (a) The frequency response of the optimum noneatsal Wiener filter is Pucle*) __ Palo) MO) = Pei) = Pele)-+ PP) Using the given power spectral densities for d(n) and 20(n) it follows that the frequency response of the Wiener filter is as shown in the figure below. (6) The minimum meansquare error is Simin = ral0)- D> h(0)rac(t) = Ef tle) me Pde) da188 Problem Solutions H(o*) hi | ; TANG =a =a/h a4 mew Evaluating the integral we find _5A_No Sin = SG ‘The mean-square error using, h(n) = 6(n) is cB) = friesChapter 7 189 7.9 We would like to estimate a process d{n) from noisy observations, n(n) = din) + v(n) where u(n) is white noise with variance #3 = 1 and d(n) is a wide-sense stationary random process with the first four values of the autocorrelation sequence given by ry = (15, 0, 1.0, oj” Assume that d(n) and v(n) are uncorrelated. Our goal is to design an FIR filter to reduce the noise in d(n). Hardware constraints, however, limit the filter to only three nonzero coefficients in W(2). (a) Derive the optimal three-multiplier causal filter W(z) = w(0) + w(a)27! + w(2)2-? for estimating d(n) and evaluate the mean-square error E{|d(n) — d(n)|?}. (b) Repeat (a) for the noncausal FIR filter W(2) = (1+ w(0) + w(t)? (o) Can you suggest a way to reduce the mean-square error below that obtained for the filters designed in parts (a) and (b) without using any more than three filter coefficients? Solution (0) The Wiener-Hopf equations for the optimal three-multiplier filter W(2) = w(0) + w(d)en* + w(2)2* for estimating a(n) are re(0) re(1) re(2) ] fF e(0) ras(0) ral) re(0) re(1) | | w() | = | rae) ra) ra) 72(0) | | w(2) ranl2) with Pa(B) = valk) + ro(k) and Pae(k) = rab) using the given values for the autocorrelation r4(k), these equations become [® 0 [8] (*] 0 25 0 |} way |=] 0 10 0 25} | w(2) 10190 Problem Solutions ‘The solution for the coefficients w(k) is w) ]_ f 05238 wi) f=] 0 w() J [0.1905 B{La(n) — dn)|*} = r4(0) ~ Yo w(k)ran() = 1.5 ~ 0.5288. (1.5) ~ 0.1905 = 0.5288 ‘The mean-square error is (b) For the noncausal FIR filter W(2) = w(—1)z + w(0) + w()e™ the Wiener-Hopf equations for the optimum coefficients may be derived as follows. With € the mean-square error 6 =B{e(n)} we st the derivative of € with respect to w(A) equal to ero as follows of piign {-mnvsin-w} 0 5 k= 0.81 Substituting for e(n) and simplifying we have YS wire) = rank) 5 k=O, Since rae(k) = ra(k); then these equations are (0) rel) re(2) ] f w(-0) ra(-1) rel) re(0) re(1) | | w(0) | =| rul0) a2) re) 74(0) | | wa) ral) Using the given antocorrelations, these become 25 0 10) [ w(-1) 0 0 25 0 || wo) |=] 15 ro 0 25] | wa) 0 w(~1) o wo) | =| 06 wil) 0 B{ld(n) ~ din)/?} = ral) — $7 w(eyras(h) = 0.6 ‘Thorefore, we have with a mean-square error © Since rg(1) = 0 and the additive noise, v(m). is white, then a(n +1) Is of no use in estimating {n). Therefore, a better estimator to use that has only three coefficients is the following W(2) = w(0) + 0(2)2°7 + w(—2)e?Chapter 7 191 7.10 Suppose that a signal x(n) is recorded and that, due to measurement errors, there are outliers in the data, ie, for some values of n there is a large error in the measured value of (n) Instead of eliminating these data values, suppose that we perform a minimum mean-square interpolation as follows. Given a “bad” data value at time n = no, consider an estimate for -»(n9) of the form, (no) = ax(ng ~ 1) + bar(no + 1) (a) Assuming that x(n) is a wide-sense stationary random process with autocorrelation sequence r2(k), find the values for a and b that minimize the mean-square error € = E{\x(n0) ~ 2(r0)!?) (b) Ire) = (0.5)! () evaluate the mean square error for the interpolator found in part (a) nuss when it may be better to use an estimator of the form (nq) = ax(ng ~ 1) + b(n ~ 2) or explain why such an estimator should not be used. (a) Given an autocorrelation sequence r(k}, derive the Wiener-Hopf equations that define the optinuun filter for interpolating (7) to produce the best estimate of x(no) in terms of the 2p data values, x(q ~ 1), 2(n9 = 2),.+.2(no — p) and x(ng + 1),2(n9 + 2),---.2(000 +p) (e) Find an expression for the mean-square error for your estimate in part (d) Solution _ («) With an interpolator of the form F(a) = ax(nq — 1) + be(no + 1) the values for a and b that minimize the mean-square error § = B{|2{(n9) ~ 2(n0)?} are found by setting the derivative of € with respect to a and b equal to zero. Assuming that (1) is @ real-valued process, we have B{-2[x{(na) ~ 2(ra)]2(n0 - 1} =0 and —2[x(m0) — 2(nro)] (mo + a} Dividing by 2 and substituting for #(n) gives B{ [2(00) ~ az(g ~ 1) ~ be(no + 1}} (m9 ~1)} = 0@) co (a Problem Solutions B{ [e(n9) ~ ae(ng ~ 1) ~ be(no + 1)) x(n + n}=o Writing these equations in matrix form in terms of the autocorrelation sequence ry(k), we have re(0) re(2) ] fa] [ ralt) [ 8) no) ] [ 6 ] . [ ral) ] Solving for @ and 6 we find to oe ie wal am 0 [Ea] a: re(1) [re(0) — re(2)) ~ FOB | nenfra0-r0)] > aoeali] ‘The mean-square error for the interpolator found in part (a) is Emin = Ef e(n0}2(na)} = B{ [2(no) — ax(no ~1) ~ ba(no +1)}2(0)} = re(0) ~ are(1) ~ brat) With re(k) = (0.5), 5 and Soin = 0.5 Whether we use the interpolator (no) = ax(r — 1) + b2{n0 ~ 2) (ng) = axr(np ~ 1) + be{m9 +1) depends on the autocorrelation sequence for x(n). If |rs(2)] > |re(1)], then the first estimate would be better. Otherwise, the second one will produce a swmaller mean-square error. With an estimate of x(n) of the form ¥ aldietn —0 a (no) the fiter coefficients that minimize the mean-square error are found by setting the derivatives af P{e?(no)} with respect to a(k) equal to zera as follows, Fags Pd} = -2E(e(no)xOn9~ 1) poke 4p Dividing by two, and substituting for the exror e(n), this becomes B{ [z60) ~ s a(0}x(019 ~ )]2{na — 6y} = 0 ‘aChapter 7 193, re(k) — > a(re(k =) where r,(k) is the autocorrelation of e(n). (©) ‘The minimum mean-square error is Y alata =H] 2(00)} Sain = Be(na}(9)} = Bf [x(00) Erin = 720) ~ FD aldra(t)194 Problem Solutions 7.11 In this problem we consider the design of an optimum smoothing filter for estimating a process a(n) from the measurements 2(n) = d(n) + vn) Our goal is to use a noneausal FIR filter that has a system function of the form: Wee) = So wea In other words, we want to produce an estimate of d(n) as follows : » din) = S> w(k)x(n— k) « (a) Derive the Wiener-Hopf equations that define the set of coefficients that minimize the § = Blan) ~ din) (b) How would the Wiener-Hopf equations derived in part (a) change if we used a causal filter with the same number of coefficients? In other words, if the system function was of the form W(z) = Duly & how would you modify your equations in (a)? (6) State qualitatively when you might prefer the noncausal filter over the eausal filter and vice versa, For example, for what types of signals and for what types of noise would you expect a. causal filter to be superior to the noncausal filter? (a) FIR digital fitters with linear phase (or zero phase) are important in many signal pro- cessing applications where frequency dispersion due to nonlinear phase is harmful. An PIR filter wilh zeww phase is characterized by the property that ea w(n) ‘Thus, the system function may be written as » We) = w(0) +o wl: a Derive the Wiener-Hopf equations that define the optimum zero phase smoothing filter. (©) With ra(k) = 4(0.5)!" and ry (k) = 6(4), find the optimum values for the filter coefficients, ‘w(0) and w(1) in the zero phase filter W(2) = w(0) + w(1)fz +27Chapter 7 195 Solution (2) Using a noncausal filter of the form the estimate of d(n) is given by Gin) = SS wika(n—) with : = Efle(n)l*} = E{ld(n) — din) ?} the coefficients that minimize the mean-square error are solutions to the following equations ee ae*(n) Since el) = d(n) — > w(a(n - mp en : 200) arin) bury and Ele(nz"(n—K)} =0 5 » ior Using the definition of e(n) this becomes Baine(a~ #)) — 5 wi EXe(n=z"(n =H} =0 and, with r,(k) = B{a(n)x*(n ~ k)} and ras(k) Wrath O = raalk) 5 RMP ae H{a(n)2"(n— B)} we have which is a set of (2p +1) linear equations in the (2p-+ 1) unknowns w(&), ‘equations may be written in matrix form as follows (0) rel) r2(2p) w(-p) rae(~B) ra() rel) reQp—1) | | wl) rae(1 ~p) rep) ra(20-1) (0) w(p) raelP)196 ) C} (a) Problem Solutions We may now evaluate the mean-squate extor, ofan fate) — 5 edt —0] } & = Plleln)?, = (elm yey} ~ So we fe(ma"(n—D} However, from the orthogonality condition, B{e(n}z"(n — k)} = 0, it follows that the second term is zero, and we have Sin = E{e(n)a°(n)} = e{[u -> woes} Finally, taking expected values we find ral) ~ D> writ) If we change the filter in (a) to a causal smoothing filter of the form Sout We) then the only thing that changes in the derivations abeve is the limits on the summations. In particular, the Wiener-Hopf equations would take the form » Ler = rie) ¢ =O)... 2 and the minimum error would be » Sin = ra(0) — } wri (0) fe If the autocorrelation sequence for d{n) is generally Inrger ovr the interval [~p,...,p] than i is over the interval 0,...,2p|, then the zero phase smoothing filter would be better to use, Other- vise, the causa iter would be better. For example, with a signal that has an autocorrelation sequence r¢ = [1,0,0,0,.9,..., you may wish to compare the performance ofthe optimum causal and noneatsal Hiters when p = 2 Using a zero phase smoothing filter W(2) = w0) + wh) [+] the estimate of d(n) is nov given by din) = w(0)x(n) + w(t) [x(n - 0) + 2(n +9]Chapter 7 197 @) Again, with = = Ble(n)P} = Bf ldo) = din) l?} we differentiate € with respect to w*(k) as follows However, since and te) be my oF WOK) | kD thon wo have B{e(n)z"(n)} =0 and Pfeonlfaln—W) tala + WP} =0 5 k= Leap If we now substitute for e(n) we find 742(0) ~ {v(0)re(0) + Dw(0)[ra() + raf] } =0 and Braet) ~ {20(O)re(k) + Y > 2w()[ralk—Drek+ 0] =O fs kel e Thus, w(O)rs(0) + 2w(K)re(K) = ras(0) and sw(0)ra(8) + Sw) [rk =1) + ra(k+D] =racl8) i R= Tyee ‘The coefficients for the thivd-order zero phase filter are the sohition to the equations, re(0) 2re(1) w(0) ] _ | rae(0) [ ra(1} r2(0)+r2(2) ] [ oa rac(1) ] Incorporating the given values for the autocorrelations, we have (2 ][28]-[2] ‘and the solution is,198 Problem Solutions 7.12 We observe a signal, x(n), in a noisy and reverberant environment, y(n) = a(n) + 0.8e(n 1) + v(x) where u(n) is white noise with variance 63, = 1 that is uncorrelated with x(n). We know that 2(n) is a wide-sense stationary AR(1) random process with autocorrelations re= (4, 2,1, 05)" {a) Pind the non-causal IIR Wiener filter, H(z), that produces the minimum mean-square estimate of 2(n). (b) Design a causal IIR Wiener filter, H(z), that produces the minimum mean-square esti- mate of 2(1) Solution Before we begin this problem, we need to determine the power spectra Since x(n) is an AR(1) process, we can determine r(k) and P,(2) from r4(0) and r,(1), which are given. Specifically, swe have = #0) _ PO = Teale ay where [re ref 1 Jo. ft [esa 0} [ot ] ~+[o] and PO=a Solving for a(1) and 6(0) we find | a(t) = HA aa and 3 ‘Thu, wo. (a) For the noneausal filter, we know that Since an) = x(n) + 0.8z(n ~ 1) + w(n) then Pylz) = (1+ 0.82-4)(1 + 0.82)P.(2) + Pula) In addition, replk) = Bfo(n +k) [2(n) + 082(n ~1) + v(n)]} = ral) + 08re(k-+ 1)Chapter 7 199 and. Poy(2) = (140.82) Pal2) ‘Therefore, the noncausal Wiener filter is (1 4082)P (2) cael 1 08-HF 08 )P =) +1 30 +082) a) 3(1 + 082-1)(1 + 0.82) + (1 = 0.52-)(1 = 0.52) which may be simplified to fn Ho (14082) (b) For the cansal Wiener filter, 10= sam la], where 0 and Q(z) are found from the spectral factorization of y(n), Pa) = 3212") With no) = BAEO8MH(L-+ 082) A) = “Goss ayr—o5e +1 an G05: 055) it fotows that Q2)= tod 5s oa * TOs 1 rasa, Since 3.5829 Oe then200 Problem Solutions 7.13 A wide-sense stationary random process has an autocorrelation sequence of the form. reli) alt! where [al < 1. Over a given time interval, [n.4,np], the process #(n) is only known at the end points, ie, the only given data is 2(n4) and 2(ng). Based on these two observations, determine the optimum estimate 3(n) = afn)x(na) + 6(n)x(np) of x(n) over each of the following intervals (a) n> np. (0) nema. (©) na ny we have (recall that np > ma) [363 ]=[ antes] Oe [33 ]-["""] (¢) Fornacn S> SAG mals mCi) and the derivative with respect to k(i,5) is ey AK) YL adarmatiom + DeaGimates)206 ) () Problem Solutions or, since A is symmetric, a oO e(KAK? aD ) Yo Mism)o(n,) = 2A For complex data, we treat K and K* as independent vasiables. Diferetioting with respect to KC we have i ewiak) =a (Ka) =0 aK Hae and : FEI KAKM) = KA Given these matrix differential formulas, the derivation of the expression for the Kalman gain is straightforward. First, with the error covariance mattix given by Pln|n) = [L~ K(njO(n)] Plan — YLT K()C(n)]"” + KWIQK™(n) wwe expand it as follows P(n|n) = Pln|n~1) - K(n)C(n)P (nln — 1) + P(nln — 1)C* (mK (x) + K(n)C(n)P(njn = 1)C™ (mp (n) + K (mJ QUE” (n) Since the trace of a sum of matrices is the sum of the traces, using the matrix differentiation formulas above we have AztLP Evin} = [1 KEE) Pein — HOM) + KHALID) =0 Finally, solving for K(n) gives the desired expression for the Kalman gain, Kin) = Plain NE" 9 [C(PCaIn— NEM) +N]Chapter 7 207 7.17 Consider a system consisting of two sensors, each making a single measurement of an unknown, constant 968)] (b) Asn 00 <1 30Chapter 7 7.21 In this problem we will derive the following expression for the Kalman guin, K(n) = Pin|nJO¥ (nJQy iin) (P7.21-1) (a) By substituting Eq, (7.113) for the Kalman gain into Eq. (7.114), show that the error covariance matrix P(njn) may be written as Plain) = Plajn—1)—P(n|n—1) CM (n)[Cln)P (n\n 1)C"(n)+Qo(n)] 1C(n)P(n-—1) (b) Using the matrix inversion lemma given in Bq. (2.28) on p. 29, show that the inverse covariance matrix may be written as Prin) = PMnjn — 1) + C4 (njQz>(n)Cin) (c) By multiplying the expression for the Kalman gain given in Eq, (7.113) on the left by P(n|n)P~"(n]n), use your results in part (b) to derive the expression for K(n) given in Eq. (P7.21-1) Solution ("This part follows by inspection, With Plain) = [I= KinjC(m)]P(nin — 1) and a Kalman gain given by (a) = Plain = HOM(w)[eP{nn— HOM) + Qo] wwe have Plain) = [F-Pain— DE" On[CMMPEaIn~ NEM) +Q.Km]“Cln)}PEaln = 2) = Plain) — Plnin = NEM) [CKm)P Kn — HEM En) + ln)] MECH PEn— 1) (0) The matrix that we want to invert is Pian) = P(njn = 1) = Pn — 10" (n)[Clne)P(njn — 1}C*(n) + Qu(n)]'C(n)P(m|n.— 1) ‘The matrix inversion lemma states that (A+ BOD)! =A} ABC! +DA“By DA ‘Therefore, we define = Pinin-1) -P(njn— 1)" (n) [emPinin — 10" in) + Qoem)] Cln)P(nn 1) caw216 Problem Solutions Note that, (C'+DA“B)* [comPinin = YC" a) + Qeln) = Cl)P(nln = Pala — P(Rjn~ O(n) Qin) erefore, P-"(njn) is Pan) = Prin 1) +PMaln — YP (nin ~ YE" (njQS(n)Cln)P nin — 1)P-(n]n— 1) = PMnja~1) + OM )QsnpCn) which is what we wanted to show. (©) The Kalman gain is K(n) = Plan — NOM (n)[C(~PEalr— NOM) +Q.0)] Multiplying on the left by P(njn)P-1(n\n) gives K(n) = PlalnyP-HenimpPnjn —}C" [OCP (HJn~ YO") + QC] Substituting the expression for P~!(n|n) desived in part (b) yields KO) = Poole) [Pala 1) + CM MQEMICL] Pima — HE" [EfmPInin — HOM) + Qotn)] = Poon [em + 6% mQz*mC(aIPKnin — HO™ op] [EMP (rin —}C*(n) + Quln)] = Plaine (nQ3"(r)[Qe(n) + Clo)PLala ~1}6%)] [ElapPlnn — HEN) + Qulvd] Plaka" (Qe"(n) which is the desired relationship.Chapter 7 a7 7.22 Consider the ARMA(1,1) process y(n) given by un) + ay(n ~ 1) = w(n) + dw(n ~ 1) where w(n) is a zero-mean white-noise process with a variance 02 (a) Show that the state-space representation for this process may be written as 1 x(n) = [ : xen [ 1 | w(n) u(r) 1 0] x(n) ‘where x(n) is a two-dimensional state vector (b) Assuming that the error covariance P(n|n) converges to a steady state value of P and is a solution of the Ricatti equation given in Prob, 7.19, show that rea["3 8] where c is a scalar that satisfies the second-order equation c= (b- a) +ate~ Ite and find the two values of ¢ that satisfy this equation. For each of these values, find the corresponding values for P. (c) Find the steady-state Kalman gain and determine the values for K that correspond to ‘the solutions for ¢ found in part (1b). {a) Let, x)= [29] [20 J=[o o][ 88 ]+[5 ] + eee ta(n) = bw(n) = a(n azy(n ~ 1) + bw(n — 1) + w(n) (a) = [1 O}x(n) is the equation we have for the ARMA(1,) process.Problem Solutions (b) For the state variable representation of the ARMA process, we have as[o 0] ae = eff 2) Jf vem sony F}-2[} A] a-=0 ws cao) ‘The algebraic Ricatti equation is P=alr-re"(cre" +a.) ‘cpa +a. P- APA" +aPc!(cro" «@.) 'epa" a0 From the Kalman fering equations, we know that P=APA" +Q0 ‘Therefore, let us denote P by =[ 9) 2) P=[%) no | Incorporating this into the equation above, we have 1) 2) ] _f -@ 17 [ wa) play 1b] [a3 wa l-[o aE Bis c]eeLs 8] find [f 2) ao 2ap(2) + pl4) + P03) 78) ° ° “Therefore, P has the form ‘Multiplying this out Now, note that cect +a, =o2fio [Ef [3] re-ssa+9 ‘Therefore, ' (9PC" +0)" = asa “Sara aie # Cs fea SLs Sha ao oe 0 ae7 a o)*ixeChapter 7 219 However, in order for this to be equal to 2ero, as dictated by the agebraie Rcatti equation, the constant c must satisfy the equation : 2» (b-a~ ae)? e= des (app Oo 8 Multiplying this out and simplifying this may be written as el-P += ‘Therefore, there are two possible values for c, |, the marine I is which s non-negative defi forall ves of Fare = 021, we have eb pan De] which is non-negative definite provided B > 1 ©) Using the following expression for the Kalman gain (see Problem 7.21) K=PCM(CPC! +Q,)7 Kh: Pl[o}-[ycro |220 Problem Solutions SOLUTIONS TO CHAPTER 8 Power Spectrum Estimation 8.1 Given N = 10,000 samples of a process -r(n), you are asked to compute the periodogram, However, with only a finite amount of memory resources, you are unable to compute a DET any longer than 1024. Using these 10,000 samples, describe haw you would be able to compute 4 periodogram that has a resolution of, an T0006 Hint: Consider how the decimation-in-time FFT algorithm works. Aw = 0.89. Solution ‘To get the maximum resolution from N’ = 10000 data values, we want to compute the periodogram of x(n) (segmenting x(n) into subsoquences reduces the resolution). ‘The question, therefore, is how to ‘compute the periodogram of x(n) using 1024-point DPT's, Recalling how the FFT works, note that Salmon = So z(10m + eS — 04S 0 + eI X(o) ‘Therefore, the procedure is to pad (n) to form a sequence of length V = 10240, and then decimate -2(n) into 10 sequences 2(n) of length M = 1024, (00-40 5 m= 0,24...41028 Next, the 1024-point DF''s of these sequences, X;(k), are computed, and combined using the “twiddle factor” exp(-, X(k) = Dette) | k= 0,1,..., 10090 Finally, squaring the maguitude of X(k) and dividing by a resolution Aw = 0.89(27/10000). 1UZ4D, we have the periodogram withChapter 8 2 8.2 A continuous-time signal x4(t) is bandlimited to 5 kHz, Le, ra(t) has a spectrum Xq(f) that is zero for [f| > 5 kHz. Only 10 seconds of the signal has been recorded and is available for processing. We would like to estimate the power spectrum of ra() using the available data in a radix-2 FFT algorithm, and itis required that the estimate have a resolution of at least 10 Hz. Suppose that we use Bartlett’s method of periodogram averaging. (a) Ifthe data is sampled at the Nyquist rate, what is the minimum section length that you may use to get the desired resolution’? (b) Using the minimum section length determined in part (a), with 10 seconds of data, how many sections are available for averaging? (c) How does your choice of the sampling rate affect the resolution and variance of your estimate? Are there any henefits ta sampling ahve the Nyquist rate? Solution (a) If we sample at the Nyquist rate, f, = 10kHz, then a resolution of Af = 10He (in analog frequency) implies that we want a resolution (in radians) of Ar Aw = 20 l i Since the resolution of the periodogram using an E-point data record is x 10-9 (b) Sampling at 10 kil, 10 seconds of data corresponds to N= (10}(10 x 10°) = x10° samples. Therefore, with a 1024-point DFT the number of sections we may have in Bartlet's method is Kk = [w/lo24] = 98, (c) If the sampling rate is increased then Avy decreases which, in tum, requires a longer section length for a given resolution, However, an increase in the sampling rate produces a corresponding increase in the total number of samples within a 7’ second interval. Therefore, since the variance (normalized) is V=LN increasing the sampling rate has no effect. ‘Thus, provided that the sampling rate is not less than the Nyquist frequency, the resolution and the variance do not depend on the sampling rate,222 Problem Solutions 8.3 Bartlett's method is used to estimate the power spectrum of a process from a sequence of N = 2000 samples. (a) What is the minimum length £ that may be used for each sequence if we are to have a resolution of Af = 0.005? (b) Explain why it would not be advantageous to increase L beyond the value found in (a). (©) The quality factor of a spectrum estimnate is defined to be the inverse of the variability, Q=y Using Bartlett's method, what is the minimum number of data samples, V, that axe necessary to achieve a resolution of Af = 0.005, and a quality factor that is five times larger than that of the periodogram? Solution (0) Sinee AF 9/1 then 989 _ 09 “BF 0005 ~ (b) Increasing L will increase the resolution, but it wil alto reslt in a decrease in the number af segments that may be averaged. ‘This iu turn will incre the variance of the spectrum ctimete (©) For the periodograr, the quality factor is Qqur = 1/Vpr = 1s The quality factor for Bartlett's method is Qy = 1/Va = K. Therefore, if we want Qyer/Qe > 5, then we must have K > 5. With M = 180 (for Af = 0.005), then we must have 180 N= KM >5 x 180=900Chapter 8 223 8.4 A random process 2(n) is generated by filtering unit variance white noise as shown in the figure below wa) [a 1 | 2) AG L_ ae where A(z) = 1-42? 40.9927; Ao(2) az! +0982? (a) Prepare a carefully labeled sketch of the power spectrum of r(n) assuming that a is small, o.g., 0 2 For the minimum variance estimate, F Porte*)] ‘which, as with the MEM spectrum estimate, is infinite in length. However, unlike either the Blackman-Tukey or the MEM estimates, the autocorrelation matching property does not hold, ie, Fe(k) # re(h) for [Bl <2,236 8.10 Problem Solutions iven that the sixth-order minimum variance spectrum estimate of a process 2(n) is Pav} T¥ acos4a + dacos 6a and the seventh-order estimate is 1 Pave ‘2a cos 2w — a.cos Tw find the seventh-order maximum entropy spectrum, Pyrem(e). Solution ‘This problem may be easily solved using the following relationship between the MEM and MV spectrum ‘estimates, Specifically, note that and ‘Taking the difference yields ‘Thos,Chapter 8 237 8.11 The first-order (p = 1) minimum variance spectrum estimate of a random process is Pave) = 7% 08 (a) Find the autocorrelations, r'-(0) and r-(1), that produced this spectrum estimate. (b) In general, given the pth-order minimum variance estimate Pyv(e), is it possible to recover the values of the autocorrelation sequence that produce this estimate? Solution, (a) The first-order MV spectrum estimate is GO) 2gCoos where r2(0) nO] rel) 7200) and q(k) is the sum along the kth diagonal of Rz?. Since [ re(0) —re(1)— re(0) then A » 00 oa | = =H and 2(r2(0) = ¥3(0)) Pave) = 50) = Brg) eos If we express Pry (e) as follows then, comparing this to the given minimam variance estimate, we have re(0) 7200) = r2(1) 0). ec) Solving these equations for r,(0) and r(1) we find rx(0)=3 ; re(l)=1 (0) In genera, the pth-order minimum variance estimate the autocorrelation sequence ra(8) may be recovered from the minimum variance spectrum estimate, Pyyy(e*). However, the procedure is not emyy. Since Re is Toeplita, if Ry is a p x p mattis, then only p entries in Ry need to be determined ~ efther the fist row or the first column. The inverse of Ry is centrosymmetsie and, given Pyyy(e!), we know the sums along the p diagonals of R;?. Expressing the inverse ‘of R;? in terms ofr, (4), and suraming along the diagonals, we obtain p nonlinear equations in punlenowens whieh, in theory, can be solved for r().238 Problem SolutionsChapter 8 239) 8.12 The second-order maximum entropy spectrum of a process 2(n) is Prum(e) = [1 — 0.5e-3 + 0.25e-2%4 (a) What is the first-order maximum entropy speetrum? (b) Pind the second-order minimum variance spectrum estimate. Solution (a) Given the second-order MEM spectrum of 2(n), it follows that the second-order all-pole model AG) = 1-052 * +0252 > ‘The inverse Levinson-Durbin recursion yields the following autocorrelation sequence ry = (2.5897, 1.0159, -0.1270]" ‘Therefore, the first-order MEM spectrum is fj Poel = Teel gt One OF 0) where o a) crs) rant) =rat)~ 2 2 () Having found the autocorrelations 7.(0), r4(1), and rs(2), we may now find the second-order MV estimate as follows. With 72(0) ra(l) 122) 25897 1.0159 -0.1270 R= | re) ra(0) ra(t) |=] 1.0159 2.5397 1.0159 re) rel) 1210) 0.1270 1.0159 2.5897 for the inverse we have 050-025 0.125 025 05037 0.5 0125-025 0.50 ‘Therefore, a0) = 15937 aa) = -05 a) = 0125 and the MV spectrum becomes, 3 3 Pave GO) + Bgl) cosas + 2912) condo ~ THT eomw + Oboe de240 Problem Solutions 8.13 Prom measurements of a process (1), we estimate the following values for the autocorrelation sequence: : r(k)=all 5 [ki Az >... > Ay > 0 be the ef ‘theorem states that if g(-) is a continuous real-valued function then ly On Le 12 tim = D> 9(Ae) olPe(e*) dx ca Nite W 2 Use Szegé's theorem to show that H(x) = 2Fi(x) + n(2re) ice., that the two entropy expressions are proportional to each other.