1-

a) Do the following for ten realizations:

i) Generate a signal y[n], N = 512 samples, using the Narrowband AR processin page 148 in the textbook. [ ] [ ]

ten different realizations of y[n] 15

10

-5

-10

-15

100

200

300

400

500

600

ii)

Estimate the parameters using the autocorrelation method.

From Equation parameters of the real system: Numerator = [ 1 0 0 0 0] Denumerator = [1 -1.6408 2.2044 -1.4808 0.814 ]

Then true spectrum of the system is :

True PowerSpectral Density 200 180 160 140 120 100 80 60 40 20 0 -4

-3

-2

-1

Using autocorrelation method I found the estimated parameters like this : Estimate numerator = [1.0692 0 0 0 0] Estimate Denumerator = [1 -1.66284812274577 2.15974478610830 -1.45767759588340 0.768503414833484]

b) Plot all pole-zero diagrams on the same Figure.

zeros-poles diagrams 1

0.5

Imaginary Part

-0.5

-1

-1

-0.5

0 Real Part

0.5

1.5

c) Plot all spectra on the same figure. After calculating the estimated parameters , I obtain the power spectral density for ten realizations.
Estimated PowerSpectral Density 250

200

150

100

50

0 -4

-3

-2

-1

And average estimate spectrum of y[n] :

Average Estimate PowerSpectralDensity of ten different realizations 160 140 120 100 80 60 40 20 0 -4

-3

-2

-1

2- COVARIANCE METHOD Estimated Parameters using covariance method :

Estimate numerator = [1.0664 0 0 0 0]

Estimate Denumerator = [1 -1.648098544485497 2.192103123925523 -1.473035990919552 0.795286555553020]

zeros-poles diagrams 1 0.8 0.6 0.4

Imaginary Part

0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -1.5 -1 -0.5 0 Real Part 0.5 1 4

Estimate Power Spectral Density plots for ten different signal

Estimated PowerSpectral Density 400 350 300 250 200 150 100 50 0 -4

-3

-2

-1

Plot of average estimate power spectral density :

Average Estimate PowerSpectralDensity of ten different realizations 200 180 160 140 120 100 80 60 40 20 0 -4

-3

-2

-1

3- Add white,Gaussian noise to one of the realizations such that SNR is 5 dB. Estimate the parameters using the autocorrelation method for different model order values and plot the spectrum. When we add noise pole-zero plot : For model order P = 4;

zeros-poles diagrams 1 0.8 0.6 0.4

Imaginary Part

0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -1 -0.5 0 Real Part 0.5 1 4

Average Estimate PowerSpectralDensity of ten different realizations 350

300

250

200

150

100

50

0 -4

-3

-2

-1

Estimate Denumerator = [1 -1.166358269907974 1.331382505298467 Estimate Numerator = [ 3.4825 0 0 0 0] For Model order P = 6 ; Pole-zero diagrams :
zeros-poles diagrams 1 0.8 0.6 0.4

-0.642160877669500 0.388587690561766]

Imaginary Part

0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -1 -0.5 0 Real Part 0.5 1 2 6

Average Estimate PowerSpectralDensity of ten different realizations 1400

1200

1000

800

600

400

200

0 -4

-3

-2

-1

Estimate Denumerator = [1 -0.955753388093975 1.076882471542758 -0.032532620043360 -0.125738012157119 0.457459488845452 0.073733723380570] Estimate Numerator = [ 2.3689 0 0 0 0 0 0] Real Denumerator = [1 -1.6408 2.2044 -1.4808 0.814 0 0] Real Numerator = [ 1 0 0 0 0 0 0]

4- Comment on the result Autocorrelation Method First of all I calculated the parameters using the autocorrelation method. Autocorrelation method is more simple method, but it produces lower resolution than the others. To understand it, I plot the Estimate spectrum and the true spectrum on the same figure. As you see , blue one ,or Estimate spectrum is not equal the true spectrum. It is because of low resolution of the autocorrelation method. In this code I used biased autocorrelation method because, unbiased autocorrelation matrix is not guaranteed to be positive definite and the variance of the spectrum estimate tends to become larger when the matrix is ill-conditioned. Therefore, biased autocorrelation estimates are generally preferred. When I use real-valued noise , observed this values: Estimate Denumerator = [1 -1.66284812274577 2.15974478610830 -1.45767759588340 0.768503414833484]

Real Denumerator = [1 -1.6408 2.2044 -1.4808 0.814 ] Estimate Numerator = [ 1.0692 Real Numerator = [ 1 0 0 0 0 0 0 0] 0]

As you see, using autocorrelation method , obtain this values which are close to real values. When I use complex noise, observed this values : Estimate Denumerator = [ 1 + 0.00000000000000i -1.60575309677584 + 0.0198466699814833i 2.12121494934365 - 0.0257820652567119i 0.768774605203358 - 0.0123184341509201i] -1.40456996565833 + 0.0260092976526211i

Estimate Numerator = [ 0.9317

0]

As you see, values are complex and close to real values but not equal. Resolution is low ! For better observation , I plot the true spectrum and the estimate spectrum on the same figure. From the figure, you can understand what I mean.
Difference Between True Spectrum & Estimate Spectrum 200 180 160 140 120 100 80 60 40 20 0 -4 Estimate Spectrum True Spectrum

-3

-2

-1

Covariance Method This method is more complex than the autocorrelation method, but it has higher resolution which is good. Covariance method has complex calculations so it takes so much time. It is disadvantage of the covariance method. The advantage of the covariance method over the autocorrelation method is that no windowing of the data is required in the formation of the covariance estimates Cxx(j,k). This method is complex because covariance matrix is hermitian symmetric, positive semi definite but not toeplitz so we cant use Levinson algorithm. So we use cholesky decomposition which has a lot of calculation. In these method , estimated poles are not guaranteed to be inside the unit circle,however usually they are inside the unit circle. When I use real-valued noise , observed this values: Estimate Denumerator = [1 -1.648098544485497 2.192103123925523 -1.473035990919552 0.795286555553020]

10

Real Denumerator = [1 -1.6408 2.2044 -1.4808 0.814 ] Estimate numerator = [1.0664 0 0 0 0] Real Numerator = [ 1 0 0 0 0]

As you see, using covariance method , obtain this values which are close to real values. When I use complex noise, observed this values : Estimate Denumerator = [1+ 0.000000000000000i 2.231556726273705 + 0.242639761266443i 0.812295745810523 + 0.055032847788367i] Estimate Numerator = [ 1.3944 0 0 0 0] -1.675746488771692 - 0.224387043236265i -1.497106766815033 - 0.353961312167855i

As you see, values are complex and close to real values . I say that this method is not guarantated to be inside the unit circle. So you can see from the figure what I mean :
zeros-poles diagrams 1 0.8 0.6 0.4

Imaginary Part

0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -1 -0.5 0 Real Part 0.5 1 4

Because of noise is obtained by randn(x) , each iteration it changed. Some of the iterations I observed poles are inside the unit circle and some of the iterations they are not. To see the difference between estimated spectrum and true spectum :

11

Difference between true spectrum and estimate spectrum 200 180 160 140 120 100 80 60 40 20 0 -4

-3

-2

-1

As you see from the figure, Estimate Spectrum is almost equal to the true spectrum. This means high resolution! Autocorrelation method by adding noise (5dB) Model order = P = 4 ;
Difference between true spectrum and estimate spectrum 350 Estimate Spectrum true spectrum 300

250

200

150

100

50

0 -4

-3

-2

-1

12

Model order = P = 6;
Difference between true spectrum and estimate spectrum 1400

1200

1000

800

600

400

200

0 -4

-3

-2

-1

When I increase the model order P , our estimate system is distorted. P = 4 is the best choice to estimate the system. P = 4 is equal to number of parameter ( except a(0) = 1). -------------------------------------------------------------ooo-----------------------------------------------------------------CODES : Power Spectral Density Calculatin :
function PowerSpectralDensityOfOutput = PSD(num,den,i) % Assume we use white noise so PSD of white noise = 1 % assume Transfer function order is 4 % num : numerator % den : denumerator % i : figure index, which figure you want to plot PowerSpectralDensityOfError = 1; % Error ~ N(0,1) w = -pi:0.001:pi; z = exp(1i*w); H = (num(1) + num(2)*z.^-1 + num(3)*z.^-2 + num(4)*z.^-3 + num(5)*z.^-4) ./(den(1) + den(2)*z.^-1 + den(3)*z.^-2 + den(4)*z.^-3 + den(5)*z.^-4); PowerSpectralDensityOfOutput = ((abs(H)).^2)*PowerSpectralDensityOfError; figure(i),hold all,plot(w,PowerSpectralDensityOfOutput,'m','lineWidth',1.5); grid on; end Autocorrelation Method : clear all,close all;

13

% GENERATING THE TRANSFER FUNCTION IN TIME DOMAIN % H(z) = 1/A(z); N = 512; P = 4 ; m = 10; w = -pi:0.001:pi; num = [1 0 0 0 0]; den = [1 -1.6408 2.2044 -1.4808 0.8145]; % Generation of Noise % Real Noise WhiteNoise(1,:) = randn(1,N*m); % Complex Noise % WhiteNoise(1,:) = (1/sqrt(2))*randn(1,N*m) + 1i*(1/sqrt(2))*randn(1,N*m); % FOR TEN DIFFERENT REALIZATION PowerSpectralDensitySum = 0; for j = 1:m % Output Signal y(j,:) = filter(num,den,WhiteNoise((j*N-N+1):N*j)); % FIGURE OF TEN DIFFERENT REALIZATIONS y[n] IN THE SAME PLOT figure(2),hold all,grid on, stem(y(j,:)),title('ten different realizations of y[n]'); % AUTOCORRELATION FUNCTION AutoCorrelation = xcorr(y(j,:),y(j,:),'biased'); % Obtaining the AutoCorrelation Matrix Ryy = zeros(P,P); for k = 1:(P) Ryy(:,(k)) = (AutoCorrelation((N+1-k):(N+P-k)))'; end ryy = (AutoCorrelation((N+1):(N+P)))'; % Estimated Parameters EstimateParams = (inv(Ryy))*(-ryy); EstimateParams = [ones(1);EstimateParams]; TruePowerSpectralDensity = PSD(num,den,3); title('True PowerSpectral Density'); sum2 = 0; for n = 1:N sum1 = 0; for k1 = 2:P+1 if (n+1-k1) > 0 sums = (EstimateParams(k1))*y(j,(n+1-k1)); else sums = 0; end sum1 = sums + sum1; end sum2 = (abs(y(j,(n)) + sum1 ))^2 + sum2; end EstimatedVariance = (1/N)*sum2; EstimateDenum = [EstimatedVariance; 0 ;0; 0; 0]; PowerSpectralDensityOfOutput = PSD(EstimateDenum,EstimateParams,7); title('Estimated PowerSpectral Density'); PowerSpectralDensitySum = PowerSpectralDensityOfOutput + PowerSpectralDensitySum; end % Plotting the average spectrum AverageEstimateQyy = PowerSpectralDensitySum/m; figure(8),hold all,grid on, plot(linspace(-pi,pi,length(AverageEstimateQyy)),AverageEstimateQyy),title('Average Estimate PowerSpectralDensity of ten different realizations'); % Pole-Zero Diagram [zeros,poles,k1] = tf2zp(num,den);

14

[EstimateZeros,EstimatePoles,k2] = tf2zp(EstimateDenum',EstimateParams'); figure(9),hold on,zplane(zeros,poles),title('zeros-poles diagrams'); figure(9),hold on,zplane(EstimateZeros,EstimatePoles);

Covariance Method : clear all,close all; % GENERATING THE TRANSFER FUNCTION IN TIME DOMAIN % H(z) = 1/A(z); N = 512; P = 4 ; m = 10; w = -pi:0.001:pi; num = [1 0 0 0 0]; den = [1 -1.6408 2.2044 -1.4808 0.8145]; % Generation of Noise % Real Noise % WhiteNoise(1,:) = randn(1,N*m); % Complex Noise WhiteNoise(1,:) = (1/sqrt(2))*randn(1,N*m) + 1i*(1/sqrt(2))*randn(1,N*m); % FOR TEN DIFFERENT REALIZATION PowerSpectralDensitySum = 0; EstimateParams = zeros(1,P); % EstimateParams = [ones(1),EstimateParams]; for i = 1:m % Output Signal y(i,:) = filter(num,den,WhiteNoise((i*N-N+1):N*i)); % FIGURE OF TEN DIFFERENT REALIZATIONS y[n] IN THE SAME PLOT

for j = 0:P for k = 0:P Cyy = 0; for n = P:(N-1) if (n-k)>=1 || (n-j)>=1 Cxx = y(i,n-k+1) * conj(y(i,n-j+1)); else Cxx = 0; end Cyy = Cxx + Cyy; end CyySeq( i, j+1 , k+1 ) = Cyy / ( N - P ); end end C( :, :, i ) = CyySeq( i, (2:end), (2:end)); c( :, i ) = -CyySeq( i, (2:end), 1 )';

DiagonalMatrix( :, : ) = diag( diag( C( :, :, i ), 0)' ); V = chol(C(:,:,i)); U = inv(sqrt(DiagonalMatrix))*V; LowerTriangularMatrix = (U)'; y_Matrix( :, i ) = inv( LowerTriangularMatrix( :, : ) ) * c( :, i ); EstimateParams = inv( U ) * inv( DiagonalMatrix( :, : ) ) * y_Matrix( :, i );

EstimateParams = [ones(1); EstimateParams]; % Finding of the Estimated Variance

15

sum2 = 0; for n = 1:N sum1 = 0; for k1 = 2:P+1 if (n+1-k1) > 0 sums = (EstimateParams(k1))*y(i,(n+1-k1)); else sums = 0; end sum1 = sums + sum1; end sum2 = (abs(y(i,(n)) + sum1 ))^2 + sum2; end EstimatedVariance = (1/(N-P))*sum2; EstimateDenum = [EstimatedVariance; 0 ;0; 0; 0]; PowerSpectralDensityOfOutput = PSD(EstimateDenum,EstimateParams,7); PowerSpectralDensitySum = PowerSpectralDensityOfOutput + PowerSpectralDensitySum; end % Plotting the average spectrum AverageEstimateQyy = PowerSpectralDensitySum/m; figure(8),grid on, plot(linspace(-pi,pi,length(AverageEstimateQyy)),AverageEstimateQyy),title('Average Estimate PowerSpectralDensity of ten different realizations'); TruePowerSpectralDensity = PSD(num,den,8); % Pole-Zero Diagram [zeros,poles,k1] = tf2zp(num,den); [EstimateZeros,EstimatePoles,k2] = tf2zp(EstimateDenum',EstimateParams'); figure(9),hold on,zplane(zeros,poles),title('zeros-poles diagrams'); figure(9),hold on,zplane(EstimateZeros,EstimatePoles);

Autocorrelation method with noise (5dB): clear all,close all; % GENERATING THE TRANSFER FUNCTION IN TIME DOMAIN % H(z) = 1/A(z); N = 512; P = 4 ; m = 10;SNR = 5; w = -pi:0.001:pi; num = [1 zeros(1,P)]; den = [1 -1.6408 2.2044 -1.4808 0.8145 zeros(1,(P-4))]; % Generation of Noise % Real Noise WhiteNoise(1,:) = randn(1,N*m); % Complex Noise % WhiteNoise(1,:) = (1/sqrt(2))*randn(1,N*m) + 1i*(1/sqrt(2))*randn(1,N*m); % FOR TEN DIFFERENT REALIZATION PowerSpectralDensitySum = 0; for j = 1:m % Output Signal y(j,:) = filter(num,den,WhiteNoise((j*N-N+1):N*j)); y_with_noise(j,:) = awgn(y(j,:),SNR); % FIGURE OF TEN DIFFERENT REALIZATIONS y[n] IN THE SAME PLOT figure(2),hold all,grid on, stem(y_with_noise(j,:)),title('ten different realizations of y[n]'); % AUTOCORRELATION FUNCTION AutoCorrelation = xcorr(y_with_noise(j,:),y_with_noise(j,:),'biased'); % Obtaining the AutoCorrelation Matrix Ryy = zeros(P,P); for k = 1:(P)

16

Ryy(:,(k)) = (AutoCorrelation((N+1-k):(N+P-k)))'; end ryy = (AutoCorrelation((N+1):(N+P)))'; % Estimated Parameters EstimateParams = (inv(Ryy))*(-ryy); EstimateParams = [ones(1);EstimateParams]; TruePowerSpectralDensity = PSD(num,den,3); sum2 = 0; for n = 1:N sum1 = 0; for k1 = 2:P+1 if (n+1-k1) > 0 sums = (EstimateParams(k1))*y_with_noise(j,(n+1-k1)); else sums = 0; end sum1 = sums + sum1; end sum2 = (abs(y_with_noise(j,(n)) + sum1 ))^2 + sum2; end EstimatedVariance = (1/N)*sum2; EstimateDenum = [EstimatedVariance; 0 ;0; 0; 0]; PowerSpectralDensityOfOutput = PSD(EstimateDenum,EstimateParams,7); PowerSpectralDensitySum = PowerSpectralDensityOfOutput + PowerSpectralDensitySum; end % Plotting the average spectrum AverageEstimateQyy = PowerSpectralDensitySum/m; figure(8),hold all,grid on, plot(linspace(-pi,pi,length(AverageEstimateQyy)),AverageEstimateQyy),title('Average Estimate PowerSpectralDensity of ten different realizations'); % Pole-Zero Diagram [zeros,poles,k1] = tf2zp(num,den); [EstimateZeros,EstimatePoles,k2] = tf2zp(EstimateDenum',EstimateParams'); figure(9),hold on,zplane(zeros,poles),title('zeros-poles diagrams'); figure(9),hold on,zplane(EstimateZeros,EstimatePoles);

---------------------------------------------------------------ooo---------------------------------------------------------------------

17