List of Figures

1 2 3 4 An example of the NLS cost function versus frequency. RMSEs of the estimates of ! and the associated CRB versus the SNR. (a) N = 32 and M = 15. (b) N = 256 and M = 50.
0

15 16 17 18

RMSEs of the estimates of ! and the associated CRB versus N . (a) SNR=5 dB. (b) SNR=30 dB.
0

MATLAB ops versus N .

Computationally E cient Parameter Estimation for Harmonic Sinusoidal Signals

Hongbin Li

a;1

, Petre Stoica

b;2

and Jian Li

c;1

Department of Electrical and Computer Engineering, Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ 07030, USA b Department of Systems and Control, Uppsala University, P.O.Box 27, SE{751 03, Uppsala, Sweden c Department of Electrical and Computer Engineering, P. O. Box 116130, University of Florida, Gainesville, FL 32611, USA

Abstract
A Markov-like weighted least squares (WLS) estimator is presented herein for harmonic sinusoidal parameter estimation. The estimator involves two distinct steps whereby it rst obtains a set of initial parameter estimates that neglect the harmonic structure by some standard sinusoidal parameter estimation technique, and then the initial parameter estimates are re ned via a WLS t. It is observed that the proposed estimator achieves similar performance to the optimal nonlinear least squares method for a moderate or large number of data samples and/or signal-to-noise ratio (SNR), but at a signi cantly reduced computational complexity. Furthermore, the former is shown to have a lower threshold SNR than the latter.
Key words: Harmonic sinusoidal signal, fundamental frequency estimation, weighted least squares.

? Please address all correspondence to: Dr. Hongbin Li, Department of Electrical

and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ 07030, USA. Phone: (201) 216-5604. Fax: (201) 216-8246. E-Mail: hli@stevens-tech.edu. 1 Partially supported by the National Science Foundation Grant MIP-9457388 and the O ce of Naval Research Grant N00014-96-0817. 2 Partially supported by the Senior Individual Grant Program of the Swedish Foundation for Strategic Research. Preprint submitted to Elsevier Preprint 11 November 1999

1 Introduction
Consider the noise-corrupted measurements of a harmonic signal
y (n) =
K X k=1 ke j (kn!0 +

k ) + v (n);

n = 0; 1; : : : ; N

1;
k

(1)
> 0 and

where ! is the fundamental frequency with 0 < j! j < =K ,

0 0

2 0; 2 ) denote the amplitude and phase, respectively, of the k-th har2 4 3T 5 2

monic component, and v(n) is the complex white Gaussian noise with zeromean and variance . Let y = y(0) : : : y(N 1) , where ( )T denotes the transpose, and v 2 C N be similarly formed from fv(n)gN . Then, (1) can n
1 1 =0

be compactly rewritten as

y = A(! ) + v;
0

(2)

where A(! ) 2 C N
0 2 4

denotes the Vandermonde matrix with the k-th column

3T 5 2 4 3T 5 ( 1) 1

given by ak = 1 ejk!0 : : : ejk N !0 , and = ej 1 : : : K ej K . It is assumed that the number of harmonic components, K , is known. (The estimation of K was discussed in, e.g., 1], 2], and 3].) The problem of interest is to estimate ! and f k ; k gK from the observations fy(n)gN . n k
0 =1 1 =0

The above problem occurs in speech and musical signal processing, angular speed determination of rotating targets illuminated by a radar, passive detection and location of helicopters and boats, among others. There have been a number of studies on harmonic sinusoidal parameter estimation. Speci cally, an adaptive comb ltering technique for harmonic signal enhancement was introduced in 4]. The method typically requires a relatively large number of 2

observations so that its convergence can be ensured. Thus, it may not be used in applications where the data length is short. Another harmonic parameter estimation algorithm was proposed in 5]. It makes use of a generalized least square criterion which is a nonlinear function of ! . As such, a 1-dimensional
0

(1D) search is in general needed, similar to the nonlinear least squares (NLS) algorithm to be discussed next. The detection of harmonic signals was investigated in 6]. It was shown that the knowledge of the harmonic structure and the fundamental frequency can be incorporated to signi cantly improve the detection performance. Yet, how to obtain an accurate estimate of such knowledge was not addressed there. Apparently, compared with the rich literature dealing with standard (i.e., non-harmonic) sinusoidal parameter estimation (see, e.g., 7]- 14] and and the references therein), the estimation of harmonic sinusoidal parameters has been far less emphasized. The reason why the attention to this topic has been somewhat de cient may be that the NLS estimate of ! , which coincides with the optimal maximum
0

likelihood (ML) estimate when v(n) is white Gaussian, can be obtained by using a seemingly simple 1D search of the NLS cost function. Speci cally, the NLS estimates of the unknown parameters are determined by (see, e.g., 13])
!0 = arg min yH ^ !0
h 0

IN A(! ) AH (! )A(! ) AH (! ) y ^ = AH (! )A(! ) AH (! )y ; ! !

h i 1 0 0 0 0 i 1 0 0

(3) (4)

0 = ^0

where IN is the N N identity matrix. Unfortunately, the NLS cost function in (3) is usually multi-modal with many local minima. An example of the NLS cost function is shown in Figure 1 where K = 5; N = 32; 3
2

= 0:1; ! =
0

0:08,

= 1 and

= =4; 8k. Hence, the minimization of the NLS

cost function requires the use of a very ne searching algorithm and may be computationally prohibitive. In this paper, we present a new method for the parameter estimation of harmonic sinusoidal signals. The proposed method makes use of some initial parameter estimates obtained by, for example, the MUSIC algorithm 7] which ignores the harmonic structure. Then, the estimates are re ned by using a Markov-like weighted least squares (WLS) technique. We show that the method is computationally more e cient than the NLS method, and yet it performs similarly to the latter and is very close to the Cramer-Rao bound (CRB), the best performance bound of any unbiased estimators, for moderate or large N and/or signal-to-noise ratio (SNR).

2 The Markov-Like WLS Estimator

The Markov-like WLS estimator rst ignores the harmonic structure and uses some standard sinusoidal parameter estimator, such as MUSIC 7], to obtain the initial estimates, !k of !k = k! , for k = 1; : : : ; K . The initial estimates, ~ f ~k ; ~k gK of f k ; k gK , can be obtained via least squares (from (4) with k k
0 =1 =1 2 4 1 1

k!0 replaced by !k ). Next, let 0 = ~

2

3T

!0 : : :
1

K!0 5

2RK ,
3 1

and

matrix S0 2 Z K
3

= !
4

3T 0 1 1

:::

R(2K+1)

. Apparently, there exists a

(2K +1)

that has full column rank so that

0 = S0 0:

(5)

The MUSIC estimate ~ 0 of

is not statistically e cient. However, for suf0

ciently large N and M , where M is length of the data subvectors used in MUSIC 7], its performance is close to the CRB corresponding to Hence, a Markov-like WLS estimate, ^ 0 of 0, can be obtained as ^ 0 = arg min ~ 0 S0
0

10].

W;
0

(6)
3 3K

where the weighting matrix W0 = CRB ( ~0), with CRB( ~0) 2 R K

1

being

the CRB matrix that does not assume the knowledge of the harmonic structure and is evaluated at ~ 0. For large N , W0 can be well approximated by 14]
2 6 6 6 6 6 6 6 6 6 6 6 6 6 4 3

W0

0 ...

W0 '

0 WK

7 7 7 7 7 7 7 7 7 7 7 7 7 5

(7)

where
2 6 6 6 6 6 6 6 6 2 6 6 6 6 6 4 3

0
2

0 Wk = 1 0 2N ~k
2 2

N ~

2 0 N ~ k 3 N ~k

7 7 7 7 7 7 7 2 2 7 k 7 7 7 7 7 3 2 5

; k = 1; 2; : : : ; K:

(8)

0 The fact that Wk is block-diagonal implies that the amplitude estimates in ^ 0 and ~ 0 are equivalent, i.e., applying WLS does not lead to re ned estimates

of f k gK . Hence, the cost function in (6) can be simpli ed by excluding the k

=1

amplitude parameters. 5

Let 2 R K and 2 R K
2 1 (

+1) 1

be similarly de ned to 0 and 0, respectively,

=1

except that the amplitude parameters f k gK have been removed. It follows k that
2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 4 3

0 1 0 ::: 0 1 0 0 ::: 0 0 0 1 ::: 0

= 2 0 0 ::: 0 ... ...

0 0 0 ::: 1
K 0 0 ::: 0

7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 5

K X k=1

(E k
2

1;k +1

4 + kE k; ) = S ;
2 1

(9)

where Ei;j is the 2K (K + 1) elementary matrix with unit ij -th element and zero elsewhere. Evidently, S has full column rank by construction. Let ~ be the corresponding initial estimate of . Similarly to (6), the Markov-like WLS estimate ^ of is given by ^ = arg min ~ S where W 2 R K
2 2K 2

W;

(10)

is block-diagonal with the diagonal blocks obtained from

0 (8) by deleting the rst row and column of each Wk . The solution to (10) is

^ = ST WS 6

ST W ~

(11)

Observe that

(see (8)) cancels out in (11) and, hence, there is no need for

an estimate of it. It should be noted that a new set of phase estimates, which are di erent from those in ^ , can be obtained from (4) by using the ! given in ^ . We have ^
0

observed that the so-obtained phase estimates are generally more accurate and should be preferred over those in ^ , especially when the SNR is relatively low. It is also worth noting that the sparse structure of S may be exploited for implementing (11). Speci cally, it can be shown that the calculation of

ST WS, which will become quite time-consuming if implemented in a direct

manner and when K is large, will reduce to only a few additions provided that the sparsity of S is properly exploited. We refer the readers to 15], where an e cient implementation of an algorithm similar to (11) was discussed in detail.

3 Cramer-Rao Bound
An asymptotic (for large N ) CRB for the case of real-valued harmonic sinusoidal signals was derived in 4]. In the following, we derive the exact CRB for the parameter estimation problem posed for the complex data model in (1).
2 3T 5 2 4 3T

Let

= ! ::: K ::: K = ! T that is a permutation of 0. We rewrite (2) as

4 0 1 1 0 0

2R

(2K +1) 1

. Observe

4 y = B(! ; ) + v = x( ) + v;

(12)

where B(! ; ) 2 C N
0

with its k-th column de ned by bk = ej k ak . By using

the Slepian-Bangs formula (see, e.g., 13]), the CRB matrix for the problem under study is given by
H CRB ( ) = 2 Re @ x@ ( ) @ x( T ) ; @
" # 1 2

(13)

where ( )H denotes the Hermitian transpose. Next, we evaluate the partial derivatives in (13) as follows:
@ x( ) 4 @ x( ) @ x( ) @ x( ) 5 = ; @ T @!0 @ T @ T
2 3

(14)

where
@ x( ) =D ; @!0 @ x( ) = B(!0; ); @ T 8 < @ x( ) = j B(!0; )diag : 1; : : : @ T

(15) (16)
9 =

4 = j B(! ; )C;
0

(17)

with the k-th column of D 2 C N

2 4

given by
3T

dk = jkej k 0 ejk!
It follows that
8 > > > > > > > > > > > > 2 > < > > > > > > > > > > > > > : 2 6 6 6 6 6 6 6 6 6 6 6 6 6 4

: : : (N

1)ejk N
(

1)!0 5

(18)

DH D

DH B

DH BC

CRB( ) = 2 Re

BH D
j CT BH D

BH B
j CT BH B

j BH BC

CT BH BC

39 1 > > 7> > 7> > 7> > 7> > 7> > 7> = 7 7 7> 7> > 7> > 7> > 7> > > 5> > > ;

; (19)

where we have omitted the dependence of B on ! and for notational sim0

plicity.

4 Numerical Results
We compare the performance of our proposed estimator and the optimal NLS method, which are referred to as the WLS and NLS estimators, respectively. We also compare the performance of the estimators with the CRB. For both estimators, we use MUSIC to obtain the initial parameter estimates. Speci cally, we take the smallest frequency estimate (and ignore the others) obtained by MUSIC as the initial estimate of ! . The NLS cost function is minimized
0

by using a gradient-type nonlinear optimization routine, fminu , provided in MATLAB. The signal consists of K = 5 harmonic components corrupted by a zero-mean complex white Gaussian noise, with ! = 2
0

0:08,

=1

and 10 log als.

k
10

= =4; 8k. The SNR for the k-th harmonic component is de ned as
k=
2 2

(dB). All results shown below are based on 200 independent tri-

The rst example considers the e ect of SNR on the parameter estimation accuracy. Figure 2(a) shows the root mean squared errors (RMSEs) of the estimates of ! , along with the CRB, as a function of the SNR for the rst
0

harmonic component when N = 32 and M = 15 (M is the length of the data subvectors used in MUSIC). The curves corresponding to the amplitude and phase estimation are similar to those for the frequency estimation and thus are not shown here. The performance of MUSIC is included in the plot to show 9

the merit of incorporating the knowledge of the harmonic structure. It is seen that, for SNR 0 dB, the performance of WLS is very close to the CRB. For small SNRs, the poor initial condition provided by MUSIC may cause NLS to converge to some local minima, as seen in the gure when SNR = 10 and 0 dB, or even not to converge at all, which is the case when SNR = 15 and 5 dB in this example. It is also noted that WLS has a lower threshold SNR than NLS. Figure 2(b) shows the counterpart results when N = 256 and M = 50. It is observed that WLS and NLS behave similarly as in the previous gure except that the threshold SNR is decreased for both algorithms. Next, we examine the e ect of the data length, N , on the performance of the methods under study. The SNR for the rst harmonic is xed at some value and N is varied from 12 to 200. For each value of N , M is chosen as N=2 rounded up to the nearest integer. The other parameters are the same as in the previous example. Figure 3(a) shows the RMSEs of the estimates of ! as
0

well as the CRB when SNR=5 dB. We observe that even for relatively large
N (such as when N = 146 in this example), the NLS algorithm may converge

to some local minimum (due to poor initial estimates), whereas the WLS method is very close to the CRB for N > 20. Figure 3(b) shows the results when the SNR is increased to 30 dB. In this case WLS and NLS are seen to perform similarly to one another for N > 20, with the latter being slightly better than the former. The performance of WLS can be slightly improved by using MODE 12] instead of MUSIC to provide the initial parameter estimates when the SNR is moderate or high. This is because MODE usually yields more accurate initial estimates than MUSIC in such a case. 10

Figure 4 shows the numbers of MATLAB ops required by the WLS and NLS algorithms in the estimation of ! in the previous example. The number
0

of ops involved in the estimation of the amplitude and phase parameters is not counted since the associated computation is negligible. Neither do we count the number of ops involved in the MUSIC algorithm because both WLS and NLS use MUSIC for initialization. Since NLS is iterative and the associated number of ops may vary from trial to trial, the numbers shown in Figure 4 for NLS are the averages over 200 trials. It is seen from this gure that WLS is computationally much simpler than NLS. Also observe that whereas the complexity of WLS is not a ected by N , NLS in general becomes more involved as N increases, which is explained by the fact the NLS cost function becomes more and more erratic as N increases 11]. We see that the number of ops involved in the NLS estimator may increase when N becomes too small. This is due to the poor initial estimates which makes the NLS estimator more di cult to converge.

5 Conclusions
We have presented a Markov-like WLS estimator for harmonic sinusoidal parameter estimation. We have shown through numerical examples that the proposed estimator performs similarly to the optimal NLS method when the SNR and/or N are moderate or high, but the former is computationally signi cantly simpler than the latter. The proposed WLS estimator may also be extended to estimate the parameters 11

of superimposed harmonic sinusoidal signals in which we have several harmonic signals present in the observed data. The extension would be straightforward if the harmonic signals are well-separated. i.e., the fundamental frequency of any harmonic signal is not close to any multiples of the other fundamental frequencies. Without this condition, it would be di cult to identify the fundamental frequencies associated with each harmonic signal from the initial frequency estimates. The reason for considering the WLS estimator for superimposed harmonic sinusoidal parameter estimation is that the NLS method in this case will need a search over an L-dimensional parameter space, where L is the number of harmonic signals, whereas the WLS estimator will still have a closed-form solution. As a result, the computational advantage of the WLS estimator over the NLS method will be even more striking. However, how to obtain a set of reliable initial estimates of the fundamental frequencies when the harmonic signals are not well-separated remains a future topic of research.

12

References
1] M. Wax and T. Kailath, \Detection of signals by information theoretic criteria," IEEE Transactions on Acoustics, Speech Signal Processing, vol. 33, pp. 387{392, April 1985. 2] B. G. Quinn, \Testing for the presence of sinusoidal components," in Time Series and Allied Processes, Papers in Honour of E. J. Hannan, J. Gani and M. Priestley (eds.). She eld: Applied Probability Trust, 1986. 3] B. G. Quinn, \Estimating the number of terms in a sinusoidal regression," Journal of Time Series Analysis, vol. 10, no. 1, pp. 71{75, 1989. 4] A. Nehorai and B. Porat, \Adaptive comb ltering for harmonic signal enhancement," IEEE Transactions on Acoustics, Speech Signal Processing, vol. 34, pp. 1124{1138, October 1986. 5] B. G. Quinn and P. J. Thomson, \Estimating the frequency of a periodic function," Biometrika, vol. 78, no. 1, pp. 65{74, 1991. 6] M. Zeytinoglu and K. M. Wong, \Detection of harmonic sets," IEEE Transactions on Signal Processing, vol. 43, pp. 2618{2630, November 1995. 7] R. O. Schmidt, \Multiple emitter location and signal parameter estimation," IEEE Transactions on Antennas and Propagation, vol. 34, pp. 276{280, March 1985. 8] E. J. Hannan and B. G. Quinn, \The resolution of closely adjacent spectral lines," Journal of Time Series Analysis, vol. 10, no. 1, pp. 13{31, 1989. 9] R. Roy and T. Kailath, \ESPRIT{estimation of signal parameters via rotational invariance techniques," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 984{995, July 1989. 10] P. Stoica and A. Nehorai, \MUSIC, maximum likelihood, and Cramer-Rao bound," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 720{741, May 1989. 11] P. Stoica, R. L. Moses, B. Friedlander, and T. Soderstrom, \Maximum likelihood estimation of the parameters of multiple sinusoids from noisy measurements," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 37, pp. 378{392, March 1989. 12] P. Stoica and K. C. Sharman, \Maximum likelihood methods for direction-ofarrival estimation," IEEE Transactions on Acoustics, Speech Signal Processing, vol. 38, pp. 1132{1143, July 1990. 13] P. Stoica and R. L. Moses, Introduction to Spectral Analysis. Upper Saddle River, NJ: Prentice Hall, 1997.

13

14] P. Stoica, A. Jakobsson, and J. Li, \Cisoid parameter estimation in the colored noise case: asymptotic Cramer-Rao bound, maximum likelihood, and nonlinear least-squares," IEEE Transactions on Signal Processing, vol. 45, pp. 2048{2059, August 1997. 15] H. Li, P. Stoica, and J. Li, \Computationally e cient maximum likelihood estimation of structured covariance matrices," IEEE Transactions on Signal Processing, vol. 47, pp. 1314{1323, May 1999.

14

200 180 160

NLS Cost Function

140 120 100 80 60 40 20 0 0.5 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 0.4 0.5

Frequency (Hz)

Fig. 1. An example of the NLS cost function versus frequency.

15

10

Angular Frequency (N=32)

MUSIC WLS NLS CRB

10

10

RMSE
10
2

10

10 15

10

10

15

20

25

30

SNR (dB)

(a)
10
4

Angular Frequency (N=256)

MUSIC WLS NLS CRB

10

10

RMSE
10
2

10

10 15

10

10

15

20

25

30

SNR (dB)

(b)
Fig. 2. RMSEs of the estimates of !0 and the associated CRB versus the SNR. (a) N = 32 and M = 15. (b) N = 256 and M = 50.

16

10 10 10 10

Angular Frequency (SNR=5)

MUSIC WLS NLS CRB

RMSE

10 10 10 10 10

20

40

60

80

100

120

140

160

180

200

Data Length

(a)
10 10 10 10
2

Angular Frequency (SNR=30)

MUSIC WLS NLS CRB

RMSE

10 10 10 10 10

20

40

60

80

100

120

140

160

180

200

Data Length

(b)
Fig. 3. RMSEs of the estimates of !0 and the associated CRB versus N . (a) SNR=5 dB. (b) SNR=30 dB.

17

10

10

WLS NLS

MATLAB Flops

10

10

10

10

20

40

60

80

100

120

140

160

180

200

Data Length

Fig. 4. MATLAB ops versus N .

18