3038 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO.

7, JULY 2008

From (50) [10] L. C. Godara and A. Cantoni, “Uniqueness and linear independence
of steering vectors in array space,” J. Acoust. Soc. Amer., vol. 70, pp.
[b; b3 ]H VH ()GGH GGH V()
b 467–475, Aug. 1981.
=0
b3 [11] B. Picinbono, “On circularity,” IEEE Trans. Signal Process., vol. 42,
no. 12, pp. 3473–3482, Dec. 1994.

) [b; b3 ]H VH ()GGH V() 3 = 0

b
b
(51)
[12] A. J. Barabell, “Improving the resolution performance of eigenstruc-
ture-based direction-finding algorithms,” in Proc. IEEE Int. Conf.
Acoust., Speech, Signal Processing (ICASSP), Boston, MA, May
1983, pp. 336–339.
where the property that PG = GGH is the projection matrix is used.
Note that (51) is in a quadrature form. We then conclude that

GH V()
b
= 0: (52)
b3
Stochastic Maximum-Likelihood DOA Estimation in the
Presence of Unknown Nonuniform Noise
Lemma 3 actually shows an equivalence between (26) and the esti-
mator fr (). Chiao En Chen, Student Member, IEEE,
() = [ba(); b3 a()3 ]. From Lemma 3 we know that if Q()
Let a
6
Flavio Lorenzelli, Senior Member, IEEE, Ralph. E. Hudson, and
drops rank at  = l , then GH a () = 0. Since A  is a full-column- Kung Yao, Life Fellow, IEEE
rank matrix that is orthogonal to G, the following statement can be

6
equivalently made:
• there exists a  = l , such that a () is a linear combination of

columns of A(:; q ), q = 1; . . . ; LR + 2LC .
From Assumption 1, we know the probability for the existence of  can
be reasonably assumed to be zero.
Uniqueness of Estimator (35)
6
If there exists a  = c and aH ()G1 GH 
1 a( ) = 0. Then the
following equations hold:
a() 0 concentrates the log-likelihood function with respect to the signal and noise
GH = 0; G H = 0: (53)
0 a()3
6
An equivalent statement is made as follows:
• there exists a  = c , such that [aT (); 0T ]T , and [0T ; aH ()]T
 (:; q ), q = 1; . . . ; LR +
is a linear combination of columns of A
2L C .
Similarly, the probability for the existence of  is considered as zero,
and the uniqueness for (35) can be guaranteed with probability one.
REFERENCES
[1] P. Chage, Y. Wang, and J. Saillard, “A Root-MUSIC algorithm for
non circular sources,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal
Processing (ICASSP), Salt Lake City, UT, May 2001, pp. 7–11.
[2] R. O. Schmidt, “A Signal Subspace Approach to Multiple Source Loca-
tion and Spectral Estimation,” Ph.D. dissertation, Stanford Univ., Stan-
ford, CA, May 1981.
[3] R. Roy, A. Paulraj, and T. Kailath, “ESPRIT-estimation of signal pa-
rameters via rotational invariance techniques,” IEEE Trans. Acoust.,
Speech, Signal Process., vol. 37, pp. 984–995, Jul. 1989.
[4] P. Stoica and K. Sharman, “Maximum likelihood methods for direction
estimation,” IEEE Trans. Acoust., Speech, Signal Process., vol. 38, pp.
1131–1143, Feb. 1990.
[5] M. Pesavento, A. B. Gershman, and K. M. Wong, “Direction finding in
partly calibrated sensor arrays composed of multiple subarrays,” IEEE
Trans. Signal Process., vol. 50, no. 9, pp. 2103–2115, Sep. 2002.
[6] J. Li, B. Halder, P. Stoica, and M. Viberg, “Computationally efficient
angle estimation for signals with known waveforms,” IEEE Trans.
Signal Process., vol. 43, no. 9, pp. 2154–2163, Sep. 1995.
[7] A. van der Veen and A. Paulraj, “An analytical constant modulus al-
gorithm,” IEEE Trans. Signal Process., vol. 44, no. 5, pp. 1136–1155,
May 1996.
[8] H. Abeida and J. P. Delmas, “Music-like estimation of direction of
arrival for noncircular sources,” IEEE Trans. Signal Process., vol. 54,
no. 7, pp. 2678–2690, Jul. 2006.
[9] K. C. Tan, S. S. Goh, and E. C. Tan, “A study of the rank-ambiguity
issues in direction-of-arrival estimation,” IEEE Trans. Signal Process.,
vol. 44, no. 4, pp. 880–887, Apr. 1996.
Abstract—This correspondence investigates the direction-of-arrival
(DOA) estimation of multiple narrowband sources in the presence of
nonuniform white noise with an arbitrary diagonal covariance matrix.
While both the deterministic and stochastic Cramér-Rao bound (CRB)
and the deterministic maximum-likelihood (ML) DOA estimator under
this model have been derived by Pesavento and Gershman, the stochastic
ML DOA estimator under the same setting is still not available in the
literature. In this correspondence, a new stochastic ML DOA estimator
is derived. Its implementation is based on an iterative procedure which
nuisance parameters in a stepwise fashion. A modified inverse iteration
algorithm is also presented for the estimation of the noise parameters.
Simulation results have shown that the proposed algorithm is able to pro-
vide significant performance improvement over the conventional uniform
ML estimator in nonuniform noise environments and require only a few
iterations to converge to the nonuniform stochastic CRB.
Index Terms—Direction-of-arrival (DOA) estimation , nonuniform noise,
sensor array processing, stochastic maximum likelihood (ML) algorithm.
I. INTRODUCTION
Direction–of-arrival (DOA) estimation has been one of the central
problems in radar, sonar, navigation, geophysics, and acoustic tracking.
A wide variety of high-resolution narrowband DOA estimators have
been proposed and analyzed in the past few decades [2]–[5]. The max-
imum likelihood (ML) estimator, which shows excellent asymptotic
performance, plays an important role among these techniques. Many of
the proposed ML estimators are derived from the uniform white noise
assumption [5]–[7], in which the noise process of each sensor is as-
sumed to be spatially uncorrelated white Gaussian with identical un-
known variance. It is shown that under this assumption the estimates of
the nuisance parameters (source waveforms and noise variance) can be
expressed as a function of DOAs [8]–[10], and, therefore, the number of
independent parameters to be estimated is substantially reduced. This
Manuscript received May 21, 2007; revised November 26, 2007. This work
was supported by in part by the NSF CENS program NSF Grant EF-0410438,
by AROD-MURI PSU Contract 50126, and by ST Microelectronics, Inc.The
associate editor coordinating the review of this correspondence and approving
it for publication was Dr. Eran Fishler.
The authors are with the Electrical Engineering Department, University
of California, Los Angeles, CA 90095 USA (e-mail: chiaoen@ucla.edu;
lorenz@ucla.edu; ralph@ucla.edu; yao@ee.ucla.edu).
Digital Object Identifier 10.1109/TSP.2008.917364

1053-587X/$25.00 © 2008 IEEE

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 7, JULY 2008 3039

procedure is called concentration which results in a substantial reduc-
tion of the parameter space required for numerical optimization.
In many applications, the uniform white noise assumption may be
unrealistic. For densely placed sensors, the sensor noise may be corre-
lated and therefore should be modeled as a colored random process. In
order to reduce the number of nuisance noise parameters, various noise
modeling techniques have been applied in the literature [11]–[15]. In
[11], [12], the noise process is assumed to follow the AR(MA) model
while in [13], [14] the noise covariance matrix is parameterized to
have a linear structure. For non-Gaussian and nonstationary environ-
ments, the generalized autoregressive conditional heteroscedasticity
(GARCH) modeling has been proposed recently [15].
In some practical applications, the sensors may be sparsely placed
so that the noise at each sensor is essentially uncorrelated. However,
the noise power of each sensor can still be different due to the nonuni-
formity of sensor noise or the imperfection of array calibration. In this
case, the noise covariance matrix can be modeled as a diagonal matrix
but the diagonal elements in general are not identical. It is crucial to
note that this scenario can not be obtained as a special case of the Au-
source with respect to the array centroid, where m = 1; . . . ; M . The
y
array output (t), observed at the tth snapshot can then be modeled as
y(t) = Ax(t) + e(t); t = 1; . . . ; N (1)
where x(t) = [x1 (t); . . . ; xM (t)]T and e(t) are the signal and the
noise vector observed at the tth snapshot respectively, and
A = [a(1); . . . ; a(M )] (2)
is the steering matrix. It will be assumed that the geometry of the sensor
A
array is such that is full rank for all DOAs of interest. As a result,
A AH will always be positive definite.
In this paper, we investigate the case where the sensor noise of each
channel is a spatially uncorrelated white Gaussian random process, and
it follows that the noise covariance matrix has the following diagonal
structure:

Efe(t)e(t)H g = Q = diagf12 ; . . . ; P2 g

toregressive Moving Average (ARMA) modeling since the noise is uni-
form and white in the simplest zero-order model but becomes colored (3)
as the model order increases [1]. Also the DOA estimators derived from
where p2 is the noise variance of the pth sensor.
x
either the uniform white noise assumption or the general colored noise
modeling techniques may not give satisfactory results in this nonuni- For the signal waveform (t), two different modeling techniques
form uncorrelated noise environment, since the uniform white noise can be applied: 1) the deterministic model which assumes (t) to be x
assumption blindly treats all sensors equally and the general colored deterministic but unknown, and 2) the stochastic model which charac-
noise modeling technique neglects the fact that the sensor noises are x
terizes (t) as a stochastic process. In [1], a ML DOA estimator is de-
uncorrelated. rived based on the deterministic signal model. Here, we take the second
The considered uncorrelated unknown noise model has a linear x
route and model (t) as a zero-mean Gaussian process with covariance
covariance structure and thus many DOA estimation algorithms can matrix
be applied. However, it appears that the ML DOA estimator does not
have a concentrated form except for the trivial single noise parameter Efx(t)x(t)H g = Rx : (4)
case [13], [14] and therefore a multidimensional search in the joint
signal-noise parameter space is generally required. To circumvent
such difficulty, many suboptimal solutions have been proposed [13],
y
From (1), (3), and (4), it follows immediately that (t) is also a zero
mean Gaussian random process and its covariance matrix can be ex-
[14], [16]–[18].
pressed as:
The deterministic and stochastic CRBs for the considered DOA esti-
mation problem are first derived by Pesavento et al. [1]. In order to mit-
igate the burden of the straightforward implementation of the ML DOA
Efy(t)y(t)H g = Ry = ARx AH + Q: (5)
estimator, a new deterministic MLE based on the stepwise concentra-
tion is also proposed [1]. However, to the best of our knowledge, no
similar work has been carried out for the stochastic case. In this paper, III. ML DOA ESTIMATION
we propose a new stochastic ML DOA algorithm under the same noise
Under the signal model defined in the previous section, we can de-
rive the joint likelihood function of the unknown parameters 9 =
model and in some sense “complete” the understanding of this problem.
The rest of the paper is organized as follows. The stochastic signal
model is introduced in Section II, and the stochastic ML DOA esti- R Q
f22; x ; g as
N
(99) = PN j1Ry jN exp 0 y(t)H R0y 1 y(t)
mator and its implementation is derived in Section III. In Section IV, we
present the modified inverse iteration algorithm for the noise variance f (6)
estimation stage of the iterative optimization procedure. In Section V, t=1
the performance of the proposed nonuniform ML estimator is studied
through extensive numerical simulations. Section VI concludes this where 2 = [1 ; . . . ; M ]T . Taking the logarithm of (6) and neglecting
paper. all the constant terms, the log-likelihood function L(9 9) can be ex-
Throughout this paper, we denote the superscripts T , 3 , H , and y pressed as
as transpose, conjugate, conjugate transpose, and pseudo inverse of a
v
matrix. The operator diagf g denotes the diagonal matrix with the L (9) = g(S; Ry ) = 0N flog jRy j + tr(Ry01S)g
v X
(7)
vector as the main diagonal, and diagf g denotes the column vector
formed from the elements of the main diagonal of square matrix . In X where
addition, trf1g and j 1 j denote the trace and determinant of a square
N
matrix respectively.
S = N1 y ( t) y ( t) H (8)
t=1
II. SIGNAL MODEL is the sample covariance matrix of the array output.
The ML estimate for 9 can then be obtained by solving
Let there be M narrowband sources in the far field of a P -element

9^ = arg max
randomly distributed array. For simplicity, we assume the sources and
the array lie in the same plane, and denote m as the DOA of the mth 9
9 ):
L(9 (9)
3040 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 7, JULY 2008

Clearly, the search space of solving (9) is huge (of dimension M + It appears that (19) is a rather complicated function of Q and no ana-
+
M 2 P ), and, therefore, we seek to reduce the optimization problem lytical solution seems to be available.
2
since the estimation of is our only interest. To this end, a new ML DOA estimator based on the stepwise concen-
We first maximize (9) with respect to Rx for fixed and Q. Taking 2 tration is proposed. The idea of this technique is to numerically concen-
partial derivatives of L (99)
with respect to Rx i;j , we obtain [ ] trate the log-likelihood function by the following iterative procedure.
@L (99)
[ ]
@ Rx i;j
Iterative procedure of the proposed ML DOA estimator

= 0N fa( ) Ry 01 a( ) 0 a( ) Ry01SR0y 1a( )g (10)

j
H
i j
H
i
Step 1) Iter = 1. Compute the MLEs
for i; j = 1; . . . ; M . Setting (10) to zero and putting it into a compact
(2^ ; R^ x ; Q^ ) under the uniform white noise
assumption, summarized as follows [10]:
form, a necessary condition for the extremum point can be written as 2^ = argmax 2 log jAR ^ x A + ^ 2 I j
H

^ x = A SA 0 ^ [A A]01
y y
P

R 2
A Ry01 [S 0 Ry ]R0
y 1 A = 0:
H H

^ 2 = trf(2I 0 AAy )Sg=(P 0 M )

H
(11)

^ = ^ I
P

Q
where Ay = (A A)01 A .
P
For a more detailed derivation of (10)–(11), see [9]. H H

Following a similar derivation as [13] and [14], the optimal Rx that

Step 2) Use (2 ^ ; R^ x ; Q^ ) as an initial estimate, and define A^ as the
solves (11) is obtained as
steering matrix evaluated at 2 ^.
^ x = [A~ H A~ ]01 [A~ H S~ A~ 0 A~ H A~ ][A~ H A~ ]01 ^
Find a refined Q so that the resultant R ^ y = A^ R^ xA^ + Q^
H

R (12)
increases the log-likelihood function.
Step 3) Use the obtained Q ^ to find an improved 2^
where ^
through 2 = argmax2 L(2 2; Q^ ) where
~ = Q01=2A L(2 2; Q) is defined as in (16). Update R^ x using
A (13)
the latest estimates (2 ^ ; Q^ ) through (12).
S~ = Q01=2 SQ01=2 : (14) Iter = Iter + 1
Substituting Ry of (5) by ARx AH ^ + Q, the ML estimate for 2
Repeat Step 2 and Step 3 until the
algorithm converges.
and Q can be obtained by solving

(2^ ; Q^ ) = arg max 2; Q):

L(2 (15)
The parameter “ Iter
” denotes the index of iteration in the procedure.
2Q ; Note that we start with a joint optimization problem (9) of dimen-
+ +
sion M M 2 P , and reduce it to a sequence of smaller problems (of
where dimension M in step 1 and 3 and of dimension P in step 2). Many nu-

(22; Q) = 0N log jQ1 2fPA~ S~ PA~ 0 PA~ + I gQ1 2 j

= =
merical algorithms exist in the literature [19]–[24] which can be used
L P (or modified) to solve the DOA estimation problem in step 1 and step
0N trfS~g + N trfPA~ S~ g (16) 3. The only remaining issue which has not been explained in this corre-
spondence is how step 2 is implemented. The modified inverse iteration
and PA ~ =A ~ [A~ A~ ]01 A~ .
H H
algorithm is proposed for this purpose.
To concentrate L(2 2; Q) further, one would fix 2 and seek an
estimator for Q that maximizes L(22; Q). Unfortunately, a separable
closed-form estimator for Q that maximizes (16) seems to be an- IV. MODIFIED INVERSE ITERATION ALGORITHM
alytically unavailable [13], [14], and that prevents us from further
simplifications. In [25], a general method based on the inverse iteration algorithm
At the same time, we can approach the problem by fixing and Rx 2 for estimating a covariance matrix of a specified structure from vector
in (7) and solve for an estimator for Q that maximizes L . Denote (99) samples of the random process is proposed. We modified this algorithm
q = diag fQg and qp as the pth element of q, then the pth element of to solve step 2 and referred to it as the “modified inverse iteration al-
the gradient vector rq L (99)
can expressed as gorithm” in the following discussion.
01 The basic idea of the this algorithm is as follows. Let D denote the
[r L(99)] = 0 N @ log@ qjRy j + @ trf@Rqy Sg set of all P 2 P diagonal matrices. In each iteration we start with
2^ ^ ^ 1
q p
p p
some initial estimates , Rx , and Q, and a Q 2 D is sought so
= 0 N ftrfRy01E g 0 trfRy01SR0y 1E gg p;p p;p ( 1 ^ )
that S 0 Q; Ry satisfies (20), i.e.
= 0 N trf[Ry01 0 Ry01SR0y 1]E g (17) p;p
01 0 1
= 0 N [Ry 0 Ry SRy ] 01 (18) p;p
[R^ y01(R^ y 0 (S 0 1Q))R^ y01] = 0 ; p = 1; . . . ; P (21)
p;p

where E is a P 2 P matrix with the (i; j )th element equals to 1

i;j

9) to zero, a necessary condition for

and 0 elsewhere. Setting r L(9 q
where R^ y = A^ R^ xA^ + Q^ . Note 1q = diagf1Qg is an improving
H

the extremum point is obtained as direction since

trf[Ry01 0 Ry01SR0y 1]E g = 0; p = 1; . . . ; P

p;p (19)
[r L(9^ )] 1q = trf[0R^ y01 + R^ y01SR^ y01]1Qg
q
T

N
or equivalently
= trf[0R^ y01 + R^ y01(S 0 1Q)R^ y01]1Qg
[R0y 1 0 Ry01SR0y 1] = 0; p = 1; . . . ; P:
p;p (20) +trf[R^ y011QR^ y01]1Qg (22)
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 7, JULY 2008 3041

= trf[R^ 0y 1 1QR^ y01]1Qg  0: (23)

The equality sign in (23) holds since the first term in (22) is zero by
construction.
The condition (21) is equivalent to

trf[R^ y01SR^ y010R^ y01(R^ y+1Q)R^ y01]E g =0; p =1; . . .; P: (24)

p;p

Putting (24) into a matrix form, then 1q, the vector of the diagonal
elements of 1Q can be solved by the following linear equation:

H1q = u (25)

where

[H] = trfE R^ y01E R^ y01g

i;j j;j i;i (26)

and

u = trf[R^ y01SR^ y01 0 R^ y01]E g

i i;i (27)
Fig. 1. Comparison of the DOA estimation RMSEs and the stochastic CRB in
= 1 ...
for all i; j ; ;P. the presence of nonuniform sensor noise (uncorrelated sources scenario).
The overall procedure of the modified inverse iteration algorithm is
summarized as follows.
V. SIMULATION RESULTS
Modified inverse iteration algorithm In this section, we present the simulation results of the proposed sto-
chastic ML estimator in comparison with the deterministic ML esti-
Step 1) Given some initial estimates ; x ; (2^ R^ Q^ )
, compute an mator [1], the power domain (PD) method [26], and the approximate
improving direction 1Q
by (25). ML (AML) algorithm [18]. In order to make a fair comparison, we ini-
Step 2) Backtracking Line Search tialize the AML algorithm with the DOA estimates of the conventional
Set t = 1. stochastic uniform ML estimator (same as the 1st iteration estimate of
while the proposed stochastic ML algorithm). The nonlinear DOA estima-
(S R^ + 1Q) (S R^ ) +
g ; y t < g ; y trq g ; y T (S R^ ) 1q tion required in step 1 and step 3 of the proposed iterative procedure
or Q^ + 1Q 0
t < , is solved through the alternating maximization (AM) algorithm [19],
implemented by an initial 1 coarse grid search followed by the golden
t = t.
end section fine search. Despite of the fact that AM guarantees only a local
Step 3) Set new Q^ Q^ + 1Q
as t optimal solution, excellent global convergence has been observed in
this type of DOA estimation problem [19].
Repeat Step 1. to Step 3. until the algorithm converges,
where  and  are two constants satisfying <  < : 0 05 The simulation settings and scenarios of this paper are intentionally
0
and <  < . 1 chosen to be identical to the settings in [1] in which the DOA esti-
mation of two narrow-band sources of equal power using a uniform
linear array (ULA) is considered. The DOAs of the narrowband sources
are set to be 7 and 13 relative to the broadside respectively, and the
Remark: The proposed modified inverse iteration algorithm has a
very interesting connection to the Newton’s method. To see this we first
compute the Hessian of L (99) q
with respect to . After some straight-
ULA is assumed to be omnidirectional with half-wavelength inter-ele-
ment spacing. In each simulation scenario, the root-mean-square-errors
forward manipulations, we have the following expression:
(RMSEs) of the DOA estimates are computed by averaging over 1000
[r2 L(99)] = N trfE R^ y01E R^ y01g Monte Carlo runs and plotted along with the stochastic CRB [1], [27].
q i;j j;j i;i

02N <ftr[E R^ y01E R^ y01SR^ y01]g (28)

jj i;i
Throughout the first four scenarios, we let P = 10 and consider the
following noise covariance matrix:
for all i; j = 1; . . . ; P . It follows immediately that the Newton’s di- Q = 2diagf[10:0; 2:0; 1:5; 0:5; 8:0; 0:7; 1:1; 3:0; 6:0; 3:0]g:
rection , 1qnt can be obtained as

1qnt = 0[r2 L(99)]01 r L(99):

In the first scenario, we assume the sources to be uncorrelated and
q q (29) set the array-SNR (ASNR) to be 5 dB, where the ASNR with respect
to the mth source is defined as [1]
However, the Newton’s method cannot be applied in our problem
since L(99) is highly nonconvex. In fact, it is observed that the Hessian
= Efjx P(t)j g
2 P
1:
(99)
m
ASNRm (30)
of L evaluated at the initial estimate is not positive semi-definite 2
1q
p
p =1
which means nt is not guaranteed to be a improving direction.
1q , on the other hand is always an ascending direction as is proved Fig. 1 shows the RMSE performances of the tested algorithms with
in (23). After comparing (26) to (28) and (27) with (17), it is interesting respect to the number of snapshots.
(9) = u
to see that rq L N . Furthermore, 0 r2q L [ (9)]
R^ = R
i;j is exactly In the second scenario, we assume the sources to be uncorrelated and
[H ] S = R S = R
N i;j if y if y and y y . As a result, the =
set N 100 snapshots. The RMSEs of the tested algorithms versus the
proposed modified inverse iteration algorithm can be viewed as a novel ASNR are shown in Fig. 2.
modification of the Newton’s method for the considered nonconvex In the third scenario, we assume the sources to be partially correlated
optimization problem. where the correlation coefficient of the two sources are assumed to be
3042
Fig. 2. Comparison of the DOA estimation RMSEs and the stochastic CRB in
the presence of nonuniform sensor noise (uncorrelated sources scenario).
Fig. 3. Comparison of the DOA estimation RMSEs and the stochastic CRB in
the presence of nonuniform sensor noise (correlated sources scenario).
0.1 (same as in [1] but due to a typo appeared as 0.9). The ASNR is set
to be 7.5 dB and the RMSE performance is plotted with respect to the
number of snapshots in Fig. 3.
In the fourth scenario, we assume the same correlated sources as the
third simulation scenario. N is fixed to be 100 and the RMSEs of the
tested algorithms are plotted in Fig. 4 versus the ASNR.
In the following two simulation scenarios, we investigate how the
nonuniformity of the sensor noise affects the performance of the tested
algorithms. As in [1], we define the worst-noise-power-ratio (WNPR)
2
max
WNPR = (31)
min
2 form algorithms provide essential performance improvement over
2
as a measure of nonuniformity of the sensor noise, where max
and
2
min are the highest and lowest sensor noise variance in the array,
respectively. In each Monte Carlo run, we fix the WNPR and ran-
domly choose two sensor locations in the ULA, one with noise vari-
2 2 2
ance min and the other max = WNPRmin . The rest of the sensors
U
are assigned their noise variances according to a uniform distribution
2 2
(min ; max ).
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 7, JULY 2008
Fig. 4. Comparison of the DOA estimation RMSEs and the stochastic CRB in
the presence of nonuniform sensor noise (correlated sources scenario).
Fig. 5. Comparison of the DOA estimation RMSEs and the stochastic CRB
versus the number of sensors.
In the fifth scenario, the RMSE of the tested algorithms versus the
number of sensors is investigated. We assume the sources to be uncor-
related and the ASNR, number of snapshots, and the WNPR are set to
be 20 dB, 100, and 65, respectively. Simulation results are shown in
Fig. 5.
In the sixth scenario, we assume uncorrelated sources again and set
the ASNR, number of snapshots, and the number of sensors to be 0
dB, 100, and 10, respectively. The RMSE performance is plotted with
respect to the WNPR in Fig. 6.
From Figs. 1–6, it can be observed that all the tested nonuni-
the conventional (uniform) ML estimator in the nonuniform noise
environment. As the nonuniformity of sensor noise becomes larger
(WNPR 1), a more substantial performance improvement of the
proposed nonuniform ML estimator over the uniform ML estimator is
observed. The performance improvement also becomes more signifi-
cant as the number of degrees of freedom (N times P ) increases.
Among the tested nonuniform algorithms, the PD method has the
lowest complexity since the parameter space is only of dimension M
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 7, JULY 2008
Fig. 6. Comparison of the DOA estimation RMSEs and the stochastic CRB
versus the WNPR.
and no additional computation is required to update the nuisance pa-
rameters [26]. However, the RMSE performance of the PD method ap-
pears to be the worst and approaches that of the uniform ML estimator
as the ASNR is sufficiently large. This is not surprising since the ob-
jective function of PD degenerates to the same ML metric of the deter-
ministic uniform ML estimator when p2 approaches zero. The RMSE
performance and the complexity of the proposed stochastic nonuniform
ML estimator is comparable to that of the deterministic nonuniform
ML estimator [1] since both algorithms implement the idea of step-
wise-concentration and both require only two iterations to converge to
a solution close to the CRB in the large sample and high ASNR cases.
However, the deterministic nonuniform ML estimator seems to have a
better threshold property as its RMSE is smaller than that of the sto-
chastic nonuniform ML estimator in the low ASNR regions. The AML
algorithm has a RMSE performance comparable to both the determin-
istic and stochastic ML estimators at the expense of higher complexity
O
(appears to be around ((M 2 + P )3 ) comparing to the complexity
O 3
(M ) of PD and the nonuniform ML estimators [26]). However,
when N is small, the RMSE performance of AML degrades consid-
erably, which is probably due to the asymptotic approximations made
in the derivations.
VI. CONCLUSION
In this correspondence, we address the problem of estimating the
DOAs of multiple narrowband sources in the presence of unknown
nonuniform sensor noise. A new stochastic ML DOA estimator has
been derived. Unlike the conventional uniform ML DOA estimator, no
separate solution for the DOA estimation seems to exist in the nonuni-
form case. To this end, we implement our algorithm by using the so
called stepwise concentration which concentrates all the nuisance pa-
rameters iteratively. A modified inverse iteration algorithm has also
been derived for the estimation of the noise covariance matrices.
The performance of the proposed algorithm is studied through ex-
tensive computer simulations. In all settings, the proposed algorithm
asymptotically converges to the CRB within two iterations and there-
fore the complexity is only a few times higher than the conventional
uniform ML estimator. Simulation results also demonstrate the perfor-
mance improvement in comparison with other nonuniform algorithms
for a variety of scenarios (number of snapshots, SNR, number of sen-
sors, etc.).
3043
REFERENCES
[1] M. Pesavento and A. M. Gershman, “Maximum-likelihood direc-
tion-of-arrival estimation in the presence of unknown nonuniform
noise,” IEEE Trans. Signal Process., vol. 49, no. 7, pp. 1310–1324,
Jul. 2001.
[2] R. Schmidt, “A signal subspace approach to multiple emitter location
and spectral estimation,” Ph.D. dissertation, Stanford Univ., Stanford,
CA, 1981.
[3] M. Wax and T. Kailath, “Optimum localization of multiple sources by
passive arrays,” IEEE Trans. Acoust., Speech Signal Process., vol. 31,
no. 5, pp. 1210–1218, Oct. 1983.
[4] T. Kailath and R. Roy, “Estimation of signal parameters via rotational
invariance techniques,” IEEE Trans. Acoust., Speech Signal Process.,
vol. 37, pp. 984–995, Jul. 1989.
[5] P. Stoica and A. Nehorai, “Music, maximum likelihood and
Cramer-Rao bound,” IEEE Trans. Acoust., Speech Signal Process.,
vol. 37, pp. 720–741, May 1989.
[6] J. F. Bohme, “Estimation of spectral parameters of correlated signals
in wavefields,” Signal Process., vol. 11, no. 4, pp. 329–337, Dec. 1986.
[7] P. Stoica and A. Nehorai, “Performance study of conditional and
unconditional direction-of-arrival estimation,” IEEE Trans. Acoust.,
Speech Signal Process., vol. 38, pp. 1783–1795, Oct. 1990.
[8] J. F. Bohme, “Separated estimation of wave parameters and spectral
parameters by maximum likelihood,” presented at the IEEE Int. Conf.
Acoustics, Speech, Signal Process. (ICASSP), Tokyo, Japan, 1986.
[9] A. G. Jaffer, “Maximum likelihood direction finding of stochastic
sources: A separable solution,” presented at the Int. Conf. Acoustics,
Speech, Signal Process. (ICASSP), New York, Apr. 1988.
[10] P. Stoica and A. Nehorai, “On the concentrated stochastic likelihood
function in array signal processing,” Circuits, Syst., Signal Process.,
vol. 14, pp. 669–674, Sep. 1995.
[11] J. Le Cadre, “Parametric methods for spatial signal processing in
the presence of unknown colored noise fields,” IEEE Trans. Acoust.,
Speech Signal Process., vol. 37, no. 7, pp. 965–983, Jul. 1989.
[12] V. Nagesha and S. Kay, “Maximum likelihood estimation for array
processing in colored noise,” IEEE Trans. Signal Process., vol. 44, no.
2, pp. 169–180, Feb. 1996.
[13] J. F. Bohme and D. Kraus, “On least squares methods for direction of
arrival estimation in the presence of unknown noise field,” presented
at the IEEE Int. Conf. Acoustics, Speech, Signal Process. (ICASSP),
New York, Apr. 1988.
[14] B. Friedlander and A. F. Weiss, “Direction finding using noise co-
variance modeling,” IEEE Trans. Signal Process., vol. 43, no. 7, pp.
1557–1567, Jul. 1995.
[15] H. Amiri, H. Amindavar, and R. L. Kirlin, “Array signal processing
using GARCH noise modeling,” presented at the IEEE Int. Conf.
Acoustics, Speech, Signal Process. (ICASSP), Montreal, QC, Canada,
May 2004.
[16] A. M. Zoubir and S. Aouada, “High resolution estimation of directions
of arrival in nonuniform noise,” presented at the IEEE Int. Conf. Acous-
tics, Speech, Signal Process. (ICASSP), Montreal, QC, Canada, May
2004.
[17] A. B. Gershman, A. L. Matveyev, and J. F. Bohme, “Maximum like-
lihood estimation of signal power in sensor array in the presence of
unknown noise field,” Proc. Inst. Elect. Eng. Radar, Sonar Navig., pp.
218–224, Oct. 1995.
[18] B. Goransson and B. Ottersten, “Direction estimation in partially un-
known noise fields,” IEEE Trans. Signal Process., vol. 47, no. 9, pp.
2375–2385, Sep. 1999.
[19] I. Ziskind and M. Wax, “Maximum likelihood localization of multiple
sources by alternating projection,” IEEE Trans. Acoust., Speech, Signal
Process., vol. 36, no. 10, pp. 1553–1560, Oct. 1988.
[20] K. Sharman, “Maximum likelihood estimation by simulated an-
nealing,” in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process.
(ICASSP), New York, pp. 2741–2744.
[21] P. Stoica and A. B. Gershman, “Maximum-likelihood DOA estimation
by data-supported grid search,” IEEE Signal Process. Lett., vol. 6, pp.
273–275, Oct. 1999.
[22] S. Kay and S. Saha, “Mean likelihood frequency estimation,” IEEE
Trans. Signal Process., vol. 48, no. 7, pp. 1937–1946, Jul. 2000.
[23] M. Li and Y. Lu, “Genetic algorithm based maximum likelihood DOA
estimation,” in Proc. Inst. Elect. Eng. Int. Radar Conf., Edinburgh,
U.K., Oct. 2002.
[24] P. J. Chung and J. F. Bohme, “DOA estimation using fast EM and
SAGE algorithms,” Signal Process., vol. 82, pp. 1753–1762, Nov.
2002.
3044
[25] J. P. Burg, D. G. Luenberger, and D. L. Wenger, “Estimation of struc-
tured covariance matrices,” Proc. IEEE, vol. 70, Sep. 1982.
[26] D. Madurasinghe, “A new DOA estimator in nonuniform noise,” IEEE
Signal Process. Lett., pp. 337–339, Apr. 2005.
[27] A. B. Gershman, P. Stoica, M. Pesavento, and E. G. Larsson, “Sto-
chastic Cramer-Rao bound for direction estimation in unknownnoise
fields,” Proc. Inst. Elect. Eng. Radar, Sonar Navig., pp. 2–8, Feb. 2002.
First-Order Perturbation Analysis of Singular Vectors in
Singular Value Decomposition
Jun Liu, Student Member, IEEE,
Xiangqian Liu, Senior Member, IEEE, and
Xiaoli Ma, Member, IEEE
Abstract—Singular value decomposition (SVD) is an important tech-
nique in signal processing. In the literature, the perturbation analysis of
SVD has been well documented in the context of subspace decomposition.
The contribution of the signal subspace to the perturbation of the singular
vectors that span the signal subspace is often ignored as it is treated as
second and higher order terms, and thus the first-order perturbation is
typically given as the column span of the noise subspace. In this corre-
spondence, we show that not only the noise subspace, but also the signal
subspace, contribute to the first-order perturbation of the singular vectors.
We further show that the contribution of the signal subspace does not
impact on the performance analysis of algorithms that rely on the signal
subspace for parameter estimation, but it affects the analysis of algorithms
that depends on the individual basis vectors. For the latter, we also give
a condition under which the contribution of the signal subspace to the
perturbation of singular vectors may be ignored in the statistical sense.
Numerical examples are provided to validate our theoretic claims.
Index Terms—Perturbation analysis, singular value decomposition
(SVD), subspace decomposition.
I. INTRODUCTION
Singular value decomposition (SVD) plays an important role in
signal processing. For example, subspace-based algorithms in array
signal processing depends on SVD or eigenvalue decomposition
to obtain a basis of the desired signal and/or noise subspaces, and
principal component analysis (PCA) uses SVD to retrieve principal
components from noisy observations. Therefore, perturbation analysis
of SVD is important for these algorithms. Consider the following
observation matrix
X = +1
X X (1)
Manuscript received February 20, 2007; revised November 29, 2007. The
associate editor coordinating the review of the manuscript and approving it for
publication was Dr. Sergiy A. Vorobyov. This work was supported in part by
the U.S. Army Research Laboratory and the U.S. Army Research Office under
grant number W911NF-06-1-0415 and W911NF-06-1-0090. Parts of this work
were presented at the IEEE Statistical Signal Processing Workshop, Madison,
WI, August 26–29, 2007.
J. Liu and X. Liu are with Department of Electrical and Computer En-
gineering, University of Louisville, Louisville, KY 40292 USA (e-mail:
jun.liu@louisville.edu; x.liu@louisville.edu).
X. Ma is with School of Electrical and Computer Engineering, Georgia Insti-
tute of Technology, Atlanta, GA 30332 USA (e-mail: xiaoli@ece.gatech.edu).
Digital Object Identifier 10.1109/TSP.2007.916137
1053-587X/$25.00 © 2008 IEEE
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 56, NO. 7, JULY 2008
where X is a perturbed version of the data matrix X , and 1
X is the
perturbation. The subspace decomposition of X is
X = 6
U V
H = U s6sV Hs + U n6nV Hn (2)
where U s , associated with nonzero singular values, spans the column
space of X , which is also called the signal subspace, and U n , associ-
ated with zero singular values n (66 = ) 0 , spans the orthogonal space
of U s , which is also called the noise subspace. Similarly, the subspace
decomposition of X is given by
H H H
X = 6
U V = s6 s
U V s + U n6nV n : (3)
1
Due to the perturbation X , all the quantities in the right-hand side
(RHS) of (3) may differ from those in the RHS of (2). For example, we
= +1
may express U s as U s U s U s , where 1
U s is the perturbation
of the singular vectors that span the signal subspace.
Perturbation analysis results for subspace decomposition have been
well documented in the literature. For example, the statistics of eigen-
vectors of sample covariance matrices is investigated in [1]. The pertur-
bation of the invariant subspace is considered in [2] and [3]. Based on
1
the results in [2] and [3], a first-order approximation of U s is given in
[4]. Furthermore, [5] and [6] give a second-order approximation using
similar arguments as in [4]. It is shown in [4]–[6] that the contribution
1
of the noise subspace spanned by U n to U s is of the first order of
1 X . It is argued that the contribution of the signal subspace spanned
1 1
by U s to U s is of the second order of X , and thus the contribution
1
of the signal subspace to U s is ignored when the first-order pertur-
bation is considered.
In this correspondence, we analyze the perturbation of U s from an-
other perspective. Different from the existing results in [4]–[6], we find
1
that the contribution of the signal subspace to U s is also of the first
1
order of X . The difference is explained in detail in Remarks 1 and
2 of Section II. Furthermore, we show that the signal subspace has an
impact on the first-order perturbation of the individual singular vectors,
but not on the first-order perturbation of the signal subspace spanned
by these vectors. For this reason, this new result has no material effect
on the performance analysis of subspace-based algorithms that utilize
the signal subspace, e.g., ESPRIT [7], MUSIC [8], the column span
methods [9], as well as the subspace-based algorithms discussed in
[4]–[6]. In other words, the same results will be obtained in the per-
formance analysis of these algorithms no matter whether the contri-
bution of the signal subspace to the first-order approximation of U s 1
is considered or not. However, our result does affect the performance
analysis of methods that depend on the individual singular vectors,
e.g., prewhitening in JADE [11] and the principal component analysis
(PCA) [10].
The rest of the correspondence is organized as follows. The perturba-
tion of the singular vectors of a perturbed matrix is given in Section II.
In Section III, we give a condition under which the contribution of the
1
signal subspace to the first-order approximation of U s can be ignored
in the statistical sense. In Section IV, we apply the perturbation result
to analyze several subspace-based algorithms. Numerical examples are
provided in Section V to validate the theoretic claims.
In the following, upper (lower) boldface letters are used for matrices
(column vectors). A 3 , A T , and AH denote the conjugate, transpose,
and Hermitian transpose of A , respectively. We will use I M for an
identity matrix of size M 2 M , 0M 2N for a zero matrix of size M 2 N ,
kAk for the Frobenius norm of A, and kAk2 for the 2 norm of A.