This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing
Robust Detection in MIMO Radar With Steering
Vector Mismatches
Jun Liu, Senior Member, IEEE, and Jian Li, Fellow, IEEE
Abstract—We address a robust detection problem for MIMO
radars in Gaussian noise with unknown covariance matrix, for
the mismatched case where the nominal transmit (or receive)
steering vector may not be aligned with the true transmit (or
receive) steering vector. Subspace models are adopted for taking
into account these mismatches. More specifically, the transmit
(or receive) steering vector is assumed to belong to a subspace.
Without requiring training data, we design adaptive detectors
according to the criteria of Wald test, Rao test, and generalized
likelihood ratio test (GLRT). It is found that the Rao test does
not exist for the detection problem to be solved. Additionally,
the proposed Wald test and GLRT exhibit constant false alarm
rate properties against the noise covariance matrix. Numerical
examples show that the proposed detectors compared to their
existing counterparts have better robustness against the steering
vector mismatches. Moreover, the proposed Wald test exhibits
better robustness than the proposed GLRT in the region of high
signal-to-noise ratio.
Index Terms—Adaptive detection, MIMO radar, generalized
likelihood ratio test (GLRT), steering vector mismatch, Rao test,
Wald test.
I. I NTRODUCTION
DAPTIVE detection in Gaussian noise with unknown
A covariance matrix has been a topic of long-standing
interest (e.g., see [1]–[6] and the references therein). In
Kelly’s pioneering work [7], a generalized likelihood ratio
test (GLRT) was proposed for the adaptive detection problem
in phased-array radars. The core of the GLRT is to replace
the unknown parameters in the likelihood ratio with their
maximum likelihood estimates (MLEs). To obtain the MLE of
the noise covariance matrix, a standard assumption is that a set
of homogeneous training data is available. It has been proven
in [8] that there is no uniformly most powerful invariant test
for the detection problem in [7]. It means that no optimality is
guaranteed for Kelly’s GLRT. Hence, it is necessary to develop
different adaptive detectors. In [9], Robey et al. proposed an
adaptive matched filter (AMF) detector in an ad hoc way. This
AMF has the same form as that derived from the criterion
of the Wald test [10]. The AMF is computationally more
This work was supported by the National Natural Science Foundation of
China under Grants 61871469 and 61771442, the Youth Innovation Promotion
Association CAS (CX2100060053), the Fundamental Research Funds for the
Central Universities (WK2100000006), the Key Research Program of the
Frontier Sciences, CAS, under Grant QYZDY-SSW-JSC035, and the National
Key Research and Development Program of China (No. 2018YFB1801105).
(Corresponding author: Jian Li)
J. Liu is with the Department of Electronic Engineering and Information
Science, University of Science and Technology of China, Hefei 230027, China
(e-mail:jun liu math@hotmail.com; junliu@ustc.edu.cn).
J. Li is with the Department of Electronic Engineering and Information
Science, University of Science and Technology of China, Hefei 230027, Chi-
na, and the Department of Electrical and Computer Engineering, University
of Florida, Gainesville, FL 32611 USA (e-mail: li@dsp.ufl.edu).
1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
efficient than Kelly’s GLRT, and exhibits better detection
performance than Kelly’s GLRT when the signal-to-noise ratio
(SNR) is high . In [11], De Maio designed an adaptive detector
according to the criterion of the Rao test, and revealed that
the Rao test has strong rejection capabilities of mismatched
signals. Interestingly, Kelly’s GLRT, the AMF, and the Rao
test all bear the constant false alarm rate (CFAR) properties
against the noise covariance matrix.
Note that a set of homogeneous training data is required in
Kelly’s GLRT, the AMF, and the Rao test. However, it may
be difficult to collect sufficient and homogeneous training data
in practice [12], [13], due to environmental non-stationarity
and the presence of interfering targets. The lack and/or het-
erogeneity of training data can result in obvious losses in the
performance of the conventional adaptive detectors, and even
make them unable to work. Therefore, it is highly desirable to
develop adaptive detection schemes without requiring training
data.
Multiple-input multiple-output (MIMO) radar is an attrac-
tive architecture, which can directly use adaptive techniques
without resorting to training data [14], [15]. MIMO radar
falls into two classes: distributed and colocated ones. The
former consists of spatially distributed antennas [16], [17],
exploiting spatial diversity; the latter is composed of closely
spaced antennas [18], [19], [20], taking advantage of waveform
diversity. In this study, our focus is on the colocated MIMO
radar, which is called MIMO radar hereafter for brevity.
In [21], target detection was examined in the MIMO radar
by assuming that the noise covariance matrix is known. A
clairvoyant detector was provided to show that the exploitation
of orthogonal transmit signals leads to gains in the detection
performance. In [22], the method of independent component
analysis was used for target detection under the same assump-
tion of known noise covariance matrix. However, the noise
covariance matrix is usually unknown in practical scenarios.
According to the GLRT principle, an adaptive detector was
designed in [15] for the MIMO radar target detection in
Gaussian noise with unknown covariance matrix. It is worth
noting that this GLRT detector for MIMO radar does not
require training data, and exhibits the CFAR property with
respect to (w.r.t.) the noise covariance matrix. In [23], two
adaptive detectors were proposed according to the criteria of
the Wald and Rao tests. Moreover, analytical performance
of the GLRT, Wald test, and Rao test for MIMO radar was
provided therein. It was found that among these three detectors
for MIMO radar, the Wald test is the most robust against
the steering vector mismatches, and the Rao test exhibits the
strongest selectivity (i.e., rejection capability of mismatched
signals). It has to be pointed out that these three adaptive
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

detectors for MIMO radar cannot change their robustness the mth transmit antenna is denoted by sm (k), k = 1, 2, . . . K,
or selectivity. In order to flexibly tuning between robustness where K is the number of samples of each transmitted signal.
and selectivity, the authors in [24] introduced a parameter to Define
develop a tunable detector which includes the Rao and Wald sk , [s1 (k), s2 (k), . . . , sM (k)]T . (1)
tests as special cases.
Denote by xn (k) the signal received by the nth receive
Note that the steering vector mismatches are not taken into
antenna, and define
account during the design procedures of all the adaptive detec-
tors for MIMO radar mentioned above. Both the transmit and xk , [x1 (k), x2 (k), . . . , xN (k)]T , k = 1, 2, . . . , K. (2)
receive steering vectors for MIMO radar are assumed known in
these adaptive detectors. However, in practice there inevitably Under the point-like target assumption, xk is given by
exist errors in the steering vectors (e.g., due to beam-pointing xk = aar (θ)a†t (θ)sk + vk , k = 1, 2, . . . , K, (3)
errors, array calibration errors, waveform distortions, and array
geometry distortions). For enhanced robustness, the steering where
vector mismatches should be taken into consideration during • a is a deterministic but unknown complex scalar account-
the design procedures of the adaptive detectors. ing for the target reflectivity and the channel propagation
In this paper, we address the MIMO radar target detection effects;
• vk ∈ C
problem in the presence of transmit and receive steering vector N
denotes the disturbance term including jam-
mismatches. Subspace models are adopted to account for these ming, out-of-bin interference and noise;
mismatches. Specifically, the transmit (or receive) steering • at is the transmit steering vector, i.e.,
vector is assumed to belong to a transmit (or receive) subspace.
at (θ) , [exp(ȷ2π f¯0 ξ1 (θ)), . . . , exp(ȷ2π f¯0 ξM (θ))]T ;
This model can tolerate the mismatched signals, and hence
(4)
can improve the robustness. Exploiting the subspace models,
• f¯0 is the carrier frequency;
we propose adaptive detectors according to the criteria of the
• ξm (θ) is the time needed by the signal sent via the mth
Wald test, Rao test, and GLRT. It is shown that the Rao test
transmit antenna to arrive at the target;
does not exist for the detection problem considered. Moreover,
• θ stands for the location parameter of the target (e.g., its
both the Wald test and GLRT bear the CFAR properties
direction-of-arrival (DOA));
against the noise covariance matrix. Simulation results are
• ar is the receive steering vector, i.e.,
provided to demonstrate that compared to the existing adaptive
detectors for MIMO radar, the proposed detectors are more ar (θ) , [exp(ȷ2π f¯0 ξ¯1 (θ)), . . . , exp(ȷ2π f¯0 ξ¯N (θ))]T ;
robust against the steering vector mismatches. In addition, the (5)
robustness of the proposed Wald test is stronger than that of • ξ¯n (θ) represents the time required by the signal reflected
the proposed GLRT in the region of high SNR. by the target located at θ to arrive at the nth receive
The remainder of this paper is organized as follows. In antenna.
Section II, the data model for MIMO radar is introduced, We write the received data in the following matrix form:
and the problem of interest is formulated. We design adaptive
detectors in Section III. In Section IV, the CFAR property X = aar (θ)a†t (θ)S + V, (6)
is proven. Simulation results are provided in Section V and
where
finally the paper is summarized in Section VI.
X = [x1 , x2 , · · · , xK ] ∈ CN ×K , (7)
Notation: Vectors (matrices) are denoted by boldface lower
M ×K
(upper) case letters. Superscripts (·)T , (·)∗ and (·)† denote S = [s1 , s2 , · · · , sK ] ∈ C , (8)
transpose, complex conjugate and conjugate transpose, respec-
tively. The notation ∼ means “is distributed as,” and CN and
denotes a circularly symmetric, complex Gaussian distribution. V = [v1 , v2 , . . . , vK ] ∈ CN ×K . (9)
E(·) is the statistical expectation. In stands for an n×n identity This data model is widely used, and can be found in [15],
matrix, and ⊗ is the Kronecker product. PB = B(B† B)−1 B† [19], [21], [23], [25]–[29].
is the orthogonal projection matrix onto the subspace spanned As in [15], [21], [29], the columns of V are assumed to
by the columns of the matrix B, and P⊥ B = I − PB . C
m×n
be IID, and have a circularly symmetric, complex Gaussian
is a complex matrix space of dimension m × n, and 0m×n distribution with zero mean and unknown covariance matrix
represents a null matrix of dimension m × n. | · | represents R, i.e., vk ∼ CN (0, R).
the modulus of a scalar and the determinant of a matrix, when Our task is to declare the presence or absence of the target
the argument is a scalar and matrix, respectively.
√ tr(·) denotes according to the observed data. With the valid hypothesis (H1 )
the trace of a square matrix, and ȷ = −1. P{·} and λmax {·} be such that the received data in the receivers contain the target
are the principal eigenvector and the corresponding maximum echoes and the alternative hypothesis (H0 ) be such that the
eigenvalue of a square matrix. received data are free of the target echoes, the target detection
problem becomes a binary hypothesis test of H1 against H0 :
II. P ROBLEM F ORMULATION {
Assume that there are M transmit and N receive antennas in H0 : X = V,
(10)
a MIMO radar. The discrete-time baseband signal emitted by H1 : X = aar (θ)a†t (θ)S + V.

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

Note that these data are collected before performing matched where
filtering. As indicated in [23], [29], the matched filtering is B = S† H ∈ CK×q . (15)
implicitly done in adaptive detectors.
For the detection problem in (10), the GLRT, Rao test, Note that the unknown complex amplitude α has been ab-
and Wald test have been designed in [15], [23]. However, no sorbed into the unknown vector a. Here, we use the same
mismatches in the steering vectors ar and at are taken into notation for brevity. As a result, the robust MIMO radar
consideration in [15], [23]. In practical scenarios, there are detection problem can be cast into
{
many factors leading to mismatches in the steering vectors H0 : X = V,
ar and at , such as waveform distortions, array calibration (16)
H1 : X = Aab† B† + V,
errors, and beam-pointing errors [30]–[33]. As shown in [23],
the steering vector mismatches result in obvious losses in the where a and b are unknown, and the noise covariance matrix
detection performance of the GLRT, Rao test, and Wald test. R is unknown. For mathematical tractability, we assume K ≥
In the current study, we take into account the steering vector N + q and N > p + q, which are usually satisfied in practice.
mismatches in the design procedures of adaptive detectors, In the following, we design robust detectors according to the
and we propose robust detectors for the cases where the true criteria of the Wald test, Rao test, and GLRT.
transmit (and/or receive) steering vector may not be aligned
with the nominal transmit (and/or receive) steering vector. A. Wald Test
Let Θ be the parameter vector partitioned as
III. ROBUST D ETECTORS
To obtain robust detectors, we adopt subspace models for Θ = [ΘTr , ΘTs ]T , (17)
the transmit and receive steering vectors to take the mismatch- where
es into consideration. Specifically, we assume Θr = [aT , bT ]T , (18)
ar = Aa, (11) and
N ×p Θs = vec(R). (19)
where A ∈ C is a known full-column-rank matrix, and
a ∈ Cp×1 is an unknown complex-valued coordinator vector. Usually, Θr and Θs are called the relevant and nuisance
Similarly, we assume parameters, respectively. Denote by f1 and f0 the joint prob-
ability density functions (PDFs) of X under H1 and H0 ,
at = Hb, (12)
respectively. The Fisher information matrix F(Θ) associated
where H ∈ CM ×q is a known full-column-rank matrix, and with f1 can be expressed as
b ∈ Cq×1 is an unknown complex-valued coordinator vector. { }
∂ ln f1 ∂ ln f1
Note that the column vectors of A (or H) span the subspace F(Θ) =E
∂Θ∗ ∂ΘT
where the transmit (or receive) steering vector belongs to. [ ] (20)
FΘr ,Θr FΘr ,Θs
Now we provide some remarks on how to choose the , .
FΘs ,Θr FΘs ,Θs
matrices A and H. Denote the true and nominal transmit
steering vectors by at (θ) and at (θ0 ), respectively, and denote The Wald test for the complex-valued signals is given by [37]
the true and nominal receive steering vectors by ar (θ) and {[ ] }−1
ar (θ0 ), respectively. In practice, the true transmit (or receive) tWald = (Θ̂r1 − Θr0 )† F−1 (Θ̂1 ) (Θ̂r1 − Θr0 ),
steering vector is not aligned with the nominal transmit (or Θr ,Θr
(21)
receive) steering vector. Note that the true (or receive) steering
where Θ̂r1 is the MLE of Θr under H1 , Θr0 is the value of Θr
vector is unknown, but the nominal transmit (or receive)
under H0 , and {[F−1 (Θ̂1 )]Θr ,Θr }−1 is the Schur complement
steering vector is known.
of FΘs ,Θs evaluated at Θ̂1 .
We can construct the subspace matrix A (or H) based on
The probability density function (PDF) of X under H1 is
the nominal receive (or transmit) steering vector. Take the
receive subspace matrix A as an example. We can use two 1 { [
f1 (X) = N exp −tr R−1 (X − Aab† B† )
column vectors (i.e., p = 2) to construct A: one is the nominal π |R| K
(22)
]}
receive steering vector ar (θ0 ), and the other is very close to × (X − Aab† B† )† .
the nominal receive steering vector (e.g., ar (θ0 − 0.5◦ )). That
is to say, Taking the logarithm of (22) and performing the derivative
with respect to (w.r.t.) a, a∗ , b, and b∗ yield
A = [ar (θ0 − 0.5◦ ), ar (θ0 )]. (13) { [ ]T
The transmit subspace matrix H can be constructed similarly
∂ lnf1 (X)
∂a = b† B† (X − Aab† B† )† R−1 A ,
(23)
as (13), which is omitted for brevity. This approach of sub-
∂ lnf1 (X)
∂a∗ = A† R−1 (X − Aab† B† )Bb,
space construction can also be found in [30], [34]–[36]. and
According to (11) and (12), the received data X can be { [ ]T
rewritten as
∂ lnf1 (X)
∂b = a† A† R−1 (X − Aab† B† )B ,
(24)
X = Aab† B† + V, (14)
∂ lnf1 (X)
∂b∗ = B† (X − Aab† B† )† R−1 Aa,

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

respectively. Let and

( ) 1
† −2
Xr = X − Aab B ∈ C † † N ×K
. (25) Ã = XP⊥
t X A ∈ CN ×p . (38)

Substituting (23) and (24) into (20) produces Define

[ † −1 X̌ = (XX† )− 2 X ∈ CN ×K ,
1
(39)
A R Xr Bbb† B† X†r R−1 A
FΘr ,Θr = E
B† X†r R−1 Aab† B† X†r R−1 A and
] (26)
A† R−1 Xr Bba† A† R−1 Xr B Ǎ = (XX† )− 2 A ∈ CN ×p .
1
(40)
.
B† X†r R−1 Aaa† A† R−1 Xr B
As derived in Appendix B, g(â1 , b) can be recast into
Let
( )
t = Bb ∈ C K×1
. (27) t† IK − X̌† X̌ t
†
g(â1 , b) = |XX | † ( )
As derived in Appendix A, we have t IK − X̌† X̌ t + t† X̌† PǍ X̌t
{[ ( )] }−1 [ ]−1 (41)
† b† B† X̌† PǍ X̌Bb
F−1 Θ̂1 = FΘr ,Θr . (28) = |XX | 1 + † † .
Θr ,Θr b B (Ik − X̌† X̌)Bb
Inserting (28) and (81) into (21), with the constant scalar The MLE of b under H1 (denoted by b̂1 ) is given by
dropped, we have the Wald test for given R, a, and b, i.e.,
b̂1 = min g(â1 , b)
tWald = b† B† Bba† A† R−1 Aa. (29) b
b† B† X̌† PǍ X̌Bb (42)
Next we obtain the MLEs of R, a, and b under H1 . The = max .
MLE of R under H1 is b b† B† (Ik − X̌† X̌)Bb
1 [( )( )† ]
R̂1 = X − Aab† B† X − Aab† B† . (30) Note that b̂1 is the eigenvector (up to a scalar) corresponding
K to the maximum eigenvalue of a matrix, i.e.,
Substituting (30) into (22) leads to {[ ]−1 † † }
( )( )† −K b̂1 = P B† (IK − X̌† X̌)B B X̌ PǍ X̌B . (43)

f1 (X) = c  X − Aab† B† X − Aab† B†  , (31)
Replacing R in (33) with R̂1 in (30), with the constant
where c is a constant. If â and b̂ can maximize f1 (X), then scalar dropped, leads to
βâ and β1 b̂ can also maximize f1 (X), where β is an arbitrary [ ]−1
nonzero number. Hence, there exists an ambiguity for the tWald = a† A† (X − Aab† B† )(X − Aab† B† )† Aa
MLEs of a and b. To eliminate this ambiguity, we impose † †
[ ⊥ †
]
† −1
= a A XPt X + (Aa − Xt)(Aa − Xt) Aa
a constraint that {( )−1 ( ) −1
b† B† Bb = 1. (32) = a† A† XP⊥ t X
†
− XP⊥ t X
†
(Aa − Xt)
[ ( ) ]−1
Then the Wald test becomes † −1
× 1 + (Aa − Xt)† XP⊥ t X (Aa − Xt)
tWald = a† A† R−1 Aa. (33) ( ) }
† −1
× (Aa − Xt)† XP⊥ t X Aa.
Define (44)
( )( )† 

g(a, b) =  X − Aab† B† X − Aab† B†  As derived in Appendix C, we have
 
 † ( )
= XP⊥ X †
+ (Aa − Xt) (Aa − Xt)  † −1
[
t
] â†1 A† XP⊥
t X (Aâ1 − Xt) = 0. (45)
†( )
† −1
= 1 + (Aa − Xt) XP⊥ t X (Aa − Xt)
  Utilizing (90) and (45), we can obtain
× XP⊥ †
t X , ( )
† −1
(34) tWald = â†1 A† XP⊥ t X Aâ1
† †
( )
where t is defined in (27). Taking the derivative of (34) w.r.t. t X̌ PǍ X̌tt† X̌†
= IN +
a and equating the result to zero, yield the MLE of a under 1 − t† X̌† P⊥Ǎ
X̌t 1 − t† X̌† X̌t
H1 as PǍ X̌t
[ ( ) ]−1 ( ) ×
† −1 † −1 1 − t† X̌† P⊥ X̌t
â1 = A† XP⊥ t X A A† XP⊥ t X Xt ∈ Cp×1 . Ǎ

(35) t† X̌† PǍ X̌t

=( )( )
Inserting (35) into (34), we have 1 − t† X̌† P⊥ X̌t 1 − t† X̌† X̌t
 ( ) Ǎ
g(â1 , b) = XP⊥ X †
1 + t † †
X̃ X̃t − t† †
X̃ P X̃t , (36) b† B† X̌† PǍ X̌Bb
=( )(
t Ã
).
where 1 − b† B† X̌† P⊥ X̌Bb 1 − b† B† X̌† X̌Bb
( ) 1
† −2
Ǎ
X̃ = XP⊥
t X X ∈ CN ×K , (37) (46)

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

Note that the vector b̂1 is the principal eigenvector (up to and it is partitioned as
a scalar) of the matrix (43). To satisfy the constraint in (32), [ ]
X11 X12
we define X̆ = , (57)
X21 X22
b̂1
)1/2 ∈ C .
q×1
b̂ = ( (47)
† † where X11 ∈ Cp×q , X12 ∈ Cp×(K−q) , X21 ∈ C(N −p)×q , and
b̂1 B Bb̂1
X22 ∈ C(N −p)×(K−q) . Define
( )−1
Using b̂ in (47) to take the place of b in (46), we obtain the M , X†21 X22 X†22 X21 ∈ Cq×q , (58)
final Wald test:
( )−1 ( )−1
tWald =b̂†1 B† X̌† PǍ X̌Bb̂1 b̂†1 B† Bb̂1 Q , X11 − X12 X†22 X22 X†22 X21 ∈ Cp×q , (59)
1
× ( )−1 W , X12 P⊥
X†
X†12 ∈ Cp×p (60)
1− b̂†1 B† X̌† P⊥ b̂†1 B† Bb̂1
22

Ǎ
X̌Bb̂1
with ( )−1
1 (48) † †
× ( )−1 P⊥
X † ,IK−q − X22 X 22 X22 X22
(61)
1− b̂†1 B† X̌† X̌Bb̂1 b̂†1 B† Bb̂1
22

∈C(K−q)×(K−q) .
H1
≷ ξWald , Following the lead of [38], the GLRT for the detection
H0 problem in (16) can be given by
where ξWald is a detection threshold. { } H1
−1
tGLRT = λmax W−1 Q (Iq + M) Q† ≷ ξGLRT , (62)
H0
B. Rao Test where ξGLRT is a detection threshold. Note that the GLRT
The Rao test for the complex-valued signals is given as [37] in [38] was designed for adaptive detection of uncertain
waveforms. It turns out that an interesting application of this
tRao
T  GLRT is to solve a robust detection problem for MIMO radar.
∂ lnf1 (X)  [ ] ∂ lnf1 (X) 
−1
= F (Θ̂ ) .
∂Θr Θ=Θ̂0 ∂Θ∗r Θ=Θ̂0
0
Θr ,Θr IV. CFAR P ROPERTY
(49) Here we consider the CFAR property of the proposed Wald
[ ] test. The CFAR of the GLRT detector has been proven in [38].
It can be easily checked that F−1 (Θ̂0 ) does not exist,
Θr ,Θr By observing the form of the Wald test in (48) and the matrix
which means that there is no meaningful Rao test. defined in (43), we know that the CFAR property against the
noise covariance matrix can be proven by showing that two
C. GLRT matrices X̌† X̌ and X̌† PǍ X̌ are independent of R.
Using (39), we have
The GLRT detector can be obtained as follows:
max{a,b, R} f1 (X) H1 X̌† X̌ = X† (XX† )−1 X. (63)
≷ ξ0 , (50)
max{R} f0 (X) H0 Define
X̂ = R−1/2 X ∈ CN ×K , (64)
where ξ0 is a detection threshold. Define
and then
A|| , A(A† A)−1/2 ∈ CN ×p , (51)
X̌† X̌ = X̂† (X̂X̂† )−1 X̂. (65)
and we have
Hence,
A†|| A|| = Ip . (52) X̂ ∼ CN (0, IN ⊗ IK ) . (66)
Then, there exists a unitary matrix UA such that †
It means that the matrix X̌ X̌ is independent of the noise
† N ×N covariance matrix R.
UA , [A|| , A⊥ ] ∈ C , (53)
Define
where A⊥ ∈ CN ×(N −p) . Define  = R−1/2 A ∈ CN ×p , (67)
B|| , B(B† B)−1/2 ∈ CK×q . (54) and then, we have
[ ]−1
Similar to (53), there exists a unitary matrix UB such that X̌† PǍ X̌ =X̂† (X̂X̂† )−1  † (X̂X̂† )−1 Â
(68)
UB , [B|| , B⊥ ] ∈ CK×K , (55) × Â† (X̂X̂† )−1 X̂.
where B⊥ ∈ CK×(K−q) . Define The QR decomposition of  can be expressed by
X̆ , UA XU†B ∈ CN ×K , (56) Â = A1 A2 , (69)

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

(a) K = 32 (a) K = 32
1 1
Proposed GLRT
Proposed GLRT
Detection Probability

Detection Probability
0.8 0.8 Proposed Wald
Proposed Wald
GLRT
GLRT
0.6 0.6 Wald
Wald
Rao
Rao
0.4 0.4

0.2 0.2

0 0
−5 0 5 10 15 20 25 30 35 −5 0 5 10 15 20 25 30 35 40 45
SNR (dB) SNR (dB)
(b) K = 40 (b) K = 40
1 1
Proposed GLRT
Proposed GLRT
Detection Probability

Detection Probability
0.8 0.8 Proposed Wald
Proposed Wald
GLRT
GLRT
0.6 0.6 Wald
Wald
Rao
Rao
0.4 0.4

0.2 0.2

0 0
−5 0 5 10 15 20 25 30 35 −5 0 5 10 15 20 25 30 35 40 45
SNR (dB) SNR (dB)

Fig. 1. Detection probability versus SNR for cos2 ϕ = 0.5099 in the Fig. 2. Detection probability versus SNR for cos2 ϕ = 0.7448 in the
mismatched case. mismatched case.

(a) K = 32
where A1 ∈ CN ×p is a slice of an N × N unitary matrix
1

U, and A2 ∈ Cp×p is an upper triangular matrix of full-rank.

Detection Probability

0.8
Note that 0.6
U† A1 = EA , (70) Proposed GLRT
0.4 Proposed Wald
GLRT
where 0.2 Wald
EA = [Ip , 0p×(N −p) ]T ∈ CN ×p . (71) Rao
0
−5 0 5 10 15 20 25 30 35 40 45
Inserting (69) and (70) into (68) yields SNR (dB)
[ ]−1 (b) K = 40
X̌† PǍ X̌ =X̄† (X̄X̄† )−1 EA E†A (X̄X̄† )−1 EA
1

(72)
Detection Probability

0.8
× E†A (X̄X̄† )−1 X̄,
0.6
Proposed GLRT
where Proposed Wald
0.4
†
X̄ = U X̂ ∼ N (0, IN ⊗ IK ) . (73) GLRT
0.2 Wald

It follows from (72) that the matrix X̌† PǍ X̌ is independent Rao
0
of R. Therefore, the proposed Wald test exhibits the CFAR −5 0 5 10 15 20 25 30 35 40 45
property against the noise covariance matrix R. SNR (dB)

Fig. 3. Detection probability versus SNR in the matched case.

V. S IMULATION R ESULTS
In this section, numerical examples demonstrate the perfor-
mismatch degree between the true receive steering vector
mance of the proposed detectors. Assume that the MIMO radar
ar (θ) and the assumed one ar (θ0 ) is defined by
receiver (or transmitter) uses a uniformly spaced linear array
with a half-wavelength spacing. The numbers of receive and |a†r (θ)R−1 ar (θ0 )|2
transmit antennas are 16 and 12 (i.e., N = 16 and M = 12), cos2 ϕ = . (74)
respectively. The (i, j)th element of the noise covariance a†r (θ)R−1 ar (θ)a†r (θ0 )R−1 ar (θ0 )
matrix is chosen to be [R]i,j = σ 2 0.9|i−j| , where σ 2 is the We select p = q = 2, and the subspace matrix A for the
noise power. Unless specifically stated, the probability of false receive steering vector is chosen as
alarm is set to be 10−3 , and the numbers of independent trials
used for simulating the detection threshold and the probability A = [ar (θ0 − 0.5◦ ), ar (θ0 )]. (75)
of detection are 100/PFA and 104 , respectively.
Assume that the true direction of arrival (DOA) of a target The subspace matrix H for the transmit steering vector is
is θ = 40◦ . The nominal DOA is denoted by θ0 , and the chosen in the similar way as that in (75). The SNR in decibel

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

(a) SNR = 16 dB
1 (a)
Proposed GLRT 1
Detection Probability

0.8 Proposed Wald Proposed Wald

GLRT 0.8
0.6

0.9
Wald

0.7
0.5
Rao

0.3
0.4 0.6

cos2φ

0.1
0.2 0.4

0 0.9
0 0.2 0.4 0.6 0.8 1 0.2 0.
9 0.7
0 0.5

0.
cos2φ 0 .5

7
0.1.3 0.3
(b) SNR = 22 dB 0 0.1
1 0 5 10 15 20 25 30 35 40
SNR (dB)
Detection Probability

0.8 Proposed GLRT (b)

1
Proposed Wald
Proposed GLRT

0.1
0.6 GLRT
Wald 0.8

0.9
0.4
Rao

0.7
0.5
0.6
0.2

0.3
cos2φ

9
0.
05.7
0 0.4 03.
0.

0.1
0 0.2 0.4 0.6 0.8 1

1
0.
cos2φ 0.9
0.2 0.7
0.5
0.3
Fig. 4. Detection probability versus cos2 ϕ for K = 32. 0.1
0
0 5 10 15 20 25 30 35 40
SNR (dB)
(c)
is defined by 1 0.9
[ ] 05.7
0.
0.3 0.1
SNR = 10 log10 |α|2 a†r (θ)R−1 ar (θ)a†t (θ)SS† at (θ) . (76) 0.9

0.1
0.8
0.7
0.5
0.3
For comparison purposes, we consider three classical detectors 0.6
cos2φ

0.1
in MIMO radars: the GLRT [15], Rao test [23], and Wald test
0.4
[23].
In Fig. 1, we plot the detection probability curves versus 0.2
SNR in the mismatched case. In this case, the nominal DOA GLRT

is 44.5◦ and cos2 ϕ = 0.5099. As we can see, the proposed 0

0 5 10 15 20 25 30 35 40
GLRT and Wald test outperform their counterparts in this SNR (dB)
mismatched case. The phenomenon demonstrates that the (d)
1
proposed detectors compared to their existing counterparts
0.3

0.9
0.1

possess stronger robustness against the mismatches in the 0.8 0.7

0.5
0.

steering vectors. Moreover, the robustness of the proposed 0.7

9

0.6 0.5
GLRT test is slightly better than that of the proposed Wald 0.1
cos φ
2

0.3
test, when K = 32 and SNR is not too high. With the 0.1
0.4
increase of K, the two proposed detectors show almost the
same performance. 0.2
Fig. 2 takes into account the less serious mismatch of Wald

steering vectors than that in Fig. 1. Here, we select θ0 = 43◦ 0

0 5 10 15 20 25 30 35 40
and cos2 ϕ = 0.7448. The other parameter settings are the SNR (dB)
(e)
same as those in Fig. 1. We can see that the proposed detectors 1 0.9
0.9 0.7
0.5
are more robust than the GLRT, Rao test and Wald test in this 0.3 0.1
0.1

0.7
0.5
mismatched case, which is consistent with the phenomenon 0.8
0.3
1
0.

in Fig. 1. In addition, comparisons between Fig. 1 and Fig.

0.6
2 show that the performance gains of the proposed detectors
cos2φ

over their existing counterparts decrease with the increase in 0.4

cos2 ϕ. Interestingly, the detection probabilities of the existing
detectors are not monotonic functions of the SNR. This is 0.2
Rao
expected, because the conventional detectors do not consider
0
the signal leakage in the mismatched case. 0 5 10 15 20 25 30 35 40
Fig. 3 shows the detection probability curves versus SNR SNR (dB)

in the matched case where no error exists in the steering

Fig. 5. Contour of constant detection probability in the mismatched case for
vectors. It can be observed that the detection performance of K = 32.
the proposed detectors are worse than that of the conventional

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

detectors. This is because that the noise energy is accumulated We proceed to partition Xr as [xr1 , xr2 , . . . , xrK ]. Then, we
due to using the subspace models for the steering vectors. have
In Fig. 4, the detection probability as a function of cos2 ϕ F(Θr12 ) = F(Θr21 )†
is shown for the cases of SNR = 16 dB and SNR = 22 dB. { }
∑
K
[ T ∗ T ∗ ]
As we can see, the proposed detectors are more robust to the † −1 T ∗
=A R E ti xri xr1 θ , xr2 θ , . . . , xrK θ B
mismatched signals when the mismatch is serious. i=1
Fig. 5 shows the contour of constant detection probability = 0p×q ,
for the case of K = 32. This type of figure is first introduced in (79)
[39]. We can observe that the proposed detectors exhibit better
where ti is the i-th element of the vector t. In addition,
robustness than the conventional detectors. In addition, the  †  
proposed Wald test show better robustness to the mismatched 
 x θ 

 r1  † 
signals than the proposed GLRT in the high SNR region. x†r2 θ
F(Θr22 ) = B† E   ...
 [θ xr1 , θ † xr2 , . . . , θ † xrK ] B


 

 
x†rK θ
VI. C ONCLUSIONS
= B† θ † RθIK B
In this paper, we have considered the MIMO radar tar- = a† A† R−1 AaB† B ∈ Cq×q .
get detection problem in the presence of Gaussian noise (80)
with unknown covariance matrix. We have focused on the
Inserting (78), (79) and (80) into (77) leads to
mismatched steering vector cases where errors exist in the [ † † ]
transmit and receive steering vectors. To account for the b B BbA† R−1 A 0p×q
FΘr ,Θr = .
mismatches, we have adopted the subspace models where 0q×p a† A† R−1 AaB† B
the transmit and receive steering vectors are assumed to lie (81)
within two subspaces. Without the need for training data, Taking the derivatives of (23) and (24) with respect to
adaptive detectors were designed according to the criteria of vecT (R∗ ) and then performing the expectation operation yield
the Wald test, Rao test, and the GLRT. Interestingly, the Rao FΘr ,Θs as a null matrix. According to the following equation
test does not exist for the detection problem considered. The [F−1 (Θ)]Θr ,Θr
proposed Wald test and GLRT exhibit the CFAR properties (82)
against the unknown noise covariance matrix. Simulation = [FΘr ,Θr − FΘr ,Θs F−1
Θs ,Θs FΘs ,Θr ]
−1
,
results demonstrate that the proposed detectors, compared to we can derive (28).
their conventional counterparts, have better robustness to the
steering vector mismatches. In addition, the proposed Wald A PPENDIX B
test is more robust than the proposed GLRT detector in the D ERIVATIONS OF (41)
region of high SNR. It follows from (32) that
Note that the disturbance is assumed to have Gaussian    
XP⊥ †  †
t X = XX − Xtt X
† †
distribution. In practice, it may be non-Gaussian. The detection [ ]
problem in the non-Gaussian background is left for future = |XX† | 1 − t† X† (XX† )−1 Xt (83)
[ ]
work. = |XX† | 1 − t† X̌† X̌t ,
where X̌ is defined in (39). It is easy to check that
A PPENDIX A ( )
† −1
( )−1
XP⊥ t X = XX† − Xtt† X†
D ERIVATIONS OF (28)
(XX† )−1 Xtt† X† (XX† )−1
= (XX† )−1 + ,
Using (27), we can rewrite (26) as 1 − t† X̌† X̌t
(84)
FΘr ,Θr
[ † −1 ] and
A R Xr tt† X†r R−1 A A† R−1 Xr tθ † Xr B ( )
† −1
=E t† X̃† X̃t = t† X† XP⊥
t X Xt
B† X†r θt† X†r R−1 A B† X†r θθ † Xr B
[ ] t† X̌† X̌t (85)
F(Θr11 ) F(Θr12 ) = .
=
F(Θr21 ) F(Θr22 )
, 1 − t† X̌† X̌t
(77) Then, we have
t† X̃† PÃ X̃t
where θ = R−1 Aa ∈ CN ×1 . According to [40], the random ( ) [ ]−1
X̌tt† X̌† ( )
⊥ † −1
vector z = Xr Bb ∈ CN ×1 has the covariance matrix †
= t X̌ IN + Ǎ A †
XPt X A
b† B† BbR under hypothesis H1 . Hence, F(Θr11 ) can be 1 − t† X̌† X̌t
( )
recast as † X̌tt† X̌†
× Ǎ IN + X̌t,
1 − t† X̌† X̌t
F(Θr11 ) = b† B† BbA† R−1 A ∈ Cp×p . (78) (86)

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

where Ǎ is defined in (40). Using (84), we obtain [7] E. J. Kelly, “An adaptive detection algorithm,” IEEE Transactions on
[ ]−1 † Aerospace and Electronic Systems, vol. 22, no. 1, pp. 115–127, March
Ǎ A† (XP⊥ † −1
t X ) A Ǎ 1986.
[ † † †
]−1 [8] S. Bose and A. O. Steinhardt, “A maximal invariant framework for
† Ǎ X̌tt X̌ Ǎ adaptive detection with structured and unstructured covariance matrices,”
= Ǎ Ǎ Ǎ + Ǎ† (87) IEEE Transactions on Signal Processing, vol. 43, no. 9, pp. 2164–2175,
1 − t† X̌† X̌t September 1995.
PǍ X̌tt† X̌† PǍ [9] F. C. Robey, D. R. Fuhrmann, E. J. Kelly, and R. Nitzberg, “A CFAR
= PǍ − . adaptive matched filter detector,” IEEE Transactions on Aerospace and
(1 − t† X̌† P⊥
Ǎ
X̌t) Electronic Systems, vol. 28, no. 1, pp. 208–216, January 1992.
[10] A. De Maio, “A new derivation of the adaptive matched filter,” IEEE
Plugging (87) into (86), we can write t† X̃† PÃ X̃t as (88). Signal Processing Letters, vol. 11, no. 10, pp. 792–793, October 2004.
Substituting (83), (85), and (88) into (36) results in [11] ——, “Rao test for adaptive detection in Gaussian interference with
unknown covariance matrix,” IEEE Transactions on Signal Processing,
1 − t† X̌† X̌t vol. 55, no. 7, pp. 3577–3584, July 2007.
g(â1 , b) = |XX† | . (89) [12] Y. Gao, H. Li, and B. Himed, “Knowledge-aided range-spread target de-
1 − t† X̌† X̌t + t† X̌† PǍ t tection for distributed MIMO radar in nonhomogeneous environments,”
( ) IEEE Transactions on Signal Processing, vol. 65, no. 3, pp. 617–627,
Applying the equation 1 − t† X̌† X̌t = t† IK − X̌† X̌ t, we February 2017.
can write (89) as (41). [13] A. De Maio, D. Orlando, C. Hao, and G. Foglia, “Adaptive detection of
point-like targets in spectrally symmetric interference,” IEEE Transac-
tions on Signal Processing, vol. 64, no. 12, pp. 3207–3220, December
A PPENDIX C 2016.
D ERIVATIONS OF (45) [14] J. Li and P. Stoica, MIMO Radar Signal Processing. Hoboken, NJ:
John Wiley & Sons, 2009.
Utilizing (35), we can write Aâ1 as (90). Then, we have [15] L. Xu, J. Li, and P. Stoica, “Target detection and parameter estimation for
( )
† −1 MIMO radar systems,” IEEE Transactions on Aerospace and Electronic
â†1 A† XP⊥ t X (Aâ1 − Xt) Systems, vol. 44, no. 3, pp. 927–939, July 2008.
† †
( )
t X̌ PǍ X̌tt† X̌† [16] A. M. Haimovich, R. S. Blum, and L. J. Cimini, “MIMO radar with
widely separated antennas,” IEEE Signal Processing Magazine, vol. 25,
= IN +
1 − t† X̌† P⊥ X̌t 1 − t† X̌† X̌t no. 1, pp. 116–129, January 2008.
( Ǎ ) [17] Q. He and R. S. Blum, “Diversity gain for MIMO Neyman-Pearson
PǍ X̌t signal detection,” IEEE Transactions on Signal Processing, vol. 59,
× − X̌t no. 3, pp. 869–881, March 2011.
1 − t† X̌† P⊥ X̌t (91) [18] J. Li and P. Stoica, “MIMO radar with colocated antennas,” IEEE Signal
Ǎ
( † †
) † †
† † IN 1 − t X̌ X̌t + X̌tt X̌ Processing Magazine, vol. 24, no. 5, pp. 106–114, September 2007.
t X̌ PǍ
= † † ⊥
[19] H. Li and B. Himed, “Transmit subaperturing for MIMO radars with co-
1 − t X̌ PǍ X̌t 1 − t† X̌† X̌t located antennas,” IEEE Journal of Selected Topics in Signal Processing,
( ) vol. 4, no. 1, pp. 55–65, February 2010.
PǍ X̌t − X̌t 1 − t† X̌† P⊥ X̌t [20] G. Cui, X. Yu, V. Carotenuto, and L. Kong, “Space-time transmit code
× Ǎ
.
1 − t† X̌† P⊥ Ǎ
X̌t and receive filter design for colocated MIMO radar,” IEEE Transactions
on Signal Processing, vol. 65, no. 5, pp. 1116– 1129, March 2017.
The numerator of (91) can be simplified as [21] I. Bekkerman and J. Tabrikian, “Target detection and localization using
[ ( ) ] MIMO radars and sonars,” IEEE Transactions on Signal Processing,
t† X̌† PǍ IN 1 − t† X̌† X̌t + X̌tt† X̌† vol. 54, no. 10, pp. 3873–3883, October 2006.
( ) [22] M. Hatam, A. Sheikhi, and M. A. Masnadi-Shirazi, “Target detection
× X̌tt† X̌† − IN P⊥ X̌t in pulse-train MIMO radars applying ICA algorithms,” Progress In
( † † ) † † Ǎ (92) Electromagnetics Research, vol. 122, pp. 413–435, 2012.
= t X̌ X̌t − 1 t X̌ PǍ IN P⊥ Ǎ
X̌t [23] W. Liu, Y. Wang, J. Liu, and W. Xie, “Adaptive detection without train-
= 0. ing data in colocated MIMO radar,” IEEE Transactions on Aerospace
and Electronic Systems, vol. 51, no. 3, pp. 2469–2479, July 2015.
Hence, (45) is obtained. [24] J. Liu, S. Zhou, W. Liu, J. Zheng, H. Liu, and J. Li, “Tunable adaptive
detection in colocated MIMO radar,” IEEE Transactions on Signal
Processing, vol. 66, no. 4, pp. 1080–1092, February 2018.
R EFERENCES [25] G. Cui, H. Li, and M. Rangaswamy, “MIMO radar waveform design
with constant modulus and similarity constraints,” IEEE Transactions
[1] F. Gini, A. Farina, and M. Greco, “Selected list of references on radar
on Signal Processing, vol. 62, no. 2, pp. 343–353, January 2014.
signal processing,” IEEE Transactions on Aerospace and Electronic
Systems, vol. 37, no. 1, pp. 329–359, January 2001. [26] B. Tang and J. Tang, “Joint design of transmit waveforms and receive
[2] W. Liu, W. Xie, J. Liu, and Y. Wang, “Adaptive double subspace signal filters for MIMO radar space-time adaptive processing,” IEEE Transac-
detection in Gaussian background–Part I: Homogeneous environments,” tions on Signal Processing, vol. 64, no. 18, pp. 4707– 4722, September
IEEE Transactions on Signal Processing, vol. 62, no. 9, pp. 2345–2357, 2016.
May 2014. [27] L. Xu and J. Li, “Iterative generalized-likelihood ratio test for MIMO
[3] K. J. Sohn, H. Li, and B. Himed, “Parametric Rao test for multichannel radar,” IEEE Transactions on Signal Processing, vol. 55, no. 6, pp. 2375–
adaptive signal detection,” IEEE Transactions on Aerospace and Elec- 2385, June 2007.
tronic Systems, vol. 43, no. 3, pp. 920–933, July 2007. [28] J. Li, P. Stoica, L. Xu, and W. Roberts, “On parameter identifiability
[4] P. Wang, H. Li, and B. Himed, “A new parametric GLRT for multichan- of MIMO radar,” IEEE Signal Processing Letters, vol. 14, no. 12, pp.
nel adaptive signal detection,” IEEE Transactions on Signal Processing, 968–971, December 2007.
vol. 58, no. 1, pp. 317–325, January 2010. [29] J. Li, L. Xu, P. Stoica, K. W. Forsythe, and D. W. Bliss, “Range
[5] V. Carotenuto, A. De Maio, D. Orlando, and L. Pallotta, “Adaptive radar compression and waveform optimization for MIMO radar: A Cramer-
detection using two sets of training data,” IEEE Transactions on Signal Rao bound based study,” IEEE Transactions on Signal Processing,
Processing, vol. 66, no. 7, pp. 1791–1801, April 1 2018. vol. 56, no. 1, pp. 218–232, January 2008.
[6] C. Hao, D. Orlando, G. Foglia, and G. Giunta, “Knowledge-based [30] F. Bandiera, D. Orlando, and G. Ricci, Advanced radar detection
adaptive detection: Joint exploitation of clutter and system symmetry schemes under mismatched signal models. Morgan & Claypool, 2009.
properties,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1489– [31] J. Li and P. Stoica, Robust Adaptive Beamforming. John Wiley & Sons,
1493, October 2016. Inc., 2005.

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

10

t† X̃† PÃ X̃t

( )( )( )
† † X̌tt† X̌† PǍ X̌tt† X̌† PǍ X̌tt† X̌†
= t X̌ IN + PǍ − IN + X̌t
1 − t† X̌† X̌t 1 − t† X̌† P⊥ X̌t 1 − t† X̌† X̌t
( ) ( † † ⊥
Ǎ
) † †
† † † † P
† † IN 1−t X̌ X̌t + X̌tt X̌ Ǎ 1−t X̌ PǍ X̌t −PǍ X̌tt X̌ PǍ
= t X̌
1 − t† X̌† X̌t 1 − t† X̌† P⊥ X̌t
( † †
) † †
Ǎ
IN 1−t X̌ X̌t + X̌tt X̌
× X̌t
1 − t† X̌† X̌t
t† X̌† −t† X̌† t† X̌† X̌t+t† X̌† X̌tt† X̌† PǍ −PǍ t† X̌† P⊥ Ǎ
X̌t−PǍ X̌tt† X̌† PǍ
=
1 − t† X̌† X̌t 1 − t† X̌† P⊥ Ǎ
X̌t
X̌t−t† X̌† X̌tX̌t+ X̌tt† X̌† X̌t
×
( 1 − t† X̌† X̌t )
t X̌ PǍ − PǍ t† X̌† X̌t + PǍ t† X̌† PǍ X̌t − PǍ X̌tt† X̌† PǍ X̌t
† † (88)
= ( ) ( ) ( )
1 − t† X̌† X̌t 1 − t† X̌† P⊥Ǎ
X̌t 1 − t† X̌† X̌t
t† X̌† PǍ X̌t − t† X̌† PǍ t† X̌† X̌tX̌t + t† X̌† PǍ t† X̌† PǍ X̌tX̌t − t† X̌† PǍ X̌tt† X̌† PǍ X̌t
= ( )( )( )
1 − t† X̌† X̌t 1 − t† X̌† P⊥ Ǎ
X̌t 1 − t† X̌† X̌t
t† X̌† PǍ X̌t − t† X̌† PǍ X̌tt† X̌† X̌t + t† X̌† PǍ X̌tt† X̌† PǍ X̌t − t† X̌† PǍ X̌tt† X̌† PǍ X̌t
= ( )( )( )
1 − t† X̌† X̌t 1 − t† X̌† P⊥ Ǎ
X̌t 1 − t† X̌† X̌t
( )
t† X̌† PǍ X̌t 1 − t† X̌† X̌t
=( )( )( )
1 − t† X̌† X̌t 1 − t† X̌† P⊥ Ǎ
X̌t 1 − t† X̌† X̌t
t† X̌† PǍ X̌t
=( )( ).
1 − t† X̌† X̌t 1 − t† X̌† P⊥
Ǎ
X̌t

[ ( ) ]−1 ( )
† −1 † −1
Aâ1 = A A† XP⊥ t X A A† XP⊥ t X Xt
[ ] ( )
† 21 †
( )
⊥ † −1
−1
† X̌tt† X̌†
= (XX ) Ǎ A XPt X A Ǎ IN + X̌t
1 − t† X̌† X̌t
( )( )
P X̌tt† X̌† P X̌tt† X̌†
= (XX† ) 2 PǍ − Ǎ † † ⊥ Ǎ
1
IN + X̌t
1 − t X̌ PǍ X̌t 1 − t† X̌† X̌t
( )
† 21
PǍ − PǍ t† X̌† P⊥Ǎ
X̌t − PǍ X̌tt† X̌† PǍ IN 1 − t† X̌† X̌t + X̌tt† X̌†
= (XX ) X̌t
1 − t† X̌† P⊥Ǎ
X̌t 1 − t† X̌† X̌t
PǍ − PǍ t† X̌† X̌t + PǍ t† X̌† PǍ X̌t − PǍ X̌tt† X̌† PǍ X̌t
= (XX† ) 2
1

† † ⊥
1 − t X̌ PǍ X̌t 1 − t† X̌† X̌t (90)
PǍ X̌t − PǍ t† X̌† X̌tX̌t + PǍ t† X̌† PǍ X̌tX̌t − PǍ X̌tt† X̌† PǍ X̌t
= (XX† )
1
2 ( )( )
1 − t† X̌† P⊥
Ǎ
X̌t 1 − t† X̌† X̌t
PǍ X̌t − PǍ X̌tt† X̌† X̌t + PǍ X̌tt† X̌† PǍ X̌t − PǍ X̌tt† X̌† PǍ X̌t
= (XX† ) 2
1
( )( )
1 − t† X̌† P⊥
Ǎ
X̌t 1 − t† X̌† X̌t
( )
PǍ X̌t 1 − t† X̌† X̌t
= (XX† ) 2 (
1
)( )
1 − t† X̌† P⊥
Ǎ
X̌t 1 − t† X̌† X̌t
PǍ X̌t
= (XX† ) 2
1
.
1 − t† X̌† P⊥
Ǎ
X̌t

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2019.2939078, IEEE
Transactions on Signal Processing

11

[32] Y. Gu, N. A. Goodman, S. Hong, and Y. Li, “Robust adaptive beam-
forming based on interference covariance matrix sparse reconstruction,”
Signal Processing, vol. 96, 2014.
[33] Y. Gu and A. Leshem, “Robust adaptive beamforming based on interfer-
ence covariance matrix reconstruction and steering vector estimation,”
IEEE Transactions on Signal Processing, vol. 60, no. 7, pp. 3881–3885,
July 2012.
[34] O. Besson, L. L. Scharf, and F. Vincent, “Matched direction detectors
and estimators for array processing with subspace steering vector
uncertainties,” IEEE Transactions on Signal Processing, vol. 53, no. 12,
pp. 4453–4463, December 2005.
[35] J. Carretero-Moya, A. De Maio, J. Gismero-Menoyo, and A. Asensio-
Lopez, “Experimental performance analysis of distributed target coher-
ent radar detectors,” IEEE Transactions on Aerospace and Electronic
Systems, vol. 48, no. 3, pp. 2216–2238, July 2012.
[36] F. Gini and A. Farina, “Matched subspace CFAR detection of hovering
helicopters,” IEEE Transactions on Aerospace and Electronic Systems,
vol. 35, no. 4, pp. 1293–1305, October 1999.
[37] W. Liu, Y. Wang, and W. Xie, “Fisher information matrix, Rao test,
and Wald test for complex-valued signals and their applications,” Signal
Processing, vol. 94, no. 1, pp. 1–5, January 2014.
[38] S. Bose and A. O. Steinhardt, “Adaptive array detection of uncertain
rank one waveforms,” IEEE Transactions on Signal Processing, vol. 44,
no. 11, pp. 2801–2809, November 1996.
[39] N. B. Pulsone and C. M. Rader, “Adaptive beamformer orthogonal
rejection test,” IEEE Transactions on Signal Processing, vol. 49, no. 3,
pp. 521–529, March 2001.
[40] W. Liu, W. Xie, J. Liu, D. Zou, H. Wang, and Y. Wang, “Detection
of a distributed target with direction uncertainty,” IET Radar, Sonar &
Navigation, vol. 8, no. 9, pp. 1177–1183, 2014.
Jian Li (S’87-M’91-SM’97-F’05) received the
M.Sc. and Ph.D. degrees in electrical engineering
from The Ohio State University, Columbus, in 1987
and 1991, respectively. She is currently a Profes-
PLACE sor in the Department of Electrical and Computer
PHOTO Engineering, University of Florida, Gainesville. Her
HERE current research interests include spectral estimation,
statistical and array signal processing, and their
applications to radar, sonar, and biomedical engi-
neering. Dr. Li’s publications include Robust Adap-
tive Beamforming (2005, Wiley), Spectral Analysis:
the Missing Data Case (2005, Morgan & Claypool), MIMO Radar Signal
Processing (2009, Wiley), and Waveform Design for Active Sensing Systems
– A Computational Approach (2011, Cambridge University Press).
Dr. Li is a Fellow of IEEE and a Fellow of IET. She is also a Fellow
of the European Academy of Sciences (Brussels). She received the 1994
National Science Foundation Young Investigator Award and the 1996 Office
of Naval Research Young Investigator Award. She was Executive Committee
Members of the 2002 and 2016 International Conferences on Acoustics,
Speech, and Signal Processing, in Orlando, Florida, May 2002, and in
Shanghai, China, March 2016, respectively. She was an Associate Editor of
the IEEE Transactions on Signal Processing from 1999 to 2005, an Associate
Editor of the IEEE Signal Processing Magazine from 2003 to 2005, and a
member of the Editorial Board of Signal Processing, a publication of the
European Association for Signal Processing (EURASIP), from 2005 to 2007.
She was a member of the Editorial Board of the IEEE Signal Processing
Magazine from 2010 to 2012. She is currently a member of the Sensor
Array and Multichannel Technical Committee of the IEEE Signal Processing
Society. She is a co-author of the paper that has received the M. Barry Carlton
Award for the best paper published in IEEE Transactions on Aerospace and
Electronic Systems in 2005. She is also a co-author of a paper published
in IEEE Transactions on Signal processing that has received the Best Paper
Award in 2013 from the IEEE Signal Processing Society.

Jun Liu (S’11-M’13-SM’16) received the B.S. de-

gree in mathematics from Wuhan University of
Technology, Wuhan, China, in 2006, the M.S. degree
in mathematics from Chinese Academy of Sciences,
PLACE China, in 2009, and the Ph.D. degree in electrical
PHOTO engineering from Xidian University, Xi’an, China,
HERE in 2012.
From July 2012 to December 2012, he was a Post-
doctoral Research Associate with the Department
of Electrical and Computer Engineering, Duke U-
niversity, Durham, NC, USA. From January 2013
to September 2014, he was a Postdoctoral Research Associate with the
Department of Electrical and Computer Engineering, Stevens Institute of
Technology, Hoboken, NJ, USA. From October 2014 to March 2018, he was
with Xidian University, Xi’an, China. He is currently an Associate Professor
with the Department of Electronic Engineering and Information Science,
University of Science and Technology of China, Hefei, China. His research
interests include statistical signal processing, optimization algorithms, and
machine learning. He is currently an Associate Editor for the IEEE SIGNAL
PROCESSING LETTERS.

1053-587X (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.