Hybrid Beamformer Codebook Design and Ordering
for Compressive mmWave Channel Estimation
Junmo Sung and Brian L. Evans
Wireless Networking and Communications Group, The University of Texas at Austin Austin, TX USA
junmo.sung@utexas.edu, bevans@ece.utexas.edu
Abstract—In millimeter wave (mmWave) communication sys-
tems, beamforming with large antenna arrays is critical to
overcome high path losses. Separating all-digital beamforming
into analog and digital stages can provide the large reduction
arXiv:1909.09861v1 [cs.IT] 21 Sep 2019
in power consumption and small loss in spectral efficiency
needed for practical implementations. Developing algorithms
with this favorable tradeoff is challenging due to the additional
degrees of freedom in the analog stage and its accompanying
hardware constraints. In hybrid beamforming systems, for ex-
ample, channel estimation algorithms do not directly observe
the channels, face a high channel count, and operate at low
SNR before transmit-receive beam alignment. Since mmWave
channels are sparse in time and beam domains, many compressed
sensing (CS) channel estimation algorithms have been devel-
oped that randomly configure the analog beamformers, digital
beamformers, and/or pilot symbols. In this paper, we propose to
design deterministic beamformers and pilot symbols for open-
loop channel estimation. We use CS approaches that rely on
low coherence for their recovery guarantees, and hence seek to
minimize the mutual coherence of the compressed sensing matrix.
We also propose a precoder column ordering to design the pilot
symbols. Simulation results show that our beamformer designs
reduce channel estimation error over competing methods.
Index Terms—millimeter wave, hybrid beamforming, channel
estimation, compressed sensing, codebook design
I. I NTRODUCTION
Hybrid analog and digital beamforming architectures in
millimeter wave (mmWave) communication systems have
drawn a great amount of attention for multiple reasons. They
can practically maintain achievable spectral efficiency as with
all-digital MIMO architectures due to the sparse nature of
mmWave channels [1], [2]. Reducing the number of RF chains
leads to almost proportional power consumption reduction.
Hybrid beamforming architectures, however, demand even
more complicated signal processing [3] because an analog
stage has fewer degrees of freedom compared with an all-
digital MIMO architecture; e.g., phase shifters are constrained
to have a discrete phase on the unit circle. Channel estimation
is not an exception in this signal processing complication.
MmWave channel measurement campaigns have revealed
that the channels are sparse in both time and angular di-
mensions [4], [5]. The sparsity finds compressed sensing
(CS) algorithms suitable for mmWave channel estimation.
For phase shifter based hybrid beamforming architecture,
the adaptive CS was proposed in [6] to narrow beams by
iteratively adapting precoders and combiners. Another popular
approach found in many publications is to formulate a sparse
channel estimation problem and apply well-known or modified
CS algorithms [7]–[11]. For example, orthogonal matching
pursuit (OMP) or its variants are used in [7], [8], and [9]
compares different channel estimation algorithms that include
OMP as a representative of CS algorithms. The adaptive
dictionary generation algorithm proposed in [12] is useful to
deal with common issues regarding the dimension deficiency.
The random configuration for the phase shifters has been
widely used in those literatures since the generated sensing
matrices are incoherent and satisfy the restricted isometry
property condition with high probability. In contrast with
the random configuration, a deterministic codebook and pilot
design was considered in [13]. The approach taken in [13] is
to obtain codebooks that provide the minimal total coherence
(MTC) of the sensing matrix. However, it still contains
randomness in beamformer column permutation.
In this paper, we propose a completely deterministic beam-
former codebook and pilot design method for CS based open-
loop narrowband mmWave channel estimation. In practice,
receivers should know about pilots and precoding schemes
in order to properly perform channel estimation even with
the random codebook. Therefore we design codebooks and
pilots that can minimize mutual coherence of the resulting
sensing matrix, which is of importance to CS algorithms. As
suggested in [13], random beamformer column permutation
plays a critical role to reduce measurement time and instances.
We propose a greedy algorithm to find the best column permu-
tation of the obtained RF beamformer codebook. Simulation
results show that codebooks obtained by the proposed method
outperforms the random codebook, the MTC codebook, and
the adaptive CS in practical situations.
II. S YSTEM M ODEL
In the downlink, we assume a single base station (BS) and
channel estimation is performed by each user. Both the BS and
user equipment (UE) are equipped with the fully-connected
phase shifter based hybrid beamforming architecture as shown
in Fig 1. The BS has Nt transmit antennas and Lt transmit RF
chains, and the UE has Nr receive antennas and Lr receive
RF chains. The number of streams is assumed to be equal to
Lt . Assuming the channel is narrowband, the received signal
after RF and baseband combining at the time instance m can
be expressed as
√
ym = H
ρWm H
HFm xm + Wm nm ∈ CLr , (1)
 +

…
RF RF
DAC + ADC

…
Chain Chain
C

RF Combiner
RF Precoder
H
Lt Lt A
Nr Lr Lr
Digital Digital
<latexit sha1_base64="DKhqvWi7aeBSy87EYl2D6XtHwg8=">AAAB9HicbVC7SgNBFJ2NrxhfUUubwSBYSNgVwZQBGwuLiMYEkiXMTu4mQ+axzMwGwpJPsNXGTmz9H8GPcZJsoYkHBg7n3Ms9c6KEM2N9/8srrK1vbG4Vt0s7u3v7B+XDoyejUk2hSRVXuh0RA5xJaFpmObQTDUREHFrR6Gbmt8agDVPy0U4SCAUZSBYzSqyTHu56tleu+FV/DrxKgpxUUI5Gr/zd7SuaCpCWcmJMJ/ATe6EkuDSpkGFGtGWUw7TUTQ0khI7IADqOSiLAhNk89BSfOaWPY6XdkxbP1d8bGRHGTETkJgWxQ7PszcT/vE5q41qYMZmkFiRdHIpTjq3CswZwn2mglk8cIVQzlxXTIdGEWtdTydURLH9+lTxdVgO/GtxfVeq1vJgiOkGn6BwF6BrV0S1qoCaiaICe0Qt69cbem/fufSxGC16+c4z+wPv8AUORkeU=</latexit> <latexit sha1_base64="DKhqvWi7aeBSy87EYl2D6XtHwg8=">AAAB9HicbVC7SgNBFJ2NrxhfUUubwSBYSNgVwZQBGwuLiMYEkiXMTu4mQ+axzMwGwpJPsNXGTmz9H8GPcZJsoYkHBg7n3Ms9c6KEM2N9/8srrK1vbG4Vt0s7u3v7B+XDoyejUk2hSRVXuh0RA5xJaFpmObQTDUREHFrR6Gbmt8agDVPy0U4SCAUZSBYzSqyTHu56tleu+FV/DrxKgpxUUI5Gr/zd7SuaCpCWcmJMJ/ATe6EkuDSpkGFGtGWUw7TUTQ0khI7IADqOSiLAhNk89BSfOaWPY6XdkxbP1d8bGRHGTETkJgWxQ7PszcT/vE5q41qYMZmkFiRdHIpTjq3CswZwn2mglk8cIVQzlxXTIdGEWtdTydURLH9+lTxdVgO/GtxfVeq1vJgiOkGn6BwF6BrV0S1qoCaiaICe0Qt69cbem/fufSxGC16+c4z+wPv8AUORkeU=</latexit> <latexit sha1_base64="o1/pw9n0hWTAODeF1cTBy8fhSA0=">AAAB9HicbVC7SgNBFJ2NrxhfUUubwSBYSNgVwZQBGwuLiMYEkiXMTu4mQ+axzMwGwpJPsNXGTmz9H8GPcZJsoYkHBg7n3Ms9c6KEM2N9/8srrK1vbG4Vt0s7u3v7B+XDoyejUk2hSRVXuh0RA5xJaFpmObQTDUREHFrR6Gbmt8agDVPy0U4SCAUZSBYzSqyTHu56uleu+FV/DrxKgpxUUI5Gr/zd7SuaCpCWcmJMJ/ATe6EkuDSpkGFGtGWUw7TUTQ0khI7IADqOSiLAhNk89BSfOaWPY6XdkxbP1d8bGRHGTETkJgWxQ7PszcT/vE5q41qYMZmkFiRdHIpTjq3CswZwn2mglk8cIVQzlxXTIdGEWtdTydURLH9+lTxdVgO/GtxfVeq1vJgiOkGn6BwF6BrV0S1qoCaiaICe0Qt69cbem/fufSxGC16+c4z+wPv8AUB1keM=</latexit> <latexit sha1_base64="o1/pw9n0hWTAODeF1cTBy8fhSA0=">AAAB9HicbVC7SgNBFJ2NrxhfUUubwSBYSNgVwZQBGwuLiMYEkiXMTu4mQ+axzMwGwpJPsNXGTmz9H8GPcZJsoYkHBg7n3Ms9c6KEM2N9/8srrK1vbG4Vt0s7u3v7B+XDoyejUk2hSRVXuh0RA5xJaFpmObQTDUREHFrR6Gbmt8agDVPy0U4SCAUZSBYzSqyTHu56uleu+FV/DrxKgpxUUI5Gr/zd7SuaCpCWcmJMJ/ATe6EkuDSpkGFGtGWUw7TUTQ0khI7IADqOSiLAhNk89BSfOaWPY6XdkxbP1d8bGRHGTETkJgWxQ7PszcT/vE5q41qYMZmkFiRdHIpTjq3CswZwn2mglk8cIVQzlxXTIdGEWtdTydURLH9+lTxdVgO/GtxfVeq1vJgiOkGn6BwF6BrV0S1qoCaiaICe0Qt69cbem/fufSxGC16+c4z+wPv8AUB1keM=</latexit>

<latexit sha1_base64="JMQP6fsz8SUEkEYonmXBtF7r6bE=">AAAB9HicbVDLSgMxFM3UV62vqks3wSK4kDIjgl0W3LiSitYW2qFk0jttaB5DkimUoZ/gVjfuxK3/I/gxpu0stPVA4HDOvdyTEyWcGev7X15hbX1jc6u4XdrZ3ds/KB8ePRmVagpNqrjS7YgY4ExC0zLLoZ1oICLi0IpGNzO/NQZtmJKPdpJAKMhAsphRYp30cNfTvXLFr/pz4FUS5KSCcjR65e9uX9FUgLSUE2M6gZ/YCyXBpUmFDDOiLaMcpqVuaiAhdEQG0HFUEgEmzOahp/jMKX0cK+2etHiu/t7IiDBmIiI3KYgdmmVvJv7ndVIb18KMySS1IOniUJxybBWeNYD7TAO1fOIIoZq5rJgOiSbUup5Kro5g+fOr5OmyGvjV4P6qUq/lxRTRCTpF5yhA16iOblEDNRFFA/SMXtCrN/bevHfvYzFa8PKdY/QH3ucPQ5WR5Q==</latexit>

. . . . . .
N Combiner
. Precoder . . . . .
. . . N . . .
E
RF L RF
DAC + ADC

…
Chain Chain
+

…
Fig. 1. System block diagram of the phase shifter based hybrid beamforming architecture

where ρ is the average transmit power, Wm ∈ CNr ×Lr and and the corresponding transmit and receive array response
Fm ∈ CNt ×Lt denote the combiner and precoder matrices, matrices can be constructed as
respectively, xm ∈ CLt denotes the pilot symbol vector, H ∈
CNr ×Nt is the channel matrix, and nm ∈ CNr ∼ CN (0, σn2 I) At = [at (ϑ0 ), at (ϑ1 ), . . . , at (ϑNp −1 )] ∈ CNt ×Np ,
denotes the additive noise. (·)H denotes the conjugate trans- Ar = [ar (θ0 ), ar (θ1 ), . . . , ar (θNp −1 )] ∈ CNr ×Np .
pose. Both the precoder and combiner matrices are a product
of RF and baseband ones, i.e., Fm = FRF,m FBB,m and With the transmit and receive array response matrices, (3) can
Wm = WRF,m WBB,m . By vectorizing the right hand side be rewritten as
in (1), it can be rewritten as
√ H = Ar Hd AH
t, (4)
sT H


ym = ρ ⊗ m vec(H) + vm ,
Wm
√ where Hd ∈ CNp ×Np is a diagonal matrix with the scaled
= ρΦm vec(H) + vm ,
channel gains on its diagonal.
H
where sm = Fm xm , vm = Wm nm , Φm = sT H
m ⊗ Wm , and
Now we define the angle dictionary matrix for sparse
T formulation. The transmit and receive angle dictionary matrix,
(·) and ⊗ denote the matrix transpose and the Kronecker
product. By stacking M instances of the received signal Āt , is defined as
vectors, we can obtain r
1
√ Āt = [at (ν0 ), at (ν1 ), . . . , at (νGt −1 )] ∈ CNt ×Gt ,
y= ρΦvec(H) + v ∈ CM Lr , (2) Nt
(5)
where y = [y1T , y2T , . . . , yM
T T
] is the total received signal
vector, Φ = [Φ1 , Φ2 , . . . , ΦT
T T T
M ] is the sensing matrix, and
where Gt is the size of transmit angle grid, and νn ∈ Θ =
T T T T
v = [v1 , v2 , . . . , vM ] . {νn = arccos φπn | φn = Gnt − 21 , n = 0, 1, . . . , Gt − 1} where
G
For the narrowband channel, we adopt the geometric chan- Gt ≥ Nt . It it worth noting that Āt ĀH
t = Nt INt where INt
nel model. Assuming Np clusters constitute the channel, the is the Nt × Nt identity matrix. The receive angle dictionary
channel matrix can be given as matrix, Ār , is also similarly defined with Gr (≥ Nr ). Then
the channel matrix in (4) can be rewritten as
Np −1
s
Nt Nr X Nr ×Nt
H= αl ar (θl )aH
t (ϑl ) ∈ C , (3) H = Ār H̄d ĀH
t, (6)
Np
l=0
where H̄d ∈ CGr ×Gt is the channel gain matrix which needs
where αl ∼ CN (0, σα2 ) is the complex channel gain, at (·) not be a diagonal matrix. Ignoring the grid quantization errors,
and ar (·) are, respectively, the transmit and receive array vec(H̄d ) is an Np -sparse vector that will be estimated.
response vectors evaluated at the angles. θ and ϑ are the By plugging (6) into (2), we can get
angle of arrival (AoA) and departure (AoD). In this paper,
the employed antenna arrays are assumed to be uniform linear √
y= ρΦ(Ā∗t ⊗ Ār )vec(H̄d ) + v
array with a half wavelength spacing. The transmit and receive √
= ρΦΨh + v,
array response vectors are then given as
r where Ψ = Ā∗t ⊗ Ār ∈ CNt Nr ×Gt Gr is the dictionary matrix,
1
at (ϑ) = [1, e−jπ cos ϑ , . . . , e−jπ cos(Nt −1)ϑ ]T ∈ CNt , and h = vec(H̄d ) ∈ CGt Gr is the sparse channel gain vector.
Nt
r The product of the sensing matrix and the dictionary matrix is
1 called the equivalent sensing matrix, i.e., ΦΨ. We assume that
ar (θ) = [1, e−jπ cos θ , . . . , e−jπ cos(Nr −1)θ ]T ∈ CNr ,
Nr the channel gain vector is estimated by using CS algorithms.
 H
P
III. C OHERENCE M INIMIZING C ODEBOOK M
P reduction. Assuming Wmr Wm r
= I, we focus on
∗ T
smt smt . It can be rewritten as
Recovery guarantees of CS algorithms can be assessed by
mutual coherence of the equivalent sensing matrix, and the Mt
X Mt
X
mutual coherence, µ, is defined by s∗mt sT
mt = F∗mt x∗mt xT T
mt Fmt
mt mt
|aHi aj | |(AH A)ij | Mf Mx
!
µ(A) = max = max , X X
i6=j kai k2 kaj k
2
i6=j kai k2 kaj k
2 = F∗mf x∗mx xT
mx FT
mf ,
mf mx
where ai denotes the i-th column vector in the matrix A, and Mf
Aij denotes the element in the i-the row and j-th column X
= F∗mf XFT
mf , (9)
of the matrix A. Therefore mutual coherence minimization
mf
can be achieved by minimizing all off-diagonal elements
2 P ∗ T
of AH A, i.e., min AH A − I F . Plugging the equivalent where X = xmx xmx , the second equality holds if we use
sensing matrix, the optimization problem is given by the same technique that is used for the transmitter and receiver
separation, and Mt = Mf Mx where Mx and Mf denote the
2
min ΨH ΦH ΦΨ − I F
. (7) number of pilot symbol vectors and precoders, P respectively.
In the ideal case, it is desired that both X and F∗mf FTmf
The objective function in (7) can be simplified as are to be the identity matrix to obtain
P ∗ T
smt smt = I. Then
2 2 Mx should be Lt , and accordingly Mf becomes N Lt . Since
t
ΨH ΦH ΦΨ − I F
= kΦH Φ − I F we do not want to lose beamforming capability, we control
M 2
X M values by adjusting Mx . The value of Mx determines the
= k ΦH
m Φm −I , rank of X. Then it boils down to the low rank approximation
m=1 F which can be expressed as
where the first equality holds due to the fact that ΨΨH = kI min kI − XkF subject to rank(X) ≤ Mx ,
Gt Gr X
and k = N t Nr
can be ignored since it scales all columns
of the equivalent sensing matrix and does not affect mutual and its well known analytic solution is to exploit truncated
coherence. Our goal, thus, is to find the sets of the pilots, singular value decomposition (SVD). It implies that the
precoders and combiners that minimize the objective function. smaller rank naturally leads X to have grater deviation from
Considering the fact that Φm is sT H
m ⊗ Wm , the summation in
the identity matrix. There exists infinite solutions; however,
the objective function can also be written as we have some criteria on choosing the solution. Firstly, all
X X elements of all xmx should not be zero to take advantage of
ΦH
m Φm = s∗m sT
m ⊗ Wm Wm .
H
all possible beams. Secondly, all elements of xmx should have
the identical magnitude to equally weight all beams. The Lt -
The above equation can readily be made the identity matrix point Discrete Fourier Transform (DFT) matrix satisfies these
if (i) the summation can be distributed over the Kronecker criteria, and any Mx columns can be chosen from the DFT
product and (ii) M is large enough. For illustration, let M matrix and conclude the pilot codebook. The pilot codebook
be M = Mt Mr where Mt and Mr denote the number of X can be expressed as
configurations of transmitter and receiver, then we have
X = {xmx : ∀mx ∈ 1, 2, . . . , Lt , ULt = [x1 , x2 , . . . , xLt ]},
M Mt X
Mr
where UN ∈ CN ×N is the N -point DFT matrix. According
X X
s∗m sT H
m ⊗ Wm Wm = s∗mt sT H
mt ⊗ Wmr Wmr
to Mx , M is determined by N t Nr
M .
m mt mr P L∗t Lr T x P H
Mt
X Mr
X We have assumed that Fmt Fmt and Wmr Wm r
are
= s∗mt sT
mt ⊗ H
Wmr Wm r
. the identity matrix. Taking into account the phase shifter based
mt mr analog beamformers, Mf = N t Nr
Lt and Mr = Lr , the column
(8) partition of the proper size DFT matrices can compose the
beamformer codebooks. Namely then can be expressed as
Since s∗mt sT H
mt is a rank one matrix, and Wmr Wmr can be 
up to a rank Lr matrix, Mt and Nt
P ∗MrT must be P at least N t F = Fmf : ∀mf ∈ {1, 2, . . . , Mf = },
and N r
Lr in order to make both s s
mt mt and Wmr Wm H
r
Lt
become a full rank matrix. Being full rank matrices is a critical UNt = [F1 , F2 , . . . , FMf ] ,
requirement for it to be the identity matrix. Then the solutions 
Nr
for making both terms the identity matrix can easily be found. W = Wmr : ∀mr ∈ {1, 2, . . . , Mr = },
Lr
We now set this as the baseline, and start to decrease M .
UNr = [W1 , W2 , . . . , WMr ]} , (10)
As M is a product of Mt and Mr , either or both can be
decreased to have a lower M value. A BS normally has far where F and W denote the precoder and combiner code-
more antennas than UEs do (Nt  N Lr ), we target Mt for
r
books, respectively. Another advantage of a DFT matrix
 n o
−1
= (S ◦ INt ) 2 ⊗ INr (S ⊗ INr ) ×
Algorithm 1 Greedy Precoder Column Ordering n
−1
o
(S ◦ INt ) 2 ⊗ INr
Input: X, U(=Nt -point DFT matrix) n o
−1 −1
Output: F = (S ◦ INt ) 2 S (S ◦ INt ) 2 ⊗ INr ,
1: Initialization: Set F to an empty matrix.
2: for n = 1 to Nt do . Choose Nt columns where the second equality comes from (8) and (10), and (11)
3: for m = 1 to the number of columns left in U do becomes
4: . Go through a pool of possible columns 
−1 −1

µ(Φ) = max (S ◦ INt ) 2 S (S ◦ INt ) 2 . (12)
5: Fm ← [F, U:,m ] i6=j ij
6: Sm ← (Fm )∗ (I ⊗ X)(Fm )T
7: Calculate µ(Φ)m with Sm by using (12). Here, S can be written in matrix form as F∗ (I Nt ⊗ X)FT by
Lt

8: end for simplifying (9). In (12), the matrix size is reduced by a factor
9: mmin = arg minm µ(Φ)m of Nr compared with (11) in both dimensions. The matrix
10: F ← [F, U:,mmin ] . Append the found vector to the inversion and square root are taken on a diagonal matrix.
codebook For pilot codebook construction, Mx column vectors are to
11: U ← U:,[...,mmin −1,mmin +1,...] . Remove the found be chosen from the Lt -point DFT matrix. As Lt is usually not
vector from the pool a large value, we perform exhaustive search for this selection.
12: end for However, the first column of the DFT matrix should always
13: return F be included in selection. This is because the elements in the
first row of Fm are identical, and a sum of elements in the
non-first column of the DFT matrix is zero. It leads to the
is that the required phase shifter resolution in bits is first elements of sm = Fm xm being a zero and the first Nr
log2 (number of antennas). columns in Φm zeros. Not having the first column of the DFT
matrix as a pilot vector eventually ends up with the first Nr
IV. P RECODER C OLUMN P ERMUTATION columns in Φ being zeros. It is also worth mentioning that
the order of pilot vectors does not matter considering X is a
With Mx < Lt , X is not the identity matrix, and neither is
sum of outer products of each pilot vector.
(9). In this case, the column order in F = [F1 , F2 , . . . , FMf ]
Taking into account all that is mentioned above, the greedy
P ∗xmTt selection result in changes not only in S =
and
column ordering algorithm is given in Algorithm 1. U:,m
smt smt , but also mutual coherence of Φ. In [13], the
denotes the m-th column of the matrix U. As Algorithm 1 is

columns are proposed to be randomly permutated; however, Lt −1
for a given set of pilots, we perform the algorithm M x −1
we want to find a deterministic order so that the codebook
times to find the best pilot selection and precoder column
can practically be used.PNote that column permutation does
order that achieves the lowest mutual coherence.
not affect the fact that F∗mt FT
mt = I.
The distribution of mutual coherence obtained by random
Since F has Nt columns and Nt is usually a large number,
column permutation and its mean value are provided in
the exhaustive search for the best column order would not be
Fig. 2 along with one obtained by the proposed algorithm.
feasible. With Nt = 64, the number of permutations is 64! ≈
For illustration, two different values (two and seven) are
1.27 × 1089 . Therefore by adopting a greedy algorithm, we
considered for Mx . As shown in the figure for both cases, the
iteratively seek one column vector from the DFT matrix that
mutual coherence of the proposed algorithm is lower than the
achieves the lowest mutual coherence with the pre-selected
mean of random permutation and is near the lowest value that
vectors for the codebook. At the same time, Φ is a M Lt ×
random permutation can achieve. This observation can also be
Nt Nr matrix, and calculating its mutual coherence directly
seen with all possible Mx values, and the evaluated mutual
from Φ is computationally expensive. Thus we first obtain
coherence values from random permutation and the proposed
a simplified formulation for mutual coherence of the sensing
algorithm are given in Table I. For all possible Mx values,
matrix. Mutual coherence can also be defined as
the proposed algorithm achieves lower mutual coherence
  than the mean of mutual coherence distribution of random
µ(Φ) = max Φ̃H Φ̃ , (11)
i6=j ij permutation, and as Mx increases, mutual coherence declines.
When Mx = Lt (= 8 in this case), the mutual coherence
where Φ̃ is the column-wise normalized version of Φ which converges to zero. It implies that higher Mx values make the
1
can also be expressed as Φ̃ = ΦΣ− 2 where Σ = INt Nr ◦ CS algorithms perform more accurate channel estimation.
(ΦH Φ) and ◦ denotes the Hadamard product. Then we have
V. N UMERICAL R ESULTS
H H − 21 − 21
Φ Φ INt Nr ◦ ΦH Φ
H



Φ̃ Φ̃ = INt Nr ◦ Φ Φ In this section, we compare performance of the determinis-
− 12 tic codebook obtained by the proposed algorithm with that of
= {INt Nr ◦ (S ⊗ INr )} (S ⊗ INr ) ×
(i) the random phase shifter and pilot configuration, (ii) the
− 12
{INt Nr ◦ (S ⊗ INr )} MTC codebook with random precoder column permutation
 0.04 5
Distribution of random permutation
0.035 Mean of random permutation M x =2 (M=64)
Proposed algorithm
0
0.03

0.025 -5
M=864
Probability

0.02
-10
0.015
M x =4 (M=128)
-15
0.01

0.005 -20
M x =8 (M=256)
0
0.55 0.6 0.65 0.7 0.75
-25
Mutual Coherence -10 -5 0 5 10 15
(a) Mx = 2

Fig. 3. NMSE vs. SNR with the proposed, random and MTC codebooks
0.045 using OMP [13] along with the adaptive CS channel estimation [6].
Distribution of random permutation
0.04 Mean of random permutation
Proposed algorithm
0.035
Fig. 3 shows NMSE as a function of the transmit SNR,
and the three codebooks and the adaptive CS are considered.
0.03
The first observation is that NMSE decreases with SNR and
Probability

0.025
the number of snapshots (M ). With an increase in M , the
0.02 estimator can take more measurements, which improves per-
0.015 formance of CS-based estimator. At the same time, however,
0.01 higher M also requires a longer measurement time and more
0.005
computation. Thus we have a performance versus time and
0
power consumption trade-off.
0.12 0.14 0.16 0.18 0.2
Mutual Coherence
0.22 0.24 0.26
Three values of M (= 64, 128, 256) are considered in Fig. 3,
(b) Mx = 7
and performance ranking of the codebooks changes depending
on the value. The proposed codebook achieves the lowest
Fig. 2. Distribution of mutual coherence of the sensing matrix generated with
20,000 random precoder column permutation. The mean of the distribution NMSE among the three when M is either 128 or 256. With
and the mutual coherence of the proposed algorithm are marked as well. The M = 256, its performance is the best together with the MTC,
parameters in this simulation is Nt = 64, Lt = 8, and (a) Mx = 2 and (b) and M = 64 makes it the worst. It implies that the proposed
Mx = 8.
algorithm works well when the number of snapshots is not
too small. This is due to the fact that X deviates from the
[13] and (iii) the adaptive CS [6]. For evaluation of the identity matrix with small Mx . Different approaches may be
codebooks, OMP is employed except for the adaptive CS. explored to address this issue, which will be our future work.
The system parameters for simulation are as follows unless The adaptive CS generally needs a large number of snap-
otherwise specified: Nt = 64, Nr = 16, Lt = 8, Lr = 4, shots which is determined by the grid size. Even with more
Gt = γNt , Gr = γNr , bP S = 6 and Np = 4 where snapshots, estimation errors are greater than the considered
γ = 1.5(> 1) is the grid multiplier. For the adaptive CS, Gt = codebooks including the proposed one. In Fig. 3, the adaptive
Gr = 96, which results in M = 864. The normalized mean CS has M = 864, but the NMSE is higher than the proposed
squared error (NMSE) is defined as E[kH − Ĥk2F /kHk2F ] codebook with M = 256 across the SNR range and than that
where H and Ĥ are the true and the estimated channel with M = 128 in medium and high SNR regimes.
matrix, respectively, SNR is defined as ρ/σ 2 , and (·)F denotes
To illustrate the relationship between performance and Mx ,
the Frobenius norm of a matrix. Multipath components of
Fig. 4 is provided where SNR is fixed at either 15 or 0 dB.
the channels, in simulations, have AoDs and AoAs that are
The NMSE of the random codebook starts from a relatively
not necessarily aligned with the grids of the dictionary. The
low NMSE and gradually declines with Mx . The other two
complete source code is available [14].
codebooks, on the other hand, start from high NMSE, drop
at low Mx , and gradually decline in medium and high Mx
regimes. Due to the steep drops in the low Mx regime, NMSE
TABLE I
M EAN COHERENCE OF RANDOM PERMUTATION AND COHERENCE OF THE
curves of the proposed and MTC codebooks cross that of
PROPOSED ALGORITHM OVER VARIOUS Mx random codebook. For 15 dB SNR, the crossover happens
at Mx = 4 with the proposed codebook which is earlier
Mx 1 2 3 4 5 6 7 8 than Mx = 5 with the MTC codebook. For 0 dB SNR,
Permutation 0.86 0.62 0.48 0.38 0.30 0.23 0.16 0
Proposed 0.75 0.52 0.39 0.31 0.25 0.19 0.13 0 the it happens at Mx = 4 for both codebooks. With system
configurations such as one used in this simulation, thus, the
 -5 nication systems based on mutual coherence minimization. We
first obtained the criteria for codebooks of pilots, precoders
and combiners, and proposed the pilot selection and precoder
-10
column ordering algorithm for further mutual coherence re-
duction. The mutual coherence distribution of random col-
-15 umn permutation was provided to show the proposed greedy
precoder column ordering algorithm achieves lower mutual
-20 coherence than random permutation.
1 2 3 4 5 6 7 8 We also provided the channel estimation simulation results
using OMP for performance comparison between the pro-
(a) SNR = 15 dB posed, the random and the MTC codebooks. The proposed
method provides the best tradeoff between channel estimation
2 performance and measurement time, and has more accurate
0 channel estimation vs. number of channel paths.
-2
R EFERENCES
-4
[1] F. Sohrabi and W. Yu, “Hybrid digital and analog beamforming design
-6 for large-scale antenna arrays,” IEEE J. Sel. Topics Signal Process.,
vol. 10, no. 3, pp. 501–513, April 2016.
-8 [2] O. E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R. W. Heath,
“Spatially sparse precoding in millimeter wave MIMO systems,” IEEE
-10
1 2 3 4 5 6 7 8 Trans. Wireless Commun., vol. 13, no. 3, pp. 1499–1513, Mar. 2014.
[3] R. W. Heath, N. Gonzlez-Prelcic, S. Rangan, W. Roh, and A. M.
(b) SNR = 0 dB Sayeed, “An overview of signal processing techniques for millimeter
wave MIMO systems,” IEEE J. Sel. Topics Signal Process., vol. 10,
Fig. 4. NMSE vs. Mx with SNR of (a) 15 dB and (b) 0 dB. no. 3, pp. 436–453, Apr. 2016.
[4] T. S. Rappaport, G. R. MacCartney, M. K. Samimi, and S. Sun,
-14 “Wideband millimeter-wave propagation measurements and channel
models for future wireless communication system design,” IEEE Trans.
-15 Commun., vol. 63, no. 9, pp. 3029–3056, Sep. 2015.
[5] T. S. Rappaport, F. Gutierrez, E. Ben-Dor, J. N. Murdock, Y. Qiao,
-16 and J. I. Tamir, “Broadband millimeter-wave propagation measurements
and models using adaptive-beam antennas for outdoor urban cellular
-17 communications,” IEEE Trans. Antennas Propag., vol. 61, no. 4, pp.
1850–1859, Apr. 2013.
-18 [6] A. Alkhateeb, O. E. Ayach, G. Leus, and R. W. Heath, “Channel
estimation and hybrid precoding for millimeter wave cellular systems,”
-19 IEEE J. Sel. Topics Signal Process., vol. 8, no. 5, pp. 831–846, Oct.
2 3 4 5 6 7 8 9 10
2014.
[7] K. Venugopal, A. Alkhateeb, N. González-Prelcic, and R. W. Heath,
“Channel estimation for hybrid architecture-based wideband millimeter
Fig. 5. NMSE vs. N p channel paths with 15 dB SNR and Mx = 4
wave systems,” IEEE J. Sel. Areas in Commun., vol. 35, no. 9, pp.
1996–2009, Sep. 2017.
[8] J. Rodriguez-Fernandez, N. Gonzalez-Prelcic, K. Venugopal, and
proposed codebook is preferred when Mx is greater than three R. W. Heath, “Frequency-domain compressive channel estimation for
as (i) it achieves NMSE that is lower or equal to that of frequency-selective hybrid mmWave MIMO systems,” IEEE Trans.
Wireless Commun., vol. 17, no. 5, pp. 2946–2960, May 2018.
the MTC codebook and (ii) is lower than that of the random [9] R. Méndez-Rial, C. Rusu, N. González-Prelcic, A. Alkhateeb, and R. W.
codebook when Mx > 3 and (iii) the proposed codebook has Heath, “Hybrid MIMO architectures for millimeter wave communica-
no randomness. tions: Phase shifters or switches?” IEEE Access, vol. 4, pp. 247–267,
2016.
The number of channel paths Np affects channel estimation [10] S. Park and R. W. Heath, “Spatial channel covariance estimation for
performance as well since it is directly related to the sparsity mmWave hybrid MIMO architecture,” in Proc. Asilomar Conf. Sig.,
Sys. and Comp., Nov. 2016, pp. 1424–1428.
of the channel vector being estimated. In Fig. 5, SNR and Mx [11] X. Gao, L. Dai, S. Han, C. L. I, and X. Wang, “Reliable beamspace
are fixed at 15 dB and four, respectively. In this figure, the pro- channel estimation for millimeter-wave massive MIMO systems with
posed codebook yields the lowest estimation error across the lens antenna array,” IEEE Trans. Wireless Commun., vol. 16, no. 9, pp.
6010–6021, Sep. 2017.
Np values. The order of the MTC and the random codebooks [12] Y. Xiao, Y. Wang, and W. Xiang, “Dimension-deficient channel esti-
varies depending on Np . Since Np varies and is determined mation of hybrid beamforming based on compressive sensing,” IEEE
by the channel environment, the proposed codebook would Access, vol. 7, pp. 13 791–13 798, Jan. 2019.
[13] J. Lee, G. T. Gil, and Y. H. Lee, “Channel estimation via orthogonal
be a good choice as it achieves the lowest channel estimation matching pursuit for hybrid MIMO systems in millimeter wave com-
error regardless of the number of channel paths. munications,” IEEE Trans. Commun., vol. 64, no. 6, pp. 2370–2386,
Jun. 2016.
VI. C ONCLUSION [14] J. Sung and B. L. Evans, “Hybrid beamformer codebook design and or-
dering for compressive mmWave channel estimation,” Software Release,
In this paper, we proposed a codebook design method for Jul. 26, 2019, https://github.com/junmo-sung/hb-cb-design-ordering.
mmWave channel estimation in hybrid beamforming commu-