CNN-Based Hybrid Precoding Design with Geometric Mean

Decomposition
Mahmoud A. Abugubba 1 , Nagia M. Gaboua 1 , Taissir Elganimi 2 , and Khaled Rabie 1

1
Affiliation not available
2
University of Tripoli

October 30, 2023

Posted on 6 Dec 2020 — CC0 1.0 — https://doi.org/10.36227/techrxiv.21632132.v1 — e-Prints posted on TechRxiv are preliminary reports that are not peer reviewed. They should not b...

Abstract
Communications over millimeter-wave (mmWave) frequencies is a key technology for the fifth generation (5G) cellular networks
due to the large bandwidth available at mmWave bands. The short wavelength of mmWave bands enables large antenna
arrays to be placed on the transceivers which forms massive multiple-input multiple-output (MIMO). Massive MIMO with
conventional fully-digital (FD) beamforming is difficult to be implemented due to high power consumption and hardware cost.
One of the most effective solutions to this problem is hybrid beamforming which can be used to balance the beamforming gain,
hardware implementation cost, and the power consumption. However, due to the non-convex constraints imposed by phase
shifters, finding the global optima for the hybrid beamforming system is very challenging with high computational complexity.
To address this issue, deep learning (DL)-based hybrid precoding with geometric mean decomposition (GMD) algorithm for
narrowband mmWave massive MIMO system is proposed in this paper, where it can directly estimate the hybrid analog and
digital precoders (combiners) from a given optimal FD precoder (combiner). Simulation results demonstrated that the proposed
hybrid precoding model can more accurately approximate the FD precoding performance.

1
 CNN-Based Hybrid Precoding Design with
Geometric Mean Decomposition
Mahmoud A. Abugubba1 , Nagia M. Gaboua1 , Taissir Y. Elganimi1 and Khaled M. Rabie2
1 Department of Electrical and Electronic Engineering, University of Tripoli, Libya
2 Department of Engineering, Manchester Metropolitan University, United Kingdom
Email: {ma.abugubba, n.gaboua, t.elganimi}@uot.edu.ly, k.rabie@mmu.ac.uk
Abstract—Communications over millimeter-wave (mmWave)
frequencies is a key technology for the fifth generation (5G) cel-
lular networks due to the large bandwidth available at mmWave
bands. The short wavelength of mmWave bands enables large
antenna arrays to be placed on the transceivers which forms
massive multiple-input multiple-output (MIMO). Massive MIMO
with conventional fully-digital (FD) beamforming is difficult to
be implemented due to high power consumption and hardware
cost. One of the most effective solutions to this problem is hybrid
beamforming which can be used to balance the beamforming
gain, hardware implementation cost, and the power consumption.
However, due to the non-convex constraints imposed by phase
shifters, finding the global optima for the hybrid beamforming
system is very challenging with high computational complexity.
To address this issue, deep learning (DL)-based hybrid precod-
ing with geometric mean decomposition (GMD) algorithm for
narrowband mmWave massive MIMO system is proposed in
this paper, where it can directly estimate the hybrid analog and
digital precoders (combiners) from a given optimal FD precoder
(combiner). Simulation results demonstrated that the proposed
hybrid precoding model can more accurately approximate the
FD precoding performance.
Index Terms—Massive MIMO, millimeter-wave, fully digital
precoding, hybrid precoding, deep learning, CNN.
I. I NTRODUCTION
A key technology for the fifth generation (5G) of wireless
communications and beyond is millimeter-wave (mmWave)
communication, which offers greater data rates on the order
of gigabit per second (Gbps), wider bandwidth, and higher
spectral efficiency than conventional cellular communications
[1]. However, the communication over the mmWave band
is very challenging due to signal propagation and channel
properties. To address these issues, massive multiple-input
multiple-output (MIMO) with beamforming techniques is re-
quired, where a large antenna arrays are used to concen-
trate the radiated energy and steer it toward the receiver
direction [2]. Higher diversity and multiplexing gains can
be attained through massive MIMO with beamforming gain,
which leads to higher spectral efficiency and higher radiated
energy efficiency [3]. Massive MIMO is costly and very
difficult to be combined with mmWave, though. One of the
promising approaches to address these problems is hybrid
beamforming, which utilizes significantly fewer power-hungry
radio frequency (RF) chains to achieve the performance of
fully digital (FD) beamforming, hence reducing the power
consumption and the system implementation complexity [4].
The hybrid beamforming design problem is a non-convex
optimization problem due to the constant modulus constraints
that imposed by phase shifters. Most existing methods reduce
the complexity by decoupling the optimization problem into
two sub-problems, where the objective of each sub-problem is
to approximate the FD precoding using matrix decomposition
[5]. In [6], a phase-extraction (PE) and manifold optimization
(MO)-based alternating minimization algorithm have been pro-
posed. Although the MO algorithm can achieve near optimal
performance by iteratively reducing the Euclidean distance
between both the hybrid beamformer and the FD beamformer,
its computational complexity prevents it from being used
in implementations. In the PE algorithm, the computational
complexity has been reduced with slight performance loss, but
it still provides a better precoding algorithm than most existing
algorithms. These algorithms are based on the singular value
decomposition (SVD) which requires complicated bit alloca-
tion schemes to achieve the channel capacity. To avoid this
issue, geometric mean decomposition (GMD) was proposed in
[7] to decompose the channel matrix into several parallel sub-
channels with equal signal-to-noise-ratios (SNRs), and hence
the simple identical bit allocation can be utilized for all sub-
channels. However, high computational complexity is still a
great challenge in hybrid beamforming design.
Recently, embedding deep learning (DL) into wireless
communications has had a great impact on solving com-
plex problems and high computation issues. For example,
the authors in [8] used deep recurrent neural network to
optimize the resource allocation problem with low compu-
tational complexity for the non-orthogonal multiple access
(NOMA) heterogeneous internet of things (IoT) network. In
[9], two convolutional neural networks (CNNs) were proposed
to address the problem of modulation classification, where it
can accurately recognize various modulation types. In another
work [10], a deep neural network (DNN) was employed
to estimate the channel and direction-of-arrival for massive
MIMO systems with better performance than the conventional
methods.
DL-based hybrid precoding schemes for mmWave massive
MIMO systems has also gained the attention of many re-
searchers. For example, in [11], a DNN is trained to optimize
the hybrid precoding using GMD. In another work [12], a
CNN-based hybrid precoding is proposed with two CNNs,
where each CNN is trained to estimate the hybrid precoders.
 1 1

2 2

Data stream RF Chain RF Chain Data stream

Digital 1 Digital
1
Precoder mm-Wave Compiner
Channel

RF Chain RF Chain

Analog Precoder Analog Combiner

Fig. 1. Block diagram of mmWave massive MIMO system with hybrid precoding.

Later, a joint hybrid precoding framework based on DNN is NtRF and NrRF RF chains, respectively, where Ns ≤ NtRF ≤ Nt
proposed with end-to-end optimization [13]. In order to reduce and Ns ≤ NrRF ≤ Nr in order to enable data stream multi-
the computation time, deep reinforcement learning has been plexing between the transmitter and receiver [5]. In Fig. 1, the
applied to hybrid beamforming designs in [14], while an auto- symbol vector s ∈ CNs ×1 that satisfy E[ss∗ ] = N1s INs is firstly
RF
encoder based on DNN is proposed in [15] for multi-user precoded by low dimensional digital precoder FBB ∈ CNt ×Ns ,
RF
scenarios. In a recent work [16], the authors designed a hybrid and then the high dimensional analog precoder FRF ∈ CNt ×Nt
precoding algorithm based on attention layer and CNN which is applied using phase shifters to shift the input phase and
is trained via unsupervised learning to maximize the spectral keep the amplitude as a constant. Therefore, all elements of
efficiency directly. (i) (i)H
FRF matrix are constrained to satisfy (FRF FRF )i,j = N1t and
Motivated by the foregoing introduction and recent work, can be represented by
a CNN-based hybrid precoding with GMD-based algorithm
is proposed in this paper. More specifically, the main contri-  F F F 
bution of this paper is to design hybrid precoding using DL ejθ11 ejθ12 ... ejθ1NRF
F F F
ejθ2NRF
 jθ21
1 e ejθ22 ...

approaches. The proposed hybrid preceding model simulate 
FRF =√  . .. .. ..
, (1)
the working operation of phase shifters, and satisfy the power  ..
Nt  . . .


constraint on digital precoder. In addition, the model can F F F

directly estimate the hybrid analog and digital precoders and ejθNt 1 ejθNt 2 ... ejθNt NRF
combiners from a given FD precoder and combiner, and also
F
is trained with GMD preceding dataset to take advantages where θij ∈ [0, 2π] for phase shifters of the analog precoder.
of GMD over SVD precoding. Simulation results verified the The transmitted signal at the transmit antenna array can be
ability of the proposed hybrid preceding model to approximate written as x = FRF FBB s, where FBB must be normalized to
the FD precoder performance. satisfy the total power constrain ∥FRF FBB ∥2F = Ns . Therefore,
The rest of this paper is organized as follows. In Section the received signal ỹ ∈ CNr ×1 can be written as
II, the system and channel models are presented for mmWave
√
massive MIMO system with hybrid precoding. The system ỹ = ρHFRF FBB s + n, (2)
model of the proposed CNN-based hybrid precoding with
GMD-based algorithm is presented in Section III. The simu- where ρ denotes the average received power, H ∈ CNr ×Nt is
lation results are discussed in Section IV, and the conclusions the channel matrix that satisfy E[∥H∥2F ] = Nt Nr , and n is the
and possible future work are given in Section V. additive white noise vector that follows an independent and
identical distribution, n ∼ CN (0, σn2 ).
II. SYSTEM MODEL
The receiver has the similar structure as the transmitter,
In this section, the system and channel models for mmWave where the received symbol vector s̃ after the combining
massive MIMO system with hybrid precoding are described, process can be represented by
and the problem formulation for maximizing the achievable
rate is also discussed.
s̃ = WH H
BB WRF ỹ
√ (3)
A. System Model = ρWBB WH
H H H
RF HFRF FBB s + WBB WRF n,

In Fig. 1, a narrowband mmWave massive MIMO system

RF RF
with hybrid precoding is considered. The Nt transmit antennas where WRF ∈ CNr ×Nr and WBB ∈ CNr ×Ns are the analog
transmit Ns independent data streams collected by Nr receive and digital combiners, respectively. WRF has a similar property
antennas. The transmitter and the receiver are equipped with as the analog precoder FRF which means it satisfies the
 (i) (i)H
constraint (WRF WRF )i,j = N1r , and the entire elements
matrix can be represented by Feature Hybrid
Generator Precoder
W
Model
 W W

ejθ11 ejθ12 . . . ejθ1NRF
W W W
ejθ22 . . . ejθ2NRF
 jθ21
e

1  Geometric Mean
WRF = √

, (4) H Decompsition
. .. .. ..

Nr  .. .

. . 
W W W
ejθNr 1 ejθNr 2 ... ejθNr NRF Feature
Hybrid
Combiner
Generator
W Model
where θij ∈ [0, 2π] for phase shifters of analog combiner.
The spectral efficiency (in bits/s/Hz) achieved by the mmWave
hybrid precoding system when Gaussian symbols are conveyed Fig. 2. The block diagram of the proposed hybrid precoding based on DL
over the wireless channel can be expressed as [17] approach.
 ρ
R = log2 INs + C−1 WH H
BB WRF HFRF FBB
Ns n (5)
 where FRF and WRF are the set of feasible analog precoders
× FH H H
BB FRF H WRF WBB , and combiners with constant amplitude, and ∥FRF FBB ∥2F = Ns
ensures the total transmitted power constraint. This optimiza-
where Cn = σn2 WH H
BB WRF WRF WBB ∈ C
Ns ×Ns
is the noise tion problem is non-convex and difficult to be solved, because
covariance matrix after combining. it requires a joint optimization over four beamforming variable
B. Channel Model matrices (FRF , FBB , WRF and WBB ). Decoupling the joint
The mmWave propagation is characterized as limited scat- design problem into hybrid precoding and combining problems
tering environment due to severe free-space pathloss. For this can overcome the difficulty of the design and provide near-
reason, the extended Saleh-Valenzuela model is used in this optimal hybrid precoding [5], where the objective of each
paper to cluster the channel into Nc scattering clusters, each problem is to minimize the Euclidean distance between the
of which contributes Nray propagation paths. Therefore, the hybrid beamformer with an FD beamformer. The decoupled
narrowband clustered channel matrix H ∈ CNr ×Nt can be hybrid precoder design problem that can provide hybrid pre-
represented by [5]: coders (FRF , FBB ) which approximately maximize R can be
expressed as
Nc N
s ray
Nt Nr X X
t H ∥Fopt − FRF FBB ∥F ,
H= αij ar (ϕrij , θij
r
)at (ϕtij , θij ) , (6) min
FRF ,FBB
Nc Nray i=1 j=1
s.t. FRF ∈ FRF , (9)
where αij is the complex gain of the j-th ray in i-th cluster,
(ϕtij , θij
t
) and (ϕrij , θij
r
) are the angle of departure and arrival ∥FRF FBB ∥2F = Ns ,
in azimuth and elevation planes, respectively. at and ar are where Fopt is the optimal FD precoder.
the normalized array response vectors of the transmit and
D. Fully Digital Precoding Based on GMD
receive antenna arrays corresponding to j-th ray in i-th cluster,
respectively. The channel matrix H can be decomposed using GMD into
The antenna array that enables the beamforming in azimuth Ns sub-channels with equal SNR as [7]:
and elevation planes is a uniform planar array (UPA) which 
 R1 ∗
  H
P1
H = QRPH = Q1 Q2

has the following response vector , (10)
0 R2 PH 2
1 h
aUPA (ϕ, θ) = √ 1, . . . , ejkd(m sin(ϕ) sin(θ)+n cos(θ)) , where P1 ∈ CNt ×Ns and Q1 ∈ CNr ×Ns are semi-unitary
N
iT (7) matrices (i.e., Q∗1 Q1 = P∗1 P1 = INs ). However, P1 can be
jkd((N1 −1) sin(ϕ) sin(θ)+(N2 −1) cos(θ)) regarded as an optimal FD precoder such that Fopt = P1 ,
...,e ,
and Q1 as an optimal FD combiner such that Wopt = Q1 .
where k = 2π λ , d is the element spacing between array ele- In addition, R1 is an upper triangular matrix whose diagonal
ments, the antenna array size is N = N1 N2 , and 0 ≤ m < N1 elements equal to the geometric mean of Ns positive singular
and 0 ≤ n < N2 are the indices of an antenna element. values, and ∗ denotes an arbitrary matrix that can be neglected
C. Problem Formulation [18], [19].
The hybrid precoder and combiner’s design aims to maxi- III. C ONVOLUTIONAL N EURAL N ETWORK -BASED
mize the spectral efficiency as follows H YBRID P RECODER AND C OMBINER
max R, This section presents the proposed hybrid precoding based
FRF ,FBB ,WRF ,WBB
on CNN approach as shown in Fig. 2. This figure shows
s.t. FRF ∈ FRF , the block diagram of the proposed hybrid precoding which
(8)
WRF ∈ WRF , composed of two models: hybrid precoder and combiner
∥FRF FBB ∥2F = Ns , models, where all models have the same building blocks. As
 CNN

Output
Dense
Layer

Layer
Convolutional Layer

Convolutional Layer

Convolutional Layer

Convolutional Layer

Pooling Layer
Zero Padding

Flatten Layer
128@2x2

64@2x2

64@2x2

32@2x2
4x4

2x2

Output
Dense
Layer

Layer
Fig. 3. The proposed hybrid precoder architecture.

shown in Fig. 2, the channel matrix H is firstly decomposed different filter size and Leaky-ReLU activation functions are
using GMD, and then Fopt and Wopt are fed to each hybrid utilized to extract complex features of the input, the first layer
precoder and hybrid combiner feature generator. The feature has 128 filters with 2×2 filter size, the second and third layers
generator is a pre-process block that allows the model to have 64 filters with 2×2 size. The last convolutional layer has
extract more features of the input, which provides better 32 filters with 2×2 size. In addition, pooling layer is applied to
training performance. The output of the feature generator reduce the output size, and to speed the computation. Flatten
regarded as the input raw data of the model. layer is also required to make a connection between all pooling
The hybrid precoder (combiner) model has two output layer activation outputs and next dense layers neurons.
layers, where the first layer produces the phase angle of analog The output of the flatten layer is shared between two
precoder (combiner) to simulate the working operation of neural networks, the first neural network for analog precoder
phase shifters, while the second layer produces the normalized has a single dense layer with 2Nt NtRF neurons and Leaky-
digital precoder (combiner). Each model is trained to minimize ReLU activation function, and output layer with 2NtRF Ns
the decoupled joint optimization problem. neurons which has been designed to meet the analog precoder
(i) (i)H
constraint (FRF FRF )i,j = N1t . Consequently, the estimated
A. Feature Generator
analog precoder with a constant amplitude F̂RF can be ex-
Feature generator receives a complex-valued matrix such as pressed as
the optimal FD precoder, Fopt , and then it is converted to the 1 F

real valued raw data by using the real, imaginary and the phase F̂RF = √ ej θ̂RF , (12)
Nt
of the input matrix. For example, if the input matrix is Fopt ,
F
the output vector xF ∈ R3Nt Ns ×1 of the feature generator can RF
where θ̂ RF ∈ RNt ×Nt is the phase angle of the estimated
be expressed as analog precoder, where each element must be between zero
F
h and 2π, i.e., (θ̂RF )i,j ∈ [0, 2π]. Therefore, the sigmoid function
xF = ℜ(Fopt )1,1 , ℜ(Fopt )1,2 , . . . , ℜ(Fopt )Nt ,Ns , with 2π amplitude is used to limit the output in [0, 2π]
ℑ(Fopt )1,1 , ℑ(Fopt )1,2 , . . . , ℑ(Fopt )Nt ,Ns , (11) range. The second neural network for the digital precoder has
iT single dense layer with 8NtRF Ns neurons and Leaky-ReLU
∠(Fopt )1,1 , ∠(Fopt )1,2 , . . . , ∠(Fopt )Nt ,Ns . activation function, and the output layer has 2NtRF Ns neurons
that produces a vectorized real and imaginary components of
Similarly, the output vector is xW ∈ R3Nr Ns ×1 when the the estimated digital precoder F̂BB . The hybrid precoder must
input matrix is Wopt . satisfy the total power constraint ∥F̂RF F̂BB ∥2F = Ns . To meet
B. Hybrid Precoder Model this constraint, F̂BB should be normalized as
√
The objective of the proposed CNN-based hybrid precoder Ns
F̂BB = F̂BB . (13)
model is to minimize the hybrid precoder design problem as ∥F̂RF F̂BB ∥F
stated in (9). Therefore, the proposed hybrid precoder model
must be trained to minimize the Euclidean distance between The model is trained with a defined loss function, that can
the optimal FD precoder Fopt and hybrid precoders (FRF , be expressed as the hybrid precoder design problem as
FBB ). Fig. 3 shows the hybrid precoder model architecture.
L(θ) = ∥Fopt − F̂RF F̂BB ∥F , (14)
As shown, the hybrid precoder model receives a feature vector
xF which contains a vectorized version of the real, imaginary where θ denotes the parameters of hybrid precoder model, and
and angular values of Fopt . The first layer is zero-padding layer the hybrid precoder matrix F̂RF F̂BB should be a unitary matrix,
with 4 × 4 padding which adds zeros with size 4 around the where all columns are orthonormal vectors, such that
input. Zero-padding layer is used to extract the features at the
H H
corner as well as the center. Four convolutional layers with F̂BB F̂RF F̂RF F̂BB = INs . (15)
 Adding this constraint to loss function as penalty term can
improve the model performance. Thus, the loss function can
be rewritten as
L(θ) = ∥Fopt −F̂RF F̂BB ∥F
H H (16)
+ λF ∥F̂BB F̂RF F̂RF F̂BB − INs ∥F ,
where λF is a non-negative constant of the penalty term
that used to satisfy F̂RF F̂BB to be semi-unitary matrix with
orthonormal vectors.
In order to minimize the lost function L(θ), Adam optimizer
is used during the training process to update the model
parameters in the direction of local minimum of L(θ). The
layers parameters at the first iteration should be initialized
at random points. In order to model a general problem as
possible, each initialized parameters have different random
distribution based on the layer activation function.
C. Hybrid Combiner Model
Hybrid combiner model is designed with the similar design
objectives of hybrid precoder model, where it has the same
building blocks: padding layer, four convolutional layers with
single pooling layer, flatten layer, dense layers and output
layers. The first output layer is designed to meet the analog
(i) (i)H
combiner constraint (WRF WRF )i,j = N1r . Consequently, the
estimated analog combiner with a constant amplitude ŴRF can
be expressed as
1 W
ŴRF = √ ej θ̂RF , (17)
Nr
W RF
where θ̂ RF ∈ RNr ×Nr is the phase angle of the estimated
W
analog combiner, where each element in θ̂RF is limited in
[0, 2π] range using sigmoid function with 2π amplitude. The
second output layer produces the estimated digital combiner
ŴBB .
The model is trained with a defined loss function, that can
be expressed as the hybrid combiner design problem as
L(θ) = ∥Wopt −ŴRF ŴBB ∥F
H H (18)
+ λW ∥ŴBB ŴRF ŴRF ŴBB − INs ∥F ,
where λW is a non-negative constant of the penalty term
that used to satisfy ŴRF ŴBB to be semi-unitary matrix with
orthonormal vectors. Adam optimizer is also used during the
training process to update the model parameters in order to
find a local minimum of L(θ).
IV. S IMULATION R ESULTS
In this section, simulation results are presented to illustrate
the spectral efficiency of the proposed CNN-based hybrid
precoding model compared to FD precoding based on GMD,
PE and MO algorithms. Throughout the simulations, a massive
MIMO with Nt = 8 × 8 and Nr = 4 × 4 UPA antennas with
half-wave element spacing is considered and the channel is
modeled as cluster environment with Nc = 5 clusters, each
cluster with Nray = 10 rays and 10◦ angle of spread.
The proposed model is constructed and processed using
Keras Python package, and the training dataset is generated
TABLE I
A SUMMARY TABLE OF TRAINING PARAMETERS .
Parameter Value
Optimizer Adam (with β1 = 0.9, β2 = 0.999 )
Learning rate (η) 0.00005
Leaky-ReLU constant (α) 0.2
λF , λW 0.15
Mini batch size 64
Size of dataset 200,000
Epochs 200
using the channel model expressed in (6) with 200,000 realiza-
tions. Adam optimizer is selected during the back propagation
process with leaning rate of 0.00005. At each iteration, the
model is trained with 64 mini batch size. Table I summarizes
the training parameters.
Fig. 4 shows the spectral efficiency performance comparison
with NtRF = NrRF = 2 chains. It can be seen from Fig. 4
that the spectral efficiency of the proposed CNN-based hybrid
precoding model, PE and MO algorithms are almost the same
as the FD precoding in the case of Ns = 1, and are almost the
same as PE and MO algorithms and within a small gap from
the FD precoding in the case of Ns = 2 data streams. This
reveals that the proposed hybrid precoder (combiner) model
can more accurately approximate the optimal FD precoder
(combiner).
To explore the performance of the proposed hybrid pre-
coding model at larger input and output dimensions, Fig. 5
shows the spectral efficiency performance comparison with
NtRF = NrRF = 4 chains. It can be noticed from Fig. 5 that
the spectral efficiency performance of the proposed hybrid
precoding model is almost the same as PE algorithm and
within a small gap from MO algorithm and FD precoding
in the case of Ns = 2, and within a small gap from the FD
precoding, PE and MO algorithms in the case of Ns = 4 data
streams.
In addition, the estimation time comparison of MO and
PE algorithms with the proposed DL-based hybrid precoding
model for NtRF = NrRF chains which equal to Ns = 2 and 4
multiplexed data streams and 1,000 realizations is shown in
Fig. 6. It can be noticed from Fig. 6 that MO algorithm takes
a long time to estimate the hybrid precoders and combiners
compared with PE algorithm and the proposed hybrid precod-
ing model, and the latter is almost 19 times faster than PE
algorithm.
V. C ONCLUSIONS
A novel CNN-based hybrid precoding model with GMD
algorithm is proposed in this paper. The main purpose of im-
plementing this proposed model in mmWave massive MIMO
is that the conventional precoding methods are deployed
numerically with high computational complexity. DL methods
have been introduced to overcome this issue by generating
the optimal hybrid precoders directly from a given optimal
 25 104
MO
FD
PE
Spectral efficiency (bits/s/Hz)

MO Proposed
20 PE 103
Proposed
Ns = 2
15 102

Time (sec)
10 101

5 Ns = 1 100

0 10-1
-30 -25 -20 -15 -10 -5 0 5 10 2 4
SNR (dB) RF Chains

Fig. 4. Spectral efficiency performance versus SNR of different precoding Fig. 6. Estimation time of different precoding algorithms with 1,000 realiza-
algorithms with NtRF = NrRF = 2. tions and NtRF = NrRF = Ns = 2 and 4.

40 [5] O. E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R.W. Heath,

FD “Spatially sparse precoding in millimeter wave MIMO systems,” IEEE
35
Spectral efficiency (bits/s/Hz)

MO Trans. Wireless Commun., vol. 13, no. 3, pp. 1499–1513, 2014.

30 PE
Proposed
25
20
15
10
5 [9] Y. Wang, M. Liu, J. Yang and G. Gui, ”Data-driven deep learning for
0 Veh. Technol., vol. 68, no. 4, pp. 4074-4077, April 2019.
-30 -25 -20 -15 -10
SNR (dB)
Fig. 5. Spectral efficiency performance versus SNR of different precoding
algorithms with NtRF = NrRF = 4.
FD precoder. The precoder and combiner models trained
with decoupled precoding and combining design problems,
respectively, with addition term to satisfy the orthogonality in
the estimated precoder. The spectral efficiency performance of
the proposed DL-based hybrid precoding model is compared
to FD precoder, PE and MO algorithms for different numbers
of RF chains and multiplexed data streams. The results showed
that the proposed hybrid precoding model can more accurately
approximate the performance of FD precoding with lower
time consumption than PE and MO algorithms. Our future
studies will focus on integrating intelligent reflecting surfaces
technology into the proposed system model.
R EFERENCES
[1] Z. Pi and F. Khan, “An introduction to millimeter-wave mobile broad-
band systems,” IEEE Commun. Mag., vol. 49, no. 6, pp. 101–107, 2011.
[2] G. R. MacCartney, M. K. Samimi, and T. S. Rappaport, “Exploiting di-
rectionality for millimeter-wave wireless system improvement,” in 2015
IEEE Int. Conf. Commun. (ICC), London, UK, 2015, pp. 2416–2422.
[3] T. L. Marzetta, E. G. Larsson, H. Yang, and H. Q. Ngo, Fundamentals
of Massive MIMO. Cambridge University Press, 2016.
[4] F. Sohrabi and W. Yu, “Hybrid digital and analog beamforming design
for large-scale antenna arrays,” IEEE J. Sel. Topics Signal Process., vol.
10, no. 3, pp. 501–513, 2016.
[6] X. Yu, J.-C. Shen, J. Zhang, and K. B. Letaief, “Alternating minimization
algorithms for hybrid precoding in millimeter wave MIMO systems,”
Ns = 4 IEEE J. Sel. Topics Signal Process., vol. 10, no. 3, pp. 485–500, 2016.
[7] Y. Jiang, W. W. Hager, and J. Li, “The geometric mean decomposition,”
Linear Algebra and its Applications, vol. 396, pp. 373-384, Feb. 2005.
[8] M. Liu, T. Song and G. Gui, ”Deep cognitive perspective: Resource
allocation for NOMA-based heterogeneous IoT with imperfect SIC,” in
IEEE Internet of Things Journal, vol. 6, no. 2, pp. 2885-2894, April
Ns = 2
2019.
automatic modulation recognition in cognitive radios,” in IEEE Trans.
-5 0 5 10
[10] H. Huang, J. Yang, H. Huang, Y. Song and G. Gui, ”Deep learning for
super-resolution channel estimation and DOA estimation based massive
MIMO system,” in IEEE Trans. Veh. Technol., vol. 67, no. 9, pp. 8549-
8560, Sept. 2018.
[11] H. Huang, Y. Song, J. Yang, G. Gui and F. Adachi, ”Deep-learning-
based millimeter-wave massive MIMO for hybrid precoding,” in IEEE
Trans. Veh. Technol., vol. 68, no. 3, pp. 3027-3032, March 2019.
[12] A. M. Elbir, ”CNN-based precoder and combiner design in mmWave
MIMO systems,” in IEEE Commun. Lett., vol. 23, no. 7, pp. 1240-1243,
July 2019.
[13] P. Dong, H. Zhang and G. Y. Li, ”Framework on deep learning-based
joint hybrid processing for mmWave massive MIMO systems,” in IEEE
Access, vol. 8, pp. 106023-106035, 2020.
[14] Q. Wang, K. Feng, X. Li and S. Jin, ”PrecoderNet: Hybrid beamforming
for millimeter wave systems with deep reinforcement learning,” in IEEE
Wireless Commun. Lett., vol. 9, no. 10, pp. 1677-1681, Oct. 2020.
[15] J. Chen et al., ”Hybrid beamforming/combining for millimeter wave
MIMO: A machine learning approach,” in IEEE Trans. Veh. Technol.,
vol. 69, no. 10, pp. 11353-11368, Oct. 2020.
[16] H. Jiang, Y. Lu, X. Li, B. Wang, Y. Zhou and L. Dai, ”Attention-based
hybrid precoding for mmWave MIMO systems,” 2021 IEEE Info. Theo.
Workshop (ITW), Kanazawa, Japan, Oct. 2021, pp. 1-6.
[17] A. Goldsmith, S. A. Jafar, N. Jindal and S. Vishwanath, ”Capacity limits
of MIMO channels,” in IEEE J. Sel. Areas Commun., vol. 21, no. 5,
pp. 684-702, June 2003.
[18] T. Y. Elganimi and F. F. Alfitouri, ”Norm-based user selection algo-
rithm for single-RF space modulation techniques with geometric mean
decomposition based precoding scheme,” SN Appl. Sci., vol. 3, no, 1,
pp. 1-13, 2021.
[19] T. Y. Elganimi and A. A. Aturki, ”Joint user selection and GMD-based
hybrid beamforming for generalized spatial modulation aided millimeter-
wave massive MIMO systems,” in the Proceedings of the 2020 IEEE 3rd
Int. Conf. Info. Commun. Signal Process. (ICICSP), Shanghai, China,
September 12–15, 2020, pp. 364–369.