The Role of Deep Learning in NOMA for 5G and Beyond
Communications
Moh. Khalid Hasan*, Md. Shahjalal, Md. Mainul Islam, Md. Morshed Alam, Md. Faisal Ahmed, and Yeong Min Jang**
Dept. of Electronics Engineering, Kookmin University, Seoul, South Korea
Email: *khalidrahman45@ieee.org, **yjang@kookmin.ac.kr
Abstract— In the coming future, it is obvious that the wireless
networks will be congested with massive amounts of data traffic
with the increasing number of users. Current multiple access
techniques will certainly not have the capability to efficiently serve
in the massively congested scenarios. In recent times, non-
orthogonal multiple access (NOMA) has been recognized as an
immensely potential technique for 5G and beyond
communications that can increase spectral efficiency to a greater
extent serving a vast number of users. However, several
circumscriptions are observed in NOMA such as the requirement
of a perfect channel state information in the transmitter and high
computational complexity in the receiver. The use of deep learning
(DL) techniques is a great resolution to deal with the challenges.
This paper discusses the applications of the DL methods in NOMA
for 5G and beyond communications. Firstly, the deep neural
networks that are employed in NOMA are listed up and discussed.
After that, their functions are studied specifically focusing on how
to improve the NOMA performance. Finally, the possible future
challenges and research issues are identified at the end of the
paper.
Index terms— Non-orthogonal multiple access (NOMA), deep
learning (DL), and 5G.
I. INTRODUCTION
Next-generation wireless networks will require an
extensive capacity to support an immense amount of advanced
multimedia applications [1, 2]. The fifth-generation (5G)
technologies face significant challenges with the explosive
growth of internet-of-thing-based large scale heterogeneous
networks. The uplink/downlink transmission of future massive
incremental data to a huge number of users will face numerous
hindrances using the current multiple-access techniques due to
their incapability to support efficient spectrum usage [3, 4]. The
necessity of a new method for 5G and beyond communication
in handling a large quantity of data is realized by researchers
over the last few years [5].
Non-orthogonal multiple access (NOMA) is a promising
technique that has drawn significant research interest in recent
years due to its potential applications in 5G and beyond
networks [5, 6]. It brings numerous advantages to the current
wireless networks such as an increase in spectral efficiency,
reliability, and connectivity reducing overall latency. NOMA
exploits the existing resources in a more efficient basis than the
conventional multiple access techniques [7, 8].
978-1-7281-4985-1/20/$31.00 ©2020 IEEE
In NOMA, a multiuser signal is multiplexed using
superposition coding in the transmitter and then sent to the users
using different power levels in a non-orthogonal basis. Usually,
the users with bad channel conditions are allocated higher
power than the users with good channel conditions. Therefore,
having proper channel state information (CSI) in the transmitter
has a profound significance in NOMA. When the signal is
collected in the receiver, the user with the stronger channel gain
retrieves its signal immediately. On the other hand, the user
with the weaker channel recognizes the other signals as
interference and performs successive interference cancellation
(SIC) technique to retrieve its original signal. In the
conventional orthogonal multiple access systems, the
orthogonality between channels is achieved using guard periods
to avoid interference, which particularly decreases the spectral
efficiency to a great extent. This problem is completely
remediated using NOMA, where no guard period is used and so
signals overlap with each other. NOMA provides fantastic
throughput with maintaining an excellent level of user fairness
[9].
In terms of theoretical perspective, SIC can be considered
as an optimal scheme for both uplink and downlink
communication [10]. With the perfect SIC, the performance of
NOMA is superior compared to the existing OFDMA scheme
[11]. Perfect CSI is very crucial for performing SIC in the
receiver. It is pretty difficult to design an efficient power
allocation scheme without having the exact CSI at the
transmitter. Acquiring a perfect or near-perfect CSI is a
challenging task. Fortunately, deep learning (DL) techniques
are already being proved to be a useful resolution to this
problem. It can be incorporated in NOMA to detect a
completely unknown and sharply changing channel condition.
In this paper, we enlist the functions of DL to improve the
performance of NOMA systems. We first provide a short
overview of the DL methods that are useful in NOMA and other
wireless networking applications. Afterward, we specifically
discuss the applications of DL in NOMA. Finally, we provide
a short overview of the challenges that can be observed using
DL methods in NOMA.
II. DEEP LEARNING IN NOMA
The current improvements in DL algorithms have brought
significant benefits to the area of wireless communications.
303 ICAIIC 2020
 Input layers Encoder layer
x xˆ ( H , P, yi )
Hidden layers
Fig. 1. DL-based NOMA communication architecture.
Data in wireless systems, such as CSI, are increasingly
heterogeneous and display intricate correlations [12]. Efficient
training of these data can exhibit better performance. In NOMA,
the employment of DL is extensively researched over the last
few years. Different kinds of learning algorithms are utilized in
different aspects and demonstrate better performance compared
to traditional schemes. In general, three types of learning
methods are developed in the literature, such as supervised,
unsupervised, and reinforcement. Supervised learning is
extensively utilized to estimate the CSI. In addition, it can also
be used in other applications such as spectrum sensing,
localization, and throughput prediction. Unsupervised learning
is mostly used in user clustering and congestion control. Deep
reinforcement learning is a different type of DL method that is
considered to have immense significance in wireless
communications. Until now, the most recognized application of
reinforcement learning is resource management. It uses a
special type of learning algorithm, called Q learning, and
produces the output without having any preliminary knowledge
about the learning environment.
In NOMA, DL is employed in different applications. DL
can be extremely useful for complex data processing and to
acquire perfect CSI. Two types of training can be identified in
NOMA, such as online and offline training. A DL-based
NOMA model [13] is depicted in Fig. 1. The hidden layer is
employed for training and recognition [13]. Hidden layers are
particularly equipped with multiple neurons. In Fig. 1, one of
the layers is named as noise layer, where the processed signal
can be corrupted using additive-white Gaussian noise. The CSI
can be acquired automatically by extensive training of the input
data using existing channel models. Arbitrary sequences of the
input signals are extensively trained using CSI of different
environments collected from simulations. This type of training
is called offline training. In addition, the output signals can be
fed into the training data to accurately predict the channel
environment. Pilot signals can be utilized to carry the CSI in
Decoder layer
Noise layer Output layer
real-time and the input signals can be trained with the help of
that data. This type of learning is called online learning. A block
diagram of the NOMA system with the integration of both
online and offline training for the auto-detection of CSI is
depicted in Fig. 2. DL can also be used for estimating the
channel considering adverse weather. For example, in [14], a
DL technique, called long-short term memory (LSTM) is used
for detecting dynamically fluctuating channel. Herein, the
hidden layers can act as a memory for the network state, which
particularly permits the DL to save, recall, and process the
preceding complex data.
DL can also be employed in NOMA for other purposes as
follows.
a. Resource allocation
Scheduling can be very important in the cases where
NOMA subchannels cannot provide sufficient bandwidth to the
users. A proper resource allocation algorithm can maximize the
overall sum rate. Therefore, the performance of the NOMA
system greatly depends on the resource allocation strategies
[11]. DL is a useful method that can be applied to overcome the
resource allocation problem in the case of multiple users [15,
16]. In addition, dynamic resource allocation is a challenge for
a fast-changing channel condition. Optimizing energy
consumption with efficient resource allocation is a significant
research issue using NOMA. Intelligent learning methods are
extensively developed to deal with this challenge. In [17], a DL
method, called deep belief network, is proposed for resource
allocation by minimizing the total transmit power.
Reinforcement learning can also be useful in solving the
dynamic resource allocation problem [15]. In reinforcement
learning, a deep deterministic policy gradient is used for
deriving the resource allocation decision. In addition, the
subcarrier assignment decision is evaluated with the help of Q
learning.
304
 Transmitter Channel Receiver

P1
Signal 1 Signal 2
Signal 1 detection detection
Signal 1 Power
Fading
Modulation allocation ∑ AWGN
channel
Signal 2 deployment P1>P2
Signal 1 Subtraction of
Signal 2 detection signal 1
P2

Offline Real-time
Simulated data CSI attainment Online training
training channel data

Channel estimation using deep learning

Fig. 2. Block diagram of the NOMA system employing online and offline DL mechanisms.

b. Signal decoding neural networks employed in NOMA provides a high

computational cost. Although NOMA itself is a complex
The conventional SIC method has several disadvantages.
networking system, DL techniques have enhanced its
In particular, it is very difficult to apply the SIC method to
performance but also increases the design complexity at the
decode the data perfectly when the number of users is increased.
same time. Therefore, reducing the computational complexity
The SIC method is also affected by the propagation error. The
is a significant research issue for the future. Moreover, using a
deep neural network can be useful to recover a discrete
specific neural network, different researchers used different
sequence from an impaired signal utilizing signal classification
architecture. The number of layers and the quality and quantity
[18]. It is also possible to optimize the SIC method by using DL
of the training dataset is different for a different architecture,
[19]. In [20], an online learning detection method for NOMA
however, the optimized system is yet to be discovered.
uplink is proposed to enhance spectral efficiency.
In addition, some of the DL methods, such as deep
c. Signal constellation design reinforcement learning is employed in NOMA in very few
literature. It is envisioned that the reinforcement learning
Efficient constellation design can be another useful
mechanism can optimize the performance of wireless networks
application of DL in NOMA [21]. The constellation of multiple
to a greater extent in terms of throughput, latency, and resource
users is superimposed while transmitting the data. The main
management. Extensive employment of reinforcement learning
theory is the mapping of the input source bits into symbol
in future NOMA systems is an essential research challenge.
sequences using DL after applying modulation techniques. To
increase energy efficiency, a new layer, called the
IV. CONCLUSIONS
normalization layer is added to connect the transmitter [21].
Another DL method to optimize the constellation design is NOMA is recognized to have immense significance in 5G
discussed in [22] based on autoencoders. The developed and beyond communication systems for its high spectral
mechanism is useful for multi-user access under various efficiency and low-latency characteristics. DL can enhance its
channel environments. performance to a considerable extent. In this study, the specific
roles of the DL methods in different NOMA applications are
DL can also be used in other applications in NOMA, such
briefly discussed. The concept of how DL techniques improve
as subchannel matching [23], power minimization [17], and
NOMA performance is explained. In addition, the specific DL
latency optimization [24]. The main applications of DL in
methods that are used in literature are listed up and their
NOMA are summarized in Table 1.
functions are elaborated. Finally, a short discussion on the
possible future research issues is conducted.
III. CHALLENGES AND FUTURE ISSUES
DL methods are widely employed in wireless ACKNOWLEDGMENT
communications to improve performance. Extensive
This research was financially supported by the Ministry of
development of DL in NOMA is already observed in recent
Trade, Industry and Energy(MOTIE) and Korea Institute for
times. However, there are still some challenges that are yet to
Advancement of Technology(KIAT) through the
be resolved by researchers. The efficient design of the core
InternationalCooperative R&D program.
neural networks is still a headache. Most of the current deep

305
 Table 1: Applications of DL-methods in NOMA
Reference Type of learning Deep learning model Application
[11] Supervised Recurrent neural network Rapid and optimized resource allocation
[14] [13] Supervised LSTM Channel estimation
[15] Reinforcement Deep Q-network Subcarrier assignment and power allocation
[16] Reinforcement Attention-based neural network Joint power allocation and channel assignment
[17] Supervised and Deep belief network Power optimization
unsupervised
[18] Supervised Deep neural network Automatic realization of the CSI
[21] Unsupervised Deep neural network Signal constellation design
[25] Supervised Deep neural network Reliability improvement of grant-free access
[26] Supervised learning Deep neural network Optimization for energy-efficient scheduling
[27] Reinforcement Deep neural network [27] based on Power allocation optimization
Q-learning [28]

[8] A. Li, Y. Lan, X. Chen, and H. Jiang, “Non-orthogonal multiple

REFERENCES
[1] M. G. Kibria, K. Nguyen, G. P. Villardi, O. Zhao, K. Ishizu, and
F. Kojima, “Big data analytics, machine learning, and artificial
intelligence in next-generation wireless networks,” IEEE
Access, vol. 6, pp. 32328-32338, 2018.
[2] W. H. Chin, Z. Fan, and R. Haines, “Emerging technologies and
research challenges for 5G wireless networks,” IEEE Wireless
Communications, vol. 21, no. 2, pp. 106-112, Apr. 2014.
[3] L. Dai, B. Wang, Y. Yuan, S. Han, C. I, and Z. Wang, “Non-
orthogonal multiple access for 5G: solutions, challenges,
opportunities, and future research trends,” IEEE
Communications Magazine, vol. 53, no. 9, pp. 74-81, Sep. 2015.
[4] M. K. Hasan, N. T. Le, M. Shahjalal, M. Z. Chowdhury, and Y.
M. Jang, “Simultaneous data transmission using multilevel LED
in hybrid OCC/LiFi system: concept and demonstration,” IEEE
Communications Letters, vol. 23, no. 12, pp. 2296-2300, Dec.
2019.
[5] S. M. R. Islam, N. Avazov, O. A. Dobre, and K. Kwak, “Power-
domain non-orthogonal multiple access (NOMA) in 5G
systems: potentials and challenges,” IEEE Communications
Surveys & Tutorials, vol. 19, no. 2, pp. 721-742, 2017.
[6] W. Shin, M. Vaezi, B. Lee, D. J. Love, J. Lee, and H. V. Poor,
“Non-orthogonal multiple access in multi-cell networks: theory,
performance, and practical challenges,” IEEE Communications
Magazine, vol. 55, no. 10, pp. 176-183, Oct. 2017.
[7] Z. Ding, Z. Yang, P. Fan, and H. V. Poor, “On the performance
of non-orthogonal multiple access in 5G systems with randomly
deployed users,” IEEE Signal Processing Letters, vol. 21, no.
12, pp. 1501-1505, Dec. 2014.
access (NOMA) for future downlink radio access of 5G,” China
Communications, vol. 12, pp. 28-37, Dec. 2015.
[9] M. Aldababsa, M. Toka, S. Gökçeli, G. K. Kurt, and O. Kucur1,
“A tutorial on nonorthogonal multiple access for 5G and
beyond,” wireless communications and mobile computing, vol.
2018, 2018.
[10] K. Higuchi and A. Benjebbour, “Non-orthogonal multiple
access (NOMA) with successive interference cancellation for
future radio access,” IEICE TRANSACTIONS on
Communications, vol. 98, pp. 403-414, 2015.
[11] M. Liu, T. Song, and G. Gui, “Deep cognitive perspective:
resource allocation for NOMA-Based heterogeneous IoT with
imperfect SIC,” IEEE Internet of Things Journal, vol. 6, no. 2,
pp. 2885-2894, Apr. 2019.
[12] M. A. Alsheikh, D. Niyato, S. Lin, H. Tan, and Z. Han, “Mobile
big data analytics using deep learning and apache spark,” IEEE
network, vol. 30, no. 3, p. 22–29, 2016.
[13] G. Gui, H. Huang, Y. Song, and H. Sari, “Deep learning for an
effective nonorthogonal multiple access scheme,” IEEE
Transactions on Vehicular Technology, vol. 67, no. 9, pp. 8440-
8450, Sep. 2018.
[14] M. AbdelMoniem, S. M. Gasser, M. S. El-Mahallawy, M. W.
Fakhr, and A. Soliman, “Enhanced NOMA system using
adaptive coding and modulation based on LSTM neural network
channel estimation,” Applied Sciences, vol. 9, no. 15, 2019.
[15] Y. Zhang, X. Wang, and Y. Xu, “Energy-efficient resource
allocation in uplink noma systems with deep reinforcement
learning,” in Proc. International Conference on Wireless
Communications and Signal Processing (WCSP), Xi'an, China,
2019.

306
[16] C. He, Y. Hu, Y. Chen, and B. Zeng, “Joint power allocation and
channel assignment for NOMA with deep reinforcement
learning,” IEEE Journal on Selected Areas in Communications,
vol. 37, no. 10, pp. 2200-2210, Oct. 2019.
[17] J. Luo, J. Tang, D. K. C. So, G. Chen, K. Cumanan, and J. A.
Chambers, “A deep learning-based approach to power
minimization in multi-carrier NOMA with SWIPT,” IEEE
Access, vol. 7, pp. 17450-17460, 2019.
[18] C. Lin, Q. Chang, and X. Li, “A Deep Learning Approach for
MIMO-NOMA Downlink Signal Detection,” Sensors, vol. 19,
2019.
[19] J. Kang, I. Kim, and C. Chun, “Deep learning-based MIMO-
NOMA with imperfect SIC decoding,” IEEE Systems Journal,
pp. 1-4.
[20] D. A. Awan, R. L.G. Cavalcante, M. Yukawa, and S. Stanczak,
“Detection for 5G-NOMA: an online adaptive machine learning
approach," arXiv:1711.00355.
[21] L. Jiang, X. Li, N. Ye, and A. Wang, “Deep learning-aided
constellation design for downlink NOMA,” in Proc.
International Wireless Communications & Mobile Computing
Conference (IWCMC), Tangier, Morocco, 2019.
[22] F. Alberge, “Constellation design with deep learning for
downlink non-orthogonal multiple access,” in Proc. IEEE
Annual International Symposium on Personal, Indoor and
Mobile Radio Communications (PIMRC), Bologna, 2018.
[23] Y. Liu, H. Zhang, K. Long, A. Nallanathan, and V. C. M. Leung,
“Energy-efficient subchannel matching and power allocation in
NOMA autonomous driving vehicular networks,” IEEE
Wireless Communications, vol. 26, no. 4, pp. 88-93, Aug. 2019.
[24] P. Yang, L. Li, W. Liang, H. Zhang, and Z. Ding, “Latency
optimization for multi-user NOMA-MEC offloading using
reinforcement learning,” in Proc. Wireless and Optical
Communications Conference (WOCC), Beijing, China, 2019.
[25] N. Ye, X. Li, H. Yu, A. Wang, W. Liu, and X. Hou, “Deep
learning aided grant-free NOMA toward reliable low-latency
access in tactile internet of things,” IEEE Transactions on
Industrial Informatics, vol. 15, no. 5, pp. 2995-3005, May 2019.
[26] L. Lei et al., “Learning-assisted optimization for energy-
efficient scheduling in deadline-aware NOMA systems,” IEEE
Transactions on Green Communications and Networking, vol.
3, no. 3, pp. 615-627, Sep. 2019.
[27] K. N. Doan, M. Vaezi, W. Shin, H. V. Poor, H. Shin, and Tony
Q. S. Quek, “Power allocation in cache-aided NOMA systems:
optimization and deep reinforcement learning approaches,”
arXiv:1909.11074.
[28] L. Xiao, Y. Li, C. Dai, H. Dai, and H. V. Poor, “Reinforcement
learning-based NOMA power allocation in the presence of smart
jamming,” IEEE Transactions on Vehicular Technology, vol.
67, no. 4, pp. 3377-3389, Apr. 2018.

307