Professional Documents
Culture Documents
net/publication/329495369
CITATIONS READS
8 4,361
2 authors, including:
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Manuel Eugenio Morocho-Cayamcela on 29 November 2019.
861
Balevi et al. [21], incorporated fog networking into a summary of the potential applications of unsupervised learning
heterogeneous cellular networks and used an unsupervised soft- in 5G wireless communication technologies.
clustering algorithm to locate the fog nodes that are upgraded
from low power nodes (LPNs) to high power nodes (HPNs). The IV. R EINFORCEMENT LEARNING IN 5G M OBILE AND
authors showed that by applying machine learning clustering to WIRELESS COMMUNICATIONS TECHNOLOGY
a priori known data like the number of fog nodes and location The philosophy of Reinforcement Learning scheme is based
of all LPNs within a cell, they were able to determine a clustering on a learning system often called agent, that reacts to the
configuration that reduced latency in the network. The latency environment. The agent performs actions and gets rewards or
calculation was performed with open- loop communications, penalties (negative rewards) in return for its actions. That means
with no ACK for transmitted packets, and compared to the that the agent has to learn by itself creating a policy that defines
Voronoi tessellation model, a classical model based on Euclidean the action that the agent should choose in a certain situation. The
distance. aim of the reinforcement-learning task is to maximize the
A typical unsupervised learning technique is K-means aforementioned reward over time.
clustering; numerous authors have investigated the applications Resource allocation in Long Term Evolution (LTE) net- works has
of this particular clustering technique in the next generation been a dilemma since the technology was introduced. To
wireless network system. Sobabe et al. [22], proposed a overcome the wireless spectrum scarcity in 5G, novel deep
cooperative spectrum-sensing algorithm using a combination of learning approaches that account the coexistence of LTE and
an optimized version of K-means clustering, Gaussian mixture LTE-Unlicensed, to model the resource allocation problem in
model and expectation maximization (EM) algorithm. They LTE-U small base stations (SBS), has been introduced in [26].
proved that their learning algorithm outperformed the energy To accomplish their contribution, the authors introduced a
vector-based algorithm. Song et al. [23], discussed how K-means reinforcement-learning algorithm based on long short-term
clustering algorithm and its classification capabilities can aid in memory (RL-LSTM) cells to allocate pro- actively the resources
selecting an efficient relay selection among urban vehicular of LTE-U over the unlicensed spectrum. The problem formulated
networks. The authors investigated the methods for multi-hop reassembles a non-cooperative game between the SBSs, where a
wireless broadcast and how K-means is a key factor in the RL-LSTM framework enables the SBSs to learn automatically
decision-making and learning steps of the base stations, that which of the unlicensed channels to use, based on the probability
learn from the distribution of the devices and chooses of future changes in terms of the WLAN activity and the LTE-U
automatically which are the most suitable devices to use as a traffic loads of the unlicensed channels. This work takes into
relay. account the value of LTE-U as a proposal that allows cellular
When a wireless network experience unusual traffic demand network operators to offload some of their data traffic, and the
at a particular time and location, it is often called an anomaly, connotation of AI in the form of RL-LSTM, as a promising
To help identify these anomalies, Parwez et al. [24], used mobile solution to long-term dependencies learning, sequence, and time-
network data for anomaly detection purposes, with the help of series problems. Nevertheless, researchers should be warned that
hierarchical clustering to identify this kind of inconsistencies. this deep learning architecture is one of the most difficult to train,
The authors claim that the detection of this data deviations helps due to the vanishing and exploding gradient problem in Recurrent
to establish regions of interest in the network that require special Neural Networks (RNN) [27], the speed of activation functions,
actions, such as resource allocation, or fault avoidance as well as the initialization of parameters for LSTM systems [28].
solutions. Reinforcement Learning has also played an important role on
Ultra-dense small cells (UDSC) is expected to increase the Heterogeneous Networks (HetNets), enabling Femto Cells (FCs)
capacity of the network, spectrum and energy efficiency. To to autonomously and opportunistically sense the radio
consider the effects of cell switching, dynamic interference, environment and tune their parameters accordingly to satisfy
time-varying user density, dynamic traffic patterns, and specific pre-set quality-of-service requirements. Alnwaimi et al.
changing frequencies, Wang et al. [25], proposed a data-driven [29], proved that by using reinforcement learning for the
resource management for UDSC using Affinity Propagation, an femtocells self-configuration, based on dynamic-learning games
unsupervised learning clustering approach, to perform data for a multi-objective fully-distributed strategy, the intra/inter-tier
analysis and extract the knowledge and behavior of the system interference can be reduced significantly. The collision and
under complex environments. Later they introduced a power reconfiguration measurements were used as a "learning cost
control and channel management system based on the results of during training. This self-organizing potential, empower FCs to
the unsupervised learning algorithm. They conclude their identify available spectrum for opportunistic use, based on the
research stating that by means of simulation, their data-driven learned parameters.
resource management framework significantly improved the
efficiency of the energy and throughput in UDSC.
Alternative clustering models like K-Means, Mini- Batch K-
Means, Mean-Shift clustering, DBSCAN, Agglomerative
Clustering, etc., can be used to associate the users to a certain
base station in order to optimize the user equipment (UE) and
base stations (BS) transmitting/receiving power. Table II, shows
862
TABLE I
SUMMARY OF SUPERVISED LEARNING-BASED SCHEMES FOR 5G MOBILE AND WIRELESS COMMUNICATIONS TECHNOLOGY.
TABLE II
SUMMARY OF UNSUPERVISED LEARNING-BASED SCHEMES FOR 5G MOBILE AND WIRELESS COMMUNICATIONS TECHNOLOGY.
TABLE III
SUMMARY OF REINFORCEMENT LEARNING-BASED SCHEMES FOR 5G MOBILE AND WIRELESS COMMUNICATIONS TECHNOLOGY.
863
Intelligence becomes crucial in optimizing this kind of McGrawHill-MachineLearning-TomMitchell.pdf
problems in the network. An additional goal of linking AI with [8] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, 1st
ed.,
5G networks would be to obtain significant improvements in the T. Dietterich, Ed. London, England: The MIT Press, 2016. [Online].
context of edge caching just by applying off-the-shelf machine Available: www.deeplearningbook.org
learning algorithms. We have shown how AI can be a solution [9] P. U. Stanford Vision Lab, Stanford University, “ImageNet.”
that can fill this gap of requirements in 5G mobile and wireless [Online].
Available: http://image-net.org/about-overview
communications, allowing base stations to predict what kind of [10] B. Bojović, E. Meshkova, N. Baldo, J. Riihijärvi, and M.
content users nearby may request in the near future, allocating Petrova, “Machine learning-based dynamic frequency and
dynamic frequencies in self-organized LTE dense small cell bandwidth allocation in self-organized LTE dense small cell
deployments, predicting path loss/link budget with deployments,” Eurasip Journal on Wireless Communications and
Networking, vol. 2016, no. 1, 2016. [Online]. Available:
approximated NN models, inferring the unobservable channel https://jwcn-eurasipjournals.springeropen.
state information from an observable channel, adjusting the com/track/pdf/10.1186/s13638-016-0679-0
TDD uplink-downlink configuration in XG-PON-LTE systems [11] S. Y. Han, N. B. Abu-ghazaleh, and S. Member, “Efficient and Con-
based on ongoing network conditions, sensing the spectrum sistent Path loss Model for Mobile Network Simulation,”
using unsupervised models, reducing the latency by IEEE/ACM Transactions on Networking, vol. PP, no. 99, pp. 1–1,
2015.
automatically configuring the clusters in Het-Nets, detecting [12] R. D. a. Timoteo, D. C. Cunha, and G. D. C. Cavalcanti, “A Proposal
anomalies/faults/intrusions in mobile wireless networks, for Path Loss Prediction in Urban Environments using Support
managing the resources in ultra-dense small cells, selecting the Vector Re- gression,” Advanced International Conference on
relay nodes in vehicular networks, allocating the resources Telecommunications, vol. 10, no. c, pp. 119–124, 2014.
[13] J. Liu, R. Deng, S. Zhou, and Z. Niu, “Seeing the unobservable:
in LTE-U networks, enabling autonomous and opportunistic Channel learning for wireless communication networks,” 2015
sensing of the radio environment in femto-cells, selecting the IEEE Global Communications Conference, GLOBECOM 2015,
optimal radio access technology (RAT) in HetNets, among 2015.
others. [14] P. Sarigiannidis, A. Sarigiannidis, I. Moscholios, and P.
Zwierzykowski, “DIANA: A Machine Learning Mechanism for
Adjusting the TDD Uplink-Downlink Configuration in XG-PON-
ACKNOWLEDGMENT LTE Systems,” Mobile Information Systems, vol. 2017, no. c, 2017.
This work was supported by the Global Excellent Technology [15] S. P. Sotiroudis, S. K. Goudos, K. A. Gotsis, K. Siakavara, J. N.
Innovation Program (10063078) funded by the Ministry of Sahalos, and L. Fellow, “Application of a Composite Differential
Evolution Algorithm in Optimal Neural Network Design for
Trade, Industry and Energy (MOTIE) of Korea; and by the Propagation Path-Loss Prediction in Mobile Communication
National Research Foundation of Korea (NRF) grant funded by Systems,” IEEE ANTENNAS AND WIRELESS PROPAGATION
the Korea government (MSIP; Ministry of Science, ICT & LETTERS, vol. 12, pp. 364–367, 2013.
Future Planning) (No. 2017R1C1B5016837). [16] J. M. Mom, C. O. Mgbe, and G. A. Igwue, “Application of Artificial
Neural Network For Path Loss Prediction In Urban Macrocellular
Envi- ronment,” American Journal of Engineering Research
(AJER), vol. 03, no. 02, pp. 270–275, 2014.
REFERENCES [17] I. Popescu, D. Nikitopoulos, I. Nafornita, and P. Constantinou,
“ANN prediction models for indoor environment,” IEEE
[1] A. Geron, “Hands-on machine learning with Scikit-Learn and
International Confer- ence on Wireless and Mobile Computing,
TensorFlow: concepts, tools, and techniques to build intelligent
Networking and Communica- tions 2006, WiMob 2006, pp. 366–
systems,” p. 543, 2017. [Online]. Available:
371, 2006.
http://shop.oreilly.com/ product/0636920052289.do
[18] S. P. Sotiroudis, K. Siakavara, and J. N. Sahalos, “A Neural Network
[2] A. Osseiran, J. F. Monserrat, and P. Marsch, 5G Mobile and Wireless
Approach to the Prediction of the Propagation Path-loss for Mobile
Communications Technology, 1st ed. United Kingdom: Cambridge
Communications Systems in Urban Environments,” PIERS Online,
University Press, 2017. [Online]. Available: www.cambridge.org/
vol. 3, no. 8, pp. 1175–1179, 2007.
9781107130098
[3] R. Li, Z. Zhao, X. Zhou, G. Ding, Y. Chen, Z. Wang, and H. Zhang, [19] M. Ayadi, A. Ben Zineb, and S. Tabbane, “A UHF Path Loss Model
“Intelligent 5G: When Cellular Networks Meet Artificial Us- ing Learning Machine for Heterogeneous Networks,” IEEE
Intelligence,” IEEE Wireless Communications, vol. 24, no. 5, pp. Transactions on Antennas and Propagation, vol. 65, no. 7, pp.
175–183, 2017. [Online]. Available: 3675–3683, 2017.
http://www.rongpeng.info/files/Paper_wcm2016.pdf [20] International Telecommunication Union, “A path-specific
[4] T. E. Bogale, X. Wang, and L. B. Le, “Machine Intelligence propagation prediction method for point-to-area terrestrial services
Techniques for Next-Generation Context-Aware Wireless in the VHF and UHF bands,” ITU P-Series Radiowave propagation,
Networks,” ITU Special Issue: The impact of Artificial Intelligence no. P.1812-4, pp. 1–35, 2015. [Online]. Available:
(AI) on communication networks and services., vol. 1, 2018. https://www.itu.int/dms_pubrec/itu-r/ rec/p/R-REC-P.1812-4-
[Online]. Available: 201507-I!!PDF-E.pdf
https://arxiv.org/pdf/1801.04223.pdfhttp://arxiv.org/abs/1801.0422 [21] E. Balevi and R. D. Gitlin, “Unsupervised machine learning in 5G
3 networks for low latency communications,” 2017 IEEE 36th
[5] C. Jiang, H. Zhang, Y. Ren, Z. Han, K. C. Chen, and L. Hanzo, International Performance Computing and Communications
“Machine Learning Paradigms for Next-Generation Wireless Conference, IPCCC 2017, vol. 2018-Janua, pp. 1–2, 2018.
Networks,” IEEE Wireless Communications, 2017. [22] G. Sobabe, Y. Song, X. Bai, and B. Guo, “A cooperative spectrum
[6] G. Villarrubia, J. F. De Paz, P. Chamoso, and F. D. la Prieta, sensing algorithm based on unsupervised learning,” 10th International
“Artificial neural networks used in optimization problems,” Congress on Image and Signal Processing, BioMedical Engineering
Neurocomputing, vol. 272, pp. 10–16, 2018. and Informatics (CISP-BMEI 2017), vol. 1, pp. 198–201, 2017.
[7] T. M. Mitchell, Machine Learning, 1st ed. McGraw- [23] W. Song, F. Zeng, J. Hu, Z. Wang, and X. Mao, “An Unsupervised-
Hill Science/Engineering/Math, 1997. [On- Learning-Based Method for Multi-Hop Wireless Broadcast Relay Se-
line]. Available: https://www.cs.ubbcluj.ro/~gabis/ml/ml-books/ lection in Urban Vehicular Networks,” IEEE Vehicular Technology
864
Conference, vol. 2017-June, 2017.
[24] M. S. Parwez, D. B. Rawat, and M. Garuba, “Big data analytics for
user-activity analysis and user-anomaly detection in mobile wireless
network,” IEEE Transactions on Industrial Informatics, vol. 13, no. 4,
pp. 2058–2065, 2017.
[25] L.-C. Wang and S. H. Cheng, “Data-Driven Resource Management
for Ultra-Dense Small Cells: An Affinity Propagation Clustering
Approach,” IEEE Transactions on Network Science and Engineering,
vol. 4697, no. c, pp. 1–1, 2018. [Online]. Available:
https://ieeexplore.ieee.org/document/8369148/
[26] U. Challita, L. Dong, and W. Saad, “Deep learning for proactive
resource allocation in LTE-U networks,” in European Wireless 2017-
23rd European Wireless Conference, 2017. [Online]. Available:
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8011311
[27] R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty
of training recurrent neural networks,” Tech. Rep., 2013.
[Online]. Available:
http://proceedings.mlr.press/v28/pascanu13.pdf?
spm=5176.100239.blogcont292826.13.57KVN0&file=pascanu13.pdf
[28] Q. V. Le, N. Jaitly, and G. E. Hinton Google, “A Simple Way to
Initialize Recurrent Networks of Rectified Linear Units,” Tech. Rep.,
2015. [Online]. Available: https://arxiv.org/pdf/1504.00941v2.pdf
[29] G. Alnwaimi, S. Vahid, and K. Moessner, “Dynamic heterogeneous
learning games for opportunistic access in LTE-based
macro/femtocell deployments,” IEEE Transactions on Wireless
Communications, vol. 14, no. 4, pp. 2294–2308, 2015.
[30] D. D. Nguyen, H. X. Nguyen, and L. B. White, “Reinforcement
Learning with Network-Assisted Feedback for Heterogeneous RAT
Selection,” IEEE Transactions on Wireless Communications, vol. 16,
no. 9, pp. 6062–6076, 2017.
865