2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)

A Machine Learning Algorithm for Unlicensed LTE

and WiFi Spectrum Sharing
Nazanin Rastegardoost and Bijan Jabbari
Department of Electrical and Computer Engineering, George Mason University, Fairfax, Virginia
Email: {nrastega,bjabbari}@gmu.edu

Abstract—Shared use of unlicensed spectrum in practice for

coexistence with WiFi is rather complex and to achieve optimum
usage can be highly challenging. While maximal utilization is
desirable for unlicensed LTE, it is essentially important not to
disturb WiFi activity in the unlicensed channel when designing
a coexistence scheme. Opportunistic exploitation of idle gaps, or
white spaces, in the WiFi channel for unlicensed LTE transmis-
sions enables achieving the above objectives. However, complex
analytical approaches to the opportunistic coexistence problem
require considerable computation and might result in excessive
latency, which would be undesirable. Machine learning schemes
may reduce the computational complexity and hence not only
reduce the latency but also help with energy consumption in
wireless communications systems.
We propose a novel algorithm based on reinforcement learning
technique for the problem of opportunistic coexistence of unli-
censed LTE and WiFi. The proposed approach in particular is
based on Q-Learning, which provides a robust and model-free
decision-making framework that enables online and distributive Fig. 1: Interference in coexisting LTE-U and WiFi networks.
coexistence of small cells with WiFi. Our approach takes into
account the latency imposed on WiFi activity by employing
carrier sensing at the base station, and aims to minimize it,
while maximizing unlicensed LTE utilization of the idle spectral
Multiple Access with Collision Avoidance (CSMA-CA) mech-
resources.
Index Terms—Unlicensed LTE, LTE-U, WiFi, ABS, coexis- anism to access the spectrum. Consequently, WiFi network
tence, white space, machine learning, 5G. performance will be pushed to the limit as a result of backing-
off for contiguous LTE-U transmissions. Therefore, in order
to protect WiFi from bandwidth starvation and performance
I. I NTRODUCTION
degradation, it is of essential importance to employ an efficient
Unlicensed LTE (for example, LTE-U used broadly for and fair coexistence mechanism for LTE-U and WiFi when
LTE in unlicensed spectrum) operating with small cells are coexisting in the unlicensed spectrum.
considered as a promising solution to increase the spectral Numerous coexistence mechanisms for LTE-U and WiFi
capacity towards the 5th -Generation (5G) wireless networks have been proposed in the literature. Many of the works
[1]. The available wide bandwidth, lightly loaded, and short that address their time domain coexistence, are based on
communication range channels in the unlicensed 5 GHz spec- muting or duty cycling. This approach is generally referred
trum, provide the opportunity for small cells to offload the to as Carrier Sensing Adaptive Transmission (CSAT), where
excess data traffic of the cellular networks by means of LTE LTE-U adaptively performs an ON/OFF duty cycle, and only
carrier aggregation [2]. However, these unlicensed bands are transmits during the ON period, leaving the OFF period for
currently occupied with incumbent technologies, such as WiFi, WiFi transmissions. Almost Blank Subframe allocation (ABS)
whose performance is shown to be seriously degraded by the is another similar approach, where a number of subframes
interference from coexisting legacy LTE operation in the same in each LTE frame are reserved for WiFi activity. The idea
unlicensed band, as illustrated in Fig. 1 [3]. was first introduced in [4], and further investigated in [1],
In addition, fundamental differences in the MAC layer where a fixed number of subframes are randomly allocated
functionality of LTE and WiFi, make their interference man- to ABS. Variations such as adaptive ABS and dynamic duty
agement and coexistence problem quite challenging. As op- cycle selection are proposed in the literature to enhance the
posed to WiFi, LTE is conventionally designed for exclusive coexistence performance of the two networks by taking into
and centrally-scheduled transmissions, without any need to account the WiFi network traffic load [5]–[8]. Although the
contend or coexist with other technologies. Whereas WiFi aforementioned works have effectively improved the LTE-U
nodes contend in a distributive fashion based on Carrier Sense and WiFi coexistence by providing a fair throughput balance,

978-1-5386-5191-9/18/$31.00 ©2018 IEEE

 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)
however, they fail to reflect the disturbance and the resulting
latency imposed on the WiFi activity due to LTE-U interrup-
tion in the ABS regime.
On the other hand, inefficient utilization of the spectrum has
resulted in extensive vacant portions known as white space,
containing nothing but white noise. White spaces in the unli-
censed spectrum caused by random activities of the incumbent
technologies, provide considerable spectral opportunities for
emerging technologies such as LTE-U, at no cost. Statistical
characterization of WiFi white spaces is carried out in [9]
using a Markov modulated batch Poisson process (MMBPP)
framework, where authors demonstrate the abundance and
duration of idle opportunities in the WiFi channel. The idea
of exploiting WiFi white spaces for opportunistic LTE-U
access is introduced in [10] and further investigated in [11],
where an MMBPP-based opportunistic coexistence mechanism
is proposed for LTE-U that enables the base station (BS)
to dynamically predict the duration of WiFi white spaces
upon sensing one, and schedule transmissions accordingly.
This approach is shown to achieve comparable throughput
to ABS, while significantly reducing the latency imposed on
WiFi activity.
However, the MMBPP-based approach for LTE-U coexis-
tence is practically difficult to implement, due to the parameter
estimation and computational complexities related to MMBPP
calculations that have to repeatedly be carried out upon every
white space occurrence. Moreover, the LTE-U probabilistic
operation is subject to error, and thus, can cause alteration
of the WiFi system statistics, which is not captured in the
model. To address the above issues, in this paper we propose
a practically simpler method to the problem of opportunistic
coexistence of LTE-U and WiFi. The proposed approach
is based on reinforcement learning, particularly Q-Learning,
which provides a robust and model-free decision-making
framework that enables online and distributive coexistence of
LTE-U small cells with WiFi. Q-Learning (QL) is employed
in [12] for a distributed channel selection strategy for LTE-
U, that dynamically identifies the utilization condition of each
channel towards optimum selection. In [6], [13] Q-Learning is
used for dynamic ABS duty cycle selection according to the
coexisting WiFi network traffic load. However, these works
ignore the interference and latency imposed on WiFi activity,
which is addressed in our proposed framework by employing
carrier sensing at the LTE-U BS.
The rest of the paper is organized as follows. Section II
provides an overview on the system structure. The Q-Learning
based opportunistic coexistence scheme is proposed in Section
III. Section IV provides simulation results. Finally, Sections
V concludes the paper.
II. S YSTEM OVERVIEW
A. Deployment Scenario
We focus on a co-channel coexistence scenario, where two
networks operate in a single unlicensed frequency channel; a
legacy WiFi network (with a single or multiple access points
(APs) and N user stations (STAs)), and a co-located LTE-U
small cell (with a single BS). Downlink (DL) and Uplink (UL)
traffic is assumed for the WiFi network, and the utilization of
the entire WiFi network, or its traffic intensity, is denoted by
ρ ∈ (0, 1).
Distributed CSMA-CA protocol is implemented among
WiFi nodes for channel access coordination based on a physi-
cal layer Clear Channel Assessment (CCA) and random back-
off mechanism. Particularly, if the WiFi receiver detects any
WiFi signal with decodable preamble that exceeds the Carrier
Sensing (CS) threshold, or any other signal exceeding the
Energy Detection (ED) threshold, it will hold a busy CCA flag
until the end of that transmission. As a result, WiFi nodes can
detect LTE-U transmissions using the ED feature.
We assume WiFi APs and STAs as well as the co-located
LTE-U BS and the associated User Equipments (UEs) are
distributed in the same geographical area such that they are all
within the CCA range of each other, i.e., their simultaneous
transmission results in severe interference, and no two nodes1
can transmit at the same time. This scenario may apply to
dense networks situations, where no other idle channels are
available and the two co-located operators have to share a
single channel. This assumption also helps with abstracting the
physical layer, which can later be improved using stochastic
geometry methods to account for spatial distributions and
randomness in the received SINR. Nevertheless, here we relax
the spatial dimension to emphasize on temporal design issues
from a MAC layer perspective.
Here, the LTE-U supplement downlink (SDL) mode is
considered, where the excess DL traffic is offloaded in the
unlicensed band, and the UL and control traffic remain in the
licensed bands, and thus, the BS is assumed to have full buffer
DL traffic. In accessing the shared unlicensed channel, LTE-U
BS also implements ED to sense WiFi transmissions and avoid
interrupting them. Once the access is granted, the BS centrally
schedules and allocates available resources for blocks of 1 ms
long (equal to one LTE subframe duration) among LTE UEs.
B. LTE-U Duty Cycle
As illustrated in Fig. 2a, the real-life WiFi traffic is sig-
nificantly bursty, leaving lots of white spaces between IEEE
802.11 frames [14]. White space is referred to the idle periods
in WiFi channel that correspond to no transmissions, excluding
the short silent gaps related to the back-off times and WiFi
protocol-related space intervals. We define a variable and adap-
tive duty cycle for LTE-U based on temporal characteristics
of WiFi traffic. As illustrated in Fig. 2b, which is an enlarged
view of Fig. 2a, the LTE-U duty cycle is comprised of an
OFF period, corresponding to WiFi frame transmission or
busy time, followed by an ON period, which relates to WiFi
white space or idle time. Idle and busy times in WiFi traffic
are naturally alternating, even though the busy times may be
comprised of consecutive frame transmissions (irrespective of
the transmitter node).
1 The term ”node” is used to refer to both service providers (AP/BS) and
users (STA/UE)
 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)
Fig. 2: (a) Traffic trace of a real-life WiFi network [14]. (b)
Closer look: WiFi cycle (variable LTE-U duty cycle), WiFi frame
transmission period (LTE-U OFF), and white space (LTE-U ON).
We assume LTE-U BS constantly performs carrier sensing
to avoid interrupting WiFi transmissions in the common chan-
nel while fully utilizing the idle resources. When the channel
is idle, the BS knows that there is a white space, and starts
the LTE-U ON period. Since our goal here is to minimize
the disturbance introduced to WiFi activity, the BS has to
stop transmission by the end of the white space, i.e. when
a packet arrives at the WiFi network. However, due to the
back-off mechanism of the CSMA-CA, the WiFi nodes will
not start transmitting as long as they sense the LTE signal.
Consequently, LTE-U will continue transmitting, and thus,
delaying WiFi traffic further and further.
To avoid this situation, in this paper we use Q-Learning to
provide the LTE-U BS with the ability to estimate the duration
of an ongoing white space at each duty cycle, such as to
vacate the channel for the upcoming WiFi frame transmission,
without requiring detailed knowledge of the WiFi system
parameters and statistics.
III. Q-L EARNING -BASED O PPORTUNISTIC C OEXISTENCE
M ECHANISM FOR LTE-U
Consider an LTE-U small cell sharing an unlicensed chan-
nel with a WiFi network in time domain. As discussed
previously, the WiFi network generates random and bursty
traffic in uplink and downlink directions, resulting in abundant
white spaces between WiFi frames. Here, we formulate a
decision-making learning-based approach to the opportunistic
coexistence problem of LTE-U with WiFi, where the LTE-
U BSs are reinforcement learning agents who autonomously
schedule transmissions while adapting their performance to the
coexisting WiFi network activity.
In the co-channel coexistence scenario, the LTE-U BS
performs carrier sensing to detect any WiFi activity in the
common unlicensed channel. As long as the channel is occu-
pied by WiFi transmissions, LTE-U remains in the OFF period
and continuously listens to the channel until it becomes idle,
or equivalently, a WiFi white space is initiated. Once a white
space is sensed, the BS starts the ON period and transmits
for a × tSf ms, where a ∈ A = {a1 , . . . , aM } is the integer
indicating the number of LTE subframes with duration of tSf ,
equal to 1 ms.
The BS uses Q-learning algorithm, discussed below, to
dynamically estimate the best value of a, and thus, the duration
of every LTE-U ON period, so that it can vacate the band for
upcoming WiFi transmissions such as to minimize the inter-
ference and latency imposed on WiFi activity. Consequently,
the BS adaptively adjusts its duty cycle and ON/OFF periods
according to WiFi activity such that opportunistic exploitation
of natural white spaces in WiFi channel is achieved.
A. Q-Learning Algorithm
Q-learning is an online model-free reinforcement learning
technique, where the learning agent improves its behavior
through interaction with the environment, and by estimating
the values of state-action pairs based on experience. In each
state, the agent is able to compare the expected utility of the
available actions and choose the best amongst them. The Q-
value, Q(s, a), is defined to be the expected discounted sum of
future payoffs achieved by taking action a in state s following
the optimal policy that minimizes the cumulative cost function,
that is, Q : S × A → R.
We consider the Q-learning algorithm with -greedy policy,
where the action at each step is selected as follows. A uniform
random number r ∈ U(0, 1) is generated first and compared
against the parameter  ∈ (0, 1). If r is smaller than , an action
is selected randomly. Otherwise, the optimal action with the
lowest Q-value is selected, i.e.,
at = arg min Q(s, a). (1)
a∈A
The -greedy parameter allows for exploration of the conse-
quences of all actions. In fact, it ensures that all the state-action
pairs are explored with some non-zero probability, , so that
all possibilities are discovered.
Once the action is taken, and upon observing a new state,
st+1 , and collecting the immediate payoff, ct , that results from
action at , the agent updates the Q-values as follows:
Q(st , at ) ← (1−α)Q(st , at )+α[ct +γmina Q(st+1 , a)]. (2)
Here α ∈ [0, 1] is the learning rate, that determines how
fast the learning can occur. If α is too small, it will take
long time to complete the learning process, whereas if it is
too big, the algorithm might not converge. Also, γ ∈ [0, 1]
is the discount factor that controls the value placed on the
future rewards. If γ is too small, learning will not depend as
much on future rewards, and immediate rewards are optimized
instead. On the contrary, if it is too big, learning will heavily
depend on future rewards. It is worth mentioning that the
above procedure involves simple computations, which helps
with efficient energy consumption.
B. QL-Based Opportunistic LTE-U Algorithm
Consider LTE-U BS as the agent who applies carrier sensing
to detect and opportunistically utilize white spaces in the WiFi
 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)

channel, and further employs Q-learning to adaptively sched-

ule LTE-U ON periods according to the estimated duration of Algorithm 1: Q-Learning Based Opportunistic LTE-U
white spaces. 1: procedure LTE-U BS: CONSTANTLY SENSE WIFI
The set of actions for LTE-U BS is denoted by A = CHANNEL
{a1 , a2 , . . . , aM }, representing the integer number of LTE 2: Initialize:
subframes scheduled for transmissions, i.e., the length of LTE- 3: for each s ∈ S and a ∈ A do
U ON period is equal to at × tSf = at ms, where at ∈ A and 4: Initialize the Q-value representation mechanism,
tSf = 1 ms. Q(s, a)
We consider a cost function, which we aim to minimize,
5: end for
defined as the absolute difference or error in the prediction
6: Evaluate the starting state s = st ∈ S
of white space duration. In fact, given that the actual WiFi
7: Wait for a WiFi frame arrival
white space is of duration wt , the goal of the BS is to choose
8: top:
the length of ON period, at , as close to wt as possible. If
9: while channel is busy do
at is shorter than wt , then the idle spectral resources are
10: LTE-U OFF
underutilized and LTE-U fails to achieve maximal utilization.
11: if channel becomes idle then
On the other hand, if at is longer than wt , the upcoming WiFi
12: Generate a random number r ∈ U(0, 1)
arrivals will find the channel busy by LTE-U transmissions,
13: if (r < ) then
and thus, WiFi latency is increased. Consequently, minimizing
14: Select action at randomly
the cost function will result in maximum LTE-U utilization
15: else
while limiting WiFi latency at the same time. We assume that
16: Select action at according to equation (1)
the BS can fully observe wt by sensing WiFi broadcast control
signalings such as RTS/CTS. The immediate cost is defined 17: end if
as follows: 18: LTE-U ON: transmit for at subframes
ct = |wt − at |. (3) 19: Receive the white space duration wt and the im-
mediate cost, ct = |wt − at |
We set the states, S = {s1 , s2 , . . . , sN }, such that N levels 20: Observe the next state st+1 ∈ S according to
for the error in white space prediction is presumed. That is, equation (4)
at step (cycle) t, state st is determined as follows: 21: Update the Q-table entry using equation (2)
Put s = st+1

 s1 , ct < cmin 22:

 23: end if
 s2 , cmin ≤ ct < cmin + ∆c



 24: Wait until next WiFi frame arrival
st = s3 , cmin + ∆c ≤ ct < cmin + 2∆c (4) 25: goto top.

 ..
.





sN , ct ≥ cmax ,

well as an ABS-plus-sensing, and the MMBPP-based oppor-
where cmin and cmax are the smallest and largest tolerable tunistic approach, specified as follows.
cost values, respectively, and the range is divided by 1) ABS: We consider an ABS mechanism where LTE-U BS
cmax − cmin allocates a number of subframes at the end of each frame to
∆c = .
N −2 WiFi network activity, such that the fraction of the frame that
The agent maintains a Q-table where each entry corresponds WiFi can use is equal to the traffic intensity, or the utilization
to a pair of state-action, with a Q-value representing the of the WiFi network. For instance, if ρ = 0.2, LTE-U is ON
estimated future cost following from that state-action pair. At during the first 8 subframes, and stops transmissions for the
each state, the BS chooses the action according to the -greedy last 2 subframes of a frame (20% ABS).
policy, that is either randomly, or the one that yields to the 2) ABS+Sensing: In ABS-plus-sensing, which is in fact
lowest estimated cost, i.e., with minimum Q-value. Once the a mixture of ABS and opportunistic schemes, the BS keeps
action is executed, the agent receives a feedback from the sensing the channel and whenever the channel is idle, starts
environment in form of the cost function, and learns about the LTE-U transmission for a fixed duration equal to the average
outcome of the deployed action in the current state. The next white space length. So basically, similar to opportunistic
state is then identified, and the previous Q-value is updated. method, the LTE-U duty cycle is afloat and variable, but the
The proposed Q-learning based opportunistic LTE-U algorithm LTE-U ON period is constant and a function of the WiFi traffic
is summarized in Algorithm 1. intensity, as in ABS.
3) MMBPP-Based Opportunistic Scheme: In this oppor-
IV. N UMERICAL R ESULTS tunistic approach, LTE-U BS constantly monitors WiFi activity
In this section we examine the proposed Q-learning algo- in the unlicensed channel through carrier sensing, similar to
rithm for the opportunistic coexistence of LTE-U and WiFi, the QL-based approach, and turns LTE-U transmissions off as
and compare the results with those of an ABS mechanism as long as the channel is sensed to be busy. The BS observes the
 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)

time instant and size of the latest WiFi packet frame arrival, 1

(Tn−1 , Kn−1 ). Once an idle channel is sensed, that indicates 0.8

MMBPP

LTE-U throughput
a white space initiation, the BS uses the following equation ABS+sensing
0.6 ABS
based on the MMBPP model to estimate WiFi frame inter-
Q-Learning
arrival time, Tban , for a = Kn−1 , 0.4

0.2

Tban = −ν a G−1
aa 1. (5) 0
0.2 0.5
ρ

This is the conditional expectation of a phase type distribution Fig. 3: Maximum achievable LTE-U throughput.
with r phases and parameters (ν a , Gaa ), where r is the
number of underlying states in the MMBPP arrival process 12

MMBPP
that represents the number of WiFi network traffic load levels, 10

WiFi latency (ms)

ABS+sensing
Gaa is the (a, a) block of the generator matrix, G, of the 8
ABS
MMBPP arrival process, and ν a = νa /(νa 1), where vector νa 6
Q-Learning
represents the initial distribution of the MMBPP process for an 4

initial arrival of size a, and 1 represents a unit column vector 2

of order r. Here, it is assumed that the information regarding 0

0.2 0.5
the statistics and parameters of the WiFi network temporal ρ

behavior are provided to or learned by the BS. Specifically,
the BS may employ maximum likelihood parameter estimation
methods, such as EM algorithm, to estimate the statistics of
the MMBPP process. Next, the BS predicts the time of next
WiFi frame arrival, Tbn , as follows,
Tbn = Tn−1 + Tban ,
(6)
and estimates the white space duration by calculating the
remainder of the time until Tbn . Consequently, the BS is able to
schedule transmissions (LTE-U ON) for the estimated duration
of WiFi white space and stop transmissions (LTE-U OFF) by
the end of this period, such that the channel is available for
the upcoming WiFi frame arrival. For details of the MMBPP
arrival process and the associated opportunistic algorithm,
refer to [9], [11].
A. Simulations
We consider a WiFi network with a single AP and N = 5
STAs with DL traffic that is generated according to an
MMBPP arrival process with r = 2 states, indicating low and
high traffic load levels. Each WiFi frame arrival is assumed to
be comprised of up to 64 IEEE 802.11ac packets according to
a state-dependent normal probability distribution. The state-
dependent arrival rates are varied to obtain different traffic
intensities (WiFi utilization), ρ. The coexisting LTE-U BS
employs the Q-Learning-based algorithm to opportunistically
exploit the white spaces in the WiFi channel and schedule
transmissions such that the channel will most likely be vacated
for upcoming WiFi transmissions.
For the Q-Learning algorithm, we put α = 0.5, γ = 0.9,
and  = 0.1. We consider the set of possible actions, A =
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, indicating number of subframes
in an LTE frame that the BS transmits (LTE-U ON period),
Fig. 4: WiFi latency.
and the set of states as follows,


 1, ct < 0.5

 2, 0.5 ≤ ct < 1



st = 3, 1 ≤ ct < 2 (7)

4 2 ≤ ct < 3





5, ct ≥ 3,

where ct is the immediate cost function, given in equation (3).
Fig. 3 demonstrates LTE-U throughput vs. the coexisting
WiFi network traffic intensity in terms of the maximum
percentage of total resources that LTE-U can exploit, for
the four aforementioned mechanisms. As expected, LTE-U
throughput generally decreases with ρ increasing, mainly
because with higher WiFi traffic the duration of white spaces
drops down. We can see that the ABS-plus-sensing achieves
higher throughput compared to ABS, mainly because of the
plenitude of exploitable white spaces in WiFi traffic that it
can identify using carrier sensing. Also, it achieves higher
throughput compared to the MMBPP-based approach because
of the fact that it is less conservative and does not vacate
the channel dynamically based on WiFi prediction. It is also
observed that the LTE-U throughput with the MMBPP-based
opportunistic mechanism is comparable to that with ABS,
which is due to the efficient exploitation of white spaces. We
can see that under the proposed QL-based algorithm, LTE-
U is generally able to achieve a relatively high throughput,
especially when the traffic load of the WiFi network is higher.
The latency imposed on the coexisting WiFi network is
shown in Fig. 4. As ρ increases, LTE-U is off for longer
periods in all the four mechanisms, and thus, is less likely to
interrupt WiFi transmissions. It is evident that ABS extremely
disturbs WiFi traffic and results in significant latency. How-
ever, the latency with the other mechanisms is considerably
 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)
(LTE-U throughput / WiFi Latency)
1.2
1 MMBPP
0.8
Power
0.6
0.4
0.2
0
0.2
ρ control the trade-off between LTE-U utilization and WiFi
Fig. 5: Power measure; ratio of the LTE-U achievable throughput to
the latency imposed on the coexisting WiFi network.
less, mainly because they employ spectrum sensing to avoid
interrupting WiFi transmissions. Also, notice that the latency
imposed by the MMBPP-based mechanism is always the
minimum. WiFi latency is equal to 0.58 ms, which is ten
times lower than that of ABS (5.5 ms), with moderate WiFi
traffic (ρ = 0.5). The reason is that by accurately estimating
the duration of each white space, LTE-U BS is very likely
to vacate the spectrum for upcoming WiFi transmissions in
a timely manner. It is observed that under the proposed QL-
based algorithm, the latency imposed on WiFi is comparably
low for lightly loaded WiFi network, and it is elevated with
increased WiFi traffic load, yet, it outperforms the ABS
mechanism.
Fig. 5 shows the power measure, i.e., the ratio of the LTE-U
achievable throughput to the latency imposed on the coexisting
WiFi network. This measure expresses the overall coexistence
performance under different schemes, which we tend to max-
imize. It is observed that the MMBPP-based scheme outper-
forms the three other mechanisms under both scenarios. This
is achieved at the cost of extensively complex computations.
However, we can see that in a lightly loaded WiFi network,
the QL-based approach provides similar performance to that of
the MMBPP-based, while no information of the WiFi network
statistics are necessary and the computational complexities are
avoided at the same time.
Note that at ρ = 0.5, higher throughput is achieved
under the QL-based approach compared to the MMBPP-based
mechanism at the price of increased latency. Yet, the power
measure performance is considerably superior to that of the
ABS scheme. It is worth mentioning that alternative definition
of the states and the cost function, would result in different
performances. They also provide a means to control the trade-
off between LTE-U utilization and WiFi latency.
V. C ONCLUSION
A Q-Learning-based approach was proposed in this paper
for a model-free decision-making implementation of oppor-
tunistic coexistence of LTE-U with WiFi, which enabled
the LTE-U BS to dynamically identify and further exploit
white spaces in the WiFi channel, without requiring detailed
knowledge of the WiFi system. By adaptively adjusting LTE-
U duty cycle to WiFi activity, the proposed algorithm enabled
maximal utilization of idle resources for LTE-U transmissions,
while decreasing the latency imposed on WiFi traffic. Simu-
lation results demonstrated superior performance as compared
ABS+sensing to the ABS mechanism, as well as similar performance as
compared to the MMBPP-based opportunistic approach in
ABS
Q-Learning lightly loaded scenarios, even though extensive computations
and parameter estimation of the Markovian solution were
avoided. The proposed approach also provided a means to
0.5
latency in the coexisting networks.
R EFERENCES
[1] H. Zhang, X. Chu, W. Guo, and S. Wang, “Coexistence of WiFi and
heterogeneous small cell networks sharing unlicensed spectrum,” IEEE
Communications Magazine, vol. 53, no. 3, pp. 158–164, March 2015.
[2] J. Li, X. Wang, D. Feng, M. Sheng, and T. Q. S. Quek, “Share in
the commons: Coexistence between LTE unlicensed and WiFi,” IEEE
Wireless Communications, vol. 23, no. 6, pp. 16–23, December 2016.
[3] A. Babaei, J. Andreoli-Fang, and B. Hamzeh, “On the impact of LTE-
U on WiFi performance,” in 2014 IEEE 25th Annual International
Symposium on Personal, Indoor, and Mobile Radio Communication
(PIMRC), Sept 2014, pp. 1621–1625.
[4] E. Almeida, A. M. Cavalcante, R. C. D. Paiva, F. S. Chaves, F. M.
Abinader, R. D. Vieira, S. Choudhury, E. Tuomaala, and K. Doppler,
“Enabling LTE/WiFi coexistence by LTE blank subframe allocation,” in
2013 IEEE International Conference on Communications (ICC), June
2013, pp. 5083–5088.
[5] J. Xiao and J. Zheng, “An adaptive channel access mechanism for LTE-
U and WiFi coexistence in an unlicensed spectrum,” in 2016 IEEE
International Conference on Communications (ICC), May 2016, pp. 1–
6.
[6] N. Rupasinghe and I. Guvenc, “Reinforcement learning for licensed-
assisted access of LTE in the unlicensed spectrum,” in 2015 IEEE
Wireless Communications and Networking Conference (WCNC), March
2015, pp. 1279–1284.
[7] S. Chatterjee, M. J. Abdel-Rahman, and A. B. MacKenzie, “Optimal
distributed allocation of almost blank subframes for LTE/WiFi co-
existence,” in 2017 15th International Symposium on Modeling and
Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), May
2017, pp. 1–6.
[8] M. G. S. Sriyananda, I. Parvez, I. Guvene, M. Bennis, and A. I. Sarwat,
“Multi-armed bandit for LTE-U and WiFi coexistence in unlicensed
bands,” in 2016 IEEE Wireless Communications and Networking Con-
ference, April 2016, pp. 1–6.
[9] N. Rastegardoost and B. Jabbari, “Statistical characterization of WiFi
white space,” IEEE Communications Letters, vol. 21, no. 12, pp. 2674–
2677, Dec 2017.
[10] N. Rastegardoost and B. Jabbari, “A stochastic-modeling approach to
MAC coexistence of LTE-U and WiFi,” in Proceedings of 2017 IEEE
28th Annual International Symposium on Personal, Indoor, and Mobile
Radio Communication (PIMRC), Oct. 2017.
[11] N. Rastegardoost and B. Jabbari, “WiFi white spaces for opportunistic
LTE-U,” in GLOBECOM 2017 - 2017 IEEE Global Communications
Conference, Dec 2017, pp. 1–6.
[12] O. Sallent, J. Prez-Romero, R. Ferrs, and R. Agust, “Learning-based
coexistence for LTE operation in unlicensed bands,” in 2015 IEEE
International Conference on Communication Workshop (ICCW), June
2015, pp. 2307–2313.
[13] Y. Y. Liu and S. J. Yoo, “Dynamic resource allocation using reinforce-
ment learning for LTE-U and WiFi in the unlicensed spectrum,” in 2017
Ninth International Conference on Ubiquitous and Future Networks
(ICUFN), July 2017, pp. 471–475.
[14] J. Huang, G. Xing, G. Zhou, and R. Zhou, “Beyond co-existence:
Exploiting WiFi white space for ZigBee performance assurance,” in 18th
IEEE ICNP, Oct 2010, pp. 305–314.