INTELLIGENT RADIO: WHEN ARTIFICIAL INTELLIGENCE MEETS THE RADIO NETWORK
Machine-Learning-Based Opportunistic
Spectrum Access in Cognitive Radio Networks
Pengcheng Zhu, Jiamin Li, Dongming Wang, and Xiaohu You
Abstract
The explosive growth of wireless devices and
data rate demands makes spectrum scarcity a seri-
ous problem. A promising solution is to employ
OSA, which enables SUs to seek and opportunis-
tically exploit the underutilized spectrum without
interrupting the data transmission of PUs. How-
ever, the real-world implementation of OSA still
faces several critical challenges including lack of
global information, the dilemma of exploration and
exploitation, and channel access competition. In
this article, we propose a machine-learning-based
OSA framework by integrating MAB and matching
theory. First, we start from the single-SU scenar-
io without global information while considering
the volatility of channel availability. We propose
an occurrence-aware OSA (OA-OSA) framework
based on the UCB algorithm, which can achieve
long-term optimal network throughput perfor-
mance and a well-balanced trade-off between
exploration and exploitation based on only local
information. Then we extend OA-OSA to the multi-
SU scenario with channel access competitions,
and derive an OCA-OSA framework by integrating
OA-OSA and the Gale-Shapley algorithm. Simula-
tion results demonstrate that the proposed frame-
works achieve superior performance in network
throughput and less deviation from optimal perfor-
mance with global information.
Introduction
With the explosive growth of wireless data traffic
and mobile service demands, spectrum scarci-
ty has become a serious problem. On the other
hand, several research works reveal that the
licensed spectrum allocated to users has not been
fully utilized in either space or time, which results
in spectrum waste named the spectrum hole [1].
To address the paradox between spectrum scar-
city and spectrum hole, opportunistic spectrum
access (OSA) has been proposed and intensively
researched [2]. OSA is one of the core technol-
ogies of cognitive radio, which emphasizes the
intelligence of the network and adapts spectrum
utilization with the external wireless environment
through learning or other means. The principle of
OSA is to enable secondary users (SUs) to seek
and opportunistically use the spectrum hole for
data transmission without violating the quality
of service (QoS) requirements of primary users
(PUs) [2]. OSA is compatible with existing cellular
technologies and can be implemented in both
Digital Object Identifier:
10.1109/MWC.001.1900234 The authors are with Southeast University, Nanjing; Jiamin Li is the corresponding author.
38 1536-1284/20/$25.00 © 2020 IEEE
Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
centralized and distributed manners [3, 4]. Spe-
cifically, we focus on distributed OSA and study
its implementation in real-world cognitive radio
networks.
However, due to the existence of multiple
channels and users, several technical challenges
arise from the opportunistic utilization of spectrum,
which are summarized below.
Performance Guarantees under Information Uncertainty
Channel availability and channel quality are often
unknown to SUs, and are highly dynamic and
unpredictable due to the random behavior of
PUs and channel fading. As a result, such channel
dynamics lead to uncertain throughput degrada-
tion. Considering the limited sensing capability of
SUs, the communication overheads to collect the
channel state information (CSI) of all the channels
are tremendous. Therefore, conventional OSA
methods which presume that perfect CSI is avail-
able will be infeasible. How to develop a distrib-
uted OSA algorithm with bounded performance
guarantees in the absence of prior knowledge is
an inevitable challenge.
Dilemma of Exploration-Exploitation
Without prior knowledge or even statistical infor-
mation about channel states, each SU has to
estimate channel availability and quality. Chan-
nel estimation can be performed either offline or
online. Compared to offline estimation, the online
approach is more efficient since transmission
can be performed during estimation. However,
it faces the dilemma of exploration-exploitation.
Specifically, in each round of decision making,
an SU must choose whether to increase the esti-
mation accuracy by exploring alternative choices,
or to take full advantage of existing estimation by
exploiting the currently optimal choice. In order
to optimize the long-term throughput, a good bal-
ance between exploration and exploitation must
be achieved.
Competition of Multiple SUs
An SU will experience channel selection collision
when other SUs compete for the same channel.
Thus, the decisions of SUs are coupled. Such cou-
plings cannot be easily analyzed by traditional
game-theoretical approaches [5], because the
perfect information of other game players as well
as CSI are unknown. How to resolve the competi-
tion of multiple SUs under information uncertainty
remains nontrivial.
IEEE Wireless Communications • February 2020
 The real-world implementation of OSA
becomes more complicated when the above
three challenges are jointly considered. To pro-
vide a tractable solution, we propose a machine-
learning-based OSA framework that combines
multi-armed bandit (MAB) with matching theory.
MAB is a powerful reinforcement learning tool
to address the sequential decision making prob-
lem. It simulates the mechanism the player uses
to decide which single arm to play in each round
with the purpose of maximizing the cumulative
reward based on only historical observations [6].
In particular, the channel selection problem in
OSA is analogous to the classic arm selection
problem in the MAB framework, where each
channel is formulated as an arm, and the SU who
selects the channel is formulated as the play-
er. The objective is to maximize the cumulative
reward by selecting while learning. Among several
MAB algorithms, we choose upper confidence
bound (UCB) [7] to achieve a well-balanced
trade-off between exploitation and exploration.
It employs the sample-mean estimations for
exploitation and simultaneously utilizes the confi-
dence interval of estimations for exploration. Fur-
thermore, the Gale-Shapley (GS) [8] algorithm is
combined with UCB to resolve the competition of
multiple SUs under information uncertainty.
In this article, we start from the single-SU sce-
nario and develop an occurrence-aware OSA
(OA-OSA) framework under information uncer-
tainty. Next, we extend OA-OSA into the multi-SU
scenario, and develop an occurrence-aware and
collision-aware OSA (OCA-OSA) framework. A
case study is provided to demonstrate that both
OA-OSA and OCA-OSA can achieve bounded
performance guarantees without any prior infor-
mation of CSI. Finally, we provide a summary and
present future research directions.
Related Works
Several researchers have already investigated the
implementation of distributed OSA. In [9], Chen
et al. developed a spatial congestion framework
to model the channel selection game with fixed
user locations and spatial reuse, and proposed a
distributed learning algorithm based on the Nash
equilibrium. A two-phase stochastic multiple chan-
nel sensing (SMCS) protocol was proposed for
distributed OSA [10], in which the aggregated
throughput of SUs is maximized subject to the
QoS constraints of PUs based on partially observ-
able channel information and sensing overhead.
However, in both [9, 10], the prior knowledge
of the channel is assumed to be available. For
instance, the game-theoretical approach devel-
oped in [9] requires statistical knowledge of the
environment, which may not be always available
in practical applications.
MAB has been widely applied in OSA to solve
channel selection problems. [11, 12]. In [11], Lai
et al. considered the scenario where the probabil-
ity of channel availability is unknown in advance,
and proposed a UCB-based channel selection
algorithm to maximize the expected through-
put of SUs. Anandkumar et al. considered the
single-SU scenario and proposed a distributed
channel selection scheme based on the e-greedy
algorithm [12]. However, classic MAB algorithms
such as UCB and e-greedy were originally devel-
IEEE Wireless Communications • February 2020
Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
oped for the scenario of a single player, which The difference between
will lead to severe collisions when multiple SUs the works mentioned
compete for the same channel [6]. Therefore, in this article and our
the interactions among multiple SUs should be work is that we con-
considered in decision making, which is actually
a combinatorial problem with high complexity. sider both information
Moreover, the preferences of SUs and channels uncertainty and inter-
are not aligned. Particularly, SUs prefer channels actions among multiple
with better channel availability and quality, while SUs. Specifically, we
channels prefer SUs who cause less co-channel incorporate occurrence
interference. Although some works have studied
the multi-SU scenario of OSA [7, 13], channel awareness and collision
volatility as well as mutual preferences of SUs and awareness into the pro-
channels are not considered. posed OSA framework,
Matching theory has been widely applied to and adapt it to learn
address the combinatorial problem in various appli- channel volatility and
cation scenarios including peer discovery, task off-
loading, user association, and so on [8, 14, 15]. matching collision.
In [8], Gu et al. provided a comprehensive survey
of the applications of matching theory in wireless
communications. In [14], Francesco et al. consid-
ered the scenario of the Internet of Things (IoT)
and developed a distributed many-to-one match-
ing strategy to provide an efficient task offloading
mechanism based on the deferred acceptance
algorithm. A matching-based social network-aware
algorithm was proposed to optimize the overall
network performance in wireless small-cell net-
works [15]. The drawback of conventional match-
ing algorithms such as the GS algorithm [8] is that
perfect information on both sides is required in
order to precisely construct the mutual preferenc-
es.
The difference between the works mentioned
above and our work is that we consider both infor-
mation uncertainty and interactions among multi-
ple SUs. Specifically, we incorporate occurrence
awareness and collision awareness into the pro-
posed OSA framework, and adapt it to learn chan-
nel volatility and matching collision.
System Model and Problem Formulation
In this section, first we introduce the system
model of OSA in multi-cell cognitive radio net-
works. Then we present the formulated optimiza-
tion problem.
Figure 1 depicts the considered multi-cell cog-
nitive radio networks, where each small cell con-
sists of a small-cell base station (SBS), J PUs, and
I SUs. We consider the uplink scenario where
both PUs and SUs transmit data to the SBS. OSA
is employed to improve spectrum efficiency. The
specific role and functionality of each entity are
illustrated as follows:
• SBS: The SBS is a fixed infrastructure network
component that provides connection service
and coordinates channel selection for users
within its coverage area.
• PUs: PUs are licensed users who are authorized
to use the spectrum with top priority. PUs are
interference-sensitive, that is, SUs are prohibit-
ed from using the channel when it is occupied
by a PU.
• SUs: SUs are not allocated dedicated channels.
They can transmit data by utilizing idle channels
that are unoccupied by PUs.
A time slot model is adopted, in which the total
time period is divided into T equalized slots. At
the tth slot, a channel is unavailable if it is unoc-
cupied by PUs or available otherwise. We assume
39
 FIGURE 1. The implementation of OSA in a cognitive radio network.
that both channel availability and channel quality
remain unchanged within a slot and vary across dif-
ferent slots. The implementation of OSA is gener-
ally composed of two stages: sensing and decision
making.
Sensing
At the beginning of each slot, an SU selects a
channel to sense and observes its occupancy.
Due to the hardware limitation, only one channel
can be sensed in a slot, and the remaining chan-
nels are left unsensed. The channel is reported as
“occupied” if the signal of a PU is detected, and is
reported as “idle” otherwise.
Decision Making
Based on the sensing results and collected histor-
ical information, an SU selects the channel with
the highest possible throughput. If more than one
SU competes for the same channel, it is randomly
allocated to one of the candidate SUs, and the
other SUs have to wait for the next slot.
The channel selection strategy of an SU is
denoted as a binary variable. For example, we use
ai,n,t = 1 to represent that SU i selects channel n at
slot t, and the corresponding throughput is denot-
ed as Qi,n,t. The achievable throughput not only
depends on CSI, but also depends on the channel
selection strategies of other SUs. The throughput
of the selected channel can even be zero if it is
unavailable, for example, it is occupied by another
PU.
The objective of OSA is to maximize the cumu-
lative throughput of the entire network over a total
period of T slots via the optimization of channel
selection, which can be denoted as
T I N ever, the variation of channel availability has an
max{ai,n,t } ∑ t=1∑ i=1∑ n=1 ai,n,tQi,n,t .
40
Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
Unavailability of Global Information
The global information related to throughput
includes both CSI and strategies of other SUs. If
there is a genie that reveals the global information
to SUs, the best solution of OSA follows the Nash
equilibrium of the noncooperative game-theo-
retical approach. However, the game-theoreti-
cal approach is infeasible if global information is
unknown for SUs.
Learning while Selecting
We propose a learning-based OSA framework
that enables SUs to learn the environment while
making decisions. For each SU, the CSI of a chan-
nel can be learned from historical observations
including empirical throughput performance,
number of selections, and occurrence time. The
objective is to maximize the expectation of cumu-
lative throughput of the entire network.
Occurrence-Aware and Collision-Aware
Opportunistic Spectrum Access
In this section, first, we develop an occur-
rence-aware OSA (OA-OSA) framework for the
single-SU scenario. Then we consider the more
practical multi-SU scenario, and develop an occur-
rence-aware and collision-aware OSA (OCA-OSA)
framework by combining the GS algorithm and
OA-OSA. The proposed OA-OSA and OCA-OSA
frameworks are shown in Fig. 2.
OA-OSA for the Single-SU Scenario
The channel selection problem is a typical online
sequential decision making problem that can be
solved by employing the MAB framework. How-
adverse impact on existing MAB algorithms such
as UCB and e-greedy, which are only applicable
IEEE Wireless Communications • February 2020
 FIGURE 2. The proposed OA-OSA and OCA-OSA frameworks for the single-SU and multi-SU scenarios, respectively.
to the MAB problem with constantly available
arms. To address this challenge, we incorporate
classical UCB with occurrence awareness and
propose an occurrence-aware OSA framework
named OA-OSA. Its implementation consists of
four stages: estimation, sensing, decision making,
and learning. Details are given as follows.
Stage 1. Estimation: At the beginning of each
slot, every SU estimates its preference toward each
channel based on historical observations. The pref-
erence of SU i toward channel n is estimated as
Q! i,n,t = Qi,n,t−1 +
( occ
β log t − ti,n ).
Ai,n,t−1
(1)
Here, the first term of Eq. 1 denotes the historical
throughput of channel n up to the current slot,
which is the sample-mean estimation of the chan-
nel throughput and belongs to local information.
The second term denotes the confidence interval
representing the uncertainty of sample-mean esti-
occ
mation. b is a positive weight of exploration. ti,n
represents the channel occurrence time. Ai,n,t–1
denotes the number of times channel n has been
selected up to the current slot, which also belongs
to local information.
Stage 2. Sensing: Denote the ultimate selected
channel as yi. Then SU i senses the occupancy of
this channel.
Stage 3. Decision Making: Based on the sens-
ing results, the preference toward channel ψi is
updated. Specifically, if it is occupied, this chan-
nel is specified as unavailable. Both the prefer-
ence and the channel throughput are set as zero.
Then update ψi as the channel with the second
highest preference. Otherwise, if ψi is reported as
idle, the corresponding preference and the chan-
nel throughput remain unchanged. Next, it selects
channel ψi, that is, ai,ψi,t = 1, and sends a scheduling
request to the SBS.
Stage 4. Learning: After data transmission, the
SU observes the achieved throughput associated
with channel ψi. At the end of the tth slot, for any
— —
channel n, update Qi,n,t = Qi,n,t–1Ai,n,t–1 + Qi,n,tai,n,t)/
(Ai,n,t–1 + 1}) and Ai,n,t = Ai,n,t–1 + ai,n,t.
The above four stages are repeated iteratively
until the maximum number of slots is reached. The
OA-OSA framework adapts to the random vari-
ations of channel availability and channel quality
due to the following properties.
Occurrence Awareness: Occurrence aware-
ness is incorporated in the confidence interval.
IEEE Wireless Communications • February 2020
Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
For any newly available channel, its small number
of selections Ai,n,t – 1 results in a large confidence
interval, which encourages OA-OSA to explore
and learn its performance. Meanwhile, OA-OSA
tends to exploit the existing channels with small-
er occurrence time since more times of selection
result in less estimation deviation.
Throughput Awareness: Throughput aware-
ness is considered in both the decision making
stage and the sensing stage. Compared to con-
ventional OSA, where the SU randomly senses a
channel, OA-OSA adopts a targeted sensing strat-
egy in which only the channel with the highest
preference is sensed, thereby avoiding throughput
performance degradation.
OCA-OSA for the Multi-SU Scenario
In the multi-SU scenario, OA-OSA will suffer from
significant performance degradation due to com-
petition among SUs. To overcome this challenge,
we augment OA-OSA with the GS algorithm[8],
which transforms the combinatorial problem into
a two-sided matching problem. We develop OCA-
OSA, which is shown in Fig. 2. It is implemented
in four stages: estimation, sensing, matching, and
learning. The first two stages of OCA-OSA are
exactly the same as those of OA-OSA. Therefore,
we emphasize the stages of matching and learn-
ing.
Stage 3. Matching: In the third stage, the
combinatorial channel selection problem can be
transformed into a two-sided one-to-one matching
game, which can easily be solved by the classic GS
algorithm, where SUs from one side are matched
with channels on the other side based on their
mutual preferences in a stable way. That is, any
SU is matched with its best available choice. The
implementation of the GS algorithm consists of
two phases: preference list construction and itera-
tive matching.
Preference List Construction: As shown in Fig.
2, if channel n – 1 is sensed to be occupied, it is
placed at the end of the preference list of SU i.
For any other available channel (e.g., channel n),
if it is a newly available channel, the preference is
set as positive infinity so that the SU has a greater
chance to be matched with it to learn its through-
put performance. Otherwise, if it has been avail-
able before, the preference is determined as Eq. 1,
which takes empirical throughput performance and
occurrence awareness into consideration. Then the
preference list of SU i is constructed by re-sorting
41

FIGURE 3. The assumptions of channel availability.
the preferences of every channel in descending
order.
Meanwhile, the preference of a channel toward
an SU is defined as positively related to the dis-
tance between the SU and the SBS, which guar-
antees fairness among SUs since cell-edge SUs
will have high priority. Furthermore, it reduces the
overall interference levels of the whole network
since cell-edge SUs are allocated high-quality chan-
nels under the same condition. The preference list
of a channel is constructed similarly to that of a SU.
Iterative Matching: In this phase, the SBS col-
lects the preference list of each SU and performs
iterative matching. In each iteration, each SU pro-
poses to its most preferred channel in its prefer-
ence list. If a channel receives only one proposal
from SUs, it will hold this SU as a potential can-
didate. Note that this candidate can be rejected
later if a better candidate appears. Otherwise, if it
receives more than one proposal, it will select the
most preferred SU based on its preference list and
reject the other SUs. At the end of this iteration,
each SU updates its preference list by removing
the channels that have rejected it.
In the next iteration, any SU who has been
rejected in the previous iteration proposes to its
currently most preferred channel. If the channel
has not held any candidate and has received only
one proposal from an SU, they are matched. Oth-
erwise, the channel will select the most preferred
SU among all the newly received proposals as well
as the previously held candidate. The iteration will
continue until every SU has been matched with a
channel or rejected by all the channels in its prefer-
ence list, and a stable matching has been derived.
Stage 4. Learning: Each SU accesses a chan-
nel channel based on the matching result. After
data transmission, it observes the corresponding
throughput of the accessed channel. Next, for any
—
channel n, Qi,n,t and Ai,n,t are updated similarly as
OA-OSA.
As for OCA-OSA, the complexity of other
stages is similar to that of OA-OSA, while the
complexity of matching depends on O(Nlog(N) +
Ilog(I)), which is much lower than that of exhaus-
tive searching. Compared to OA-OSA, OCA-OSA
adapts to the multi-SU scenario due to the follow-
ing property.
Collision Awareness: By observing matching
results, an SU can continuously learn the relation-
ship among its preferences toward channels, the
strategies of other SUs, and the preferences of
channels. Thus, the impacts of matching collision
and mutual preferences have been incorporated
into the learning process.
Performance Evaluation
In this section, preliminary simulation results are
provided to validate the performance of the pro-
posed scheme. Simulations are carried out in
MATLAB for both the single-SU and multi-SU sce-
42
Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
narios. We evaluate the performance in terms of
average network throughput and learning regret.
Learning regret is defined as the cumulative per-
formance loss of the proposed scheme compared
to the optimal channel selection policy with glob-
al information.
Performance under the Single-SU Scenario
We consider one SU and four channels. The
dynamics of channel availability are shown in
Fig. 3, where channels marked A are available
channels, and channels marked U are unavailable
channels. The achievable throughput of channel n
at each slot follows a uniform distribution within
— — —
the range [0.8 Qn,1.2 Qn], where Qn represents the
—
average throughput. We have Qn = 0.5, 0.3, 0.2,
0.1 when n = 1, 2, 3, 4. The weight of exploration
b is set as 2. Two baseline algorithms are taken for
comparison. Baseline 1 is the channel selection
algorithm proposed in [6], which does not consid-
er either occurrence awareness or collision aware-
ness. Baseline 2 is obtained by enhancing baseline
1 with occurrence awareness. However, collision
awareness is still not considered.
Figures 4a and 4b show the normalized average
network throughput and learning regret under the
single-SU scenario, respectively. When t = 500, the
proposed OA-OSA scheme outperforms baseline 1
and baseline 2 by 29.5 and 6.5 percent, respective-
ly. Besides, its performance also converges faster
than the other two baseline algorithms. This huge
performance gain is achieved because it takes both
occurrence awareness and collision awareness
into consideration. Among all the three algorithms,
baseline 1 performs the worst since both occur-
rence awareness and collision awareness are not
considered. The achieved throughput is even zero
during some slots due to the fact that an occupied
channel has been mistakenly selected. From Fig.
4b, it is clear that OA-OSA always achieves the
lowest learning regret. Its learning regret is 38 and
6.2 percent lower than those of baseline 1 and
baseline 2, respectively.
Performance under the Multi-SU Scenario
We consider the multi-SU scenario with three SUs
and four channels. The settings of channel occu-
pancy and throughput remain the same as the
single-SU scenario. In both baseline 1 and base-
line 2 algorithms, if multiple SUs select the same
channel, the channel is randomly allocated to one
of the candidate SUs, and the other SUs have to
wait for the next slot.
Figures 5a and 5b show the normalized average
network throughput and learning regret for the
multi-SU scenario, respectively. Numerical results
of Fig. 5a indicate that the total network through-
put of OCA-OSA over T = 2000 slots is 47.7 per-
cent larger than that of baseline 1. Compared to
baseline 2, OCA-OSA can improve the total net-
work throughput by 8.2 percent over T = 2000
slots. Numerical results of Fig. 5b also demonstrate
that OCA-OSA can achieve superior performance
in terms of learning regret. Compared to the base-
line 1 and baseline 2 algorithms, OCA-OSA can
reduce learning regret by 93.2 and 76.1 percent,
respectively, when t = 2000. Therefore, the joint
consideration of occurrence awareness and colli-
sion awareness can significantly improve the per-
formance.
IEEE Wireless Communications • February 2020
 1 600
Baseline 1
0.9 OA-OSA
Baseline 2
Normalized Average Network Throughput
Baseline 2
500 OA-OSA
0.8 Baseline 1

0.7
400 66

Learning Regret
0.6
64

0.5 300 62
200 205 210
0.4
200
0.3

0.2
100
0.1

0 0
0 200 400 600 800 1000 1200 1400 1600 1800 2000 0 200 400 600 800 1000 1200 1400 1600 1800 2000
Slot Slot
(a) The normalized average network throughput. (b) The learning regret.

FIGURE 4. The throughput performance under the single-SU scenario: a) the normalized average network throughput; b) the learning regret.

1 400
OCA-OSA Baseline 1
0.9 Baseline 2 Baseline 2
350
Normalized Average Network Throughput

Baseline 1 OSA-OSA
0.8
300
0.7
Learning Regret

250
0.6

0.5 200

0.4
150
0.3
100
0.2
50
0.1

0 0
0 200 400 600 800 1000 1200 1400 1600 1800 2000 0 200 400 600 800 1000 1200 1400 1600 1800 2000
Slot Slot
(a) The normalized average network throughput. (b) The learning regret.

FIGURE 5. The throughput performance under the multi-SU scenario: a) the normalized average network throughput; b) the learning regret.

Conclusion and Open Issues Context Awareness: In addition to the occur-

In this article, we propose a machine-learning- rence awareness and collision awareness con-
based OSA framework that incorporates both sidered in this work, there are still numerous
occurrence awareness and collision aware- context factors that influence the learning perfor-
ness. We start from the single-SU scenario and mance, including energy status, service priority,
develop a distributed, low-complexity, online location, QoS requirements, and so on. How to
OSA framework based on only local informa- reasonably incorporate these factors to design a
tion. Then we extend it to the more complicat- context-aware framework for more accurate esti-
ed multi-SU scenario and develop a OCA-OSA mation is an open issue.
framework that enables SUs to learn the inter- Personalized Service Demand of SUs: This
actions among SUs from the matching results. work assumes that all the SUs are identical, and
Numerical results demonstrate that OCA-OSA only considers the QoS performance indicator
can improve the throughput performance by 8.2 in terms of throughput. However, this simplified
percent if collision awareness is considered, and optimization model does not consider other per-
by 47.7 percent if both occurrence awareness formance metrics such as latency and fairness,
and collision awareness are considered. Com- and may not capture the personalized service
pared to the conventional OSA scheme, the demands of SUs well. Therefore, an important
learning regret can also be reduced by 38.0 and research direction is to conduct research on OSA
93.2 percent under the single-SU and multi-SU from the perspective of quality of experience
scenarios, respectively. (QoE), and design a machine-learning-based OSA
Finally, we outline some open issues that require framework to support differentiated service pro-
further research efforts. visioning.

IEEE Wireless Communications • February 2020 43

Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
 Channel Switching Cost: Due to dynam-
ic channel availability, SUs have to frequently
switch channels and readjust relevant transmission
parameters, resulting in extra signaling overhead
and additional switching cost. This cost should
also be taken into consideration in the optimiza-
tion process, and its impact on the learning regret
should be analyzed.
Acknowledgments
This work was supported by the Natural Science
Foundation of Jiangsu Province under Grant
BK20180011, the National Natural Science Foun-
dation of China under Grant 61971127, and the
National Science and Technology Major Project
of China under Grant 2018ZX03001008-002.
References
[1] S. Haykin, “Cognitive Radio: Brain-Empowered Wireless
Communications,” IEEE JSAC, vol. 23, no. 2, Feb. 2005, pp.
201–20.
[2] S. Stotas et al., “On the Throughput and Spectrum Sensing
Enhancement of Opportunistic Spectrum Access Cognitive
Radio Networks,” IEEE Trans. Wireless Commun., vol. 11, no.
1, Jan. 2012, pp. 97–107.
[3] M. Rashid et al., “Opportunistic Spectrum Scheduling for
Multiuser Cognitive Radio: A Queueing Analysis,” IEEE Trans.
Vehic. Tech., vol. 8, no. 10, Oct. 2009, pp. 5259–69.
[4] Y. Liang et al., “Cognitive Radio Networking and Communi-
cations: An Overview,” IEEE Trans. Vehic. Tech., vol. 60, no.
7, Sept. 2011, pp. 3386–3407.
[5] X. Kang et al., “Incentive Mechanism Design for Hetero-
geneous Peer-to-Peer Networks: A Stackelberg Game
Approach,” IEEE Trans. Mobile Comp., vol. 14, no. 5, May
2015, pp. 1018–30.
[6] Y. Xu et al., “Decision-Theoretic Distributed Channel Selec-
tion for Opportunistic Spectrum Access: Strategies, Chal-
lenges and Solutions,” IEEE Commun. Surveys & Tutorials,
vol. 15, no. 4, Apr. 2013, pp. 1689–1713.
[7] N. Modi et al., “QoS Driven Channel Selection Algorithm for
Cognitive Radio Network: Multi-User Multi-Armed Bandit
Approach,” IEEE Trans. Cognitive Commun. Net., vol. 3, no.
1, Mar. 2017, pp. 49–66.
[8] Y. Gu et al., “Dynamic Path to Stability in LTE-Unlicensed
With User Mobility: A Matching Framework,” IEEE Trans.
Wireless Commun., vol. 16, no. 7, Dec. 2017, pp. 4547–61.
[9] X. Chen et al., “Distributed Spectrum Access with Spatial
Reuse,” IEEE JSAC, vol. 31, no. 3, Mar. 2013, pp. 593–603.
[10] K. Feng et al., “ Novel Design on Multiple Channel Sensing
for Partially Observable Cognitive Radio Networks,” IEEE
Trans. Mobile Comp., vol. 16, no.8, Aug. 2017, pp. 2260–75.
44
Authorized licensed use limited to: UNIVERSIDAD DE MONTERREY (UDEM). Downloaded on November 28,2022 at 18:37:36 UTC from IEEE Xplore. Restrictions apply.
[11] L. Lai et al., “Cognitive Medium Access: Exploration,
Exploitation, and Competition,” IEEE Trans. Mobile Comp.,
vol. 10, no. 2, Feb. 2011, pp. 239–53.
[12] A. Anandkumar et al., “Distributed Algorithms for Learning
and Cognitive Medium Access with Logarithmic Regret,”
IEEE JSAC, vol. 29, no. 4, Apr. 2011, pp. 731–45.
[13] J. Chen et al., “Interference-Aware Online Distributed
Channel Selection for Multicluster FANET: A Potential Game
Approach,” IEEE Trans. Vehic. Tech., vol. 68, no. 4, Apr.
2019, pp. 3792–3804.
[14] F. Chiti et al., “A Matching Theory Framework for Tasks
Offloading in Fog Computing for IoT Systems,” IEEE Trans.
Commun., vol. 66, no. 11, Nov. 2018, pp. 5526–38.
[15] M. I. Ashraf et al., “Dynamic Clustering and User Associ-
ation in Wireless Small-Cell Networks with Social Consider-
ations,” IEEE Trans. Vehic. Tech., vol. 66, no. 7, July 2017,
pp. 6553–68.
Biographies
Pengcheng Zhu [M’09] received his B.S. and M.S. degrees in
electrical engineering from Shandong University, Jinan, China,
in 2001 and 2004, respectively, and his Ph.D. degree in com-
munication and information science from Southeast University,
Nanjing, China, in 2009. He is an associate professor with the
National Mobile Communications Research Laboratory, South-
east University. His research interests lie in the areas of wireless
communications and mobile networks.
J iamin L i received his B.S. and M.S. degrees in communica-
tion and information systems from Hohai University, Nanjing,
China, in 2006 and 2009, respectively, and his Ph.D. degree
in information and communication engineering from Southeast
University in 2014. He has been a lecturer with the Nation-
al Mobile Communications Research Laboratory, Southeast
University, since 2014. His research interests include massive
MIMO, distributed antenna systems, and cooperative commu-
nications.
D ongming W ang [M’06] received his B.S. degree from
Chongqing University of Posts and Telecommunications, China,
in 1999, his M.S. degree from Nanjing University of Posts and
Telecommunications, China, in 2002, and his Ph.D. degree from
Southeast University in 2006. He is a professor with the Nation-
al Mobile Communications Research Laboratory, Southeast
University. His research interests include distributed antenna
systems and large-scale MIMO systems.
Xiaohu You [F’11] received his B.S., M.S., and Ph.D. degrees
in electrical engineering from Nanjing Institute of Technology
in 1982, 1985, and 1989, respectively. He is a professor with
the National Mobile Communications Research Laboratory,
Southeast University. His research interests include mobile com-
munications, adaptive signal processing, and artificial neural
networks with applications to communications and biomedical
engineering.
IEEE Wireless Communications • February 2020