Special Collection Article

International Journal of Distributed

Sensor Networks
2018, Vol. 14(2)
Adaptive access-point and channel Ó The Author(s) 2018
DOI: 10.1177/1550147718761584
selection method using Markov journals.sagepub.com/home/dsn

approximation

Tomotaka Kimura1, Kouji Hirata2 and Masahiro Muraguchi1

Abstract
This article proposes an access-point and channel selection method for Internet of Things environments. Recently, the
number of wireless nodes has increased with the growth of Internet of Things technologies. In order to accommodate
traffic generated by the wireless nodes, we need to utilize densely placed wireless access-points. This article introduces
a joint optimization problem of access-point and channel selection for such an environment. The proposed method deals
with the optimization problem, using Markov approximation which adapts to dynamic changes in network conditions.
Markov approximation is a distributed optimization framework, where a network is optimized by individual behavior of
users forming a time-reversible continuous-time Markov chain. The proposed method searches optimal solution for the
access-point and channel selection problem on the time-reversible continuous-time Markov chain. Simulation experi-
ments demonstrate the effectiveness of the proposed method.

Keywords
Internet of Things, access-point selection, channel selection, Markov approximation, optimization problem

Date received: 16 May 2017; accepted: 26 January 2018

Handling Editor: Sang-Woon Jeon

Introduction Under current IEEE 802.11 specifications, wireless

Recently, a huge number of network devices have con-
nected to the Internet, which leads to the realization of
Internet of Things (IoT).1–5 The number of network
devices will further increase with the growth of IoT
technologies. In IoT environments, many wireless nodes
and access-points are assumed to be densely distributed,
and they construct wireless local area networks.6 In
fact, the number of access-points has increased rapidly
in public spaces such as train stations and airports.
When there are some access-points in a given area,
wireless nodes can select an access-point from among
them. Each access-point selects a channel to communi-
cate and then communicates with wireless nodes on the
channel. The quality of the communications strongly
depends on the selected access-points and channels.
Therefore, access-point and channel selection has
become an important technical issue.7–10
nodes select the access-point with the highest received
signal strength indicator (RSSI). In this RSSI-based
selection, traffic loads among access-points are often
uneven. Some access-points have many wireless nodes,
whereas others have relatively few wireless nodes.
When many wireless nodes connect to an access-point,
the throughput drastically decreases because of traffic
congestion. To alleviate this traffic congestion, some
wireless nodes should switch to access-points that have
lower loads but somewhat lower RSSIs. Moreover,
1
Faculty of Engineering, Tokyo University of Science, Tokyo, Japan
2
Faculty of Engineering Science, Kansai University, Osaka, Japan
Corresponding author:
Tomotaka Kimura, Faculty of Engineering, Tokyo University of Science, 6-
3-1 Niijuku, Katsushika-ku, Tokyo 125-8585, Japan.
Email: kimura@ee.kagu.tus.ac.jp

Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License
(http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without
further permission provided the original work is attributed as specified on the SAGE and Open Access pages (http://www.uk.sagepub.com/aboutus/
openaccess.htm).
2 International Journal of Distributed Sensor Networks
RSSI-based selection causes another problem in terms
of frequency resource utilization of access-points
because the RSSI values are independent of channels of
access-points. In cases where neighboring access-points
use the same channel, we should avoid simultaneously
using the access-points even if those RSSIs are high.
Frequency resources can be effectively used by selecting
an access-point that has a lower RSSI and uses a differ-
ent channel.
In the past, several access-point selection methods
have been considered.7–9 Most of those methods use
techniques such as mathematical programming and
heuristic algorithms to select the optimal access-point
based on the number of wireless nodes and RSSI levels
at a given time.7 These methods must conduct the opti-
mization process in a centralized manner each time the
number of users changes or any RSSI value is updated.
Hence, they are not suitable for access-point selection
in situations where network conditions change dynami-
cally. In public spaces such as train stations and air-
ports, network conditions frequently change. Typically,
users stay in public spaces for a short period, and most
users briefly connect to particular access-points.
Especially, in IoT environments, namely, densely
deployed wireless networks, network conditions more
frequently change because there are a lot of access-
points and wireless nodes in a small area. Therefore, an
access-point and channel selection method that can
flexibly accommodate such dynamic situations is
needed for IoT environments.
To effectively select access-points and/or channels in
IoT environments, some methods have been proposed
in the literature.10–15 Karimi et al.12 have proposed an
access-point and channel selection method. In this
method, multiple access-points collaborate with each
other in order to accommodate users’ traffic. The
authors have formulated the proposed method as a
mathematical programming and maximized the
throughput of the users. However, this method is cen-
tralized optimization and does not consider dynamic
situations. Wu et al.13,14 have proposed channel selec-
tion and access scheduling methods for wireless mesh
networks and sensor networks. These methods select a
channel for each wireless node and schedule packet
transmission among wireless nodes based on Latin
squares. Although these methods enhance the through-
put of wireless nodes, they also do not consider
dynamic situations. Zhu et al.15 have proposed a chan-
nel selection method for sensor networks in IoT envir-
onments, which utilizes the multi-armed bandit model.
This method assumes dynamic situations, but does not
consider an access-point selection problem. Bai et al.11
have proposed an access-point selection for enterprise
wireless local area networks (WLANs), which collects
information on networks and selects access-point based
on stochastic dominance tools. Xu et al.10 have
proposed an online access-point selection method
named SmartAssoc that aims at maximizing the mini-
mum throughput among users. SmartAssoc considers
dynamic situations. When a new user arrives, the user
connects to an access-point based on p-norm of current
loads for access-points. Although these access-point
selection methods work under dynamic situations, they
do not consider a channel selection problem.
In this article, we deal with a joint optimization
problem of access-point and channel selection for IoT
environments, considering dynamic situations. We first
introduce a mixed-integer programming (MIP) for the
joint optimization problem of access-point and channel
selection. The MIP represents ideal performance, but it
is difficult to apply the MIP to large-scale problems in
dynamic situations. We then propose a new access-
point and channel selection method using Markov
approximation in order to adapt to dynamic changes in
network conditions. Markov approximation is a dis-
tributed optimization framework16 and provides
approximate solution for optimization problems. In
Markov approximation, a network system is designed
in such a way that it follows a time-reversible continu-
ous-time Markov chain, where its states correspond to
network conditions. The sojourn time in each state
depends on the utility value of the network system. By
doing so, the time average of the utility value
approaches the optimal value. The proposed method
searches approximate solution for the joint optimiza-
tion problem of access-point and channel selection on
the time-reversible continuous-time Markov chain,
aiming at maximizing the minimum throughput of
wireless nodes. To do so, the proposed method repre-
sents combinations of access-points selected by wireless
nodes and channels selected by access-points as states
of the Markov chain.
The motivation for this article is as follows. Markov
approximation optimizes the network configurations
by adaptively changing independent behaviors of users
(i.e. wireless nodes) with time based on minimum neces-
sary information. It flexibly responds to dynamic char-
acteristics of the network conditions caused by arrival
and departure of wireless nodes because the network
configurations are optimized by just the behaviors of
the mobile users without network operators. Therefore,
it is suitable for situations where network conditions
frequently change. In IoT environments, network con-
ditions often change in a given area. In such environ-
ments, as mentioned above, conventional optimization
techniques used in access-point and channel selection
methods do not work well. They are done in centralized
control by resolving optimization problems according
to a network condition at a certain time. In these meth-
ods, network operators collect information on the net-
work, and they optimize and configure the whole
network, which has large overhead, every time the
Kimura et al.
network condition is updated. Therefore, they are not
suitable for IoT environments. To overcome this diffi-
culty, we apply Markov approximation to the access-
point and channel selection problem.
Note that the previous works about Markov approx-
imation assumed static situations where the network
condition do not change despite the characteristic of
Markov approximation, which is suitable for IoT envir-
onments. In this article, we focus on the adaptability of
Markov approximation to dynamic situations. The pro-
posed method applies Markov approximation to the
access-point and channel selection problem so as to
adapt to the dynamic characteristic of the IoT environ-
ments and we reveal the impact of the proposed method
through simulation experiments.
This article is an extended version of our conference
paper. The conference paper introduced the main idea
of our proposed method and showed its fundamental
performance. In this article, we modify the proposed
method in order to improve not only total throughput
of users but also throughput fairness of users by chang-
ing the definition of the objective function of the opti-
mization problem. Furthermore, this article provides
the MIP for the optimization problem to obtain its
optimal solution. We also add the results of simulation
experiments in several scenarios for depicting the per-
formance of the proposed method in detail.
The rest of this article is organized as follows. We
first describe the system model and access-point selec-
tion problem. We then introduce the MIP for the prob-
lem and provide the proposed dynamic selection
method using Markov approximation. We present
experimental results to evaluate the performance of our
proposed method. Finally, we conclude this article.
System model and access-point selection
problem
System model
We assume that there are multiple users (wireless nodes)
and multiple access-points in a given area, as shown in
Figure 1. Let K(t), A, and C denote the set of users at
time t, the set of access-points, and the set of channels,
respectively. Each access-point a 2 A selects one of the
jCj channels, and each user k 2 K(t) selects one of the
jAj access-points. We also assume dynamic situations
where users dynamically arrive at and depart from the
area with time t.
The system state is represented by combinations of
channels selected by access-points and access-points
selected by users. We define the 1 3 jAj vector x as the
access-point state, in which the ith element
xi 2 f1, 2, . . . , jCjg is the channel selected by access-
point i. We also define the 1 3 jK(t)j vector y as the
user state, in which the jth element yj 2 f1, 2, . . . , jAjg
3
Figure 1. System model (jK(t)j = 5, jAj = 3).
is the access-point selected by user j. With these defini-
tions, the system state at time t is denoted by (x, y). We
further define the set Z(t) of feasible system states at
time t as follows
Z(t)
= f(x, y)jxi 2 f1, 2, . . . , jCjg, yj 2 f1, 2, . . . , jAjg
i = 1, 2, . . . , jAj, j = 1, 2, . . . , jK(t)jg
In this article, we refer to z 2 Z(t) as a strategy.
When there is no need to distinguish between access-
points and users, each of them is collectively referred to
as a player.
Let Nk (z) and uk (z) denote the total number of users
that use the same channel as user k (including user k)
and the throughput of user k, respectively, under strat-
egy z. Let mk, a denote the maximum throughput that
user k achieves when the user connects to access-point
a and exclusively occupies a single channel. We assume
that when there are multiple users on a channel, the fre-
quency resources on the channel are evenly available
for their communications by means of time-division
multiplexing (i.e. uk (z) = mk, a =Nk (z) for user k connect-
ing access-point a under strategy z).
Access-point and channel selection problem
This article deals with a problem that maximizes the
minimum throughput of users at a given time t. The
objective function of the problem is as follows
 
max min uk (z) ð1Þ
z2Z(t) k2K(t)
Equation (1) is a combinatorial optimization problem
which is known to be non-deterministic polynomial-
time (NP)-hard. Therefore, a brute-force search is the
only way to correctly obtain the optimal solution.
However, even if the numbers of users and
4 International Journal of Distributed Sensor Networks

Table 1. List of symbols in MIP.

Symbol Meaning

c Real variable that represents the minimum throughput of users.

Yk, a Binary variable that is equal to 1 if user k 2 K(t) uses access-point k; otherwise, 0.
Xa, c Binary variable that is equal to 1 if access-point a 2 A uses channel c 2 C; otherwise, 0.
dk, a, c Binary variable that is equal to 1 if user k 2 K(t) connects to access-point a 2 A with channel c 2 C; otherwise, 0.
gk, k0 , c Binary variable that is equal to 1 if user k 2 K(t) and user k0 2 K(t) use the same channel c 2 C; otherwise, 0.
Nk Positive integer variable that indicates the number of users that use the same channel as user k 2 K(t).
M Sufficiently large value.
sk, n Binary variable that is equal to 1 if Nk = n; otherwise, 0.
mk, a, n Binary variable that is equal to 1 if user k 2 K(t) uses access-point a 2 A and Nk = n.

access-points are small, the total number of strategies 8k 2 K(t), a 2 A, n = 1, 2, . . . , jK(t)j;

jZ(t)j becomes extremely large. This makes it difficult Yk, a + sk, n  1  mk, a, n ð9Þ
to obtain the optimal solution with a brute-force search Yk, a  mk, a, n , sk, n  mk, a, n
in a reasonable time. Moreover, under situations where
jK(t)j
XX
users dynamically arrive and depart, we need to obtain
8k 2 K(t); mk, a mk, a, n =n  c ð10Þ
the optimal strategy z 2 Z(t) in order to maximize the n = 1 a2A
utility every time the system conditions change. Our
proposed method thus uses Markov approximation16 Equation (2) is the objective function, which indi-
to obtain approximate solution to equation (1) in a cates the minimum throughput of users. Equation (3)
dynamic manner. means that user k connects to one of the access-points.
Equation (4) indicates that access-point a selects one of
the channels. Equations (5)–(7) provide the number Nk
MIP of users that use the same channel as user k. Equations
Before we explain the proposed access-point and chan- (8)–(10) calculate the throughput of user k, and equa-
nel selection method, we introduce the following MIP tion (10) represents that the throughput of each user is
that maximizes the minimum throughput of users at a equal to or more than c. Therefore, equations (2) and
given time t. The symbols used in the MIP are listed in (10) maximize the minimum throughput of users, which
Table 1. correspond to equation (1).
Maximize Although the MIP provides the optimal solution, it
is not suitable for large-scale problems and dynamic
c ð2Þ situations because the MIP is NP-hard. Therefore, the
proposed access-point and channel selection method
subject to
aims to obtain approximation solution, using Markov
X approximation.
8k 2 K(t); Yk, a = 1 ð3Þ
a2A
X
8a 2 A; Xa, c = 1 ð4Þ Dynamic selection using Markov
c2C approximation
8k 2 K(t), c 2 C, a 2 A; Xa, c + Yk, a  1  dk, a, c This section first gives an overview of Markov approxi-
ð5Þ
Xa, c  dk, a, c , Yk, a  dk, a, c mation and then describes the proposed access-point
0 selection method that uses Markov approximation
P k 2 K(t),P
8k, c 2 C;
under dynamic situations.
dk, a, c + dk 0 , a, c  1  g k, k 0 , c
P
a2A a2A P ð6Þ
dk, a, c  gk, k 0 , c , dk 0 , a, c  gk, k 0 , c
a2A a2A Markov approximation
X X
8k 2 K(t); Nk = 1 + gk, k 0 , c ð7Þ Under static situations, equation (1) is equivalent to16,17
k 0 2K(t)nfkg c2C X
max pz F z
8k 2 K(t), n = 1, 2, . . . , jK(t)j; p(t)0
z2Z(t)
Nk  n  M(1  sk, n ) ð8Þ X ð11Þ
subject to pz = 1
(Nk  n)  M(1  sk, n )
z2Z(t)
Kimura et al. 5

where pz is the percentage of time that strategy z 2 Z(t) With pz , the approximate value of equation (11) is
is selected, p(t) is the 1 3 jZ(t)j vector (p1 p2    pjZ(t)j ), given by
and Fz = mink2K(t) uk (z). X
Markov approximation obtains approximate solu- pz Fz ð14Þ
tion to an optimization problem using the following z2Z(t)
relation
As b increases, the accuracy of the approximation
improves, and the value of equation (14) approaches
maxfw1 , w2 , . . . , wn g
! the optimal value.
1 Xn
We now consider a time-reversible continuous-time
 log exp(bwi )
b i=1
Markov chain, where states are elements in Z(t) and
each steady-state probability is equal to pz . In order to
1
 maxfw1 , w2 , . . . , wn g + logn construct such a time-reversible continuous-time
b
Markov chain, for each pair of adjacent states
where b is a positive constant, and w1 , w2 , . . . , wn are z, z0 2 Z(t), the local balance equation should be satis-
non-negative variables. From this relation, we have the fied as follows16
log-sum-exponential (LSE) function as follows
pz qz, z0 = pz0 qz0 , z
!
1 Xn
where qz, z0 represents the transition rate from the state z
max(w1 , w2 , . . . , wn ) = lim log exp(bwi )
b!‘ b
i=1
to its adjacent state z0 . One way to satisfy the local bal-
ance equation is to set the transition rate qz, z0 to
When we define wi as
a(t)
qz, z0 = ð15Þ
wi = Fzi , zi 2 Z(t), i = 1, 2, . . . , jZ(t)j exp(bFz )

we have the following approximation where a(t) is a parameter controlling the transition rate.
! As we can see from equation (15), the transition rate
1 X depends on the utility value. Specifically, transition
max Fz ’ log exp(bFz ) ð12Þ rates from states with high utility values to other states
z2Z(t) b z2Z(t) are small, and thus, the sojourn time of the states
becomes large. However, the sojourn time of states with
From the above, we can see that the accuracy of the
low utility values is small. As a result, the time average
approximation depends on b, and we obtain the opti-
of the utility values approaches to the optimal value. In
mal value when b = ‘.
the literature,16 performance evaluations were con-
Owing to the fact that the LSE function is a closed
ducted under static situations with fixed a(t). In
and convex function, we obtain the following approxi-
dynamic situations, however, we should dynamically
mation for equation (11) from equation (12)16
adjust the value of a(t) according to system conditions
X 1 X because the transition rate qz, z0 depends on the number
max pz Fz  pz logpz of users.
p(t)0
z2Z(t)
b z2Z(t)
X
subject to pz = 1 Proposed method
z2Z(t)
In our proposed method, each access-point selects a
This is a nonlinear programming problem, and thus, channel and each user selects an access-point according
we can obtain an optimal solution pz by solving the to a time-reversible continuous-time Markov chain on
Karush–Kuhn–Tucker conditions Z(t). To do so, each player follows the flow in Figure 2.
Each player first generates a counter r that is random
Fz  b1 logpz  b1 + h = 0, 8z 2 Z(t) value according to the exponential distribution with a
P 
pz = 1 parameter l(t) depending on the current utility Fz (i.e.
z2Z(t)
minimum throughput of users). For access-points, l(t)
h0
is defined as
where h is a Lagrange multiplier. Solving the above
problem, pz (z 2 Z(t)) is given by a(t)(jCj  1)
l(t) = ð16Þ
exp(bFz )
exp(bFz )
pz = P ð13Þ and for users, l(t) is defined as
exp(bFz0 )
z0 2Z(t)
6 International Journal of Distributed Sensor Networks
Figure 2. Player operation.
a(t)(jAj  1)
l(t) = ð17Þ
exp(bFz )
Note that each player knows the current utility by
exchanging information on its current throughput.
After setting r, each player decreases the counter r with
time.
When the counter r of a player reaches zero, the
player changes a current strategy as follows. In the case
where the player is access-point a, it randomly selects a
channel from Cnfcg (i.e. a channel other than the cur-
rent channel c) and switches to that channel. Similarly,
in the case where the player is user k, it randomly selects
an access-point from Anfag (i.e. an access-point other
than the current access-point a) and reconnects to the
selected access-point. Immediately after the new strat-
egy znew is adopted, the player changing the strategy
sends a notification to all other players. When receiving
the notification, each player then estimates and broad-
casts its current throughput. By doing so, each player
knows the new utility Fznew . Furthermore, each player
generates a new counter r with Fznew and repeats the
above procedure.
When the number of users changes, all players are
also notified of the change in the utility. Specifically,
when a new user arrives, the new user sends a notifica-
tion to participate. Moreover, when a user departs, it
sends a notification immediately before the departure.
Players receiving these notifications halt their count-
downs and then reset their counters to random value
according to the exponential distribution with the para-
meter l(t).
In our proposed method, we define a(t) in equations
(16) and (17) as
Figure 3. Example of change in M(t).
Figure 4. Feasible transitions from strategy z.
j  exp(bM(t))
a(t) =
jAj(jCj  1) + jK(t)j(jAj  1)
where j is a positive constant, and M(t) is the maxi-
mum utility from the occurrence of a change in the user
number to time t (Figure 3). M(t) is updated as follows.
When a notification of a change to strategy z is received
at time t, if Fz .M(t), then M(t) is updated by setting
M(t) :¼ Fz . This update is also performed when the
number of users changes at time t. Because the number
of feasible transitions from strategy z is equal to
jAj(jCj  1) + jK(t)j(jAj  1) (see Figure 4), the transi-
tion rate qz (t) from strategy z is given by
a(t)ðjAj(jCj  1) + jK(t)j(jAj  1)Þ
qz (t) =
exp(bFz )
j  exp(bM(t))
=
exp(bFz )
When Fz = M(t), the strategy changes according to
the exponential distribution with the rate qz (t) = j. In
contrast, when Fz \M(t), we have qz (t).j, and thus,
the mean time over which strategy z is adopted is
shorter than that in strategies that attain M(t) as shown
in Figure 5.
Performance evaluation
Evaluation model
We evaluate our proposed method through simulation
experiments. We consider two scenarios with different
Kimura et al.
Figure 5. Strategy changes in our proposed method.
values for maximum throughput mk, a . In the first sce-
nario, we set mk, a = 50 for each user k 2 K(t) and
access-point a 2 A. In the second scenario, mk, a is ran-
domly selected from among f40, 45, 50g. In the follow-
ing, we refer to the first scenario as the homogeneous
scenario and the second scenario as the heterogeneous
scenario.
We assume that immediately after the strategy of a
player changes, other players can know the change. We
set the number jAj of access-points to 20, the number
jCj of channels to 10, and the parameter j to 0:01. At
time t = 0, each user randomly selects an access-point,
and each access-point randomly selects a channel.
We use three indices as performance metrics: the
mean utility, the mean number of iterations, and the
mean total throughput. The mean utility is the time
average of the utility value normalized by f, where f
denotes the time P average of the optimal value of the
total throughput k2K(t) uk (z) divided by the number
jK(t)j of users. Note that the optimal value of the total
throughput can be easily obtained, that is,
jCj 3 maxfmk, a g = 10 3 50 = 500. The mean number
of iterations is the mean number of strategy changes
per player per unit time, and the mean total throughput
is the time average of the total throughput of users.
Unless stated otherwise, in each simulation experiment,
the simulation runs until 50,000 time units.
For the sake of comparison, we show the results of
the RSSI-based method, the simulated annealing
method, SmartAssoc method,10 and the proposed
method, the objective function of which is maximizing
the total throughput (abbreviated as the throughput-
based method).18
The RSSI-based method. A player is randomly chosen
among K(t) [ A according to a Poisson process with
rate 0.1 and then the player changes its strategy. In the
case where the player is a user, it selects the access-
point that has the highest maximum throughput (i.e.
mk, a = 50). If there are multiple access-points with the
highest maximum throughput, the access-point is ran-
domly selected among them. In contrast, in the case
where the player is an access-point, it randomly selects
the channel among C.
7
The simulated annealing method. The optimization prob-
lem equation (1) is solved by the simulated annealing
algorithm,19 which is one of the most widely used opti-
mization techniques. This algorithm simulates the cool-
ing process by gradually decreasing the temperature
until the temperature is sufficiently low. The initial state
zinit of the system is randomly chosen from Z(t) and the
temperature T is initially set to be 106 . Until the tem-
perature T decreases below the threshold th (th = 1), the
following procedure is repeated: we randomly choose a
solution znew from the neighboring set N z of the current
state z, where N z is defined as the set of the feasible
states that transition from the current state z (Figure 4).
The solution znew is accepted with the following prob-
ability PA (T )

1 (Fz  Fznew  0)
PA (T ) = ð18Þ
exp(Fz Fznew )=T (Otherwise)
If the solution znew is accepted, then the current state
z is updated to the solution znew , that is, z :¼ znew . After
that, the temperature T is updated to T :¼ 0:95T . In
our experiments, we conducted 100 trails for different
initial states, and obtained the best solution among
solutions in these trails.
SmartAssoc. SmartAssoc is an online access-point selec-
tion method for maximizing throughput fairness.10
When a user arrives, it selects an access-point according
to the following procedure. Let Ac denote a set of
access-point using channel c 2 C. For each access-point
a 2 Ac in each set Ac , the user calculates Lp norm
loads for all the access-points in Ac in the case where
P a. The Lp norm
the user connects to the access-point
load P is defined as Lp = ( a2Ac Lpa )1=p , where
La = k2Ka (1=mk, a ), Ka denotes the set of users using
access-point a, and p is the natural logarithm of the
number of the access-points in Ac (i.e. p = ln(jAc j)).
After calculating Lp norm loads for all access-points in
A, the user selects the best candidate access-point that
has the smallest Lp norm load. However, the access-
point selects a channel randomly.
The throughput-based method. The objective function
equation (1) is substituted
P by the normalized total
throughput, that is, Fz = k2K uk (z)=jKj.18
Evaluation results
To evaluate the basic performance of our proposed
method, we first consider static situations where the
number of users is fixed. Figure 6 shows the mean util-
ity value as a function of the parameter b. In both sce-
narios, the mean utility value of our proposed method
increases with b. This result indicates that, for large b,
8 International Journal of Distributed Sensor Networks

Figure 6. Mean utility value: (a) homogeneous scenario and (b) heterogeneous scenario.

Figure 7. Mean number of iterations: (a) homogeneous scenario and (b) heterogeneous scenario.

our proposed method can select access-points effec-
tively in static situations. The proposed method can
obtain the optimal value when b = ‘, which is the char-
acteristic of LSE function. Therefore, the proposed
method works well when b is large.
Figure 7 shows the mean number of iterations as a
function of b. In the homogeneous scenario, the mean
number of iterations is relatively small, whereas in the
heterogeneous scenario the mean number of iterations
rapidly increases with b. These results imply that the
utility value in the homogeneous scenario tends to be
larger than that in the heterogeneous scenario. In the
proposed method, strategies with high utility values
remain for long periods because the transition rates
from the strategies to other strategies are set to be low
values (see equations (16) and (17)). However, strategies
with low utility values are frequently changed to other
strategies. In the homogeneous scenario, the maximum
throughput that users can achieve is the same for each
access-point (i.e. mk, a = 50). Therefore, even if a user
changes an access-point, it is not easy to decrease the
throughput. In contrast, in the heterogeneous scenario,
the maximum throughput of users is different, and thus,
the throughput tends to decrease when a user changes
an access-point. As a result, the number of iterations in
the heterogeneous scenario becomes large.
Figure 8 shows the normalized utility value as
a function of elapsed time t, where b = 5.
‘‘Instantaneous’’ refers to the instantaneous utility
value, and ‘‘Average’’ is the average utility value at
each time. In both scenarios, strategy changes occur
more frequently when jK(t)j = 50. The reason is that
strategies with small utility values are more likely to be
selected when jK(t)j = 50 than when jK(t)j = 20.
Moreover, there is also a higher probability of selecting
strategies with small utility values in the heterogeneous
scenario than in the homogeneous scenario. This results
in frequent strategy changes in the heterogeneous
scenario.
We then compare the proposed method with the
RSSI-based method, the simulated annealing method
and the throughput-based method. Figure 9 shows the
Kimura et al.
Figure 8. Changes in utility value with elapsed time t (static situation): (a) jK(t)j = 20 (homogeneous scenario), (b) jK(t)j = 50
(homogeneous scenario), (c) jK(t)j = 20 (heterogeneous scenario), and (d) jK(t)j = 50 (heterogeneous scenario).
mean utility value and the mean total throughput as
a function of b, where jK(t)j = 50. In the figures,
‘‘Proposal (Fairness),’’ ‘‘Proposal (Throughput),’’
‘‘Simulated Annealing,’’ ‘‘SmartAssoc,’’ and ‘‘RSSI’’
refer to our proposed method, the throughput-based
method, the simulated annealing method, SmartAssoc
method, and the RSSI-based method, respectively. In
SmartAssoc, when new users arrive, they sequentially
select access-points. We also plot the optimal values of
the total throughput in the results. As we can see from
Figure 9(a) and (c), for large b, the mean utility value
of our proposed method is larger than that of other
methods. Specifically, our proposed method can
improve the throughput fairness of users. Moreover,
from Figure 8, we observe that the total throughput of
our proposed method is comparable with that of the
throughput-based method and the optimal value of the
total throughput. We also observe that our proposed
method outperforms the RSSI-based method,
SmartAssoc and the simulated annealing method in
terms of the total throughput. These results indicate
that the proposed method increases not only the
throughput fairness of users but also the total through-
put. However, for small b, the total throughput of our
9
proposed method is smaller than that of the RSSI-
based method in the heterogeneous scenario. This
stems from the fact that our proposed method ran-
domly selects the access-point among all access-points,
whereas the RSSI-based method selects the access-
point among the access-points with the highest RSSI
value. This result implies that the performance of our
proposed method could be improved by restricting the
selection of access-point to access-points that have high
RSSI value. We left the evaluation of the effectiveness
of the restriction for the future work.
To demonstrate that our proposed method is effec-
tive under the dynamic situations where the number of
users changes dynamically with time, we consider the
following situation: jK(t)j = 50 for 0\t  20, 000,
jK(t)j = 30 for 20, 000\t  40, 000, jK(t)j = 50 for
40, 000\t  60, 000, jK(t)j = 20 for 60, 000\t 
80, 000, and jK(t)j = 40 for 80, 000\t  100, 000. We
also show the results of SmartAssoc and the simulated
annealing method. As for the simulated annealing
method, the simulated annealing algorithm is con-
ducted every time the number of users in the system is
changed. Note that in SmartAssoc, we sequentially add
all the users to the system in random order. Moreover,
10 International Journal of Distributed Sensor Networks

Figure 9. Performance comparison (jK(t)j = 50): (a) mean utility (homogeneous scenario), (b) mean total throughput
(homogeneous scenario), (c) mean utility (heterogeneous scenario), and (d) mean total throughput (heterogeneous scenario).

Figure 10. Changes in utility with elapsed time t (dynamic situation): (a) homogeneous scenario and (b) heterogeneous scenario.

to fairly compare the performance, we modify
SmartAssoc in such a way that players can dynamically
change their strategies. Specifically, a player is ran-
domly chosen among K(t) [ A according to a Poisson
process with rate 0.1 and then the player changes its
strategy. When the player is a user, it selects the access-
point using SmartAssoc algorithm. By contrast, when
the player is an access-point, it randomly selects the
channel among C.
Figure 10 shows the normalized utility value as a
function of elapsed time t, where b = 5. From this fig-
ure, we observe that the normalized utility of
SmartAssoc is stable because users periodically selects
access-points with small Lp norm load. We also observe
Kimura et al.
that immediately after the number of users jK(t)j
changes, the utility value of our proposed method
dropped significantly. As the time goes by, the mean
utility value of our proposed method becomes large
and is larger than the utility values of SmartAssoc and
the simulated annealing method, because the selection
of strategies with large utility values increases. This
result indicates that our proposed method can adapt to
sudden changes in the number of users and is an effec-
tive method for selecting access-points.
Conclusion
We have proposed a new method for selecting access-
points and channels using Markov approximation.
Through simulation experiments, we showed that our
proposed method can obtain the high mean utility
value under static situations. Therefore, our proposed
method is effective at improving the throughput fair-
ness of users. We also showed that our proposed
method adapts to dynamic changes in the number of
users. In this article, we assumed instantaneous notifi-
cation of strategy changes throughout the network. In
practice, however, there is some latency in the exchange
of information. We thus will consider the effects of
latency in the future work.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
References
1. Al-Fuqaha A, Guizani M, Mohammadi M, et al. Inter-
net of Things: a survey on enabling technologies, proto-
cols, and applications. IEEE Commun Surv Tut 2015; 17:
2347–2376.
2. Gazis V. A survey of standards for Machine-to-Machine
(M2M) and the Internet of Things (IoT). IEEE Commun
Surv Tut 2017; 19: 482–511.
3. Gubbi J, Buyya R, Marusic S, et al. Internet of Things
(IoT): a vision, architectural elements, and future direc-
tions. Future Gener Comp Sy 2013; 29: 1645–1660.
4. Whitmore A, Agarwal A and Xu LD. The Internet of
Things–a survey of topics and trends. Inform Syst Front
2015; 17: 261–274.
5. Xu LD, He W and Li S. Internet of things in industries:
a survey. IEEE T Ind Inform 2014; 10: 2233–2243.
11
6. Balachandran A, Bahl P and Voelker GM. Hot-spot con-
gestion relief in public-area wireless networks. In: Pro-
ceedings of the 4th IEEE workshop on mobile computing
systems and applications (WMCSA’02), Callicoon, NY,
20–21 June 2002. New York: IEEE.
7. Akl RG and Park S. Optimal access point selection and
traffic allocation in IEEE 802.11 networks. In: Proceed-
ings of the 9th world multi-conference on systemics, cyber-
netics and informatics (WMSCI’05), Orlando, FL, 10–19
July 2005.
8. Hamamoto R, Takano C, Obata H, et al. An access
point selection mechanism based on cooperation of
access points and users movement. In: Proceedings of the
2015 IFIP/IEEE international symposium on integrated
network management (IM’15), Ottawa, ON, Canada,
11–15 May 2015. New York: IEEE.
9. Koutsopoulos I and Tassiulas L. Joint optimal access
point selection and channel assignment in wireless net-
works. IEEE ACM T Network 2007; 15: 521–532.
10. Xu F, Zhu X, Tan C, et al. SmartAssoc: decentralized
access point selection algorithm to improve throughput.
IEEE T Parall Distr 2013; 24: 2482–2491.
11. Bai Y, Vuran M, Batur D, et al. Stoop: stochastically-
dominant access point selection in enterprise WLANs. In:
Proceedings of the 2017 IEEE international conference on
smart computing, Hong Kong, China, 29–31 May 2017.
New York: IEEE.
12. Karimi OB, Liu J and Rexford J. Optimal collaborative
access point association in wireless networks. In: Pro-
ceedings of the 2014 IEEE INFOCOM, Toronto, ON,
Canada, 27 April–2 May 2014. New York: IEEE.
13. Wu D, Bao L, Regan AC, et al. Large-scale access sche-
duling in wireless mesh networks using social centrality. J
Parallel Distr Com 2013; 73: 1049–1065.
14. Wu D, Bao L and Liu CH. Scalable channel allocation
and access scheduling for wireless Internet-of-Things.
IEEE Sens J 2013; 13: 3596–3604.
15. Zhu J, Song Y, Jiang D, et al. Multi-armed bandit chan-
nel access scheme with cognitive radio technology in wire-
less sensor networks for the Internet of Things. IEEE
Access 2016; 4: 4609–4617.
16. Chen M, Liew SC, Shao Z, et al. Markov approximation
for combinatorial network optimization. IEEE T Inform
Theory 2013; 59: 6301–6327.
17. Chen S, Gan X, Feng X, et al. Markov approximation
for Multi-RAT selection. In: Proceedings of the 2015
IEEE international conference on communications
(ICC’15), London, 8–12 June 2015. New York: IEEE.
18. Kimura T, Hirata K and Muraguchi M. Dynamic access-
point selection method using Markov approximation. In:
Proceedings of the 2017 international conference on infor-
mation networking (ICOIN’17), Da Nang, Vietnam, 11–
13 January 2017. New York: IEEE.
19. Van Laarhoven PJM and Aarts EHL. Simulated anneal-
ing. In: Van Laarhoven PJM and Aarts EHL (eds) Simu-
lated annealing: theory and applications, vol. 37.
Dordrecht: Springer, 1987, pp.7–15.