2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications

(GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)

Mobile Device Training Strategies in Federated

Learning: An Evolutionary Game Approach
Yuze Zou:˚ , Shaohan Feng: , Dusit Niyato: , Yutao Jiao: , Shimin Gong; , and Wenqing Cheng˚
: School of Computer Science and Engineering, Nanyang Technological University, Singapore
˚ School of Electronic Information and Communications, Huazhong Univ. of Science & Technology, China
; School of Intelligent Systems Engineering, Sun Yat-sen University, China

Abstract—With the tremendous success of ma-
chine learning and increasingly powerful mobile
devices, federated learning has gained growing
attention from both academia and industry. It
capitalizes on vast number of distributed data to
support machine learning based applications while
maintaining data privacy. In this paper, we con-
sider a federated learning system, in which the
mobile devices allocate their data and computation
resources among the machine learning applications,
i.e., model owners. Specifically, we formulate an
evolutionary game mode for the mobile devices
with bounded rationality to adapt their training
strategies aiming to maximize the device’s indi-
vidual utility. The uniqueness and stability of the
equilibrium of the game are analysed theoretically.
Besides, Extensive experiments are conducted to
determine the functions fitted for the accuracy and
energy consumption metrics.
Index Terms—Federated Learning, Evolutionary
Game, Equilibrium
I. Introduction
Recently, due to the great success in machine learn-
ing, extensive applications based on machine learning
techniques, e.g., face recognition, image classification,
and natural language processing, have been develope-
d [1]. However, the conventional centralized machine
learning techniques are facing thorny issues concern-
ing data privacy, e.g., data abuse and leakage, which
hinder the machine learning based applications from
touching the massive data generated by the mobile
devices. To address the issues, federated learning has
been introduced as one of the most promising solu-
tions. Lately, Google have built a scalable production
system for federated learning in the domain of mobile
devices, based on TensorFlow. [2] Briefly, federated
learning is the collaborative machine learning that
fully utilizes powerful mobile devices, takes advan-
tage of the intelligence at the end of the network,
and accomplishes the learning tasks in a distributed
manner [3], [4]. In the federated learning system, a
centralized server working as a coordinating cloud is
responsible for training machine learning models for
the model owners by using the federation of the mobile
devices, e.g., averaging the weights trained by the
federation. However, with the springing up machine
learning based applications, there are a number of
emerging federated learning based applications, each
of which is run by an individual model owner. The
applications and model owners need the data and
training services from the federation of the mobile
devices. In contrast, asking the mobile devices to work
as sacrificial volunteers is not a viable and sustainable
option. Therefore, mobile devices with limited re-
sources have to choose machine learning applications,
i.e., the model owners, to contribute their training
services. This selection is done based on the utility
which consists of incentives from the model owners
and the costs of executing different training services.
Moreover, the bounded rationality generally exists in
the federated learning environment. For example, the
accuracy of one model is determined by all mobile
devices participating in the training for that model
while the training strategy of one mobile device is
generally not known to others. Motivated by these
observations, we propose an evolutionary game model
to investigate the training strategies of the mobile
devices. Specifically, the contributions of this paper
mainly lie in two folds:
‚ We proposed two metrics, i.e., the accuracy and
energy consumption of federated learning per-
formance, which correspond to the benefit and
cost, respectively. Extensive experiments are con-
ducted to determine the function approximations
fitted for the accuracy and energy consumption
metrics. Accordingly, the power law function and

978-1-7281-2980-8/19/$31.00 ©2019 IEEE 874

DOI 10.1109/iThings/GreenCom/CPSCom/SmartData.2019.00157
 ś Ř 'RZQORDG PRGHO
ř 7UDLQ PRGHO
Ś 8SORDG ZHLJKWV
ś $YHUDJH ZHLJKWV
Ř Ś
Ś
Ś Ř Ř
ř ř ‫ڮ‬ ř 'HYLFHV
(a) Federated learning workflow (b) System Model
Fig. 1. Federated learning workflow, system model and training service process
linear model are obtained respectively.
‚ An evolutionary game model is formulated to
investigate the mobile devices’ training strategies
under the scenario of multiple model owners. The
uniqueness and stability of the evolutionary equi-
librium are analysed, and the stability is further
validated numerically.
The remainder of the paper is organized as fol-
lows: Section II gives a brief introduction of federated
learning and describes the system model. Evolutionary
game is applied to formulate the problem in the Sec-
tion III, where the uniqueness and Lyapunov stability
are proved. Numerical performance and conclusion are
given in Section IV and Section V, respectively.
II. System Description
A. Preliminary of Federated Learning
The federated learning system works in four phases
as shown in Fig. 1(a), which are: 1) The central
coordinator, i.e., model owner distributes its model
to the mobile devices. 2) The mobile devices train
the machine learning model received from the model
owner using its local dataset. 3) After training, mobile
devices upload the trained weights back to the model
owner. 4) Then, the model owner averages all the
weights uploaded by the mobile devices. This process
repeats until the machine learning model achieves the
desired performance.
B. System Model
We consider the federated learning system that
consists of K model owners served by N mobile
devices as depicted in Fig. 1(b). The index sets of
model owners and mobile devices are denoted by K fi
t1, 2, . . . , K u and N fi t1, 2, . . . , N u, respectively.
Each model owner has a particular machine learning
model to train. For example, the model owners may
have different types of models, such as different neural
0RELOH'HYLFH
0RGHO
‫ ڮ‬2ZQHUV
0RGHO 8WLOLW\
5HFHLYHU 0RQLWRU
7UDLQLQJ
8QLW
:HLJKWV 'HFLVLRQ
‫ڮ‬ 0RELOH 6HQGHU 0DNHU
'DWDVHW
(c) Training service process
networks, e.g., convolutional neural network (CNN),
recurrent neural network (RNN), and long short-term
memory (LSTM), or the same type of models with
different parameter settings, e.g., the number of layers
and neurons. Likewise, the mobile devices also have
different types, such as smart phones, tablets, and
wearable devices.
The training service process of a mobile device is
shown in Fig. 1(c). The mobile device feeds the model
received from the model owner to the training unit and
the utility monitor records the utility of the training
strategy. After that, the decision maker determines the
training strategy for the next period according to the
utility. In the sequel, certain amount of the dataset
is selected for the training service. After training, the
updated weights are sent back to the model owner.
Let Mid denote the dataset size of mobile de-
vice i and xi,k denote the probability that mobile
device i provide training service to model owner k.
Let xi fi rxi,1 , xi,2 , . . . , xi,K s denote the training s-
trategy of mobile device i and X fi rx1J , x2J , . . . , xN J sJ
denote the training profile of the federation of mobile
devices. Then, the total amount ř of datad offered to
model owner k is given by N i“1 xi,k Mi . Let fk p¨q
denote the update function, which describes training
benefit of model owner k against training dataset
size, e.g., accuracy of the model. The update that
model ř owner k receives from mobile devices is given
by fk p N i“1 xi,k Mi q. In the sequel, we assume that
d
the payoff that model owner k offers to the mobile
devicesřis proportional to the update it receives, i.e.,
γi,k fk p N
i“1 xi,k Mi q, where γi,k is the payoff coefficient
d
of model owner k. Furthermore, mobile devices will
receive the payoff from model owners proportionally
according to their training contributions to the model
owners. For example, the contribution of mobile de-
x Md
vice i made to model owner k is given by řN i,kx iM d .
i“1 i,k i
Accordingly, the payoff that mobile device i can re-
875
ceive from model owner k is given by
ÿ
N
xi,k Mid
pi,k “ γi,k fk p xi,k Mid q řN ` αk ,
i“1 xi,k Mi
d
i“1
where αk ě 0 is a bias term of model owner k, which
offers an additional incentive for mobile devices to
train for it.
On the other hand, model training incurs costs,
e.g., energy consumption, due to the usage of the
computing resources. Let ci,k denote the cost per unit
data of mobile device i when training for the model
owner k. Then, the cost incurred by mobile device i
for training owner k’s model is given by
ci,k “ ηi,k xi,k Mid ,
where ηi,k denotes the cost coefficient of mobile de-
vice i’s training process for model owner k. The
coefficient may differ due to different complexities of
the training models. In summary, we define the utility
that mobile device i can gain from the training for
model owner k as follows:
ui,k “ pi,k ´ ci,k (1)
Thus, we can define ui fi rui,1 , ui,2 , . . . , ui,K sJ as
the utility vector of mobile device i.
III. Evolutionary Game Formulation
In this section, we give the evolutionary game
formulation for training strategies of the mobile de-
vices in the federated learning system. The replicator
dynamics is used to model the strategy adaptation
process. [5]
A. Game Formulation
The federated learning problem can be formulated
as an evolutionary game as follows:
Set of players pN q: The set of mobile devices N de-
notes the set of players of the game, i.e., the federation
of mobile devices in the system.
Set of actions pAq: The mobile devices, i.e., the
players aim to obtain the training strategies that
maximize their individual utilities. According to the
system model, mobile device i’s training strategy is
given by xi . Accordingly, we define Ś the set of action
for each mobile device as A fi kPK r0, 1s.
Utility function: The utility of mobile device i
training for model owner k evaluates its performance
of choosing training strategy xi,k , which is elaborated
in (1). Let U fi ru1 , u2 , . . . , uN s denote the utility
function matrix for the federation of mobile devices.
876
With the above definitions, we define the evolution-
ary game G as follows:
G “ pN , A, U q . (2)
B. Dynamics of Training Strategy Adaptation
In the context of the evolutionary game, each mo-
bile device will dynamically adapt its strategy accord-
ing to its utility. This is referred to as the evolution
of the game where the strategy adaptation of mobile
devices will affect the data assignment for model
owners, and therefore, the training strategy will evolve
over time.
In this case, the data assignment xi,k is function of
time t which can be denoted as xi,k ptq. The strategy
adaptation process and the corresponding training
strategy evolution can be modelled and analysed by
replicator dynamics [5], which is a set of ordinary
differential equations defined as follows:
x9 i,k ptq “ xi,k ptqpui,k ptq ´ ūi ptqq, (3)
for all xi,k P r0, 1s, with the initial data assignment
strategy xi,k p0q “ x0i,k , where ūi is the average utility
of the trainingřstrategy for mobile device i, which is
given by ūi “ kPK ui,k xi,k .
The replicator dynamics governs the rate of strategy
adaptation of the training strategy. As the game is
repeated, each mobile device observes its own utility
and compares it with the average utility. Then, in the
next period, the mobile device forms another strategy
if its utility is less than the average. By replicator
dynamics, the probability of a mobile device training
for model owner k will increase if the corresponding
utility is higher than the average (i.e., ui,k ą ūi ).
C. Uniqueness and Stability Analysis of the Evolution-
ary Equilibrium
Definition 1. Evolutionary Equilibrium: The evolu-
tionary equilibrium is the solution of the game defined
in (3).
Firstly, the uniqueness of the evolutionary equilib-
rium is guaranteed by the following theorem.
Theorem 1. The evolutionary game defined in (3) is
uniquely solvable and hence admits a unique evolution-
ary equilibrium.
Proof. The following proposition 1 guarantees that
x9 i,k ptq satisfies Lipschitz condition with respect to all
xj,l , pj, lq P N ˆ K. Hence, the evolutionary game de-
fined in (3) is uniquely solvable according to Cauchy-
Lipschitz theorem [6].
Proposition 1. Let gi,k ptq fi xi,k ptq pui,k ptq ´ ūi ptqq, TABLE I
then the first derivatives of gi,k ptq over xj,l ptq, for all Parameters Settings
pj, lq P N ˆ K are bounded. Value Value
N 3 K 3
Proof. Please see Appendix A. Md r100, 50000s η cf. Table II
Secondly, according to Lyapunov’s second method f p¨q » cf. (6) fi αk » [1
fi 1 2]
for stability [7], we justify the stability of the evolu- 0.2 0.5 0.3 1
X0 – 0.3 0.5 0.2 fl γi,k – 0.8 fl ˆ r8,32,64s
tionary equilibrium to the evolutionary game defined 0.2 0.5 0.3 0.6
100

in (3) as presented in Theorem 2.

Theorem 2. The evolutionary game defined in (3) 

admits a stable evolutionary equilibrium. 







(QHUJ\&RQVXPSWLRQ-


Proof. From the Lyapunov’s second method for sta-  



$FFXUDF\
 

bility, we design a Lyapunov function as follows: 





  

ř ř  
     
V pxi,k ptqq“t kPK xi,k ptqu ,
2
iPI (4) 



where V pxi,k ptqq satisfies 

  

          
$ 'DWDVHW6L]H 'DWDHW6L]H 

’
&“ 0, if xi,k “ 0, @i P N , k P K,
V pxi,k ptqq
’
(5)
%ą 0, otherwise.
By using Lyapunov function defined in (4), the evolu-
tionary equilibrium to the evolutionary game defined
in (3) can be proven to be stable. Please refer to
Appendix B for details.
IV. Performance Evaluation
A. Parameters Setting and Measurement
We evaluate three neural networks with different
number of hidden layer’s units (H). Basically, the
more hidden units a model has, the higher accuracy
it can yield at the cost of computational complexity,
which leads to higher energy consumption for mobile
devices. Models with H P t0, 64, 512u are evaluat-
ed.1 The parameters settings are listed in Table I.
According to Table I, model owner 3 provides the
highest γ and α compared with the other two model
owners, which represents the highest incentive for
mobile devices. Apart from that, all model owners
prefer mobile device 1 to the other two devices in
terms of γ. The reason may be that mobile device 1
has the highest reputation according to its training
history for the model owners.
1) Update Function Fitting: To obtain the update
function f p¨q of machine learning model(s), we con-
duct a set of machine learning experiments. In par-
ticular, we use the multi-class classification dataset,
MNIST [8]. The update function is a function that
elaborates the model performance, e.g., accuracy, over
1
To hold the expression consistency, we use H “ 0 here to
denote NN with no hidden layer.
(a) Accuracy (b) Energy Consumption
Fig. 2. Accuracy and Energy Consumption Fitting for Different
Learning Models
training dataset size (m). The update function may
vary from different models. Generally, however, the
update functions are nondecreasing, which means that
more training data contributes to a more accurate
model or at least the model with same accuracy. In
order to obtain a precise expression of f p¨q, we train
the model with varied training dataset size from 100
to 50000 and then test its accuracy on the test dataset.
The results and fitted update functions are given in
Fig. 2. We use power law function to fit the accuracy
curve, which is given by
f pmq “ c ´ am´b (6)
where a, b, c ą 0. c captures the upper bound of the
accuracy, i.e., lim f pmq “ c. Similar results also
mÑ8
appeared in [9]. The fitted parameters for the accuracy
models are listed in Table II. It is worth noting that
the fitness of these models are qualified with coefficient
of determination higher than 0.99, i.e., R2 ą 0.99.
2) Energy Consumption Measurement: To measure
the energy consumption of mobile devices, we adopt
Raspberry Pi 3 Model B [10], to mimic the mobile
device, which is a portable single-board computer that
has comparable hardware with mobile phones. The
energy consumption of the three models above are
depicted in Fig. 2(b) along with the linear regression
models. The energy consumption coefficients η are
listed in Table II, which are in mJ per 100 samples

877
 TABLE II with the setting that γ1,k ą γ2,k ą γ3,k , which means
Fitted Model Coefficients for Accuracy and Energy that the model owners prefer mobile device 1.
Consumption against Training Set Size
2) Data Assignment Variations: The dataset size
Accuracy Energy variations of model owners against time are shown in
H a b c R2 η R2 Fig. 4(b). Similarly, the dataset sizes converge to the
0 20.74 0.8090 0.9228 0.9979 1.2303 0.9991
64 10.18 0.7017 0.9608 0.9902 2.9587 0.9987 equilibrium after fluctuations. Besides, D1 ą D3 ą D2
512 2.26 0.4686 0.9875 0.9909 21.4793 1.0000 at the stable state. This shows that although model 3
has the highest accuracy, but is not the most energy
efficient one. In contrast, model 1 achieves the highest


 
(YROXWLRQDU\
HTXLOLEULXP energy efficiency, and thus mobile devices tend to

 offer their training service for model owner 1. On



 the other hand, no model owner obtain all data from

(YROXWLRQDU\

mobile devices. The reason is that the accuracies of
 HTXLOLEULXP 



       
(a)
Fig. 3. Direction Field of the replicator dynamics showing the
stability of the evolutionary equlibrium
training, i.e., 10´5 J / sample training. Similarly, the
fitness of the models are also qualified with R2 ą 0.99.
B. Demonstration of the Equilibrium Stability
The stability of the evolutionary equilibrium can
be verified and presented by using a direction field
of the replicator dynamics. We use the evolutionary
process training strategies of mobile device 1 and 2
for model owner 1 as well as training strategies of
mobile device 1 for all model owners as examples. The
direction fields of px1,1 , x2,1 q and px1,1 , x1,2 , x1,3 q are
shown in Fig. 3(a) and (b), respectively. As shown in
Fig. 3, any unstable strategy will follow the arrow to
reach the evolutionary equilibrium, which is marked
by the black circle. This demonstrates the stability
of the evolutionary equilibrium and is consistent with
Theorem 2.
C. Evolutionary Game Dynamics
Next, we investigate the dynamics of mobile devices’
utilities and training strategies along with the learning
rate of evolutionary game. The dataset size of mobile
devices are the same, where each of them has 20000
training data items. The dynamics are shown in Fig. 4.
1) Utility Variations: The utility variations of mo-
bile devices against time are shown in Fig. 4(a). The
utilities fluctuate first over time and then converge
to the evolutionary equilibrium. At the evolutionary
equilibrium, the utilities of three mobile devices show
an order of ū1 ą ū2 ą ū3 . These results are consistent


the models tend to saturate when dataset size is
 


 



larger than 10000 according to Fig. 2(a) while energy
consumptions still increase linearly with the dataset
(b) size according to Fig. 2(b). As a result, mobile devices
prefer to train models dispersively.
3) Training Strategy Variations: Figures 4(c)-(e)
depict the variations of training strategies of mobile
device 1, 2,and 3 against time, respectively. Although
these three devices are initialized with the same
amount of data, they converge to different training
strategies due to the different preferences of the model
owners with respect to these mobile devices.
4) Learning Rate Variations: Figure 4(f) shows
the learning rate of the evolutionary dynamics a-
gainst time. We define learning rate (lr ) as the L1
9
ř ř of the dynamic replicators X, i.e., lr fi
distance
iPN kPK |x
9 i,k ptq|. As shown in Fig. 4(f), the learning
rate decreases dramatically with time, which indicates
that the evolutionary converges quickly. In particular,
lr approaches to 0 when t ě 30.
V. Conclusion
In this paper, we have investigated mobile devices’
training strategy in federated learning system. Evo-
lutionary game theory has been adopted to model
dynamic strategies of the mobile devices with bounded
rationality. Two metrics, i.e., accuracy and energy
consumption of federated learning performance, which
define the benefit and cost, respectively, have been
measured and modeled for three different learning
models. Based on the measurements, we have con-
ducted the numerical evaluations and the results have
validated the stability of the evolutionary equilibrium.
These results can be applied for the scenario consists
of multiple model owners that served by a federation
of mobile devices, which is the trend of machine
learning era.

878
      


     


    

'DWD$VVLJQPHQW

/HDUQLQJUDWH


3DUWLWLRQ

3DUWLWLRQ

3DUWLWLRQ
0RGHO2ZQHU
8WLOLW\

  0RGHO2ZQHU    

0RELOH'HYLFH
0RGHO2ZQHU
0RELOH'HYLFH 
    
0RELOH'HYLFH

    


     

                                                            
7LPHW 7LPHW 7LPHW 7LPHW 7LPHW 7LPHW

(a) Utility (b) Data Assignment (c) Mobile Device 1 (d) Mobile Device 2 (e) Mobile Device 3 (f) Learning Rate
Fig. 4. Evolutionary Game Dynamics: Utility and Training Strategy Variations of Mobile Devices, Learning Rate

VI. Acknowledgment ∇t pyptqq ď 0 for all values of yptq ‰ 0. The ∇t pyptqq

This work was supported in part by the “Nation- is given as follows:
« ff
al Key R&D Program of China”(2018YFB1004504), ÿ ÿ ÿ ÿ
A*STAR-NTU-SUTD Joint Research Grant Call on ∇t pyptqq “ 2 xi,k ptq x9 i,k ptq
iPN kPK
Artificial Intelligence for the Future of Manufactur- ÿ ÿiPN kPK
ing RGANS1906, WASP/NTU M4082187 (4080), Sin- “ 2N xi,k ptq pui,k ptq ´ ūi ptqq
gapore MOE Tier 1 under Grant 2017-T1-002-007 iPN kPK
˜ ¸
RG122/17, MOE Tier 2 under Grant MOE2014-T2- ÿ ÿ ÿ
2-015 ARC4/15, Singapore NRF2015-NRF-ISF001- “ 2N xi,k ptqui,k ptq ´ ūi
iPN kPK iPN
2277, and Singapore EMA Energy Resilience under
Grant NRF2017EWT-EP003-041. “ 0.
This states that the evolutionary game defined in
Appendix A (3) follows Lyapunov stability, i.e., the equilibrium is
Proof of Proposition 1 stable.
To prove Proposition 1, we give the derivative of
References
gi,k over xj,l as follows:
ˆ ˙ [1] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,”
dgi,k dxi,k dui,k dūi nature, vol. 521, no. 7553, pp. 436–444, 2015.
“ pui,k ´ ūi q ` xi,k ´ , [2] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. In-
dxj,l dxj,l dxj,l dxj,l
german, V. Ivanov, C. Kiddon, J. Konecny, S. Mazzocchi,
dui,k and H. B. McMahan, “Towards federated learning at scale:
where is given by
dxj,l System design,” arXiv preprint arXiv:1902.01046, 2019.
„ [3] H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas,
xi,k Mid
“ γk f 1 phpxk qqMjd ` “Federated learning of deep networks using model averag-
dui,k
dxj,l
hpxk q ing,” CoRR, vol. abs/1602.05629, 2016.
˜ ¸ff [4] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B.
dxi,k Mjd xi,k Mid Mjd McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth,
fk phpxk qq ´
dxj,l hpxk q h2 pxk q “Practical secure aggregation for federated learning on
user-held data,” arXiv preprint arXiv:1611.04482, 2016.
dxi,k [5] J. Hofbauer and K. Sigmund, “Evolutionary game dy-
, ´ ηi,k Mid namics,” Bulletin of the American Mathematical Society,
dxj,l
vol. 40, no. 4, pp. 479–519, 2003.
ř
where hpxk q fi iPN xi,k Mid . Besides, t is omitted [6] E. of Mathematics, “Cauchy-lipschitz theorem,”
https://www.encyclopediaofmath.org/index.php/
here for notationalˇ ˇ
convenient. In the sequel, for all Cauchy-Lipschitz theorem.
ˇ du ˇ
pi, k q P N ˆ K, ˇ dxi,k
j,l
ˇ is bounded for all pj, lq P N ˆ K [7] S. Sastry, “Lyapunov stability theory,” Nonlinear Systems,
1 pp. 182–234, 1999.
due
ˇ ˇ the boundedness of f p¨q and f p¨q. Similarly,
to ˇ ˇ [8] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner,
ˇ dūi ˇ ˇ dg ˇ
ˇ dxj,l ˇ is also bounded. These facts admit that ˇ dxi,k j,l
ˇ “Gradient-based learning applied to document recogni-
is bounded, which completes the proof. tion,” Proc. of the IEEE, vol. 86, no. 11, pp. 2278–2324,
1998.
[9] B. Gu, F. Hu, and H. Liu, “Modelling classification per-
Appendix B formance for large data sets,” in International Conference
Proof of Theorem 2 on Web-Age Information Management, 2001, pp. 317–328.
[10] R. Foundation, “Raspberry Pi 3 model B,” https://www.
To prove that the function defined in (4) meet raspberrypi.org/products/raspberry-pi-3-model-b/.
the Lyapunov conditions, we need to verify that

879