,(((:LUHOHVV&RPPXQLFDWLRQVDQG1HWZRUNLQJ&RQIHUHQFH :&1&
ChainsFL: Blockchain-driven Federated Learning
from Design to Realization
Shuo Yuan, Bin Cao∗ , Mugen Peng, Yaohua Sun
2021 IEEE Wireless Communications and Networking Conference (WCNC) | 978-1-7281-9505-6/20/$31.00 ©2021 IEEE | DOI: 10.1109/WCNC49053.2021.9417299
State Key Laboratory of Networking and Switching Technology,
Beijing University of Posts and Telecommunications, Beijing, 100876, China
Email: {yuanshuo, caobin, pmg, sunyaohua}@bupt.edu.cn
Abstract—Despite the advantages of Federated Learning (FL),
such as devolving model training to intelligent devices and
preserving data privacy, FL still faces the risk of the single
point of failure and attack from malicious participants. Recently,
blockchain is considered a promising solution that can transform
FL training into a decentralized manner and improve security
during training. However, traditional consensus mechanisms and
architecture for blockchain can hardly handle the large-scale FL
task due to the huge resource consumption, limited throughput,
and high communication complexity. To this end, this paper
proposes a two-layer blockchain-driven FL framework, called
as ChainsFL, which is composed of multiple Raft-based shard
networks (layer-1) and a Direct Acyclic Graph (DAG)-based
main chain (layer-2) where layer-1 limits the scale of each
shard for a small range of information exchange, and layer-
2 allows each shard to update and share the model in parallel
and asynchronously. Furthermore, FL procedure in a blockchain
manner is designed, and the refined DAG consensus mechanism
to mitigate the effect of stale models is proposed. In order to
provide a proof-of-concept implementation and evaluation, the
shard blockchain base on Hyperledger Fabric is deployed on the
self-made gateway as layer-1, and the self-developed DAG-based
main chain is deployed on the personal computer as layer-2. The
experimental results show that ChainsFL provides acceptable
and sometimes better training efficiency and stronger robustness
comparing with the typical existing FL systems.
I. I NTRODUCTION
Internet of Everything brings not only massive amounts of
data but also considerable distributed computation and storage
resources. However, it is challenging to use these data and
resources effectively while preserving privacy. To this end,
Federated Learning (FL) as a distributed training technology
has been proposed to collaborate devices (e.g., mobile phones
or vehicles) to train a model while keeping the training data
locally [1].
In FL, the global model is aggregated by a central server
based on local models trained by each device over the local
raw data [2]. To enhance the availability of FL, on the other
hand, many existing works dedicate to investigate and optimize
the performance of FL in wireless networks. For example, [3]
formulates the FL over a wireless network as an optimization
problem to analyze the impact of wireless environment for the
time of FL task. Besides, the radio resources are optimized in
[4] to schedule devices and minimize the convergence time
of FL. However, the security and efficiency of the traditional
∗ Bin Cao is the corresponding author.
978-1-7281-9505-6/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 17,2021 at 04:24:19 UTC from IEEE Xplore. Restrictions apply.
centralized FL are challenging in the widespread application
and can be summarized as follows:
• Security. The traditional FL that relies on a central
FL server to orchestrate the entire training process is
vulnerable to a single point of failure and targeted attacks,
which will lead to service paralysis. Moreover, valid
local models will be distorted by the fallacious global
model aggregated at the FL server. Besides, traditional
FL cannot handle the effect of malicious devices on the
accuracy of the global model.
• Efficiency. Nowadays, most FL architectures run in a
synchronous manner, and the training will inevitably be
slow down by stragglers which are devices that take a
long time to complete one training round [5]. On the
other hand, in the asynchronous training, stale models
trained from an older version of the global model may
be used to update the latest global model and make the
global model unstable [6].
To address the aforementioned challenges, the blockchain
technology [7] [8] with decentralized and tamper-resistant
characteristics is introduced into FL. By leveraging
blockchain, [9] proposed BlockFL to carry out synchronous
training in a decentralized manner. Thus, the single point
of failure and targeted attacks can be overcome, and all the
local model updates will be verified during the Proof-of-Work
(PoW) consensus. Besides, Multi-access Edge Computing
(MEC) servers are adopted in [10] to provide computing
resources to the resources-constraint devices and as miners to
process received local models. Although these architectures
improve the security of the FL training process, the PoW
consensus brings a huge computation cost and limits the
throughput of the system.
Different from [9] and [10] which use the public blockchain,
we employ a consortium blockchain suitable for the MEC
environment, where edge nodes are used as blockchain nodes,
and devices need to be licensed to access the blockchain
[11]. Moreover, compared with PoW, the Raft consensus [12]
avoids the flaws, such as high computation costs and long
confirmation delay, and has a significant improvement in
throughput. Thus, we adopt the Raft to achieve a fast low-
complexity consensus to support decentralized FL. However,
the throughput of Raft is limited by the maximum performance
of a single node with limited resources. So how to improve
 ,(((:LUHOHVV&RPPXQLFDWLRQVDQG1HWZRUNLQJ&RQIHUHQFH :&1&
the scalability of the Raft-based blockchain to support a
large-scale FL task?
To this end, we split edge nodes into multiple groups
where each group is one shard to deploy one Raft-based
blockchain to interact with the devices joined the task, and
each shard executes the FL training independently. Hence,
limited scalability and high communication complexity caused
by a single large-scale Raft-based blockchain will be tackled
well. However, the time cost of each shard to complete one
iteration of synchronous training is different, and it is hard to
aggregate and update the global model from all the shards.
So another critical problem is: how to efficiently coordinate
and integrate the models trained by all shards to update the
global model?
To address this problem, we construct a main chain to
store and share models trained from shards and adopt the
Directed Acyclic Graph (DAG) consensus [13] to process
these models asynchronously. Thus, a two-layer blockchain-
driven FL framework called ChainsFL is set up in this paper.
On the other hand, the combination of synchronous training
and asynchronous training is implemented in ChainsFL finally.
Besides, both the aggregating of the basic model for the next
shard training iteration and the validating of the transactions
in DAG are combined as one process in ChainsFL.
Through the above observations and discussions, the main
contributions of this work can be summarized as follows.
• We propose ChainsFL, a federated learning framework
driven by the two-layer blockchain. Then, we design
a Raft-based blockchain sharding architecture to im-
prove scalability and a refined DAG-based main chain
to achieve cross-shard interaction. To the best of our
knowledge, ChainsFL is the first framework using the
DAG to coordinate multiple shard blockchain networks
to improve the security and scalability of FL systems.
• We define the operation process and design a set of
interaction rules for ChainsFL to perform FL tasks. To
improve the FL efficiency, synchronous and asynchronous
training are combined in ChainsFL. Moreover, the virtual
pruning mechanism is proposed based on the refined
DAG consensus to eliminate the impact of stale models
on the global model.
• We establish the sharding network prototype based on
Hyperledger Fabric to implement the layer-1 and develop
a DAG-based main chain to implement the layer-2 as
well as carry out cross-layer interaction. The extensive
evaluation results show that ChainsFL provides accept-
able and sometimes better convergence rates compared
to FedAvg [1] and AsynFL [6] for CNNs and enhances
the robustness of the training architecture.
II. F RAMEWORK OF C HAINS FL
In this section, we illustrate the framework and key compo-
nents of ChainsFL. ChainsFL consists of two-layer blockchain
which the layer-1 formed by multiple Raft-based blockchains
and layer-2 formed by one DAG-based main chain. For layer-
1, a set of edge nodes be partitioned into multiple independent
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 17,2021 at 04:24:19 UTC from IEEE Xplore. Restrictions apply.
shards to deploy these blockchains. Besides, the main chain is
deployed on many distributed nodes to coordinate and validate
transactions committed by shards in a decentralized manner.
For simplicity, the descriptions below do not include the details
of the interaction between shards and the main chain, which
are described in Section III.
A. Raft-based Shard Blockchain
The Raft-based blockchain is deployed in each shard formed
by several edge nodes with heavy computation and storage
capabilities and responsible for coordinate the devices served
by these nodes to complete the training task. We assume that
the device set {d1 , d2 , · · · , dn } is selected by the blockchain
of shard #1 to participate in the FL training task, and the
datasets of these devices are {D1 , D2 , · · · , Dn }. Without loss
of generality, assume that each training sample in dataset
is a set of input-output pairs (x, y). The parameters set of
the FL model is denoted as w. For each sample i, the
loss function of the machine learning problem is defined as
fi (w) = l(xi , yi |w). Thus, the loss function on the mini-
batch bj which samples from Dj and contains k samples can
be written as fbj (w). The goal of device j is to solve the
following optimization problem:
min Fj (w) = Ebj ∼Dj fbj (w). (1)
The local model of device j can be updated according to wj ←
wj − μ∇fbj (wj ),where μ is learning rate of the device.
In addition, the FederatedAveraging algorithm [1] is adopted
by each shard to aggregate updated local models. Then the
loss function of shard #1 on the decentralized datasets can be
expressed as:
n
|Dj |
Gs1 (w) = Fj (w), (2)
j=1
D
n
where D = j=1 |Dj | is total size of datasets used in
this shard training round. As the devices selected in round
t upload the parameters of the updated local models, the
model parameters of shard #1 ws1 will be updated through the
weighted aggregation of all updated local models’ parameters,
i.e., n
 |Dj |wj (t)
ws1 (t) = . (3)
D
j=1
After the sharding training iteration is completed, the up-
dated sharding model will be packed as a transaction and then
published to the main chain.
B. DAG-based Main Chain
The DAG-based asynchronous consensus mechanism
(known as DAG consensus or tangle consensus [13] [14])
is adopted in the main chain to handle the interactions with
shards. In the main chain, a tangle graph can be constructed
where vertices represent transactions, and the edges denote
approval of another transaction. Each transaction consists
of one model’s parameters trained by the shard. Besides,
transactions that are not approved by any other transaction are
called tips. Different from other blockchain, the DAG-based
main chain does not rely on single chain being the single
 ,(((:LUHOHVV&RPPXQLFDWLRQVDQG1HWZRUNLQJ&RQIHUHQFH :&1&

Shard #1 Shard #M Algorithm 1 Updated Local Model Validation

Task Node in shard executes:
Publisher
receive the transactions from devices
for all received the transactions do
Anew ← validate the accuracy of the model in the
Training
Round #1

transaction
Training

1st 1st
if Anew > Aτ then
iteration

iteration
forward the transaction to the leader in shard
else
Several
rounds ago mark invalid and discard
end if
Training
Round #1

2nd 2nd end for

Training
iteration

iteration
Several
rounds ago
DAG-base mainchain
Raft-based blockchain
deployed on edge nodes Device
Shard gets the genesis block Shard uploads the
aggregated shard model
from main chain
Devices download the basic Shard selects two tips
model from shard blockchain and aggregates them to
for next round obtain the basic model
Devices upload the updated local for current iteration
models to the shard blockchain
Fig. 1. Overview of the FL procedure in ChainsFL.
source of confidence thanks to the tangle graph. Each node in
the DAG network maintains a local ledger that can be used to
construct a tangle graph. Besides, the node belonging to the
shard blockchain network can also join the DAG network.
III. F EDERATED L EARNING P ROCEDURE IN C HAINS FL
To perform the FL task in a decentralized manner, we define
the operation process and design a set of interaction rules to
orchestrate the devices and compute nodes in ChainsFL, as
shown in Fig. 1. More details about this procedure are given
as follows.
Phase 1: Task Publication. The task publisher signs a smart
contract to release the FL task. Besides, the information of this
task, such as the structure and parameters of the initial model,
the configurations for shard training, and the conditions for
end this task, will be stated in this contract. Then the smart
contract will create the genesis transaction (denoted as g0 ) of
the main chain which includes these information and a test
dataset. Meanwhile, related shard networks will be triggered
by this smart contract to run the training task.
Phase 2: Shard Training. The genesis transaction will be
pulled by the leader of triggered shard blockchains. Then the
configurations and requirements of this task will be stored in
the distributed ledger of the shard and used to initiate the
training process. The interactions between devices and the
shard blockchain and details of training process in one shard
are described as follows:
• 1) Device Selection: The leader of shard blockchain
selects participants for shard training based on the avail-
ability such as power status and network status which
are reported by devices periodically. Only devices that
are willing to participate in training and have abundant
battery and stable network coverage will be selected.
Then, these devices will be authorized to access the shard
blockchain to download the basic model for local training
and upload the transactions. It is worth noting that the
device selection in each shard is independent.
• 2) Local Update: Based on the basic model obtained from
the shard blockchain, the device will run the training
process over the local raw data to solve problem (1).
Afterward, the updated local model will be sent to the
blockchain node after the training goal, such as preset
the number of local epochs or accuracy, is achieved.
• 3) Model Aggregation: As shown in Algorithm 1, each
blockchain node will receive local models and validate
the accuracy based on the test dataset stored in g0 . Where
Aτ is the preset threshold and could be set as the accuracy
of the basic model used in the current training round.
Next, the leader of the shard blockchain will aggregate
valid local models according to (3) to update the shard
model. Due to the synchronous training in each shard, the
aggregation will be triggered if enough devices upload the
local models in time, otherwise, the round is abandoned.
The process from device selection to model aggregation
is called one shard training round. If the current iteration
has not ended, the updated shard model will be packed
as a transaction of shard blockchain and then published
in the shard to provide the basic model for the next shard
training round.
Phase 3: Shard Model Committing and Basic Model
Aggregation. When the shard has completed rounds needed
to execute in the current iteration, the latest aggregated shard
model will be packed as a transaction of the main chain and
then be committed to the main chain. The details of these
processes are shown in Algorithm 2.
In addition, the smart contract will monitor the main chain
and take a similar operation as the basic model aggregation
in Algorithm 2 to aggregate the global model. The number of
tips used to aggregate the global model will be stated in the
smart contract. Besides, the task publisher is able to aggregate
the global model from any location where can access the main
chain. On the other hand, devices licensed to access the edge

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 17,2021 at 04:24:19 UTC from IEEE Xplore. Restrictions apply.
 ,(((:LUHOHVV&RPPXQLFDWLRQVDQG1HWZRUNLQJ&RQIHUHQFH :&1&
Algorithm 2 Tips Choosing and Shard Model Committing
if the current is 1st iteration then
wbm ← extract the initial parameters from g0
ApproveSet = (g0 )
else
  
(w1 , w2 , ..., wη ) ← choose η tips from the tangle graph
  
(A1 , A2 , ..., Aη ) ← validate the accuracy of the model in
each chosen λtip 
wbm ← ( 1 wi /λ) aggregate λ(λ < η) tips with the
highest accuracy
ApproveSet = these λ tips
end if
wnew ← ShardTrain(wbm )
g ← package wnew and the ID of all transactions in
ApproveSet as a transaction
commit the g to the main chain
shard blockchain are able to select to join the training task to
improve the model accuracy or aggregate the latest model to
get the latest intelligent service, anytime and anywhere without
any centralized management. Thus, the vision of “providing
AI for every person and every organization at everywhere” can
be realized in an efficient way.
IV. C ONSENSUS IN C HAINS FL
As described in Section II, the Raft consensus and the DAG
consensus are adopted in layer-1 and layer-2, respectively.
A. Raft Consensus
The edge nodes in one Raft consensus-based shard
blockchain network are divided into leader and followers. The
details of leader selection are not the focus of this paper and
can be found in [12]. As shown in Algorithm 1, followers are
responsible for validating received transactions (updated local
models) and then forwarding the valid ones to the leader. The
leader will sort these transactions according to the generation
time. When the cumulative size of transactions reaches the
threshold or the period is over, the leader will create a block
and broadcast it to all followers. Moreover, the consensus for
this block will be achieved when the leader received half of
all followers’ responses.
By partitioning a large-scale blockchain network into multi-
ple independent shards to parallel consensus and separate data
storage, the system’s overall throughput is scaled effectively.
Besides, most of the data just need synchronizing within the
shard instead of broadcasting to the entire network, which
considerably reduces communication rounds and speeds the
processing of transactions. On the other hand, the influence
of stragglers is limited to the shards to which the stragglers
belong instead of the entire network.
B. DAG Consensus-based Virtual Pruning
The main chain in ChainsFL is used to share the model
trained by each shard and validate these models’ validity. As
described in Algorithm 2, the leader will aggregate the basic
model using λ tips for the next shard training iteration. Then,
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 17,2021 at 04:24:19 UTC from IEEE Xplore. Restrictions apply.
Tip 1 Approve New tip Normal growth
Freshness time
Commit tip 1 to
DAG
Selected as
candidate
Tip 1 approved by other
transaction
Tip 2 Approve New tip
Freshness time
No one will select
Commit tip 2 to Selected as Judged as stale Tip 2 approved by other
DAG candidate model transaction
Tip 3 Freshness time
Virtual Pruning
No one will select
Commit tip 3 to Selected as
DAG candidate Judged as stale model
Fig. 2. Three situations in lifecycle of each tip.
these tips will be approved by this shard model trained from
this basic model in the current iteration. In the main chain, the
model can be regarded as reach a consensus in the main chain
when it is directly or indirectly approved by all new models
committed to the main chain.
Due to the graph structure of the main chain, models not
approved by other models for a long time (called stale models)
will always be treated as tips. Thus, the ratio of stale models to
all tips will increase if they are not dealt with timely. Then the
probability that the shard selects the stale model to aggregate
the basic model will increase. To eliminate stale models, a
waiting time, called freshness time, is set in the main chain. As
shown in Fig. 2, each tip will experience one of three situations
during the entire lifecycle and tips in the third situation will be
judged as stale and will not be selected by shards. On the other
hand, with the validation during the DAG consensus described
in Algorithm 2, the model with low accuracy in the selected
η tips has a high probability to be discarded. Therefore, most
of the tips judged as stale are models with lower accuracy and
will be pruned from the tangle graph.
V. I MPLEMENTATION
To implement and then evaluate the proposed ChainsFL, we
set up the Raft-based blockchain based on the Hyperledger
Fabric [15], which includes the Raft consensus. Besides, a
refined DAG-based main chain is developed. Due to the limited
size of the block in Fabric, it is hard to store the model
file in the main body of the block directly. Thus, the peer-
to-peer file system called InterPlanetary File System (IPFS)
[16] is used in ChainsFL to implement the off-chain storage.
As one file is added to IPFS, a hash value that uniquely
identifies the contents of this file will be returned by IPFS.
Moreover, the hash value can be used to locate and then
download the related file across the system [16]. Then, the
transactions in ChainsFL store the hash of the parameter file
to guarantee its immutability. The details of Fabric and IPFS
are not presented in this paper to save space. The source code
of the implementation is available on GitHub1 .
1 https://github.com/shuoyuan/ChainsFL-implementation
 ,(((:LUHOHVV&RPPXQLFDWLRQVDQG1HWZRUNLQJ&RQIHUHQFH :&1&
Deployment in Each Shard
Leader
Layer-2
Follower Follower
DAG Host (PC)
Layer-1
Shard #1 (Gateway) Shard #2 (Gateway) Shard #3 (Gateway)
Fig. 3. The implementation of ChainsFL in the real environment.
The implementation of ChainsFL without participating de-
vices are shown in Fig. 3, and the following details are
considered:
• Device: the lightweight computer is used as the partici-
pant of the FL task to train the local model.
• Raft-based Blockchain Node: multiple self-made gate-
ways are used as the blockchain nodes to deploy the
Fabric blockchain and as the access points to provide
radio access for devices.
• Main Chain: a personal computer (PC) is used to deploy
the self-developed DAG-based main chain.
To accelerate and simplify the executions of massive experi-
ments, the main chain is deployed on one computer (one node)
instead of many distributed nodes in our real environment
setup2 . Due to the limited number of the gateway, a complete
Fabric blockchain containing one leader and two followers is
configured in one gateway3 . Thus, the gateway and devices the
gateway served are one shard network defined in ChainsFL.
VI. E VALUATION AND D ISCUSSION
In this section, we empirically evaluate the ChainsFL based
on its convergence and robustness against the malicious model
attacks from devices and shards.
A. Evaluation Specifics
With the implemented ChainsFL described in Section V, the
object classification task is adopted to evaluate the efficiency
and security of ChainsFL. The image dataset MNIST, which
includes 60000 training images and 10000 testing images,
is used. The synchronous algorithm of FedAvg [1], and the
asynchronous algorithm of AsynFL [6] are used as baselines.
To simulate the real data characteristics, the Non-IID data
setting is adopted in all experiments. Thus, we divide the
training set of MNIST into 200 groups after sorting these by
digit label, and each group has 300 images. We assign each
of 100 devices 2 groups.
The Convolutional Neural Network (CNN) with three con-
volutional layers and two full connected layers is used in the
training task. Moreover, we take the learning rate μ = 0.01,
mini-batch size B = 10, and local epoch E = 5 for all devices.
2 Due to interactions between the leader of the shard blockchain and the
main chain will not be affected by this deployment scheme, the performance
of federated learning over ChainsFL will not be disturbed.
3 At least three nodes (one leader and two followers) need to be deployed
in the Raft consensus of Fabric.
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 17,2021 at 04:24:19 UTC from IEEE Xplore. Restrictions apply.
(a) Accuracy on Testing Set. (b) Cross Entropy on Training Set.
Fig. 4. Metrics vs. # of global epochs for the MNIST CNN (non-IID).
(a) Accuracy on Testing Set. (b) Cross Entropy on Training Set.
Fig. 5. Metrics vs. # of gradients for the MNIST CNN (non-IID).
For ChainsFL, one shard training round is configured for each
shard training iteration. There are three shards configured in
ChainsFL, and k = 10 devices are non-overlapping selected
for each shard. The parameters related to the aggregation
of the basic model are set as λ = 2, and η = 3. For
FedAvg, 10 devices are selected in each global epoch. For
AsynFL, one device with local model wlm will be selected
from 100 devices and be used to update the global model
wgm in each global epoch according to the update rule of
wgm ← 12 wgm + 12 wlm . To compare three training paradigms
fairly, we conduct two comparisons: metrics versus the number
of global epochs and metrics versus the number of gradients.
For the first comparison, the basic model aggregated by one
shard can be treated as the global model. Thus, one shard
training iteration can be treated as one global epoch. On the
other hand, the gradient generated in each local epoch of
devices is counted for the second comparison. For FedAvg,
in each global epoch, since the local epochs is 5, and 10
devices are selected, 50 gradients are generated in each global
epoch. Similar to FedAvg, 50 gradients are used in each shard
iteration in ChainsFL. For AsynFL, 5 gradients are used in
each global epoch. In all experiments, 10 times run for each.
Then, the averages of these experiments results are drawn as
curves, and the standard deviations of these curves are also
shown in the figures.
B. Convergence
The convergence with the number of global epochs and
gradients increases are shown in Fig. 4 and Fig. 5, respectively.
In Fig. 4(a), we can observe that ChainsFL converges faster
than FedAvg and AsynFL at the same amount of global
epochs. The reason is that the shard aggregates the basic
model using the tips from DAG and this means that updated
shard models are shared with all shards in ChainsFL. Besides,
AsynFL uses only one device for one global epoch, which
makes it converge slow than ChainsFL and FedAvg. Actually,
 ,(((:LUHOHVV&RPPXQLFDWLRQVDQG1HWZRUNLQJ&RQIHUHQFH :&1&
(a) Accuracy of different FL (b) Accuracy of ChainsFL with dif-
paradigms with malicious devices. ferent # of malicious devices.
Fig. 6. Test accuracy with malicious devices for the MNIST CNN (non-IID).
this comparison scheme is not fair to AsynFL due to the
computation of AsynFL in each global epoch is much less than
FedAvg and ChainsFL. As shown in Fig. 5(a), the comparison
based on the number of gradients shows that AsynFL achieves
the fastest convergence, which is in line with the conclusion
of [6]. As the training progresses, the accuracy of ChainsFL
will gradually get close to AsynFL. Besides, the train loss of
these experiments also presents the same conclusions as to test
accuracy.
C. Robustness
To compare the robustness of these three paradigms, we
configure 20% of selected devices as malicious devices. The
model from malicious devices or malicious shards is set as
the same model with a test accuracy of 0.07. For detail,
two malicious devices are configured in 10 selected devices
for FedAvg, and 20 malicious devices are configured in 100
devices for AsynFL. As shown in Fig. 6(a), the accuracy of
ChainsFL with two malicious devices per shard is higher than
other paradigms thanks to the validation of shard blockchain
nodes and virtual pruning in the main chain. It is clear that the
accuracy of FedAvg has dropped significantly, and the curve
of AsynFL is chaotic and shows no signs of convergence.
To evaluate the impact of malicious devices or shard on the
global model of ChainsFL, we design three attack schemes:
two malicious devices per shard, five malicious devices per
shard, and one malicious shard. As shown in Fig. 6(b),
compared with schemes of each shard containing two and
five malicious devices, a malicious shard has less impact on
the performance of ChainsFL. The reason is that the DAG
consensus-based virtual pruning is able to effectively eliminate
the model uploaded by the malicious shard. Although mali-
cious devices are detected and eliminated during the shard
consensus for the first two schemes, the number of valid
devices that participated in each shard iteration is also reduced.
VII. C ONCLUSION
In this paper, a two-layer blockchain-driven federated learn-
ing framework is proposed to ensure the security and efficiency
of the distributed training. Through the Raft consensus used
in layer-1 and DAG consensus used in layer-2, asynchronous
optimization and synchronous optimization are effectively
combined to deal with stragglers and stale models, and a cross-
layer operation procedure is proposed to achieve the consensus
among all shards on updated shard models. Besides, the
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 17,2021 at 04:24:19 UTC from IEEE Xplore. Restrictions apply.
prototype of ChainsFL is developed, and massive experiments
are run in this prototype. The experimental results demonstrate
the effectiveness and robustness of ChainsFL.
VIII. ACKNOWLEDGEMENT
This work was supported in part by the National Nat-
ural Science Foundation of China under Grant 61921003,
Grant 61925101, Grant 61831002, Grant 61701059, and Grant
61941114, in part by the Beijing Natural Science Founda-
tion under Grant JQ18016, in part by the Eighteenth Open
Foundation of State Key Lab of Integrated Services Net-
works of Xidian University under Grant ISN20-05, in part
by the Chongqing Technological Innovation and Application
Development Projects under Grant cstc2019jscx-msxm1322,
in part by the Fundamental Research Funds for the Central
Universities under Grant 2020RC10 and Grant 2020RC11, and
in part by the BUPT Excellent Ph.D. Students Foundation
under Grant CX2020106.
R EFERENCES
[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas,
“Communication-Efficient Learning of Deep Networks from Decentral-
ized Data,” in Proc. Int. Conf. Artif. Intell. Stat., AISTATS’17, Fort
Lauderdale, Florida, USA, Apr. 2017, pp. 1273–1282.
[2] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman,
V. Ivanov, C. Kiddon et al., “Towards Federated Learning at Scale:
System Design,” arXiv preprint arXiv:1902.01046, Mar. 2019.
[3] N. H. Tran, W. Bao, A. Zomaya, M. N. H. Nguyen, and C. S. Hong,
“Federated Learning over Wireless Networks: Optimization Model De-
sign and Analysis,” in Proc. IEEE INFOCOM’19, Apr. 2019, pp. 1387–
1395.
[4] M. Chen, H. V. Poor, W. Saad, and S. Cui, “Convergence Time
Optimization for Federated Learning over Wireless Networks,” arXiv
preprint arXiv:2001.07845, Jan. 2020.
[5] S. Dutta, G. Joshi, S. Ghosh, P. Dube, and P. Nagpurkar, “Slow and Stale
Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed
SGD,” in Proc. Int. Conf. Artif. Intell. Stat., AISTATS’18, Playa Blanca,
Lanzarote, Canary Islands, Spain, Aug. 2018.
[6] C. Xie, S. Koyejo, and I. Gupta, “Asynchronous Federated Optimiza-
tion,” arXiv preprint arXiv:1903.03934, Sep. 2019.
[7] S. Nakamoto, “Bitcoin: A Peer-to-Peer Electronic Cash System,” bit-
coin.org, pp. 1–9, 2008.
[8] B. Cao, Y. Li, L. Zhang, L. Zhang, S. Mumtaz, Z. Zhou, and M. Peng,
“When Internet of Things Meets Blockchain: Challenges in Distributed
Consensus,” IEEE Netw., vol. 33, no. 6, pp. 133–139, Nov. 2019.
[9] H. Kim, J. Park, M. Bennis, and S.-L. Kim, “Blockchained On-Device
Federated Learning,” IEEE Commun. Lett., pp. 1–4, 2019.
[10] U. Majeed and C. S. Hong, “FLchain: Federated Learning via MEC-
enabled Blockchain Network,” in Proc. Asia-Pacific Netw. Oper. Manag.
Symp., APNOMS’19, Matsue, Japan, Sep. 2019, pp. 1–4.
[11] R. Yang, F. R. Yu, P. Si, Z. Yang, and Y. Zhang, “Integrated Blockchain
and Edge Computing Systems: A Survey, Some Research Issues and
Challenges,” IEEE Commun. Surv. Tutor., vol. 21, no. 2, pp. 1508–1532,
Jan. 2019.
[12] D. Ongaro and J. Ousterhout, “In Search of an Understandable Consen-
sus Algorithm,” in Proc. USENIX Annu. Tech. Conf., USENIX ATC’14,
Philadelphia, PA, USA, 2014, pp. 305–319.
[13] S. Popov, “The Tangle,” IOTA, White Paper, Apr. 2018.
[14] Y. Li, B. Cao, M. Peng, L. Zhang, L. Zhang, D. Feng, and J. Yu, “Direct
Acyclic Graph-Based Ledger for Internet of Things: Performance and
Security Analysis,” IEEE/ACM Trans. Networking, pp. 1–14, 2020.
[15] “A Blockchain Platform for the Enterprise — Hyperledger Fabric,”
https://hyperledger-fabric.readthedocs.io.
[16] J. Benet, “IPFS - Content Addressed, Versioned, P2P File System,” arXiv
preprint arXiv:1407.3561, Jul. 2014.