17070
18, 15 SEPTEMBER 2022
Sharding-Hashgraph: A High-Performance
Blockchain-Based Framework for Industrial
Internet of Things With Hashgraph Mechanism
Ningjie Gao , Ru Huo , Shuo Wang, Tao Huang , Senior Member, IEEE, and Yunjie Liu
Abstract—In recent years, with the development and
widespread use of blockchain, many projects have introduced
blockchain technology to solve the increasingly serious security
problems of the Industrial Internet of Things (IIoT). However,
due to the conflict between the operational performance and secu-
rity of the blockchain system, the conflict between transparency
and privacy, and the compatibility issues with a large number
of IIoT devices running together, the mainstream blockchain
system cannot be applied to IIoT scenarios. In order to solve
these problems, in this article, we propose an IIoT distributed
data system based on blockchain technology. We provide a novel
system architecture for different IIoT devices to deploy high-
performance blockchain systems in many scenarios, such as
smart factory networks. To improve the performance of the
blockchain network, we adopt the sharding hashgraph consensus
mechanism and introduce a node evaluation mechanism based
on the state of the node, which is applied to divide a large
number of nodes into many shards dynamically. We abstract
the node sharding problem as a joint optimization problem and
use deep reinforcement learning to solve it. Finally, we com-
pared with asynchronous Byzantine consensus algorithms, such
as HoneybadgerBFT and BEAT, which validated the performance
of this system architecture.
Index Terms—Asynchronous Byzantine fault tolerance,
blockchain, Industrial Internet of Things (IIoT), sharding.
I. I NTRODUCTION
ITH the development and progress of enabling tech-
W nologies, such as the Internet of Things, 5G, and
artificial intelligence, the Industrial Internet of Things (IIoT)
has opened the prelude to the fourth industrial revolution,
Manuscript received 30 August 2021; revised 30 October 2021; accepted
4 November 2021. Date of publication 10 November 2021; date of current
version 7 September 2022. This work was supported in part by the MIIT of
China 2020 (Identification Resources Search System for Industrial Internet of
Things) and in part by the National Key Research and Development Program
of China under Grant 2018YFB1800502. (Corresponding author: Ru Huo.)
Ningjie Gao is with the State Key Laboratory of Networking and
Switching Technology, Beijing University of Posts and Telecommunications,
Beijing 100876, China.
Ru Huo is with the Information Department, Beijing University of
Technology, Beijing 100124, China, and also with Purple Mountain
Laboratories, Nanjing 211111, China (e-mail: huoru@bjut.edu.cn).
Shuo Wang and Tao Huang are with the State Key Laboratory of
Networking and Switching Technology, Beijing University of Posts and
Telecommunications, Beijing 100876, China, and also with Purple Mountain
Laboratories, Nanjing 211111, China.
Yunjie Liu is with the State Key Laboratory of Networking and
Switching Technology, Beijing University of Posts and Telecommunications,
Beijing 100876, China, also with the Information Department, Beijing
University of Technology, Beijing 100124, China, and also with Purple
Mountain Laboratories, Nanjing 211111, China.
Digital Object Identifier 10.1109/JIOT.2021.3126895
2327-4662 
c 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
IEEE INTERNET OF THINGS JOURNAL, VOL. 9, NO.
and the development of productive forces of human soci-
ety has achieved a qualitative leap [1], [2]. In recent years,
ICT-enabling technologies represented by big data, artificial
intelligence, cloud computing, and the Internet of Things have
begun to penetrate social production activities. In order to meet
the demands of ubiquitous network connections and industrial
network stability, IIoT has introduced technologies, such as
industrial PON, industrial SDN, edge computing, and identifier
resolution. These technologies are used to realize the access
and interaction of massive heterogeneous data in IIoT. With the
application of IIoT in many fields, security attacks and failures
have caused huge troubles in IIoT networks around the world,
seriously affecting the income of IIoT [3]. Therefore, the cur-
rent industrial network is far from meeting the requirements
of IIoT in terms of security and scalability of deployment [4].
In recent years, with the emergence of blockchain technol-
ogy and its widespread application in the field of data security,
the idea of combining blockchain technology with the IIoT has
received considerable attention [5], [6]. By using the functions
of the anti-tampering and decentralized consensus mechanism
in the blockchain, we can solve the challenges mentioned
above of the IIoT system [7]. The core of the blockchain is
a novel data structure. Through this data structure, we can
achieve data privacy and security.
There have been some studies on the application of
blockchain technology in the IIoT. For example, Sarah and
Rafidah proposed a decentralized reputation management
system based on blockchain technology [8], which uses the
blockchain to determine the reputation of IIoT devices for
task offloading. However, the project uses PoET as the con-
sensus mechanism of the blockchain, which will lead to
reduced security. Mohammed uses blockchain to realize the
secure transmission of IIoT data, but they did not consider
data privacy issues [9], such as access control issues among
multiple users in the system. In addition, they all use chain-
like blockchains in IIoT software systems, which will make
many low-power devices in industrial scenarios unable to oper-
ate normally. At the same time, the introduction of blockchain
innovation into the IIoT will encounter many other problems.
We summarized three main issues as follows.
1) Tradeoff Between Security and Device Power
Consumption: As we all know, introducing the blockchain
and consensus mechanisms into the Internet can significantly
improve the security of the Internet. Blockchain exploits a
variety of different consensus mechanisms to defend against
malicious attacks. As a typical application of the consensus
GAO et al.: SHARDING-HASHGRAPH: HIGH-PERFORMANCE BLOCKCHAIN-BASED FRAMEWORK
mechanism, the proof-of-work (POW) consensus mechanism
forces the blockchain nodes to run a high-complexity crypto-
graphic algorithm, proving workload and reaching consensus.
However, for low-power devices in industrial scenarios,
it is difficult for them to run a typical POW consensus
mechanism. At the same time, choosing some lightweight
consensus mechanisms will reduce the security of the system.
Therefore, how to balance the device power consumption
requirement and security requirements of the IIoT system is
the first problem [10].
2) Conflict Between Transparency and Privacy:
Blockchain is a distributed, immutable ledger. Most
blockchain ledgers are also transparent, and all users can
access any data on the ledgers. However, for many scenarios
of the IIoT, such transparency is unacceptable because much
data in the scenarios involve key links in industrial production
and transaction processes. These data need to be protected
and can only be accessed by the specified people. Therefore,
how to ensure the security and privacy of data is the second
problem we will encounter.
3) Performance of the Blockchain is Difficult to Meet
Actual Needs: The IIoT environment usually has many device
access and high concurrent data circulation, which requires the
blockchain system to have good scalability and high through-
put. Unfortunately, most of the existing blockchains have
poor scalability and low throughput, derived from their secu-
rity consensus mechanism based on complex cryptography. In
addition, the chain structure of the blockchain cannot make full
use of a large number of devices in the IIoT scene, such as
low-power sensors. Therefore, how to improve the through-
put of the blockchain to meet the needs of high concurrent
transactions in IIoT is the third problem we need to solve.
These three issues prompted us to design a high-
performance, scalable, and secure blockchain-based IIoT
system. To improve the compatibility of the system, we
proposed a hierarchical blockchain data management archi-
tecture, which can realize the access of different industrial
equipment. We adopted the sharding-hashgraph consensus
mechanism to improve the throughput and consensus effi-
ciency of the blockchain network. In addition, we introduced
an evaluation mechanism based on the state of the node.
We abstracted the sharding problem as a joint optimization
problem and we used deep reinforcement learning to solve
the problem. Our unique contributions to the article are as
follows.
1) We propose a high-performance, scalable, and secure
blockchain system architecture for IIoT, which has a
layered architecture for different IIoT devices (such as
power-constrained devices). In addition, we provide a
node evaluation mechanism based on the state of the
node and an efficient access control scheme.
2) We use the blockchain of the Hashgraph structure as the
underlying structure instead of using the chain-structured
blockchain to build our software system. In order to fur-
ther improve the performance of the system, we have
introduced a sharding mechanism in hashgraph, which
is applied to divide the geographically close nodes into
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
17071
one shard. The nodes in each shard only contribute to
the consensus of events in their nodes.
3) We designed and built a proof-of-concept demo for the
IIOT scenario. Through comparison with other consen-
sus algorithms, such as BEAT and HoneybadgerBFT
under the same network configuration, the verifica-
tion results show that the IIoT system we proposed is
efficient.
The remainder of this article is organized as follows.
Section II briefly introduces the progress of related research.
Section III introduces the architecture of our IIoT system,
including the system and mechanism design. In Section IV,
we introduced the detailed design of the blockchain system
used in this article. Section V introduced the simulation setup
and test results. Section VI is the summary of this article and
the prospect of future work.
II. R ELATED W ORKS
In this section, we present the main definition and technical
framework of IIoT and blockchain.
A. Industrial Internet of Things
With the development of next-generation network technolo-
gies, IoT, 5G, Cloud computing, Edge/Fog computing, and
other advanced network and communication technologies are
used to accelerate the process of industrial-technological inno-
vation [1], [11]. The concept of IIoT has also emerged and
has been widely concerned by the industry. IIoT is gener-
ally defined as a communication network with high reliability,
low latency, and high controllability that provides ubiquitous
interconnection for intelligent industrial objects [12].
It includes the new industrial network architecture based
on cloud computing or fog/edge computing, and the next-
generation wireless sensor network (WSN), M2M communi-
cation, industrial SDN, industrial Ethernet, and deterministic
networking (DetNet) and other technologies. In recent years,
global firms have concentrated on their technological compe-
tency in developing and supplying software and platform for
IIoT. In general, IIoT can be described with a three-tier archi-
tecture, as shown in Fig. 1 [13]. Basic IIoT architecture is
composed of edge tier, platform tier, and enterprise tier.
IIOT devices do not belong to the three-tier architecture
of the IIoT. These devices are connected to the edge layer
and generate data streams for processing at the edge layer.
The edge tier deals with the protocol compatibility issues
of IT/OT and connects to the field-level production network,
then provides computing support for the underlying intelli-
gent devices by using distributed edge resources. The platform
layer includes various data and operation services of the IIoT
and supporting value increment of industrial data. Operation
domains include specific services for manufacturing which can
automate production by issuing control tasks, such as mon-
itoring and diagnostics, prognostics, and optimization. The
enterprise tier includes open capabilities for enterprise-level
industrial applications and third-party applications, integrat-
ing and encapsulating the atomic capabilities provided by the
platform tier and the edge tier.
17072
Fig. 1. Typical three-tier architecture of the IIoT.
The fusion of AI and big data has expanded the digiti-
zation scope of IIoT. The ability of collecting, analyzing,
and processing production data can enrich the functions of
industrial applications and improve the efficiency of opera-
tion and management [14]. Moreover, many technologies are
also applied in various scenarios. For example, in manufactur-
ing, 3-D printing technology realizes fast and low-cost product
customization functions. In the factory network, SDN, edge
computing and other technologies can be used to transform
the internal network of the factory to observe and schedule
the entire network uniformly.
IIoT can realize the transformation and upgrading of tradi-
tional industries, such as aerospace, manufacturing, medical,
oil, and natural gas development, bringing profound changes.
B. Blockchain
As a revolutionary technology in network security,
blockchain is widely regarded as a promising technology
to solve the security issues and data privacy issues of the
IIoT [15]. The main function of the blockchain is to enable a
group of entities that do not trust each other to reach a con-
sensus on a set of shared events, data, and states, and to be
supported by complex cryptographic technology and consen-
sus models [16]. Blockchain can be divided into two types
according to the structure: 1) the blockchain with chain struc-
ture and 2) the other is the blockchain with DAG structure. As
shown in Fig. 2, Bitcoin is a typical blockchain system with
a chain structure.
The chain structure of the blockchain has higher require-
ments for the network environment. For example, the classic
PBFT consensus algorithm requires the network to be an asyn-
chronous or semisynchronous network, which cannot adapt
to an overly complex and harsh network environment. In
addition, the complex security encryption mechanism of the
chain structure is power intensive and is not suitable for
power-constrained IoT devices.
In order to enable the blockchain network to have higher
throughput and better scalability, the industry has proposed
a new blockchain architecture based on the structure of a
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
IEEE INTERNET OF THINGS JOURNAL, VOL. 9, NO. 18, 15 SEPTEMBER 2022
directed acyclic graph. Typical applications include Tangle and
Hashgraph.
Hashgraph is an asynchronous Byzantine consensus algo-
rithm (ABFT), ensuring that it can maintain a stable operation
in harsh network environments. Since the ABFT protocol is
designed to run in a harsh network environment, most ABFT
consensus algorithms cannot have better performance in a
healthy network environment, but HoneyBadgerBFT [17] and
BEAT [18] are two exceptions.
HoneyBadgerBFT and BEAT use either Bracha’s broadcast
or one based on erasure codes to broadcast transactions and
then optimize a binary Byzantine agreement (BBA) protocol to
cast votes and reach an agreement. Their networks proceed in
rounds, wherein each round, n broadcast instances are executed
alongside n BBA instances, the latter of which only con-
clude when sufficiently many broadcast instances have been
completed to ensure progress. Although HoneyBadgerBFT
and BEAT successfully reduce communication complexity and
broadcast overhead and perform better than the PBFT consen-
sus algorithm in a small-scale environment, the performance
of the ABFT consensus algorithm can be further improved by
performing broadcast and voting at the same time. In addition,
hashgraph uses its unique broadcast mechanism and voting
mechanism to have huge performance advantages. In order
to build a high-performance industrial Internet blockchain
system, this article will optimize the ABFT consensus protocol
based on Hashgraph to enhance the throughput and scalability
of the blockchain.
III. S YSTEM A RCHITECTURE AND OVERVIEW
In this section, we outline the blockchain-based IIoT dis-
tributed data management system proposed in this article.
We will introduce the detailed system design from three
parts: 1) the system architecture; 2) the composite evaluation
mechanism of the node status; and 3) the privacy protec-
tion mechanism. We first introduced the system architecture
designed for the IIoT research.
A. Architecture of IIoT Distributed Data Management System
The system is built on the blockchain of the Hashgraph
structure, and each entity is a node in the blockchain-based
IIoT system. From the functional division, it can be divided
into two categories: 1) sensor network and 2) sharding-
hashgraph network. As shown in Fig. 3, The components of
the sensor network refer to those power-constrained devices
like IoT devices, and their representative devices are vari-
ous types of sensors. Because their power, storage capacity,
and computing capacity are limited, they do not participate
in the operation of the blockchain system and do not store
blockchain information. Their function is to collect and upload
data. The sharding-hashgraph network is a blockchain network
with hashgraph as the bottom layer, the nodes in the network
are full nodes. The full node is a large-scale device that has the
ability to carry blockchain programs and store large amounts
of data. Each full node is responsible for a group of lightweight
IIoT devices (sensor network) in a certain area, receiving all
data uploaded by the sensor network and storing these data.
GAO et al.: SHARDING-HASHGRAPH: HIGH-PERFORMANCE BLOCKCHAIN-BASED FRAMEWORK 17073

Fig. 2. Typical chain-structured blockchain and an example block.

node status can improve the internal throughput and consen-

sus delay of a single shard, thereby improving the overall
performance of the system.
To accurately evaluate the node’s credit status, this article
proposes a node credit cross-evaluation mechanism.
For a time range with a series of time slots
(t0 , . . . , ty , . . . , tN ), the reputation evaluation of node i
on node j within time ty can be represented by a vector
t
 t t t

γi,jy = τi,jy , si,jy , ui,jy (1)
where τ , s, and u stand for the degree of trust, degree of
suspicion, and uncertainty, respectively. The three components
add up to 1. Therefore, we can get the following formula:
Fig. 3. IIoT data management system architecture based on blockchain. ⎧ ty
⎪
⎪ τ
ty
= (1 −
ty
)
αy
⎪
⎪ u i,j α +β ty
t
⎨ i,j i
y
ty
i
ty ty βy (2)
The full node encrypts the data content that needs security ⎪ s = (1 − u )
⎪
⎪ t
i,j i,j α ty +β ty
protection through an encryption algorithm, packs the cipher- ⎪
⎩ u y = 1 − qty
i i

text and the corresponding operation authority into an event, i,j i,j
and uploads it to the blockchain network. After the consensus where α and β represent the number of confirmed (achieved
is completed, the data are stored locally or uploaded to the consensus) and discarded messages within the time ty range.
cloud network. q represents the number of processing completion messages.
Therefore, the direct reputation evaluation of node i to node
j is
B. Node Status Comprehensive Evaluation Mechanism t t t t
Ri,jy = τi,jy − si,jy + aui,jy (3)
In order to improve the scalability and consensus
performance (throughput, latency, etc.) of the hashgraph where a is a value of 0–1, which represents the influence
network, we need to divide the blockchain network into coefficient of uncertainty on direct reputation.
multiple shards. This article proposes a comprehensive evalu- In order to control the influencing factors of credit evalua-
ation mechanism based on node status, dividing the hashgraph tion more flexibly and dynamically, we set different weights
network into multiple shards with good performance. for positive interactions and negative interactions. The formula
This mechanism comprehensively considers the node’s geo- can be rewritten as
⎧ ty
graphic location, credit score, network status, CPU resources ⎪
⎪ τ
ty
=
ty ζ αy
⎪
⎪ q t ty
and divides all nodes in the network into multiple shards. The ⎨ i,j i,j y
ζ αi +ηβi
ty
nodes in each shard are geographically close, the network sta- t t ηβ (4)
⎪ si,jy = qi,jy ty y ty
tus is generally good, and the credit rating is similar. The ⎪
⎪ ζ αi +ηβi
⎪
⎩ uty = 1 − qty
introduction of a comprehensive evaluation mechanism for i,j i,j

Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
17074 IEEE INTERNET OF THINGS JOURNAL, VOL. 9, NO. 18, 15 SEPTEMBER 2022

Fig. 4. Privacy protection and access control mechanism of hashgraph system.

where ζ and η represent the different weights, and the sum of The method of protecting data confidentiality in a trans-
ζ and η is 1. parent system is encryption. There are two main types of
Since the reputation evaluation of a node may become less encryption algorithms, namely symmetric key encryption and
reliable over time, in order to describe the state of the node asymmetric key encryption (public-private key pair encryp-
more truthfully, we introduce the influence of time on the rep- tion). Considering the security of the encryption algorithm,
utation: θy , where y belongs to [1, y]. Therefore, the reputation we choose asymmetric key encryption. In addition, due to
evaluation formula of a node can be rewritten as the complexity of asymmetric keys and the massive amount
⎧ Y ty of data in the IIOT scenario, this mechanism chooses not to
⎪
⎪ ty y=1 θy τi,j
⎪
⎪ τ =  upload all the data to the blockchain system but only encrypts
⎪
⎪
i,j Y
θy
⎨ Y y=1 ty the required data and uploads it to the blockchain network.
ty y=1 θy si,j
si,j = Y (5) If node A wants to access the data of node B, the data
⎪
⎪ y=1 θy
⎪
⎪  t transaction between the two parties will be completed through
⎪
⎪ t
Y
θy ui,j
y
⎩ ui,jy = y=1Y . a smart contract. A data transaction is divided into three steps,
y=1 θy
and the data transmission process is shown in Fig. 4. First,
Since node j not only interacts with node i, it is necessary the smart contract sends the public key of node A to node B.
to introduce the reputation opinions of other nodes on node j After receiving it, node B uses the public key of node A to
when comprehensively evaluating the reputation of j. ωx rep- encrypt the target data and upload it to the hashgraph network.
resents the evaluation weight of node x to node j’s reputation. After the upload is complete, node A obtains the data from the
So, we can rewrite the formula as a summation form hashgraph network and uses the corresponding private key to
⎧ X ty decrypt it to realize the encrypted transmission of the data.
⎪
⎪ τ x=1 ωx τx,j
final =  Without any central credit server and the third party to
⎪ j
⎪
⎪
X
ω
⎨ X x=1 xty obtain the original data, this privacy protection scheme uses
x=1 ωx sx,j
final
sj = X (6) the public/private key of each node to realize the encrypted
⎪
⎪ ω
⎪
⎪ final X x=1 xty transmission of data. Therefore, it effectively ensures data pri-
⎪
⎩u x=1 ωx ux,j
j =  X . vacy in the transparent system. In addition, if necessary, each
x=1 ωx
node can flexibly update its own key pair.
After obtaining the reputation evaluation of node i, this
mechanism can combine the geographic location, credit,
network performance, and CPU resources of the node to IV. B LOCKCHAIN S YSTEM D ESIGN
divide the network into multiple shards and use DQN to make In this section, we introduce the detailed design of the
decisions during the sharding process. blockchain system used in this article.

C. Privacy Protection and Access Control Mechanism A. Sharding-Hashgraph Mechanism

Due to the transparency of the blockchain, all industrial data
stored in the blockchain will be public. However, for the IIoT,
much data are private and require access control. Therefore,
we propose a privacy protection method to support the access
control of sensor data in the hashgraph system.
We designed a sharding-Hashgraph mechanism to improve
the throughput and scalability of the blockchain system. We
will give a detailed introduction to the hashgraph mechanism.
The hashgraph protocol uses the gossip propagation pro-
tocol for communication. In the hashgraph network, node A

Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
GAO et al.: SHARDING-HASHGRAPH: HIGH-PERFORMANCE BLOCKCHAIN-BASED FRAMEWORK
Fig. 5. Hashgraph system can determine the ancestor hash of an event
through the hash signature.
randomly selects another node, such as node B, and then node
A sends what node A knows but node B does not know to
node B, and vice versa. Node A will repeat this process ran-
domly for any other nodes and the same for all other nodes.
Eventually, the information is propagated to each node through
the network.
Fig. 5 describes the data storage structure of Hashgraph-
event. The node creates an event when it receives other
people’s gossip synchronization. The event is a tuple consist-
ing of the hash value of the two-parent events, the transaction
content, the timestamp, and the signature of the rest of the
tuple.
The hashgraph node only accepts events with a valid sig-
nature and hash value that references its existing events. For
example, in Fig. 5, node C receives A3 and can use the sig-
nature of the event to verify that A3 was created by node A
when node B synchronized with it. Node C can request A3’s
ancestor event and it uses the hash contained in the event to
verify it.
In general, Hashgraph is an asynchronous distributed con-
sensus architecture, following the Byzantine failure model,
with high security (consensus probability of 1) and high
throughput. However, Hashgraph has higher requirements for
blockchain network nodes. In order to ensure the stable
operation of the blockchain system, nodes loaded with the
Hashgraph protocol generally require a large amount of stor-
age space and sufficient CPU resources. In addition, the
scalability of the hashgraph protocol is also relatively poor.
In order to further improve the scalability of the blockchain
network, this article introduces a sharding mechanism. The
core of the sharding mechanism is to divide a blockchain
network with many nodes into several subnetworks, and each
subnetwork contains a part of nodes, which is a “shard.” At
the same time, transactions in the network will also be divided
into different “shards” for processing. Each node only needs
to process a small part of the incoming transactions, and dif-
ferent nodes can process transactions in parallel. The sharding
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
17075
mechanism can increase the concurrency of transaction pro-
cessing and verification, thereby improving the throughput of
the entire network.
This article shards the full node based on its geographic
location, credit, and network performance. The scales of the
fragments are different, and they are redistributed regularly.
The nodes in each shard only contribute to the consensus
of events in their own nodes and do not participate in the
consensus of other shards.
The core of the Hashgraph consensus protocol is the vir-
tual voting mechanism. In this article, we have improved the
original virtual voting mechanism and introduced node vot-
ing weight based on the comprehensive state of the node to
improve the consensus security and consensus performance of
the system.
B. Weight-Based Virtual Voting Mechanism
In order to propose a virtual voting mechanism based on
weights, the following definitions are given.
Node Voting Weight: The shard is performed according to
the geographic location, credit, and network performance of
the complete node. The size of the shard is different, and it is
redistributed regularly. The nodes in each shard only contribute
to the consensus of events in their own nodes and only store
the cross-shard events without consensus. Each node has its
own voting weight, and votes are calculated according to the
node status when counting the votes.
Absolute Majority: More than 2/3 of the node weight in the
network.
Strongly Visible: When event B can find the weights of
nodes that cross the absolute majority in all paths of event
A, then event B is strongly visible to event A.
Round: When each node synchronizes a new event, it
immediately calculates whether to create a new round.
Witness: The first event created by each node in each round
is the witness event.
Famous Witness: If a witness in round R can be seen by an
absolute majority of witnesses in round R + 1 (visible witnesses
have an absolute majority weight), then it is a famous witness.
How to Start a New Round: The system divides rounds
according to the time horizon. Once there is an event where
two-thirds of the witnesses (not including myself) can be seen,
it is the beginning of a new round from that event.
Event Creation Round: If event A is strongly visible to more
than 2/3 of the witnesses whose creation round is x, then the
creation round of event A is x + 1, otherwise it is the largest
round among the ancestor events of event A.
Event Acceptance Round: If all well-known witnesses in
round R (creation round) can see a common event, then the
acceptance round of the event is round R.
Virtual Voting Mechanism: Witnesses in each round will
vote for the previous round, such as A3, B3, C3, and D3 will
vote for B2. The voting rule is to vote yes if it is visible;
otherwise, vote no. These voting results will be counted in
later rounds.
Counting Votes: The counting of votes for round x will start
at round x+2, and only strong visible votes will be counted. If
17076
Fig. 6. Hashgraph system uses a virtual voting mechanism to identify famous
witness.
an absolute majority of yes votes are collected, then it will be
a famous witness. As shown in Fig. 6, B4 counts whether B2
is a famous witness. The top B4 can see A3, B3, C3, and D3.
The collected votes are four yes, so B2 is a famous witness. If
this round of witnesses cannot make a decision, the next round
of witnesses will count the votes until a definite conclusion is
reached.
The core of the Hashgraph consensus protocol is divide
rounds, Decision Fame, and find order. In order to reach a
consensus, each node performs the following operations on
its hashgraph:
Divide Rounds: According to the definition and execution
process, hashgraph is divided into several rounds. In addition,
each node votes on each event and selects witnesses for each
round.
Decision Fame: The hashgraph protocol runs an election for
each witness, and each node votes on the witness to select a
famous witness for each round.
Find Order: Once all well-known witnesses have been iden-
tified in a certain round, the acceptance round and consensus
timestamp can be determined for other common events in
this round. According to the acceptance round and consensus
timestamp, all events of the round can be sorted.
After all events in this round are sequenced, the consensus
is achieved in this round.
V. S IMULATION R ESULTS AND D ISCUSSION
In this section, we evaluate how the sharding-Hashgraph
system affects the performance of the blockchain network.
Then, we discuss the simulation results
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
IEEE INTERNET OF THINGS JOURNAL, VOL. 9, NO. 18, 15 SEPTEMBER 2022
TABLE I
P ERFORMANCE C OMPARISON B ETWEEN H ASHGRAPH C ONSENSUS
A LGORITHM AND PBFT C ONSENSUS A LGORITHM
A. Implementation Details
Hashgraph system data transmission uses two encryption
methods: 1) digital signature and 2) hash function. In this test,
the system uses RSA with a 3072-bit key for digital signatures
and uses SHA384 as the hash function. All data communica-
tion is through TLS1.2 communication protocol, using Diffie
Hellman with ephemeral keys and AES256-GCM.
Hashgraph mechanism uses gossip protocol for data broad-
casting. In this system, each node can execute ten random syn-
chronization processes in parallel. In addition, each package
completed event can contain up to 1024 transactions.
B. Evaluation and Comparison
We use Amazon EC2 to evaluate the performance of the
hashgraph system under the following settings.
1) Each Hashgraph node has four virtual CPUs.
2) The running memory of each Hashgraph node is a
random value between 4 and 32 GB.
3) The network performance of each Hashgraph node is a
random value between 1 and 2 Gb/s.
4) All Hashgraph instances are evenly distributed in two
regions.
In order to better reflect the performance of the sharding-
hashgraph system, we deployed a batch of new amazon EC2
nodes, gave these nodes the same performance as other nodes
(random in the same range), and deployed Honey on these
nodes Badger BFT agreement and BEAT0 consensus agree-
ment. These protocols are all ABFT consensus protocols, and
they implement universal state machine replication.
The latency mentioned in this section refers to the average
delay from when the client submits a transaction to the node
for the first time until the node knows the consensus order and
timestamp of the transaction.
In order to prove the superiority of the ABFT consensus
algorithm, we deployed the PBFT consensus mechanism on
these nodes and conducted a simple comparison experiment.
As shown in Table I, HoneyBadgerBFT has the same level of
throughput as PBFT when the number of nodes is small, but
as the number of nodes increases, the throughput of the PBFT
algorithm drops rapidly, and HoneBadgeryBFT has a larger
performance gap with PBFT. It proves the advantages of the
GAO et al.: SHARDING-HASHGRAPH: HIGH-PERFORMANCE BLOCKCHAIN-BASED FRAMEWORK
Fig. 7. Impact of event size on system latency.
Fig. 8. Superiority of the sharding-hashgraph system.
ABFT consensus algorithm in terms of throughput and scala-
bility. In addition, the Hashgraph algorithm has very obvious
performance advantages in any period.
Fig. 7 shows the impact of event size on system latency.
It can be seen that whether it is a Hashgraph system with
four nodes, eight nodes or 16 nodes, as a single transac-
tion becomes larger, the system latency will also become
significantly higher. This is caused by the data transmission
mechanism of the hashgraph system. Since the hashgraph
system uses a virtual voting mechanism to reach a consen-
sus without frequent interaction between nodes, the delay of
the system mainly comes from the transmission delay of the
event on the network. The larger the event, the more time it
takes to upload and download, and the higher the system delay.
Fig. 8 demonstrates the superiority of the sharding-
hashgraph system over other ABFT systems. It can be seen
that with the increase in throughput, the delay of each system
has increased to a certain extent. For example, the delay of
the hashgraph system with four nodes has risen from 0.1
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
17077
Fig. 9. Hashgraph system with sharding mechanism has a better performance
than the hashgraph system without the introduction of the sharding mecha-
nism.
Fig. 10. Hashgraph system with sharding mechanism has good geographic
scalability.
to 0.5 s, and the delay of the hashgraph system with 16
nodes has increased from 0.1 to 0.5 s. 0.4 rose to 1.7 s.
However, it can be seen that whether it is a vertical compar-
ison of latency or a horizontal comparison of throughput, the
remaining consensus algorithms, and the sharding-hashgraph
consensus algorithm have a huge gap in magnitude. This result
comes from the gap in system time complexity: the time com-
plexity of HoneybadgerBFT and BEAT consensus algorithms
are both O(NlogN) level, while the time complexity of the
sharding-hashgraph system is only 0 (B) level, and B is the
size of an event.
Fig. 9 demonstrates the performance gap between the
sharding-hashgraph and the hashgraph system without shard-
ing. It can be seen that the hashgraph consensus system that
does not introduce sharding increases the system delay sig-
nificantly with the increase of the number of nodes, while
the sharding-hashgraph system stabilizes the system delay at
a low level. By introducing the sharding mechanism, the hash-
graph network can be divided into multiple shards, and each
subnetwork has a certain number of nodes, thereby achieving
17078
a significant increase in system scalability and a significant
reduction in system consensus delay.
Fig. 10 demonstrates the performance gap between
sharding-hashgraph and the hashgraph system without shard-
ing from another perspective. It can be seen that the geograph-
ical distribution of hashgraph system nodes will also lead to
a substantial increase in system latency. When the nodes are
distributed in two different areas, the system delay is almost
doubled. However, the sharding-hashgraph system stabilizes
the system consensus delay at a low level. By introducing the
sharding mechanism, the hashgraph network can be divided
into multiple shards according to geographic distribution. The
nodes in each shard are nodes in the same area, thereby sig-
nificantly reducing the impact of the distribution of nodes in
different areas on the system delay.
VI. C ONCLUSION
In this article, we proposed an IIoT distributed data man-
agement system based on blockchain technology and related
core mechanisms. In order to solve the problems of much IIoT
access equipment, equipment power consumption, and data
processing capabilities, this article proposed a novel system
architecture, which divides the system into two layers: 1) the
sensor network and 2) the blockchain network, which can
be used in intelligent deploy high-performance blockchain
systems in scenarios, such as factories. In order to improve
the performance of the blockchain network, we adopted the
sharding-hashgraph mechanism and introduced a comprehen-
sive evaluation mechanism based on the node status to shard a
large number of nodes dynamically. Simulation and compari-
son results showed that the IIoT distributed data management
system proposed in this article has good security, privacy, and
scalability.
The system proposed in this article still has some limita-
tions, such as a large amount of useless information in the
data uploaded by the sensor and limited storage space of the
complete node, which is not conducive to the data manage-
ment of the blockchain network. We will explore how to use
blockchain to better manage and utilize industrial data in the
future direction.
R EFERENCES
[1] J.-Q. Li, F. R. Yu, G. Deng, C. Luo, Z. Ming, and Q. Yan, “Industrial
Internet: A survey on the enabling technologies, applications, and chal-
lenges,” IEEE Commun. Surveys Tuts., vol. 19, no. 3, pp. 1504–1526,
3rd Quart., 2017.
[2] Y. Zhang, R. Wang, M. S. Hossain, M. F. Alhamid, and M. Guizani,
“Heterogeneous information network-based content caching in the
Internet of Vehicles,” IEEE Trans. Veh. Technol., vol. 68, no. 10,
pp. 10216–10226, Oct. 2019.
[3] P. Urien, “Blockchain IoT (BIoT): A new direction for solving Internet
of Things security and trust issues,” in Proc. 3rd Cloudification Internet
Things (CIoT), 2018, pp. 1–4.
[4] X. Wang, S. Garg, H. Lin, G. Kaddoum, J. Hu, and M. S. Hossain,
“PPCS: An intelligent privacy-preserving mobile-edge crowdsensing
strategy for Industrial IoT,” IEEE Internet Things J., vol. 8, no. 13,
pp. 10288–10298, Jul. 2021.
[5] C. Qiu, H. Yao, F. R. Yu, C. Jiang, and S. Guo, “A service-oriented per-
missioned blockchain for the Internet of Things,” IEEE Trans. Services
Comput., vol. 13, no. 2, pp. 203–215, Mar./Apr. 2020.
Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.
IEEE INTERNET OF THINGS JOURNAL, VOL. 9, NO. 18, 15 SEPTEMBER 2022
[6] L. D. Xu, Y. Lu, and L. Li, “Embedding blockchain technology into
IoT for security: A survey,” IEEE Internet Things J., vol. 8, no. 13,
pp. 10452–10473, Jul. 2021.
[7] H.-N. Dai, Z. Zheng, and Y. Zhang, “Blockchain for Internet of Things:
A survey,” IEEE Internet Things J., vol. 6, no. 5, pp. 8076–8094,
Oct. 2019.
[8] S. Iqbal, R. M. Noor, A. W. Malik, and A. U. Rahman, “Blockchain-
enabled adaptive-learning-based resource-sharing framework for IIoT
environment,” IEEE Internet Things J., vol. 8, no. 19, pp. 14746–14755,
Oct. 2021.
[9] M. H. S. Mohammed, “A hybrid framework for securing data trans-
mission in Internet of Things (IoTs) environment using blockchain
approach,” in Proc. IEEE Int. IOT Electron. Mechatronics Conf.
(IEMTRONICS), 2021, pp. 1–10.
[10] H. Yao, T. Mai, J. Wang, Z. Ji, C. Jiang, and Y. Qian, “Resource trading
in blockchain-based Industrial Internet of Things,” IEEE Trans. Ind.
Informat., vol. 15, no. 6, pp. 3602–3609, Jun. 2019.
[11] Y. Zhang, Y. Qian, D. Wu, M. S. Hossain, A. Ghoneim, and M. Chen,
“Emotion-aware multimedia systems security,” IEEE Trans. Multimedia,
vol. 21, no. 3, pp. 617–624, Mar. 2019.
[12] X. Li, D. Li, J. Wan, C. Liu, and M. Imran, “Adaptive transmission
optimization in SDN-based Industrial Internet of Things with edge
computing,” IEEE Internet Things J., vol. 5, no. 3, pp. 1351–1360,
Jun. 2018.
[13] W. Khan, M. Rehman, H. Zangoti, M. Afzal, N. Armi, and
K. Salah, “Industrial Internet of Things: Recent advances,
enabling technologies and open challenges,” Comput. Electr.
Eng., vol. 81, Jan. 2020, Art. no. 106522. [Online]. Available:
https://www.sciencedirect.com/science/article/pii/S0045790618329550
[14] J. C. Chen and V. S. Gabriel, “Revolution of 3D printing technology and
application of six sigma methodologies to optimize the output quality
characteristics,” in Proc. IEEE Int. Conf. Ind. Technol. (ICIT), 2016,
pp. 904–909.
[15] X. Xu et al., “Mitigating conflicting transactions in hyperledger fabric-
permissioned blockchain for delay-sensitive IoT applications,” IEEE
Internet Things J., vol. 8, no. 13, pp. 10596–10607, Jul. 2021.
[16] Z. Zheng, S. Xie, H. Dai, X. Chen, and H. Wang, “An overview
of blockchain technology: Architecture, consensus, and future trends,”
in Proc. IEEE Int. Congr. Big Data (BigData Congress), 2017,
pp. 557–564.
[17] A. Miller, Y. Xia, K. Croman, E. Shi, and D. Song, “The honey badger
of BFT protocols,” in Proc. ACM SIGSAC Conf. Comput. Commun.
Security, 2016, pp. 31–42.
[18] S. Duan, M. K. Reiter, and H. Zhang, “BEAT: Asynchronous BFT made
practical,” in Proc. ACM SIGSAC Conf., 2018, pp. 2028–2041.
Ningjie Gao received the B.S. degree in communi-
cation engineering from Beijing University of Posts
and Telecommunications, Beijing, China, in 2018,
where he is currently pursuing the Ph.D. degree
with the State Key Laboratory of Networking and
Switching Technology.
His current research interests include software-
defined networking, named data networks, multi-
access computing, blockchain, and reinforcement
learning.
Ru Huo received the B.S. degree in electronics and
information engineering from Harbin Engineering
University, Harbin, Heilongjiang, China, in 2011,
and the Ph.D. degree in information and communi-
cation engineering from Beijing University of Posts
and Telecommunications, Beijing, China.
From September 2015 to September 2016, she
studied with The University of British Columbia,
Vancouver, BC, Canada, as a visiting Ph.D. student.
She is currently a Lecturer with Beijing University of
Technology, Beijing. Her current research interests
include software-defined networking, information-centric networking, multi-
access edge computing, blockchain, IIOT, and resource scheduling.
GAO et al.: SHARDING-HASHGRAPH: HIGH-PERFORMANCE BLOCKCHAIN-BASED FRAMEWORK 17079

Shuo Wang received the B.S. degree in commu-
nication engineering from Zhengzhou University,
Zhengzhou, China, in July 2013, and the Ph.D.
degree in information and communication engi-
neering from Beijing University of Posts and
Telecommunications, Beijing, China, in July 2018.
He has been a Postdoctoral Researcher with
Beijing University of Posts and Telecommunications
since 2018. His research interests include data center
networking and software-defined networking.
Yunjie Liu received the B.S. degree in technical
physics from Peking University, Beijing, China, in
1968.
He is currently an Academician with China
Academy of Engineering, the Chief of the Science
and Technology Committee of China Unicom,
and the Dean of the School of Information and
Communication Engineering, Beijing University of
Posts and Telecommunications, Beijing. His current
research interests include next-generation networks,
network architecture, and management.

Tao Huang (Senior Member, IEEE) received the

B.S. degree in communication engineering from
Nankai University, Tianjin, China, in 2002, and
the M.S. and Ph.D. degrees in communication and
information system from Beijing University of Posts
and Telecommunications, Beijing, China, in 2004
and 2007, respectively.
He is currently a Professor with Beijing University
of Posts and Telecommunications. His current
research interests include network architecture,
network artificial intelligence, routing and forward-
ing, and network virtualization.

Authorized licensed use limited to: Wuhan University. Downloaded on November 22,2023 at 02:38:23 UTC from IEEE Xplore. Restrictions apply.