Professional Documents
Culture Documents
SATOSHI FUJITA
Department of Information Engineering, Hiroshima University,
Kagamiyama 1-4-1, Higashi-Hiroshima, 739-8527, Japan
fujita@se.hiroshima-u.ac.jp
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
This paper considers the problem of improving the routing performance of hierarchical
Delaunay networks. Delaunay network is a network topology for peer-to-peer systems
based on the Delaunay triangulation of a set of points associated with a set of peers.
It is known that Delaunay networks have a favorable property as a topology for peer-
to-peer systems such that a greedy routing scheme always delivers a given message to its
destination without encountering a dead-end. The key idea used in the proposed method is
to apply a hash function to the address of participant peers. More concretely, by applying
a hash function to the coordinate point of the peers and by associating several points to
each peer, we could realize an overlay so that the number of hops to the destination in the
original network could be significantly reduced.
1. Introduction
Development of sustainable publish/subscribe systems is an urgent issue for many
countries and regions including Japan, Korea and Taiwan which frequently suffer
from country-wide disasters such as big earthquake and typhoon. A lessen learnt
from the Great East Japan earthquake happened on March 11, 2011, is that many
network applications such as bulletin board stop their service due to the lost of
servers, regardless of the aliveness of the underlying network infrastructure. This
fact motivates us to realize such network services in a peer-to-peer manner which is
expected to attain a high availability compared with classical client/server systems.
∗ An earlier version of this paper was presented as “Hierarchical Delaunay Network for Peer-to-Peer
Overlay with Address Hashing,” at PDCAT 2017 by the same author.
This is an Open Access article published by World Scientific Publishing Company. It is distributed
under the terms of the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC) License
which permits use, distribution and reproduction, provided that the original work is properly cited,
the use is non-commercial and no modifications or adaptations are made.
1950010-1
December 10, 2019 17:31 JOIN S0219265919500105 page 2
S. Fujita
the Delaunay network is given in Sec. 2). In addition, as will be explained later, we
can check whether a given network structure is a Delaunay network in a distributed
manner. In other words, it could adjust itself to satisfy the favorable property even
if a part of the network is damaged by artificial attacks and/or natural disasters.
This paper focuses on the routing performance of Delaunay networks. A tri-
angulation of point set P is a partition of the convex hull of P by line segments
so that the shape of each region is a triangle. Since links in a Delaunay network
correspond to line segments used in the corresponding Delaunay triangulation and
the number of such line segments is proportional to |P |, the maximum distance
between two peers in a Delaunay
p network with respect to the number of links on a
path is proportional √ to |P |. In other words, in a Delaunay network consisting of
N peers, it takes Ω( N ) hops to deliver a message to its destination in the worst
case. To overcome such a weakness of Delaunay networks, several methods have
been proposed in the literature, such as the introduction of hierarchical structure
to the overlay and a hybrid with other network topology.17,18 The routing in such
hierarchical networks generally consists of three phases; namely (1) the routing to
a gateway peer participating in both upper and lower layers through a path in the
lower later, (2) the routing to a gateway peer closest to the destination through a
path in the upper layer, and (3) the routing to the destination through the lower
layer. The performance of such a three-phase routing is affected by the ratio of peers
participating in the upper layer; namely if it is too small, it takes long time before
reaching the gateway and if it is too large, it takes long time for the routing in the
upper layer. Theoretically, this value is balanced when the upper layer consists of
O(N 2/3 ) peers, and could not be improved as long as the overlay consists of two
Delaunay networks. In addition, although it could be overcome by using three or
more layers, it incurs additional cost for peers selected for the member of higher
layers.
In this paper we propose a method to improve the routing performance of (hier-
archical) Delaunay networks without incurring additional cost for each participant.
1950010-2
December 10, 2019 17:31 JOIN S0219265919500105 page 3
The key idea is to apply a hash function to the coordinate point of each peer. More
concretely, we propose a three-tier network consisting of upper, lower and middle
layers each of which is a Delaunay network of given point set. By applying a hash
function to the coordinate point of peers participating in the upper layer, we could
realize an overlay of the middle layer so that the number of hops to the destination
could be effectively reduced. The results of experimental evaluations indicate that
although we could reduce the average hop count to the destination in the original
Delaunay network to 80% by applying conventional hierarchical Delaunay network,
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
with future works. This paper is an extended version of a conference paper pre-
sented at PDCAT 2017.5 Although it contains several additional contents, the main
difference to the conference version is that it contains experimental results which
were not given in the conference version.
2. Preliminaries
This section describes an overview of Delaunay networks with a formal definition of
Delaunay triangulation.
a This paper assumes that points in P are located in “general” position in the sense that any four
points are not on the circumference of a single circle.
b More precisely, there might exist another point in the circle only when those four points are
arranged on the circumference of a circle. Note that we exclude such cases by assuming that points
are located at general positions.
1950010-3
December 10, 2019 17:31 JOIN S0219265919500105 page 4
S. Fujita
circumscribed circle
c
a flip
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
x
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
b
Fig. 1. Flip of line segment in procedure GENERATE DT. In this example, line segment bc is
replaced by ax by the flipping.
(a) Let ∆abc be a triangle which contains x in its circumscribed circle. Without
loss of generality, assume that x is on the opposite side of a with respect to
line bc and if there are several such points, a vertex of a triangle to have b, c
as two vertices is selected as x.
(b) Replace segment bc by segment ax. In the following, we will call this operation
“flip.”
See Fig. 1 for illustration. Note that in Step 2a, such a vertex x can be uniquely
determined since we are assuming a triangulation. As for Step 2b, since the radius
of the circumscribed circle of ∆abx is smaller than that of ∆abc, and since both
circumscribed circles have segment ab as a chord in common, the circumscribed
circle of ∆abx does not contain c. Since similar claim holds for ∆cbx, every flip
reduces the number of violations by at least one. According to the Euler’s formula
concerned with plane graphs, any triangulation of N points contains at most O(N )
triangles. Thus the maximum number of flips necessary to obtain a DT is at most
O(N ). In fact, it is known that DT can be calculated in Θ(N log N ) time since DT
is the dual of Voronoi diagram and the problem of calculating the Voronoi diagram
of a given point set is as hard as the sorting of N elements.
1950010-4
December 10, 2019 17:31 JOIN S0219265919500105 page 5
Liebeherr et al.10 proposed a P2P overlay such that peers are associated with
points on a two-dimensional space (e.g., geographical location of the peer acquired
through GPS) and communication links are associated with line segments in a DT
of the point set. This overlay has several advantages such that: the physical distance
between two adjacent peers connected by a link is short, and each peer has six neigh-
bors on average. In addition, it has a favorable property so that the greedy routing
of a message towards the direction of the point associated with the destination peer
always reaches the destination without encountering a dead-end.10
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
Let p(i) denote the coordinate point associated with peer i and assume that i
knows the coordinate points of peers in subset N (i), where N (i) denotes the set
of peers adjacent with i. In the greedy routing, after receiving a message m from
a neighbor, peer i reads out the coordinate point p(m) of the destination from the
header of the message, and selects peer j ∗ satisfying the following equation from
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
N (i):
dist(p(j ∗ ), p(m)) = min {dist(p(j), p(m)},
j∈N (i)
where dist(p, q) denotes the Euclidean distance between p and q. It then forwards
the received message to the selected peer j ∗ .
Delaunay network is organized (and maintained) by applying flip operations to
the initial triangulation in a distributed manner,10 while avoiding simultaneous flips
conflicting at a link to guarantee the correctness, which is realized by “locking” all
peers concerned with triangles adjacent with the target triangles of the current flip.
Such a lock can be realized by solving a variant of the maximal independent set
problem in a distributed manner. Since any unlocked flips can be executed concur-
rently, it significantly reduces the convergence time compared with the sequential
execution of flips.
1950010-5
December 10, 2019 17:31 JOIN S0219265919500105 page 6
S. Fujita
Upper layer
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
Lower layer
Fig. 2. Hierarchical Delaunay overlay network consisting of two layers. Red circles represent peers
selected from the lower layer. The Delaunay network in the upper layer is constructed using the
same metric with the lower layer.
by the hyper-cubic shortcut links, and the fact that the underlying graph is organized
according to DT is not used except for the calculation of the intersection of Voronoi
regions with the given bands.13
Skip Delaunay Network (SDN)18 proposed by the same authors is a more explicit
extension of the skip graph. Recall that the basic idea of skip graph is to assign a
random bit sequence to each peer as the ID and to construct ring structure defined
by the lexicographical order of IDs in a hierarchical manner.2 The baseline is a
single ring consisting of all peers and the depth of the hierarchy is determined
by the prefix length. More concretely, the ith level (i ≥ 0) of the hierarchy is a
collection of rings each of which consists of peers to have ID with the same prefix of
length i. Note that by definition, the ith level consists of (at most) 2i independent
rings. In SDN, on the other hand, several peers to have ID with the same prefix
of length i are connected with a Delaunay network instead of a ring. For example,
the 0th level is an ordinary Delaunay network consisting of all peers and the first
level consists of two independent Delaunay networks, one consists of peers to have
ID with prefix 0 and the other consists of peers to have ID with prefix 1. Such a
partitioning of the set of peers proceeds until the resulting set becomes a singleton,
which indicates that the expected number of Delaunay networks relevant with each
peer is log N . SDN is simple and elegant compared with LRC. However, the cost
spent by each peer becomes log N times of the cost of ordinary Delaunay network,
while the improvement of the routing performance is much less than log N times.
1950010-6
December 10, 2019 17:31 JOIN S0219265919500105 page 7
1950010-7
December 10, 2019 17:31 JOIN S0219265919500105 page 8
S. Fujita
Upper layer
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
Middle layer
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
Lower layer
Fig. 3. Proposed overlay network. The middle layer consisting of 2|U | = 8 points is the third
Delaunay network in the overlay network.
{p(i) : i ∈ U } ∪ {h(p(i)) : i ∈ U },
where U is the set of peers participating in the upper layer and h is an appropriate
hash function from R × R to R × R. Those 2|U | points are connected through a
Delaunay network similar to the upper layer. A key point is that each peer in U is
associated with two points in the middle layer which can be used to deliver a message
to its destination (in the middle layer) by repeating greedy forwarding. Of course
those points in the middle layer have specific roles; i.e., points in {p(i) : i ∈ U } play
the role of “bridge” connecting to the other two layers, and points in {h(p(i)) : i ∈ U }
play the role of “tunnel” to a remote point in the middle layer. Namely every message
could pass through the tunnel connecting p(i) and h(p(i)) with no cost even if the
distance of those points in the Euclidean space is very large. The reader should
note that since the average degree of any Delaunay network is bounded by six, the
degree of each peer in the middle layer associated with two points is bounded by 12
on average.
1950010-8
December 10, 2019 17:31 JOIN S0219265919500105 page 9
radius of the convex hull is almost twice of the unit length. Intuitively speaking, this
function swaps the inside and the outside of a unit circle. More precisely, it maps
a point far from the origin to a point close to the origin and vise versa. With this
function, peers corresponding to the vertices of the convex hull in the upper layer
are likely to be connected by a link in the middle layer.
The third function h3 realizes a kind of transposition in a higher dimensional
space. Suppose that the coordinate value of each point p(·) is represented by an
integer of 2n bits. Let I be the set of 2n bit integers. Let g be a function from I to
I which is defined as follows: for each b2n−1 b2n−2 · · · b0 ∈ I,
g(b2n−1 b2n−2 · · · b0 )
=bn−1 bn−2 · · · b0 b2n−1 b2n−2 · · · bn .
Note that g realizes a transposition of points in the 2n dimensional space. With this
function g, the third hash function h3 is defined as follows:
h3 (x, y) = (g(x), g(y)).
1950010-9
December 10, 2019 17:31 JOIN S0219265919500105 page 10
S. Fujita
note that when we use an ordinary two-tier Delaunay network without hash func-
tion, the routing time is minimized when the time for Phase 1 and Phase 2 are
balanced; i.e., the effect of shortcut increases as the number of selected peers de-
creases, but it would take long time before reaching the bridge peer if the number of
selected peers is too small. In contrast, since the proposed scheme realizes random
shortcut links with no additional cost, it could significantly reduce the routing time
without reducing the number of selected peers.
There are several issues to be overcome to effectively realize such an idea in
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
actual networks. The first issue is how to deliver a given message to a bridge peer as
soon as possible in Phase 1, and the second issue is how to deliver the given message
to a bridge peer closest to the destination as soon as possible in Phase 2. In the
following, we describe our solution to the above issues in detail.
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
4.4.1. Phase 1
One possible implementation of Phase 1 is to conduct a greedy routing toward the
destination of the message and to switch to Phase 2 when it meets a bridge peer on
the routing path. Although it is simple and natural, such a naive scheme could not
guarantee the usage of shortcut links, which implies that the worst case performance
of the ordinary Delaunay network could not be improved. To overcome this issue,
the proposed routing scheme uses the notion of spread net. More concretely, as a
preprocessing, it firstly spreads a “net of paths” obtained by connecting nearby
bridge peers in the lower layer, which is used to “catch” message transmitted by the
source peer. A message caught by a path in the net is lead to an end vertex of the
path (i.e., to a bridge peer); thus it could always lead a transmitted message to a
bridge peer.
A detailed description of the procedure is as follows. Recall that U denotes the
set of peers selected for the upper layer. Let Nup (u)(⊆ U ) denote the set of adjacent
peers of u in the upper layer. At first, as a preprocessing, each peer conducts the
following steps.
Preprocessing for Phase1
(1) Each u ∈ U transmits spoke message to all peers v in Nup (u) through the lower
layer, which contains: (1) the coordinate point p(u) of the source of the message
and (2) the number of links traveled by the message, where the latter value is
incremented by one for every time of relaying the message.
(2) The transmitted spoke message is delivered to its destination v using a greedy
routing. Each peer w on the delivery path from u to v could learn from the
message that: (1) there is a bridge peer u around w with (2) the number of hops
to u.
The trajectory of spoke messages forms a net of paths. In other words, each peer
w on a path in the net can know the coordinate value and the hop count to its nearby
1950010-10
December 10, 2019 17:31 JOIN S0219265919500105 page 11
bridge peers in the lower layer. Thus, if w receives a message to destination v ∗ from
its neighbor and if v ∗ is sufficiently far from w and its nearby bridge peers, i.e., if it
recognizes that the use of shortcut links reduces the hop count to destination v ∗ , it
heads the message to its nearby bridge peer as follows:
Heading to Bridge
(1) Let N ∗ (w) be the set of bridge peers recognized by w. Recall that w knows the
hop count to peers in N ∗ (w) in addition to their coordinate value.
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
(2) w selects a peer x with the smallest hop count from N ∗ (w), sets the tentative
destination of the message to peer x, and transmits it out towards the tentative
destination x.
(3) The transmitted message is delivered to its tentative destination using a greedy
routing.
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
(4) After x receiving the message, x starts the delivery of the message in Phase 2
toward a bridge peer closest to its actual destination.
Since each path organized by a spoke message emulates a link in the upper layer,
any message issued by a peer will always be “caught” by the net of paths.
4.4.2. Phase 2
Recall that the set of points in the middle layer is a union of two subsets {p(i) : i ∈
U } and {h(p(i)) : i ∈ U }. For brevity, in the following exposition, we call points in
the former subset red and points in the latter subset white. Note that every bridge
is red.
The objective of Phase 2 is to deliver a message given to a red point in the
middle layer to another red point closest to the destination as quickly as possible.
To this end, we design a routing scheme so that: it firstly delivers the message to
a point which is closest to the destination in the middle layer and is either red or
white (Step 1); if the arrived point is not red, then it forwards the message to a
nearby red point through the middle layer (Step 2); and delivers the message to a
red point closest to the destination in the upper layer (Step 3). Step 3 is the same
as the routing in ordinary Delaunay network. In contrast, the routing in Step 1 is
accelerated by effectively utilizing the tunneling and Step 2 is realized by keeping
a pointer to nearby red points for each white point. A concrete description of the
procedure is as follows:
(1) Two peers u and v are said to be adjacent in the middle layer if their corre-
sponding points are connected by a link in the middle layer. Let Nmid (u) be the
set of peers adjacent to u in the middle layer. Assume that each peer u in U
knows Nmid (u) with their corresponding coordinate values.
1950010-11
December 10, 2019 17:31 JOIN S0219265919500105 page 12
S. Fujita
(2) After receiving a message m from a neighbor, peer w refers to the coordinate
point p(m) of the destination. It then selects a peer x in Nmid (w) such that:
min{dist(h(p(x)), p(m)), dist(p(x), p(m))}
is the smallest among peers in Nmid (w) and it is smaller than
min{dist(h(p(w)), p(m)), dist(p(w), p(m))}.
If there is no such x, then go to Step 3. Otherwise, w forwards the message to
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
5. Evaluation
5.1. Setup
In this section, we evaluate the routing performance of the proposed method by
experiments. In the experiments, we measure the average hop count over all pairs
of peers in each of the following four networks and compare them (note that in
general, the hop count required for the routing of a message from a peer to another
peer in an overlay is different from the distance between those two peers in the
overlay):
(1) ordinary Delaunay network SIMPLE constructed over a given set of peers which
are randomly distributed on a two dimensional plane,
(2) HDOV17 consisting of two layers,
(3) an enhancement of HDOV with a technique given in Sec. 4.4.1 denoted PHASE1,
and
(4) an enhancement of PHASE1 with a technique given in Sec. 4.4.2 denoted
PROPOSED.
The number of peers N is selected from [100,1600], and peers in the upper layer
are selected through random sampling with probability 1 − p, so that a larger p
indicates fewer peers in the upper layer. The reader should note that such a random
sampling would be enough for evaluating the best performance of HDOV since we
are assuming that the location of each peer is randomly given.
5.2. Results
At first, as a baseline of the discussion, we measure the average hop count of SIMPLE.
Figure 4 summarizes the results. The average hop count monotonically increases as
1950010-12
December 10, 2019 17:31 JOIN S0219265919500105 page 13
20
18
16 SIMPLE
Average number of hops
14
12
10
8
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
0
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
√
N increases, which is almost proportional to N , and takes about 14 hops when
N = 1000, indicating that there remains enough room for the improvement.
Next we compare the performance of PHASE1 with SIMPLE and HDOV. Figure 5
summarizes the results, where the vertical axis is the hop count normalized by the
hop count of SIMPLE. Figure 5(a) and (b) show the impact of parameters N and
p to the normalized performance of each scheme, respectively. By Fig. 5(a), we can
find that although HDOV beats PHASE1 for small N ’s less than 300, when N ≥ 400,
PHASE1 significantly improves the performance of other schemes; for example, when
N = 1500, HDOV reduces the hop count of SIMPLE to 80%, which is further reduces
to 60% by applying PHASE1.
The above effect is apparently due to the reduction of the hop count taken during
Phase 1; i.e., the reduction of the length of Phase 1. To verify this, we compare the
length of Phase1 taken by each scheme, and found that although the length of Phase
1 in HDOV monotonically increases as N increases, that of PHASE1 is bounded by
a constant independent of N , which gradually increases as p increases, while the
constant grows more slowly than HDOV for any fixed N .
Finally we compare the performance of PROPOSED with PHASE1. Figure 6 sum-
marizes the results when we use random function h1 in the middle layer. Although
we tried several candidates for the hash function in the experiments (see Sec. 4.3
for the details), we could not find significant difference between them, which is
probably because the (original) location of peers is randomly given in our setting.
A theoretical analysis of the class of hash functions which maximizes the routing
performance for a given distribution of the location of peers, is left as an interesting
open problem. As the figure shows, the introduction of the middle layer with a hash
1950010-13
December 10, 2019 17:31 JOIN S0219265919500105 page 14
S. Fujita
P=0.95
1.4
Average # of hops normalized by simple Delaunay
HDOV
1.2
PHASE1
1
0.8
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
0.6
0.4
0.2
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
0
100 300 500 700 900 1100 1300 1500
Number of node
(a) When p = 0.95.
500 nodes
0.9
HDOV
Average # of hops normalized by simple Delaunay
0.85 PHASE1
0.8
0.75
0.7
0.65
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95
probability p
function significantly reduces the hop count especially for large N ’s; for example,
when N = 1000, PROPOSED reduces the average hop count of SIMPLE to 30%,
indicating that we could realize a four or five step routing in the resulting overlay,
on average.
1950010-14
December 10, 2019 17:31 JOIN S0219265919500105 page 15
P=0.95
1.4
Average # of hops normalized by simple Delaunay
PHASE1
1.2
PROPOSED
1
0.8
0.6
by 116.102.32.160 on 04/22/23. Re-use and distribution is strictly not permitted, except for Open Access articles.
0.4
0.2
0
J. Inter. Net. 2019.19. Downloaded from www.worldscientific.com
6. Concluding Remarks
This paper proposes a hierarchical Delaunay network for peer-to-peer overlays. The
key idea of the proposed method is to associate each peer with two addresses, one is
the given coordinate in the original Delaunay network and the other is the address
obtained by applying an appropriate hash function. We propose a routing scheme
for the resulting hierarchical network. A future work is to build a concrete network
application using the resulting overlay.
Acknowledgment
This work was partially supported by KAKENHI, the Grant-in-Aid for Scientific
Research (B), Grant Number 16H02807.
References
1. F. Araujo and L. Rodrigues, “GeoPeer: A Location-Aware Peer-to-Peer System,” In
Proc. the 3rd IEEE International Symposium on Network Computing and Applications
(2004), pages 36–46.
2. J. Aspnes and G. Shah, “Skip graphs,” ACM Trans. Algorithms 3(4) (2007).
3. H. M. N. D. Bandara and A. P. Jayasumana, “Community-Based Caching for Enhanced
Lookup Performance in P2P Systems,” IEEE Trans. Parallel and Distributed Systems
24(9) (2013) 1201–1217.
4. R. Baraglia et al., “GoDel: Delaunay overlays in P2P networks via Gossip,” In Proc. the
12th IEEE International Conference on Peer-to-Peer Computing (2012), pages 1–12.
5. S. Fujita, “Hierarchical Delaunay Network for Peer-to-Peer Overlay with Address Hash-
ing,” In Proc. PDCAT 2017, pages 305–312.
1950010-15
December 10, 2019 17:31 JOIN S0219265919500105 page 16
S. Fujita
1950010-16