UNIVERSIDAD NACIONAL DEL CENTRO DE LA PROVINCIA DE BUENOS AIRES

FACULTAD DE CIENCIAS EXACTAS
DOCTORADO EN CIENCIAS DE LA COMPUTACIÓN
Un soporte de comunicación P2P
para Organizaciones Virtuales
por
Pablo Sebastián Gotthelf
Tesis sometida a evaluación como requisito parcial
para la obetción del grado de
Doctor en Ciencias de la Computación
Director: Prof. Dr. Marcelo Campo
Co-Director: Prof. Dr. Alejandro Zunino
Tandil, Diciembre de 2008
2
3
UNIVERSIDAD NACIONAL DEL CENTRO DE LA PROVINCIA DE BUENOS AIRES
FACULTAD DE CIENCIAS EXACTAS
DOCTORADO EN CIENCIAS DE LA COMPUTACIÓN
A Peer-to-Peer Communication Support
for Virtual Organizations
by
Pablo Sebastián Gotthelf
A Thesis report submitted in partial fullfilment
of the requirements for the degree of
Doctor in Computer Science
Advisor: Dr. Marcelo Campo
Co-Advisor: Dr. Alejandro Zunino
Tandil, December de 2008
4
5
Abstract
The evolution of the Internet brings newcollaboration possibilities. Distributed resources
become available to a broad number of geographically dispersed users, institutions and
organizations. Several efforts are being made to allow these resources to be shared in an
efficient way. The terms Virtual Organization (VO) stand for a context in which tempo-
rary coalitions of independent, self-organized individuals and/or institutions take place.
VOs are being adopted by different communities, such as Grid computing, multi-agent
systems and collaborative applications to shift from local stable configurations to large
scale dynamic group conformations in the Internet.
Members of a VOinteract with one another in an autonomous way to achieve their objec-
tives. Therefore, VOs are a suitable collaboration environment for equal standing mem-
bers. When the collaboration is driven by equal standing members, no entity should be
in charge of regulating the VO, since control must be shared through different autonomic
administrative domains. As a consequence, relying in a central authority/control could
jeopardize the non-hierarchical nature of the collaboration.
Accordingly, to support VOs in the Internet, Peer to Peer (P2P) systems seems to be an ap-
propriate alternative towards a decentralized communication solution, due to their scal-
ability, robustness and self-organization features. However, providing a decentralized
solution to support equal standing members in the Internet is not easy. The dynamism of
these kind of communities in both their complexion and content adds further complex-
ity. In accordance, the goal of this Thesis is to provide a decentralized middleware to
support the collaboration of equal standing members within a VO, considering a flexible,
scalable, robust, fast and easy deployable solution.
This Thesis proposes a novel peer-to-peer communication support to VOs, enabling a
large number of entities to collaborate in a robust, fair, fast and easy deployable way,
without requiring high capacity servers or the deployment of any other special network
infrastructure. This communication support is based on a binary tree overlay network,
distributing responsibilities among the members, including membership management
and message delivery, making it an inexpensive and easily deployable solution for equal
standing members in the Internet.
To validate the approach, a communication middleware was implemented. It has shown
to be an effective communication support for MoviLog mobile agent platforms and two
groupware applications, Chatero and Science-Peer, for synchronous and for asynchronous
collaboration respectively. Moreover, simulations comparing GMACwith other approaches,
in aspects such as end to end propagation times, group latency, broadcast throughput,
protocol overhead, failure recovery and link stress in different VO settings have shown
that GMAC is a scalable and robust alternative to support VOs in the Internet.
6
7
Resumen
Gracias a la evolución de Internet, han surgido nuevas posibilidades de colaboración.
Usuarios, instituciones y organizaciones dispersas geográficamente logran acceder a un
importante número de recursos distribuidos. Muchos esfuerzos se han abocado a permi-
tir que dichos recursos puedan ser compartidos de forma efectiva. El concepto de Orga-
nización Virtual (VO) surge como el contexto de colaboración adecuado para la realiza-
ción de coaliciones temporales entre individuos y/u organizaciones independientes. Este
concepto está siendo adoptado en diferentes áreas, como en computación Grid, sistemas
multi-agentes y aplicaciones colaborativas. Principalmente, las VOs son utilizadas como
el medio de transición para la colaboración entre configuraciones estáticas a nivel local,
hacia conformaciones dinámicas de mayor escala en Internet.
Los miembros de una VO interactúan entre sí en forma autónoma para alcanzar sus ob-
jetivos. Por lo tanto el concepto de VO resulta apropiado cuando los miembros son ho-
mogéneos, es decir, cuando los integrantes se encuentran a un mismo nivel jerárquico de
colaboración.
Para brindar soporte a VOs, es esencial contar con un soporte de comunicaciones que per-
mita que la colaboración se realice en forma descentralizada. Esto se debe a que ninguna
entidad en particular debe estar a cargo de regular la VO, ya que dicha responsabilidad
debe ser compartida entre diferentes dominios administrativos que la integran. Conse-
cuentemente, en esquemas de participación estrictamente no jerárquicos, depender de
una autoridad o control central puede comprometer la naturaleza de la colaboración.
Por lo tanto, para soportar VOs en forma descentralizada, los sistemas Peer to peer (P2P)
parecen ser una solución adecuada, gracias a sus características de escalabilidad y ro-
bustez. Sin embargo, el dinamismo existente en este tipo de comunidades, tanto en su
constitución como en su contenido, incorpora complejidad adicional al procurar brindar
un soporte comunicacional distribuido.
Por lo tanto, en esta Tesis se propone un nuevo enfoque de comunicación P2P para so-
portar VOs, permitiendo a entidades colaborar en forma robusta, justa, de rápida y fácil
adopción, sin necesidad de disponer de un servidor de alta capacidad o la implementa-
ción de protocolos específicos a nivel de red. Este enfoque se basa en un árbol binario
como estructura overlay, distribuyendo equitativamente entre los participantes las res-
ponsabilidades tanto de membresía como de distribución de mensajes. De esta forma se
provee una solución económica y efectiva para comunicar comunidades homogeneas en
Internet.
Para validar el enfoque propuesto, se implementó un middleware que luego fue utili-
zado para soportar la comunicación de plataformas de agentes moviles MoviLog y dos
8
applicaciones colaborativas: Chatero y Science-peer, para colaboración sincrónica y asin-
crónica respectivamente.
Resultados expermientales en aspectos tales como tiempos de distribución, redundancia
del protocolo, tasa de transferencia y estrés en los vínculos de red han mostrando que
el enfoque propuesto es una alternativa efectiva para soportar VOs en forma escalable,
robusta y de fácil y rápida adopción.
9
Acknowledgments
There are many people I would like to thank for their support and help in the realization
of this work. Above all, I would like to thank my advisors, Marcelo Campo and Alejandro
Zunino, for sharing their knowledge, for their guidance, patience and encouragement in
all the time of research and writing of this Thesis. I also want to express my gratitude to
the National Agency for Science and Technology Promotion (ANCPyT) and the National
Council of Scientific and Technical Research (CONICET) for the financial support for this
work. I am deeply grateful to all the research fellows from the ISISTAN, in particular to
Cristian Mateos, Marco Crasso and Alfredo Teyseire, for sharing experiences, suggesting
several hints and being always willing to lend a hand. Once again, I would like to thank
Marcelo Campo for providing the means necessary for realizing this work, which, in
many cases, extended beyond his responsibilities as the director of the Institute. Finally
I would like to thank my parents Eduardo and Sara, and my sister Gabriela for their
encouragement and support during all these years. Without the support and assistance
of all these people, this Thesis would not have been possible.
P. S. G.
10
Contents
1 Introduction 21
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.2 The Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.4 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2 Background 31
2.1 Virtual Organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2 Distributed Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3 Multicast Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4 Peer-to-peer Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5 Internet Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.5.1 Firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.5.2 Network Address Translators . . . . . . . . . . . . . . . . . . . . . . 40
2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3 Related Work 43
3.1 Virtual Organizations Communication Requirements . . . . . . . . . . . . 43
3.2 Application Level Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.2.1 NARADA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.2 NICE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
11
12 CONTENTS
3.2.3 LARK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3 Peer to Peer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3.1 Napster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.3.2 GNUTELLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.3.3 Chord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.3.4 CAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.3.5 JXTA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3.6 JGroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3.7 Others . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4 Overlay Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4 GMAC 63
4.1 Aims and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2 GMAC Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3 Overlay Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.4 GMAC Message Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.4.1 Multicast Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.4.2 Control Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.4.3 Summarizing Messages . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.4.4 Welcome Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.5 Distributed VO Conformation and Mainteance . . . . . . . . . . . . . . . . 75
4.5.1 Improved Join Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.5.2 Enabling Connectivity Restricted Hosts (CRHs) . . . . . . . . . . . 77
4.5.3 VOs-bootstrap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.5.4 Failure Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.5.4.1 Multicast Messages . . . . . . . . . . . . . . . . . . . . . . 85
4.5.4.2 Summarizing Messages . . . . . . . . . . . . . . . . . . . . 85
4.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
CONTENTS 13
5 Implementation 87
5.1 Concrete Materialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2 Application Programming Interface . . . . . . . . . . . . . . . . . . . . . . 90
5.3 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.3.1 Connectivity-Restricted Hosts . . . . . . . . . . . . . . . . . . . . . 91
5.3.2 VOs-bootstrap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.3.3 Failure Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4 Simulators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.4.1 Application Level Simulator (ALS) . . . . . . . . . . . . . . . . . . . 93
5.4.2 Underlying Network Aware Simulator (UNAS) . . . . . . . . . . . 94
5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6 Virtual Organizations Supported by GMAC 95
6.1 MoviLog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.2 Chatero . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.3 Science-Peer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
7 Experimental Results 105
7.1 Metrics and Evaluation Methodology . . . . . . . . . . . . . . . . . . . . . 105
7.2 General Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.2.1 End-to-end propagation time . . . . . . . . . . . . . . . . . . . . . . 108
7.2.2 Broadcast Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . 109
7.2.3 Protocol Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
7.2.4 Group Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.2.5 Network Level Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.2.6 Failure Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.3 Movilog Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.3.1 End-to-end group propagation delay . . . . . . . . . . . . . . . . . 117
7.3.2 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
14 CONTENTS
7.4 Science-Peer Experimental Results . . . . . . . . . . . . . . . . . . . . . . . 118
7.4.1 Searching times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.4.2 End-to-end propagation delay . . . . . . . . . . . . . . . . . . . . . 120
7.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
8 Conclusions and Future Work 125
8.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
8.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
8.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
8.3.1 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
8.3.2 GMAC Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
8.3.3 Heartbeat messages . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
8.3.4 Indexable Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8.3.5 End-to-end Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8.3.6 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
8.4 Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
List of Figures
2.1 Overlay Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2 Firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.3 Network address translator . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.1 Narada source MST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2 Nice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.3 Lark propagation example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.4 Napster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.5 Gnutella . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.6 Chord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.7 CAN: Bi-dimensional space with 5 nodes . . . . . . . . . . . . . . . . . . . 54
3.8 JXTA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.9 Overlay Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1 GMAC tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2 Sequential unicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.3 GMAC message propagation example . . . . . . . . . . . . . . . . . . . . . 68
4.4 Summarizing message example . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5 Improved Join Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.6 Rotation of a connection restricted child . . . . . . . . . . . . . . . . . . . . 78
4.7 Heuristic for allowing CRHs . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
15
16 LIST OF FIGURES
4.8 VOs-bootstrap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.9 Failure recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.1 GMAC tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.2 GMAC Class Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.1 MoviLog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.2 Chatero, a synchronous groupware example . . . . . . . . . . . . . . . . . 98
6.3 Science-Peer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.4 Science-Peer papers pre-processing . . . . . . . . . . . . . . . . . . . . . . . 100
6.5 Science-Peer query distribution . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.6 Science-Peer summarizing answer . . . . . . . . . . . . . . . . . . . . . . . 102
6.7 Science-Peer query results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
7.1 Time to send a 100KB multicast message . . . . . . . . . . . . . . . . . . . . 108
7.2 Time to send a 100KB message (without unicast) . . . . . . . . . . . . . . . 109
7.3 Maximum VO broadcast throughput . . . . . . . . . . . . . . . . . . . . . . 110
7.4 Protocol overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.5 Protocol overhead (without NARADA) . . . . . . . . . . . . . . . . . . . . 113
7.6 Group latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7.7 Average link stress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.8 Link stress distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.9 GMAC failure recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
7.10 Failure recovery on a 100-node VO . . . . . . . . . . . . . . . . . . . . . . . 118
7.11 Time to send a 2KB MoviLog multicast message . . . . . . . . . . . . . . . 119
7.12 MoviLog bandwidth requirements . . . . . . . . . . . . . . . . . . . . . . . 120
7.13 Science-Peer vs centralized approach . . . . . . . . . . . . . . . . . . . . . . 121
7.14 Science-Peer vs centralized approach . . . . . . . . . . . . . . . . . . . . . . 122
7.15 Time required to deliver a 6KB query to the whole VO . . . . . . . . . . . 123
7.16 Time required to deliver a 6KB query to the whole VO (without Unicast) . 123
8.1 GMAC proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
8.2 Heartbeat summarizing messages example . . . . . . . . . . . . . . . . . . 131
List of Tables
2.1 Client/server vs P2P paradigm . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.1 Search strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2 Properties of the different approaches . . . . . . . . . . . . . . . . . . . . . 61
7.1 Protocol overhead (in KB/s) . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
17
18 LIST OF TABLES
List of Algorithms
1 Sending a multicast message to the whole VO . . . . . . . . . . . . . . . . 71
2 Receiving and forwarding a multicast message . . . . . . . . . . . . . . . . 71
3 Sending a summarizing message request . . . . . . . . . . . . . . . . . . . 73
4 Receiving and forwarding a summarizing message request . . . . . . . . . 74
5 Sending a summaring answer . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6 Receiving a summaring answer . . . . . . . . . . . . . . . . . . . . . . . . . 75
7 Joining a Virtual Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 77
8 Receiving a Joining request . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
9 Handling a Join request, full heuristic. . . . . . . . . . . . . . . . . . . . . . 79
10 Recovering from a parent failure . . . . . . . . . . . . . . . . . . . . . . . . 83
11 Recovering from a child failure . . . . . . . . . . . . . . . . . . . . . . . . . 83
19
20 LIST OF ALGORITHMS
Acronyms
ACS Autonomic Computing System
ALM Application Level Multicast
ALMI An Application Level Multicast Infrastructure
API Application Programming Interface
CRH Connectivity-Restricted Host
DNS Domain Name System
DHT Distributed Hash Table
ISP Internet Service Provider
HTTP HyperText Transfer Protocol
LAN Local Area Network
MAS Multi Agent System
MST Minimum Spanning Tree
NAT Network Address Translator
P2P Peer to Peer
RDF Resource Description Format
RDP Relative Delay Penalty
TTL Time To Live
VO Virtual Organization
Chapter 1
Introduction
The evolution of the Internet has brought new collaboration possibilities. A growing
number of autonomous entities, users, institutions and organizations spread across the
Internet are willing to collaborate with each other, sharing their resources, which include
data, processing power, disk storage, and applications. Several efforts are being made to
allow the collaboration of these autonomic entities and to enable them to share resources
in an effective way.
The Grid, according to Foster et al. (2001), is concerned with coordinated resource sharing
and problem solving in dynamic, multi-institutional VOs. In analogy with the electrical
power grid, computer resources should be available just like electricity from a power
outlet, where the user do neither has to be concerned about where the power comes
from, nor the complexity hidden behind (Chetty and Buyya, 2002). The term VO stands
for a temporary coalition of independent, self-organized individuals and/or institutions
defined by sharing rules (i.e. what is shared and under what conditions the sharing
occurs) (Czajkowski et al., 2001; Foster et al., 2001; Nami and Sharifi, 2007).
Members of a VOinteract with one another in an autonomous way to achieve their objec-
tives. In a VOenvironment, where members can share resources and skills, it is suitable to
use autonomic behavior such as Multi Agent Systems (MASs), a collection of interacting
autonomous agents, when building self-organizing VOs (Nami and Sharifi, 2007). Nami
and Sharifi (2007) consider Autonomic Computing Systems (ACSs) as an inexpensive
approach for developing large scale distributed systems, since they manage themselves,
hiding their complexity from the users’ view. ACSs are formed by autonomic elements
whose interactions enable self-managing behavior. An agent is defined as an entity with
its own state, memory, and mechanisms to sense and interact with the environment. Since
agents in MASs are autonomous, they can be effective in conforming reconfigurable VOs.
The Grid adopts VOs as a “bridge concept” in which the transition from controlled sta-
21
22 CHAPTER 1. INTRODUCTION
ble multi-clusters running over high speed networks to temporary coalitions of dynamic
autonomic cooperating entities in the Internet takes place (Foster et al., 2001; Foster and
Iamnitchi, 2003). From this perspective is understandable the diversity of VOs configu-
rations and the broad and extensive usage the concept encompass. Nevertheless, there
is a clear understanding of the need to support distributed, self-managing, collaborating
members within a dynamic environment.
To allow these interaction to take place in the Internet, several considerations should be
made. In contrast to Local Area Networks (LANs), network latency and bandwidth con-
straints restrain the collaboration options. IP multicast was considered as the solution to
support many-to-many communication over an IP network. In order to support IP mul-
ticast on the Internet, the MBONE (Multicast Backbone) has been built. The MBONE is
a virtual network extended across the Internet that supports IP multicast traffic (Eriks-
son, 1994). Despite the need for multicast services, the usefulness of the MBONE is still
limited (Diot et al., 2000; Zhang et al., 2006), mainly because of its low adoption.
An alternative for group communications are overlay networks (Touch, 2001; Pendarakis
et al., 2001; Chu et al., 2002). An overlay network is a network built on top of another
network (Touch, 2001). In the Internet, an overlay network can be very advantageous as
the whole communication can be supported by user-level applications, relying only on
unicast connections. In this manner, neither the deployment special routers, nor Inter-
net Service Provider (ISP) involvement is required. Nonetheless, entities communicating
through the Internet using unicast connections still have to face some difficulties such as
having to traverse several routers, firewalls and Network Address Translators (NATs).
Overlay networks have been extensively used in Application Level Multicast (ALM) al-
ternatives and in P2P systems. ALM alternatives are mostly focused on supporting one
sender disseminating media such as audio or video streaming. On the other hand, P2P
systems are centered in supporting file and resource sharing.However, the interaction
schemes required to support VOs demand special communication primitives. For in-
stance, Malone (1990) already discussed the need for autonomic entities communication
support, and suggested to combine ideas from artificial intelligence and organization
theory to route information within organizations. More precisely, a global blackboard is
proposed as interaction module to adapt to different communication patterns in a flexible
way, as entities can communicate with one another. Recently, Iamnitchi and Talia (2005)
have discussed specifically the environmental aspects that influences the design of VOs,
as will be described in Section 3.1.
Therefore, the communication characteristics that take place in VOs differ slightly from
the requirements ALM alternatives and P2P systems were designed for. Although, mul-
ticast communications are required to make VO announcements, such as events, status
information, requests, etc., they take the form of small sporadic messages and moreover
1.1. MOTIVATION 23
every member should be able to send these kind of messages to the whole VO. Resource
sharing between entities is also desired. However, instead of sharing static or indexable
content -as supported by P2P systems-, the implicit dynamism of VOs, which embraces
members and resources, also requires to support content that is frequently changing, as
well as gathering information from data spread across a VO.
VOs should also be fast and easily deployed. The autonomous entities taking part in
VOs should participate in an even and fair way. In other words, the non-hierarchical
nature of the collaboration should be driven by equal standing members. Accordingly,
the utilization of overlay networks seem to be an adequate approach to provide a fast
and easy deployable solution for equal standing members.
In contrast to supporting the communication with a central server, overlay networks can
be build quickly and easily without requiring the deployment and configuration of a high
capacity server. Overlay networks also enable decentralization, which, in addition to
providing robustness (no single point of failure), enable to achieve collaboration without
the intervention of third party services or high capacity centralized servers that could
jeopardize the non-hierarchical nature of the collaboration.
At this stage, it is important to mention that the terms collaboration and coordination
are not used interchangeably in this Thesis. The term collaboration is referred as the
application-level interaction activities being supported, in other words, the goal of this
work is to support the collaboration of equal standing members in VOs. On the other
hand, the term coordination is referred to the synchronized actions required by these
entities to make the collaboration possible.
In summary, a fast and effective communication support for the collaboration of equal
standing members in VOs is required. Accordingly, this Thesis proposes a Peer-to-peer
communication support for virtual organizations, considering the communication char-
acteristics of equal standing members in the Internet.
1.1 Motivation
To enable the collaboration of equal standing members, a communication middleware is
required to effectively support VOs in the Internet. This middleware should allow mem-
bers to join a VO and provide them with the communication primitives needed to inter-
act within the VO environment. Therefore, there are certain communication premises,
requirements and characteristics to be considered when supporting VOs in the Inter-
net (Desanctis and Monge, 1999; Foster and Iamnitchi, 2003; Foster et al., 2001; Wang
et al., 2005):
24 CHAPTER 1. INTRODUCTION
• Burst Communication: Members rely on small sporadic messages to coordinate the
VOs. These messages are usually used to make group announcements, request
resources, request VO related information, etc. In other words, there is no video or
audio streaming.
• Fairness: Members must cooperate in a fair way to build and maintain the VOs.
• Homogeneous groups and network links: VO members have similar behavior and
communication characteristics. Therefore, transmission probability, connection re-
sources and communication demands are similar.
• Scalability: Agrowing number of both members and resources should be supported.
Accordingly, it is important that building, maintenance and coordination be effi-
cient with respect to the number of members and resources they provide.
• Unknown Topology: Information about the underlying network is not available.
From the application perspective, it is very difficult to determine or take advan-
tage of the underlying Internet topology and physical paths (Kwon and Fahmy,
2002), making difficult the construction of a topology aware overlay.
• Connectivity-restricted clients: These are hosts restricted by firewalls or NATs, which
prevent them from being reachable by other members from the Internet. This type
of hosts should not be excluded from integrating a VO, though they may receive a
degraded service.
• Robustness: Hosts failures should not jeopardize the rest of the VO. Therefore, it is
desirable that members assume the same role or responsibility in the VO, in contrast
to relying in a central server.
• Fast and easy deployment: Members should be able to build and maintain a VO with-
out requiring neither mayor configuration efforts, nor servers spread across the
Internet.
Decentralization is an essential aspect to be covered. On one hand, as control should be
shared through different autonomic administrative domains, no entity is (or should be) in
charge of regulating a VO. Since having a centralized server often makes participants feel
like they relinquish control or privacy over their actions (even when privacy is not really
compromised). The collaboration of equal standing members should take place without
requiring any central authority/control that could jeopardize the non-hierarchical nature
of the collaboration.
On the other hand, decentralization should also be achieved for technological reasons,
since the deployment of a central server might be hard and time consuming, as it re-
1.1. MOTIVATION 25
quires administrative tasks and a high-speed Internet connection. Is becoming increas-
ingly evident that centralized solutions might not only compromise availability, because
they can be seen as a single point of failure, but also scalability, because they often act as
a performance bottleneck, resulting in inefficient network usage (Osais et al., 2006), poor
performance and limited scalability. Moreover, a decentralized solution should be easy
and fast to deploy.
Taking all these factors into consideration, an overlay network that can provide a flexi-
ble and decentralized solution to support VOs for equal standing members is required.
When building such solution, the following aspects, some of them coincident with the
ones previously identified by Iamnitchi (2003), should be considered:
• Membership protocol: A membership protocol specifies how new nodes join an over-
lay and how the awareness of other members is achieved. Ideally every member
would knoweach other, however, in practice this is very hard to accomplish (Chan-
dra et al., 1996), thus, several strategies are adopted. For example, some approaches
use a quite aggressive membership protocol, where each node keeps information of
a significant number of members, hence requiring high communication costs (Ri-
peanu et al., 2002). Some enhancements such as structuring this information or
utilizing controlled epidemic “gossip” based membership protocols have shown to
achieve an acceptable scalability (Golding and Taylor, 1992).
• Propagation mechanism: Propagation mechanisms allow queries and messages in
general to get to the proper destination. Some messages must reach all the VO
members. This is why multicast services are often required for supporting VOs.
In most cases, the propagation scheme is dictated by how the interconnections be-
tween members are structured. For instance, if the overlay follows a known struc-
ture, messages could be propagated efficiently. In contrast, in ad-hoc overlays mes-
sage propagation is often inefficient.
• Overlay Construction and maintenance: The overlay network must be able to allowthe
arrival and departure of members. This implies an overlay restructure that must be
done fast and effectively to avoid interfering with the normal activities taking place
in the VO. Some approaches try to interconnect nodes intelligently, producing an
optimized overlay network, and periodically running optimization and stabiliza-
tion mechanisms to keep consistency (Ratnasamy et al., 2001; Stoica et al., 2001).
However, these mechanisms require additional network resources and processing
power, which also increase with the number of members and the dynamism of the
environment. On the other hand, unstructured overlays are simpler to maintain
and allow better robustness (Ripeanu et al., 2002).
26 CHAPTER 1. INTRODUCTION
• Preprocessing: Some previous activities must be done before the overlay is fully
functional. These activities may encompass resource indexing, metadata organi-
zation, etc. For example, it is possible to disseminate resource descriptions in order
to improve the search performance for resource discovery purposes. Moreover,
overlay structures could be adapted considering the utilization patterns. How-
ever, it is not evident how such strategies could be adopted in dynamic environ-
ments (Iamnitchi et al., 2002), where resources might become unavailable and mem-
bers’ behavior could change suddenly.
When designing a decentralized communication support for VOs, all these aspects
should be taken into account. Having an optimized overlay allows better message prop-
agation schemes. Nevertheless, optimized structures require expensive overlay construc-
tion and maintenance costs, which, depending on the collaboration activities, could de-
mand more network resources than the VO messages themselves. In this regard, ad-
hoc overlays are simpler to build and maintain, but they introduce significant traffic to
the network, disseminating messages unintelligently. Structuring the overlay, enhances
the message propagation schemes, however these approaches are mostly based on Dis-
tributed Hash Tables (DHTs), using indexable data which might lead to inconsistencies
in dynamic environments.
It is therefore required a robust, scalable, fast and easy deployable solution to support
collaboration of equal standing members, in the context of VOs, in a fully decentralized
way (Gotthelf et al., 2008a).
1.2 The Thesis
Taking into account the new interaction possibilities the Internet offers and the commu-
nication problems discussed above, it is clear that a flexible, scalable, robust, fast and
easy deployable communication support to allow collaboration of dynamic equal stand-
ing members in a decentralized way is needed. In this sense, this Thesis proposes a
decentralized approach to suport the required collaboration in the context of a VOs. This
type of collaboration encompasses a wide variety of distributed communities, such as the
Grid, multi-agent systems and computer supported collaborative work just to mention a
few.
In this Thesis a new middleware called GMAC (Group Management Agent Cast) (Got-
thelf et al., 2005, 2007, 2008b,a) to support homogeneous VO communities is proposed.
In order to limit the scope of this work, we will focus in achieving the fast formation of
VOs, its maintenance and collaboration primitives in a scalable, flexible, robust and easy
1.2. THE THESIS 27
deployable way. Since GMAC intends to provide support for autonomous collaborat-
ing members, this Thesis will address how decentralized cooperation of equal standing
members, in the context of VOs can be achieved, given the requirements introduced in
Section 1.1.
In order to provide easy and fast support for Virtual Organizations, the basic idea is to
build a P2P overlay based on a binary tree, formed by VO members as nodes. Nodes
could be hosts, agents, or end users, while the links of the binary tree are network con-
nections. In this way, each node is connected to other three nodes at most, the parent,
and the left and right child.
This middleware provides participants with the means required to establish and partic-
ipate in a VO. These services encompass membership management and group commu-
nication primitives. The whole functionality is achieved in a collaborative and decentral-
ized manner, as the responsibilities for message delivery, tree building and failure recov-
ery, are distributed among the group members in a fair way. Therefore, when members
either join a VO, leave unexpectedly, need to send group announcements or search for
certain resources, the remaining members also take part, balancing the load and relieving
burdened members.
In opposition to other approaches (Chu et al., 2002; Pendarakis et al., 2001; Jannotti et al.,
2000; Chawathe, 2003; Francis, 2000; Nami and Sharifi, 2007; Banerjee et al., 2002), where
the overlay structure is based on information inferred about the underlying network, the
approach proposed in this Thesis does not attempt to infer information about the under-
lying network when building the overlay. The generation and maintenance of an overlay
structure that considers the underlying network requires performing several measure-
ments, which also use network resources, producing significant protocol overhead and
compromising scalability. Consequently, optimized overlay structures are best suited for
transmitting heavy traffic such as audio or video streaming. Each node participates in the
construction and maintenance of the VOs. However, other interactions are more likely to
be using the network resources in a more intensive manner. As a consequence, it might
be inappropriate using a host with a better Internet connection as a group re-transmitter,
or burdening it with extra tasks. Besides, equal standing members are supposed to have
similar connection capabilities. Accordingly, GMAC guarantees that each group member
will retransmit data to two members at most.
On the other hand, ad-hoc overlay networks such as GNUTELLA (Ripeanu et al., 2002),
also present some of the problems listed above. In ad-hoc networks the node degrees
usually follow the power-law distribution, this is, most nodes have a small number of
links and few nodes have a large number of links, which guarantees low dissemination
latency (Aberer et al., 2005). However, the power-law distribution overlooks the fair-
ness requirement, as nodes degrees are not balanced, suggesting that some nodes are
28 CHAPTER 1. INTRODUCTION
more loaded that others. Moreover, in these kind of networks, message flooding is used
as propagation mechanism, introducing high network bandwidth consumption (Aberer
et al., 2005).
For these reasons, GMAC uses a binary tree without considering each node capabilities
in particular, which should be rather similar since VOs are formed by equal standing
members. In addition to avoid performing measurements, the protocol overhead is also
reduced by using a fast failure detection and recovery strategy, instead of a failure avoid-
ance scheme which would require additional maintenance costs.
1.3 Contributions
This thesis introduces the following important contributions in the distributed collabora-
tion context:
• A new overlay network model is proposed to support VO of homogeneous com-
munities in a decentralized way. The overlay proposed is based on a binary tree as
overlay network, distributing responsibilities among the members, enabling self-
organization of the VO with low overhead, and conforming a simple, yet powerful
solution to establish VOs in the Internet. This novel approach is oriented to provide
communications when the collaboration is driven by equal standing members. In
VOs this is generally the case, as control should be shared through different auto-
nomic administrative domains. Therefore, decentralization is not only essential as
a technological factor, but also for conceptual reasons, since coordination of equal
standing members requiring a central authority/control could jeopardize the non-
hierarchical nature of the collaboration.
The binary tree structure helps bounding the node degree, avoiding flooding and
achieving logarithmic distribution times, whereas a higher tree degree would in-
crease the load imbalance between leaves and internal nodes. In addition, the root
or nodes higher in the hierarchy are not overloaded, since every node stores and
transmit the same amount of control information thanks to the recursive nature of
the solution. This recursive feature also enables different kind of enhancements,
such as task parallelization and efficient data gathering.
• Based on this model, a new middleware was developed as a Java library to support
the collaboration of equal standing members in a scalable, robust, fast and easy
deployable way. In addition, Connectivity-Restricted Hosts (CRHs) are supported
as leaves of the tree, allowing half of the VO members to be CRHs.
• The GMAC middleware was tested in the following VO settings:
1.4. ORGANIZATION 29
– MoviLog: the mobile agent platform MoviLog (Zunino et al., 2005; Mateos
et al., 2007) was improved to use GMAC multicast communication services.
Mobile agents platforms are a special case of equal standing VOs. Mobile
agents reduce network usage by migrating code, allowing them to work with
resources locally. In order to reduce network load, MoviLog platforms need
to send small, sporadic, multicast messages announcing available resources
such as code, data or services, to provide communication between agents and
for managing their mobility. The coordination of these autonomous platforms
was successfully achieved with the GMACmiddleware (Gotthelf et al., 2008b),
in an autonomous way, without relying in a central server, as mobile agent
platforms usually belong to independent organizations/institutions.
– Groupware: many groupware applications also take the form of VOs. In this
context, different adoptions of GMAC were studied, as well as how to tackle
the different collaboration requirements in each particular case. More pre-
cisely, two collaborative tools were developed over GMAC:
*
Chatero (Gotthelf et al., 2007) is a synchronous brainstorming application
that allows members to chat, drawover fixed background images and vote
for courses of action.
*
Science-Peer (Gotthelf et al., 2008a) is an asynchronous P2P tool for find-
ing scientists with similar research interests.
• Experimental results have shown that the middleware is a practical solution for
supporting VOs in the Internet. Several simulations considering aspects such as
end to end propagation times, group latency, broadcast throughput, protocol over-
head, failure recovery and link stress were performed in the different VO settings,
showing that GMAC is a scalable and robust alternative in comparison with other
approaches, such as NARADA, NICE and LARK.
All in all, the goal of this Thesis is to address how decentralized collaboration of dynamic
homogeneous members, in the context of VOs can be achieved, considering a flexible,
scalable, robust, fast and easy deployable solution.
1.4 Organization
The rest of this document is organized as follows:
Chapter 2 introduces the main concepts upon this Thesis is based. It also discusses some
of the previous efforts and alternatives for group communications.
30 CHAPTER 1. INTRODUCTION
Chapter 3 describes and analyzes different communication approaches, from application
level multicast alternatives to Peer-to-peer systems. A detailed comparison of these ap-
proaches is presented and their drawbacks are highlighted from the VO requirements
perspective.
Chapter 4 describes the generalized Group Management Agent Cast (GMAC) model,
a P2P communication support for Virtual Organizations. In this chapter the message
primitives as well as the construction and maintenance algorithms are also presented.
Chapter 5 discusses relevant aspects related to the design and implementation of GMAC
and its primitives. The basic architecture of GMAC and some of its components are de-
scribed as well as some of the most relevant implementation details, such as the protocol
connection handshaking process.
Chapter 6 presents some of the applications built over GMAC to allow the collaboration
in different VO settings. The first VO presented is the one formed by Movilog Platforms,
which was partially derived from (Gotthelf et al., 2005, 2008b). Additionally, in this chap-
ter GMAC is also shown in some particular cases of groupware applications, where col-
laboration in a VOs environment could be advantageous, derived from the works (Got-
thelf et al., 2007, 2008a).
Chapter 7 describes the middleware related metrics concerning the VO communication
characteristics, as well as the experimental evaluation specifically for the different VO
setups in which GMAC was used. Details on how this results were obtain, such as simu-
lations and models are also explained in this chapter.
Finally, the conclusions and contributions of this Thesis are presented in Chapter 8, where
future possibilities and limitations are discussed.
Chapter 2
Background
VOs can be seen as the natural context of collaboration among autonomous entities in
the Internet. In this collaboration environment, equal standing members share and con-
sume distributed resources. A communication support for VOs is required in the Inter-
net. However, due to the inherently complex and heterogeneous characteristics of the
Internet, supporting VOs is a difficult task. VOs members should be able to commu-
nicate in a fast and effective way, without requiring a central server that could jeopar-
dize the non-hierarchical nature of the collaboration, since no central control/authority
should be regulating the VO. Decentralized group communication schemes in the Inter-
net are present in application level multicast and peer to peer systems. ALM alternatives
were proposed after the deployment and scalability problems of IP multicast in the In-
ternet (MBONE) (Diot et al., 2000). On the other hand, P2P systems came along as a new
paradigm in which, in contrast to the conventional client/server model, participants not
only consume, but also provide resources to their peers. Both, ALM and P2P systems
strongly rely on overlay networks to provide their services and, from a different perspec-
tive, enable participants or members to organize in groups.
This chapter provides an insight of the related concepts and terms used in this Thesis. The
next section describes the components and features of VOs. Section 2.2 describes further
details about the characteristics of distributed resources. Alternatives to multicast and
peer to peer systems are introduced in Sections 2.3 and 2.4, respectively. In Section 2.5
some of the connectivity restrictions present in the Internet are described.
2.1 Virtual Organizations
According to Foster et al. (2001), the termVOstands for a temporary coalition of indepen-
dent, self-organized individuals and/or institutions defined by sharing rules, i.e. what is
31
32 CHAPTER 2. BACKGROUND
shared and under what conditions the sharing occurs.
The Grid community adopts VOs as a “bridge concept” in which the transition from con-
trolled stable multi-clusters running over high speed networks to temporary coalitions
of dynamic autonomic cooperating entities in the Internet takes place (Foster et al., 2001;
Foster and Iamnitchi, 2003). Fromthis perspective is understandable the diversity of VOs
configurations and the broad and extensive usage the concept encompass. Nevertheless,
there is a clear understanding of the need to support distributed, self-managing, collabo-
rating members within a dynamic environment.
Examples of VOs could be: storage service providers, members of an industrial consor-
tium bidding on a new aircraft, a crisis management team and the simulation systems
that are used to plan a respond to an emergency, or personal software agents exchanging
users’ profile information. As these examples show, VOs vary enormously in their pur-
pose, scope, size, duration and community constitution. However, there is a broad set of
common concerns and requirements that are needed to support VOs in the Internet.
Groupware applications allow people to work together in the Internet (Ellis et al., 1991;
Wheeler et al., 1999). As in some cases this collaboration is also driven by the coalition of
independent self-organized individuals, there is an inherent association with VOs. These
kind of collaborative systems can be categorized by the way their participants are inter-
connected and how messages are transmitted (Ma et al., 2003; Ehrlich et al., 1989). In
centralized architectures, organizations or enterprises provide the means for supporting
group interaction. This is suitable when the members of an organization work together
in a specific task, and guidance or event tracking of the group activity is desirable. How-
ever, in some cases, no entity is (or should be) in charge of regulating the group. As a con-
sequence, having a centralized server often makes participants feel like they relinquish
control or privacy over their actions, even when privacy is not compromised. Moreover,
with the evolution and adoption of large groupware applications, it is becoming increas-
ingly evident that centralized solutions might not only compromise availability, but also
scalability, due to their inefficient network usage (Osais et al., 2006). Furthermore, in
many cases, it is merely not possible to afford or to have a central server available timely,
for instance in emergency situations (Scalem et al., 2004).
According to Camarinha-Matos (2004), the cooperation of independent organizations is
supported by computer networks and partners in a VOs should collaborate in order to
achieve business opportunities. Trust among them and operation according to common
agreements are essential for collaborating. Although in some occasions additional repu-
tation systems for identifying trustworthy members might be required.
Malone and Crowston (1994) extensively analyze coordination in different disciplines to
achieve a better insight and understanding when building cooperative tools. In a for-
mer work, Malone (1990) combines ideas from artificial intelligence and organization
2.2. DISTRIBUTED RESOURCES 33
theory to route information within organizations. More precisely, a global blackboard is
proposed as interaction module, which adapts to different communication patters in a
flexible way, as entities can communicate with one another.
In this context, resource discovery (Iamnitchi and Foster, 2001; Hillenbrand and Müller,
2005) is a considered an important requirement for coordination (Harchol-Balter et al.,
1999). Resource discovery is the previous step before interactions could be conducted
in a direct way, such as two members adopting the client/server model. A member
assuming the client role must first know who has a particular resource before starting
a direct request/negotiation with the member assuming the server role. Therefore, it is
important to knowwhere the most suitable resources reside, since only once the potential
providers are found, the member acting as client can establish a direct communication
with the provider.
2.2 Distributed Resources
Distributed resources shared within a VO encompass computers, storage space, sensors,
data or applications. These resources belong to different entities, organizational admin-
istrations or users. Resources have different characteristics and can be characterized ac-
cording to different classes (Talia and Trunfio, 2004).
A resource class is a model for representing resources of the same type. The characteris-
tics of a resource class are specified by a set of attributes. Therefore, a specific resource
would be an instance of a resource class. For example, the characteristics of computing
resources could be defined by a class named “computing resource”. The attributes of this
class could be “OS name”, “CPU speed”, and “Total RAM”, and an instance would have
values on these attributes: “OS name=Linux”, “CPUspeed = 2800MHz” and “Total RAM
= 2048MB”.
Attributes are said to be static if their values do not change within an instance. For exam-
ple, the attribute “CPU speed” is static, -i.e. the processor speed remains the same-. On
the other hand, dynamic attributes are those which values change frequently over time.
Examples of dynamic attributes could be “CPUload”, “available memory” and “free disk
space”.
Resource instances are distributed in the Internet, and the process of finding a particular
instance is called resource discovery. This process is driven by special queries, which could
take the form of exact matching queries, range queries and general purpose queries.
Exact matching queries are used when a requester knows exactly the value the attribute
he is looking for must take. In a similar way, range queries are used when a requester
34 CHAPTER 2. BACKGROUND
specifies a range where the values must be. General purpose queries are used for less
restrictive lookups, for example in partial matching or semantic searches.
One of the mayor problems of highly distributed systems is supporting dynamic at-
tributes due to the traffic required to inform changes in resources (Talia and Trunfio,
2004). For this reason, instead of keeping track of changes, it might be better to ask
directly the resource provider on each query (on demand), since frequent updates are
expensive in terms of network traffic (Carzaniga et al., 2001; Raman et al., 1999).
When consumers keep interest in certain resources, queries take the form of subscrip-
tions. This behavior is generally found in distributed monitoring systems (Czajkowski
et al., 2001).
Resources and their characteristics should be considered when building a communica-
tion support for VOs, since collaboration usually involve members sharing resources.
Queries and messages in general need to be transmitted throughout a VOs. The next sec-
tion describes a basic group communication mechanism known as multicast. A message
sent to a multicast group, is received by each member of the group.
2.3 Multicast Communication
Basically, since VO members are distributed, an effective group communication mecha-
nism is required. In order to deliver messages characterized as one-to-many or many-to-
many, IP Multicast was defined (Eriksson, 1994; Diot et al., 2000), allowing every member
in a multicast group to receive a message addressed to it. Initially, IP multicast was hailed
as the approach to efficiently support traffic with many receivers, such as collaborative
applications supporting video or audio streaming (Parnes et al., 1997; Meissner et al.,
2001; Callahan et al., 1997; Fisher, 2002). IP Multicast uses the network infrastructure
efficiently by requiring the source to send a packet only once, even if it needs to be deliv-
ered to a large number of receivers (Deering and Cheriton, 1990). In order to support IP
multicast in the Internet, the MBONE (Multicast Backbone) has been built. The MBONE
is a virtual network extended across the Internet that supports IP multicast traffic (Eriks-
son, 1994). Despite the need for multicast services, the MBONE (Diot et al., 2000) does
not reach all Internet users, since neither all routers nor ISPs support it.
Several alternatives to the MBONE have been developed to support multicast communi-
cations. Overcast (Jannotti et al., 2000) and Scattercast (Chawathe, 2003) provide support
for data streaming, such as video and audio broadcast by disseminating servers strate-
gically across the Internet, aiming to maximize the throughput for groups with a single
sender. REUNITE (Stoica et al., 2000) uses recursive unicast trees to implement multicast
services. REUNITE can be incrementally deployed, as it works even if only a subset of
2.3. MULTICAST COMMUNICATION 35
the routers implements it. These alternatives have been designed to provide data broad-
cast services, such as audio and video streaming. As a consequence, they are best suited
for communications with a single sender and multiple receivers. Moreover, they depend
on special servers spread across the Internet leading to deployability problems.
P
h
y
s
i
c
a
l

n
e
t
w
o
r
k
O
v
e
r
l
a
y

n
e
t
w
o
r
k
overlay nodes
physical nodes
Figure 2.1: Overlay Network
An alternative for multicast communications that do not require special routers are over-
lay networks (Pendarakis et al., 2001; Chu et al., 2002). An overlay network is a computer
network which is built on top of another network. As Figure 2.1 depicts, an overlay
network could be seen as a plane on top of a physical network. Each overlay node is re-
lated to a real physical node, and links between two overlay nodes could require several
physical network links, as traverse physical nodes.
Therefore, instead of being supported at the network level, overlay networks are sup-
ported by user-level applications relying only on unicast as the subjacent transport. In
this way, neither special routers nor ISP involvement is required, avoiding the deploy-
ment problems IP multicast encountered.
Several alternatives have been proposed to provide group communication services on
the Internet through overlay networks. For example, An Application Level Multicast
Infrastructure (ALMI) (Pendarakis et al., 2001) creates a Minimum Spanning Tree (MST)
as an overlay structure among the hosts of a group. In order to build the MST, latency
measures between hosts are taken. The main drawback of this approach is that it de-
pends on a centralized component to generate and maintain the MST, limiting scala-
bility and robustness. In addition, ALMI is restricted to small groups, since the over-
head required for generating the MST increases exponentially with the size of the group.
NARADA (Chu et al., 2002) improves ALMI by achieving decentralization, though it is
still restricted to small groups. NICE (Banerjee et al., 2002) and LARK (Kandula et al.,
2003) reduce NARADA protocol overhead in order to improve scalability, but still focus
36 CHAPTER 2. BACKGROUND
on data streaming and suffer from long failure recovery delays.
Based on overlay networks, peer to peer systems emerged as an alternative to the pre-
dominant client/server paradigm. The next section introduces these kind of systems.
2.4 Peer-to-peer Systems
In contrast to a client/server based system, P2P systems allow participants to communi-
cate with one another and share their resources. In a typical client/server model, clients
only interact with a central server. However, a solution involving a central server to
support VOs has two main drawbacks:
• Since no entity is (or should be) in charge of regulating a VO, having a central-
ized server often makes participants feel like they relinquish control or privacy over
their actions (even when privacy is not really compromised). Accordingly, different
problems arise, such as who is responsible for the server administration and how
deployment costs should be afforded without jeopardizing the non-hierarchical na-
ture of the collaboration.
• From a technological perspective, the deployment of a central server requires spe-
cial considerations in terms of robustness and availability, as a single server could
be seen a as performance bottleneck and single point of failure, respectively.
P2P systems based on overlay networks (Rowstron and Druschel, 2001a; Ripeanu et al.,
2002; Stoica et al., 2000, 2001; Ratnasamy et al., 2001) are an attracting solution to re-
searchers due to their scalability, robustness and self-organization features, which are
also desired for the construction of VOs. P2P systems, similar to the concept of VOs, also
facilitate the dynamic creation of cooperating peers that have a common set of goals and
are governed by a set of policies.
Analogous to a telephone conversation, P2P was described as the point to point connec-
tion between two equal participants (peers). The Internet started as a P2P system, where
the firsts hosts in the ARPANET were US universities, and the connection between these
already independent computing sites was not in a master/slave or client/server fashion,
but rather as equal peers.
From the late 1960s until 1994 the connectivity model of the Internet assumed that ma-
chines were always on and connected, and they were assigned a permanent IP address.
A change in the IP address was unusual, and could take days until the system was up-
dated. Thereafter, a new model emerged, where users with dial-up modems connected
to the Internet. Therefore, as a growing number of users entered and leaved the network
2.4. PEER-TO-PEER SYSTEMS 37
unpredictably, ISPs started to run out of IP addresses and began to assign IP addresses
dynamically for each session. This new dynamic behavior prevented PCs from having
permanent DNS entries and therefore from hosting data or network based applications.
Although treating PCs as clients worked well, recently, as Internet connections became
more stable and hardware and software improved, making use of available resources
behind previous second-class connectivity PCs began to be taken seriously. However,
new difficulties were found when implementing these kind of distributed systems, such
as scalability, reliability and interoperability (Taylor, 2004).
P2P systems emerged as an attracting solution to these problems. According to Oram
(2001), a P2P system is a self-organizing system of equal, autonomous entities (peers)
which aims for the shared usage of distributed resources in a networked environment
avoiding central services.
Accordingly, P2P systems seems to be suitable to support autonomous VOs in the Inter-
net, and a convergence between P2P systems and the Grid in the near future is suggested
in several works (Foster and Iamnitchi, 2003; Mauthe and Heckmann, 2005; Iamnitchi
and Talia, 2005; Talia and Trunfio, 2003; Pierre, 2007).
In a similar fashion, lately, there has been a shift of paradigm in the Internet communi-
cations: from coordination to cooperation, from centralization to decentralization, and
from control to incentives (Steinmetz and Wehrle, 2005). P2P systems clearly lead this
shift, where, for example, a fair balance between give and take among peers should exist.
Some of the P2P features in comparison to a typical client/server solution are presented
in Table 2.1. P2P systems, according how they build the overlay network, are catego-
rized into structured and unstructured. Ad-hoc networks, such as Gnutella (Ripeanu
et al., 2002), are considered unstructured. Overlay links are established arbitrarily, thus
messages are either flooded, inefficiently spreading a large number of messages through
the group, or propagated, using techniques such as random walk to improve efficiency,
though at the expense of not reaching all group members (Gkantsidis et al., 2006).
On the other hand, structured networks are usually based on DHTs to build overlay
networks in specific topologies (Dabek et al., 2001; Kubiatowicz et al., 2000). In these ap-
proaches, resources previously defined by some attributes are indexed to avoid flooding,
facilitating the latter retrieval. This strategy is sound for sharing files or easily index-
able data. However, for more general, rich and complex data, these alternatives are not
suitable. P2P systems are further discussed in Section 3.3.
Based on P2P technologies, some alternatives to support collaborative applications
such as Groove (Eikemeier and Lechner, 2004), COMTELLA (Vassileva, 2004) and
Edutella (Nejdl et al., 2002) have been developed. In the Internet, groupware technol-
ogy share similar VO requirements. Groups are usually formed as temporary coali-
38 CHAPTER 2. BACKGROUND
Client/Server Peer-to-peer
Asymmetric: client and servers carry out
different tasks
Symmetric: each node carries out the same
tasks; No distinction between Client and
Server
Global knowledge: servers have a global view Local knowledge: nodes only know a small set
of other nodes
Centralization: communications and
management are centralized
Decentralization: no global knowledge, only
local interactions
Single point of failure: a server failure brings
down the whole system
Robustness: several nodes may fail without
jeopardizing the whole system
Limited scalability: servers easily overloaded
(bottleneck)
High scalability: load is distributed among
members
Expensive: server storage and bandwidth
capacity are required
Low-cost: storage and bandwidth are
contributed by members
Table 2.1: Client/server vs P2P paradigm
tions of independent users and these users are generally homogeneous, thus, the equal
standing rule would also apply. For example, single users sharing a brainstorming ses-
sion (Munkvold et al., 2006) or autonomous personal agents exchanging profile infor-
mation (Chen et al., 2000). Groove (Eikemeier and Lechner, 2004) is a P2P groupware
tool that provides extensive support for collaboration. Resource sharing is supported
through workspaces, where data, documents and messages can be stored. Although this
mechanism enables users to work offline and automatically synchronize changes upon
reconnection, workspaces are based on special relay servers, acting as a single point of
failure (Divac-Krnic and Ackermann, 2005). COMTELLA is a P2P collaborative applica-
tion based on Gnutella (Ripeanu et al., 2002) that enables students to share and search
for relevant articles, summarize and rate them. Edutella, also based on Gnutella, is an
open source P2P application for searching shared resources by their semantic metadata
expressed in Resource Description Format (RDF).
In general, P2P systems are mostly based on loose cooperation, thus they do not require
complex models of collaboration. Users share resources in an asynchronous way, and
P2P systems only provide the search and match-making mechanisms. Therefore, a large
proportion of P2P collaborative systems are usually enhanced file sharing applications,
adding metadata or other kinds of extended search capabilities.
One very important effort towards providing support for VO-like communications is
JXTA (Gong, 2001; Traversat et al., 2002). JXTA is a de-facto standard programming
framework , designed to be the basis of P2P applications. This platform provides many
useful software and hardware independent abstractions for simplifying network pro-
2.5. INTERNET ENVIRONMENT 39
gramming. One drawback of JXTA is that it relies on IP multicast for achieving effi-
cient group communication. As a consequence, it is severely limited by IP multicast low
adoption on the Internet. In networks where IP multicast is not supported, JXTA relies
on unicast, compromising performance and increasing protocol overhead. In addition,
JXTA may use special rendezvous peers to provide an application level routing mecha-
nism based on loosely consistent DHTs that allows the propagation of messages pushed
by peers. However, this mechanismstarts having peer-visibility problems in groups with
more than 45 rendezvous (Antoniu et al., 2007). In Section 3.3.5 JXTAis further discussed.
Similarly to JXTA, JGroups is a flexible toolkit for group communication developed in
Java (Ban, 1998). JGroups can use several transport protocols such as unicast or multi-
cast. For reaching group members that do not support IP multicast, JGroups relies on
a centralized component called GossipRouter that acts as a message forwarder in a star
topology. Clearly, this approach not only compromises scalability and robustness
1
, re-
quiring a server with an expensive high-speed Internet connection, but also compromises
the non-hierarchical nature of the collaboration as explained earlier in Section 1.1.
2.5 Internet Environment
Building a VO in the Internet, where every autonomous peer collaborates in a fair way,
is far from being an easy task. Internet is inherently a complex and heterogeneous envi-
ronment. For a message to reach its destination, it must first traverse a path of different
routers, gateways, firewalls, NATs, etc., each of which might have or impose restrictions.
The presence of these components entails making special considerations when building
an overlay network. This section describes the connectivity restrictions a host behind a
firewall or a NAT might encounter.
2.5.1 Firewalls
A firewall system is used to prevent unauthorized access to a private network. All net-
work traffic to/from the Internet from/to the local network must pass through the fire-
wall. Although firewalls enable different kind of configurations, they are generally con-
figured to block traffic from the outside (Internet) to the inside (private network), but to
allow users from inside to communicate freely with the outside, as the arrows in Fig-
ure 2.2 suggests. Firewalls are also important to provide logging and auditing functions
to network administrators, letting them know the amount of traffic passed though it and
the attempts there were to break into it. In P2P systems, peers often have to traverse
firewalls to communicate with one another.
1
http://www.javagroups.com/javagroupsnew/docs/manual/html/user-advanced.html#d0e2702
40 CHAPTER 2. BACKGROUND
Internet
LAN
Firewall
Figure 2.2: Firewall
2.5.2 Network Address Translators
Anetwork address translator systemallows multiple hosts on a private network to access
the Internet (public network) using a single public IP address. As Figure 2.3 suggests, this
is done by a special device, such as a router, which maps private IP addresses from the
local network to a unique public IP address. All traffic appears to outside parties as
if it was originated from the NAT device. This also means that the private network is
masqueraded, making their members not reachable from the outside.
LAN
Internet
NAT
Public IP address
Private IP addresses
Figure 2.3: Network address translator
Most NATs allow the network administrator to configure translation tables entries, also
known as static NAT or port-forwarding, enabling traffic from the outside to reach the
designated hosts in the masqueraded network. However, this requires the intervention
of the network administrators which, alleging security issues, are usually reluctant to
enable this feature.
2.6. CONCLUSIONS 41
2.6 Conclusions
Supporting VO for equal standing members is not an easy task. Within a VO, members
should collaborate in order to make an efficient use of distributed resources. The com-
munication middleware should consider the characteristics of VO members and their re-
sources. Resources can be static or dynamic. Managing information of dynamic resources
is costly since they require to be updated frequently. For these cases it might be better to
ask directly the resource provider each time a query comes along. Moreover, members
could enter and leave the VO unexpectedly, adding more dynamism to the equation.
In order to interact, members should rely on a group communication mechanism that
enables them sending messages to every VO member. As IP multicast did not succeed
in the Internet, several alternatives were proposed to deliver this service. Most of the
alternatives build an overlay network -a virtual network over the physical one- that ease
the deployment of the communication protocols by implementing themat the application
level. Based on overlay networks, peer to peer systems emerged as an alternative to
the predominant client/server paradigm. Now, instead of having a centralized server
delivering services and clients consuming them, peers are both consumers and providers,
interacting with each other in a collaborative fashion. The Internet heterogeneous and
complex characteristics should also be considered when building a support for VOs, since
communication between peers must traverse several hops and deal with connectivity
restrictions imposed by firewalls and NATs.
The next chapter presents the VO communication requirements, and describes in more
detail some of the most relevant ALM alternatives and P2P systems. Further analysis
of overlay network topologies and their implications are also presented, since these ap-
proaches strongly rely on overlay networks to achieve their goals.
42 CHAPTER 2. BACKGROUND
Chapter 3
Related Work
When designing a communication middleware to support virtual organizations, different
aspects such as robustness, scalability and fairness should be considered.
In this chapter further details will be covered related to the existent communication al-
ternatives to support VOs. In order to have a better insight of the communication pos-
sibilities, first ALM alternatives are discussed. Next, the most relevant P2P approaches
will be described. Both ALM and P2P approaches strongly rely on overlay networks -a
virtual network built at the application level over the physical one- to provide their ser-
vices. These approaches are discussed and analyzed considering their overlay structure
and propagation schemes.
In the next section the communication requirements and characteristics of VOs are de-
scribed. In order to have a better insight of the different types of solutions overlay net-
works provide, Section 3.2 describes ALM alternatives, while in Section 3.3 P2P systems
are described. Subsequently, in Section 3.4 the characteristics and features of the different
overlay topologies are analyzed and discussed. Finally, Section 5.5 presents the chapter
conclusions.
3.1 Virtual Organizations Communication Requirements
There are several requirements and characteristics that need to be considered when de-
signing a communication support for VOs. As already mentioned in Section 1.1, equal
standing members should collaborate without a central control/authority, considering
the non-hierarchical nature of the collaboration. Due to the dynamism present in the In-
ternet, members should be able to build an maintain a VOin an effective and inexpensive
way. In this context, resources should be shared in the less restrictive possible manner,
i.e. general purpose queries should be allowed.
43
44 CHAPTER 3. RELATED WORK
According to Iamnitchi and Talia (2005), the following environmental aspects should be
considered when designing a solution:
• Resource information, distribution and density: Some members share an important
number of resources, while others just a few. In addition, some resources shared
are commonly found such as PCs running Linux, whereas other resources are scarce
such as specific services or data.
• Dynamism: Some resources change their value frequently (e.g. CPU load, or avail-
able memory) while others remain the same for such a long time that could be
considered static, such as CPU speed, operating system version, etc.
Dynamism is also present in the nodes or members that conforms the VO, as they
can join and leave unexpectedly. This incorporates an additional dynamism factor
that prevents even static resources from being available from time to time.
• Request popularity distribution: The request popularity for certain resources follow
different distribution models. For example, studies have shown that HyperText
Transfer Protocol (HTTP) requests follow a Zipf distribution (Breslau et al., 1999),
while for scientific collaboration, Iamnitchi and Ripeanu (2003) suggest that re-
quests are closer to the uniform distribution.
• Peer participation: Some members participate in a VO for longer periods of time
than others. This time varies depending on the type of collaboration held in the
VO. Ideally, peers joining a VO should collaborate for relatively long periods of
time and not just to perform a specific task and then leave.
• Failure rate: When a node fails, it also stops taking part in the overlay maintenance
and construction. The actions that could be triggered to reestablish the overlay
should support the expected failure rates. Also a failure in a node could be seen as
a special case of resource dynamism, since the resources the node provided would
be no longer available.
Each VO might have a different configuration of these aspects, which should be consid-
ered when designing an overlay network for their support. However, there is a general
consensus in providing flexibility and supporting dynamism in both, participants and
resources (Desanctis and Monge, 1999).
ALM and P2P systems strongly rely on overlay networks to provide group communica-
tion mechanisms and collaboration or sharing environments respectively. ALM alterna-
tives try to provide an effective way of disseminating messages to a group, distinguish-
ing only membership. On the other hand P2P systems enable resource sharing by letting
their members to find the peers that provide the desired resources.
3.2. APPLICATION LEVEL MULTICAST 45
The number of members for ALM alternatives and P2P systems handles differs consider-
ably in size. In ALMalternatives, when multiple sources must be supported, it is unlikely
that the number of members reaches a hundred. On the other hand, P2P systems are able
to scale to millions of members due to the fact that generally only small-sized queries are
sporadically transmitted in these systems. VOs stand somewhere in between. Accord-
ingly, the expected size of VOs usually ranges from less than a hundred to thousands of
members, considering the collaborations and activities held in each kind of VO.
3.2 Application Level Multicast
Due to the problems that IP Multicast presents (Diot et al., 2000), specially its low adop-
tion, and the increasingly number of applications requiring multicast services on the In-
ternet, several alternatives have been proposed. Some of the most relevant approaches
are:
• Overcast (Jannotti et al., 2000) provides support for diffusion of information, such
as video and audio streaming, for a single sender, using special servers across the
network.
• YOID (Francis, 2000) and HMTP (Host Multicast Tree Protocol) (Zhang et al., 2006)
build distribution trees in order to join IP multicast islands. In YOID members
must discover and select a parent to join the group. In HMTP, UDP tunnels are
used between designated members to connect IP multicast islands.
• Scattercast (Chawathe, 2003) is also broadcast oriented, but based on special
application-level proxies called SCXs spread across the Internet.
• REUNITE (Stoica et al., 2000) uses recursive unicast trees to implement multicast
services. REUNITE uses special routing tables in its protocol. One drawback of
REUNITE is its reliance on special routers, thus it is very difficult and expensive
to deploy. However, REUNITE can be incrementally deployed in the sense that it
works even if only a subset of the routers implement it.
• ALMI (Pendarakis et al., 2001) creates a MST as an overlay structure among the
hosts forming a group. In order to build the MST, latency measures between hosts
are taken. The main drawback of this approach is that it depends on a centralized
component to generate and maintain the MST. In addition, ALMI is restricted to
small groups, since the cost of generating the MST increases exponentially as the
size of the group grows.
46 CHAPTER 3. RELATED WORK
• NARADA (Chu et al., 2002) improves ALMI by achieving decentralization, though
it is still restricted to small groups (less than 200) due to its exponential protocol
overhead costs.
• NICE (Banerjee et al., 2002) and LARK (Kandula et al., 2003) reduce NARADA pro-
tocol overhead in order to achieve scalability, but still focus on data streaming and
suffers from long failure recovery delays.
Next, considering the VO requirements, NARADA, Nice and Lark ALM approaches are
described in detail, since, besides admiting multiple senders, they do not require deploy-
ing special servers across the network.
3.2.1 NARADA
Source
Figure 3.1: Narada source MST
Narada (Chu et al., 2002) is an application level multicast system designed for confer-
encing applications, allowing all members to send and receive data at the same time.
To join the the overlay, a host first obtains a list of already connected hosts from a ren-
dezvous point and randomly connects to a few of them. An mesh overlay is constructed
by periodically adding and dropping connections between hosts. Hosts exchange control
messages with their neighbors in order to maintain connectivity and build a routing table
with the shortest widest path algorithm, considering bandwidth and latency as primary
and secondary metrics respectively. Host arrival and departure information is also prop-
agated through these control message exchanges, eventually reaching all the connected
hosts. On top of the mesh, Narada constructs a per-source (reverse) shortest path span-
ning tree for data delivery. A periodic mesh refinement algorithm is used to improve the
3.2. APPLICATION LEVEL MULTICAST 47
overlay by adding or dropping connections. A host probes if by adding a connection the
overall delay can be reduced. In case a certain reduction threshold is reached, the new
connection is included. In a similar manner, edges are dropped if a link cost if above
a threshold, obtained from the times the link is included in the shortest paths. Control
message exchanges are also used to detect problems in links, which are dropped in case
they can not be reestablished. Figure 3.1 shows a Narada tree example considering a
particular source. The tree is depcited by straight lines, while control messages between
hosts are depicted with dashed lines.
Narada has serious scalability problems. Routing tables are as large as the group size
and since control messages contains the states of all hosts in the group, the total control
overhead in the system is of the order of O(n).
3.2.2 NICE
Clusters
layer 0
layer 1
layer 2
Cluster leaders
Layers
Figure 3.2: Nice
Nice (Banerjee et al., 2002) aims to provide a scalable solution for low-bandwidth data
streaming. The idea is to reduce the overhead building a multilayer hierarchical control
topology. As Figure 3.2 shows, members are grouped in clusters that are organized in a
multilayer topology. The lowest layer (layer 0) includes all the participants. Layers are
divided into a set of clusters, each cluster of size k and 3k − 1, with k a constant related
to the number of members close to each other. Latency between members is used as the
distance metric for organizing the overlay. Each cluster has a leader which is determined
by the minimum maximum distance to all other peers in the cluster. To join a group,
a new member is bootstrapped to the highest layer in the hierarchy by a rendezvous.
In each layer, the joining host contacts the members and selects the one which is closer
in terms of latency. This member informs the new node about the members from the
layer below, and then the process is iterated. Once the node reaches the lowest layer it is
connected to its corresponding cluster.
48 CHAPTER 3. RELATED WORK
To send a message to the group, a member sends the message to all its peers in the cluster.
The message is propagated only to the members of a different cluster. Therefore, cluster
leaders are responsible for propagating the messages between the different layers.
Members periodically probe members in its supercluster (defined as the leader cluster
in the above layer) to see if they belong to the correct cluster. Cluster leaders are also
responsible for splitting and merging clusters to satisfy cluster size limits. The control
overhead in Nice is O(klog
k
N), and the farthest path between any pair of nodes (the
number of application level hops) is O(log
k
N).
On failures, nodes will tend to reconnect to the previous cluster. However, this behav-
ior is often criticized, since nodes reconnects around the failure origin, e.g. the cluster
leader (Kandula et al., 2003).
3.2.3 LARK
Lark Kandula et al. (2003) is a light-weigh application level multicast protocol designed
to achieve scalability. To achieve this, Lark is organized as small groups of nodes called
cliques. Nodes in a clique exchange control messages with each other. In order to join a
group, Lark assumes that already connected nodes can be found by a sort of rendezvous
or bootstrap. The joining host randomly issues a join request to an already connected
node. Dn, Dc and Dp are general constraints that bounds the per-node, per-clique degree
and the number of peers each node keeps, respectively. Each node knows its current dn
and dc values, which can not exceed the global Dn and Dc constraints. If a join request
violates these limits -e.g. the clique is full-, the requested node can slice off the clique and
create a new one, including the requesting node in it. Thereafter, the new node acts as
a special bridge node between the old clique and the new one, keeping the inter-clique
connectivity.
1
2
3
13 14
4
5
6
7
8
15
9
10
11
12
Source
Figure 3.3: Lark propagation example
3.3. PEER TO PEER 49
To send a group message, as Figure 3.3 shows, the source node propagates the message
to each clique member and bridge nodes propagate it through the different cliques.
Conversely, control messages are propagated only intra-clique. Failure in nodes may re-
quire a clique reconfiguration. In addition, bridge node failures can partition the overlay.
To tackle this problem, nodes keep information about their entry node (parent node) and
random nodes in other cliques. Therefore, each node in Lark holds O(Dn + Dp) state
information. In case the parent node fails, nodes try to reconnect to the same cliques they
were connected to.
3.2.4 Discussion
Each one of the described alternatives offer multicast support for different communica-
tion requirements, based on the specific nature of the multicast communications they
intend to support. Overcast and Scattercast have been created with the objective of pro-
viding broadcast services, such as audio and video streaming. These approaches are
suited for communications with a single sender and multiple receivers. On the other
hand, ALMI and NARADA allow every group member to send data.
YOID and HMTP achieve a logarithmic scaling behavior. However, they are intricate
and inefficient. For example YOID employs expensive and complex techniques for loop
detection and avoidance. In HMTP nodes are constantly looking for a better parent,
producing a considerable overhead.
REUNITE, Overcast and Scattercast are not well suited for multiple senders. In addi-
tion, although they can be incrementally adopted, they depend on specific routers spread
across the Internet, sharing some of the problematic characteristics of the MBONE.
Though ALMI and NARADA implement diffusion groups by creating overlay networks,
both still suffer scalability problems restricting them to small groups.
NICE and LARK achieve better scalability. However, in case of failures, Nice incurs in
O(k
2
) messages to reestablish the cluster leader. Lark reduces this overhead, but still
suffer from high failure recovery delays. Moreover, both Nice and Lark overlooks the
fairness requirements, since cluster leaders in the case of Nice, and bridge nodes in the
case of Lark, have significant more responsibilities than normal nodes.
3.3 Peer to Peer
In contrast to ALM, which are used for group communication purposes, P2P systems
build overlay networks to enable resource sharing among its members. Therefore, the
50 CHAPTER 3. RELATED WORK
P2P system allows peers to find the desired resources, i.e. resource discovery. Once a
resource is located, the requesting peer can interact directly with the peer holding the
resource of particular interest.
P2P systems can be characterized by the decentralization level and the way the overlay
is structured to follow certain rules and propagation schemes (Mischke and Stiller, 2004):
• Decentralization level:
– Centralized: Participants connect to a centralized server that keeps an index of
all the available resources. Then, when the server receives a query from a host,
answers it by indicating which is the best suited node that matches the query.
The best suited node might be the faster, the closer, the most available or the
less expensive. Once the best suited node is found, the host could initiate a
direct connection, without the intervention of the central server.
– Completely decentralized: Every node in the overlay assumes the same role.
Collaboration takes place without requiring any central coordination.
– Partially centralized: Some nodes, besides assuming their usual role, act as
centralized registries or indexes of a group of nodes. These indexes are known
as super-peers or super-nodes, and are interconnected with each other forming
a new overlay layer. These P2P systems are also known as hybrid systems.
• Structure:
– Unstructured: Resources are not related with the overlay topology. Search-
ing mechanisms range from brute force methods such as flooding, to more
sophisticated strategies, such as random walk (Lv et al., 2002) or routing in-
dexes (Crespo and Garcia-Molina, 2002), to enhance scalability.
– Structured: In structured systems the overlay and the propagation rules re-
quired are strongly correlated, they emerged mainly to address the scalability
issues that unstructured systems were originally faced with (Androutsellis-
Theotokis and Spinellis, 2004). The overlay topology is built to provide a map-
ping between content and its location, allowing to efficiently route queries to
the nodes responsible for a desired content. Using mechanisms such as DHTs,
these systems offer a scalable solution for exact-match queries, i.e. when the
exact identifier of the requested resource is provided.
Some of the most relevant P2P approaches from the structure and propagation scheme
perspective are described next.
3.3. PEER TO PEER 51
3.3.1 Napster
Napster (Eberspächer and Schollmeier, 2005) is based on a central index to allow users to
share files in the Internet. Although this index is deployed as a central server, machines
connected to it were not characterized as clients, but as equal peer machines that offer
and consume resources.
Napster central index
direct connections once discovered
Figure 3.4: Napster
As Figure 3.4 depicts, every member is connected to the server, which stores informa-
tion about the shared resources. The server answers keyword-based queries, indicating
which peer is sharing a specific content. Subsequently, the transfer is done between the
requesting node and the provider, without the intervention of the central server.
Although the central server only keeps updated information about resources shared by
the peers and answers queries about previously indexed information, the load grows lin-
early with the number of participants, making Napster unscalable. In addition, although
resources shared are static, indexed information must be updated since peers may join
and leave the system unpredictably.
Each node only has to keep a connection with the central server. If the central server fails,
peers would be not be capable to find the resources desired and hence unable cooperate,
constituting a single point of failure. In the special case of Napter, as it was used to
share illegal content, the single point of failure was made first apparent for legal/political
reasons rather than from technical matters.
3.3.2 GNUTELLA
Gnutella (Ripeanu et al., 2002) is a completely decentralized and unstructured system
that builds an ad-hoc overlay network. Neither central authority nor central coordination
are required. Each peer is connected with a small number of peers in a random ad-hoc
52 CHAPTER 3. RELATED WORK
Figure 3.5: Gnutella
basis. Messages are propagated with a flooding mechanism, this is, each peer forwards
messages to all its neighbors, which in turn, do the same.
If a host wants to join the Gnutella network, it must contact an already connected node.
Initially, to find nodes in the overlay, users tried to connect to Gnutella nodes published
in IRCs (Internet Relay Chat) or in Web pages until one worked. In a revised version
of the Gnutella protocol, a cache was used as a server that helped to find nodes in the
Gnutella network. Once a node in known, joining nodes discover new peers issuing a
special query and awaiting for connection invitations.
The application level protocol of Gnutella provides four types of messages: ping, pong,
query and query hits. Once a node is connected, it spreads a ping message to announce
itself. The nodes that receive the ping message reply with a pong message to identify
themselves and also propagate the ping message to their neighbors. Both ping and query
messages are flooded through the overlay limiting the number of hosts a message could
traverse with a Time To Live (TTL) parameter. When the TTL parameter reaches zero, the
flooding mechanism stops forwarding the message/query. The flooding mechanism is
implemented by assigning each message a unique identifier and managing at each host
a dynamic routing table of message identifiers and node addresses used to avoid loops
and drop duplicates. Queries are answered with Query hits messages. To answer these
messages, since the response messages contain the same IDas the original messages, each
host checks its routing table to determine along which link the response message should
be replied.
Each Gnutella node stores information about a small number of neighbors. Therefore,
as no information about resources is indexed or stored in DHTs, maintenance costs are
reduced. General purpose queries and dynamic resources are well supported since each
resource provider is asked on demand. Nevertheless, the flooding propagation mecha-
nism introduces high network bandwidth consumption (Aberer et al., 2005). Efficiency
could be improved by using complex techniques such as randomwalk, but at the expense
of not reaching all group members (Gkantsidis et al., 2006).
3.3. PEER TO PEER 53
3.3.3 Chord
Chord (Stoica et al., 2001, 2003) is a structured and decentralized approach that organizes
resources in a ring-like overlay network topology. This is achieved by storing key/value
pairs for distributed resources in order to map content to a specific node, allowing for-
warding look-ups queries directly. Chord is based on DHTs, with m-bit key identifiers
assigned to each node, forming an unidimensional virtual ring.
N8 Finger Table:
Idx ID Successor
0 N8+1 N10
1 N8+2 N10
2 N8+4 N15
3 N8+8 N18
4 N8+16 N24
5 N8+32 N43
N8
N10
N15
N18
N24
N43
N48
N57
N1
N20
N35
N40
Figure 3.6: Chord
Nodes and data items are associated with an identifier referred as id and key respectively.
Each node in the ring is responsible for managing a key space, allowing to map the re-
sources whose key matches the node id. Each node is connected to its successor and
predecessor in the ring, and holds a finger table as shown in Figure 3.6. Using this table,
queries are propagated according to their value, allowing to reach the desired node in a
logarithmic number of steps. Each entry i, in the finger table of node n, contains the id of
the successor of the key n +2
i−1
, 1 ≤ i < m, being m the size of the key space. Therefore,
each node maintains a routing table with approximately logN entries, allowing look-ups
to take place in O(logN) steps. In fact, Chord is similar to a binary search, where the
searching space is reduced by half after a search/routing-hop.
If a node n wants to join the ring, it first must know a connected node already partic-
ipating in the system. The new node n determines an identifier which is used later to
know its position in the ring. By querying the already known connected node, the new
node n can determine its position in the ring and notify the corresponding successor. The
finger table is built by querying iteratively the known node for n +2
1
, n +2
2
, n +2
3
, etc.
However, as finger tables on other nodes are not updated, Chord periodically performs
maintenance and stabilization tasks, which also helps keeping the consistency of the fin-
ger tables on node departures. Since finger table updates are costly, a lazy update policy
is usually adopted.
It worth noticing that the Chord DHT mechanism, based on an unidimentional ring for
54 CHAPTER 3. RELATED WORK
lookups, does not work for general purpose requests, but only for exact matching queries,
limiting its applicability.
3.3.4 CAN
CAN (Ratnasamy et al., 2001) (Content Addressable Network) is structured as a virtual
D-dimensional Cartesian coordinate space. Each node is assigned a space coordinate. A
CAN node keeps a table with the adjacent neighbors of their space coordinate. Similarly
to Chord, CAN uses a hash function to map a key to a point in the D-dimensional space.
In this way, it is possible to forward a query to a specific zone in the space.
N2
N1
N3 N4
N2
N3
N4
N5
N1
Figure 3.7: CAN: Bi-dimensional space with 5 nodes
To join the network, a bootstrap is used to let the new node to contact an already con-
nected node, next, the new node picks a random point in the space an contacts the node
that matches the zone of the point. Then, the contected node zone is partitioned in two,
making the new node responsible for one of the partitions. This could be the case of N3
in Figure 3.7, which assigns half of its space to the new node N5.
Periodically, to refresh lost entries as well as to avoid stale ones, nodes insert (key, value)
messages into the network. Additionally, each node sends periodic update messages to
each of its neighbors providing them with the its zone coordinate and a list of itsneigh-
bors and their zone coordinates. With the aid of these messages, when a node fails, a re-
covery mechanism is triggered to let the failing node neighbors take over the coordinate
zone. However, these messages also introduce a considerable overhead to the network.
With this strategy, an average path length of O(
d

N) is achieved, while each node keeps
information of 2D neighbors independently of the nodes participating in the whole
system. Increasing the number of dimensions reduces the propagation time, allowing
multiple-attribute lookups. However, this also entails keeping information about a larger
number of neighbors, increasing maintenance costs.
3.3. PEER TO PEER 55
3.3.5 JXTA
JXTA (Traversat et al., 2002) is an hybrid P2P approach. The main goal of JXTA is to
define a generic P2P overlay, which may be used to implement a wide variety of P2P
applications and services. Peers in a JXTAnetwork are self-organized in a set of peers that
share common interests called peergroups. JXTA hybrid overlay network is based on edge
and rendezvous peers. Rendezvous peers index edge peers advertisements to facilitate the
discovery of resources in a peergroup. A peergroup can have as many rendezvous peers
as required to support the peergroup size. Rendevous peers are organized into a loosely-
coupled network to reduce maintenance costs. As Figure 3.8 shows, this hierarchy allows
to propagate queries only between rendezvous peers, significantly reducing the amount of
peers that need to be searched.
Rvx
Px Edge Peer:
P1
P2
Rv1
Rv2
Rv3
Rv5
Rv4
Rv6
Advertisement
Rendezvous Peer:
Figure 3.8: JXTA
Similarly to Gnutella, each rendezvous peer maintains a list of known rendezvous in the
peergroup. A flooding like technique is employed to disseminate rendezvous peer infor-
mation. This hybrid approach combines the use of a limited-range rendezvous walker in
a loosely-consistent DHT. The term loosely-consistent DHT is employed since rendezvous
peers are not required to maintain a consistent view of the distributed hash index, in
opposition to most DHT approaches, such as Chord, CAN, etc. This strategy has the
advantage of not requiring a strong-consistency DHT maintenance. However, in very
large peergroups the system may suffer considerably from the resulting inconsistency,
as reaching a destination becomes arduous, introducing significant performance prob-
lems (Darlagiannis, 2005).
3.3.6 JGroups
Similarly to JXTA, JGroups is a flexible toolkit for group communication developed in
Java (Ban, 1998). JGroups provides a protocol stack that enables processes to join a group,
56 CHAPTER 3. RELATED WORK
send messages to all group members or to a specific one. JGroups manages membership
keeping track of active members in each group. A group is identified by its name and
members can be located on the same host, within the same LAN, or across a WAN.
JGroups is suitable to support clusters within a fast local network. However, for reach-
ing group members in the Internet that do not support IP multicast, JGroups relies on
a centralized component called GossipRouter that acts as a message forwarder in a star
topology. Clearly, this approach not only compromises scalability and robustness
1
, but
also requires a server with an expensive high-speed Internet connection.
3.3.7 Others
Groove (Eikemeier and Lechner, 2004) is a P2P groupware tool that provides extensive
support for collaboration, including discussion boards, chat, calendar and whiteboards.
Resource sharing is supported through workspaces, where data, documents and messages
can be stored. This mechanism enables users to work offline and automatically synchro-
nize changes upon reconnection. However workspaces are based on special relay servers,
acting as a single point of failure (Divac-Krnic and Ackermann, 2005; Kurmanowytsch
et al., 2003). COMTELLA is a P2P collaborative application based on GNUTELLA (Ri-
peanu et al., 2002) that enables students to share and search for relevant articles, summa-
rize and rate them. Edutella, also based on GNUTELLA, is an open source P2P applica-
tion for searching shared resources by their semantic metadata expressed in the resource
description format (RDF).
In addition to the systems described, different P2P approaches were developed (Row-
stron and Druschel, 2001a; Zhao et al., 2004; Rowstron and Druschel, 2001b; May-
mounkov and Mazières, 2002; Manku et al., 2003; Malkhi et al., 2002). Most of these
approaches are either based on DHT, are similar, could be considered variants of the pre-
viously described systems, or are not of particular interest fromthe perspective described
in Sections 1.1 and 3.1, to the type of collaboration held in VOs.
3.3.8 Discussion
In general, P2P systems are mostly based on loose cooperation, thus they do not require
complex models of collaboration. Users share resources in an asynchronous way, and
P2P systems only provide the search and match-making mechanisms. Therefore, a large
proportion of P2P collaborative systems are usually enhanced file sharing applications,
adding metadata or other kinds of extended search capabilities.
1
http://www.javagroups.com/javagroupsnew/docs/manual/html/user-advanced.html#d0e2702
3.3. PEER TO PEER 57
Napster is based on a single central server, which constitutes a single point of failure
and performance bottleneck. Moreover, in order to keep scalability, the computational
power as well as the storage capacity must growproportionally with the number of users.
Gnutella adopts a completely decentralized approach, ensuring robustness. However
the flooding mechanism disseminates a large number of messages inefficiently. Enhance-
ments such as random walk (Gkantsidis et al., 2006), although improve efficiency, are
incomplete, since it might not reach all members (Lv et al., 2002).
To avoid this problems, structured P2P systems based on DHTs were developed relating
the overlay topology with the content. Each node in the overlay structure is responsible
for a range of keys, which can be searched in logarithmic times. Although some enhance-
ments improve efficiency for inexact search, such as limited range and multi-attribute
queries (Andrzejak and Xu, 2002; Cai et al., 2004; Mastroianni et al., 2005), this kind of
systems are usually restricted to share files or any other easily indexable content.
Another important aspect to consider is the amount of data each node of the overlay
must store. Structured approaches require to keep updated indexes, routing tables, and
failure recovery information, which also means more message exchanges and, therefore,
significant overhead. Here, dynamism plays an important role, since the increment of
both resource and membership dynamism would also imply more maintenance costs.
Table 3.1 summarizes the usual look-up strategies for supporting different search types
and resource dynamism.
Static Resources Dynamic Resources
Exact Matching DHT Flooding
Range queries DHT Flooding
General purpose Flooding Flooding
Table 3.1: Search strategies
DHTs are not suitable for general purpose queries, as these kind of queries could take
any form. In a similar way, resources that change very often, such as CPU load, would
require continuous updates of the DHTs, introducing significant overhead. For these
cases, flooding is usually adopted spreading queries to all possible providers. Hybrid
approaches, such as JXTA (Traversat et al., 2002) try to get a favorable trade-off between
structured and unstructured schemes. However they also share some of their problems:
incomplete peer visibility, which leads to inconsistencies. Besides, load is not fairly dis-
tributed, since some nodes have more responsibilities than others, which is a significant
drawback for VOs with equal standing members.
It is worth mentioning that P2P systems are often categorized as 1
st
, 2
nd
and 3
dr
genera-
tion systems in the literature. The first generation systems started with Napster central
58 CHAPTER 3. RELATED WORK
directory and peers offering and consuming resources. Second generation systems, such
as GNUTELLA solved the central coordination problem by achieving decentralization.
However, the flooding mechanisms introduces unnecessary network traffic, causing scal-
ability problems. An enhancement of the flooding method in 2
nd
generation systems was
the introduction of super-peers, which characterized hybrid overlays. 3
dr
generation P2P
systems were considered as an improvement of 2
nd
generation systems by structuring the
overlay in such a way that queries could be directed to the desired node. This is achieved
with DHTs that could match a specific resource query with a location or destination in
the overlay. As mentioned earlier, this strategy is only suitable for exact-matching queries
or some special case of range and multi-attribute queries. However, for general purpose
queries is not clear how DHT systems could be used, taking in consideration that in dy-
namic environments (i.e. when the indexed resources or the group conformation changes
frequently) this solution not suitable. Therefore, in this Thesis, since we want to focus in
supporting VOs, we do not adopt this evolutionary categorization.
3.4 Overlay Topology
Multicast alternatives as well as P2P systems rely on a overlay network to deliver their
services. When VOs are conformed in this way, i.e. hosts participating in the overlay are
considered VO members, the topology of the overlay network has a significant effect on
the characteristics and features the communication will have. Figure 3.9 shows some of
the typical P2P and ALM alternatives overlay topologies.
VOs require sending messages, queries and status information to the whole VO. Each
topology has particular characteristics and features that admits different dissemination
strategies and possibilities. Additionally, the overlay topology also allows different types
of cooperation and self-organization. Scalability and fairness are also features affected by
the overlay topology. Next, considering the overlay topology, some of these features and
characteristics are described:
• Centralized: A centralized approach (Figure 3.9(a)) requires a server, which consti-
tutes a single point of failure and could act as a performance bottleneck. This is
would be the case of Napster, where the server only provided a simple indexing
system. The server has n − 1 connections, whereas a peer node has only one link
(to the server). Each node must only have information about the server, whereas
the server knows every node or peer in the VO. Considering every peer as equal,
including the central server, this is the most unbalanced scheme. To send a mes-
sage to the whole VO, the result is similar to the sequential unicast solution, where
a node must send a copy of the message to each other node, however in this case
3.4. OVERLAY TOPOLOGY 59
c) MST
c) Mesh d) Ring
b) Ad−Hoc a) Centralized
f) Hybrid
Figure 3.9: Overlay Topologies
the server carry out the delivery (e.g. an IRC chat server). Although the end-to-end
latency might be good, as any message traverses two nodes at most, some of the
underling links near the server could be very stressed and could act as bottlenecks,
introducing significant delays in the delivery time.
• Ad-hoc: Ad-hoc networks (Figure 3.9(b)) are widely adopted due to their robust-
ness, simplicity and ease of deployment. The unstructured nature of these networks
simplifies the overlay construction and maintenace, as it is not required to keep in-
formation about the location of resources. In unstructured networks the node de-
grees follow the power-lawdistribution, this is, most nodes have a small number of
links and few nodes have a large number of links, specifically, in a power-law net-
work the fraction of nodes with L links is proportional to L
−k
, where k is a network
dependent constant Ripeanu et al. (2002).
Flooding is used to forward queries, this is, each peer tries to answer the queries
itself and also propagates them to all of its neighbors. In this way, general purpose
queries and dynamism are supported, since each provider try to answer queries
with the information of its current available resources. Although by using an un-
structured network, maintenance overheads are reduced, spreading queries with
strategies such as flooding introduces high network bandwidth consumption, lim-
iting scalability. Besides, reaching all VO members is not guaranteed, due to the
restraining action of the TTL parameter. Although the number of links per node
may be bounded, the power-law distribution overlooks the fairness requirement,
60 CHAPTER 3. RELATED WORK
as some nodes will be considerably more loaded than others.
• MSTs: Minimum spanning trees overlay networks (Figure 3.9(c)) try to maximize
the usage of the available network resources. As loops are avoided, the dissemina-
tion strategy is straight forward. However, there are some serious drawbacks that
hinders the achievement of a scalable solution to support VOs:
– in order to build the MST many measurements must be taken, generating a
significant amount of overhead.
– the dynamism of the environment (node arrivals and departures) would im-
ply either continuously triggering expensive reoptimization mechanisms or
keeping a suboptimal overlay tree.
– in MSTs, nodes with higher capacity are used as retransmitters. However,
network resources in VOs are shared among various applications and thence
cannot be used only for the overlay management. Moreover, using nodes as
retransmitters would also overlook the fairness requirement.
The number of links per node depends on the available bandwidth, or could be
bounded to a specific number. Depending whether the optimization is decentral-
ized or not, each node keeps track of several or even all nodes in the overlay, to
make measurements for optimization purposes. This, however, introduces a con-
siderable overhead which also increases with the number of nodes (e.g. exponen-
tially in the case of NARADA (Chu et al., 2002)).
• Ring: In a ring overlay, (Figure 3.9(d)) each node is linked to other two, a successor
and a predecessor. Chord (Stoica et al., 2003), although conceived as a ring, actu-
ally relies on special routing tables. For the lookup and dissemination mechanisms,
messages are not forwarded to the node successors and predecessors, but to the
nodes appearing in the finger table, reaching nodes farther in the overlay. There-
fore, in Chord each node knows a logarithmic number of nodes. Accordingly, the
ring topology is only used for organization purposes, to cover the space provided
by the hashing function that enables the localization of nodes. On the other hand, in
a pure ring overlay, to reach every node, a message would have to traverse half of
the nodes, making the distribution time increase lineally with the number of nodes.
• Mesh: In a mesh overlay (Figure 3.9(e)), each node is usually connected to a fixed
number of nodes. For instace in CAN (Ratnasamy et al., 2001), each node is con-
nected to other 2D nodes, depending on the dimension of a D-Cartesian space. The
idea is that every peer has the same responsibility in the overlay, achieving a decen-
tralized protocol. Generally, mesh overlays are based on DHT to propagate queries,
selecting the neighbors closer to the desired node or location. This also allows a
3.4. OVERLAY TOPOLOGY 61
more efficient broadcast strategy than flooding, forwarding messages in a more ef-
ficient way. However, to achieve this, stabilization mechanisms are required, which
consume network resources.
• Hybrid: Hybrid overlays (Figure 3.9(f)) are based on a two layered system. They
are generally constructed as superpeers connected by an ad-hoc network, for ex-
ample having several Napster servers connected with Gnutella. This mixed setting
provides the robustness of Gnutella and also scalability, since Gnutella is used only
to connect superpeers nodes and hence the flooding mechanism can be afforded.
However, hybrid overlays also have some drawbacks, as superpeers are mostly re-
sponsible for the whole coordination, the collaboration is unbalanced. Superpeers
are also considerably more loaded than normal peers, discordant with the fairness
participation requirement.
Approach Scalability Per Node
State
Robust-
ness
Topology
Aware-
ness
Fairness Maintenance
Over-
head
Max. ap-
plication
level hops
Failure
Recov-
ery
Type of
Overlay
Narada low O(N) medium yes low high O(logN) slow MST
over a
mesh
NICE high O(1) medium yes low medium O(log
k
N) slow Hierarchical
clusters
LARK high O(1) high no low low not
bounded
slow cliques
NAPSTER low O(N) low no low low O(1) none centralized
server
GNUTELLA medium O(1) high no medium low not
bounded
fast ad-hoc
Chord high O(2log
2
N) medium no high medium O(
1
2
log
2
N) medium ring
CAN high O(2D) medium no high medium O(
D
2
d

N) medium mesh
JXTA medium O(logN) high no low low O(
N
2
/logN) fast hybrid
Table 3.2: Properties of the different approaches
Table 3.2 summarizes the properties of the different approaches presented in this chapter.
The per node state value is the maximum amount of information about other nodes in
the overlay a node must keep. Robustness is considered lowwhen single points of failure
are present, and medium if a system keeps working, but requires a considerable time to
become completely functional again. The fairness value depends on the imbalance be-
tween node responsibilities. Maximum application level hops is the maximum number
62 CHAPTER 3. RELATED WORK
of nodes a message might have to traverse to reach the farthest node in the overlay. Fail-
ure recovery is slow/medium/fast taking into account the time required by the overlay
or stabilization protocol to recover.
3.5 Conclusions
An overlay network solution to support VOs of equal standing members should provide
robustness, scalability, fast and easy deployment, in a fully decentralized and inexpensive
way. Due to the dynamism and flexibility required to support this type of VOs, none of
the previous alternatives completely satisfies the mentioned requirements.
Group communication support was first provided by alternatives to the MBONE. How-
ever, these approaches, such as REUNITE, Overcast and Scattercast, were hard to adopt,
requiring specific routers spread across the Internet. Besides, they were intended for sin-
gle source media streaming. Based on overlay networks, ALM alternatives such as ALMI
and NARADA allow the whole communication to be managed by end hosts. Neverthe-
less, they are still restricted to small groups. NICE and Lark achieve greater scalability,
but still suffer from long failure recovery delays and overlook the fairness requirement.
Ad-hoc P2P approaches have problems reaching all group members efficiently and have a
considerable load imbalance. Structured approaches have high maintenance costs, which
further increase in very dynamic environments. Hybrid approaches are not completely
suitable for equal standing members, since some nodes have more responsibility than
others.
The way an overlay is built has a significant impact on the results and features of each
particular solution. MSTs or other optimized approaches tend to generate a consider-
able amount of overhead, often overlooking the fairness requirement. The collaboration
should be decentralized, without requiring the intervetion of special servers that could
jeopardize the nature of the collaboration, and even entail deployment costs, such as de-
ploying a high capacity server. Nonetheless, a rendezvous or bootstrap might still be
needed as a meeting point to reach or find already connected nodes.
The next chapter describes the proposed approach to support VOs in the Internet.
Chapter 4
GMAC
To effectively support VOs in the Internet, a communication middleware to allow the
collaboration of equal standing autonomous entities is needed. Overlay networks have
shown to be an effective way of communicating autonomous entities in the Internet with-
out relying on a central server. However, on one hand, ALM alternatives are usually fo-
cused in providing media streaming services, while P2P systems are mostly designed to
share files and resources. Although these approaches are based on overlay networks to
allow autonomous entities to establish and communicate in groups, neither of them was
designed specifically to support decentralized collaboration of equal standing members.
Therefore, when designing a communication mechanism for the collaboration of equal
standing members, special requirements should be taken into account.
The conformation and maintenance of a VO should be done in a fair way. The non-
hierarchical nature of the collaboration implies that no single entity should be in charge
of regulating the VO, since participants could feel that they relinquish control or privacy
over their actions. In addition, equal standing members are supposed to have similar
transmission probability, connection resources and communication demands. Therefore,
members should participate evenly in building and maintaining the VO.
Since information about the underlying network is not available from the application
perspective, it is very difficult to determine or take advantage of the underlying Internet
topology when building an overlay network (Kwon and Fahmy, 2002). Accordingly, tak-
ing measurements for determining the underlying physical network would necessarily
increase protocol overhead. Protocol overhead is network traffic that does not encom-
pass application related data, such as network resources used to build and maintain an
overlay network. Scalability is closely related to protocol overhead, since, in many cases,
protocol overhead increases significantly as the number of members grow.
A VO is established only as a collaboration environment for their members. These, in
63
64 CHAPTER 4. GMAC
turn, rely on small sporadic messages to coordinate their actions, such as VO announce-
ments, requests for either resources or general related VO information. Therefore, since
VOs do not have high bandwidth communication demands (i.e. they are not intended for
media streaming), it might be possible that optimization strategies could demand even
more network resources than the VO data messages themselves.
Hosts restricted by firewalls or NATs, which prevent them from being reachable by other
members in the Internet, should not be excluded from integrating a VO. In addition,
members should be able to build and maintain a VO without requiring neither mayor
configuration efforts, nor the previous deployment of special proxies/routers spread
across the Internet. Finally, failures in host or links should not jeopardize the whole VO.
Considering the lack of proper support for dynamic VOs of equal standing members
in the Internet, this Thesis proposes GMAC (Group Management Agent Cast), an over-
lay network middleware to support homogeneous VO communities. The main goal of
GMAC is to allow equal standing members to establish VOs in a scalable, robust and
easy deployable way.
The next section describes the aims and scope of the approach proposed in this The-
sis. Section 4.2 introduces the approach adopted by GMAC. In Section 4.3 the design
decisions about of the overlay structure are justified. Section 4.4 describes the supported
message types, while the whole distributed mechanisms for the overlay construction and
maintenance, including the joining and failure recovery strategies are explained in Sec-
tion 4.5. Finally the conclusions are presented in Section 4.6.
4.1 Aims and Scope
GMAC is a new approach to enable the constitution of dynamic VOs in the Internet.
The main goal of GMAC is to provide a fast and effective support for autonomous equal
standing entities in the Internet. These entities could be final users, software applications
or agents established in a VO. GMACprovides membership services and communication
primitives required for sending announcements, requests and obtaining summarized in-
formation from data distributed in the VO.
In GMAC, the VO construction and maintenance is realized in a decentralized and fair
way, not relying on special routers or proxies spread across the Internet or the deploy-
ment of a high capacity server.
GMAC considers a VO as the collaboration context where services for membership man-
agement and group communication primitives are provided, enabling to define common
collaboration policies, such as sharing rules, etc. Therefore, in contrast to ALM alter-
natives, which are focused on media streaming, VOs communication demands are not
4.2. GMAC MODEL 65
considered critical. Members are supposed to rely on small sporadic messages to col-
laborate within the VO context, whereas other kinds of interactions are supposed to be
using the network resources in a more intensive manner. For example, certain resource
owner could be located within the VO, resulting in a direct interaction between himand a
resource requester, which is not necessarily managed by the VO middleware. Although
VOs communication requirements are low in terms of bandwidth, reaching every VO
member as fast as possible is often desired. For this reason, GMAC bounds logarithmi-
cally the maximum path length of application level hops for a message traversing the
VO. This is particularly useful to send a message to the whole VO, for example to gather
information from data distributed throughout the VO or to spread global requests.
Structured P2P systems organize the overlay network to determine easily the location of
distributed resources. Based on DHTs, the node responsible for the required resource is
reached using an indexable description, for instance by means of exact matching queries.
DHTs also balance the load in each node, since each peer is responsible for approxi-
mately the same number of keys. On the other hand, GMAC is intended for general
purpose queries over dynamic resources. In this context, keeping the DHT indexes up-
dated would introduce a significant amount of traffic to the network. From these per-
spective, GMAC is closer to the idea of unstructured networks, where general purpose
queries are spread through the overlay and answered by resource providers themselves.
By not relating the overlay network to the provided resources, maintence costs are re-
duced. However, in unstructured networks, the inefficient flooding mechanism used to
spread messages still produces a significant amount of overhead. Therefore, by means
of a distribution tree, GMAC avoids duplicates messages and loops. In some scenarios,
exact matching queries could be useful to certain VOs. However, as several solutions
based on DHTs have already proven to be successful and in order to reduce the scope of
the Thesis, this type of service is not covered in GMAC.
In summary, the idea behind GMAC is to achieve logarithmic distribution times, while
keeping protocol overhead as low as possible, reducing the overlay construction and
maintenance costs, and thus enhancing scalability.
It is worth mentioning that the decentralized approach of GMAC is suitable to establish
dynamic VOs in a rapid way. GMAC does not aim to support static or well constituted
organizations, where, for example, control and security matters start to become critical
and might need to be regulated by third parties or designated entities.
The next section introduces the approach proposed in this Thesis.
66 CHAPTER 4. GMAC
Normal Host
Host behind a firewall or NAT (CRH)
Key:
Figure 4.1: GMAC tree
4.2 GMAC Model
GMAC is based on a binary tree as overlay network, where VOs members are nodes in
the tree, while the links are unicast connections. In this way, each node is connected to
other three nodes at most, the parent and the left and right children (Figure 4.1).
GMAC provides participants with the services required to establish and participate in
a VO. These services encompass membership management and group communication
primitives. The whole functionality is achieved in a collaborative and decentralized man-
ner, as the responsibilities for message delivery, tree building and failure recovery are
distributed among the VO members. The key idea that makes this scheme possible is to
take advantage of the recursive nature of the overlay structure, allowing every member
to contribute in the same proportion. Therefore, when members either join , leave unex-
pectedly, need to send VO announcements or search for certain resources, the remaining
members also take part, balancing the load and relieving burdened members.
To provide this functionality in a decentralized way, ascending control messages are
used, since only knowing the information downward the tree is required in each node.
Therefore, each host receives control messages from its children, updates its control state
and sends it upward to its own parent. Only changes in the overlay structure, such as
host arrivals and departures, will trigger these control messages.
GMAC also supports hosts with connectivity restrictions. This kind of hosts are behind
firewalls or NATs that do not allow having open ports or using port-forwarding mech-
anisms. These hosts will be referred as CRHs as they are not able to accept incoming
connections and hence are incapable of communicating between each other. GMAC only
supports CRHs as leaves of the binary tree, limiting them to at most N/2| +1, where N
is the total number of VO members. Studies with eDonkey and Gnutella reveal that as
many as 36% of the hosts may be CRHs, claiming it as a real problem that needs to be ad-
4.2. GMAC MODEL 67
dressed (Wang et al., 2004). As GMAC allows 50% of the nodes to be CRHs, this problem
is correctly addressed.
There is also a component common to all VOs called VOs-bootstrap. The main property
of the VOs-bootstrap is that it is globally accessible in the Internet, thus it works as a
rendezvous or “meeting place” for hosts willing to join a group, as it provides them with
information needed to join any specific group. The VOs-bootstrap will be explained in
more detail in Section 4.5.3.
To illustrate the approach, for example, one of the main services GMAC provides is to
send messages to every VO member, a.k.a. multicast messages. This kind of messages
are required to support group announcements, disseminate queries, or to implement
more complex propagation mechanisms such as summarizing messages, which will be
explained in Section 4.4.3.
In order to provide this functionality, each host in a GMAC VO sends data only to their
neighbors. These, in turn, forwards the received data in the same manner, relieving the
source host.
P14 Sends a message to
the whole VO.
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
Figure 4.2: Sequential unicast
The logarithmic propagation behavior of this approach can be appreciated by compar-
ing it with sequential unicast, a.k.a. naïve unicast, where each group member has to
send a copy of the message to all the other group members (Figure 4.2). To illustrate
the approach, if we assume that a message transmission between two unicast links takes
one second, considering that unicast transmission times are restricted only by bandwidth
constraints (e.g. a 20KB message in a 160kbps unicast link), and that all group members
have the same connection capabilities, for example ADSL. As Figure 4.3 shows, GMAC
would take 11 seconds for every node to receive a message, while sending the message
68 CHAPTER 4. GMAC
28 20
12
4
2 3
1
30 22
14
6
29 21
13
5
31 23
15
7
24 16
8
25 17
9
27 19
11
26 18
10
1sec
2sec
3sec
4sec
5sec
3sec
6sec
4sec
5sec
5sec
6sec
6sec
7sec
7sec
7sec
7sec
8sec
8sec
8sec
8sec
8sec
9sec
9sec 9sec 9sec 10sec 9sec 10sec 11sec 10sec
Figure 4.3: GMAC message propagation example
to each member would require 30 seconds with the sequential unicast approach. In this
scenario, similar to a centralized server, a bottleneck would be produced, since a message
would have to be sent to each of the remaining members (N − 1 times). Although usu-
ally a centralized server has considerably more network bandwidth than normal users
and it could send multiple messages simultaneously, the goal of GMAC is to support
collaboration among users with homogeneous network capabilities.
Although this is a simplified example, it is useful to show how the propagation behavior
of the approach can benefit members with equal connection capabilities.
It is worth noting that when doubling the number of group members in the above exam-
ple, GMACwould take 14 seconds (only 3 seconds more), as it has a logarithmic behavior,
whereas sequential unicast would demand 61 seconds to reach all group members.
The next Section describes why a binary tree as overlay topology is prefered. The mes-
sage types supported by GMAC are described in Section 4.4, while in Section 4.5 the dis-
tributed algorithms for the overlay construction and maintenance are explained. Finally,
conclusions are presented in Secion 4.6.
4.3 Overlay Structure
To support decentralized collaboration, in opposition to approaches that use optimized
overlay structures (Chu et al., 2002; Pendarakis et al., 2001; Jannotti et al., 2000; Chawathe,
2003; Francis, 2000; Banerjee et al., 2002), such as MST, a simple binary tree overlay struc-
ture is preferred by GMAC for the following reasons:
• Scalability: Generally the cost of building and maintaining an optimized structure,
like a MST, is greatly increased as the number of group members grows. As a con-
4.3. OVERLAY STRUCTURE 69
sequence, MSTs are usually confined to groups with a limited number of members.
Examples that use MSTs are ALMI and NARADA.
• Dynamism of the environment: Using an optimized structure assumes that members
have dissimilar connection capabilities, and the overlay structure is build based on
those parameters. However if those parameters change very often, the structure
should be re-optimized frequently. Therefore an optimized structure is usually not
suitable in very dynamic environments.
• Fairness: Equal standing members have similar connection capabilities and respon-
sibilities should be shared evenly. Accordingly, GMAC guarantees that each group
member will retransmit data to other two members at most. Moreover, the Internet
connection in a host is normally shared by several applications. GMACis used only
for VO coordination, while it is expected that other interactions will be using the
network resources in a more intensive manner. As a consequence, it might be inap-
propriate to use a host with a better connection than the rest as group re-transmitter,
or burdened with extra tasks.
• Decentralization: Decentralization is a key factor to provide a robust solution. The
computation used to build and maintain an optimized structure generally relies
on a central component, decentralization is not easy and leads to expensive and
suboptimal solutions (Jin and Bestavros, 2006). Accordingly, GMAC uses a self-
organizing approach, since all members share the same responsibility and cooper-
ate in the formation and maintenance of the tree structure in a decentralized way.
• Fault tolerance: In GMAC hosts are reconnected in a fast and effective way. How-
ever, the cost for handling failure recovery in optimized structures is often high
because it usually implies running stabilization protocols to reconfigure the over-
lay, which is a time and resource consuming process.
• Protocol overhead: Optimized overlay structures often achieve better resource uti-
lization than unoptimized overlays. However, the construction and maintenance of
an optimized structure require performing several measurements, which also use
network resources. Considering that GMAC is usually used for transmitting small
sporadic messages, taking many measurements might use more network resources
than the communication itself.
In opposition to other approaches (Chu et al., 2002; Pendarakis et al., 2001; Jannotti et al.,
2000; Chawathe, 2003; Francis, 2000), the overlay structure proposed in this Thesis does
not take into account the underlying network. The generation and maintenance of an op-
timized structure demand network resources, producing significant protocol overhead.
70 CHAPTER 4. GMAC
Consequently, optimized overlay structures are best suited for transmitting heavy traffic,
such as audio or video.
On the other hand, non-optimized ad-hoc overlay networks, such as GNUTELLA, also
present some of the problems mentioned above. In unstructured networks the node de-
gree follow the power-law distribution, this is, most nodes have a small number of links
and few nodes have a large number of links, which guarantees low dissemination la-
tency (Aberer et al., 2005). Message flooding, however, introduces high network band-
width consumption (Aberer et al., 2005). Besides, the power-law distribution, overlooks
the fairness requirement, as nodes degrees are not balanced, suggesting that some nodes
are more loaded than others.
For these reasons, GMAC uses a binary tree without considering each node capabilities
in particular, which should be rather similar since VOs are formed by equal standing
members. Therefore, protocol overhead is kept low, since performing measurements is
avoided. In addition, by adopting a fast failure detection and recovery strategy, instead
of a failure avoidance scheme, additional maintenance costs are also reduced.
Although having a higher tree degree may improve end to end latency, the overall group
throughput would be reduced. In other words, to forward a message in a k-ary tree, in-
ternal nodes would have to divide their available bandwidth by k receivers. On the other
hand, external nodes (leaves) do not act as message forwarders, as they only transmit
their own messages. Since the ratio between the number of internal and external nodes
in a k-ary tree with i internal nodes is given by:
Internal nodes
External nodes
=
i
i(k −1) +1
·
1
k
(4.1)
, the more the tree degree k, the more the load imbalance between internal and external
nodes, this is, having less internal nodes which also are more loaded forwarding mes-
sages. It is important to notice that in a binary tree (k = 2) the number of internal nodes
is maximized, being approximately equal to the number of external nodes: i/(i +1). Ac-
cordingly, to achieve a logarithmic distribution behavior, GMAC relies on a binary tree
as overlay structure.
4.4 GMAC Message Types
GMAC provides four different type of messages: multicast messages, control messages,
summarizing messages and welcome messages. These messages, along with the self-
organizing overlay, provide the functionality required to communicate and organize
VOs.
4.4. GMAC MESSAGE TYPES 71
4.4.1 Multicast Messages
Multicast messages are used to make announcements to the whole VO. In order to pro-
vide this functionality, each host in a group sends data only to their neighbors. These, in
turn, retransmit the received data in the same manner, relieving the source host. GMAC
behaves in a logarithmic way relative to the number of VO members to transmit a mes-
sage to the whole VO.
1: procedure SENDMULTICASTMESSAGE(MMessage)
2: if connectionParent = null then
3: send(MMessage,connectionParent)
4: end if
5: if connectionLe f tChild = null then
6: send(MMessage,connectionLe f tChild)
7: end if
8: if connectionRightChild = null then
9: send(MMessage,connectionRightChild)
10: end if
11: end procedure
Algorithm 1: Sending a multicast message to the whole VO
1: procedure MULTICASTMESSAGEARRIVED(MMessage,connectionSource)
2: if connectionParent = connectionSource and connectionParent = null then
3: send(MMessage,connectionParent)
4: end if
5: if connectionLe f tChild = connectionSource and connectionLe f tChild = null then
6: send(MMessage,connectionLe f tChild)
7: end if
8: if connectionRightChild = connectionSource and connectionRigthChild = null
then
9: send(MMessage,connectionRightChild)
10: end if
11: send(MMessage, application) to the application layer
12: end procedure
Algorithm 2: Receiving and forwarding a multicast message
Algorithms 1 and 2 describe the multicast message propagation scheme from each node
perspective. When a message arrives to a node, the node forwards it before delivering it
to the application layer.
The best case would happen when the root sends a multicast message, thus, the largest
path number of end hosts a message must traverse to reach every VO members is:
tb = log
2
N| (4.2)
72 CHAPTER 4. GMAC
, this is, the path to the farthest leaf.
The worst case arises when a leaf sends a multicast message. In this case, the message
delivery would encompass the longest path of the tree to reach the farthest leaves. Con-
sequently, the largest path for the worst case is:
tw = 2(log
2
N| −1) (4.3)
4.4.2 Control Messages
This type of messages are used by the overlay structure to let a node knowits related con-
trol information downward the tree. Control message flow is always ascending. There-
fore, each host receives control messages from its children, summarizing the control state
of the left and right branch.
Control messages take part in the tree contruction and maintenance, described in Sec-
tion 4.5.
4.4.3 Summarizing Messages
Asks for
X Average
(a) Average requested
(24x1+20x1+25)/3=23
Asks for
X Average
(b) Query processing
(23x3+20x2+23)/6=22
(22x6+15)/7=21
(c) Partial answer propagation
(21x11+16x1+22x1+18)/14= 20.5
X average=20.5
(d) Final result
Figure 4.4: Summarizing message example
4.4. GMAC MESSAGE TYPES 73
1: procedure SENDSUMMARINZREQUESTMESSAGE(sumReqMessage)
2: W ←∅ stores destinations (for future answers)
3: A ←∅ stores answers
4: summarizingSource ← mysel f stores the source
5: if connectionParent = null then
6: send(sumReqMessage, connectionParent)
7: W ← W ∪ ¦(connectionParent)¦
8: end if
9: if connectionLe f tChild = null then
10: send(sumReqMessage, connectionLe f tChild)
11: W ← W ∪ ¦(connectionLe f tChild)¦
12: end if
13: if connectionRigthChild = null then
14: send(sumReqMessage, connectionRightChild)
15: W ← W ∪ ¦(connectionRightChild)¦
16: end if
17: end procedure
Algorithm 3: Sending a summarizing message request
VOs often require information summarizing data that is spread over the whole group.
This information might encompass ratings, census, profile matchings, or any other spe-
cific group information (Thorpe, 5-8 Jan. 2004; Preguiça et al., 2006; Dourish, 1995).
For example, a member might be interested in obtaining the average age of the other
members. Figure 4.4 shows how this is done by using the summarizing message mecha-
nism. First a member asks for the average age with a summarizing request (Figure 4.4(a)).
A node that receives this request propagates it to their neighbors and waits for their an-
swer. The response comes in the form of a tuple with the partial average and a coefficient
representing the number of nodes involved in the average partial result. If a node has
no neighbors it answers immediately with its age and number “1” as coefficient. As Fig-
ure 4.4(b) shows, once a node receives the expected answer, it calculates a new partial
result, including its age value, and propagates it to its neighbor. Figure 4.4(d) shows how
the final result reaches the initial requesting node. This strategy greatly reduces the num-
ber of transmitted messages, taking 2 ∗ N messages, one to propagate the query and one
to receive the answer back from each node. Besides the whole computation is equally
distributed among the group members, as each member contributes providing network
bandwidth and processing power, following the incremental solving scheme described.
Algorithms 3, 4, 5 and 6 describe how summarizing messages are handled in each host.
The propagation of the summarizing message request is similar to the multicast message,
with the difference that each time a node receives a request, it stores the information
related to the source and destination paths, in order to know how many and from whom
74 CHAPTER 4. GMAC
1: procedure SUMMARIZINGREQUESTMESSAGEARRIVED(sumReqMessage, connec-
tionSource)
2: W ←∅ stores destinations (for future answers)
3: A ←∅ stores answers
4: summarizingSource ← connectionSource stores request source
5: if connectionParent = connectionSource and connectionParent = null then
6: send(sumReqMessage, connectionParent)
7: W ← W ∪ ¦(connectionParent)¦
8: end if
9: if connectionLe f tChild = connectionSource and connectionLe f tChild = null then
10: send(sumReqMessage, connectionLe f tChild)
11: W ← W ∪ ¦(connectionLe f tChild)¦
12: end if
13: if connectionRightChild = connectionSource and connectionRigthChild = null
then
14: send(sumReqMessage, connectionRightChild)
15: W ← W ∪ ¦(connectionRightChild)¦
16: end if
17: send(sumReqMessage, application)
18: if W = ∅ then no answers missing
19: Send(sumReqMessage, application, f alse)
20: else
21: Send(sumReqMessage, application, true)
22: end if
23: end procedure
Algorithm 4: Receiving and forwarding a summarizing message request
answers are expected, and where to forward the resulting summarizing answer.
Thereafter, the application processes the request and waits for missing summarizing an-
swers, which are sent to back the application only when all of them have arrived. The
resulting answer is generated by combining the answers received with the one locally
produced. If no answers are expected, the application is informed that no combination is
required, thus, the summarizing answer generated will only represent itself.
It is important to mention that identifiers are used for each summarizing message, in or-
der to differentiate summarizing messages and handle multiple summarizing message
requests. Therefore, this identifier is used for storing and retrieving summarizing mes-
sages answers and awainting messages.
4.4.4 Welcome Messages
Members receive a welcome message when they join a group. Welcome messages are
suitable for telling a new member information already established in the VO, such as
4.5. DISTRIBUTED VO CONFORMATION AND MAINTEANCE 75
1: procedure SENDSUMMARIZINGANSWER(sumAnswerMessage)
2: if connectionParent = summarizingSource then
3: send(sumAnswerMessage, connectionParent)
4: else
5: if connectionLe f tChild = summarizingSource then
6: send(sumAnswerMessage, connectionLe f tChild)
7: else
8: if connectionRightChild = summarizingSource then
9: send(sumAnswerMessage, connectionRightChild)
10: end if
11: end if
12: end if
13: end procedure
Algorithm 5: Sending a summaring answer
1: procedure SUMMARINGANSWERARRIVED(sumAnswerMessage, summarizingAnswerSource)
2: if W ∩ ¦(summarizingAnswerSource)¦ = ∅ then
3: W ← W ` ¦(summarizingAnswerSource)¦
4: if W = ∅ then there are more answers missing
5: A ← A ∪ ¦(SumAnswerMessage)¦
6: else
7: ProcessAnswers(A, application) Generates the SummarizingAnswer
combination
8: end if
9: end if
10: end procedure
Algorithm 6: Receiving a summaring answer
things defined by the group in its absence, member status, policies, etc. Accordingly,
parent nodes transmit this kind of messages as soon as a connection with a new member
is established.
4.5 Distributed VO Conformation and Mainteance
The general idea to achieve decentralization is that when a node receives a join request
it incorporates the requesting node as a child. If there is no room at that node (i.e. it
already has two children), it will delegate the join request to its least weighted child (i.e.
the one with the smallest subtree). Therefore, a host willing to join a VO will descend the
tree until it is inserted as a leaf. As the overlay structure is a binary tree, a host joining a
group will have to traverse at most log
2
N nodes, being N the total number of nodes in
the tree.
76 CHAPTER 4. GMAC
In the following subsections a more complex join heuristic, used by GMAC to reduce
connections and allow CRHs, is explained.
4.5.1 Improved Join Heuristic
The main idea is that each host in the tree knows where the next connection point down-
ward the tree is, so as to delegate an incoming host directly to its joining point node.
In order to provide such functionality, a variable called nextParent, consisting in the ad-
dress of the node corresponding to the next connection point, is held at each host. The
nextParent value in a node is determined by the control messages received from its chil-
dren.
1
2 3
4
8
6 7 5
TreeWeight : 1
Lweight : 0Rweight : 0
TreeWeight : 2
Lweight : 1Rweight : 0
TreeWeight : 4
Lweight : 2Rweight : 1
TreeWeight : 8
Lweight : 4 Rweight : 3
Next Parent:8
Next Parent:4
Next Parent:6
Next Parent:6
Next Parent:5
Next Parent:5
Next Parent:5
Next Parent:7
TreeWeight : 1
Lweight: 0 Rweight : 0
TreeWeight : 1
Lweight : 0Rweight : 0
TreeWeight : 1
Lweight: 0 Rweight : 0
TreeWeight : 3
Lweight : 1Rweight : 1
Next Parent Host
9
Figure 4.5: Improved Join Heuristic
In order to join a VO, a node follows Algorithm 7, while each node already estabished in
the tree follows Algorithm 8 to deal with join requests.
To delegate a joining host to the proper connection point, each node receives control mes-
sages from its children with their nextParent and tree weight values. With these values,
a node calculates its own nextParent and tree weight values and sends them to its own
parent. With this process the subtree state belloweach node is known. In case the overlay
structure changes, ascending control messages are triggered by the affected nodes (if the
tree weight and nextParent values have changed).
An example of the improved heuristic is shown in Figure 4.5. The nextParent values are
determined in each node as follows:
• Node 8 is a leaf of the tree, thus its own address is used as the nextParent value.
Node 4 does the same thing, as it has only one child (i.e. it has room for one more
4.5. DISTRIBUTED VO CONFORMATION AND MAINTEANCE 77
1: procedure JOINVO(groupName, password) Joining a VO
2: rootAddress ← getRootAddress(VOsBootstrap,groupName,password)
3: if rootAddress = myAddress then
4: connected ← true is the root
5: else
6: connectionOutgoing=connectToHost(rootAddress, groupName, password)
7: if isAcceptedAsChild(connectionOutgoing) then
8: connected ← true
9: connectionParent ← connectionOutgoing
10: else
11: while connected = f alse do
12: nextParentAddress ← getNextParent(connectionOutgoing) the
nextParentAddress of the least weighted child
13: endConnection(connectionOutgoing)
14: connectionOutgoing ← connectToHost(nextParentAddress, group-
Name, password)
15: if isAcceptedAsChild(connectionOutgoing)=true then
16: connected ← true
17: connectionParent ← connectionOutgoing
18: end if
19: end while
20: end if
21: end if
22: end procedure
Algorithm 7: Joining a Virtual Organization
host). Node 2 updates its nextParent with the information received fromits children,
selecting the nextParent information from node 6 (where the least weighted subtree
is) and sends this information to its parent (node 1).
• Node 3 sets its nextParent with the value received from node 5 (in equal conditions
selects by default the value from the left) and sends it to node 1.
• Node 1 will also set its nextParent to the one received from its least weighted sub-
tree, in this case, the address of node 5, (the nextParent received fromits right child).
When a new host (node 9) wanting to join the VO arrives, it will be immediately
forwarded to the node 5, which will incorporate it as a leaf.
4.5.2 Enabling Connectivity Restricted Hosts (CRHs)
To allow hosts behind firewalls or NATs, a slight extension to the previous heuristic is
required to allow them only as leaves of the overlay tree.
78 CHAPTER 4. GMAC
1: procedure RECEIVECONNECTIONREQUEST(connectionIncoming)
2: if connectionLe f tChild = null then
3: connectionLe f tChild ← connectionIncoming
4: sendConfirmationToChild(connectionLe f tChild)
5: else
6: if ConnectionRigthChild = null then
7: connectionRightChild ← connectionIncoming
8: sendConfirmationToChild(connectionRightChild)
9: else
10: sendConnectoToHost(connectionIncoming, nexparentAddress)
11: end if
12: end if
13: end procedure
Algorithm 8: Receiving a Joining request
When all the leaves of a subtree are CRHs, it will not be possible to incorporate more
hosts of this kind. As a consequence, a new boolean variable called allowCRHs is stored
by each node to know whether a subtree is capable of accepting join requests from CRHs.
This information, together with the nextParent and tree weight values, is packed in control
messages.
9
2
3
2
3 9
4
2
3
4
1
Key:
Host behind a firewall or NAT (CRH)
Normal Host
R
e
c
o
n
n
e
c
t
C
o
n
n
e
c
t
Join
Figure 4.6: Rotation of a connection restricted child
To keep CRHs as leaves, when a host having both children and at least a CRH gets a
connection request from a non-CRH, it will incorporate it by performing a rotation. As
depicted in Figure 4.6, a rotation is performed as follows:
1. The new non-CRH host requests to join the VO to a node with a CRH child.
2. The CRH leaf is asked to reconnect to the new incoming host.
4.5. DISTRIBUTED VO CONFORMATION AND MAINTEANCE 79
3. The new host is accepted as a child in the place where the CRH was.
As a consequence, every host having a CRH child will set its nextParent variable to his
own address, as a non-CRH will always be incorporated via a rotation.
1: procedure RECEIVINGACONNECTIONREQUEST(connectionIncoming)
2: if connectionLe f tChild=null then
3: connectionLe f tChild ← connectionIncoming
4: sendConfirmationToChild(connectionLe f tChild)
5: else
6: if connectionRigthChild=null then
7: connectionRightChild ← connectionIncoming
8: sendConfirmationToChild(connectionRightChild)
9: else
10: if isNotCRH(connectionIncoming) then rotation to replace CRH children
11: if isCRH(connectionLe f tChild) then
12: sendConnectToHost(connectionLe f tChild, connectionIncoming)
13: connectionLe f tChild ← connectionIncoming
14: sendConfirmationToChild(connectionLe f tChild)
15: else
16: if isCRH(connnectionRightChild) then
17: sendConnectToHost(connectionRightChild, connectionIncoming)
18: connectionRightChild ← connectionIncoming
19: sendConfirmationToChild(connectionRightChild)
20: else
21: sendConnectoToHost(connectionIncoming, nextparentAddress)
22: end if
23: end if
24: else
25: if CRHallowed then
26: sendConnectToHost(connectionIncoming, nextParentCRHAddress)
27: else
28: sendNoMoreCRHsAllowedNotificationToChild(connectionIncoming)
29: end if
30: end if
31: end if
32: end if
33: end procedure
Algorithm 9: Handling a Join request, full heuristic.
Algorithm 9 describes the full heuristic for allowing all types of hosts, including the ro-
tation of a CRH child by replacing it with a non-CRH joining host.
It is worth mentioning that in GMAC VOs are formed incrementally. Consequently,
CRHs join-requests will be refused if the current tree does not support more CRHs. Nev-
ertheless, the application using GMAC could implement a mechanism where CRHs retry
80 CHAPTER 4. GMAC
to connect after a certain amount of time, waiting for a non-CRH connection or a CRHs
departure.
Next Parent:2
AllowCRHs= false
Next ParentR : 3allowCRHs= TRUE
Next ParentL : 2allowCRHs= FALSE
AllowCRHs= True
Next Parent:5
AllowCRHs= true
Next Parent:5
AllowCRHs= true
Node 9 willing to Join the group.
The root will delegate it depending on if
it is aCRH or not.
TreeWeight : 1
TreeWeight : 3
Lweight : 1Rweight : 1
TreeWeight : 1
TreeWeight : 8
Lweight : 3Rweight : 4
TreeWeight : 2
Lweight : 1Rweight : 0
TreeWeight : 1
TreeWeight : 4
Lweight : 2Rweight : 1
N
onC
R
H

C
R
H

TreeWeight : 1
2
4 6
8
7
3
5
9 1
Normal Host
Key:
CRHs
Figure 4.7: Heuristic for allowing CRHs
An example of the full heuristic, allowing CRHs, is shown in Figure 4.7:
• Hosts 4, 6, 7 and 8 are CRHs.
• Both children of Node 2 are CRHs, thus it sets allowCRHs to false (it is not capable
of accepting more CRHs).
• Node 5 sets allowCRHs to true, as it only has one child.
• Node 3 gets this information and sets allowCRHs to true, as node 5 is able to ac-
cept join requests from CRHs. Nevertheless, it sets the nextParent value to its own
address, as a rotation would be needed if a normal host wants to join in the future.
• Node 1 (the root node) will get:
– From node 2: the address of node 2 as nextParent, 3 as the tree weight, and that
it does not allow any more CRHs.
– From node 3: the address of node 3 as nextParent, 4 as the tree weight, and that
it allows CRHs.
4.5. DISTRIBUTED VO CONFORMATION AND MAINTEANCE 81
Thus, when a new host (host number 9) wants to join the VO, it will send to node 1 (the
root) a connection request, which will be delegated depending on whether the new host
is a CRH or not.
It is important to notice that when the allowCRHs values of a node’s children differ, for
example node 3 in Figure 4.7, it must set its nextParent value to its own address, as it will
have to decide where to delegate join requests depending on the connectivity capabilities
of the incoming host.
Even though others solutions are possible, allowing CRHs as leaves of the tree leads to
a simple, clean and convenient solution. Others possibilities, such as having levels of
intermediate CRHs (where CRHs and non-CRHs alternate) would add extra complexity
to the joining and failure recovery algorithms. In addition, this would admit at most
66.7% CRHs, compared to 50% of having CRHs only as leaves. Other possible approach
is NAT traversal (Rosenberg et al., 2003; Francis, 2003; Ford et al., 2005), which enables
CRHs to connect directly to each other. However, this approach was avoided as it could
bypass local network restrictions and policies.
4.5.3 VOs-bootstrap
As explained earlier, the main requirement of the VOs-bootstrap is that it has to be glob-
ally accessible in the Internet, working as a rendezvous or starting place for hosts willing
to join a VO.
New Host
Join VO−1
VO−1
VO−2
VOs−bootstrap
Figure 4.8: VOs-bootstrap
The main responsibility of the VOs-bootstrap (Figure 4.8) is to publish the root addresses
of the VOs. Therefore the root address of a particular VO is sent to a host intending to
join it, provided that the correct VO name and password is given.
Additionally, the VOs-Bootstrap also has to replace the published roots in case of depar-
ture or failure. Therefore a mechanism for replacing a published VO root is provided.
82 CHAPTER 4. GMAC
4.5.4 Failure Recovery
GMAC, rather than implementing a failure avoidance strategy, uses a fast failure recovery
mechanism. Instead of having redundant links, broken links are detected and reestab-
lished on failure. In case a node fails or leaves a VO, the tree must be restructured to
continue providing communication support to VOs. This reorganization is done by the
remaining members in a decentralized way, as follows:
• The parent node, which had the failing node as a child, just closes the connections
to it, updates its state information and sends it to its own parent.
• Children nodes, which had the failing node as its parent, must reconnect to the tree
by sending a connection request to the root.
For the non-adjacent nodes downward the one that failed, reorganization is transparent,
as they will be reconnected along with the orphan nodes.
In this way, a node failure is handled by reconnecting its children, which is accomplished
in a decentralized manner. In addition, tree reorganization involves at most two node
reconnections, thus the computational cost of a failure is still logarithmic over the number
of nodes.
A special case arises when the node that leaves the VO is the root itself, since orphan
nodes will fail when attempting to reconnect to it. In order to establish a new root, as or-
phen nodes notice the root is missing or not responding they will ask the VOs-bootstrap
to become the root themselves via a claimRoot message. This claimRoot message contains
the address of the supposed failing root in order to let the VOs-bootstrap check if the re-
ceived root address corresponds to the published one (i.e. that it has not been previously
changed). Then the VOs-bootstrap will verify that it is really failing, and provided this is
true, it will accept the root claim message and publish the new root address for the VO.
A claimRoot message is rejected if the included root information does not match the VOs-
bootstrap published root information. In such cases, a message containing the updated
root information is sent back to the claiming node. This will happen after a root fails
and both orphan children send a claimRoot message to the VOs-bootstrap; the first one to
arrive will succeed, while the second will be rejected. However the second one, when re-
jected, will get the updated root information, which in this case, is the succeeding orphan
address, i.e. the new root for the VO. For this special case, there will be an extra recovery
delay, due to the waiting time the non succeeding orphan node must wait until it gets the
new root address.
The VOs-bootstrap is the known rendezvous point to join a VO, however, any host will-
ing to join a group could ask an already connected host for the root address. Only a
4.5. DISTRIBUTED VO CONFORMATION AND MAINTEANCE 83
failure in both, the root and VOs-bootstrap would prevent the root substitution strategy,
in which case the orphan nodes would be unable to replace the root, splitting the VO
in two. Although this is unlikely to occur, additional measures could be taken in the
VOs-bootstrap so as not to compromise availability.
1: procedure PARENTFAILUREDETECTED
2: connected ← f alse
3: while rootFound=false do
4: connectionOutgoing ← connectToHost(rootAddress,groupName,password)
5: if root not responding then
6: rootAddress ← claimToBeTheRoot(VOsBootstrap, rootAddress, group-
Name, password, myAddress)
7: if rootAddress = myAddress then
8: rootFound ← true claim succeded, otherwise someone claimed first,
the new root will be tried next loop
9: end if
10: else
11: rootFound ← true the root responded
12: end if
13: end while
14: goto Algorithm 7 line 6.
15: end procedure
Algorithm 10: Recovering from a parent failure
1: procedure CHILDFAILUREDETECTED(connectionChild)
2: if ( thenconnectionChild=connectionLeftChild)
3: connectionLeftChild=null
4: else
5: connectionRightChild=null
6: end if
7: control Message ← generateControl Message()
8: sendControlMessage(ConnectionParent, control Message)
9: end procedure
Algorithm 11: Recovering from a child failure
In Algorithms 10 and 11 the failure recovery algorithm is described from the child and
the parent point of view respectively.
Figure 4.9 depicts the GMACfailure recovery mechanism. Suppose node 3 leaves the VO.
The root, which had node 3 as a child, just releases the connection. Both “orphan” nodes 5
and 7 will reconnect to the root keeping their current children, thus actually a subtree is
being reconnected. In this case, the root first handles node’s 5 request incorporating it
as its right child. Next, the root accepts node 7 request by delegating the join request to
84 CHAPTER 4. GMAC
Node 3 fails
1
2
3
4
1
2
6
10 14
4
8 12
16
5
9 13
17
7
11 15
3
1
2
6
10 14
4
8 12
16
5
9 13
17
7
11 15
Reconnection Request
1
2
6
10 14
4
8
16
5
9
13
17
7
11 15
1
2
6
10 14
4
8 12
16
5
9
13
17 7
11 15
12
VOs−bootstrap
VOs−bootstrap
VOs−bootstrap VOs−bootstrap
Figure 4.9: Failure recovery
node 13.
Although the final tree is not completely balanced, the overall overlay performance is
not compromised. Besides, the tree will tend to recover its balance as new nodes join
the tree. The failure recovery mechanism in GMAC is fast and simple, and neither extra
control links, nor extra control state information needs to be maintained in the nodes for
failure resilience purposes. As a consequence, GMACapproach is to detect and reconnect
disconnected nodes immediately, rather than avoiding nodes disconnection by adding
redundancy or complex heuristics.
When a node or a link fails, messages traversing the overlay may be lost. Design de-
cisions in GMAC are always made towards simplicity and keeping low the overhead,
being consistent with the decentralized and open approach proposed. Consequently,
GMAC does not provide a reliable mechanism to recover lost messages, since it would
require the implementation of expensive methods, encompassing buffers, message num-
bering, etc. Nevertheless the applications running over GMAC are free to implement any
desired recovery strategy when needed. However, for the supported VOs, which will be
described in Chapter 6, lost messages do not require to be recovered, since reconnected
subtrees can be aware of essential changes on the VO through welcome messages.
Next, how a node failure is handled in each type of message is briefly described.
4.6. CONCLUSIONS 85
4.5.4.1 Multicast Messages
Every time subtrees get disconnected, current messages traversing the tree may be lost.
To solve this problem, a first attempt might be to store undelivered messages in buffers
every time an adjacent node fails. Next, the reconnected subtree sends a multicast mes-
sage to contact the nodes that were adjacent of its previous parent. Subsequently, missing
messages could be exchanged. However, this approach fails when a parent and a child
node fail at nearly the same time. To avoid this problem the reconnected node should ask
for all undelivered messages in the tree, generating many replicated messages and possi-
bly congestion. This is why message recovery is optional in GMAC, allowing the specific
applications using GMAC to decide which approach to use depending on their commu-
nication requirements. For the case of the supported VOs, which will be described in
Chapter 6 lost messages do not require to be recovered. Nevertheless, the reconnected
subtree is aware of changes on the VO through welcome messages.
4.5.4.2 Summarizing Messages
Summarizing messages are also affected when subtrees get disconnected. Since summa-
rizing messages gather data from every member, when a subtree gets disconnected, the
mechanism will be unable to derive information from the whole VO. However, the incre-
mental method employed allows to prune the subtree, excluding their nodes to take part
in the summarizing answer. This happens when a summarizing request has been sent to
a node that fails before it has sent the summarizing answer. This pruning is done in the
following way: each time a node fails, the adjacent nodes look if summarizing answers
are expected from the failing node and removes them, hence, the answer is generated
excluding the branch contained by the failing node.
Summarizing messages are generally independent of the number of participants. For
instance, considering the example of Section 4.4.3, where the average age of participants
is obtained, even if some nodes fail during the summarizing message lookup, thanks to
the pruning mechanism, the age average of the remaining nodes is still obtained. This
strategy is usually enough, since in a dynamic VOs it is unlikely that enquiring every
member will be mandatory.
4.6 Conclusions
GMAC is an application level solution to support VOs in a robust and scalable way.
GMAC provides the basic services to support decentralized communication within a VO,
providing different message types to establish and collaborate in a fair way.
86 CHAPTER 4. GMAC
The recursive nature of the overlay structure allows to reach any member of the VO
traversing only a logarithmic number of nodes. In addition, keeping only summarized
information relative to the branch of the tree each node represents, responsibilities of
the construction and maintenance of the overlay are shared in a fair way, keeping the
overhead low and thus achieving scalability.
The goal of GMAC is to allow users with an inexpensive Internet connection to establish
and collaborate in a fast and easy deployable way, without requiring the disposal of a
high capacity server or special routers spread across the Internet. GMAC also employs a
fast failure recovery mechanism, rather than keeping redundant connections that would
imply additional overhead and computational costs.
The approach adopted by GMACis always oriented towards the most simple, yet flexible
solution, considering that VOs mostly work as a collaboration context with low commu-
nication demands, exchanging small and sporadic messages.
The next chapter, a materialization of GMAC and its implementation details are pre-
sented. Several technical issues are discussed and the solutions adopted are described.
Chapter 5
Implementation
GMAC has been materialized as a library implemented in Java. Java has many desirable
properties that ease the application development and deployment. Some of them are
multi-threading, inheritance, object orientation, portability and a well known Applica-
tion Programming Interface (API), just to mention a few.
For the implementation of the GMAC model, a practical yet flexible approach has been
adopted. For instance, although it would be useful to have a fully reliable implemen-
tation, where lost messages could be recovered, this would also imply a considerable
reduction of efficiency and a significant increase of the protocol overhead and the overall
complexity.
This chapter describes an implementation of the GMAC model used to validate the ap-
proach. The next section describes the model concrete materialization and its architec-
ture, including the explanation for its most relevant Java classes. The Application Pro-
gramming interface is described in Section 5.2, while specific implementation details are
shown in Section 5.3. In Section 5.4, two different simulators, developed to evaluate
GMAC in VO settings of considerable size, are described. Finally, in Section 5.5 the chap-
ter conclusions are presented.
5.1 Concrete Materialization
For the concrete materialization of GMAC, as Figure 5.1 shows, the overlay network is
built connecting nodes by two unicast links:
• Control link: through which control messages, related to the overlay tree building
and maintenance, are transmitted.
87
88 CHAPTER 5. IMPLEMENTATION
Normal Host
Host behind a firewall or NAT (CRH)
Key:
Data link
Control link
Figure 5.1: GMAC tree
• Data link: through which data messages are sent and forwarded. For example, when
a multicast message arrives, it is immediately retransmitted to other neighbors and
sent to the application layer for processing.
Handling control and data messages separately simplifies the implementation of the
model and improves efficiency, although using a single link is also possible.
A VO member is identified by an Internet address and port, which are used by the rest
of the participants to establish their connections. GMAC uses TCP connections for both
data and control links. Therefore, at most, a host will have six permanent TCP connec-
tions. UDP packets could have been used instead, however reliable connection helps
simplifying aspects such as host connectivity.
Figure 5.2 depicts the simplified Unified Modeling Language (UML) class diagram of
GMAC implementation.
Considering the purpose of the classes, the architecture can be divided in the three layers:
the communication level, the overlay level and the application level. The classes in the
communication level handle unicast connections. The overlay construction and mainte-
nance takes place at the overlay level and finally at the application level the API is pro-
vided. At each level, the main responsibilities are provided by the ConnectionManager,
TreeMaintenace and GroupCast classes respectively.
The whole design was done to favor flexibility. The GroupCast class provides the appli-
cation interface for GMAC services, the ConnectionManager issues all TCP connections
and the TreeMaintenace is responsible for creating and keeping the overlay structure. In-
heritance was used to distinguish standard hosts from connectivity-restricted ones, and
to keep open the possibility of adding different implementation strategies.
5.1. CONCRETE MATERIALIZATION 89
ConnectionServer
NodeAdministrator
ParentAdministrator
Connection
ChildAdministrator
ConnectionControl ConnectionData
GroupCast
Abstract-TreeMaintenance
ConnectionManager
TreeMaintenance TreeMaintenanceNAT
Application Level
Overlay Level
Connection Level
Figure 5.2: GMAC Class Diagram
The ConnectionManager class manages all connections. It has an instance of the
ConnectionServer, which listens for incoming connections on an open port. Each time a
new connection arrives, a new thread handles it and the handshaking process is started.
Four types of connections are distinguished:
• Incoming Connection: A connection issued by another host.
• Outgoing Connection: A connection a host starts.
• Control Connection: The connection in which ascending control messages are sent.
• Data Connection: The connection in which data messages are transmitted.
When a new host is willing to join the VO, the handshaking process, from an already
connected node perspective, is done as follows: The ConnectionServer listens on the
open port for incoming connections. If the joining request can be satisfied, the incoming
90 CHAPTER 5. IMPLEMENTATION
connection turns into a control connection. Once this control connection is established, a
data connection is created with a similar process. It is worth mentioning that, once the
handshaking process is done, an incoming or outgoing connection turns into a data or
control connection, which, consequently, are considered secure.
The TreeMaintenace class uses a NodeAdministrator class that, in turn, has an instance
of the ParentAdministrator class and two instances of the ChildAdministrator class,
one for each child. These instances are responsible for the control and data connections
for the parent and children nodes. Besides, the ChildAdministrator instances store their
current nextParent, weigh and allowCRHs values for each subtree.
5.2 Application Programming Interface
GMAC has been developed as a Java package to allow developers to use GMAC services
easily. For example, to join a VO, GMACmust be provided with the VOname, password,
and the VOs-bootstrap IP address and port. The following code shows the steps for
joining a GMAC VO called VONAME1:
GroupCast groupCast = new GroupCast("VONAME1", "PASSWORD1", gmac.com.ar, 2000);
groupCast.startListeningInPort (3333);
groupCast.joinGroup();
Hosts behind NATs or firewalls, which are not allowed by network administration poli-
cies to either use port-forwarding or having an open port will be treated as CRHs. For
non-CRH, only one reachable open port in the range 1024–65535 must be provided so as
not to be considered a CRHs by GMAC. This port is used by the ConnectionServer to
attend incoming connections.
Once the connection is established, the host becomes an active member of the VO and are
able to send and receive the different message types.
Java provides an API for broadcast and multicast messages represented as an instance of
the DatagramPacket class. Many existing applications already use this API. GMACbrings
support for these kind of messages in the Internet. Accordingly, the DatagramPacket ab-
straction is adopted to facilitate porting to GMAC applications based on the Java API for
broadcasting and multicasting, though TCP connections are actually used underneath.
Additionally, GMAC provides a set of tools for encapsulating and retrieving serializable
objects to and from a DatagramPacket.
The different message types are handled with the following primitives:
groupcast.sendMulticastMessage(DatagramPacket packet);
groupcast.sendSummarizingRequest(DatagramPacket packet ,String id);
groupcast.sendSummarizingAnswer(DatagramPacket packet ,String id);
5.3. IMPLEMENTATION DETAILS 91
These primitives are used to send multicast messages, summarizing requests and an-
swers. In either case, the messages are sent as DatagramPackets. For summarizing
messages an identifier must be provided. Multicast messages can optionally implement
an identifier and encapsulate it in the serialized object instance. However, for summariz-
ing messages, an identifier is compulsory, since the application requires to know which
answers matches which request.
A message is received by the blocking method: receiveAnyMessage(). These method
runs in a separate thread which is triggered on a message arrival. A special data-
gram packet called DataPacketGMCast is received. These object includes the original
DatagramPacket and the kind of message received. Usually, each application creates a
special serializable object to be transmitted in each case.
At these point it is worth mentioning that for receiving summarizing answers, GMAC
provides a special object in which all the answers received are included. Since expected
summarizing answers are stored and only sent to the application level when every an-
swer has arrived.
public void run(){
DataPacketGMCast dpGM;
while (true) {
dpGM=this.groupcast.receiveAnyMessage (); //blocking
application.PacketReceived(dpGM);
}
5.3 Implementation Details
The matching between the GMAC model and the implementation is quite straight for-
ward. However, there are certain technological factors that could be addressed in several
ways, though always preserving the the GMAC model features and advantajes. This sec-
tion presents the most relevant technological decisions made during the development of
GMAC Java library.
5.3.1 Connectivity-Restricted Hosts
GMAC supports CRHs. These hosts are restricted by NATs or firewalls, where local net-
work policies forbids them to have open ports or to implement port forwarding mecha-
nisms (e.g. Universal Plug and Play (UPnP; Jeronimo and Weast, 2003) is not available).
As these hosts can not accept incoming connections, they are not able to communicate to
each other. GMAC allows CRHs as leaves of the binary tree, thus they only have one con-
nection to their parent, which is a non-CRH. The parent node acts (as any other internal
node) as message forwarder from and to the CRH.
92 CHAPTER 5. IMPLEMENTATION
The maximum number of CRHs supported in a VO is at most n/2| + 1, where n is
the total number of VO members. When every leaf is a CRH, no more CRHs would
be allowed, and in this case, the departure of a non-CRH would also imply that a CRH
would no longer be supported. Therefore, when no room is available for more CRHs,
a non-CRH departure would also mean the disconnection of one CRH. This is what
was meant in the introduction by degraded service to CRHs. Studies with eDonkey and
Gnutella reveal that as many as 36% of the hosts may be CRHs, claiming it as problem
that needs to be addressed (Wang et al., 2004). GMAC allows 50% of the nodes to be
CRHs, thus this problem is correctly addressed.
Nevertheless, for an exceptional setting, if more than half of the VO members are ex-
pected to be CRHs, non-CRHs could be provided to act as support for CRHs, i.e. behav-
ing in a passive way just as message forwarders. Other possible approach is using NAT
traversal, which enables CRHs to connect directly to each other (Rosenberg et al., 2003;
Francis, 2003; Ford et al., 2005). However, the use of this approach is discouraged as it
could bypass local network restrictions and policies.
5.3.2 VOs-bootstrap
The VOs-bootstrap is implemented as an independent Java application. When a node
ask the VOs-bootstrap to join a certain VO given the VO name and password and pro-
viding their IP address and reachable port. If the given VO name does not exist, the
VOs-bootstrap sets up a new VO with the joining node as the root. The root substitution
mechanism is synchronized at VO level in other to attend only one claim root at the time,
while other requests are processed concurrently. Periodically, the VOs-bootstrap issues a
connection to each published VO root to check if they are is still running, removing the
non responding ones.
Additionally, if a machine has several network interfaces, and thence several addresses,
the IP address and port used to reach the VOs-bootstrap is considered the one that should
be employed by GMAC without the users intervention. However, for some network
settings, this might not be enough. For instance when several firewalls and NATs are
traversed and the outgoing connections might take a different route than incoming ones.
In such cases it is necessary that the user specifies the IP address and port manually.
The VOs-bootstrap could handle a large number of VOs without requiring a high ca-
pacity Internet connection. However, if a new host is neither able to access the VOs-
bootstrap nor find an already connected node, it will not be capable to join the desired
VO. Nevertheless, instead of relying on a single VOs-boostrap, several VOs-bootstraps
could be used. Amazingly enough, this group of VOs-bootstrap could be seen as sort of
VO in which members are well known and static VOs-bootstraps. Consequently, these
5.4. SIMULATORS 93
VOs-bootstraps could rely in GMAC, with the slight difference that each VOs-bootstrap
should keep an ordered reference list of the other VOs-bootstraps, to know which one is
entitled to be the root and thus avoiding contention problems.
5.3.3 Failure Recovery
When a parent fails, the orphan nodes try to reconnect the root of the tree. In case the
root is not responding, after three attempts, the orphan node will claim to be the root to
the VOs-bootstrap, providing their IP address and port, and the supposed failing root
address and port. In turn, the VOs-bootstrap either accepts the claim and publishes the
new root, or provides the orphan with the address and port of the current root.
The improved Join Heuristic might generate loops if some precautions are not taken. This
could happen in case of a node failure, which orphan child issues a reconnection before
the control message reaches the root. In this case, the root could issue the orphan node
to reconnect to a node in its own subtree, and thus forming a loop. For this reason, the
orphan nodes: stores a list of the last nextParent values included in control messages sent
to their parent, so as to compare if the root delegates the orphan node to a node in its
own subtree. Therefore, loops are prevented when a node reconnection is delegated to
a nextparent which matches a nextparent in its list. When this happens, the orphan node
waits an estimated period of time for the tree to stabilize.
5.4 Simulators
Since GMAC is intended for VOs of over a hundred members, it is very difficult to evalu-
ate its behaviour in real conditions. Moreover, the decentralized nature of GMAC makes
even more complex to obtain the relevant metrics, which will be presented in Chapter 7.
Accordingly, two simulators were developed to properly evaluate GMAC behaviour in
VO of considerable size.
5.4.1 Application Level Simulator (ALS)
An application level simulator (ALS), resembling a configurable binary tree overlay net-
work, was developed in Java. The ALS has the advantage that VOs of a given size can
be created enabling the unicast links between nodes to take different latency and band-
width values. For latency, a random value within a defined range is used. These values
were derived placing hosts in a bi-dimensional space and relating distance values to the
94 CHAPTER 5. IMPLEMENTATION
lantency ranges desired. A predefined set of values are used for bandwidth. For exam-
ple, in most of the experimental results, the values used were either 128 Kbps, 256 Kbps,
384 Kbps or 512 Kbps in each link.
Once the VO is establish, messages can be send and propagated through the simulator,
which is able to log the different distribution times. The size of these messages are easily
inferred from the real application running over GMAC (see Chapter 6).
Accordingly, different VO sizes can be easily tested, and the simulation can be repeated
to obtain different statistical data, such as standard deviation values, etc.
5.4.2 Underlying Network Aware Simulator (UNAS)
In order to consider the underlying Internet network topology, a special Underlying Net-
work Aware Simulator was implemented. This simulator takes as input topologies cre-
ated with the BRITE (Medina et al., 2001) topology generator. In order to simulate an
Internet-like underlying topology, the waxman (Waxman, Dec 1988) graph model was
used, since its distance-dependent model of link formation have shown to describe real
Internet topologies remarkably well (Lakhina et al., 2003; Medina et al., 2000; Calvert
et al., 1997).
Once the UNAS is feed with the generated topologies, it is able to build the GMAC over-
lay tree, considering random nodes from the input graph. To establish the unicast links
between two GMAC nodes the minimum underlying path is computed. Considering
the important sizes of the graphs, the calculation of the unicast links is a very expensive
process in terms of CPU resources.
5.5 Conclusions
In this chapter the concrete materialization of the GMAC model was presented. The
main classes of the object-oriented architecture were described. In addition, the Ap-
plication Programming Interface shows how GMAC must be used to send and receive
packets using the different message primitives. As the API shows, the adopted approach
is adequate for providing multicast support over the Internet to already implemented
applications, without requiring significant modification efforts.
Additionally, several implementation details were explained. This details are mostly re-
lated to technological solutions for problems encountered when adapting the model to
the concrete implemented solution.
The next chapter presents concrete examples of virgual organizations supported by the
implemented middleware.
Chapter 6
Virtual Organizations Supported by
GMAC
GMAC allows homogeneous entities to collaborate in the context of a VO. GMAC is ad-
vantageous to a number of communities, which share the communication requirements
described in Section 3.1.
In this chapter, three different VO supported by GMAC are presented. They have been
used to validate GMAC in different collaboration scenarios, showing its feasibility and
flexibility to support VOs in the Internet.
The next section presents MoviLog (Zunino et al., 2005; Mateos et al., 2007), a platform
for executing mobile software agents that relies on GMAC for communication purposes.
Additionally, two groupware applications were developed. The interaction schemes re-
quired for groupware applications demand special communication support. In some
cases, when the collaboration is driven by equal standing members and its communi-
cation demands are not critical, VOs are a convenient collaboration environment. Con-
sequently, two groupware applications supported by GMAC were developed: Chatero,
for synchronous collaboration and Science-Peer, for asynchronous collaboration. These
applications are presented in Sections 6.2 and 6.3 respectively.
6.1 MoviLog
GMAC, as described in (Gotthelf et al., 2005, 2008b) was used to allow the coordination
of MoviLog mobile agent platforms (Zunino et al., 2005; Mateos et al., 2007). Here the
term coordination is used from the MoviLog platforms’ perspective, not encompassing
VO construction and maintenance actions.
95
96 CHAPTER 6. VIRTUAL ORGANIZATIONS SUPPORTED BY GMAC
Mobile agents platforms are a special case of equal standing VOs, where the coordination
should be achieved in an autonomous way, without relying in a central server, as mobile
agent platforms usually belong to independent organizations/institutions. Mobile soft-
ware agents reduce network usage by migrating code to the suited platform, and thus
working with their resources locally. In order to enable this coordination, MoviLog plat-
forms need to send small, sporadic, multicast messages announcing available resources
such as code, data or services, to provide communication between agents and for man-
aging their mobility.
MoviLog (Zunino et al., 2002) is based upon the concept of Reactive Mobility by Failure
(RMF) (Zunino et al., 2005). RMF aims at simplifying mobile agent development by al-
lowing the programmer to delegate decisions on when and where to migrate a mobile
agent based on its resource requirements. The idea is that when a mobile agent needs -at
some point of its execution- a resource that is not available at the local site, RMF acts by
transferring the agent to a site where the required resource is offered. As a consequence
mobility is transparent to the programmer, agents are smaller and migrate faster (Zunino
et al., 2005).
MoviLog Platform
PNS Agents
PNS Agents
Agent
R2
R1
MoviLog Platform
R3
PNS Agents
R4
R3
MoviLog Platform
GMAC for resource
announcement and
control information
Mobile Agent
requires R2 and R4
MARlet
MARlet
MARlet
(a) Chat and whiteboard
Figure 6.1: MoviLog
As Figure 6.1 depicts, MoviLog relies on servers called MARlets, which provide the run-
time support for executing mobile agents. In addition, MARlets provide services for shar-
ing resources such as programs, data or any other agent accessible asset. The backbone of
the RMF run-time support is a distributed multi-agent system composed of non-mobile
agents called Protocol Name Servers (PNS). This system is in charge of managing infor-
mation about resources available at each host capable of executing mobile agents. Each
PNS can announce to others about new available resources, resources that are no longer
6.2. CHATERO 97
available and useful information for managing mobility such as network links status,
CPU and memory usage at each host, etc.
Therefore, to coordinate code mobility, MoviLog platforms rely on multicast services
mainly for two different purposes:
• Resource Announcement: Sporadically, each MoviLog platform needs to announce
its new available resources, as well as disposing the old ones. For this kind of
information, MoviLog sends, approximately two messages every 10 minutes. Each
message has an approximated size of 2KB.
• Control Information: Some specific MoviLog platform information must be shared
periodically. This information includes values on specific MoviLog particular met-
rics and current control state information such as CPU load, number of active
agents, memory usage, etc. This type of messages are sent every 30 seconds and
are 150 Bytes long.
In order to manage this information, PNSs must coordinate using a multicast communi-
cation mechanism. This communication was supported through message broadcasting,
limited to local area networks or sequential unicast, hence causing scalability problems
in the Internet due to excessive network traffic.
It is worth mentioning that since MoviLog was developed previously to GMAC, it was
only adapted to send multicast messages through GMAC, and hence, summarizing mes-
sages were not adopted. GMAC is suitable for addressing these issues, since most ap-
proaches for providing multicast services rely either on special routers, are designed for
single sender media streaming, or are unable to handle large VOs, hosts behind firewalls
or NATs in a decentralized way. In addition, existing approaches demand more network
resources for managing the overlay than the messages sent by the PNSs.
Accordingly, GMAC provides multicast support to meet MoviLog requirements. In Sec-
tion 7.3 experimental results concerning to MoviLog are presented, showing that GMAC
is a scalable solution in comparisons against alternatives such as NARADA (Chu et al.,
2002), NICE (Banerjee et al., 2002) and LARK (Kandula et al., 2003).
6.2 Chatero
Chatero is a synchronous collaborative application that enables people to join in a chat
group, vote for courses of actions and drawover fixed background images. In most cases,
collaborative applications are formed by well-known members that work together within
the scope of an organization, where practices such as organization guidance and tracking
98 CHAPTER 6. VIRTUAL ORGANIZATIONS SUPPORTED BY GMAC
are desirable. Conversely, Chatero is a discussion/brainstorming application, useful for
people who join a VO in an ad-lib way to work on a transient task. In many cases the
collaboration may also demand certain urgency, thus these systems often require easy
and fast deployment, decentralization and dynamism (Scalem et al., 2004; Ochoa et al.,
2007; Haake et al., 2004; Munkvold et al., 2006).
In order to join a group in Chatero, participants must provide a group name and pass-
word. Participants are identified by nicknames, and once joined, they can immediately
start chatting or drawing in the whiteboard.
(a) Chat and whiteboard (b) Voting
Figure 6.2: Chatero, a synchronous groupware example
Figure 6.2 shows Chatero in a crisis management situation, specifically to fight forest
fires. A fireman can join the group to discuss and organize the way to extinguish the fire.
Participants are able to draw over a fixed background image, in this case, a satellite view
of the affected area. It is also possible for the participants to vote for a proposal, a course
of action, or anything they wish. For the voting system a user must declare the voting
topic and a set of possible answers to select. Afterwards, a voting panel is displayed
on every group member’s screen, and options are shown, as depicted in Figure 6.2 (b).
After a specified period of time, the panel expires and the participant losses his voting
opportunity.
Chatero, as explained in Gotthelf et al. (2007), relies on GMAC, enabling any member in
the Internet to collaborate while keeping the distribution times logarithmic in relation to
the VO size. When a new user joins the VO, he sends a multicast message to announce
himself. A “welcome message” informs the new user who is currently connected and
the last chat messages transmitted. Summarizing messages are used to generate and
propagate voting results. The answer is stored in each host until the remaining neighbors
6.3. SCIENCE-PEER 99
transmit their answers, then the joint results are transmitted. Once the whole answer
is assembled in the voting requester, the results are shown to the whole group. It is
important to mention that the voting mechanism is kept transparent to users. Thus, they
do not really know whether the answer is transmitted immediately or not, as it could be
waiting for other partial results. The same happens with the voting requester, as the final
results are shown to the group without its intervention.
Chatero is not well suited for hard real-time interaction in the Internet, such as mak-
ing gestures with telepointers (Gutwin et al., 2003; Gutwin and Greenberg, 1999). In
informal tests, when VOs were large and several users were drawing simultaneously,
Chatero tended to lose fidelity, this is, the degree to which different participants’ views
agree. Nevertheless, although drawings did not appear quite smoothly on the partic-
ipants’ screens when VOs were large, the overall response was good enough to allow
them to show their viewpoint and to vote for a course of action.
6.3 Science-Peer
Science-Peer (Gotthelf et al., 2008a) is a groupware application for asynchronous collabo-
ration. It enables researchers to share their articles and contacts with each other. Besides
document sharing capabilities, Science-Peer is intended to find researchers with similar
interests.
Figure 6.3: Science-Peer
The main feature of Science-Peer is to help scientists to find researchers whose works
are related. Most of the existing approaches allow researchers to find related papers that
have already been published. Conversely, Science-Peer enables to search among papers
that are currently in progress.
100 CHAPTER 6. VIRTUAL ORGANIZATIONS SUPPORTED BY GMAC
The idea is to show how an asynchronous collaboration can be supported with GMAC,
thus we are going to focus on the communication and collaboration features, while spe-
cific application aspects will not be discussed in detail.
A user willing to find a related researcher must indicate the paper that will be compared
to the ones provided by other researchers. Then, Science-Peer will find the researchers
whose papers are related to the one provided. As papers might not been published
yet, confidentiality issues must be addressed. Researchers are understandably reluctant
to give away their current papers in progress, as it could jeopardize intellectual prop-
erty and copyright protection. Therefore Science-Peer adopts a distributed approach, in
which comparisons are local and it is not necessary to send sensible information through
the network.
In order to establish a similarity measure between papers, Science-Peer considers the
introductory text, titles of cited papers, authors and journal/conference names. Citation
and content analysis are commonly used bibliometric methods (Shapiro, 1992). These
methods are used to measure texts and information. Science-Peer uses the notion of
bibliographic coupling (Kessler, 1963), which relates papers that cite the same articles.
Usually, these methods are implemented over centralized approaches, where the whole
data must be first gathered together before it can be processed. Conversely, Science-
Peer uses GMAC summarizing messages to support recursive job decomposition. In this
way, GMAC takes advantage of each user’s computer power to parallelize the search
and similarity comparisons, speeding it up with respect to a traditional server based
approach.
Lyx document
Latex File
Ms Word Document
Term vectors
Text file
XML Docbook
XML MODS
Introduction/Abstract
Ja
b
R
e
f
B
ib
2
X
M
L
Bibtex File
Text
[<central,0.293>,<commun,0.293>,<groupwar,0.293>,<network,0.293>,
<applic,0.260>,<server,0.260>,<internet,0.228>,<overlai,0.228>,
<group,0.195>,<member,0.195>,<requir,0.163>,<topolog,0.163>,...]
Authors
[<Zhang,0.324>,<Jamin,0.243>,<Kaashoek,0.243>,<Kubiatowicz,0.243>,
<Mateos,0.243>,<Stoica,0.243>,<Wang,0.243>,<Zhao,0.243>,<Zunino,0.243>,
,<Campo,0.162>,<Dabek,0.162>,<Druschel,0.162>,<Huang,0.162>,...]
Titles
[<peer,0.629>,<multicast,0.314>,<network,0.244>,<applic,0.209>,
<scalabl,0.209>,<system,0.209>,<obil,0.174>,<architectur,0.139>,
<collabor,0.139>,<scale,0.139>,<ulticast,0.139>,<decentr,0.104>,
<distribut,0.104>,<end,0.104>,<groupwar,0.104>,...]
Journals
[<{{IEEE}J.Sel.AreasCommun.,0.566>,<Commun.{ACM},0.527>,
<ComputerNetworks,0.377>, <ACMComput.Surv.,0.188>,
<FirstMonday,0.188>, <IEEE/ACMTransactionsonNetworking,0.188>,
<MultimediaSystems,0.188>, <SIGMISDatabase,0.188>,...]
Figure 6.4: Science-Peer papers pre-processing
As Figure 6.4 shows, papers are preprocessed to be converted into normalized term vec-
tors, removing stop words and applying stemming (Baeza-Yates and Ribeiro-Neto, 1999)
to the document content and cited titles.
Once these vectors are obtained, they can be compared using the cosine similarity mea-
6.3. SCIENCE-PEER 101
sure (Baeza-Yates and Ribeiro-Neto, 1999). This measure is based on the cosine of the an-
gle between two vectors and assuming that two documents with a small angle between
their vector representations are related to each other. As the angle between the vectors
shortens, the cosine angle approaches 1, i.e., meaning that the similarity of whatever is
represented by the vectors increases. Interestingly, only two term vectors are required to
obtain the similarity value, making this method suitable for distributed processing.
In order to answer a query, Science-Peer is based on the summarizing message scheme
described in Section 4.4.3. First, the query, formed by the generated vectors and search
parameters, such as number of expected answers, vector weights etc., is propagated us-
ing a summarizing request (Figure 6.5).
[<peer,0.629>,<multicast,0.314>,<network,0.244>,
<applic,0.209>,<scalabl,0.209>,<system,0.209>,
<obil,0.174>,<architectur,0.139>, <collabor,0.139>,
<scale,0.139>,...]
Find Similar work:
V1:
V1
V1
V1
V1
V1
V1
Figure 6.5: Science-Peer query distribution
Each node, after receiving a summarizing request, calculates the similarity value for each
local paper, and waits for missing answers. This is the case of node n3 from Figure 6.6,
which calculates the partial answer a3
/
, but still waits for the answers of n1 and n2. Leaf
nodes, as they do not have to wait for more answers, just send a summarizing answer
which encompass only the local papers. The answers and sent back to the node from
which the request was received. Once the nodes obtain all the missing answers, they
recalculate and forward the best answer, which includes the most similar papers from
the answers received and the values locally obtained (a3 for the case of n3 in Figure 6.6).
In this way, the best answers are generated and propagated towards the node which had
originally issued the query recursively (node n9 in Figure 6.7).
With this mechanism the n most similar papers can be found in logarithmic time, allow-
ing the comparisons in each node to execute in parallel.
It is worth noticing that as papers in progress are kept at the researchers’ machines, offline
researchers’ papers do not participate in queries. Nevertheless, Science-Peer saves the
102 CHAPTER 6. VIRTUAL ORGANIZATIONS SUPPORTED BY GMAC
Find 3 most Similar works:
V1
CS (v1, lv1)=0.33
CS (v1, lv2)=0.42
CS (v1, lv3)=0.62
*Get 3 most Similar Local
CS (v1, lv1)=0.14
CS (v1, lv2)=0.19
CS (v1, lv3)=0.17
*Get 3 most Similar Local
a1=(lv3n1,lv2n1,lv1n1)
a2=(lv2n2,lv3n2,lv1n2)
CS (v1, lv1)=0.04
CS (v1, lv2)=0.39
*Get 3 most Similar Local
CS (v1, lv4)=0.13
a3’=(lv2n3,lv1n3)
a2 a1
a3=Vmax(a1,a2,a3’) =
(lv3n1, lv2n1, lv2n3)
n1 n2
n3
a3
* Parallel processing
Figure 6.6: Science-Peer summarizing answer
information gathered from previous queries, allowing researchers to compare different
query results.
Additionally, very little information is transmitted over the network, as queries only con-
tain term vectors and answers are summarized, containing only similarity values and
researcher’s contact information. Experimental results relative to different group sizes
are presented in Section 7.4.
6.4 Conclusions
The main contribution of GMAC is to provide a communication support for VOs in a
fast and easy deployable way, allowing members to cooperate among each other in a fair
way, minimizing protocol overhead and thus achieving scalability.
In this chapter, three different collaboration examples were presented, showing that
GMACadequately supports VOs that share some characteristics in their communications
requirements.
MoviLog mobile agent platforms are a special case of homogeneous members forming
a VO. These platforms rely in GMAC to announce available resources and status infor-
mation and thus, enabling code mobility. Initially, MoviLog was confined to local area
networks or sequential unicast in the Internet, where to address membership matters,
each platforms needed to know each other connected platform, leading to configuration,
scalability and availability problems.
6.4. CONCLUSIONS 103
Final Answer: =Vmax(a11,a7,a8)
a5
a6
a8
a7
a11
n8
n9
n7
n5
n11
n6
a8’= Get 3 most Similar Local
a8=Vmax(a5,a6,a8’)
Figure 6.7: Science-Peer query results
Software mobile agents reduce network resource usage by moving the code and thus
working with the resources locally. Mobile agent platforms only require to use the net-
work for coordination purposes -to announce available resources-, and hence they do not
have high communication demands.
Some groupware applications could also be considered a special case of VOs. GMAC
has shown to be a suitable communication support at least for two particular groupware
applications: Chatero, a crisis management synchronous collaborative application and
Science-Peer, an asynchronous collaborative application for finding scientists with simi-
lar research interests based on bibliometrics methods.
Although several solutions exist for groups which are formed by fewparticipants, GMAC
aims to allowa relatively large number of people to interact on the Internet in a decentral-
ized way. GMAC not only achieves scalability, robustness and availability, but also lever-
ages group members’ computational resources, such as network links and CPU time.
Despite some of these goals can be achieved by using server farms or other decentralized
architectures, grouping participants in a VO allows another collaboration perspective,
where all the collaboration is carried out by equal standing participants without requir-
ing the involvement of third parties, high capacity links as centralized approaches do, nor
handling complex network administrative issues. Therefore, the goal is to allow users to
interact relying only on a typical dial-up or ADSL Internet connection.
104 CHAPTER 6. VIRTUAL ORGANIZATIONS SUPPORTED BY GMAC
Chapter 7
Experimental Results
GMAC is intended to allow the collaboration of autonomous equal standing entities in
VOs. Some of the alternatives described in Chapter 3 provide support to applications
with different communication requirements. For instance, ALM alternatives generally
support a single sender broadcasting media, and try to maximize bandwidth, while con-
ferencing oriented applications require lowlatency as well, though they may allowgrace-
ful degradation.
On the other hand, GMAC was not designed for streaming media, but to enable the
collaboration of equal standing members. Accordingly, the goal is to allow distributed
members to interact in a fair, scalable and decentralized way, with low communication
demands. Nevertheless, it is desirable to analyze and evaluate GMAC performance to
determine how it meets the different VOs requirements.
The next section describes the metrics and evaluation methodology employed. In Sec-
tion 7.2 experimental results for the metrics discussed are presented. Next, in Sections 7.3
and 7.4, the experimental results for the particular VOcases of MoviLog and Science-Peer
are shown. Finally, in Section 7.5 the conclusions are presented.
7.1 Metrics and Evaluation Methodology
In the Internet, two entities communicate with each other issuing unicast connections.
Unicast connections are usually measured in terms of bandwidth and latency. On the
other hand, when the communication is driven by several entities, organized in an over-
lay network, the most relevant aspects to consider are:
• End-to-end propagation time: is the time required for a message to reach the farthest
VO member.
105
106 CHAPTER 7. EXPERIMENTAL RESULTS
• Broadcast throughput: is the maximum broadcast throughput, from the receiving ap-
plications point of view, that a VO can support. Within a distribution tree, nodes
with less bandwidth may act as bottlenecks, thus limiting the overall VO band-
width.
• Protocol overhead: this metric takes into account the network traffic that does not in-
volve application data, including control messages required to build and maintain
the overlay structure.
• Group latency: is the maximum delay, from the application point of view, between
a sender and the receivers. Within a distribution tree, latency is the longest end-to-
end delay between any pair of nodes.
• Link stress: is the number of duplicate packets on the same physical link.
• Network distance: At the application level, network distance is generally related to
latency, assuming that more latency implies more cost.
• Failure recovery: the time required by the overlay to recover from failures in nodes.
In order to evaluate the proposed approach in the different VOsetups, a GMACsimulator
was developed to obtain information about the overlay performance and its scalability,
enabling to configure parameters such as different number of VO members, bandwidth
and latency values, messages sizes, etc.
Two different simulators were developed, ALS (Application Level Simulator) and UAS
(Underlying Aware Simulator) , considering the overlay characteristics from an applica-
tion point of view and using a generated Internet-like underlying network respectively:
• Application Level Simulator (ALS): A simulator that builds the GMAC overlay tree
was implemented considering standard Internet bandwidth and latency values.
These values where derived placing hosts in a bi-dimensional space. Each host
was assigned randomly a bandwidth of either 128 Kbps, 256 Kbps, 384 Kbps or
512 Kbps, while latency varied from 5 ms to 200 ms between any pair of them.
These are relatively normal end-user bandwidth and latency values for an Inter-
net connection. It was assumed that these magnitudes were symmetrical (A→B =
B←A). Furthermore, any host could transmit data messages with equal probability.
• Underlying Network Aware Simulator (UNAS): A second simulation approach, con-
sidering the underlying Internet network topology was also implemented in order
to have a more rigorous scenario when necessary. For this case, several underly-
ing topologies were created with the BRITE (Medina et al., 2001) topology gener-
ator using a waxman (Waxman, Dec 1988) graph model to compare GMAC with
7.2. GENERAL EXPERIMENTAL RESULTS 107
other protocols in an Internet-like underlying topology. This particular class of ran-
dom graphs are used to evaluate routing algorithms, since they have a distance-
dependent model of link formation among routers, describing remarkably well the
real world (Lakhina et al., 2003; Medina et al., 2000; Calvert et al., 1997).
Different configuration of VOs were used for the simulations, which were repeated
20 times to obtain the average and standard deviation values of the metrics.
It is worth mentioning that the UNAS is required when underlying metrics, such as link
stress, must be considered. An overlay network built on top of the generated waxman
graph connote an extremely computationally expensive model to work with, since for
example, to send an unicast message between to overlay nodes, the minimumunderlying
path between these nodes must be previously computed. Therefore, the ALS is used in
most of the evaluations unless explicitly stated.
Several simulations were performed comparing GMAC with sequential unicast and two
MSTs variants, one maximizing the bandwidth and the other minimizing latency. This
could be seen as the approaches adopted by NARADA and ALMI, respectively. It is
important to notice that it would be very difficult to build these MST in real conditions,
since it would be necessary to knowthe physical network topology and its metrics values
in advance and, moreover, to assume they are static, something not likely to happen in
the Internet. However, building these MSTs is useful for comparative reasons, working
as boundaries for the metrics they optimize.
Additionally, the implementations of the approaches described in Chapter 3 were not
always available or did not support the size ranges VOs have, therefore the number of
experimental comparisons possible was limited. Nevertheless, some approaches, such as
NARADA, NICE and LARK, were compared when their experimental results were made
available by the authors. Consequently, these experiments were reproduced to evaluate
GMAC under the same conditions. However, it is worth mentioning that in some cases
the settings used by the alternatives differ significantly fromthe characteristics VOs have.
Nevertheless, they are still useful at least to compare GMAC with the approaches found
in the literature.
Finally, as GMAC follows a well known structure, in same cases it was useful to show
the theoretical boundaries imposed by the model.
7.2 General Experimental Results
Several simulations were prerformed to evaluate GMAC in general purpose VOs. Ac-
cordingly, in this section, results considering different VO sizes and results wich are com-
108 CHAPTER 7. EXPERIMENTAL RESULTS
mon to every VO are shown. The results for MoviLog and Science-Peer specific VOs are
shown in Section 7.3 and Section 7.4 respectively.
7.2.1 End-to-end propagation time
0
500
1000
1500
2000
2500
3000
0 100 200 300 400 500 600 700 800 900 10001100
T
i
m
e

t
o

s
e
n
d

1
0
0
K
B

t
o

t
h
e

V
O

[
s
e
c
]
Number of nodes
Unicast
GMAC AVG
GMAC Worst Case
MST Latency
MST Bandwidth
Figure 7.1: Time to send a 100KB multicast message
Figures 7.15 and 7.16 show the time required by a node to send 100KB to all the group
members. An excessivly large message of 100KB was used to have a better insight of how
the node bandwidth affects the delivery times. Results in more realistic VOscenarios will
be shown in Sections 7.3 and 7.4, where two VOs are analyzed.
Figure 7.15 shows that the alternative of using only unicast is not viable, since the source
node gets quickly overloaded sending the message to every member. This case is similar
to having a central server, which would be expensive and difficult to scale, as a consider-
ably better Internet connection would be required.
Figure 7.16 shows in more detail the results for GMAC (average and worst case) and the
Latency and Bandwidth MSTs alternatives. The MST Latency approach has poor per-
formance as it does not take into account bandwidth metrics and some bottleneck nodes
get burdened having to forward the message to many nodes. The MST Bandwidth was
built optimizing a 100KB message transmission, thus this is the best that can be achieved
in a distribution tree. In average, GMAC scales well, approximating to the values the
MST Bandwidth has. Standard deviations of about 9% can be seen as error bars, showing
that the delivery time vary only a few seconds. This is a predictable behaviour, since this
7.2. GENERAL EXPERIMENTAL RESULTS 109
0
50
100
150
200
250
300
200 400 600 800 1000
T
i
m
e

t
o

s
e
n
d

1
0
0
K
B

t
o

t
h
e

V
O

[
s
e
c
]
Number of nodes
GMAC AVG
GMAC Worst Case
MST Latency
MST Bandwidth
Figure 7.2: Time to send a 100KB message (without unicast)
value is affected by the different values each link in the path has, tending to average the
result.
It is worth mentioning that as the number of nodes grows, the MSTs optimization im-
pact tends to be less decisive, making the improvement, in relation with non-optimized
alteratives such as GMAC, less significant.
7.2.2 Broadcast Throughput
Figure 7.3 shows the VO broadcast throughput that can be achieved with different ap-
proaches. This metric is of particular interest to applications requiring support for au-
dio/video streaming, which is not the case when supporting VOs. Nevertheless it is
useful to know the maximum throughput that could be achieved by a node in each over-
lay.
In distribution trees, less capable nodes act as bottlenecks, limiting the overall through-
put to their bandwidth. In GMAC, bottleneck nodes may have to retransmit messages,
thus the overall throughput is restricted to half of the less capable host bandwidth. It is
worth noting that GMAC was developed for equal standing members, thus, as they are
also expected to have similar connection capabilities, bottleneck nodes are less likely to
appear. As mentioned earlier, GMAC is not concerned about providing media stream-
ing. However, if necessary, a minimum bandwidth requirement could be established
depending on the VO broadcast transfer rate desired.
110 CHAPTER 7. EXPERIMENTAL RESULTS
0
2
4
6
8
10
12
14
16
20 30 50 70 100 150 300 500 900
T
h
r
o
u
g
h
p
u
t

[
K
B
/
s
]
Number of nodes (log−scale)
GMAC
MST Bandwidth
MST Latency
Unicast
Figure 7.3: Maximum VO broadcast throughput
7.2.3 Protocol Overhead
The protocol overhead metric considers network traffic that does not represent useful
application data. In GMAC the information required to build and maintain the binary
tree is minimal, since this information is transmitted only when a node leaves or joins
the VO. In contrast, alternatives using some kind of optimization introduce considerable
overhead. This is due to the fact that many measurements between hosts are required for
building and maintaining such overlay structures. For instance, NARADA introduces
significant protocol overhead, of 2% for 64 members and 4% for 128, with a single host
streaming at 128Kbps (Chu et al., 2002; Jin and Bestavros, 2006). Measurements in opti-
mized overlay networks introduces a considerable amount of overhead, confining them
to small VOs, or restricting them to worse approximations based on complex heuristics.
NICE and LARK are application level protocols intended to reduce the exponential con-
trol overhead of NARADA. Using a hierarchical structure NICE achieves a logarithmic
protocol overhead, while the clustered approach of LARK achieves a constant protocol
overhead. In GMAC the protocol overhead is reduced even further, since only when the
overlay changes, that is, on node arrival or departure, control information is transmitted.
The failure rates used for NICE were 16 members in 100 seconds in a group of 128 (Kan-
dula et al., 2003). Therefore this failure rate was used in GMAC to get fair comparisons.
Table 7.1 shows the protocol overhead bandwidth produced by NARADA (with 30 sec-
7.2. GENERAL EXPERIMENTAL RESULTS 111
Table 7.1: Protocol overhead (in KB/s)
Nodes NARADA-30 NICE LARK GMAC
8 0.61 1.54 - 0.000818
16 2.94 0.87 - 0.001646
32 9.23 1.03 - 0.003302
64 26.20 1.20 1.42 0.006645
128 65.62 1.19 1.39 0.013190
256 96.62 1.36 1.40 0.026281
512 199.96 1.93 1.42 0.052393
1024 - 2.81 1.41 0.104827
1560 - 3.28 - 0.16154
2048 - 5.18 - 0.210454
4000 - - - 0.412099
onds periodic refresh rate), NICE, LARK and GMAC according to different group sizes.
Figure 7.4 shows that NARADA exponential protocol overhead prevents it from scaling.
In Figure 7.5 the same metrics are shown with a different scale. It is worth mentioning
that some values for larger group sizes are missing since they were not available for the
alternatives compared (Kandula et al., 2003; Jin and Bestavros, 2006).
While NICE and LARK reduce NARADA exponential overhead, in GMAC, as no opti-
mization is made, the protocol overhead is minimal, and it is only affected by the overall
joining or departure rate. In order to evaluate GMAC the failure rates proposed by the
alternatives where used, where 12.5% of the nodes from a group fail within 100 seconds.
The expected failure rates in VOs are much lower (less than 0.5% in 100 seconds). In
addition, the protocol overhead presented in the alternatives considered only the traffic
introduced for optimization purposes, excluding the overhead generated by the respec-
tive failure recovery mechanisms.
7.2.4 Group Latency
In overlay networks, group latency is determined by the longest path between any pair
of nodes in the overlay. Figure 7.6 shows the logarithmic behavior of GMAC. The latency
range for the underlying unicast links varied from 100ms to 665ms, a latency range likely
to be found in the Internet.
In this case, sequential unicast is the better approach, since the sender contacts directly
each receiver. The MST Bandwidth has poor latency, as this approach is often used for
media streaming, where latency is not crucial, as it is frequently also relinquished to
avoid jitter. It is important to notice that neither approach is suitable for hard-realtime
groupware collaboration, where latencies above 240ms produce significant coordination
errors (Gutwin, 2001). Accordingly, latencies commonly found in the Internet hinder this
112 CHAPTER 7. EXPERIMENTAL RESULTS
0
10
20
30
40
50
60
70
80
8 16 32 64 128 256 512 1024 2048
O
v
e
r
h
e
a
d

[
K
B
/
s
]
Number of nodes
NARADA
NICE
LARK
GMAC
Figure 7.4: Protocol overhead
type of collaboration for large groups. Nevertheless, real-time collaboration awareness is
often desired for small groups. As the results show, the few seconds required by GMAC
to reach all group members should be good enough for typical VO configuration.
It is worth mentioning that the standard deviations (shown as error bars) are relatively
small since the paths latency is counterbalanced by high and low latency links.
7.2.5 Network Level Metrics
GMAC does not consider the underlying network properties when building the overlay.
One of the drawbacks of this approach is that messages may be sent several times over
the same physical link. The number of duplicate packets that traverses the same physical
link is known as link stress. This metric is used to have an idea of resource utilization,
since having higher link stress would also mean that more network resources are being
wasted. In an optimal network level solution, no message should traverse a physical link
more than once.
Besides, having a high stress in some links would also represent possible bottlenecks,
since these links would require higher bandwidth, as they need to transport multiple
times the same message. Clearly, for our purposes, where nodes are expected to have
similar capabilities, link stress should be evenly balanced.
To evaluate link stress, the UNAS was used to create the underlying network topology.
7.2. GENERAL EXPERIMENTAL RESULTS 113
0
0.5
1
1.5
2
2.5
3
8 16 32 64 128 256 512 1024 2048
O
v
e
r
h
e
a
d

[
K
B
/
s
]
Number of nodes
NICE
LARK
GMAC
Figure 7.5: Protocol overhead (without NARADA)
Figure 7.7 shows the average link stress, for several Internet topology configurations cre-
ated with the BRITE topology generator (Medina et al., 2001, 2000).
The link stress distribution of GMAC, NARADA, sequential unicast and DVMRP (Dis-
tance Vector Multicast Routing Protocol) are shown in Figure 7.8, reproducing the wax-
man generated topology (1024 nodes, and 3072) described in Chu et al. (2002). The stress
of any physical link is at most 1 for DVMRP. In sequential unicast some physical links
have a stress above 16, while in GMAC and NARADA the distribution is more balanced
and no physical links have a stress larger that 9. This behavior is understandable, as the
approach followed by GMAC is to relief the overloaded senders. It is worth mention-
ing that a centralized server stress distribution is equivalent to the the sequential unicast
case, where the links in the server neighborhood are significantly overloaded.
To improve resource utilization, some efforts take into account the underlying network
when building overlay networks (Stoica et al., 2000; Ren et al., 2004; Liu et al., 2004;
Zeinalipour-Yazti and Kalogeraki, 2006; Ratnasamy et al., 2002). As explained in Sec-
tion 4.3, GMAC refrains from using optimized schemes to minimize the protocol over-
head and ease the overlay construction, maintenance and failure recovery mechanisms.
Other approaches such as LARK (Kandula et al., 2003) also relinquishes resource utiliza-
tion to increase robustness. On the other hand, media streaming applications should be
more concerned about resource utilization. As high bandwidth traffic is transmitted, it is
worthwhile to afford measurements costs in order to reduce the overall resource utiliza-
tion. Conversely, in GMAC, as sporadic small messages are sent, it is more important to
114 CHAPTER 7. EXPERIMENTAL RESULTS
0.5
1
2
5
10
100 200 300 400 500 600 700 800 900 1000
G
r
o
u
p

L
a
t
e
n
c
y

[
s
e
c
]

(
l
o
g

s
c
a
l
e
)
Number of nodes
GMAC
MST Bandwidth
MST Latency
Unicast
Figure 7.6: Group latency
not overload the network by exchanging metrics between nodes, since the management
of an optimized structure may imply more resource utilization costs.
7.2.6 Failure Recovery
Another important aspect to be considered is the possibility of failure of some nodes in
the overlay. Even though the probability of a node failure might be small, when VOs are
large, this starts to be a significant matter. When a failure occurs in a distribution tree,
members depending on the failing host, as well as the following dependents in the chain,
can get disconnected.
Figure 7.9 shows the GMAC behavior for larger VOs when 10 nodes fail simultaneously.
In addition to the average recovery time, the time including forcing always a root failure
is shown. For the latter case, one of the orphan nodes will first replace the missing root,
and afterward receive the reconnection requests from the other nodes. In both cases
GMAC quickly recovers from failures in a scalable way.
In order to evaluate GMAC with LARK and NICE, their experimental settings were re-
produced. Accordingly the time required to recover from simultaneous failing hosts in
a VO with 100 members are shown in Figure 7.10. As explained in Subsection 4.5.4,
GMAC quickly recovers from node failures by reincorporating immediately the orphan
nodes along with their subtrees in a decentralized way. In addition to the fact that the
tree is not affected when a leaf fails, which, in the case of GMAC, corresponds to half of
7.3. MOVILOG EXPERIMENTAL RESULTS 115
0.5
1
1.5
2
2.5
3
3.5
100 200 300 400 500 600 700 800 900 1000
A
v
g

L
i
n
k

S
t
r
e
s
s
Number of nodes
20000 links, 10000 Nodes Underlay
12000 links, 6000 Nodes Underlay
18000 links, 6000 Nodes Underlay
DVMRP
Figure 7.7: Average link stress
the tree nodes, thus reducing the activation of the failure recovery mechanism to half the
failure probability.
However, failure recovery in optimized overlay structures, such as ALMI, is more ex-
pensive because a host failure may trigger the reoptimization of the whole structure,
producing additional protocol overhead. Moreover, complex heuristics may be required
in order to keep the diffusion group working until a new optimization is performed.
Accordingly, as the number of VO members grow and thus, more node failures are ex-
pected, optimized overlay structures tend to be less effective (Jin and Bestavros, 2006).
This is one of the reasons that has hindered scalability in approaches based on optimized
overlay structures.
7.3 Movilog Experimental results
Experiments were done to evaluate GMAC with MoviLog specific VOs communication
requirements. To handle code mobility efficiently, each MoviLog platform needs to mul-
ticast information in the following settings:
• Setting A: Each MoviLog platform needs to announce its new available resources,
as well as disposing the old ones. For this kind of information, MoviLog sends, in
average, two messages every 10 minutes. Each message has an approximated size
of 2KB.
116 CHAPTER 7. EXPERIMENTAL RESULTS
1
2
4
8
16
32
64
128
256
512
1 2 4 8 16 32 64 128
#

o
f

P
h
y
s
i
c
a
l

L
i
n
k
s

(
l
o
g

s
c
a
l
e
)
Stress of Physical Link (log−scale)
Unicast
NARADA
GMAC
DVMRP
Figure 7.8: Link stress distribution
• Setting B: Some specific MoviLog platforminformation must be shared periodically.
This information includes MoviLog metrics and current control state information
such as CPU load, number of active agents, memory usage, etc. This type of mes-
sages are sent twice in a minute and are 150 bytes long.
MoviLog was developed previously to GMAC. Accordingly, the summarizing possibili-
ties of GMACwere not considered. Therefore MoviLog relies only on multicast messages.
In order to avoid traffic congestion, the maximum number of participants in a VOs is de-
termined by the node with less bandwidth capacity. If the number of participants grows
further, the node with lower bandwidth would act as a bottleneck, leading to congestion.
For example, the number of members supported for the case of MoviLog, when 15KB/s
is the bandwidth of a node acting as bottleneck, is bounded by:
• Setting A: 2KB * 2/600s = 0.00666KB/s per node. With a 7.5KB/s group throughput:
1125 hosts.
• Setting B: 0.150KB * 2/60s = 0.005KB/s per node. With a 7.5KB/s group through-
put: 1500 hosts.
• Both: 0.005+0.00666 = 0.011666KB/s per node = 642 hosts.
These values provide an estimate of the feasible scalability of GMAC, where a node lim-
ited to a bandwidth of only 15KB/s is retransmitting messages. A typical MoviLog de-
7.3. MOVILOG EXPERIMENTAL RESULTS 117
0
1
2
3
4
5
0 500 1000 1500 2000 2500 3000 3500 4000
T
i
m
e

t
o

r
e
c
o
v
e
r

[
s
e
c
]
Number of nodes
GMAC Recovery Time
GMAC Recovery Time (root always fails)
Figure 7.9: GMAC failure recovery
ployment consists of no more than 400 hosts, therefore although GMAC does not cur-
rently address congestion control, the numbers shown above suggests that GMAC is an
adequate solution given its scalability requirements.
To simulate MoviLog platforms settings, Simulator A was used considering bandwidth
ranges from 15KB/s to 70KB/s, and latencies ranges from 100ms to 665ms. These over
sized latency values for the simulation to really test the approach in an unfavorable set-
ting.
7.3.1 End-to-end group propagation delay
The results corresponding to MoviLog requirements are shown in Figure 7.11. The time
required until the last node receives a 2KB message for MoviLog setting A is shown,
both for the simulation results and the model worst and best case. For the model worst
case, 15KB/s bandwidth and 665ms latency links were assumed in equation 4.3, whereas
for the model best case, 70KB/s and 100ms values were used. These ranges work as
over-dimensioned boundaries for the MoviLog setting. The results show how GMAC
scales, since just a few seconds are required for a message to reach the whole group with
a logarithmic behavior.
118 CHAPTER 7. EXPERIMENTAL RESULTS
0
10
20
30
40
50
60
70
80
10 20 30 40 50 60 70
T
i
m
e

t
o

r
e
c
o
v
e
r

[
s
e
c
]
% of failures (100 nodes)
LARK
NICE
GMAC
Figure 7.10: Failure recovery on a 100-node VO
7.3.2 Scalability
For the MoviLog settings, the minimum bandwidth required in each node is depicted
in Figure 7.12. A typical MoviLog deployment with 400 hosts requires only 10KB/s in
every node in order to work properly under GMAC. NICE minimizes the control message
exchanges, thus reducing the protocol overhead. Conversely, other approaches, such
as NARADA, consume too much bandwidth and network resources for optimization
purposes. In GMAC, scalability is not compromised, as the required bandwidth in each
node is only affected by the amount of data that node is forwarding and, as detailed next,
the protocol overhead is minimum and not affected by the group size.
7.4 Science-Peer Experimental Results
GMAC not only takes advantage of each host communication capabilities for message
forwarding, but also from its computational power, since the summarizing message
scheme can be used to distribute workload among the connected computers. This is par-
ticularly useful for achieving scalability in CPU intensive applications such as Science-
Peer, as we will show in the rest of this section.
Several simulations were performed for different Science-Peer network settings using
statistical data derived from the application execution. Computation values were in-
ferred considering that in each node the application was running on a standard Pentium
7.4. SCIENCE-PEER EXPERIMENTAL RESULTS 119
0
5
10
15
20
0 200 400 600 800 1000
T
i
m
e

t
o

s
e
n
d

2
K
B

t
o

t
h
e

V
O

[
s
e
c
]
Number of nodes
Model Worst
Model Best
Simulation Result
Figure 7.11: Time to send a 2KB MoviLog multicast message
4, 800MHz, 1GB RAM. Each researcher was considered to provide 5 papers, thus the
number of papers to analyze grows with the number of researchers in the VO.
7.4.1 Searching times
Figures 7.13 and 7.14 show the query search time and the maximum number of queries
supported for different VOsizes respectively. To have a better insight of Science-Peer per-
formance, these settings were compared with centralized approaches, where the whole
process is performed by a single machine. Two centralized implementations were con-
sidered. The only difference between the two centralized implementations is that in one
case papers were already preprocessed to avoid this step in each comparison. It is impor-
tant to mention that Science-Peer always re-generates the vectors to work with the latest
version of the papers. However, a better strategy could be to generate the vectors on
start-up and when a query arrives, regenerate them only if the documents have changed.
In both centralized approaches the whole information was locally available, thus the
transfer times of papers, queries and responses were zero. Therefore, the experiments
show the parallelization and transfer times required in GMAC compared only to the pro-
cessing time of the centralized approaches, considering all hosts with the same CPU and
network connections.
As it can be seen, obtaining the most similar papers is a time consuming process. The
distributed approach of Science-Peer enables the parallelization of the process and the
120 CHAPTER 7. EXPERIMENTAL RESULTS
0
10
20
30
40
50
60
70
0 500 1000 1500 2000
B
a
n
d
w
i
d
t
h

U
s
a
g
e

[
K
B
/
s
]
Number of nodes
GMAC
NARADA
NICE
Figure 7.12: MoviLog bandwidth requirements
logarithmic distribution times of both queries and answers. In both Figures 7.13 and 7.14
can be noticed that for VOs with more that 50 participants (250 papers), Science-Peer in
the ADSL setting is better than even the optimized centralized approach. For large VOs,
Science-Peer proved to outperform both centralized approaches, since they get quickly
overloaded by CPU intensive operations such as the computation of the similarity dis-
tances. It is worth noticing that, since only the processing time is being considered for the
centralized approaches, actually, the task parallelization features are being disregarded
as the communication mechanism (although present in Science-Peer) is not considered
for the centralized approaches results.
7.4.2 End-to-end propagation delay
Figures 7.15 and 7.16 show the average time required to send a 6KB message to all the
group. This is the approximate size of a Science-Peer query, which includes four vectors
with 40 terms each and the query parameters.
As Figure 7.15 shows, having to send a unicast message to each VOmember is not feasible
for large groups, since the available bandwidth in a node must be shared to send every
message copy. In a similar way, the MST Latency approach has poor performance, as it
does not take bandwidth metrics into account (Kosti´ c et al., 2008).
GMAC scales well and, as the MST Bandwidth heuristic, only requires a few seconds to
send a 6KB message to all the group members. Although most MST-based approaches
7.5. CONCLUSIONS 121
10
20
30
40
50
60
70
80
90
100
20 35 50 100 200 300 500 800 1500
S
e
a
r
c
h

T
i
m
e

[
s
e
c
]
Number of researchers (log−scale)
Science−Peer, ADSL (16Kbps)
Science−Peer, Dial−up (5Kbps)
Central Server
Central Server (Cached Vectors)
Figure 7.13: Science-Peer vs centralized approach
try to minimize the overall link costs, they have been criticized for not bounding their
degree or diameter (Young et al., 7-11 March 2004; Helder and Jamin, 21-24 May 2002).
Therefore, MSTs are subject to hot-spots and long latencies, which also occur in group
latency and throughput metrics, as we will describe in the following subsections.
7.5 Conclusions
GMAC allows equal standing members to collaborate in the context of a VO, enabling
them to spread requests, make announcements and obtain information from distributed
data. GMAC is intended to provide services to VOs of considerable size with small
throughput transmission requirements.
Approaches that optimize the overlay to provide more throughput present a number
of problems such as protocol overhead for generating and maintaining these structures,
which besides, increases with VO size. In GMAC, these problems are not present, since
the complexity of generating its structure is not affected by the size of the group. Accord-
ingly GMAC does not take additional metrics for optimization purposes reducing the
protocol overhead and achieving a fast and effective failure recovery mechanism. Em-
ploying an application level communication solution makes network resource utilization
rather poor. This is one of the main drawbacks of GMAC, however this problem is also
present in most overlay based approaches. Nevertheless the link stress distribution is
balanced in GMAC, in contrast to having a single node sending all the copies, in which
122 CHAPTER 7. EXPERIMENTAL RESULTS
0
5
10
15
20
25
20 35 50 100 200 300 500 800 1500
Q
u
e
r
i
e
s

p
e
r

m
i
n
u
t
e
Number of researchers (log−scale)
Science−Peer, ADSL (16Kbps)
Science−Peer, Dial−up (5Kbps)
Central Server
Central Server (Cached Vectors)
Figure 7.14: Science-Peer vs centralized approach
physical links near the source might get overloaded.
Results considering MoviLog and Science-Peer requirements show that GMAC provides
a suitable communication solution to different VOs requirements. By relieving the source
from sending all messages, MoviLog platforms are able to scale in the Internet with-
out requiring high capacity connections. In addition, Science-Peer also takes advantage
of GMAC task parallelization capabilities, making use of the distributed computational
power in each member.
Considering the results obtained, GMAC has shown to be a suitable communication sup-
port for equal standing members in the Internet, to only due to its logarithmic distribu-
tion times, bounded node degree, low protocol overhead and fast failure recovery, but
also for its summarizing message features which enables tasks parallelization.
The next chapter presents the final conclusions and future research directions.
7.5. CONCLUSIONS 123
1
2
5
10
20
50
120
100 200 300 400 500 600 700 800 900 1000
T
i
m
e

t
o

s
e
n
d

6
K
B

t
o

t
h
e

V
O

[
s
e
c
]

(
l
o
g

s
c
a
l
e
)
Number of nodes
GMAC
MST Bandwidth
MST Latency
Unicast
Figure 7.15: Time required to deliver a 6KB query to the whole VO
0
5
10
15
20
50 100 150 200 250 300
T
i
m
e

t
o

s
e
n
d

6
K
B

t
o

t
h
e

V
O

[
s
e
c
]
Number of nodes
GMAC
MST Bandwidth
MST Latency
Figure 7.16: Time required to deliver a 6KB query to the whole VO (without Unicast)
124 CHAPTER 7. EXPERIMENTAL RESULTS
Chapter 8
Conclusions and Future Work
The Internet enables new collaboration possibilities and access to geographically dis-
persed resources. The research community is adopting VOs as the context where tem-
porary coalitions of dynamic collaborating entities takes place. Members of a VO inter-
act autonomously to achieve their objectives. In several research circles VOs are being
adopted as a dynamic collaboration context capable of scaling to the Internet (Nami and
Sharifi, 2007). More stable or reduced collaboration contexts can afford to address the
whole administration and configuration of the communication support manually. How-
ever, in collaboration contexts such as the Internet, in which there is an inherent dy-
namism and scalability is desired, relying on a self-organized VO is very advantageous.
This is specially true for large VOs, in which failures in either members or their connec-
tions are frequent, making them arduous to be addressed manually.
When the collaboration is driven by equal standing members, decentralization is essen-
tial. A central authority/control could jeopardize the non-hierarchical nature of the col-
laboration, as control should be shared through different autonomic administrative do-
mains.
Accordingly, this Thesis proposes a communication support for VOs, named GMAC. The
goal of GMAC is to provide a fair collaboration support for equal standing members in
the Internet. The main features of GMAC are flexibility, scalability, robustness and fast
and easy deployment. To provide this communication support, GMAC builds a binary
tree overlay network, being able to distribute responsibilities among its members and
achieving self-organization. Accordingly, GMAC relieves the burst transmitting member
by delegating the transmission to its neighbors.
The binary tree structure helps bounding the node degree and achieving logarithmic dis-
tribution times, whereas a higher tree degree would increase the load imbalance between
leaves and internal nodes. The recursive nature of the solution makes possible to balance
125
126 CHAPTER 8. CONCLUSIONS AND FUTURE WORK
the distribution of responsibilities, as nodes stores and transmit approximately the same
amount of data. Furthermore, this scheme also allows different kinds of enhancements,
such as task parallelization.
GMAC can be widely deployed, because it relies neither on special routers spread across
the Internet nor requires a high capacity server. In addition, it does not require perform-
ing complex network administrative and configuration tasks. Moreover, its application
program interface is compatible with the well known Java native multicast service, al-
lowing already implemented applications to scale to the Internet, without requiring sig-
nificant modification efforts. Furthermore, GMAC only assumes unicast as subjacent ser-
vice, allowing any host with an Internet connection to use it, even those with connectivity
restrictions such as hosts behind NATs or firewalls.
GMAC has been materialized as a Java library. A practical approach has been adopted
for the implementation of the model, always favoring simplicity and flexibility, to allow
the implementation of other strategies when needed.
GMAC has been successfully adopted to support MoviLog (Zunino et al., 2005; Mateos
et al., 2007) -mobile agent platforms- in the Internet. MoviLog mobile agent platforms
are a special case of equal standing member VO. MoviLog platforms send small, spo-
radic, multicast messages announcing available resources such as code, data or services,
to provide communication between agents and for managing their mobility, and hence
they do not have high communication demands. MoviLog was confined to local area
networks or sequential unicast in the Internet, where each platform needed to be manu-
ally configured to address membership matters, leading to configuration, scalability and
availability problems. However, these problems are correctly addressed using GMAC to
announce available resources and status information, enabling MoviLog to scale to the
Internet.
GMAC has also shown to be a suitable alternative to support equal standing groupware
applications that share the VO communication requirements described in Section 3.1.
Groupware collaboration can be done in a synchronous or an asynchronous way. Rely-
ing on GMAC as the communication support, two collaborative applications have been
developed: Chatero, a crisis management application for synchronous collaboration, and
Science-Peer, an asynchronous collaborative application for finding scientists with simi-
lar research interests.
As asynchronous groupware usually does not have a high interaction rate, P2P systems
seems to be intuitively suitable. GMAC have shown to be an effective collaboration
support for Science-Peer (Gotthelf et al., 2008a), an asynchronous collaborative applica-
tion for finding scientists with similar research interests based on bibliometrics methods.
Science-Peer is a good example of howto taking advantage of GMAC task parallelization
capabilities, making use of the distributed computational power in each member.
127
On the other hand, synchronous collaboration requires real-time interaction, thus a de-
centralized overlay communication solution might not seem appropriate. However, for
the case of Chatero (Gotthelf et al., 2007), a crisis management synchronous collabora-
tive application, the logarithmic delivery rates of GMAC has shown to be good enough
to support this type of collaboration. Here, the notion of awareness plays an important
role, as it determines how much information from other users a participant must handle,
and hence, how much information must be transmitted. For example, in a 250 member
group, undoubtedly all members will not be simultaneously painting in a whiteboard.
This is because in a collaborating group there is an implicit relation between the number
of participants, their actions and the awareness required. This relation reduces transmis-
sion requirements, rendering synchronous collaboration possible, as it allows large VOs
to collaborate (250 members requires just an 8 level tree), where surely some participants
will be actively interacting (drawing), and others observing.
Both collaboration examples rely on the summarizing message primitives to retrieve in-
formation from distributed data in a logarithmic time allowing the parallelization of
tasks. This mechanism is implemented in Chatero for incrementally obtaining ratings
and voting results, and in Science-Peer for the similarity lookup. This, in addition to
reducing the network load, as the data transmitted is incrementally summarized, also
enables task parallelization by the recursive decomposition of tasks.
Experimental results show that GMAC is a good and simple choice to support equal
standing VOs in the Internet. The complexity of generating its structure is not affected
by the size of the group, protocol overhead is minimal and failure recovery is fast and
effective.
GMAC does not take additional metrics for optimization purposes. This reduces con-
struction and maintenance costs, since, depending on each VO characteristics, perform-
ing measurements might require more network resources than the communication itself.
Consequently, the underlying network utilization is not efficient. This is one of the main
drawbacks of GMAC, and also of most application level approaches. Nevertheless, the
link stress distribution is balanced in contrast to sequential unicast or centralized ap-
proaches.
Failure recovery in GMAC is fast and simple, and neither extra control links nor ex-
tra control state information needs to be maintained for failure resilience purposes. As
a consequence, GMAC detects and reconnects disconnected nodes immediately, rather
than avoiding disconnections by adding redundancy or complex heuristics that might
increase considerably the protocol overhead.
Since new nodes are connected as leaves of the overlay tree, an interesting emergent
behavior of GMAC is that stable nodes tend to stay in the higher levels of the tree, while
more dynamic nodes stay as leaves. This is a desirable feature, since the overlay structure
128 CHAPTER 8. CONCLUSIONS AND FUTURE WORK
is not affected on leaves’s departures, considering that half of the overlay nodes are leaves
and they have a higher probability of departure.
The following Section presents the contributions of this Thesis. Limitations of the ap-
proach are mentioned in Section 8.2. The possible future work and directions are pre-
sented in Section 8.3, while final remarks are presented in Section 8.4.
8.1 Contributions
This Thesis introduces the following contributions in the context of distributed collabo-
ration:
• A novel overlay network model to support VOs of equal standing members is
proposed. This new approach enables self-organization of VOs with low over-
head,providing a simple yet powerful solution to collaborate in the Internet. The
whole collaboration is achieved without relying on special routers spread across
the Internet or a high capacity server, since the responsibilities for the VO construc-
tion and maintenance, as well as the message propagation schemes, are distributed
among its members in a fair way. The recursive nature of the overlay enables differ-
ent kind of enhancements, such as task paralellization and incremental data sum-
marization.
• Based on this model, a new middleware was developed as a Java library. This
middleware enables equal standing members to collaborate establishing VOs in
the Internet, without requiring significant configuration or deployment efforts, and
achieving scalability and robustness. In addition, CRHs are supported as leaves of
the tree, allowing half of the VO members to be CRHs.
• The GMAC middleware has shown to support adequately the following VO set-
tings:
– MoviLog: Relying on GMAC multicast communication services, the mobile
agent platformMoviLog (Zunino et al., 2005; Mateos et al., 2007) was enhanced
achieving Internet scalability. These autonomous platforms were successfully
supported with the GMAC middleware (Gotthelf et al., 2008b).
– Chatero: Chatero (Gotthelf et al., 2007) is a crisis management synchronous col-
laborative application developed to allow members to chat, draw over fixed
background images and vote for courses of action. Chatero is a good example
of a groupware application that could be supported with GMAC, achieving ro-
bustness and mostly important fast and easy deployment, since in emergency
situations it might be difficult to have a high capacity server timely available.
8.2. LIMITATIONS 129
– Science-Peer: Science-Peer (Gotthelf et al., 2008a) is an asynchronous collab-
orative tool developed for finding scientists with similar research interests.
Relying on GMAC Science-peer leverages group members’ computational re-
sources, such as network links and CPU time.
• Experimental results have shown that the middleware is a practical solution for
supporting VOs in the Internet. Two different simulators were developed allow-
ing to properly evaluate aspects such as end to end propagation times, group la-
tency, broadcast throughput, protocol overhead, failure recovery and link stress for
different VO settings, showing that GMAC is a scalable and robust alternative in
comparison with other approaches.
8.2 Limitations
GMAC is a new approach that enables equal standing members to collaborate in VOs.
However, there are certain aspects which are either not yet supported or do not strictly
adhere to the proposed equal standing VO communication requirements and GMAC
might not be suitable.
• GMAC brings support for dynamic VOs. However, for some particular cases, in
which easily indexable content is shared in a relatively static VO setting, a DHT
overlay network could be more suitable than GMAC.
• GMAC is intended for scalable and easy deployable lightweight communications.
GMAC does not perform any measurements to infer the underlying network, as
these measurements might require more network resources than the collaboration
itself. Therefore, if a VOhas a rather stable member participation and high commu-
nication requirements, such as media streaming, considering optimized strategies
could be advisable, as the overhead introduced by the measurements might be af-
forded.
• The current implementation of GMAC does not consider security issues. In order
to use the GMAC middleware, any user is able to join an already connected VO
node by providing the correct VO name and password. At the present, neither data
nor control links are encrypted, and malicious nodes are not prevented.
• GMAC is not suitable for hard real-time synchronous collaboration. GMAC is
intended to provide support to VOs of a considerable size. Therefore, only
lightweight real-time synchronous collaboration is possible -e.g. Chatero-, when
130 CHAPTER 8. CONCLUSIONS AND FUTURE WORK
the awareness needed helps reducing the required network traffic, since it is usu-
ally inversely related to the number of participants (i.e. a normal user will not be
able to handle for instance 50 other users making telepointers gestures at the same
time).
• Equal standing members are supposed to have similar connection capabilities. In
case an internal node has considerable less network resources than other members,
it could act as a bottleneck. In Section 8.3.6 a way to mitigate this problem will be
explained.
8.3 Future Work
In this Thesis, proposes a novel communication support for VOs. Therefore new possi-
bilities of future investigation arise, as well as optimizations and improvements on the
current implementation.
8.3.1 Security
Although, to reduce the scope of this work, security issues were not undertaken in this
Thesis, this is one of the main missing aspect to be covered. Security issues deserve a deep
study, and many directions could be followed in the future. A first step should be to en-
hance the handshaking joining mechanism, encrypting data and control links. Members
of the VO should be able to detect and restrain malicious or selfish nodes. In distributed
applications, this an important area of research. Different combination of strategies for
authentication (Stallings, 1999), accounting (Osipkov et al., 2006), and trust/reputation
systems (Marti and Garcia-Molina, 2006) could be integrated to GMAC.
8.3.2 GMAC Proxy
If several hosts of the same LAN are willing to join a VO, GMAC could implement a
proxy-node to improve the performance, and thus avoid connecting nodes fromthe same
LAN using external nodes (Figure 8.1). Therefore, the communication among these hosts
can be done through IP multicast, usually supported in LANs. As a consequence, every
local host would reach the VO through the proxy-node and this in turn would retransmit
VO messages to the local members. Nevertheless, relying on IP multicast over LANs
still might cause problems, since, for instance, to support summarizing messages, the
proxy node must handle membership over the LAN hosts and be able to ensemble the
summarizing answers. A simpler solution could be to reproduce a GMAC tree inside the
LAN keeping LAN hosts adjacent to each other, and thus avoiding these problems.
8.3. FUTURE WORK 131
GMAC proxy
LAN
Figure 8.1: GMAC proxy
8.3.3 Heartbeat messages
At the stage of writing this Thesis, a special case of summarizing messages are being in-
troduced to GMAC. These messages are time-triggered and flowin an ascending manner,
allowing the summarization activities to take place in each node. This kind of messages
are very useful for monitoring purposes, allowing to obtain information, such best or
most suitable resources, averages, etc., that summarizes the dynamic state of the partici-
pating VO members.
a2
a1
(n1AVG*n1P + n2AVG*n2P + n3AVG)
(n1P+n2P+1)
n1 n2
n3
a3
Resource Manager
AVG CPU=0.322; 1
AMem = n1, 1,706,063
AMem = −
AMem = −
...
AVG CPU=0.612 ; 1
AMem = n2, 1,306,279
AMem = −
AMem = −
...
AVG CPU=0.41066 ; 3
AMem = n1, 1,706,063
AMem = n3, 1,508,841
AMem = n2, 1,306,279
...
n4
AVG CPU=0.812 ; 1
AMem = n4, 0,706,279
AMem = −
AMem = −
...
AVG CPU=0.612
AMEM=1,306,279 AVG CPU=0.322
AMEM=1,706,063
AVG CPU=0.812
AMEM=0,706,279
AVG CPU=0.298
AMEM=1,508,841
(0.322*1 + 0.612*1 + 0.298)
(1+1+1)
n3 AVG CPU=
n3 AVG CPU=
= 0.41066
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.5242 ; 14
AMem = n9, 2,032,223
AMem = n1, 1,706,063
AMem = n3, 1,508,841
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
n9
AVG CPU=0.454
AMEM=2,032,223
Sumarized Data:
Average CPU Load and
three Volunteers with more memory available
n9: volunteer with
more memory available
a1
a2
a3
AVG CPU=0.x ; 1
AMem = nx, xxxx
AMem = −
AMem = −
...
Figure 8.2: Heartbeat summarizing messages example
An illustrative example is shown in Figure 8.2, showing how the heartbeat mechanism
132 CHAPTER 8. CONCLUSIONS AND FUTURE WORK
works in a VO formed by desktop grid volunteers -i.e. desktop computers sharing CPU
resources- to obtain from the whole VO the average CPU load and the three machines
with more memory available.
In this way, each node has the information that summarizes the branch they represent.
Thus, the information gathered by the root, which represents the whole VO status, could
be sent to a resource manager component. The whole computation is equally distributed
among the members. Similar to a summarizing query, but including the expiration time,
heartbeat queries are multicasted. Subsequently, leaves of the tree periodically send their
heartbeat answers to their parents. Accordingly, each volunteer contributes providing
network bandwidth and processing power, following an incremental solving scheme.
If a node fails, although the information that a node and branch represents would be lost,
the mechanismwould be still completely functional. The parent node would stop waiting
for the message from that child and continue to propagate the information. As soon as
the orphan children are reconnected they will continue to provide the information in
future heartbeat messages. Unlike summarizing messages, heartbeat messages requests
prevail on time. Therefore, when a node arrives, similar to welcome messages, the active
heartbeat request are transferred to the arriving node.
8.3.4 Indexable Content
GMAC is designed to support dynamism in both VO members and their resources.
Nonetheless, in some cases, VO members could be interested in sharing static and eas-
ily indexable content such as files. GMAC is not suited to provide this kind of services,
since, even though queries are distributed in a logarithmic delivery time with respect to
the number of members, every VO member would be ask for the specified file. GMAC
could implement a DHT based lookup service in the future, however there are many
P2P alternatives capable of providing this type of service adequately (Stoica et al., 2003;
Ratnasamy et al., 2001; Maymounkov and Mazières, 2002).
8.3.5 End-to-end Reliability
TCP connection links in the overlay network do not ensure reliability (Amir and Danilov,
2003). When a host in the VO fails, the data it was supposed to forward may not be de-
livered. The same happens on the subtrees that get temporarily disconnected. A first
attempt to solve this problem could be to store undelivered messages in buffers every
time an adjacent node fails. Next, the reconnected subtree would exchange missing mes-
sages after contacting the nodes that stored messages. However, this might generate
many replicated messages and a considerable overhead. All the same, for the supported
8.4. FINAL REMARKS 133
VOs described in Chapter 6, welcome messages were used to inform the reconnected
nodes which was the current VO state.
8.3.6 Optimization
GMAC does not implement any kind of optimization on its structure. Nevertheless by
making some node swaps it would be possible to improve the global service perfor-
mance, always preserving the binary overlay structure introduced by the model. Al-
though equal standing members should have similar connection capabilities, bottlenecks
might arise in case the connection capabilities of some internal nodes are comparatively
small. In such cases, bottleneck nodes could be detected and moved as leaves of the tree,
so as to prevent them from forwarding messages.
8.4 Final Remarks
The Internet brings new collaboration possibilities. The context that enables temporary
coalitions of independent, self-organized individuals and/or institutions is referred as
VO. VO are being adopted by different research communities, such as Grid computing,
multi-agent systems and collaborative applications to shift from local stable configura-
tions to large scale dynamic self-organizing group conformations in the Internet.
When the collaboration is driven by equal standing members, no entity should be in
charge of regulating the VO, since control must be shared through different autonomic
administrative domains.
Many approaches have been developed to provide communication support to distributed
entities. ALM alternatives are focus in broadcasting services, while P2P systems are
mostly committed to provide indexable content lookups. VOs stand somewhere in be-
tween. The interaction schemes in ALM alternatives are intended for a reduced number
of members exchanging considerable traffic, such as live audio/video conferencing ap-
plications, while P2P systems are mostly intended to support sporadic lookup queries to
millions of users, such as file sharing. Consequently, ALM alternatives and current P2P
systems either lack the proper scalability or are unable to properly address the implicit
Internet dynamism.
Accordingly, this Thesis proposes a novel communication support for VOs, enabling au-
tonomous entities to collaborate in a fair, inexpensive, robust, fast and easy deployable
way, achieving membership management and logarithmic message distribution times
relative to the number of members.
134 CHAPTER 8. CONCLUSIONS AND FUTURE WORK
Bibliography
K. Aberer, A. Datta, and M. Hauswirth. P-grid: Dynamics of self-organizing processes
in structured peer-to-peer systems. In R. Steinmetz and K. Wehrle, editors, Peer-to-
Peer Systems and Applications, volume 3485 of Lecture Notes in Computer Science, pages
137–153. Springer, 2005. ISBN 3-540-29192-X.
Y. Amir and C. Danilov. Reliable communication in overlay networks. In Proc. of the
IEEE International Conference on Dependable Systems and Networks (DSN03), Los Alami-
tos, CA, USA, 2003. IEEE Computer Society. ISBN 0-7695-1952-0. doi: 10.1109/DSN.
2003.1209961.
S. Androutsellis-Theotokis and D. Spinellis. Asurvey of peer-to-peer content distribution
technologies. ACM Comput. Surv., 36(4):335–371, 2004. ISSN 0360-0300. doi: 10.1145/
1041680.1041681.
A. Andrzejak and Z. Xu. Scalable, efficient range queries for grid information services.
In R. L. Graham and N. Shahmehri, editors, Peer-to-Peer Computing, pages 33–40. IEEE
Computer Society, 2002. ISBN 0-7695-1810-9. URL http://csdl.computer.org/comp/
proceedings/p2p/2002/1810/00/18100033abs.htm.
G. Antoniu, L. Cudennec, M. Jan, and M. Duigou. Performance scalability of the JXTA
P2P framework. In Parallel and Distributed Processing Symposium, 2007. IPDPS 2007.
IEEE International, pages 1–10, 2007.
R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley, 1999.
B. Ban. Design and implementation of a reliable group communication toolkit for Java.
Technical report, Cornell University, Sept. 1998.
S. Banerjee, B. Bhattacharjee, and C. Kommareddy. Scalable application layer multicast.
In SIGCOMM ’02: Proc. of the 2002 conference on Applications, technologies, architectures,
and protocols for computer communications, pages 205–217. ACM Press, 2002. ISBN 1-
58113-570-X. doi: 10.1145/633025.633045.
135
136 BIBLIOGRAPHY
L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and zipf-like distribu-
tions: Evidence and implications. In Proceedings of Infocom’99 (New York, March 21-25,
1999), pages 126–134, Piscataway, NJ, 1999. IEEE Operations Center.
M. Cai, M. R. Frank, J. Chen, and P. A. Szekely. MAAN: A multi-attribute addressable
network for grid information services. J. Grid Comput, 2(1):3–14, 2004. URL http:
//www.springerlink.com/index/10.1007/s10723-004-1184-y.
J. Callahan, T. Montgomery, and B. Whetten. High-performance, reliable multicasting
foundations for future internet groupware applications. Technocal report, NASAIV&V
Facility, Fairmont, West Virginia, 1997.
K. Calvert, M. Doar, and E. Zegura. Modeling internet topology. 35(6):160–163, 1997.
ISSN 0163-6804. doi: 10.1109/35.587723.
L. Camarinha-Matos. Virtual Organizations: Systems and Practices. Springer-Verlag Telos,
2004. ISBN 0387237550.
A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Design and evaluation of a wide-area
event notification service. ACM Trans. Comput. Syst, 19(3):332–383, 2001. doi: 10.1145/
380749.380767.
T. D. Chandra, V. Hadzilacos, S. Toueg, and B. Charron-Bost. On the impossibility of
group membership. In PODC ’96: Proceedings of the fifteenth annual ACM symposium
on Principles of distributed computing, pages 322–330, New York, NY, USA, 1996. ACM
Press. ISBN 0-89791-800-2. doi: 10.1145/248052.248120.
Y. Chawathe. Scattercast: an adaptable broadcast distribution framework. Multimedia
Systems, 9(1):104–118, July 2003. ISSN 0942-4962. doi: 10.1007/s00530-002-0082-z.
J. R. Chen, S. R. Wolfe, and S. D. Wragg. A distributed multi-agent system for collab-
orative information management and sharing. In CIKM ’00: Proceedings of the ninth
international conference on Information and knowledge management, pages 382–388, New
York, NY, USA, 2000. ACM. ISBN 1-58113-320-0. doi: 10.1145/354756.354844.
M. Chetty and R. Buyya. Weaving computational Grids: How analogous are they with
electrical grids? Computing in Science and Engineering, 4(4):61–71, July/Aug. 2002. ISSN
1521-9615. URL http://csdl.computer.org/dl/mags/cs/2002/04/c4061.pdf.
Y.-H. Chu, S. G. Rao, and H. Zhang. A case for end system multicast. IEEE J. Sel. Areas
Commun., 2002. URL citeseer.ist.psu.edu/chu01case.html.
A. Crespo and H. Garcia-Molina. Routing indices for peer-to-peer systems. In 22nd In-
ternational Conference on Distributed Computing Systems (ICDCS ’02), pages 23–34, Wash-
ington - Brussels - Tokyo, July 2002. IEEE. ISBN 0-7695-1585-1.
BIBLIOGRAPHY 137
K. Czajkowski, C. Kesselman, S. Fitzgerald, and I. T. Foster. Grid information services for
distributed resource sharing. In HPDC, pages 181–194. IEEE Computer Society, 2001.
ISBN 0-7695-1296-8.
F. Dabek, M. F. Kaashoek, D. R. Karger, R. Morris, and I. Stoica. Wide-area cooperative
storage with CFS. In SOSP, pages 202–215, 2001. doi: 10.1145/502034.502054.
V. Darlagiannis. Hybrid peer-to-peer systems. In R. Steinmetz and K. Wehrle, editors,
Peer-to-Peer Systems and Applications, volume 3485 of Lecture Notes in Computer Science,
pages 353–366. Springer, 2005. ISBN 3-540-29192-X.
S. E. Deering and D. R. Cheriton. Multicast routing in datagram internetworks and
extended LANs. ACM Trans. Comput. Syst., 8(2):85–110, 1990. ISSN 0734-2071. doi:
10.1145/78952.78953.
G. Desanctis and P. Monge. Introduction to the special issue: Communication processes
for virtual organizations. Organization Science, 10(6):693–703, 1999. ISSN 1526-5455.
C. Diot, B. N. Levine, B. Lyles, H. Kassem, and D. Balensiefen. Deployment issues for the
IP multicast service and architecture. IEEE Network, 14(1):78–88, Jan. / Feb. 2000.
L. Divac-Krnic and R. Ackermann. Security-related issues in peer-to-peer networks. In
Peer-to-Peer Systems and Applications, pages 529–545, 2005. doi: 10.1007/11530657_31.
P. Dourish. The parting of the ways: Divergence, data management and collaborative
work. In European Conference on Computer Supported Cooperative Work ECSCW’95, Sept.
1995. URL ftp://parcftp.xerox.com/pub/europarc/jpd/ecscw95-divergence.ps.
J. Eberspächer and R. Schollmeier. Past and future. In Peer-to-Peer Systems and Applica-
tions, pages 17–23. 2005.
S. F. Ehrlich, T. Bikson, W. Mackay, and J. C. Tang. Tools for supporting cooperative work
near and far: highlights from the CSCW conference. In Proc. of the ACM Human Factors
in Computing Systems Conference, pages 353–356. ACM Press, 1989. ISBN 0-89791-301-9.
doi: 10.1145/67449.67517.
C. Eikemeier and U. Lechner. Peer-to-peer and group collaboration - do they always
match? In Proc. of the 13
th
WETICE, pages 101–106, 2004. ISBN 0-7695-2183-5. doi:
10.1109/ENABL.2004.49.
C. A. Ellis, S. J. Gibbs, and G. Rein. Groupware: some issues and experiences. Commun.
ACM, 34(1):39–58, 1991. ISSN 0001-0782. doi: 10.1145/99977.99987.
H. Eriksson. MBONE: the multicast backbone. Commun. ACM, 37(8):54–60, Aug. 1994.
ISSN 0001-0782.
138 BIBLIOGRAPHY
H. Fisher. Multicast issues for collaborative virtual environments. IEEE Computer Graphics
and Applications, 22(5):68–75, 2002. ISSN 0272-1716. doi: 10.1109/MCG.2002.1028728.
B. Ford, P. Srisuresh, and D. Kegel. Peer-to-peer communication across network ad-
dress translators. In USENIX Annual Technical Conference, General Track, pages 179–192.
USENIX, 2005.
I. Foster, C. Kesselman, and S. Tuecke. The anatomy of the Grid: Enabling scalable virtual
organization. The International Journal of High Performance Computing Applications, 15(3):
200–222, Fall 2001. ISSN 1094-3420.
I. T. Foster and A. Iamnitchi. On death, taxes, and the convergence of peer-to-peer and
grid computing. In M. F. Kaashoek and I. Stoica, editors, IPTPS, volume 2735 of Lecture
Notes in Computer Science, pages 118–128. Springer, 2003. ISBN 3-540-40724-3.
P. Francis. Yoid: Extending the internet multicast architecture. Technical report, AT&T
Center for Internet Research at ICSI (ACIRI), 2000.
P. Francis. Is the internet going NUTSS? IEEE Internet Computing, 7(6):94–96, 2003.
C. Gkantsidis, M. Mihail, and A. Saberi. Random walks in peer-to-peer networks: Algo-
rithms and evaluation. Perform. Eval, 63(3):241–263, 2006. doi: 10.1016/j.peva.2005.01.
002.
R. A. Golding and K. Taylor. Group membership in the epidemic style. Technical report,
University of California at Santa Cruz, Santa Cruz, CA, USA, 1992.
L. Gong. Jxta: a network programming environment. Internet Computing, IEEE, 5(3):
88–95, 2001. ISSN 1089-7801.
P. Gotthelf, M. Mendoza, A. Zunino, and C. Mateos. GMAC: An Overlay Multicast Net-
work for Mobile Agents. In Proc. of the VI Argentine Symposium on Computing Technology
(AST 2005 - 34 JAIIO), 2005.
P. Gotthelf, A. Zunino, and M. Campo. A decentralized middleware for groupware ap-
plications. In J. M. Haake, S. F. Ochoa, and A. Cechich, editors, Groupware: Design, Im-
plementation, and Use, volume 4715 of Lecture Notes in Computer Science, pages 191–206.
Springer-Verlag, 2007. ISBN 978-3-540-74811-3. doi: 10.1007/978-3-540-74812-0_15.
P. Gotthelf, A. Zunino, and M. Campo. A peer-to-peer communication infrastructure for
groupware applications. International Journal of Cooperative Information Systems, 17(4):
523–554, December 2008a. doi: 10.1142/S0218843008001920.
P. Gotthelf, A. Zunino, C. Mateos, and M. Campo. Gmac: An overlay multicast network
for mobile agent platforms. Journal of Parallel and Distributed Computing, 68(8):1081–
1096, 2008b. ISSN 0743-7315. doi: 10.1016/j.jpdc.2008.04.002.
BIBLIOGRAPHY 139
C. Gutwin. The effects of network delays on group work in real-time groupware. In
ECSCW’01: Proceedings of the seventh conference on European Conference on Computer Sup-
ported Cooperative Work, pages 299–318, Norwell, MA, USA, 2001. Kluwer Academic
Publishers. ISBN 0-7923-7162-3.
C. Gutwin and S. Greenberg. The effects of workspace awareness support on the usability
of real-time distributed groupware. ACM Trans. Comput.-Hum. Interact., 6(3):243–281,
1999. ISSN 1073-0516. doi: 10.1145/329693.329696.
C. Gutwin, J. Dyck, and J. Burkitt. Using cursor prediction to smooth telepointer jitter. In
GROUP ’03: Proceedings of the 2003 international ACM SIGGROUP conference on Support-
ing group work, pages 294–301, New York, NY, USA, 2003. ACM. ISBN 1-58113-693-5.
doi: 10.1145/958160.958207.
J. M. Haake, A. Haake, T. Schümmer, M. Bourimi, and B. Landgraf. End-user controlled
group formation and access rights management in a shared workspace system. In
CSCW ’04: Proceedings of the 2004 ACM conference on Computer supported cooperative work,
pages 554–563, New York, NY, USA, 2004. ACM. ISBN 1-58113-810-5. doi: 10.1145/
1031607.1031702.
M. Harchol-Balter, T. Leighton, and D. Lewin. Resource discovery in distributed net-
works. In PODC ’99: Proceedings of the eighteenth annual ACM symposium on Principles of
distributed computing, pages 229–237, New York, NY, USA, 1999. ACM. ISBN 1-58113-
099-6. doi: 10.1145/301308.301362.
D. A. Helder and S. Jamin. End-host multicast communication using switch-trees proto-
cols. Cluster Computing and the Grid, 2002. 2nd IEEE/ACM International Symposium on,
pages 419–419, 21-24 May 2002. doi: 10.1109/CCGRID.2002.1017172.
M. Hillenbrand and P. Müller. Web services and peer-to-peer. In R. Steinmetz and
K. Wehrle, editors, Peer-to-Peer Systems and Applications, volume 3485 of Lecture Notes
in Computer Science, pages 207–224. Springer, 2005. ISBN 3-540-29192-X.
A. Iamnitchi and I. T. Foster. On fully decentralized resource discovery in grid environ-
ments. In GRID ’01: Proceedings of the Second International Workshop on Grid Computing,
pages 51–62, London, UK, 2001. Springer-Verlag. ISBN 3-540-42949-2.
A. Iamnitchi and M. Ripeanu. Myth and reality: Usage behavior in a large data-intensive
physics project. In Technical Report GriPhyN, November 2003. URL http://citeseer.
ist.psu.edu/iamnitchi02myth.html.
A. Iamnitchi and D. Talia. P2P computing and interaction with grids. Future Generation
Computer Systems, 21(3):331–332, Mar. 2005. ISSN 0167-739X.
140 BIBLIOGRAPHY
A. Iamnitchi, I. Foster, and D. C. Nurmi. A peer-to-peer approach to resource location in
grid environments. In HPDC ’02: Proceedings of the 11th IEEE International Symposium
on High Performance Distributed Computing, page 419, Washington, DC, USA, 2002. IEEE
Computer Society. ISBN 0-7695-1686-6.
A. I. Iamnitchi. Resource discovery in large resource-sharing environments. PhD thesis, The
University of Chicago, 2003. Adviser-Ian Foster.
J. Jannotti, D. K. Gifford, K. L. Johnson, M. F. Kaashoek, and J. W. O’Toole, Jr. Overcast:
Reliable Multicasting with an Overlay Network. In Proc. of the 4
th
USENIX sympo-
sium on Operating Systems Design and Implementation, Berkeley, CA, USA, Oct. 2000.
USENIX. ISBN 1-880446-16-2. URL http://www.usenix.org/publications/library/
proceedings/osdi2000/jannotti.html.
M. Jeronimo and J. Weast. UPnP Design by Example: A Software Developer’s Guide to Uni-
versal Plug and Play. Intel Press, 2003. ISBN 0971786119.
S. Jin and A. Bestavros. Small-World Internet Topologies: Possible Causes and Implica-
tions on Scalability of End-System Multicast. Computer Networks, 50(6):648–666, 2006.
doi: 10.1016/j.comnet.2005.04.016.
S. Kandula, J.-K. Lee, and J. C. Hou. Lark: a light-weight, resilient application-level
multicast protocol. In IEEE 18 Annual Workshop on computer Communications. IEEE, Oct.
2003. URL http://ieeexplore.ieee.org/iel5/8786/27817/01240811.pdf.
M. M. Kessler. Bibliographic coupling between scientific papers. American Documentation,
14:10–25, 1963.
D. Kosti´ c, A. C. Snoeren, A. Vahdat, R. Braud, C. Killian, J. W. Anderson, J. Albrecht,
A. Rodriguez, and E. Vandekieft. High-bandwidth data dissemination for large-scale
distributed systems. ACM Trans. Comput. Syst., 26(1):1–61, 2008. ISSN 0734-2071. doi:
10.1145/1328671.1328674.
J. Kubiatowicz, D. Bindel, Y. Chen, S. E. Czerwinski, P. R. Eaton, D. Geels, R. Gummadi,
S. C. Rhea, H. Weatherspoon, W. Weimer, C. Wells, and B. Y. Zhao. Oceanstore: An
architecture for global-scale persistent storage. In ASPLOS, pages 190–201, 2000. doi:
10.1145/356989.357007.
R. Kurmanowytsch, E. Kirda, C. Kerer, and S. Dustdar. OMNIX: Atopology-independent
P2P middleware. In J. Eder, R. Mittermeir, and B. Pernici, editors, CAiSE Work-
shops, volume 75 of CEUR Workshop Proceedings. CEUR-WS.org, 2003. ISBN 86-435-
0552-8. URL http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS/
Vol-75/files/UMICS_04.pdf.
BIBLIOGRAPHY 141
M. Kwon and S. Fahmy. Topology-aware overlay networks for group communication.
In NOSSDAV ’02: Proceedings of the 12th international workshop on Network and operating
systems support for digital audio and video, pages 127–136, New York, NY, USA, 2002.
ACM. ISBN 1-58113-512-2. doi: 10.1145/507670.507688.
A. Lakhina, J. Byers, M. Crovella, and I. Matta. On the geographic location of internet
resources. 21(6):934–948, 2003. ISSN 0733-8716. doi: 10.1109/JSAC.2003.814667.
Liu, Liu, Xiao, Ni, and Zhang. Location-aware topology matching in P2P systems. In
INFOCOM: The Conference on Computer Communications, joint conference of the IEEE Com-
puter and Communications Societies, volume 4, pages 2220–2230, March 2004.
Q. Lv, P. Cao, E. Cohen, K. Li, and S. Shenker. Search and replication in unstructured peer-
to-peer networks. In Proceedings of the 16th International Conference on Supercomputing
(ICS-02), pages 84–95, New York, June 22–26 2002. ACM Press.
J. Ma, M. Shizuka, J. Lee, and R. Huang. A P2P groupware system with decentralized
topology for supporting synchronous collaborations. In Proc. of the International Con-
ference on Cyberworlds, page 54, Washington, DC, USA, 2003. IEEE Computer Society.
ISBN 0-7695-1922-9.
D. Malkhi, M. Naor, and D. Ratajczak. Viceroy: a scalable and dynamic emulation of the
butterfly. In PODC ’02: Proceedings of the twenty-first annual symposium on Principles of
distributed computing, pages 183–192, New York, NY, USA, 2002. ACM. ISBN 1-58113-
485-1. doi: 10.1145/571825.571857.
T. W. Malone. Organizing information processing systems: parallels between human
organizations and computer systems. In Cognition, computing, and cooperation, pages
56–83. Ablex Publishing Corp., Norwood, NJ, USA, 1990. ISBN 0-89391-536-X.
T. W. Malone and K. Crowston. The interdisciplinary study of coordination. ACM Com-
put. Surv., 26(1):87–119, 1994. ISSN 0360-0300. doi: 10.1145/174666.174668.
G. S. Manku, M. Bawa, and P. Raghavan. Symphony: Distributed hashing in a small
world. In USENIX Symposium on Internet Technologies and Systems, 2003. URL http:
//www.usenix.org/events/usits03/tech/manku.html.
S. Marti and H. Garcia-Molina. Taxonomy of trust: Categorizing P2P reputation systems.
Computer Networks, 50(4):472–484, 2006. doi: 10.1016/j.comnet.2005.07.011.
C. Mastroianni, D. Talia, and O. Verta. Asuper-peer model for resource discovery services
in large-scale grids. Future Generation Comp. Syst, 21(8):1235–1248, 2005. doi: 10.1016/
j.future.2005.06.001.
142 BIBLIOGRAPHY
C. Mateos, A. Zunino, and M. Campo. Extending movilog for supporting web services.
Computer Languages, Systems & Structures, 31(1):11–31, 2007. ISSN 0096-0551.
A. Mauthe and O. Heckmann. Distributed computing - GRIDcomputing. In R. Steinmetz
and K. Wehrle, editors, Peer-to-Peer Systems and Applications, volume 3485 of Lecture
Notes in Computer Science, pages 193–206. Springer, 2005. ISBN 3-540-29192-X.
P. Maymounkov and D. Mazières. Kademlia: A peer-to-peer information system based
on the XOR metric. In P. Druschel, M. F. Kaashoek, and A. I. T. Rowstron, edi-
tors, IPTPS, volume 2429 of Lecture Notes in Computer Science, pages 53–65. Springer,
2002. ISBN 3-540-44179-4. URL http://link.springer.de/link/service/series/
0558/bibs/2429/24290053.htm.
A. Medina, I. Matta, and J. Byers. On the origin of power laws in internet topologies.
SIGCOMM Comput. Commun. Rev., 30(2):18–28, 2000. ISSN 0146-4833. doi: 10.1145/
505680.505683.
A. Medina, A. Lakhina, I. Matta, and J. Byers. BRITE: An approach to universal topology
generation. In MASCOTS ’01: Proceedings of the Ninth International Symposium in Model-
ing, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS’01),
page 346, Washington, DC, USA, 2001. IEEE Computer Society.
A. Meissner, L. Wolf, W. Schoenfeld, and R. Steinmetz. A framework for group integrity
management in multimedia multicasting. In Euromicro Conference, 2001. Proceedings.
27th, pages 346–353, 2001.
J. Mischke and B. Stiller. A methodology for the design of distributed search in P2P
middleware. IEEE Network, 18(1):30–37, 2004.
B. E. Munkvold, K. Eim, and Ø. S. Husby. A case study of information systems decision-
making: Process characteristics and collaboration technology support. Int. J. Coopera-
tive Inf. Syst, 15(2):179–204, 2006. doi: 10.1142/S0218843006001335.
M. R. Nami and M. Sharifi. Autonomic computing: A new approach. ams, 0:352–357,
2007. doi: 10.1109/AMS.2007.20.
W. Nejdl, B. Wolf, C. Qu, S. Decker, M. Sintek, A. Naeve, M. Nilsson, M. Palmér, and
T. Risch. Edutella: a p2p networking infrastructure based on rdf. In Proc. of the 11
th
international conference on WWW, pages 604–615, NewYork, NY, USA, 2002. ACMPress.
ISBN 1-58113-449-5. doi: 10.1145/511446.511525.
S. F. Ochoa, A. Neyem, J. A. Pino, and M. R. S. Borges. Supporting group decision mak-
ing and coordination in urban disasters relief efforts. International Journal of Decision
Systems, 16(2), 2007.
BIBLIOGRAPHY 143
A. Oram, editor. Peer-to-peer: Harnessing the Power of Disruptive Technologies. O’Reilly,
Sebastopol, California, 2001.
Y. Osais, S. Abdala, and A. Matrawy. Amultilayer peer-to-peer framework for distributed
synchronous collaboration. Internet Computing, IEEE, 10(6):33–41, 2006. ISSN 1089-
7801.
I. Osipkov, P. Wang, and N. Hopper. Robust accounting in decentralized P2P storage
systems. In 26th IEEE International Conference on Distributed Computing Systems (26th
ICDCS’2006), page 14, Lisboa, Portugal, July 2006. IEEE Computer Society.
P. Parnes, K. Synnes, D. Schefström, P. Parnes, K. Synnes, and D. Schefström. The
cdt mstar environment: scalable distributed teamwork in action. In GROUP ’97:
Proceedings of the international ACM SIGGROUP conference on Supporting group work,
pages 167–176, New York, NY, USA, 1997. ACM Press. ISBN 0-89791-897-5. doi:
10.1145/266838.266894.
D. Pendarakis, S. Shi, D. Verma, and M. Waldvogel. ALMI: An application level multicast
infrastructure. In Proc. of the 3
rd
USENIXSymposiumon Internet Technologies and Systems.
USENIX, Mar. 2001. ISBN1-880446-14-6. URL http://www.usenix.org/publications/
library/proceedings/usits01/shi.html.
G. Pierre. How cn and p2p technologies may help build next-generation grids. In UP-
GRADE ’07: Proceedings of the second workshop on Use of P2P, GRID and agents for the
development of content networks, pages 25–28, New York, NY, USA, 2007. ACM Press.
ISBN 978-1-59593-718-6. doi: 10.1145/1272980.1272982.
N. M. Preguiça, J. L. Martins, H. J. L. Domingos, and S. Duarte. Supporting multi-
synchronous groupware: Data management problems and a solution. Int. J. Cooperative
Inf. Syst, 15(2):229–258, 2006. doi: 10.1142/S0218843006001359.
R. Raman, M. Livny, and M. H. Solomon. Matchmaking: An extensible framework for
distributed resource management. Cluster Computing, 2(2):129–138, 1999.
S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Schenker. A scalable content-
addressable network. In SIGCOMM ’01: Proceedings of the 2001 conference on Appli-
cations, technologies, architectures, and protocols for computer communications, pages 161–
172, New York, NY, USA, 2001. ACM Press. ISBN 1-58113-411-8. doi: 10.1145/383059.
383072.
S. Ratnasamy, M. Handley, R. M. Karp, and S. Shenker. Topologically-aware overlay
construction and server selection. In Proceedings of the 21st Annual Joint Conference of
the IEEE Computer and Communications Society (INFOCOM-02), volume 3 of Proceedings
144 BIBLIOGRAPHY
IEEE INFOCOM 2002, pages 1190–1199, Piscataway, NJ, USA, June 23–27 2002. IEEE
Computer Society.
S. Ren, L. Guo, S. Jiang, and X. Zhang. Sat-match: A self-adaptive topology matching
method to achieve low lookup latency in structured p2p overlay networks. ipdps, 01:
83a, 2004. doi: 10.1109/IPDPS.2004.1303022.
M. Ripeanu, A. Iamnitchi, and I. Foster. Mapping the gnutella network. IEEE Internet
Computing, 6(1):50–57, 2002. ISSN 1089-7801. doi: 10.1109/4236.978369.
J. Rosenberg, J. Weinberger, C. Huitema, and R. Mahy. STUN – Simple Traversal of User
Datagram Protocol (UDP) through Network Address Translators (NATs). Internet En-
gineering Task Force: RFC 3489, March 2003.
A. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location and routing
for large-scale peer-to-peer systems. In Proc. of the 18
th
IFIP/ACM Int. Conference on
Distributed Systems Platforms, volume 2218 of LNCS, pages 329–350. Springer-Verlag,
Nov. 2001a. URL http://www.research.microsoft.com/~antr/PAST/pastry.pdf.
A. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location, and routing
for large-scale peer-to-peer systems. In Proceedings of the 18
th
IFIP/ACM International
Conference on Distributed Systems Platforms (Middleware), volume 2218 of Lecture Notes
In Computer Science, pages 329–350. Springer-Verlag, Nov. 2001b.
M. Scalem, S. Bandyopadhyay, and A. K. Sircar. An approach towards a decentralised
disaster management information network. In S. Manandhar, J. Austin, U. B. Desai,
Y. Oyanagi, and A. K. Talukder, editors, Applied Computing, Second Asian Applied Com-
puting Conference, AACC 2004, volume 3285 of Lecture Notes in Computer Science, pages
287–295, Kathmandu, Nepal, 2004. Springer.
F. R. Shapiro. Origins of bibliometrics, citation indexing, and citation analysis: The ne-
glected legal literature. JASIS, 43(5):337–339, 1992.
W. Stallings. The HMAC algorithm: Key hashing for message authentication. Dr. Dobbs
Journal, 24(4):46, 48–49, Apr. 1999. ISSN 1044-789X. URL http://www.ddj.com/ftp/
1999/1999_04/hmac.txt.
R. Steinmetz and K. Wehrle. What is this "peer-to-peer" about? In R. Steinmetz and
K. Wehrle, editors, Peer-to-Peer Systems and Applications, volume 3485 of Lecture Notes
in Computer Science, pages 9–16. Springer, 2005. ISBN 3-540-29192-X.
I. Stoica, T. S. E. Ng, and H. Zhang. REUNITE: A Recursive Unicast Approach to Multi-
cast. In Proc. of INFOCOM00, pages 1644–1653, Los Alamitos, Mar. 26–30 2000. IEEE.
BIBLIOGRAPHY 145
I. Stoica, R. Morris, D. R. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable
peer-to-peer lookup service for internet applications. In SIGCOMM, pages 149–160,
2001. doi: 10.1145/383059.383071.
I. Stoica, R. Morris, D. Liben-Nowell, D. R. Karger, M. F. Kaashoek, F. Dabek, and H. Bal-
akrishnan. Chord: A scalable peer-to-peer lookup service for internet applications.
IEEE/ACM Transactions on Networking, 11(1):17–32, Feb. 2003. doi: 10.1109/TNET.2002.
808407.
D. Talia and P. Trunfio. Toward a synergy between p2p and grids. IEEE Internet Comput-
ing, 7(4):94–96, July 2003. doi: 10.1109/MIC.2003.1215667.
D. Talia and P. Trunfio. Web services for peer-to-peer resource discovery on the grid. In
M. Agosti, H.-J. Schek, and C. Türker, editors, DELOS Workshop: Digital Library Archi-
tectures, pages 73–84. Edizioni Libreria Progetto, Padova, 2004.
I. J. Taylor. From P2P to Web Services and Grids: Peers in a Client/Server World. Springer,
2004. ISBN ISBN: 1-85233-869-5. URL http://p2pgridbook.com/.
C. Thorpe, J.; Albrecht. Characteristics of large group support systems. System Sciences,
2004. Proceedings of the 37th Annual Hawaii International Conference on, pages 8 pp.–, 5-8
Jan. 2004. doi: 10.1109/HICSS.2004.1265152.
J. D. Touch. Overlay networks. Computer Networks, 36(2/3):115–116, 2001. doi: 10.1016/
S1389-1286(01)00171-2.
B. Traversat, M. Abdelaziz, M. Duigou, J. C. Hugly, E. Pouyoul, and B. Yeager. Project
JXTA virtual network, February 2002. URL http://www.jxta.org/project/www/docs/
JXTAprotocols.pdf.
UPnP. The universal plug and play. URL http://www.upnp.org.
J. Vassileva. Harnessing p2p power in the classroom. In Proc. of Intelligent Tutoring Sys-
tems, pages 305–314, 2004.
W. Wang, H. Chang, A. Zeitoun, and S. Jamin. Characterizing guarded hosts in peer-to-
peer file sharing systems. In In Proc. of IEEE Global Communications Conference, Global
Internet and Next Generation Networks, 2004.
W. Wang, C. Jin, and S. Jamin. Network overlay construction under limited end-
to-end addressability. In Proc. of the IEEE 24
th
Annual Joint Conference of the IEEE
Computer and Communications Societies, volume 3, pages 2124–2134. IEEE, Mar. 2005.
doi: 10.1109/INFCOM.2005.1498488. URL http://citeseer.ist.psu.edu/657376.
html;http://www.eecs.umich.edu/techreports/cse/2004/CSE-TR-489-04.pdf.
146 BIBLIOGRAPHY
B. Waxman. Routing of multipoint connections. Selected Areas in Communications, IEEE
Journal on, 6(9):1617–1622, Dec 1988. ISSN 0733-8716. doi: 10.1109/49.12889.
B. C. Wheeler, A. R. Dennis, and L. I. Press. Groupware comes to the internet: charting a
new world. SIGMIS Database, 30(3-4):8–21, 1999. ISSN 0095-0033. doi: 10.1145/344241.
344242.
A. Young, J. Chen, Z. Ma, A. Krishnamurthy, L. Peterson, and R. Wang. Overlay mesh
construction using interleaved spanning trees. INFOCOM 2004. Twenty-third Annu-
alJoint Conference of the IEEE Computer and Communications Societies, 1:–407, 7-11 March
2004. ISSN 0743-166X. doi: 10.1109/INFCOM.2004.1354512.
D. Zeinalipour-Yazti and V. Kalogeraki. Structuring topologically aware overlay net-
works using domain names. Computer Networks, 50(16):3064–3082, 2006. doi: 10.1016/
j.comnet.2005.12.003.
B. Zhang, W. Wang, S. Jamin, D. Massey, and L. Zhang. Universal ip multicast delivery.
Computer Networks, 50(6):781–806, Apr. 2006. doi: 10.1016/j.comnet.2005.07.016.
B. Y. Zhao, L. Huang, J. Stribling, S. C. Rhea, A. D. Joseph, and J. D. Kubiatowicz.
Tapestry: A resilient global-scale overlay for service deployment. IEEE J. Sel. Areas
Commun., 22(1):41–53, Jan. 2004. doi: 10.1109/JSAC.2003.818784.
A. Zunino, M. Campo, and C. Mateos. Simplifying Mobile Agent Development Through
Reactive Mobility by Failure. In G. Bittencourt and G. Ramalho, editors, Advances in
Artificial Intelligence, volume 2507 of Lecture Notes in Computer Science, pages 163–174.
Springer-Verlag, Nov. 2002. ISBN 3-540-00124-7.
A. Zunino, C. Mateos, and M. Campo. Reactive Mobility by Failure: When Fail Means
Move. Information Systems Frontiers. Special Issue on Mobile Computing and Communica-
tions, 7(2):141–154, 2005. ISSN 1387-3326.