You are on page 1of 77

Project No.

FP6-004265

CoreGRID

European Research Network on Foundations, Software Infrastructures and
Applications for large scale distributed, GRID and Peer-to-Peer Technologies

Network of Excellence

GRID-based Systems for solving complex problems

D.STE.07 – Design Methodology of the Generic
Component-based Grid Platform

Due date of deliverable: 30 April 2008
Actual submission date: 6 May, 2008

Start date of project: 1 September 2004 Duration: 48 months

Organisation name of lead contractor for this deliverable:
University of Westminster

Project co-funded by the European Commission within the Sixth Framework Programme
(2002–2006)
Dissemination level
PU Public PU

Keyword list: Design methodologies, component model, tools/system com-
ponents, Problem Solving Environments, portal components

CoreGRID FP6-004265 1

Contents
1 Executive Summary 4

2 Introduction 5

3 Platform Architecture 6
3.1 Middleware Architecture and Design . . . . . . . . . . . . . . . . 6
3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2.1 Advantages for using the GCM . . . . . . . . . . . . . . . 7
3.3 Collective Communications Support . . . . . . . . . . . . . . . . 9
3.3.1 Brief recall of the GCM collective interfaces . . . . . . . . 9
3.3.2 Use of collective interfaces to build up a Grid-aware mid-
dleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Dynamic Service Aggregation Platform . . . . . . . . . . . . . . . 14
3.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4.2 The Virtual Clusters Platform . . . . . . . . . . . . . . . 14
3.4.3 WSPeer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4.4 Integration and Contribution to Platform Architecture . . 17

4 Interoperability Issues 19
4.1 Interoperability in Component-based Platforms . . . . . . . . . . 19
4.1.1 Introduction to Interoperability in Component-based Plat-
forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.1.2 Component Model and Framework Interoperability . . . . 19
4.1.3 Composition-level interoperability . . . . . . . . . . . . . 21
4.1.4 Interoperability – discussion . . . . . . . . . . . . . . . . . 23
4.2 Workflow Level Interoperability . . . . . . . . . . . . . . . . . . . 23

5 Development and Execution Environment 27
5.1 Component-Based Integrated Toolkit . . . . . . . . . . . . . . . . 27
5.1.1 Integration in a Generic Component Platform . . . . . . . 27
5.1.2 GCM-Based Design . . . . . . . . . . . . . . . . . . . . . 28
5.1.3 Life-Cycle Controller Modification to Prevent Deadlocks . 30
5.2 Grid Integrated Development Environment - GIDE . . . . . . . . 31
5.2.1 User Groups . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2.2 Composition . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.2.3 Deployment Perspective . . . . . . . . . . . . . . . . . . . 34
5.2.4 Monitoring Perspective . . . . . . . . . . . . . . . . . . . 34
5.2.5 Steering Perspective . . . . . . . . . . . . . . . . . . . . . 35
5.3 Mediator Component Framework . . . . . . . . . . . . . . . . . . 36
5.3.1 Classification of mediator components . . . . . . . . . . . 38
5.3.2 GCM-based design . . . . . . . . . . . . . . . . . . . . . . 39
5.4 gUSE - A Service based Environment for Grid Application De-
velopers and End Users . . . . . . . . . . . . . . . . . . . . . . . 43
5.4.1 gUSE as a Grid application developer environment . . . . 43
5.4.2 gUSE as an end user environment . . . . . . . . . . . . . 44

CoreGRID - Network of Excellence

CoreGRID FP6-004265 2

5.4.3 gUSE as a scalable, fault tolerant layer to ease Grid sys-
tem management . . . . . . . . . . . . . . . . . . . . . . . 44
5.4.4 gUSE system architecture . . . . . . . . . . . . . . . . . . 45

6 Autonomic Support 47
6.1 Autonomic support of non functional features in GCM . . . . . . 47
6.1.1 Autonomic components . . . . . . . . . . . . . . . . . . . 47
6.1.2 Autonomic behaviour support in GCM . . . . . . . . . . . 49
6.1.3 Experimental results . . . . . . . . . . . . . . . . . . . . . 50
6.1.4 Specific contributions to STE institute . . . . . . . . . . . 51
6.2 Automation of Application Deployment on Grids . . . . . . . . . 51
6.2.1 Application Submission . . . . . . . . . . . . . . . . . . . 53
6.2.2 Resource Discovery . . . . . . . . . . . . . . . . . . . . . . 54
6.2.3 Resource Selection . . . . . . . . . . . . . . . . . . . . . . 54
6.2.4 Deployment Planning . . . . . . . . . . . . . . . . . . . . 55
6.2.5 Deployment Enactment . . . . . . . . . . . . . . . . . . . 55
6.2.6 Application Execution . . . . . . . . . . . . . . . . . . . . 56
6.2.7 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7 Security Issues in Component-Based Grid Systems 57
7.1 Security Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.2 Existing security infrastructures . . . . . . . . . . . . . . . . . . . 58
7.2.1 Grid Security Infrastructure . . . . . . . . . . . . . . . . . 58
7.2.2 Shibboleth . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.3 Security Issues – Conclusion . . . . . . . . . . . . . . . . . . . . . 60

8 Use Cases – Overview and Examples 61
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8.2 Wrapping legacy software . . . . . . . . . . . . . . . . . . . . . . 61
8.2.1 GENIE Overview . . . . . . . . . . . . . . . . . . . . . . . 61
8.2.2 Component-based version of GENIE . . . . . . . . . . . . 62
8.3 Componentizing existing applications . . . . . . . . . . . . . . . . 63
8.3.1 Jem3D overview . . . . . . . . . . . . . . . . . . . . . . . 63
8.3.2 Componentising Jem3D . . . . . . . . . . . . . . . . . . . 63
8.4 Developing component-based codes . . . . . . . . . . . . . . . . . 64
8.4.1 GriCoL . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
8.4.2 Use Case: Biochemistry Application . . . . . . . . . . . . 65
8.5 Use Cases – Conclusions . . . . . . . . . . . . . . . . . . . . . . . 66

9 Software Licensing Issues1 67
9.1 Some GPL facts . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2 Incompatibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2.1 Apache License . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2.2 Eclipse Public License (EPL) . . . . . . . . . . . . . . . . 67
9.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
1
This section is based on text originally written by Tony Arbona and Gaston Freire from
Grid Systems, S.A. and provided via the EU GridCOMP project.

CoreGRID - Network of Excellence

CoreGRID FP6-004265 3 10 Conclusions and Future Work 70 CoreGRID .Network of Excellence .

To be practical and useful.1. Extending the concept of homogenous sets of component level interfaces between software systems. del- egation. and then work- flow level interoperability in Section 4. giving an overview of the Grid Component Model (GCM) in Section 3. which is a framework that enables developers to develop Grid applications that transparently use grid resources. Section 5.2. authorization. Section 8 presents some use cases exploring the approaches investigated in the course of the project. and virtual organization building. performance and robustness.CoreGRID FP6-004265 4 1 Executive Summary This deliverable reports on the considerations and methodology for the design and implementation of a Generic Component-based Grid Platform. CoreGRID .4 introduces an alternative workflow-based Grid application development and enactor infrastructure . deployment. execution. The Virtual Clusters project is a further extension which provides support for mo- bile devices through functional services known as “Aggregators” (Section 3. which enables the creation.1). which automatically take care of some non-functional properties aimed at achieving some autonomic reconfigura- tion goal. Section 5.Network of Excellence . any Grid platform must be adequately supported by development and monitoring tools. contribut- ing to scalability. This section discusses authentication. paying particular attention to the middleware and the extensions to the communication support that it provides in Section 3. but it also involves direct connection between components. We first outline the platform architecture. The modular design of a component-based runtime environment in- cludes components for performance monitoring. we address the need for an appropriate licensing strategy for the dissemination of results. monitoring and steering of component applications. Section 6 focuses on the sup- port provided through autonomic components.2. Finally. the specific requirements of addressing security for component applications through component containers is discussed. Its goal is to integrate system- component capabilities into application code. moni- toring and sharing of Grid applications through intuitive graphical interfaces. communication.gUSE (Grid User Support Environment). testing. Autonomic support is a key aspect of any generic Grid infrastructure. It collates the innovative efforts of a number of partners which together address the es- sential aspects of an effective Generic Component Platform. In Section 9. data dependencies and task scheduling. We address then address interoperability issues in Section 4.1 presents a component based integrated toolkit. Here. A distributed component application requires communication not only between the client and components. is the Mediator Component Framework presented in Section 5. The GIDE (Section 5. All these features are brought together for the user with a homoge- nous interface in the Grid Integrated Development Environment (GIDE). we conclude and summarize in Section 10. In particular.2) is an Eclipse based set of plugins which provides integrated tools for component composition and programming.3). sandboxing.3. by first discussing the interoperability in component based platforms (Section 4. achieving both application steer- ing and performance adaptation to achieve the most efficient execution on the available Grid resources. the need to address legacy code is discussed and a biochemistry use case application is presented. Section 7 addresses security issues.

the lack of longer-term experience and the complexity of the tar- get systems demand much more research results in the field.Network of Excellence . The component-oriented design methodology supports flexible component adaptation statically and dynamically. users. Additionally. the feature-rich philosophy introduces considerable software and administrative overheads. Our generic Grid platform design methodology relies on this advanced approach based on the GCM and on composable component frameworks addressing different concerns of the platform functionalities. existing. support extensi- bility of different aspects of the platform and allow easy to use and consistency- preserving reconfigurations. In our proposed design methodology. In many cases. the CORBA Component Model (CCM) [56]. As a consequence. such that the requirements of end users are a subset of the available functionalities. component-based software technologies have emerged as a modern approach to software development for distributed and Grid applications.CoreGRID FP6-004265 5 2 Introduction Grid technology has the potential to enable coordinated sharing and utilisation of resources in large-scale applications. which could scale automatically based on the context of use. even for relatively simple demands. and the emerging Grid Component Model (GCM) [18]. The wide adoption of component-based software development and in particular the use of suitable programming models for compatibility and interoperability are key issues towards building effective future Grids. It is the cohesive bond enabling virtualisation and control of resources. In Grid infrastructures the platform is the heart of the system. This limits the scalability of the platform to be unidirectional – always scal- ing upwards. Examples of component models applicable to this field include the Common Component Architecture (CCA) [8]. contemporary Grid platforms are feature-rich. The compo- nent frameworks facilitate designing the platform architecture. the majority of the platforms cannot easily be adopted for a specific purpose. we address this problem by developing our Generic Component-based Grid Platform. Recently. However. CoreGRID . Instead. policies and applications regardless of geographic or administrative boundaries. they involve substantial consideration for changes.

Each com- ponent is either composite (composed of other components).1 Middleware Architecture and Design 3.Network of Excellence . reference allocation and other relevant services. The Fractal specification [15] proposes a generic. Though CCA enables seamless runtime interoperabil- ity between components. or primitive (a black box encapsulating a basic functional block). recursively composing components to form more complex. However. Component models specify the way in which components are defined and interact. Another key feature is its support for extensible reflective facilities: each component is asso- ciated with an extensible set of controllers that enable inspecting and modifying internal features of the component. which are interfaces pointing to method invocations. In CCA. The CCM specification introduces the concept of components and the defini- tion of a comprehensive set of interfaces and techniques for specifying imple- mentation. Fractal [32] is a model that organizes components hierarchically. The CCM is an extension of the CORBA object model defined to overcome its complexities. The CCM [6] is a language-independent. server-side component model which defines features and services to enable application developers to build. By means of these interfaces. Controllers provide a means for capturing extra-functional behaviours such as varying the sub-components of a composite component dynamically or intercepting incoming and outgoing operation in- vocations. CoreGRID . that is. The Common Component Architecture (CCA) [8] specifies the means for in- teraction among components. typed component model in which components are runtime entities that communicate exclusively through interfaces. the CCM does not permit hierarchical composition. in contrast to the Fractal component model. deploy and manage components to integrate with other CORBA services. one of the main weaknesses of the CCA is the lack of support for hierarchical component composition and for control mechanisms thereof. thus constituting a component assembly. composite components.CoreGRID FP6-004265 6 3 Platform Architecture 3. One of the crucial features of this model is its support for hierarchical composition. components interact using ports.2 Background A component is a software entity that gives a standardized description of what it provides (server interfaces) and requires (client interfaces). and deployment of components. The GCM proposal [18] is an extension of the Fractal component model that specifically targets Grid environments. The enclosing framework provides support services such as connection pooling. Components in this model define provides-ports to provide interfaces and uses-ports to make use of non-local in- terfaces. components can be connected (bound) together and communicate. Dynamic construc- tion and destruction of component instances is also supported along with local and non-local binding. packaging. The CCM provides the capabilities for composing components (through receptacles) and permits con- figuration through attributes.

Collective interaction patterns GCM supports collective interaction pat- terns. taking care of component per- formance tuning. XML based ADL The GCM component program architecture is described using a proper XML-based Architecture Description Language (ADL) that decouples functional program development from the underlying tasks needed to deploy. which defines the service policy of the requests. Its main features are sketched in Figure 1. Figure 1: Main features of the GCM Currently. These reactions are imple- CoreGRID . built on top of the already existing Fractal model. a reference implementation of our Generic GCM-based Grid Plat- form is being developed on top of ProActive [61] within the GridCOMP STREP FP6 EU project [38]. data ports (supporting data sharing among components) and event ports. with different. Autonomic management of notable composite components (i. elementary components can be used. reusable. The developer provides the functional (business) code. composite components are first class components. In particular. GCM components have a request queue. Dynamic reconfiguration is crucial in a component-based system. multicast.CoreGRID FP6-004265 7 3. and programmers need not necessarily perceive these components as compos- ite. we distinguish two essential roles: the developer and the assembler. As a result. unless they explicitly want to consider this feature. taking into account the requirements of the application. Autonomic management of components GCM supports Manager com- ponents as well as standard controllers in the component membrane. Concerning the development of a component application. while the assembler is in charge of composing the system. parametric parallelism exploitation patterns) is also supported. Moreover.e. collective broadcast. com- posite components modelling common. GCM provides stream ports (support- ing data flow streams between components).Network of Excellence .2. in that they can be used in every context where non-composite. The new. Non standard ports In addition to standard use/provide (client/server) ports and to the collective ports.1 Advantages for using the GCM The GCM [18] is the CoreGRID component model for Grid computing. 1-to-N and N-to-1 ports. and an activity. which contains the received requests. gather and reduce patterns are provided as primitives. Hierarchical GCM users have the possibility of programming GCM compo- nents as compositions of existing GCM components. run and control the components on the component framework. It means reacting to situations unforeseen by the developer. Following ProActive design solutions this reference im- plementation of GCM uses asynchronous method calls with futures for dealing with possibly high latency. scatter. configurable distribution and collection policies.

we focus on the NF interfaces of the host component.Network of Excellence .1. All these interfaces give the membrane a better structure and enforce decoupling between the membrane and its externals. and then CoreGRID . they give a high level view of the structure suggested for the component management: • hns: Host component external NF Server interfaces as exist in the Fractal specification. GCM also promotes the idea of designing the membrane of a GCM component as a GCM-based system of components. they give inner func- tional components access to interfaces of inner NF components. • hnc: Host component external NF Client interfaces. making references to Figure 2. For example.CoreGRID FP6-004265 8 mented by the assembler inside non-functional (NF) pieces of code. NF = non-functional nns -First letter (component): Controller object nfs h/n/f/e: host/inner NF/inner F/external F Controller b2 -Second letter (interface): Component b4 n/f/i/m: external NF/ external F/internal F/internal NF nfc -Third letter (interface): c/s: client/server Membrane b3 -Example: hmc hns = external NF server interface of the host component ffs b9 fns b5 ffc hfs b8 hfc b6 fnc b10 hic b7 his Functional Content hms ens b11 hnc efs efc Figure 2: New structure for the membrane of Fractal/GCM components Here we give a more detailed explanation about some possible roles of NF interfaces exposed by the membrane. For sim- plicity. the two above mentioned roles are relevant. At this non- functional level. This non-functional system is then diluted into the functional hosting component and explicitly defines what are the subtle interactions that can arise between the functional and non-functional parts as shown in Figure 2. to connect nfc with fns. • hmc: Host component internal NF Client interfaces. • hms: Host component internal NF Server interfaces. This is why having the ability to specify at will the control part of any GCM component is important. GCM adds an additional stage: we have first to perform binding b3. they allow external entities to access controllers inside the membrane. they give to inner NF components access to interfaces of inner functional components. As all those non-functional pieces of code may end up to be complex and interact in a subtle way. they give to inner NF components access to external NF interfaces. hns Labels of Interfaces : Controller b1 Component F = functional. A detailed presentation of our work and results on autonomic support of non-functional features in GCM is included in Section 6.

as well as redistribution policies for invocations return values. b10) except that no interception of the communications on these bindings is allowed in Fractal. This avoids nfc to be strongly coupled with nfs: to connect nfc to another interface of kind nfs.Network of Excellence . which is one of the main reasons for defining this kind of interface: it enables parallel invocations. When a single invocation is transformed into a list of invocations.g. some of the links are represented with dashed arrows. Invocations forwarded to the connected server interfaces may occur in parallel. the source interface is the alias and it is “merged” with the destination interface. is built by the concatenation of a gathercast and a multicast interface: • the server gather-multicast (Fig. • the client gather-multicast (Fig. 3. To conclude. by transforming a list of invocations into a single invocation. 3. specific to composite components. by transforming a single invocation onto a list of invocations. Multicast interfaces provide abstractions for one-to-many communications. or a reduction of it. Syn- chronization barriers and gathering operations are customizable by means of an aggregation policy. A gathercast in- terface coordinates incoming invocations before continuing the invocation flow: it can define synchronization barriers and gather incoming data. which allow membrane to be designed in a component-oriented way. There is also an other kind of interface called gather-multicast (Fig. These bindings are similar to the export/import bindings existing in Fractal (b6. This interface. The semantics of the propaga- tion of the invocation and of the distribution of the invocation parameters are customizable by means of a parameter distribution policy. one key element of the Generic component-based Grid Platform. only binding b9 has to be changed.1. Those links are not real bindings but “alias” bindings (e. and the result of an invocation on a multicast interface is a list of results. and to be reconfigurable. In Figure 2.2 3. these generated invocations are forwarded to connected server interfaces.3 Collective Communications Support 3. The aim is to show the applicability of GCM to the Integrated Toolkit.b) connects internal components on a CoreGRID .1 Brief recall of the GCM collective interfaces The GCM adds the idea of collective interfaces to the Fractal component model following previous work and results in the area of multiparadigm communica- tions for Grid computing [36].CoreGRID FP6-004265 9 binding b9.3. This is detailed further in Section 5.a) exposes its gathercast interface and is connected to internal components through a multicast interface. b3). Invocation return values are automatically redistributed to the invoking components. 3). Gathercast interfaces provide abstractions for many-to-one communications. the GCM provides possibilities consisting in implementing as components (part of) the membrane and thus benefiting from the component structure.

we have introduced new controllers associated to primitive components and to the gather-multicast in- terfaces. In general. expected by the non-optimized version of the server interface.CoreGRID FP6-004265 10 gathercast interface. The next subsection depicts how the gather-multicast interface and its MxN optimization have been used to implement a GCM-based runtime to support MPI-like SPMD applications. and server interfaces in the ’N’ server components and connect them directly. Efficient communication requires some form of direct bindings between the inner components accord- ing to the redistribution pattern. of the non-optimized version. = . • a dispatch function (D): associated to the generated client interfaces. It is also in charge of dispatching method calls. according to data distribution. . creating dynamically new client in- terfaces on the ’M’ client components. it transforms the original single call. These controllers are capable of optimizing these collective interfaces by distributing the ’MxN’ behavior i. into a list of calls. client interface . . client interface b. internal multicast external multicast a. . it is re- sponsible for the adaptation of method call signatures. . = . a multicast client interface and a gather- cast server interface) and the direct binding scheme. it is responsible for the adaptation of the method call signatures. For this reason. it trans- forms the received calls into a single call. CoreGRID . . The optimization takes three main arguments: • a reconfiguration function (R): used by a special controller to create the necessary interfaces (in general. Succeeding to host communication intensive ap- plications while achieving good performances is a strong argument in favor of using a component-based approach to design runtime and middleware for grid computing in general. . . In general. external gathercast internal gathercast client interface client interface Figure 3: GCM gather-multicast interface type Obviously.Network of Excellence . • a gather function(G): associated to the generated server interfaces.e. (see Figure 4). . exposing them externally through a multicast inter- face. not only for supporting HPC scientific applications. . the naive solution for performing collective communications among composites bound by a gather-multicast interface creates bottlenecks both in the gather and the multicast sides of the interface.

i. The DiscoGrid communication API defines classical point-to-point primitives such as Send and Receive as well as collectives ones such as Barrier. Direct (optimized) bindings 2:1 Original bindings Figure 4: Example of a GCM gather-multicast usage and its optimisation using an MxN interface type 3.CoreGRID FP6-004265 11 and Original single interfaces and Generated collective interfaces 1:0 1:1 .3. It wraps an MPI process and is CoreGRID . which is an automated way of updating data shared between processes. the Update primitive is useful to synchronize borders of each domain. The DiscoGrid Runtime is a component-based infrastructure that models a hierarchy of resources forming a computing Grid. of the underlying physical resource topology. we have de- signed a GCM-based runtime support for hiererchical MPI-like SPMD applica- tions. The API also defines the Update primitive. at the program level.. This hierarchy is the expression. and a Receive can be seen as a gathercast.. Reduce. programmers can perform either point-to.. a leaf or a node. The originality of the DiscoGrid communication API resides in using hierarchical identifiers to involve different sets of processes.. In the case of domain-decomposition problems. and offers support to the con- cepts and programming primitives of the DiscoGrid API [50].. such as data location and neighboring. The grid resources are organized as a tree and. Point-to-point primitives have a special semantic where a Send operation targeting a node in the resource hierarchy can be seen as a multicast. 0:2 0:3 2:0 . The DiscoGrid Runtime is a composition of two sorts of components: • the primitive wrapper is the most elementary component used to form the entire component infrastructure.2 Use of collective interfaces to build up a Grid-aware middle- ware In the context of the French ANR-funded project DiscoGrid [24].e. using an associated meta-information. 0:0 1:2 0:1 . AllReduce.. depending on hierarchical identifiers of the source and the destination.point or multi-point commu- nications. The DiscoGrid API intends to provide programmers with a notion of hierarchy in their dis- tributed applications.Network of Excellence . respectively.

that also generate the method calls and invoke them on the proper bound interfaces. These gather- multicast interfaces implement the main collective operations (reduce. The non- optimized implementation of the update operation uses the initial compo- nent bindings and gather-multicast interfaces. that instead of just triggering a one shot communication between two sets of components (of size M and N). In general. that exposes externally the entire DiscoGrid application. If necessary. In practice. By default. it only presents a single server interface and a single client interface that are bound to a generic clustering component that represents the upper level of the hierarchy. By default. Depending on the operation. broadcast and barriers) as aggregation and dispatch poli- cies. So. trigger a set of such MxN communications CoreGRID . The interest of such encapsulation is that one can envisage to take profit of GCM features to compose complex simulations by loosely coupling simpler applications or even expose one whole application as a WebService. High-level DiscoGrid communication support • Point-to-point communications: A point-to-point communication between processes on different clusters happens when two process do not have the same hierarchical identifier prefix.Network of Excellence . selected dynamically according to the requested operation. sent upwards in the hierarchy and routed through composite interfaces until it reaches the destination. • Neighborhood-based communication: This kind of communication is rep- resented by the update primitive. the aggregation policy also reduces the data. gather. a message is properly tagged in the wrapper component. scatter. standard collective communica- tions assume that all processes of a given level must participate on the process. capa- ble of receiving messages from other clustering components and dispatch- ing messages to inner components and a gather-multicast client inter- face capable of gathering messages from inner-components and dispatch them to other clustering components of the same level. defined in the DiscoGrid API. a collective communication is said to take place at a given level. The primitive wrapper is also responsible for the encapsulation of MPI messages into objects with information useful to route messages through the component infrastructure. It presents two interfaces: a gather-multicast server interface. messages are gathered on the most external clustering component that represents the given level. This update communica- tion can be considered a generalization of the MxN communication. the entire component model is enclosed in a generic composite.CoreGRID FP6-004265 12 responsible for the MPI to/from Java communication. • the composite clustering is a generic component capable of clustering lower-level components (both primitive wrappers and composite cluster- ing). the parameters are properly split by the parameter dispatch policy. • Collective communications: As in MPI.

splitted up into pieces and dispatched properly to the correct destinations using a dispatch policy that looks like a scatter. messages are then gathered in the most external clustering component of the level. However. this is a good point in favor of such an approach.CoreGRID FP6-004265 13 (i. Some performance benchmarks confirm that the optimized update operation implementation is worth considering (see Figure 5). the usage of the GCM collective interface types and the introduc- tion of numerous optimisations that we have been able to conduct prove that a middleware for grid computing based upon a component-based approach can exhibit flexibility but also. And G blocks until messages from neighbors are arrived on the gathercast interface and order them properly. 12000 DiscoGrid Non-Optimized Pure MPI DiscoGrid Optimized 10000 8000 Time (ms) 6000 4000 2000 0 8 16 32 64 128 Number of Nodes Figure 5: Poisson-3D update operation duration Overall. D splits the parameter of the update call and dispatches the data directly to the connected neighbors. but taking into account neighborhood information to split and determine who receives each piece of data. there is some room for optimization. Thanks to gather-multicast in- terfaces. the runtime can do this generic MxN communication at once: each process participates with its shared data of the given level. As the Generic Component-based Grid platform may face communication intensive patterns.Network of Excellence . CoreGRID . which is done as follows. good performances in supporting performance de- manding inter-component interactions. a generalized MxN communication). The optimization of gather-multicast interfaces is defined by three functions: R defines the creation of a multicast client interface and a gathercast server interface (the generated interfaces attached to each primitive component in the Figure 4) and the bindings according to the neighborhood information (the arrows crossing the composite component boundaries in the Figure 4). before passing the message to the wrapped MPI process.e.

Tools and Environments. minimal administrative and management overhead due to the large number of mobile devices integrated in a relatively stable system. ”Efficient“ in this con- text refers to a number of desired attributes the integrated system should have: performance efficiency in dealing with many mobile nodes at the same time.4. 3. failure masking. This happens in a standardized manner through WS technologies (HTTP/SOAP) despite the potential hetero- geneity and multitude of binding protocols available “behind” the VC abstrac- tion.4 Dynamic Service Aggregation Platform 3. clients only access the avail- able functionality through the Aggregators. that provide high level. and the further opportunities for in- tegration with WSPeer. Instead. and very dynamic mobile domain.CoreGRID FP6-004265 14 3. heterogeneous. then move further into the most significant as- pects of the projects in regards to work done in the Institute of Systems.2 The Virtual Clusters Platform The Virtual Clusters Platform (from now on “VC”) is a proxy-based middle- ware that enables the efficient integration of mobile and potentially small-scale devices (collectively referred to from now on as simply “mobile services” or “mobile devices”) in Grid systems as service providers -and not just service consumers. and finally lay the roadmap for future integration activities. Overview/Architecture The VC is based around the concept of Aggrega- tor services (or simply “Aggregators”). failure tolerance. Aggregators are proxies to services available in mobile devices.4. abstract views of the underlying. collective operations and more. The Aggregators reside in proxy nodes. which can be either dedicated nodes (preferably on the “edge” of the static part of the hybrid system) or typical functional nodes since Aggregators can be duplicated or migrated and the middleware does not have a large footprint [40]. In order for Aggregators to assume their proxy responsibilities. We will first provide some overview of the Virtual Clusters platform and WSPeer.1 Introduction In this section we briefly present the Virtual Clusters platform. and it is Virtual because potential clients do not and cannot “see” the mobile services directly. mobile devices that offer a specific service are grouped to form what is called a Virtual Cluster. the related contribution to WP7 stemming from it. Dynamic Properties Here we will mention only the dynamic properties of the VC platform that are relevant to the Platform Architecture deliverable CoreGRID .Network of Excellence . It is a cluster because it is a collection of functional services (although it does not bear much similarity to the traditional computational clusters). introduction of efficient programming models and techniques on top of this integrated system. towards what are called “hybrid systems”. Aggregators are not simple proxy services though: the are assisted by a number of well orchestrated middleware services that provide support for im- portant non functional aspects of the system such as mobility monitoring.

It is purely distance-oriented only and does not include issues related to location or personalisation. In brief. are able to detect and automatically activate mobile services that have registered previously in a seamless manner. there is a mobility prediction mechanism – a Virtual Distance mechanism [39] – to calculate the probability of a device moving out of coverage. mo- bile devices offering the same service will only have to register with the relevant Aggregator -a process that is already semi-automated and in the near future will require human intervention only in case of security con- siderations or special policy enforcing. This entails some sort of ability to forecast or predict changes due to mobility. CoreGRID .Network of Excellence . In the VC platform. • Mobility Prediction and Monitoring Amongst the main design goals and the core driving force behind the VC platform was the inherent support for mobile services. • Dynamic Service Aggregation Aggregators are generated only once for each VC: the first time that a particular service becomes available in the system. service mobility in particular. For more information the reader is referred to [41]. and a flexible but powerful mechanism for monitoring and adapting to environmental changes in general. Aggregators -with the help of the middleware services. this is based on an algorithm that keeps track of a short history of mobility patterns of the device in question.CoreGRID FP6-004265 15 Figure 6: High Level Architecture of the Virtual Clusters Platform context. From then on. which makes it appealing for such dynamic systems. This mechanism is non intrusive for the mobile device and it has no requirements on the device provider.

the device might actually come back online almost immediately. or the LSC (see below: “Binding Extensions”). Furthermore. possibly utiliz- ing different binding protocols. WSPeer is not based on a container model in which appli- CoreGRID . along with the detailed description of the VC approach to dependability. If. there is enough flex- ibility in the VC platform. a temporary discon- nection of the device. Server-side components available in the underlying fabric layer can make use of a variety of transport protocols. without any consequences in the higher layers. enables a unified and consis- tent interface to multiple and dynamic services/resources. then there is build-in support for auto- matic redistribution of the task to available services. To achieve fault tolerance. At the programming level. the VC abstraction also enables the en- capsulation of internal failures. and failure recovery – to support both. due to intermittent wireless connectivity. and will provide the foundations for collective (group) operations on the VC. as well as support for migratable services in case of predicted failures.3 WSPeer WSPeer is an API for hosting and invoking Web services.Network of Excellence . the VC takes advantage of the cardinality of the underlying services in the mobile domain. In such case. 3. First of all. failures within the cluster generate notifications that only go up to the proxy layer while they are dealt with internally with mechanisms such as state migration or redistribution. such as WS. the VC abstraction will significantly ease software de- velopment and simplify some high-level programming models. protecting this way the rest of the system from the inherent instability of the underlying nodes.CoreGRID FP6-004265 16 • Abstraction Layer The implementation of the VC abstraction. such as the master-worker and the voting models. • Failure Management Support Dealing with the consequences of device failures has a three-fold meaning: fault tolerance – for high reliability. However. the failure is permanent or non-recoverable. and without the need to redistribute or restart the execution. for failure recovery. and works as a bridging layer that provides unified access to heterogeneous services. RMI. For failure masking. to automatically mirror the execution of a task to more than one available services. In [39] an extended discussion about dependability issues in such hybrid environments is presented. it provides trans- parency from the underlying binding protocols and dynamic addressing schemes. is regarded as a failure in a traditional system. slightly different techniques are required. and since the nature of failures in the mo- bile domain is fundamentally different than in traditional static domains. Finally. The VC abstraction enables failure masking by dealing with failures without requiring the intervention of the requesting client. Unlike most Web service middleware. however.4. failure masking – for stabilized avail- ability. to allow the immediate reactivation of the device. Finally.

and. 3. WSPeer has evolved out of an interest in combining the strengths of Web services. Instead an event-based architecture is used in which applications add themselves as listeners to events generated by WSPeer. beyond the base network forming and maintenance. WSPeer is architected to allow for multiple protocol and middleware bindings to facilitate Grid and P2P interoperability. (ii) communication between the VC Aggregators and the underlying mobile ser- vice nodes. a J2ME version of WSPeer. the VC environment would benefit on three different and distinct levels: (i) communication and discovery among the VC layer. WSPeer already has bindings for JXTA. Lastly. leaving the appli- cation in control of how these are handled.CoreGRID FP6-004265 17 cations are placed into the container environment and managed by it. mobile nodes can make use of decentralized P2P infrastructure to dynamically discover and directly interact with one another using WSPeer’s unicast and multicast protocols.4 Integration and Contribution to Platform Architecture By integrating the Virtual Clusters architecture with WSPeer discovery and invocation capabilities. specifically the interoperability benefits of WSDL and SOAP and the decentralized resource sharing and discovery mechanisms of P2P systems. Web service XML standards allow interoperable service definition and mes- sage exchange while decentralised resource sharing and discovery have proven themselves to be scalable and resilient to node failure while enabling transient nodes to interact. In addition. which would expand the current capabilities of the VC Aggregators to allow these communication protocols. a node exposed with WSPeer functionality can bridge static Grid environments with more dynamic ones. The WSPeer communication and discovery capabilities. P2PS. Currently. Events are generated as the result of deployment. WSPeer can also be beneficial in the second aspect. and the arrival of SOAP messages. and Styx.4. This makes the architecture simple for integrating applications with it. Hierarchical Structures Hierarchical structures and topologies are becom- ing important in distributed systems because of their flexibility and available CoreGRID . This allows for the complimenting of standard Web services means for message trans- fer and service description with P2P mechanisms for discovery as well as Net- work Address Translation System (NAT) and firewall traversal.Network of Excellence . (iii) direct P2P interaction between mobile nodes. WSPeer supports P2P communication using JXTA and P2PS and Grid and Web services through WS-RF and WS-Notification. The first aspect would allow for the VC layer to form a dynamic network that is able to self discover and heal in the case that Aggregators are leaving and joining the network. Because multiple bindings can coexist. also provides the tools needed for VCs to discover and communicate with one another when migration events are triggered. publishing. that of commu- nication between the VC and the underlying mobile and transient nodes in the cluster. there is also support for firewall traversal using Styx as well as an in-house lightweight XML communication protocol for mobile devices bundled within WSKPeer.

Globus or Tomcat/Axis). Binding Extensions This is important in the context of mobile and po- tentially limited environments because of the inadequacy of mobile devices to support traditional service middleware (e.g.Network of Excellence . CoreGRID . Integration with WSPeer will extend the range of immediately supported bindings from the existing HTTP/SOAP. WSPeer would bring in an- other dimension in the form of P2P-style bindings and communication. different protocols and different policies could be en- forced in different branches of the hierarchy for better fine tuning and more application-specific support. While the VC platform is inherently ready to provide the backbone for hierarchical clustering. Better Non-Functional Properties Support The P2P style interactions of WSPeer will assist the VC proxies in a number of significant non-functional properties.CoreGRID FP6-004265 18 options. Yet another mobile provider could go for the least “expensive” approach of all that comes in the form of the Lightweight Service Container (LSC) developed in UoW [41]. It is therefore desired to have a flexible system that can understand a variety of binding protocols so that mobile users can select the one that better suits their own device. Another might choose a less “heavy” approach such as RMI or even RMI-OP for Java micro devices. This will relax a number of constraints of the VC platform and will enable more straightforward communication between proxies at the same hierarchy level. and LSC. to also include Styx. RMI. a process that takes place in response to (and in support of) mobility in order to forward execution results back to the requesting client. some of the stronger mobile devices can opt for a full fledged web services container with all the benefits and the problems this will bring. thus providing even more options for mobile service providers. For example. Direct communication between VC proxies in a P2P manner maps directly to high level VC proxy chaining. P2PS and JXTA. For example. It will also enable flexible topologies and mobility at every hierarchical branch to be supported seamlessly as the VC proxies will be able to dynamically adapt to environmental changes easily and in a cooperative fashion thanks to WSPeer P2P bindings. Migration techniques will also take advantage of WSPeer since cooperation between VC proxies becomes more efficient through notification mechanisms that are directly supported through WSPeer.

1.1.CoreGRID FP6-004265 19 4 Interoperability Issues 4. must be able to interoperate with existing component frame- works and allow applications to be built from a mixture of various kinds of components. Beside it there are alternative interoperability approaches: our idea is to introduce mediators and adapters to build up an ad-hoc interoperability layer between selected component frame- works without superimposing on them another general-purpose interoperability framework (like a CORBA bus. or a meta-component model implemented on top of some selected existing component frameworks [59]). or the model abstraction [66]. The first solution enables the creation (instantiation) of a CCA component as a primitive Fractal component in a single address space. we describe two levels of interoperability: • Component model and framework interoperability • Interoperability at the composition level The first case is exemplified on our experience with the GCM and CCA com- ponent models [51] and the specific implementations: ProActive and MOCCA [52]. including CCA and GCM components as well as Web Services and legacy jobs. It relies on a wrapper that encapsulates a CCA component. which proposes a high-level scripting approach to component application composition and allows integrat- ing multiple middleware technologies and component models. The second case is demonstrated on the example of GridSpace programming environment. in the context of heterogeneous systems.Services interface to a CCA component (see Fig. Today. the execution environment.1 Interoperability in Component-based Platforms 4. a popular solu- tion for interoperability between components is Web Services where standard- ized protocols based on XML provide the common language for applications to communicate [30]. wrapped as GRID. We demonstrated how these models can be mapped to each other and how the implementations can interoperate. to be seen as a GCM component.it components [1]. 4.2 Component Model and Framework Interoperability We identified two approaches of interoperability between CCA and GCM: the encapsulation of a single CCA component and of a complete CCA system. Interop- erability has been outlined as a requisite for the GCM: a realistic programming model. Naturally. component frameworks and legacy software. In this section. and exposes cca.Network of Excellence . This has been successfully applied also to high-performance modules like ASSIST modules. 7).1 Introduction to Interoperability in Component-based Plat- forms Interoperability can be defined as an ability of two or more entities to commu- nicate and cooperate despite differences in the implementation language. the GCM proposes to achieve general component in- teroperability through standardized web services. Before instantiation we should know the type of CoreGRID . consisting of several CCA components.

and translates the Fractal invocations to CCA invocations and reversely.Network of Excellence . MOCCA). Integration of a single component was realized as outlined above. and ability to interact with it from a Fractal framework as if it was a Fractal composite component. The wrapper uses the BuilderService to connect exported CCA ports to corresponding GluePorts using CCA framework. b) Interoperability between CCA and Fractal components a component in order to define the Fractal type of the component. The wrapper instanti- ates the CCA component as a local object and it invokes setServices (this) CoreGRID .CoreGRID FP6-004265 20 Figure 7: a) Integration of a single CCA component into a Fractal one . In the second case CCA components are created in their own framework and they are connected to Fractal components running in their framework. the Wrapper component is both a CCA and a Fractal component. The solution we propose is based on a wrapper which adds a Membrane to a CCA assembly. So. we wrap the component assembly as a Fractal component in such a way that it can be connected to other Fractal components. A wrapper which encapsulates a CCA component and which exposes Services interface to a CCA component is created by the Fractal framework.g. and they can be connected (bound) to other Fractal components using BindingController and Component interfaces of the wrapper. The GluePorts expose Fractal interfaces to the outside world. In other words. In this case. In order to verify the proposed above solution a prototype using Java-based ProActive and MOCCA implementations was developed. The implementation of a GluePort is framework specific. we have a CCA component or a set of CCA components which are created and connected among themselves by a CCA framework (e. so the commu- nication between CCA component assembly and GluePorts is handled by the CCA framework. this might be obtained from a provided ADL description. Complete interoperability between two frameworks requires instantiation of a whole CCA assembly. The wrapper should interact with the CCA framework only via BuilderService external interface (obtained in framework dependent manner).

The architecture of the proposed system is shown in Figure 8. As the goal of our Ruby-based GScript language is to provide constructs for component deployment. Moreover. which stores the same information in the Ruby script format.CoreGRID FP6-004265 21 on a CCA component. being focused on the Grid environment. containers and the state of the Grid resources. there is a need to provide an execution engine. and is also possible to use a local registry. It was the implementation choice to create a Server Glue as ProActive com- ponent which includes the MOCCA code. The registry is available as a Web service. allowing a rich set of control constructs of component application (workflow). forming the existing CCA assembly. A scripting approach provides also the full flexibility of using a programming language. or Java API). It would be also valuable if the solution could facilitate such aspects as component configuration and passing parameters to the application. deployment and execution tool for the Grid. CoreGRID .Network of Excellence . passing the reference to itself. since many models are available for programming Grid applications. whereas a Client Glue is created as MOCCA component with an “embedded” ProActive one. a script. Our approach is in line with the suggestion that any component-based platform to be successful should support scripting as a mechanism for glueing application components together [57]. Registry is used for storing all technical information about available components. The high-level notation allows hiding all the details of underlying Grid infrastructure. It should be noted that both Client and Server Glue components are conceptually symmetric and their role is to translate invocations from one framework to the other. A system designed to meet the above mentioned requirements based on a scripting approach is called GridSpace [54]. and consequently the wrapper can create direct (local) bindings to exported CCA ports. The approach should not be limited to a single component model.g. automating the process of component deployment and resource selection where possible. In the real interoperability scenario we assume that there are CCA compo- nents running in a framework and connected using a mechanism specific to this framework (e. 4. so the programmer may focus on the application logic and automating the process of resource selection and component deploy- ment. based on a scripting paradigm.1. There is a need for a high-level program- ming approach which would enable combining the composition in space and composition in time in a way which is flexible and convenient for a program- mer. responsible for providing the information on components in use and to hide the complexity of the underly- ing grid infrastructure. prototyping and experiment- ing scenarios. composition and invocation the component methods. The CCA component registers its uses and provides ports. This solution is especially well suited for rapid application development (RAD). Such a solution would eventually form a powerful application development.3 Composition-level interoperability Our research was focused also on the high-level approach for component compo- sition. it should conceal the complexity of the un- derlying infrastructure. updated by Moni- toring system.

More details on the invoker and the architecture of the system can be found in [11]. Composition-level interoperability is supported by the GridSpace environ- ment in the following way. it is possible to combine components from different models into a single application. Script invo- cations are translated to underlying Fractal. It is also possible to integrate modules developed in other technolo- gies. to enable translation of invocations between them. If more than one component model is supported. it is possible to introduce such a generic glue which can bridge components from different models and frameworks. those glue components allow composi- tion in space (direct connections) between the two frameworks [51]. The Invoker module transparently performs remote calls on component ports of different frameworks. In the case of a component workflow. As our research on interoperability between GCM and CCA [51] suggests. the runtime system is the central point of inter-component communication. e. For Web Services the adapter can use a client from a standard library of a scripting language. Web services or WSRF in such a workflow. there are adapters for communicating respectively with MOCCA or ProActive components. when direct links between components are involved. The role of optimizer is similar to the deployment framework as proposed in the following sections of this document. so it acts as an intermediary passing results from one component invocation to another. The invoker has an extensible architecture which allows plugging in adapters responsible for interacting with different technologies.CoreGRID FP6-004265 22 Figure 8: Architecture of GridSpace scripting environment which is useful for quick development and debugging. CoreGRID . CCA or GCM APIs. There is also an adapter for the MOCCA-ProActive Glue components. it may be neces- sary to introduce glue ports between the heterogeneous components.g.Network of Excellence . As of component tech- nologies. The Optimizer module is responsible for supporting decisions on (automatic) component deployment. The scripting API for composition and deployment of components is neutral with respect to the used component model. In the case of composition in space.

One aspect is the execution of workflow components on computing resources spanning several grids. The P-GRADE grid portal and application development environment [44] was utilized to implement the generic interoperability scenario above. Components of a P-GRADE workflow can be executed in various Grids and can send input/output data to each other. as well as to create the glue code which enables inter- framework interoperability. The analysis of CCA and GCM component models. shows that despite some differences. Figure 10 illustrates a P-GRADE workflow where the different jobs are mapped to different production Grids (EGEE. These grids can be based on different grid middleware and may require different user certificates for authentication. On the composition-level interoperability. GT4 and g-Lite. then the generation of wrappers and glue code bridging two different component frameworks can be generic and thus automated.2 Workflow Level Interoperability Grid interoperation at the level of workflows can be achieved inside one workflow or within several different workflow systems. we can talk about intra. In both cases.1. The input files of the workflow can come from file systems. CoreGRID .CoreGRID FP6-004265 23 4. UK-NGS. We observed that if the properties of two different component models can be well under- stood. The multi-grid support in the P- GRADE portal allows downloading multiple certificate proxies to one portal session and mapping these to multiple Grids. The generic requirements towards intra-workflow interoperability are shown in Figure 9 [45]. we can also note that the vari- ous component models can be equivalent to some extent and it is possible to propose an environment which can conceal the differences between the under- lying frameworks. The other is the movement of data between heterogeneous data resources and workflow components. while making them still available for more advanced users who need full control over the application ecexution.4 Interoperability – discussion On the level of inter-component model interoperability we can draw the follow- ing conclusions. including a non-trivial application (simulation of gold cluster formation[53]) and integrated with the ProActive library. The prototype functionality has been verified with a number of examples. 4.Network of Excellence . Both the file systems and the databases can be located in different production grids. Therefore we can achieve interoperability and facilitate the application composition by hiding the low-level details. Therefore. it is feasible to integrate components from one model into another framework. or from database management systems and the result can also be fed into any of these solutions. we have to examine two different aspects of interoperation.or inter workflow interoperation. The job submission mechanisms of the portal also support submitting jobs to various Grid middleware such as GT2. like SRM or SRB. respectively. and the jobs of the workflow can also be mapped to different grids. SZTAKI and University of Westminster are collaborating within the framework of the CoreGrid project to address these interoperability aspects and were concentrating on intra-workflow interoperability so far.

In order to achieve workflow level interoperation of SRB and the already supported data sources (GirdFTP catalogues and EGEE storage elements). and also to utilise databases exposed by OGSA-DAI. the natural way to integrate SRB with P-GRADE was to extend it with a new port type called SRB. The P-GRADE portal allows data to be fed from and to GridFTP. The second challenge was enabling the creation of SRB input and output ports in the P-GRADE workflow editor. to access the SRB data collection to retrieve the selected files before job execution. And finally. Intra- workflow interoperation of grid data resources allows data to be input from or output to different file storage systems or database solutions. The job executable can be provided by the user or can be selected form a legacy code repository realized by the GEMLCA [23] solution. The second aspect of intra-workflow interoperability is data access. When moving these CoreGRID . As input and output files in P-GRADE are represented as ports. our current work extended this towards SRB and OGSA- DAI. The first challenge is setting up and configuring the SRB client environment to enable access to multiple SRB servers at the same time and by extending the multi-Grid capabilities of P- GRADE to SRB resources. and also to copy SRB output files back to the SRB resource. workflow level integration of P-GRADE and SRB was necessary [46]. located in several different grids.CoreGRID FP6-004265 24 Figure 9: Generic requirements towards intra-workflow interoperation of grid data resources TeraGrid) and are running on multiple grid middleware. SRB and SRM based file systems.Network of Excellence . As P-GRADE already supported accessing GridFTP and SRM catalogues. This integration raises three challenges.

Network of Excellence . and the computing resources available on the database server are not adequate to run this analysis. Jobs of one workflow can run in different grids (US OSG. local) from these different grids. EGEE) and utilise data resources based on different technologies (SRB. UK NGS. It is quite common in an e-science scenario that the result of a query produces a set of data that forms the input of a parameter sweep experiment. P-GRADE CoreGRID . This is justified by the huge size of databases and also by the fact that the service representation requires the conversion of data to SOAP-XML format. a query).CoreGRID FP6-004265 25 Figure 10: Workflow level interoperability with P-GRADE Grid Portal files the portal machine can easily become a bottleneck if all SRB file transfers occur and terminate on this machine. When integrating OGSA-DAI to P-GRADE we have to keep this very important feature of OGSA-DAI in- tact and move the data only when it is absolutely necessary. GridFTP. The primary aim when accessing a database in the grid is to move computa- tion to the data rather than the other way round. In order to achieve the generic interoperation of grid data resources data coming from relational or other databases has to also be accommodated into workflows. SRM. OGSA-DAI was selected as the middleware level solution to be integrated with P-GRADE due to its flexible architecture and widespread utilization by the grid community. Moving the data is inevitable when some complex analysis has to be carried out on a data set delivered as a result of a database operation (e. GridFTP file systems and EGEE storage elements (based on SRM) at the level of P-GRADE workflows. A generic tool exposing database operations as services is OGSA- DAI. In order to overcome this shortcoming the implemented solution utilises direct file transfer between the SRB resource and the executor site whenever it is possible. This workflow level integration now allows the seamless interoperation of SRB catalogues.g.

Based on the above. an important requirement towards the integration is that data coming as a result of OGSA-DAI queries should be able to serve as input to these param- eter sweeps. to form the input of parameter study workflows in P-GRADE. The integration of OGSA-DAI and P-GRADE requires the definition of a new job type. the major principles of OGSA-DAI P-GRADE integra- tion are the following: • Allow a set of database operations to be delivered to OGSA-DAI resources as P-GRADE workflow components (move operations to data). an OGSA-DAI job in P-GRADE is a job that receives its input as a perform document. The job itself is responsible for interpreting this document. It is then the responsibility of the following jobs in the portal to interpret this response document and use it. CoreGRID .Network of Excellence . the only feasible solution is to move the data to computational resources where the analysis is carried out. as explained in [4]. for example as input for parameter sweep workflows. when it is required. connecting to the defined OGSA-DAI service and returning the results as a response document. called OGSA-DAI job in P-GRADE. As the parameter study workflows may require hundreds or ever thousands of resources to run. There- fore. The most important ques- tion here is what an OGSA-DAI job is doing and at what level of abstraction it should be defined by the user. OGSA-DAI receives these operations in the form of a perform document and returns them as a response document. An OGSA-DAI job describes a set of database operations.CoreGRID FP6-004265 26 supports the creation of parameter study workflows. • Allow delivering the results of OGSA-DAI operations. Therefore. Work is currently undertaken to fully implement this solution.

those that transparently use the Grid resources. to use very few API methods.1.1 Component-Based Integrated Toolkit The Integrated Toolkit.. described in detail in 5. 5.. Grid−unaware application Grid−aware application PSE integrated toolkit user portal application tuning application−level steering . automatically deciding which tasks can be run at every moment. The best suitable applications for the Integrated Toolkit are those with large granularity tasks. is a framework that enables the easy development of Grid-unaware applications.CoreGRID FP6-004265 27 5 Development and Execution Environment 5. On the other hand.Network of Excellence . the Integrated Toolkit runtime. depicted in Figure 11. • Data-dependency checking at task level. PSEs and portals. i. and it builds on top of the Mediator Components and the Service and Resource Abstraction Layer. It gives support to different kinds of Grid applications. It is mainly formed by an interface and a runtime. On the one hand. manager component information cache component steering service and resource abstraction layer interface security context application resource application information monitoring persistence broker meta−data services services service services repository Figure 11: Generic component platform CoreGRID . 64]. previously presented in [20.1 Integration in a Generic Component Platform The Integrated Toolkit is a part of our Generic Component-based Platform.2.1. optionally. the Integrated Toolkit interface offers a simple program- ming model which only requires the user to select the parts of the application that will be executed on the Grid (the so-called tasks) and. • Task scheduling and resource selection taking into account task con- straints and performance aspects.e. has the following features: • Performance optimization of the application by exploiting its inherent concurrency.

etc. thanks to the dynamic and reconfigurable features of the GCM. a set of user-defined constraints for the task. CoreGRID . the available Grid resources and their capabilities. This set of com- ponents allow to integrate Grid resources and services into one overall system with homogeneous component interfaces. file transfer. by means of a uniform interface for job submission. Concerning the lower layers in the component platform. in order to provide the Integrated Toolkit with graphical deploy- ment.2. corresponding to the default controller interfaces (such as the LifeCycleController [32]) and to a customized AutonomicController interface [18]. resource management. These services would allow the user to reconfigure and steer the Integrated Toolkit runtime components through the Grid IDE. This component could change its scheduling strategy on demand. The runtime components of the Integrated Toolkit could offer a set of non-functional services.2 GCM-Based Design The design of the Integrated Toolkit is based on the GCM [17] and. depending on which functionality interests the programmer. each of them in charge of a given functionality. achieving both steering and performance adaptation. it could be used together with the Grid IDE (Section 5. monitoring and steering of the application. detailed in Section 5. it looks for data dependencies between the new task and all previ- ous ones. It implements the Integrated Toolkit interface used by the application to submit tasks: when such a request arrives. and third. its runtime is defined as a set of GCM components. 5.CoreGRID FP6-004265 28 Although the main purpose of the Integrated Toolkit is to provide a straight- forward programming model for Grid-unaware applications. therefore. building a task dependency graph. Furthermore.3) offer system-component capabilities to the Integrated Toolkit.Network of Excellence . comprises the following components: • Task Analyser : receives incoming tasks and detects their precedence. When a task has all its dependencies solved. inspired on the GRID superscalar framework [9]. while heterogeneous software architec- tures and technologies are situated underneath. makes possible to use it as a whole or to deploy solely specific subcomponents.1. it could also be an alternative to develop Grid-aware applications: the componentised structure of the Integrated Toolkit. The design. the location of the data required by the task. the Task Analyser sends it to the Task Scheduler. the Integrated Toolkit also relies on a Service and Resource Abstraction Layer which represents an abstraction from the underlying Grid middleware. second. Moreover. • Task Scheduler : decides where to execute the dependency-free tasks re- ceived from the Task Analyser. as well as to obtain dynamic information about the execution of the application.1. the Mediator Com- ponents (Section 5. This decision is made accordingly to a certain scheduling algorithm and taking into account three information sources: first.2).

The former gathers all in- formation related with files: what kind of file accesses have been done. and non-functional interfaces.Network of Excellence . provided by default or custom controllers. which versions of each file exist and where they are located. When the transfers for a task are completed. multicast com- munications).CoreGRID FP6-004265 29 • Job Manager : in charge of job submission and monitoring. and then controls the proper completion of the job. The latter is the component that actually transfers the files from one host to another. it transforms the task into a Grid job in order to submit it for execution on the Grid. • Synchronous and asynchronous communications. • File Manager : takes care of all the operations where files are involved. • Separation between functional interfaces. It could implement some fault-tolerance mechanisms in response to a job failure. Task Task Job Analyser Scheduler Manager File Manager File File Information Transfer Provider Manager Figure 12: Design of the Integrated Toolkit runtime The design presented above benefits from the following GCM properties: • Hierarchical composition. it also informs the File Information Provider about the new location of files. CoreGRID . • Collective interactions between components (in particular. It is a composite component which encompasses the File Information Provider and the File Transfer Manager components. • ADL-based description of the component structure. It receives the scheduled tasks from the Task Scheduler and delegates the necessary file transfers to the File Manager. used to access the implementa- tion of the functionalities that the component offers.

A would remain blocked waiting for the result of the call and it could never serve the stop control request. If one invokes the stop method of the Integrated Toolkit life-cycle controller (stopFc. File Information Provider (FIP). otherwise. etc. is concretely the following one: Task Analyser (TA). we could experience a deadlock.CoreGRID FP6-004265 30 5. The problem arises if B is stopped before it can serve the request from A. the Integrated Toolkit runtime can experience errors of different kinds: a job submission that has failed. Job Manager (JM). the synchronous calls between subcom- ponents lead to the dependencies shown in Figure 13. Both of these proposals focus on the ‘stopping’ CoreGRID . and such dependencies impose a stop order that must be respected. File Transfer Manager (FTM). in the case of the Integrated Toolkit. the components that form the Integrated Toolkit cannot be stopped in any arbitrary order because they have data dependencies. two ver- sions of an algorithm that automatically determines the proper stopping order for a structure of components.1. two alternatives are proposed: on the one hand. a failure in one of them could impede the overall system to work properly. on the other. a language for the assembler (the person in charge of composing the system) to specify the stopping sequence of the subcomponents of a composite. TA TS JM FM FIP FTM Figure 13: Data dependencies between Integrated Toolkit subcomponents In order to address this problem. an exception in some point of the code. see [32]) the call is forwarded to all the hierarchy of components in an a priori unknown order. however. A dependency between two components A and B appears when A invokes a synchronous method on B and waits for its result. Task Scheduler (TS). a solution has been presented in [63]. managing an error produced inside the Integrated Toolkit while it is working is not a trivial issue. Since all its subcomponents are interconnected and commu- nicate constantly.Network of Excellence . It basically redefines the behaviour of the life-cycle controller to ensure that a structure of components is stopped in an adequate order that. a problem with a file transfer. The general response to such a situation should be to stop the components as quickly as possible. Unfortunately. Nevertheless. In that sense. in that case.3 Life-Cycle Controller Modification to Prevent Deadlocks During the execution of the application.

2 Grid Integrated Development Environment . Figure 15: Component-Based Program Development Pipeline Our philosophy is to restrict the programmer as little as possible. Given that the underlying component model for our platform is GCM we use Java and Eclipse as the development platform. this time extending the ‘started’ state. providing means to orchestrate the internal operation of this state. Grid Interactive Development Environment Obtaining Application Program / Monitor & Compose Deploy Solution (Algorithm) GCM Steer Metadata Description incl.1) and the ProActive library [61].3 will present a complementary modification of the GCM component life cycle. As can be seen from Figure 15 this includes graphical composition. deployment. ADL. This enables the maximum integration with the Integrated Toolkit (Section 5. as well as coding directly in a suitable computer language using some middleware API.Network of Excellence . etc.CoreGRID FP6-004265 31 receiveStop()? started stopping sendStopToSubcomp()! [all subcomponents are start()? in stopped state] sendStopToParent()! stopped Figure 14: Default behaviour of the life-cycle controller state shown in Figure 14. and enable the developer full access to all levels of the language hierarchy. Later in this document. Eclipse is also well known for its extensibility via the development of suitable plugins and CoreGRID . monitoring and steering of grid based ap- plications. By language hierarchy we mean that the developer will be capable of switching between developing graphically via a component model view.GIDE Our vision for the GIDE is to provide the developer with a single environment for the development pipeline. Section 5. 5.

1 User Groups The Grid IDE is aimed at supporting a number of different user groups. The model is well supported by a front end based on the Graphical Editing Framework (GEF) [25] and inherited features from GEF. Figure 16 gives a block diagram representation of the GIDE design. In addition to this. monitoring and steering are also being developed as plug-ins. includ- CoreGRID . The underlying architecture for the composition view relies on this model for the functionalities. This ensures that the target user groups can benefit from a richer set of functionalities [33]. The main advantage of relying on this plug-in-based approach is that specific features can be activated (plugged-in) on demand.Network of Excellence . This approach aligns with industrial efforts in building applications through graphical composition [27]. we followed the model driven approach using the Graphical Modelling Framework (GMF) supported by the Eclipse platform. Some monitoring capability is already present in the In- teractive Control and Debugging of Distribution (IC2D) application [13] which provides graphical monitoring of Active Objects.CoreGRID FP6-004265 32 hence provides a seamless path from code to component model in a uniform development platform. such as event handlers.2. APPLICATIONS (USE CASES) USES (Requirements) API Eclipse Framework Development IDE IC2D Data Centre IDE Composition Deployment Monitoring Steering GIDE Toolset ADL Code/ Node ADL Finalization Component Install/ Parser/ ADL Test Debug Resource Start/Stop Renderer Deployment Monitor Remove Verifier Generator Monitor Figure 16: GIDE Block Diagram In designing the front-end. 5. deployment. We are in the process of ex- tending this functionality in order to enable the deployment and monitoring of components. providing support for developing Grid applications poses additional requirements. However. We can classify the user groups as follows: Application Developers: Application developers require support for developing Grid applications through graphical composition as well as having to support source-code based development.

Eclipse acts as the host platform to our environment. which are then routed to the dedicated event handlers or providers of the environment. GMF-Runtime. This functionality is also shared with the application developers who need such facilities to test the application during development. These event handlers or providers are linked to the underlying model so that the changes are reflected upon editing. The Graphical Editing Framework. and steering perspectives. CoreGRID . monitoring of both component status and resource.2.2 Composition The composition process is enabled via a fully interactive environment. These events are captured by the host platform through a message loop processed by the Eclipse. Components can also be im- ported from existing ADL files and stored in the component palette. Addi- tional tools are necessary to enable deployment. Hence there is a need for a design that would facilitate fast handovers and enable other operators to assist newcomers in coming to terms with the applications quickly. Composition is achieved by drawing a membrane around a group of components and defining interfaces. the monitoring of deployed applications. modified and stored. The underpinning feature which enables such interactivity is the event driven ap- proach.Network of Excellence . Components can then be resized and moved. and a view of the fractal description of the component as an ADL file. Also. The monitoring process provides a set of opportunities for concerned users to monitor their running application in real-time. 5. Data Centre Operators: Data centres have high turnover rates. and the complexities of deploying these components over distributed systems. The central area focuses the user on the graphical composition view which provides the developer with a palette of available components that can be dragged and dropped on the composition canvas. ADL files conform to the GCM-specification for describing compositions such as in [31]. so that a uniform design is visible to all operators in order to enhance handover and communication. In order to achieve this the GIDE is also available as a standalone Rich Client Platform (RCP) application. The following are the four GIDE perspectives that meet these needs. The developer is able to switch between the graphical view. monitoring. A prototype has been completed for the Composition perspective (see Fig- ure 17). The ADL file can then be exported and used for deployment. personalisation of views should be limited as far as possible. which provides only the key functionalities that would be required in a data centre environment. Application Users: The GIDE should facilitate the deployment of applications and subsequently. and steering of components to maximise resource utilisa- tion. These features are arranged within the deployment.CoreGRID FP6-004265 33 ing support for Grid component models and composite components. Connec- tions between the interfaces can be drawn directly between the components using the connection tool. and Eclipse facilitate handling of different events within the environment.

See Figure 18 for an example Monitor perspective consisting of component and resource monitor views. Three types of monitoring are necessary in order to enable proper management of applications.4 Monitoring Perspective The monitoring perspective will provide the views that data centre operators need in order to properly monitor the environment in which components oper- ate. To complement this view. 5. hard disk CoreGRID .CoreGRID FP6-004265 34 Figure 17: Component Composition Perspective 5.Network of Excellence . a view of the hosts and their resource statuses is also provided.3 Deployment Perspective This perspective will consist of views needed for application deployment. Within the deployment perspective the oper- ator is able to launch components simply via drag-and-drop operations before moving on to steering. monitoring of resources provides the hardware status of hosts.2. giving a developer the ability to associate sets of hosts with each deployment descriptor.2. Deployment descriptors are used to associate components with virtual nodes. The main view is of a deployment descriptor editor to map physical hosts to virtual nodes. A developer may have a set of these deployment descriptors to be used for deployment to different hardware con- figurations. This includes CPU utilization. Firstly. Virtual nodes are included in ADL files to specify the number of vir- tual nodes that the component will need.

This view graphically shows the components location and their status. Finally. monitoring of the GCM components themselves provides status and location information along with a zoom-in feature for monitoring sub-components. CoreGRID . Figure 18: Monitor Perspective 5.5 Steering Perspective More useful for data centre operators.2. the aim of the steering perspective will be to provide views to enable the operator to start. the operator has the facility to start. Secondly. stop or move components from one virtual node to another while monitoring their status to ensure correct execution. and other platform specific status information. Based on these views. as well as the components that are running on them. stop and relocate components.CoreGRID FP6-004265 35 space. virtual nodes. Building on the monitoring and host views. An additional view shows the geography and resource availability of the hosts. it has as its main focus a component monitoring view.Network of Excellence . we allow monitoring of active objects. which is necessary for developers/composers to debug and monitor applications during the development phase.

or even both. in the following. It implements a set of component interfaces to various kinds of Grid services and resources. For example. achieving both steering of the application and performance adaptation by the application to achieve the most efficient execution on the available resources offered by the Grid. The concrete set of mediator components is considered to be flexible and extensible to future application needs. Application-level meta-data repository This repository is supposed to store meta data about a specific applica- tion. or the Java-GAT [55]. as shown in Figure 11. The mediator components are an integral part of the generic component platform. The collected information is used by other components to support resource management (location and selection) and to optimize further CoreGRID . Adding such an interface contributes to the complete integration of application and system components by enabling communication in both directions.Network of Excellence . Its goal is to integrate system-component capabilities into application code. file systems. via mediator components to the underlying system software. it provides an abstraction layer between application components and system components. By introducing such a set of components.CoreGRID FP6-004265 36 5. etc. PSE’s. Steering interface A dedicated part of the service and resource abstraction layer is the steer- ing interface. timing or resource requirements from previous. SAGA [43]. as shown in Figure 11. Service and resource abstraction layer This is the lowest layer of an application runtime stack. independent of the service architecture actually deployed..g. or an application manager) like any other component in the sys- tem. e. This component-level interface is supposed to make appli- cations accessible by system and user-interface components (like portals. like OGSA/OGSI or WSRF. storing. The interfaces are supposed to be implemented by a delegation mechanism that forwards invocations to service providers. like job schedulers. One obvious additional use of such an interface would be a debugging interface for Grid applications. Doing so. related runs. we use the term Grid services for referering to services offered within a Grid platform. Examples of such runtime environments are the GAT [4]. resources and services in the Grid get integrated into one overall system with homogeneous component interfaces.3 Mediator Component Framework The mediator component toolkit has been introduced in [19]. Canonical example components are discussed in the following. The strength of such a component-based approach is that it provides a homogeneous set of well-defined (component-level) interfaces to and between all software systems in a Grid platform. The advantage of such a component system is that it abstracts from the many software architectures and technologies used underneath. ranging from portals and applications.

this component may have to prefetch (poll) information from the various sources to provide them to the application in time.g. CoreGRID . which can be used to improve the execution time of applications automatically as well as for improving ser- vices and tools which are involved in the environment. Application-level information cache This component is supposed to provide a unified interface to deliver all kinds of meta-data (e. including mechanisms for service and information discovery. Such an application manager is in charge of guaranteeing such successful completion in spite of temporary error conditions or per- formance limitations. carrying their own threads of ac- tivity. Steering and tuning components are supposed to utilize both the application-level meta data repository and the application-level information cache components. from application-level meta data) to the application. The meta data to be stored in this repository needs to use application-dependent schemas. For the latter purpose. as well as available steering components. user portals. An implementation of this component has been described in [5]. Its purpose is twofold. the application meta data repository and cache. This way. in charge of tracking an application from submission to successful completion. the crux of its design will be to provide efficient mechanisms for storage and retrieval without built-in knowledge about data structures. user authentication and authorization become important issues. The organisation of such a repository with namespaces for users and applications poses fur- ther challenges. e. the application man- ager needs to interoperate with most of the other mediator components. but also for im- plementing pro-active steering systems.g. (In other settings.) For performing its task. a monitoring system. via application managers. requires a component-level interface to give external components access to the application. such an active component might be referred to as an agent. from a GIS. like the application itself (via the steering interface). also dedicated steering components are necessary. Application manager component The application manager component establishes a pro-active user inter- face.. this application-level cache is supposed to deliver the information really fast. it is supposed to provide a unifying component interface to all data (independent of its actual storage). the steering components provides a framework for performance tuning. Second. Besides the steer- ing interface. both for mediating between application and system components. Application steering and tuning components Controlling and steering of applications by the user. cutting down access times of current implementations like Globus GIS (up to multiple seconds) to the order of a method invocation. and PSE’s. First. These components have to interface with Grid monitoring services and Grid resource brokers.Network of Excellence . As this repository encompasses access to stable storage.CoreGRID FP6-004265 37 runs of the applications automatically.

1 Classification of mediator components With respect to their functionality. This group of components is intended to be integrated directly with the application code. manager cache runtime environment app. app. info.Network of Excellence .3. These components are merely passive code. We distinguish two sub categories: (a) Passive components. They provide Grid-related functional- ity to the application.CoreGRID FP6-004265 38 application steering tuning app. These are the following: • runtime environment providing the general-purpose interface to the Grid. their implementation. Fig. comp. running within the appli- cation’s process environment. 1. the proposed mediator components can be categorized as in the following. and their integration into an overall Grid component platform. 11. providing a proxy to the application persis- tence service (see below) • application-level information cache providing up-to-date status information about resources and their performance CoreGRID . 19 shows this categorization of the components and services from Fig.− appli− level cation comp. including steering interface and security context (not shown in the figure) • application manager in a passive variant. persis− persis− compute compute tence tence data server server service appli− moni− resource file cation toring meta data broker service server repository meta data services Grid resources Figure 19: Mediator component classification 5. and the corresponding process environment at runtime. Application-level components.

CoreGRID FP6-004265 39 • tuning component in a passive variant. possibly for paus- ing or migrating an application • tuning component proactively monitoring resource performance and actively deliv- ering related events to an application 2. both of applica- tions and of other mediator components. using status information to tune application performance • steering component providing a steering interface to the application (b) Active components. Likely.3. 3. These are the following components: • service proxy components providing a component interface wrapper to existing services like resource brokers or monitoring services • application persistence service providing a persistent service that ensures applications can run to completion in spite of transient error conditions.2 GCM-based design In the following. as published CoreGRID . without further user interaction. Such components are typi- cally bundled with an instance of the runtime environment to form an individual (application) process. This group of components provides access to meta data. but independent from the application itself.Network of Excellence . Service-level components. Meta-data components. we will present the design by which application components and mediator components are integrated with each other using GCM. An implementation of such a service has been described in [47]. For pro-active functionality. in behalf of a given application. • application persistence data repository providing persistent status information to the application persistence service • application-level meta data repository providing persistent status information to individual applications 5. This group of components is closely-related to Grid services that are op- erating persistently within a Grid platform (virtual organization). Examples are: • application manager providing external requests to an application. active components are beneficial that operate as individual processes. a single implementation will be able to serve all necessary instances of this group.

CoreGRID FP6-004265 40

in [28]. For adaptation either by tuning and management components, or by the
user via the steering component, the application components have to be called
by the mediator components. For this purpose, we propose the interface shown
in Figure 20, with specialized controllers that are added to the application
components’ membrane.
Based on experience gathered when investigating the GCM component frame-
work, we propose two extensions. First, in order to effectively modify the struc-
ture of a running application, we propose to implement an explorer component.
Thus, the user could switch between different implementations of his/her algo-
rithms without the need to stop and re-run their application.
Second, the mediator toolkit can greatly benefit from implementing different
control aspects of the application separately, namely by using controllers. We
propose to introduce into the architecture the following controllers, as depicted
in Figure 20:

• steering – for modifying application parameters, which would allow for
computational steering during runtime

• persistence – for handling checkpoints: initiating checkpoints, as well as
starting (from checkpoint or from scratch) and stopping the application

• distribution – for optimal utilization of allocated resources, and for adapt-
ing to changes in environment (releasing and acquiring resources, changes
in quality of network connections)

• component – for investigating the application’s structure (in terms of com-
ponents) and modifying it (e.g. switching to alternative implementation,
replacing subcomponents)

Note that the component controller is already implemented in GCM. How-
ever, the other controllers have to be added according to the necessary func-
tionality. Another important observation is that communication with the ap-
plication is via its controllers only.

Persistence Controller and Life Cycle Controller The Persistence Con-
troller is the manager of an application instance. Not only is it responsible for
checkpointing, but also for starting (from scratch or from a checkpoint) and
stopping an application. For this to be accomplished, we propose bounding the
Persistence Controller with GCM’s LifeCycleController. The latter is a simple
state automaton (with two states: started and stopped). We propose extending
the state-cycle to service checkpointing, shown in Figure 21.
We propose to extend the started state of the component by adding substates
representing different stages of the running application (created, initialized, run-
ning, and finished), and checkpointing.
The GCM LifeCycleController is responsible for starting and stopping the
component. There are certain conditions under which a component can be
stopped. For example, all method invocations on this component should have

CoreGRID - Network of Excellence

CoreGRID FP6-004265 41

application
components
application Grid−unaware with integrated tookit or Grid−aware

steering persistence distribution component
controller controller controller controller

app−level
components

steering application tuning info cache explorer
component manager component component
toolkit

component

security context
components

application
system

PSE resource information monitoring meta−data
user portal broker system system repository

Figure 20: Generic component platforms with application controllers

STARTED

CREATED INITIALIZED
STOPPED

FINISHED RUNNING

CHECKPOINTING

Compound state of GCM LifeCycleController
State of extended LifeCycleController

Figure 21: Extended states of LifeCycleController

finished (a special interceptor keeps a counter of active method invocations).
Similarly, only a component with all mandatory interfaces bound can be started.
Our system also benefits from this approach. Transitions between stopped
and started states are limited to only a few started substates. The component
must not be allowed to stop while checkpointing is in progress. Additionally,
stopping an application in the running state could mean interrupting the ap-
plication (transition to finished) first.

Application controllers The proposed application controllers (steering – sc,
persistence – pc, distribution – dc, and component – cc) are implemented as
GCM-controllers, part of the membrane, shown in Figure 22 (left).

Dealing with GCM-aware and unaware applications The GCM Media-
tor Toolkit is ready to run not only with applications that have been developed

CoreGRID - Network of Excellence

CoreGRID FP6-004265 42

application
component

application
component sc

pc
dc cc

sc pc dc cc

Figure 22: Controllers inside the component membrane (left) or as subcompo-
nents (right).

with GCM in mind, but also with application objects (rather than components),
from “legacy” applications, as shown in Figure 23.

GCM-unaware application GCM-aware application
Default application component
Default Distribution
Controller User application component
Application manager Application manager dc
Default Persistence
Controller
dc
Application
pc
Default Componentpc
Controller
cc
Default Steering
am cc Controller am
sc

sc

Figure 23: Integrating applications with mediator controllers.

GCM-unaware applications The framework is able to cooparate with ap-
plications that do not use the GCM framework. In that case, a set of default
controllers is created. A user application is encapsulated by the default persis-
tence controller as this component is responsible for starting and stopping the
application, and it has direct access to the application object. This controller,
together with the default distribution, component, and steering controllers, are
integrated to a default application component, which is bound to the rest of
the framework via the application manager.
The default implementations of controllers are very simple. Only the persis-
tence controller is able to perform some actions – starting and terminating the
application. All other methods in this and the remaining controllers throw a not
implemented exception, as they cannot be provided without specific knowledge
about the application.

GCM-aware applications These are very easy to connect to the framework.
The only requirement towards the application developer is to deliver a GCM

CoreGRID - Network of Excellence

pc. • Workflow components can be Web services. • Workflows can be combined with collector components that perform some collective evaluation of the partial results that are generated by the various workflow instances. CoreGRID . production level scientific e-infrastructures. • Workflows can be combined with generator components that create input parameters automatically for parameter study simulations. 5. Short computations can be executed locally on a gUSE server. scalable and flexible services for Grid and Web service users. Grid sys- tem administrators can benefit from the loosely-coupled gUSE architecture to deliver stable. graphical interfaces. Internally. gUSE enables the creation.Network of Excellence .4 gUSE . 5.1 gUSE as a Grid application developer environment Application developers use gUSE to compose complex applications from ele- mentary Web services and batch jobs. end users can apply gUSE to select pre-defined calculations and execute them with their own custom parameters.A Service based Environment for Grid Application Developers and End Users gUSE (Grid User Support Environment) is a flexible and scalable Grid appli- cation development and enactor infrastructure that provides user environment on top of widely adopted. Grid application develop- ers can apply gUSE to design complex calculations on top of Web service and Grid service infrastructures. avoiding communication with remote host and resulting better performance.4. execution. cc. monitoring and sharing of distributed applications through intuitive. while the connections between the nodes represent data transfer between these algorithms. Globus Toolkit 4 or gLite service hosts. realizing hierarchical appli- cation structures in which any component of a workflow can also be the same workflow. where the nodes of a workflow are able to perform an elementary (or complex) computation. Such calculations are represented as data-driven workflows. testing. or jobs that are able to run as batch computation on Globus Toolkit 2. right) with exported interfaces for each of the controllers (dc. they are expected to be bound to the user’s implementation of the controllers. • Grid Application Developers can connect batch programs with Web ser- vices into data-driven workflows that perform complex calculations based on elementary computational steps. see Figure 22. • Applications can be nested into each other. and sc). • Any part of a workflow or even the whole workflow can be executed mul- tiple times in a parameter study fashion in order to perform the same computation for input different parameters.CoreGRID FP6-004265 43 component (user application component.

CoreGRID . or by a remote process. Globus Toolkit 4. application end users and their collab- oration. gLite and Web service networks with functions spe- cialized for application developers. fixed parameters cannot. • End user can be notified in email about relevant events related to the progress of the Grid application. • The execution of an application can be monitored and visualized by graph- ical interfaces. 5. file storage. A graph defines the graph topology that can be extended to a concrete workflow by specify- ing services or grid jobs to it. Any service can be replaced by another implementation. work- flow management or job submission. each of them playing a well defined role in the Grid application lifecycle . fault tolerant layer to ease Grid system management • gUSE is implemented as a set of services. any service-to-service communication protocol can be re-implemented. • The start of a user application can be initiated manually through the gUSE GUI.4.Network of Excellence . well tested applications that are ready for execution. e. at the level of concrete workflows and at the level of a workflow instance.4. or can be left open.g. accessing the gUSE application repos- itory with a Web service call. Reusabil- ity provides flexible inheritance between applications from the different abstract levels. Open parameters can be changed by end users. A workflow instance includes all the input parameters and it is mapped to Grid resources for execution.CoreGRID FP6-004265 44 • A Grid application can be fed with input parameters from files or from databases that are accessible through JDBC interfaces (Java Database Connectivity). • Additional support is to produce reusable Grid Applications: a grid ap- plication can be abstracted at the level of a graph. tested and monitored to result a well tested application that can be offered to end users.2 gUSE as an end user environment • Built-on application repositories can be used to load pre-defined. These Web services discover each other through a central gUSE information system and communicate using SOAP messages. • gUSE services extend the basic but generic functionality of Globus Toolkit 2. • Any parameter of a pre-defined application can be fixed.3 gUSE as a scalable. • Grid applications can be executed. 5.

or it can be a higher level service that e. 5. • gUSE provides APIs to easily interface the system to other Grid systems or distributed computing platforms. can be started multiple times to avoid that function becoming a performance bottleneck.Network of Excellence . WS-PGRADE hides the communication protocols and sequences behind JSR168 compliant portlets. WS-PGRADE uses the client APIs of gUSE services to turn user requests into sequences of gUSE specific Web service calls. load balances between existing gUSE job submitter services.g.4. A graph editor component can be downloaded from WS-PGRADE via the browser to the user CoreGRID .4 gUSE system architecture gUSE is a set of Web services that bind together in flexible ways to deliver user services in Grid and/or Web services environments. WS-PGRADE is one implemen- tation and can be replaced by specialized user interfaces if necessary. Figure 24: The three tiers of the gUSE architecture End users can access WS-PGRADE via Web browsers.CoreGRID FP6-004265 45 • Any member of the gUSE service set can be stopped and restarted without stopping the whole user environment. Such an additional service can be for example an interface submitter service to utilize a Grid or cluster job manager. Any gUSE service can be dupli- cated. User interfaces for gUSE services are provided by the WS-PGRADE Web application (See Figure 24). • WS-PGRADE (Web Services Parallel Grid Runtime and Application De- veloper Environment) is the graphical services through which the rest of the gUSE service stack can be utilized. WS-PGRADE is a Web portal hosted in Gridsphere framework.

gUSE services provide data services for users (user database.). data management on behalf of the users. job submission. The lowest level of the architecture is the services of Web service.Network of Excellence . and control services in the system (e. The middle tier of the architecture constitutes of gUSE services. The editor can be used to define the static skeleton of workflows. application code database. workflow database. Grid services typically provide job execution. etc. workflow enactment. These services discover each other using a special gUSE service. proxy credential database. gUSE services connect to this tier to perform computation. CoreGRID . the information system. etc. large scale file storage and brokering services in a secured way. while Web services make pre- defined computational algorithms accessible via SOAP messages. Grid service networks.g.CoreGRID FP6-004265 46 machine. while the HTML pages of WS-PGRADE provide interfaces to add content to graphs. application history and change log database.). to generate complete Grid/Web service applications. gUse services can be hosted on one ore more server machines in order to optimize resource usage or end-to-end per- formance.

aspects that affect how the final results are com- puted rather than what results are actually computed. last but not least.e. Here we con- sider a particular subset of autonomic components. In particular. performance tuning and. parallel composite components. performance problems include proper parallel activity scheduling and mapping. in such a way these primitive com- ponents are assembled according to a well known parallel pattern in a new composite component. In this section we present some of the results related to component autonomic management of non functional properties. we call this kind of components behavioural skeletons. as the composite components we are taking into account structure the inner components of the composite according to well known (parallel) algorith- mic skeletons. 6. The term borrows the skeleton keyword from the algorithmic skeleton research track. that is components conform- ing to the Fractal [32] and GCM [17] specifications that are hierarchically built from other. As described in [3]. to the aspects related to dynamic adaptation of execution to the target architecture features. more specific problems. security and fault tolerance.1. W are the inner component instances.CoreGRID FP6-004265 47 6 Autonomic Support 6. load balancing.1 Autonomic components Autonomic components are components that autonomously take care of some non functional properties aimed at achieving some specific goal. adap- tation of program configuration to varying features of the target architecture. As an example. then we discuss the techniques used to introduce autonomic behaviour within components and eventually we discuss results achieved with autonomic GCM components so far. in particular. programmers must deal with a full set of problems that are not directly related to the functionality of the application but rather to non-functional aspects. Figure 25 shows a functional replication behavioural skeleton.e. whose autonomic control takes care of all the aspects related to efficiency in com- posite component execution and. i. just to name a few. All these “general” problem classes.1 Autonomic support of non functional features in GCM The development of efficient component based Grid applications raises some problems that are specifically related to the dynamic features of the Grid target architectures. S represents the port collecting tasks from the the outside and dispatching them to the W s and C represents the port collecting CoreGRID . i. namely: • structured. include other.Network of Excellence . We first introduce autonomic com- ponents. • whose autonomic control takes care of performance optimization. Such problems include performance tuning. in turn. simpler components. The adjective behavioural reflects the fact autonomic control co-designed with the skeleton structure of the composite adapts behaviour of the composite to the “local” (in temporal and spatial dimension) properties of the target architecture.

that can be summarized as follows: ac.3 the corrective actions are executed exploiting again the interfaces provided by the autonomic controller. In this task farm behavioural skeleton: • the composite component hosts a number of identical inner components. corrective actions are planned according to one of pre-defined strategies programmed in the manager and specific of the parallel computation pattern modelled by the behavioural skeleton ac. business data filtering. tak- ing care of the non functional aspect related to the performance and effi- ciency achieved while computing the embarrassingly parallel computation • the composite component hosts a passive autonomic controller. The autonomic manager of the task farm skeleton implements a classical autonomic control cycle.Network of Excellence . providing methods to access composite component status and performance parame- ters and measures.CoreGRID FP6-004265 48 (partial) results from the W s and delivering them for further processing to other components. and the cycle is restarted Sample autonomic policies implemented in the task farm autonomic man- ager include the possibility to increase the number of instances of the inner component when computation requests arrive more frequently than actual re- sult dispatching as well as substitution of non responding inner component instances (e.1 performance parameters are collected exploiting the autonomic controller interfaces ac.g. each computing the application producing output data from an input task data set • the composite component hosts an autonomic manager component. etc. as well as methods implementing the actions eventually invoked by the autonomic manager • the composite component provides interfaces to submit tasks to be com- puted and to gather results (functional interface) as well as interfaces to interact with the autonomic manager and with the autonomic controller (non functional interfaces).2 performance figures are analysed. that is computations where a set of independent input tasks (data sets) are processed by the same application code to produce a(n ordered) set output data sets. In particular they are compared to theoretical figures and. in case of mismatch. An example of behavioural skeleton is the one modelling embarrassingly parallel computations. parameter sweeping. CoreGRID . due to network faults or overloads). This kind of parallel pattern is well know and largely used in grids to perform image processing.

Network of Excellence . the features discussed in Sec. Within our GCM task farm behavioural skeleton template. ORC) Figure 25: Generic functional replication GCM behavioural skeleton 6.g. adding an inner component instance (a worker) 3. Task farm behavioural skeleton has been developed as a (prototype) representative of the full class of behavioural skeletons.1. It provides the composite component the interfaces needed to: 1.CoreGRID FP6-004265 49 AC AM W Functional Functional server port S W C client port W skeleton behaviour (e.2 Autonomic behaviour support in GCM We discuss how the behavioural skeletons are implemented in GCM by de- tailing the implementation of the task farm behavioural skeleton. start/stop autonomic manager activity 2. We already have a prototype implementation of the managers with policies expressed via “business CoreGRID . removing an inner component instance • Autonomic manager (AM) is implemented currently as an (active) object. actually. 6. The ABC provides the composite task farm component some interfaces: 1. namely the Autonomic Behavioural Controller (ABC). accessing service time and interarrival time of the composite 2.1 above are implemented as follows: • Autonomic controller is a plain Fractal controller. Its implementation has been cus- tomized from classes of a more general functional replication skeleton that can be specialized to implement data parallel (stateless and stateful) behavioural skeletons as well. We have plans to provide it as a full fledged Fractal membrane component in the future. provide/substitute autonomic management policies Autonomic policies are actually currently implemented directly program- ming Java methods in the manager object.1.

Medium to coarse grain applica- tions have been demonstrated to scale on all the architectures considered. • behavioural skeletons timely react to changes in the load of the resources used to run the composite component. such as in the task farm behavioural skeleton and in case of stateful computations.25) and $workerBean. the former being an access to the “getter” interface providing service time measure.1. • behavioural skeletons timely react to increases in task submission speed by augmenting the instances of the inner worker components used. mainly coming from GridCOMP use cases workpackage. Such rules look like the following: rule "AdaptUsageFactor" when $workerBean: WorkpoolBean(serviceTime > 0.CoreGRID FP6-004265 50 rules” [42] and the prototype is currently being integrated in the main GCM autonomic controllers.Network of Excellence . improvements achieved in the application service and completion time. ex- ploiting the ProActive based GCM reference implementation currently being developed within the EU STREP project GridCOMP. and the latter being an access to the “actuator” ABC interface adding an inner component instance to the task farm. With all the applications we have been able to demonstrate: • scalability of the behavioural skeletons.3 Experimental results We used behavioural skeletons (that is GCM autonomic components) to run several different applications on several distributed target architectures. When additional charge is put CoreGRID . rule based implementa- tion of GCM autonomic managers we used JBoss rules. In the prototype. The target architectures considered range from network of laptops and PCs to workstation clusters (in- terconnected through Fast Ethernet) and the Grid5000 architecture. Efficiency is close to the ideal one in case the computational grain of the inner worker components is suitably coarse.t. the when clause defines the pre- conditions activating the rule and the then clause defines the consequences of (or actions related to) rule activation. 6.r.25) then $workerBean. the manager arranges to release one or more of the inner component instances in such a way efficiency is kept close to the ideal one.addWorkerToNode("") represent accesses to the ABC con- troller. In case submission speed decreases. We run simple. such as in the stateful data parallel behavioural skeletons) is negligible w. synthetic applications and more complex application as well. $workerBean: WorkpoolBean(serviceTime > 0. Re- configuration time (both in case of stateless computations. end AdaptUsageFactor is the name of the rule.addWorkerToNode("").

2 Automation of Application Deployment on Grids A generic component-based grid platform faces the issue of deploying its ele- ments.t. This is quite an “artificial” policy (JBoss rule) we used to show the overhead of reconfiguration (in this case lower than 5% w. a component contains a struc- tured set of binary executables and requirements for their instantiation. Hence. for instance.r. a deployment mediator is a solution to abstract the grid platform from the complex process of actually deploying components. finding and acquiring suitable resources. and configuring what is needed.4 Specific contributions to STE institute The design of autonomic components is currently going to be integrated in the design of IDE presented in Sec. 5. The autonomic component development also defines a design and imple- mentation methodology that has influenced other research topics in the STE institute. such as mediator components. It is worth pointing out that in principle such notable composite components and their internal autonomic management of performance features is completely trans- parent to the final users. launching processes if needed. Overall. 6. the manager detects a mismatch among the pre- dicted performance and the actually achieved values and suddenly starts a reconfiguration process. transferring required files. equivalent skeleton components not providing autonomic managers. Such a mediator provides an automatic service of application deployment because it should not rely on any human intervention [19].t. this will provide user friendly pos- sibilities to exploit autonomic composites (behavioural skeletons). inner com- ponent instances are deallocated to increase efficiency. the deployment operation involves all the phases implied from a high level order such as deploy a set of components: translating an appli- cation specific description to a system level description. Specific widgets and tools will be eventu- ally available in the IDE to allow autonomic component users to customize or even rewrite from scratch the policies and mechanisms provided by both ABC controller and the AM manager. if resources previously overloaded become more and more free. Overall. that only perceive some additional (possibly ignored) non functional interfaces w. to the total execution time. The implementation of the GCM component in this case is a prototype directly implemented on top of SCA/Tuscany and the whole GCM behavioural skeleton is provided as a plain web service accessible simply exploiting his WSDL to submit tasks. As in the case of interarrival time changes. The lower plot shows the effect of the intervention of the manager that decided to double the number of instances of the inner components after computing half of the total tasks submitted. The CoreGRID .CoreGRID FP6-004265 51 on the used resources.Network of Excellence .2. Here. this allows to provide limited but effective tool support within the IDE.r. 6.1. Figure 26 plots scalability (upper part) and reconfiguration behaviour (lower part) of the rule based implementation of the GCM task farm behavioural skeleton. From the point of view of the execution.

CoreGRID FP6-004265 52 1400 800 Measured (coarse grain) Completion time (secs) (coarse grain) Ideal (coarse grain) Completion time (secs) (fine grain) 1200 Measured (fine grain) 700 Ideal (fine grain) 600 1000 500 800 400 600 300 400 200 200 100 0 0 2 4 6 8 10 12 14 16 #worker 9 about 170 secs for 1/2 tasks with 8 N=4 inner component instances 7 6 #worker 5 4 3 Reconfiguration 2 takes place here (workers doubled) about 85 secs for 1/2 tasks with 1 K=2N inner component instances 0 0 50 100 150 200 250 300 Computation time (secs) Figure 26: Scalability and reconfiguration effect of GCM task farm behavioural skeleton CoreGRID .Network of Excellence .

and the resource ranking – an CoreGRID . can be iterated until a suitable set of resources is found. written in an application specific language. I/O data and configuration files which make up each module (e. Some steps have to be re-executed when the application configuration is changed at run-time. storage. In the following we describe the activities involved in the deployment of an application on a Grid. First. as depicted in Figure 27. b) to dynamically alter a previous configu- ration.1 Application Submission The process takes into input the information necessary to drive the following phases. 6. of their interactions. Planning. Execution).Network of Excellence . middleware systems) that must be installed to satisfy application dependencies. objectives include generating deployment plans a) to deploy components in a multi-middleware environment. the module description deals with the executable files. Application Description The description of (the components of) the sub- mitted application. like the resource constraints – characteristics that Grid resources (computational. each process). network) must possess to execute the application. is composed of various kinds of data. location within a specific network do- main. We also detail the inputs that must be provided by the user or the deployment framework to perform such activities. This information is provided through a (logical) file containing a de- scription of the components of the application.g. when a complete restart from a previous checkpoint is needed (severe performance degradation or failure of several resources). and of the required resource characteristics. Second. Selection. that interact closely. co-location. the placement policies – restrictions or hints for the placement of subsets of application processes (e. the execution platform constraints – software (libraries. adding new computational resources to a running application. Discovery. The logical order of the activities is fixed (Submission.CoreGRID FP6-004265 53 Figure 27: Activities involved in the deployment process of an application. the steps in the grey box. Enactment.2. A framework for the automatic execution of applications can be composed of several interacting entities in charge of distinct activities.g. or network performance requirements). there is information to guide the stages related to mapping the application onto resources. c) for re-deployment. Moreover.

or set of them. Information retrieved from different sources is mapped to a standard schema for resource description that can be exploited in the following activities inde- pendently from the information source.g. OS. runtime en- vironments). and to start the actual execution.g. CPU rating). must be collected. each with its own description (metadata).g. Third. including network topology and characteristics.2.2. then there should be no need for additional application description provided as input from the application developer or user unless this is needed for the composition phase prior to deployment. libraries. 6. while others can report dynamic information about resource state and performance. an intermediate translation layer between the requirements needed by the user and the information provided is necessary. access to a shared file system). some placement policies are implic- itly aggregate requirements.2 Resource Discovery This activity is aimed at finding the resources compatible with the execution of the application. services needed to deploy components (e.g. for each suitable resource found. requirements that a set of resources or a module group must respect at the same time (e.g. custom). all the resources on the same LAN. If an application is built from existing components and services. specific file transfer protocols). and particular execution services (e..CoreGRID FP6-004265 54 objective function provided by the user. The GIS can be composed of various software systems. the information needed to per- form resource selection (that considers also aggregate requirements). Then. 6. MDS-3. Some of these systems provide only static in- formation. the proper resources that will host the execution of the application must be selected. the deployment directives determine the tasks that must be performed to set up the application runtime environment. The requirements can specify hardware characteristics (e. Resource ranking is exploited to select the best resource. As discussed in the following subsections. stating the optimization goal of appli- cation mapping. implementing infor- mation providers that communicate with different protocols (MDS-2. and as aggregate requirements. NWS. iGrid. i.3 Resource Selection When information about available resources is collected.e. interacting with Grid Information Services (GIS). software ones (e.g.Network of Excellence . to con- figure the application execution environment). disk space). MDS-4. among those satisfying the given requirements for a single application process. accessible TCP ports. compilers. In the application description several requirements can be specified that available resources must respect to be eligible for execution. the provided application descrip- tion information is used throughout the deployment process. that is requirements that must be respected by a single module or resource (e. and the dif- CoreGRID . Resources satisfying unitary requirements can be discovered. CPU rating. available memory. In order to interact with such different entities. Resource constraints can be expressed as unitary requirements.

and restart the whole planning process. Thus. which is in charge of the execution of the tasks needed to deploy the application. thus the elimination of one or more eligible resources can force to find new resources. in order to set up and start the runtime environment of the application. At the end of this phase. also exploiting dynamic information. that describes the workflow of actions needed to set up the hosting environment before the actual execution can start. and finally the transfer can occur. GridFTP) must be selected.g. Moreover.CoreGRID FP6-004265 55 ferent parts of each component have to be mapped on some of the selected resources. HTTP. This activity can require repeated interactions with the resource discovery and selection phases because some problems about the transformation from the deployment schema to the deployment plan can arise. repeated interaction with the resource discovery mechanisms may be needed to find the best set of resources. the objective function must be evaluated against the char- acteristics and available services of the resources (expressed in the normalized resource description schema). The abstract plan is then mapped on the resources. the related services and resources must be started or configured. 6. additional services can be transferred/activated on the resources or configured to set up the hosting envi- ronment. the execution environment of the application must be ready to start the actual execution. This activity also implies satisfying all the aggregate requirements within the application. This activity must deal with different kinds of software and middleware systems. The implementation of the services that will perform this activity must CoreGRID .4 Deployment Planning A component-based application can require different services installed on the selected resources to host its execution. 6.Network of Excellence . identifying all the services and protocols that will be exploited in the next phase on each re- source. At this point. to transfer files a protocol (e. and turned into a concrete plan. the selection of the right ones depends on the concrete deployment plan. At the end of this phase. the concrete deployment plan has been generated. After resource selection. specifying every single task to perform to deploy the application.2. Each of these ancillary applications has a well-defined deployment schema.5 Deployment Enactment The concrete deployment plan developed in the previous phase is submitted to the execution framework.2. an abstract deployment plan is computed by gath- ering the deployment schema of all application modules. This service must ensure a correct execution of the deployment tasks while respecting the precedences described in the deployment plan. establishing a resource ranking where appropriate in order to find a suitable solution. For example.

it architecture the Grid Abstract Machine (GAM. and actually im- plements part of the GAM. Several activities have to be performed while the application is active. this phase can require interactions with the previous one. Hence.2. and actually the deployment system must rely on at least one permanent process or daemon. Adage [48] (Automatic Deployment of Applications in a Grid Environment) is a research project that aims at studying the deploy- ment issues related to multi-technology applications.6 Application Execution The deployment process for adaptive Grid applications does not finish when the application is started. CoreGRID . and to release resources when the normal termination is reached. In the ASSIST/Grid. (secure) data and code staging and execution. [2]) is a software level providing the abstrac- tions of security mechanisms.7 Validation The deployment process presented in this section has been implemented in two deployment systems GEA and Adage. 6. Changes in the state of the resources can force a new deployment plan for some tasks. The Grid Execution Agent (GEA. 6. as well as to add mechanisms to deal with new services. in order to support new resource requests for application adaptation.Network of Excellence .CoreGRID FP6-004265 56 be flexible enough to implement the functionalities to interact with different services. [22]) is the tool to run complex component-based Grid applications. One of its originality is to use a generic application description model (GADe) [49] to handle several middleware systems.2. resource selection. to schedule a restart if an application failure is detected. The whole application life-cycle must be managed. These monitoring and controlling activities have to be mediated by the deployment support (actual mechanisms depend on the middleware). resource discovery.

however. In such scenarios as dynamic deployment of component code on shared containers. Component model. The distributed component application requires communication not only between the client and components. MPI or Service- oriented architectures. • Delegation – answers the question how can a code running in Component A authenticate itself to Component B on behalf of a client. upon which the components are deployed. but without the possibility of losing the control over the delegated rights. • Authorization – answers the question what is permitted and which should be denied. 7. due to the possibility of CoreGRID .Network of Excellence . we focus on a simplified scenario of such system usage. • Virtual organization building – this is related to community-based management of access policies. Another requirement is the Single Sign-On (SSO). Therefore the following issues have to be taken into account when analyzing the security model for such applications. . • Communication security – in distributed and often Internet-wide ap- plications. we have the component containers running on remote and shared resources. This issue is especially important in the case of dynamic deployment of component into shared containers. we give overview of the existing Grid-related security infrastructures and we discuss their applicability to the component-based systems.CoreGRID FP6-004265 57 7 Security Issues in Component-Based Grid Systems In this section we describe our research towards the analysis of security-related issues in component-based grid systems. As shown in Fig. • Authentication – answers the questions how to verify the identity of the communicating parties. 28.1 Security Issues When analyzing the security issues in component-based grid systems. After introducing important concepts and issues. but it also involves direct connection between components. such as e. using proper cryptography solutions. • Sandboxing – answers the question how to protect the code running on shared (not owned!) computational resources from interfering with the others. such issues as message integrity and privacy should be ad- dressed. to relieve the users from management of multiple logins and passwords to access multiple resources. it may involve potentially complex policies. as well as distributed policy management. The above mentioned issues have a lot in common with problems related to running distributed application on the Grid. possibly dynamically evolving partnerships and contracts.g. The client credentials should be thus delegated.

g. providing groups. we give now the overview of existing solutions and discuss their applicability to the component- based world. Extensions of GSI include CAS and VOMS systems for vir- tual organization management. 7.CoreGRID FP6-004265 58 Figure 28: A simplified view on a component-based application dynamic deployment of applications onto shared resources.1 Grid Security Infrastructure GSI [67] was developed as a part of Globus project. They provide community servers for managing lists of users and their roles within VO. It is based on Public Key Infrastructure (PKI) using X. which constitute short-lived credentials allowing single sign- on and delegation. CoreGRID . which is the cumbersome and often created a entry barrier for beginners.Network of Excellence . transport security based on SSL. NGS. MyProxy credential management system can be used for portal integration.2.509 certificates. roles or attributed for more scalable policy management.). 7. GSI implements extensions for proxy certificates. The advantage of GSI comes from its wide deployment on production Grid infrastructures (e. facilitating usage of the Grid for users who prefer not to manage their certificates. etc. requires special han- dling and combining the solutions known from Grid-enabled MPI or Web Ser- vices.2 Existing security infrastructures Having discussed the issues with component-based systems. The proxy certificate del- egation mechanism has been widely used in parallel-distributed applications. and WS-Security for Web Service based ap- plications. The problems related to GSI come from the requirement that the users need to manage their personal certificates. TeraGrid. EGEE. as well as the capabilities offered by H2O system which is a base for MOCCA component framework. Below we discuss how existing technologies known from the Grid and Web areas can be effectively used for assuring the security in component-based systems. As a feasibility study of using GSI for component-based Grid applications. authorization. GSI supports such features as authentication. including MPICH-G2 and it can be directly used to provide delegation of cre- dentials for inter-component connections. We have analyzed GSI and Shibboleth standards.

As a result. Also. standard IdPs are prepared to interact with clients which are using Web- browsers. the access to MOCCA framework can be granted only to such clients who can provide a valid proxy certificate which is listed in registered users database. Shibboleth is used mainly for integrating Web resources of educational institutions. To overcome that limitation. It extends the H2O container used by MOCCA with a GSI-based authenticator module. The authorization is performed based on attributes.CoreGRID FP6-004265 59 Figure 29: Using Shibboleth to access components on the Grid we have build a prototype support for GSI into MOCCA [52] framework. The problem with the above described scenario is that Shibboleth does not provide credential delegation mechanism. Then. 7. we have developed a standalone IdP CoreGRID .2.2 Shibboleth Shibboleth [62] is a federated Web Single Sign-On framework based on SAML (Security Assertion Markup Language). It is an attribute exchange framework. developed by Internet2 consortium. Service provider (component container) redirects the client to IdP (Identity Provider). the SP requests attributes based on the handle. The SSO is achieved by letting users to use their home organi- zation logins and passwords to access remote resources. Shibboleth features attribute-based access control and by mutual agreements between participating institutions it allows decentralized building of virtual organizations. In contrast to GSI it re- quires no user certificates. In such scenario all SPs and IdPs must trust each other. as it is implemented in GSI. 29. how- ever it is the subject of interest for Grid research to overcome problems with user certificates which are required by GSI.Network of Excellence . The IdP returns a handle (an authentication token) after successful authentication. The client contacts the component container. open source. acting as Shibboleth resource provider. since all login requests are redirected to user’s home organization. The proposal of integrating Shibboleth with component container is shown in Fig.

In addition to such infrastructures as GSI or Shibboleth. The prototype of component container which will act as a Shibboleth service provider is under development within MOCCA framework. code execution sandboxing based on classloader separation. single sign-on. and con- figurable security policies. we have to men- tion that Grid component frameworks such as ProActive or MOCCA are based on Java platform-based security solutions. since all distributed component frameworks for Grid programming face similar challenges from the security point of view.CoreGRID FP6-004265 60 client.Network of Excellence . CCA and others. credential delegation and VO-management applies to all component-based systems. Our analysis of the issues such as authentication. 7. providing such features as security managers. which is a library that can be used to obtain Shibboleth credentials from any application.3 Security Issues – Conclusion In this section we have described briefly our experience with the security-related issues which can be met when discussing component-based Grid systems. Distributed policy management and decision infras- tructures such as PERMIS and XACML-based solutions are also the subject of current research. CoreGRID . including those based on GCM.

An important aspect of this strategy is to guarantee relatively small overhead introduced by the wrapper software layer. 8. since the lifespan of Grid infrastructures has been so brief. Interactions between different (instances CoreGRID . vegetation and soil and ice sheets. In both cases the wrapper development process can significantly be simplified by adopting the most efficient strategy and using various tools for reducing the time for development. however. Each Earth module is bound to the natural laws of physics that govern the exchange of energy. and hand-coded wrapping [65].Network of Excellence . in particular. portability and support for dynamic properties. their software environments rarely reach maturity making the software crisis especially acute. This. The main aim of this section is to present our ex- perience in applying component-based development using different approaches depending on the status and the properties of different application codes [35]. ocean. Figure 30 shows a sample Earth system model (or configuration). which are in fact different configurations suited for different simulation scenarios. Depending on the properties and the development status of different legacy codes two approaches can be considered automatically generated wrapping [34]. are critical issues in enabling large production applications for Grid computing systems. The model uses different instances of each of the Earth system modules (instance types are shown within parenthesises). is considered the most complex artefact in distributed computing.1 Introduction It is generally accepted that component-based software development is becom- ing the most cost-effective approach to application construction for complex distributed and Grid systems. 8. Arguably the most serious obstacle to the acceptance of modern component technologies is the so-called software crisis. Hence. in general. The framework includes different variants of earth modules such as the atmosphere. fluxes. Application software. makes the search for the most appropriate programming model and corresponding programming environments even more important than before. Further.CoreGRID FP6-004265 61 8 Use Cases – Overview and Examples 8. marine sediments. land surface.2 Wrapping legacy software Giving high level of attention and support to legacy applications by providing enabling approaches and tools for their seamless integration into state-of-the- art component-oriented systems is a high priority issue. each Earth module uses their own module-specific computational models and module-specific data representations for representing boundary surfaces where the exchanges occur.2. water and other physical or biogeochemical tracers. A typical Earth system model is a composition of different variants of these earth modules (or different instances of earth modules). sea-ice.1 GENIE Overview GENIE (Grid ENabled Integrated Earth system modelling) [37] is a scalable modular platform used to produce and simulate a range of Earth system models in order to understand the long-term changes in Earth’s climate.

CoreGRID FP6-004265 62 Atmosphere (EMBM) Surface Flux Sea−Ice (Goldstein) Land (EMBM) Ocean (Goldstein) Figure 30: A sample configuration representing an Earth system model of) Earth modules are shown by arrows. 8. To facilitate inter-operatability between different language-specific implementations and to expose these components on the Web-Services front. In the context of Grid. a configuration may also use modules to represent exchanged or accumulated physical or biogeochemical tracers (for example. the overall composition of these components (and thus different earth system models) may include remote com- ponents and remote services. With these wrappers in place.2. building automated submission. these components have been wrapped through Java in- terfaces [58]. management. This includes creating component repositories. inter. CoreGRID . problem solving environments or problem solving portals are finding broader range of audience including scientists. In addressing this challenge. Evolving GENIE as a. or as a part of a portal is an interesting challenge. the GENIE was implemented in pure FOR- TRAN without any explicit notion of components and services.2 Component-based version of GENIE In the original implementation. The frame- work included a driver module for orchestrating the execution. At present. scheduling and execution mechanisms and building/supporting visualisation platforms. In addition to Earth system modules.Network of Excellence . Surface flux in the Figure). there are still issues that need to be addressed.and intra- module communications and data exchanges for a given configuration. although we could potentially make use of existing techniques to handle different issues which may arise within. these earth-modules have been componentised by wrapping the legacy code into software components.

At each time step.1 Jem3D overview Jem3D is a numerical solver for the 3D Maxwell’s equations modelling the time domain propagation of electromagnetic waves [10]. The original Jem3D builds on the ProActive library.. i. The reason for the decomposition was to make the factory reusable beyond Jem3D.e. neighbouring subdomains must communicate to compute the values of those border facets. passing them to the factory. A later iteration of the activity grouped the factory and the subdomains into a composite domain component. 8. the domain division is controlled by another triplet (d1 × d2 × d3) that determines the number of subdomains on each axis. and connecting the subdomains.CoreGRID FP6-004265 63 8. and that it minimally supports the same communication styles as the object platform (e. This is done through setting the mesh size. Figure 31 shows the static structure of the resulting component-based Jem3D using a UML component diagram (multicast interfaces are represented as stereo- CoreGRID . Implementing the interface-based version served to increase confidence in the new component ar- chitecture and drastically simplified the final component-based implementation. The process assumes that the target component platform allows connecting components via provided and required interfaces. During the component architecture design. It relies on a finite volume approximation method operating on unstructured tetrahedral meshes. The run-time view of the original architecture was described using UML object diagrams and UML interaction diagrams. initialising..3. the method evaluates flux balances as the combination of elementary fluxes computed through the four facets of a tetrahedron. streams. the triplet (m1 × m2 × m3) that specifies the number of points on the x.2 Componentising Jem3D Jem3D was componentised using the approach presented in [60]. the launcher entity (an executing Java program) was decomposed into a subdomain factory compo- nent and an activator component. the former is assigned the responsibilities for creating.3. Since some facets are located on the boundary between subdomains.3 Componentizing existing applications This approach consists of a general componentisation process in order to trans- form an object-based system into a component-based system. and z axes used for building the mesh.Network of Excellence . The component-based implementation involved wrapping classes to form Frac- tal components and replacing a large part of the injector logic with Fractal ADL descriptions. Parallelisation relies on dividing the computational domain into a number of subdomains. y. Most of the effort was spent on the architecture recovery activity because of the undocu- mented and degraded structure of the system. remote method invocation. exploiting the hierarchical composition feature of Fractal/ProActive. as seen next. The complexity of the calculation can be changed by modifying the number of tetrahedra in the domain. and starting the computation. while the latter the responsibilities for obtaining the input data.g. and events) [60]. 8.

Since allowable configurations follow a fixed. gUSE. canonical structure in the form of a 3D mesh. A multicast interface is also used to connect each agent to all other agents. which depends on the dynamically-determined domain division. The runtime configuration con- sists of multiple subdomains. The runtime con- figuration also includes a dynamically varying number of steering agents. This approach nor- mally involves the use of a component-based development environment such as GIDE. which is used to configure the factory with the required subdomain implementation. as seen in Figure 4 (pseudo code is used for brevity). The main collector is connected to the current set of agents via a multicast interface. the Fractal ADL includes currently no variability mechanisms for ex- pressing such descriptions.Network of Excellence . SEGL. a parameterised descrip- tion would be useful for automatically generating subdomain configurations.4 Developing component-based codes The development process for new application codes can be simplified by follow- ing the component-oriented development paradigm [12]. etc. a problem solving environment capable of uti- lizing the resources of the Grid to execute and manage complex scientific Grid CoreGRID . Note that the ADL is not used to express the configuration of subdomains. with each sub- domain connected to its neighbours via multicast interfaces. The initial configuration of Jem3D is described using the Fractal ADL. logically arranged in a 3D mesh. a component-based language for the description of Grid experiments and SEGL. For our example in this section we use an integrated system is composed of GriCoL. However.CoreGRID FP6-004265 64 * Steering Agent * MainCollector Activator Domain * :SubDomain Factory :SubDomain[*] * Multicast interfaces Client Server Figure 31: Component-based Jem3D structure typed UML interfaces with special notation). The ADL does include a simple parameterisation mechanism. 8.

This implies enormous time and effort for a user when setting up. each system is simulated with 10 different starting velocities. The common use of beta-lactamase inhibitors has led to resistances against them through natural evolution of the beta-lactamase enzyme. An additional property of the lan- guage is that it is multi-tiered. The main elements of this language are com- ponents. They are combined with 10 different substrates.1 GriCoL GriCoL [21.CoreGRID FP6-004265 65 applications.and data-intensive tasks without being tied to a specific application domain. GriCoL combines the component model of programming with the method- ology of parallel programming. In the first stage. This stage consists of two phases: An en- ergy minimization phase in order to eliminate energetically unfavorable starting CoreGRID .4. 14] is a parallel language for programming complex computer. the preparation stage. Another property of the language is its extensibility through the use of components. The language is based on the principle of wrapping functionality in components in order to utilize the capacities of super- computer applications and to enable interaction with other language elements and structures. For a better statistical evaluation and a better sampling of the conformational space. strongly depending on the numbers of CPUs available and the size of the system. which have a defined internal structure and interact with each other through a defined set of interfaces. bacteria have acquired resistance against beta-lactam antibiotics with the help of enzymes called beta-lactamases. The multi-tiered model of organization enables the user when describing the experiment to concentrate primarily on the com- mon logic of the experiment program and subsequently on the description of the individual fragments of the program.2 Use Case: Biochemistry Application Penicillins and cephalosporins are the most commonly used beta-lactam antibi- otics. In the second stage. resulting in 300 different topologies. To prevent this resistance one tries to block the action of the beta- lactamase enzymes with so-called beta-lactamase inhibitors. running and analyzing this large number of simulations by hand. However. These enzymes efficiently hydrolyze the antibiotic and therefore render them ineffective.Network of Excellence . which results in 300 different systems that need to be simulated. 8. 8. They inhibit the growth of bacteria by inhibiting the enzymes responsible for the cell wall building process. The language enables new functional components to be added to the language library. For this purpose all possible enzyme-inhibitor-variants are created. The project starts from a testing series of 30 enzyme variants provided by the user. The following use case describes how the Institute for Technical Biochemistry in Stuttgart models the effect of mu- tations on the inhibitor resistance of beta-lactamases. the input data required for the sim- ulation is prepared. the equilibra- tion stage.4. each system to be investigated is brought to equilibrium. Program codes for the parallel machine are wrapped in the standard language capsule. This pro- cess usually needs days to weeks.

8. the wrap- per software approach can be adopted in its two flavours hand-written or au- tomatically generated wrapper code. In order to re-use legacy codes. CoreGRID . Beneficial is also the reusability of the computation modules: once defined for a specific program. We compare and contrast the three approaches and highlight their ad- vantages and weaknesses. Subsequently. such as deformed bond lengths and bond angles. This work can also serve as a starting point for future developments in the area of component-based methodologies for con- structing Grid applications. the data from the production phase is analyzed and different properties are calculated from it.Network of Excellence . Be- side the time saved for setting up.CoreGRID FP6-004265 66 conditions. submitting. is therefore indispensable for the user. We present our experience in selecting and applying the most appropriate approach depending on the status and the properties of different application codes. The storage of simulation results in the experiment database enables the scientist to later retrieve and compare the results easily. the analysis. simulation carries on without any restraints until the user defined require- ments for equilibrium are fulfilled. A corresponding tool. the system is simulated with gradually decreasing restraints on the atomic coordinates to slowly adapt it to the simulation conditions. The calculated properties are averaged over the 10 MD simulations with dif- ferent starting conditions for each system. which supports the modeling of experiment processes and executes them in a Grid environment. After that. a practical realization of such experiments is possible with the help of Grid technology. Because of the high processing capacities required. The third approach is component-oriented development from scratch. It was also possible to generate an executable program of maximum efficiency. the base for common errors like misspelling input files is also minimized. A global evaluation then compares the property under investigation between all systems to find significant differ- ences. it can be reused for other control flows and can be shared among colleagues requiring that program. it enters the production phase. in the restraint phase. Another approach applicable to existing object-oriented software is to componentise the code by introducing appropri- ate modifications. The equilibration control helps to minimize simulation overhead as simulations which have already reached equi- librium are submitted to the production phase while others that have not are simulated further. The three main approaches can be integrated into a single development framework. After the system has reached equilibrium. In this stage data of the system is collected for later analysis. which can then be correlated by the user to experimentally determined biochemical activity. This GriCoL component-based language offers all elements which are necessary to describe the application flow of a real experiment [21]. and monitoring the thousands of jobs.5 Use Cases – Conclusions This section provides an overview of three different approaches to developing component-based Grid applications. In the final stage of the experiment. There were obvious benefits from using the Grid management system.

This should be taken into account when choosing the version of GPL to be used. There are 2 ways of distributing third-party plug-ins: • As simple Eclipse plug-ins. to avoid conflicts.html.Network of Excellence . and provided via the EU GridCOMP project. install the 2 This section is based on text originally written by Tony Arbona and Gaston Freire from Grid Systems. • It is worth mentioning. and so on). • GPL is “viral”: any application that uses GPL code. once the application is acquired.1 Some GPL facts • The GPL requires the software vendor to provide the source code of the application (or a reasonable mechanism to get it) when distributing it. GPL v3 is the best option.A. the source code will be delivered.CoreGRID FP6-004265 67 9 Software Licensing Issues2 9.2. some- thing unusual for commercial licenses and that can cause the rejection of the software industry. derby.2. the GPL gives the receptor the freedom to redistribute it. S. in order to be dis- tributed. 9. This will re- quire the user to have Eclipse previously installed and then. However.2 Eclipse Public License (EPL) This is the license of Eclipse and the plug-ins included in its distribution. must be licensed under the GPL. The applications under the GPL are often accompanied by a compressed file with the source code. The full list of licenses whose compatibility with GPL has been studied can be read here: http://www.1 Apache License This license is used by some libraries included in ProActive (commons. axis. but not GPL v2. to avoid common misunderstandings. the fact that licensing an application under the GPL does not make it automatically free. The Apache Software Foundation itself does not consider them incompatible [6]. CoreGRID .gnu. But the incompatibility between Apache and GPLv2 is not consensual. It is compatible with the GPL v3. 9. 9. In this case. In this case. only the third-party-developed plug-ins are packaged and distributed under some license. Anyway. A GPL application can be sold. and it is not legal to distribute applications containing combinations of incompatible licenses among their base libraries [29]. all the out- puts from a prototype software (employing previously developed software.2 Incompatibilities The GPL is incompatible with several other open-source licenses. To be distributed. in the sense of invoking its libraries or API) would need to be licensed as GPL (that means distributing its source code).org/philosophy/license-list. along with the bina- ries.

net/projects/azureus/ • Talend Open Studio: data integration http://www. both the third-party-developed plug-ins and other (essential) ones coming from the Eclipse framework are packaged and distributed. The usual way to avoid conflict is to distribute Eclipse GPL plug-ins separately.talend.eclipseplugincentral. the official repository of third-party plug-ins for Eclipse.com/ • GumTree: physics experiments GUI http://sourceforge. and then violating the license.eclipse. our advice is to distribute Eclipse plug-ins.CoreGRID FP6-004265 68 new plug-ins. Nevertheless. there are RCP applications licensed under GPL. The EPL-GPL incompatibility mainly affects the RCP applications. For certain kinds of users (not developers) this represents a loss of usability.com. we can find a multitude of GPL plug-ins.Network of Excellence . The EPL license is incompatible with both version 2 and 3 of the GPL.net/projects/gumtree/ • JCrypTool: a crypto tool http://sourceforge. having a single Eclipse instance with all development tools within will enhance usability and reduce memory consumption.net/projects/jcryptool/ • JTAF: small sports events manager http://www. Moreover.net/projects/relations-rcp/ A list of open sourced (under different licenses) RCP applications can be found here: http://www. so this form of distribution should be avoided (this applies both to GIDE as IC2D). It is not legal to distribute applications that combine plug-ins licensed under both licenses (EPL and GPL) [26]. The resulting RCP application is independent from Eclipse and does not require additional installation steps for the user. This will not be a major drawback. CoreGRID . Examples of this are: • Azureus: one of the most famous Bittorrent clients http://sourceforge. In http://www.org/community/rcpos.php?article id=1&clang=1 • Relations RCP: a personal wiki http://sourceforge. not RCP applications. In this case. taking into account that prototype software is targeted mainly to developers.ch/rex/index.php Even so.jtaf. • As an RCP (Rich Client Platform) application.

and should be distributed separately as Eclipse plug-ins. Software developed under Eclipse will have to be licensed as GPL as well. This prevents them from being packaged as RCPs.Network of Excellence .3 Conclusions We recommend GPL v3 (not GPL v2) to avoid illegal (according to the Free Software Foundation) license combinations with Apache.CoreGRID FP6-004265 69 9. CoreGRID .

the lack of longer-term experience and the complexity of the target systems demand more research results in the field. Automation is vital to address scalability and robustness and is a key area of investigation. These were legacy code wrapping. as previously stated. One of the major results of the research was to highlight the difficulties a number of topics central to the generic grid paradigm. in order to enable the maximum adaptation of the grid model. CoreGRID . However. and automation.CoreGRID FP6-004265 70 10 Conclusions and Future Work We have presented the results of a collaborative approach to the design of a Generic component-based Grid platform.Network of Excellence . Many of the results presented here are still in progress and will require some more time for completion. The results presented here represent the culmination of the research efforts of a number of partners with particu- lar emphasis on the integration of each approach into a unified Generic Grid Architecture.

. Dahlgren. Spain. IEEE CS Press. Baduel. H. Keahey. and T. Ravazzolo. Aldinucci. Zoccolo. and S. Allen. Rana. and B. S. Programming Grid Applications with GRID superscalar. F. Labarta. [5] G. Kilpatrick. S. Cafaro. Seidel. K. Italy. Danelutto. [6] Apache Software Foundation. Proc. T. Boon. Paternesi. In: CoreGRID Integration Workshop 2005. [8] R. Proc. A Parallel Object-Oriented Application for 3D Electromagnetism. Lezzi. and F. and R. p. R. and P. A. Vanneschi. R. ASSIST as a research framework for high-performance Grid programming environments. Building Interoperable Grid-aware ASSIST Applications via WebServices. Gombas. E. Vanneschi. T. Geist. S. Schtt. Merzky. of IEEE Int. Delbe. Armstrong. Euromicro PDP 2008: Parallel Distributed and network-based Processing. T. Proc. [7] R. Toward a Common Component Architecture for High-Performance Scientific Computing. N. P.apache. [9] R. Balaton. Proceedings of the IEEE. Kumfert. pp. Parker.M. Gama. Journal of GRID Computing. Comput. IEEE CS Press. McInnes. M. Bour- geois. Laforenza. Sirvent.Network of Excellence . Lanteri. 145–152. 18(2):215–229.org/licenses/GPL- compatibility. S. Integrating Resource and Service Discovery in the CoreGrid Information Cache Mediator Component. Reinefeld. pp. R. Epicoco. 93(3). HPDC Sym- posium. IEEE CS Press. 2008. [10] L. 1999. Gannon. D. K. Parker. Aldinucci. C. pp. Behavioural skeletons in GCM: autonomic management of grid components.M. Kasmi. CoreGRID . Spies. Davis. editors. of Intl. 18th Int. G. Dazzi. In PARCO 2005: Parallel Computing. and M. 13. Campa. P. Cunha and O. 2006. Kacsuk. Danelutto. Parallel and Distributed Processing Symposium. The CCA component model for high- performance scientific computing. M. J. vol. F. Aloisio. J. T. Aldinucci. 2004. A. E. Kohn. Kielmann. L. Pisa. Smolinski. June 2003. G. van Nieuwpoort. J. Danelutto. pp.html. Z. M. : Pract. M. 2006. P. Kielmann. Malaga. March 2005. Allan. A. M. vol. Schintke. C. Kaiser. [4] G. A.V. F. Badia. Epperly. J. 2005. A. Armstrong. Hutanu. D. I. 151–170. Concurr. 54–63. In J. Goodale. T. Cela. http://www. 534–550. Sottile. Grima. Prez. M.M. Baude. M. [2] M. and C. M.CoreGRID FP6-004265 71 References [1] M. [3] M. L. Grid Computing: Software environments and Tools. B. editors. Jan. The Grid Application Toolkit: Towards Generic and Easy Application Programming Interfaces for the Grid. In D. D. 1(2). McInnes. Tonellotto. S. B. Springer. and D. Baz. N. Coppola. Ullmer. Caromel. December 2005. Exper. Vanneschi.

C. Bruneton. Zoccolo. and J. Deliverable Report D.02. Component- oriented Development Environment for Grid: Design and Implementation. Springer. Basic Features of the Grid Component Model (assessed). pp. In Parallel Processing and Applied Mathematics. C. C. Future Gen- eration Grids. Coppola. and A. in Making Grids Work. [13] F. M. Coupaye. CoreGRID Workshop on Inte- grated research in Grid Computing. E-SCIENCE ’06: Proceedings of the 2nd IEEE Int. [18] CoreGrid NoE Institute on Programming Model. Paccosi. Resch. F. Stefani. J. Currle-Linde.05. Isaiadis. Recursive and Dynamic Soft- ware Composition with Sharing. [20] CoreGrid NoE Institute on Grid Systems. Int. [17] CoreGrid NoE Institute on Programming Model. 1068–1077. Laforenza. Lacour. IC2D: Interactive Control and Debugging of Distribution. Zoccolo. Perez. Deliverable Report D. Tonellotto.B. Reinefeld.Network of Excellence . [15] E. Tools. Design of the Integrated Toolkit with Supporting Mediator Components. Lecture Notes in Computer Science. Con- ference on Large-Scale Scientific Computing. T. In V. Springer. A. 2008 (to appear). 2001. HPC Application Execution on Grids. Danelutto editors. Gorlatch and M. Resch. 7th Int. Bouziane. [14] H. [21] N. and Environments. 4967. 2007. Towards a common deployment model for Grid systems. 2008. Currle-Linde. Proc. Bergel. 7th International Conference. 2002. Baraglia. 2006. and J. T. N. O. Italy. 2179:193–200. Pisa. Revised Selected Papers. [16] M. Danelutto. vol. Deliverable Report D. In.CoreGRID FP6-004265 72 [11] T. Vol. 31–40. editors. 2008 (to appear).PM. Pro- posal for Mediator Component Toolkit. Huet. Proc. Lecture Notes in Computer Science. 2005. Proposals for a Grid Component Model. M. Analysis of Com- ponent Model Extensions to support the GriCoL Language. M. Bubak. and Environments. Tonellotto. Bos. Priol. Bartynski. Basukoski. Conference on e-Science and Grid Computing. and S. and C.04. M. and M. Laforenza. R. 2006. In S. Amsterdam. Getov.02. Getov. Prez. D. PPAM 2007. 2006.PM. N. pp. 2006. T. November 2005. Danelutto. Universal Grid Client: Grid Operation Invoker. GriCoL: A Language for Scientific Grids.ETS. Malawski. Making grids Work. [19] CoreGrid NoE Institute on Grid Systems. Tools. and M. [22] M. V. CoreGRID . Baude. Springer. and A. IST. Deliver- able Report D. Springer. N. Fagni. [12] A. Vayssiere. Thiyagalingam. D. Nano. F. D. Caromel. T. Gubala.STE. Vanneschi. CoreGRID. S. Workshop on Component- Oriented Programming.

T. http://gridcomp. 44(10). M.html [33] T. Commercial Exploitation of Grid technologies and Services. Final Report. Philippsen.org. In: CoreGRID Symposium. http://www. Grid Development Tools for Eclipse. No. [31] The Fractal Project. Smith. [29] S. In Proc. [28] M. 118–125. Science. 2005.genie. Blackman. 2006. Terstyanszky. [35] V. 2008. Getov. vol. Integrated Framework for Development and Execution of Component-based Grid Applications.org/specification/ index. G.uk [38] The GridCOMP Project. 1-2. Service-oriented science. [37] Grid ENabled Integrated Earth system model (GENIE).CoreGRID FP6-004265 73 [23] T. 2008. Proc. Kielmann. [36] V.Network of Excellence .ercim. Getov. U. Eclipse Technology eXchange Workshop eTX at ECOOP. IEEE John Vincent Atanasoff 2006 Interna- tional Symposium on Modern Computing (JVA’06). Proc. Dependability in Hybrid Grid Systems: a Virtual Clusters Approach.org/gef/ [26] Eclipse Public License. [27] Eclipse . N.ac. Foster. http://www. 2005.An open development platform. A Mixed-Language Programming Methodology for High Per- formance Java Computing. Integrating Ap- plication and System Components with the Grid Component Model. and T. http://fractal. Friese. IPDPS. pp.org/wiki/Eclipse Public License. CoreGRID . S. G. and P. Delaittre. Journal of Grid Computing. Lanteri/DiscoGrid [25] Eclipse Graphical Editing Framework. Las Palmas. http://en. Forge and C. 3. http://www. Kiss. [34] V. 308(5723):814–817. http://www. pp.eclipse. GEMLCA: Running Legacy Code Applications as Grid Services. Springer. Getov. Goyeneche. in The Architecture of Scientific Software. Kluwer Academic Publishers. von Laszewski. [24] The DiscoGrid Project. Foster. 52–57.eclipse. Isaiadis and V. 2001. M. Multi-Paradigm Communications in Java for Grid Computing. Communications of the ACM. November 2006. I. SCF Associates.wikipedia. IEEE CS Press.org/ [32] Fractal specification.fr/caiman/Stephane. Freisleben. 333–347.objectweb. and B. Spain. IEEE CS Press. Ejdys. pp 75–90. 2001. http://fractal. Herman-Izycka. [30] I. Lal. Kacsuk. Nov. pp.inria. 2006. Winter.objectweb. Getov.org [39] S. A.

Henrio. 3-4. I. pp. IASTED Int. September 2007. Sipos. MOCCA – Towards a Distributed CCA Framework for Metacomputing. Baude. A Service for Reliable Execution of Grid Applications. and T. pp. and G. pp. of the UK E-Science All Hands Meeting. Kurzyniec. Lopez and C. http://www. Perez. G. 2007. February 2008. Springer. 174a. and T. 643–650. No. Sirvent. Interoperability of Grid Component Models: GCM and CCA Case Study. CoreGRID Symposium. 2008.CoreGRID FP6-004265 74 [40] S. Bubak. Caromel. Improving MPI Support for Applications on Hi- erarchically Distributed Resources In Proc. Kelley.com/products/rules. V. [52] M. Workflow Level In- teroperation of Grid Data Resources. Tools. T. November 2005.M. pp 221-238. Tudose. Acta Press. France. In Proc. In Proc. Proc. Terstyanszky. Multi-Grid. Perez. Conf. Kaiser. pp. Getov. Workshop on Grid Computing (Grid2005). Kiss.. In Proc. Vol. 8th IEEE Int. [41] S. Kacsuk and G.Network of Excellence . C. A. February 2008. [45] T. Merzky. 3. Krepska. and I. Badia. [46] T. 2007. Technical Report TR-0094. Morel. 194-201. 3rd IEEE Int. L. 1st Francophone Conference On Software Deployment and (Re)Configuration (DECOR’2004). [48] S. Lacour. [44] P. 179-192. 2008. P. Weidner. Kacsuk. H. Terstyanszky. Getov. Isaiadis and V. EuroPVM/MPI Conference. Lacour. pp. Multi-User Workflows in the P- GRADE Grid Portal. M. CoreGRID.jboss. Sunderam. and R. LNCS. Perez. vol. Evaluation of Dynamic Clustering Architecture for Utilising Mobile Res ources. 2005. and V. IEEE CS Press. 2007 [51] M. and O. A. SRB Data Resources in Computational Grid Workflows. In Proc. Kacsuk. Conference on Par- allel and Distributed Computing and Networks. Taylor. 6th IEEE/ACM Int. Isaiadis. Grid Interoperability at the Application Level Using SAGA. Symposium on Cluster Computing and the Grid (CCGrid-2008). IEEE CS Press. D. Winter. 187–192. 2005. [50] R. Proc. [42] Jboss rules home page. R. P. 4757. [47] E. Grenoble. Priol. Malawski. In: Achievements in European Research on Grid Systems. [49] S. In the Proc. C. In: Proc. Dynamic Service-Based In- tegration of Mobile Clusters in Grids. October 2004. Springer. and Environments. p. pp. Springer. Kiss. and M. D. Kielmann. A Software Architecture for Automatic Deployment of CORBA Components Using Grid Technologies. 10th HIPS Workshop in Conjunction with IPDPS. 584–591. and S. 2008. 117–125. 2005. Conference on e-Science and Grid Computing. CoreGRID . F. Institute on Grid Systems. Priol. [43] S. Jha. Journal of Grid Computing. Generic Application Description Model: Toward Automatic Deployment of Applications on Computational Grids. Malawski. Springer.

D. K. T. SIAM Press. Bubak. Damevski. HPC-GECO/CompFrame Workshop in Conjunction with HPDC. Rivera. Darlington.M.corba. 2008 (to appear). Simon (Eds. Henrio. [56] Object Management Group Inc. and V. Springer. Gubala. pp. T. P. M. 2006. Dalmasso. De- sign Support for Componentising and Grid-enabling Scientific Applica- tions.0. and V. R. M. 1998. D.V. Malawski.R. 165–178. Thiyagalingam. Parker. Computer. M. [64] E. J. [54] M.fr/oasis/ proactive/ [62] Shibboleth. and H. In: Proc.inria. Panagiotidi. Zhang. Panagiotidi. France. Kasztelnik.internet2. and J. 31(3):23–30. The CORBA Component Model. Baude. 2006. ACM CompFrame07 Symposium. In All Hands Meeting. Caromel. Tejedor. Heroux. Bal. P. [65] J. Service-enabling Legacy Applications for the GENIE Project. User-friendly and Reliable Grid Computing Based on Imperfect Middleware. In Proc. V. Integrating component-based scientific computing software. Johnson. and C. Chapter 15. and L. T. Paris. http://shibboleth. Kielmann. Proc. Cohen. Core- GRID Integration Workshop. Sunderam. K. [55] R. 2008 (to appear). Or- chestrating a safe functional suspension of GCM components. Morel. 2007. van Nieuwpoort. Ex- periments with Distributed Component Computing Across Grid Bound- aries. F. Frontiers of Parallel Processing For Scientific Computing. [57] J.Network of Excellence .CoreGRID FP6-004265 75 [53] M. http://www. 2007. High-level scripting approach for building component-based applications on the grid. http://www-sop. Placek. T. 2007. and H. in Making Grids Work. M. in Making Grids Work. Parlavantzas. and D. [59] S. V. Baude.M. Tejedor. ACM/IEEE Conference on Supercomputing (SC’07). Springer. [58] S. Naoumenko. 271–288. A Component-Based Integrated Toolkit. Bartynski. Beckmann. In M. Dar- lington. Kurzyniec. In: Achievements in European Research on Grid Systems. and C. O. 2006. Domain-Specific Metadata for Model Validation and Performance Optimisation. pp. [61] ProActive Java Grid Middleware. and E. ACM Press. Getov. Getov. Kielmann. Scripting: Higher-level programming for the 21st cen- tury. [60] N. Nottingham. M. Springer. 109–116.edu/ [63] E. J. 31–38. Raghavan.). CoreGRID . September 2006. Proc. 2008 (to appear). K. Bubak. Malawski. Getov. F.E. Marko Krznarić. M. Badia . Springer. Revision V4. pp. A. Katsiri.org/. Badia. R. S. Ousterhout. pp.

Siebenlist. CoreGRID . Gawor. C. Pearlman.509 Proxy Certificates for Dynamic Delegation. TR ITI-2000-37. Component interoperability. Mulmo. J. and J. Uni- versity of Malaga. Proc.lcc.uma. Welch. L. Vallecillo. O. 3rd Annual PKI R&D Workshop. Troya. 2004.es/~av/Publicaciones/ 00/Interoperability.pdf. Dept. J. 2000.Network of Excellence . Foster. Kesselman. Hernandez. Tuecke. S. [67] V. de Lenguajes y Ciencias de la Computacion. http://www. and F. S.M. X.CoreGRID FP6-004265 76 [66] A. Meder. I.