The Anatomy of the Grid

Enabling Scalable Virtual Organizations

24/05/20
 Agenda
Review of Grid Problems
Virtual Organization
Grid Architecture
Grid Requirements
Importance of Grid
Comparison of Grid with other
Technologies
Conclusion
24/05/20
 Need of Grid
Grid
– “Grid” computing has emerged as an
important new Technology, distinguished from
conventional distributed computing by its
focus on large-scale resource sharing,
resources use in dynamics and scalable
virtual organizations(VO).

24/05/20
 Grid Problems
First Problem
– Flexible, secure, coordinated resource
sharing among dynamic collections of
individuals, institutions, resources and VO.
Second Problem
– We encounter unique authentication,
authorization, resource access, resource
discovery and other challenges.

These problems are addressed by Grid

24/05/20 Technologies.
 Concept of Grid

It is coordinated resource sharing and

problem solving in dynamic, multi-
institutional virtual organizations.

24/05/20
 Resource sharing
and problem-solving
The sharing is not primarily file exchange
but rather direct access to computers,
software,data and other resources.
It is required by a range of collaborative
problem-solving and resource brokering in
industry and science.
It is highly controlled with resource
providers and consumers
24/05/20
 Virtual Organizations

A set of individuals,institutions defined by

such sharing rules form what we call a
Virtual Organization.

24/05/20
 Need of VO
Car manufacturer to perform scenario
evaluation during planning for a new factory
Bidding on a new aircraft
Simulation systems that they use to plan a
response to an emergency situation
High energy physics collaboration

24/05/20
 Types of VO
VOs vary tremendously in their:
– Purpose,scope,size,duration,structure,commun
ity and sociology.

24/05/20
 Grid Requirements
Highly flexible sharing relationship ranging
from client-server to peer-to-peer.
Precise levels of Control over how shared
resources are used, access control,
delegation and application of local and
global policies.
sharing of varied resources, ranging from
programs, files and data to
computers,sensors and networks.
24/05/20
 Grid Requirements
Diverse usage modes ranging from single
user to multi-user and from performance
sensitive to cost-sensitive.
Embracing issues of quality of service,
scheduling, co-allocation and accounting.

24/05/20
 Limitations of current Technology
Internet Technology address communication and
Information exchange among computers but do
not provide integrated approaches to the
coordinated user of resources at multiple sites.
Business-to-business exchanges focus on
information sharing (via. centralized servers).
Distributed computing companies seek to
harness idle computers on an international scale
but it supports only highly centralized access to
those resources.
24/05/20
 Limitations of current Technology
Enterprise Technologies such as CORBA and
Enterprise Java enable resource sharing within a
single organization.
Distributed Computing Environment supports
secure resource sharing but it is inflexible and
burden.
Storage service provider (SSP) and application
service provider (ASP) are typically linked with
customer via a Virtual Private Network.
24/05/20
 Limitations of current Technology

 Dynamic, cross-organizational sharing are not

used in distributed computing. It is achieved by
Grid Technology.

24/05/20
 Overview of Grid Technology
The Grid community has produced protocols,service and tools that
addresses when we seek to build scalable Vos.
Security solutions

– It supports Management of credentials and policies when

computations span multiple institutions.
Resource management protocols and services
– It supports secure remote access and data resources and the co-
allocation of multiple resources.

24/05/20
 Overview of Grid Technology
 Information query protocols and services
It provides configuration and status
information about resources,
organizations and services.
 Data management and services
It locates and transport datasets
between storage systems and
applications

24/05/20
 Nature of Grid Architecture
 The establishment,management and
exploitation of dynamic,cross-organizational
VO sharing relationships require new
technology.
A Standards-based Open architecture
facilitates extensibility,Interoperability,potability
and code sharing.

24/05/20
 Issues of Grid
 Interoperability
It means common protocol
 Protocols
VO users and resource negotiate, establish,
manage and exploit sharing relationships. It is easy
to define standard services.
 Services
It provides enhanced capabilities
 Application Programming Interface (API) and Software
Development Kit (SDK)

24/05/20
 Issues of Grid
 Application Programming Interface (API) and
Software Development Kit (SDK)
It provides the programming abstractions
required to create a usable Grid.

24/05/20
 What? why?
 Wha is Middleware?
The services needed to support a common set of
applications in a distributed network environment.
 Why we need interoperability?
To ensure that sharing relationships can be
initiated among arbitrary parties, accommodating
new participants dynamically, across different
platforms, languages and programming
environments.

24/05/20
 why?
 Why we need protocols?
How distributed system elements interact with one
another in order to achieve a specified behaviour,
and the structure of the information exchanged
during this interaction.
Protocols govern the interaction between
components, and not the implementation of
components, local control is preserved.
This focuses on external rather than internal
interactions.

24/05/20
 why?
 Why we need services?
The service is defined solely by the
protocol that it speaks and the behaviours
that it implements.
The services for access to
computation,access to data,resource
discovery, co-scheduling and data
replication.
It is also to abstract away resource
24/05/20 specific details.
 why?
 Why we consider API and SDK?
It is more to VOs than interoperability,protocol and
services.
Developers must be able to develop an applications in
complex and dynamic execution environment.
Users must be able to operate these applications.
Standard abstractions, APIs and SDKs can accelerate
code development, enable code sharing and enhance
application portability.
API and SDK are adjunct to protocols, not an alternative.

24/05/20
 Layers of the Grid Architecture
 The layers of grid will follow the principles of the
hourglass model.
 The hourglass defines a small set of core
abstractions and protocols, onto which many different
high-level behaviours can be mapped and which
themselves can be mapped onto many different
underlying technologies.
 The number of protocols defined at the neck must be
small.

24/05/20
 Various layers
 The neck of the hourglass consists of Resource and
connectivity protocols, which facilitate the sharing of
individual resources.
 It can be implemented on top of a diverse range of
resource types, defined at the fabric layer.
 It can be used to construct a wide range of global services
and application specific behaviours at the collective layer
and they involve the coordinated use of multiple
resources.

24/05/20
 Fabric Layer
 Interfaces to Local control.
 It provides the resources to which shared access is
mediated by grid protocols.
 It implement the local, resource specific operations
that occur on specific resources as a result of
sharing operations at higher level.
 If we place few demands on fabric elements, then
deployment of grid infrastructure is simplified.

24/05/20
 Fabric Elements
 Resource level support for advanced reservations
makes it possible for higher level service to aggregate
(coschedule) resources in interesting ways.
 A requirement for advance reservation increases the
cost of incorporating new resources into a Grid.
 The resources should implement enquiry mechanisms
that permit discovery of their structure, state and
capabilities.
 The resource management mechanisms that provide
some control of delivered quality of service.
24/05/20
 Grid Resources
 Computational Resources
Mechanisms are required for starting programs and
for monitoring and controlling the execution of the
resulting processes.
Management mechanisms that allow control over
the resources allocated to processes are useful.
Enquiry functions are needed for determining
hardware and software characteristics and state
information such as load and queue state in the
case of scheduler-management resources.

24/05/20
 Grid Resources
 Storage Resources
Mechanisms are required for putting and getting
files.
Management mechanisms that allow control over
the resources allocated to data transfers
(space,disk bandwidth, network bandwidth, CPU)
are useful.
Enquiry functions are needed for determining
hardware and software characteristics and load
information such as available space and bandwidth
utilization.
24/05/20
 Grid
 Network Resources
Resources
Management mechanisms that provide control over the
resources allocated to network transfers (i.e
prioritization,reservation) are useful.
Enquiry functions are needed for determining network
characteristics and load.
 Code Repositories and Catalogs
This specialized form of storage resource requires
mechanisms for managing versioned source and object
code (like CVS) and implement catalog query and update
operations (like RDBMS).

24/05/20
 Support of GT in Fabric
 Globus Toolkit designed to use existing fabric components, including
vendor supplied protocols and interfaces.
 If vendor does not provide the necessary fabric layer behaviour, the GT
includes the missing functionality.
 For example, enquiry software is provided for discovering structure and
state information of resource and for packaging this information in a form
that facilitates at the resource layer.
 Resource management is, the domain of local resource
managers(GARA(slot), PBS, Condor (Adv. Res.))

24/05/20
 Connectivity Layer
 Communication easily and securely.
 It provides core communication and authentication
protocols required for Grid-specific network transactions.
 Communication protocol enable the exchange of data
between fabric layer resources. It include transport, routing
and naming. It will not demand new protocols.
 Authentication protocols build on communication services
to provide secure mechanisms for verifying the identity of
users and resources.

24/05/20
 Authentication solutions
 Single sign on
Users must be able to “log on” (authenticate) just once and
then have access to multiple Grid resources.
 Delegation
A user must be able to endow a program with the ability to run
on that user’s behalf, so that the program is able to access the
resources on which the user is authorized. The program also be
able to conditionally delegate a subset of its rights to another
program.

24/05/20
 Authentication solutions
 Integration with Various local security solutions
 Each resource providers may employ any of a
variety of local security solutions, including kerberos
and unix security. Grid solutions must be able to
interoperate with these various local solutions.
 User-based trust relationships
A user to use resources from multiple providers
together, the security system must not require each
of the resource providers to interact with each other
in configuring security environment.

24/05/20
 Support of GT in Connectivity
The Internet protocols are used for communication.

– TCP/IP protocol stack ( Internet -IP and ICMP, transport-

TCP &UDP, application - DNS
Grid Security Infrastructure (GSI) protocols are used for
authentication, communication protection and authorization. It
is build on and extends the Transport layer protocol (TLS).
X.509 format identity certificates are used. Authorization
supported via Generic Authorization and Access(GAA) to
allow resources owners to integrate local policies.

24/05/20
 Resource Layer
Sharing single resources
The resource layer build on connectivity layer. To
define protocols (API and SDK) for the secure
negotiation, initiation, monitoring, control,
accounting and payment of sharing operations on
individual resources.
These protocols call fabric layer functions to
access and control local resources.

24/05/20
 Resource Layer protocol
Information Protocols

– It is used to obtain information about the structure

and state of a resource (load and policy).
Management protocols

– It is used to negotiate access to shared

resources. It support accounting,payment,
monitoring the status of an operation and
controlling the operation.
24/05/20
Support of GT in Resource - client
A Grid Resource Information Protocol (GRIP, it is
based on LDAP). The Grid Resource Registration
Protocol (GRRP) is used to register resources with
Grid Index Information servers.
HTTP-based Grid Resource Access and
Management (GRAM) protocol is used to allocation
of computational resources and monitor and control.
GridFTP is a management protocol for data access
LDAP is also used as catalog access protocol
24/05/20
Support of GT in Resource - server
A Grid Resource Information service (GRIS)
implements server side LDAP functionality.
Gatekeeper, which provides the interaction of GSI
and GRAM.
The Generic Security Services (GSS) API is used to
acquire, forward and verify authentication
credentials.

24/05/20
 Collective
It is global in nature and capture interactions across
collections of resources.
Collective layer protocols are:

– Directory services may allow its users to query for

resources by name, attribute. It construct by
GRRP and GRIP.

– Co-allocation,scheduling and brokering service to

allocate and scheduling the task on resource.
(AppLes, Condor-G and Nimrod-G,DRM broker)
24/05/20
 Collective layer protocol
Monitoring and diagnostics services support failure,
adversial attack (ID), overload.
Data replication services support the VO storage
resource to maximize data access with respect
response time,reliability and cost.
Grid enabled programming system (like MPI and
manager-worker framework)
Workload management system and collaboration
framework also know as problem solving
envrionment (PSE) (like workflow)
24/05/20
 Collective layer protocol
Software discovery services discover and select the
best software implementation and execution platform
based on the parameters of the problem. (e.g
NetSolve, Ninf)
Community authorization servers enforce community
policies governing resource access and generating
capabilities. It provides global policy enforcement
service.
Community accounting and payment services gather
resource usage information for accounting and
payment,
24/05/20
limitting of resource usage by users.
 Collective layer protocol

Collaborative services support the coordinated

exchange of information within user
communities. (e.g CAVERNsoft, Access Grid)

24/05/20
Support of GT in Collective layer

Meta Directory Service (MDS) which introduces

Grid Information Index Service (GIIS)
Replica catalog and replica management service
Online credential repository service (Myproxy)
DUROC co-allocation library provides an SDK
and API for resource allocation.

24/05/20
 Application

It comprise the user applications that operate

within a VO Environment.
It construct in terms of calling upon, services
defined at any layer.
Using frameworks and libraries to access the
services. (e.g SciRun, CORBA,Cactus, workflow
systems)

24/05/20
 Intergrid Protocols

The core Internet protocols enable different

computer networks to interoperate and exchange
information.
It enable different organizations to interoperate
and exchange or share resources.

24/05/20
 Comparisons
World Wide Web

– It is using TLS for authentication, but do not

support single sign on or delegation. (VO
portals will support)

– GSI delegation capabilities would permit a

browser client to delegate capabilities to a
web server so that the server could act on the
client behalf.
24/05/20
 Comparisons
Application and storage service provider

– Security tends to be handled by using VPN

technology. It is hard to achive. It means ASP
application impossible to access data located
on SSP. VPN is not a VO (dynamic and control
remote resources)

– Ssp offer file-sharing services via HTTP, FTP

and webDAV with user ids,password, and
access control lists controlling access.
24/05/20
 Comparisons
Enterprise computing system

– Sharing arrangements are typically relatively

static and restricted to occur within a single
organization. The interaction is client-server
rather than coordinated use of multiple
resources.
Internet and peer-to-peer

– Shared infrastructure and Interoperability

– File sharing with no access control and

24/05/20
 Conclusion
This paper will prove useful to the emerging grid
community, by improving understanding of our
problem and providing a common language for
describing solutions. We also hope that analysis will
help establish connections among grid developers
and proponents of related technologies.

24/05/20
 Definitions
Protocol defines a set of rules that end points in a
telecommunication system use when
exchanging infromation
IP defines an unreliable packet transfer protocol
TCP builds on IP to define a reliable data delivery
protocol
TLS defines a protocol to provide privacy and
data integrity between to applications. It is on
top of a TCP
LDAP defines a query response protocol for
querying the state of a remote database. (on
24/05/20
 Definitions
Protocol defines a set of rules that end points
telecommunication system use when exchan
infromation
IP defines an unreliable packet transfer protocol
TCP builds on IP to define a reliable data del
protocol
TLS defines a protocol to provide privacy and
integrity between to applications. It is on top of a
LDAP defines a query response protocol for que
the state of a remote database. (on TCP)
24/05/20
 Definitions
Service is a network enabled entity that
provides a specific capability
( service=protocol(interact)
+behavior(response))
FTP server interact speaks the FTP and
supports remote read and write access to a
file.
LDAP server interact speaks the LDAP and
supports response to queries.
24/05/20
 Definitions
API defines a standard interface (e.g set of
subroutine calls or objects & methods of an
OO API) for invoking a specified set of
functionality.
GSS API for verifying identify of
communicating parties, encrypting
messages.
MPI API for parallel computing.
SDK denotes a set of code designed to be
linked with and invoked from within, an
24/05/20
 Thank you

24/05/20