You are on page 1of 14

Architecture for Grid

Tein Horng Yuan 2003/02/24


Why Grid
Large amounts of data in petabyte / Hardware / Software over the Internet Institutions / Individuals around the world need to collaborate together to do the problem solving in a dynamic and flexible way

Virtual Organization
VOs enable disparate groups of organizations and/or individuals to share resources in a controlled fashion, so that members may collaborate to achieve a shared goal. (ref .1)

Grid Architecture
Fabric Layer Connectivity Layer Resource Layer Collective Layer Application Layer

Layered Grid Architecture

Application Collective Resource Connectivity Fabric

High-Throughput Computing System

App High Throughput Computing System job management,

API SDK C-point Protocol Checkpoint Repository

ollective Dynamic checkpoint, (App) failover, staging

ollective Generic) Brokering, certificate authorities

API SDK Access Protocol Compute Resource 6

Resource Access to data, access to computers, access to network performance data

Connect Communication, service discovery (DNS), authentication, authorization, delegation Fabric Storage systems, schedulers

Data Grid Architecture

App Discipline-Specific Data Grid Application

ollective Coherency control, replica selection, task management, (App) virtual data catalog, virtual data code catalog,

ollective Replica catalog, replica management, co-allocation, Generic) certificate authorities, metadata catalogs, Access to data, access to computers, access to network performance data,


Communication, service discovery (DNS), Connect authentication, authorization, delegation Fabric Storage systems, clusters, networks, network caches,

EDG middleware architecture Globus hourglass

Current EDG architectural functional blocks: Basic Services ( authentication, authorization, Replica Catalog, secure file transfer,Info Providers) rely on Globus 2.0 (GSI, GRIS/GIIS,GRAM, MDS)

Specific application layer VO common application layer

GRID middleware





Other apps


Other apps

High level GRID middleware Basic Services OS & Net services



The DataGrid Project WP1 (Workload Management System) WP2 (Data Management) WP3 (Information and Monitoring Services) WP4 (Fabric Management) WP5 (Mass Storage Management) WP6 (Testbed and Demonstrators) WP7 (Networking) Applications
WP8 (High Energy Physics) WP9 (Earth Observation) WP10 (Biology)

WP11 (Dissemination) WP12 (Project Management)


Local Computing

Local Database Local Database Metadata Metadata Management Management Object to Object to File File Mapping Mapping

Local Application Local Application


Grid Application Layer Grid Application Layer

Job Job Management Management Data Data Management Management

Collective Services Collective Services

Information Information & Monitoring & Monitoring Replica Replica Manager Manager Grid Grid Scheduler Scheduler

Underlying Grid Services Underlying Grid Services

Database Database Services Services Computing Computing Element Element Services Services Storage Storage Element Element Services Services Replica Replica Catalog Catalog Authorization Authorization Authentication Authentication & Accounting & Accounting Logging & Logging & BookBookkeeping keeping



Fabric services Fabric services

Resource Resource Management Management Configuration Configuration Management Management Monitoring Monitoring and Fault and Fault Tolerance Tolerance Node Node Installation & Installation & Management Management Fabric Storage Fabric Storage Management Management


Client Applications
Web Command line D0 Framework C++ codes Python codes, Java codes

Request Formulator and Planner

Request Manager

Cache Manager

Job Manager

Storage Manager

Connectivity and Resource

SAM-specific user, group, node, station registration

Authentication and Security Fabric

Tape Storage Elements Disk Storage Elements Compute Elements LANs and WANs Code Repostory Resource and Services Catalog Replica Catalog Meta-data Catalog

Collective Services

Dataset Editor

Project Master

Station Master

Station Master
Job Services

File Storage Server

Data Mover

SAM Resource Management

Batch Systems - LSF, FBS, PBS, Condor

Significant Event Logger Naming Service Catalog Manager

Database Manager



Catalog protocols

File transfer protocols ftp, bbftp, rcp


Mass Storage systems protocols e.g. encp, hpss


Bbftp cookie

Indicates component that will be replaced


or added

using PPDG and Grid tools

Name in quotes is SAM-given software component name

GriPhyN/PPDG Data Grid Architecture

12 From: GriPhyN/PPDG Data Grid Architecture, Toolkit, and Roadmap V2

1. 2. Anatomy of the Grid: Enabling Scalable Virtual Organizations Grid Architecture EDG Tutorial Architecture Overview Tutorial - Architecture Overview.ppt GriPhyN/PPDG Data Grid Architecture, Toolkit, and Roadmap V2 2.09.pdf DO-SAM and the Particle Physics Data Grid


4. 5.

Tein Horng Yuan Academia Sinica Computing Centre PC Farm project leader since 1997 Oct.