Grid Computing

lecture by Camiel Plevier

• History of Computation Infrastructure

• Grid Concept
• Basic Architecture
– Resources
– Human Interfaces
– Grid Middleware
• Grid Topologies
• Security
• Application Development
• Dutch Space & Grid
• Demonstration
Grid Computing by Camiel Plevier 1

History of Computation Infrastructure

IT evolution
• 1960’s - (super)
computers with user
terminals
• 1970’s - desktop PC
• 1980’s - networks
and clusters
(homogenous grid)
• 1990’s - Internet connectivity (WWW, HTML, optical
networks) & open source (Linux)

Do you need more?

Grid Computing by Camiel Plevier 2
 Grid Concept

• Many heterogeneous computers over the whole world

can be used to provide a lot of CPU power
and data storage capacity
• Applications can be
executed at several
locations
• Combining
geographically
distributed
services
• Collaboration
• Seamless access,
Web services

Grid Computing by Camiel Plevier 3

Basic Architecture
User portals
Problem Solving
General Environments Application Science
Grid Access and Project tools and Scientific resources and
information experimental devices devices

Service layer (Grid services)

Network Information
Security, scheduling, events, Resource and data
monitoring and maintenance. management.

Resource layer (Physical resources)

Standalone
computers
Super- Cluster
computers Online instruments

Databases

Corporate network/Intranet
Grid Computing by Camiel Plevier 4
 Resources

Anything that your application might need

• Computing – CPU type and speed, memory size
• Data Centres – capacity, sharing
• Communications Network – connectivity, bandwidth
• Software and Licences – OS, local tools and libraries
• Special equipment, services
Non-Technical Characteristics
User portals
Problem Solving
General Environments Application Science
Grid Access and Project tools and Scientific resources and
information experimental devices devices

• Authorisation Policies Service layer (Grid services)

Network
Security, scheduling, events,
monitoring and maintenance.
Information
Resource and data
management.

– for example: for medical use only Resource layer (Physical resources)
Standalone
computers
Super- Cluster
computers Online instruments

• Accounting Databases

Corporate network/Intranet

Grid Computing by Camiel Plevier 5

Human Interfaces of Grid

• User portal or client tools

– Job definition, submission, control, monitoring and result
collection
– Available grid capacity monitoring
• Resource Provider
– Sharing based on characteristics
– Installation, administration and maintenance.
• Administrator User portals
General
Problem Solving
Environments Application Science
Grid Access and Project tools and Scientific resources and
information experimental devices devices

– Grid status monitoring

Service layer (Grid services)

– User and resource management Network

Security, scheduling, events,
monitoring and maintenance.
Information
Resource and data
management.

– Middleware and server maintenance Resource layer (Physical resources)

Super- Cluster
Standalone
computers
Online instruments
computers

Databases

Corporate network/Intranet

Grid Computing by Camiel Plevier 6

 Grid Middleware

Software that connects other software components or

applications to provide the following functions:
• Run applications on suitable available resources
– Brokering, Scheduling
• Provide uniform, high-level access to resources
– Semantic interfaces
– Web Services, Service Oriented Architectures
• Address inter-domain issues of User portals
General
Grid Access and
information
Problem Solving
Environments
Project tools and
Application Science
Scientific resources and
devices
experimental devices

security, policy, etc. Service layer (Grid services)

Network Information

– Federated Identities Security, scheduling, events,

monitoring and maintenance.
Resource and data
management.

• Provide application-level status

Resource layer (Physical resources)
Standalone
computers
Super- Cluster
computers Online instruments

Databases

monitoring and control Corporate network/Intranet

Grid Computing by Camiel Plevier 7

Grid Topologies

• Intragrid
– Local grid within an organisation
– Trust based on personal contracts
• Extragrid
– Resources of a consortium of organisations connected through a
(Virtual) Private Network
– Trust based on Business to Business contracts
• Intergrid
– Global sharing of resources through the internet
– Trust based on certification

Grid Computing by Camiel Plevier 8

 Security

• Authentication, Authorisation, Accounting, Assurance

• Public-key cryptography, Certificates

Grid Computing by Camiel Plevier 9

Grid Application Development

• From Existing Programs

• Parallel Speedup vs. Sequential Fraction
• Formulating new algorithms
– Reduce dependencies between calculation parts
• Embarrassingly Parallel
– Monte Carlo Simulations
– Repeated application to unrelated data (for example: spatial
distribution)

Grid Computing by Camiel Plevier 10

 Parallel Speedup vs. Sequential Fraction
application parallelizability
10
9
Amdahl’s law

100%
98% 8

speedup factor
95%
7
90%
75% 6
50% 5
4
3
2
1
1 2 3 4 5 6 7 8 9 10
number of processors

• speedup assumed proportional with number of processors

• viewpoint from existing application; new applications will be fitted to
the possibilities
Grid Computing by Camiel Plevier 11

Dutch Space & Grid

Grid Computing by Camiel Plevier 12

 Demonstration

Tide_grid_workflow_demo.avi

Grid Computing by Camiel Plevier 13

Thank you for your attention

Gri ?
d? Middleware
?
er

?
Scheduling
ok

Au
Br

tho
r iza
e?

tion
ic

?
rv
Se

Au
the
ntic Job?
atio
?

n?
re
nt
ce
ta
Da

Grid Computing by Camiel Plevier 14

 Exercise

• Pick a role from the stack.

• Play your role in an anticipating fashion; try to prepare as
much as possible
• Every data communication should be on paper, passed
on one-on-one (no broadcasting)
– User <-> Middleware
– Middleware <-> Resource
– NO resource <-> resource (third party transfers)
• Middleware actors should discuss to divide the work.
Write discoveries and agreements on the role definition
paper.

Grid Computing by Camiel Plevier 15

Role overview and Workflow

• User
• Middleware
– User Portal
– Controller
– Scheduler
– ... Need more?
– Broker
– Agents
• Resources
Grid Computing by Camiel Plevier 16
 Grid Role Play Scenarios

• Static
• Dynamic resources
• Failing jobs
• Cancel application
• Different topology
– grouping resources into cluster
– dividing middleware into middleware groups

Grid Computing by Camiel Plevier 17