You are on page 1of 13

Resource

deployment

CERN openlab II quarterly review


20 September 2006

Xavier Gréhant
Resource deployment in production

ƒ CERN servers hosting services


https://websvc02.cern.ch/WinServices/

snapshot from Emmanuel Ormancey, Alberto Pace, hepix.caspur.it/spring2006/TALKS/7apr.pace.virtsrv.pdf

CERN openlab presentation – 2006 2


Resource deployment in production

ƒ Amazon Elastic Compute Cloud (EC2)


ƒ Elastic = variable number of servers

ƒ Fixed server characteristics


• 1.7Ghz Xeon CPU
• 1.75GB of RAM
• 160GB of local disk
• 250Mbps of network bandwidth.

ƒ Pricing
• 10 cents per server-hour consumed
• 20 cents per GB transferred outside of Amazon
• 15 cents per GB-month of Amazon S3 storage for images.

Source: Grid Today

CERN openlab presentation – 2006 3


Introducing Tycoon

ƒ Fixed CPU capacity allocation


• e.g: Batch system, Amazon EC2, CERN WoD

33% CPU
time

job job
started terminated

ƒ Variable computational unit share


• e.g: DGAS, Tycoon

time

comp. share
unit % total CPU

CERN openlab presentation – 2006 4


Tycoon architecture

ƒ Architecture
ƒ Auctioneer: Xen VM management with Python
bids shares
VM
Physical
node

ƒ Market
bank
fund

client auct
adjust

list heartbeat
sls

CERN openlab presentation – 2006 5


Testing Tycoon Maturity

ƒ Former version tested


ƒ Multiple CPUs not handled
ƒ Memory, disk not handled
ƒ Memory overflow : crash

ƒ New version
ƒ Bid on memory
ƒ Overbid (=overflow) : account closed

ƒ Live migration?
ƒ Ok if we restart your job elsewhere …
ƒ … from 0 after 10h run?

CERN openlab presentation – 2006 6


Investigating Tycoon relevance

ƒ Good to use Xen: no VM overhead.

ƒ Too fine-grained for scientific grids?


ƒ HEP vs Biomed & generic apps
ƒ Sharing CPU: ok. But memory?

ƒ What configuration?
ƒ Empty fc6 image. Happy?
ƒ No. We want slc4 with Geant4, AFS client ...

ƒ Outside the grid


ƒ Independent cluster
ƒ Service level agreements?

CERN openlab presentation – 2006 7


Focus on VM management

ƒ SmartFrog
ƒ Orchestrating components easily

ƒ Wherever they sit

CERN openlab presentation – 2006 8


Our VM manager

CERN openlab presentation – 2006 9


Orchestrating VM managers

CERN openlab presentation – 2006 10


Orchestrating VM managers

CERN openlab presentation – 2006 11


Objectives

ƒ VM managers
ƒ Migrateable
ƒ Fine-grained, dynamic resource binding

ƒ Orchestration
ƒ Multi-VMs node, single image
ƒ Load balancing, migration
ƒ Grid integrated service

CERN openlab presentation – 2006 12


People and collaboration
ƒ Tycoon
ƒ Project from HP Labs Palo Alto
ƒ Summer student: Salman Toor
ƒ Grid relevance & integration
• Discussions with EGEE
ƒ Large-scale tests to come
• Kurchatov institute
• Greek Research and Technology Network SmartFrog
• EGEE’06 presentation
ƒ SmartFrog
ƒ Core from HP Labs Bristol
ƒ Summer student: Olivier Pernet
ƒ SA3 certification testbed
• Andreas Unterkircher and Omer Khalid
ƒ PhD thesis
ƒ ENST Paris, Pr. Isabelle Demeure
“Loose virtual environments
for efficient computing resources supply on a grid.“

CERN openlab presentation – 2006 13