You are on page 1of 13



CERN openlab II quarterly review

20 September 2006

Xavier Gréhant
Resource deployment in production

ƒ CERN servers hosting services

snapshot from Emmanuel Ormancey, Alberto Pace,

CERN openlab presentation – 2006 2

Resource deployment in production

ƒ Amazon Elastic Compute Cloud (EC2)

ƒ Elastic = variable number of servers

ƒ Fixed server characteristics

• 1.7Ghz Xeon CPU
• 1.75GB of RAM
• 160GB of local disk
• 250Mbps of network bandwidth.

ƒ Pricing
• 10 cents per server-hour consumed
• 20 cents per GB transferred outside of Amazon
• 15 cents per GB-month of Amazon S3 storage for images.

Source: Grid Today

CERN openlab presentation – 2006 3

Introducing Tycoon

ƒ Fixed CPU capacity allocation

• e.g: Batch system, Amazon EC2, CERN WoD

33% CPU

job job
started terminated

ƒ Variable computational unit share

• e.g: DGAS, Tycoon


comp. share
unit % total CPU

CERN openlab presentation – 2006 4

Tycoon architecture

ƒ Architecture
ƒ Auctioneer: Xen VM management with Python
bids shares

ƒ Market

client auct

list heartbeat

CERN openlab presentation – 2006 5

Testing Tycoon Maturity

ƒ Former version tested

ƒ Multiple CPUs not handled
ƒ Memory, disk not handled
ƒ Memory overflow : crash

ƒ New version
ƒ Bid on memory
ƒ Overbid (=overflow) : account closed

ƒ Live migration?
ƒ Ok if we restart your job elsewhere …
ƒ … from 0 after 10h run?

CERN openlab presentation – 2006 6

Investigating Tycoon relevance

ƒ Good to use Xen: no VM overhead.

ƒ Too fine-grained for scientific grids?

ƒ HEP vs Biomed & generic apps
ƒ Sharing CPU: ok. But memory?

ƒ What configuration?
ƒ Empty fc6 image. Happy?
ƒ No. We want slc4 with Geant4, AFS client ...

ƒ Outside the grid

ƒ Independent cluster
ƒ Service level agreements?

CERN openlab presentation – 2006 7

Focus on VM management

ƒ SmartFrog
ƒ Orchestrating components easily

ƒ Wherever they sit

CERN openlab presentation – 2006 8

Our VM manager

CERN openlab presentation – 2006 9

Orchestrating VM managers

CERN openlab presentation – 2006 10

Orchestrating VM managers

CERN openlab presentation – 2006 11


ƒ VM managers
ƒ Migrateable
ƒ Fine-grained, dynamic resource binding

ƒ Orchestration
ƒ Multi-VMs node, single image
ƒ Load balancing, migration
ƒ Grid integrated service

CERN openlab presentation – 2006 12

People and collaboration
ƒ Tycoon
ƒ Project from HP Labs Palo Alto
ƒ Summer student: Salman Toor
ƒ Grid relevance & integration
• Discussions with EGEE
ƒ Large-scale tests to come
• Kurchatov institute
• Greek Research and Technology Network SmartFrog
• EGEE’06 presentation
ƒ SmartFrog
ƒ Core from HP Labs Bristol
ƒ Summer student: Olivier Pernet
ƒ SA3 certification testbed
• Andreas Unterkircher and Omer Khalid
ƒ PhD thesis
ƒ ENST Paris, Pr. Isabelle Demeure
“Loose virtual environments
for efficient computing resources supply on a grid.“

CERN openlab presentation – 2006 13