Oracle Real Application (lusters
10 - the loundation íor
Lnterprise Grid (omputing
.v Oracte !bite Paper
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 2
Oracle Real Application (lusters 10 - the
loundation íor Lnterprise Grid (omputing
ORACLE REAL APPLICATION CLUSTERS - FUNDAMENTAL TO
ACHIEVING ENTERPRISE GRID COMPUTING BENEFITS
In the past íew vears. there ha·e been a number oí signiíicant impro·ements in
computing hardware. networking. chip architectures. data storage svstems and in
standards-based open-source operating svstems. Oracle is the íirst ·endor that
oííers a complete enterprise computing soítware inírastructure that le·erages all oí
these impro·ements to deli·er signiíicant economic return on in·estment through
better a·ailabilitv. scalabilitv. ílexibilitv. and manageabilitv. Oracle Real Application
(lusters RA(, is a kev element in this unique. popular and successíul solution.
\hether vou run an enterprise data center ser·ing 200.000 users. a small to
medium-sized operation ser·ing 200 users or a web-based business handling 25.000
transactions per hour. vou are concerned with economicallv maximizing the
a·ailabilitv. scalabilitv. ílexibilitv and the manageabilitv oí vour computing resource.
Oracle`s Real Application (lusters. introduced in 2001. oííers a pro·en alternati·e
to traditional expensi·e hardware and proprietarv soítware solutions íor achie·ing
these goals. 1his paper presents se·eral scenarios representing the common
questions or decisions I1 executi·es íace and how enterprise customers ha·e used
Oracle Real Application (lusters to address these business needs.
OracIe ReaI AppIication CIusters - the foundation for Enterprise Grid
In June 2001. Oracle released Oracle9i with Real Application (lusters. Oracle9i
RA( made it possible íor a collection oí database ser·ers to cooperate
transparentlv in the management oí a single Oracle database. 1his allowed Oracle
to deli·er greater scalabilitv and a·ailabilitv than had pre·iouslv been possible while
simultaneouslv reducing costs and impro·ing ílexibilitv. In Januarv 2004 Oracle
released the second generation oí its clustered database technologv. Oracle Real
Application (lusters 10g release 1 and in June oí 2005. 10g Release 2.
OracIe ReaI AppIication
CIusters (RAC) is a key enabIing
technoIogy for OracIe
Enterprise Grid computing.
OracIe RAC 10g ReIease 2
offers a compIete grid
infrastructure for each Iayer of
the enterprise grid -
appIication, database and
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 3
Figure 1: OracIe RAC -cIustering database servers - foundation for Enterprise Grid
Computing deIivering high avaiIabiIity, scaIabiIity and fIexibiIity.
QUESTIONS AND DECISIONS
\e speak with manv companies throughout the course oí the vear. manv (IOs.
(1Os and executi·es responsible íor the computing inírastructure íor a multitude
oí enterprises. large to small. Increasinglv. thev look to ·endors such as Oracle íor
ad·ice and guidance on the decisions and questions thev íace like:
- ´My SMP infrastructure reaches its financial and technology
refresh point in fourteen months. What do we do? Is this an
opportunitv to consider an enterprise grid architecture based upon
Oracle 10g Real Application (lusters?" A major on||ne reta||er made the
sh|ft and we've documented |t here.
- ´Our business is successful and growing ÷ that's the good news
and the bad news. We seem to reach capacity on our critical
systems and it requires a forklift upgrade to get more scalability.
Lxpensi·e and disrupti·e. \ill Oracle Real Application (lusters 10g
and the Oracle Grid ·ision pro·ide a less disrupti·e. more ílexible and
RAC: The CIuster Database
Point Of FaiIure
Storage Area Net work
ReaI AppIication CIusters
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 4
scalable solution·` 1he 6h|cago 8tock Exchange tack|ed th|s concern
w|th 0rac|e RA6 as to|d be|ow.
- ´Our average server utilization is well below 50° -- sure we ha·e
spikes to o·er 90° but that happens at diííerent times oí the month íor
each oí the ·arious applications we`·e dedicated to these svstems.` \ill
Oracle`s 10g grid solution impro·e our utilization AND enhance the
a·ailabilitv. scalabilitv and ílexibilitv oí mv en·ironment Ta|k Amer|ca
|s a b|g proponent of 0rac|e 10 Rea| App||cat|on 6|usters because |t
enab|es a 8erv|ce 0r|ented Arch|tecture wh|ch v|rtua||zes the under|y|ng
resources prov|d|ng better hardware ut|||zat|on, f|ex|b|||ty, |mproved 8LAs
and |ower cost.
E-RETAILER - SMP REFRESH? NO - OPTING INSTEAD FOR LINUX-
BASED REAL APPLICATION CLUSTERS FOR THEIR MISSION
\ell-run companies manage their capital budgets and inírastructure spending ·erv
careíullv. 1wo vears ago. the core one oí the largest e-retailers computing
inírastructure was a large SMP svstem handling OL1P and Data \arehousing
acti·ities. 1hat asset was due íor reíresh´replacement. 1his svstem ran a number oí
applications and Oracle databases írom their (atalog. (ustomer database and their
\hat thev thought was going to be a simple contract renewal wasn`t. 1his customer
suííered se·ere sticker shock when presented with a nearlv >10M million quote to
reíresh the SMP UNIX en·ironment that had been their mainstav. At this point.
the customer had to look íor other wavs to achie·e their high a·ailabilitv and
scalabilitv goals while maintaining their períormance le·els.
Instead. thev architected a cluster oí Intel-based commoditv ser·ers running Linux.
Oracle Real Application (lusters with its embedded clusterware and cluster íile
svstem. Deploving on twentv-eight nodes in three clusters one containing 16
nodes,. this e-retailer was able to migrate to an a·ailable and scalable en·ironment
íor a total capital cost oí 25° oí what was initiallv proposed bv the hardware
·endor sa·ing them nearlv >6M in capital outlavs. 1he annual support and
maintenance íees were correspondinglv reduced bv ¯0°. Alreadv íamiliar with the
Oracle Database operating in a single instance mode on their SMP svstem. this e-
retailer íelt comíortable considering a Real Applications (lusters solution.
High AvaiIabiIity: One oí the kev dri·ers to mo·e to Oracle RA( íor this e-Retailer
was its compelling acti·e´acti·e` architecture and the high a·ailabilitv that oííered
Sticker Shock of nearIy $10M for a
UNIX-SMP pIatform refresh made the
decision to move to a of RAC Linux-
based cIuster easy for this weII-
known e-retaiIer. RAC running on Iow
cost servers enabIed them to avoid $6
million in capital outlays.
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 5
while pro·iding better resource utilization.. \ith Oracle RA(. ií a node in the
cluster íails. the database keeps on running. All other nodes in the cluster remain up
and users continue processing. \ith a single SMP ser·er. ií anvthing íails the whole
machine goes down - user processing is interrupted and can take 20 to 30 minutes
to resume on a cold standbv acti·e´passi·e, svstem. In a cold´standbv
en·ironment. when the primarv svstem íails. administrators ha·e to manuallv
dismount and remount storage ·olumes. user connections ha·e to be established to
the new ser·er and the buííer cache in the standbv`s memorv needs to be populated
with the most írequentlv used data. A signiíicant part oí the cost sa·ings this e-
Retailer experienced was due to not ha·ing to purchase and warehouse a cold
íailo·er SMP svstem.
Oracle Real Application (luster`s acti·e´acti·e svstem oííers ·irtuallv uninterrupted
processing íor users and signiíicantlv better. more complete utilization oí
computing resources as compared to the traditional cold standbv solutions. Oracle`s
RA( eliminates the need íor these reco·erv steps - all cluster nodes are alwavs
connected to all the storage so no ·olume dismounting and remounting is required.
Users on the sur·i·ing nodes remain connected onlv the users on the íailed nodes
need to reconnect and RA( does this:
- rapidlv and transparentlv such that users oíten don`t e·en know a
íailure has occurred
- automaticallv - no need íor administrator inter·ention and pre-set
íailo·er preíerences to speciíied nodes can be established through
Oracle Lnterprise Manager.
Oracle RA( svstems can be coníigured to ha·e no single point oí íailure. e·en
when running on low-cost. commoditv hardware and storage. In an Oracle RA(
en·ironment ií database ser·ers íail their applications simplv keep running. lail-
o·er is írequentlv transparent to applications and occurs in seconds. Oracle also
protects írom major site íailures bv combining Oracle RA( with Oracle Data
Guard - part oí Oracle`s Maximum A·ailabilitv Architecture MAA,.
THE CHICAGO STOCK EXCHANGE - MORE CAPACITY WITHOUT THE
1he (hicago Stock Lxchange (lX, is the third most acti·e stock exchange in the
United States bv ·olume. It is a technologicallv ad·anced exchange. pro·iding íast.
qualitv executions. Due to the nature oí the business. continuous uptime. scalabilitv
on demand. and rock-solid períormance are absolutelv critical.
lacing shrinking re·enues in the late 1990s. (lX's I1 department needed to íind a
wav to do more with less monev. At the same time. it needed to eliminate
bottlenecks created bv massi·e data loads on hea·v trading davs. Seeking peak
reliabilitv and períormance. (lX chose Oracle Database with Real Application
(lusters on lP hardware íor its optimal solution. (lX`s needs were growing. most
Our onIy question was: ReaI
AppIication CIusters or not?" said
David MiIne, director of database
technoIogies for the Chicago Stock
Exchange. "Oracle's clustering
technology was irresistible. Most
important was its reIiabiIity and
avaiIabiIity. Its seIIing points were
ease and fIexibiIity in adding Iarge or
smaII hardware as needed, where
needed, and no changes required to
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 6
likelv requiring it to procure the next class oí SMP machine. But. (lX was
concerned about signiíicantlv o·er-buving or under-buving ser·er capacitv - trading
·olumes spiked bv 2 times in 2000 but dropped signiíicantlv in 2002. 1hev also
were concerned that as newer technologv emerged. thev would not be able to take
ad·antage oí it without replacing the entire ser·er inírastructure.
In a two node cluster. a hardware íailure on one node could negati·elv impact all
users. and result in lost trading re·enues Migrating to a íour node RA(´Grid
en·ironment signiíicantlv reduced the exposure to a single hardware íailure. Oracle
RA( allowed the (hicago Stock Lxchange to easilv and quicklv add more low-cost
ser·ers to the cluster. 1his has signiíicantlv reduced their exposure and extended
the useíul liíe oí existing hardware. \ith Oracle RA( a hardware íailure on one
node is not catastrophic. the workload is automaticallv shiíted to other hardware in
the Grid. Most oí the time the users are not aware oí the íailure. In addition. the
results oí a ser·er íailure are less urgent in the Grid en·ironment. while still a
critical e·ent. it is not necessarilv the all hands on deck` scenario.
Improved ScaIabiIity, FIexibiIity and ManageabiIity can be achieved
Lnterprises ha·e also had to o·er-pro·ision to accommodate changes in business
transaction ·olume and in the spikiness oí web-based transaction acti·itv. Scaling
in a traditional SMP en·ironment required long lead time ad·ance planning due to
the lead times in obtaining the hardware. preparing the en·ironment and shiíting
ScaIe Out: Oracle RA( allows multiple ser·ers in a cluster to transparentlv manage a
single database. Oracle RA( allows database svstems to scale-out rather than
ha·ing to scale-up to meet additional workload increases. In the scale-up model oí
computing. once a ser·er has been íullv coníigured with (PUs and memorv. the
next step is an expensi·e "íork-liít upgrade". but with Oracle RA( users just plug
in another ser·er. ií additional processing power is needed.
RA( also pro·ides the opportunitv to rapidlv retire excess capacitv and re-deplov
that íor a diííerent purpose either as a node in an application ser·er mid-tier laver
or as an additional database cluster node in a diííerent cluster.
In addition to eliminating scalabilitv constraints. RA( has pro·en to scale better
than SMP when e·aluated in a head-to-head comparison. Gi·en the same number
and tvpe oí (PUs. the same amount oí total memorv. RA( has pro·en to oííer
greater scalabilitv and processing throughput than a big SMP en·ironment.
Project MegaGrid - Proving "scaIe out" as an economicaIIy attractive aIternative
As a íounding partner oí Project MegaGrid. in phase one oí this multi-phase
collaboration with Dell (omputers. LM( and Intel. compelling e·idence oí the
power oí a low-cost RA( en·ironment was gathered. Starting in 2004. Oracle.
Dell. LM( and Intel came together to ·alidate Oracle`s grid computing ·ision and
establish grid computing best practices as a wav to encourage adoption and reduce
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 7
market coníusion and hesitation to embrace this new model. In phase 1. a standard
telecom ser·ice pro·isioning application írom (ramer Svstems. used bv telecom
companies such as British 1elecom and Bell South. was benchmarked on both a
large ¯2(PU SMP svstem and an Oracle RA( cluster oí low-cost commoditv Dell
ser·ers. 1he goal was to process as manv transactions oí a mixed workload per
hour while maintaining a maximum íour-second response time. 1he >2.8 million
SMP ser·er en·ironment was able to handle 550.000 transactions per hour. 1he
same ·olume oí transactions while maintaining the SLAs was achie·ed on a 10-
node. 20 (PU Dell PowerLdge 1¯50 ser·er cluster costing about >60.000. Another
test but maintaining the same SLAs was achie·ed on a íour-node íour (PU 64-bit
Dell ¯250 ser·er cluster costing >160.000.
Figure 2: CIustered, Iow-cost servers running ReaI AppIication CIusters can offer an
economic, avaiIabiIity and scaIabiIity advantages over traditionaI architectures
In business íor 16 vears. 1alk America pro·ides local and long distance telephone
ser·ices to residential and small business customers in the United States. O·er the
last íi·e vears. their emphasis has shiíted more to local telephone ser·ice where the
marketplace íor those ser·ices is ·erv acti·e and where customer ser·ice and where
1alk America s customer ser·ice and responsi·eness is a competiti·e ad·antage -
something thev take so seriouslv thev were awarded the J.D. Powers Ser·ice
"The beauty of OracIe 10g
and a RAC-enabIed grid is
that the choice is mine. I can
manage SLAs based upon
my business needs."
Laurence Grant - Vice
President of Enterprise
Computing Systems - TaIk
Cost Comparison: SMP vs. Grid
TeIecom WorkIoad - 550,000
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 8
Lxcellence award íor top customer satisíaction rating in the telecommunications
segment. (ore to 1alk America`s competiti·eness is their I1 operation and the
proprietarv integrated order processing. pro·isioning. billing. pavment. collection.
and customer ser·ice iníormation svstems which enables them to oííer
competiti·elv priced ser·ice plans. high-qualitv ser·ice and simplicitv through
consolidated billing and responsi·e customer care.
lundamental to high-qualitv customer ser·ice is a highlv a·ailable svstems
en·ironment. 1alk America`s legacv en·ironment. based upon 20 diííerent
Iníormix databases. was pre·enting them írom meeting their business goals -
database reliabilitv was a problem. Additionallv. 1alk America was íacing a number
oí other challenges that was stressing their I1 en·ironment:
- Rapid growth oí their business and the exponential growth oí call
records and transaction ·olumes
- Need to pro·ide a 24x¯ operation
- Increasing svstems dependencies - less time íor maintenance
- Shrinking nightlv batch windows
- Growing multitude oí svstems to satisív growing end-user. internal
users sales´ser·ice, and increased reporting requirements
- Much larger data ·olumes growing to 60 terabvtes írom onlv 1
terabvte eight vears earlier
1alk America had to make some changes - changes that mandated an upgrade oí
their inírastructure to a high períormance. highlv a·ailable. ílexible en·ironment.
Laurence Grant. Vice President oí Lnterprise (omputing Svstems at 1alk America.
and Vishal Anand. Director oí Database Administration. were concerned that
simplv upgrading their silo`ed en·ironment would be expensi·e and would not
reallv sol·e the problems thev íaced. In·ariablv. some decisions about ser·er and
storage capacitv would result in an o·er allocation íor some applications and under
allocation íor others.
1alk America decided to pursue Oracle Database 10g and Real Application (lusters
as their core database and high a·ailabilitv inírastructure. L·en though the solution
was ·erv new at the time. Larrv and Vishal belie·ed that the ad·antages 10g brought
to their problems íar outweighed the risks oí going with a relati·elv new product
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 9
OracIe Database 10 and ReaI AppIication CIusters - the underpinning
of a Service Oriented Architecture:
1alk America was interested in the economies that could be gained ií thev
consolidated databases and migrated to a ser·ice-oriented architecture. It meant
greater hardware utilization. more ílexibilitv to mo·e resources to accommodate
spikes in workload demands. and cost sa·ings bv not ha·ing to in·est in an
expensi·e. dedicated solution íor each major svstem. 1alk America had experience
with big iron solutions and thev didn`t want to buv hardware e·erv six. nine or
twel·e months to keep pace with the business - thev wanted to a·oid the business
disruption and the additional costs oí shuííling equipment in and out. Oracle`s 10g
RA( was the kev that enabled the high a·ailabilitv and workload sharing and
balancing allowing 1alk America to gain better resource utilization and a·oid
changing the hardware íootprint e·erv íew months.
1alk America wanted an en·ironment that would enable them to manage resources
to satisív the ·arious ser·ice le·el agreements SLAs, established íor their users.
1hev took ad·antage oí the 10g Grid capabilities and consolidated all their 20
separate databases into one 601B Oracle database. 1he ·arious ser·ices such as
Order Processing. Pro·isioning. Leads Processing. Data Marts. (RM etc. run on
the RA( cluster and share access to the single database - administration is easier.
a·ailabilitv is higher bv ·irtue oí Oracle RA(. and ílexibilitv is greater in terms oí
adding or remo·ing processing power to match the needs oí the application at anv
Leveraging AII the Hardware AII the Time
1he Oracle Database 10g ser·ice-oriented architecture enabled bv Oracle Real
Application (lusters. binds ser·ices to the underlving hardware - but in a ílexible.
·irtual wav. It is ·erv easv within a 10g architecture to run OL1P applications and
data warehousing within the same RA( cluster. And it`s easv to change the
allocation oí resources as business processing needs change - hence le·eraging all
the hardware all the time.
Larrv Grant explains the one oí the kev ad·antages an Oracle Real Application
(lusters 10g - enabled grid pro·ides.
. .ivpte e·avpte. .vppo.e í bare Oí)P .errice attocatea to tovr voae.. ít ove voae goe. aorv
í`v .titt bigbty araitabte becav.e tbree voae. are vp ava rvvvivg attbovgb yov vigbt be rvvvivg
ritb aegraaea pertorvavce. )bat vay be acceptabte. borerer. a. tovg a. yov`re rvvvivg. ßvt rbat
Oracte R.C ]0g .ay.. í`re got tovr voae. rvvvivg Oí)P ava tro voae. aoivg reportivg iv vy
.i· voae ctv.ter. Reportivg i. vot vi..iov·criticat. borerer. Ove ot vy tovr Oí)P voae. goe.
aorv vor iv.teaa ot rvvvivg iv.t ov tbree voae.. í`v goivg to reairect .ove ot vy trattic to ove ot
tbo.e otber reportivg voae.. í`v goivg to be rvvvivg Oí)P ritb tovr voae. ttbree .vrririvg voae.
ava ove re·pvrpo.ea reportivg voae). í av cov.ciov.ty bac/ivg ott vy reportivg .o iv.teaa ot vy
Oí)P rvvvivg aegraaea. it. goivg to rvv at tvtt .teav ava í`v goivg to tet vy reportivg .y.tev
aegraae. )be beavty ot Oracte ]0g ava a R.C·evabtea gria i. tbat tbe cboice i. vive. í cav
vavage ´í.. ba.ea vpov vy bv.ive.. veea..
OracIe ReaI AppIication CIusters 10 - the Foundation for Grid Computing Page 10
Figure 3. TaIk America's typicaI service aIIocation among the six OracIe RAC nodes
impIemented on a 96 CPU two HP Superdome environment. Services can be reIocated
and resource aIIocation expanded or contracted as daiIy, monthIy, year-end processing
1alk America has achie·ed greater hardware utilization. better SLA achie·ement.
and signiíicant cost sa·ings bv ·irtue oí the íull technologv stack supplied bv Oracle
Database 10g and Real Application (lusters no need to acquire >500.000 oí third
partv clustering soítware,. Again - le·eraging all the hardware all the time.
Oracle 10g Real Application (lusters oííers enterprises:
- Radical economic beneíits when migrating írom an aging SMP
en·ironment to a modern enterprise grid oí low cost ser·ers enabled with
- An agile response to changes in business requirements through the
addition or subtraction or re-purposing oí ser·er resources as transaction
·olumes. processing schedules. number oí user sessions or application mix
- Increased hardware utilization which enables vour I1 íunction to íund
more strategic initiati·es bv re-directing budget awav írom under-utilized
legacv hardware´soítware inírastructure to strategic de·elopment
initiati·es. 1his results in enhanced competiti·eness and customer
OracIe ReaI AppIication CIusters 10- the Foundation for Enterprise Grid Computing
Author: R. Hietter
500 OracIe Parkway
Redwood Shores, CA 94065
Copyright © 2005, OracIe. AII rights reserved.
This document is provided for information purposes onIy and the
contents hereof are subject to change without notice.
This document is not warranted to be error-free, nor subject to any
other warranties or conditions, whether expressed oraIIy or impIied
in Iaw, incIuding impIied warranties and conditions of merchantabiIity
or fitness for a particuIar purpose. We specificaIIy discIaim any
IiabiIity with respect to this document and no contractuaI obIigations
are formed either directIy or indirectIy by this document. This document
may not be reproduced or transmitted in any form or by any means,
eIectronic or mechanicaI, for any purpose, without our prior written permission.
OracIe, JD Edwards, and PeopIeSoft are registered trademarks of
OracIe Corporation and/or its affiIiates. Other names may be trademarks
of their respective owners.