Professional Documents
Culture Documents
Expert Oracle Exadata
Uploaded by
vbpanchalCopyright
Available Formats
Share this document
Did you find this document useful?
Is this content inappropriate?
Report this DocumentCopyright:
Available Formats
Expert Oracle Exadata
Uploaded by
vbpanchalCopyright:
Available Formats
Expert Oracle Exadata
Kerry Osborne
Randy Johnson
Tanel Pöder
Expert Oracle Exadata
#OPYRIGHT Ú BY +ERRY /SBORNE 2ANDY *OHNSON 4ANEL 0DER
!LL RIGHTS RESERVED .O PART OF THIS WORK MAY BE REPRODUCED OR TRANSMITTED IN ANY FORM OR BY ANY MEANS
ELECTRONIC OR MECHANICAL INCLUDING PHOTOCOPYING RECORDING OR BY ANY INFORMATION STORAGE OR RETRIEVAL
SYSTEM WITHOUT THE PRIOR WRITTEN PERMISSION OF THE COPYRIGHT OWNER AND THE PUBLISHER
)3".
)3".
(eBook)
4RADEMARKED NAMES LOGOS AND IMAGES MAY APPEAR IN THIS BOOK 2ATHER THAN USE A TRADEMARK SYMBOL
WITH EVERY OCCURRENCE OF A TRADEMARKED NAME LOGO OR IMAGE WE USE THE NAMES LOGOS AND IMAGES ONLY
IN AN EDITORIAL FASHION AND TO THE BENEFIT OF THE TRADEMARK OWNER WITH NO INTENTION OF INFRINGEMENT OF
THE TRADEMARK
4HE USE IN THIS PUBLICATION OF TRADE NAMES TRADEMARKS SERVICE MARKS AND SIMILAR TERMS EVEN IF THEY ARE
NOT IDENTIFIED AS SUCH IS NOT TO BE TAKEN AS AN EXPRESSION OF OPINION AS TO WHETHER OR NOT THEY ARE SUBJECT
TO PROPRIETARY RIGHTS
0RESIDENT AND 0UBLISHER 0AUL -ANNING
,EAD %DITOR *ONATHAN 'ENNICK
4ECHNICAL 2EVIEWER +EVIN #LOSSON
%DITORIAL "OARD 3TEVE !NGLIN -ARK "ECKNER %WAN "UCKINGHAM 'ARY #ORNELL *ONATHAN 'ENNICK
*ONATHAN (ASSELL -ICHELLE ,OWMAN *AMES -ARKHAM -ATTHEW -OODIE *EFF /LSON *EFFREY 0EPPER
&RANK 0OHLMANN $OUGLAS 0UNDICK "EN 2ENOW
#LARKE $OMINIC 3HAKESHAFT -ATT 7ADE
4OM 7ELSH
#OORDINATING %DITOR !DAM (EATH
#OPY %DITOR *AMES #OMPTON
#OMPOSITOR "YTHEWAY 0UBLISHING 3ERVICES
)NDEXER 30) 'LOBAL
!RTIST 30) 'LOBAL
#OVER $ESIGNER !NNA )SHCHENKO
$ISTRIBUTED TO THE BOOK TRADE WORLDWIDE BY 3PRINGER 3CIENCE "USINESS -EDIA ,,# 3PRING 3TREET
TH &LOOR .EW 9ORK .9 0HONE
302).'%2 FAX
E
MAIL ^aSTab ]h/b_aX]VTa
bQ\ R^\ OR VISIT fff b_aX]VTa^][X]T R^\
&OR INFORMATION ON TRANSLATIONS PLEASE E
MAIL aXVWcb/P_aTbb R^\ OR VISIT fff P_aTbb R^\
!PRESS AND FRIENDS OF %$ BOOKS MAY BE PURCHASED IN BULK FOR ACADEMIC CORPORATE OR PROMOTIONAL USE
E"OOK VERSIONS AND LICENSES ARE ALSO AVAILABLE FOR MOST TITLES &OR MORE INFORMATION REFERENCE OUR
3PECIAL "ULK 3ALESnE"OOK ,ICENSING WEB PAGE AT fff P_aTbb R^\ Qd[Z bP[Tb
4HE INFORMATION IN THIS BOOK IS DISTRIBUTED ON AN hAS ISv BASIS WITHOUT WARRANTY !LTHOUGH EVERY
PRECAUTION HAS BEEN TAKEN IN THE PREPARATION OF THIS WORK NEITHER THE AUTHORS NOR !PRESS SHALL HAVE
ANY LIABILITY TO ANY PERSON OR ENTITY WITH RESPECT TO ANY LOSS OR DAMAGE CAUSED OR ALLEGED TO BE CAUSED
DIRECTLY OR INDIRECTLY BY THE INFORMATION CONTAINED IN THIS WORK
Contents at a Glance
About the Authors................................................................................................ xvi
About the Technical Reviewer ............................................................................ xvii
Acknowledgments ............................................................................................. xviii
Introduction ......................................................................................................... xix
Chapter 1: What Is Exadata?...................................................................................1
Chapter 2: Offloading / Smart Scan ......................................................................23
Chapter 3: Hybrid Columnar Compression............................................................65
Chapter 4: Storage Indexes.................................................................................105
Chapter 5: Exadata Smart Flash Cache...............................................................125
Chapter 6: Exadata Parallel Operations ..............................................................143
Chapter 7: Resource Management......................................................................175
Chapter 8: Configuring Exadata ..........................................................................237
Chapter 9: Recovering Exadata...........................................................................275
Chapter 10: Exadata Wait Events........................................................................319
Chapter 11: Understanding Exadata Performance Metrics.................................345
Chapter 12: Monitoring Exadata Performance....................................................379
Chapter 13: Migrating to Exadata.......................................................................419
Chapter 14: Storage Layout ................................................................................467
Chapter 15: Compute Node Layout .....................................................................497
Chapter 16: Unlearning Some Things We Thought We Knew .............................511
III
#/.4%.43 !4 ! ',!.#%
Appendix A: CellCLI and dcli ...............................................................................535
Appendix B: Online Exadata Resources ..............................................................545
Appendix C: Diagnostic Scripts...........................................................................547
Index ...................................................................................................................551
IV
Contents
About the Authors................................................................................................ xvi
About the Technical Reviewer ............................................................................ xvii
Acknowledgments ............................................................................................. xviii
Introduction ......................................................................................................... xix
Chapter 1: What Is Exadata?...................................................................................1
An Overview of Exadata.....................................................................................................2
History of Exadata..............................................................................................................3
Alternative Views of What Exadata Is ................................................................................5
Data Warehouse Appliance....................................................................................................................... 5
OLTP Machine........................................................................................................................................... 6
Consolidation Platform ............................................................................................................................. 6
Configuration Options ........................................................................................................7
Exadata Database Machine X2-2.............................................................................................................. 7
Exadata Database Machine X2-8.............................................................................................................. 8
Hardware Components ......................................................................................................9
Operating Systems ................................................................................................................................. 10
Database Servers ................................................................................................................................... 11
Storage Servers...................................................................................................................................... 11
InfiniBand ............................................................................................................................................... 11
Flash Cache ............................................................................................................................................ 12
Disks....................................................................................................................................................... 12
V
#/.4%.43
Bits and Pieces ....................................................................................................................................... 12
Software Components .....................................................................................................12
Database Server Software...................................................................................................................... 12
Storage Server Software ........................................................................................................................ 16
Software Architecture......................................................................................................18
Summary .........................................................................................................................21
Chapter 2: Offloading / Smart Scan ......................................................................23
Why Offloading Is Important ............................................................................................23
What Offloading Includes.................................................................................................27
Column Projection .................................................................................................................................. 27
Predicate Filtering .................................................................................................................................. 32
Storage Indexes...................................................................................................................................... 33
Simple Joins (Bloom Filters)................................................................................................................... 34
Function Offloading ................................................................................................................................ 37
Compression/Decompression................................................................................................................. 39
Encryption/Decryption ............................................................................................................................ 40
Virtual Columns ...................................................................................................................................... 41
Data Mining Model Scoring .................................................................................................................... 43
Non-Smart Scan Offloading.................................................................................................................... 44
Smart Scan Prerequisites ................................................................................................45
Full Scans ............................................................................................................................................... 45
Direct Path Reads ................................................................................................................................... 45
Exadata Storage ..................................................................................................................................... 46
Smart Scan Disablers ......................................................................................................49
Simply Unavailable ................................................................................................................................. 49
Reverting to Block Shipping ................................................................................................................... 49
Skipping Some Offloading ...................................................................................................................... 50
VI
#/.4%.43
How to Verify That Smart Scan is Happening ..................................................................50
10046 Trace............................................................................................................................................ 51
Performance Statistics (v$sessstat) ....................................................................................................... 52
Offload Eligible Bytes.............................................................................................................................. 54
SQL Monitoring ....................................................................................................................................... 58
Parameters ......................................................................................................................61
Summary .........................................................................................................................64
Chapter 3: Hybrid Columnar Compression............................................................65
Oracle Storage Review ....................................................................................................65
Oracle Compression Mechanisms ...................................................................................68
BASIC...................................................................................................................................................... 68
OLTP ....................................................................................................................................................... 68
HCC......................................................................................................................................................... 69
HCC Mechanics................................................................................................................72
HCC Performance.............................................................................................................73
Load Performance .................................................................................................................................. 73
Query Performance................................................................................................................................. 78
DML Performance................................................................................................................................... 83
Expected Compression Ratios .........................................................................................91
Compression Advisor.............................................................................................................................. 91
Real World Examples.............................................................................................................................. 94
Restrictions/Challenges.................................................................................................100
Moving Data to a non-Exadata Platform............................................................................................... 100
Disabling Serial Direct Path Reads ....................................................................................................... 100
Locking Issues...................................................................................................................................... 101
Single Row Access ............................................................................................................................... 102
Common Usage Scenarios.............................................................................................102
VII
#/.4%.43
Summary .......................................................................................................................104
Chapter 4: Storage Indexes.................................................................................105
Structure........................................................................................................................105
Monitoring Storage Indexes...........................................................................................107
Database Statistics............................................................................................................................... 107
Tracing.................................................................................................................................................. 108
Monitoring Wrap Up.............................................................................................................................. 111
Controlling Storage Indexes...........................................................................................111
_kcfis_storageidx_disabled ................................................................................................................. 111
_kcfis_storageidx_diag_mode ............................................................................................................. 112
_cell_storidx_mode.............................................................................................................................. 112
Storage Software Parameters .............................................................................................................. 112
Behavior.........................................................................................................................113
Performance ..................................................................................................................114
Special Optimization for Nulls .............................................................................................................. 116
Physical Distribution of Values ............................................................................................................. 117
Potential Issues .............................................................................................................118
Incorrect Results .................................................................................................................................. 119
Moving Target....................................................................................................................................... 119
Partition Size ........................................................................................................................................ 122
Incompatible Coding Techniques ......................................................................................................... 122
Summary .......................................................................................................................124
Chapter 5: Exadata Smart Flash Cache...............................................................125
Hardware .......................................................................................................................125
Cache vs. Flash Disk......................................................................................................127
Using Flash Memory as Cache ............................................................................................................. 128
How ESFC is Created ............................................................................................................................ 132
VIII
#/.4%.43
Controlling ESFC Usage ........................................................................................................................ 134
Monitoring .....................................................................................................................135
At the Storage Layer............................................................................................................................. 135
At the Database Layer .......................................................................................................................... 139
Performance ..................................................................................................................140
Summary .......................................................................................................................141
Chapter 6: Exadata Parallel Operations ..............................................................143
Parameters ....................................................................................................................143
Parallelization at the Storage Tier .................................................................................145
Auto DOP........................................................................................................................146
Operation and Configuration................................................................................................................. 146
I/O Calibration....................................................................................................................................... 149
Auto DOP Wrap Up ................................................................................................................................ 152
Parallel Statement Queuing ...........................................................................................152
The Old Way.......................................................................................................................................... 153
The New Way........................................................................................................................................ 153
Controlling Parallel Queuing ................................................................................................................. 157
Parallel Statement Queuing Wrap Up ................................................................................................... 165
In-Memory Parallel Execution........................................................................................165
Summary .......................................................................................................................175
Chapter 7: Resource Management......................................................................175
Database Resource Manager.........................................................................................176
Consumer Groups ................................................................................................................................. 178
Plan Directives...................................................................................................................................... 181
Resource Plan....................................................................................................................................... 183
Resource Manager Views ..................................................................................................................... 184
The Wait Event: resmgr: cpu quantum ................................................................................................. 185
IX
#/.4%.43
DBRM Example ..................................................................................................................................... 185
Testing a Resource Plan....................................................................................................................... 190
Instance Caging .............................................................................................................198
Configuring and Testing Instance Caging............................................................................................. 199
Over-Provisioning ................................................................................................................................. 204
I/O Resource Manager ...................................................................................................204
How IORM Works.................................................................................................................................. 206
Methods for Managing I/O on Exadata ................................................................................................. 209
Bringing It All Together......................................................................................................................... 215
IORM Monitoring and Metrics ............................................................................................................... 219
Summary .......................................................................................................................235
Chapter 8: Configuring Exadata ..........................................................................237
Exadata Network Components ......................................................................................237
The Management Network ................................................................................................................... 237
The Client Access Network................................................................................................................... 238
The Private Network ............................................................................................................................. 239
About the Configuration Process ...................................................................................240
Configuring Exadata.......................................................................................................241
Step 1: The Configuration Worksheet................................................................................................... 242
Step 2: The DBM Configurator .............................................................................................................. 253
Step 3: Upload Parameter and Deployment Files ................................................................................. 254
Step 4: CheckIP (RWTRZX_ bW) ............................................................................................................ 258
Step 5: Firstboot ................................................................................................................................... 260
Step 6: Staging the Installation Media.................................................................................................. 262
Step 7: Running OneCommand............................................................................................................. 263
Upgrading Exadata.........................................................................................................268
Creating a New RAC Cluster ................................................................................................................. 268
Upgrading the Existing Cluster ............................................................................................................. 270
X
#/.4%.43
Summary .......................................................................................................................273
Chapter 9: Recovering Exadata...........................................................................275
Exadata Diagnostic Tools...............................................................................................275
Sun Diagnostics: sundiag.sh ................................................................................................................ 276
HealthCheck ......................................................................................................................................... 278
CellCLI................................................................................................................................................... 279
Backing Up Exadata.......................................................................................................284
Backing Up the Database Servers ........................................................................................................ 284
Backing Up the Storage Cell ................................................................................................................. 287
Backing Up the Database...............................................................................................293
Disk-Based Backups............................................................................................................................. 293
Tape-Based Backups............................................................................................................................ 293
Backup from Standby Database ........................................................................................................... 294
Exadata Optimizations for RMAN.......................................................................................................... 295
Recovering Exadata .......................................................................................................296
Restoring the Database Server............................................................................................................. 296
Recovering the Storage Cell ................................................................................................................. 299
Summary .......................................................................................................................318
Chapter 10: Exadata Wait Events........................................................................319
Events Specific to Exadata ............................................................................................319
The “cell” Events.................................................................................................................................. 320
Plan Steps That Trigger Events ............................................................................................................ 321
Exadata Wait Events in the User I/O Class.....................................................................323
cell smart table scan ............................................................................................................................ 323
cell smart index scan ........................................................................................................................... 326
cell single block physical read ............................................................................................................. 328
cell multiblock physical read................................................................................................................ 330
XI
#/.4%.43
cell list of blocks physical read ............................................................................................................ 331
cell smart file creation.......................................................................................................................... 332
cell statistics gather ............................................................................................................................. 333
Exadata Wait Events in the System I/O Class ................................................................333
cell smart incremental backup............................................................................................................. 334
cell smart restore from backup ............................................................................................................ 335
Exadata Wait Events in the Other and Idle Classes .......................................................337
cell smart flash unkeep ........................................................................................................................ 337
Old Events......................................................................................................................338
direct path read .................................................................................................................................... 338
enq: KO—fast object checkpoint.......................................................................................................... 339
reliable message .................................................................................................................................. 340
Resource Manager Events .............................................................................................341
resmgr:cpu quantum ............................................................................................................................ 341
resmgr:pq queued ................................................................................................................................ 342
Summary .......................................................................................................................343
Chapter 11: Understanding Exadata Performance Metrics.................................345
Measuring Exadata’s Performance Metrics...................................................................345
Revisiting the Prerequisites for Exadata Smart Scans ..................................................346
Exadata Smart Scan Performance........................................................................................................ 346
Understanding Exadata Smart Scan Metrics and Performance Counters ........................................... 350
Exadata Dynamic Performance Counters ......................................................................350
When and How to Use Performance Counters...................................................................................... 351
The Meaning and Explanation of Exadata Performance Counters........................................................ 354
Exadata Performance Counter Reference ............................................................................................ 356
Understanding SQL Statement Performance .................................................................374
Summary .......................................................................................................................377
XII
#/.4%.43
Chapter 12: Monitoring Exadata Performance....................................................379
A Systematic Approach..................................................................................................379
Monitoring SQL Statement Response Time ...................................................................380
Monitoring SQL Statements with Real-Time SQL Monitoring Reports................................................. 381
Monitoring SQL Statements using V$SQL and V$SQLSTATS ................................................................ 393
Monitoring the Storage Cell Layer .................................................................................395
Accessing Cell Metrics in the Cell Layer Using CellCLI......................................................................... 395
Accessing Cell Metrics Using the Grid Control Exadata Storage Server Plug-In ................................. 396
Which Cell Metrics to Use?................................................................................................................... 405
Monitoring Exadata Storage Cell OS-level Metrics............................................................................... 406
Summary .......................................................................................................................418
Chapter 13: Migrating to Exadata.......................................................................419
Migration Strategies ......................................................................................................420
Logical Migration ...........................................................................................................421
Extract and Load................................................................................................................................... 422
Copying Data over a Database Link...................................................................................................... 427
Replication-Based Migration ................................................................................................................ 443
Logical Migration Wrap Up ................................................................................................................... 451
Physical Migration .........................................................................................................451
Backup and Restore.......................................................................................................452
Full Backup and Restore....................................................................................................................... 452
Incremental Backup.............................................................................................................................. 454
Transportable Tablespaces (and XTTS) ................................................................................................ 455
Physical Standby .................................................................................................................................. 458
ASM Rebalance .................................................................................................................................... 460
Post-Migration Tasks............................................................................................................................ 463
Wrap Up Physical Migration Section..................................................................................................... 464
XIII
#/.4%.43
Summary .......................................................................................................................465
Chapter 14: Storage Layout ................................................................................467
Exadata Disk Architecture .............................................................................................467
Failure Groups ...................................................................................................................................... 469
Grid Disks ............................................................................................................................................. 471
Storage Allocation ................................................................................................................................ 473
Creating Grid Disks ........................................................................................................477
Creating Grid Disks............................................................................................................................... 478
Grid Disk Sizing .................................................................................................................................... 479
Creating FlashDisk-Based Grid Disks ................................................................................................... 483
Storage Strategies .........................................................................................................484
Configuration Options ........................................................................................................................... 485
Isolating Storage Cell Access ............................................................................................................... 485
Cell Security...................................................................................................................487
Cell Security Terminology..................................................................................................................... 488
Cell Security Best Practices.................................................................................................................. 489
Configuring ASM-Scoped Security ....................................................................................................... 489
Configuring Database-Scoped Security................................................................................................ 490
Removing Cell Security......................................................................................................................... 493
Summary .......................................................................................................................495
Chapter 15: Compute Node Layout .....................................................................497
Provisioning Considerations ..........................................................................................497
Non-RAC Configuration..................................................................................................500
RAC Clusters ..................................................................................................................504
Typical Exadata Configuration .......................................................................................506
Exadata Clusters............................................................................................................507
Summary .......................................................................................................................509
XIV
#/.4%.43
Chapter 16: Unlearning Some Things We Thought We Knew .............................511
A Tale of Two Systems ..................................................................................................511
OLTP-Oriented Workloads..............................................................................................512
Exadata Smart Flash Cache (ESFC) ...................................................................................................... 512
Scalability ............................................................................................................................................. 513
Write-Intensive OLTP Workloads .......................................................................................................... 513
DW-Oriented Workloads ................................................................................................514
Enabling Smart Scans .......................................................................................................................... 514
Things that Can Cripple Smart Scans................................................................................................... 516
Other Things to Keep in Mind ............................................................................................................... 526
Mixed Workloads ...........................................................................................................528
To Index or Not to Index?...................................................................................................................... 529
The Optimizer Doesn’t Know ................................................................................................................ 530
Using Resource Manager ..................................................................................................................... 533
Summary .......................................................................................................................534
Appendix A: CellCLI and dcli ...............................................................................535
CellCLI Command Syntax...............................................................................................535
Getting Familiar with CellCLI ................................................................................................................ 536
Sending Commands from the Operating System ................................................................................. 540
Configuring and Managing the Storage Cell......................................................................................... 540
dcli Command Syntax ....................................................................................................542
Summary .......................................................................................................................544
Appendix B: Online Exadata Resources ..............................................................545
Exadata MOS Support Notes..........................................................................................545
Helpful Exadata MOS Support Notes .............................................................................545
Exadata Bulletins and Blogs ..........................................................................................546
Appendix C: Diagnostic Scripts...........................................................................547
Index ...................................................................................................................551
XV
About the Author
Kerry Osborne BEGAN WORKING WITH /RACLE VERSION IN (E HAS
WORKED AS BOTH A DEVELOPER AND A $"! &OR THE PAST SEVERAL YEARS HE HAS
BEEN FOCUSED ON UNDERSTANDING /RACLE INTERNALS AND SOLVING PERFORMANCE
PROBLEMS (E IS AN /AK4ABLE MEMBER AND AN /RACLE !CE $IRECTOR +ERRY IS A
FREQUENT SPEAKER AT /RACLE CONFERENCES (E IS ALSO A CO
FOUNDER OF %NKITEC
AN /RACLE
FOCUSED CONSULTING COMPANY HEADQUARTERED IN $ALLAS 4EXAS (E
BLOGS AT ZTaah^bQ^a]T ^aPR[T Vdh R^\
Randy Johnson IS A 0RINCIPAL #ONSULTANT AT %NKITEC A CONSULTING FIRM
SPECIALIZING IN /RACLE 2ANDY HAS OVER YEARS OF EXPERIENCE WITH /RACLE
BEGINNING WITH /RACLE IN THE EARLY S -UCH OF HIS CAREER HAS COMBINED
/RACLE $"! WORK WITH 5NIX ADMINISTRATION DUTIES /VER THE LAST FIVE YEARS HE
HAS FOCUSED ALMOST EXCLUSIVELY ON 2!# AND !3- (E IS ALSO AN AUTHORITY ON
/RACLE BACKUP AND RECOVERY VIA 2-!. HAVING WRITTEN A POPULAR UTILITY FOR
AUTOMATING 2-!. SCRIPTS CALLED $IXIE 2ANDY OCCASIONALLY BLOGS AT
Q[^V T]ZXcTR R^\
Tanel Pöder IS ONE OF THE LEADING /RACLE PERFORMANCE SPECIALISTS IN THE
WORLD HAVING HELPED SOLVING COMPLEX PROBLEMS FOR CUSTOMERS IN OVER
COUNTRIES ON FIVE CONTINENTS (E SPECIALIZES IN ADVANCED PERFORMANCE TUNING
END
TO
END TROUBLESHOOTING AND OTHER COMPLEX AND THEREFORE INTERESTING
TASKS SUCH AS MIGRATING 6,$"S WITH VERY LOW DOWNTIME 4ANEL HAS OPTIMIZED
THE PERFORMANCE OF %XADATA INSTALLATIONS STARTING FROM %XADATA 6 AND HE
PLANS TO GO EVEN DEEPER WITH HIS CURRENT %XADATA PERFORMANCE AND
TROUBLESHOOTING RESEARCH
(E IS ONE OF THE FIRST /RACLE #ERTIFIED -ASTERS IN THE WORLD AN /RACLE !#%
$IRECTOR AND A PROUD MEMBER OF THE /AK4ABLE .ETWORK
4ANEL REGULARLY SPEAKS AT CONFERENCES WORLDWIDE AND PUBLISHES HIS ARTICLES SCRIPTS AND TOOLS IN HIS BLOG
AT Wcc_) Q[^V cP]T[_^STa R^\
XVI
About the Technical Reviewer
Kevin Closson IS A 4ECHNOLOGY $IRECTOR AND 0ERFORMANCE !RCHITECT IN THE
$ATA #OMPUTING $IVISION OF %-# &ROM THROUGH +EVIN WAS A
0ERFORMANCE !RCHITECT IN /RACLE #ORPORATIONS %XADATA DEVELOPMENT
ORGANIZATION WITHIN THE 3ERVER 4ECHNOLOGY 'ROUP (IS YEAR CAREER FOCUS HAS
BEEN SYSTEM AND SOFTWARE PERFORMANCE ENGINEERING AND ARCHITECTURE
COMPETITIVE BENCHMARKING SUPPORT AND APPLICATION DEVELOPMENT ON HIGH
END
3-0 AND #LUSTERED 5NIX AND ,INUX PLATFORMS (IS WORK PRIOR TO /RACLE AT
(00OLY3ERVE 6ERITAS AND )"-3EQUENT WAS FOCUSED ON SCALABILITY AND
AVAILABILITY ENHANCEMENTS OF THE /RACLE SERVER AND PLATFORMS THAT SUPPORT IT (IS
/RACLE PORT
LEVEL WORK LED TO HIS 53 PATENTS IN 3-0.5-! LOCKING AND
DATABASE CACHING METHODS +EVIN MAINTAINS A VERY POPULAR
TECHNOLOGY BLOG AT ZTeX]R[^bb^] f^aS_aTbb R^\
XVII
Acknowledgments
!S ALWAYS ITS THE PEOPLE CLOSEST TO YOU THAT PAY THE PRICE FOR YOUR INDISCRETIONS 3O )D LIKE TO TAKE THIS
OPPORTUNITY TO THANK MY FAMILY FOR PUTTING UP WITH ME WHILE ) STRUGGLED THROUGH THIS PROJECT )T WAS FAR
MORE WORK THAN ) EVER EXPECTED BUT WHAT REALLY SURPRISED ME WAS THAT THERE WAS A SENSE OF IMPENDING
DOOM HANGING OVER ME EVERY DAY FOR ALMOST A YEAR ) JOKED THAT EVERYTHING WAS MEASURED IN PARAGRAPHS
) COULD GO OUT TO DINNER WITH MY WIFE OR ) COULD WRITE ANOTHER TWO PARAGRAPHS !CTUALLY ) COULDNT WRITE
THEM THAT FAST BUT YOU KNOW WHAT ) MEAN 3O AGAIN )D LIKE TO THANK MY WIFE *ILL AND MY KIDS FOR PUTTING
UP WITH THE DISTANT STARES AND THE SELF IMPOSED RAIN CLOUD HANGING OVER ALL OF OUR HEADS
+ERRY /SBORNE
) WANT TO SAY THANK YOU TO MY KIDS "RANDON AND #HARIS WHO ENDURED THE MANY LONG NIGHTS AND
WEEKENDS ) SPENT AT THE KITCHEN TABLE WRITING THIS BOOK 4HEY ARE SUCH AN INSPIRATION TO ME -Y DAUGHTER
#HARIS A TRULY GIFTED WRITER IS SO CREATIVE AND BOLD -Y SON "RANDON THE BRILLIANT SCIENTIST AND STRATEGIC
THINKER OF THE FAMILY NEVER GIVES UP 7ATCHING THEM GROW INTO YOUNG ADULTS HAS TAUGHT ME THE VALUE OF
CREATIVITY AND PERSEVERANCE ESPECIALLY WHEN THE TASK SEEMS INSURMOUNTABLE ) LOVE YOU KIDS
4O MY PARENTS FAMILY AND FRIENDS )D LIKE TO SAY THANK YOU FOR ALL YOUR ENCOURAGEMENT AND
SUPPORT 9OU WERE ALWAYS THERE FOR ME WHEN ) NEEDED YOU ) WONT TRY TO NAME EVERYONE HERE BUT YOU
KNOW WHO YOU ARE
!ND FINALLY )D LIKE TO SAY A SPECIAL THANKS TO +ERRY FOR INVITING ME ON THIS JOURNEY WITH HIM !
FEW YEARS AGO IF YOUD TOLD ME ) WAS GOING TO WRITE A BOOK SOMEDAY )D HAVE LAUGHED OUT LOUD "UT +ERRY
HAS A TALENT FOR NUDGING ME OUT OF MY COMFORT ZONE )N ADDITION TO BEING A MENTOR OVER THE YEARS HE HAS
BECOME A TRUSTED FRIEND
2ANDY *OHNSON
4HIS IS THE FIRST BOOK )gVE CO
AUTHORED SO THIS ACKNOWLEDGMENT IS GOING TO BE A THANK
YOU NOTE TO ANYONE
WHO HAS EVER HELPED ME WITH MY CAREER AND SUPPORTED MY PASSION FOR SOLVING PROBLEMS WITH DATA
ALGORITHMS AND COMPUTERS )F YOU HAVE EVER TAUGHT ME HELPED ME OR GIVEN ME ADVICE THANK YOU )gD
LIKE TO GIVE A SPECIAL THANKS TO MY PARENTS WHO STRONGLY SUPPORTED MY INTEREST IN TECHNOLOGY
ELECTRONICS AND COMPUTERS WHEN ) WAS A KID !DDITIONALLY A BIG THANKS TO MY WIFE *ANIKA WHO DESPITE
NOT BEING A COMPUTER GEEK UNDERSTANDS THAT ITgS PERFECTLY NORMAL TO SOMETIMES STARE AT A COMPUTER
SCREEN FOR HOURS IN A ROW
4ANEL 0DER
XVIII
Introduction
4HANK YOU FOR PURCHASING THIS BOOK 7E WORKED HARD ON IT FOR A LONG TIME /UR HOPE IS THAT YOU FIND IT
USEFUL AS YOU BEGIN TO WORK WITH %XADATA 7EVE TRIED TO INTRODUCE THE TOPICS IN A METHODICAL MANNER
AND MOVE FROM GENERALIZATIONS TO SPECIFIC TECHNICAL DETAILS 7HILE SOME OF THE MATERIAL PAINTS A VERY
BROAD PICTURE OF HOW %XADATA WORKS SOME IS VERY TECHNICAL IN NATURE AND YOU MAY FIND THAT HAVING
ACCESS TO AN %XADATA SYSTEM WHERE YOU CAN TRY SOME OF THE TECHNIQUES PRESENTED WILL MAKE IT EASIER TO
UNDERSTAND .OTE THAT WEVE USED MANY UNDOCUMENTED PARAMETERS AND FEATURES TO DEMONSTRATE HOW
VARIOUS PIECES OF THE SOFTWARE WORK $O NOT TAKE THIS AS A RECOMMENDED APPROACH FOR MANAGING A
PRODUCTION SYSTEM 2EMEMBER THAT WE HAVE HAD ACCESS TO A SYSTEM THAT WE COULD TEAR APART WITH LITTLE
WORRY ABOUT THE CONSEQUENCES THAT RESULTED FROM OUR ACTIONS 4HIS GAVE US A HUGE ADVANTAGE IN OUR
INVESTIGATIONS INTO HOW %XADATA WORKS )N ADDITION TO THIS PRIVILEGED ACCESS WE WERE PROVIDED A GREAT
DEAL OF SUPPORT FROM PEOPLE BOTH INSIDE AND OUTSIDE OF /RACLE FOR WHICH WE ARE EXTREMELY GRATEFUL
The Intended Audience
4HIS BOOK IS INTENDED FOR EXPERIENCED /RACLE PEOPLE 7E DO NOT ATTEMPT TO EXPLAIN HOW /RACLE WORKS
EXCEPT AS IT RELATES TO THE %XADATA PLATFORM 4HIS MEANS THAT WE HAVE MADE SOME ASSUMPTIONS ABOUT THE
READERS KNOWLEDGE 7E DO NOT ASSUME THAT YOU ARE AN EXPERT AT PERFORMANCE TUNING ON /RACLE BUT WE
DO EXPECT THAT YOU ARE PROFICIENT WITH 31, AND HAVE A GOOD UNDERSTANDING OF BASIC /RACLE ARCHITECTURE
How We Came to Write This Book
)N THE SPRING OF %NKITEC BOUGHT AN %XADATA 6 1UARTER 2ACK 7E PUT IT IN THE TINY COMPUTER ROOM AT
OUR OFFICE IN $ALLAS 7E DONT HAVE A RAISED FLOOR OR ANYTHING VERY FANCY BUT THE ROOM DOES HAVE ITS OWN
AIR CONDITIONING SYSTEM )T WAS ACTUALLY MORE DIFFICULT THAN YOU MIGHT THINK TO GET /RACLE TO LET US
PURCHASE ONE 4HEY HAD MANY CUSTOMERS THAT WANTED THEM AND THEY WERE UNDERSTANDABLY PROTECTIVE OF
THEIR NEW BABY 7E DIDNT HAVE A TOP NOTCH DATA CENTER TO PUT IT IN AND EVEN THE POWER REQUIREMENTS
HAD TO BE DEALT WITH BEFORE THEY WOULD DELIVER ONE TO US !T ANY RATE SHORTLY AFTER WE TOOK DELIVERY
THROUGH A SERIES OF CONVERSATIONS WITH *ONATHAN 'ENNICK 2ANDY AND ) AGREED TO WRITE THIS BOOK FOR
!PRESS 4HERE WAS NOT A WHOLE LOT OF DOCUMENTATION AVAILABLE AT THAT TIME AND SO WE FOUND OURSELVES
PESTERING ANYONE WE COULD FIND WHO KNEW ANYTHING ABOUT IT +EVIN #LOSSON AND $AN .ORRIS WERE BOTH
GRACIOUS ENOUGH TO ANSWER MANY OF OUR QUESTIONS AT THE (OTSOS 3YMPOSIUM IN THE SPRING OF +EVIN
CONTACTED ME SOME TIME LATER AND OFFERED TO BE THE OFFICIAL TECHNICAL REVIEWER 3O 2ANDY AND ) STRUGGLED
THROUGH THE SUMMER AND EARLY FALL ATTEMPTING TO LEARN EVERYTHING COULD
) RAN INTO 4ANEL AT /RACLE /PEN 7ORLD IN 3EPTEMBER AND WE TALKED ABOUT A CLIENT USING
%XADATA THAT HE HAD DONE SOME MIGRATION WORK FOR /NE THING LED TO ANOTHER AND EVENTUALLY HE AGREED
TO JOIN THE TEAM AS A CO AUTHOR !T /PEN 7ORLD /RACLE ANNOUNCED THE AVAILABILITY OF THE NEW 8 MODELS
SO WE HAD BARELY GOTTEN STARTED AND WE WERE ALREADY BEHIND ON THE TECHNOLOGY
XIX
).42/$5#4)/.
)N *ANUARY OF THE 8 PLATFORM WAS BEGINNING TO SHOW UP AT CUSTOMER SITES %NKITEC AGAIN
DECIDED TO INVEST IN THE TECHNOLOGY AND WE BECAME THE PROUD PARENTS OF AN 8
QUARTER RACK !CTUALLY
WE DECIDED TO UPGRADE OUR EXISTING 6 QUARTER RACK TO A HALF RACK WITH 8 COMPONENTS 4HIS SEEMED LIKE A
GOOD WAY TO LEARN ABOUT DOING UPGRADES AND TO SEE IF THERE WOULD BE ANY PROBLEMS MIXING COMPONENTS
FROM THE TWO VERSIONS THERE WERENT 4HIS BRINGS ME TO AN IMPORTANT POINT
A Moving Target
,IKE MOST NEW SOFTWARE %XADATA HAS EVOLVED RAPIDLY SINCE ITS INTRODUCTION IN LATE 4HE CHANGES
HAVE INCLUDED SIGNIFICANT NEW FUNCTIONALITY )N FACT ONE OF THE MOST DIFFICULT PARTS OF THIS PROJECT HAS
BEEN KEEPING UP WITH THE CHANGES 3EVERAL CHAPTERS UNDERWENT MULTIPLE REVISIONS BECAUSE OF CHANGES IN
BEHAVIOR INTRODUCED WHILE WE WERE WRITING THE MATERIAL 4HE LAST VERSION WE HAVE ATTEMPTED TO COVER IN
THIS BOOK IS DATABASE VERSION WITH BUNDLE PATCH AND RT[[bae VERSION .OTE THAT THERE
HAVE BEEN MANY PATCHES OVER THE LAST TWO YEARS AND THAT THERE ARE MANY POSSIBLE COMBINATIONS OF
DATABASE VERSION PATCH LEVEL AND RT[[bae VERSIONS 3O IF YOU ARE OBSERVING SOME DIFFERENT BEHAVIOR THAN
WE HAVE DOCUMENTED THIS IS A POTENTIAL CAUSE .EVERTHELESS WE WELCOME YOUR FEEDBACK AND WILL BE
HAPPY TO ADDRESS ANY INCONSISTENCIES THAT YOU FIND )N FACT THIS BOOK HAS BEEN AVAILABLE AS PART OF
!PRESSS !LPHA PROGRAM WHICH ALLOWS READERS TO DOWNLOAD EARLY DRAFTS OF THE MATERIAL 0ARTICIPANTS IN
THIS PROGRAM HAVE PROVIDED QUITE A BIT OF FEEDBACK DURING THE WRITING AND EDITING PROCESS 7E ARE VERY
THANKFUL FOR THAT FEEDBACK AND SOMEWHAT SURPRISED AT THE DETAILED INFORMATION MANY OF YOU PROVIDED
Thanks to the Unofficial Editors
7E HAVE HAD A GREAT DEAL OF SUPPORT FROM A NUMBER OF PEOPLE ON THIS PROJECT (AVING OUR OFFICIAL
TECHNICAL REVIEWER ACTUALLY WRITING BITS THAT WERE DESTINED TO END UP IN THE BOOK WAS A LITTLE WEIRD )N
SUCH A CASE WHO REVIEWS THE REVIEWERS WRITING &ORTUNATELY !RUP .ANDA VOLUNTEERED EARLY IN THE
PROJECT TO BE AN UNOFFICIAL EDITOR 3O IN ADDITION TO THE AUTHORS REVIEWING EACH OTHERS STUFF AND +EVIN
REVIEWING OUR CHAPTERS !RUP READ AND COMMENTED ON EVERYTHING INCLUDING +EVINS COMMENTS )N
ADDITION MANY OF THE /AK 4ABLE .ETWORK MEMBERS GAVE US FEEDBACK ON VARIOUS CHAPTERS THROUGHOUT
THE PROCESS -OST NOTABLY &RITS (OOGLAND AND 0ETER "ACH PROVIDED VALUABLE INPUT
7HEN THE BOOK WAS ADDED TO !PRESSS !LPHA 0ROGRAM WE GAINED A WHOLE NEW SET OF REVIEWERS
3EVERAL PEOPLE GAVE US FEEDBACK BASED ON THE EARLY VERSIONS OF CHAPTERS THAT WERE PUBLISHED IN THIS
FORMAT 4HANKS TO ALL OF YOU WHO ASKED US QUESTIONS AND HELPED US CLARIFY OUR THOUGHTS ON SPECIFIC
ISSUES )N PARTICULAR 4YLER -UTH AT /RACLE TOOK A VERY ACTIVE INTEREST IN THE PROJECT AND PROVIDED US WITH
VERY DETAILED FEEDBACK (E WAS ALSO INSTRUMENTAL IN HELPING TO CONNECT US WITH OTHER RESOURCES INSIDE
/RACLE SUCH AS 3UE ,EE WHO PROVIDED A VERY DETAILED REVIEW OF THE 2ESOURCE -ANAGEMENT CHAPTER
&INALLY )D LIKE TO THANK THE TECHNICAL TEAM AT %NKITEC 4HERE WERE MANY WHO HELPED US KEEP ON
TRACK AND HELPED PICK UP THE SLACK WHILE 2ANDY AND ) WERE WORKING ON THIS PROJECT INSTEAD OF DOING OUR
REAL JOBS 4HE LIST OF PEOPLE WHO HELPED IS PRETTY LONG SO ) WONT CALL EVERYONE BY NAME )F YOU WORK AT
%NKITEC AND YOU HAVE BEEN INVOLVED WITH THE %XADATA WORK OVER THE LAST COUPLE OF YEARS YOU HAVE
CONTRIBUTED TO THIS BOOK ) WOULD LIKE TO SPECIFICALLY THANK 4IM &OX WHO GENERATED A LOT OF THE GRAPHICS
FOR US IN SPITE OF THE FACT THAT HE HAD NUMEROUS OTHER IRONS IN THE FIRE INCLUDING HIS OWN BOOK PROJECT
7E ALSO OWE !NDY #OLVIN A VERY SPECIAL THANKS AS A MAJOR CONTRIBUTOR TO THE PROJECT (E WAS
INSTRUMENTAL IN SEVERAL CAPACITIES &IRST HE WAS PRIMARILY RESPONSIBLE FOR MAINTAINING OUR TEST
ENVIRONMENT INCLUDING UPGRADING AND PATCHING THE PLATFORM SO THAT WE COULD TEST THE NEWEST FEATURES
AND CHANGES AS THEY BECAME AVAILABLE 3ECOND HE HELPED US HOLD DOWN THE FORT WITH OUR CUSTOMERS WHO
XX
).42/$5#4)/.
WERE IMPLEMENTING %XADATA WHILE 2ANDY AND ) WERE BUSY WRITING 4HIRD HE WAS INSTRUMENTAL IN HELPING
US FIGURE OUT HOW VARIOUS FEATURES WORKED PARTICULARLY WITH REGARD TO INSTALLATION CONFIGURATION AND
CONNECTIONS TO EXTERNAL SYSTEMS )T WOULD HAVE BEEN DIFFICULT TO COMPLETE THE PROJECT WITHOUT HIM
Who Wrote That?
4HERE ARE THREE AUTHORS OF THIS BOOK FOUR IF YOU COUNT +EVIN )T WAS REALLY A COLLABORATIVE EFFORT AMONG
THE FOUR OF US "UT IN ORDER TO DIVIDE THE WORK WE EACH AGREED TO DO A NUMBER OF CHAPTERS )NITIALLY 2ANDY
AND ) STARTED THE PROJECT AND 4ANEL JOINED A LITTLE LATER SO HE GOT A LIGHTER LOAD IN TERMS OF THE
ASSIGNMENTS BUT WAS A VERY VALUABLE PART OF TEAM HELPING WITH RESEARCH ON AREAS THAT WERE NOT
SPECIFICALLY ASSIGNED TO HIM 3O HERES HOW THE ASSIGNMENTS WORKED OUT
+ERRY #HAPTERS n
2ANDY #HAPTERS n n AND ABOUT HALF OF
4ANEL #HAPTERS n AND ABOUT HALF OF
+EVIN %ASILY IDENTIFIABLE IN THE h+EVIN 3AYSv SECTIONS
Online Resources
7E USED A NUMBER OF SCRIPTS IN THIS BOOK 7HEN THEY WERE SHORT OR WE FELT THE SCRIPTS THEMSELVES WERE OF
INTEREST WE INCLUDED THEIR CONTENTS IN THE TEXT 7HEN THEY WERE LONG OR JUST NOT VERY INTERESTING WE
SOMETIMES LEFT THE CONTENTS OF THE SCRIPTS OUT OF THE TEXT 9OU CAN FIND THE SOURCE CODE FOR ALL OF THE
SCRIPTS WE USED IN THE BOOK ONLINE AT fff 4g_Tac>aPR[T4gPSPcP R^\ !PPENDIX # ALSO CONTAINS A LISTING OF
ALL THE DIAGNOSTIC SCRIPTS ALONG WITH A BRIEF DESCRIPTION OF THEIR PURPOSE
A Note on “Kevin Says”
+EVIN #LOSSON SERVED AS OUR PRIMARY TECHNICAL REVIEWER FOR THE BOOK +EVIN WAS THE CHIEF PERFORMANCE
ARCHITECT AT /RACLE FOR THE 3!'% PROJECT WHICH EVENTUALLY TURNED INTO %XADATA SO HE IS EXTREMELY
KNOWLEDGEABLE NOT ONLY ABOUT HOW IT WORKS BUT ALSO ABOUT HOW IT SHOULD WORK AND WHY (IS DUTIES AS
TECHNICAL REVIEWER WERE TO REVIEW WHAT WE WROTE AND VERIFY IT FOR CORRECTNESS 4HE GENERAL WORKFLOW
CONSISTED OF ONE OF THE AUTHORS SUBMITTING A FIRST DRAFT OF A CHAPTER AND THEN +EVIN WOULD REVIEW IT AND
MARK IT UP WITH COMMENTS !S WE STARTED WORKING TOGETHER WE REALIZED THAT IT MIGHT BE A GOOD IDEA TO
ACTUALLY INCLUDE SOME OF +EVINS COMMENTS IN THE BOOK WHICH PROVIDES YOU WITH A SOMEWHAT UNIQUE
LOOK INTO THE PROCESS +EVIN HAS A UNIQUE WAY OF SAYING A LOT IN VERY FEW WORDS /VER THE COURSE OF THE
PROJECT ) FOUND MYSELF GOING BACK TO SHORT COMMENTS OR EMAILS MULTIPLE TIMES AND OFTEN FOUND THEM
MORE MEANINGFUL AFTER ) WAS MORE FAMILIAR WITH THE TOPIC 3O ) WOULD RECOMMEND THAT YOU DO THE SAME
2EAD HIS COMMENTS AS YOURE GOING THROUGH A CHAPTER BUT TRY TO COME BACK AND REREAD HIS COMMENTS
AFTER FINISHING THE CHAPTER ) THINK YOULL FIND THAT YOU WILL GET MORE OUT OF THEM ON THE SECOND PASS
How We Tested
7HEN WE BEGAN THE PROJECT THE CURRENT RELEASE OF THE DATABASE WAS 3O SEVERAL OF THE CHAPTERS
WERE INITIALLY TESTED WITH THAT VERSION OF THE DATABASE AND VARIOUS PATCH LEVELS ON THE STORAGE CELLS 7HEN
XXI
).42/$5#4)/.
BECAME AVAILABLE WE WENT BACK AND RETESTED 7HERE THERE WERE SIGNIFICANT DIFFERENCES WE TRIED
TO POINT THAT OUT BUT THERE ARE SOME SECTIONS THAT WERE NOT WRITTEN UNTIL AFTER WAS AVAILABLE 3O
ON THOSE TOPICS WE MAY NOT HAVE MENTIONED DIFFERENCES WITH BEHAVIOR 7E USED A COMBINATION
OF 6 AND 8 HARDWARE COMPONENTS FOR OUR TESTING 4HERE WAS BASICALLY NO DIFFERENCE OTHER THAN THE 8
BEING FASTER
Schemas and Tables
9OU WILL SEE A COUPLE OF DATABASE TABLES USED IN SEVERAL EXAMPLES THROUGHOUT THE BOOK 4ANEL USED A
TABLE CALLED 4 THAT LOOKS LIKE THIS
BHB/B0=31>G -/cPQ[TNbcPcb
>f]Ta)C0=4;
CPQ[T)C
=P\T=d[[.Ch_T
>F=4AE0A270A!"
=0<4E0A270A!"
CH?4E0A270A! !
;8=4=D<14A
C4GCE0A270A!#
A>F=D<=D<14A
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CPQ[TBcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
C01;4N=0<4)C
;0BCN0=0;HI43) 0?A ! ")!')$$
346A44)
?0AC8C8>=43)=>
=D<NA>FB)%!('$(((
2708=N2=C)
1;>2:B) '$!$$
4<?CHN1;>2:B)
0E6NB?024)
0E6NA>FN;4=) #
<>=8C>A8=6)H4B
B0<?;4NB8I4)%!('$(((
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2^[d\]BcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
=P\T0]P[hiTS=3E3T]bXch=d[[b1dRZTcbBP\_[T
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
>F=4A # ! ! #&% ( %!('$(((
=0<4 # ! $# & '$ %!('$(((
CH?4 # ! ( %!('$(((
;8=4 # ! !"$#' #! %!('$(((
XXII
).42/$5#4)/.
C4GC # ! " "%#' " %!('$(((
A>F=D< # ! %!('$(((
) USED SEVERAL VARIATIONS ON A TABLE CALLED 3+%7 4HE ONE ) USED MOST OFTEN IS 3+%7 AND IT
LOOKED LIKE THIS
BHB/B0=31>G -/cPQ[TNbcPcb
>f]Ta):B>
CPQ[T)B:4F"
=P\T=d[[.Ch_T
?:N2>;=D<14A
2>; =D<14A
2>;!E0A270A!"
2>;"30C4
2>;#E0A270A!
=D;;N2>;E0A270A!
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CPQ[TBcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
C01;4N=0<4)B:4F"
;0BCN0=0;HI43) 90= ! ()#()
346A44)
?0AC8C8>=43)=>
=D<NA>FB)"'# #'
2708=N2=C)
1;>2:B) ($'%$#
4<?CHN1;>2:B)
0E6NB?024)
0E6NA>FN;4=)""
<>=8C>A8=6)H4B
B0<?;4NB8I4)"'# #'
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2^[d\]BcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
=P\T0]P[hiTS=3E3T]bXch=d[[b1dRZTcbBP\_[T
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
?:N2>; ! " ( (''' ! "'# "%
2>; ! ( !'#' # "'# ##
2>;! ! ! $ ! "'# "%
2>;" ! $ ! ! "'# "%
2>;# ! " """""" ! "'# "%
=D;;N2>; ! "'"((( #( (((
XXIII
).42/$5#4)/.
4HIS DETAILED INFORMATION SHOULD NOT BE NECESSARY FOR UNDERSTANDING ANY OF OUR EXAMPLES BUT
IF YOU HAVE ANY QUESTIONS ABOUT THE TABLES THEY ARE HERE FOR YOUR REFERENCE !LSO BE AWARE THAT WE USED
OTHER TABLES AS WELL BUT THESE ARE THE ONES WE USED MOST OFTEN
Good Luck
7E HAVE HAD A BLAST DISCOVERING HOW %XADATA WORKS ) HOPE YOU ENJOY YOUR EXPLORATIONS AS MUCH AS WE
HAVE AND ) HOPE THIS BOOK PROVIDES A PLATFORM FROM WHICH YOU CAN BUILD YOUR OWN BODY OF KNOWLEDGE )
FEEL LIKE WE ARE JUST BEGINNING TO SCRATCH THE SURFACE OF THE POSSIBILITIES THAT HAVE BEEN OPENED UP BY
%XADATA 'OOD LUCK WITH YOUR INVESTIGATIONS AND PLEASE FEEL FREE TO ASK US QUESTIONS AND SHARE YOUR
DISCOVERIES WITH US AT fff 4g_Tac>aPR[T4gPSPcP R^\
XXIV
CHAPTER 1
What Is Exadata?
.O DOUBT YOU ALREADY HAVE A PRETTY GOOD IDEA WHAT %XADATA IS OR YOU WOULDNT BE HOLDING THIS BOOK IN
YOUR HANDS )N OUR VIEW IT IS A PRECONFIGURED COMBINATION OF HARDWARE AND SOFTWARE THAT PROVIDES A
PLATFORM FOR RUNNING /RACLE $ATABASE VERSION G 2ELEASE AS OF THIS WRITING 3INCE THE %XADATA
$ATABASE -ACHINE INCLUDES A STORAGE SUBSYSTEM NEW SOFTWARE HAS BEEN DEVELOPED TO RUN AT THE STORAGE
LAYER 4HIS HAS ALLOWED THE DEVELOPERS TO DO SOME THINGS THAT ARE JUST NOT POSSIBLE ON OTHER PLATFORMS )N
FACT %XADATA REALLY BEGAN ITS LIFE AS A STORAGE SYSTEM )F YOU TALK TO PEOPLE INVOLVED IN THE DEVELOPMENT OF
THE PRODUCT YOU WILL COMMONLY HEAR THEM REFER THE STORAGE COMPONENT AS %XADATA OR 3!'% 3TORAGE
!PPLIANCE FOR 'RID %NVIRONMENTS WHICH WAS THE CODE NAME FOR THE PROJECT
%XADATA WAS ORIGINALLY DESIGNED TO ADDRESS THE MOST COMMON BOTTLENECK WITH VERY LARGE DATABASES
THE INABILITY TO MOVE SUFFICIENTLY LARGE VOLUMES OF DATA FROM THE DISK STORAGE SYSTEM TO THE DATABASE
SERVERS /RACLE HAS BUILT ITS BUSINESS BY PROVIDING VERY FAST ACCESS TO DATA PRIMARILY THROUGH THE USE OF
INTELLIGENT CACHING TECHNOLOGY !S THE SIZES OF DATABASES BEGAN TO OUTSTRIP THE ABILITY TO CACHE DATA
EFFECTIVELY USING THESE TECHNIQUES /RACLE BEGAN TO LOOK AT WAYS TO ELIMINATE THE BOTTLENECK BETWEEN THE
STORAGE TIER AND THE DATABASE TIER 4HE SOLUTION THEY CAME UP WITH WAS A COMBINATION OF HARDWARE AND
SOFTWARE )F YOU THINK ABOUT IT THERE ARE TWO APPROACHES TO MINIMIZING THIS BOTTLENECK 4HE FIRST IS TO
MAKE THE PIPE BIGGER 7HILE THERE ARE MANY COMPONENTS INVOLVED AND ITS A BIT OF AN OVERSIMPLIFICATION
YOU CAN THINK OF )NFINI"AND AS THAT BIGGER PIPE 4HE SECOND WAY TO MINIMIZE THE BOTTLENECK IS TO REDUCE
THE AMOUNT OF DATA THAT NEEDS TO BE TRANSFERRED 4HIS THEY DID WITH 3MART 3CANS 4HE COMBINATION OF THE
TWO HAS PROVIDED A VERY SUCCESSFUL SOLUTION TO THE PROBLEM "UT MAKE NO MISTAKE REDUCING THE VOLUME
OF DATA FLOWING BETWEEN THE TIERS VIA 3MART 3CAN IS THE GOLDEN GOOSE
+EVIN 3AYS 4HE AUTHORS HAVE PROVIDED AN ACCURATE LIST OF APPROACHES FOR ALLEVIATING THE HISTORICAL BOTTLENECK
BETWEEN STORAGE AND #05 FOR $7") WORKLOADSIF THAT IS THE UNDERLYING MANDATE IS TO CHANGE AS LITTLE IN THE
CORE /RACLE $ATABASE KERNEL AS POSSIBLE &ROM A PURE COMPUTER SCIENCE PERSPECTIVE THE LIST OF SOLUTIONS TO THE
GENERIC PROBLEM OF DATA FLOW BETWEEN STORAGE AND #05 INCLUDES OPTIONS SUCH AS CO
LOCATING THE DATA WITH THE
DATABASE INSTANCETHE hSHARED
NOTHINGv -00 APPROACH 7HILE IT IS WORTHWHILE TO POINT THIS OUT THE AUTHORS ARE
RIGHT NOT TO SPEND TIME DISCUSSING THE OPTIONS DISMISSED BY /RACLE
)N THIS INTRODUCTORY CHAPTER WELL REVIEW THE COMPONENTS THAT MAKE UP %XADATA BOTH HARDWARE AND
SOFTWARE 7ELL ALSO DISCUSS HOW THE PARTS FIT TOGETHER THE ARCHITECTURE 7ELL TALK ABOUT HOW THE
DATABASE SERVERS TALK TO THE STORAGE SERVERS 4HIS IS HANDLED VERY DIFFERENTLY THAN ON OTHER PLATFORMS SO
WELL SPEND A FAIR AMOUNT OF TIME COVERING THAT TOPIC 7ELL ALSO PROVIDE SOME HISTORICAL CONTEXT "Y THE
#(!04%2 7(!4 )3 %8!$!4!
END OF THE CHAPTER YOU SHOULD HAVE A PRETTY GOOD FEEL FOR HOW ALL THE PIECES FIT TOGETHER AND A BASIC
UNDERSTANDING OF HOW %XADATA WORKS 4HE REST OF THE BOOK WILL PROVIDE THE DETAILS TO FILL OUT THE SKELETON
THAT IS BUILT IN THIS CHAPTER
+EVIN 3AYS )N MY OPINION $ATA 7AREHOUSING "USINESS )NTELLIGENCE PRACTITIONERS IN AN /RACLE ENVIRONMENT
WHO ARE INTERESTED IN %XADATA MUST UNDERSTAND #ELL /FFLOAD 0ROCESSING FUNDAMENTALS before ANY OTHER ASPECT OF
THE %XADATA $ATABASE -ACHINE !LL OTHER TECHNOLOGY ASPECTS OF %XADATA ARE MERELY ENABLING TECHNOLOGY IN SUPPORT
OF #ELL /FFLOAD 0ROCESSING &OR EXAMPLE TAKING TOO MUCH INTEREST TOO EARLY IN %XADATA )NFINI"AND COMPONENTRY IS
SIMPLY NOT THE BEST WAY TO BUILD A STRONG UNDERSTANDING OF THE TECHNOLOGY 0UT ANOTHER WAY THIS IS ONE OF THE RARE
CASES WHERE IT IS BETTER TO FIRST APPRECIATE THE WHOLE CAKE BEFORE SCRUTINIZING THE INGREDIENTS 7HEN ) EDUCATE ON
THE TOPIC OF %XADATA ) START WITH THE TOPIC OF #ELL /FFLOAD 0ROCESSING )N DOING SO ) QUICKLY IMPART THE FOLLOWING FOUR
FUNDAMENTALS
#ELL /FFLOAD 0ROCESSING 7ORK PERFORMED BY THE STORAGE SERVERS THAT WOULD OTHERWISE HAVE TO BE EXECUTED IN THE
DATABASE GRID )T INCLUDES FUNCTIONALITY LIKE 3MART 3CAN DATA FILE INITIALIZATION 2-!. OFFLOAD AND (YBRID #OLUMNAR
#OMPRESSION (## DECOMPRESSION IN THE CASE WHERE )N
-EMORY 0ARALLEL 1UERY IS NOT INVOLVED
3MART 3CAN 4HE MOST RELEVANT #ELL /FFLOAD 0ROCESSING FOR IMPROVING $ATA 7AREHOUSE "USINESS )NTELLIGENCE
QUERY PERFORMANCE 3MART 3CAN IS THE AGENT FOR OFFLOADING FILTRATION PROJECTION 3TORAGE )NDEX EXPLOITATION AND
(## DECOMPRESSION
&ULL 3CAN OR )NDEX &AST &ULL 3CAN 4HE REQUIRED ACCESS METHOD CHOSEN BY THE QUERY OPTIMIZER IN ORDER TO TRIGGER
A 3MART 3CAN
$IRECT 0ATH 2EADS 2EQUIRED BUFFERING MODEL FOR A 3MART 3CAN 4HE FLOW OF DATA FROM A 3MART 3CAN CANNOT BE
BUFFERED IN THE 3'! BUFFER POOL $IRECT PATH READS CAN BE PERFORMED FOR BOTH SERIAL AND PARALLEL QUERIES $IRECT PATH
READS ARE BUFFERED IN PROCESS 0'! HEAP
An Overview of Exadata
! PICTURES WORTH A THOUSAND WORDS OR SO THE SAYING GOES &IGURE
SHOWS A VERY HIGH
LEVEL VIEW OF THE
PARTS THAT MAKE UP THE %XADATA $ATABASE -ACHINE
#(!04%2 7(!4 )3 %8!$!4!
Exadata Database Servers Exadata Storage Servers
Infiniband
11gR2 / ASM
iDB / RDS
cellsrv
Figure 1-1. High-level Exadata components
7HEN CONSIDERING %XADATA IT IS HELPFUL TO DIVIDE THE ENTIRE SYSTEM MENTALLY INTO TWO PARTS THE
STORAGE LAYER AND THE DATABASE LAYER 4HE LAYERS ARE CONNECTED VIA AN )NFINI"AND NETWORK )NFINI"AND
PROVIDES A LOW LATENCY HIGH THROUGHPUT SWITCHED FABRIC COMMUNICATIONS LINK )T PROVIDES REDUNDANCY
AND BONDING OF LINKS 4HE DATABASE LAYER IS MADE UP OF MULTIPLE 3UN SERVERS RUNNING STANDARD /RACLE
G2 SOFTWARE 4HE SERVERS ARE GENERALLY CONFIGURED IN ONE OR MORE 2!# CLUSTERS ALTHOUGH 2!# IS NOT
ACTUALLY REQUIRED 4HE DATABASE SERVERS USE !3- TO MAP THE STORAGE !3- IS REQUIRED EVEN IF THE
DATABASES ARE NOT CONFIGURED TO USE 2!# 4HE STORAGE LAYER ALSO CONSISTS OF MULTIPLE 3UN SERVERS %ACH
STORAGE SERVER CONTAINS DISK DRIVES AND RUNS THE /RACLE STORAGE SERVER SOFTWARE cellsrv
#OMMUNICATION BETWEEN THE LAYERS IS ACCOMPLISHED VIA I$" WHICH IS A NETWORK BASED PROTOCOL THAT IS
IMPLEMENTED USING )NFINI"AND I$" IS USED TO SEND REQUESTS FOR DATA ALONG WITH METADATA ABOUT THE
REQUEST INCLUDING PREDICATES TO cellsrv )N CERTAIN SITUATIONS cellsrv IS ABLE TO USE THE METADATA TO
PROCESS THE DATA BEFORE SENDING RESULTS BACK TO THE DATABASE LAYER 7HEN cellsrv IS ABLE TO DO THIS IT IS
CALLED A 3MART 3CAN AND GENERALLY RESULTS IN A SIGNIFICANT DECREASE IN THE VOLUME OF DATA THAT NEEDS TO BE
TRANSMITTED BACK TO THE DATABASE LAYER 7HEN 3MART 3CANS ARE NOT POSSIBLE cellsrv RETURNS THE ENTIRE
/RACLE BLOCKS .OTE THAT I$" USES THE 2$3 PROTOCOL WHICH IS A LOW LATENCY PROTOCOL THAT BYPASSES
KERNEL CALLS BY USING REMOTE DIRECT MEMORY ACCESS 2$-! TO ACCOMPLISH PROCESS TO PROCESS
COMMUNICATION ACROSS THE )NFINI"AND NETWORK
History of Exadata
%XADATA HAS UNDERGONE A NUMBER OF SIGNIFICANT CHANGES SINCE ITS INITIAL RELEASE IN LATE )N FACT ONE
OF THE MORE DIFFICULT PARTS OF WRITING THIS BOOK HAS BEEN KEEPING UP WITH THE CHANGES IN THE PLATFORM
DURING THE PROJECT (ERES A BRIEF REVIEW OF THE PRODUCTS LINEAGE AND HOW IT HAS CHANGED OVER TIME
#(!04%2 7(!4 )3 %8!$!4!
+EVIN 3AYS )D LIKE TO SHARE SOME HISTORICAL PERSPECTIVE "EFORE THERE WAS %XADATA THERE WAS 3!'%3TORAGE
!PPLIANCE FOR 'RID %NVIRONMENTS WHICH WE MIGHT CONSIDER 6 )N FACT IT REMAINED 3!'% UNTIL JUST A MATTER OF
WEEKS BEFORE ,ARRY %LLISON GAVE IT THE NAME %XADATAJUST IN TIME FOR THE /PEN 7ORLD LAUNCH OF THE PRODUCT IN
AMID HUGE CO
BRANDED FANFARE WITH (EWLETT
0ACKARD !LTHOUGH THE FIRST EMBODIMENT OF 3!'% WAS A (EWLETT
0ACKARD EXCLUSIVE /RACLE HAD NOT YET DECIDED THAT THE PLATFORM WOULD BE EXCLUSIVE TO (EWLETT
0ACKARD MUCH LESS
THE EVENTUAL TOTAL EXCLUSIVITY ENJOYED BY 3UN -ICROSYSTEMSBY WAY OF BEING ACQUIRED BY /RACLE )N FACT /RACLE
LEADERSHIP HADNT EVEN ESTABLISHED THE RIGID ,INUX /PERATING 3YSTEM REQUIREMENT FOR THE DATABASE HOSTS THE
PORTING EFFORT OF I$" TO (0
58 )TANIUM WAS IN VERY LATE STAGES OF DEVELOPMENT BEFORE THE 3UN ACQUISITION WAS
FINALIZED "UT 3!'% EVOLUTION WENT BACK FURTHER THAN THAT
V1: 4HE FIRST %XADATA WAS RELEASED IN LATE )T WAS LABELED AS 6 AND WAS A
COMBINATION OF (0 HARDWARE AND /RACLE SOFTWARE 4HE ARCHITECTURE WAS SIMILAR
TO THE CURRENT 8
VERSION WITH THE EXCEPTION OF THE &LASH #ACHE WHICH WAS
ADDED TO THE 6 VERSION %XADATA 6 WAS MARKETED AS EXCLUSIVELY A DATA
WAREHOUSE PLATFORM 4HE PRODUCT WAS INTERESTING BUT NOT WIDELY ADOPTED )T
ALSO SUFFERED FROM ISSUES RESULTING FROM OVERHEATING 4HE COMMONLY HEARD
DESCRIPTION WAS THAT YOU COULD FRY EGGS ON TOP OF THE CABINET -ANY OF THE
ORIGINAL 6 CUSTOMERS REPLACED THEIR 6S WITH 6S
V2: 4HE SECOND VERSION OF %XADATA WAS ANNOUNCED AT /PEN 7ORLD IN 4HIS
VERSION WAS A PARTNERSHIP BETWEEN 3UN AND /RACLE "Y THE TIME THE
ANNOUNCEMENT WAS MADE /RACLE WAS ALREADY IN THE PROCESS OF ATTEMPTING TO
ACQUIRE 3UN -ICROSYSTEMS -ANY OF THE COMPONENTS WERE UPGRADED TO BIGGER
OR FASTER VERSIONS BUT THE BIGGEST DIFFERENCE WAS THE ADDITION OF A SIGNIFICANT
AMOUNT OF SOLID
STATE BASED STORAGE 4HE STORAGE CELLS WERE ENHANCED WITH '
OF %XADATA 3MART &LASH #ACHE 4HE SOFTWARE WAS ALSO ENHANCED TO TAKE
ADVANTAGE OF THE NEW CACHE 4HIS ADDITION ALLOWED /RACLE TO MARKET THE
PLATFORM AS MORE THAN A $ATA 7AREHOUSE PLATFORM OPENING UP A SIGNIFICANTLY
LARGER MARKET
X2: 4HE THIRD EDITION OF %XADATA ANNOUNCED AT /RACLE /PEN 7ORLD IN WAS
NAMED THE 8 !CTUALLY THERE ARE TWO DISTINCT VERSIONS OF THE 8 4HE 8
FOLLOWS THE SAME BASIC BLUEPRINT AS THE 6 WITH UP TO EIGHT DUAL
#05 DATABASE
SERVERS 4HE #05S WERE UPGRADED TO HEX
CORE MODELS WHERE THE 6S HAD USED
QUAD
CORE #05S 4HE OTHER 8 MODEL WAS NAMED THE 8
)T BREAKS THE SMALL
5 DATABASE SERVER MODEL BY INTRODUCING LARGER DATABASE SERVERS WITH ¾ CORE
#05S AND A LARGE 4" MEMORY FOOTPRINT 4HE 8
IS MARKETED AS A MORE ROBUST
PLATFORM FOR LARGE /,40 OR MIXED WORKLOAD SYSTEMS DUE PRIMARILY TO THE LARGER
NUMBER OF #05 CORES AND THE LARGER MEMORY FOOTPRINT
#(!04%2 7(!4 )3 %8!$!4!
Alternative Views of What Exadata Is
7EVE ALREADY GIVEN YOU A RATHER BLAND DESCRIPTION OF HOW WE VIEW %XADATA (OWEVER LIKE THE WELL
KNOWN TALE OF THE BLIND MEN DESCRIBING AN ELEPHANT THERE ARE MANY CONFLICTING PERCEPTIONS ABOUT THE
NATURE OF %XADATA 7ELL COVER A FEW OF THE COMMON DESCRIPTIONS IN THIS SECTION
$ATA 7AREHOUSE !PPLIANCE
/CCASIONALLY %XADATA IS DESCRIBED AS A data warehouse appliance (DW Appliance) 7HILE /RACLE HAS
ATTEMPTED TO KEEP %XADATA FROM BEING PIGEONHOLED INTO THIS CATEGORY THE DESCRIPTION IS CLOSER TO THE
TRUTH THAN YOU MIGHT INITIALLY THINK )T IS IN FACT A TIGHTLY INTEGRATED STACK OF HARDWARE AND SOFTWARE THAT
/RACLE EXPECTS YOU TO RUN WITHOUT A LOT OF CHANGES 4HIS IS DIRECTLY IN
LINE WITH THE COMMON
UNDERSTANDING OF A $7 !PPLIANCE (OWEVER THE VERY NATURE OF THE /RACLE DATABASE MEANS THAT IT IS
EXTREMELY CONFIGURABLE 4HIS FLIES IN THE FACE OF THE TYPICAL $7 !PPLIANCE WHICH TYPICALLY DOES NOT HAVE
A LOT OF KNOBS TO TURN (OWEVER THERE ARE SEVERAL COMMON CHARACTERISTICS THAT ARE SHARED BETWEEN $7
!PPLIANCES AND %XADATA
Exceptional Performance: 4HE MOST RECOGNIZABLE CHARACTERISTIC OF %XADATA AND
$7 !PPLIANCES IN GENERAL IS THAT THEY ARE OPTIMIZED FOR DATA WAREHOUSE TYPE
QUERIES
Fast Deployment: $7 !PPLIANCES AND %XADATA $ATABASE -ACHINES CAN BOTH BE
DEPLOYED VERY RAPIDLY 3INCE %XADATA COMES PRECONFIGURED IT CAN GENERALLY BE
UP AND RUNNING WITHIN A WEEK FROM THE TIME YOU TAKE DELIVERY 4HIS IS IN STARK
CONTRAST TO THE NORMAL /RACLE CLUSTERED DATABASE DEPLOYMENT SCENARIO WHICH
GENERALLY TAKES SEVERAL WEEKS
Scalability: "OTH PLATFORMS HAVE SCALABLE ARCHITECTURES 7ITH %XADATA
UPGRADING IS DONE IN DISCRETE STEPS 5PGRADING FROM A HALF RACK CONFIGURATION TO
A FULL RACK INCREASES THE TOTAL DISK THROUGHPUT IN LOCK STEP WITH THE COMPUTING
POWER AVAILABLE ON THE DATABASE SERVERS
Reduction in TCO: 4HIS ONE MAY SEEM A BIT STRANGE SINCE MANY PEOPLE THINK
THE BIGGEST DRAWBACK TO %XADATA IS THE HIGH PRICE TAG "UT THE FACT IS THAT BOTH
$7 !PPLIANCES AND %XADATA REDUCE THE OVERALL COST OF OWNERSHIP IN MANY
APPLICATIONS /DDLY ENOUGH IN %XADATAS CASE THIS IS PARTIALLY THANKS TO A
REDUCTION IN THE NUMBER OF /RACLE DATABASE LICENSES NECESSARY TO SUPPORT A
GIVEN WORKLOAD 7E HAVE SEEN SEVERAL SITUATIONS WHERE MULTIPLE HARDWARE
PLATFORMS WERE EVALUATED FOR RUNNING A COMPANYS /RACLE APPLICATION AND HAVE
ENDED UP COSTING LESS TO IMPLEMENT AND MAINTAIN ON %XADATA THAN ON THE OTHER
OPTIONS EVALUATED
High Availability: -OST $7 !PPLIANCES PROVIDE AN ARCHITECTURE THAT SUPPORTS AT
LEAST SOME DEGREE OF high availability (HA) 3INCE %XADATA RUNS STANDARD /RACLE
G SOFTWARE ALL THE (! CAPABILITIES THAT /RACLE HAS DEVELOPED ARE AVAILABLE OUT
OF THE BOX 4HE HARDWARE IS ALSO DESIGNED TO PREVENT ANY SINGLE POINT OF FAILURE
Preconfiguration: 7HEN %XADATA IS DELIVERED TO YOUR DATA CENTER A 3UN
ENGINEER WILL BE SCHEDULED TO ASSIST WITH THE INITIAL CONFIGURATION 4HIS WILL
INCLUDE ENSURING THAT THE ENTIRE RACK IS CABLED AND FUNCTIONING AS EXPECTED "UT
LIKE MOST $7 !PPLIANCES THE WORK HAS ALREADY BEEN DONE TO INTEGRATE THE
COMPONENTS 3O EXTENSIVE RESEARCH AND TESTING ARE NOT REQUIRED
#(!04%2 7(!4 )3 %8!$!4!
Limited Standard Configurations: -OST $7 !PPLIANCES ONLY COME IN A VERY
LIMITED SET OF CONFIGURATIONS SMALL MEDIUM AND LARGE FOR EXAMPLE %XADATA IS
NO DIFFERENT 4HERE ARE CURRENTLY ONLY FOUR POSSIBLE CONFIGURATIONS 4HIS HAS
REPERCUSSIONS WITH REGARDS TO SUPPORTABILITY )T MEANS IF YOU CALL SUPPORT AND
TELL THEM YOU HAVE AN 8 (ALF 2ACK THE SUPPORT PEOPLE WILL IMMEDIATELY
KNOW ALL THEY NEED TO KNOW ABOUT YOUR HARDWARE 4HIS PROVIDES BENEFITS TO THE
SUPPORT PERSONNEL AND THE CUSTOMERS IN TERMS OF HOW QUICKLY ISSUES CAN BE
RESOLVED
2EGARDLESS OF THE SIMILARITIES /RACLE DOES NOT CONSIDER %XADATA TO BE A $7 !PPLIANCE EVEN THOUGH
THERE ARE MANY SHARED CHARACTERISTICS 'ENERALLY SPEAKING THIS IS BECAUSE %XADATA PROVIDES A FULLY
FUNCTIONAL /RACLE DATABASE PLATFORM WITH ALL THE CAPABILITIES THAT HAVE BEEN BUILT INTO /RACLE OVER THE
YEARS INCLUDING THE ABILITY TO RUN ANY APPLICATION THAT CURRENTLY RUNS ON AN /RACLE DATABASE AND IN
PARTICULAR TO DEAL WITH MIXED WORKLOADS THAT DEMAND A HIGH DEGREE OF CONCURRENCY WHICH $7
!PPLIANCES ARE GENERALLY NOT EQUIPPED TO HANDLE
+EVIN 3AYS 7HETHER %XADATA IS OR IS NOT AN APPLIANCE IS A COMMON TOPIC OF CONFUSION WHEN PEOPLE ENVISION
WHAT %XADATA IS 4HE /RACLE %XADATA $ATABASE -ACHINE IS NOT AN APPLIANCE (OWEVER THE STORAGE GRID DOES
CONSIST OF %XADATA 3TORAGE 3ERVER CELLSWHICH ARE APPLIANCES
/,40 -ACHINE
4HIS DESCRIPTION IS A BIT OF A MARKETING PLOY AIMED AT BROADENING %XADATAS APPEAL TO A WIDER MARKET
SEGMENT 7HILE THE DESCRIPTION IS NOT TOTALLY OFF
BASE IT IS NOT AS ACCURATE AS SOME OTHER MONIKERS THAT
HAVE BEEN ASSIGNED TO %XADATA )T BRINGS TO MIND THE CLASSIC QUOTE
It depends on what the meaning of the word “is” is.
"ILL #LINTON
)N THE SAME VEIN /,40 /NLINE 4RANSACTION 0ROCESSING IS A BIT OF A LOOSELY DEFINED TERM 7E
TYPICALLY USE THE TERM TO DESCRIBE WORKLOADS THAT ARE VERY LATENCY SENSITIVE AND CHARACTERIZED BY SINGLE
BLOCK ACCESS VIA INDEXES "UT THERE IS A SUBSET OF /,40 SYSTEMS THAT ARE ALSO VERY WRITE INTENSIVE AND
DEMAND A VERY HIGH DEGREE OF CONCURRENCY TO SUPPORT A LARGE NUMBER OF USERS %XADATA WAS NOT DESIGNED
TO BE THE FASTEST POSSIBLE SOLUTION FOR THESE WRITE INTENSIVE WORKLOADS (OWEVER ITS WORTH NOTING THAT
VERY FEW SYSTEMS FALL NEATLY INTO THESE CATEGORIES -OST SYSTEMS HAVE A MIXTURE OF LONG RUNNING
THROUGHPUT SENSITIVE 31, STATEMENTS AND SHORT DURATION LATENCY SENSITIVE 31, STATEMENTS 7HICH LEADS
US TO THE NEXT VIEW OF %XADATA
#ONSOLIDATION 0LATFORM
4HIS DESCRIPTION PITCHES %XADATA AS A POTENTIAL PLATFORM FOR CONSOLIDATING MULTIPLE DATABASES 4HIS IS
DESIRABLE FROM A TOTAL COST OF OWNERSHIP 4#/ STANDPOINT AS IT HAS THE POTENTIAL TO REDUCE COMPLEXITY
AND THEREFORE COSTS ASSOCIATED WITH THAT COMPLEXITY REDUCE ADMINISTRATION COSTS BY DECREASING THE
NUMBER OF SYSTEMS THAT MUST BE MAINTAINED REDUCE POWER USAGE AND DATA CENTER COSTS THROUGH
#(!04%2 7(!4 )3 %8!$!4!
REDUCING THE NUMBER OF SERVERS AND REDUCE SOFTWARE AND MAINTENANCE FEES 4HIS IS A VALID WAY TO VIEW
%XADATA "ECAUSE OF THE COMBINATION OF FEATURES INCORPORATED IN %XADATA IT IS CAPABLE OF ADEQUATELY
SUPPORTING MULTIPLE WORKLOAD PROFILES AT THE SAME TIME !LTHOUGH IT IS NOT THE PERFECT /,40 -ACHINE THE
&LASH #ACHE FEATURE PROVIDES A MECHANISM FOR ENSURING LOW LATENCY FOR /,40
ORIENTED WORKLOADS 4HE
3MART 3CAN OPTIMIZATIONS PROVIDE EXCEPTIONAL PERFORMANCE FOR HIGH
THROUGHPUT $7
ORIENTED
WORKLOADS 2ESOURCE -ANAGEMENT OPTIONS BUILT INTO THE PLATFORM PROVIDE THE ABILITY FOR THESE SOMEWHAT
CONFLICTING REQUIREMENTS TO BE SATISFIED ON THE SAME PLATFORM )N FACT ONE OF THE BIGGEST UPSIDES TO THIS
ABILITY IS THE POSSIBILITY OF TOTALLY ELIMINATING A HUGE AMOUNT OF WORK THAT IS CURRENTLY PERFORMED IN MANY
SHOPS TO MOVE DATA FROM AN /,40 SYSTEM TO A $7 SYSTEM SO THAT LONG
RUNNING QUERIES DO NOT NEGATIVELY
AFFECT THE LATENCY
SENSITIVE WORKLOAD )N MANY SHOPS SIMPLY MOVING DATA FROM ONE PLATFORM TO ANOTHER
CONSUMES MORE RESOURCES THAN ANY OTHER OPERATION %XADATAS CAPABILITIES IN THIS REGARD MAY MAKE THIS
PROCESS UNNECESSARY IN MANY CASES
Configuration Options
3INCE %XADATA IS DELIVERED AS A PRECONFIGURED INTEGRATED SYSTEM THERE ARE VERY FEW OPTIONS AVAILABLE !S
OF THIS WRITING THERE ARE FOUR VERSIONS AVAILABLE 4HEY ARE GROUPED INTO TWO MAJOR CATEGORIES WITH
DIFFERENT MODEL NAMES THE 8
AND THE 8
4HE STORAGE TIERS AND NETWORKING COMPONENTS FOR THE
TWO MODELS ARE IDENTICAL 4HE DATABASE TIERS HOWEVER ARE DIFFERENT
%XADATA $ATABASE -ACHINE 8
4HE 8
COMES IN THREE FLAVORS QUARTER RACK HALF RACK AND FULL RACK 4HE SYSTEM IS BUILT TO BE
UPGRADEABLE SO YOU CAN UPGRADE LATER FROM A QUARTER RACK TO HALF RACK FOR EXAMPLE (ERE IS WHAT YOU
NEED TO KNOW ABOUT THE DIFFERENT OPTIONS
Quarter Rack 4HE 8
1UARTER 2ACK COMES WITH TWO DATABASE SERVERS AND
THREE STORAGE SERVERS 4HE HIGH
CAPACITY VERSION PROVIDES ROUGHLY 4" OF
USABLE DISK SPACE IF IT IS CONFIGURED FOR NORMAL REDUNDANCY 4HE HIGH
PERFORMANCE VERSION PROVIDES ROUGHLY ONE THIRD OF THAT OR ABOUT 4" OF USABLE
SPACE AGAIN IF CONFIGURED FOR NORMAL REDUNDANCY
Half Rack 4HE 8
(ALF 2ACK COMES WITH FOUR DATABASE SERVERS AND SEVEN
STORAGE SERVERS 4HE HIGH
CAPACITY VERSION PROVIDES ROUGHLY 4" OF USABLE DISK
SPACE IF IT IS CONFIGURED FOR NORMAL REDUNDANCY 4HE HIGH
PERFORMANCE VERSION
PROVIDES ROUGHLY 4" OF USABLE SPACE IF CONFIGURED FOR NORMAL REDUNDANCY
Full Rack 4HE 8
1UARTER 2ACK COMES WITH EIGHT DATABASE SERVERS AND
FOURTEEN STORAGE SERVERS 4HE HIGH
CAPACITY VERSION PROVIDES ROUGHLY 4" OF
USABLE DISK SPACE IF IT IS CONFIGURED FOR NORMAL REDUNDANCY 4HE HIGH
PERFORMANCE VERSION PROVIDES ABOUT 4" OF USABLE SPACE IF CONFIGURED FOR
NORMAL REDUNDANCY
#(!04%2 7(!4 )3 %8!$!4!
.OTE (ERES HOW WE CAM UP WITH THE ROUGH USEABLE SPACE ESTIMATES 7E TOOK THE ACTUAL SIZE OF THE DISK AND
SUBTRACTED '" FOR /3$"&3 SPACE !SSUMING THE ACTUAL DISK SIZES ARE '" AND '" FOR HIGH CAPACITY
(# AND HIGH PERFORMANCE (0 DRIVES THAT LEAVES '" FOR (# AND '" FOR (0 -ULTIPLY THAT BY THE NUMBER
OF DISKS IN THE RACK OR $IVIDE THAT NUMBER BY OR DEPENDING ON WHETHER YOU ARE USING NORMAL OR
HIGH REDUNDANCY TO GET USABLE SPACE +EEP IN MIND THAT THE USABLE FREE MB THAT ASMCMD REPORTS TAKES INTO
ACCOUNT THE SPACE NEEDED FOR A REBALANCE IF A FAILGROUP WAS LOST REQ?MIR?FREE?-" 5SABLE FILE SPACE FROM
ASMCMDgS LSDG IS CALCULATED AS FOLLOWS
&REE?-" REDUNDANCY REQ?MIR?FREE?-"
(ALF AND FULL RACKS ARE DESIGNED TO BE CONNECTED TO ADDITIONAL RACKS ENABLING MULTIPLE RACK
CONFIGURATIONS 4HESE CONFIGURATIONS HAVE AN ADDITIONAL )NFINI"AND SWITCH CALLED A spine switch )T IS
INTENDED TO BE USED TO CONNECT ADDITIONAL RACKS 4HERE ARE ENOUGH AVAILABLE CONNECTIONS TO CONNECT AS
MANY AS EIGHT RACKS ALTHOUGH ADDITIONAL CABLING MAY BE REQUIRED DEPENDING ON THE NUMBER OF RACKS YOU
INTEND TO CONNECT 4HE DATABASE SERVERS OF THE MULTIPLE RACKS CAN BE COMBINED INTO A SINGLE 2!#
DATABASE WITH DATABASE SERVERS THAT SPAN RACKS OR THEY MAY BE USED TO FORM SEVERAL SMALLER 2!#
CLUSTERS #HAPTER CONTAINS MORE INFORMATION ABOUT CONNECTING MULTIPLE RACKS
%XADATA $ATABASE -ACHINE 8
4HERE IS CURRENTLY ONLY ONE VERSION OF THE 8
)T HAS TWO DATABASE SERVERS AND FOURTEEN STORAGE CELLS )T
IS EFFECTIVELY AN 8
&ULL 2ACK BUT WITH TWO LARGE DATABASE SERVERS INSTEAD OF THE EIGHT SMALLER DATABASE
SERVERS USED IN THE 8
!S PREVIOUSLY MENTIONED THE STORAGE SERVERS AND NETWORKING COMPONENTS ARE
IDENTICAL TO THE 8
MODEL 4HERE ARE NO UPGRADES SPECIFIC TO X
AVAILABLE )F YOU NEED MORE CAPACITY
YOUR OPTION IS TO ADD ANOTHER 8
ALTHOUGH IT IS POSSIBLE TO ADD ADDITIONAL STORAGE CELLS
Upgrades
1UARTER RACKS AND HALF RACKS MAY BE UPGRADED TO ADD MORE CAPACITY 4HE CURRENT PRICE LIST HAS TWO
OPTIONS FOR UPGRADES THE (ALF 2ACK 4O &ULL 2ACK 5PGRADE AND THE 1UARTER 2ACK TO (ALF 2ACK 5PGRADE
4HE OPTIONS ARE LIMITED IN AN EFFORT TO MAINTAIN THE RELATIVE BALANCE BETWEEN DATABASE SERVERS AND
STORAGE SERVERS 4HESE UPGRADES ARE DONE IN THE FIELD )F YOU ORDER AN UPGRADE THE INDIVIDUAL
COMPONENTS WILL BE SHIPPED TO YOUR SITE ON A BIG PALLET AND A 3UN ENGINEER WILL BE SCHEDULED TO INSTALL
THE COMPONENTS INTO YOUR RACK !LL THE NECESSARY PARTS SHOULD BE THERE INCLUDING RACK RAILS AND CABLES
5NFORTUNATELY THE LABELS FOR THE CABLES SEEM TO COME FROM SOME OTHER PART OF THE UNIVERSE 7HEN WE DID
THE UPGRADE ON OUR LAB SYSTEM THE LACK OF LABELS HELD US UP FOR A COUPLE OF DAYS
4HE QUARTER
TO
HALF UPGRADE INCLUDES TWO DATABASE SERVERS AND FOUR STORAGE SERVERS ALONG WITH AN
ADDITIONAL )NFINI"AND SWITCH WHICH IS CONFIGURED AS A SPINE SWITCH 4HE HALF
TO
FULL UPGRADE INCLUDES
FOUR DATABASE SERVERS AND SEVEN STORAGE SERVERS 4HERE IS NO ADDITIONAL )NFINI"AND SWITCH REQUIRED
BECAUSE THE HALF RACK ALREADY INCLUDES A SPINE SWITCH
4HERE IS ALSO THE POSSIBILITY OF ADDING STANDALONE STORAGE SERVERS TO AN EXISTING RACK !LTHOUGH THIS
GOES AGAINST THE BALANCED CONFIGURATION PHILOSOPHY /RACLE DOES ALLOW IT /DDLY ENOUGH THEY DO NOT
#(!04%2 7(!4 )3 %8!$!4!
SUPPORT PLACING THE STORAGE SERVERS IN THE EXISTING RACK EVEN IF THERE IS SPACE AS IN THE CASE OF A QUARTER
RACK OR HALF RACK FOR EXAMPLE
4HERE ARE A COUPLE OF OTHER THINGS WORTH NOTING ABOUT UPGRADES -ANY COMPANIES PURCHASED
%XADATA 6 SYSTEMS AND ARE NOW IN THE PROCESS OF UPGRADING THOSE SYSTEMS 3EVERAL QUESTIONS NATURALLY
ARISE WITH REGARD TO THIS PROCESS /NE HAS TO DO WITH WHETHER IT IS ACCEPTABLE TO MIX THE NEWER 8
SERVERS WITH THE OLDER 6 COMPONENTS 4HE ANSWER IS YES ITS /+ TO MIX THEM )N OUR LAB ENVIRONMENT
FOR EXAMPLE WE HAVE A MIXTURE OF 6 OUR ORIGINAL QUARTER RACK AND 8
SERVERS THE UPGRADE TO A HALF
RACK 7E CHOSE TO UPGRADE OUR EXISTING SYSTEM TO A HALF RACK RATHER THAN PURCHASE ANOTHER STANDALONE
QUARTER RACK WITH 8
COMPONENTS WHICH WAS ANOTHER VIABLE OPTION
4HE OTHER QUESTION THAT COMES UP FREQUENTLY IS WHETHER ADDING ADDITIONAL STANDALONE STORAGE
SERVERS IS AN OPTION FOR COMPANIES THAT ARE RUNNING OUT OF SPACE BUT THAT HAVE PLENTY OF #05 CAPACITY ON
THE DATABASE SERVERS 4HIS QUESTION IS NOT AS EASY TO ANSWER &ROM A LICENSING STANDPOINT /RACLE WILL SELL
YOU ADDITIONAL STORAGE SERVERS BUT REMEMBER THAT ONE OF THE GOALS OF %XADATA WAS TO CREATE A MORE
BALANCED ARCHITECTURE 3O YOU SHOULD CAREFULLY CONSIDER WHETHER YOU NEED MORE PROCESSING CAPABILITY AT
THE DATABASE TIER TO HANDLE THE ADDITIONAL THROUGHPUT PROVIDED BY THE ADDITIONAL STORAGE (OWEVER IF ITS
SIMPLY LACK OF SPACE THAT YOU ARE DEALING WITH ADDITIONAL STORAGE SERVERS ARE CERTAINLY A VIABLE OPTION
Hardware Components
9OUVE PROBABLY SEEN MANY PICTURES LIKE THE ONE IN &IGURE
)T SHOWS AN %XADATA $ATABASE -ACHINE
&ULL 2ACK 7EVE ADDED A FEW GRAPHIC ELEMENTS TO SHOW YOU WHERE THE VARIOUS PIECES RESIDE IN THE
CABINET )N THIS SECTION WELL COVER THOSE PIECES
#(!04%2 7(!4 )3 %8!$!4!
Storage
Servers
Database
Servers
Infiniband Cisco Network
Leaf Switch, ILOM,
Switches and KVM
Database
Servers
Storage
Servers
Infiniband
Spine
Switch
Figure 1-2. An Exadata Full Rack
!S YOU CAN SEE MOST OF THE NETWORKING COMPONENTS INCLUDING AN %THERNET SWITCH AND TWO REDUNDANT
)NFINI"AND SWITCHES ARE LOCATED IN THE MIDDLE OF THE RACK 4HIS MAKES SENSE AS IT MAKES THE CABLING A
LITTLE SIMPLER 4HERE IS ALSO A 3UN )NTEGRATED ,IGHTS /UT -ANAGER ),/- MODULE AND +6- IN THE CENTER
SECTION 4HE SURROUNDING EIGHT SLOTS ARE RESERVED FOR DATABASE SERVERS AND THE REST OF THE RACK IS USED FOR
STORAGE SERVERS WITH ONE EXCEPTION 4HE VERY BOTTOM SLOT IS USED FOR AN ADDITIONAL )NFINI"AND hSPINEv
SWITCH THAT CAN BE USED TO CONNECT ADDITIONAL RACKS IF SO DESIRED )T IS LOCATED IN THE BOTTOM OF THE RACK
BASED ON THE EXPECTATION THAT YOUR %XADATA WILL BE IN A DATA CENTER WITH A RAISED FLOOR ALLOWING CABLING
TO BE RUN FROM THE BOTTOM OF THE RACK
/PERATING 3YSTEMS
4HE CURRENT GENERATION 8 HARDWARE CONFIGURATIONS USE )NTEL
BASED 3UN SERVERS !S OF THIS WRITING ALL THE
SERVERS COME PREINSTALLED WITH /RACLE ,INUX /RACLE HAS ANNOUNCED THAT THEY INTEND TO SUPPORT TWO
VERSIONS OF THE ,INUX KERNELTHE STANDARD 2EDHAT
COMPATIBLE VERSION AND AN ENHANCED VERSION CALLED
#(!04%2 7(!4 )3 %8!$!4!
THE 5NBREAKABLE %NTERPRISE +ERNEL 5%+ 4HIS OPTIMIZED VERSION HAS SEVERAL ENHANCEMENTS THAT ARE
SPECIFICALLY APPLICABLE TO %XADATA !MONG THESE ARE NETWORK
RELATED IMPROVEMENTS TO )NFINI"AND USING
THE 2$3 PROTOCOL /NE OF THE REASONS FOR RELEASING THE 5%+ MAY BE TO SPEED UP /RACLES ABILITY TO ROLL OUT
CHANGES TO ,INUX BY AVOIDING THE LENGTHY PROCESS NECESSARY TO GET CHANGES INTO THE STANDARD /PEN
3OURCE RELEASES /RACLE HAS BEEN A STRONG PARTNER IN THE DEVELOPMENT OF ,INUX AND HAS MADE SEVERAL
MAJOR CONTRIBUTIONS TO THE CODE BASE 4HE STATED DIRECTION IS TO SUBMIT ALL THE ENHANCEMENTS INCLUDED IN
THE %5+ VERSION FOR INCLUSION IN THE STANDARD RELEASE
/RACLE HAS ALSO ANNOUNCED THAT THE 8 DATABASE SERVERS WILL HAVE THE OPTION OF RUNNING 3OLARIS
%XPRESS !ND SPEAKING OF 3OLARIS WE ARE FREQUENTLY ASKED ABOUT WHETHER /RACLE HAS PLANS TO RELEASE A
VERSION OF %XADATA THAT USES 30!2# #05S !T THE TIME OF THIS WRITING THERE HAS BEEN NO INDICATION THAT
THIS WILL BE A FUTURE DIRECTION )T SEEMS MORE LIKELY THAT /RACLE WILL CONTINUE TO PURSUE THE 8
BASED
SOLUTION
3TORAGE SERVERS FOR BOTH THE 8
AND 8
MODELS WILL CONTINUE TO RUN EXCLUSIVELY ON /RACLE ,INUX
/RACLE VIEWS THESE SERVERS AS A CLOSED SYSTEM AND DOES NOT SUPPORT INSTALLING ANY ADDITIONAL SOFTWARE ON
THEM
$ATABASE 3ERVERS
4HE CURRENT GENERATION 8
DATABASE SERVERS ARE BASED ON THE 3UN &IRE 8 - SERVERS %ACH SERVER
HAS TWO ¾ #ORE )NTEL 8EON 8 PROCESSORS '(Z AND '" OF MEMORY 4HEY ALSO HAVE FOUR
INTERNAL '" + 20- 3!3 DRIVES 4HEY HAVE SEVERAL NETWORK CONNECTIONS INCLUDING TWO 'B AND
FOUR 'B %THERNET PORTS IN ADDITION TO THE TWO 1$2 )NFINI"AND 'BS PORTS .OTE THAT THE 'B PORTS
ARE OPEN AND THAT YOULL NEED TO PROVIDE THE CORRECT CONNECTORS TO ATTACH THEM TO YOUR EXISTING COPPER OR
FIBER NETWORK 4HE SERVERS ALSO HAVE A DEDICATED ),/- PORT AND DUAL HOT
SWAPPABLE POWER SUPPLIES
4HE 8
DATABASE SERVERS ARE BASED ON THE 3UN &IRE 8 SERVERS 4HEY ARE DESIGNED TO HANDLE
SYSTEMS THAT REQUIRE A LARGE AMOUNT OF MEMORY 4HE SERVERS ARE EQUIPPED WITH EIGHT X #ORE )NTEL 8EON
8 PROCESSORS '(Z AND 4" OF MEMORY 4HIS GIVES THE FULL RACK SYSTEM A TOTAL OF CORES AND
TERABYTES OF MEMORY
3TORAGE 3ERVERS
4HE CURRENT GENERATION OF STORAGE SERVERS ARE THE SAME FOR BOTH THE 8
AND THE 8
MODELS %ACH
STORAGE SERVER CONSISTS OF A 3UN &IRE 8 - AND CONTAINS DISKS $EPENDING ON WHETHER YOU HAVE
THE HIGH
CAPACITY VERSION OR THE HIGH
PERFORMANCE VERSION THE DISKS WILL EITHER BE 4" OR '" 3!3
DRIVES %ACH STORAGE SERVER COMES WITH '" OF MEMORY AND TWO X #ORE )NTEL 8EON 8 PROCESSORS
RUNNING AT '(Z 4HESE ARE THE SAME #05S AS ON THE 8
DATABASE SERVERS "ECAUSE THESE #05S ARE
IN THE 7ESTMERE FAMILY THEY HAVE BUILT IN !%3 ENCRYPTION SUPPORT WHICH ESSENTIALLY PROVIDES A
HARDWARE ASSIST TO ENCRYPTION AND DECRYPTION %ACH STORAGE SERVER ALSO CONTAINS FOUR '" 3UN &LASH
!CCELERATOR & 0#)E CARDS 4HIS PROVIDES A TOTAL OF '" OF FLASH BASED STORAGE ON EACH STORAGE CELL
4HE STORAGE SERVERS COME PRE
INSTALLED WITH /RACLE ,INUX
)NFINI"AND
/NE OF THE MORE IMPORTANT HARDWARE COMPONENTS OF %XADATA IS THE )NFINI"AND NETWORK )T IS USED FOR
TRANSFERRING DATA BETWEEN THE DATABASE TIER AND THE STORAGE TIER )T IS ALSO USED FOR INTERCONNECT TRAFFIC
BETWEEN THE DATABASE SERVERS IF THEY ARE CONFIGURED IN A 2!# CLUSTER )N ADDITION THE )NFINI"AND
NETWORK MAY BE USED TO CONNECT TO EXTERNAL SYSTEMS FOR SUCH USES AS BACKUPS %XADATA PROVIDES
REDUNDANT
PORT 1$2 )NFINI"AND SWITCHES FOR THESE PURPOSES 4HE SWITCHES PROVIDE 'B3EC OF
THROUGHPUT 9OU WILL OCCASIONALLY SEE THESE SWITCHES REFERRED TO AS hLEAFv SWITCHES )N ADDITION EACH
DATABASE SERVER AND EACH STORAGE SERVER ARE EQUIPPED WITH $UAL
0ORT 1$2 )NFINI"AND (OST #HANNEL
#(!04%2 7(!4 )3 %8!$!4!
!DAPTERS !LL BUT THE SMALLEST QUARTER RACK %XADATA CONFIGURATIONS ALSO CONTAIN A THIRD )NFINI"AND
SWITCH INTENDED FOR CHAINING MULTIPLE %XADATA RACKS TOGETHER 4HIS SWITCH IS GENERALLY REFERRED TO AS A
hSPINEv SWITCH
&LASH #ACHE
!S MENTIONED EARLIER EACH STORAGE SERVER COMES EQUIPPED WITH '" OF FLASH
BASED STORAGE 4HIS
STORAGE IS GENERALLY CONFIGURED TO BE A CACHE /RACLE REFERS TO IT AS %XADATA 3MART &LASH #ACHE %3&#
4HE PRIMARY PURPOSE OF %3&# IS TO MINIMIZE THE SERVICE TIME FOR SINGLE BLOCK READS 4HIS FEATURE PROVIDES
A SUBSTANTIAL AMOUNT OF DISK CACHE ABOUT 4" ON A HALF RACK CONFIGURATION
$ISKS
/RACLE PROVIDES TWO OPTIONS FOR DISKS !N %XADATA $ATABASE -ACHINE MAY BE CONFIGURED WITH EITHER
HIGH
CAPACITY DRIVES OR HIGH
PERFORMANCE DRIVES !S PREVIOUSLY MENTIONED THE HIGH
CAPACITY OPTION
INCLUDES 4" 20- DRIVES WHILE THE HIGH
PERFORMANCE OPTION INCLUDES '" 20- 3!3
DRIVES /RACLE DOES NOT ALLOW A MIXTURE OF THE TWO DRIVE TYPES 7ITH THE LARGE AMOUNT OF FLASH CACHE
AVAILABLE ON THE STORAGE CELLS IT SEEMS THAT THE HIGH
CAPACITY OPTION WOULD BE ADEQUATE FOR MOST READ
HEAVY WORKLOADS 4HE FLASH CACHE DOES A VERY GOOD JOB OF REDUCING THE SINGLE
BLOCK
READ LATENCY IN THE
MIXED
WORKLOAD SYSTEMS WEVE OBSERVED TO DATE
"ITS AND 0IECES
4HE PACKAGE PRICE INCLUDES A 5 RACK WITH REDUNDANT POWER DISTRIBUTION UNITS !LSO INCLUDED IN THE
PRICE IS AN %THERNET SWITCH 4HE SPEC SHEETS DONT SPECIFY THE MODEL FOR THE %THERNET SWITCH BUT AS OF THIS
WRITING THEY ARE SHIPPING A SWITCH MANUFACTURED BY #ISCO 4O DATE THIS IS THE ONE PIECE OF THE PACKAGE
THAT /RACLE HAS AGREED TO ALLOW CUSTOMERS TO REPLACE )F YOU HAVE ANOTHER SWITCH THAT YOU LIKE BETTER YOU
CAN REMOVE THE INCLUDED SWITCH AND REPLACE IT AT YOUR OWN COST 4HE 8
INCLUDES A +6- UNIT AS WELL
4HE PACKAGE PRICE ALSO INCLUDES A SPARES KIT THAT INCLUDES AN EXTRA FLASH CARD AN EXTRA DISK DRIVE AND
SOME EXTRA )NFINI"AND CABLES TWO EXTRA FLASH CARDS AND TWO EXTRA DISK DRIVES ON FULL RACKS 4HE PACKAGE
PRICE DOES NOT INCLUDE 3&0 CONNECTORS OR CABLES FOR THE '" %THERNET PORTS 4HESE ARE NOT STANDARD
AND WILL VARY BASED ON THE EQUIPMENT USED IN YOUR NETWORK 4HE PORTS ARE INTENDED FOR EXTERNAL
CONNECTIONS OF THE DATABASE SERVERS TO THE CUSTOMERS NETWORK
Software Components
4HE SOFTWARE COMPONENTS THAT MAKE UP %XADATA ARE SPLIT BETWEEN THE DATABASE TIER AND THE STORAGE TIER
3TANDARD /RACLE DATABASE SOFTWARE RUNS ON THE DATABASE SERVERS WHILE /RACLES RELATIVELY NEW DISK
MANAGEMENT SOFTWARE RUNS ON THE STORAGE SERVERS 4HE COMPONENTS ON BOTH TIERS USE A PROTOCOL CALLED
I$" TO TALK TO EACH OTHER 4HE NEXT TWO SECTIONS PROVIDE A BRIEF INTRODUCTION TO THE SOFTWARE STACK THAT
RESIDES ON BOTH TIERS
$ATABASE 3ERVER 3OFTWARE
!S PREVIOUSLY DISCUSSED THE DATABASE SERVERS RUN /RACLE ,INUX /F COURSE THERE IS THE OPTION TO RUN
3OLARIS %XPRESS BUT AS OF THIS WRITING WE HAVE NOT SEEN ONE RUNNING 3OLARIS
#(!04%2 7(!4 )3 %8!$!4!
4HE DATABASE SERVERS ALSO RUN STANDARD /RACLE G 2ELEASE SOFTWARE 4HERE IS NO SPECIAL VERSION OF
THE DATABASE CODE THAT IS DIFFERENT FROM THE CODE THAT IS RUN ON ANY OTHER PLATFORM 4HIS IS ACTUALLY A
UNIQUE AND SIGNIFICANT FEATURE OF %XADATA COMPARED TO COMPETING DATA WAREHOUSE APPLIANCE PRODUCTS
)N ESSENCE IT MEANS THAT ANY APPLICATION THAT CAN RUN ON /RACLE G2 CAN RUN ON %XADATA WITHOUT
REQUIRING ANY CHANGES TO THE APPLICATION 7HILE THERE IS CODE THAT IS SPECIFIC TO THE %XADATA PLATFORM I$"
FOR EXAMPLE /RACLE CHOSE TO MAKE IT A PART OF THE STANDARD DISTRIBUTION 4HE SOFTWARE IS AWARE OF
WHETHER IT IS ACCESSING %XADATA STORAGE AND THIS hAWARENESSv ALLOWS IT TO MAKE USE OF THE %XADATA
SPECIFIC OPTIMIZATIONS WHEN ACCESSING %XADATA STORAGE
!3- /RACLE !UTOMATIC 3TORAGE -ANAGEMENT IS A KEY COMPONENT OF THE SOFTWARE STACK ON THE
DATABASE SERVERS )T PROVIDES FILE SYSTEM AND VOLUME MANAGEMENT CAPABILITY FOR %XADATA STORAGE )T IS
REQUIRED BECAUSE THE STORAGE DEVICES ARE NOT VISIBLE TO THE DATABASE SERVERS 4HERE IS NO DIRECT
MECHANISM FOR PROCESSES ON THE DATABASE SERVERS TO OPEN OR READ A FILE ON %XADATA STORAGE CELLS !3-
ALSO PROVIDES REDUNDANCY TO THE STORAGE BY MIRRORING DATA BLOCKS USING EITHER NORMAL REDUNDANCY TWO
COPIES OR HIGH REDUNDANCY THREE COPIES 4HIS IS AN IMPORTANT FEATURE BECAUSE THE DISKS ARE PHYSICALLY
LOCATED ON MULTIPLE STORAGE SERVERS 4HE !3- REDUNDANCY ALLOWS MIRRORING ACROSS THE STORAGE CELLS
WHICH ALLOWS FOR THE COMPLETE LOSS OF A STORAGE SERVER WITHOUT AN INTERRUPTION TO THE DATABASES RUNNING
ON THE PLATFORM 4HERE IS NO FORM OF HARDWARE OR SOFTWARE BASED 2!)$ THAT PROTECTS THE DATA ON %XADATA
STORAGE SERVERS 4HE MIRRORING PROTECTION IS PROVIDED EXCLUSIVELY BY !3-
7HILE 2!# IS GENERALLY INSTALLED ON %XADATA DATABASE SERVERS IT IS NOT ACTUALLY REQUIRED 2!# DOES
PROVIDE MANY BENEFITS IN TERMS OF HIGH AVAILABILITY AND SCALABILITY THOUGH &OR SYSTEMS THAT REQUIRE MORE
#05 OR MEMORY RESOURCES THAN CAN BE SUPPLIED BY A SINGLE SERVER 2!# IS THE PATH TO THOSE ADDITIONAL
RESOURCES
4HE DATABASE SERVERS AND THE STORAGE SERVERS COMMUNICATE USING THE )NTELLIGENT $ATABASE PROTOCOL
I$" I$" IMPLEMENTS WHAT /RACLE REFERS TO AS A function shipping ARCHITECTURE 4HIS TERM IS USED TO
DESCRIBE HOW I$" SHIPS INFORMATION ABOUT THE 31, STATEMENT BEING EXECUTED TO THE STORAGE CELLS AND
THEN RETURNS PROCESSED DATA PREFILTERED FOR EXAMPLE INSTEAD OF DATA BLOCKS DIRECTLY TO THE REQUESTING
PROCESSES )N THIS MODE I$" CAN LIMIT THE DATA RETURNED TO THE DATABASE SERVER TO ONLY THOSE ROWS AND
COLUMNS THAT SATISFY THE QUERY 4HE FUNCTION SHIPPING MODE IS ONLY AVAILABLE WHEN FULL SCANS ARE
PERFORMED I$" CAN ALSO SEND AND RETRIEVE FULL BLOCKS WHEN OFFLOADING IS NOT POSSIBLE OR NOT DESIRABLE
)N THIS MODE I$" IS USED LIKE A NORMAL )/ PROTOCOL FOR FETCHING ENTIRE /RACLE BLOCKS AND RETURNING THEM
TO THE /RACLE BUFFER CACHE ON THE DATABASE SERVERS &OR COMPLETENESS WE SHOULD MENTION THAT IT IS REALLY
NOT A SIMPLE ONE WAY OR THE OTHER SCENARIO 4HERE ARE CASES WHERE WE CAN GET A COMBINATION OF THESE TWO
BEHAVIORS 7ELL DISCUSS THAT IN MORE DETAIL IN #HAPTER
I$" USES THE 2ELIABLE $ATAGRAM 3OCKETS 2$3 PROTOCOL AND OF COURSE USES THE )NFINI"AND FABRIC
BETWEEN THE DATABASE SERVERS AND STORAGE CELLS 2$3 IS A LOW
LATENCY LOW
OVERHEAD PROTOCOL THAT
PROVIDES A SIGNIFICANT REDUCTION IN #05 USAGE COMPARED TO PROTOCOLS SUCH AS 5$0 2$3 HAS BEEN
AROUND FOR SOME TIME AND PREDATES %XADATA BY SEVERAL YEARS 4HE PROTOCOL IMPLEMENTS A DIRECT MEMORY
ACCESS MODEL FOR INTERPROCESS COMMUNICATION WHICH ALLOWS IT TO AVOID THE LATENCY AND #05 OVERHEAD
ASSOCIATED WITH TRADITIONAL 4#0 TRAFFIC
#(!04%2 7(!4 )3 %8!$!4!
+EVIN 3AYS 2$3 HAS INDEED BEEN AROUND FOR QUITE SOME TIME ALTHOUGH NOT WITH THE %XADATA USE CASE IN MIND
4HE HISTORY OF 2$3 GOES BACK TO THE PARTNERING BETWEEN 3ILVER3TORM ACQUIRED BY 1LOGIC #ORPORATION AND /RACLE TO
ADDRESS THE REQUIREMENTS FOR LOW LATENCY AND HIGH BANDWIDTH PLACED UPON THE 2EAL !PPLICATION #LUSTERS NODE
INTERCONNECT VIA libskgxp FOR $,- LOCK TRAFFIC AND TO A LESSER DEGREE FOR 0ARALLEL 1UERY DATA SHIPPING 4HE LATTER
MODEL WAS FIRST PROVEN BY A 4" SCALE 40#
( CONDUCTED WITH /RACLE $ATABASE G ON THE NOW DEFUNCT
0!.4!3YSTEMS PLATFORM ,ATER /RACLE ALIGNED ITSELF MORE CLOSELY WITH -ELLANOX
4HIS HISTORY LESSON TOUCHES ON AN IMPORTANT POINT I$" IS BASED ON libskgxp WHICH ENJOYED MANY YEARS OF
HARDENING IN ITS ROLE OF INTERCONNECT LIBRARY DATING BACK TO THE FIRST PHASE OF THE #ACHE &USION FEATURE IN /RACLEI
4HE ABILITY TO LEVERAGE A TRIED AND TRUE TECHNOLOGY LIKE libskgxp CAME IN HANDY DURING THE MOVE TO TAKE 3!'% TO
MARKET
)T IS IMPORTANT TO UNDERSTAND THAT NO STORAGE DEVICES ARE DIRECTLY PRESENTED TO THE OPERATING SYSTEMS
ON THE DATABASE SERVERS 4HEREFORE THERE ARE NO OPERATING
SYSTEM CALLS TO OPEN FILES READ BLOCKS FROM
THEM OR THE OTHER USUAL TASKS 4HIS ALSO MEANS THAT STANDARD OPERATING
SYSTEM UTILITIES LIKE iostat WILL
NOT BE USEFUL IN MONITORING YOUR DATABASE SERVERS BECAUSE THE PROCESSES RUNNING THERE WILL NOT BE
ISSUING )/ CALLS TO THE DATABASE FILES (ERES SOME OUTPUT THAT ILLUSTRATES THIS FACT
KSO@SANDBOX1> @whoami
USERNAME SID SERIAL# PREV_HASH_VALUE SCHEMANAME OS_PID
--------------- ---------- ---------- --------------- ---------- -------
KSO 689 771 2334772408 KSO 23922
KSO@SANDBOX1> select /* avgskew3.sql */ avg(pk_col) from kso.skew3 a where col1 > 0;
...
> strace -cp 23922
Process 23922 attached - interrupt to quit
Process 23922 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
49.75 0.004690 0 10902 5451 setsockopt
29.27 0.002759 0 6365 poll
11.30 0.001065 0 5487 sendmsg
9.60 0.000905 0 15328 4297 recvmsg
0.08 0.000008 1 16 fcntl
0.00 0.000000 0 59 read
0.00 0.000000 0 3 write
0.00 0.000000 0 32 12 open
0.00 0.000000 0 20 close
#(!04%2 7(!4 )3 %8!$!4!
0.00 0.000000 0 4 stat
0.00 0.000000 0 4 fstat
0.00 0.000000 0 52 lseek
0.00 0.000000 0 33 mmap
0.00 0.000000 0 7 munmap
0.00 0.000000 0 1 semctl
0.00 0.000000 0 65 getrusage
0.00 0.000000 0 32 times
0.00 0.000000 0 1 semtimedop
------ ----------- ----------- --------- --------- ----------------
100.00 0.009427 38411 9760 total
)N THIS LISTING WE HAVE RUN strace ON A USERS FOREGROUND PROCESS SOMETIMES CALLED A SHADOW
PROCESS 4HIS IS THE PROCESS THATS RESPONSIBLE FOR RETRIEVING DATA ON BEHALF OF A USER !S YOU CAN SEE THE
VAST MAJORITY OF SYSTEM CALLS CAPTURED BY strace ARE NETWORK RELATED setsockopt poll sendmsg AND
recvmsg "Y CONTRAST ON A NON %XADATA PLATFORM WE MOSTLY SEE DISK )/ RELATED EVENTS PRIMARILY SOME
FORM OF THE READ CALL (ERES SOME OUTPUT FROM A NON %XADATA PLATFORM FOR COMPARISON
KSO@LAB112> @whoami
USERNAME SID SERIAL# PREV_HASH_VALUE SCHEMANAME OS_PID
--------------- ---------- ---------- --------------- ---------- -------
KSO 249 32347 4128301241 KSO 22493
KSO@LAB112> @avgskew
AVG(PK_COL)
-----------
16093749.8
...
[root@homer ~]# strace -cp 22493
Process 22493 attached - interrupt to quit
Process 22493 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
88.86 4.909365 3860 1272 pread64
10.84 0.599031 65 9171 gettimeofday
0.16 0.008766 64 136 getrusage
0.04 0.002064 56 37 times
0.02 0.001378 459 3 write
0.02 0.001194 597 2 statfs
0.02 0.001150 575 2 fstatfs
0.02 0.001051 350 3 read
0.01 0.000385 96 4 mmap2
0.00 0.000210 105 2 io_destroy
0.00 0.000154 77 2 io_setup
0.00 0.000080 40 2 open
0.00 0.000021 11 2 fcntl64
------ ----------- ----------- --------- --------- ----------------
100.00 5.524849 10638 total
#(!04%2 7(!4 )3 %8!$!4!
.OTICE THAT THE MAIN SYSTEM CALL CAPTURED ON THE NON %XADATA PLATFORM IS )/ RELATED pread64 4HE
POINT OF THE PREVIOUS TWO LISTINGS IS TO SHOW THAT THERE IS A VERY DIFFERENT MECHANISM IN PLAY IN THE WAY
DATA STORED ON DISKS IS ACCESSED WITH %XADATA
3TORAGE 3ERVER 3OFTWARE
#ELL 3ERVICES cellsrv IS THE PRIMARY SOFTWARE THAT RUNS ON THE STORAGE CELLS )T IS A MULTI
THREADED
PROGRAM THAT SERVICES )/ REQUESTS FROM A DATABASE SERVER 4HOSE REQUESTS CAN BE HANDLED BY RETURNING
PROCESSED DATA OR BY RETURNING COMPLETE BLOCKS DEPENDING IN THE REQUEST cellsrv ALSO IMPLEMENTS THE
2ESOURCE -ANAGER DEFINED )/ DISTRIBUTION RULES ENSURING THAT )/ IS DISTRIBUTED TO THE VARIOUS
DATABASES AND CONSUMER GROUPS APPROPRIATELY
4HERE ARE TWO OTHER PROGRAMS THAT RUN CONTINUOUSLY ON %XADATA STORAGE CELLS -ANAGEMENT 3ERVER
-3 IS A *AVA PROGRAM THAT PROVIDES THE INTERFACE BETWEEN cellsrv AND THE #ELL #OMMAND ,INE
)NTERFACE cellcli UTILITY -3 ALSO PROVIDES THE INTERFACE BETWEEN cellsrv AND THE 'RID #ONTROL %XADATA
PLUG
IN WHICH IS IMPLEMENTED AS A SET OF cellcli COMMANDS THAT ARE RUN VIA Rsh 4HE SECOND UTILITY IS
2ESTART 3ERVER 23 23 IS ACTUALLY A SET OF PROCESSES THAT IS RESPONSIBLE FOR MONITORING THE OTHER PROCESSES
AND RESTARTING THEM IF NECESSARY /37ATCHER IS ALSO INSTALLED ON THE STORAGE CELLS FOR COLLECTING HISTORICAL
OPERATING SYSTEM STATISTICS USING STANDARD 5NIX UTILITIES SUCH AS vmstat AND netstat .OTE THAT /RACLE
DOES NOT AUTHORIZE THE INSTALLATION OF ANY ADDITIONAL SOFTWARE ON THE STORAGE SERVERS
/NE OF THE FIRST THINGS YOU ARE LIKELY TO WANT TO DO WHEN YOU FIRST ENCOUNTER %XADATA IS TO LOG ON TO
THE STORAGE CELLS AND SEE WHATS ACTUALLY RUNNING 5NFORTUNATELY THE STORAGE SERVERS ARE GENERALLY OFF
LIMITS TO EVERYONE EXCEPT THE DESIGNATED SYSTEM ADMINISTERS OR $"!S (ERES A QUICK LISTING SHOWING THE
OUTPUT GENERATED BY A ps COMMAND ON AN ACTIVE STORAGE SERVER
> ps -eo ruser,pid,ppid,cmd
RUSER PID PPID CMD
root 12447 1 /opt/oracle/.../cellsrv/bin/cellrssrm -ms 1 -cellsrv 1
root 12453 12447 /opt/oracle/.../cellsrv/bin/cellrsbmt -ms 1 -cellsrv 1
root 12454 12447 /opt/oracle/.../cellsrv/bin/cellrsmmt -ms 1 -cellsrv 1
root 12455 12447 /opt/oracle/.../cellsrv/bin/cellrsomt -ms 1 -cellsrv 1
root 12456 12453 /opt/oracle/.../bin/cellrsbkm
-rs_conf /opt/oracle/.../cellsrv/deploy/config/cellinit.ora
-ms_conf /opt/oracle/cell
root 12457 12454 /usr/java/jdk1.5.0_15//bin/java -Xms256m -Xmx512m
-Djava.library.path=/opt/oracle/.../cellsrv/lib
-Ddisable.checkForUpdate=true -jar /opt/oracle/cell11.2
root 12460 12456 /opt/oracle/.../cellsrv/bin/cellrssmt
-rs_conf /opt/oracle/.../cellsrv/deploy/config/cellinit.ora
-ms_conf /opt/oracle/cell
root 12461 12455 /opt/oracle/.../cellsrv/bin/cellsrv 100 5000 9 5042
root 12772 22479 /usr/bin/mpstat 5 720
root 12773 22479 bzip2 --stdout
root 17553 1 /bin/ksh ./OSWatcher.sh 15 168 bzip2
root 20135 22478 /usr/bin/top -b -c -d 5 -n 720
root 20136 22478 bzip2 --stdout
root 22445 17553 /bin/ksh ./OSWatcherFM.sh 168
root 22463 17553 /bin/ksh ./oswsub.sh HighFreq ./Exadata_vmstat.sh
root 22464 17553 /bin/ksh ./oswsub.sh HighFreq ./Exadata_mpstat.sh
root 22465 17553 /bin/ksh ./oswsub.sh HighFreq ./Exadata_netstat.sh
root 22466 17553 /bin/ksh ./oswsub.sh HighFreq ./Exadata_iostat.sh
#(!04%2 7(!4 )3 %8!$!4!
root 22467 17553 /bin/ksh ./oswsub.sh HighFreq ./Exadata_top.sh
root 22471 17553 /bin/bash /opt/oracle.cellos/ExadataDiagCollector.sh
root 22472 17553 /bin/ksh ./oswsub.sh HighFreq
/opt/oracle.oswatcher/osw/ExadataRdsInfo.sh
root 22476 22463 /bin/bash ./Exadata_vmstat.sh HighFreq
root 22477 22466 /bin/bash ./Exadata_iostat.sh HighFreq
root 22478 22467 /bin/bash ./Exadata_top.sh HighFreq
root 22479 22464 /bin/bash ./Exadata_mpstat.sh HighFreq
root 22480 22465 /bin/bash ./Exadata_netstat.sh HighFreq
root 22496 22472 /bin/bash /opt/oracle.oswatcher/osw/ExadataRdsInfo.sh HighFreq
3O AS YOU CAN SEE THERE ARE A NUMBER OF PROCESSES THAT LOOK LIKE cellrsvXXX 4HESE ARE THE PROCESSES
THAT MAKE UP THE 2ESTART 3ERVER !LSO NOTICE THE FIRST BOLDED PROCESS THIS IS THE *AVA PROGRAM THAT WE
REFER TO AS -ANAGEMENT 3ERVER 4HE SECOND BOLDED PROCESS IS cellsrv ITSELF &INALLY YOULL SEE SEVERAL
PROCESSES ASSOCIATED WITH /37ATCHER .OTE ALSO THAT ALL THE PROCESSES ARE STARTED BY ROOT 7HILE THERE ARE
A COUPLE OF OTHER SEMI PRIVILEGED ACCOUNTS ON THE STORAGE SERVERS IT IS CLEARLY NOT A SYSTEM THAT IS SETUP
FOR USERS TO LOG ON TO
!NOTHER INTERESTING WAY TO LOOK AT RELATED PROCESSES IS TO USE THE ps –H COMMAND WHICH PROVIDES AN
INDENTED LIST OF PROCESSES SHOWING HOW THEY ARE RELATED TO EACH OTHER 9OU COULD WORK THIS OUT FOR
YOURSELF BY BUILDING A TREE BASED ON THE RELATIONSHIP BETWEEN THE PROCESS )$ 0)$ AND PARENT PROCESS )$
00)$ IN THE PREVIOUS LISTING BUT THE –H OPTION MAKES THAT A LOT EASIER (ERES AN EDITED SNIPPET OF
OUTPUT FROM A ps –H COMMAND
cellrssrm <= main Restart Server
cellrsbmt
cellrsbkm
cellrssmt
cellrsmmt
java - .../oc4j/ms/j2ee/home/oc4j.jar <= Management Server
cellrsomt
cellsrv
)TS ALSO INTERESTING TO SEE WHAT RESOURCES ARE BEING CONSUMED ON THE STORAGE SERVERS (ERES A
SNIPPET OF OUTPUT FROM top
top - 18:20:27 up 2 days, 2:09, 1 user, load average: 0.07, 0.15, 0.16
Tasks: 298 total, 1 running, 297 sleeping, 0 stopped, 0 zombie
Cpu(s): 6.1%us, 0.6%sy, 0.0%ni, 93.30%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 24531712k total, 14250280k used, 10281432k free, 188720k buffers
Swap: 2096376k total, 0k used, 2096376k free, 497792k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12461 root 18 0 17.0g 4.5g 11m S 105.9 19.2 55:20.45 cellsrv
1 root 18 0 10348 748 620 S 0.0 0.0 0:02.79 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.14 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:01.45 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
4HE OUTPUT FROM top SHOWS THAT cellsrv IS USING MORE THAN ONE FULL #05 CORE 4HIS IS COMMON ON
BUSY SYSTEMS AND IS DUE TO THE MULTI THREADED NATURE OF THE cellsrv PROCESS
#(!04%2 7(!4 )3 %8!$!4!
Software Architecture
)N THIS SECTION WELL BRIEFLY DISCUSS THE KEY SOFTWARE COMPONENTS AND HOW THEY ARE CONNECTED IN THE
%XADATA ARCHITECTURE 4HERE ARE COMPONENTS THAT RUN ON BOTH THE DATABASE AND THE STORAGE TIERS &IGURE
DEPICTS THE OVERALL ARCHITECTURE OF THE %XADATA PLATFORM
Oracle Database Server
CKPT RECO
PMON SMON
SGA
Shared Pool
Database Buffer Cache
Other
DBWR LGWR
Shadow
Processes
ASM
IDR / RDS over infiniband
Exadata Storage Servers
MS
Management
Server
Storage Server
CEL SRV
Shared Memory
RS
Restart
Server
OSWatcher
Cellinit.ora
Non-Exadata Disks
alert.log
Exadata Cell Disks
Figure 1-3. Exadata architecture diagram
#(!04%2 7(!4 )3 %8!$!4!
4HE TOP HALF OF THE DIAGRAM SHOWS THE KEY COMPONENTS ON ONE OF THE DATABASE SERVERS WHILE THE
BOTTOM HALF SHOWS THE KEY COMPONENTS ON ONE OF THE STORAGE SERVERS 4HE TOP HALF OF THE DIAGRAM SHOULD
LOOK PRETTY FAMILIAR AS IT IS STANDARD /RACLE G ARCHITECTURE )T SHOWS THE 3YSTEM 'LOBAL !REA 3'!
WHICH CONTAINS THE BUFFER CACHE AND THE SHARED POOL )T ALSO SHOWS SEVERAL OF THE KEY PROCESSES SUCH AS
,OG 7RITER ,'72 AND $ATABASE 7RITER $"72 4HERE ARE MANY MORE PROCESSES OF COURSE AND MUCH
MORE DETAILED VIEWS OF THE SHARED MEMORY THAT COULD BE PROVIDED BUT THIS SHOULD GIVE YOU A BASIC
PICTURE OF HOW THINGS LOOK ON THE DATABASE SERVER
4HE BOTTOM HALF OF THE DIAGRAM SHOWS THE COMPONENTS ON ONE OF THE STORAGE SERVERS 4HE
ARCHITECTURE ON THE STORAGE SERVERS IS PRETTY SIMPLE 4HERE IS REALLY ONLY ONE PROCESS cellsrv THAT
HANDLES ALL THE COMMUNICATION TO AND FROM THE DATABASE SERVERS 4HERE ARE ALSO A HANDFUL OF ANCILLARY
PROCESSES FOR MANAGING AND MONITORING THE ENVIRONMENT
/NE OF THE THINGS YOU MAY NOTICE IN THE ARCHITECTURE DIAGRAM IS THAT cellsrv USES AN init.ora FILE
AND HAS AN ALERT LOG )N FACT THE STORAGE SOFTWARE BEARS A STRIKING RESEMBLANCE TO AN /RACLE DATABASE
4HIS SHOULDNT BE TOO SURPRISING 4HE cellinit.ora FILE CONTAINS A SET OF PARAMETERS THAT ARE EVALUATED
WHEN cellsrv IS STARTED 4HE ALERT LOG IS USED TO WRITE A RECORD OF NOTABLE EVENTS MUCH LIKE AN ALERT LOG ON
AN /RACLE DATABASE .OTE ALSO THAT !UTOMATIC $IAGNOSTIC 2EPOSITORY !$2 IS INCLUDED AS PART OF THE
STORAGE SOFTWARE FOR CAPTURING AND REPORTING DIAGNOSTIC INFORMATION
!LSO NOTICE THAT THERE IS A STANDALONE PROCESS THAT IS NOT ATTACHED TO ANY DATABASE INSTANCE
DISKMON WHICH PERFORMS SEVERAL TASKS RELATED TO %XADATA 3TORAGE !LTHOUGH IT IS CALLED DISKMON IT IS
REALLY A NETWORK
AND CELL
MONITORING PROCESS THAT CHECKS TO VERIFY THAT THE CELLS ARE ALIVE DISKMON IS ALSO
RESPONSIBLE TO PROPAGATING $ATABASE 2ESOURCE -ANAGER $"2- PLANS TO THE STORAGE SERVERS DISKMON
ALSO HAS A SINGLE SLAVE PROCESS PER INSTANCE WHICH IS RESPONSIBLE FOR COMMUNICATING BETWEEN !3- AND
THE DATABASE IT IS RESPONSIBLE FOR
4HE CONNECTION BETWEEN THE DATABASE SERVER AND THE STORAGE SERVER IS PROVIDED BY THE )NFINI"AND
FABRIC !LL COMMUNICATION BETWEEN THE TWO TIERS IS CARRIED BY THIS TRANSPORT MECHANISM 4HIS INCLUDES
WRITES VIA THE $"72 PROCESSES AND ,'72 PROCESS AND READS CARRIED OUT BY THE USER FOREGROUND OR
SHADOW PROCESSES
&IGURE
PROVIDES ANOTHER VIEW OF THE ARCHITECTURE WHICH FOCUSES ON THE SOFTWARE STACK AND HOW IT
SPANS MULTIPLE SERVERS IN BOTH THE DATABASE GRID AND THE STORAGE GRID
#(!04%2 7(!4 )3 %8!$!4!
Single Instance Database RAC Database
Database Server Database Server Database Server
Database Instance Database Instance Database Instance
DBRM DBRM DBRM
Single ASM
ASM ASM ASM
Cluster
LIBCELL LIBCELL LIBCELL
iDB over Infiniband
Storage Cell Storage Cell Storage Cell
CELLSRV CELLSRV CELLSRV
IORM IORM IORM
RS MS RS MS RS MS
Figure 1-4. Exadata software architecture
!S WEVE DISCUSSED !3- IS A KEY COMPONENT .OTICE THAT WE HAVE DRAWN IT AS AN OBJECT THAT CUTS
ACROSS ALL THE COMMUNICATION LINES BETWEEN THE TWO TIERS 4HIS IS MEANT TO INDICATE THAT !3- PROVIDES
THE MAPPING BETWEEN THE FILES AND THE OBJECTS THAT THE DATABASE KNOWS ABOUT ON THE STORAGE LAYER !3-
DOES NOT ACTUALLY SIT BETWEEN THE STORAGE AND THE DATABASE THOUGH AND IT IS NOT A LAYER IN THE STACK THAT
THE PROCESSES MUST TOUCH FOR EACH hDISK ACCESSv
&IGURE
ALSO SHOWS THE RELATIONSHIP BETWEEN $ATABASE 2ESOURCE -ANAGER $"2- RUNNING ON
THE INSTANCES ON THE DATABASE SERVERS AND )/ 2ESOURCE -ANAGER )/2- WHICH IS IMPLEMENTED INSIDE
cellsrv RUNNING ON THE STORAGE SERVERS
4HE FINAL MAJOR COMPONENT IN &IGURE
IS LIBCELL WHICH IS A LIBRARY THAT IS LINKED WITH THE /RACLE
KERNEL LIBCELL HAS THE CODE THAT KNOWS HOW TO REQUEST DATA VIA I$" 4HIS PROVIDES A VERY NONINTRUSIVE
MECHANISM TO ALLOW THE /RACLE KERNEL TO TALK TO THE STORAGE TIER VIA NETWORK
BASED CALLS INSTEAD OF
OPERATING SYSTEM READS AND WRITES I$" IS IMPLEMENTED ON TOP OF THE 2ELIABLE $ATAGRAM 3OCKETS 2$3
PROTOCOL PROVIDED BY THE /PEN&ABRICS %NTERPRISE $ISTRIBUTION 4HIS IS A LOW
LATENCY LOW
#05
OVERHEAD
PROTOCOL THAT PROVIDES INTERPROCESS COMMUNICATIONS 9OU MAY ALSO SEE THIS PROTOCOL REFERRED TO IN SOME
OF THE /RACLE MARKETING MATERIAL AS THE :ERO
LOSS :ERO
COPY :$0 )NFINI"AND PROTOCOL &IGURE
IS A
BASIC SCHEMATIC SHOWING WHY THE 2$3 PROTOCOL IS MORE EFFICIENT THAN USING A TRADITIONAL 4#0 BASED
PROTOCOL LIKE 5$0
#(!04%2 7(!4 )3 %8!$!4!
5SER -ODE
Oracle / RDS Oracle / UDS
+ERNAL -ODE
RDS TCP
IP
IPoIB
(ARDWARE
Host Channel Adapter
Figure 1-5. RDS schematic
!S YOU CAN SEE FROM THE DIAGRAM USING THE 2$3 PROTOCOL TO BYPASS THE 4#0 PROCESSING CUTS OUT A
PORTION OF THE OVERHEAD REQUIRED TO TRANSFER DATA ACROSS THE NETWORK .OTE THAT THE 2$3 PROTOCOL IS ALSO
USED FOR INTERCONNECT TRAFFIC BETWEEN 2!# NODES
Summary
%XADATA IS A TIGHTLY INTEGRATED COMBINATION OF HARDWARE AND SOFTWARE 4HERE IS NOTHING MAGICAL ABOUT
THE HARDWARE COMPONENTS THEMSELVES 4HE MAJORITY OF THE PERFORMANCE BENEFITS COME FROM THE WAY THE
COMPONENTS ARE INTEGRATED AND THE SOFTWARE THAT IS IMPLEMENTED AT THE STORAGE LAYER )N THE NEXT CHAPTER
WELL DIVE INTO THE OFFLOADING CONCEPT WHICH IS WHAT SETS %XADATA APART FROM ALL OTHER PLATFORMS THAT RUN
/RACLE DATABASES
CHAPTER 2
Offloading / Smart Scan
/FFLOADING IS THE SECRET SAUCE OF %XADATA )TS WHAT MAKES %XADATA DIFFERENT FROM EVERY OTHER PLATFORM
THAT /RACLE RUNS ON /FFLOADING REFERS TO THE CONCEPT OF MOVING PROCESSING FROM THE DATABASE SERVERS TO
THE STORAGE LAYER )T IS ALSO THE KEY PARADIGM SHIFT PROVIDED BY THE %XADATA PLATFORM "UT ITS MORE THAN
JUST MOVING WORK IN TERMS OF #05 USAGE 4HE PRIMARY BENEFIT OF /FFLOADING IS THE REDUCTION IN THE
VOLUME OF DATA THAT MUST BE RETURNED TO THE DATABASE SERVER 4HIS IS ONE OF THE MAJOR BOTTLENECKS OF MOST
LARGE DATABASES
4HE TERMS /FFLOADING AND 3MART 3CAN ARE USED SOMEWHAT INTERCHANGEABLY /FFLOADING IS A BETTER
DESCRIPTION IN OUR OPINION AS IT REFERS TO THE FACT THAT PART OF THE TRADITIONAL 31, PROCESSING DONE BY THE
DATABASE CAN BE hOFFLOADEDv FROM THE DATABASE LAYER TO THE STORAGE LAYER )T IS A RATHER GENERIC TERM
THOUGH AND IS USED TO REFER TO MANY OPTIMIZATIONS THAT ARE NOT EVEN RELATED TO 31, PROCESSING INCLUDING
IMPROVING BACKUP AND RESTORE OPERATIONS
3MART 3CAN ON THE OTHER HAND IS A MORE FOCUSED TERM IN THAT IT REFERS ONLY TO %XADATAS
OPTIMIZATION OF 31, STATEMENTS 4HESE OPTIMIZATIONS COME INTO PLAY FOR SCAN OPERATIONS TYPICALLY &ULL
4ABLE 3CANS ! MORE SPECIFIC DEFINITION OF A 3MART 3CAN WOULD BE ANY SECTION OF THE /RACLE KERNEL CODE
THAT IS COVERED BY THE 3MART 3CAN WAIT EVENTS 4HERE ARE ACTUALLY TWO WAIT EVENTS THAT INCLUDE THE TERM
h3MART 3CANv IN THEIR NAMES #ELL 3MART 4ABLE 3CAN AND #ELL 3MART )NDEX 3CAN 7ELL DISCUSS BOTH OF
THESE WAIT EVENTS IN DETAIL A BIT LATER IN #HAPTER 7HILE ITS TRUE THAT h3MART 3CANv HAS A BIT OF A
MARKETING FLAVOR IT DOES HAVE SPECIFIC CONTEXT WHEN REFERRING TO THE CODE COVERED BY THESE WAIT EVENTS
!T ANY RATE WHILE THE TERMS ARE SOMEWHAT INTERCHANGEABLE KEEP IN MIND THAT /FFLOADING CAN REFER TO
MORE THAN JUST SPEEDING UP 31, STATEMENT EXECUTION
)N THIS CHAPTER WE WILL FOCUS ON 3MART 3CAN OPTIMIZATIONS 7ELL COVER THE VARIOUS OPTIMIZATIONS THAT
CAN COME INTO PLAY WITH 3MART 3CANS THE MECHANICS OF HOW THEY WORK AND THE REQUIREMENTS THAT MUST
BE MET FOR 3MART 3CANS TO OCCUR 7ELL ALSO COVER SOME TECHNIQUES THAT CAN BE USED TO HELP YOU
DETERMINE WHETHER 3MART 3CANS HAVE OCCURRED FOR A GIVEN 31, STATEMENT 4HE OTHER OFFLOADING
OPTIMIZATIONS WILL ONLY BE MENTIONED BRIEFLY AS THEY ARE COVERED ELSEWHERE IN THE BOOK
Why Offloading Is Important
7E CANT EMPHASIZE ENOUGH HOW IMPORTANT THIS CONCEPT IS 4HE IDEA OF MOVING DATABASE PROCESSING TO
THE STORAGE TIER IS A GIANT LEAP FORWARD 4HE CONCEPT HAS BEEN AROUND FOR SOME TIME )N FACT RUMOR HAS IT
THAT /RACLE APPROACHED AT LEAST ONE OF THE LARGE 3!. MANUFACTURERS SEVERAL YEARS AGO WITH THE IDEA 4HE
MANUFACTURER WAS APPARENTLY NOT INTERESTED AT THE TIME AND /RACLE DECIDED TO PURSUE THE IDEA ON ITS
OWN /RACLE SUBSEQUENTLY PARTNERED WITH (0 TO BUILD THE ORIGINAL %XADATA 6 WHICH INCORPORATED THE
/FFLOADING CONCEPT &AST FORWARD A COUPLE OF YEARS AND YOU HAVE /RACLES ACQUISITION OF 3UN
-ICROSYSTEMS 4HIS PUT THE COMPANY IN A POSITION TO OFFER AN INTEGRATED STACK OF HARDWARE AND SOFTWARE
AND GIVES IT COMPLETE CONTROL OVER WHICH FEATURES TO INCORPORATE INTO THE PRODUCT
#(!04%2 /&&,/!$).' 3-!24 3#!.
/FFLOADING IS IMPORTANT BECAUSE ONE OF THE MAJOR BOTTLENECKS ON LARGE DATABASES IS THE TIME IT TAKES
TO TRANSFER THE LARGE VOLUMES OF DATA NECESSARY TO SATISFY $7
TYPE QUERIES BETWEEN THE DISK SYSTEMS AND
THE DATABASE SERVERS THAT IS BECAUSE OF BANDWIDTH 4HIS IS PARTLY A HARDWARE ARCHITECTURE ISSUE BUT THE
BIGGER ISSUE IS THE SHEER VOLUME OF DATA THAT IS MOVED BY TRADITIONAL /RACLE DATABASES 4HE /RACLE
DATABASE IS VERY FAST AND VERY CLEVER ABOUT HOW IT PROCESSES DATA BUT FOR QUERIES THAT ACCESS A LARGE
AMOUNT OF DATA GETTING THE DATA TO THE DATABASE CAN STILL TAKE A LONG TIME 3O AS ANY GOOD PERFORMANCE
ANALYST WOULD DO /RACLE FOCUSED ON REDUCING THE TIME SPENT ON THE THING THAT ACCOUNTED FOR THE
MAJORITY OF THE ELAPSED TIME $URING THE ANALYSIS THE TEAM REALIZED THAT EVERY QUERY THAT REQUIRED DISK
ACCESS WAS VERY INEFFICIENT IN TERMS OF HOW MUCH DATA HAD TO BE RETURNED TO AND PROCESSED BY THE
DATABASE SERVERS /RACLE HAS MADE A LIVING BY DEVELOPING THE BEST CACHE
MANAGEMENT SOFTWARE
AVAILABLE BUT FOR REALLY LARGE DATA SETS IT IS JUST NOT PRACTICAL TO KEEP EVERYTHING IN MEMORY ON THE
DATABASE SERVERS
■ +EVIN 3AYS 4HE AUTHORS MAKE A GOOD POINT BASED ON A historical PERSPECTIVE OF /RACLE QUERY PROCESSING
(OWEVER ) ROUTINELY FIND MYSELF REMINDING PEOPLE THAT MODERN COMMODITY X SERVERS ARE NO LONGER
ARCHITECTURALLY CONSTRAINED TO SMALL MEMORY CONFIGURATIONS &OR EXAMPLE SERVERS BASED ON )NTEL 8EON
PROCESSORS WITH 1UICK 0ATH )NTERCONNECT SUPPORT LARGE NUMBERS OF MEMORY CHANNELS EACH WITH LARGE NUMBER OF
$)-- SLOTS #OMMODITY
BASED SERVERS WITH MULTIPLE TERABYTES OF MAIN MEMORY ARE QUITE COMMON )N FACT THE 8
%XADATA MODEL SUPPORTS TWO TERABYTES OF MAIN MEMORY IN THE DATABASE GRID AND THAT CAPACITY WILL INCREASE
NATURALLY OVER TIME ) EXPECT THIS BOOK TO REMAIN RELEVANT LONG ENOUGH FOR FUTURE READERS TO LOOK BACK ON THIS
COMMENT AS ARCANE SINCE THE TREND TOWARD EXTREMELY LARGE MAIN MEMORY X SYSTEMS HAS ONLY JUST BEGUN 4HE
IMPORTANT THING TO REMEMBER ABOUT %XADATA IS THAT IT IS EVERYTHING /RACLE DATABASE OFFERS PLUS %XADATA 3TORAGE
3ERVERS 4HIS POINT IS RELEVANT BECAUSE CUSTOMERS CAN CHOOSE TO COMBINE DEEP COMPRESSION FOR EXAMPLE %XADATA
(YBRID #OLUMNAR #OMPRESSION WITH THE )N
-EMORY 0ARALLEL 1UERY FEATURE FOR THOSE CASES WHERE RULING OUT
MAGNETIC MEDIA ENTIRELY IS THE RIGHT SOLUTION FOR MEETING SERVICE LEVELS
)MAGINE THE FASTEST QUERY YOU CAN THINK OF A SINGLE COLUMN FROM A SINGLE ROW FROM A SINGLE TABLE
WHERE YOU ACTUALLY KNOW WHERE THE ROW IS STORED ROWID /N A TRADITIONAL /RACLE DATABASE AT LEAST ONE
BLOCK OF DATA HAS TO BE READ INTO MEMORY TYPICALLY + TO GET THE ONE COLUMN ,ETS ASSUME YOUR TABLE
STORES AN AVERAGE OF ROWS PER BLOCK 9OUVE JUST TRANSFERRED EXTRA ROWS TO THE DATABASE SERVER THAT
ARE SIMPLY OVERHEAD FOR THIS QUERY -ULTIPLY THAT BY A BILLION AND YOU START TO GET AN IDEA OF THE
MAGNITUDE OF THE PROBLEM IN A LARGE DATA WAREHOUSE %LIMINATING THE TIME SPENT ON TRANSFERRING
COMPLETELY UNNECESSARY DATA BETWEEN THE STORAGE AND THE DATABASE TIER IS THE MAIN PROBLEM THAT
%XADATA WAS DESIGNED TO SOLVE
/FFLOADING IS THE APPROACH THAT WAS USED TO SOLVE THE PROBLEM OF EXCESSIVE TIME SPENT MOVING
IRRELEVANT DATA BETWEEN THE TIERS /FFLOADING HAS THREE DESIGN GOALS ALTHOUGH THE PRIMARY GOAL FAR
OUTWEIGHS THE OTHERS IN IMPORTANCE
v 2EDUCE THE VOLUME OF DATA TRANSFERRED FROM DISK SYSTEMS TO THE DATABASE SERVERS
v 2EDUCE #05 USAGE ON DATABASE SERVERS
v 2EDUCE DISK ACCESS TIMES AT THE STORAGE LAYER
#(!04%2 /&&,/!$).' 3-!24 3#!.
2EDUCING THE VOLUME WAS THE MAIN FOCUS AND PRIMARY GOAL 4HE MAJORITY OF THE OPTIMIZATIONS
INTRODUCED BY /FFLOADING CONTRIBUTE TO THIS GOAL 2EDUCING #05 LOAD IS IMPORTANT AS WELL BUT IS NOT THE
PRIMARY BENEFIT PROVIDED BY %XADATA AND THEREFORE TAKES A BACK SEAT TO REDUCING THE VOLUME OF DATA
TRANSFERRED !S YOULL SEE HOWEVER DECOMPRESSION IS A NOTABLE EXCEPTION TO THAT GENERALIZATION AS IT IS
PERFORMED ON THE STORAGE SERVERS 3EVERAL OPTIMIZATIONS TO REDUCE DISK ACCESS TIME WERE ALSO
INTRODUCED AND WHILE SOME OF THE RESULTS CAN BE QUITE STUNNING WE DONT CONSIDER THEM TO BE THE
BREAD
AND
BUTTER OPTIMIZATIONS OF %XADATA
%XADATA IS AN INTEGRATED HARDWARESOFTWARE PRODUCT THAT DEPENDS ON BOTH COMPONENTS TO PROVIDE
SUBSTANTIAL PERFORMANCE IMPROVEMENT OVER NON
%XADATA PLATFORMS (OWEVER THE PERFORMANCE BENEFITS
OF THE SOFTWARE COMPONENT DWARF THE BENEFITS PROVIDED BY THE HARDWARE (ERE IS AN EXAMPLE
SYS@SANDBOX> alter session set cell_offload_processing=false;
Session altered.
Elapsed: 00:00:00.06
SYS@SANDBOX> select count(*) from kso.skew3 where col1 < 0;
COUNT(*)
----------
2
1 row selected.
Elapsed: 00:00:51.09
SYS@SANDBOX> alter session set cell_offload_processing=true;
Session altered.
Elapsed: 00:00:00.07
SYS@SANDBOX> select count(*) from kso.skew3 where col1 < 0;
COUNT(*)
----------
2
1 row selected.
Elapsed: 00:00:00.15
4HIS EXAMPLE SHOWS THE PERFORMANCE OF A SCAN AGAINST A SINGLE TABLE WITH MILLION ROWS 7E RAN IT
ONCE WITH /FFLOADING DISABLED EFFECTIVELY USING ALL THE HARDWARE BENEFITS OF %XADATA AND NONE OF THE
SOFTWARE BENEFITS 9OULL NOTICE THAT EVEN ON %XADATA HARDWARE THIS QUERY TOOK ALMOST A MINUTE +EEP IN
MIND THAT THIS WAS ONLY SPREAD ACROSS THREE STORAGE SERVERS ON OUR 6 QUARTER RACK AND DID NOT UTILIZE THE
FLASH CACHE AT ALL 7E THEN RE
ENABLED /FFLOADING AND THE QUERY COMPLETED IN SUBSTANTIALLY LESS THAN A
SECOND /BVIOUSLY THE HARDWARE IN PLAY WAS THE SAME IN BOTH EXECUTIONS 4HE POINT IS THAT ITS THE
SOFTWARES ABILITY VIA /FFLOADING THAT MADE THE DIFFERENCE
#(!04%2 /&&,/!$).' 3-!24 3#!.
A GENERIC VERSION OF EXADATA?
4HE TOPIC OF BUILDING A GENERIC VERSION OF %XADATA COMES UP FREQUENTLY 4HE IDEA IS TO BUILD A HARDWARE
PLATFORM THAT IN SOME WAY MIMICS %XADATA PRESUMABLY AT A LOWER COST THAN WHAT /RACLE CHARGES FOR
%XADATA /F COURSE THE FOCUS OF THESE PROPOSALS IS TO REPLICATE THE HARDWARE PART OF %XADATA BECAUSE THE
SOFTWARE COMPONENT CANNOT BE REPLICATED 4HIS REALIZATION ALONE SHOULD MAKE YOU STOP AND QUESTION
WHETHER THIS APPROACH IS EVEN FEASIBLE .EVERTHELESS THE IDEA OF BUILDING YOUR OWN %XADATA SOUNDS
ATTRACTIVE BECAUSE THE INDIVIDUAL HARDWARE COMPONENTS CAN BE PURCHASED FOR LESS THAN THE PACKAGE PRICE
/RACLE CHARGES 4HERE ARE A FEW FLAWS WITH THIS THINKING HOWEVER
1. 4HE HARDWARE COMPONENT THAT TENDS TO GET THE MOST ATTENTION IS THE FLASH CACHE
9OU CAN BUY A 3!. OR .!3 WITH A LARGE CACHE 4HE MIDDLE
SIZE %XADATA PACKAGE
RACK SUPPLIES AROUND 4ERABYTES OF FLASH CACHE ACROSS THE STORAGE
SERVERS 4HATS A PRETTY BIG NUMBER BUT WHATS CACHED IS AS IMPORTANT AS THE SIZE
OF THE CACHE ITSELF %XADATA IS SMART ENOUGH NOT TO CACHE DATA THAT IS UNLIKELY TO
BENEFIT FROM CACHING &OR EXAMPLE IT IS NOT HELPFUL TO CACHE MIRROR COPIES OF
BLOCKS SINCE /RACLE ONLY READS PRIMARY COPIES UNLESS A CORRUPTION IS DETECTED
/RACLE HAS A LONG HISTORY OF WRITING SOFTWARE TO MANAGE CACHES 3O IT SHOULD COME
AS NO SURPRISE THAT IT DOES A VERY GOOD JOB OF NOT FLUSHING EVERYTHING OUT WHEN A
LARGE TABLE SCAN IS PROCESSED SO THAT FREQUENTLY ACCESSED BLOCKS WOULD TEND TO
REMAIN IN THE CACHE 4HE RESULT OF THIS DATABASE
AWARE CACHING IS THAT A NORMAL
3!. OR .!3 WOULD NEED A MUCH LARGER CACHE TO COMPETE WITH %XADATAS FLASH
CACHE +EEP IN MIND ALSO THAT THE VOLUME OF DATA YOU WILL NEED TO STORE WILL BE
MUCH LARGER ON NON
%XADATA STORAGE BECAUSE YOU WONT BE ABLE TO USE (YBRID
#OLUMNAR #OMPRESSION
2. 4HE MORE IMPORTANT ASPECT OF THE HARDWARE WHICH ODDLY ENOUGH IS OCCASIONALLY
OVERLOOKED BY THE $)9 PROPOSALS IS THE THROUGHPUT BETWEEN THE STORAGE AND
DATABASE TIERS 4HE %XADATA HARDWARE STACK PROVIDES A MORE BALANCED PATHWAY
BETWEEN STORAGE AND DATABASE SERVERS THAN MOST CURRENT IMPLEMENTATIONS 3O THE
SECOND AREA OF FOCUS IS GENERALLY THE BANDWIDTH BETWEEN THE TIERS )NCREASING THE
EFFECTIVE THROUGHPUT BETWEEN THE TIERS IS NOT AS SIMPLE AS IT SOUNDS THOUGH
%XADATA PROVIDES THE INCREASED THROUGHPUT VIA )NFINI"AND AND THE 2ELIABLE
$ATAGRAM 3OCKETS 2$3 PROTOCOL /RACLE DEVELOPED THE I$" PROTOCOL TO RUN ACROSS
THE )NFINIBAND NETWORK 4HE I$" PROTOCOL IS NOT AVAILABLE TO DATABASES RUNNING ON
NON
%XADATA HARDWARE 4HEREFORE SOME OTHER MEANS FOR INCREASING BANDWIDTH
BETWEEN THE TIERS IS NECESSARY 3O YOU CAN USE )0/" ON A 'E NETWORK AND USE
I3#3) OR .&3 OR YOU CAN USE HIGH
SPEED FIBER
BASED CONNECTIONS )N ANY CASE YOU
WILL NEED MULTIPLE INTERFACE CARDS IN THE SERVERS WHICH WILL NEED TO BE ATTACHED VIA
A FAST BUS 4HE STORAGE DEVICE OR DEVICES WILL ALSO HAVE TO BE CAPABLE OF
DELIVERING ENOUGH OUTPUT TO MATCH THE PIPE AND CONSUMPTION CAPABILITIES THIS IS
WHAT /RACLE MEANS WHEN THEY TALK ABOUT A BALANCED CONFIGURATION 9OULL ALSO
HAVE TO DECIDE WHICH HARDWARE COMPONENTS TO USE AND TEST THE WHOLE THING TO
MAKE SURE THAT ALL THE VARIOUS PARTS YOU PICK WORK WELL TOGETHER WITHOUT HAVING A
MAJOR BOTTLENECK AT ANY POINT IN THE PATH FROM DISK TO DATABASE SERVER
#(!04%2 /&&,/!$).' 3-!24 3#!.
3. 4HE THIRD COMPONENT THAT THE $)9 PROPOSALS GENERALLY ADDRESS IS THE DATABASE
SERVERS THEMSELVES 4HE %XADATA HARDWARE SPECIFICATIONS ARE READILY AVAILABLE SO
IT IS A SIMPLE MATTER TO BUY EXACTLY THE SAME 3UN MODELS 5NFORTUNATELY YOULL
NEED TO PLAN FOR MORE #05S SINCE YOU WONT BE ABLE TO OFFLOAD ANY PROCESSING TO
THE #05S ON THE %XADATA STORAGE SERVERS 4HIS IN TURN WILL DRIVE UP THE NUMBER OF
/RACLE DATABASE LICENSES
4. !SSUMING WE COULD MATCH THE %XADATA HARDWARE PERFORMANCE IN EVERY AREA WE
WOULD STILL NOT EXPECT TO BE ABLE COME CLOSE TO THE PERFORMANCE PROVIDED BY
%XADATA 4HATS BECAUSE IT IS THE SOFTWARE THAT PROVIDES THE LIONS SHARE OF THE
PERFORMANCE BENEFIT OF %XADATA 4HIS IS EASILY DEMONSTRATED BY DISABLING
/FFLOADING ON %XADATA AND RUNNING COMPARISONS 4HIS ALLOWS US TO SEE THE
PERFORMANCE OF THE HARDWARE WITHOUT THE SOFTWARE ENHANCEMENTS ! BIG PART OF
WHAT %XADATA SOFTWARE DOES IS ELIMINATE TOTALLY UNNECESSARY WORK SUCH AS
TRANSFERRING COLUMNS AND ROWS THAT WILL EVENTUALLY BE DISCARDED BACK TO THE
DATABASE SERVERS
!S OUR FRIEND #ARY -ILLSAP LIKES TO SAY h4HE FASTEST WAY TO DO ANYTHING IS TO NOT DO ITv
What Offloading Includes
4HERE ARE MANY OPTIMIZATIONS THAT CAN BE LUMPED UNDER THE /FFLOADING BANNER 4HIS CHAPTER FOCUSES ON
31, STATEMENT OPTIMIZATIONS THAT ARE IMPLEMENTED VIA 3MART 3CANS 4HE BIG THREE 3MART 3CAN
OPTIMIZATIONS ARE #OLUMN 0ROJECTION 0REDICATE &ILTERING AND 3TORAGE )NDEXES 4HE PRIMARY GOAL OF MOST
OF THE 3MART 3CAN OPTIMIZATIONS IS TO REDUCE THE AMOUNT OF DATA THAT NEEDS TO BE TRANSMITTED BACK TO THE
DATABASE SERVERS DURING SCAN EXECUTION (OWEVER SOME OF THE OPTIMIZATIONS ALSO ATTEMPT TO OFFLOAD
#05 INTENSIVE OPERATIONS DECOMPRESSION FOR EXAMPLE 7E WONT HAVE MUCH TO SAY ABOUT OPTIMIZATIONS
THAT ARE NOT RELATED TO 31, STATEMENT PROCESSING IN THIS CHAPTER SUCH AS 3MART &ILE #REATION AND 2-!.
RELATED OPTIMIZATIONS 4HOSE TOPICS WILL BE COVERED IN MORE DETAIL ELSEWHERE IN THE BOOK
■ +EVIN 3AYS 4HIS ASPECT OF /FFLOAD 0ROCESSING SEEMS QUITE COMPLICATED 4HE AUTHORS ARE CORRECT IN STATING THAT
THE PRIMARY BENEFIT OF 3MART 3CAN IS PAYLOAD REDUCTION BETWEEN STORAGE AND THE DATABASE GRID !ND ITS TRUE THAT
SOME #05
OFFLOAD BENEFIT IS ENJOYED BY DECOMPRESSING %XADATA (YBRID #OLUMNAR #OMPRESSION UNITS IN THE STORAGE
CELLS (OWEVER THEREIN LIES ONE CASE WHERE /FFLOAD 0ROCESSING ACTUALLY AIMS TO increase THE PAYLOAD BETWEEN THE
CELLS AND THE DATABASE GRID 4HE TRADE
OFF IS IMPORTANT HOWEVER )T MAKES SENSE TO DECOMPRESS %(## DATA IN THE
CELLS AFTER FILTRATION IN SPITE OF THE FACT THAT MORE DATA IS SENT TO THE DATABASE GRID DUE TO THE DECOMPRESSION !LL
TECHNOLOGY SOLUTIONS HAVE TRADE
OFFS
#OLUMN 0ROJECTION
4HE TERM Column Projection REFERS TO %XADATAS ABILITY TO LIMIT THE VOLUME OF DATA TRANSFERRED BETWEEN
THE STORAGE TIER AND THE DATABASE TIER BY ONLY RETURNING COLUMNS OF INTEREST THAT IS THOSE IN THE SELECT LIST
#(!04%2 /&&,/!$).' 3-!24 3#!.
OR NECESSARY FOR JOIN OPERATIONS ON THE DATABASE TIER )F YOUR QUERY REQUESTS FIVE COLUMNS FROM A
COLUMN TABLE %XADATA CAN ELIMINATE MOST OF THE DATA THAT WOULD BE RETURNED TO THE DATABASE SERVERS BY
NON
%XADATA STORAGE 4HIS FEATURE IS A MUCH BIGGER DEAL THAN YOU MIGHT EXPECT AND IT CAN HAVE A VERY
SIGNIFICANT IMPACT ON RESPONSE TIMES (ERE IS AN EXAMPLE
SYS@SANDBOX1> alter system flush shared_pool;
System altered.
Elapsed: 00:00:00.12
SYS@SANDBOX1> alter system flush buffer_cache;
System altered.
Elapsed: 00:00:00.13
SYS@SANDBOX1> alter session set "_serial_direct_read"=true;
Session altered.
Elapsed: 00:00:00.00
SYS@SANDBOX1> alter session set cell_offload_processing=false;
Session altered.
Elapsed: 00:00:00.01
SYS@SANDBOX1> select count(col1) from kso.skew3;
COUNT(COL1)
-----------
384000044
1 row selected.
Elapsed: 00:00:51.32
SYS@SANDBOX1> alter session set cell_offload_processing=true;
Session altered.
Elapsed: 00:00:00.00
SYS@SANDBOX1> select count(col1) from kso.skew3;
COUNT(COL1)
-----------
384000044
1 row selected.
Elapsed: 00:00:26.27
4HIS EXAMPLE DESERVES A LITTLE DISCUSSION &IRST WE USED A TRICK TO FORCE DIRECT PATH READS WITH THE
_SERIAL_DIRECT_READ PARAMETER MORE ON THAT LATER .EXT WE DISABLED 3MART 3CANS BY SETTING
CELL_OFFLOAD_PROCESSING TO FALSE 9OU CAN SEE THAT OUR TEST QUERY DOESNT HAVE A WHERE CLAUSE 4HIS
#(!04%2 /&&,/!$).' 3-!24 3#!.
MEANS THAT 0REDICATE &ILTERING AND 3TORAGE )NDEXES CANNOT BE USED TO CUT DOWN THE VOLUME OF DATA THAT
MUST BE TRANSFERRED FROM THE STORAGE TIER BECAUSE THOSE TWO OPTIMIZATIONS CAN ONLY BE DONE WHEN THERE
IS A WHERE CLAUSE WELL DISCUSS THOSE OPTIMIZATIONS SHORTLY 4HAT LEAVES #OLUMN 0ROJECTION AS THE ONLY
OPTIMIZATION IN PLAY !RE YOU SURPRISED THAT #OLUMN 0ROJECTION ALONE COULD CUT A QUERYS RESPONSE TIME
IN HALF 7E WERE THE FIRST TIME WE SAW IT BUT IT MAKES SENSE IF YOU THINK ABOUT IT 9OU SHOULD BE AWARE
THAT COLUMNS IN THE SELECT LIST ARE NOT THE ONLY COLUMNS THAT MUST BE RETURNED TO THE DATABASE SERVER
4HIS IS A VERY COMMON MISCONCEPTION *OIN COLUMNS IN THE WHERE CLAUSE MUST ALSO BE RETURNED !S A
MATTER OF FACT IN EARLY VERSIONS OF %XADATA THE #OLUMN 0ROJECTION FEATURE WAS NOT AS EFFECTIVE AS IT COULD
HAVE BEEN AND ACTUALLY RETURNED ALL THE COLUMNS INCLUDED IN THE WHERE CLAUSE WHICH IN MANY CASES
INCLUDED SOME UNNECESSARY COLUMNS
4HE DBMS_XPLAN PACKAGE CAN DISPLAY INFORMATION ABOUT COLUMN PROJECTION ALTHOUGH BY DEFAULT IT
DOES NOT 4HE PROJECTION DATA IS STORED IN THE PROJECTION COLUMN IN THE V$SQL_PLAN VIEW AS WELL (ERE IS
AN EXAMPLE
SYS@SANDBOX> select count(s.col1),avg(length(s.col4))
2 from kso.skew s, kso.skew2 s2
3 where s.pk_col = s2.pk_col
4 and s.col1 > 0
5 and s.col2='asddsadasd';
COUNT(S.COL1) AVG(LENGTH(S.COL4))
------------- -------------------
127999992 1
1 row selected.
SYS@SANDBOX> select sql_id, child_number, sql_text
2 from v$sql where sql_text like '%skew%';
SQL_ID CHILD SQL_TEXT
------------- ------ --------------------------------------------------------------
8xa3wjh48b9ar 0 select count(s.col1),avg(length(s.col4)) from kso.skew s, kso.
1 row selected.
SYS@SANDBOX> select * from
2 table(dbms_xplan.display_cursor('&sql_id','&child_no','+projection'));
Enter value for sql_id: 8xa3wjh48b9ar
Enter value for child_no:
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------
SQL_ID 8xa3wjh48b9ar, child number 0
-------------------------------------
select count(s.col1),avg(length(s.col4)) from kso.skew s, kso.skew2 s2
where s.pk_col = s2.pk_col and s.col1 > 0 and s.col2='asddsadasd'
Plan hash value: 3361152066
#(!04%2 /&&,/!$).' 3-!24 3#!.
-------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | | 360K(100)| |
| 1 | SORT AGGREGATE | | 1 | 30 | | | |
|* 2 | HASH JOIN | | 64M| 1836M| 549M| 360K (1)| 01:12:02 |
|* 3 | TABLE ACCESS STORAGE FULL| SKEW | 16M| 366M| | 44585 (2)| 00:08:56 |
| 4 | TABLE ACCESS STORAGE FULL| SKEW2| 128M| 732M| | 178K (1)| 00:35:37 |
-------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("S"."PK_COL"="S2"."PK_COL")
3 - storage(("S"."COL2"='asddsadasd' AND "S"."COL1">0))
filter(("S"."COL2"='asddsadasd' AND "S"."COL1">0))
Column Projection Information (identified by operation id):
-----------------------------------------------------------
1 - (#keys=0) COUNT(LENGTH("S"."COL4"))[22], COUNT("S"."COL1")[22],
SUM(LENGTH("S"."COL4"))[22]
2 - (#keys=1) "S"."COL4"[VARCHAR2,1], "S"."COL1"[NUMBER,22]
3 - "S"."PK_COL"[NUMBER,22], "S"."COL1"[NUMBER,22], "S"."COL4"[VARCHAR2,1]
4 - "S2"."PK_COL"[NUMBER,22]
33 rows selected.
SYS@SANDBOX> select projection from v$sql_plan
2 where projection is not null
3 and sql_id = '8xa3wjh48b9ar';
PROJECTION
-------------------------------------------------------------------------------------------
(#keys=0) COUNT(LENGTH("S"."COL4"))[22], COUNT("S"."COL1")[22], SUM(LENGTH("S"."COL4"))[22]
(#keys=1) "S"."COL4"[VARCHAR2,1], "S"."COL1"[NUMBER,22]
"S"."PK_COL"[NUMBER,22], "S"."COL1"[NUMBER,22], "S"."COL4"[VARCHAR2,1]
"S2"."PK_COL"[NUMBER,22]
4 rows selected.
3O AS YOU CAN SEE THE PLAN OUTPUT SHOWS THE PROJECTION INFORMATION BUT ONLY IF YOU USE THE
+PROJECTION ARGUMENT IN THE CALL TO THE DBMS_XPLAN PACKAGE .OTE ALSO THAT THE PK_COL COLUMNS FROM
BOTH TABLES WERE LISTED IN THE PROJECTION SECTION BUT THAT NOT ALL COLUMNS IN THE WHERE CLAUSE ARE
INCLUDED /NLY THOSE COLUMNS THAT NEED TO BE RETURNED TO THE DATABASE JOIN COLUMNS SHOULD BE LISTED
.OTE ALSO THAT THE PROJECTION INFORMATION IS NOT UNIQUE TO %XADATA BUT IS A GENERIC PART OF THE DATABASE
CODE
4HE V$SQL FAMILY OF VIEWS CONTAIN COLUMNS THAT DEFINE THE VOLUME OF DATA THAT MAY BE SAVED BY
/FFLOADING IO_CELL_OFFLOAD_ELIGIBLE_BYTES AND THE VOLUME OF DATA THAT WAS ACTUALLY RETURNED BY THE
STORAGE SERVERS IO_INTERCONNECT_BYTES .OTE THAT THESE COLUMNS ARE CUMULATIVE FOR ALL THE EXECUTIONS
OF THE STATEMENT 7ELL BE USING THESE TWO COLUMNS THROUGHOUT THE BOOK BECAUSE THEY ARE KEY INDICATORS
#(!04%2 /&&,/!$).' 3-!24 3#!.
OF OFFLOAD PROCESSING (ERES A QUICK DEMONSTRATION TO SHOW THAT PROJECTION DOES AFFECT THE AMOUNT OF
DATA RETURNED TO THE DATABASE SERVERS AND THAT SELECTING FEWER COLUMNS RESULTS IN LESS DATA TRANSFERRED
SYS@SANDBOX> select /* single col */ avg(pk_col)
2 from kso.skew3;
AVG(PK_COL)
-----------
16093750.3
1 row selected.
Elapsed: 00:00:32.13
SYS@SANDBOX> select /* multi col */ avg(pk_col),sum(col1)
2 from kso.skew3;
AVG(PK_COL) SUM(COL1)
----------- ----------
16093750.3 1.9003E+14
1 row selected.
Elapsed: 00:00:45.32
SYS@SANDBOX> set timing off
SYS@SANDBOX> select sql_id,sql_text from v$sql
2 where sql_text like '%col */ avg(pk_col)%';
SQL_ID SQL_TEXT
------------- ----------------------------------------------------------------------
bb3z4aaa9du7j select /* single col */ avg(pk_col) from kso.skew3
555pskb8aaqct select /* multi col */ avg(pk_col),sum(col1) from kso.skew3
2 rows selected.
SYS@SANDBOX> select sql_id, IO_CELL_OFFLOAD_ELIGIBLE_BYTES eligible,
2 IO_INTERCONNECT_BYTES actual,
3 100*(IO_CELL_OFFLOAD_ELIGIBLE_BYTES-IO_INTERCONNECT_BYTES)
4 /IO_CELL_OFFLOAD_ELIGIBLE_BYTES "IO_SAVED_%", sql_text
5 from v$sql where sql_id in ('bb3z4aaa9du7j','555pskb8aaqct');
SQL_ID ELIGIBLE ACTUAL IO_SAVED_% SQL_TEXT
------------- ---------- ---------- ---------- ------------------------------------
bb3z4aaa9du7j 1.6025E+10 4511552296 71.85 select /* single col */ avg(pk_col)
555pskb8aaqct 1.6025E+10 6421233960 59.93 select /* multi col */ avg(pk_col),s
2 rows selected.
SYS@SANDBOX> @fsx4
Enter value for sql_text: %col */ avg(pk_col)%
Enter value for sql_id:
#(!04%2 /&&,/!$).' 3-!24 3#!.
SQL_ID CHILD OFFLOAD IO_SAVED_% AVG_ETIME SQL_TEXT
------------- ------ ------- ---------- ------------- --------------------
6u7v77c2f8x5r 0 Yes 59.93 45.15 select /* multi col
d43dr7hvmw3yb 0 Yes 71.85 31.94 select /* single col
2 rows selected.
.OTE THAT THE EXTRA COLUMN RESULTED IN A GREAT DEAL OF EXTRA TIME REQUIRED TO COMPLETE THE QUERY AND
THAT THE COLUMNS IN V$SQL VERIFIED THE INCREASED VOLUME OF DATA THAT HAD TO BE TRANSFERRED 7EVE ALSO
SHOWN THE OUTPUT OF A MODIFIED VERSION OF THE fsx.sql SCRIPT WHICH WELL DISCUSS IN MORE DETAIL LATER IN
THIS CHAPTER &OR NOW PLEASE JUST ACCEPT THAT IT SHOWS US WHETHER A STATEMENT WAS OFFLOADED OR NOT
0REDICATE &ILTERING
4HE SECOND OF THE BIG THREE 3MART 3CAN OPTIMIZATIONS IS Predicate Filtering 4HIS TERM REFERS TO %XADATAS
ABILITY TO RETURN ONLY ROWS OF INTEREST TO THE DATABASE TIER 3INCE I$" INCLUDES THE PREDICATE INFORMATION
IN ITS REQUESTS THIS IS ACCOMPLISHED BY PERFORMING THE STANDARD FILTERING OPERATIONS AT THE STORAGE CELLS
BEFORE RETURNING THE DATA /N DATABASES USING NON
%XADATA STORAGE FILTERING IS DONE ON THE DATABASE
SERVERS 4HIS GENERALLY MEANS THAT A LARGE NUMBER OF RECORDS THAT WILL EVENTUALLY BE DISCARDED WILL BE
RETURNED TO THE DATABASE TIER &ILTERING THESE ROWS AT THE STORAGE LAYER CAN PROVIDE A VERY SIGNIFICANT
DECREASE IN THE VOLUME OF DATA THAT MUST BE TRANSFERRED TO THE DATABASE TIER 7HILE THIS OPTIMIZATION ALSO
RESULTS IN SOME SAVINGS IN #05 USAGE ON THE DATABASE SERVERS THE BIGGEST ADVANTAGE IS GENERALLY THE
REDUCTION IN DATA TRANSFER
(ERE IS AN EXAMPLE
SYS@SANDBOX> alter session set cell_offload_processing=false;
Session altered.
Elapsed: 00:00:00.01
SYS@SANDBOX> select count(pk_col) from kso.skew3;
COUNT(PK_COL)
-------------
384000036
Elapsed: 00:00:48.45
SYS@SANDBOX> alter session set cell_offload_processing=true;
Session altered.
Elapsed: 00:00:00.01
SYS@SANDBOX> select count(pk_col) from kso.skew3;
COUNT(PK_COL)
-------------
384000036
Elapsed: 00:00:26.61
SYS@SANDBOX> -- disable storage indexes
SYS@SANDBOX> alter system set "_kcfis_storageidx_disabled"=true;
#(!04%2 /&&,/!$).' 3-!24 3#!.
System altered.
Elapsed: 00:00:00.17
SYS@SANDBOX> select count(pk_col) from kso.skew3 where col1 < 0;
COUNT(PK_COL)
-------------
2
Elapsed: 00:00:08.53
&IRST WE COMPLETELY DISABLED /FFLOADING USING THE CELL_OFFLOAD_PROCESSING PARAMETER AND RAN A
QUERY WITHOUT A WHERE CLAUSE 7ITHOUT THE BENEFIT OF /FFLOADING THIS QUERY TOOK ABOUT SECONDS 7E
THEN ENABLED /FFLOADING AND RE
RAN THE QUERY 4HIS TIME THE QUERY TOOK ONLY ABOUT SECONDS 4HE
SAVINGS OF APPROXIMATELY SECONDS WAS DUE STRICTLY TO #OLUMN 0ROJECTION BECAUSE WITHOUT A WHERE
CLAUSE FOR FILTERING THERE WERE NO OTHER OPTIMIZATIONS THAT COULD COME INTO PLAY 7E THEN USED A TRICK TO
DISABLE STORAGE INDEXES BY SETTING THE HIDDEN PARAMETER _KCFIS_STORAGEIDX_DISABLED TO TRUE WELL
DISCUSS THAT MORE IN THE NEXT SECTION AND ADDED A WHERE CLAUSE WHICH REDUCED THE EXECUTION TIME TO
ABOUT SECONDS 4HIS REDUCTION OF AN ADDITIONAL SECONDS OR SO WAS THANKS TO 0REDICATE &ILTERING .OTE
THAT WE HAD TO DISABLE STORAGE INDEXES TO BE SURE THAT WE WERENT GETTING ANY BENEFIT FROM THAT
OPTIMIZATION AND THAT ALL THE IMPROVEMENT WAS DUE TO 0REDICATE &ILTERING WHICH BRINGS US TO THE NEXT
TOPIC
3TORAGE )NDEXES
3TORAGE )NDEXES PROVIDE THE THIRD LEVEL OF OPTIMIZATION FOR 3MART 3CANS 3TORAGE )NDEXES ARE IN
MEMORY
STRUCTURES ON THE STORAGE CELLS THAT MAINTAIN A MAXIMUM AND MINIMUM VALUE FOR EACH -" DISK STORAGE
UNIT FOR UP TO EIGHT COLUMNS OF A TABLE 3TORAGE )NDEXES ARE A LITTLE DIFFERENT THAN MOST 3MART 3CAN
OPTIMIZATIONS 4HE GOAL OF 3TORAGE )NDEXES IS NOT TO REDUCE THE AMOUNT OF DATA BEING TRANSFERRED BACK TO
THE DATABASE TIER )N FACT WHETHER THEY ARE USED ON A GIVEN QUERY OR NOT THE AMOUNT OF DATA RETURNED TO
THE DATABASE TIER REMAINS CONSTANT /N THE CONTRARY 3TORAGE )NDEXES ARE DESIGNED TO ELIMINATE TIME
SPENT READING DATA FROM DISK ON THE STORAGE SERVERS THEMSELVES 4HINK OF THIS FEATURE AS A PRE
FILTER 3INCE
3MART 3CANS PASS THE QUERY PREDICATES TO THE STORAGE SERVERS AND 3TORAGE )NDEXES CONTAIN A MAP OF
VALUES IN EACH -" STORAGE REGION ANY REGION THAT CANT POSSIBLY CONTAIN A MATCHING ROW CAN BE
ELIMINATED WITHOUT EVER BEING READ 9OU CAN ALSO THINK OF 3TORAGE )NDEXES AS AN ALTERNATE PARTITIONING
MECHANISM $ISK )/ IS ELIMINATED IN ANALOGOUS FASHION TO PARTITION ELIMINATION )F A PARTITION CANT
CONTAIN ANY RECORDS OF INTEREST THE PARTITIONS BLOCKS WILL NOT BE READ 3IMILARLY IF A STORAGE REGION
CANNOT CONTAIN ANY RECORDS OF INTEREST THAT STORAGE REGION NEED NOT BE READ
3TORAGE )NDEXES CANNOT BE USED IN ALL CASES AND THERE IS LITTLE THAT CAN BE DONE TO AFFECT WHEN OR HOW
THEY ARE USED "UT IN THE RIGHT SITUATIONS THE RESULTS FROM THIS OPTIMIZATION TECHNIQUE CAN BE
ASTOUNDING
(ERE IS AN EXAMPLE
SYS@SANDBOX> -- disable storage indexes
SYS@SANDBOX> alter system set "_kcfis_storageidx_disabled"=true;
System altered.
Elapsed: 00:00:00.22
SYS@SANDBOX> select count(pk_col) from kso.skew3 where col1 < 0;
#(!04%2 /&&,/!$).' 3-!24 3#!.
COUNT(PK_COL)
-------------
2
Elapsed: 00:00:08.74
SYS@SANDBOX> -- enable storage indexes
SYS@SANDBOX> alter system set "_kcfis_storageidx_disabled"=false;
System altered.
Elapsed: 00:00:00.03
SYS@SANDBOX> select count(pk_col) from kso.skew3 where col1 < 0;
COUNT(PK_COL)
-------------
2
Elapsed: 00:00:00.08
)N THIS EXAMPLE WE DISABLED STORAGE INDEXES USING THE _KCFIS_STORAGEIDX_DISABLED PARAMETER TO
REMIND YOU OF THE ELAPSED TIME REQUIRED TO READ THROUGH MILLION ROWS USING #OLUMN 0ROJECTION AND
0REDICATE &ILTERING 2EMEMBER THAT EVEN THOUGH THE AMOUNT OF DATA RETURNED TO THE DATABASE TIER IS
EXTREMELY SMALL IN THIS CASE THE STORAGE SERVERS STILL HAD TO READ THROUGH EVERY BLOCK CONTAINING DATA FOR
THE SKEW3 TABLE AND THEN HAD TO CHECK EACH ROW TO SEE IF IT MATCHED THE WHERE CLAUSE 4HIS IS WHERE THE
MAJORITY OF THE SECONDS WAS SPENT 7E THEN RE
ENABLED STORAGE INDEXES AND RERAN THE QUERY WHICH
REDUCED THE EXECUTION TIME TO ABOUT SECONDS 4HIS REDUCTION IN ELAPSED TIME IS A RESULT OF STORAGE
INDEXES BEING USED TO AVOID VIRTUALLY ALL OF THE DISK )/ AND THE TIME SPENT FILTERING THROUGH THOSE
RECORDS
*UST TO REITERATE #OLUMN 0ROJECTION AND 0REDICATE &ILTERING AND MOST OTHER 3MART 3CAN
OPTIMIZATIONS IMPROVE PERFORMANCE BY REDUCING THE VOLUME OF DATA BEING TRANSFERRED BACK TO THE
DATABASE SERVERS AND THUS THE AMOUNT OF TIME TO TRANSFER THE DATA 3TORAGE )NDEXES IMPROVE
PERFORMANCE BY ELIMINATING TIME SPENT READING DATA FROM DISK ON THE STORAGE SERVERS AND FILTERING THAT
DATA 3TORAGE )NDEXES ARE COVERED IN MUCH MORE DETAIL IN #HAPTER
3IMPLE *OINS "LOOM &ILTERS
)N SOME CASES JOIN PROCESSING CAN BE OFFLOADED TO THE STORAGE TIER AS WELL /FFLOADED JOINS ARE
ACCOMPLISHED BY CREATING WHAT IS CALLED A bloom filter "LOOM FILTERS HAVE BEEN AROUND FOR A LONG TIME
AND HAVE BEEN USED BY /RACLE SINCE /RACLE $ATABASE 6ERSION G 2ELEASE 3O THEY ARE NOT SPECIFIC TO
%XADATA /NE OF THE MAIN WAYS /RACLE USES THEM IS TO REDUCE TRAFFIC BETWEEN PARALLEL QUERY SLAVES 4HEY
HAVE THE ADVANTAGE OF BEING VERY SMALL RELATIVE TO THE DATA SET THAT THEY REPRESENT (OWEVER THIS COMES
AT A PRICE THEY CAN RETURN FALSE POSITIVES 4HAT IS ROWS THAT SHOULD NOT BE INCLUDED IN THE DESIRED RESULT
SET CAN OCCASIONALLY PASS A BLOOM FILTER &OR THAT REASON AN ADDITIONAL FILTER MUST BE APPLIED AFTER THE
BLOOM FILTER TO ENSURE THAT ANY FALSE POSITIVES ARE ELIMINATED 4HE INTERESTING THING ABOUT BLOOM FILTERS
FROM AN %XADATA PERSPECTIVE IS THAT THEY MAY BE PASSED TO THE STORAGE SERVERS AND EVALUATED THERE 4HIS
TECHNIQUE CAN RESULT IN A LARGE DECREASE IN THE VOLUME OF DATA THAT MUST BE TRANSMITTED BACK TO DATABASE
SERVERS
(ERES AN EXAMPLE
#(!04%2 /&&,/!$).' 3-!24 3#!.
SYS@SANDBOX> -- disable bloom filter offloading
SYS@SANDBOX> alter session set "_bloom_predicate_pushdown_to_storage"=false;
Session altered.
Elapsed: 00:00:00.82
SYS@SANDBOX> @bloom_join2.sql
COL2 SUM(A.COL1)
------------------------------ -----------
2342 144
asddsadasd 153598416
2 rows selected.
Elapsed: 00:11:39.39
SYS@SANDBOX> -- enable bloom filter offloading
SYS@SANDBOX> alter session set "_bloom_predicate_pushdown_to_storage"=true;
Session altered.
Elapsed: 00:00:00.82
SYS@SANDBOX> @bloom_join2.sql
COL2 SUM(A.COL1)
------------------------------ -----------
asddsadasd 153598416
2342 144
2 rows selected.
Elapsed: 00:02:06.13
SYS@SANDBOX> @dplan
Enter value for sql_id: 09m6t5qpgkywx
Enter value for child_no: 0
PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------
SQL_ID 09m6t5qpgkywx, child number 0
-------------------------------------
select /*+ bloom join 2 use_hash (skew temp_skew) */ a.col2,
sum(a.col1) from kso.skew3 a, kso.skew2 b where a.pk_col = b.pk_col and
b.col1 = 1 group by a.col2
Plan hash value: 466947137
#(!04%2 /&&,/!$).' 3-!24 3#!.
-----------------------------------------------------------------------------------------
| Id | Operation | Name | TQ |IN-OUT| PQ Distrib |
-----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | |
| 1 | PX COORDINATOR | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10002 | Q1,02 | P->S | QC (RAND) |
| 3 | HASH GROUP BY | | Q1,02 | PCWP | |
| 4 | PX RECEIVE | | Q1,02 | PCWP | |
| 5 | PX SEND HASH | :TQ10001 | Q1,01 | P->P | HASH |
| 6 | HASH GROUP BY | | Q1,01 | PCWP | |
|* 7 | HASH JOIN | | Q1,01 | PCWP | |
| 8 | BUFFER SORT | | Q1,01 | PCWC | |
| 9 | PX RECEIVE | | Q1,01 | PCWP | |
| 10 | PX SEND BROADCAST | :TQ10000 | | S->P | BROADCAST |
| 11 | TABLE ACCESS BY INDEX ROWID| SKEW2 | | | |
|* 12 | INDEX RANGE SCAN | SKEW2_COL1 | | | |
| 13 | PX BLOCK ITERATOR | | Q1,01 | PCWC | |
|* 14 | TABLE ACCESS STORAGE FULL | SKEW3 | Q1,01 | PCWP | |
-----------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
7 - access("A"."PK_COL"="B"."PK_COL")
12 - access("B"."COL1"=1)
14 - storage(:Z>=:Z AND :Z<=:Z)
filter(SYS_OP_BLOOM_FILTER(:BF0000,"A"."PK_COL"))
36 rows selected.
SYS@SANDBOX> @dplan
Enter value for sql_id: 09m6t5qpgkywx
Enter value for child_no: 1
PLAN_TABLE_OUTPUT
-----------------------------------------------------------------------------------------
SQL_ID 09m6t5qpgkywx, child number 1
-------------------------------------
select /*+ bloom join 2 use_hash (skew temp_skew) */ a.col2,
sum(a.col1) from kso.skew3 a, kso.skew2 b where a.pk_col = b.pk_col and
b.col1 = 1 group by a.col2
Plan hash value: 466947137
-----------------------------------------------------------------------------------------
| Id | Operation | Name | TQ |IN-OUT| PQ Distrib |
-----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | |
| 1 | PX COORDINATOR | | | | |
| 2 | PX SEND QC (RANDOM) | :TQ10002 | Q1,02 | P->S | QC (RAND) |
#(!04%2 /&&,/!$).' 3-!24 3#!.
| 3 | HASH GROUP BY | | Q1,02 | PCWP | |
| 4 | PX RECEIVE | | Q1,02 | PCWP | |
| 5 | PX SEND HASH | :TQ10001 | Q1,01 | P->P | HASH |
| 6 | HASH GROUP BY | | Q1,01 | PCWP | |
|* 7 | HASH JOIN | | Q1,01 | PCWP | |
| 8 | BUFFER SORT | | Q1,01 | PCWC | |
| 9 | PX RECEIVE | | Q1,01 | PCWP | |
| 10 | PX SEND BROADCAST | :TQ10000 | | S->P | BROADCAST |
| 11 | TABLE ACCESS BY INDEX ROWID| SKEW2 | | | |
|* 12 | INDEX RANGE SCAN | SKEW2_COL1 | | | |
| 13 | PX BLOCK ITERATOR | | Q1,01 | PCWC | |
|* 14 | TABLE ACCESS STORAGE FULL | SKEW3 | Q1,01 | PCWP | |
-----------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
7 - access("A"."PK_COL"="B"."PK_COL")
12 - access("B"."COL1"=1)
14 - storage(:Z>=:Z AND :Z<=:Z AND SYS_OP_BLOOM_FILTER(:BF0000,"A"."PK_COL"))
filter(SYS_OP_BLOOM_FILTER(:BF0000,"A"."PK_COL"))
36 rows selected.
)N THIS LISTING WE USED A HIDDEN PARAMETER ?BLOOM_PREDICATE_PUSHDOWN_TO_STORAGE TO DISABLE THIS
FEATURE FOR COMPARISON PURPOSES .OTICE THAT OUR TEST QUERY RAN IN ABOUT MINUTES WITH /FFLOADING AND
MINUTES WITHOUT )F YOU LOOK CLOSELY AT THE 0REDICATE )NFORMATION OF THE PLANS YOU WILL SEE THAT THE
SYS_OP_BLOOM_FILTER(:BF0000,"A"."PK_COL") PREDICATE WAS RUN ON THE STORAGE SERVERS FOR THE SECOND
RUN 4HE OFFLOADED VERSION RAN FASTER BECAUSE THE STORAGE SERVERS WERE ABLE TO PRE
JOIN THE TABLES WHICH
ELIMINATED A LARGE AMOUNT OF DATA THAT WOULD OTHERWISE HAVE BEEN TRANSFERRED BACK TO THE DATABASE
SERVERS
&UNCTION /FFLOADING
/RACLES IMPLEMENTATION OF 31, INCLUDES MANY BUILT
IN 31, FUNCTIONS 4HESE FUNCTIONS CAN BE USED
DIRECTLY IN 31, STATEMENTS 4HEY MAY BE DIVIDED INTO TWO MAIN GROUPS SINGLE
ROW FUNCTIONS AND MULTI
ROW FUNCTIONS 3INGLE
ROW FUNCTIONS RETURN A SINGLE RESULT ROW FOR EVERY ROW OF A QUERIED TABLE 4HESE
SINGLE ROW FUNCTIONS CAN BE FURTHER SUBDIVIDED INTO THE FOLLOWING GENERAL CATEGORIES
v .UMERIC FUNCTIONS SIN COS FLOOR MOD LOG x
v #HARACTER FUNCTIONS CHR LPAD REPLACE TRIM UPPER LENGTH x
v $ATETIME FUNCTIONS ADD_MONTHS TO_CHAR TRUNC x
v #ONVERSION FUNCTIONS CAST HEXTORAW TO_CHAR TO_DATE x
6IRTUALLY ALL OF THESE SINGLE
ROW FUNCTIONS CAN BE OFFLOADED TO %XADATA STORAGE 4HE SECOND MAJOR
GROUP OF 31, FUNCTIONS OPERATE ON A SET OF ROWS 4HERE ARE TWO SUBGROUPS IN THIS MULTI
ROW FUNCTION
CATEGORY
#(!04%2 /&&,/!$).' 3-!24 3#!.
v !GGREGATE FUNCTIONS AVG COUNT SUM x
v !NALYTIC FUNCTIONS AVG COUNT DENSE_RANK LAG x
4HESE FUNCTIONS RETURN EITHER A SINGLE ROW AGGREGATE FUNCTIONS OR MULTIPLE ROWS ANALYTIC
FUNCTIONS .OTE THAT SOME OF THE FUNCTIONS ARE OVERLOADED AND BELONG TO BOTH GROUPS .ONE OF THESE
FUNCTIONS CAN BE OFFLOADED TO %XADATA 7HICH MAKES SENSE BECAUSE MANY OF THESE FUNCTIONS REQUIRE
ACCESS TO THE ENTIRE SET OF ROWS WHICH INDIVIDUAL STORAGE CELLS DO NOT HAVE
4HERE ARE SOME ADDITIONAL FUNCTIONS THAT DONT FALL NEATLY INTO ANY OF THE PREVIOUSLY DESCRIBED
GROUPINGS 4HESE FUNCTIONS ARE A MIXED BAG IN TERMS OF /FFLOADING &OR EXAMPLE DECODE AND NVL ARE
OFFLOADABLE BUT THE 8-, FUNCTIONS ARE NOT 3OME OF THE $ATA -INING FUNCTIONS ARE OFFLOADABLE AND SOME
ARE NOT !LSO KEEP IN MIND THAT THE LIST OF OFFLOADABLE FUNCTIONS MAY CHANGE AS NEWER VERSIONS ARE
RELEASED 4HE DEFINITIVE LIST OF WHICH FUNCTIONS ARE OFFLOADABLE FOR YOUR PARTICULAR VERSION IS CONTAINED IN
V$SQLFN_METADATA
SYS@SANDBOX> select distinct name, version, offloadable
2 from V$SQLFN_METADATA
3 order by 1,2;
NAME VERSION OFF
------------------------------ ------------ ---
!= SQL/DS YES
!= V6 Oracle YES
< SQL/DS YES
< V6 Oracle YES
<= SQL/DS YES
<= V6 Oracle YES
= SQL/DS YES
= V6 Oracle YES
> SQL/DS YES
> V6 Oracle YES
>= SQL/DS YES
>= V6 Oracle YES
ABS V10 Oracle YES
ABS V10 Oracle YES
ABS V6 Oracle YES
ACOS V10 Oracle YES
ACOS V73 Oracle YES
. . .
VSIZE V6 Oracle YES
WIDTH_BUCKET V82 Oracle NO
XMLCAST V11R1 Oracle NO
XMLCDATA V10 Oracle NO
XMLCOMMENT V10 Oracle NO
XMLCONCAT V92 Oracle NO
XMLDIFF V11R1 Oracle NO
XMLEXISTS2 V11R1 Oracle NO
XMLISNODE V92 Oracle NO
XMLISVALID V92 Oracle NO
XMLPATCH V11R1 Oracle NO
XMLQUERY V10 Oracle NO
XMLTOOBJECT V11R1 Oracle NO
#(!04%2 /&&,/!$).' 3-!24 3#!.
XMLTRANSFORM V92 Oracle NO
XMLTRANSFORMBLOB V10 Oracle NO
XS_SYS_CONTEXT V11R1 Oracle NO
921 rows selected.
/FFLOADING FUNCTIONS DOES ALLOW THE STORAGE CELLS TO DO SOME OF THE WORK THAT WOULD NORMALLY BE
DONE BY THE #05S ON THE DATABASE SERVERS (OWEVER THE SAVING IN #05 USAGE IS GENERALLY A RELATIVELY
MINOR ENHANCEMENT 4HE BIG GAIN USUALLY COMES FROM LIMITING THE AMOUNT OF DATA TRANSFERRED BACK TO
THE DATABASE SERVERS "EING ABLE TO EVALUATE FUNCTIONS CONTAINED IN WHERE CLAUSES ALLOWS STORAGE CELLS TO
SEND ONLY ROWS OF INTEREST BACK TO THE DATABASE TIER 3O AS WITH MOST /FFLOADING THE PRIMARY GOAL OF THIS
OPTIMIZATION IS TO REDUCE THE AMOUNT OF TRAFFIC BETWEEN THE STORAGE AND DATABASE TIERS
#OMPRESSION$ECOMPRESSION
/NE %XADATA FEATURE THAT HAS RECEIVED QUITE A BIT OF ATTENTION IS (YBRID #OLUMNAR #OMPRESSION (##
%XADATA OFFLOADS THE DECOMPRESSION OF DATA STORED IN (## FORMAT DURING 3MART 3CAN OPERATIONS 4HAT IS
COLUMNS OF INTEREST ARE DECOMPRESSED ON THE STORAGE CELLS WHEN THE COMPRESSED DATA IS ACCESSED VIA
3MART 3CANS 4HIS DECOMPRESSION IS NOT NECESSARY FOR FILTERING SO ONLY THE DATA THAT WILL BE RETURNED TO
THE DATABASE TIER WILL BE DECOMPRESSED .OTE THAT ALL COMPRESSION IS CURRENTLY DONE AT THE DATABASE TIER
HOWEVER $ECOMPRESSION MAY ALSO BE DONE AT THE DATABASE TIER WHEN DATA IS NOT ACCESSED VIA A 3MART
3CAN 3O TO MAKE IT SIMPLE 4ABLE
SHOWS WHERE THE WORK IS DONE
Table 2-1. HCC Compression/Decompression Offloading
Operation Database Servers Storage Servers
#OMPRESSION !LWA YS .EVER
$ECOMPRESSION .ON 3MART 3CAN 3MART 3CAN
$ECOMPRESSING DATA AT THE STORAGE TIER RUNS COUNTER TO THE THEME OF MOST OF THE OTHER 3MART 3CAN
OPTIMIZATIONS -OST OF THEM ARE GEARED TO REDUCING THE VOLUME OF DATA TO BE TRANSPORTED BACK TO THE
DATABASE SERVERS "ECAUSE DECOMPRESSION IS SUCH A #05 INTENSIVE TASK PARTICULARLY WITH THE HIGHER
LEVELS OF COMPRESSION THE DECISION WAS MADE TO DO THE DECOMPRESSION ON THE STORAGE SERVERS 4HIS
DECISION IS NOT BE LOCKED IN STONE HOWEVER AS IN SOME SITUATIONS THERE MAY BE AMPLE #05 RESOURCES
AVAILABLE TO MAKE DECOMPRESSING DATA ON THE DATABASE SERVERS AN ATTRACTIVE OPTION THAT IS IN SOME
SITUATIONS THE REDUCTION IN DATA TO BE SHIPPED MAY OUTWEIGH THE REDUCTION IN DATABASE SERVER #05
CONSUMPTION )N FACT AS OF CELLSRV VERSION %XADATA DOES HAVE THE ABILITY TO RETURN COMPRESSED
DATA TO THE DATABASE SERVERS WHEN THE STORAGE CELLS ARE BUSY
#(!04%2 /&&,/!$).' 3-!24 3#!.
+EVIN 3AYS 4HE AUTHORS ARE CORRECT TO SURMISE THAT THE DELINEATION OF RESPONSIBILITY BETWEEN THE DATABASE GRID
AND THE STORAGE GRID ARE QUITE FLUIDAT LEAST FROM AN ARCHITECTURAL STANDPOINT !LLOW ME TO EXPLAIN $URING 3MART
3CAN PROCESSING %XADATA 3TORAGE 3ERVER SOFTWARE PERFORMS OFFLOAD PROCESSING SUCH AS FILTRATION PROJECTION AND
DECOMPRESSION OF %(## DATA ON -" CHUNKS OF DATA )N TRUE 3MART 3CAN FORM THE PRODUCT OF 3MART 3CAN FILTERED
AND PROJECTED DATA FLOWS OVER I$" INTO THE 0'! OF THE REQUESTING PROCESS IN THE DATABASE GRID (OWEVER THERE
HAVE ALWAYS BEEN CONDITIONS WHERE 3MART 3CAN IS FORCED TO HALT INTELLIGENT PROCESSING WITHIN ONE OF THESE -"
CHUNKS AND RETURN DATA IN ORIGINAL BLOCK FORM /NE SUCH CASE IS WHEN 3MART 3CAN ENCOUNTERS A CHAINED ROW
DURING FILTRATION /NLY THE DATABASE GRID CAN DETERMINE THE LOCATION OF THE BLOCK THAT CONTAINS THE CHAINED ROW
%VEN IF CELLS WERE GIVEN THE INTELLIGENCE TO DETERMINE THE LOCALE OF THE CHAINED ROW IT WOULD MOST LIKELY NOT BE ON
THE SAME CELL AND CELLS HAVE NO DIRECT COMMUNICATION PATHS AMONG THEMSELVES ANYWAY 3O WITH THIS EXAMPLE IN
MIND IT IS EASY TO SEE THAT 3MART 3CAN CAN REVERT TO A BLOCK SERVER WHEN IT needs TO &OR THAT MATTER 3MART 3CAN
CAN REVERT TO A BLOCK SERVER WHEN IT wants TO ) EXPECT THE SHELF
LIFE OF THIS BOOK TO OUTLIVE SOME OF THE RIGID
DESCRIPTIONS WEVE GIVEN REGARDING ROLES AND THE DELINEATION OF RESPONSIBILITIES BETWEEN THE DATABASE GRID AND THE
STORAGE GRID OF THE %XADATA $ATABASE -ACHINE #ONSIDER A SCENARIO WHERE THE CELLS ARE SERVICING A MODERATELY
SELECTIVE QUERY OF MINIMAL COMPLEXITY FOR EXAMPLE FEW OR NO JOINS AND LIGHT AGGREGATION AND SORTING AGAINST
DEEPLY COMPRESSED DATA 'IVEN THESE PARAMETERS THE PROCESSOR UTILIZATION WOULD SKEW HEAVILY TOWARD THE STORAGE
GRID AS THE COST OF FILTERING PROJECTING AND DECOMPRESSING THE DATA IS MUCH GREATER THAN THE EFFORT BEING
EXPENDED IN THE DATABASE GRID 7E KNOW 3MART 3CAN CAN REVERT TO A BLOCK SERVER WHEN IT needs TO 4HE SCENARIO )
JUST DESCRIBED MAY INDEED BE A CASE WHEN 3MART 3CAN WOULD want TO REVERT TO BLOCK SERVER FOR AT LEAST SOME OF ITS
PAYLOAD )NDEED ITS BETTER NOT TO BROKER WORK TO FULLY SATURATED PROCESSORS IF THERE ARE OTHER PROCESSORS NEAR IDLE
"Y THE WAY THERE IS A HIDDEN PARAMETER THAT CONTROLS WHETHER DECOMPRESSION WILL BE OFFLOADED AT
ALL 5NFORTUNATELY IT DOESNT JUST MOVE THE DECOMPRESSION BACK AND FORTH BETWEEN THE STORAGE AND
DATABASE TIERS )F THE _CELL_OFFLOAD_HYBRIDCOLUMNAR PARAMETER IS SET TO A VALUE OF FALSE 3MART 3CANS WILL
BE COMPLETELY DISABLED ON (## DATA
%NCRYPTION$ECRYPTION
%NCRYPTION AND DECRYPTION ARE HANDLED IN A MANNER VERY SIMILAR TO COMPRESSION AND DECOMPRESSION OF
(## DATA %NCRYPTION IS ALWAYS DONE AT THE DATABASE TIER WHILE DECRYPTION CAN BE DONE BY THE STORAGE
SERVERS OR BY THE DATABASE SERVERS 7HEN ENCRYPTED DATA IS ACCESSED VIA 3MART 3CAN IT IS DECRYPTED ON
THE STORAGE SERVERS /THERWISE IT IS DECRYPTED ON THE DATABASE SERVERS .OTE THAT THE 8
AND X
PLATFORMS BOTH USE )NTEL 8EON 7ESTMERE CHIPS IN THE STORAGE SERVERS 8
USES THE SAME CHIPS IN THE
DATABASE SERVERS BY THE WAY 4HESE CHIPS CONTAIN A SPECIAL INSTRUCTION SET )NTEL !%3
.) THAT EFFECTIVELY
ADDS A HARDWARE BOOST TO PROCESSES DOING ENCRYPTION OR DECRYPTION .OTE THAT /RACLE $ATABASE 2ELEASE
IS NECESSARY TO TAKE ADVANTAGE OF THE NEW INSTRUCTION SET
%NCRYPTION AND (## COMPRESSION WORK WELL TOGETHER 3INCE COMPRESSION IS DONE FIRST THERE IS LESS
WORK NEEDED FOR PROCESSES DOING ENCRYPTION AND DECRYPTION ON (## DATA .OTE THAT THE
#(!04%2 /&&,/!$).' 3-!24 3#!.
CELL_OFFLOAD_DECRYPTION PARAMETER CONTROLS THIS BEHAVIOR AND THAT AS IT DOES WITH THE HIDDEN
PARAMETER _CELL_OFFLOAD_HYBRIDCOLUMNAR SETTING THE PARAMETER TO A VALUE OF FALSE COMPLETELY DISABLES
3MART 3CANS ON ENCRYPTED DATA WHICH ALSO DISABLES DECRYPTION AT THE STORAGE LAYER
6IRTUAL #OLUMNS
Virtual columns PROVIDE THE ABILITY TO DEFINE PSEUDO
COLUMNS THAT CAN BE CALCULATED FROM OTHER COLUMNS
IN A TABLE WITHOUT ACTUALLY STORING THE CALCULATED VALUE 6IRTUAL COLUMNS MAY BE USED AS PARTITION KEYS
USED IN CONSTRAINTS OR INDEXED #OLUMN LEVEL STATISTICS CAN ALSO BE GATHERED ON THEM 3INCE THE VALUES OF
VIRTUAL COLUMNS ARE NOT ACTUALLY STORED THEY MUST BE CALCULATED ON THE FLY WHEN THEY ARE ACCESSED 4HESE
CALCULATIONS CAN BE OFFLOADED WHEN ACCESS IS VIA 3MART 3CANS
SYS@SANDBOX1> alter table kso.temp_skew add col1_plus_pk as (col1+pk_col);
Table altered.
SYS@SANDBOX1> select col1_plus_pk from kso.temp_skew where rownum < 10;
COL1_PLUS_PK
------------
27998260
27998258
27998256
27998254
27998252
27998250
27998248
27998246
27998244
9 rows selected.
SYS@SANDBOX1> select count(*) from kso.temp_skew where col1_plus_pk=27998244;
COUNT(*)
----------
2
SYS@SANDBOX> @fsx4
Enter value for sql_text: select count(*) from kso.temp_skew where col1_plus_pk=27998244
Enter value for sql_id:
SQL_ID CHILD OFFLOAD AVG_ETIME IO_SAVED_% SQL_TEXT
------------- ------ ------- ------------- ---------- --------------------
35tqjjq5vzg4b 0 Yes 1.14 99.99 select count(*) from
1 row selected.
SYS@SANDBOX1> @dplan
Enter value for sql_id: 35tqjjq5vzg4b
#(!04%2 /&&,/!$).' 3-!24 3#!.
Enter value for child_no:
PLAN_TABLE_OUTPUT
---------------------------------------------------------------------------------------
SQL_ID 35tqjjq5vzg4b, child number 0
-------------------------------------
select count(*) from kso.temp_skew where col1_plus_pk=27998244
Plan hash value: 725706675
----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
----------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 44804 (100)| |
| 1 | SORT AGGREGATE | | 1 | 13 | | |
|* 2 | TABLE ACCESS STORAGE FULL| TEMP_SKEW | 320K| 4062K| 44804 (2)| 00:08:58 |
----------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - storage("COL1"+"PK_COL"=27998244)
filter("COL1"+"PK_COL"=27998244)
20 rows selected.
SYS@SANDBOX1> alter session set "_cell_offload_virtual_columns"=false;
Session altered.
SYS@SANDBOX1> @flush_sql
Enter value for sql_id: 35tqjjq5vzg4b
PL/SQL procedure successfully completed.
SYS@SANDBOX1> select count(*) from kso.temp_skew where col1_plus_pk=27998244;
COUNT(*)
----------
2
1 row selected.
SYS@SANDBOX1> @fsx4
Enter value for sql_text: select count(*) from kso.temp_skew where col1_plus_pk=27998244
Enter value for sql_id:
Enter value for inst_id:
SQL_ID CHILD OFFLOAD AVG_ETIME IO_SAVED_% SQL_TEXT
------------- ------ ------- ------------- ---------- --------------------
#(!04%2 /&&,/!$).' 3-!24 3#!.
35tqjjq5vzg4b 0 Yes 3.00 59.79 select count(*) from
1 row selected.
SYS@SANDBOX1> alter session set "_cell_offload_virtual_columns"=true;
Session altered.
SYS@SANDBOX1> select count(*) from kso.temp_skew where col1_plus_pk=27998244;
COUNT(*)
----------
2
1 row selected.
SYS@SANDBOX1> @fsx4
Enter value for sql_text: select count(*) from kso.temp_skew where col1_plus_pk=27998244
Enter value for sql_id:
Enter value for inst_id:
SQL_ID CHILD OFFLOAD AVG_ETIME IO_SAVED_% SQL_TEXT
------------- ------ ------- ------------- ---------- --------------------
35tqjjq5vzg4b 0 Yes 3.00 59.79 select count(*) from
35tqjjq5vzg4b 0 Yes 1.19 99.99 select count(*) from
2 rows selected.
4HIS EXAMPLE SHOWS THAT VIRTUAL COLUMN EVALUATION CAN BE OFFLOADED )T ALSO SHOWS THAT THE
OPTIMIZATION CAN BE CONTROLLED USING THE _CELL_OFFLOAD_VIRTUAL_COLUMNS PARAMETER .OTE THAT STORAGE
INDEXES ARE NOT BUILT ON VIRTUAL COLUMNS !S WITH &UNCTION /FFLOADING THE REAL ADVANTAGE OF OFFLOADING
VIRTUAL COLUMN CALCULATIONS HAS MORE TO DO WITH REDUCING THE VOLUME OF DATA RETURNED TO THE DATABASE
SERVERS THAN WITH REDUCING #05 USAGE ON THE DATABASE TIER
$ATA -INING -ODEL 3CORING
3OME OF THE DATA MODEL SCORING FUNCTIONS CAN BE OFFLOADED 'ENERALLY SPEAKING THIS OPTIMIZATION IS
AIMED AT REDUCING THE AMOUNT OF DATA TRANSFERRED TO THE DATABASE TIER AS OPPOSED TO PURE #05
OFFLOADING !S WITH OTHER FUNCTION /FFLOADING YOU CAN VERIFY WHICH DATA MINING FUNCTIONS CAN BE
OFFLOADED BY QUERYING V$SQLFN_METADATA 4HE OUTPUT LOOKS LIKE THIS
SYS@SANDBOX> select distinct name, version, offloadable
2 from V$SQLFN_METADATA
3 where name like 'PREDICT%'
4 order by 1,2;
NAME VERSION OFFLOADABLE
------------------------------ ------------ -----------
PREDICTION V10R2 Oracle YES
PREDICTION_BOUNDS V11R1 Oracle NO
PREDICTION_COST V10R2 Oracle YES
PREDICTION_DETAILS V10R2 Oracle NO
#(!04%2 /&&,/!$).' 3-!24 3#!.
PREDICTION_PROBABILITY V10R2 Oracle YES
PREDICTION_SET V10R2 Oracle NO
6 rows selected.
!S YOU CAN SEE SOME OF THE FUNCTIONS ARE OFFLOADABLE AND SOME ARE NOT 4HE ONES THAT ARE
OFFLOADABLE CAN BE USED BY THE STORAGE CELLS FOR 0REDICATE &ILTERING (ERES AN EXAMPLE QUERY THAT SHOULD
ONLY RETURN RECORDS THAT MEET THE SCORING REQUIREMENT SPECIFIED IN THE WHERE CLAUSE
select cust_id
from customers
where region = 'US'
and prediction_probability(churnmod,'Y' using *) > 0.8;
4HIS OPTIMIZATION IS DESIGNED TO OFFLOAD #05 USAGE AS WELL AS REDUCE THE VOLUME OF DATA TRANSFERRED
(OWEVER IT IS MOST BENEFICIAL IN SITUATIONS WHERE IT CAN REDUCE THE DATA RETURNED TO THE DATABASE TIER
SUCH AS IN THE PREVIOUS EXAMPLE
.ON
3MART 3CAN /FFLOADING
4HERE ARE A FEW OPTIMIZATIONS THAT ARE NOT RELATED TO QUERY PROCESSING !S THESE ARE NOT THE FOCUS OF THIS
CHAPTER WE WILL ONLY TOUCH ON THEM BRIEFLY
Smart File Creation
4HIS OPTIMIZATION HAS A SOMEWHAT MISLEADING NAME )T REALLY IS AN OPTIMIZATION DESIGNED TO SPEED UP
BLOCK INITIALIZATION 7HENEVER BLOCKS ARE ALLOCATED THE DATABASE MUST INITIALIZE THEM 4HIS ACTIVITY
HAPPENS WHEN TABLESPACES ARE CREATED BUT IT ALSO OCCURS WHEN FILES ARE EXTENDED FOR ANY NUMBER OF
OTHER REASONS /N NON %XADATA STORAGE THESE SITUATIONS REQUIRE THE DATABASE SERVER TO FORMAT EACH BLOCK
AND THEN WRITE THEM BACK TO DISK !LL THAT READING AND WRITING CAUSES A LOT OF TRAFFIC BETWEEN THE DATABASE
SERVERS AND THE STORAGE CELLS !S YOU ARE NOW AWARE ELIMINATING TRAFFIC BETWEEN THE LAYERS IS A PRIMARY
GOAL OF %XADATA 3O AS YOU MIGHT IMAGINE THIS TOTALLY UNNECESSARY TRAFFIC HAS BEEN ELIMINATED "LOCKS ARE
FORMATTED BY THE STORAGE CELLS THEMSELVES WITHOUT HAVING TO SEND THEM TO THE DATABASE SERVERS 4IME
SPENT WAITING ON THIS ACTIVITY IS RECORDED BY THE 3MART &ILE #REATION WAIT EVENT 4HIS WAIT EVENT AND THE
OPERATIONS THAT INVOKE IT ARE COVERED IN MORE DETAIL IN #HAPTER
RMAN Incremental Backups
%XADATA SPEEDS UP INCREMENTAL BACKUPS BY INCREASING THE GRANULARITY OF BLOCK CHANGE TRACKING /N NON
%XADATA PLATFORMS BLOCK CHANGES ARE TRACKED FOR GROUPS OF BLOCKS ON %XADATA CHANGES ARE TRACKED FOR
INDIVIDUAL BLOCKS 4HIS CAN SIGNIFICANTLY DECREASE THE NUMBER OF BLOCKS THAT MUST BE BACKED UP RESULTING
IN SMALLER BACKUP SIZES LESS )/ BANDWIDTH AND REDUCED TIME TO COMPLETE INCREMENTAL BACKUPS 4HIS
FEATURE CAN BE DISABLED BY SETTING THE _DISABLE_CELL_OPTIMIZED_BACKUPS PARAMETER TO A VALUE OF TRUE
4HIS OPTIMIZATION IS COVERED IN #HAPTER
RMAN Restores
4HIS OPTIMIZATION SPEEDS UP THE FILE INITIALIZATION PORTION WHEN RESTORING FROM BACKUP ON A CELL
!LTHOUGH RESTORING DATABASES FROM BACKUPS IS RARE THIS OPTIMIZATION CAN ALSO HELP SPEED UP CLONING OF
#(!04%2 /&&,/!$).' 3-!24 3#!.
ENVIRONMENTS 4HE OPTIMIZATION REDUCES #05 USAGE ON THE DATABASE SERVERS AND REDUCES TRAFFIC BETWEEN
THE TWO TIERS )F THE _CELL_FAST_FILE_RESTORE PARAMETER IS SET TO A VALUE OF FALSE THIS BEHAVIOR WILL BE
DISABLED 4HIS OPTIMIZATION IS ALSO COVERED IN #HAPTER
Smart Scan Prerequisites
3MART 3CANS DO NOT OCCUR FOR EVERY QUERY RUN ON %XADATA 4HERE ARE THREE BASIC REQUIREMENTS THAT MUST
BE MET FOR 3MART 3CANS TO OCCUR
v 4HERE MUST BE A FULL SCAN OF AN OBJECT
v 4HE SCAN MUST USE /RACLES $IRECT 0ATH 2EAD MECHANISM
v 4HE OBJECT MUST BE STORED ON %XADATA STORAGE
4HERE IS A SIMPLE EXPLANATION AS TO WHY THESE REQUIREMENTS EXIST /RACLE IS A # PROGRAM 4HE
FUNCTION THAT PERFORMS 3MART 3CANS kcfis_read IS CALLED BY THE DIRECT PATH READ FUNCTION kcbldrget
WHICH IS CALLED BY ONE OF THE FULL SCAN FUNCTIONS )TS THAT SIMPLE 9OU CANT GET TO THE kcfis_read FUNCTION
WITHOUT TRAVERSING THE CODE PATH FROM FULL SCAN TO DIRECT READ !ND OF COURSE THE STORAGE WILL HAVE TO BE
RUNNING /RACLES SOFTWARE IN ORDER TO PROCESS 3MART 3CANS
7ELL DISCUSS EACH OF THESE REQUIREMENTS IN TURN
&ULL 3CANS
)N ORDER FOR QUERIES TO TAKE ADVANTAGE OF %XADATAS /FFLOADING CAPABILITIES THE OPTIMIZER MUST DECIDE TO
EXECUTE A STATEMENT WITH A &ULL 4ABLE 3CAN OR A &AST &ULL )NDEX 3CAN .OTE THAT ) AM USING THESE TERMS
SOMEWHAT GENERICALLY 'ENERALLY SPEAKING THESE TERMS CORRESPOND TO TABLE ACCESS FULL AND INDEX FAST
FULL SCAN OPERATIONS OF AN EXECUTION PLAN 7ITH %XADATA THESE FAMILIAR OPERATIONS HAVE BEEN RENAMED
SLIGHTLY TO SHOW THAT THEY ARE ACCESSING %XADATA STORAGE 4HE NEW OPERATION NAMES ARE TABLE ACCESS
STORAGE FULL AND INDEX STORAGE FAST FULL SCAN .OTE THAT THERE ARE ALSO SOME MINOR VARIATIONS OF
THESE OPERATIONS SUCH AS MAT_VIEW ACCESS STORAGE FULL THAT ALSO QUALIFY FOR 3MART 3CANS 9OU SHOULD
HOWEVER BE AWARE THAT THE FACT THAT YOUR EXECUTION PLAN SHOWS A TABLE ACCESS STORAGE FULL OPERATION
DOES NOT MEAN THAT YOUR QUERY WAS PERFORMED WITH A 3MART 3CAN )T MERELY MEANS THAT THIS PREREQUISITE
HAS BEEN SATISFIED 7ELL DISCUSS HOW TO VERIFY WHETHER A STATEMENT WAS ACTUALLY /FFLOADED VIA A 3MART
3CAN A LITTLE LATER IN THIS CHAPTER
$IRECT 0ATH 2EADS
)N ADDITION TO REQUIRING &ULL 3CAN OPERATIONS 3MART 3CANS ALSO REQUIRE THAT THE READ OPERATIONS BE
EXECUTED VIA /RACLES $IRECT 0ATH 2EAD MECHANISM $IRECT 0ATH 2EADS HAVE BEEN AROUND FOR A LONG TIME
4RADITIONALLY THIS READ MECHANISM HAS BEEN USED BY SLAVE PROCESSES WHICH SERVICE PARALLEL QUERIES
"ECAUSE PARALLEL QUERIES WERE ORIGINALLY EXPECTED TO BE USED FOR ACCESSING VERY LARGE AMOUNTS OF DATA
TYPICALLY MUCH TOO LARGE TO FIT IN THE /RACLE BUFFER CACHE IT WAS DECIDED THAT THE PARALLEL SLAVES SHOULD
READ DATA DIRECTLY INTO THEIR OWN MEMORY ALSO KNOWN AS THE PROGRAM GLOBAL AREA OR 0'! 4HE $IRECT
0ATH 2EAD MECHANISM COMPLETELY BYPASSES THE STANDARD /RACLE CACHING MECHANISM OF PLACING BLOCKS IN
THE BUFFER CACHE 4HIS WAS A VERY GOOD THING FOR VERY LARGE DATA SETS AS IT ELIMINATED EXTRA WORK THAT WAS
EXPECTED TO NOT BE HELPFUL CACHING FULL TABLE SCAN DATA THAT WOULD PROBABLY NOT BE REUSED AND KEPT
THEM FROM FLUSHING OTHER DATA OUT OF THE CACHE !ND AS WE PREVIOUSLY MENTIONED THE kcfis KERNEL FILE
INTELLIGENT STORAGE FUNCTIONS ARE BURIED UNDER THE kcbldrget KERNEL BLOCK DIRECT READ GET FUNCTION
4HEREFORE 3MART 3CANS CAN ONLY BE PERFORMED IF THE DIRECT PATH READ MECHANISM IS BEING USED
#(!04%2 /&&,/!$).' 3-!24 3#!.
)N ADDITION TO PARALLEL SLAVES DIRECT PATH READS ARE POSSIBLE FOR NON PARALLEL 31, STATEMENTS WHEN
CONDITIONS ARE RIGHT 4HERE IS A HIDDEN PARAMETER _SERIAL_DIRECT_READ WHICH CONTROLS THIS FEATURE
7HEN THIS PARAMETER IS SET TO ITS DEFAULT VALUE AUTO /RACLE AUTOMATICALLY DETERMINES WHETHER TO USE
DIRECT PATH READS FOR NON PARALLEL SCANS 4HE CALCULATION IS BASED ON SEVERAL FACTORS INCLUDING THE SIZE OF
THE OBJECT THE SIZE OF THE BUFFER CACHE AND HOW MANY OF THE OBJECTS BLOCKS ARE ALREADY IN THE BUFFER
CACHE 4HERE IS ALSO A HIDDEN PARAMETER _SMALL_TABLE_THRESHOLD WHICH PLAYS A ROLE IN DETERMINING
HOW BIG A TABLE MUST BE BEFORE IT WILL BE CONSIDERED FOR SERIAL DIRECT PATH READS 4HE ALGORITHM FOR
DETERMINING WHETHER TO USE THE DIRECT PATH READ MECHANISM ON NON PARALLEL SCANS IS NOT PUBLISHED
7HILE THE ABILITY TO DO SERIAL DIRECT PATH READS HAS BEEN AROUND FOR SOME TIME IT HAS ONLY RECENTLY
BECOME A RELATIVELY COMMON OCCURRENCE /RACLE $ATABASE G2 HAS A MODIFIED VERSION OF THE
CALCULATIONS USED TO DETERMINE WHETHER TO USE DIRECT PATH READS FOR NON PARALLEL SCANS 4HE NEW
MODIFICATIONS TO THE ALGORITHM MAKE THE DIRECT PATH READ MECHANISM MUCH MORE LIKELY TO OCCUR THAN IT
WAS IN PREVIOUS VERSIONS 4HIS WAS PROBABLY DONE AS A RESULT OF %XADATAS 3MART 3CAN OPTIMIZATIONS AND
THE DESIRE FOR THEM TO BE TRIGGERED WHENEVER POSSIBLE 4HE ALGORITHM MAY BE SOMEWHAT OVERLY AGGRESSIVE
ON NON %XADATA PLATFORMS
■ .OTE -Y /RACLE 3UPPORT .OTE CONTAINS THE FOLLOWING STATEMENT
There have been changes in 11g in the heuristics to choose between direct path reads or reads through buffer
cache for serial table scans. In 10g, serial table scans for "large" tables used to go through cache (by default)
which is not the case anymore. In 11g, this decision to read via direct path or through cache is based on the size
of the table, buffer cache size and various other stats. Direct path reads are faster than scattered reads and have
less impact on other processes because they avoid latches.
%XADATA 3TORAGE
/F COURSE THE DATA BEING SCANNED MUST BE STORED ON %XADATA STORAGE IN ORDER FOR 3MART 3CANS TO OCCUR )T
IS POSSIBLE TO CREATE !3- DISK GROUPS THAT ACCESS NON
%XADATA STORAGE ON %XADATA DATABASE SERVERS !ND
OF COURSE IT MAKES SENSE THAT ANY 31, STATEMENTS ACCESSING OBJECTS DEFINED USING THESE NON
%XADATA
DISKGROUPS WILL NOT BE ELIGIBLE FOR /FFLOADING 7HILE IT IS UNUSUAL IT IS ALSO POSSIBLE TO CREATE !3-
DISKGROUPS USING A COMBINATION OF %XADATA AND NON
%XADATA STORAGE 4HIS MIGHT BE DONE TO FACILITATE A
MIGRATION VIA AN !3- REBALANCE FOR EXAMPLE 1UERIES AGAINST OBJECTS WHOSE SEGMENTS RESIDE ON THESE
MIXED STORAGE DISKGROUPS ARE ALSO NOT ELIGIBLE FOR /FFLOADING 4HERE IS ACTUALLY AN ATTRIBUTE ASSIGNED TO
!3- DISK GROUPS cell.smart_scan_capable THAT SPECIFIES WHETHER A DISK GROUP IS CAPABLE OF PROCESSING
3MART 3CANS 4HIS ATTRIBUTE MUST BE SET TO FALSE BEFORE NON
%XADATA STORAGE CAN BE ASSIGNED TO AN !3-
DISK GROUP (ERES A LISTING SHOWING THE PROCESS OF CREATING A MIXED STORAGE DISK GROUP AND THE EFFECTS ON
QUERIES AGAINST A TABLE WITH SEGMENTS STORED IN THAT DISK GROUP .OTE THAT WE ARE JUMPING AHEAD IN THIS
EXAMPLE SO SOME OF THE COMMANDS MAY NOT MAKE A LOT OF SENSE YET $ONT WORRY WELL COVER THE DETAILS
IN DUE TIME &OR NOW WE JUST WANT YOU TO SEE THAT YOU CAN ACCESS NON
%XADATA STORAGE BUT THAT IT WILL
DISABLE THE %XADATA STORAGE
BASED OPTIMIZATIONS
SYS@+ASM> -- Add non-Exadata storage
SYS@+ASM> alter diskgroup SMITHERS add failgroup LOCAL disk '/dev/raw/raw5','/dev/raw/raw6';
#(!04%2 /&&,/!$).' 3-!24 3#!.
alter diskgroup SMITHERS add failgroup LOCAL disk '/dev/raw/raw5','/dev/raw/raw6'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15285: disk '/dev/raw/raw5' violates disk group attribute cell.smart_scan_capable
ORA-15285: disk '/dev/raw/raw6' violates disk group attribute cell.smart_scan_capable
SYS@+ASM> alter diskgroup smithers set attribute 'cell.smart_scan_capable' = 'FALSE';
Diskgroup altered.
SYS@+ASM> alter diskgroup SMITHERS add failgroup LOCAL disk '/dev/raw/raw5','/dev/raw/raw6';
Diskgroup altered.
SYS@+ASM> select name, total_mb from v$asm_diskgroup where state='MOUNTED'
NAME TOTAL_MB
-------------------------------------------------- --------
SMITHERS 512,000
SMITHERS_LOCAL 1,562
SYS@+ASM> select g.name "diskgroup", d.path "disk", d.failgroup "failgroup", d.total_mb "disk
size" from v$asm_diskgroup g, v$asm_disk d where g.group_number=d.group_number and
g.state='MOUNTED'
diskgroup disk failgroup disk size
--------------- ------------------------------------- ----------------------- ---------
SMITHERS /dev/raw/raw5 LOCAL 102,400
SMITHERS /dev/raw/raw6 LOCAL 102,400
SMITHERS o/192.168.12.3/SMITHERS_CD_05_cell01 ENKCEL01 102,400
SMITHERS o/192.168.12.4/SMITHERS_CD_05_cell02 ENKCEL02 102,400
SMITHERS o/192.168.12.5/SMITHERS_CD_05_cell03 ENKCEL03 102,400
SMITHERS_LOCAL /dev/raw/raw1 SMITHERS_LOCAL_0000 781
SMITHERS_LOCAL /dev/raw/raw2 SMITHERS_LOCAL_0001 781
7 rows selected.
7E STARTED OUT WITH A DISKGROUP ON %XADATA STORAGE THAT CONTAINED A TABLE COMPRESSED WITH (##
7HEN WE INITIALLY TRIED TO ADD THE NON
%XADATA STORAGE TO THAT DISK GROUP WE GOT AN ERROR SAYING WE HAD
VIOLATED THE cell.smart_scan_capable DISK GROUP ATTRIBUTE /NCE WE CHANGED THE ATTRIBUTE SETTING IT TO
FALSE WE WERE ABLE TO ADD THE NON
%XADATA STORAGE OF COURSE CHANGING THIS SETTING DISABLES 3MART
3CANS ON ANY OBJECT WITH SEGMENTS STORED IN THIS DISK GROUP 7E THEN LOGGED ONTO THE DATABASE AND
TRIED TO ACCESS OUR COMPRESSED TABLE
SYS@SMITHERS> select table_name, compression, compress_for
2 from dba_tables where owner='ACOLVIN';
#(!04%2 /&&,/!$).' 3-!24 3#!.
TABLE_NAME COMPRESS COMPRESS_FOR
------------------------------ -------- ------------
SKEW3 ENABLED QUERY HIGH
SYS@SMITHERS> @table_size
Enter value for owner: ACOLVIN
Enter value for table_name: SKEW3
Enter value for type: TABLE
OWNER SEGMENT_NAME TYPE TOTALSIZE_MEGS TS
-------------------- ------------------------------ ------------------ -------------- -----
ACOLVIN SKEW3 TABLE 1,020.0 USERS
--------------
sum 1,020.0
SYS@SMITHERS> select count(*) from acolvin.skew3 where col1<0;
select count(*) from acolvin.skew3 where col1<0
*
ERROR at line 1:
ORA-64307: hybrid columnar compression is only supported in tablespaces residing on Exadata«
storage
SYS@SMITHERS> alter table acolvin.skew3 move nocompress;
Table altered.
SYS@SMITHERS> select /*+ parallel (a 8) */ count(*) from acolvin.skew3 a;
COUNT(*)
----------
384000048
Elapsed: 00:03:24.64
SYS@SMITHERS> @fsx4
Enter value for sql_text: select /*+ parallel (a 8) */ count(*) from acolvin.skew3 a
Enter value for sql_id:
SQL_ID CHILD OFFLOAD IO_SAVED_% AVG_ETIME SQL_TEXT
------------- ------ ------- ---------- ------------- --------------------
5y9jm9pfbrg7q 0 No .00 204.58 select /*+ parallel
1 row selected.
SYS@SMITHERS> @table_size
Enter value for owner: ACOLVIN
Enter value for table_name: SKEW3
Enter value for type: TABLE
OWNER SEGMENT_NAME TYPE TOTALSIZE_MEGS TS
-------------------- ------------------------------ ------------------ -------------- -----
ACOLVIN SKEW3 TABLE 13,814.0 USERS
--------------
#(!04%2 /&&,/!$).' 3-!24 3#!.
sum 13,814.0
1 row selected.
3O ADDING THE NON
%XADATA STORAGE ALSO DISABLED (## 7E HAD TO DECOMPRESS THE TABLE SIMPLY TO
ACCESS IT /NCE THIS WAS DONE WE EXECUTED A PARALLEL QUERY ON THE TABLE AND AS YOU CAN SEE THE QUERY
WAS NOT /FFLOADED 3O ALL THAT WAS TO SHOW THAT YOU NEED %XADATA STORAGE TO DO 3MART 3CANS
Smart Scan Disablers
4HERE ARE SITUATIONS WHERE 3MART 3CANS ARE EFFECTIVELY DISABLED 4HE SIMPLE CASE IS WHERE THEY HAVE NOT
BEEN ENABLED IN THE CODE YET AND SO 3MART 3CANS DONT HAPPEN AT ALL 4HERE ARE OTHER CASES WHERE
/RACLE STARTS DOWN THE 3MART 3CAN PATH BUT THE STORAGE SOFTWARE EITHER DECIDES OR IS FORCED TO REVERT TO
BLOCK SHIPPING MODE 'ENERALLY THIS DECISION IS MADE ON A BLOCK BY BLOCK BASIS
3IMPLY 5NAVAILABLE
$URING THE DISCUSSION OF 3MART 3CAN OPTIMIZATIONS WE HAVE COVERED THE PREREQUISITES THAT MUST BE MET
TO ENABLE 3MART 3CANS (OWEVER EVEN WHEN THOSE CONDITIONS ARE MET THERE ARE CIRCUMSTANCES THAT
PREVENT 3MART 3CANS (ERE ARE A FEW OTHER SITUATIONS THAT ARE NOT RELATED TO SPECIFIC OPTIMIZATIONS BUT
WHERE 3MART 3CANS SIMPLY CANNOT BE USED AT LEAST AS OF cellsrv VERSION
v 3MART 3CANS CANNOT BE USED ON CLUSTERED TABLES
v 3MART 3CANS CANNOT BE USED ON )NDEX /RGANIZED 4ABLES )/4S
v 3MART 3CANS CANNOT BE USED ON TABLES WITH ROWDEPENDENCIES ENABLED
2EVERTING TO "LOCK 3HIPPING
4HERE ARE SITUATIONS WHERE 3MART 3CANS ARE USED BUT FOR VARIOUS REASONS cellsrv REVERTS TO BLOCK
SHIPPING MODE 4HIS IS A VERY COMPLEX TOPIC AND WE STRUGGLED WITH WHETHER TO INCLUDE IT IN AN
INTRODUCTORY CHAPTER ON OFFLOADING "UT IT IS A FUNDAMENTAL CONCEPT AND SO IN THE END WE DECIDED TO
DISCUSS IT HERE
7EVE DESCRIBED HOW 3MART 3CANS AVOID TRANSFERRING LARGE AMOUNTS OF DATA TO THE DATABASE LAYER BY
RETURNING PREFILTERED DATA DIRECTLY TO THE 0'! 4HE KEY CONCEPT TO UNDERSTAND HERE IS THAT 3MART 3CANS
CAN CHOOSE OR BE FORCED TO RETURN COMPLETE BLOCKS TO THE 3'! "ASICALLY ANY SITUATION THAT WOULD CAUSE
/RACLE TO HAVE TO READ ANOTHER BLOCK TO COMPLETE A RECORD WILL CAUSE THIS TO HAPPEN ! CHAINED ROW IS
PERHAPS THE SIMPLEST EXAMPLE 7HEN /RACLE ENCOUNTERS A CHAINED ROW THE HEAD PIECE WILL CONTAIN A
POINTER TO THE BLOCK CONTAINING THE SECOND ROW PIECE 3INCE THE STORAGE CELLS DO NOT COMMUNICATE
DIRECTLY WITH EACH OTHER AND IT IS UNLIKELY THAT THE CHAINED BLOCK RESIDES ON THE SAME STORAGE CELL
cellsrv SIMPLY SHIPS THE ENTIRE BLOCK AND ALLOWS THE DATABASE LAYER TO DEAL WITH IT
3O IN THIS VERY SIMPLE CASE THE 3MART 3CAN IS PAUSED MOMENTARILY AND A SINGLE BLOCK READ IS
EFFECTIVELY PERFORMED WHICH MOTIVATES ANOTHER SINGLE BLOCK READ TO GET THE ADDITIONAL ROW PIECE +EEP IN
MIND THAT THIS IS A VERY SIMPLE CASE 4HIS SAME BEHAVIOR COMES INTO PLAY WHEN /RACLE MUST DEAL WITH
READ CONSISTENCY ISSUES &OR EXAMPLE IF /RACLE NOTICES THAT A BLOCK IS hNEWERv THAN THE CURRENT QUERY THE
PROCESS OF FINDING AN AGE
APPROPRIATE VERSION OF THE BLOCK IS LEFT FOR THE DATABASE LAYER TO DEAL WITH 4HIS
EFFECTIVELY PAUSES THE 3MART 3CAN PROCESSING WHILE THE DATABASE DOES ITS TRADITIONAL READ CONSISTENCY
PROCESSING
3O IS THIS REALLY IMPORTANT AND WHY SHOULD YOU CARE 4HE ANSWER OF COURSE IS THAT IT DEPENDS )N
MOST CASES YOU PROBABLY WONT CARE /RACLE GUARANTEES THAT READS WILL BE CONSISTENT EVEN WHEN DOING
#(!04%2 /&&,/!$).' 3-!24 3#!.
3MART 3CANS 4HIS IS AN IMPORTANT POINT BY THE WAY 4HE FACT THAT /RACLE BEHAVES EXACTLY THE SAME FROM
THE APPLICATION STANDPOINT REGARDLESS OF WHETHER 3MART 3CANS ARE USED OR NOT IS A BIG DEAL 4HE FACT THAT
/RACLE MAY DO SOME SINGLE BLOCK READS ALONG WITH ITS 3MART 3CAN IS OF LITTLE CONCERN IF THE RESULTS ARE
CORRECT AND THE PERFORMANCE IS NOT SEVERELY IMPACTED AND IN MOST CASES IT WONT BE 4HERE ARE CASES
THOUGH WHERE CHOOSING TO DO A 3MART 3CAN AND THEN REVERTING TO BLOCK SHIPPING MODE CAN BE PAINFUL
FROM A PERFORMANCE STANDPOINT 4HESE ARE THE CASES WHERE ITS IMPORTANT TO UNDERSTAND WHATS GOING ON
UNDER THE COVERS 9OULL FIND MORE INFORMATION ON THIS ISSUE IN #HAPTER
3KIPPING 3OME /FFLOADING
!NOTHER VERY COMPLEX BEHAVIOR THAT WE WILL ONLY MENTION BRIEFLY IS THE ABILITY OF CELLSRV TO REFUSE TO DO
SOME OF THE NORMAL OFFLOAD PROCESSING 4HIS CAN BE DONE TO AVOID OVERLOADING THE #05 RESOURCES ON THE
STORAGE CELLS FOR EXAMPLE ! GOOD EXAMPLE OF THIS BEHAVIOR OCCURS WHEN DECOMPRESSING (## DATA
$ECOMPRESSION IS AN EXTREMELY #05 INTENSIVE TASK ESPECIALLY FOR THE HIGHER LEVELS OF COMPRESSION )N
LATER VERSIONS OF THE STORAGE SOFTWARE AND LATER CELLSRV CAN CHOOSE TO SKIP THE DECOMPRESSION
STEP ON SOME PORTION OF THE DATA WHEN THE #05S ON THE STORAGE CELLS ARE VERY BUSY AND THE DATABASE
HOSTS #05 ARE NOT VERY BUSY 4HIS EFFECTIVELY MOVES SOME OF THE WORKLOAD BACK TO THE DATABASE TIER BY
FORCING THE DATABASE HOSTS TO DO THE DECOMPRESSION )N THIS CASE SOME STEPS SUCH AS PROJECTION MAY STILL
BE DONE ON THE STORAGE CELLS DESPITE THE FACT THAT THE DECOMPRESSION STEP IS SKIPPED !S +EVIN HAS
ALREADY POINTED OUT THIS SELECTIVE DETERMINATION OF WHICH OFFLOAD PROCESSING TO DO CAN MAKE THE
BOUNDARIES BETWEEN THE TIERS VERY FLUID AND DIFFICULT TO MONITOR #HAPTER COVERS SOME TECHNIQUES FOR
MONITORING THIS BEHAVIOR
How to Verify That Smart Scan is Happening
/NE OF THE MOST IMPORTANT THINGS YOU CAN LEARN ABOUT %XADATA IS HOW TO IDENTIFY WHETHER A QUERY HAS
BEEN ABLE TO TAKE ADVANTAGE OF 3MART 3CANS 4HIS IS NOT AS EASY AS IT SOUNDS 5NFORTUNATELY THE NORMAL
EXECUTION PLAN OUTPUT PRODUCED BY THE DBMS_XPLAN PACKAGE WILL NOT SHOW YOU WHETHER A 3MART 3CAN WAS
USED OR NOT (ERES AN EXAMPLE
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------
SQL_ID 05cq2hb1r37tr, child number 0
-------------------------------------
select avg(pk_col) from kso.skew a where col1 > 0
Plan hash value: 568322376
-----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 44486 (100)| |
| 1 | SORT AGGREGATE | | 1 | 11 | | |
|* 2 | TABLE ACCESS STORAGE FULL| SKEW | 32M| 335M| 44486 (1)| 00:08:54 |
-----------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
#(!04%2 /&&,/!$).' 3-!24 3#!.
2 - storage("COL1">0)
filter("COL1">0)
.OTICE THAT THE OPTIMIZER CHOSE A TABLE ACCESS STORAGE FULL OPERATION AND THAT THE PREDICATE
SECTION SHOWS A storage() PREDICATE ASSOCIATED WITH STEP OF THE PLAN "OTH OF THESE CHARACTERISTICS
INDICATE THAT A 3MART 3CAN WAS POSSIBLE BUT NEITHER PROVIDES A DEFINITIVE VERIFICATION )N FACT THE
STATEMENT IN THIS LISTING WAS not EXECUTED WITH A 3MART 3CAN
■ .OTE !N INTERESTING FEATURE OF THE PLAN OUTPUT IS WORTH MENTIONING .OTICE THAT IN THE PREDICATE SECTION THERE IS A
storage() CLAUSE AND THERE IS A MATCHING filter() CLAUSE THAT BOTH PERFORM THE SAME COMPARISON 7E
SCRATCHED OUR HEADS ABOUT THIS FOR A WHILE WONDERING WHETHER THESE CLAUSES REPRESENTED SEPARATE PARTS OF THE
PLAN OR WHETHER IT WAS JUST A QUIRK OF THE 80,!. OUTPUT 4HERE IS ACTUALLY A VERY SIMPLE EXPLANATION FOR THIS
BEHAVIOR 4HE CELLS HAVE TO FALL BACK TO REGULAR BLOCK )/ IN A NUMBER OF CASES 7HEN A 3MART 3CAN HITS A CHAINED
ROW FOR EXAMPLE THAT BLOCK MUST BE RETURNED IN ITS ENTIRETY TO THE $" LAYER 3INCE THERE IS NO SUCH THING AS A
GUARANTEED hPUREv 3MART 3CAN A FILTER OPERATION MUST BE INCLUDED IN THE PLAN IN ADDITION TO THE FILTERING PROVIDED
BY THE STORAGE CELLS REPRESENTED BY THE storage() CLAUSE 3O THE TWO PREDICATES ACTUALLY REPRESENT TWO DISTINCT
OPERATIONS +EEP IN MIND THOUGH THAT THEY WILL NOT OVERLAP 4HE filter() OPERATION WILL BE DONE ON ROWS RETURNED
VIA THE BLOCK SHIPPING MODE WHILE THE storage() OPERATION WILL BE PERFORMED ON THE STORAGE CELLS FOR THE ROWS
THAT CAN BE RETURNED DIRECTLY TO THE 0'! VIA THE NORMAL 3MART 3CAN MECHANISM
4HE FACT THAT EXECUTION PLANS DO NOT SHOW WHETHER A 3MART 3CAN WAS PERFORMED IS A BIT FRUSTRATING
(OWEVER THERE ARE SEVERAL TECHNIQUES THAT WE CAN USE TO WORK AROUND THIS ISSUE 7ELL COVER A FEW
OPTIONS IN THE NEXT SEVERAL SECTIONS
4RACE
/NE OF THE MOST STRAIGHTFORWARD WAYS TO DETERMINE WHETHER A 3MART 3CAN WAS USED IS TO ENABLE A
TRACE ON THE STATEMENT IN QUESTION 5NFORTUNATELY THIS IS A BIT CUMBERSOME AND DOESNT ALLOW YOU TO DO
ANY INVESTIGATION INTO WHAT HAS HAPPENED WITH PAST EXECUTIONS .EVERTHELESS TRACING IS A FAIRLY FOOLPROOF
WAY TO VERIFY WHETHER A 3MART 3CAN WAS USED OR NOT )F 3MART 3CAN WAS USED THERE WILL BE CELL SMART
TABLE SCAN OR CELL SMART INDEX SCAN EVENTS IN THE TRACE FILE (ERE IS AN EXCERPT FROM THE TRACE FILE
COLLECTED FOR THE PREVIOUS STATEMENT
PARSING IN CURSOR #47387827351064 len=49 dep=0 uid=0 oct=3 lid=0 tim=1297219338278533
hv=3279003447 ad='2c8743808' sqlid='05cq2hb1r37tr'
select avg(pk_col) from kso.skew a where col1 > 0
END OF STMT
PARSE #47387827351064:c=57991,e=78256,p=25,cr=199,cu=0,mis=1,r=0,dep=0,
og=1,plh=568322376,tim=12
EXEC #47387827351064:c=0,e=14,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,plh=568322376,tim=1297
WAIT #47387827351064: nam='SQL*Net message to client' ela= 2 . . .
WAIT #47387827351064: nam='cell single block physical read' ela= 487 . . .
WAIT #47387827351064: nam='cell multiblock physical read' ela= 25262 . . .
#(!04%2 /&&,/!$).' 3-!24 3#!.
*** 2011-02-08 20:42:19.106
WAIT #47387827351064: nam='cell multiblock physical read' ela= 20303 . . .
WAIT #47387827351064: nam='gc cr multi block request' ela= 493 . . .
WAIT #47387827351064: nam='gc cr multi block request' ela= 271 . . .
WAIT #47387827351064: nam='cell multiblock physical read' ela= 2550 . . .
*** 2011-02-08 20:42:20.107
WAIT #47387827351064: nam='cell multiblock physical read' ela= 3095 . . .
WAIT #47387827351064: nam='gc cr multi block request' ela= 548 . . .
WAIT #47387827351064: nam='gc cr multi block request' ela= 331 . . .
WAIT #47387827351064: nam='cell multiblock physical read' ela= 22930 . . .
.OTICE THAT THERE ARE NO 3MART 3CAN WAIT EVENTS IN THE TRACE FILE OUTPUT &OR COMPARISON HERE IS A
BRIEF EXCERPT FROM A STATEMENT USING 3MART 3CAN
PARSING IN CURSOR #2 len=32 dep=0 uid=0 oct=3 lid=0 hv=123 ad='196' sqlid='162wjnvwyybhn'
select sum(pk_col) from kso.skew
END OF STMT
PARSE #2:c=2000,e=2424,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=1,plh=568322376
EXEC #2:c=0,e=34,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,plh=568322376
WAIT #2: nam='SQL*Net message to client' ela= 3 driver id=1650815232 #bytes=1 p3=0 obj#=-1
WAIT #2: nam='ges message buffer allocation' ela= 2 pool=0 request=1 allocated=0 obj#=-1
WAIT #2: nam='KJC: Wait for msg sends to complete' ela= 10 msg=6674450368 dest|rcvr=65536
WAIT #2: nam='reliable message' ela= 1107 channel context=6712270872 channel handle=66967991
WAIT #2: nam='ges message buffer allocation' ela= 1 pool=0 request=1 allocated=0 obj#=-1
WAIT #2: nam='enq: KO - fast object checkpoint' ela= 104 name|mode=126 2=65575 0=1 obj#=-1
WAIT #2: nam='ges message buffer allocation' ela= 1 pool=0 request=1 allocated=0 obj#=-1
WAIT #2: nam='enq: KO - fast object checkpoint' ela= 103 name|mode=126 2=65575 0=2 obj#=-1
WAIT #2: nam='cell smart table scan' ela= 162 cellhash#=2133459483 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 244 cellhash#=379339958 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 181 cellhash#=3176594409 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 1285 cellhash#=2133459483 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 1327 cellhash#=379339958 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 1310 cellhash#=3176594409 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 19755 cellhash#=3176594409 p2=0 p3=0 obj#=66849
WAIT #2: nam='cell smart table scan' ela= 39 cellhash#=3176594409 p2=0 p3=0 obj#=66849
!S YOU CAN SEE THIS TRACE FILE CONTAINS SEVERAL CELL SMART TABLE SCAN WAIT EVENTS 4HERE IS NO DOUBT
THAT THIS STATEMENT WAS OFFLOADED 7E COVER %XADATA SPECIFIC AND RELATED WAIT EVENTS IN DETAIL IN
#HAPTER
0ERFORMANCE 3TATISTICS VSESSSTAT
/F COURSE WE CAN ALSO LOOK AT SOME OF THE PERFORMANCE VIEWS SUCH AS V$SESSSTAT AND
V$ACTIVE_SESSION_HISTORY 4ANEL 0ODERS 3NAPPER SCRIPT PROVIDES A GREAT WAY TO SEE WHAT WAIT EVENTS ARE
BEING GENERATED WHILE A STATEMENT IS RUNNING BUT AGAIN YOU MUST CATCH IT IN THE ACT TO VERIFY WHATS
HAPPENING !CTIVE 3ESSION (ISTORY !3( IS GOOD AS WELL BUT SINCE THE DATA IS SAMPLED THERE IS NO
GUARANTEE THAT IT WILL CATCH THE WAIT EVENT YOU ARE LOOKING FOR .EVERTHELESS PERFORMANCE STATISTICS
PROVIDE A RELIABLE SOURCE OF DATA AS LONG AS YOU CAN ACCESS THE SYSTEM DURING THE EXECUTION OF THE
STATEMENT YOU ARE INVESTIGATING (ERES AN EXAMPLE USING V$MYSTATS WHICH IS SIMPLY A VERSION OF
V$SESSSTAT THAT LIMITS DATA TO YOUR CURRENT SESSION &OR THIS EXAMPLE WELL LOOK AT THE cell scans STATISTIC
WHICH IS INCREMENTED WHEN A 3MART 4ABLE 3CAN OCCURS
#(!04%2 /&&,/!$).' 3-!24 3#!.
SYS@dbm1> set echo on
SYS@dbm1> @mystats
SYS@dbm1> select name, value
2 from v$mystat s, v$statname n
3 where n.statistic# = s.statistic#
4 and name like nvl('%&name%',name)
5 order by 1
6 /
Enter value for name: cell scans
NAME VALUE
---------------------------------------------------------------------- ---------------
cell scans 833
1 row selected.
SYS@dbm1> set echo off
SYS@dbm1> select avg(pk_col) from kso.skew2 a where col1 > 0;
AVG(PK_COL)
-----------
16093748.8
1 row selected.
SYS@dbm1> @mystats
Enter value for name: cell scan
NAME VALUE
---------------------------------------------------------------------- ---------------
cell scans 834
1 row selected.
SYS@dbm1> alter session set cell_offload_processing=false;
Session altered.
SYS@dbm1> select avg(pk_col) from kso.skew2 a where col1 > 0;
AVG(PK_COL)
-----------
16093748.8
1 row selected.
SYS@dbm1> @mystats
Enter value for name: cell scans
#(!04%2 /&&,/!$).' 3-!24 3#!.
NAME VALUE
---------------------------------------------------------------------- ---------------
cell scans 834
1 row selected.
3O AS YOU CAN SEE THE FIRST TIME THE STATEMENT WAS EXECUTED IT WAS OFFLOADED AND THE cell scans
STATISTIC WAS INCREMENTED FROM TO 7E THEN TURNED OFF 3MART 3CANS AND RAN THE STATEMENT AGAIN
4HIS TIME THE STATISTIC WAS NOT INCREMENTED 3O THIS APPROACH WORKS WELL AS LONG AS WE CAN CATCH THE
STATEMENTS OF INTEREST IN ACTION .OTE THAT /RACLE PERFORMANCE STATISTICS ARE COMPLEMENTARY TO THE
/RACLE WAIT EVENT INTERFACE AND PROVIDE INFORMATION THAT IS JUST NOT AVAILABLE ELSEWHERE 7E COVER THE
%XADATA
RELATED STATISTICS IN DETAIL IN #HAPTER
/FFLOAD %LIGIBLE "YTES
4HERE IS ANOTHER CLUE TO WHETHER A STATEMENT USED A 3MART 3CAN OR NOT !S WEVE ALREADY MENTIONED THE
V$SQL FAMILY OF VIEWS CONTAIN A COLUMN CALLED IO_CELL_OFFLOAD_ELIGIBLE_BYTES WHICH SHOWS THE
NUMBER OF BYTES THAT ARE ELIGIBLE FOR /FFLOADING 4HIS COLUMN CAN BE USED AS AN INDICATOR OF WHETHER A
STATEMENT USED A 3MART 3CAN )T APPEARS THAT THIS COLUMN IS SET TO A VALUE GREATER THAN ONLY WHEN A
3MART 3CAN IS USED 7E CAN MAKE USE OF THIS OBSERVATION TO WRITE A LITTLE SCRIPT fsx.sql THAT RETURNS A
VALUE OF 9%3 OR ./ DEPENDING ON WHETHER THAT COLUMN IN V$SQL HAS A VALUE GREATER THAN 4HE OUTPUT OF
THE SCRIPT IS A LITTLE TOO WIDE TO FIT IN A BOOK FORMAT SO WEVE USED A COUPLE OF CUT
DOWN VERSIONS IN OUR
EXAMPLES !ND OF COURSE ALL OF THE VERSIONS WILL BE AVAILABLE IN THE ONLINE CODE REPOSITORY 9OUVE ALREADY
SEEN THE SCRIPT IN ACTION IN SEVERAL OF THE PREVIOUS SECTIONS (ERES WHATS INSIDE THE SCRIPT AND AN
EXAMPLE OF ITS USE
SYS@SANDBOX1> !cat fsx.sql
----------------------------------------------------------------------------------------
--
-- File name: fsx.sql
--
-- Purpose: Find SQL and report whether it was Offloaded and % of I/O saved.
--
-- Usage: This scripts prompts for two values.
--
-- sql_text: a piece of a SQL statement like %select col1, col2 from skew%
--
-- sql_id: the sql_id of the statement if you know it (leave blank to ignore)
--
-- Description:
--
-- This script can be used to locate statements in the shared pool and
-- determine whether they have been executed via Smart Scans.
--
-- It is based on the observation that the IO_CELL_OFFLOAD_ELIGIBLE_BYTES
-- column in V$SQL is only greater than 0 when a statement is executed
-- using a Smart Scan. The IO_SAVED_% column attempts to show the ratio of
-- of data received from the storage cells to the actual amount of data
-- that would have had to be retrieved on non-Exadata storage. Note that
-- as of 11.2.0.2, there are issues calculating this value with some queries.
--
-- Note that the AVG_ETIME will not be acurate for parallel queries. The
#(!04%2 /&&,/!$).' 3-!24 3#!.
-- ELAPSED_TIME column contains the sum of all parallel slaves. So the
-- script divides the value by the number of PX slaves used which gives an
-- approximation.
--
-- Note also that if parallel slaves are spread across multiple nodes on
-- a RAC database the PX_SERVERS_EXECUTIONS column will not be set.
--
---------------------------------------------------------------------------------------
set pagesize 999
set lines 190
col sql_text format a70 trunc
col child format 99999
col execs format 9,999
col avg_etime format 99,999.99
col "IO_SAVED_%" format 999.99
col avg_px format 999
col offload for a7
select sql_id, child_number child, plan_hash_value plan_hash, executions execs,
(elapsed_time/1000000)/decode(nvl(executions,0),0,1,executions)/
decode(px_servers_executions,0,1,px_servers_executions/
decode(nvl(executions,0),0,1,executions)) avg_etime,
px_servers_executions/decode(nvl(executions,0),0,1,executions) avg_px,
decode(IO_CELL_OFFLOAD_ELIGIBLE_BYTES,0,'No','Yes') Offload,
decode(IO_CELL_OFFLOAD_ELIGIBLE_BYTES,0,0,
100*(IO_CELL_OFFLOAD_ELIGIBLE_BYTES-IO_INTERCONNECT_BYTES)
/decode(IO_CELL_OFFLOAD_ELIGIBLE_BYTES,0,1,IO_CELL_OFFLOAD_ELIGIBLE_BYTES)) "IO_SAVED_%",
sql_text
from v$sql s
where upper(sql_text) like upper(nvl('&sql_text',sql_text))
and sql_text not like 'BEGIN :sql_text := %'
and sql_text not like '%IO_CELL_OFFLOAD_ELIGIBLE_BYTES%'
and sql_id like nvl('&sql_id',sql_id)
order by 1, 2, 3
/
SYS@SANDBOX1> select avg(pk_col) from kso.skew3 where col1 < 0;
AVG(PK_COL)
-----------
1849142.5
Elapsed: 00:00:00.07
SYS@SANDBOX1> alter session set cell_offload_processing=false;
Session altered.
Elapsed: 00:00:00.00
SYS@SANDBOX1> select avg(pk_col) from kso.skew3 where col1 < 0;
#(!04%2 /&&,/!$).' 3-!24 3#!.
AVG(PK_COL)
-----------
1849142.5
Elapsed: 00:00:49.68
SYS@SANDBOX1> @fsx4
Enter value for sql_text: select avg(pk_col) from kso.skew3 where col1 < 0
Enter value for sql_id:
SQL_ID CHILD OFFLOAD IO_SAVED_% AVG_ETIME SQL_TEXT
------------- ------ ------- ---------- ---------- ---------------------------------
a6j7wgqf84jvg 0 Yes 100.00 .07 select avg(pk_col) from kso.skew3
a6j7wgqf84jvg 1 No .00 49.68 select avg(pk_col) from kso.skew3
Elapsed: 00:00:00.04
)N THE fsx SCRIPT YOU CAN SEE THAT THE OFFLOAD COLUMN IS JUST A DECODE THAT CHECKS TO SEE IF THE
IO_CELL_OFFLOAD_ELIGIBLE_BYTES COLUMN IS EQUAL TO OR NOT 4HE IO_SAVED_% COLUMN IS CALCULATED USING
THE IO_INTERCONNECT_BYTES FIELD AND IT ATTEMPTS TO SHOW HOW MUCH DATA WAS RETURNED TO THE DATABASE
SERVERS 4HE EXAMPLE SHOWS THE SAME STATEMENT RUN ONCE WITH 3MART 3CAN ENABLED AND ONCE WITH 3MART
3CAN DISABLED 7E USED THE CELL_OFFLOAD_PROCESSING PARAMETER TO TURN 3MART 3CANS ON AND OFF FOR THIS
PURPOSE #HANGING THIS PARAMETER CAUSED THE ORIGINAL CURSOR TO BE INVALIDATED BECAUSE OF AN OPTIMIZER
MISMATCH 4HIS RESULTED IN TWO CHILD CURSORS FOR THE STATEMENT )N OUR EXAMPLE THE OUTPUT OF THE SCRIPT
SHOWS THAT ONE VERSION OF THE STATEMENT WAS OFFLOADED USING 3MART 3CANS AND TOOK LESS THAN A SECOND
4HE SECOND EXECUTION OF THE STATEMENT THOUGH WAS NOT /FFLOADED AND TOOK ALMOST A MINUTE
4HE TECHNIQUE USED BY THE fsx SCRIPT SEEMS TO WORK PRETTY WELL MOST OF THE TIME (OWEVER IN
SITUATIONS WHERE A SINGLE CHILD CURSOR IS USED REPEATEDLY IT IS POSSIBLE THAT SOME EXECUTIONS MAY BE
EXECUTED USING 3MART 3CANS WHILE OTHERS ARE NOT 7HILE THIS IS AN UNUSUAL SITUATION IT CAN CAUSE
CONFUSION BECAUSE THE IO_CELL_OFFLOAD_ELIGIBLE_BYTES COLUMN CONTAINS A CUMULATIVE VALUE FOR ALL
EXECUTIONS 4HAT IS TO SAY THAT EACH EXECUTION ADDS ITS ELIGIBLE BYTES COUNT TO THE RUNNING TOTAL 7HEN
SOME EXECUTIONS USE 3MART 3CANS AND SOME DONT THE IO_CELL_OFFLOAD_ELIGIBLE_BYTES COLUMN WILL BE
GREATER THAN 4HIS IS A FAIRLY RARE OCCURRENCE AND YOU MAY NEVER RUN INTO IT .EVERTHELESS HERE IS AN
EXAMPLE
SYS@SANDBOX1> alter session set "_serial_direct_read"=true;
Session altered.
Elapsed: 00:00:00.01
SYS@SANDBOX1> -- execution 1
SYS@SANDBOX1> select avg(pk_col) from kso.skew a where col1 > 0;
AVG(PK_COL)
-----------
16093748.8
1 row selected.
Elapsed: 00:00:03.51
SYS@SANDBOX1> @fsx3
Enter value for sql_text: %skew%
Enter value for sql_id:
#(!04%2 /&&,/!$).' 3-!24 3#!.
SQL_ID CHILD OFFLOAD EXECS ELIGIBLE_BYTES SQL_TEXT
------------- ------ ------- ----- -------------- --------------------
05cq2hb1r37tr 0 Yes 1 38797312 select avg(pk_col) f
1 row selected.
Elapsed: 00:00:00.01
SYS@SANDBOX1> alter session set "_serial_direct_read"=false;
Session altered.
Elapsed: 00:00:00.00
SYS@SANDBOX1> -- execution 2
SYS@SANDBOX1> select avg(pk_col) from kso.skew a where col1 > 0;
AVG(PK_COL)
-----------
16093748.8
1 row selected.
Elapsed: 00:00:04.71
SYS@SANDBOX1> @fsx3
Enter value for sql_text: %skew%
Enter value for sql_id:
SQL_ID CHILD OFFLOAD EXECS ELIGIBLE_BYTES SQL_TEXT
------------- ------ ------- ----- -------------- --------------------
05cq2hb1r37tr 0 Yes 2 38797312 select avg(pk_col) f
1 row selected.
Elapsed: 00:00:00.01
SYS@SANDBOX1>
SYS@SANDBOX1> alter session set "_serial_direct_read"=true;
Session altered.
Elapsed: 00:00:00.01
SYS@SANDBOX1> -- execution 3
SYS@SANDBOX1> select avg(pk_col) from kso.skew a where col1 > 0;
AVG(PK_COL)
-----------
16093748.8
1 row selected.
Elapsed: 00:00:03.54
SYS@SANDBOX1> @fsx3
Enter value for sql_text: %skew%
Enter value for sql_id:
#(!04%2 /&&,/!$).' 3-!24 3#!.
SQL_ID CHILD OFFLOAD EXECS ELIGIBLE_BYTES SQL_TEXT
------------- ------ ------- ----- -------------- --------------------
05cq2hb1r37tr 0 Yes 3 58195968 select avg(pk_col) f
1 row selected.
Elapsed: 00:00:00.01
)N THIS EXAMPLE WE USED THE _SERIAL_DIRECT_READ PARAMETER TO DISABLE 3MART 3CANS 4HIS PARAMETER
DOES NOT INVALIDATE CURSORS SO THE SAME CURSOR WAS USED FOR ALL THREE EXECUTIONS OF THE STATEMENT )F YOU
LOOK AT THE SECOND EXECUTION YOU CAN PROBABLY GUESS THAT IT DIDNT USE A 3MART 3CAN AS IT WAS SLOWER
THAN THE PREVIOUS EXECUTION 9OU CAN VERIFY THAT ASSUMPTION BY NOTING THAT THE %LIGIBLE "YTE #OUNT DID
NOT INCREASE (OWEVER THE fsx SCRIPT SIMPLY CHECKS TO SEE IF THE VALUE OF THE
IO_CELL_OFFLOAD_ELIGIBLE_BYTES COLUMN IS GREATER THAN !S YOU CAN SEE THE VALUE OF THE COLUMN
REMAINED THE SAME BETWEEN EXECUTIONS CAUSING THE fsx SCRIPT TO REPORT THAT THE STATEMENT WAS
OFFLOADED EVEN THOUGH THE SECOND EXECUTION WAS NOT OFFLOADED 3O KEEP IN MIND THAT OUR INDICATOR IS A
CUMULATIVE VALUE FOR ALL THE EXECUTIONS OF THIS CURSOR .OTE THAT IN THE PREVIOUS EXAMPLE WE SET UP A
RATHER UNUSUAL SITUATION WHERE A SINGLE CURSOR IS OFFLOADED ON ONE EXECUTION AND NOT ON ANOTHER 4HIS
RARELY HAPPENS IN REAL LIFE
4HE TECHNIQUE DEMONSTRATED BY THE fsx SCRIPT PROVIDES A VERY USEFUL ALTERNATIVE TO TRACING OR USING
SESSION STATISTICS TO VERIFY WHETHER 3MART 3CANS ARE BEING PERFORMED 4HE BIGGEST ADVANTAGE IS THAT YOU
DONT HAVE TO TRY TO CATCH 3MART 3CAN IN REAL TIME 4HE IO_CELL_OFFLOAD_ELIGIBLE_BYTES COLUMN IS STORED
IN V$SQL AND RELATED VIEWS WHICH MEANS ITS ALSO CAPTURED BY !72 4HIS PROVIDES US WITH A HISTORICAL
VIEW INTO HOW STATEMENTS HAVE BEEN PROCESSED IN THE PAST )T IS ONE OF THE MAIN TOOLS THAT WE USE FOR A
QUICK
AND
DIRTY VERIFICATION OF WHETHER A 3MART 3CAN HAS BEEN USED OR NOT
31, -ONITORING
4HERE IS ONE OTHER TOOL THAT IS VERY USEFUL FOR DETERMINING WHETHER A 31, STATEMENT WAS /FFLOADED 4HE
REPORT_SQL_MONITOR PROCEDURE IS PART OF THE NEW 2EAL 4IME 31, -ONITORING FUNCTIONALITY THAT WAS
ADDED WITH G )T IS BUILT INTO THE DBMS_SQLTUNE PACKAGE AND PROVIDES A GREAT DEAL OF INFORMATION NOT
ONLY ON WHETHER A STATEMENT WAS OFFLOADED BUT ALSO ON WHICH STEPS IN A PLAN WERE OFFLOADED (ERES AN
EXAMPLE FIRST OF A STATEMENT THAT WAS NOT OFFLOADED AND THEN OF THE SAME STATEMENT WHEN IT WAS
OFFLOADED
SYS@SANDBOX1> alter session set cell_offload_processing=false;
Session altered.
SYS@SANDBOX1> set echo off
SYS@SANDBOX1> @avgskew3
AVG(PK_COL)
-----------
16093750.2
SYS@SANDBOX1> @fsx4
Enter value for sql_text: %skew3%
Enter value for sql_id:
#(!04%2 /&&,/!$).' 3-!24 3#!.
SQL_ID CHILD OFFLOAD IO_SAVED_% AVG_ETIME SQL_TEXT
------------- ------ ------- ---------- ---------- ----------------------------------------
6uutdmqr72smc 0 No .00 57.95 select /* avgskew3.sql */ avg(pk_col) fr
SYS@SANDBOX1> @report_sql_monitor
Enter value for sid:
Enter value for sql_id: 6uutdmqr72smc
Enter value for sql_exec_id:
REPORT
------------------------------------------------------------------------------------------
SQL Monitoring Report
SQL Text
------------------------------
select /* avgskew3.sql */ avg(pk_col) from kso.skew3 a where col1 > 0
Global Information
------------------------------
Status : DONE (ALL ROWS)
Instance ID : 1
Session : SYS (3:5465)
SQL ID : 6uutdmqr72smc
SQL Execution ID : 16777216
Execution Started : 03/15/2011 15:26:11
First Refresh Time : 03/15/2011 15:26:19
Last Refresh Time : 03/15/2011 15:27:09
Duration : 58s
Module/Action : sqlplus@enkdb01.enkitec.com (TNS V1-V3)/-
Service : SYS$USERS
Program : sqlplus@enkdb01.enkitec.com (TNS V1-V3)
Fetch Calls : 1
Global Stats
==========================================================================================
| Elapsed | Cpu | IO | Application | Cluster | Fetch | Buffer | Read | Read |
| Time(s) | Time(s) | Waits(s) | Waits(s) | Waits(s) | Calls | Gets | Reqs | Bytes |
==========================================================================================
| 58 | 35 | 23 | 0.00 | 0.00 | 1 | 2M | 15322 | 15GB |
==========================================================================================
#(!04%2 /&&,/!$).' 3-!24 3#!.
SQL Plan Monitoring Details (Plan Hash Value=2684249835)
======================================================================================
| Id | Operation | Name | ... | Activity | Activity Detail |
| | | | | (%) | (# samples) |
======================================================================================
| 0 | SELECT STATEMENT | | | | |
| 1 | SORT AGGREGATE | | | 41.38 | Cpu (24) |
| 2 | TABLE ACCESS STORAGE FULL | SKEW3 | | 58.62 | Cpu (11) |
| | | | | | direct path read (23) |
======================================================================================
SYS@SANDBOX1> @ss_on
SYS@SANDBOX1> alter session set cell_offload_processing=true;
Session altered.
SYS@SANDBOX1> @avgskew3
AVG(PK_COL)
-----------
16093750.2
SYS@SANDBOX1> @fsx4
Enter value for sql_text: %skew3%
Enter value for sql_id:
SQL_ID CHILD OFFLOAD IO_SAVED_% AVG_ETIME SQL_TEXT
------------- ------ ------- ---------- ---------- ----------------------------------------
6uutdmqr72smc 0 Yes 71.85 34.54 select /* avgskew3.sql */ avg(pk_col) fr
SYS@SANDBOX1> @report_sql_monitor
Enter value for sid:
Enter value for sql_id: 6uutdmqr72smc
Enter value for sql_exec_id:
REPORT
----------------------------------------------------------------------------------------
SQL Monitoring Report
SQL Text
------------------------------
select /* avgskew3.sql */ avg(pk_col) from kso.skew3 a where col1 > 0
Global Information
------------------------------
Status : DONE (ALL ROWS)
Instance ID : 1
Session : SYS (3:5467)
SQL ID : 6uutdmqr72smc
SQL Execution ID : 16777219
Execution Started : 03/15/2011 15:36:11
#(!04%2 /&&,/!$).' 3-!24 3#!.
First Refresh Time : 03/15/2011 15:36:15
Last Refresh Time : 03/15/2011 15:36:45
Duration : 34s
Module/Action : sqlplus@enkdb01.enkitec.com (TNS V1-V3)/-
Service : SYS$USERS
Program : sqlplus@enkdb01.enkitec.com (TNS V1-V3)
Fetch Calls : 1
Global Stats
=========================================================================================
| Elapsed | Cpu | IO | Application | Fetch | Buffer | Read | Read | Cell |
| Time(s) | Time(s) | Waits(s) | Waits(s) | Calls | Gets | Reqs | Bytes | Offload |
=========================================================================================
| 35 | 31 | 3.85 | 0.00 | 1 | 2M | 18422 | 15GB | 71.83% |
=========================================================================================
SQL Plan Monitoring Details (Plan Hash Value=2684249835)
============================================================================================
| Id | Operation | Name | | Cell | Activity | Activity Detail |
| | | | | Offload | (%) | (# samples) |
============================================================================================
| 0 | SELECT STATEMENT | | | | | |
| 1 | SORT AGGREGATE | | | | 50.00 | Cpu (17) |
| 2 | TABLE ACCESS STORAGE FULL | SKEW3 | | 71.83% | 50.00 | Cpu (12) |
| | | | | | | cell smart tab... (5) |
============================================================================================
.OTE THAT WE CUT OUT A NUMBER OF COLUMNS FROM THE REPORT BECAUSE IT IS VERY WIDE AND DOESNT FIT
NICELY IN BOOK FORMAT .EVERTHELESS YOU CAN SEE THAT THE REPORT SHOWS WHICH STEPS WERE /FFLOADED #ELL
/FFLOAD WHERE THE STATEMENT SPENT ITS TIME !CTIVITY AND WHAT IT SPENT TIME DOING !CTIVITY $ETAIL
4HIS CAN BE EXTREMELY USEFUL WITH MORE COMPLEX STATEMENTS THAT HAVE MULTIPLE STEPS ELIGIBLE FOR
OFFLOADING !LSO NOTE THAT MONITORING OCCURS AUTOMATICALLY ON PARALLELIZED STATEMENTS AND ON STATEMENTS
THAT THE OPTIMIZER ANTICIPATES WILL RUN FOR A LONG TIME )F /RACLE IS NOT AUTOMATICALLY CHOOSING TO MONITOR
A STATEMENT THAT IS OF INTEREST YOU CAN USE THE MONITOR HINT TO TELL /RACLE TO MONITOR THE STATEMENT
Parameters
4HERE ARE SEVERAL PARAMETERS THAT APPLY TO /FFLOADING 4HE MAIN ONE IS CELL_OFFLOAD_PROCESSING WHICH
TURNS /FFLOADING ON AND OFF 4HERE ARE SEVERAL OTHERS THAT ARE OF LESS IMPORTANCE 4ABLE
SHOWS A LIST OF
THE NON
HIDDEN PARAMETERS THAT AFFECT /FFLOADING AS OF /RACLE DATABASE VERSION .OTE THAT WE
HAVE ALSO INCLUDED THE HIDDEN PARAMETER _SERIAL_DIRECT_READ WHICH CONTROLS THIS VERY IMPORTANT
FEATURE
Table 2-2. Database Parameters Controlling Offloading
Parameter Default Description
cell_offload_compaction ADAPTIVE 4HIS PARAMETER IS RESERVED FOR FUTURE USE 4HE ONLINE
DESCRIPTION SAYS h#ELL PACKET COMPACTION STRATEGYv
#(!04%2 /&&,/!$).' 3-!24 3#!.
Parameter Default Description
cell_offload_decryption TRUE #ONTROLS WHETHER DECRYPTION IS OFFLOADED .OTE THAT WHEN THIS
PARAMETER IS SET TO FALSE 3MART 3CANS ARE COMPLETELY DISABLED
ON ENCRYPTED DATA
cell_offload_parameters 4HIS PARAMETER IS RESERVED FOR FUTURE USE
cell_offload_plan_display AUTO #ONTROLS WHETHER %XADATA OPERATION NAMES ARE USED IN
EXECUTION PLAN OUTPUT FROM 80,!. AUTO MEANS TO DISPLAY
THEM ONLY IF THE DATABASE IS USING %XADATA STORAGE
cell_offload_processing TRUE 4URNS /FFLOADING ON OR OFF
_serial_direct_read AUTO #ONTROLS THE SERIAL DIRECT PATH READ MECHANISM 4HE VALID
VALUES ARE AUTO TRUE FALSE ALWAYS AND NEVER
)N ADDITION TO THE NORMAL /RACLE APPROVED PARAMETERS THERE ARE A NUMBER OF SO CALLED HIDDEN
PARAMETERS THAT AFFECT VARIOUS ASPECTS OF /FFLOADING 4HE FOLLOWING LISTING SHOWS ALL THE CELL PARAMETERS
INCLUDING THE HIDDEN PARAMETERS ALONG WITH THEIR DESCRIPTIONS
SYS@POC1> @parmsd
Enter value for parameter: cell
Enter value for isset:
Enter value for show_hidden: Y
NAME DESCRIPTION
--------------------------------------------- ---------------------------------------------
cell_offload_compaction Cell packet compaction strategy
cell_offload_decryption enable SQL processing offload of encrypted
data to cells
cell_offload_parameters Additional cell offload parameters
cell_offload_plan_display Cell offload explain plan display
cell_offload_processing enable SQL processing offload to cells
_allow_cell_smart_scan_attr Allow checking smart_scan_capable Attr
_cell_fast_file_create Allow optimized file creation path for Cells
_cell_fast_file_restore Allow optimized rman restore for Cells
_cell_file_format_chunk_size Cell file format chunk size in MB
_cell_index_scan_enabled enable CELL processing of index FFS
_cell_offload_capabilities_enabled specifies capability table to load
_cell_offload_hybridcolumnar Query offloading of hybrid columnar
compressed tables to exadata
_cell_offload_predicate_reordering_enabled enable out-of-order SQL processing offload to
cells
_cell_offload_timezone enable timezone related SQL processing
offload to cells
#(!04%2 /&&,/!$).' 3-!24 3#!.
_cell_offload_virtual_columns enable offload of predicates on virtual
columns to cells
_cell_range_scan_enabled enable CELL processing of index range scans
_cell_storidx_mode Cell Storage Index mode
_db_check_cell_hints
_disable_cell_optimized_backups disable cell optimized backups
_kcfis_cell_passthru_enabled Do not perform smart IO filtering on the cell
_kcfis_kept_in_cellfc_enabled Enable usage of cellsrv flash cache for kept
objects
_kcfis_nonkept_in_cellfc_enabled Enable use of cellsrv flash cache for
non-kept objects
22 rows selected.
7EVE USED A FEW OF THESE PARAMETERS IN THE EXAMPLES IN THIS CHAPTER AND WELL USE SEVERAL IN THE
UPCOMING CHAPTERS 4HERE ARE SEVERAL OTHER kcfis KERNEL FILE INTELLIGENT STORAGE PARAMETERS AS WELL
(ERES A LISTING OF THEM WITH DESCRIPTIONS
SYS@SMITHERS> @parmsd
Enter value for parameter: kcfis
Enter value for isset:
Enter value for show_hidden: Y
NAME DESCRIPTION
---------------------------------------- --------------------------------------------------
_kcfis_block_dump_level Smart IO block dump level
_kcfis_caching_enabled enable kcfis intra-scan session caching
_kcfis_cell_passthru_enabled Do not perform smart IO filtering on the cell
_kcfis_control1 Kcfis control1
_kcfis_control2 Kcfis control2
_kcfis_control3 Kcfis control3
_kcfis_control4 Kcfis control4
_kcfis_control5 Kcfis control5
_kcfis_control6 Kcfis control6
_kcfis_disable_platform_decryption Don't use platform-specific decryption on the
storage cell
_kcfis_dump_corrupt_block Dump any corrupt blocks found during smart IO
_kcfis_fast_response_enabled Enable smart scan optimization for fast response
(first rows)
_kcfis_fast_response_initiosize Fast response - The size of the first IO in
logical blocks
_kcfis_fast_response_iosizemult Fast response - (next IO size = current IO size *
this parameter)
_kcfis_fast_response_threshold Fast response - the number of IOs after which
smartIO is used
#(!04%2 /&&,/!$).' 3-!24 3#!.
_kcfis_fault_control Fault Injection Control
_kcfis_io_prefetch_size Smart IO prefetch size for a cell
_kcfis_ioreqs_throttle_enabled Enable Smart IO requests throttling
_kcfis_kept_in_cellfc_enabled Enable usage of cellsrv flash cache for kept
objects
_kcfis_large_payload_enabled enable large payload to be passed to cellsrv
_kcfis_max_cached_sessions Sets the maximum number of kcfis sessions cached
_kcfis_max_out_translations Sets the maximum number of outstanding
translations in kcfis
_kcfis_nonkept_in_cellfc_enabled Enable use of cellsrv flash cache for non-kept
objects
_kcfis_oss_io_size KCFIS OSS I/O size
_kcfis_rdbms_blockio_enabled Use block IO instead of smart IO in the smart IO
module on RDBMS
_kcfis_read_buffer_limit KCFIS Read Buffer (per session) memory limit in
bytes
_kcfis_spawn_debugger Decides whether to spawn the debugger at kcfis
initialize
_kcfis_stats_level sets kcfis stats level
_kcfis_storageidx_diag_mode Debug mode for storage index on the cell
_kcfis_storageidx_disabled Don't use storage index optimization on the
storage cell
_kcfis_test_control1 kcfis tst control1
_kcfis_trace_bucket_size KCFIS tracing bucket size in bytes
_kcfis_trace_level sets kcfis tracing level
_kcfis_work_set_appliances Working Set of appliances in a KCFIS session
34 rows selected.
.OTE THAT HIDDEN PARAMETERS SHOULD NOT BE USED ON PRODUCTION SYSTEMS WITHOUT PRIOR DISCUSSION
WITH /RACLE SUPPORT "UT THEY DO PROVIDE VALUABLE CLUES ABOUT HOW SOME OF THE %XADATA FEATURES WORK
AND ARE CONTROLLED
Summary
/FFLOADING REALLY IS THE SECRET SAUCE OF %XADATA 7HILE THE HARDWARE ARCHITECTURE DOES A GOOD JOB OF
PROVIDING MORE BALANCE BETWEEN THE STORAGE LAYERS ABILITY TO DELIVER DATA AND THE DATABASE LAYERS ABILITY
TO CONSUME IT THE BULK OF THE PERFORMANCE GAINS ARE PROVIDED BY THE SOFTWARE 3MART 3CANS ARE LARGELY
RESPONSIBLE FOR THESE GAINS 4HE PRIMARY FOCUS OF MOST OF THESE OPTIMIZATIONS IS TO REDUCE THE AMOUNT OF
DATA TRANSFERRED BETWEEN THE STORAGE TIER AND THE DATABASE TIER
CHAPTER 3
Hybrid Columnar Compression
(YBRID #OLUMNAR #OMPRESSION (## IS PROBABLY ONE OF THE LEAST UNDERSTOOD OF THE FEATURES THAT ARE
UNIQUE TO %XADATA 4HE FEATURE WAS ORIGINALLY ROLLED OUT IN A BETA VERSION OF G2 AND WAS ENABLED ON
BOTH %XADATA AND NON
%XADATA PLATFORMS 4HE PRODUCTION RELEASE OF G2 RESTRICTED THE FEATURE TO
%XADATA PLATFORMS ONLY /STENSIBLY THIS DECISION WAS MADE BECAUSE THE ADDITIONAL PROCESSING POWER
AVAILABLE ON THE %XADATA STORAGE SERVERS "ECAUSE THE FEATURE IS RESTRICTED TO %XADATA THE RECENT
DOCUMENTATION REFERS TO IT AS %XADATA (YBRID #OLUMNAR #OMPRESSION %(##
4HIS CHAPTER WILL FIRST PRESENT BASIC INFORMATION ON BOTH OF /RACLES OLDER ROW
MAJOR FORMAT
COMPRESSION METHODS "!3)# AND /,40 7E WILL THEN EXPLORE IN DETAIL THE MECHANICS OF /RACLES NEW
(## FEATUREHOW IT WORKS INCLUDING HOW THE DATA IS ACTUALLY STORED ON DISK WHERE AND WHEN THE
COMPRESSION AND DECOMPRESSION OCCUR AND WHEN ITS USE IS APPROPRIATE &INALLY WELL PROVIDE SOME REAL
WORLD EXAMPLES OF (## USAGE
Oracle Storage Review
!S YOU PROBABLY ALREADY KNOW /RACLE STORES DATA IN A BLOCK STRUCTURE 4HESE BLOCKS ARE TYPICALLY + AND
FROM A SIMPLISTIC POINT OF VIEW THEY CONSIST OF A BLOCK HEADER A ROW DIRECTORY ROW DATA AND FREE SPACE
4HE BLOCK HEADER STARTS AT THE TOP OF A BLOCK AND WORKS ITS WAY DOWN WHILE THE ROW DATA STARTS AT THE
BOTTOM AND WORKS ITS WAY UP 4HE FREE SPACE GENERALLY SITS IN THE MIDDLE &IGURE SHOWS A
REPRESENTATION OF A TRADITIONAL /RACLE BLOCK FORMAT
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
#OMMON AND
6ARIABLE (EADER
4ABLE $IRECTORY
2OW $IRECTORY
&REE 3PACE
2OW $ATA
$ATABASE
"LOCK
Figure 3-1. The standard Oracle block format (row-major storage)
2OWS ARE STORED IN NO SPECIFIC ORDER BUT COLUMNS ARE GENERALLY STORED IN THE ORDER IN WHICH THEY
WERE DEFINED IN THE TABLE &OR EACH ROW IN A BLOCK THERE WILL BE A ROW HEADER FOLLOWED BY THE COLUMN
DATA FOR EACH COLUMN &IGURE SHOWS HOW THE PIECES OF A ROW ARE STORED IN A STANDARD /RACLE BLOCK
.OTE THAT WE CALLED IT A row piece BECAUSE OCCASIONALLY A ROWS DATA MAY BE STORED IN MORE THAN ONE
CHUNK )N THIS CASE THERE WILL BE A POINTER TO THE NEXT PIECE
#OMMON AND
6ARIABLE (EADER
4ABLE $IRECTORY
2OW $IRECTORY
2OW (EADER
&REE 3PACE #OLUMN $ATA
#OLUMN ,ENGTH
#OLUMN ,ENGTH
ATA
2OW $ Column
Value
$ATABASE Column
"LOCK Value
Figure 3-2. The standard Oracle row format (row-major storage)
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
.OTE THAT THE ROW HEADER MAY CONTAIN A POINTER TO ANOTHER ROW PIECE 7ELL COME BACK TO THIS A LITTLE
LATER BUT FOR NOW JUST BE AWARE THAT THERE IS A MECHANISM TO POINT TO ANOTHER LOCATION !LSO NOTE THAT
EACH COLUMN IS PRECEDED BY A SEPARATE FIELD INDICATING THE LENGTH OF THE COLUMN .OTHING IS ACTUALLY
STORED IN THE #OLUMN 6ALUE FIELD FOR NULLS 4HE PRESENCE OF A NULL COLUMN IS INDICATED BY A VALUE OF IN
THE COLUMN LENGTH FIELD 4RAILING NULLS DONT EVEN STORE THE COLUMN LENGTH FIELDS AS THE PRESENCE OF A NEW
ROW HEADER INDICATES THAT THERE ARE NO MORE COLUMNS WITH VALUES IN THE CURRENT ROW
?2C5A44 IS A KEY VALUE ASSOCIATED WITH BLOCKS IT CONTROLS HOW SPACE IS USED IN A BLOCK )TS PURPOSE IS
TO RESERVE SOME FREE SPACE IN EACH BLOCK FOR UPDATES 4HIS IS NECESSARY TO PREVENT ROW MIGRATION MOVING
ROWS TO NEW BLOCKS THAT WOULD BE CAUSED BY LACK OF SPACE IN THE ROWS ORIGINAL BLOCK WHEN A ROW
INCREASES IN SIZE 7HEN ROWS ARE EXPECTED TO BE UPDATED WITH VALUES THAT REQUIRE MORE SPACE MORE
SPACE IS GENERALLY RESERVED 7HEN ROWS ARE NOT EXPECTED TO INCREASE IN SIZE BECAUSE OF UPDATES VALUES AS
LOW AS MAY BE SPECIFIED FOR ?2C5A44 7ITH COMPRESSED BLOCKS IT IS COMMON TO USE VERY LOW VALUES OF
?2C5A44 BECAUSE THE GOAL IS TO MINIMIZE SPACE USAGE AND THE ROWS ARE GENERALLY NOT EXPECTED TO BE
UPDATED FREQUENTLY IF EVER &IGURE
SHOWS HOW FREE SPACE IS RESERVED BASED ON THE VALUE OF ?2C5A44
ACE
&REE 3P 20 % of the space in the
bolck is reserved for
updates to existing records
A
$AT $AT
AB 2OW
"LOC ASE
K
0#4
&2%
%
Figure 3-3. Block free space controlled by PCTFREE
&IGURE
SHOWS A BLOCK THAT RESERVES PERCENT OF ITS SPACE FOR UPDATES ! BLOCK WITH A ?2C5A44
SETTING OF PERCENT WOULD ALLOW INSERTS TO FILL THE BLOCK ALMOST COMPLETELY 7HEN A RECORD IS UPDATED
AND THE NEW DATA WILL NOT FIT IN THE AVAILABLE FREE SPACE OF THE BLOCK WHERE THE RECORD IS STORED THE
DATABASE WILL MOVE THE ROW TO A NEW BLOCK 4HIS PROCESS IS CALLED row migration )T DOES NOT COMPLETELY
REMOVE THE ROW FROM THE ORIGINAL BLOCK BUT LEAVES A REFERENCE TO THE NEWLY RELOCATED ROW SO THAT IT CAN
STILL BE FOUND BY ITS ORIGINAL ROWID WHICH BASICALLY CONSISTS OF A FILE NUMBER BLOCK NUMBER AND ROW
NUMBER WITHIN THE BLOCK 4HIS ABILITY OF /RACLE TO RELOCATE ROWS WILL BECOME RELEVANT WHEN WE DISCUSS
WHAT HAPPENS IF YOU UPDATE A ROW THAT HAS BEEN COMPRESSED .OTE THAT THE MORE GENERIC TERM FOR
STORING ROWS IN MORE THAN ONE PIECE IS row chaining 2OW MIGRATION IS A SPECIAL CASE OF ROW CHAINING IN
WHICH THE ENTIRE ROW IS RELOCATED &IGURE
SHOWS WHAT A BLOCK WITH A MIGRATED ROW MIGHT LOOK LIKE
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
"LOCK &LAGGED FOR
/,40 #OMPRESSION
0OINTER TO
5PDATED 2OW
e &REE 3PACE
Spac
Free
5NCOMPRESSE
D
5PDATED 2OW
Figure 3-4. Row migration
4HIS DIAGRAM REPRESENTS A SITUATION WHERE THE ENTIRE ROW HAS BEEN RELOCATED LEAVING BEHIND ONLY A
POINTER TO THE NEW LOCATION
Oracle Compression Mechanisms
/RACLE PROVIDES SEVERAL COMPRESSION MECHANISMS IN ADDITION TO (## 4HE NAMING IS SOMEWHAT
CONFUSING DUE TO THE COMMON MARKETING TERMS AND SOME CHANGES IN NOMENCLATURE ALONG THE WAY (ERE
WELL REFER TO THE THREE MAIN FLAVORS OF COMPRESSION USED BY /RACLE AS "!3)# /,40 AND (##
"!3)#
4HIS COMPRESSION METHOD IS A BASE FEATURE OF /RACLE $ATABASE G %NTERPRISE %DITION )T COMPRESSES
DATA ONLY ON DIRECT PATH LOADS -ODIFICATIONS FORCE THE DATA TO BE STORED IN AN UNCOMPRESSED FORMAT AS
DO INSERTS THAT DO NOT USE THE DIRECT PATH LOAD MECHANISM 2OWS ARE STILL STORED TOGETHER IN THE NORMAL
ROW
MAJOR FORM 4HE COMPRESSION UNIT IS A SINGLE /RACLE BLOCK "!3)# IS THE DEFAULT COMPRESSION
METHOD FROM A SYNTAX STANDPOINT &OR EXAMPLE "!3)# COMPRESSION WILL BE USED IF YOU ISSUE THE
FOLLOWING COMMAND
2A40C4C01;4q2><?A4BB*
"ASIC COMPRESSION WAS INTRODUCED IN /RACLE $ATABASE VERSION I 4HIS FORM OF COMPRESSION WAS ALSO
REFERRED TO AS $33 #OMPRESSION IN THE PAST 4HE SYNTAX 2><?A4BB5>A38A42CN;>03>?4A0C8>=B CAN STILL
BE USED TO ENABLE "!3)# COMPRESSION ALTHOUGH THIS SYNTAX HAS NOW BEEN DEPRECATED
/,40
4HE /,40 COMPRESSION METHOD ALLOWS DATA TO BE COMPRESSED FOR ALL OPERATIONS NOT JUST DIRECT PATH
LOADS )T IS PART OF AN EXTRA
COST OPTION CALLED !DVANCED #OMPRESSION AND WAS INTRODUCED IN /RACLE
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
$ATABASE VERSION G 2ELEASE 4HE STORAGE FORMAT IS ESSENTIALLY THE SAME AS "!3)# USING A SYMBOL
TABLE TO REPLACE REPEATING VALUES /,40 COMPRESSION ATTEMPTS TO ALLOW FOR FUTURE UPDATES BY LEAVING
PERCENT FREE SPACE IN EACH BLOCK VIA THE ?2C5A44 SETTING "!3)# COMPRESSION USES A ?2C5A44 VALUE OF
PERCENT 4HEREFORE TABLES COMPRESSED FOR /,40 WILL OCCUPY SLIGHTLY MORE SPACE THAN IF THEY WERE
COMPRESSED WITH "!3)# ASSUMING DIRECT PATH LOADS ONLY AND NO UPDATES 4HE SYNTAX FOR ENABLING THIS
TYPE OF COMPRESSION IS
2A40C4C01;4q2><?A4BB5>A>;C?*
!LTERNATIVELY THE SYNTAX 2><?A4BB5>A0;;>?4A0C8>=B MAY BE USED ALTHOUGH THIS SYNTAX IS NOW
DEPRECATED /,40 COMPRESSION IS IMPORTANT BECAUSE IT IS THE FALLBACK METHOD FOR TABLES THAT USE (##
COMPRESSION )N OTHER WORDS BLOCKS WILL BE STORED USING /,40 COMPRESSION IN CASES WHERE (## CANNOT
BE USED NON
DIRECT PATH LOADS FOR EXAMPLE /NE IMPORTANT CHARACTERISTIC OF /,40 COMPRESSION IS THAT
UPDATES AND NON
DIRECT PATH INSERTS ARE NOT COMPRESSED INITIALLY /NCE A BLOCK BECOMES hFULLv IT WILL BE
COMPRESSED 7ELL REVISIT THIS ISSUE IN THE PERFORMANCE SECTION OF THIS CHAPTER
(##
(## IS ONLY AVAILABLE FOR TABLES STORED ON %XADATA STORAGE !S WITH "!3)# COMPRESSION DATA WILL ONLY BE
COMPRESSED IN (## FORMAT WHEN IT IS LOADED USING DIRECT PATH LOADS #ONVENTIONAL INSERTS AND UPDATES
CAUSE RECORDS TO BE STORED IN /,40 COMPRESSED FORMAT )N THE CASE OF UPDATES ROWS ARE MIGRATED TO NEW
BLOCKS 4HESE BLOCKS ARE MARKED FOR /,40 COMPRESSION SO WHEN ONE OF THESE NEW BLOCKS IS SUFFICIENTLY
FULL IT WILL BE COMPRESSED USING THE /,40 ALGORITHM
(## PROVIDES FOUR LEVELS OF COMPRESSION AS SHOWN IN 4ABLE
.OTE THAT THE EXPECTED
COMPRESSION RATIOS ARE VERY ROUGH ESTIMATES AND THAT THE ACTUAL COMPRESSION RATIO WILL VARY GREATLY
DEPENDING ON THE DATA THAT IS BEING COMPRESSED
Table 3-1. HCC Compression Types
Compression Expected
Description
Type Compression Ratio
@D4AH;>F (## LEVEL USES THE ,:/ COMPRESSION ALGORITHM 4HIS LEVEL ¾
PROVIDES THE LOWEST COMPRESSION RATIOS BUT REQUIRES THE LEAST #05
FOR COMPRESSION AND DECOMPRESSION OPERATIONS 4HIS ALGORITHM IS
OPTIMIZED FOR MAXIMIZING SPEED RATHER THAN COMPRESSION
$ECOMPRESSION IS VERY FAST WITH THIS ALGORITHM 9OU MAY SEE SOME
DOCUMENTS REFER TO THIS LEVEL AS F0A47>DB4;>F
@D4AH7867 (## LEVEL USES THE :,)" GZIP COMPRESSION ALGORITHM 3OME ¾
DOCUMENTS REFER TO THIS LEVEL AS F0A47>DB47867
0A278E4;>F (## LEVEL USES THE :,)" GZIP COMPRESSION ALGORITHM AS WELL ¾
BUT AT A HIGHER COMPRESSION LEVEL THAN @D4AH7867 $EPENDING ON
THE DATA HOWEVER THE COMPRESSION RATIOS MAY NOT BE MUCH
HIGHER THAN WITH @D4AH7867
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
Compression Expected
Description
Type Compression Ratio
0A278E47867 (## LEVEL COMPRESSION USES "ZIP COMPRESSION 4HIS IS THE ¾
HIGHEST LEVEL OF COMPRESSION AVAILABLE BUT IS FAR AND AWAY THE MOST
#05
INTENSIVE #OMPRESSION TIMES ARE OFTEN SEVERAL TIMES SLOWER
THAN FOR LEVELS AND "UT AGAIN DEPENDING ON THE DATA THE
COMPRESSION RATIO MAY NOT BE THAT MUCH HIGHER THAN WITH 0A278E4
;>F 4HIS LEVEL IS FOR SITUATIONS WHERE THE TIME TO COMPRESS DATA IS
RELATIVELY UNIMPORTANT OR WHERE SPACE IS IN CRITICALLY SHORT SUPPLY
COMPRESSION ALGORITHMS
4HE REFERENCES TO THE SPECIFIC ALGORITHMS ASSOCIATED WITH (## COMPRESSION ,:/ :,)" OR ":)0 ARE ALL
INFERRED FROM THE FUNCTION NAMES USED IN THE /RACLE CODE 4HE >A0341D6 UTILITY PROVIDES A DIRECT INTERFACE
INTO THE /RACLE CALL STACK VIA THE B7>ACNBC02: COMMAND /PERATING SYSTEM UTILITIES SUCH AS PSTACK CAN
ALSO PROVIDE THE SAME INFORMATION !S AN EXAMPLE HERE IS A PART OF THE STACK DUMP FOR A PROCESS DOING
0A278E47867 COMPRESSION
-ZVRRQiX_!_bTdS^S^
-ZVRRQiX_!S^
-1I!NQi2^\_aTbb
-WP]S[TNR^\_aTbb
-1I!NR^\_aTbb1[^RZ
4ABLES MAY BE COMPRESSED WITH (## USING THE FOLLOWING SYNTAX
2A40C4C01;4 2><?A4BB5>A@D4AH;>F*
2A40C4C01;4 2><?A4BB5>A@D4AH7867*
2A40C4C01;4 2><?A4BB5>A0A278E4;>F*
2A40C4C01;4 2><?A4BB5>A0A278E47867*
9OU MAY ALSO CHANGE A TABLES COMPRESSION ATTRIBUTE BY USING THE 0;C4AC01;4 STATEMENT (OWEVER
THIS COMMAND HAS NO EFFECT ON EXISTING RECORDS UNLESS YOU ACTUALLY REBUILD THE SEGMENT USING THE <>E4
KEYWORD 7ITHOUT THE <>E4 KEYWORD THE 0;C4AC01;4 COMMAND MERELY NOTIFIES /RACLE THAT FUTURE DIRECT
PATH INSERTS SHOULD BE STORED USING (## "Y THE WAY YOU CAN SEE WHICH FORM OF COMPRESSION A TABLE IS
ASSIGNED IF ANY USING A QUERY LIKE THIS
BHB/B0=31>G -bT[TRc^f]TacPQ[TN]P\TR^\_aTbbNU^a
!Ua^\SQPNcPQ[Tb
"fWTaTR^\_aTbbX^],y4=01;43z
#P]SR^\_aTbbNU^a[XZT]e[U^a\PcR^\_aTbbNU^a
$^aSTaQh !*
>F=4AC01;4N=0<42><?A4BBN5>A
:B>B:4F!N10B8210B82
:B>B:4F!N722 @D4AH;>F
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
:B>B:4F!N722!@D4AH7867
:B>B:4F!N722"0A278E4;>F
:B>B:4F!N722#0A278E47867
:B>B:4F!N>;C?>;C?
%a^fbbT[TRcTS
/F COURSE THE CURRENT ASSIGNMENT MAY HAVE NOTHING TO DO WITH THE STORAGE FORMAT IN USE FOR ANY OR
ALL OF THE BLOCKS THAT STORE DATA FOR A GIVEN TABLE (ERE IS A QUICK EXAMPLE
BHB/B0=31>G -RPccPQ[TNbXiT! b`[
R^\_dcTbd\^Uc^cP[bXiTN\TVb^]aT_^ac
QaTPZ^]aT_^ac
R^[^f]TaU^aP!
R^[bTV\T]cN]P\TU^aP"
R^[bTV\T]cNch_TU^aP
R^[c^cP[bXiTN\TVbU^a(((((( (
bT[TRcb ^f]TabTV\T]cN]P\T
bd\QhcTb !# !#Pbc^cP[bXiTN\TVbR^\_aTbbNU^a
Ua^\SQPNbTV\T]cbbSQPNcPQ[Tbc
fWTaTb ^f]Ta,c ^f]Ta
P]Sc cPQ[TN]P\T,b bTV\T]cN]P\T
P]Sb ^f]Ta[XZT]e[^f]Tab ^f]Ta
P]Sc cPQ[TN]P\T[XZT]e[cPQ[TN]P\TbTV\T]cN]P\T
Va^d_Qhb ^f]TabTV\T]cN]P\TR^\_aTbbNU^a
^aSTaQh"*
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4F #% "
bd\ #% "
a^fbT[TRcTS
4[P_bTS) ) ) "
BHB/B0=31>G -P[cTacPQ[TZb^ bZTfR^\_aTbbU^a0A278E47867*
CPQ[TP[cTaTS
4[P_bTS) ) ) !
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4F #% "0A278E47867
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
bd\ #% "
a^fbT[TRcTS
4[P_bTS) ) ) !
BHB/B0=31>G - 8cbPhb0A278E47867Qdc^QeX^db[hcWTSPcPWPb]^cQTT]R^\_aTbbTS
BHB/B0=31>G -
BHB/B0=31>G -P[cTacPQ[TZb^ bZTf\^eTR^\_aTbbU^a`dTah[^f_PaP[[T["!*
CPQ[TP[cTaTS
4[P_bTS) ) ) ( %
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4F" "@D4AH;>F
bd\" "
4[P_bTS) ) ) !
9OU COULD PROBABLY GUESS THAT THE FIRST 0;C4A COMMAND WITHOUT THE <>E4 KEYWORD DIDNT REALLY DO
ANYTHING TO THE EXISTING DATA )T ONLY TOOK A FEW HUNDREDTHS OF A SECOND AFTER ALL !ND AS YOU CAN SEE
WHEN WE LOOKED AT THE SIZE OF THE TABLE IT DID NOT COMPRESS THE EXISTING DATA EVEN THOUGH THE DATA
DICTIONARY NOW SAYS THE TABLE IS IN 0A278E47867 MODE 7HEN WE ADDED THE <>E4 KEYWORD THE TABLE WAS
ACTUALLY REBUILT WITH THE NEW COMPRESSION SETTING USING DIRECT PATH INSERTS AND AS YOU CAN SEE THE DATA
HAS BEEN COMPRESSED FROM THE ORIGINAL -" TO -"
)N THIS SECTION WE BRIEFLY DESCRIBED EACH OF THE THREE TYPES OF COMPRESSION AVAILABLE IN /RACLE 3INCE
THIS CHAPTER IS FOCUSED ON (## WE WONT DISCUSS THE OTHER METHODS FURTHER EXCEPT AS THEY RELATE TO HOW
(## WORKS
HCC Mechanics
(## WORKS BY STORING DATA IN A NONTRADITIONAL FORMATNONTRADITIONAL FOR /RACLE ANYWAY $ATA STORED
USING (## STILL RESIDES IN /RACLE BLOCKS AND EACH BLOCK STILL HAS A BLOCK HEADER "UT THE DATA STORAGE HAS
BEEN REORGANIZED )N THE FIRST PLACE THE BLOCKS ARE COMBINED INTO LOGICAL STRUCTURES CALLED compression
units OR #5S ! #5 CONSISTS OF MULTIPLE /RACLE BLOCKS USUALLY ADDING UP TO + OR + &IGURE
SHOWS A LOGICAL REPRESENTATION OF HOW #5S ARE LAID OUT
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
Compression Unit
Block Header Block Header Block Header Block Header
Compression Unit
Column 4 Column 5
Header
Column 3
Column 1 Column 6
Column 5
Column 2 Column 4
Figure 3-5. Layout of an HCC Compression Unit
.OTICE THAT THE ROWS ARE NO LONGER STORED TOGETHER )NSTEAD THE DATA IS ORGANIZED BY COLUMN WITHIN
THE COMPRESSION UNIT 4HIS IS NOT A TRUE COLUMN ORIENTED STORAGE FORMAT BUT RATHER A CROSS BETWEEN
COLUMN ORIENTED AND ROW ORIENTED 2EMEMBER THAT THE SORTING IS DONE ONLY WITHIN A SINGLE #5 4HE NEXT
#5 WILL START OVER WITH #OLUMN AGAIN 4HE ADVANTAGE OF THIS FORMAT IS THAT IT ALLOWS ANY ROW TO BE READ
IN ITS ENTIRETY BY READING A SINGLE #5 7ITH A TRUE COLUMN ORIENTED STORAGE FORMAT YOU WOULD HAVE TO
PERFORM A SEPARATE READ FOR EACH COLUMN 4HE DISADVANTAGE IS THAT READING AN INDIVIDUAL RECORD WILL
REQUIRE READING A MULTI
BLOCK #5 INSTEAD OF A SINGLE BLOCK /F COURSE FULL TABLE SCANS WILL NOT SUFFER
BECAUSE ALL THE BLOCKS WILL BE READ 7ELL TALK MORE ABOUT THIS TRADE
OFF A LITTLE LATER BUT YOU SHOULD
ALREADY BE THINKING THAT THIS LIMITATION COULD MAKE (## LESS ATTRACTIVE FOR TABLES THAT NEED TO SUPPORT
LOTS OF SINGLE ROW ACCESS
4HE SORTING BY COLUMN IS ACTUALLY DONE TO IMPROVE THE EFFECTIVENESS OF THE COMPRESSION ALGORITHMS
NOT TO GET PERFORMANCE BENEFITS OF COLUMN ORIENTED STORAGE 4HIS IS WHERE THE NAME h(YBRID #OLUMNAR
#OMPRESSIONv COMES FROM AND WHY %XADATA HAS NOT BEEN MARKETED AS A COLUMN ORIENTED DATABASE 4HE
NAME IS ACTUALLY VERY DESCRIPTIVE OF HOW THE FEATURE ACTUALLY WORKS
HCC Performance
4HERE ARE THREE AREAS OF CONCERN WHEN DISCUSSING PERFORMANCE RELATED TO TABLE COMPRESSION 4HE FIRST
load performance IS HOW LONG IT TAKES TO COMPRESS THE DATA 3INCE COMPRESSION ONLY TAKES PLACE ON
DIRECT PATH LOADS THIS IS ESSENTIALLY A MEASUREMENT OF THE IMPACT OF LOADING DATA 4HE SECOND AREA OF
CONCERN query performance IS THE IMPACT OF DECOMPRESSION AND OTHER SIDE EFFECTS ON QUERIES AGAINST
THE COMPRESSED DATA 4HE THIRD AREA OF CONCERN DML performance IS THE IMPACT COMPRESSION
ALGORITHMS HAVE ON OTHER $-, ACTIVITIES SUCH AS 5PDATES AND $ELETES
,OAD 0ERFORMANCE
!S YOU MIGHT EXPECT LOAD TIME TENDS TO INCREASE WITH THE AMOUNT OF COMPRESSION APPLIED !S THE SAYING
GOES h4HERE IS NO SUCH THING AS A FREE PUPPYv 7HEN YOU COMPARE COSTS IN TERMS OF INCREASED LOAD TIME
WITH THE BENEFIT PROVIDED BY INCREASED COMPRESSION RATIO THE TWO :LIB
BASED OPTIONS @D4AH;>F AND
0A278E47867 APPEAR TO OFFER THE BEST TRADE
OFF (ERES A LISTING SHOWING THE SYNTAX FOR GENERATING
COMPRESSED VERSIONS OF A ' TABLE ALONG WITH TIMING INFORMATION
BHB/B0=31>G -/WRRNQdX[S"
BHB/B0=31>G -bTccX\X]V^]
BHB/B0=31>G -bTcTRW^^]
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"N]^]T]^[^VVX]V_PaP[[T['
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) )#! &%
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"NQPbXR]^[^VVX]V_PaP[[T['R^\_aTbb
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) )"$ (&
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"N^[c_]^[^VVX]V_PaP[[T['R^\_aTbbU^a^[c_
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) )!# $'
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"NWRR ]^[^VVX]V_PaP[[T['R^\_aTbbU^a`dTah[^f
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) )$% $&
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"NWRR!]^[^VVX]V_PaP[[T['R^\_aTbbU^a`dTahWXVW
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) )$% #(
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"NWRR"]^[^VVX]V_PaP[[T['R^\_aTbbU^aPaRWXeT[^f
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) )$" #"
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"NWRR#]^[^VVX]V_PaP[[T['R^\_aTbbU^aPaRWXeTWXVW
!PbbT[TRc _PaP[[T[P' Ua^\PR^[eX] bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) ')$$ $'
BHB/B0=31>G -bTccX\X]V^UU
BHB/B0=31>G -bTcTRW^^UU
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta)
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B
02>;E8=B:4F"C01;4 $"#& $
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
bd\ $"#& $
a^fbT[TRcTS
BHB/B0=31>G-RPcR^\_NaPcX^ b`[
R^\_dcTbd\^Uc^cP[bXiTN\TVb^]aT_^ac
QaTPZ^]aT_^ac
R^[^f]TaU^aP
R^[bTV\T]cN]P\TU^aP!
R^[bTV\T]cNch_TU^aP
R^[c^cP[bXiTN\TVbU^a(((((( (
R^[R^\_aTbbX^]NaPcX^U^a((( (
bT[TRc^f]TabTV\T]cN]P\TbTV\T]cNch_Tch_T
bd\QhcTb !# !#Pbc^cP[bXiTN\TVb
^aXVX]P[NbXiT bd\QhcTb !# !#PbR^\_aTbbX^]NaPcX^
Ua^\SQPNbTV\T]cb
fWTaT^f]Ta[XZT]e[^f]Ta^f]Ta
P]SbTV\T]cN]P\T[XZT]e[cPQ[TN]P\TbTV\T]cN]P\T
P]SbTV\T]cNch_T[XZT]e[ch_TbTV\T]cNch_T
Va^d_Qh^f]TabTV\T]cN]P\TcPQ[Tb_PRTN]P\TbTV\T]cNch_T
^aSTaQh$*
BHB/B0=31>G -/R^\_NaPcX^
4]cTaeP[dTU^a^aXVX]P[NbXiT) $"#& $
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B2><?A4BB8>=NA0C8>
:B>B:4F"N=>=4C01;4 $"& '
:B>B:4F"N>;C?C01;4 & ! & #
:B>B:4F"N10B82C01;4(%# & %
:B>B:4F"N722 C01;4"&( #
:B>B:4F"N722"C01;4""% (#$ %
:B>B:4F"N722!C01;4""% &#$ %
:B>B:4F"N722#C01;4!&# "$$ (
bd\!$ ( #
&a^fbbT[TRcTS
4HE LISTING SHOWS THE COMMANDS USED TO CREATE COMPRESSED VERSIONS OF THE B:4F" TABLE 7E ALSO
LOADED AN UNCOMPRESSED VERSION FOR A TIMING REFERENCE .OTE ALSO THAT THE B:4F" TABLE IS HIGHLY
COMPRESSIBLE DUE TO MANY REPEATING VALUES IN A SMALL NUMBER OF COLUMNS )TS A LITTLE HARD TO PICK OUT
THE INFORMATION FROM THE LISTING SO 4ABLE
SUMMARIZES THE DATA IN A MORE EASILY DIGESTED FORMAT
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
Table 3-2. HCC Load Times vs.Compression Ratios for SKEW3
Table Name Compress For Compression Ratio Load Time Load Time Ratio
B:4F"
B:4F"N=>=4
B:4F"N>;C?>;C?
B:4F"N10B8210B82
B:4F"N722 @D4AH ;>F
B:4F"N722!@D4AH 7867
B:4F"N722"0A278E4 ;>F
B:4F"N722#0A278E4 7867
3O AS YOU CAN SEE @D4AH7867 AND 0A278E4;>F COMPRESSION LEVELS RESULTED IN ALMOST EXACTLY THE
SAME COMPRESSION RATIOS FOR THIS DATASET AND TOOK ROUGHLY THE SAME AMOUNT OF TIME TO LOAD
,OADING IS DEFINITELY SLOWER WITH COMPRESSION AND FOR THIS DATASET WAS SOMEWHERE BETWEEN AND
TIMES SLOWER WITH THE EXCEPTION OF 0A278E47867 WHICH WELL COME BACK TO .OTICE THE HUGE JUMP IN
COMPRESSION BETWEEN @D4AH;>F AND @D4AH7867 7HILE THE LOAD TIME ROUGHLY DOUBLED THE COMPRESSION
RATIO IMPROVED BY A FACTOR OF &OR THIS DATASET THIS IS CLEARLY THE SWEET SPOT WHEN COMPARING LOAD TIME
TO COMPRESSION RATIO
.OW LETS TURN OUR ATTENTION TO THE 0A278E47867 COMPRESSION SETTING )N THE PREVIOUS TEST WE DID NOT
ATTEMPT TO MAXIMIZE THE LOAD TIME /UR CHOICE OF IS ACTUALLY A RATHER PEDESTRIAN SETTING FOR PARALLELISM
ON %XADATA )N ADDITION OUR PARALLEL SLAVE PROCESSES WERE LIMITED TO A SINGLE NODE VIA THE
?0A0;;4;N5>A24N;>20; PARAMETER WELL TALK MORE ABOUT THAT IN #HAPTER 3O OUR LOAD PROCESS WAS USING
A TOTAL OF EIGHT SLAVES ON A SINGLE NODE (ERES SOME OUTPUT FROM THE 5NIX c^_ COMMAND SHOWING HOW
THE SYSTEM WAS BEHAVING DURING THE LOAD
,,,5Xabc722#ad]'b[PeTb
c^_ ()$#) #d_!SPhb )" $dbTab[^PSPeTaPVT)! &( (
CPbZb)'"!c^cP[(ad]]X]V'!"b[TT_X]V bc^__TS i^\QXT
Cpu(s): 50.5%us %bh ]X#' 'XS fP WX bX bc
<T\)&# !&&$!Zc^cP[!(#($ ' ZdbTS##$"!%&!ZUaTT '!'ZQdUUTab
BfP_) %&& '$!Zc^cP[! ! (##ZdbTS #%$ ( 'ZUaTT!$ $!(!ZRPRWTS
?83DB4A?A=8E8ACA4BB7AB2?D<4<C8<42><<0=3
(## ^aPR[T!$ V'%\$$\A(( ( )! !$^aPN_ NB0=31>G
(#$ ^aPR[T!$ V'%\$$\A(( ( )! ! ^aPN_ !NB0=31>G
(#%$^aPR[T!$ V'%\$$\A(( ( )! "#^aPN_ "NB0=31>G
(#%'^aPR[T!$ V'&\$$\A(( ( )! !!^aPN_ #NB0=31>G
(#&(^aPR[T!$ V'%\$$\A(( ( )! ! ^aPN_ $NB0=31>G
($ $^aPR[T!$ V'%\$#\A(( ( )! '^aPN_ %NB0=31>G
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
($ &^aPR[T!$ V''\$ \A(( ( )!& $(^aPN_ &NB0=31>G
(# ^aPR[T!$ V'&\$#\A(( $ )! " ^aPN_ NB0=31>G
#LEARLY LOADING DATA INTO AN 0A278E47867 COMPRESSED TABLE IS A #05 INTENSIVE PROCESS "UT NOTICE
THAT WERE STILL ONLY USING ABOUT HALF THE PROCESSING POWER ON THE SINGLE $ATABASE 3ERVER !DDING MORE
PROCESSORS AND MORE SERVERS SHOULD MAKE IT GO CONSIDERABLY FASTER "Y THE WAY ITS USUALLY WORTHWHILE TO
LOOK AT THE #05 USAGE DURING THE UNCOMPRESSED LOAD FOR COMPARISON PURPOSES (ERES ANOTHER SNAPSHOT
FROM c^_ TAKEN DURING THE LOADING OF THE UNCOMPRESSED VERSION OF THE B:4F" TABLE IN THIS AND SIMILAR
SNAPSHOTS WEVE HIGHLIGHTED OUTPUT ITEMS OF PARTICULAR INTEREST
,,,=^2^\_aTbbX^];^PS
c^_ ()#%)$$d_!SPhb )!"%dbTab[^PSPeTaPVT) ! % !
CPbZb)'""c^cP[!ad]]X]V'" b[TT_X]V bc^__TS i^\QXT
Cpu(s): 22.3%us #bh ]X&$ (XS fP WX !bX bc
<T\)&# !&&$!Zc^cP[!(!&"$"!ZdbTS##&$#!! ZUaTT "&%ZQdUUTab
BfP_) %&& '$!Zc^cP[! "$"%'ZdbTS #%"%#'#ZUaTT!$ &#%&!ZRPRWTS
?83DB4A?A=8E8ACA4BB7AB2?D<4<C8<42><<0=3
$(((^aPR[T % V&!\%#\B$# ' ) # '(^aPN_ NB0=31>G
% ^aPR[T % V&!\%#\B$ $ ) # &!^aPN_ NB0=31>G
% !#^aPR[T % V%(\% \B#' $ ) " (&^aPN_ &NB0=31>G
% ! ^aPR[T % V& \% \B## % ) # %^aPN_ %NB0=31>G
% "^aPR[T % V& \%!\B#" % ) # #!^aPN_ !NB0=31>G
% #^aPR[T % V& \%!\B#! ( ) # !%^aPN_ $NB0=31>G
% &^aPR[T % V%(\% \B# ( ) # !'^aPN_ #NB0=31>G
% $^aPR[T $ V&!\%#\A"' " ) # #$^aPN_ "NB0=31>G
.OTICE THAT WHILE THE NUMBER OF ACTIVE PROCESSES IS THE SAME THE #05 USAGE IS SIGNIFICANTLY LESS
WHEN THE DATA IS NOT BEING COMPRESSED DURING THE LOAD 4HE OTHER LEVELS OF COMPRESSION USE SOMEWHAT
LESS #05 BUT ARE MUCH CLOSER TO (## THAN TO THE NONCOMPRESSED LOAD
4O SPEED UP THE 0A278E47867 LOADING WE COULD ADD MORE PROCESSES OR WE COULD ALLOW THE SLAVES TO
RUN ON MULTIPLE NODES (ERES A QUICK EXAMPLE
BHB/B0=31>G -P[cTabhbcT\bTc_PaP[[T[NU^aRTN[^RP[,UP[bT*
BhbcT\P[cTaTS
4[P_bTS) ) ) (
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTf"NWRR#]^[^VVX]V_PaP[[T["!R^\_aTbbU^aPaRWXeTWXVW
!PbbT[TRc _PaP[[T[P"! Ua^\Zb^ bZTf"P*
CPQ[TRaTPcTS
4[P_bTS) ) ") ' (%
3ETTING THE ?0A0;;4;N5>A24N;>20; PARAMETER TO &!,3% ALLOWED SLAVES TO BE SPREAD ACROSS BOTH
NODES 3ETTING THE PARALLEL DEGREE TO ALLOWED SLAVES TO RUN ON EACH OF THE TWO NODES IN OUR QUARTER
RACK TEST SYSTEM 4HIS EFFECTIVELY UTILIZED ALL THE #05 RESOURCES ON BOTH NODES (ERES ONE LAST SNAPSHOT
OF c^_ OUTPUT FROM ONE OF THE NODES DURING THE LOAD 4HE OTHER NODE SHOWED THE SAME BASIC PROFILE
DURING THE LOAD
,,,BTR^]S722#ad]"!b[PeTb
c^_ ')"!)#"d_!) !dbTab[^PSPeTaPVT) ' $ & # &
CPbZb)'%!c^cP[ (ad]]X]V'#"b[TT_X]V bc^__TS i^\QXT
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
Cpu(s): 97.3%us #bh ]X! !XS fP WX bX bc
<T\)&# !&&$!Zc^cP["$ # '%#ZdbTS"'''$'''ZUaTT (!$#'ZQdUUTab
BfP_) %&& '$!Zc^cP[ ZdbTS %&& '$!ZUaTT" %#$! 'ZRPRWTS
?83DB4A?A=8E8ACA4BB7AB2?D<4<C8<42><<0=3
! %$&^aPR[T!$ V \&!\A(( !$)! %^aPN_ NB0=31>G!
! %%"^aPR[T!$ V "\' \A(( !$) ^aPN_ #NB0=31>G!
!%#' ^aPR[T!$ V'(\$#\A(( ") & "&^aPN_ 'NB0=31>G!
!%#(%^aPR[T!$ V'(\$#\A(( ") % ''^aPN_ $NB0=31>G!
! %%&^aPR[T!$ V \&"\A(' $ !$) % (^aPN_ %NB0=31>G!
!%#'"^aPR[T!$ V'(\$"\A(' $ ") % %"^aPN_ (NB0=31>G!
!%#''^aPR[T!$ V( \$!\A(' $ ") ' & ^aPN_ NB0=31>G!
!%#'$^aPR[T!$ V( \$#\A(& ( ") # $#^aPN_ NB0=31>G!
!%#( ^aPR[T!$ V( \$#\A(& ( ") # #%^aPN_ !NB0=31>G!
! %$$^aPR[T!$ V $\& \A(& " $) " !!^aPN_ NB0=31>G!
!%#(#^aPR[T!$ V'(\$!\A(& " ") " #!^aPN_ #NB0=31>G!
! %% ^aPR[T!$ V %\&"\A($ # $) ! %$^aPN_ "NB0=31>G!
!%#(!^aPR[T!$ V'(\$#\A($ # ") ' "^aPN_ "NB0=31>G!
! %$(^aPR[T!$ V #\&(\A(# ' !$) " #!^aPN_ !NB0=31>G!
! %%(^aPR[T!$ V &\&!\A( $ $) (^aPN_ &NB0=31>G!
! %%$^aPR[T!$ V &\%&\A'% ! $) ' ' ^aPN_ $NB0=31>G!
1UERY 0ERFORMANCE
/F COURSE LOAD TIME IS NOT THE ONLY PERFORMANCE METRIC OF INTEREST 1UERY TIME IS MORE CRITICAL THAN LOAD
TIME FOR MOST SYSTEMS SINCE THE DATA IS ONLY LOADED ONCE BUT QUERIED MANY TIMES 1UERY PERFORMANCE IS
A MIXED BAG WHEN IT COMES TO COMPRESSION $EPENDING ON THE TYPE OF QUERY COMPRESSION CAN EITHER
SPEED IT UP OR SLOW IT DOWN $ECOMPRESSION CERTAINLY ADDS OVERHEAD IN THE WAY OF ADDITIONAL #05 USAGE
BUT FOR QUERIES THAT ARE BOTTLENECKED ON DISK ACCESS REDUCING THE NUMBER OF BLOCKS THAT MUST BE READ
CAN OFTEN OFFSET AND IN MANY CASES MORE THAN MAKE UP FOR THE ADDITIONAL OVERHEAD +EEP IN MIND THAT
DEPENDING ON THE ACCESS MECHANISM USED THE DECOMPRESSION CAN BE DONE ON EITHER THE STORAGE CELLS
SMART SCANS OR ON THE DATABASE NODES (ERES AN EXAMPLE OF RUNNING A #05
INTENSIVE PROCEDURE
BHB/B0=31>G -RPcVPcWTaNcPQ[TNbcPcb b`[
QTVX]
SQ\bNbcPcb VPcWTaNcPQ[TNbcPcb
^f]TacPQ[TN]P\T
STVaTT,-"!
\TcW^SN^_c,-U^aP[[R^[d\]bbXiT
*
T]S*
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta)02>;E8=
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ) ! #
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta):B>
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"N>;C?
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ) ! &$
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"N10B82
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ) ! %
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"N722
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ) # !
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"N722!
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ) # (#
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"N722"
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ) # !#
BHB/B0=31>G -/VPcWTaNcPQ[TNbcPcb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F"N722#
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) )! ""
!ND AGAIN FOR CLARITY 4ABLE
SHOWS THE TIMINGS IN A MORE READABLE FORMAT
Table 3-3. Statistics Gathering Timing for SKEW3
Table Name Compress For Compression Ratio Run Time Run Time Ratio
B:4F"
B:4F"N>;C?>;C?
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
Table Name Compress For Compression Ratio Run Time Run Time Ratio
B:4F"N10B8210B82
B:4F"N722 @D4AH ;>F
B:4F"N722!@D4AH 7867
B:4F"N722"0A278E4 ;>F
B:4F"N722#0A278E4 7867
'ATHERING STATISTICS IS A VERY #05 INTENSIVE OPERATION 3PREADING THE STAT GATHERING ACROSS SLAVE
PROCESSES PER NODE ALMOST COMPLETELY UTILIZED THE #05 RESOURCES ON THE $" SERVERS !S YOU CAN SEE THE
COMPRESSION SLOWED DOWN THE PROCESSING ENOUGH TO OUTWEIGH THE GAINS FROM THE REDUCED NUMBER OF
DATA BLOCKS THAT NEEDED TO BE READ 4HIS IS DUE TO THE #05 INTENSIVE NATURE OF THE WORK BEING DONE
(ERES A SNAPSHOT OF c^_ OUTPUT TO VERIFY THAT THE SYSTEM IS #05 BOUND
c^_ #)# )$ d_#SPhb%) & dbTab[^PSPeTaPVT) ' # %# $"
CPbZb)'# c^cP[! ad]]X]V'! b[TT_X]V bc^__TS i^\QXT
Cpu(s): 96.1%us &bh ]X! (XS fP WX "bX bc
<T\)&# !&&$!Zc^cP["##(##!#ZdbTS"($"""!'ZUaTT"#$##'ZQdUUTab
BfP_) %&& '$!Zc^cP[$%'&$%ZdbTS %! " (%ZUaTT!(!!%" !ZRPRWTS
?83DB4A?A=8E8ACA4BB7AB2?D<4<C8<42><<0=3
% !&^aPR[T!$ # "V #%\ %\A(& % ! ) ( %^aPN_ N?>2
% $#^aPR[T!$ # "V #$\ (\A(& % ! ) &$^aPN_ #N?>2
% "&^aPR[T!$ # "V ##\ "\A(% ( ! ) # ! ^aPN_ %N?>2
% !$^aPR[T!$ # "V "(\ '\A(% % ! )!# ^aPN_ N?>2
% ""^aPR[T!$ # "V #&\ (\A(% % ! )!& #^aPN_ #N?>2
% #$^aPR[T!$ # "V #$\ &\A(% ! ! )! !$^aPN_ N?>2
% "$^aPR[T!$ # "V $#\ !\A($ ( ! ) % $ ^aPN_ $N?>2
% #"^aPR[T!$ # "V "$\ %\A($ ( ! )!! ""^aPN_ (N?>2
% " ^aPR[T!$ # "V $%\ (\A($ % ! ) % &"^aPN_ "N?>2
% # ^aPR[T!$ # "V #"\ $\A($ % ! ) ' %!^aPN_ 'N?>2
% $ ^aPR[T!$ # "V $$\ ! \A($ % ! ) $#^aPN_ "N?>2
% #&^aPR[T!$ # "V # \ "\A(# ( ! ) & (!^aPN_ N?>2
% "(^aPR[T!$ # "V $$\ #\A(# % ! ) " !%^aPN_ &N?>2
% #(^aPR[T!$ # "V $%\ !#\A(" ( ! )! $^aPN_ !N?>2
% !(^aPR[T!$ # "V $&\ !%\A(" " ! ) ( ((^aPN_ !N?>2
% $%^aPR[T!$ # "V # \ \A(" " ! ) % &%^aPN_ $N?>2
.OW LETS LOOK AT A QUERY THAT IS )/ INTENSIVE 4HIS TEST USES A QUERY WITHOUT A F74A4 CLAUSE THAT
SPENDS MOST OF ITS TIME RETRIEVING DATA FROM THE STORAGE LAYER VIA CELL SMART TABLE SCANS
BHB/B0=31>G -/WRRNcTbc"
BHB/B0=31>G -bTccX\X]V^]
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\PR^[eX] bZTf"P*
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) % !
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\Zb^ bZTf"N^[c_P*
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) $ &(
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\Zb^ bZTf"NQPbXRP*
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) $ !%
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\Zb^ bZTf"NWRR P*
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) " $%
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\Zb^ bZTf"NWRR!P*
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) " "(
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\Zb^ bZTf"NWRR"P*
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) " "%
BHB/B0=31>G -bT[TRc _PaP[[T[P"! bd\_ZNR^[Ua^\Zb^ bZTf"NWRR#P*
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
BD<?:N2>;
% ' 4 $
a^fbT[TRcTS
4[P_bTS) ) ) # &'
4ABLE
SHOWS THE TIMINGS FOR THE UNQUALIFIED QUERY IN A TABULAR FORMAT
Table 3-4. Unqualified Query Timing on SKEW3
Table Name Compress For Compression Ratio Run Time Run Time Ratio
B:4F"
B:4F"N>;C?>;C?
B:4F"N10B8210B82
B:4F"N722 @D4AH ;>F
B:4F"N722!@D4AH 7867
B:4F"N722"0A278E4 ;>F
B:4F"N722#0A278E4 7867
4HE QUERY DOES NOT REQUIRE A LOT OF #05 RESOURCES ON THE $" SERVER !S A RESULT THE SAVINGS IN )/
TIME MORE THAN OFFSET THE INCREASED #05 .OTE THAT THE RATHER SIGNIFICANT #05 REQUIREMENTS OF THE
0A278E47867 DECOMPRESSION CAUSED THE ELAPSED TIME TO INCREASE FOR THIS TEST OVER THE LESS #05
INTENSIVE ALGORITHMS .EVERTHELESS IT WAS STILL FASTER THAN THE UNCOMPRESSED RUN (ERES A SNAPSHOT OF
c^_ OUTPUT WHILE ONE OF THESE QUERIES WAS RUNNING
c^_ $)"&) "d_#SPhb&) #(dbTab[^PSPeTaPVT)$ ! ! $
CPbZb)'%&c^cP[&ad]]X]V'% b[TT_X]V bc^__TS i^\QXT
Cpu(s): 25.2%us (bh ]X&" #XS $fP WX bX bc
<T\)&# !&&$!Zc^cP["$ ("'(%ZdbTS"''""'$%ZUaTT"#%$%'ZQdUUTab
BfP_) %&& '$!Zc^cP[$%'#''ZdbTS %! ""%#ZUaTT!((&%'$%ZRPRWTS
?83DB4A?A=8E8ACA4BB7AB2?D<4<C8<42><<0=3
!$# #^aPR[T % V# \" \B"" ) " &$^aPN_ NB0=31>G
!$#&#^aPR[T $ V"(\" \B"! # ) " $#^aPN_ !NB0=31>G
!$#&!^aPR[T $ V"&\!(\B!' ' ) " #$^aPN_ NB0=31>G
!$#! ^aPR[T % V"$\!'\A!& ' ) " & ^aPN_ #NB0=31>G
!$# '^aPR[T $ V"(\" \A!& # ) " &'^aPN_ "NB0=31>G
!$#"$^aPR[T $ V"'\" \B!& # ) " #%^aPN_ 'NB0=31>G
!$#&'^aPR[T $ V"'\" \B!& ) " &$^aPN_ #NB0=31>G
!$#!'^aPR[T $ V"'\" \B!# ' ) " ' ^aPN_ $NB0=31>G
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
!$#"&^aPR[T $ V"'\" \A!# $ ) " ''^aPN_ (NB0=31>G
!$# %^aPR[T $ V"(\"!\B!# ) " ' ^aPN_ !NB0=31>G
!$#" ^aPR[T $ V"(\" \A!! ' ) " ^aPN_ %NB0=31>G
!$#""^aPR[T % V# \"!\B!! ' ) ! (#^aPN_ &NB0=31>G
!$#& ^aPR[T $ V"'\" \A!! ' ) " &"^aPN_ NB0=31>G
!$#'#^aPR[T $ V"'\" \B! ! ) " !"^aPN_ $NB0=31>G
!$#&%^aPR[T $ V"&\" \B ' $ ) " " ^aPN_ "NB0=31>G
!$# ^aPR[T $ V"'\" \A " ! ) ! %&^aPN_ NB0=31>G
$-, 0ERFORMANCE
'ENERALLY SPEAKING RECORDS THAT WILL BE UPDATED SHOULD NOT BE COMPRESSED 7HEN YOU UPDATE A RECORD
IN AN (## TABLE THE RECORD WILL BE MIGRATED TO A NEW A BLOCK THAT IS FLAGGED AS AN /,40 COMPRESSED
BLOCK /F COURSE A POINTER WILL BE LEFT BEHIND SO THAT YOU CAN STILL GET TO THE RECORD VIA ITS OLD ROWID BUT
THE RECORD WILL BE ASSIGNED A NEW ROWID AS WELL 3INCE UPDATED RECORDS ARE DOWNGRADED TO /,40
COMPRESSION YOU NEED TO UNDERSTAND HOW THAT COMPRESSION MECHANISM WORKS ON UPDATES &IGURE
DEMONSTRATES HOW NON
DIRECT PATH LOADS INTO AN /,40 BLOCK ARE PROCESSED
%MPTY )NITIALLY #OMPRESSED
"LOCK 5NCOMPRESSED 0ARTIALLY
"LOCK
"LOCK #OMPRESSED #OMPRESSED
"LOCK "LOCK
Figure 3-6. The OLTP compression process for non-direct path loads
4HE PROGRESSION OF STATES MOVES FROM LEFT TO RIGHT 2OWS ARE INITIALLY LOADED IN AN UNCOMPRESSED
STATE !S THE BLOCK FILLS TO THE POINT WHERE NO MORE ROWS CAN BE INSERTED THE ROW DATA IN THE BLOCK IS
COMPRESSED 4HE BLOCK IS THEN PLACED BACK ON THE FREELIST AND IS CAPABLE OF ACCEPTING MORE
UNCOMPRESSED ROWS 4HIS MEANS THAT IN AN /,40
COMPRESSED TABLE BLOCKS CAN BE IN VARIOUS STATES OF
COMPRESSION !LL ROWS CAN BE COMPRESSED SOME ROWS CAN BE COMPRESSED OR NO ROWS CAN BE
COMPRESSED 4HIS IS EXACTLY HOW RECORDS IN (## BLOCKS BEHAVE WHEN THEY ARE UPDATED ! COUPLE OF
EXAMPLES WILL DEMONSTRATE THIS BEHAVIOR 4HE FIRST WILL SHOW HOW THE SIZE OF A TABLE CAN BALLOON WITH
UPDATES
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4F #% "
bd\ #% "
a^fbT[TRcTS
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTfNWRR"]^[^VVX]V_PaP[[T[ %R^\_aTbbU^aPaRWXeT[^f
!PbbT[TRcUa^\Zb^ bZTfP*
CPQ[TRaTPcTS
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4FN722"
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4FN722"!" 0A278E4;>F
bd\!"
a^fbT[TRcTS
BHB/B0=31>G -d_SPcT _PaP[[T[P"! Zb^ bZTfNWRR"PbTcR^[ ,R^[ *
"! #a^fbd_SPcTS
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4FN722"
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4FN722"( % 0A278E4;>F
bd\( %
a^fbT[TRcTS
BHB/B0=31>G - 2WTRZW^fcWXbR^\_PaTbc^SXaTRc_PcW[^PS>;C?cPQ[T
BHB/B0=31>G -
BHB/B0=31>G -RaTPcTcPQ[TZb^ bZTfN^[c_]^[^VVX]V_PaP[[T[ %R^\_aTbbU^a^[c_
!PbbT[TRc _PaP[[T[P % Ua^\Zb^ bZTfP*
CPQ[TRaTPcTS
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4FN>;C?
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4FN>;C?( " >;C?
bd\( "
a^fbT[TRcTS
BHB/B0=31>G -/R^\_NaPcX^
4]cTaeP[dTU^a^aXVX]P[NbXiT) #% "
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B2><?A4BB8>=NA0C8>
:B>B:4FC01;4 #% "
:B>B:4FN722"C01;4( % %
:B>B:4FN>;C?C01;4( " %
bd\"!&( #
"a^fbbT[TRcTS
4HIS OUTPUT SHOWS THAT UPDATING ALL THE RECORDS IN AN (## 0A278E4;>F TABLE EXPANDED IT TO A
SLIGHTLY LARGER FOOTPRINT THAN AN /,40
COMPRESSED VIA DIRECT PATH LOAD VERSION OF THE SAME TABLE
)NTERESTINGLY ENOUGH THE UPDATE STATEMENT SPENT MOST OF ITS TIME WAITING ON 1D554A1DBHF08CB
!PPARENTLY THERE IS NOT A SPECIFIC WAIT EVENT FOR WHEN A PROCESS IS WAITING WHILE A BLOCK IS BEING
COMPRESSED (ERE IS THE COMPLETE SET OF EVENTS THAT THE UPDATE STATEMENT WAITED ON FROM A TRACE
FILE PROCESSED WITH cZ_a^U ALONG WITH PER
SECOND SESSION STATISTICS BOTH FROM ONE OF THE PARALLEL SLAVE
PROCESSES
4[P_bTScX\TbX]R[dSTfPXcX]V^]U^[[^fX]VTeT]cb)
4eT]cfPXcTS^]CX\Tb<Pg FPXcC^cP[FPXcTS
FPXcTS
Rdab^a)_X]BfPXc^]G
?G3T`)4gTRdcX^]<bV %%& #"%' $
RT[[\d[cXQ[^RZ_WhbXRP[aTPS # !
aTb\Va)R_d`dP]cd\( #
buffer busy waits 2077 3.54 139.84
[PcRW)RPRWTQdUUTabRWPX]b! %' "#% &$
T]`)CG R^]cT]cX^]%% !& ""
T]`)7F R^]cT]cX^] #! $$ % $%
[PcRW)aTS^P[[^RPcX^]"% !$ $
V[^QP[T]`dTdTTg_P]SfPXc (
VRRdaaT]c\d[cXQ[^RZaT`dTbc%& "
:92)FPXcU^a\bVbT]Sbc^R^\_[TcT %
[^VQdUUTab_PRT!" !' ! ##
[PcRW)T]`dTdTWPbWRWPX]b!'
[PcRW)VTbaTb^daRTWPbW[Xbc! $
[^VUX[TbfXcRWR^\_[TcX^]$ " ''
T]`)51 R^]cT]cX^]% " !
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
T]`)DB R^]cT]cX^]#"(" (''! %#
3XbZUX[T^_TaPcX^]b8 > !
R^]ca^[UX[TbT`dT]cXP[aTPS&! & %#
RT[[bX]V[TQ[^RZ_WhbXRP[aTPS " $ !%
SQUX[TbX]V[TfaXcT&! % !"
:BE\PbcTafPXc#" ! $ !#
0B<UX[T\TcPSPcP^_TaPcX^] ##
ZUZ)Pbh]RSXbZ8>"% ( "$
RT[[b\PacUX[TRaTPcX^]#&&( ' $
2BB^_TaPcX^])PRcX^]
T]`)25 R^]cT]cX^]"% ( !#
R^]ca^[UX[T_PaP[[T[faXcT ' ( ($
35B[^RZWP]S[T!$! "'" %
d]S^bTV\T]cTgcT]bX^] %" !( ($
; eP[XSPcX^]!
aT[XPQ[T\TbbPVT$ $
[PcRW)a^fRPRWT^QYTRcb
VRRaVaP]c! fPh
VRRa\d[cXQ[^RZaT`dTbc#
[^VUX[TbfXcRWRWTRZ_^X]cX]R^\_[TcT%" !(# $
[PcRW)d]S^V[^QP[SPcP"
[PcRWUaTT& $ &
[PcRW)VRT[T\T]c#
fPXc[Xbc[PcRWUaTT!
[^VUX[Tbh]R
[PcRW)^QYTRc`dTdTWTPSTa^_TaPcX^]"
BcPc=P\T4eT]cb BTR
7B2>;C?B_PRTBPeX]V" #'$
7B2>;C?2^\_aTbbTS1[^RZb
7B22^\_aTbbTSBTV\T]c1[^RZ2WP]VTb'" #
BC0C7B27TP_BTV\T]c1[^RZ2WP]VTb'" #
BC0C7B2>;C?=^]2^\_aTbbXQ[T1[^RZb
BC0C7B2>;C?_^bXcXeTR^\_aTbbX^]!
7B2>;C?X][X]TR^\_aTbbX^]!
4HE SECOND EXAMPLE DEMONSTRATES THAT A ROW IS MIGRATED FROM AN (## BLOCK WHEN IT IS UPDATED
"ASICALLY WELL UPDATE A SINGLE ROW SEE THAT ITS ROWID HAS CHANGED VERIFY THAT WE CAN STILL GET TO THE
RECORD VIA ITS ORIGINAL ROWID AND CHECK TO SEE IF THE C01;454C272>=C8=D43A>F STATISTIC GETS UPDATED
WHEN WE ACCESS THE ROW VIA ITS ORIGINAL ROWID
BHB/B0=31>G -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4FN722"
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>B:4FN722" ' %0A278E4;>F
bd\ ' %
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTfNWRR"fWTaT_ZNR^[, %"%&*
2>D=C
BHB/B0=31>G -bT[TRca^fXS^[SNa^fXSa^fXS^[SNa^fXSNU^a\Pc
!Ua^\Zb^ bZTfNWRR"fWTaT_ZNR^[, %"%&*
A>F83>;3NA>F83N5>A<0C
000C21008005'dB5R(' $$(##! !!"""
BHB/B0=31>G - B^^daa^fXbX]UX[T'Q[^RZ $$(##!a^f!!"""
BHB/B0=31>G -
BHB/B0=31>G -d_SPcTZb^ bZTfNWRR"bTcR^[ ,R^[ fWTaT_ZNR^[, %"%&*
a^fd_SPcTS
BHB/B0=31>G -bT[TRca^fXS^[SNa^fXSa^fXS>;3NA>F83N5>A<0C
!Ua^\Zb^ bZTfNWRR"fWTaT_ZNR^[, %"%&*
A>F83>;3NA>F83N5>A<0C
000C2100700<6<<000& " & '#
BHB/B0=31>G - 7PCWTa^fXSWPbRWP]VTS~cWTa^f\^eTSc^UX[T&
BHB/B0=31>G -
BHB/B0=31>G - ;TcbbTTXUfTRP]bcX[[VTcc^XceXPcWT^aXVX]P[a^fXS
BHB/B0=31>G -
BHB/B0=31>G -bT[TRc_ZNR^[Ua^\Zb^ bZTfNWRR"fWTaTa^fXS,000C21008005'dB5R(*
?:N2>;
%"%&
BHB/B0=31>G - HTbfTRP]~RP]fTdbTcWT]Tfa^fXS.
BHB/B0=31>G -
BHB/B0=31>G -bT[TRc_ZNR^[Ua^\Zb^ bZTfNWRR"fWTaTa^fXS,000C2100700<6<<000*
?:N2>;
%"%&
BHB/B0=31>G - CWPcf^aZbc^^~8czbP\XVaPcTSA^f
BHB/B0=31>G - ;TczbeTaXUhfXcW{R^]cX]dTSa^f|bcPc
BHB/B0=31>G -
BHB/B0=31>G -/\hbcPcb
4]cTaeP[dTU^a]P\T)cPQ[TUTcRWR^]cX]dTSa^f
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
=0<4E0;D4
cPQ[TUTcRWR^]cX]dTSa^f!(#&
BHB/B0=31>G - bT[TRceXPcWT^aXVX]P[a^fXS
BHB/B0=31>G -
BHB/B0=31>G -bT[TRc_ZNR^[Ua^\Zb^ bZTfNWRR"fWTaTa^fXS,000C21008005'dB5R(*
?:N2>;
%"%&
BHB/B0=31>G -/\hbcPcb
4]cTaeP[dTU^a]P\T)cPQ[TUTcRWR^]cX]dTSa^f
=0<4E0;D4
cPQ[TUTcRWR^]cX]dTSa^f!(#'
BHB/B0=31>G - BcPcXbX]RaT\T]cTS~b^STUX]XcT[hP\XVaPcTSa^f
3O THE ROW HAS DEFINITELY BEEN MIGRATED .OW LETS VERIFY THAT THE MIGRATED ROW IS NOT COMPRESSED
7E CAN DO THIS BY DUMPING THE BLOCK WHERE THE NEWLY MIGRATED RECORD RESIDES "UT BEFORE WE LOOK AT THE
MIGRATED ROW LETS HAVE A LOOK AT THE ORIGINAL BLOCK
BHB/B0=31>G -RPcSd\_NQ[^RZ b`[
/UX]SNcaPRT
P[cTabhbcT\Sd\_SPcPUX[TUX[T]^Q[^RZQ[^RZ]^*
BHB/B0=31>G -/Sd\_NQ[^RZ
CA02458;4N=0<4
d P__ ^aPR[T SXPV aSQ\b bP]SQ^g B0=31>G caPRT B0=31>G N^aPN$ ( caR
4]cTaeP[dTU^aUX[T]^)&
4]cTaeP[dTU^aQ[^RZ]^)" & '#
BhbcT\P[cTaTS
.OW LETS LOOK AT THE TRACE FILE PRODUCED IN THE TRACE DIRECTORY (ERE IS AN EXCERPT FROM THE BLOCK
DUMP
1[^RZWTPSTaSd\_) g U %" R
>QYTRcXS^]1[^RZ.H
seg/obj:0x13081RbR) g T $&#S#XcR)"U[V)4ch_) 30C0
Qa]) QSQP) g U %" eTa) g ^_R)
X]R) TgU[V)
8c[GXSDQP5[PV;RZBR] 5bR
g g !U " # g TTR"'" U! ## UbR g
g ! g g UbR g
g " g g UbR g
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
QSQP) g U %" R
SPcPNQ[^RZNSd\_SPcPWTPSTaPc g!Q'#(R' " &R
,,,,,,,,,,,,,,,
cbXi) g U'
WbXi) g #
_Q[) g!Q'#(R' " &R
&%$#"!
U[PV,
]cPQ,
nrow=1
UaaT,
UbQ^, g #
UbT^, g U%
Peb_, g U#R
c^b_, g U#R
gT)_cXJ L]a^f, ^UUb,
g !)_aXJ L^UUb, g U%
Q[^RZNa^fNSd\_)
cPQ a^f / g U%
c[)"!UQ) 7 5; [Q) g RR)$
col 0: [ 4] c3 02 40 44
R^[ )J!LR !
R^[!)J L% &"%#%#&"% %#% &"%#
R^[")J&L&'%P & $ $ Q"!
R^[#)J L$(
T]SN^UNQ[^RZNSd\_
4HE BLOCK IS NOT COMPRESSED AND CONFORMS TO THE NORMAL /RACLE BLOCK FORMAT .OTICE THAT THERE IS
ONLY ONE ROW IN THE BLOCK ]a^fb, !LSO NOTICE THAT cWTSPcPN^QYTRcNXS IS INCLUDED IN THE BLOCK IN HEX
FORMAT bTV ^QY) g " ' 4HE TABLE HAS FIVE COLUMNS 4HE VALUES ARE DISPLAYED ALSO IN HEX FORMAT *UST
TO VERIFY THAT WE HAVE THE RIGHT BLOCK WE CAN TRANSLATE THE SPcPN^QYTRcNXS AND THE VALUE OF THE FIRST
COLUMN AS FOLLOWS
BHB/B0=31>G -RPc^QYNQhNWTg b`[
R^[^QYTRcN]P\TU^aP"
bT[TRc^f]Ta^QYTRcN]P\T^QYTRcNch_T
Ua^\SQPN^QYTRcb
fWTaTSPcPN^QYTRcNXS,c^N]d\QTaaT_[PRTWTgNeP[dT gGGGGGG*
BHB/B0=31>G -/^QYNQhNWTg
4]cTaeP[dTU^aWTgNeP[dT) g " '
>F=4A>1942CN=0<4>1942CNCH?4
:B>B:4FN722"C01;4
BHB/B0=31>G -STbRZb^ bZTfNWRR"
=P\T=d[[.Ch_T
?:N2>;=D<14A
2>; =D<14A
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
2>;!E0A270A!"
2>;"30C4
2>;#E0A270A!
BHB/B0=31>G -/SXb_[PhNaPf
4]cTaeP[dTU^abcaX]V)R" !# ##
4]cTaeP[dTU^ach_T)=D<14A
E0;D4
%"%&
!S YOU CAN SEE THIS IS THE RECORD THAT WE UPDATED IN THE B:4FN722" TABLE *UST AS AN ASIDE IT IS
INTERESTING TO SEE HOW THE COMPRESSED BLOCK FORMAT DIFFERS FROM THE STANDARD UNCOMPRESSED FORMAT
(ERE IS A SNIPPET FROM THE ORIGINAL BLOCK IN OUR EXAMPLE THAT IS COMPRESSED FOR 0A278E4;>F
,,,,,,,,,,,,,,,
cbXi) g U'
WbXi) g R
_Q[) g!Q$S&S$RTP&R
&%$#"!
U[PV,
]cPQ,
]a^f,
UaaT,
UbQ^, g R
UbT^, g"
Peb_, g #
c^b_, g #
a N(Xa!, g
\TRNZSQW(Xa!, g
&%$#"!
bWRUNZSQW(Xa!,
&%$#"!
U[PVN(Xa!, A Archive compression: Y
UR[bN(Xa!J L,jl
g %)_cXJ L]a^f, ^UUb,
g P)_aXJ L^UUb, g"
Q[^RZNa^fNSd\_)
cPQ a^f / g"
c[)' %UQ) 7 5 =[Q) g!RR)
]aXS) g ! &RQS#
R^[ )J' #L
Compression level: 03 (Archive Low)
;T]VcW^U2Da^f)' #
ZSiWaW) ?221;:)!BcPacB[^c)
=D<?) !
?=D<) ?>55)&(&#?A83) g ! &RQS#
?=D<) ?>55) $(( ?A83) g ! &RQS$
2DWTPSTa)
2DeTabX^]) 2D\PVXR]d\QTa) g#Q##$P"
2DRWTRZbd\) gU $!($'!
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
2Dc^cP[[T]VcW)! # %
2DU[PVb)=2 D 2A3 >?
]R^[b)$
nrows: 32759
P[V^)
2DSTR^\_[T]VcW) &!%([T] eP[dT[T]VcW) # ((&
a^f_XTRTb_Taa^f)
num deleted rows: 1
deleted rows: 0,
BC0ACN2D)
U## U ! ! U!% ! &RQS# "T&% !
&RQS$ #Q##$P" '!($$!U $"(TTQ % $&UU&
.OTICE THAT THIS BLOCK SHOWS THAT IT IS COMPRESSED AT LEVEL 0A278E4;>F !LSO NOTICE THAT ONE
RECORD HAS BEEN DELETED FROM THIS BLOCK MIGRATED WOULD BE A MORE ACCURATE TERM AS THIS IS THE RECORD
THAT WE UPDATED EARLIER 4HE LINE THAT SAYS ST[TcTSa^fb) ACTUALLY SHOWS A LIST OF THE ROWS THAT HAVE BEEN
MIGRATED 2EMEMBER THAT WE UPDATED THE RECORD WITH ROWID MEANING FILE BLOCK
SLOT 3O THIS LINE TELLS US THAT THE ONE DELETED ROW WAS IN SLOT
■ +EVIN 3AYS ) LIKE TO REMIND PEOPLE TO BE MINDFUL OF THE POTENTIAL PERFORMANCE IRREGULARITIES THAT MAY OCCUR
SHOULD A TABLE COMPRISED OF %(## DATA METAMORPHOSE INTO A TABLE THAT IS OF MIXED COMPRESSION TYPES WITH
INDIRECTION 4HERE SHOULD BE NO EXPECTED LOSS IN FUNCTIONALITY HOWEVER THERE IS AN EXPECTED YET UNPREDICTABLE
IMPACT TO THE REPEATABILITY OF SCAN PERFORMANCE 4HE UNPREDICTABLE NATURE IS DUE TO THE FACT THAT IT IS COMPLETELY
DATA
DEPENDENT
Expected Compression Ratios
(## CAN PROVIDE VERY IMPRESSIVE COMPRESSION RATIOS 4HE MARKETING MATERIAL HAS CLAIMED ¾
COMPRESSION RATIOS AND BELIEVE IT OR NOT THIS IS ACTUALLY A VERY ACHIEVABLE NUMBER FOR MANY DATASETS /F
COURSE THE AMOUNT OF COMPRESSION DEPENDS HEAVILY ON THE DATA AND WHICH OF THE FOUR ALGORITHMS IS
APPLIED 4HE BEST WAY TO DETERMINE WHAT KIND OF COMPRESSION CAN BE ACHIEVED ON YOUR DATASET IS TO TEST
IT /RACLE ALSO PROVIDES A UTILITY OFTEN REFERRED TO AS THE #OMPRESSION !DVISOR TO COMPRESS A SAMPLE OF
DATA FROM A TABLE IN ORDER TO CALCULATE AN ESTIMATED COMPRESSION RATIO 4HIS UTILITY CAN EVEN BE USED ON
NON %XADATA PLATFORMS AS LONG AS THEY ARE RUNNING OR LATER 4HIS SECTION WILL PROVIDE SOME
INSIGHT INTO THE #OMPRESSION !DVISOR AND PROVIDE COMPRESSION RATIOS ON SOME SAMPLE REAL WORLD
DATASETS
#OMPRESSION !DVISOR
)F YOU DONT HAVE ACCESS TO AN %XADATA BUT STILL WANT TO TEST THE EFFECTIVENESS OF (## YOU CAN USE THE
#OMPRESSION !DVISOR FUNCTIONALITY THAT IS PROVIDED IN THE 31<BN2><?A4BB8>= PACKAGE 4HE
64CN2><?A4BB8>=NA0C8> PROCEDURE ACTUALLY ENABLES YOU TO COMPRESS A SAMPLE OF ROWS FROM A SPECIFIED
TABLE 4HIS IS NOT AN ESTIMATE OF HOW MUCH COMPRESSION MIGHT HAPPEN THE SAMPLE ROWS ARE INSERTED
INTO A TEMPORARY TABLE 4HEN A COMPRESSED VERSION OF THAT TEMPORARY TABLE IS CREATED 4HE RATIO RETURNED
IS A COMPARISON BETWEEN THE SIZES OF THE COMPRESSED VERSION AND THE UNCOMPRESSED VERSION !S OF
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
/RACLE $ATABASE VERSION THIS PROCEDURE MAY BE USED ON A NON %XADATA PLATFORM TO ESTIMATE
COMPRESSION RATIOS FOR VARIOUS LEVELS OF (## AS WELL AS /,40 COMPRESSION 4HE !DVISOR DOES NOT WORK ON
NON %XADATA PLATFORMS RUNNING $ATABASE VERSIONS PRIOR TO ALTHOUGH THERE MAY BE A PATCH
AVAILABLE TO ENABLE THIS FUNCTIONALITY
4HE #OMPRESSION !DVISOR MAY ALSO BE USEFUL ON %XADATA PLATFORMS /F COURSE YOU COULD JUST
COMPRESS A TABLE WITH THE VARIOUS LEVELS TO SEE HOW WELL IT COMPRESSES BUT IF THE TABLES ARE VERY LARGE THIS
MAY NOT BE PRACTICAL )N THIS CASE YOU MAY BE TEMPTED TO CREATE A TEMPORARY TABLE BY SELECTING THE
RECORDS WHERE a^f]d\+G AND DO YOUR COMPRESSION TEST ON THAT SUBSET OF ROWS !ND THATS BASICALLY WHAT
THE !DVISOR DOES ALTHOUGH IT IS A LITTLE SMARTER ABOUT THE SET OF RECORDS IT CHOOSES (ERES AN EXAMPLE OF
ITS USE
BHB/B0=31>G -RPcVTcNR^\_aTbbX^]NaPcX^ b`[
bTcb`[Q[P]Z[X]Tb^]
bTcUTTSQPRZ^UU
PRRT_c^f]Ta
_a^\_c4]cTaEP[dTU^a^f]Ta)
PRRT_ccPQ[TN]P\T
_a^\_c4]cTaEP[dTU^acPQ[TN]P\T)
PRRT_cR^\_Nch_T
_a^\_c4]cTaEP[dTU^aR^\_aTbbX^]Nch_T>;C?)
STUPd[c>;C?
342;0A4
[NQ[ZR]cNR\_18=0AHN8=C464A*
[NQ[ZR]cNd]R\_18=0AHN8=C464A*
[Na^fNR\_18=0AHN8=C464A*
[Na^fNd]R\_18=0AHN8=C464A*
[NR\_NaPcX^=D<14A*
[NR^\_ch_TNbcaE0A270A!! *
[NR^\_ch_T]d\QTa*
1468=
RPbTR^\_Nch_T
fWT]>;C?cWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aN^[c_*
fWT]@D4AHcWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aN`dTahN[^f*
fWT]@D4AHN;>FcWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aN`dTahN[^f*
fWT]@D4AHN7867cWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aN`dTahNWXVW*
fWT]0A278E4cWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aNPaRWXeTN[^f*
fWT]0A278E4N;>FcWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aNPaRWXeTN[^f*
fWT]0A278E4N7867cWT][NR^\_ch_T),31<BN2><?A4BB8>= R^\_NU^aNPaRWXeTNWXVW*
4=320B4*
31<BN2><?A4BB8>= VTcNR^\_aTbbX^]NaPcX^
bRaPcRWcQb]P\T,-DB4AB
^f]]P\T,-^f]Ta
cPQ]P\T,-cPQ[TN]P\T
_Pac]P\T,-=D;;
R^\_ch_T,-[NR^\_ch_T
Q[ZR]cNR\_,-[NQ[ZR]cNR\_
Q[ZR]cNd]R\_,-[NQ[ZR]cNd]R\_
a^fNR\_,-[Na^fNR\_
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
a^fNd]R\_,-[Na^fNd]R\_
R\_NaPcX^,-[NR\_NaPcX^
R^\_ch_TNbca,-[NR^\_ch_TNbca
*
SQ\bN^dc_dc _dcN[X]T*
31<BN>DC?DC _dcN[X]T4bcX\PcTS2^\_aTbbX^]APcX^dbX]Vkk[NR^\_ch_TNbcakk)kk
a^d]S[NR\_NaPcX^"*
SQ\bN^dc_dc _dcN[X]T*
4=3*
d]STU^f]Ta
d]STUcPQ[TN]P\T
d]STUR^\_Nch_T
bTcUTTSQPRZ^]
BHB/B0=31>G -/VTcNR^\_aTbbX^]NaPcX^ b`[
4]cTaEP[dTU^a^f]Ta):B>
4]cTaEP[dTU^acPQ[TN]P\T)B:4F"
4]cTaEP[dTU^aR^\_aTbbX^]Nch_T>;C?)
4bcX\PcTS2^\_aTbbX^]APcX^dbX]V2^\_aTbb5^a>;C?) #
4[P_bTS) ) ) & $
BHB/B0=31>G -/VTcNR^\_aTbbX^]NaPcX^ b`[
4]cTaEP[dTU^a^f]Ta):B>
4]cTaEP[dTU^acPQ[TN]P\T)B:4F"
4]cTaEP[dTU^aR^\_aTbbX^]Nch_T>;C?)@D4AH;>F
2^\_aTbbX^]0SeXb^abT[U RWTRZeP[XSPcX^]bdRRTbbUd[ bT[TRcR^d]c^]Q^cWD]R^\_aTbbTSP]S
47222^\_aTbbTSU^a\Pc, a^fb
4bcX\PcTS2^\_aTbbX^]APcX^dbX]V2^\_aTbb5^a@dTah;^f)#
4[P_bTS) ) ) # #
BHB/B0=31>G -/VTcNR^\_aTbbX^]NaPcX^ b`[
4]cTaEP[dTU^a^f]Ta):B>
4]cTaEP[dTU^acPQ[TN]P\T)B:4F"
4]cTaEP[dTU^aR^\_aTbbX^]Nch_T>;C?)@D4AH7867
2^\_aTbbX^]0SeXb^abT[U RWTRZeP[XSPcX^]bdRRTbbUd[ bT[TRcR^d]c^]Q^cWD]R^\_aTbbTSP]S
47222^\_aTbbTSU^a\Pc, a^fb
4bcX\PcTS2^\_aTbbX^]APcX^dbX]V2^\_aTbb5^a@dTah7XVW)#! #
4[P_bTS) ) ) #!
BHB/B0=31>G -/VTcNR^\_aTbbX^]NaPcX^ b`[
4]cTaEP[dTU^a^f]Ta):B>
4]cTaEP[dTU^acPQ[TN]P\T)B:4F"
4]cTaEP[dTU^aR^\_aTbbX^]Nch_T>;C?)0A278E4;>F
2^\_aTbbX^]0SeXb^abT[U RWTRZeP[XSPcX^]bdRRTbbUd[ bT[TRcR^d]c^]Q^cWD]R^\_aTbbTSP]S
47222^\_aTbbTSU^a\Pc, a^fb
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
4bcX\PcTS2^\_aTbbX^]APcX^dbX]V2^\_aTbb5^a0aRWXeT;^f)#" $
4[P_bTS) ) ) &
BHB/B0=31>G -/VTcNR^\_aTbbX^]NaPcX^ b`[
4]cTaEP[dTU^a^f]Ta):B>
4]cTaEP[dTU^acPQ[TN]P\T)B:4F"
4]cTaEP[dTU^aR^\_aTbbX^]Nch_T>;C?)0A278E47867
2^\_aTbbX^]0SeXb^abT[U RWTRZeP[XSPcX^]bdRRTbbUd[ bT[TRcR^d]c^]Q^cWD]R^\_aTbbTSP]S
47222^\_aTbbTSU^a\Pc, a^fb
4bcX\PcTS2^\_aTbbX^]APcX^dbX]V2^\_aTbb5^a0aRWXeT7XVW)$# &
4[P_bTS) ) ) ' (
.OTICE THAT THE PROCEDURE PRINTS OUT A VALIDATION MESSAGE TELLING YOU HOW MANY RECORDS WERE USED
FOR THE COMPARISON 4HIS NUMBER CAN BE MODIFIED AS PART OF THE CALL TO THE PROCEDURE IF SO DESIRED 4HE
GET?COMPRESSION?RATIOSQL SCRIPT PROMPTS FOR A TABLE AND A #OMPRESSION 4YPE AND THEN EXECUTES THE
31<BN2><?A4BB8>= 64CN2><?A4BB8>=NA0C8> PROCEDURE /NCE AGAIN THE PERTINENT DATA IS A LITTLE HARD TO
PICK OUT OF THE LISTING SO 4ABLE
COMPARES THE ACTUAL COMPRESSION RATIOS TO THE ESTIMATES PROVIDED BY
THE #OMPRESSION !DVISOR
Table 3-5. Compression Advisor Accuracy
Actual Estimated
Table Name Compress For Compression Compression Error Percentage
Ratio Ratio
B:4F"N>;C?>;C?
B:4F"N722 @D4AH ;>F
B:4F"N722!@D4AH 7867
B:4F"N722"0A278E4 ;>F
B:4F"N722#0A278E4 7867
4HE ESTIMATES ARE FAIRLY CLOSE TO THE ACTUAL VALUES AND WHILE THEY ARE NOT PERCENT ACCURATE THE
TRADEOFF IS PROBABLY WORTH IT IN CASES WHERE THE OBJECTS ARE VERY LARGE AND A FULL TEST WOULD BE TOO TIME
CONSUMING OR TAKE UP TOO MUCH DISK SPACE 4HE ABILITY TO RUN THE #OMPRESSION !DVISOR ON NON %XADATA
PLATFORMS IS ALSO A REAL PLUS )T SHOULD BE ABLE TO PROVIDE YOU WITH ENOUGH INFORMATION TO MAKE
REASONABLE DECISIONS PRIOR TO ACTUALLY MIGRATING DATA TO %XADATA
2EAL 7ORLD %XAMPLES
!S 9OGI "ERRA ONCE SAID YOU CAN LEARN A LOT JUST BY WATCHING -ARKETING SLIDES AND BOOK AUTHOR CLAIMS
ARE ONE THING BUT REAL DATA IS OFTEN MORE USEFUL *UST TO GIVE YOU AN IDEA OF WHAT KIND OF COMPRESSION IS
REASONABLE TO EXPECT HERE ARE A FEW COMPARISONS OF DATA FROM DIFFERENT INDUSTRIES 4HE DATA SHOULD
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
PROVIDE YOU WITH AN IDEA OF THE POTENTIAL COMPRESSION RATIOS THAT CAN BE ACHIEVED BY (YBRID #OLUMNAR
#OMPRESSION
Custom Application Data
4HIS DATASET CAME FROM A CUSTOM APPLICATION THAT TRACKS THE MOVEMENT OF ASSETS 4HE TABLE IS VERY
NARROW CONSISTING OF ONLY COLUMNS 4HE TABLE HAS CLOSE TO BILLION ROWS BUT MANY OF THE COLUMNS
HAVE A VERY LOW NUMBER OF DISTINCT VALUES .$6 4HAT MEANS THAT THE SAME VALUES ARE REPEATED MANY
TIMES 4HIS TABLE IS A PRIME CANDIDATE FOR COMPRESSION (ERE ARE THE BASIC TABLE STATISTICS AND THE
COMPRESSION RATIOS ACHIEVED
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CPQ[TBcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
C01;4N=0<4)2?N308;H
;0BCN0=0;HI43)!( 342 ! !")$$) %
346A44)
?0AC8C8>=43)H4B
=D<NA>FB)(!$!# !#
2708=N2=C)
1;>2:B) $ "%%'
4<?CHN1;>2:B)
0E6NB?024)
0E6NA>FN;4=) #
<>=8C>A8=6)H4B
B0<?;4NB8I4)(!$!# !#
TOTALSIZE_MEGS : 118019
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2^[d\]BcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
=P\T0]P[hiTS=d[[.=3E3T]bXch=d[[b1dRZTcb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
?:N02C8E8CHN3C;N83 ! !( ! =>C=D;;(!$!# !#
5:N02C8E8CHN83 ! !( ! =>C=D;;#""''(!'
5:N34=><8=0C8>=N83 ! !( ! "' ''&(& #("'
0<>D=C ! !( ! !&"('#
5:N1A0=27N83 ! !( ! =>C=D;; " !'
;>68=N83 ! !( ! =>C=D;;" """""
30C4C8<4NBC0<? ! !( ! =>C=D;;& !&!
;0BCN<>385HN;>68=N83 ! !( ! =>C=D;;" """""
<>385HN30C4C8<4NBC0<? ! !( ! =>C=D;;#% !!# !
02C8E4N5;06 ! !( ! =>C=D;;! !
5:N106N83 ! !( ! !'($"% '"%%("$"$
2A438CN30C4 ! !( ! $#( '! '"%%("$"$
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
BHB/?>2 -/cPQ[TNbXiT!
4]cTaeP[dTU^a^f]Ta)
4]cTaeP[dTU^acPQ[TN]P\T)2?N308;HN8=EN02C8E8CHN3C;
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
>F=4AB46<4=CN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>2?N308;HN8=EN02C8E8CHN3C; ' ' '
bd\ ' ' '
BHB/?>2 -/R^\_NaPcX^
4]cTaeP[dTU^a^aXVX]P[NbXiT) ' ' '
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)2?N308;H
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B2><?A4BB8>=NA0C8>
:B>2?N308;HN722 C01;4&#'' $ '
:B>2?N308;HN722"C01;4!##! "#' "
:B>2?N308;HN722!C01;4! '# &$#
:B>2?N308;HN722#C01;4 ' & '%$ "
bd\ "(!! '
!S EXPECTED THIS TABLE IS EXTREMELY COMPRESSIBLE 3IMPLE QUERIES AGAINST THESE TABLES ALSO RUN MUCH
FASTER AGAINST THE COMPRESSED TABLES AS YOU CAN SEE IN THIS LISTING
BHB/?>2 -bT[TRcbd\P\^d]cUa^\Zb^ 2?N308;HN722"fWTaTRaTSXcNSPcT, ^Rc ! *
BD<0<>D=C
# !&&(% # (
a^fbT[TRcTS
4[P_bTS) ) ) ! "&
BHB/?>2 -bT[TRcbd\P\^d]cUa^\Zb^ 2?N308;HfWTaTRaTSXcNSPcT, ^Rc ! *
BD<0<>D=C
# !&&(% # (
a^fbT[TRcTS
4[P_bTS) ) )#! $'
4HIS SIMPLE QUERY RAN ROUGHLY TIMES FASTER USING THE 0A278E4;>F COMPRESSED TABLE THAN WHEN IT
WAS RUN AGAINST THE UNCOMPRESSED TABLE
Telecom Call Detail Data
4HIS TABLE CONTAINS CALL DETAIL RECORDS FOR A TELECOM COMPANY 4HERE ARE APPROXIMATELY BILLION
RECORDS IN THE TABLE -ANY OF THE COLUMNS IN THIS TABLE ARE UNIQUE OR NEARLY SO )N ADDITION MANY OF THE
COLUMNS CONTAIN LARGE NUMBERS OF NULLS .ULLS ARE NOT COMPRESSIBLE SINCE THEY ARE NOT STORED IN THE
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
NORMAL /RACLE BLOCK FORMAT 4HIS IS NOT A TABLE WE WOULD EXPECT TO BE HIGHLY COMPRESSIBLE (ERE ARE THE
BASIC TABLE STATISTICS AND THE COMPRESSION RATIOS
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CPQ[TBcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
C01;4N=0<4)B44
;0BCN0=0;HI43)!( B4? ! ) !) $
346A44)'
?0AC8C8>=43)H4B
=D<NA>FB) #&#&&%'&#
2708=N2=C)
1;>2:B)$&$"!&"
4<?CHN1;>2:B)
0E6NB?024)
0E6NA>FN;4=)!'!
<>=8C>A8=6)H4B
B0<?;4NB8I4) #&#&&%'&#
TOTALSIZE_MEGS : 455821
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
BHB/?>2 -/R^\_NaPcX^
4]cTaeP[dTU^a^aXVX]P[NbXiT)#$$'!
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B44N722
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B2><?A4BB8>=NA0C8>
:B>B44N722 C01;4 %'%( ! &
:B>B44N722!C01;4(% #! # &
:B>B44N722"C01;4'&#$ '$ !
:B>B44N722#C01;4&!" ( % "
bd\#!#% !
Financial Data
4HE NEXT TABLE IS MADE UP OF FINANCIAL DATA REVENUE ACCRUAL DATA FROM AN ORDER ENTRY SYSTEM TO BE EXACT
(ERE ARE THE BASIC TABLE STATISTICS
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CPQ[TBcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
C01;4N=0<4)A4EN022AD0;
;0BCN0=0;HI43) & 90= ! )#!)#&
346A44)
?0AC8C8>=43)H4B
=D<NA>FB) #&"%%'%
2708=N2=C)
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
1;>2:B) $!!$(
4<?CHN1;>2:B)
0E6NB?024)
0E6NA>FN;4=)( &
<>=8C>A8=6)H4B
B0<?;4NB8I4) #&"%%'%
TOTALSIZE_MEGS : 120019
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
3O THE NUMBER OF ROWS IS NOT THAT GREAT ONLY ABOUT MILLION BUT THE TABLE IS WIDE )T HAS
COLUMNS AND THE AVERAGE ROW LENGTH IS BYTES )T IS A BIT OF A MIXED BAG WITH REGARDS TO COMPRESSIBILITY
THOUGH -ANY OF THE COLUMNS CONTAIN A HIGH PERCENTAGE OF NULLS /N THE OTHER HAND MANY OF THE
COLUMNS HAVE A VERY LOW NUMBER OF DISTINCT VALUES 4HIS TABLE MAY BE A CANDIDATE FOR REORDERING THE DATA
ON DISK AS A STRATEGY TO IMPROVE THE COMPRESSION RATIO !T ANY RATE HERE ARE THE COMPRESSION RATES
ACHIEVED ON THIS TABLE AT THE VARIOUS (## LEVELS
BHB/?>2 -/R^\_NaPcX^
4]cTaeP[dTU^a^aXVX]P[NbXiT) ! (
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)A4EN022AD0;N722
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B2><?A4BB8>=NA0C8>
:B>A4EN022AD0;N722 C01;4" (&! %" '
:B>A4EN022AD0;N722!C01;4 & '! (&
:B>A4EN022AD0;N722"C01;4 #" # "' #
:B>A4EN022AD0;N722#C01;4 !$# %( %
bd\&$( #
Retail Sales Data
4HE FINAL TABLE IS MADE UP OF SALES FIGURES FROM A RETAILER 4HE TABLE CONTAINS ABOUT BILLION RECORDS AND
OCCUPIES WELL OVER HALF A 4ERABYTE 4HERE ARE VERY FEW COLUMNS AND THE DATA IS HIGHLY REPETITIVE )N FACT
THERE ARE NO UNIQUE FIELDS IN THIS TABLE 4HIS IS A VERY GOOD CANDIDATE FOR COMPRESSION (ERE ARE THE BASIC
TABLE STATISTICS
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CPQ[TBcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
C01;4N=0<4)B0;4B
;0BCN0=0;HI43)!" 342 ! ") ")##
346A44)
?0AC8C8>=43)=>
=D<NA>FB)$'$"&'#"%$
2708=N2=C)
1;>2:B)&( '"'%!
4<?CHN1;>2:B)
0E6NB?024)
0E6NA>FN;4=)("
<>=8C>A8=6)H4B
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
B0<?;4NB8I4)$'$"&'#"%$
TOTALSIZE_MEGS : 618667
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2^[d\]BcPcXbcXRb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
=P\T0]P[hiTS=d[[.=3E3T]bXch=d[[b1dRZTcbBP\_[T
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
CA0=BN83 ! !" ! "'(' ' !' $'$"&'#"%$
CA0=BN;8=4N=> ! !" ! !% &("& $'$"&'#"%$
D=8CN83 ! !" ! !'% ' $'$"&'#"%$
30H ! !" ! " """""" $'$"&'#"%$
CA0=BNB4@ ! !" ! !!("! ## $'$"&'#"%$
1468=N30C4 ! !" ! # !$ $'$"&'#"%$
4=3N30C4 ! !" ! # !$ $'$"&'#"%$
D=8CNCH?4 ! !" ! $'$"&'#"%$
B:DNCH?4 ! !" ! $#''# ' $'$"&'#"%$
@CH ! !" ! # (% $ $'$"&'#"%$
?A824 ! !" ! %!! % ' $'$"&'#"%$
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
(ERE ARE THE COMPRESSION RATIOS ACHIEVED FOR THIS TABLE !S EXPECTED THEY ARE VERY GOOD
BHB/34<> -/R^\_NaPcX^
4]cTaeP[dTU^a^aXVX]P[NbXiT)% '%%&
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B0;4BN722
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B2><?A4BB8>=NA0C8>
:B>B0;4BN722 C01;4# %$# % # (
:B>B0;4BN722!C01;4!%$#! !" "
:B>B0;4BN722"C01;4!%$"' $!" "
:B>B0;4BN722#C01;4 (%"" " $
bd\ #"%'
Summary of the Real World Examples
4HE EXAMPLES IN THIS SECTION CAME FROM REAL APPLICATIONS 4HEY SHOW A FAIRLY EXTREME VARIATION IN DATA
COMPRESSIBILITY 4HIS IS TO BE EXPECTED AS THE SUCCESS OF COMPRESSION ALGORITHMS IS VERY DEPENDENT ON
THE DATA BEING COMPRESSED 4ABLE PRESENTS THE DATA FROM ALL FOUR EXAMPLES
Table 3-6. Real-World Examples Compared
Data Type Base Table Name Characteristics Compression Ratios
!SSET 4RACKING 2?N308;H 3KINNY 4ABLE -ANY ,OW .$6 #OLUMNS ¾ ¾
#ALL $ETAIL 2ECORDS B44 -ANY .5,,S -ANY 5NIQUE #OLUMNS ¾ ¾
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
Data Type Base Table Name Characteristics Compression Ratios
&INANCIAL $ATA A4EN022AD0; 7IDE 4ABLE -ANY .5,,S -ANY ,OW .$6 ¾ ¾
#OLUMNS
2ETAIL 3ALES $ATA B0;4B 3KINNY 4ABLE -OSTLY ,OW .$6 #OLUMNS ¾ ¾
(OPEFULLY THIS DATA GIVES YOU SOME FEEL FOR THE RANGE OF COMPRESSION RATIOS THAT YOU CAN EXPECT FROM
(## AND THE TYPES OF DATASETS THAT WILL BENEFIT MOST /F COURSE THE BEST WAY TO PREDICT HOW COMPRESSIBLE
A PARTICULAR TABLE MAY BE IS TO ACTUALLY TEST IT 4HIS FACT CANNOT BE OVEREMPHASIZED
Restrictions/Challenges
4HERE ARE A FEW CHALLENGES WITH USING (## -ANY OF THEM HAVE TO DO WITH THE FACT THAT (## IS NOT
AVAILABLE ON NON
%XADATA PLATFORMS 4HIS FACT POSES CHALLENGES FOR RECOVERY AND HIGH AVAILABILITY
SOLUTIONS 4HE OTHER MAJOR CHALLENGE IS THAT (## DOESNT PLAY WELL WITH DATA THAT IS BEING ACTIVELY
UPDATED )N PARTICULAR SYSTEMS CHARACTERIZED BY LOTS OF SINGLE
ROW UPDATES WHICH WE OFTEN DESCRIBE AS
/,40 WORKLOADS WILL PROBABLY NOT WORK WELL WITH (##
-OVING $ATA TO A NON
%XADATA 0LATFORM
0ROBABLY THE LARGEST HURDLE WITH USING (## HAS BEEN MOVING THE DATA TO NON
%XADATA PLATFORMS &OR
EXAMPLE WHILE 2-!. AND $ATAGUARD BOTH SUPPORT THE (## BLOCK FORMAT AND WILL HAPPILY RESTORE DATA
TO A NON
%XADATA ENVIRONMENT A DATABASE RUNNING ON SUCH AN ENVIRONMENT WILL NOT BE ABLE TO DO
ANYTHING WITH THE DATA UNTIL IT IS DECOMPRESSED 4HIS CAN MEAN A LENGTHY DELAY BEFORE BEING ABLE TO
ACCESS THE DATA IN A CASE WHERE A FAILOVER TO A STANDBY ON A NON
%XADATA PLATFORM OCCURS 4HE SAME ISSUE
HOLDS TRUE FOR DOING AN 2-!. RESTORE TO A NON
%XADATA PLATFORM 4HE RESTORE WILL WORK BUT THE DATA IN
(## FORMATTED BLOCKS WILL NOT BE ACCESSIBLE UNTIL THE DATA HAS BEEN MOVED INTO A NON
(## FORMAT 4HIS
CAN BE DONE WITH THE 0;C4AC01;4<>E4=>2><?A4BB COMMAND BY THE WAY
■ .OTE 4HE ABILITY TO DECOMPRESS (## DATA ON NON
%XADATA PLATFORMS ONLY BECAME AVAILABLE IN /RACLE DATABASE
VERSION !TTEMPTING THIS ON VERSION WOULD RESULT IN AN ERROR #HECK WITH /RACLE 3UPPORT FOR A
PATCH THAT MAY ENABLE THIS BEHAVIOR ON
)N ADDITION TO THE LENGTHY DELAY ASSOCIATED WITH DECOMPRESSING DATA BEFORE BEING ABLE TO ACCESS IT
THERE IS ALSO THE ISSUE OF SPACE )F (## IS PROVIDING A ¾ COMPRESSION FACTOR YOU WILL NEED TO HAVE
TIMES THE SPACE YOU ARE CURRENTLY USING AVAILABLE ON THE TARGET ENVIRONMENT TO HANDLE THE INCREASED SIZE
OF THE DATA &OR THESE REASONS $ATAGUARD IS RARELY SET UP WITH A STANDBY ON A NON
%XADATA PLATFORM
$ISABLING 3ERIAL $IRECT 0ATH 2EADS
!S WE DISCUSSED IN #HAPTER 3ERIAL $IRECT 0ATH 2EADS ALLOW NONPARALLELIZED SCAN OPERATIONS TO USE THE
DIRECT PATH READ MECHANISM WHICH IS A PREREQUISITE FOR ENABLING THE 3MART 3CAN FEATURES OF %XADATA
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
3ERIAL $IRECT 0ATH 2EADS ARE ENABLED BASED ON A CALCULATION THAT DEPENDS ON THE SIZE OF THE OBJECT BEING
SCANNED RELATIVE TO THE AVAILABLE BUFFER CACHE )N SIMPLISTIC TERMS ONLY LARGE OBJECTS WILL BE CONSIDERED
FOR 3ERIAL $IRECT 0ATH 2EADS (##S EFFECTIVENESS CAN ACTUALLY WORK AGAINST IT HERE 3INCE THE COMPRESSION
REDUCES THE SIZE OF THE OBJECTS SO DRASTICALLY IT CAN CAUSE STATEMENTS THAT WOULD NORMALLY BENEFIT FROM A
3MART 3CAN TO USE THE STANDARD READ MECHANISM DISABLING MANY OF %XADATAS OPTIMIZATIONS 4HIS IS
GENERALLY NOT A HUGE PROBLEM BECAUSE THE NUMBER OF BLOCKS IS CONSIDERABLY REDUCED BY (## AND THE
DATABASE IS MAKING THIS DECISION AT RUN TIME 4HE PROBLEM COMES IN THOUGH WHEN AN OBJECT IS
PARTITIONED 4HE CALCULATION IS BASED ON THE SIZE OF THE OBJECT BEING SCANNED IN THE CASE OF A PARTITIONED
OBJECT THIS MEANS THE SIZE OF THE PARTITION 3O IN CASES WHERE PARTITIONING IS USED WITH (## WE OFTEN SEE
SOME PARTITIONS USING 3MART 3CANS AND SOME UNABLE TO USE 3MART 3CANS +EEP IN MIND THAT THIS ALSO
MEANS DECOMPRESSION CANNOT BE DONE AT THE STORAGE LAYER AS THIS CAPABILITY IS ENABLED ONLY WHEN
PERFORMING 3MART 3CANS
■ +EVIN 3AYS #OMPRESSION ALSO HAS A LOT TO DO WITH THE OFTEN OVERLOOKED )N -EMORY 0ARALLEL 1UERY FEATURE OF
/RACLE $ATABASE G 6ERY EFFECTIVE COMPRESSION COMBINED WITH MODERN LARGE MEMORY SERVERS MAKES )N -EMORY
0ARALLEL 1UERY A USABLE FEATURE FOR PRODUCTION PURPOSES #USTOMERS WOULD DO WELL TO CONSIDER THE BEST
COMPRESSION TECHNOLOGY THAT SUITS THEIR $-, PROCESSING REQUIREMENTS WHILE POTENTIALLY EXPLOITING THE POWER OF )N
-EMORY 0ARALLEL 1UERY
,OCKING )SSUES
4HE %XADATA DOCUMENTATION SAYS THAT UPDATING A SINGLE ROW OF A TABLE COMPRESSED WITH (## LOCKS THE
ENTIRE COMPRESSION UNIT CONTAINING THE ROW 4HIS CAN CAUSE EXTREME CONTENTION ISSUES FOR /,40
TYPE
SYSTEMS 4HIS IS THE MAIN REASON THAT (## IS NOT RECOMMENDED FOR TABLES OR PARTITIONS WHERE THE DATA
WILL BE UPDATED (ERES A DEMONSTRATION OF THE LOCKING BEHAVIOR
:B>/B0=31>G -bT[TRca^fXS^[SNa^fXSa^fXS^[SNa^fXS_ZNR^[Ua^\Zb^ bZTfNWRR"
!fWTaTa^f]d\+ *
A>F83>;3NA>F83?:N2>;
000C2100700<ZhG000& "!(%# & !&(((# (
000C2100700<ZhG001& "!(%# & !&(((# '
000C2100700<ZhG002& "!(%# & !!&(((# &
000C2100700<ZhG003& "!(%# & "!&(((# %
000C2100700<ZhG004& "!(%# & #!&(((# $
000C2100700<ZhG005& "!(%# & $!&(((# #
000C2100700<ZhG006& "!(%# & %!&(((# "
000C2100700<ZhG007& "!(%# & &!&(((# !
000C2100700<ZhG008& "!(%# & '!&(((#
(a^fbbT[TRcTS
:B>/B0=31>G -d_SPcTZb^ bZTfbTcR^[ ,R^[ fWTaT_ZNR^[,!&(((# (*
a^fd_SPcTS
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
BHB/B0=31>G -bT[TRcR^[ Ua^\Zb^ bZTfNWRR"fWTaT_ZNR^[,!&(((# (U^ad_SPcT]^fPXc*
bT[TRcR^[ Ua^\Zb^ bZTffWTaT_ZNR^[, %'$'#"&U^ad_SPcT]^fPXc
4AA>APc[X]T )
>A0 $#)aTb^daRTQdbhP]SPR`dXaTfXcW=>F08Cb_TRXUXTS^acX\T^dcTg_XaTS
BHB/B0=31>G - 4g_TRcTSQTRPdbTcWXba^fWPbQTT]d_SPcTSQhP]^cWTa_a^RTbb
BHB/B0=31>G -
BHB/B0=31>G -bT[TRcR^[ Ua^\Zb^ bZTfNWRR"fWTaT_ZNR^[,!&(((# U^ad_SPcT]^fPXc*
bT[TRcR^[ Ua^\Zb^ bZTffWTaT_ZNR^[, %'$'#!(U^ad_SPcT]^fPXc
4AA>APc[X]T )
>A0 $#)aTb^daRTQdbhP]SPR`dXaTfXcW=>F08Cb_TRXUXTS^acX\T^dcTg_XaTS
BHB/B0=31>G - =^c]^a\P[>aPR[T[^RZX]VQTWPeX^a
#LEARLY THIS BEHAVIOR WOULD BE DISASTROUS TO MANY /,40 SYSTEMS %SPECIALLY WHEN YOU CONSIDER THE
LARGE NUMBER OF RECORDS THAT CAN BE STORED IN AN (## BLOCK )N THIS CASE THAT NUMBER IS APPROXIMATELY
ROWS PER BLOCK 4HIS MEANS THAT A SINGLE UPDATE COULD LOCK WELL OVER ROWS
3INGLE 2OW !CCESS
(## IS BUILT FOR FULL TABLE SCAN ACCESS $ECOMPRESSION IS A #05
INTENSIVE TASK 3MART 3CANS CAN
DISTRIBUTE THE DECOMPRESSION WORK TO THE #05S ON THE STORAGE CELLS 4HIS MAKES THE #05
INTENSIVE TASK
MUCH MORE PALATABLE (OWEVER 3MART 3CANS ONLY OCCUR WHEN &ULL 3CANS ARE PERFORMED 4HIS MEANS
THAT OTHER ACCESS MECHANISMS INDEX ACCESS FOR EXAMPLE MUST USE THE $" SERVER #05S TO PERFORM
DECOMPRESSION 4HIS CAN PUT AN ENORMOUS #05 LOAD ON $" SERVERS IN HIGH VOLUME /,40
TYPE SYSTEMS
)N ADDITION SINCE DATA FOR A SINGLE ROW IS SPREAD ACROSS MULTIPLE BLOCKS IN A #5 RETRIEVING A COMPLETE
ROW CAUSES THE ENTIRE #5 TO BE READ 4HIS CAN HAVE A DETRIMENTAL EFFECT ON THE OVERALL DATABASE EFFICIENCY
FOR SYSTEMS THAT TEND TO ACCESS DATA USING INDEXES EVEN IF THE ACCESS IS READ
ONLY
Common Usage Scenarios
(## PROVIDES SUCH HIGH LEVELS OF COMPRESSION THAT IT HAS BEEN USED AS AN ALTERNATIVE TO TRADITIONAL ),-
STRATEGIES WHICH GENERALLY INVOLVE MOVING OLDER HISTORICAL DATA OFF THE DATABASE ENTIRELY 4HESE ),-
STRATEGIES USUALLY ENTAIL SOME TYPE OF DATE RANGE PARTITIONING AND A PURGE PROCESS 4HIS IS DONE TO FREE
STORAGE AND IN SOME CASES TO IMPROVE PERFORMANCE /FTEN THE DATA MUST BE RETAINED IN SOME BACKUP
FORMAT SO THAT IT CAN BE ACCESSED IF REQUIRED AT SOME LATER DATE 7ITH (## IT IS POSSIBLE IN MANY CASES TO
RETAIN DATA INDEFINITELY BY COMPRESSING THE OLDEST PARTITIONS 4HIS APPROACH HAS MANY ADVANTAGES OVER
THE TRADITIONAL APPROACH OF MOVING THE DATA
&IRST AND FOREMOST THE DATA REMAINS AVAILABLE VIA THE STANDARD APPLICATION INTERFACES .O ADDITIONAL
WORK WILL NEED TO BE DONE TO RESTORE A BACKUP OF OLD DATA BEFORE IT CAN BE ACCESSED 4HIS ADVANTAGE ALONE
IS OFTEN ENOUGH TO JUSTIFY THIS APPROACH 4HIS APPROACH TYPICALLY ENTAILS LEAVING ACTIVE PARTITIONS
UNCOMPRESSED WHILE COMPRESSING OLD PARTITIONS MORE AGGRESSIVELY (ERES A SHORT EXAMPLE OF CREATING A
PARTITIONED TABLE WITH MIXED COMPRESSION MODES
BHB/34<> -2A40C4C01;4:B> 2;0BBNB0;4BN?
!CA0=BN83E0A270A!"
"D=8CN83=D<14A"
#30H=D<14A"
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
$CA0=BNB4@E0A270A!"
%4=3N30C430C4
&1468=N30C430C4
'D=8CNCH?4E0A270A!"
(2DBCNCH?4E0A270A!
;>03N30C430C4
2DAA4=2HNCH?4270A
!?2C5A44 ?2CDB43# 8=8CA0=B <0GCA0=B!$$=>;>668=6
"BC>A064
#1D554AN?>>;3450D;C5;0B7N202743450D;C24;;N5;0B7N202743450D;C
$C01;4B?0242;0BBN30C0
%?0AC8C8>=1HA0=641468=N30C4
&?0AC8C8>=? E0;D4B;4BBC70=C>N30C4
'! ' ( % ) ) BHHHH << 3377!#)<8)BB=;BN20;4=30A,6A46>A80=
(B46<4=C2A40C8>=8<<4380C4
! ?2C5A44 ?2CDB43# 8=8CA0=B <0GCA0=B!$$NOCOMPRESS=>;>668=6
! BC>A0648=8C80;'"''% '=4GC #'$&%<8=4GC4=CB <0G4GC4=CB! #&#'"%#$
!!?2C8=2A40B4 5A44;8BCB 5A44;8BC6A>D?B 1D554AN?>>;3450D;C
!"5;0B7N202743450D;C24;;N5;0B7N202743450D;C
!#C01;4B?0242;0BBN30C0
!$?0AC8C8>=?!E0;D4B;4BBC70=C>N30C4
!%! ' ( & ) ) BHHHH << 3377!#)<8)BB=;BN20;4=30A,6A46>A80=
!&B46<4=C2A40C8>=8<<4380C4
!'?2C5A44 ?2CDB43# 8=8CA0=B <0GCA0=B!$$COMPRESS FOR QUERY HIGH=>;>668=6
!(BC>A0648=8C80;'"''% '=4GC #'$&%<8=4GC4=CB <0G4GC4=CB! #&#'"%#$
" ?2C8=2A40B4 5A44;8BCB 5A44;8BC6A>D?B 1D554AN?>>;3450D;C
" 5;0B7N202743450D;C24;;N5;0B7N202743450D;C
"!C01;4B?0242;0BBN30C0
""?0AC8C8>=?"E0;D4B;4BBC70=C>N30C4
"#! ' ( ' ) ) BHHHH << 3377!#)<8)BB=;BN20;4=30A,6A46>A80=
"$B46<4=C2A40C8>=8<<4380C4
"%?2C5A44 ?2CDB43# 8=8CA0=B <0GCA0=B!$$COMPRESS FOR ARCHIVE LOW=>;>668=6
"&BC>A0648=8C80;'"''% '=4GC #'$&%<8=4GC4=CB <0G4GC4=CB! #&#'"%#$
"'?2C8=2A40B4 5A44;8BCB 5A44;8BC6A>D?B 1D554AN?>>;3450D;C
"(5;0B7N202743450D;C24;;N5;0B7N202743450D;C
# C01;4B?0242;0BBN30C0*
CPQ[TRaTPcTS
BHB/34<> -/_PacNbXiT! b`[
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)
#(!04%2 (9"2)$ #/,5-.!2 #/-02%33)/.
>F=4AB46<4=CN=0<4?0ACN=0<4C>C0;B8I4N<46B2><?A4BBN5>A
:B>2;0BBNB0;4BN?? !#
?!!# @D4AH7867
?"!# 0A278E4;>F
bd\&!
Summary
)NTRODUCED IN /RACLE G 2 (YBRID #OLUMNAR #OMPRESSION PROVIDES EXCEPTIONAL COMPRESSION
CAPABILITIES THAT ARE FAR BEYOND ANYTHING AVAILABLE IN PRIOR RELEASES 4HIS IS THANKS IN LARGE PART TO THE
ADOPTION OF INDUSTRY
STANDARD COMPRESSION ALGORITHMS AND AN INCREASE IN THE SIZE OF THE COMPRESSION
UNIT FROM A SINGLE DATABASE BLOCK TYPICALLY + TO A LARGER UNIT OF + OR + 4HE FEATURE IS ONLY
APPROPRIATE FOR DATA THAT IS NO LONGER BEING MODIFIED THOUGH BECAUSE OF LOCKING ISSUES AND THE FACT THAT
UPDATED ROWS ARE MOVED INTO A MUCH LESS COMPRESSED FORMAT /,40 COMPRESSION FORMAT &OR THIS
REASON (## SHOULD ONLY BE USED WITH DATA THAT IS NO LONGER BEING MODIFIED OR ONLY OCCASIONALLY
MODIFIED 3INCE COMPRESSION CAN BE DEFINED AT THE PARTITION LEVEL IT IS COMMON TO SEE TABLES THAT HAVE A
MIXTURE OF COMPRESSED AND UNCOMPRESSED PARTITIONS 4HIS TECHNIQUE CAN IN MANY CASES REPLACE ),-
APPROACHES THAT REQUIRE MOVING DATA TO ALTERNATE STORAGE MEDIA AND THEN PURGING IT FROM THE DATABASE
CHAPTER 4
Storage Indexes
3TORAGE )NDEXES ARE THE MOST USEFUL %XADATA FEATURE THAT YOU NEVER HEAR ABOUT 4HEY ARE NOT INDEXES THAT
ARE STORED IN THE DATABASE LIKE /RACLES TRADITIONAL "
4REE OR BITMAPPED INDEXES )N FACT THEY ARE NOT
INDEXES AT ALL IN THE TRADITIONAL SENSE 4HEY ARE NOT CAPABLE OF IDENTIFYING A SET OF RECORDS THAT HAS A
CERTAIN VALUE IN A GIVEN COLUMN 2ATHER THEY ARE A FEATURE OF THE STORAGE SERVER SOFTWARE THAT IS DESIGNED
TO ELIMINATE DISK )/ 4HEY ARE SOMETIMES DESCRIBED AS hREVERSE INDEXESv 4HATS BECAUSE THEY IDENTIFY
LOCATIONS WHERE THE REQUESTED RECORDS ARE NOT INSTEAD OF THE OTHER WAY AROUND 4HEY WORK BY STORING
MINIMUM AND MAXIMUM COLUMN VALUES FOR DISK STORAGE UNITS WHICH ARE -EGABYTE -" BY DEFAULT
"ECAUSE 31, PREDICATES ARE PASSED TO THE STORAGE SERVERS WHEN 3MART 3CANS ARE PERFORMED THE STORAGE
SOFTWARE CAN CHECK THE PREDICATES AGAINST THE 3TORAGE )NDEX METADATA MAXIMUM AND MINIMUM VALUES
BEFORE DOING THE REQUESTED )/ !NY STORAGE REGION THAT CANNOT POSSIBLY HAVE A MATCHING ROW IS SKIPPED
)N MANY CASES THIS CAN RESULT IN A SIGNIFICANT REDUCTION IN THE AMOUNT OF )/ THAT MUST BE PERFORMED
+EEP IN MIND THAT SINCE THE STORAGE SOFTWARE NEEDS THE PREDICATES TO COMPARE TO THE MAXIMUM AND
MINIMUM VALUES IN THE 3TORAGE )NDEXES THIS OPTIMIZATION IS ONLY AVAILABLE FOR 3MART 3CANS
4HE STORAGE SOFTWARE PROVIDES NO DOCUMENTED MECHANISM FOR ALTERING OR TUNING 3TORAGE )NDEXES
ALTHOUGH THERE ARE A FEW UNDOCUMENTED PARAMETERS THAT CAN BE SET PRIOR TO STARTING RT[[bae ON THE
STORAGE SERVERS )N FACT THERE IS NOT EVEN MUCH AVAILABLE IN THE WAY OF MONITORING &OR EXAMPLE THERE IS
NO WAIT EVENT THAT RECORDS THE AMOUNT OF TIME SPENT WHEN A 3TORAGE )NDEX IS ACCESSED OR UPDATED %VEN
THOUGH THERE ARE NO COMMANDS TO MANIPULATE 3TORAGE )NDEXES THEY ARE AN EXTREMELY POWERFUL FEATURE
AND CAN PROVIDE DRAMATIC PERFORMANCE IMPROVEMENTS &OR THAT REASON IT IS IMPORTANT TO UNDERSTAND
HOW THEY WORK
■ +EVIN 3AYS 4O KEEP THE ROLE OF 3TORAGE )NDEXES STRAIGHT IN MY MIND ) GENERALLY PICTURE AN OPTIMIZATION THAT
DRAMATICALLY IMPROVES SEARCHING FOR A NEEDLE IN A HAYSTACK AS OPPOSED TO FINDING WHERE CERTAIN PIECES OF STRAW
EXIST IN A HAYSTACK
Structure
3TORAGE )NDEXES CONSIST OF A MINIMUM AND A MAXIMUM VALUE FOR UP TO EIGHT COLUMNS 4HIS STRUCTURE IS
MAINTAINED FOR -" CHUNKS OF STORAGE STORAGE REGIONS 3TORAGE )NDEXES ARE STORED IN MEMORY ONLY AND
ARE NEVER WRITTEN TO DISK
#(!04%2 34/2!'% ).$%8%3
■ +EVIN 3AYS 3TORAGE )NDEXES ARE STORED IN THE HEAP OF RT[[bae SO TECHNICALLY SPEAKING THEY COULD END UP ON
DISK SWAP UNDER INSANE CONDITIONSxSO NOT NEVERx
&IGURE
SHOWS A CONCEPTUAL VIEW OF THE DATA CONTAINED IN A 3TORAGE )NDEX
Based on the
predicate, the
storage index
SELECT cust_state on this column
FROM customers will be used
WHERE cust_age > 35
Because only this 1MB
region of data satisfies 1MB
the predicate, only this Range
data will be scanned
1MB
Range
Figure 4-1. Conceptual diagram of a Storage Index
!S YOU CAN SEE IN THE DIAGRAM THE FIRST STORAGE REGION IN THE #USTOMER TABLE HAS A MAXIMUM VALUE OF
INDICATING THAT ITS POSSIBLE FOR IT TO CONTAIN ROWS THAT WILL SATISFY THE QUERY PREDICATE RdbcNPVT-"$
4HE OTHER STORAGE REGIONS IN THE DIAGRAM DO NOT HAVE MAXIMUM VALUES THAT ARE HIGH ENOUGH TO CONTAIN
ANY RECORDS THAT WILL SATISFY THE QUERY PREDICATE 4HEREFORE THOSE STORAGE REGIONS WILL NOT BE READ FROM
DISK
)N ADDITION TO THE MINIMUM AND MAXIMUM VALUES THERE IS A FLAG TO INDICATE WHETHER ANY OF THE
RECORDS IN A STORAGE REGION CONTAIN NULLS 4HE FACT THAT NULLS ARE REPRESENTED AT ALL IS SOMEWHAT SURPRISING
GIVEN THAT NULLS ARE NOT STORED IN TRADITIONAL /RACLE INDEXES 4HIS ABILITY OF 3TORAGE )NDEXES TO TRACK NULLS
MAY ACTUALLY HAVE REPERCUSSIONS FOR DESIGN AND IMPLEMENTATION DECISIONS 4HERE ARE SYSTEMS THAT DONT
USE NULLS AT ALL 3!0 FOR EXAMPLE USES A SINGLE SPACE CHARACTER INSTEAD OF NULLS 3!0 DOES THIS SIMPLY TO
INSURE THAT RECORDS CAN BE ACCESSED VIA "
4REE INDEXES WHICH DO NOT STORE NULLS !T ANY RATE 3TORAGE
)NDEXES PROVIDE THE EQUIVALENT OF A BIT
MAPPED INDEX ON NULLS WHICH MAKES FINDING NULLS A VERY EFFICIENT
PROCESS ASSUMING THEY REPRESENT A LOW PERCENTAGE OF THE VALUES
#(!04%2 34/2!'% ).$%8%3
Monitoring Storage Indexes
4HE ABILITY TO MONITOR 3TORAGE )NDEXES IS VERY LIMITED 4HE OPTIMIZER DOESNT KNOW WHETHER A 3TORAGE
)NDEX WILL BE USED FOR A PARTICULAR 31, STATEMENT .OR DO !72 OR !3( CAPTURE ANY INFORMATION ABOUT
WHETHER 3TORAGE )NDEXES WERE USED BY PARTICULAR 31, STATEMENTS 4HERE IS A SINGLE STATISTIC THAT TRACKS
3TORAGE )NDEX USAGE AT THE DATABASE LEVEL AND AN UNDOCUMENTED TRACING MECHANISM
$ATABASE 3TATISTICS
4HERE IS ONLY ONE DATABASE STATISTIC RELATED TO STORAGE INDEXES 4HE STATISTIC RT[[_WhbXRP[8>QhcTb
bPeTSQhbc^aPVTX]STg KEEPS TRACK OF THE ACCUMULATED )/ THAT HAS BEEN AVOIDED BY THE USE OF 3TORAGE
)NDEXES 4HIS STATISTIC IS EXPOSED IN ebTbbcPc AND ebhbbcPc AND RELATED VIEWS )TS A STRANGE STATISTIC
THAT CALCULATES A PRECISE VALUE FOR SOMETHING IT DIDNT DO .EVERTHELESS IT IS THE ONLY EASILY ACCESSIBLE
INDICATOR AS TO WHETHER 3TORAGE )NDEXES HAVE BEEN USED 5NFORTUNATELY SINCE THE STATISTIC IS CUMULATIVE
LIKE ALL STATISTICS IN ebTbbcPc IT MUST BE CHECKED BEFORE AND AFTER A GIVEN 31, STATEMENT IN ORDER TO
DETERMINE WHETHER 3TORAGE )NDEXES WERE USED ON THAT PARTICULAR STATEMENT (ERE IS AN EXAMPLE
BHB/4G31 -bTcTRW^^]
BHB/4G31 -/bX
BHB/4G31 -R^[]P\TU^aP&
BHB/4G31 -R^[eP[dTU^a((((((((((((((
BHB/4G31 -bT[TRc]P\TeP[dT
!Ua^\e\hbcPcbebcPc]P\T]
"fWTaT] bcPcXbcXR,b bcPcXbcXR
#P]S]P\T[XZTbc^aPVT*
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf!fWTaTR^[ Xb]d[[*
0E6?:N2>;
"!
BHB/4G31 -bTcTRW^^UU
BHB/4G31 -/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg"('#(#(!#'
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf!fWTaTR^[ Xb]d[[*
0E6?:N2>;
"!
#(!04%2 34/2!'% ).$%8%3
BHB/4G31 -/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg&(%('('#(%
!S YOU CAN SEE THE bX b`[ SCRIPT QUERIES e\hbcPc FOR A STATISTIC THAT CONTAINS THE WORD hSTORAGEv
4HE VALUE FOR THIS STATISTIC WILL BE UNTIL A 31, STATEMENT THAT USES A 3TORAGE )NDEX HAS BEEN EXECUTED IN
THE CURRENT SESSION )N OUR EXAMPLE THE QUERY USED A 3TORAGE )NDEX THAT ELIMINATED ABOUT BILLION BYTES
OF DISK )/ 4HIS IS THE AMOUNT OF ADDITIONAL )/ THAT WOULD HAVE BEEN NECESSARY WITHOUT 3TORAGE
)NDEXES .OTE THAT e\hbcPc IS A VIEW THAT EXPOSES CUMULATIVE STATISTICS FOR YOUR CURRENT SESSION 3O IF YOU
RUN THE STATEMENT A SECOND TIME THE VALUE SHOULD INCREASE TO TWICE THE VALUE IT HAD AFTER THE FIRST
EXECUTION /F COURSE DISCONNECTING FROM THE SESSION BY EXITING 31,
0LUS FOR EXAMPLE RESETS MOST
STATISTICS EXPOSED BY e\hbcPc INCLUDING THIS ONE TO
4RACING
4HERE IS ANOTHER WAY TO MONITOR WHAT IS GOING ON WITH 3TORAGE )NDEXES AT THE INDIVIDUAL STORAGE CELL
LEVEL 4HE RT[[bae PROGRAM HAS THE ABILITY TO CREATE TRACE FILES WHENEVER 3TORAGE )NDEXES ARE ACCESSED
4HIS TRACING CAN BE ENABLED BY SETTING THE N24;;NBC>A064N8=34GN3806N<>34 PARAMETER TO 425% IN THE
RT[[X]Xc ^aP FILE ON ONE OF THE STORAGE CELLS 9OU WILL NEED TO RESTART THE RT[[bae PROGRAM ONCE THIS
PARAMETER IS SET )N GENERAL THIS SHOULD NOT CAUSE ANY INTERRUPTION TO THE CLUSTERWARE OR DATABASES
RUNNING ON THE DATABASE MACHINES THANKS TO THE REDUNDANCY PROVIDED BY !3- (OWEVER YOU SHOULD BE
AWARE THAT IF YOU HAVE ISSUES ON OTHER STORAGE CELLS ITS POSSIBLE THAT RESTARTING A RT[[bae PROCESS COULD
CAUSE AN OUTAGE 4RACING CAN ALSO BE ENABLED ON ALL STORAGE SERVERS BY SETTING THE HIDDEN DATABASE
PARAMETER N:258BNBC>A06483GN3806N<>34 TO A VALUE OF 3INCE THIS TRACING MECHANISM IS COMPLETELY
UNDOCUMENTED IT SHOULD NOT BE USED WITHOUT APPROVAL FROM /RACLE SUPPORT "ETTER SAFE THAN SORRY
"ECAUSE THE RT[[bae PROCESS IS MULTITHREADED THE TRACING FACILITY CREATES MANY TRACE FILES 4HE RESULT
IS SIMILAR TO TRACING A SELECT STATEMENT THAT IS EXECUTED IN PARALLEL ON A DATABASE SERVER IN THAT THERE ARE
MULTIPLE TRACE FILES THAT NEED TO BE COMBINED TO SHOW THE WHOLE PICTURE 4HE NAMING CONVENTION FOR THE
TRACE FILES IS SVTRC? FOLLOWED BY A PROCESS )$ FOLLOWED BY A THREAD IDENTIFIER 4HE PROCESS )$ MATCHES THE
OPERATING SYSTEM PROCESS )$ OF THE RT[[bae PROCESS 3INCE RT[[bae ENABLES ONLY THREADS BY DEFAULT
THE FILE NAMES ARE REUSED RAPIDLY AS REQUESTS COME INTO THE STORAGE CELLS "ECAUSE OF THIS RAPID REUSE ITS
QUITE EASY TO WRAP AROUND THE THREAD NUMBER PORTION OF THE FILE NAME 3UCH WRAPPING AROUND DOESNT
WIPE OUT THE PREVIOUS TRACE FILE BUT RATHER APPENDS NEW DATA TO THE EXISTING FILE !PPENDING HAPPENS
WITH TRACE FILES ON /RACLE DATABASE SERVERS AS WELL BUT IS MUCH LESS COMMON BECAUSE THE PROCESS )$
PORTION OF THE DEFAULT FILE NAME COMES FROM THE USERS SHADOW PROCESS 3O BASICALLY EACH SESSION GETS ITS
OWN NUMBER
4HERE IS ANOTHER RELATED CELLSRV PARAMETER N24;;NB8N<0GN=D<N3806N<>34N3D<?B THAT SETS A
MAXIMUM NUMBER OF TRACE FILES THAT WILL BE CREATED BEFORE THE TRACING FUNCTIONALITY IS TURNED OFF 4HE
PARAMETER DEFAULTS TO A VALUE OF 0RESUMABLY THE PARAMETER IS A SAFETY MECHANISM TO KEEP THE DISK
FROM GETTING FILLED BY TRACE FILES SINCE A SINGLE QUERY CAN CREATE A LARGE NUMBER OF FILES
(ERE IS A SNIPPET FROM A TRACE FILE GENERATED ON OUR TEST SYSTEM
CaPRTUX[T
^_c ^aPR[T RT[[ ! ! ! N;8=DG G%#N ! % ! [^V SXPV Pb\ RT[[ T]ZRT[ " caPRT becaRN "!$"N
caR
>A02;4N7><4, ^_c ^aPR[T RT[[ ! ! ! N;8=DG G%#N ! % !
BhbcT\]P\T);X]dg
=^ST]P\T)T]ZRT[ " T]ZXcTR R^\
AT[TPbT)! % ' (# " " T[$
#(!04%2 34/2!'% ).$%8%3
ETabX^]) B<?CdT0dV" !!)# ) "43C!
<PRWX]T)g'%N%#
24;;BFETabX^])>BBN ! " N;8=DG G%#N ! % !
! ! & )# )# !&
DbTaCWaTPS);F?83) "'"#dbTa8S) ZTa]T[8S) _cWaTPS83) g!PPT%Q%'((#
! ! & !) )!% #( (& )5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbRP[[TSW^bc
T]ZSQ ! T]ZXcTR R^\J_XS)'$ (L]d\QTa^UUT]RX]VX]_a^VaTbb aTXS
RXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\],![]XS, & (( "((VXS,!"VX], V\], d\
T\XS, ^_XS,#'^_b], [e[,_a^RTbbWSa, gUTRT
! ! & !) )!% #(&!&&)5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbbTc]d\QTa^UUT]RX]VX]
_a^VaTbb aTXS
RXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\],![]XS, & (( "((VXS,!"VX], V\], d\
T\XS, ^_XS,#'^_b], [e[,_a^RTbbWSa, gUTRT
! ! & !)#$)!$ ( #!' )5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbRP[[TSW^bc
T]ZSQ T]ZXcTR R^\J_XS)("!%L]d\QTa^UUT]RX]VX]_a^VaTbb aTXS
RXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\], []XS, & (( "((VXS,
! #&#'"%#!VX], V\],"d\T\XS,"^_XS,#!^_b],"[e[,_a^RTbbWSa, gUTRT
! ! & !)#$)!$ ( $$(!)5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbbTc]d\QTa^UUT]RX]VX]
_a^VaTbb aTXSRXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\], []XS, & (( "((VXS,
! #&#'"%#!VX], V\],"d\T\XS,"^_XS,#!^_b],"[e[,_a^RTbbWSa, gUTRT
! ! & !)#$)# '&&')5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbRP[[TSW^bc
T]ZSQ T]ZXcTR R^\J_XS)("!%L]d\QTa^UUT]RX]VX]_a^VaTbb aTXS
RXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\], []XS, & (( "((VXS,
! #&#'"%#!VX], V\],"d\T\XS,"^_XS, ^_b], [e[,\T\QTaWSa, gUTRT
! ! & !)#$)# !!!$%)5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbbTc]d\QTa^UUT]RX]VX]
_a^VaTbb aTXSRXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\], []XS, & (( "((VXS,
! #&#'"%#!VX], V\],"d\T\XS,"^_XS, ^_b], [e[,\T\QTaWSa, gUTRT
! ! ! !) !)"# #%$"(')5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbRP[[TSW^bc
T]ZSQ ! T]ZXcTR R^\J_XS)'$ (L]d\QTa^UUT]RX]VX]_a^VaTbb aTXS
RXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\],![]XS, & (( "((VXS,
! #&#'"%#"VX], V\], d\T\XS, ^_XS,"(^_b], [e[,_a^RTbbWSa, gUTRT
! ! ! !) !)"# #& # ')5T]RT<PbcTa)>BBN8>2C;N54=24N4=C8CHXbbTc]d\QTa^UUT]RX]VX]
_a^VaTbb aTXSRXS,"RQ S "RSQ(RUU$TUU'Q'QUQ ( R%UT(XRX], & (( "((]\],![]XS, & (( "((VXS,
! #&#'"%#"VX], V\], d\T\XS, ^_XS,"(^_b], [e[,_a^RTbbWSa, gUTRT
! ! !" () &)"' !&&'!!)A83G g!PPT!U!(UTTRU^aB@;83%Sg!#&aehZa&!UX[cTa
! ! !" () &)"' !&&'!!)A83G g!PPT!U!(UTTR)bc!eP[XS1Xc<P_&UUUUUUUUUUUUUUUcPQ]
XSj&$&$(#"" #&& "('l
! ! !" () &)"' !&&'!!)A83G)bcac"!T]S! #'^UUbTc$""%$!'#'%# bXiT "! (!aV]8Sg
$ '(" AV]>UUbTc %"'#bR]) g &"US P&WXbc) g(
! ! !" () &)"' !&&'!!)A83GeP[XSPcX^]WXbc^ah) )?PacXP[ATPS )?PacXP[ATPS!)D]STU
")D]STU#)D]STU$)D]STU%)D]STU&)D]STU')D]STU()D]STU
! ! !" () &)"' !&&'!!)2^[XSJ!L]d\5X[c! U[V!)
! ! !" () &)"' !&&'!!)[^)R !
! ! !" () &)"' !&&'!!)WX)R"%#$ #Q
! ! !" () &)"' !&&'!!)2^[XSJ"L]d\5X[c U[V!)
! ! !" () &)"' !&&'!!)[^)% &"%#%#&"% %#%
! ! !" () &)"' !&&'!!)WX)% &"%#%#&"% %#%
! ! !" () &)"' !&&'!!)2^[XSJ$L]d\5X[c U[V!)
! ! !" () &)"' !&&'!!)[^)#T
! ! !" () &)"' !&&'!!)WX)$(
#(!04%2 34/2!'% ).$%8%3
! ! !" () &)"' !( $")A83G g!PPSUTP"S!S U^aB@;83%Sg!#&aehZa&!UX[cTa
! ! !" () &)"' !( $")A83G g!PPSUTP"S!S )bc!eP[XS1Xc<P_UUUUUUUUUUUUUUUUcPQ]
XSj&$&$(#"" #&& "('l
! ! !" () &)"' !( $")A83G)bcac T]S! #'^UUbTc$#%" "( %(%bXiT #'$&%aV]8Sg
$! ((%AV]>UUbTc bR]) g &"US"#QWXbc) g P
! ! !" () &)"' !( $")A83GeP[XSPcX^]WXbc^ah) )5d[[ATPS )?PacXP[FaXcT!)D]STU
")D]STU#)D]STU$)D]STU%)D]STU&)D]STU')D]STU()D]STU
! ! !" () &)"' !( $")2^[XSJ!L]d\5X[c! U[V!)
! ! !" () &)"' !( $")[^)R !
! ! !" () &)"' !( $")WX)R"%#$R'
! ! !" () &)"' !( $")2^[XSJ"L]d\5X[c U[V!)
! ! !" () &)"' !( $")[^)% &"%#%#&"% %#%
! ! !" () &)"' !( $")WX)% &"%#%#&"% %#%
! ! !" () &)"' !( $")2^[XSJ$L]d\5X[c U[V!)
! ! !" () &)"' !( $")[^)$(
! ! !" () &)"' !( $")WX)$(
! ! !" () &)"' !(!#$()A83G g!PPSUTP"S"SRU^aB@;83%Sg!#&aehZa&!UX[cTa
! ! !" () &)"' !(!#$()A83G g!PPSUTP"S"SR)bc!eP[XS1Xc<P_UUUUUUUUUUUUUUUUcPQ]
XSj&$&$(#"" #&& "('l
! ! !" () &)"' !(!#$()A83G)bcac T]S! #'^UUbTc$#%" #($ !&!bXiT #'$&%aV]8Sg
$! ((&AV]>UUbTc bR]) g &"US"#QWXbc) g P
! ! !" () &)"' !(!#$()A83GeP[XSPcX^]WXbc^ah) )5d[[ATPS )?PacXP[FaXcT!)D]STU
")D]STU#)D]STU$)D]STU%)D]STU&)D]STU')D]STU()D]STU
! ! !" () &)"' !(!#$()2^[XSJ!L]d\5X[c! U[V!)
! ! !" () &)"' !(!#$()[^)R !
! ! !" () &)"' !(!#$()WX)R"%#!&#U
! ! !" () &)"' !(!#$()2^[XSJ"L]d\5X[c U[V!)
! ! !" () &)"' !(!#$()[^)% &"%#%#&"% %#%
! ! !" () &)"' !(!#$()WX)% &"%#%#&"% %#%
! ! !" () &)"' !(!#$()2^[XSJ$L]d\5X[c U[V!)
! ! !" () &)"' !(!#$()[^)$(
! ! !" () &)"' !(!#$()WX)$(
3EVERAL THINGS ARE WORTH POINTING OUT IN THIS TRACE FILE
v 4HE FIRST SEVERAL LINES ARE THE STANDARD TRACE FILE HEADER WITH FILE NAME AND
SOFTWARE VERSION
v 4HE LINES THAT BEGIN WITH TIMESTAMPS ARE EACH ASSOCIATED WITH A SINGLE STORAGE
REGION
v 4HE B@;83 OF THE STATEMENT THAT GENERATED THE TRACE FILE IS IDENTIFIED FOR EACH
STORAGE REGION
v %ACH LINE CONTAINING THE B@;83 KEYWORD BEGINS DATA FOR A NEW STORAGE REGION
v 4HE LINE BELOW THE B@;83 LINE CONTAINS AN )$ WHICH CONTAINS THE SPcPN^QYTRcNXS
FROM 310N>1942CB FOR THE TABLE BEING SCANNED
v 4HE aTVX^]BXiT FIELDS SHOW THAT THE STORAGE REGIONS REALLY ARE -"
v )T LOOKS LIKE EACH 3TORAGE )NDEX ENTRY OCCUPIES + OF MEMORY BASED ON THE bcac
AND T]S FIELD VALUES
#(!04%2 34/2!'% ).$%8%3
v &OR EACH COLUMN EVALUATED THERE IS AN XS FIELD THAT CORRELATES TO ITS POSITION IN THE
TABLE
v &OR EACH COLUMN EVALUATED THERE IS A U[V FIELD )T APPEARS THAT IT IS THE DECIMAL
REPRESENTATION OF A BIT MASK )T ALSO APPEARS THAT THE FIRST BIT INDICATES WHETHER
NULLS ARE CONTAINED IN THE CURRENT COLUMN OF THE STORAGE REGION 4HAT IS AND
BOTH INDICATE THAT NULLS ARE PRESENT
v &OR EACH COLUMN EVALUATED THERE IS A [^ AND A WX VALUE STORED AS HEX
v 4HE [^ AND WX VALUES ARE ONLY BYTES INDICATING THAT THE 3TORAGE )NDEXES WILL BE
INEFFECTIVE ON COLUMNS WHERE THE LEADING PORTION OF THE VALUES ARE NOT DISTINCT
EMPIRICAL EVIDENCE BEARS THIS OUT BY THE WAY
7HILE GENERATING AND READING TRACE FILES IS VERY INFORMATIVE IT IS NOT VERY EASY TO DO AND REQUIRES
DIRECT ACCESS TO THE STORAGE SERVERS /N TOP OF THAT THE APPROACH IS COMPLETELY UNDOCUMENTED )T IS
PROBABLY BEST USED FOR INVESTIGATIONS IN NONPRODUCTION ENVIRONMENTS
-ONITORING 7RAP 5P
.EITHER THE DATABASE STATISTIC NOR THE TRACING IS A PARTICULARLY SATISFYING WAY OF MONITORING 3TORAGE )NDEX
USAGE )T WOULD BE NICE TO BE ABLE TO TRACK 3TORAGE )NDEX USAGE AT THE STATEMENT LEVEL VIA A COLUMN IN
631, FOR EXAMPLE )N THE MEANTIME THE RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg STATISTIC IS
THE BEST OPTION WE HAVE
Controlling Storage Indexes
4HERE IS NOT MUCH YOU CAN DO TO CONTROL 3TORAGE )NDEX BEHAVIOR (OWEVER THE DEVELOPERS HAVE BUILT IN A
FEW HIDDEN PARAMETERS THAT PROVIDE SOME FLEXIBILITY
4HERE ARE THREE DATABASE PARAMETERS THAT DEAL WITH 3TORAGE )NDEXES THAT WERE AWARE OF
v NZRUXbNbc^aPVTXSgNSXbPQ[TS DEFAULT IS &!,3%
v NZRUXbNbc^aPVTXSgNSXPVN\^ST DEFAULT IS
v NRT[[Nbc^aXSgN\^ST DEFAULT IS %6!
.ONE OF THESE PARAMETERS ARE DOCUMENTED SO YOU NEED TO BE CAREFUL WITH THE METHODS WE DISCUSS
IN THIS SECTION .EVERTHELESS WELL TELL YOU A LITTLE BIT ABOUT SOME OF THESE PARAMETERS AND WHAT THEY CAN
DO
?KCFIS?STORAGEIDX?DISABLED
4HE NZRUXbNbc^aPVTXSgNSXbPQ[TS PARAMETER ALLOWS 3TORAGE )NDEXES TO BE DISABLED !S WITH ALL HIDDEN
PARAMETERS ITS BEST TO CHECK WITH /RACLE SUPPORT BEFORE SETTING IT BUT AS HIDDEN PARAMETERS GO THIS ONE
IS RELATIVELY INNOCUOUS 7E HAVE USED IT EXTENSIVELY IN TESTING AND HAVE NOT EXPERIENCED ANY NEGATIVE
CONSEQUENCES
9OU CAN SET THE PARAMETER AT THE SESSION LEVEL WITH THE P[cTabTbbX^] STATEMENT
P[cTabTbbX^]bTcNZRUXbNbc^aPVTXSgNSXbPQ[TS,cadT*
.OTE THAT ALTHOUGH SETTING NZRUXbNbc^aPVTXSgNSXbPQ[TS TO 425% DISABLES 3TORAGE )NDEXES FOR READS
THE SETTING DOES NOT DISABLE THE MAINTENANCE OF EXISTING 3TORAGE )NDEXES 4HAT IS TO SAY THAT EXISTING
#(!04%2 34/2!'% ).$%8%3
3TORAGE )NDEXES WILL STILL BE UPDATED WHEN VALUES IN A TABLE ARE CHANGED EVEN IF THIS PARAMETER IS SET TO
425%
?KCFIS?STORAGEIDX?DIAG?MODE
4HE SECOND PARAMETER NN:258BNBC>A06483GN3806N<>34 LOOKS EERILY LIKE THE RT[[X]Xc ^aP PARAMETER
N24;;NBC>A064N8=34GN3806N<>34 WHICH WAS DISCUSSED EARLIER !S YOU MIGHT EXPECT SETTING THIS
PARAMETER AT THE DATABASE LAYER CAUSES TRACE FILES TO BE GENERATED ACROSS ALL THE AFFECTED STORAGE CELLS
3ETTING IT TO A VALUE OF ENABLES TRACING /DDLY SETTING IT TO A VALUE OF DISABLES 3TORAGE )NDEXES
5NFORTUNATELY THE TRACE FILES ARE CREATED ON THE STORAGE CELLS "UT THIS METHOD OF GENERATING THEM IS
MUCH LESS INTRUSIVE THAN RESTARTING THE RT[[bae PROCESS ON A STORAGE SERVER
9OU CAN SET THE PARAMETER AT THE SESSION LEVEL WITH THE P[cTabTbbX^] STATEMENT
P[cTabTbbX^]bTcNZRUXbNbc^aPVTXSgNSXPVN\^ST,!*
4HERE MAY BE OTHER VALID VALUES FOR THE PARAMETER THAT ENABLE DIFFERENT LEVELS OF TRACING +EEP IN
MIND THAT THIS WILL PRODUCE A LARGE NUMBER OF TRACE FILES ON EVERY STORAGE CELL THAT IS INVOLVED IN A QUERY
THAT USES 3TORAGE )NDEXES
?CELL?STORIDX?MODE
4HE N24;;NBC>A83GN<>34 PARAMETER WAS ADDED IN THE SECOND POINT RELEASE OF /RACLE $ATABASE G2
7HILE THIS PARAMETER IS UNDOCUMENTED IT APPEARS THAT IT CONTROLS WHERE 3TORAGE )NDEXES WILL
BE APPLIED 4HERE ARE THREE VALID VALUES FOR THIS PARAMETER 4E0:3BC0;; 4E0 AND :3BC ARE /RACLE KERNEL
FUNCTION NAMES
9OU CAN SET THE PARAMETER AT THE SESSION LEVEL WITH THE P[cTabTbbX^] STATEMENT
P[cTabTbbX^]bTcNRT[[Nbc^aXSgN\^ST,0;;*
4HE EFFECTS OF THIS PARAMETER HAVE VARIED ACROSS RELEASES !S OF CELLSRV VERSION %6! THE
DEFAULT SUPPORTS ALL THE VALID COMPARISON OPERATORS 9OU SHOULD NOTE THAT IN OLDER VERSIONS THE %6!
SETTING DID NOT SUPPORT THE )3 .5,, COMPARISON OPERATOR )TS ALSO IMPORTANT TO KEEP IN MIND THAT THE
DATABASE PATCHING IS TIED TO THE STORAGE SOFTWARE PATCHING 5PGRADING THE VERSION OF CELLSRV WITHOUT
PATCHING THE DATABASE SOFTWARE CAN RESULT IN UNPREDICATABLE BEHAVIOR DISABLING STORAGE INDEXES FOR
EXAMPLE
3TORAGE 3OFTWARE 0ARAMETERS
)N ADDITION TO THE DATABASE PARAMETERS THERE ARE ALSO A NUMBER OF UNDOCUMENTED STORAGE SOFTWARE
PARAMETERS THAT ARE RELATED TO 3TORAGE )NDEX BEHAVIOR 4HESE PARAMETERS CAN BE MODIFIED BY ADDING
THEM TO THE RT[[X]Xc ^aP FILE AND THEN RESTARTING RT[[bae .OTE THAT CELLINITORA WILL BE DISCUSSED IN MORE
DETAIL IN #HAPTER (ERE IS A LIST OF THE CELLINITORA 3TORAGE )NDEX PARAMETERS ALONG WITH THEIR DEFAULT
VALUES
v NRT[[NT]PQ[TNbc^aPVTNX]STgNU^aN[^PSb,CAD4
v NRT[[NT]PQ[TNbc^aPVTNX]STgNU^aNfaXcTb,CAD4
v NRT[[NbXN\PgN]d\NSXPVN\^STNSd\_b,!
v NRT[[Nbc^aPVTNX]STgNR^[d\]b,
v NRT[[Nbc^aPVTNX]STgNSXPVN\^ST,50;B4
#(!04%2 34/2!'% ).$%8%3
v NRT[[Nbc^aPVTNX]STgN_PacXP[NaSNbTRc^ab,$ !
v NRT[[Nbc^aPVTNX]STgN_PacXP[NaTPSbNcWaTbW^[SN_TaRT]c,'$
v NRT[[Nbc^aPVTNX]STgNbXiX]VNUPRc^a,!
9OUVE ALREADY SEEN THE TRACING PARAMETERS N24;;NBC>A064N8=34GN3806N<>34AND
N24;;NB8N<0GN=D<N3806N<>34N3D<?B IN THE SECTION h-ONITORING 3TORAGE )NDEXESv 4HESE TWO PARAMETERS
ARE THE MOST USEFUL IN OUR OPINION ALTHOUGH YOU SHOULD GET THE IDEA FROM THE LIST THAT THERE IS ALSO SOME
BUILT IN ABILITY TO MODIFY BEHAVIORS SUCH AS THE AMOUNT OF MEMORY TO ALLOCATE FOR STORAGE INDEXES AND THE
NUMBER OF COLUMNS THAT CAN BE INDEXED PER TABLE
Behavior
4HERE IS NOT A LOT YOU CAN DO TO CONTROL WHEN 3TORAGE )NDEXES ARE USED AND WHEN THEY ARE NOT /THER
THAN THE PARAMETER FOR DISABLING THEM THERE IS LITTLE YOU CAN DO 4HERE IS NO SPECIFIC HINT TO ENABLE OR
DISABLE THEIR USE !ND UNFORTUNATELY THE >?CN?0A0< HINT DOES NOT WORK WITH THE
N:258BNBC>A06483GN38B01;43 PARAMETER EITHER 4HE FACT THAT THERE IS NO WAY TO FORCE THE USE OF A 3TORAGE
)NDEX MAKES IT EVEN MORE IMPORTANT TO UNDERSTAND WHEN THIS POWERFUL OPTIMIZATION WILL AND WILL NOT BE
USED
)N ORDER FOR A STORAGE INDEX TO BE USED A QUERY MUST INCLUDE OR MAKE USE OF ALL THE FOLLOWING
Smart Scan: 3TORAGE )NDEXES CAN ONLY BE USED WITH STATEMENTS THAT DO 3MART
3CANS 4HIS COMES WITH A WHOLE SET OF REQUIREMENTS AS DETAILED IN #HAPTER
4HE MAIN REQUIREMENTS ARE THAT THE OPTIMIZER MUST CHOOSE A FULL SCAN AND THAT
THE )/ MUST BE DONE VIA THE DIRECT PATH READ MECHANISM
At Least One Predicate: )N ORDER FOR A STATEMENT TO USE A 3TORAGE )NDEX THERE
MUST BE A F74A4 CLAUSE WITH AT LEAST ONE PREDICATE
Simple Comparison Operators: 3TORAGE )NDEXES CAN BE USED WITH THE FOLLOWING
SET OF OPERATORS
,+-14CF44=-,+,8=8B=D;;8B=>C=D;;
)F A QUERY MEETS THE REQUIREMENTS OF HAVING AT LEAST ONE PREDICATE INVOLVING SIMPLE COMPARISON
OPERATORS AND IF THAT QUERYS EXECUTION MAKES USE OF 3MART 3CAN THEN THE STORAGE SOFTWARE CAN MAKE
USE OF STORAGE INDEXES 4HEY CAN BE APPLIED TO ANY OF THE FOLLOWING ASPECTS OF THE QUERY
Multi-Column Predicates 3TORAGE )NDEXES CAN BE USED WITH MULTIPLE
PREDICATES ON THE SAME TABLE
Joins: 3TORAGE )NDEXES CAN BE USED ON STATEMENTS ACCESSING MULTIPLE TABLES TO
MINIMIZE DISK )/ BEFORE THE *OIN OPERATIONS ARE CARRIED OUT
Parallel Query: 3TORAGE )NDEXES CAN BE USED BY PARALLEL QUERY SLAVES )N FACT
SINCE DIRECT PATH READS ARE REQUIRED TO ENABLE 3TORAGE )NDEXES PARALLEL QUERIES
ARE VERY USEFUL FOR ENSURING THAT 3TORAGE )NDEXES CAN BE USED
HCC: 3TORAGE )NDEXES WORK WITH (## COMPRESSED TABLES
Bind Variables: 3TORAGE )NDEXES WORK WITH BIND VARIABLES 4HE VALUES OF THE
BIND VARIABLES APPEAR TO BE PASSED TO THE STORAGE CELLS WITH EACH EXECUTION
#(!04%2 34/2!'% ).$%8%3
Partitions –3TORAGE )NDEXES WORK WITH PARTITIONED OBJECTS )NDIVIDUAL
STATEMENTS CAN BENEFIT FROM PARTITION ELIMINATE AND STORAGE INDEXES DURING THE
SAME EXECUTION
Sub-queries: 3TORAGE )NDEXES WORK WITH PREDICATES THAT COMPARE A COLUMN TO A
VALUE RETURNED BY A SUB
QUERY
Encryption: 3TORAGE )NDEXES WORK ON ENCRYPTED TABLES
4HERE ARE OF COURSE LIMITATIONS &OLLOWING ARE SOME FEATURES AND SYNTAX THAT PREVENT THE USE OF
STORAGE INDEXES
CLOBs 3TORAGE )NDEXES ARE NOT CREATED ON #,/"S
, 3TORAGE )NDEXES DO NOT WORK WITH PREDICATES THAT USE THE , COMPARISON
OPERATOR
Wildcards 3TORAGE )NDEXES DO NOT WORK ON PREDICATES THAT USE THE WILDCARD
! FURTHER LIMITATION IS THAT STORAGE INDEXES MAY CONTAIN ONLY EIGHT COLUMNS 4HEY ARE CREATED AND
MAINTAINED FOR EIGHT
COLUMNS PER TABLE 4HIS DOES NOT MEAN THAT QUERIES WITH MORE THAN PREDICATES
CANNOT MAKE USE OF 3TORAGE )NDEXES )N SUCH CASES THE STORAGE SOFTWARE CAN USE THE INDEXES THAT EXIST
BUT BY DEFAULT THERE WILL BE A MAXIMUM OF EIGHT COLUMNS THAT CAN BE INDEXED )T DOES APPEAR THAT THE
DEVELOPERS HAVE PARAMETERIZED THIS SETTING SO IT MAY BE POSSIBLE TO CHANGE THIS VALUE WITH HELP FROM
/RACLE SUPPORT
&INALLY BEAR IN MIND THAT STORAGE INDEXES ARE NOT PERSISTED TO DISK 4HE STORAGE CELL MUST REBUILD
THEM WHENEVER THE RT[[bae PROGRAM IS RESTARTED 4HEY ARE GENERALLY CREATED DURING THE FIRST SMART SCAN
THAT REFERENCES A GIVEN COLUMN AFTER A STORAGE SERVER HAS BEEN RESTARTED 4HEY CAN ALSO BE CREATED WHEN A
TABLE IS CREATED VIA A 2A40C4C01;40BB4;42C STATEMENT OR DURING OTHER DIRECT
PATH LOADS !ND OF
COURSE THE STORAGE CELL WILL UPDATE STORAGE INDEXES IN RESPONSE TO CHANGES THAT APPLICATIONS MAKE TO THE
DATA IN THE TABLES
Performance
3TORAGE )NDEXES PROVIDE SOME OF THE MOST DRAMATIC PERFORMANCE BENEFITS AVAILABLE ON THE %XADATA
PLATFORM $EPENDING ON THE CLUSTERING FACTOR OF A PARTICULAR COLUMN THAT IS HOW WELL THE COLUMNS DATA
IS SORTED ON DISK THE RESULTS CAN BE SPECTACULAR (ERES A TYPICAL EXAMPLE SHOWING THE PERFORMANCE OF A
QUERY WITH AND WITHOUT THE BENEFIT OF 3TORAGE )NDEXES
BHB/4G31 -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,UP[bT*
BTbbX^]P[cTaTS
BHB/4G31 -P[cTabTbbX^]bTcNZRUXbNbc^aPVTXSgNSXbPQ[TS,cadT*
BTbbX^]P[cTaTS
#(!04%2 34/2!'% ).$%8%3
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
a^fbT[TRcTS
4[P_bTS) ) )#$ "(
BHB/4G31 -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,cadT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
a^fbT[TRcTS
4[P_bTS) ) )!" &
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT_ZNR^[,& *
2>D=C
!
4[P_bTS) ) ) " &#
BHB/4G31 -P[cTabTbbX^]bTcNZRUXbNbc^aPVTXSgNSXbPQ[TS,UP[bT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT_ZNR^[,& *
2>D=C
!
4[P_bTS) ) ) %
!T THE START OF THIS DEMONSTRATION ALL OFFLOADING WAS DISABLED VIA THE DATABASE INITIALIZATION
PARAMETER 24;;N>55;>03N?A>24BB8=6 3TORAGE )NDEXES WERE ALSO DISABLED VIA THE HIDDEN PARAMETER
N:258BNBC>A06483GN38B01;43 ! QUERY WITHOUT A F74A4 CLAUSE WAS RUN AND WAS COMPLETED USING DIRECT
PATH READS BUT WITHOUT OFFLOADING 4HAT QUERY TOOK MINUTE AND SECONDS TO DO THE FULL TABLE SCAN AND
#(!04%2 34/2!'% ).$%8%3
RETURNED ENTIRE BLOCKS TO THE DATABASE GRID JUST AS IT WOULD ON NON %XADATA STORAGE ENVIRONMENTS
/FFLOADING WAS THEN RE ENABLED AND THE QUERY WAS REPEATED 4HIS TIME IT COMPLETED IN ABOUT SECONDS
4HE IMPROVEMENT IN ELAPSED TIME WAS PRIMARILY DUE TO COLUMN PROJECTION SINCE THE STORAGE LAYER ONLY
HAD TO RETURN A COUNTER OF ROWS INSTEAD OF RETURNING ANY OF THE COLUMN VALUES
! VERY SELECTIVE F74A4 CLAUSE WAS THEN ADDED TO THE QUERY IT REDUCED THE TIME TO ABOUT SECONDS
4HIS IMPROVEMENT WAS THANKS TO PREDICATE FILTERING 2EMEMBER THAT 3TORAGE )NDEXES WERE STILL TURNED
OFF ! COUNTER FOR ONLY ROWS HAD TO BE RETURNED TO THE DATABASE MACHINE BUT THE STORAGE CELLS STILL HAD
TO READ ALL THE DATA TO DETERMINE WHICH ROWS TO RETURN &INALLY THE 3TORAGE )NDEXES WERE RE ENABLED BY
SETTING N:258BNBC>A06483GN38B01;43 TO &!,3% AND THE QUERY WITH THE F74A4 CLAUSE WAS EXECUTED AGAIN
4HIS TIME THE ELAPSED TIME WAS ONLY ABOUT SECOND 7HILE THIS PERFORMANCE IMPROVEMENT SEEMS
EXTREME IT IS RELATIVELY COMMON WHEN 3TORAGE )NDEXES ARE USED
3PECIAL /PTIMIZATION FOR .ULLS
.ULLS ARE A SPECIAL CASE FOR 3TORAGE )NDEXES 4HERE IS A SEPARATE FLAG IN THE 3TORAGE )NDEX STRUCTURE THAT IS
USED TO INDICATE WHETHER A STORAGE REGION CONTAINS NULLS OR NOT 4HIS SEPARATE FLAG MAKES QUERIES LOOKING
FOR NULLS OR THE ABSENCE OF NULLS EVEN MORE EFFICIENT THAN THE NORMAL MINIMUM AND MAXIMUM
COMPARISONS THAT ARE TYPICALLY DONE (ERES AN EXAMPLE COMPARING TYPICAL PERFORMANCE WITH AND
WITHOUT THE SPECIAL NULL OPTIMIZATION
BHB/4G31 -bTccX\X]V^]
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[ , *
2>D=C
4[P_bTS) ) ) # #&
BHB/4G31 -/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg"( $&&%
4[P_bTS) ) )
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[ Xb]d[[*
2>D=C
#
4[P_bTS) ) ) !
BHB/4G31 -/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg $($#& #%!#
4[P_bTS) ) )
)N THIS EXAMPLE YOU CAN SEE THAT RETRIEVAL OF A FEW NULLS WAS EXTREMELY FAST 4HIS IS BECAUSE THERE IS
NO POSSIBILITY THAT ANY STORAGE REGION THAT DOESNT CONTAIN A NULL WILL HAVE TO BE READ SO NO FALSE
#(!04%2 34/2!'% ).$%8%3
POSITIVES WILL SLOW DOWN THIS QUERY 7ITH ANY OTHER VALUE EXCEPT THE MINIMUM OR MAXIMUM VALUE FOR A
COLUMN THERE WILL MOST LIKELY BE STORAGE REGIONS THAT CANT BE ELIMINATED EVEN THOUGH THEY DONT
ACTUALLY CONTAIN A VALUE THAT MATCHES THE PREDICATES 4HIS IS EXACTLY THE CASE IN THE PREVIOUS EXAMPLE
WHERE NO RECORDS WERE RETURNED FOR THE FIRST QUERY EVEN THOUGH IT TOOK SECONDS TO READ ALL THE DATA
FROM DISK .OTICE ALSO THAT THE AMOUNT OF )/ SAVED BY THE NULL QUERY IS ALMOST GIGABYTES '" WHILE
THE AMOUNT SAVED BY THE FIRST QUERY WAS ONLY ABOUT -" 4HAT MEANS THAT THE FIRST QUERY FOUND ALMOST
NO STORAGE REGIONS THAT IT COULD ELIMINATE
0HYSICAL $ISTRIBUTION OF 6ALUES
3TORAGE )NDEXES BEHAVE VERY DIFFERENTLY FROM NORMAL INDEXES 4HEY MAINTAIN A FAIRLY COARSE PICTURE OF THE
VALUES THAT ARE STORED ON DISK (OWEVER THEIR MECHANISM CAN BE VERY EFFECTIVE AT ELIMINATING LARGE
AMOUNTS OF DISK )/ IN CERTAIN SITUATIONS WHILE STILL KEEPING THE COST OF MAINTAINING THEM RELATIVELY LOW
)TS IMPORTANT TO KEEP IN MIND THAT THE PHYSICAL DISTRIBUTION OF DATA ON DISK WILL HAVE A LARGE IMPACT ON
HOW EFFECTIVE THE 3TORAGE )NDEXES ARE !N ILLUSTRATION WILL MAKE THIS CLEARER
3UPPOSE YOU HAVE A TABLE THAT HAS A COLUMN WITH UNIQUE VALUES THAT IS NO VALUE IS REPEATED )F THE
DATA IS STORED ON DISK IN SUCH A MANNER THAT THE ROWS ARE ORDERED BY THAT COLUMN THEN THERE WILL BE ONE
AND ONLY ONE STORAGE REGION FOR ANY GIVEN VALUE OF THAT COLUMN !NY QUERY WITH AN EQUALITY PREDICATE ON
THAT COLUMN WILL HAVE TO READ AT MOST ONE STORAGE REGION &IGURE
SHOWS A CONCEPTUAL PICTURE OF A
3TORAGE )NDEX FOR A SORTED COLUMN
Figure 4-2. A Storage Index on a sorted column
!S YOU CAN SEE FROM THE DIAGRAM IF YOU WANTED TO RETRIEVE THE RECORD WHERE THE VALUE WAS YOU
WOULD ONLY HAVE ONE STORAGE REGION THAT COULD POSSIBLY CONTAIN THAT VALUE
3UPPOSE NOW THAT THE SAME DATA SET IS STORED ON DISK IN A RANDOM ORDER (OW MANY STORAGE REGIONS
WOULD YOU EXPECT TO HAVE TO READ TO LOCATE A SINGLE ROW VIA AN EQUALITY PREDICATE )T DEPENDS ON THE
NUMBER OF ROWS THAT FIT INTO A STORAGE REGION BUT THE ANSWER IS CERTAINLY MUCH LARGER THAN THE STORAGE
REGION THAT WOULD BE REQUIRED WITH THE SORTED DATA SET
#(!04%2 34/2!'% ).$%8%3
■ +EVIN 3AYS #HOOSING TO SORT DATA AT LOAD TIME IS A hPAY NOW OR PAY LATERv PROPOSITION )F SORTING DATA FITS WITHIN
THE OPPORTUNITY WINDOW FOR LOADING DATA THE EXTRA EFFORT WILL PAY DIVIDENDS IN INCREASED 3TORAGE )NDEX
EFFECTIVENESS AND QUITE OFTEN IMPROVED (YBRID #OLUMNAR #OMPRESSION RATIOS
)TS JUST THAT SIMPLE 3TORAGE INDEXES WILL BE MORE EFFECTIVE ON SORTED DATA &ROM A PERFORMANCE
PERSPECTIVE THE BETTER SORTED THE DATA IS ON DISK THE FASTER THE AVERAGE ACCESS TIME WILL BE WHEN USING
3TORAGE )NDEXES &OR A COLUMN THAT IS COMPLETELY SORTED THE ACCESS TIME SHOULD BE VERY FAST AND THERE
SHOULD BE LITTLE VARIATION IN THE ACCESS TIME REGARDLESS OF WHAT VALUES ARE REQUESTED &OR UNSORTED DATA
THE ACCESS TIMES WILL BE FASTER TOWARD THE ENDS OF THE RANGE OF VALUES BECAUSE THERE ARE NOT MANY STORAGE
REGIONS THAT WILL HAVE RANGES OF VALUES CONTAINING THE QUERIED VALUE 4HE AVERAGE ACCESS TIMES FOR VALUES
IN THE MIDDLE OF THE DISTRIBUTION WILL VARY WIDELY &IGURE
IS A CHART COMPARING ACCESS TIMES USING
3TORAGE )NDEXES FOR SORTED AND UNSORTED DATA
8
4 Unsorted Values
3 Sorted Values
0
0 5 10 15 20 25
Figure 4-3. Storage Index access times – sorted vs. unsorted
!S YOU CAN SEE SORTED DATA WILL PROVIDE BETTER AND MORE CONSISTENT RESULTS 7HILE WERE ON THE
SUBJECT ) SHOULD POINT OUT THAT THERE ARE MANY CASES WHERE SEVERAL COLUMNS WILL BENEFIT FROM THIS
BEHAVIORAL CHARACTERISTIC OF 3TORAGE )NDEXES )TS COMMON IN DATA WAREHOUSE ENVIRONMENTS TO HAVE DATA
THAT IS PARTITIONED ON A DATE COLUMN !ND THERE ARE OFTEN MANY OTHER COLUMNS THAT TRACK THE PARTITION
KEY SUCH AS ASSOCIATED DATES ORDER DATE SHIP DATE INSERT DATE RETURN DATE FOR EXAMPLE OR SEQUENTIALLY
GENERATED NUMBERS LIKE ORDER NUMBERS 1UERIES AGAINST THESE COLUMN ARE OFTEN PROBLEMATIC DUE THE FACT
THAT PARTITION ELIMINATE CANNOT HELP THEM 3TORAGE )NDEXES WILL PROVIDE A SIMILAR BENEFIT TO PARTITION
ELIMINATION AS LONG AS CARE IS TAKEN TO ENSURE THAT THE DATA IS PRE
SORTED PRIOR TO LOADING
Potential Issues
4HERES NO SUCH THING AS A FREE PUPPY !S WITH EVERYTHING IN LIFE THERE ARE A FEW ISSUES WITH 3TORAGE
)NDEXES THAT YOU SHOULD BE AWARE OF
#(!04%2 34/2!'% ).$%8%3
)NCORRECT 2ESULTS
"Y FAR THE BIGGEST ISSUE WITH 3TORAGE )NDEXES HAS BEEN THAT IN EARLY RELEASES OF THE %XADATA 3TORAGE
3OFTWARE THERE WERE A HANDFUL OF BUGS REGARDING INCORRECT RESULTS 4HAT IS TO SAY THAT IN CERTAIN SITUATIONS
USAGE OF 3TORAGE )NDEXES COULD ELIMINATE STORAGE REGIONS FROM CONSIDERATION THAT ACTUALLY CONTAINED
RECORDS OF INTEREST 4HIS INCORRECT ELIMINATION COULD OCCUR DUE TO TIMING ISSUES WITH CONCURRENT $-,
WHILE A 3MART 3CAN WAS BEING DONE USING 3TORAGE )NDEXES 4HESE BUGS HAVE BEEN ADDRESSED IN
AND THE LATEST PATCHES ON THE STORAGE SERVERS )F YOU RUN INTO THIS ISSUE DISABLING 3TORAGE )NDEX USE VIA
THE HIDDEN PARAMETER N:258BNBC>A06483GN38B01;43 MAY BE THE ONLY OPTION AVAILABLE UNTIL THE PROPER
PATCHES ARE APPLIED 4HIS PARAMETER CAN BE SET WITH AN P[cTabTbbX^] COMMAND SO THAT ONLY
PROBLEMATIC QUERIES ARE AFFECTED /F COURSE YOU SHOULD CHECK WITH /RACLE 3UPPORT BEFORE ENABLING ANY
HIDDEN PARAMETERS
-OVING 4ARGET
3TORAGE )NDEXES CAN BE A LITTLE FRUSTRATING BECAUSE THEY DONT ALWAYS KICK IN WHEN YOU EXPECT THEM TO
!ND WITHOUT ANY HINTS TO TELL /RACLE THAT YOU REALLY WANT A 3TORAGE )NDEX TO BE USED THERE IS LITTLE YOU CAN
DO OTHER THAN TRY TO UNDERSTAND WHY THEY ARE NOT USED IN CERTAIN CIRCUMSTANCES SO YOU CAN AVOID THOSE
CONDITIONS IN THE FUTURE
)N EARLY VERSIONS OF THE STORAGE SERVER SOFTWARE ONE OF THE MAIN REASONS THAT 3TORAGE )NDEXES WERE
DISABLED WAS DUE TO IMPLICIT DATA TYPE CONVERSIONS /VER THE YEARS /RACLE HAS GOTTEN BETTER AND BETTER AT
DOING hSMARTv DATA TYPE CONVERSIONS THAT DONT HAVE NEGATIVE PERFORMANCE CONSEQUENCES &OR EXAMPLE
IF YOU WRITE A 31, STATEMENT WITH A F74A4 CLAUSE THAT COMPARES A DATE FIELD TO A CHARACTER STRING /RACLE
WILL USUALLY APPLY A c^NSPcT FUNCTION TO THE CHARACTER STRING INSTEAD OF MODIFYING THE DATE COLUMN WHICH
COULD HAVE THE UNPLEASANT SIDE EFFECT OF DISABLING AN INDEX 5NFORTUNATELY WHEN THE %XADATA STORAGE
SOFTWARE WAS RELATIVELY NEW ALL THE NUANCES HAD NOT BEEN WORKED OUT AT LEAST TO THE DEGREE WERE USED TO
FROM THE DATABASE SIDE $ATES HAVE BEEN PARTICULARLY PERSNICKETY (ERE IS AN EXAMPLE USING CELLSRV
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[",! >2C $*
2>D=C
4[P_bTS) ) ) #
BHB/4G31 -/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg
#(!04%2 34/2!'% ).$%8%3
4[P_bTS) ) )
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[",! >2C ! $*
2>D=C
4[P_bTS) ) ) &
BHB/4G31 -/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg $($#""&&(!
4[P_bTS) ) )
)N THIS VERY SIMPLE EXAMPLE THERE IS A QUERY WITH A PREDICATE COMPARING A DATE COLUMN COL TO A
STRING CONTAINING A DATE )N ONE CASE THE STRING CONTAINED A FOUR
DIGIT YEAR )N THE OTHER ONLY TWO DIGITS
WERE USED /NLY THE QUERY WITH THE FOUR
DIGIT YEAR FORMAT USED THE 3TORAGE )NDEX ,ETS LOOK AT THE PLANS
FOR THE STATEMENTS TO SEE WHY THE TWO QUERIES WERE TREATED DIFFERENTLY
BHB/4G31 -/Ubg!
4]cTaeP[dTU^ab`[NcTgc)bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[",
4]cTaeP[dTU^ab`[NXS)
4]cTaeP[dTU^aX]bcNXS)
B@;N830E6N4C8<4?G>55;>038>NB0E43B@;NC4GC
!b$']%S"\iZ\] & HTb bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT
R^[",! >2C ! $
UdW\V(W`SQS'# # HTb(( ((bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT
R^[",! >2C $
!a^fbbT[TRcTS
BHB/4G31 -bT[TRcUa^\cPQ[TSQ\bNg_[P] SXb_[PhNRdab^ab`[NXSRWX[SN]^ch_XRP[*
4]cTaeP[dTU^ab`[NXS)UdW\V(W`SQS'#
4]cTaeP[dTU^aRWX[SN]^)
?;0=NC01;4N>DC?DC
B@;N83UdW\V(W`SQS'#RWX[S]d\QTa
bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[",! >2C $
?[P]WPbWeP[dT)!%'#!#('"$
#(!04%2 34/2!'% ).$%8%3
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckkkk$"$: kk
k kB>AC066A460C4kk k'kkk
k!kC01;40224BBBC>A0645D;;kB:4F"k"'#k" &!k$"$:!k )#&) #k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! bc^aPVT2>;",! >2C $
UX[cTa2>;",! >2C $
! a^fbbT[TRcTS
BHB/4G31 -
4]cTaeP[dTU^ab`[NXS)!b$']%S"\iZ\]
4]cTaeP[dTU^aRWX[SN]^)
?;0=NC01;4N>DC?DC
B@;N83!b$']%S"\iZ\]RWX[S]d\QTa
bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[",! >2C ! $
?[P]WPbWeP[dT)!%'#!#('"$
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckkkk$" : kk
k kB>AC066A460C4kk k'kkk
k!kC01;40224BBBC>A0645D;;kB:4F"k"'#k" &!k$" : k )#%)!#k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! bc^aPVT2>;",C>N30C4! $ ! ) ) bhhhh \\ SS
WW!#)\X)bb
UX[cTa2>;",C>N30C4! $ ! ) ) bhhhh \\ SS
WW!#)\X)bb
!!a^fbbT[TRcTS
)T APPEARS THAT THE OPTIMIZER DIDNT RECOGNIZE THE TWO
DIGIT DATE AS A DATE !T THE VERY LEAST THE
OPTIMIZER FAILED TO APPLY THE c^NSPcT FUNCTION TO THE LITERAL AND SO THE 3TORAGE )NDEX WAS NOT USED
&ORTUNATELY MOST OF THESE TYPES OF DATA CONVERSION ISSUES HAVE BEEN RESOLVED WITH THE LATER RELEASES
(ERES THE SAME TEST USING CELLSRV
#(!04%2 34/2!'% ).$%8%3
BHB/B0=31>G-/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg
BHB/B0=31>G-bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[",! >2C $*
2>D=C
BHB/B0=31>G-/bX
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg % !#$!%'#'
3O AS YOU CAN SEE THIS CONVERSION ISSUE HAS BEEN RESOLVED 3O WHY BRING IT UP 7ELL THE POINT IS THAT
THE BEHAVIOR OF 3TORAGE )NDEXES HAVE UNDERGONE NUMEROUS CHANGES AS THE PRODUCT HAS MATURED !S A
RESULT WE HAVE BUILT A SET OF TEST CASES THAT WE USE TO VERIFY BEHAVIOR AFTER EACH PATCH IN OUR LAB /UR TEST
CASES PRIMARILY VERIFY COMPARISON OPERATORS LIKE )3 .5,, ETCx AND A FEW OTHER SPECIAL CASES SUCH
AS ,/"S COMPRESSION AND ENCRYPTION /F COURSE ITS ALWAYS A GOOD PRACTICE TO TEST APPLICATION BEHAVIOR
AFTER ANY PATCHING BUT IF YOU HAVE SPECIFIC CASES WHERE 3TORAGE )NDEXES ARE CRITICAL TO YOUR APPLICATION
YOU MAY WANT TAKE SPECIAL CARE TO TEST THOSE PARTS OF YOU APPLICATION
0ARTITION 3IZE
3TORAGE )NDEXES DEPEND ON 3MART 3CANS WHICH DEPEND ON DIRECT PATH READS !S WE DISCUSSED IN #HAPTER
/RACLE WILL GENERALLY USE SERIAL DIRECT PATH READS FOR LARGE OBJECTS (OWEVER WHEN AN OBJECT IS
PARTITIONED /RACLE MAY FAIL TO RECOGNIZE THAT THE OBJECT IS hLARGE v BECAUSE /RACLE LOOKS AT THE SIZE OF
EACH INDIVIDUAL SEGMENT 4HIS MAY RESULT IN SOME PARTITIONS NOT BEING READ VIA THE 3MART 3CAN
MECHANISM AND THUS DISABLING ANY 3TORAGE )NDEXES FOR THAT PARTITION 7HEN HISTORICAL PARTITIONS ARE
COMPRESSED THE PROBLEM BECOMES EVEN MORE NOTICEABLE AS THE REDUCED SIZE OF THE COMPRESSED
PARTITIONS WILL BE EVEN LESS LIKELY TO TRIGGER THE SERIAL DIRECT PATH READS 4HIS ISSUE CAN BE WORKED AROUND
BY NOT RELYING ON THE SERIAL DIRECT PATH READ ALGORITHM AND INSTEAD SPECIFYING A DEGREE OF PARALLELISM FOR
THE OBJECT OR USING A HINT TO FORCE THE DESIRED BEHAVIOR
)NCOMPATIBLE #ODING 4ECHNIQUES
&INALLY THERE ARE SOME CODING TECHNIQUES THAT CAN DISABLE 3TORAGE )NDEXES (ERES AN EXAMPLE SHOWING
THE EFFECT OF THE cad]R FUNCTION ON DATE COLUMNS
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTcad]RR^[",! >2C ! $*
2>D=C
#
#(!04%2 34/2!'% ).$%8%3
a^fbT[TRcTS
4[P_bTS) ) )$& $
BHB/4G31 -/Ubg!
4]cTaeP[dTU^ab`[NcTgc)bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTcad]RR^["
4]cTaeP[dTU^ab`[NXS)
B@;N830E6N4C8<4?G>55;>038>NB0E43B@;NC4GC
"R f(%RPhWdc($% (# HTb(( ((bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT
cad]RR^[",! >2C ! $
"R f(%RPhWdc($& $ HTb(( ((bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT
cad]RR^[",! >2C ! $
!a^fbbT[TRcTS
4[P_bTS) ) )
BHB/4G31 -bT[TRcUa^\cPQ[TSQ\bNg_[P] SXb_[PhNRdab^ab`[NXSRWX[SN]^ch_XRP[*
4]cTaeP[dTU^ab`[NXS)"R f(%RPhWdc(
4]cTaeP[dTU^aRWX[SN]^)
?;0=NC01;4N>DC?DC
B@;N83"R f(%RPhWdc(RWX[S]d\QTa
bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTcad]RR^[",! >2C ! $
?[P]WPbWeP[dT)!%'#!#('"$
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckkkk$# : kk
k kB>AC066A460C4kk k'kkk
k!kC01;40224BBBC>A0645D;;kB:4F"k#k"!k$# :"k )#')!#k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! bc^aPVTCAD=28=C4A=0;N5D=2C8>=2>;",C>N30C4! $ !
) ) bhhhh \\ SSWW!#)\X)bb
UX[cTaCAD=28=C4A=0;N5D=2C8>=2>;",C>N30C4! $ !
) ) bhhhh \\ SSWW!#)\X)bb
=^cT
RPaSX]P[XchUTTSQPRZdbTSU^acWXbbcPcT\T]c
#(!04%2 34/2!'% ).$%8%3
)N THIS EXAMPLE A FUNCTION WAS APPLIED TO A DATE COLUMN WHICH AS YOU MIGHT EXPECT DISABLES THE
3TORAGE )NDEX 4HE FACT THAT APPLYING A FUNCTION TO A COLUMN DISABLES THE 3TORAGE )NDEX IS NOT TOO
SURPRISING BUT APPLICATION OF THE cad]R FUNCTION IS A COMMONLY SEEN CODING TECHNIQUE -ANY DATES HAVE
A TIME COMPONENT AND MANY QUERIES WANT DATA FOR A SPECIFIC DAY )T IS WELL KNOWN THAT TRUNCATING A DATE
IN THIS MANNER WILL DISABLE NORMAL " 4REE INDEX USAGE )N THE PAST THAT GENERALLY DIDNT MATTER 1UERIES
IN MANY DATA WAREHOUSE ENVIRONMENTS WERE DESIGNED TO DO FULL SCANS ANYWAY SO THERE WAS REALLY NO
NEED TO WORRY ABOUT DISABLING AN INDEX 3TORAGE )NDEXES CHANGE THE GAME FROM THIS PERSPECTIVE AND MAY
FORCE US TO RE THINK SOME OF OUR APPROACHES 7ELL DISCUSS THIS ISSUE IN MORE DETAIL IN #HAPTER
Summary
3TORAGE )NDEXES ARE AN OPTIMIZATION TECHNIQUE THAT IS AVAILABLE WHEN THE DATABASE IS ABLE TO UTILIZE
3MART 4ABLE 3CANS 4HEY CAN PROVIDE DRAMATIC PERFORMANCE IMPROVEMENTS 3TORAGE INDEXES CAN BE
THOUGHT OF AS AN ALTERNATE PARTITIONING STRATEGY BUT WITHOUT THE NORMAL RESTRICTIONS ASSOCIATED WITH
PARTITIONING 4HEY ARE ESPECIALLY EFFECTIVE WITH QUERIES THAT ACCESS DATA VIA AN ALTERNATE KEY THAT TRACKS
THE PRIMARY PARTITION KEY
(OW THE DATA IS PHYSICALLY STORED IS AN IMPORTANT CONSIDERATION AND HAS A DRAMATIC IMPACT ON THE
EFFECTIVENESS OF 3TORAGE )NDEXES #ARE SHOULD BE TAKEN WHEN MIGRATING DATA TO THE %XADATA PLATFORM TO
ENSURE THAT THE DATA IS CLUSTERED ON DISK IN A MANNER THAT WILL ALLOW 3TORAGE )NDEXES TO BE USED
EFFECTIVELY
CHAPTER 5
Exadata Smart Flash Cache
4HE MARKETING GUYS AT /RACLE MUST LIKE THE TERM hSMARTv 4HEY HAVE APPLIED IT TO A DOZEN OR SO DIFFERENT
FEATURES ON THE %XADATA PLATFORM 4HEY ALSO SEEM TO LIKE THE TERM hFLASH v WHICH IS ASSOCIATED WITH AT
LEAST A HALF DOZEN FEATURES AS WELL 4O ADD TO THE CONFUSION THERE ARE TWO FEATURES IN /RACLE $ATABASE G
2ELEASE THAT HAVE ALMOST EXACTLY THE SAME NAMES $ATABASE 3MART &LASH #ACHE $"&# AND %XADATA
3MART &LASH #ACHE %3&# 7HILE BOTH FEATURES MAKE USE OF FLASH
BASED MEMORY DEVICES THEY ARE VERY
DIFFERENT 4HIS CHAPTER IS FOCUSED ON %3&# SO WELL ONLY MENTION $"&# IN PASSING
#LEARY ONE OF THE GOALS WITH %XADATA 6 AND NOW 8 WAS TO EXPAND %XADATA CAPABILITIES TO
IMPROVE ITS PERFORMANCE WITH /,40 WORKLOADS %3&# WAS THE KEY COMPONENT THAT WAS ADDED TO THE 6
CONFIGURATION TO ACCOMPLISH THIS GOAL 4HE ADDITION PROVIDES OVER 4" OF CACHE IN A FULL RACK )TS
IMPORTANT TO UNDERSTAND THAT THIS CACHE IS MANAGED BY /RACLE SOFTWARE THAT IS AWARE OF HOW THE DATA IS
BEING USED BY THE DATABASES THAT THE STORAGE IS SUPPORTING /RACLE HAS BEEN WORKING ON SOFTWARE FOR
EFFECTIVELY MANAGING DATABASE CACHES FOR OVER YEARS 3INCE THE STORAGE SOFTWARE KNOWS WHAT THE
DATABASE IS ASKING FOR IT HAS A MUCH BETTER IDEA OF WHAT SHOULD AND SHOULDNT BE CACHED THAN A
CONVENTIONAL STORAGE ARRAY
DBFC VS. ESFC
$"&# AND %3&# ARE TWO COMPLETELY DIFFERENT THINGS $"&# IS AN EXTENSION OF THE BUFFER CACHE ON THE
DATABASE SERVER )T IS A STANDARD PART OF G AND IS IMPLEMENTED AS A TIER BUFFER CACHE FOR AN INSTANCE )T
IS ONLY SUPPORTED ON 3OLARIS AND /RACLE %NTERPRISE ,INUX )T IS ENABLED BY ADDING A FLASH CARD TO A DATABASE
SERVER AND TELLING A SINGLE INSTANCE TO USE IT )F A USER NEEDS A BLOCK THAT IS NOT IN THE BUFFER CACHE IT WILL
LOOK IN THE $"&# TO SEE IF IT IS THERE BEFORE REQUESTING AN )/ 7HEN BLOCKS ARE AGED OUT OF THE BUFFER POOL
THEY ARE MOVED TO THE $"&# INSTEAD OF BEING SIMPLY FLUSHED %3&# IS OF COURSE THE DISK CACHE ON THE
%XADATA STORAGE SERVERS )T CACHES DATA FOR ALL INSTANCES THAT ACCESS THE STORAGE CELL
Hardware
%ACH %XADATA STORAGE SERVER HAS 3UN &LASH !CCELERATOR & 0#)E CARDS %ACH CARD HOLDS ' FOR A TOTAL
OF ' ON EACH STORAGE SERVER 4HESE ARE THE NUMBERS THAT YOU NORMALLY SEE QUOTED IN THE
SPECIFICATIONS ALTHOUGH THE AMOUNT THATS AVAILABLE FOR USE IS SLIGHTLY LESS 4HE CARDS ARE ACTUALLY MADE
UP OF SOLID
STATE FLASH MODULES &-ODS SOMETIMES CALLED DISK ON MODULES $/-S %ACH OF THE
MODULES CONTAIN EIGHT '" 3,# .!.$ COMPONENTS 3O EACH OF THE &-ODS HAS '" OF STORAGE OF
WHICH '" IS ADDRESSABLE 7ITH &-ODS ON EACH OF THE CARDS YOU HAVE OF THESE MODULES IN EACH
STORAGE SERVER 4HE MODULES ARE PRESENTED TO THE /3 SEPARATELY %ACH MODULE RESERVES AN ADDITIONAL
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
' OF ITS ADDRESSABLE SPACE FOR ITS OWN PURPOSES YIELDING A USABLE SPACE OF ' PER MODULE 4HIS
YIELDS A TOTAL OF ' OF USABLE FLASH STORAGE PER STORAGE CELL 4HATS OVER A TERABYTE 4" ON THE SMALLEST
QUARTER RACK CONFIGURATION AND OVER 4" ON A FULL RACK
4HE & CARDS ALSO INCLUDE AN ENERGY STORAGE MODULE %3- SOMETIMES ALSO REFERRED TO AS A 3UPER
#APACITOR 9OU CAN THINK OF IT AS A BATTERY ALTHOUGH IT ACTUALLY USES A CAPACITOR WHEREAS A TRADITIONAL
BATTERY RELIES ON CHEMICAL REACTIONS !T ANY RATE THE PURPOSE OF THE %3- IS TO PROVIDE ENOUGH POWER TO
FLUSH ANY DATA IN ITS VOLATILE MEMORY TO THE NONVOLATILE FLASH MEMORY IN THE CASE OF A SUDDEN POWER LOSS
4HE VOLATILE MEMORY CONSISTS OF - OF $2!- PER &-OD AND IS USED TO BUFFER WRITES TO THE NONVOLATILE
FLASH MEMORY 9OU SHOULD NOTE THAT IF THE %3- FAILS WRITES WILL BY
PASS THE $2!- AND BE SENT DIRECTLY TO
THE FLASH MEMORY !S A RESULT WRITE SPEEDS WILL BE SIGNIFICANTLY DECREASED IN THE CASE OF %3- FAILURE 4HIS
DOESNT PRESENT A MAJOR ISSUE WHEN THE CARDS ARE CONFIGURED AS %3&# SINCE THE /RACLE STORAGE SOFTWARE
TREATS THE %3&# AS A WRITE
THROUGH CACHE AND THEREFORE WRITES DIRECTLY TO DISK BYPASSING THE CACHE )T CAN
BE AN ISSUE THOUGH IF THE CARDS ARE USED AS &LASH $ISKS 3UN RECOMMENDS THAT THE %3- MODULES BE
REPLACED EVERY TWO YEARS AS THE STORED ENERGY DEGRADES OVER TIME LIKE A BATTERY 4HE ),/- SYSTEM
MANAGEMENT FIRMWARE MONITORS %3- USAGE AND ISSUES WARNINGS AS THE %3- APPROACHES THE END OF ITS
LIFESPAN 3EE THE 3UN &LASH !CCELERATOR & 0#)E #ARD 5SERS 'UIDE FOR FURTHER DETAILS
!CCORDING TO THE 3UN DOCUMENTATION THE STORAGE SERVERS ALLOW 0#)E CARDS TO BE REPLACED WHILE THE
SYSTEM IS RUNNING (OWEVER THE /RACLE %XADATA 3TORAGE 3OFTWARE 5SERS 'UIDE RECOMMENDS POWERING
DOWN THE STORAGE SERVERS BEFORE REPLACING ONE OF THESE CARDS &ORTUNATELY YOU CAN ACCOMPLISH THIS
WITHOUT EXPERIENCING AN OUTAGE AS !3- REDUNDANCY ALLOWS ENTIRE STORAGE CELLS TO BE OFFLINE WITHOUT
AFFECTING THE DATABASES THEY ARE SUPPORTING .OTE THAT REPLACING ONE OF THE & CARDS SHOULD NOT REQUIRE
ANY RECONFIGURATION &IGURE
SHOWS A CONCEPTUAL DIAGRAM OF ONE OF THE &LASH !CCELERATOR CARDS
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
Figure 5-1. Conceptual diagram of the Sun F20 PCIe Flash Card
Cache vs. Flash Disk
4HE FLASH MEMORY ON THE STORAGE SERVERS CAN BE USED IN TWO WAYS )T CAN BE CONFIGURED AS A LARGE DISK
CACHE %3&# OR IT CAN BE CARVED UP AND PRESENTED AS SOLID STATE DISKS WHICH CAN BE USED BY !3- FOR
DATABASE STORAGE 4HESE TWO MODES ARE NOT MUTUALLY EXCLUSIVE 4HE FLASH MEMORY CAN BE ALLOCATED TO
EITHER FORMAT IN WHATEVER PERCENTAGES ARE DESIRED 4HE RECOMMENDED CONFIGURATION IS TO USE ALL THE FLASH
MEMORY AS CACHE 4HIS CONFIGURATION SIGNIFICANTLY SPEEDS UP RANDOM ACCESS READS (OWEVER IT PROVIDES
NO DIRECT BENEFIT TO WRITE SPEEDS
)N MANY SYSTEMS THE APPROACH OF ALLOCATING ALL FLASH MEMORY AS CACHE WORKS VERY WELL (OWEVER FOR
SYSTEMS THAT ARE VERY WRITE INTENSIVE IT MAY BE BENEFICIAL TO USE SOME OF THE FLASH MEMORY AS A DISK
+EEP IN MIND THAT DEPENDING ON THE !3- REDUNDANCY LEVEL USED .ORMAL OR (IGH CHOOSING THIS OPTION
WILL CONSUME OR TIMES THE AMOUNT OF FLASH STORAGE THAT THE OBJECTS ACTUALLY OCCUPY 4HIS FACT ALONE
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
MAKES THE OPTION LESS PALATABLE !LSO KEEP IN MIND THAT WRITES TO DATA FILES ARE DONE IN THE BACKGROUND BY
THE $"72 PROCESSES 3O CHOOSING TO USE PART OF THE FLASH BASED STORAGE AS A DISK MAY NOT PROVIDE AS
MUCH BENEFIT AS YOU MIGHT HOPE &ORTUNATELY ITS PRETTY EASY TO REALLOCATE FLASH STORAGE SO TESTING YOUR
SPECIFIC SITUATION SHOULD NOT PROVE TOO DIFFICULT (OWEVER SINCE THIS CHAPTER FOCUSES ON THE %XADATA
3MART &LASH #ACHE FEATURE WE WILL ONLY BRIEFLY COVER USING THE &S AS FLASH DISKS
■ .OTE ! COMMON MISCONCEPTION IS THAT PUTTING ONLINE REDO LOGS ON FLASH STORAGE WILL SIGNIFICANTLY SPEED UP
WRITES TO REDO LOGS AND THUS INCREASE THE THROUGHPUT OF HIGH
TRANSACTION SYSTEMS 7HILE ITS TRUE THAT SMALL RANDOM
WRITES ARE FASTER ON 33$
BASED STORAGE THAN ON TRADITIONAL DISKS WRITES TO REDO LOGS ON HIGH
TRANSACTION SYSTEMS
GENERALLY DO NOT FALL INTO THAT BUCKET AND ACTUALLY DO NOT BENEFIT THAT MUCH FROM BEING STORED ON 33$ STORAGE )N
ADDITION 33$ WRITE MECHANICS CAUSE A LOT OF VARIABILITY IN INDIVIDUAL WRITE TIMES 4HERE MAY BE INDIVIDUAL WRITES
THAT TAKE ORDERS OF MAGNITUDE LONGER THAN THE AVERAGE 4HIS CAN CAUSE PROBLEMS ON VERY BUSY SYSTEMS AS WELL 3O
BEFORE YOU PUT YOUR ONLINE REDO LOGS ON TO YOUR VALUABLE %XADATA &LASH STORAGE YOU SHOULD TEST TO MAKE SURE THAT
THE BENEFITS OUTWEIGH THE COSTS
5SING &LASH -EMORY AS #ACHE
! VERY SIMPLIFIED DESCRIPTION OF HOW A DISK CACHE WORKS GOES SOMETHING LIKE THIS 7HEN A READ REQUEST
COMES IN THE )/ SUBSYSTEM CHECKS THE CACHE TO SEE IF THE REQUESTED DATA EXISTS IN CACHE )F THE DATA IS IN
THE CACHE IT IS RETURNED TO THE REQUESTING PROCESS )F THE REQUESTED DATA DOESNT RESIDE IN THE CACHE IT IS
READ FROM DISK AND RETURNED TO THE REQUESTING PROCESS )N THE CASE OF UNCACHED DATA THE DATA IS LATER
COPIED TO THE CACHE AFTER IT IS RETURNED TO THE REQUESTING PROCESS 4HIS IS DONE TO ENSURE THAT THE CACHE
DOESNT SLOW DOWN THE )/ FOR THE REQUESTING PROCESS
7ITH %XADATA DISK CACHING IS NOT QUITE AS SIMPLE AS THE GENERAL CASE WE JUST DESCRIBED )N AN EFFORT
TO MAXIMIZE THE POTENTIAL THROUGHPUT THE RT[[bae PROGRAM MAY ACTUALLY FIRE OFF ASYNC )/ REQUESTS TO
BOTH THE DISK AND THE FLASH CACHE 'ENERALLY SPEAKING IF THE REQUESTED DATA IS IN THE CACHE THE REQUESTS
WILL BE FULFILLED BY THE FLASH CACHE BEFORE THE DISK READS WILL BE ABLE TO COMPLETE (OWEVER WHEN THE
SYSTEM IS HEAVILY LOADED IT IS POSSIBLE FOR SOME REQUESTS TO BE FULFILLED BY THE FLASH CACHE WHILE OTHERS ARE
FULFILLED BY THE DISKS 4HIS TWO
PRONGED ATTACK EFFECTIVELY INCREASES THE AMOUNT OF THROUGHPUT THAT THE
SYSTEM CAN DELIVER
■ +EVIN 3AYS h4HE HARDWARE UPON WHICH %XADATA 3TORAGE 3ERVER IS BASED OFFERS FIVE HIGH
BANDWIDTH 0#) SLOTS
FOR STORAGE ATTACHMENT 7ITHOUT ATTACKING BOTH &LASH AND (ARD $ISK ASSETS IN PARALLEL EACH CELL WOULD BE LIMITED TO
ROUGHLY '"S SCAN THROUGHPUT 7ITH RT[[bae SCANNING BOTH &LASH AND (ARD $ISK CONCURRENTLY THE SCAN
THROUGHPUT INCREASES TO ROUGHLY '"S
&IGURE SHOWS THE )/ PATH OF READS USING /RACLES %3&#
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
Flash Cache Reads - I / O Path Architecture
Read Operation Read Operation
on cached data on uncached data
DB DB
1 1
Read Request
Read Request
3 3
cellsrv cellsrv
2 2 2 4
DB DB
Cache Cache
Step 2 is a Race Condition.
The fastest read wins.
Figure 5-2. Conceptual model of read operation I/O path
3INCE %3&# IS A WRITE THROUGH CACHE WRITES BYPASS THE CACHE AND GO DIRECTLY TO DISK (OWEVER AFTER
SENDING AN ACKNOWLEDGEMENT BACK TO THE DATABASE SERVER /RACLES STORAGE SOFTWARE THEN COPIES THE DATA
INTO THE CACHE ASSUMING IT IS SUITABLE FOR CACHING 4HIS IS A KEY POINT 4HE METADATA THAT IS SENT WITH THE
WRITE REQUEST LETS THE STORAGE SOFTWARE KNOW IF THE DATA IS LIKELY TO BE USED AGAIN AND IF SO THE DATA IS ALSO
WRITTEN TO THE CACHE 4HIS STEP IS DONE AFTER SENDING AN ACKNOWLEDGEMENT TO THE DATABASE TIER IN ORDER TO
ENSURE THAT THE WRITE OPERATION CAN COMPLETE AS QUICKLY AS POSSIBLE
&IGURE SHOWS THE )/ PATH OF WRITES USING /RACLES %3&#
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
Flash Cache Writes - I / O Path Architecture
Write Operation
3 DB
Acknowledgement
cellsrv
2 4
DB
Cache
Figure 5-3. Conceptual model of write operation I/O path
!S YOU MIGHT GUESS 3MART 3CANS GENERALLY IGNORE THE %3&# AND READ DIRECTLY FROM DISK (OWEVER IF
THE OBJECT BEING SCANNED HAS BEEN DESIGNATED FOR PREFERENTIAL CACHING BY SETTING THE STORAGE CLAUSE
ATTRIBUTE 24;;N5;0B7N20274 TO :44? THEN EVEN 3MART 3CANS WILL ATTEMPT TO READ FROM FLASH CACHE 4HE
FOLLOWING EXAMPLE DEMONSTRATES THIS BEHAVIOR
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb"$&
4[P_bTS) ) )
BHB/4G31 -P[cTacPQ[TZb^ bZTf"bc^aPVTRT[[NU[PbWNRPRWTZTT_*
CPQ[TP[cTaTS
4[P_bTS) ) )
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf"fWTaTR^[ - *
0E6?:N2>;
% $"" ' $
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
4[P_bTS) ) )"# "$
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb"$&
4[P_bTS) ) )
BHB/4G31 - ]^cWX]VX]cWTU[PbWRPRWTcWTUXabccX\TcahPVPX]
BHB/4G31 -
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf"fWTaTR^[ - *
0E6?:N2>;
% $"" ' $
4[P_bTS) ) )!& '!]^cXRTcWPccWTT[P_bTScX\TXbb^\TfWPcUPbcTa
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb!##
4[P_bTS) ) )
BHB/4G31 -P[cTacPQ[TZb^ bZTf"bc^aPVTRT[[NU[PbWNRPRWTSTUPd[c*
CPQ[TP[cTaTS
4[P_bTS) ) ) $
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf"fWTaTR^[ - *
0E6?:N2>;
% $"" ' $
4[P_bTS) ) )" $#
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb!##
4[P_bTS) ) )
BHB/4G31 -
BHB/4G31 -P[cTacPQ[TZb^ bZTf"bc^aPVTRT[[NU[PbWNRPRWTZTT_*
CPQ[TP[cTaTS
4[P_bTS) ) )
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf"fWTaTR^[ - *
0E6?:N2>;
% $"" ' $
4[P_bTS) ) )!& (
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb#$!!$
4[P_bTS) ) )
)N THE PRECEDING EXAMPLE YOU CAN SEE THAT THE 3MART 3CAN IS FASTER WHEN THE 24;;N5;0B7N20274
STORAGE CLAUSE ATTRIBUTE IS SET TO :44? AND MORE IMPORTANTLY THAT A LARGE NUMBER OF hCELL FLASH CACHE READ
HITSv ARE GENERATED IN THAT CASE 7HEN THE 24;;N5;0B7N20274 ATTRIBUTE IS SET TO 3450D;C THE QUERY IS
SLOWER AND NO hCELL FLASH CACHE READ HITSv ARE GENERATED
(OW %3&# IS #REATED
3TORAGE CONFIGURATION IS COVERED IN DETAIL IN #HAPTER BUT THE BASIC COMMANDS FOR CREATING AND
MANAGING FLASH CACHE WILL BE COVERED BRIEFLY IN THIS SECTION AS WELL 4HE CACHE IS CREATED AND MANAGED
USING THE RT[[R[X UTILITY ON EACH STORAGE SERVER THE RT[[R[X COMMAND SYNTAX IS COVERED IN DETAIL IN
!PPENDIX ! !LTERNATIVELY THE SR[X UTILITY WHICH REPLICATES COMMANDS ACROSS MULTIPLE STORAGE CELLS CAN
BE USED 4HE COMMAND TO CREATE THE CACHE IS 2A40C45;0B720274 (ERES AN EXAMPLE
2T[[2;8-RaTPcTU[PbWRPRWTP[[bXiT," V
5[PbWRPRWTRT[[ "N5;0B720274bdRRTbbUd[[hRaTPcTS
4HIS FORM OF THE COMMAND TELLS THE STORAGE SOFTWARE TO SPREAD THE CACHE ACROSS ALL THE &-ODS ON ALL
THE FLASH CARDS )N THIS CASE THE SIZE PARAMETER WAS USED TO SPECIFY THAT THE ENTIRE CACHE SHOULD OCCUPY
ONLY ' LEAVING ROUGHLY ' AVAILABLE FOR USE AS A FLASH DISK 4O USE ALL AVAILABLE SPACE FOR FLASH CACHE
SIMPLY LEAVE OFF THE SIZE PARAMETER
2T[[2;8-RaTPcTU[PbWRPRWTP[[
5[PbWRPRWTRT[[ "N5;0B720274bdRRTbbUd[[hRaTPcTS
4HIS COMMAND USES ALL AVAILABLE STORAGE ON ALL AVAILABLE FLASH CARDS .OTE THAT THE FLASH CACHE IS
AUTOMATICALLY ASSIGNED A NAME THAT INCLUDES THE NAME OF THE STORAGE CELL 4O SEE THE SIZE OF THE
FLASHCACHE YOU CAN ISSUE THE ;8BC5;0B72027434C08; COMMAND
2T[[2;8-[XbcU[PbWRPRWTSTcPX[
]P\T)RT[[ "N5;0B720274
RT[[3XbZ)53N $NRT[[ "53N !NRT[[ "53N #NRT[[ "53N NRT[[ "53N NRT[[ "
53N "NRT[[ "53N !NRT[[ "53N 'NRT[[ "53N &NRT[[ "53N NRT[[ "53N (NRT[[ "53N $NRT[[ "
53N %NRT[[ "53N #NRT[[ "53N "NRT[[ "53N NRT[[ "
RaTPcX^]CX\T)! &C ()#$)"! %)
STVaPSTS2T[[SXbZb)
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
TUUTRcXeT2PRWTBXiT)" 6
XS)' P# RR( "& R #R(U '$QU UUQT$' (&%R
bXiT)" 6
bcPcdb)]^a\P[
.OTICE THAT THE FLASH CACHE IS SPREAD ACROSS CELL DISKS 4HERE IS ONE CELL DISK FOR EACH &-OD ON
EACH FLASH CARD 4O GET MORE INFORMATION ABOUT THE CELL DISKS THAT MAKE UP THE FLASH CACHE YOU CAN USE
THE ;8BC24;;38B: COMMAND SHOWN HERE
2T[[2;8-[XbcRT[[SXbZPccaXQdcTb]P\TSXbZCh_TbXiTfWTaT]P\T[XZT53
53N NRT[[ "5[PbW3XbZ!! '&$6
53N NRT[[ "5[PbW3XbZ!! '&$6
53N !NRT[[ "5[PbW3XbZ!! '&$6
53N "NRT[[ "5[PbW3XbZ!! '&$6
53N #NRT[[ "5[PbW3XbZ!! '&$6
53N $NRT[[ "5[PbW3XbZ!! '&$6
53N %NRT[[ "5[PbW3XbZ!! '&$6
53N &NRT[[ "5[PbW3XbZ!! '&$6
53N 'NRT[[ "5[PbW3XbZ!! '&$6
53N (NRT[[ "5[PbW3XbZ!! '&$6
53N NRT[[ "5[PbW3XbZ!! '&$6
53N NRT[[ "5[PbW3XbZ!! '&$6
53N !NRT[[ "5[PbW3XbZ!! '&$6
53N "NRT[[ "5[PbW3XbZ!! '&$6
53N #NRT[[ "5[PbW3XbZ!! '&$6
53N $NRT[[ "5[PbW3XbZ!! '&$6
3INCE THE FLASH CACHE SITS ON TOP OF CELL DISKS THE CELL DISKS MUST BE CREATED BEFORE THE FLASH CACHE
4HIS CAN BE DONE WITH THE 2A40C424;;38B: COMMAND
2T[[2;8-RaTPcTRT[[SXbZP[[U[PbWSXbZ
2T[[3XbZ53N NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N !NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N "NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N #NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N $NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N %NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N &NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N 'NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N (NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N !NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N "NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N #NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[3XbZ53N $NRT[[ "bdRRTbbUd[[hRaTPcTS
9OU CAN ALSO CREATE A FLASH CACHE ON A LIMITED SET OF &-ODS BY SPECIFYING A SPECIFIC LIST OF CELL DISKS
)N MOST CASES THIS IS NOT DONE BUT IT IS POSSIBLE (ERES AN EXAMPLE
2T[[2;8-RaTPcTU[PbWRPRWTRT[[SXbZ,53N NRT[[ "53N NRT[[ "bXiT,# 6
5[PbWRPRWTRT[[ "N5;0B720274bdRRTbbUd[[hRaTPcTS
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
2T[[2;8-[XbcU[PbWRPRWTSTcPX[
]P\T)RT[[ "N5;0B720274
RT[[3XbZ)53N NRT[[ "53N NRT[[ "
RaTPcX^]CX\T)! (C $)!()!' %)
STVaPSTS2T[[SXbZb)
TUUTRcXeT2PRWTBXiT)# 6
XS)PS$%PP(S ST# #& " '$U! (& "#$!"TQQ
bXiT)# 6
bcPcdb)]^a\P[
/NCE AGAIN USING RT[[R[X TO MANAGE STORAGE INCLUDING FLASH CACHE IS COVERED IN MORE DETAIL IN
#HAPTER BUT THIS SECTION SHOULD GIVE YOU A BASIC UNDERSTANDING OF HOW THE FLASH CACHE IS CREATED
#ONTROLLING %3&# 5SAGE
'ENERALLY SPEAKING OBJECTS WILL BE CACHED IN THE %3&# BASED ON THE STORAGE SOFTWARES AUTOMATIC
CACHING POLICY (OWEVER YOU CAN OVERRIDE THE AUTOMATIC POLICY FOR INDIVIDUAL DATABASE OBJECTS BY USING
THE 24;;N5;0B7N20274 STORAGE CLAUSE ATTRIBUTE 4HE ATTRIBUTE HAS THREE VALID VALUES
=>=4 .EVER CACHE THIS OBJECT
3450D;C 4HE !UTOMATIC CACHING MECHANISM IS IN EFFECT 4HIS IS THE DEFAULT
VALUE
:44? 4HE OBJECT SHOULD BE GIVEN PREFERENTIAL STATUS .OTE THAT THIS DESIGNATION
ALSO CHANGES THE DEFAULT BEHAVIOR OF 3MART 3CANS ALLOWING THEM TO READ FROM
BOTH THE CACHE AND DISK
9OU CAN SPECIFY THE STORAGE CLAUSE WHEN AN OBJECT IS CREATED 3OME OPTIONS OF THE STORAGE CLAUSE CAN
BE MODIFIED USING THE 0;C4A COMMAND AS WELL (ERES AN EXAMPLE OF CHANGING THE 24;;N5;0B7N20274
STORAGE CLAUSE USING THE 0;C4A COMMAND
P[cTacPQ[TZb^ bZTfbc^aPVTRT[[NU[PbWNRPRWTZTT_*
9OU CAN ALSO SEE WHETHER OBJECTS HAVE BEEN DESIGNATED FOR MORE AGGRESSIVE CACHING BY LOOKING AT
THE RT[[NU[PbWNRPRWT COLUMN OF SQPNcPQ[Tb OR SQPNX]STgTb
BHB/4G31 -/TbURNZTT_NcPQ[Tb
BHB/4G31 -bT[TRc^f]TacPQ[TN]P\TbcPcdb[PbcNP]P[hiTS
!]d\Na^fbQ[^RZbSTVaTTRT[[NU[PbWNRPRWT
"Ua^\SQPNcPQ[Tb
#fWTaTRT[[NU[PbWNRPRWT[XZT]e[RT[[NU[PbWNRPRWT:44?
$
4]cTaeP[dTU^aRT[[NU[PbWNRPRWT)
>F=4AC01;4N=0<4BC0CDB;0BCN0=0;=D<NA>FB1;>2:B346A4424;;N5;0B7N20274
:B>B:4FE0;83 ' B4? "! # %!!(# :44?
:B>B:4F"E0;83 # >2C "'# #' (#(&!& :44?
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
Monitoring
9OUR OPTIONS FOR MONITORING %3&# ARE LIMITED ON THE DATABASE SERVERS 4HERE IS ONLY ONE STATISTIC RELATED
TO %3&# IN ebhbbcPc AND RELATED VIEWS RT[[U[PbWRPRWTaTPSWXcb 4HE STORAGE SERVERS HOWEVER
PROVIDE A MUCH LARGER ARRAY OF DIAGNOSTIC INFORMATION VIA THE RT[[R[X UTILITY 5NFORTUNATELY SINCE THESE
METRICS ARE PROVIDED BY RT[[R[X ON INDIVIDUAL STORAGE CELLS THEY ARE LIMITED TO THAT SINGLE STORAGE SERVER
4HERE IS NO COMPREHENSIVE SET OF DIAGNOSTIC INFORMATION THAT COVERS THE ENTIRE STORAGE GRID AT THIS POINT
!T THE 3TORAGE ,AYER
4HERE ARE A NUMBER OF STATISTICS THAT ARE CAPTURED FOR %3&# USAGE AT THE STORAGE CELL LAYER 4HE RT[[R[X
UTILITY ALLOWS US TO VIEW THOSE METRICS 4HE ;8BC<4CA823458=8C8>= COMMAND PROVIDES A DESCRIPTION OF
THE VARIOUS STATISTICS THAT ARE COLLECTED &OR EXAMPLE
2T[[2;8-;8BC<4CA823458=8C8>=PccaXQdcTb]P\TSTbRaX_cX^]F74A4^QYTRcCh_T,5;0B720274
4ABLE
SHOWS THE OUTPUT OF THE COMMAND 4HE TABLE PROVIDES A BRIEF DESCRIPTION FOR EACH STATISTIC
TRACKED AT THE STORAGE LAYER THAT YOU CAN REPORT ON
Table 5-1. Flash Cache Metric Definitions
Metric Description
52N1H:44?N>E4AFA .UMBER OF MEGABYTES PUSHED OUT OF THE FLASH CACHE BECAUSE OF SPACE LIMIT FOR
gKEEPg OBJECTS
52N1H:44?N>E4AFANB42 .UMBER OF MEGABYTES PER SECOND PUSHED OUT OF THE FLASH CACHE BECAUSE OF
SPACE LIMIT FOR gKEEPg OBJECTS
52N1H:44?NDB43 .UMBER OF MEGABYTES USED FOR gKEEPg OBJECTS ON FLASH CACHE
52N1HNDB43 .UMBER OF MEGABYTES USED ON FLASH CACHE
52N8>N1H:44?NA .UMBER OF MEGABYTES READ FROM FLASH CACHE FOR gKEEPg OBJECTS
52N8>N1H:44?NANB42 .UMBER OF MEGABYTES READ PER SECOND FROM FLASH CACHE FOR gKEEPg OBJECTS
52N8>N1H:44?NF .UMBER OF MEGABYTES WRITTEN TO FLASH CACHE FOR gKEEPg OBJECTS
52N8>N1H:44?NFNB42 .UMBER OF MEGABYTES PER SECOND WRITTEN TO FLASH CACHE FOR gKEEPg OBJECTS
52N8>N1HNA .UMBER OF MEGABYTES READ FROM FLASH CACHE
52N8>N1HNAN<8BB .UMBER OF MEGABYTES READ FROM DISKS BECAUSE NOT ALL REQUESTED DATA WAS IN
FLASH CACHE
52N8>N1HNAN<8BBNB42 .UMBER OF MEGABYTES READ FROM DISKS PER SECOND BECAUSE NOT ALL REQUESTED
DATA WAS IN FLASH CACHE
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
Metric Description
52N8>N1HNANB42 .UMBER OF MEGABYTES READ PER SECOND FROM FLASH CACHE
52N8>N1HNANB:8? .UMBER OF MEGABYTES READ FROM DISKS FOR )/ REQUESTS WITH A HINT TO BYPASS
FLASH CACHE
52N8>N1HNANB:8?NB42 .UMBER OF MEGABYTES READ FROM DISKS PER SECOND FOR )/ REQUESTS WITH A HINT
TO BYPASS FLASH CACHE
52N8>N1HNF .UMBER OF MEGABYTES WRITTEN TO FLASH CACHE
52N8>N1HNFNB42 .UMBER OF MEGABYTES PER SECOND WRITTEN TO FLASH CACHE
52N8>N4AAB .UMBER OF )/ ERRORS ON FLASH CACHE
52N8>NA@:44?NA .UMBER OF READ )/ REQUESTS FOR gKEEPg OBJECTS SATISFIED FROM FLASH CACHE
52N8>NA@:44?NAN<8BB .UMBER OF READ )/ REQUESTS FOR gKEEPg OBJECTS THAT DID NOT FIND ALL DATA IN FLASH
CACHE
52N8>NA@:44?NAN<8BBNB42.UMBER OF READ )/ REQUESTS PER SECOND FOR gKEEPg OBJECTS THAT DID NOT FIND ALL
DATA IN FLASH CACHE
52N8>NA@:44?NANB42 .UMBER OF READ )/ REQUESTS FOR gKEEPg OBJECTS PER SECOND SATISFIED FROM FLASH
CACHE
52N8>NA@:44?NANB:8? .UMBER OF READ )/ REQUESTS FOR gKEEPg OBJECTS WITH A HINT TO BYPASS FLASH
CACHE
52N8>NA@:44?NANB:8?NB42.UMBER OF READ )/ REQUESTS PER SECOND FOR gKEEPg OBJECTS WITH A HINT TO
BYPASS FLASH CACHE
52N8>NA@:44?NF .UMBER OF )/ REQUESTS FOR gKEEPg OBJECTS THAT RESULTED IN FLASH CACHE BEING
POPULATED WITH DATA
52N8>NA@:44?NFNB42 .UMBER OF )/ REQUESTS PER SECOND FOR gKEEPg OBJECTS THAT RESULTED IN FLASH
CACHE BEING POPULATED WITH DATA
52N8>NA@NA .UMBER OF READ )/ REQUESTS SATISFIED FROM FLASH CACHE
52N8>NA@NAN<8BB .UMBER OF READ )/ REQUESTS THAT DID NOT FIND ALL DATA IN FLASH CACHE
52N8>NA@NAN<8BBNB42 .UMBER OF READ )/ REQUESTS PER SECOND THAT DID NOT FIND ALL DATA IN FLASH
CACHE
52N8>NA@NANB42 .UMBER OF READ )/ REQUESTS SATISFIED PER SECOND FROM FLASH CACHE
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
Metric Description
52N8>NA@NANB:8? .UMBER OF READ )/ REQUESTS WITH A HINT TO BYPASS FLASH CACHE
52N8>NA@NANB:8?NB42 .UMBER OF READ )/ REQUESTS PER SECOND WITH A HINT TO BYPASS FLASH CACHE
52N8>NA@NF .UMBER OF )/ REQUESTS THAT RESULTED IN FLASH CACHE BEING POPULATED WITH DATA
52N8>NA@NFNB42 .UMBER OF )/ REQUESTS PER SECOND THAT RESULTED IN FLASH CACHE BEING
POPULATED WITH DATA
4HE METRICS IN 4ABLE
ARE CUMULATIVE SINCE CT[[bae WAS STARTED 4HE ;8BC<4CA822DAA4=C
COMMAND SHOWS THE CURRENT VALUES OF THE METRICS FOR A SINGLE STORAGE CELL &OLLOWING ARE EXAMPLES OF A
COUPLE OF RT[[R[X COMMANDS SHOWING FIRST ALL FLASH CACHE
RELATED METRICS AND THEN ONLY THE FLASH CACHE
METRICS THAT ARE RELATED TO OBJECTS DESIGNATED FOR AGGRESSIVE CACHING THAT IS OBJECTS WHOSE STORAGE
CLAUSE SPECIFIES 24;;N5;0B7N20274,:44?
2T[[2;8-;8BC<4CA822DAA4=CF74A4^QYTRcCh_T,5;0B720274
52N1H:44?N>E4AFA5;0B720274 <1
52N1H:44?N>E4AFANB425;0B720274 <1 bTR
52N1H:44?NDB435;0B720274$ &!<1
52N1HNDB435;0B720274$#"#<1
52N8>N1H:44?NA5;0B720274' !''<1
52N8>N1H:44?NANB425;0B720274 <1 bTR
52N8>N1H:44?NF5;0B720274 "#<1
52N8>N1H:44?NFNB425;0B720274 <1 bTR
52N8>N1HNA5;0B720274"$ !( '<1
52N8>N1HNAN<8BB5;0B720274(##(#!<1
52N8>N1HNAN<8BBNB425;0B720274 <1 bTR
52N8>N1HNANB425;0B720274 <1 bTR
52N8>N1HNANB:8?5;0B720274 !!#&' <1
52N8>N1HNANB:8?NB425;0B720274 "<1 bTR
52N8>N1HNF5;0B720274"$#&#" <1
52N8>N1HNFNB425;0B720274 #<1 bTR
52N8>N4AAB5;0B720274
52N8>NA@:44?NA5;0B720274( !" 8>aT`dTbcb
52N8>NA@:44?NAN<8BB5;0B720274 $$8>aT`dTbcb
52N8>NA@:44?NAN<8BBNB425;0B720274 8> bTR
52N8>NA@:44?NANB425;0B720274 8> bTR
52N8>NA@:44?NANB:8?5;0B720274 $"!8>aT`dTbcb
52N8>NA@:44?NANB:8?NB425;0B720274 8> bTR
52N8>NA@:44?NF5;0B720274 &&(#8>aT`dTbcb
52N8>NA@:44?NFNB425;0B720274 8> bTR
52N8>NA@NA5;0B720274#! $#% ((8>aT`dTbcb
52N8>NA@NAN<8BB5;0B720274!'($(( &8>aT`dTbcb
52N8>NA@NAN<8BBNB425;0B720274 8> bTR
52N8>NA@NANB425;0B720274 8> bTR
52N8>NA@NANB:8?5;0B720274' (& !!8>aT`dTbcb
52N8>NA@NANB:8?NB425;0B720274!$ #8> bTR
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
52N8>NA@NF5;0B720274" #!'% $'8>aT`dTbcb
52N8>NA@NFNB425;0B720274!! #8> bTR
2T[[2;8-;8BC<4CA822DAA4=CF74A4^QYTRcCh_T,5;0B7202740=3]P\T;8:4 :44?
52N1H:44?N>E4AFA5;0B720274 <1
52N1H:44?N>E4AFANB425;0B720274 <1 bTR
52N1H:44?NDB435;0B720274$ &!<1
52N8>N1H:44?NA5;0B720274' !''<1
52N8>N1H:44?NANB425;0B720274 <1 bTR
52N8>N1H:44?NF5;0B720274 "#<1
52N8>N1H:44?NFNB425;0B720274 <1 bTR
52N8>NA@:44?NA5;0B720274( !" 8>aT`dTbcb
52N8>NA@:44?NAN<8BB5;0B720274 $$8>aT`dTbcb
52N8>NA@:44?NAN<8BBNB425;0B720274 8> bTR
52N8>NA@:44?NANB425;0B720274 8> bTR
52N8>NA@:44?NANB:8?5;0B720274 $"!8>aT`dTbcb
52N8>NA@:44?NANB:8?NB425;0B720274 8> bTR
52N8>NA@:44?NF5;0B720274 &&(#8>aT`dTbcb
52N8>NA@:44?NFNB425;0B720274 8> bTR
)N ADDITION TO THE PERFORMANCE METRICS YOU CAN ALSO SEE WHAT OBJECTS ARE IN THE CACHE 4HE ;8BC
5;0B7202742DAA4=C COMMAND DISPLAYS WHATS IN THE CACHE 4HIS COMMAND SHOWS A SINGLE ENTRY FOR EACH
CACHED OBJECT THE AMOUNT OF SPACE IT IS OCCUPYING AND VARIOUS OTHER STATISTICS (ERES AN EXAMPLE OF
LOOKING AT THE FLASH CACHE CONTENT
2T[[2;8-STbRaXQTU[PbWRPRWTR^]cT]c
RPRWTS:TT_BXiT
RPRWTSBXiT
SQ83
SQD]X`dT=P\T
WXc2^d]c
W^dabC^4g_XaPcX^]
\Xbb2^d]c
^QYTRc=d\QTa
cPQ[TB_PRT=d\QTa
2T[[2;8-[XbcU[PbWRPRWTR^]cT]cfWTaTSQD]X`dT=P\T[XZT4G31
-P]SWXcR^d]c-
-PccaXQdcTbSQD]X`dT=P\T^QYTRc=d\QTaRPRWTS:TT_BXiT
-RPRWTSBXiTWXc2^d]c\Xbb2^d]c
4G31! # (#" #$'&! '
4G31# !#!#'"!"&%%
4G31"'" ! #%" # &!&
4G31#& # & #!#!&#'$
4G31#&$ $'' '$%$ " !"
4G31$ " $" '# %%%(#$$
4G31%"&" !$"($! !!"(
4G31%%'#(#"'&%"$! #"'&%"$! #! ""!!
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
4G31& #(& !! % '!$$ $
4G31& &&$ %$ &' (% !%"! '%'
5NFORTUNATELY THE OBJECT NAME IS NOT INCLUDED IN THE LIST OF ATTRIBUTES 4HIS MEANS YOU MUST GO BACK
TO THE DATABASE TO DETERMINE WHICH OBJECT IS WHICH BY QUERYING SQPN^QYTRcb FOR EXAMPLE .OTE THAT THE
>QYTRc=d\QTa ATTRIBUTE IN RT[[R[X IS EQUIVALENT TO THE SPcP?^QYTRcNXS IN THE DATABASE VIEWS SUCH AS
SQPN^QYTRcb
BHB/4G31 -bT[TRc^f]Ta^QYTRcN]P\T^QYTRcNch_T
!Ua^\SQPN^QYTRcbfWTaTSPcPN^QYTRcNXS,%%'#(*
>F=4A>1942CN=0<4>1942CNCH?4
:B>B:4FC01;4
!T THE $ATABASE ,AYER
4HE DATABASE REALLY DOESNT PROVIDE MUCH VISIBILITY INTO HOW %3&# IS USED 4HERE IS ONLY ONE STATISTIC
AVAILABLE IN ebhbbcPc AND RELATED VIEWS REGARDING ITS USE
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb! ''
!S WITH ALL SUCH STATISTICS hCELL FLASH CACHE READ HITSv IS CUMULATIVE )N ebhbbcPc THE STATISTIC REFLECTS
THE SYSTEM TOTAL SINCE THE INSTANCE WAS STARTED )N e\hbcPc THE STATISTIC REFLECTS THE SESSION TOTAL SINCE
THE SESSION WAS STARTED 3O THE ONLY REAL WAY TO MAKE USE OF THE INFORMATION IS TO LOOK AT THE VALUE BEFORE
AND AFTER THE EXECUTION OF A 31, STATEMENT OF INTEREST
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb&%(
BHB/4G31 -bTcTRW^^]
BHB/4G31 -/PeVbZTf
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfPfWTaTR^[ -
!
0E6?:N2>;
% ("&$ '
BHB/4G31 -bTcTRW^^UU
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbW
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb!$$
Performance
3O HOW MUCH BENEFIT DOES THE %XADATA 3MART &LASH #ACHE ACTUALLY PROVIDE 0ROBABLY ONE OF THE MOST
EFFECTIVE MEASURES OF THE EFFECTIVENESS OF THE CACHE IS THE SINGLE BLOCK READ TIMES (ERES A SNIPPET FROM
AN !72 REPORT FOR A FAIRLY REPRESENTATIVE MIXED WORKLOAD PRODUCTION ENVIRONMENT
4eT]cFPXcb+ \b+!\b+#\b+'\b+ %\b+"!\b+, b- b
RT[[bX]V[TQ[^RZ_WhbXRP[!( #:(% & $ ! # # " #
4HIS OUTPUT WAS TAKEN FROM A ONE
HOUR SNAPSHOT AND AS YOU CAN SEE THERE WERE ALMOST
SINGLE BLOCK READS OF WHICH ABOUT TOOK LESS THAN MS 4HIS IS FAIRLY REPRESENTATIVE OF THE PRODUCTION
SYSTEMS WEVE OBSERVED AND COMPARES FAVORABLY WITH 33$
BASED SYSTEMS 3YSTEMS THAT RUN ENTIRELY ON
33$ STORAGE GENERALLY HAVE AVERAGE SINGLE BLOCK ACCESS TIMES IN THE MS RANGE WHICH IS MARGINALLY
FASTER THAN WHAT WE GENERALLY SEE ON %XADATA (OWEVER THE 33$ SYSTEMS ARE USUALLY NOT CAPABLE OF
SCALING THE )/ THE WAY %XADATA CAN %XADATA ENJOYS THIS ADVANTAGE PRIMARILY BECAUSE IT IS DESIGNED TO BE
A BALANCED CONFIGURATION MEANING THAT THE COMPONENTS IN THE )/ PATH ARE MORE CLOSELY MATCHED THAN
IN MOST CONVENTIONAL STORAGE ARRAYS .OTHING IS WITHOUT LIMITATIONS OF COURSE BUT MOST STORAGE SYSTEMS
ARE BOTTLENECKED AT VARIOUS POINTS IN THE )/ PATH AT THE DISK CONTROLLERS FOR EXAMPLE
!N EXAMPLE MAY BE USEFUL IN ILLUSTRATING THIS POINT 7E RECENTLY DID A BENCHMARKING EXERCISE PITTING
AN %XADATA 6 (IGH #APACITY 3!4! QUARTER RACK SYSTEM AGAINST A 3UN - WITH CORES USING 33$ FOR
ALL DATABASE STORAGE 4HE TEST WAS A BATCH PROGRAM THAT SPENT MOST OF ITS TIME DOING WHAT WE WOULD
TYPICALLY CALL /,40
TYPE PROCESSING 4HERE WAS A LARGE TABLE WITH A BILLION PLUS ROWS BUT IT WAS ACCESSED
VIA "
4REE INDEXES A ROW AT A TIME /N THE - THE PROCESS TOOK ABOUT HOURS AND ON THE %XADATA IT
TOOK A LITTLE LESS THAN AN HOUR 7E HAVE HAD SIMILAR EXPERIENCES WITH OTHER /,40
TYPE WORKLOADS AS WELL
7E GENERALLY DONT SEE THE ENORMOUS PERFORMANCE BENEFITS OVER OTHER SYSTEMS THAT WE SEE ON DATA
WAREHOUSE ORIENTED WORKLOADS BUT WE HAVE SEEN THE PLATFORM TURN IN IMPROVED PERFORMANCE NUMBERS
ON EVERY SYSTEM WEVE COMPARED TO DATE
3O HOW ELSE COULD WE MEASURE THE EFFECTIVENESS OF %3&# )TS TEMPTING TO TRY TO CALCULATE A HIT RATIO
FOR THE FLASH CACHE .O DOUBT IT WOULD BE INTERESTING TO KNOW WHAT PERCENTAGE OF REQUESTED )/S ARE
SATISFIED BY THE %3&# "UT THIS CALCULATION IS NOT STRAIGHTFORWARD 3INCE %3&# CACHES MORE THAN JUST TABLE
AND INDEX BLOCKS ITS DIFFICULT TO GET A REASONABLE ESTIMATE 9OU COULD ARGUE THAT OTHER ACCESSES %3&#
BUFFERS CONTROL FILES ACCESS AS WELL ARE NOT SIGNIFICANT ENOUGH TO MAKE ANY DIFFERENCE AND THAT MAY BE
TRUE "UT THE FACT OF THE MATTER IS THERE IS NO EASY WAY TO DO ANYTHING OTHER THAN MAKE A SOMEWHAT
EDUCATED GUESS KNOWING FULL WELL THAT IT WONT BE COMPLETELY ACCURATE .EVERTHELESS HERE IS A SCRIPT
TbURNWXcNaPcX^ b`[ THAT MAKES AN ATTEMPT 4HE SCRIPT ADDS THE SINGLE BLOCK AND MULTIBLOCK PHYSICAL
READS AND COMPARES THOSE WITH THE CELL FLASH CACHE HITS )N MANY SYSTEMS THIS WILL BE REASONABLY CLOSE
9OU SHOULD BE AWARE THOUGH THAT 3MART 3CANS CAN ALSO ACCESS THE %3&# FOR OBJECTS THAT HAVE BEEN
DESIGNATED FOR AGGRESSIVE CACHING VIA THE 24;;N5;0B7N20274 STORAGE PARAMETER
BHB/4G31 -/TbURNWXcNaPcX^
BHB/4G31 -R^[d\]R WTPSX]V4eT]ck=P\TU^a\PcP" cad]R
BHB/4G31 -R^[d\]R!WTPSX]VC^cP[kFPXcbU^a\Pc((((((((
BHB/4G31 -R^[d\]R"WTPSX]VBTR^]SbkFPXcX]VU^a\Pc(((((((
BHB/4G31 -R^[d\]R$WTPSX]V0eVkFPXck\bU^a\Pc(((( (
BHB/4G31 -R^[d\]R%WTPSX]V5[PbW2PRWT7XcbU^a((((((((((((
BHB/4G31 -R^[WXcNaPcX^WTPSX]V7XcAPcX^U^a((( (((
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
BHB/4G31 -
BHB/4G31 -bT[TRc
!RT[[bX]V[T\d[cXQ[^RZaTPSbR
"R!R"R$R%
#R% STR^ST]e[R! R!WXcNaPcX^
$Ua^\
%bT[TRc
&bd\c^cP[NfPXcbR!
'PeVeP[dTR%
(bd\cX\TNfPXcTS R"
PeVPeTaPVTNfPXc R$
Ua^\
!bhb eNbhbcT\NTeT]cebhbbcPcbb
"fWTaT
#TeT]cX]
$RT[[bX]V[TQ[^RZ_WhbXRP[aTPS
%RT[[\d[cXQ[^RZ_WhbXRP[aTPS
&P]S
']P\T[XZTRT[[U[PbWRPRWTaTPSWXcb
(P]S
! TeT]c]^c[XZT8S[T
! ^aSTaQh
!!R"
!"*
0eV
4eT]cC^cP[BTR^]SbFPXc7Xc
=P\TFPXcbFPXcX]V\b5[PbW2PRWT7XcbAPcX^
RT[[bX]V[T\d[cXQ[^RZaTPSb %%#!"&#" $!#&($ ! "!$
!S YOU CAN SEE THE SCRIPT VASTLY OVERESTIMATES THE HIT RATIO ON THIS PARTICULAR SYSTEM COMING UP WITH
A HIT RATIO OF OVER 4HIS HAPPENS BECAUSE THIS SYSTEM HAS SEVERAL OBJECTS THAT ARE DESIGNATED FOR
AGGRESSIVE CACHING RESULTING IN MANY CACHE HITS FOR 3MART 3CAN EVENTS WHICH ARE NOT INCLUDED IN THE
SINGLE BLOCK AND MULTI
BLOCK READ STATISTICS &OR SYSTEMS THAT DONT USE THE :44? ATTRIBUTE THE NUMBERS
SHOULD BE CLOSER TO REALITY (OWEVER CALCULATED HIT RATIOS ARE PROBABLY NOT THE BEST APPROACH )T IS
GENERALLY MORE ENLIGHTENING TO LOOK AT THE ACTUAL SINGLE BLOCK ACCESS TIMES THAN TO TRY TO CALCULATE A HIT
RATIO
Summary
%XADATA 3MART &LASH #ACHE PROVIDES YET ANOTHER WAY TO PARE DOWN THE )/ COST ASSOCIATED WITH /RACLE
DATABASES -OST OF THE OPTIMIZATIONS THAT THE %XADATA PLATFORM PROVIDES REQUIRE THE USE OF 3MART 3CANS
FULL TABLE OR INDEX SCANS %3&# DOES NOT DEPEND ON 3MART 3CANS AND IN FACT IS MOST USEFUL FOR SPEEDING
UP ACCESS ON RANDOM READS OF SINGLE BLOCKS 4HESE OPERATIONS ARE GENERALLY ASSOCIATED WITH /,40
WORKLOADS AND SO %3&# IS THE KEY COMPONENT OF %XADATA FOR /,40 OR MIXED WORKLOADS 4HE FACT THAT
%3&# DOES NOT PROVIDE A WRITE
BACK CACHE THOUGH SEVERELY LIMITS ITS EFFECTIVENESS WITH SYSTEMS THAT ARE
BOTTLENECKED ON WRITES .EVERTHELESS THE LARGE CACHE AND THE INTELLIGENT CACHING ALGORITHMS USED BY THE
/RACLE STORAGE SOFTWARE ALLOW %3&# TO PROVIDE READ PERFORMANCE SIMILAR TO SOLID
STATE BASED DISK
#(!04%2 %8!$!4! 3-!24 &,!3( #!#(%
SYSTEMS /FFLOADING LARGE PORTIONS OF RANDOM READ ACTIVITY FROM THE DISKS ALSO PROVIDES AN INDIRECT
BENEFIT TO PROCESSES ATTEMPTING TO WRITE TO THOSE SAME DISKS
7HILE %3&# IS GENERALLY THOUGHT OF AS AN OPTIMIZATION AIMED AT REDUCING LATENCY FOR SMALL READS IT
CAN ALSO BE USED QUITE EFFECTIVELY FOR LARGE $7
TYPE QUERIES )N FACT THE LARGE THROUGHPUT NUMBERS THAT
/RACLE QUOTES DEPEND ON DISK AND FLASH CACHE BEING SCANNED CONCURRENTLY 4HE FLASH CACHE ACTUALLY
SHOULDERS THE LIONS SHARE OF THE BURDEN IN THIS CASE 9OU CAN ENABLE THIS ABILITY ON SPECIFIC TABLES BY
SETTING THE 24;;N5;0B7N20274 ATTRIBUTE ON THE TABLE TO :44? 4HIS ABILITY SHOULD NOT BE OVERLOOKED FOR KEY
TABLES AS THE PERFORMANCE DIFFERENCE CAN BE DRAMATIC
CHAPTER 6
Exadata Parallel Operations
%XADATA DOESNT HAVE A SPECIAL WAY OF EXECUTING PARALLEL OPERATIONS THAT IS NOT AVAILABLE ON OTHER
PLATFORMS RUNNING G2 (OWEVER PARALLEL PROCESSING IS A KEY COMPONENT OF %XADATA BECAUSE EFFICIENT
HANDLING OF $ATA 7AREHOUSE WORKLOADS WAS A PRIMARY DESIGN GOAL FOR %XADATA )N ADDITION BECAUSE
/FFLOADING3MART 3CAN DEPENDS ON DIRECT PATH READS WHICH ARE USED BY PARALLEL QUERY SLAVES PARALLEL
OPERATIONS TAKE ON A WHOLE NEW IMPORTANCE 4RADITIONALLY THE USE OF PARALLEL QUERY HAS REQUIRED CAREFUL
CONTROL OF CONCURRENCY IN ORDER TO MAXIMIZE THE USE OF AVAILABLE RESOURCES WITHOUT OVERWHELMING THE
SYSTEM /RACLES PREVIOUS ATTEMPTS AT THROTTLING PARALLEL OPERATIONS TO ALLOW THEM TO BE USED IN MULTIUSER
ENVIRONMENTS HAVE NOT BEEN ENTIRELY SUCCESSFUL G2 PROVIDES SOME NEW CAPABILITIES FOR CONTROLLING
PARALLEL OPERATIONS )N PARTICULAR A QUEUING MECHANISM HAS BEEN INTRODUCED THAT ALLOWS THE NUMBER OF
CONCURRENT PARALLEL PROCESSES TO BE MANAGED MORE EFFECTIVELY 4HIS APPROACH APPEARS TO BE MUCH BETTER
SUITED TO ALLOWING A HIGH DEGREE OF PARALLELISM WITHOUT OVERWHELMING THE AVAILABLE RESOURCES THAN
PREVIOUS ATTEMPTS G2 ALSO INTRODUCED THE ABILITY FOR /RACLE TO AUTOMATICALLY CALCULATE A DEGREE OF
PARALLELISM ON A STATEMENT
BY
STATEMENT BASIS
4HIS CHAPTER IS ABOUT PARALLEL QUERY /THER FORMS OF PARALLELISM PROVIDED BY THE DATABASE SUCH AS
RECOVERY PARALLELISM WILL NOT BE COVERED HERE 7E WILL BRIEFLY COVER PARALLEL QUERY MECHANICS AND
DEMONSTRATE SPECIFICALLY HOW THE QUEUING MECHANISM AND AUTOMATIC DEGREE OF PARALLELISM WORK WITH
%XADATA 4HIS CHAPTER WILL ALSO BRIEFLY COVER THE NEW G2 )N
-EMORY 0ARALLEL %XECUTION FEATURE AND
DISCUSS HOW %XADATA STORAGE PARALLELIZES )/ OPERATIONS )T WILL NOT COVER ALL THE DETAILS OF PARALLEL QUERY
OR PARALLEL $-, AND ASSUMES THAT THE READER IS FAMILIAR WITH BASIC /RACLE PARALLEL CONCEPTS .OTE ALSO
THAT THE DISCUSSION AND EXAMPLES WILL REFER TO BEHAVIOR IN /RACLE $ATABASE G 2ELEASE )N
MOST CASES THE COMMENTS APPLY EQUALLY TO VERSION AS WELL %XCEPTIONS ARE EXPLICITLY CALLED OUT
Parameters
"EFORE DESCRIBING THE VARIOUS NEW FEATURES AND HOW THEY APPLY TO %XADATA YOU SHOULD REVIEW THE
PARAMETERS THAT AFFECT HOW PARALLEL QUERIES ARE HANDLED BY THE DATABASE 4ABLE
SHOWS THE PARAMETERS
ALONG WITH A BRIEF DESCRIPTION OF EACH ONES PURPOSE
#(!04%2 -!+).' 0#"3
Table 6-1. Database Parameters Controlling 11gR2 Parallel Features
Parameter Default Description
_PaP[[T[NPSP_cXeTN\d[cXNdbTaCAD4 /LD MECHANISM OF THROTTLING PARALLEL STATEMENTS BY
DOWNGRADING )N MOST CASES THIS PARAMETER SHOULD
BE SET TO 50;B4 ON %XADATA FOR REASONS WELL DISCUSS
LATER IN THE CHAPTER
_PaP[[T[NPdc^\PcXRNcd]X]V50;B4 $EPRECATED SINCE G 4HIS PARAMETER ENABLED AN
AUTOMATIC $/0 CALCULATION ON OBJECTS FOR WHICH A
PARALLELISM ATTRIBUTE IS SET
_PaP[[T[NSTVaTTN[X\Xc2?D 4HIS PARAMETER SETS AN UPPER LIMIT ON THE $/0 THAT
CAN BE APPLIED TO A SINGLE STATEMENT 4HE DEFAULT
MEANS THAT /RACLE WILL CALCULATE A VALUE FOR THIS LIMIT
BASED ON THE SYSTEMS CHARACTERISTICS
_PaP[[T[NSTVaTTN_^[XRh<0=D0; #ONTROLS SEVERAL PARALLEL FEATURES INCLUDING
!UTOMATIC $EGREE OF 0ARALLELISM AUTO $/0 0ARALLEL
3TATEMENT 1UEUING AND )N
MEMORY 0ARALLEL
%XECUTION
_PaP[[T[NTgTRdcX^]N\TbbPVTNbXiT %"'# 4HE SIZE OF PARALLEL MESSAGE BUFFERS IN BYTES
_PaP[[T[NU^aRTN[^RP[50;B4 $ETERMINES WHETHER PARALLEL QUERY SLAVES WILL BE
FORCED TO EXECUTE ONLY ON THE NODE THAT INITIATED THE
QUERY CAD4 OR WHETHER THEY WILL BE ALLOWED TO
SPREAD ON TO MULTIPLE NODES IN A 2!# CLUSTER 50;B4
_PaP[[T[NX]bcP]RTNVa^d_ 5SED TO RESTRICT PARALLEL SLAVES TO CERTAIN SET OF
INSTANCES IN A 2!# CLUSTER
_PaP[[T[NX^NRP_NT]PQ[TS50;B4 5SED IN CONJUNCTION WITH THE
31<BNA4B>DA24N<0=064A 20;81A0C4N8> FUNCTION TO
LIMIT DEFAULT $/0 CALCULATIONS BASED ON THE )/
CAPABILITIES OF THE SYSTEM
_PaP[[T[N\PgNbTaeTab 4HE MAXIMUM NUMBER OF PARALLEL SLAVE PROCESS THAT
MAY BE CREATED ON AN INSTANCE 4HE DEFAULT IS
CALCULATED BASED ON SYSTEM PARAMETERS INCLUDING
2?DN2>D=C AND ?0A0;;4;NC7A403BN?4AN2?D /N MOST
SYSTEMS THE VALUE WILL WORK OUT TO BE ¯2?DN2>D=C
_PaP[[T[N\X]N_TaRT]c /LD THROTTLING MECHANISM )T REPRESENTS THE
MINIMUM PERCENTAGE OF PARALLEL SERVERS THAT ARE
NEEDED FOR A PARALLEL STATEMENT TO EXECUTE
#(!04%2 -!+).' 0#"3
Parameter Default Description
_PaP[[T[N\X]NbTaeTab 4HE MINIMUM NUMBER OF PARALLEL SLAVE PROCESSES
THAT SHOULD BE KEPT RUNNING REGARDLESS OF USAGE
5SUALLY SET TO ELIMINATE THE OVERHEAD OF CREATING AND
DESTROYING PARALLEL PROCESSES
_PaP[[T[N\X]NcX\TNcWaTbW^[S0DC> 4HE MINIMUM ESTIMATED SERIAL EXECUTION TIME THAT
WILL BE TRIGGER AUTO $/0 4HE DEFAULT IS 0DC> WHICH
TRANSLATES TO SECONDS 7HEN THE
?0A0;;4;N346A44N?>;82H PARAMETER IS SET TO 0DC> OR
;8<8C43 ANY STATEMENT THAT IS ESTIMATED TO TAKE
LONGER THAN THE THRESHOLD ESTABLISHED BY THIS
PARAMETER WILL BE CONSIDERED A CANDIDATE FOR AUTO
$/0
_PaP[[T[NbTaeTaCAD4 50;B4 (AS NOTHING TO DO WITH PARALLEL QUERIES 3ET TO TRUE OR
FALSE DEPENDING ON WHETHER THE DATABASE IS 2!#
ENABLED OR NOT 4HIS PARAMETER WAS DEPRECATED LONG
AGO AND HAS BEEN REPLACED BY THE 2;DBC4AN30C010B4
PARAMETER
_PaP[[T[NbTaeTaNX]bcP]RTb OR THE (AS NOTHING TO DO WITH PARALLEL QUERIES )T IS SET TO THE
NUMBER OF NUMBER OF INSTANCES IN A 2!# CLUSTER
2!# INSTANCES
_PaP[[T[NbTaeTabNcPaVTc 4HE UPPER LIMIT ON THE NUMBER OF PARALLEL SLAVES THAT
MAY BE IN USE ON AN INSTANCE AT ANY GIVEN TIME IF
PARALLEL QUEUING IS ENABLED 4HE DEFAULT IS CALCULATED
AUTOMATICALLY
_PaP[[T[NcWaTPSbN_TaNR_d 5SED IN VARIOUS PARALLEL CALCULATIONS TO REPRESENT THE
NUMBER OF CONCURRENT PROCESSES THAT A #05 CAN
SUPPORT
Parallelization at the Storage Tier
%XADATA HAS A LOT OF PROCESSING POWER AT THE STORAGE LAYER 2EGARDLESS OF WHETHER YOU ARE USING A 6
QUARTER RACK OR AN 8 FULL RACK YOU STILL HAVE MORE #05 RESOURCES AVAILABLE AT THE STORAGE LAYER THAN
YOU HAVE AT THE DATABASE LAYER 3INCE 3MART 3CANS OFFLOAD A LOT OF PROCESSING TO THE STORAGE CELLS EVERY
QUERY EXECUTED VIA 3MART 3CAN IS EFFECTIVELY PARALLELIZED ACROSS THE #05S ON THE STORAGE CELLS 4HIS TYPE
OF PARALLELIZATION IS COMPLETELY INDEPENDENT OF THE DATABASE PARALLEL PROCESSING CAPABILITIES BY THE WAY
3O THIS KIND OF PARALLELIZATION OCCURS EVEN WHEN THE ACTIVITY IS DRIVEN BY A SINGLE PROCESS ON A SINGLE
DATABASE SERVER 4HIS INTRODUCES SOME INTERESTING ISSUES THAT SHOULD BE CONSIDERED WITH REGARD TO
NORMAL PARALLELIZATION AT THE DATABASE TIER 3INCE ONE OF THE PRIMARY JOBS OF A PARALLELIZED QUERY IS TO
ALLOW MULTIPLE PROCESSES TO PARTICIPATE IN THE )/ OPERATIONS AND SINCE THE )/ OPERATIONS ARE ALREADY
SPREAD ACROSS MULTIPLE PROCESSES THE DEGREE OF PARALLELISM REQUIRED BY STATEMENTS RUNNING ON THE
%XADATA PLATFORM SHOULD BE SMALLER THAN ON OTHER PLATFORMS
#(!04%2 -!+).' 0#"3
Auto DOP
/NE OF THE MAJOR CHANGES TO PARALLEL OPERATIONS IN /RACLE $ATABASE G 2ELEASE WAS THE ADDITION OF A
FEATURE AFFECTIONATELY KNOWN AS !UTO $/0 )T WAS DESIGNED TO OVERCOME THE PROBLEMS ASSOCIATED WITH
THE FACT THAT THERE IS RARELY A SINGLE $/0 VALUE THAT IS APPROPRIATE FOR ALL QUERIES TOUCHING A PARTICULAR
OBJECT 0RIOR TO G2 $/0 COULD BE SPECIFIED AT THE STATEMENT LEVEL VIA HINTS OR AT THE OBJECT LEVEL VIA
THE 346A44 AND 8=BC0=24 SETTINGS 2EALISTICALLY USING HINTS AT THE STATEMENT LEVEL MAKES MORE SENSE IN
MOST SITUATIONS FOR THE REASON JUST MENTIONED "UT IT REQUIRES THAT THE DEVELOPERS UNDERSTAND THE
PLATFORM THAT THE STATEMENTS WILL BE RUNNING ON AND THE WORKLOAD THAT THE HARDWARE WILL BE SUPPORTING
AT THE TIME OF EXECUTION 'ETTING THE SETTINGS CORRECT CAN BE A TEDIOUS TRIAL
AND
ERROR PROCESS AND
UNFORTUNATELY $/0 CANNOT BE CHANGED WHILE A STATEMENT IS RUNNING /NCE IT STARTS YOUR ONLY OPTIONS
ARE TO LET IT COMPLETE OR KILL IT AND TRY AGAIN 4HIS MAKES FINE TUNING IN A hLIVEv ENVIRONMENT A PAINFUL
PROCESS
/PERATION AND #ONFIGURATION
7HEN !UTO $/0 IS ENABLED /RACLE EVALUATES EACH STATEMENT TO DETERMINE WHETHER IT SHOULD BE RUN IN
PARALLEL AND IF SO WHAT $/0 SHOULD BE USED "ASICALLY ANY STATEMENT THAT THE OPTIMIZER CONCLUDES WILL
TAKE LONGER THAN SECONDS TO RUN SERIALLY WILL BE A CANDIDATE TO RUN IN PARALLEL 4HE
SECOND THRESHOLD
CAN BE CONTROLLED BY SETTING THE ?0A0;;4;N<8=NC8<4NC7A4B7>;3 PARAMETER BY THE WAY 4HIS DECISION IS
MADE REGARDLESS OF WHETHER ANY OF THE OBJECTS INVOLVED IN THE STATEMENT HAVE BEEN DECORATED WITH A
PARALLEL DEGREE SETTING OR NOT
!UTO $/0 IS ENABLED BY SETTING THE ?0A0;;4;N346A44N?>;82H PARAMETER TO A VALUE OF 0DC> OR ;8<8C43
4HE DEFAULT SETTING FOR THIS PARAMETER IS <0=D0; WHICH DISABLES ALL THREE OF THE NEW G2 PARALLEL
FEATURES !UTO $/0 0ARALLEL 3TATEMENT 1UEUING )N
MEMORY 0ARALLEL %XECUTION 5NFORTUNATELY
?0A0;;4;N346A44N?>;82H IS ONE OF THOSE PARAMETERS THAT CONTROL MORE THAN ONE THING 4HE FOLLOWING LIST
SHOWS THE EFFECTS OF THE VARIOUS SETTINGS FOR THIS PARAMETER
<0=D0; )F ?0A0;;4;N346A44N?>;82H IS SET TO <0=D0; NONE OF THE NEW G2
PARALLEL FEATURES WILL BE ENABLED 0ARALLEL PROCESSING WILL WORK AS IT DID IN
PREVIOUS VERSIONS 4HAT IS TO SAY STATEMENTS WILL ONLY BE PARALLELIZED IF A HINT IS
USED OR AN OBJECT IS DECORATED WITH A PARALLEL SETTING
;8<8C43 )F ?0A0;;4;N346A44N?>;82H IS SET TO ;8<8C43 ONLY !UTO $/0 IS ENABLED
WHILE 0ARALLEL 3TATEMENT 1UEUING AND )N
MEMORY 0ARALLEL %XECUTION REMAIN
DISABLED )N ADDITION ONLY STATEMENTS ACCESSING OBJECTS THAT HAVE BEEN
DECORATED WITH THE DEFAULT PARALLEL SETTING WILL BE CONSIDERED FOR !UTO $/0
CALCULATION
0DC>) )F ?0A0;;4;N346A44N?>;82H IS SET TO 0DC> ALL THREE OF THE NEW FEATURES ARE
ENABLED 3TATEMENTS WILL BE EVALUATED FOR PARALLEL EXECUTION REGARDLESS OF ANY
PARALLEL DECORATION AT THE OBJECT LEVEL
!LTHOUGH THE ONLY DOCUMENTED WAY TO ENABLE 0ARALLEL 3TATEMENT 1UEUING AND )N
MEMORY 0ARALLEL
%XECUTION ARE VIA THE ALL
OR
NOTHING SETTING OF 0DC> THE DEVELOPERS HAVE THOUGHTFULLY PROVIDED HIDDEN
PARAMETERS THAT PROVIDE INDEPENDENT CONTROL OF THESE FEATURES 4ABLE
SHOWS THE PARAMETERS AND HOW
THE SETTINGS OF ?0A0;;4;N346A44N?>;82H ALTER THE HIDDEN PARAMETERS
#(!04%2 -!+).' 0#"3
Table 6-2. Hidden Parameters Affected by PARALLEL_DEGREE_POLICY
Parallel_Degree_Policy Parameter Value
<0=D0;N_PaP[[T[NbcPcT\T]cN`dTdX]V 50;B4
N_PaP[[T[NR[dbcTaNRPRWTN_^[XRh 030?C8E4
;8<8C43N_PaP[[T[NbcPcT\T]cN`dTdX]V 50;B4
N_PaP[[T[NR[dbcTaNRPRWTN_^[XRh 030?C8E4
0DC>N_PaP[[T[NbcPcT\T]cN`dTdX]V CAD4
N_PaP[[T[NR[dbcTaNRPRWTN_^[XRh 202743
)TS PRETTY OBVIOUS WHAT THE N?0A0;;4;NBC0C4<4=CN@D4D8=6 PARAMETER CONTROLS 7HEN IT IS SET TO
425% QUEUING IS ENABLED 4HE PURPOSE OF THE N?0A0;;4;N2;DBC4AN20274N?>;82H PARAMETER IS A LITTLE LESS
OBVIOUS )T TURNS OUT THAT IT CONTROLS )N
MEMORY 0ARALLEL %XECUTION 3ETTING THE VALUE OF THE
N?0A0;;4;N2;DBC4AN20274N?>;82H PARAMETER TO 202743 ENABLES )N
MEMORY 0ARALLEL %XECUTION 9OU
SHOULD NOTE THAT )N
MEMORY 0ARALLEL %XECUTION IS ARGUABLY OF LESS VALUE ON THE %XADATA PLATFORM
BECAUSE THE 3MART 3CAN OPTIMIZATIONS WILL NOT BE AVAILABLE WHEN USING THIS FEATURE TO SCAN THE BUFFER
CACHE 7ELL DISCUSS THAT IN MORE DETAIL A LITTLE LATER )N THE MEANTIME HERES AN EXAMPLE SHOWING !UTO
$/0 IN ACTION
BHB/4G31 -/SQPNcPQ[Tb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
>F=4AC01;4N=0<4BC0CDB;0BCN0=0;=D<NA>FB1;>2:B346A44
:B>B:4FE0;83!$ =>E "! # %!!(#
BHB/4G31 -P[cTabhbcT\bTc_PaP[[T[NSTVaTTN_^[XRh,Pdc^*
BhbcT\P[cTaTS
BHB/4G31 -bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfPfWTaTR^[ - *
0E6?:N2>;
% ("&#( "
a^fbT[TRcTS
BHB/4G31 -/Ub
4]cTaeP[dTU^ab`[NcTgc)bZTf
4]cTaeP[dTU^ab`[NXS)
#(!04%2 -!+).' 0#"3
B@;N830E6N4C8<40E6N;8>B@;NC4GC
$R`!WQ a"&ca!' && %!#!&bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfP
fWTaTR^[ -
a^fbbT[TRcTS
BHB/4G31 -RPcS_[P] b`[
bTc[X]Tb $
bT[TRcUa^\cPQ[TSQ\bNg_[P] SXb_[PhNRdab^ab`[NXSRWX[SN]^ch_XRP[
BHB/4G31 -/S_[P]
4]cTaeP[dTU^ab`[NXS) $R`!WQ a"&ca
4]cTaeP[dTU^aRWX[SN]^)!
?;0=NC01;4N>DC?DC
B@;N83 $R`!WQ a"&caRWX[S]d\QTa!
bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfPfWTaTR^[ -
?[P]WPbWeP[dT)$&'"%% &
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\TkC@k
k kB4;42CBC0C4<4=Ckkkk%" ' kkk
k kB>AC066A460C4kk k!#kkkk
k!k?G2>>A38=0C>Akkkkkkk
k"k?GB4=3@2A0=3><k)C@ k k!#kkk@ k
k#kB>AC066A460C4kk k!#kkk@ k
k$k?G1;>2:8C4A0C>Akk"!<k&"!<k%" ' k ) ) %k@ k
k%kC01;40224BB5D;;kB:4Fk"!<k&"!<k%" ' k ) ) %k@ k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
% PRRTbb)I-,)I0=3)I+,)I
UX[cTa2>; -
=^cT
Pdc^\PcXR3>?)2^\_dcTS3TVaTT^U?PaP[[T[Xb\Xb$
!'a^fbbT[TRcTS
#(!04%2 -!+).' 0#"3
3O AS YOU CAN SEE ENABLING !UTO $/0 ALLOWED THE STATEMENT TO BE PARALLELIZED EVEN THOUGH THE
TABLE WAS NOT DECORATED WITH A PARALLEL SETTING !LSO NOTICE THAT THE PLAN OUTPUT PRODUCED BY 31<BNG?;0=
SHOWS THAT !UTO $/0 WAS ENABLED AND THE CALCULATED $/0 WAS
/RACLE HAS PROVIDED SOME CONTROL OVER THE !UTOMATIC $/0 CALCULATIONS AS WELL WITH THE
?0A0;;4;N346A44N;8<8C PARAMETER 4HE DEFAULT VALUE FOR THIS PARAMETER IS 2?D WHICH TELLS /RACLE TO
CALCULATE AN h)DEAL $/0v BASED ON THE AMOUNT OF DATA BUT THEN CAPS IT WITH A FORMULA BASED ON
2?DN2>D=C C7A403BN?4AN2?D AND 02C8E4N8=BC0=24N2>D=C
)/ #ALIBRATION
/RACLE $ATABASE VERSION INTRODUCED A RESTRICTION TO !UTO $/0 REQUIRING THAT THE )/ SYSTEM BE
CALIBRATED BEFORE STATEMENTS WILL BE AUTOMATICALLY PARALLELIZED 4HE CALIBRATION IS DONE BY THE
31<BNA4B>DA24N<0=064A 20;81A0C4N8> PROCEDURE WHICH GENERATES A RANDOM READ
ONLY WORKLOAD AND
SPREADS IT ACROSS ALL INSTANCES IN A 2!# CLUSTER )T RUNS FOR SEVERAL MINUTES LONGER ON 2!# DATABASES
BECAUSE IT MUST RUN ON EACH INSTANCE 4HE PROCEDURE CAN PUT A SIGNIFICANT LOAD ON THE SYSTEM AND THE
DOCUMENTATION RECOMMENDS RUNNING IT WHEN THE SYSTEM IS IDLE OR VERY LIGHTLY LOADED (ERES AN
EXAMPLE OF WHAT HAPPENS IF THE CALIBRATION PROCEDURE HAS NOT BEEN RUN ON
BHB/;01 ! !-/S_[P]
4]cTaeP[dTU^ab`[NXS) $R`!WQ a"&ca
4]cTaeP[dTU^aRWX[SN]^)
?;0=NC01;4N>DC?DC
B@;N83 $R`!WQ a"&caRWX[S]d\QTa
bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfPfWTaTR^[ -
?[P]WPbWeP[dT)$%'"!!"&%
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckkkk##!(' kk
k kB>AC066A460C4kk k kkk
k!kC01;40224BB5D;;kB:4Fk"!<k""$<k##!(' k ) )!(k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! UX[cTa2>; -
=^cT
Pdc^\PcXR3>?)bZX__TSQTRPdbT^U8>RP[XQaPcTbcPcXbcXRbPaT\XbbX]V
!"a^fbbT[TRcTS
!S YOU CAN SEE WHEN THE )/ CALIBRATION HAS NOT BEEN DONE !UTO $/0 IS DISABLED AND THE OPTIMIZER
GENERATES A SERIAL EXECUTION PLAN 4HERE ARE TWO VIEWS THAT PROVIDE ADDITIONAL INFORMATION ABOUT THE
#(!04%2 -!+).' 0#"3
CALIBRATION PROCESS 4HE E8>N20;81A0C8>=NBC0CDB VIEW SHOWS WHETHER THE CALIBRATION HAS BEEN DONE
WHILE THE 310NABA2N8>N20;81A0C4 VIEW SHOWS THE RESULTS OF THE PROCEDURE (ERES AN EXAMPLE SHOWING
HOW TO USE THE PROCEDURE
BHB/4G31 -bT[TRcUa^\E8>N20;81A0C8>=NBC0CDB*
BC0CDB20;81A0C8>=NC8<4
=>C0E08;01;4
a^fbT[TRcTS
BHB/4G31 -RPcRP[XQaPcTNX^ b`[
B4CB4AE4A>DC?DC>=
342;0A4
[Pc8=C464A*
X^_b8=C464A*
\Q_b8=C464A*
1468=
31<BNA4B>DA24N<0=064A 20;81A0C4N8>+38B:B-+<0GN;0C4=2H-X^_b\Q_b[Pc*
31<BNA4B>DA24N<0=064A 20;81A0C4N8>]^N^UNSXbZb X^_b\Q_b[Pc*
31<BN>DC?DC ?DCN;8=4\PgNX^_b,kkX^_b*
31<BN>DC?DC ?DCN;8=4[PcT]Rh,kk[Pc*
SQ\bN^dc_dc _dcN[X]T\PgN\Q_b,kk\Q_b*
T]S*
BHB/4G31 -bTccX\X]V^]
BHB/4G31 -/RP[XQaPcTNX^
4]cTaeP[dTU^a]^N^UNSXbZb)"%
bfXcRWc^^cWTabTbbX^]^]31bTaeTa
-c^_
c^_ ! ) !)"!d_ SPhb()!&"dbTab[^PSPeTaPVT) #( ## #&
CPbZb)' #c^cP[ ad]]X]V' "b[TT_X]V bc^__TS i^\QXT
2_db) #db bh ]X(( #XS fP WX bX bc
<T\)&# !&&$!Zc^cP[$!!!$ 'ZdbTS! ' !%##ZUaTT'!(""%ZQdUUTab
BfP_) %&& '$!Zc^cP[ ZdbTS %&& '$!ZUaTT#" &'$ 'ZRPRWTS
?83DB4A?A=8E8ACA4BB7AB2?D<4<C8<42><<0=3
#'!$^aPR[T $ $!!$\!!\ (\B & ) $^aPNRb !N4G31
#'!&^aPR[T $ $!!$\!!\ (\B ) #^aPNRb "N4G31
#'" ^aPR[T $ $!!$\!!\ (\B ) #^aPNRb $N4G31
' ^aPR[T $ ! !V$#\"(\B % !)## %&^aPNSXP N?>2
#'! ^aPR[T $ $!!$\!!\ (\B % ) "^aPNRb N4G31
#'!"^aPR[T $ $!!$\!!\ (\B % ) $^aPNRb N4G31
#'!(^aPR[T $ $!!$\!!\ (\B % ) !^aPNRb #N4G31
$!"(^bQ^a]T % "!%' %#'' %A % ) "c^_
!' '^aPR[TAC ""$\ ""\$!\B % !'')! "#
#(!04%2 -!+).' 0#"3
bfXcRWc^^]T^UcWTRT[[b
JT]ZRT[ )a^^cL a^^c-e\bcPc!
_a^Rb \T\^ah bfP_ X^ bhbcT\ R_d
aQbf_SUaTTQdUURPRWTbXb^QXQ^X]RbdbbhXSfPbc
( #&! " &"% "$" (! "( !(' ('
( #&$ #" &"% "$" (! ! !"" &"!! "'$ (#%
( #&$ #" &"% "$" (! ' !# %&$! !(# (#$
( #' " &"% "$" "% ( & $& & $! & $ (#$
( #&'&%" &"% "$" # ! '"$& && ! (!' (#$
( #$&&%" &"% "$" # ( %!(' & "! '% (#$
bfXcRWQPRZc^^aXVX]P[bTbbX^]
\PgNX^_b, #
[PcT]Rh,
\PgN\Q_b,!!
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4[P_bTS) ) ")# !&
BHB/4G31 -bTccX\X]V^UU
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)RT[[U[PbWRPRWT
=0<4E0;D4
RT[[U[PbWRPRWTaTPSWXcb
BHB/4G31 -bT[TRcUa^\E8>N20;81A0C8>=NBC0CDB*
BC0CDB20;81A0C8>=NC8<4
A403H 342 ( ! !$#?<
BHB/4G31 -bT[TRcc^NRWPabcPacNcX\TSS \^] hhbcPacNcX\T
!<0GN8>?B<0GN<1?B<0GN?<1?B;0C4=2H=D<N?7HB820;N38B:B
"Ua^\310NABA2N8>N20;81A0C4*
BC0ACNC8<<0GN8>?B<0GN<1?B<0GN?<1?B;0C4=2H=D<N?7HB820;N38B:B
STR #!! "%
4HIS LISTING IS A LITTLE LONG SO LET@S WALK THROUGH WHAT WE DID
7E CHECKED THE CURRENT STATUS IN E8>N20;81A0C8>=NBC0CDB
7E RAN THE 31<BNA4B>DA24N<0=064A 20;81A0C4N8> PROCEDURE USING A SCRIPT
CALLED RP[XQaPcTNX^ b`[
#(!04%2 -!+).' 0#"3
7E THEN JUMPED TO ANOTHER SESSION TO SEE WHAT KIND OF ACTIVITY WAS GENERATED
ON THE DATABASE SERVER USING c^_
.EXT WE CREATED A SESSION ON ONE OF THE STORAGE SERVERS TO SEE THE ACTIVITY THERE
USING e\bcPc
7E THEN SWITCHED BACK TO THE ORIGINAL SESSION AND AFTER ALMOST MINUTES THE
PROCEDURE COMPLETED
&INALLY WE CHECKED THE STATUS AND RESULTS IN E8>N20;81A0C8>=NBC0CDB AND
310NABA2N8>N20;81A0C4
!UTO $/0 7RAP 5P
4HE END RESULT OF SETTING ?0A0;;4;N346A44N?>;82H TO 0DC> IS THAT ALL KINDS OF STATEMENTS WILL BE RUN IN
PARALLEL EVEN IF NO OBJECTS HAVE BEEN SPECIFICALLY DECORATED WITH A PARALLEL DEGREE SETTING 4HIS IS TRULY
AUTOMATIC PARALLEL PROCESSING BECAUSE THE DATABASE DECIDES WHAT TO RUN IN PARALLEL AND WITH HOW MANY
SLAVES /N TOP OF THAT BY DEFAULT THE SLAVES MAY BE SPREAD ACROSS MULTIPLE NODES IN A 2!# DATABASE
5NFORTUNATELY THIS COMBINATION OF FEATURES IS A LITTLE LIKE THE 7ILD 7EST WITH THINGS RUNNING IN PARALLEL ALL
OVER THE PLACE "UT THE ABILITY TO QUEUE PARALLEL STATEMENTS DOES PROVIDE SOME SEMBLANCE OF ORDER WHICH
LEADS US TO THE NEXT TOPIC
Parallel Statement Queuing
) RECENTLY ATTENDED A 4ECH $AY WHERE ONE OF THE SPEAKERS RECALLED THE PRESENTATION ,ARRY %LLISON DID
WHEN /RACLE FIRST ANNOUNCED THE 0ARALLEL 1UERY FEATURE /RACLE VERSION )N THE DEMO ,ARRY HAD A
MULTIPROCESSOR COMPUTER ALL TO HIMSELF ) DONT REMEMBER HOW MANY PROCESSORS IT HAD BUT ) REMEMBER
HE HAD SOME KIND OF GRAPHIC SHOWING INDIVIDUAL #05 UTILIZATION ON ONE SCREEN WHILE HE FIRED UP A
PARALLEL QUERY ON ANOTHER SCREEN 4HE MONITORING SCREEN LIT UP LIKE A #HRISTMAS TREE %VERY ONE OF THE
#05S WAS PEGGED DURING HIS DEMO 7HEN THE SPEAKER WAS TELLING THE STORY HE SAID THAT HE HAD WONDERED
AT THE TIME WHAT WOULD HAVE HAPPENED IF THERE HAD BEEN OTHER USERS ON THE SYSTEM DURING THE DEMO
4HEIR EXPERIENCE WOULD PROBABLY NOT HAVE BEEN A GOOD ONE 4HATS EXACTLY THE ISSUE THAT 0ARALLEL
3TATEMENT 1UEUING ATTEMPTS TO RESOLVE
/RACLES PARALLEL CAPABILITIES HAVE BEEN A GREAT GIFT BUT THEY HAVE ALSO BEEN A CURSE BECAUSE
CONTROLLING THE BEAST IN AN ENVIRONMENT WHERE THERE ARE MULTIPLE USERS TRYING TO SHARE THE RESOURCES IS
DIFFICULT AT BEST 4HERE HAVE BEEN ATTEMPTS TO COME UP WITH A REASONABLE WAY OF THROTTLING BIG PARALLEL
STATEMENTS "UT TO DATE ) DONT THINK THOSE ATTEMPTS HAVE BEEN OVERLY SUCCESSFUL
/NE OF THE MOST PROMISING ASPECTS OF %XADATA IS ITS POTENTIAL TO RUN MIXED WORKLOADS /,40 AND
$7 WITHOUT CRIPPLING ONE OR THE OTHER )N ORDER TO DO THAT /RACLE NEEDS SOME MECHANISM TO SEPARATE
THE WORKLOADS AND JUST AS IMPORTANTLY TO THROTTLE THE RESOURCE INTENSIVE PARALLEL QUERIES 0ARALLEL
3TATEMENT 1UEUING APPEARS TO BE JUST SUCH A TOOL !ND WHEN COMBINED WITH 2ESOURCE -ANAGER IT
PROVIDES A PRETTY ROBUST MECHANISM FOR THROTTLING THE WORKLOAD TO A LEVEL THAT THE HARDWARE CAN SUPPORT
#(!04%2 -!+).' 0#"3
4HE /LD 7AY
"EFORE WE GET TO THE NEW 0ARALLEL 1UEUING FUNCTIONALITY WE SHOULD PROBABLY REVIEW HOW IT WAS DONE IN
PREVIOUS VERSIONS 4HE BEST TOOL WE HAD AT OUR DISPOSAL WAS 0ARALLEL !DAPTIVE -ULTIUSER WHICH PROVIDES
THE ABILITY TO AUTOMATICALLY DOWNGRADE THE DEGREE OF PARALLELISM FOR A GIVEN STATEMENT BASED ON THE
WORKLOAD WHEN A QUERY EXECUTES )T IS ACTUALLY A POWERFUL MECHANISM AND IT IS THE BEST APPROACH WEVE
HAD PRIOR TO G2 4HIS FEATURE IS ENABLED BY SETTING THE ?0A0;;4;N030?C8E4N<D;C8NDB4A PARAMETER TO
425% 4HIS IS STILL THE DEFAULT IN G2 BY THE WAY SO THIS IS DEFINITELY A PARAMETER THAT YOU MAY WANT TO
CONSIDER CHANGING 4HE DOWNSIDE OF THIS APPROACH IS THAT PARALLELIZED STATEMENTS CAN HAVE WILDLY
VARYING EXECUTION TIMES !S YOU CAN IMAGINE A STATEMENT THAT GETS SLAVES ONE TIME AND THEN GETS
DOWNGRADED TO SERIAL EXECUTION THE NEXT TIME WILL PROBABLY NOT MAKE THE USERS VERY HAPPY
4HE ARGUMENT FOR THIS TYPE OF APPROACH IS THAT STUFF IS GOING TO RUN SLOWER IF THE SYSTEM IS BUSY
REGARDLESS OF WHAT YOU DO AND THAT USERS EXPECT IT TO RUN SLOWER WHEN THE SYSTEM IS BUSY 4HE FIRST PART OF
THAT STATEMENT MAY BE TRUE BUT ) DONT BELIEVE THE SECOND PART IS AT LEAST IN MOST CASES 4HE BIGGER
PROBLEM WITH THE DOWNGRADE MECHANISM THOUGH IS THAT THE DECISION ABOUT HOW MANY SLAVES TO USE IS
BASED ON A SINGLE POINT IN TIME THE POINT WHEN THE PARALLEL STATEMENT STARTS 2ECALL THAT ONCE THE DEGREE
OF PARALLELISM $/0 IS SET FOR AN EXECUTION PLAN IT CANNOT BE CHANGED 4HE STATEMENT WILL RUN TO
COMPLETION WITH THE NUMBER OF SLAVES IT WAS ASSIGNED EVEN IF ADDITIONAL RESOURCES BECOME AVAILABLE
WHILE IT IS RUNNING
3O CONSIDER THE STATEMENT THAT TAKES ONE MINUTE TO EXECUTE WITH SLAVES !ND SUPPOSE THAT SAME
STATEMENT GETS DOWNGRADED TO SERIAL DUE TO A MOMENTARILY HIGH LOAD !ND SAY THAT A FEW SECONDS AFTER IT
STARTS THE SYSTEM LOAD DROPS BACK TO MORE NORMAL LEVELS 5NFORTUNATELY THE SERIALIZED STATEMENT WILL
CONTINUE TO RUN FOR NEARLY MINUTES WITH ITS SINGLE PROCESS EVEN THOUGH ON AVERAGE THE SYSTEM IS NOT
BUSIER THAN USUAL 4HIS SORT OF ERRATIC PERFORMANCE CAN WREAK HAVOC FOR THOSE USING THE SYSTEM AND FOR
THOSE SUPPORTING IT
4HE .EW 7AY
.OW LETS COMPARE 0ARALLEL !DAPTIVE -ULTI 5SER THE OLD WAY WITH THE NEW MECHANISM INTRODUCED IN
G2 THAT ALLOWS PARALLEL STATEMENTS TO BE QUEUED 4HIS MECHANISM SEPARATES LONG RUNNING PARALLEL
QUERIES FROM THE REST OF THE WORKLOAD 4HE MECHANICS ARE PRETTY SIMPLE 4URN THE FEATURE ON 3ET A TARGET
NUMBER OF PARALLEL SLAVES USING THE ?0A0;;4;NB4AE4ABNC0A64C PARAMETER 2UN STUFF )F A STATEMENT THAT
REQUIRES EXCEEDING THE TARGET TRIES TO START IT WILL BE QUEUED UNTIL THE REQUIRED NUMBER OF SLAVES BECOME
AVAILABLE 4HERE ARE OF COURSE MANY DETAILS TO CONSIDER AND OTHER CONTROL MECHANISMS THAT CAN BE
APPLIED TO MANAGE THE PROCESS 3O LETS LOOK AT HOW IT BEHAVES
BHB/4G31 -P[cTabhbcT\bTc_PaP[[T[NSTVaTTN_^[XRh,Pdc^*
BhbcT\P[cTaTS
BHB/4G31 -P[cTabhbcT\bTc_PaP[[T[NbTaeTabNcPaVTc, *
BhbcT\P[cTaTS
BHB/4G31 -/_Pa\b
4]cTaeP[dTU^a_PaP\TcTa)_PaP[[T[
4]cTaeP[dTU^aXbbTc)
4]cTaeP[dTU^abW^fNWXSST])
#(!04%2 -!+).' 0#"3
=0<4E0;D48B3450D;
_PaP[[T[NPSP_cXeTN\d[cXNdbTa50;B4CAD4
_PaP[[T[NPdc^\PcXRNcd]X]V50;B4CAD4
_PaP[[T[NSTVaTTN[X\Xc'50;B4
_PaP[[T[NSTVaTTN_^[XRh0DC>50;B4
_PaP[[T[NTgTRdcX^]N\TbbPVTNbXiT %"'#CAD4
_PaP[[T[NU^aRTN[^RP[CAD450;B4
_PaP[[T[NX]bcP]RTNVa^d_CAD4
_PaP[[T[NX^NRP_NT]PQ[TS50;B4CAD4
_PaP[[T[N\PgNbTaeTab"! CAD4
_PaP[[T[N\X]N_TaRT]c CAD4
_PaP[[T[N\X]NbTaeTab %50;B4
_PaP[[T[N\X]NcX\TNcWaTbW^[S!CAD4
_PaP[[T[NbTaeTaCAD4CAD4
_PaP[[T[NbTaeTaNX]bcP]RTb!CAD4
_PaP[[T[NbTaeTabNcPaVTc 50;B4
_PaP[[T[NcWaTPSbN_TaNR_d!CAD4
%a^fbbT[TRcTS
BHB/4G31 -bb bWPeVbZTf b`[! Zb^ Zb^
bcPacX]V! R^_XTb^UPeVbZTf b`[
BHB/4G31 -bTcTRW^^]
BHB/4G31 -/`dTdTSNb`[
BHB/4G31 -R^[b`[NcTgcU^aP% cad]R
BHB/4G31 -B4;42CbXSb`[NXSb`[NTgTRNXSb`[NcTgc
!Ua^\eb`[N\^]Xc^a
"F74A4bcPcdb,@D4D43
#^aSTaQh"
$
B83B@;N83B@;N4G42N83B@;NC4GC
#(# $R`!WQ a"&ca %&&&! 'bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfP
fWTaTR^[ -
$# $R`!WQ a"&ca %&&&!!
$! $R`!WQ a"&ca %&&&!!
! $R`!WQ a"&ca %&&&!!!
% $R`!WQ a"&ca %&&&!!"
! $R`!WQ a"&ca %&&&!!#
"(& $R`!WQ a"&ca %&&&!!$
!$ $R`!WQ a"&ca %&&&!!%
%(" $R`!WQ a"&ca %&&&!!&
$# $R`!WQ a"&ca %&&&!!'
### $R`!WQ a"&ca %&&&!!(
&#" $R`!WQ a"&ca %&&&!"
" $R`!WQ a"&ca %&&&!"
$ $R`!WQ a"&ca %&&&!"!
$" $R`!WQ a"&ca %&&&!""
#(!04%2 -!+).' 0#"3
%#! $R`!WQ a"&ca %&&&!"#
"#' $R`!WQ a"&ca %&&&!"$
! $R`!WQ a"&ca %&&&!"%
!$ $R`!WQ a"&ca %&&&!"&
(a^fbbT[TRcTS
BHB/4G31 -/RWTRZN_g
BHB/4G31 -bT[TRcUa^\E?GN?A>24BBNBHBBC0CfWTaTbcPcXbcXR[XZT8]DbT*
BC0C8BC82E0;D4
BTaeTab8]DbT'
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT'
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT'
BHB/4G31 -bTcTRW^^UU
BHB/4G31 -/b]P__TaPbW,bXSTeT]cfPXcNR[PbbPbW ,_[b`[N^QYTRcNXSb`[NXS$ P[[
BP\_[X]VfXcWX]cTaeP[$bTR^]Sb cX\Tb
BTbbX^]B]P__Tae" QhCP]T[?^STa/4!B=Wcc_) cTRW T!b] R^\
0RcXeTkB83k4E4=CkF08CN2;0BB
k" kaTb\Va)_``dTdTSkBRWTSd[Ta
k"#'kaTb\Va)_``dTdTSkBRWTSd[Ta
k! kaTb\Va)_``dTdTSkBRWTSd[Ta
k! kaTb\Va)_``dTdTSkBRWTSd[Ta
k&#"kaTb\Va)_``dTdTSkBRWTSd[Ta
k!$ kaTb\Va)_``dTdTSkBRWTSd[Ta
k%("kaTb\Va)_``dTdTSkBRWTSd[Ta
k%#!kaTb\Va)_``dTdTSkBRWTSd[Ta
k$kaTb\Va)_``dTdTSkBRWTSd[Ta
k$# kaTb\Va)_``dTdTSkBRWTSd[Ta
#(!04%2 -!+).' 0#"3
0RcXeTk?;B@;N>194kB@;N83
! $!kk $R`!WQ a"&ca
&kk
4]S^U0B7b]P_ T]S,! ! % ()!$)#"bTR^]Sb,$bP\_[TbNcPZT],#!
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4HERE ARE SEVERAL THINGS WORTH MENTIONING IN THIS LISTING 4O SET UP THE DESIRED CONDITIONS WE TURNED
ON !UTO $/0 WHICH ALSO ENABLES 0ARALLEL 1UEUING AND THEN SET THE ?0A0;;4;NB4AE4ANC0A64C PARAMETER
TO A VERY LOW NUMBER IN ORDER TO TRIGGER QUEUING MORE EASILY 7E THEN USED A SHELL SCRIPT bb bW TO
FIRE OFF COPIES OF THE PeVbZTf b`[ SCRIPT IN RAPID SUCCESSION 1UERYING EB@;N<>=8C>A SHOWED THAT THE
STATEMENTS WERE INDEED QUEUING 4HIS IS AN IMPORTANT POINT !LL STATEMENTS USING PARALLEL QUERY WILL
SHOW UP IN THE EB@;N<>=8C>A VIEW )F THEY HAVE A STATUS OF @D4D43 THEY ARE NOT ACTUALLY EXECUTING BUT ARE
INSTEAD WAITING UNTIL ENOUGH PARALLEL SLAVES BECOME AVAILABLE 7E ALSO QUERIED E?GN?A>24BBNBHBBC0C A
FEW TIMES TO SEE HOW MANY PARALLEL SLAVES WERE BEING USED &INALLY WE RAN 4ANEL 0ODERS SNAPPER SCRIPT
TO SEE WHAT EVENT THE QUEUED STATEMENTS WERE WAITING ON !S YOU CAN SEE IT WAS THE aTb\Va)_``dTdTS
WAIT EVENT
■ .OTE 4HERE IS ONE OTHER THING YOU SHOULD BE AWARE OF WITH REGARD TO THE WAIT EVENTS 4HERE IS A WAIT EVENT
CHANGE THAT RELATES TO PARALLEL QUEUING 4HIS EXAMPLE WAS CREATED USING /RACLE $ATABASE VERSION )F YOU
ARE USING YOU WILL SEE A DIFFERENT SET OF WAIT EVENTS THERE ARE TWO 4HE FIRST IS ?G@dTdX]V)bcPcT\T]c
`dTdT 4HIS IS THE EVENT THAT A STATEMENT WAITS ON WHEN IT IS NEXT TO RUN 4HE OTHER IS T]`)9G B@;bcPcT\T]c
`dTdT 4HIS EVENT IS WHAT A STATEMENT WAITS ON WHEN THERE ARE OTHER STATEMENTS AHEAD OF IT IN THE QUEUE 4HIS
SCHEME SEEMS QUITE UNWIELDY WHICH IS PROBABLY WHY IT WAS CHANGED IN THE LATER RELEASE
#ONTROLLING 0ARALLEL 1UEUING
4HERE ARE SEVERAL MECHANISMS IN PLACE FOR CONTROLLING HOW THE 0ARALLEL 3TATEMENT 1UEUING FEATURE
BEHAVES 4HE BASIC APPROACH IS TO USE A FIRST
IN FIRST
OUT QUEUING MECHANISM "UT THERE ARE WAYS TO
PRIORITIZE WORK WITHIN THE QUEUING FRAMEWORK )TS ALSO POSSIBLE TO COMPLETELY BY
PASS THE QUEUING
MECHANISM VIA A HINT !ND CONVERSELY IT IS POSSIBLE TO ENABLE QUEUING FOR A STATEMENT VIA A HINT EVEN
WHEN THE PARALLEL 3TATEMENT 1UEUING FEATURE IS NOT ENABLED AT THE DATABASE LEVEL 4HERE ARE ALSO A FEW
PARAMETERS THAT AFFECT THE QUEUING BEHAVIOR !ND FINALLY 2ESOURCE -ANAGER HAS THE CAPABILITY TO AFFECT
HOW STATEMENTS ARE QUEUED
Controlling Queuing with Parameters
4HERE ARE A HANDFUL OF PARAMETERS THAT AFFECT HOW 0ARALLEL 1UEUING BEHAVES 4HE MAIN PARAMETER IS
?0A0;;4;NB4AE4ABNC0A64C WHICH TELLS /RACLE HOW MANY PARALLEL SERVER PROCESSES TO ALLOW BEFORE STARTING
TO HOLD STATEMENTS BACK IN THE QUEUE 4HE DEFAULT VALUE FOR THIS PARAMETER IS CALCULATED AS FOLLOWS
#(!04%2 -!+).' 0#"3
¾ #05?COUNT ¾ PARALLEL?THREADS?PER?CPU ¾ ACTIVE?INSTANCES
3O ON AN %XADATA 8 WITH A DATABASE THAT SPANS 2!# NODES THE DEFAULT VALUE WOULD BE
CALCULATED AS
¾ ¾ ¾
4HIS CALCULATED VALUE IS ALMOST CERTAINLY HIGHER THAN YOU WOULD WANT FOR MOST MIXED WORKLOAD
SYSTEMS AS IT IS GEARED AT COMPLETELY CONSUMING THE AVAILABLE #05 RESOURCES WITH PARALLEL QUERY
PROCESSES !LLOWING LONG RUNNING PARALLEL STATEMENTS TO CONSUME THE SERVER COMPLETELY MEANS THAT
RESPONSE SENSITIVE /,40 TYPE STATEMENTS COULD SUFFER 9OU SHOULD ALSO NOTE THAT IT IS POSSIBLE TO HAVE
MORE SERVER PROCESSES ACTIVE THAN THE PARAMETER ALLOWS 3INCE THE NUMBER OF SLAVES ASSIGNED TO A QUERY
MAY BE TWICE THE $/0 THE TARGET CAN OCCASIONALLY BE EXCEEDED
4HE SECOND PARAMETER THAT DESERVES SOME DISCUSSION IS THE HIDDEN PARAMETER
N?0A0;;4;NBC0C4<4=CN@D4D8=6 WHICH TURNS THE FEATURE ON AND OFF !S ALREADY DISCUSSED IN THE !UTO $/0
SECTION THIS PARAMETER IS SET TO CAD4 WHEN THE ?0A0;;4;N346A44N?>;82H PARAMETER IS SET TO 0DC>
(OWEVER THE HIDDEN PARAMETER CAN ALSO BE SET MANUALLY TO TURN 0ARALLEL 1UEUING OFF AND ON
INDEPENDENTLY
!UTO $/0 CALCULATIONS ARE STILL A LITTLE SCARY 3O ITS NICE THAT THERE IS A WAY TO TURN ON THE 0ARALLEL
1UEUING FEATURE WITHOUT ENABLING /RACLE TO TAKE COMPLETE CONTROL OF WHICH STATEMENTS RUN IN PARALLEL
/F COURSE SINCE THIS INVOLVES SETTING A HIDDEN PARAMETER YOU SHOULD NOT DO THIS IN A PRODUCTION
ENVIRONMENT WITHOUT APPROVAL FROM /RACLE SUPPORT .EVERTHELESS HERES ANOTHER QUICK EXAMPLE
SHOWING THAT QUEUING CAN BE TURNED ON WITHOUT ENABLING !UTO $/0 OR )N MEMORY 0ARALLEL %XECUTION
BHB/4G31 -P[cTabhbcT\bTc_PaP[[T[NSTVaTTN_^[XRh,\P]dP[*
BhbcT\P[cTaTS
BHB/4G31 -P[cTacPQ[TZb^ bZTf_PaP[[T[STVaTT'*
CPQ[TP[cTaTS
BHB/4G31 -/_Pa\b
4]cTaeP[dTU^a_PaP\TcTa)_PaP[[T[
4]cTaeP[dTU^aXbbTc)
4]cTaeP[dTU^abW^fNWXSST])
=0<4E0;D48B3450D;
_PaP[[T[NPSP_cXeTN\d[cXNdbTa50;B4CAD4
_PaP[[T[NPdc^\PcXRNcd]X]V50;B4CAD4
_PaP[[T[NSTVaTTN[X\Xc'50;B4
_PaP[[T[NSTVaTTN_^[XRh<0=D0;50;B4
_PaP[[T[NTgTRdcX^]N\TbbPVTNbXiT %"'#CAD4
_PaP[[T[NU^aRTN[^RP[CAD450;B4
_PaP[[T[NX]bcP]RTNVa^d_CAD4
_PaP[[T[NX^NRP_NT]PQ[TS50;B4CAD4
_PaP[[T[N\PgNbTaeTab"! CAD4
_PaP[[T[N\X]N_TaRT]c CAD4
_PaP[[T[N\X]NbTaeTab %50;B4
_PaP[[T[N\X]NcX\TNcWaTbW^[S!CAD4
_PaP[[T[NbTaeTaCAD4CAD4
_PaP[[T[NbTaeTaNX]bcP]RTb!CAD4
#(!04%2 -!+).' 0#"3
_PaP[[T[NbTaeTabNcPaVTc 50;B4
_PaP[[T[NcWaTPSbN_TaNR_d!CAD4
%a^fbbT[TRcTS
BHB/4G31 -bb bWPeVbZTf b`[! Zb^ Zb^
bcPacX]V! R^_XTb^UPeVbZTf b`[
BHB/4G31 -/`dTdTSNb`[
]^a^fbbT[TRcTS
BHB/4G31 -/`dTdTSNb`[
]^a^fbbT[TRcTS
BHB/4G31 -/RWTRZN_g
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT#'
a^fbT[TRcTS
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT$%
a^fbT[TRcTS
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT%#
a^fbT[TRcTS
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT#'
a^fbT[TRcTS
#(!04%2 -!+).' 0#"3
BHB/4G31 -
BC0C8BC82E0;D4
BTaeTab8]DbT$%
a^fbT[TRcTS
BHB/4G31 -`dTdTSNb`[
]^a^fbbT[TRcTS
BHB/4G31 - ]^`dTdX]V[^czb^U_PaP[[T[b[PeTbad]]X]V
BHB/4G31 -
BHB/4G31 -P[cTabhbcT\bTcN_PaP[[T[NbcPcT\T]cN`dTdX]V,cadT* T]PQ[T@dTdX]V
BhbcT\P[cTaTS
BHB/4G31 -bb bWPeVbZTf b`[! Zb^ Zb^
bcPacX]V! R^_XTb^UPeVbZTf b`[
BHB/4G31 -/`dTdTSNb`[
B83B@;N83B@;N4G42N83B@;NC4GC
' $R`!WQ a"&ca %&&&"%"bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfP
fWTaTR^[ -
! % %&&&"%#
## %&&&"%$
!&" %&&&"%%
!"# %&&&"%&
% %&&&"%'
%&&&"%(
!"" %&&&"&
" %&&&"&
"' %&&&"&!
& %&&&"&"
!(" %&&&"&#
"& %&&&"&$
'" %&&&"&%
!$ %&&&"&&
%% %&&&"&'
!" %&&&"&(
($ %&&&"'
'a^fbbT[TRcTS
BHB/4G31 - ]^ffTWPeTR^]ca^[^UfWXRWbcPcT\T]cbPaT_PaP[[T[XiTSP]SfTWPeT`dTdX]V
#(!04%2 -!+).' 0#"3
4HIS LISTING SHOWS THAT YOU CAN HAVE 0ARALLEL 3TATEMENT 1UEUING WITHOUT TURNING ON !UTO $/0 /F
COURSE ITS USING A HIDDEN PARAMETER SO YOU PROBABLY DONT WANT TO DO THIS ON A PRODUCTION SYSTEM
WITHOUT DISCUSSING IT WITH /RACLE 3UPPORT FIRST
Controlling Queuing with Hints
4HERE ARE TWO HINTS THAT CAN BE USED TO CONTROL 0ARALLEL 3TATEMENT 1UEUING AT THE STATEMENT LEVEL /NE
HINT =>NBC0C4<4=CN@D4D8=6 ALLOWS THE QUEUING PROCESS TO BE COMPLETELY BYPASSED EVEN IF THE FEATURE IS
TURNED ON AT THE INSTANCE LEVEL 4HE OTHER HINT BC0C4<4=CN@D4D8=6 TURNS ON THE QUEUING MECHANISM
EVEN IF THE FEATURE IS NOT ENABLED AT THE INSTANCE LEVEL 4HE BC0C4<4=CN@D4D8=6 HINT PROVIDES A
DOCUMENTED AVENUE FOR USING THE QUEUING FEATURE WITHOUT ENABLING !UTO $/0 (ERE IS AN EXAMPLE
SHOWING HOW TO BYPASS THE QUEUING BY USING THE =>NBC0C4<4=CN@D4D8=6 HINT
BHB/4G31 -bTcTRW^^]
BHB/4G31 -/cTbcN_g`NWX]c
BHB/4G31 - cWXbbW^d[SRPdbT`dTdTX]VQdccWTWX]cTSbcPcT\T]cb
BHB/4G31 - bW^d[SYd\_cWT`dTdT
BHB/4G31 -P[cTabhbcT\bTcN_PaP[[T[NbcPcT\T]cN`dTdX]V,cadT*
BhbcT\P[cTaTS
BHB/4G31 -P[cTabhbcT\bTc_PaP[[T[NbTaeTabNcPaVTc,"!*
BhbcT\P[cTaTS
BHB/4G31 -bb bWcTbcN_g`NWX]c b`[ Zb^ Zb^
bcPacX]V R^_XTb^UcTbcN_g`NWX]c b`[
BHB/4G31 -/`dTdTSNb`[
B83B@;N83B@;N4G42N83B@;NC4GC
#'#QVR\\Rhhhe_V( %&&&!"(bT[TRc _PaP[[T[P % PeV_ZNR^[Ua^\Zb^ bZTf"P
fWTaTR^[ -
&&#QVR\\Rhhhe_V( %&&&!#
$'#QVR\\Rhhhe_V( %&&&!#
((QVR\\Rhhhe_V( %&&&!#!
%&(QVR\\Rhhhe_V( %&&&!#"
"( QVR\\Rhhhe_V( %&&&!##
% QVR\\Rhhhe_V( %&&&!#$
&a^fbbT[TRcTS
BHB/4G31 -bb bWcTbcN_g`NWX]c! b`[!Zb^ Zb^
bcPacX]V!R^_XTb^UcTbcN_g`NWX]c! b`[
#(!04%2 -!+).' 0#"3
BHB/4G31 -/`dTdTSNb`[
B83B@;N83B@;N4G42N83B@;NC4GC
&&#QVR\\Rhhhe_V( %&&&!# bT[TRc _PaP[[T[P % PeV_ZNR^[Ua^\Zb^ bZTf"P
fWTaTR^[ -
$'#QVR\\Rhhhe_V( %&&&!#
((QVR\\Rhhhe_V( %&&&!#!
%&(QVR\\Rhhhe_V( %&&&!#"
"( QVR\\Rhhhe_V( %&&&!##
% QVR\\Rhhhe_V( %&&&!#$
%a^fbbT[TRcTS
BHB/4G31 -/Pb
B83B@;N83278;3?;0=N70B7NE0;D44G42B0E6N4C8<4B@;NC4GC
#$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
##'$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
'$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
&$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
##%$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
"$#$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
!$&$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
!$"$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
% $e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
$%$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
%$$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
(%($e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
! $e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
!(&$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
"("$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
"(#$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
#'%$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
#''$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
$'$$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
$'%$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
&&'$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
'%'$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
'%($e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
'&!$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
(%$$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
(%&$e%VacefZQ'"% # #$' & !! '(bT[TRc =>NBC0C4<4=CN@D4D8=6
!$$QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
&&#QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
#'#QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
%&(QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
((QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
"( QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
% QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
#(!04%2 -!+).' 0#"3
$'#QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
$&QVR\\Rhhhe_V( # #$' & #! # ( bT[TRc _PaP[[T[P %
"$a^fbbT[TRcTS
4HIS LISTING SHOWS THAT STATEMENTS USING THE =>NBC0C4<4=CN@D4D8=6 HINT COMPLETELY BYPASS THE
QUEUING MECHANISM 4HE STEPS WE EXECUTED WERE TO ENABLE QUEUING FIRE OFF COPIES OF A SELECT
STATEMENT THAT WAS HINTED TO USE A $/0 OF 4HESE STATEMENTS WERE QUEUED AS SHOWN BY THE
`dTdTSNb`[ b`[ SCRIPT 7E THEN FIRED OFF COPIES OF A STATEMENT USING THE =>NBC0C4<4=CN@D4D8=6 HINT
4HE `dTdTSNb`[ b`[ SCRIPT SHOWED THAT THE NEW STATEMENTS WERE NOT AT THE BOTTOM OF THE QUEUE )N FACT
THEY WERE NOT QUEUED AT ALL 4HEY WERE RUN IMMEDIATELY 4HE Pb b`[ SCRIPT SHOWS ACTIVE SESSIONS AND AS
YOU CAN SEE A SIDE EFFECT OF BYPASSING THE QUEUING IS THAT THE NUMBER OF ACTIVE SLAVES CAN END UP BEING
CONSIDERABLY HIGHER THAN THE SETTING OF THE ?0A0;;4;NB4AE4ABNC0A64C PARAMETER
Controlling Queuing with Resource Manager
/RACLES $ATABASE 2ESOURCE -ANAGER $"2- PROVIDES ADDITIONAL CAPABILITY TO CONTROL 0ARALLEL
3TATEMENT 1UEUING 7HILE A THOROUGH DISCUSSION OF $"2- IS BEYOND THE SCOPE OF THIS CHAPTER WE WILL
COVER SOME SPECIFIC FEATURES RELATED TO 0ARALLEL 1UERY #HAPTER COVERS $"2- IN MORE DETAIL
7ITHOUT $"2- THE PARALLEL STATEMENT QUEUE BEHAVES STRICTLY AS A FIRST IN FIRST OUT &)&/ QUEUE
$"2- PROVIDES SEVERAL DIRECTIVE ATTRIBUTES THAT CAN BE USED TO PROVIDE ADDITIONAL CONTROL ON A CONSUMER
GROUP BASIS -ANY OF THESE CONTROLS WERE INTRODUCED IN VERSION 4ABLE CONTAINS A LIST OF
ADDITIONAL CAPABILITIES PROVIDED BY $"2-
Table 6-3. DBRM Parallel Statement Queuing Controls
Control Description
3PECIFY A 4IMEOUT 4HE ?0A0;;4;N@D4D4NC8<4>DC DIRECTIVE ATTRIBUTE CAN BE USED TO SET A
MAXIMUM QUEUE TIME FOR A CONSUMER GROUP 4HE TIME LIMIT IS SET IN
SECONDS AND ONCE IT HAS EXPIRED THE STATEMENT WILL TERMINATE WITH
AN ERROR /2!
.OTE THAT THIS DIRECTIVE DID NOT BECOME
AVAILABLE UNTIL VERSION OF THE DATABASE
3PECIFY -AXIMUM $/0 4HE ?0A0;;4;N346A44N;8<8CN? DIRECTIVE ATTRIBUTE SETS A MAXIMUM
NUMBER OF PARALLEL SLAVES THAT MAY BE ASSIGNED TO AN INDIVIDUAL
STATEMENT 4HIS IS EQUIVALENT TO THE ?0A0;;4;N346A44N;8<8C DATABASE
PARAMETER BUT IS USED TO SET LIMITS FOR DIFFERENT SETS OF USERS BASED ON
CONSUMER GROUPS
-ANAGE /RDER OF $EQUEUING 4HE <6<CN? <6<CN?! x <6<CN?' DIRECTIVE ATTRIBUTES CAN BE USED TO
ALTER THE NORMAL &)&/ PROCESSING 4HIS ATTRIBUTE ALLOWS PRIORITIZATION
OF DEQUEUING %ACH OF THE EIGHT ATTRIBUTES ESSENTIALLY PROVIDES A
DISTINCT DEQUEUING PRIORITY LEVEL !LL STATEMENTS WITH AN <6<CN?
ATTRIBUTE WILL BE DEQUEUED PRIOR TO ANY STATEMENT WITH <6<CN?! )N
ADDITION TO THE DEQUEUING PRIORITY A PROBABILITY NUMBER CAN BE
ASSIGNED TO REGULATE DEQUEUING OF STATEMENTS WITHIN THE SAME LEVEL
#(!04%2 -!+).' 0#"3
Control Description
,IMIT 0ERCENTAGE OF 0ARALLEL 3LAVES 4HE ?0A0;;4;NC0A64CN?4A24=C064 DIRECTIVE ATTRIBUTE CAN BE USED TO
LIMIT A CONSUMER GROUP TO A PERCENTAGE OF THE PARALLEL SLAVES
AVAILABLE TO THE SYSTEM 3O IF A SYSTEM ALLOWED SLAVES TO BE ACTIVE
BEFORE STARTING TO QUEUE AND ?0A0;;4;NC0A64CN?4A24=C064 WAS SET TO
THE CONSUMER GROUP WOULD ONLY BE ABLE TO CONSUME SLAVES
.OTE THAT THIS DIRECTIVE DID NOT BECOME AVAILABLE UNTIL VERSION
OF THE DATABASE
1UEUE -ULTIPLE 31,S AS A 3ET 4HE 1468=NB@;N1;>2: AND 4=3NB@;N1;>2: PROCEDURES IN
$"-3?2%3/52#%?-!.!'%2 WORK WITH 0ARALLEL 3TATEMENT
1UEUING BY TREATING INDIVIDUAL STATEMENTS AS IF THEY HAD BEEN
SUBMITTED AT THE SAME TIME 4HE MECHANISM REQUIRES SURROUNDING
INDEPENDENT 31, STATEMENTS WITH CALLS TO THE 1468= AND 4=3
PROCEDURES .OTE THAT THIS PROCEDURE DID NOT BECOME AVAILABLE UNTIL
VERSION OF THE DATABASE
4HE $"2- DIRECTIVES CAN BE QUITE INVOLVED (ERE IS AN EXAMPLE FROM THE Oracle® Database VLDB
and Partitioning Guide 11g Release 2 (11.2) THAT SHOWS FOUR DIRECTIVES MAKING USE OF THE 0ARALLEL
3TATEMENT 1UEUING FEATURES
31<BNA4B>DA24N<0=064A 2A40C4N?;0=N38A42C8E4
A4?>ACBN?;0=BHBN6A>D?3XaTRcXeTU^abhbPRcXeXch
\V\cN_ ,- *
31<BNA4B>DA24N<0=064A 2A40C4N?;0=N38A42C8E4
A4?>ACBN?;0=>C74AN6A>D?B3XaTRcXeTU^abW^ac ad]]X]V`dTaXTb
\V\cN_!,-&
_PaP[[T[NSTVaTTN[X\XcN_ ,-#
bfXcRWNcX\T,-%
bfXcRWNTbcX\PcT,-CAD4
bfXcRWNU^aNRP[[,-CAD4
bfXcRWNVa^d_,-<438D<NB@;N6A>D?*
31<BNA4B>DA24N<0=064A 2A40C4N?;0=N38A42C8E4
A4?>ACBN?;0=<438D<NB@;N6A>D?3XaTRcXeTU^a\TSXd\ ad]]X]V`dTaXTb
\V\cN_!,-!
_PaP[[T[NcPaVTcN_TaRT]cPVT,-'
bfXcRWNcX\T,-(
bfXcRWNTbcX\PcT,-CAD4
bfXcRWNU^aNRP[[,-CAD4
bfXcRWNVa^d_,-;>=6NB@;N6A>D?*
31<BNA4B>DA24N<0=064A 2A40C4N?;0=N38A42C8E4
A4?>ACBN?;0=;>=6NB@;N6A>D?3XaTRcXeTU^a[^]V ad]]X]V`dTaXTb
\V\cN_!,-
_PaP[[T[NcPaVTcN_TaRT]cPVT,-$
_PaP[[T[N`dTdTNcX\T^dc,- ## *
4HIS EXAMPLE BEARS SOME EXPLANATION
#(!04%2 -!+).' 0#"3
4HE FIRST DIRECTIVE IS FOR bhbPRcXeXch 3O SESSIONS IN THE CONSUMER GROUP
BHBN6A>D? WILL HAVE PERCENT OF THE HIGHEST DEQUEUING PRIORITY <6<CN?
4HE SECOND DIRECTIVE IS FOR bW^ac ad]]X]V`dTaXTb 4HIS APPLIES TO ALL SESSIONS
IN THE >C74AN6A>D?B CONSUMER GROUP THE DEFAULT CONSUMER GROUP 4HIS
DIRECTIVE SPECIFIES THAT /RACLE SHOULD EVALUATE EACH 31, STATEMENT AND IF THE
ESTIMATED EXECUTION TIME IS LONGER THAN SECONDS BF8C27NC8<4 THE SESSIONS
SHOULD BE SWITCHED TO THE <438D<NB@;N6A>D? CONSUMER GROUP "Y THE WAY THE
SESSION SWITCHES BACK TO /4(%2?'2/503 WHEN THE STATEMENT IS COMPLETE
BECAUSE BF8C27N5>AN20;;,-CAD4 4HE DEQUEUING PRIORITY <6<CN?! IS SET TO
PERCENT MEANING THAT STATEMENTS SHOULD BE DEQUEUED AFTER ANY <6<CN?
STATEMENTS WITH A PROBABILITY OF PERCENT WHEN COMPARED WITH OTHER <6<CN?!
STATEMENTS &INALLY THE MAXIMUM $/0 IS SET TO BY THE
?0A0;;4;N346A44N;8<8CN? ATTRIBUTE
4HE THIRD DIRECTIVE IS FOR \TSXd\ ad]]X]V`dTaXTb )T APPLIES TO SESSIONS IN THE
<438D<NB@;N6A>D? CONSUMER GROUP 4HIS DIRECTIVE ALSO INCLUDES A SWITCH
CAUSING SESSIONS TO BE MOVED TO THE ;>=6NB@;N6A>D? IF /RACLE ESTIMATES A 31,
STATEMENT WILL TAKE LONGER THAN MINUTES !DDITIONALLY THIS DIRECTIVE SETS THE
DEQUEUING PRIORITY TO BE PERCENT OF THE SECOND PRIORITY GROUP <6<CN?!
4HIS DIRECTIVE ALSO PUTS A LIMIT ON THE PERCENTAGE OF PARALLEL SLAVES THAT MAY BE
USED ?0A0;;4;NC0A64CN?4A24=C064 )N THIS CASE PERCENT OF THE TOTAL SLAVES
ALLOWED ON THE SYSTEM IS THE MAXIMUM THAT MAY BE USED BY SESSIONS IN THIS
CONSUMER GROUP
4HE LAST DIRECTIVE IS FOR [^]V ad]]X]V`dTaXTb )T APPLIES TO SESSIONS IN THE
;>=6NB@;N6A>D? CONSUMER GROUP 3ESSIONS IN THIS GROUP HAVE A VERY LOW
DEQUEUING PRIORITY AS COMPARED TO THE OTHERS <6<CN?!, )T ALSO LIMITS THE
PERCENTAGE OF PARALLEL SLAVES TO PERCENT &INALLY SINCE STATEMENTS IN THIS
GROUP MAY BE QUEUED FOR A LONG TIME THE ?0A0;;4;N@D4D4NC8<4>DC ATTRIBUTE
HAS BEEN SET TO SECONDS 3O IF A STATEMENT IS QUEUED FOR HOURS IT WILL
FAIL WITH A TIMEOUT ERROR
0ARALLEL 3TATEMENT 1UEUING 7RAP 5P
4HIS SECTION HAS BEEN CONSIDERABLY MORE DETAILED THAN THE COVERAGE OF THE OTHER TWO NEW PARALLEL
FEATURES IN G2 4HATS BECAUSE THIS FEATURE IS A CRITICAL COMPONENT IN ALLOWING %XADATA TO HANDLE
MIXED WORKLOADS EFFECTIVELY )T ENABLES US TO EFFECTIVELY HANDLE A MIXTURE OF STATEMENTS THAT ARE SENSITIVE
TO BOTH THROUGHPUT AND RESPONSE TIME 7ITHOUT THIS FEATURE IT WOULD BE VERY DIFFICULT TO PROVIDE
ADEQUATE RESOURCES WITHOUT SEVERELY COMPROMISING ONE OR THE OTHER
In-Memory Parallel Execution
4HE THIRD NEW FEATURE OF G2 RELATED TO PARALLEL OPERATIONS IS PROBABLY THE LEAST WELL KNOWN 0RIOR TO
G2 QUERIES THAT WERE PARALLELIZED TOTALLY IGNORED THE BUFFER CACHE /RACLE ASSUMED THAT PARALLEL
QUERIES WOULD ONLY BE DONE ON VERY LARGE TABLES THAT WOULD PROBABLY NEVER HAVE A LARGE PERCENTAGE OF
THEIR BLOCKS IN THE BUFFER CACHE 4HIS ASSUMPTION LED TO THE CONCLUSION THAT IT WOULD BE FASTER TO JUST READ
THE DATA FROM DISK )N ADDITION FLOODING THE BUFFER CACHE WITH A LARGE NUMBER OF BLOCKS FROM A FULL TABLE
SCAN WAS NOT DESIRABLE AND SO /RACLE DEVELOPED A MECHANISM CALLED DIRECT PATH READS WHICH BYPASSED
#(!04%2 -!+).' 0#"3
THE NORMAL CACHING MECHANISM IN FAVOR OF READING BLOCKS DIRECTLY IN TO THE USERS 0ROGRAM 'LOBAL !REA
0'!
4HE )N
-EMORY 0ARALLEL %XECUTION FEATURE TAKES A DIFFERENT APPROACH )T ATTEMPTS TO MAKE USE OF THE
BUFFER CACHE FOR PARALLEL QUERIES 4HE FEATURE IS CLUSTER
AWARE AND IS DESIGNED TO SPREAD THE DATA ACROSS
THE CLUSTER NODES THAT IS THE 2!# DATABASE SERVERS 4HE DATA BLOCKS ARE ALSO AFFINITIZED TO A SINGLE NODE
REDUCING THE AMOUNT OF COMMUNICATION AND DATA TRANSFERS BETWEEN THE NODES 4HE GOAL OF COURSE IS TO
SPEED UP THE PARALLEL QUERY BY ELIMINATING DISK )/ 4HIS CAN BE A VIABLE TECHNIQUE BECAUSE MANY
SYSTEMS NOW HAVE VERY LARGE AMOUNTS OF MEMORY WHICH OF COURSE CAN PROVIDE A SIGNIFICANT SPEED
ADVANTAGE OVER DISK OPERATIONS 4HERE ARE SOME DOWNSIDES TO THIS APPROACH THOUGH 4HE BIGGEST
DISADVANTAGE WITH REGARD TO %XADATA IS THAT ALL THE 3MART 3CAN OPTIMIZATIONS ARE DISABLED BY THIS FEATURE
■ +EVIN 3AYS )T IS TRUE THAT )N
-EMORY 0ARALLEL 1UERY )/ IS BUFFERED IN THE 3'! AND REMAINS CACHEDTHUS 3MART
3CAN IS NOT INVOLVED 4HIS WOULD ONLY BE A DISADVANTAGE IF A QUERY EXECUTES FASTER WITH 3MART 3CAN THAN WITH )N
-EMORY 0ARALLEL 1UERY #OMPARING THE PROCESSING DYNAMIC OF A QUERY SERVICED BY 3MART 3CAN AS OPPOSED TO )N
-EMORY 0ARALLEL 1UERY IS A VERY COMPLEX TOPICESPECIALLY WHEN COMPRESSED DATA IS INVOLVED )N THE SAME WAY
3MART 3CAN IS ABLE TO FILTER VAST AMOUNTS OF DEEPLY
COMPRESSED DATA SPECIFICALLY (## SO IS )N
-EMORY 0ARALLEL
1UERY 4HAT IS (## DATA IS FILTERED IN ITS COMPRESSED FORM REGARDLESS OF WHO IS DOING THE FILTRATION )N THE CASE OF
3MART 3CAN HOWEVER THE DATA THAT SIFTS THROUGH FILTRATION MUST BE DECOMPRESSION AND SENT TO THE DATABASE GRID
VIA I$" FOR DATA
INTENSIVE PROCESSING SUCH AS JOIN SORT OR AGGREGATE 7ITH )N
-EMORY 0ARALLEL 1UERY THERE IS NO
DATA FLOW OVER I$" &URTHERMORE A TABLE THAT IS ACCESSED FREQUENTLY BY 3MART 3CAN REQUIRES THE SAME REPETITIOUS
PHYSICAL )/ FILTRATION DECOMPRESSION WITH THE EXCEPTION OF RESULTS
CACHED QUERIES EACH TIME IT IS ACCESSED
7ITH )N
-EMORY 0ARALLEL 1UERY THE )/ AND I$" EFFORT IS ELIMINATED AND AS WE KNOW MEMORY IS FASTER THAN BOTH
HARD DISK AND FLASH 7HAT CUSTOMERS MUST ANALYZE IS WHETHER THERE ARE SUFFICIENT PROCESSOR CYCLES AVAILABLE IN THE
DATABASE GRID TO PERFORM THE REPETITIOUS FILTRATION AND DECOMPRESSION OF POPULAR TABLES 2EMEMBER IN THE CASE OF
3MART 3CAN THAT EFFORT IS OFFLOADED TO THE CELLS 4O CAP THIS TRAIN OF THOUGHT )LL SIMPLY SAY THERE IS NO ABSOLUTE
ANSWER IN THE COMPARISON BETWEEN )N
-EMORY 0ARALLEL 1UERY AND 3MART 3CAN 4HE VARIABLES NEED TO BE PLUGGED IN
BY THE CUSTOMER USING THE KNOWLEDGE GAINED FROM READING THIS BOOK /RACLE DOCUMENTATION AND PERHAPS BLOGS
SMILEY FACE
.OTE THAT WE HAVENT ACTUALLY SEEN )N MEMORY 0ARALLEL 1UERY IN THE WILD ON %XADATA WHICH IS
PROBABLY GOOD SINCE MANY OF THE OPTIMIZATIONS BUILT INTO %XADATA RELY ON /FFLOADING WHICH DEPENDS ON
DIRECT READS /F COURSE DIRECT READS WILL NOT BE DONE IF BLOCKS ARE BEING ACCESSED IN MEMORY ON THE
DATABASE SERVERS /N MOST PLATFORMS MEMORY ACCESS WOULD BE MUCH FASTER THAN DIRECT READS FROM DISK
"UT WITH %XADATA ELIMINATING THE DISK )/ ALSO ELIMINATES A LARGE NUMBER OF #05S THAT COULD BE APPLIED
TO FILTERING AND OTHER OPERATIONS 4HIS MEANS THAT ACCESSING THE BUFFER CACHE FOR PARALLEL EXECUTION COULD
ACTUALLY BE LESS EFFICIENT THAN ALLOWING THE STORAGE SERVERS TO PARTICIPATE IN THE EXECUTION OF THE 31,
STATEMENT
! LITTLE DEMONSTRATION IS PROBABLY IN ORDER AT THIS POINT )TS WORTH NOTING THAT GETTING THIS FEATURE TO
KICK IN AT ALL TAKES QUITE A BIT OF EFFORT (ERE ARE THE BASIC STEPS WE HAD TO TAKE IN ORDER TO GET IT TO WORK
&IRST WE HAD TO FIND A QUERY THAT THE OPTIMIZER ESTIMATED WOULD RUN FOR LONGER THAN THE NUMBER OF
SECONDS SPECIFIED BY THE ?0A0;;4;N<8=NC8<4NC7A4B7>;3 PARAMETER ASSUMING THE STATEMENT WASNT
#(!04%2 -!+).' 0#"3
PARALLELIZED 4HE DEFAULT FOR THIS PARAMETER IS 0DC> MEANING SECONDS 7E SET THIS PARAMETER TO
SECONDS TO MAKE IT EASIER TO TRIGGER THE )N MEMORY 0ARALLEL BEHAVIOR 4HIS QUERY ALSO HAD TO BE ON A TABLE
THAT WOULD ALMOST COMPLETELY FIT IN THE AGGREGATE BUFFER CACHE PROVIDED BY THE COMBINATION OF ALL THE
PARTICIPATING 2!# INSTANCES 4O SIMPLIFY THINGS WE LIMITED THE PROCESSING TO A SINGLE INSTANCE BY SETTING
THE ?0A0;;4;N5>A24N;>20; PARAMETER TO CAD4 /F COURSE WE HAD TO SET ?0A0;;4;N346A44N?>;82H TO 0DC> TO
ENABLE THE FEATURE 7E ALSO SET VERY LOW VALUES FOR ?0A0;;4;NB4AE4ABNC0A64C AND ?0A0;;4;N346A44N;8<8C
(ERE ARE THE PARAMETER SETTINGS AND PERTINENT INFORMATION ABOUT THE QUERY WE USED TO TEST THE FEATURE
BHB/4G31 -/_Pa\b
4]cTaeP[dTU^a_PaP\TcTa)_PaP[[T[
4]cTaeP[dTU^aXbbTc)
4]cTaeP[dTU^abW^fNWXSST])
=0<4E0;D48B3450D;
UPbcNbcPacN_PaP[[T[Na^[[QPRZ;>FCAD4
_PaP[[T[NPSP_cXeTN\d[cXNdbTa50;B4CAD4
_PaP[[T[NPdc^\PcXRNcd]X]V50;B4CAD4
_PaP[[T[NSTVaTTN[X\Xc'50;B4
_PaP[[T[NSTVaTTN_^[XRh0DC>50;B4
_PaP[[T[NTgTRdcX^]N\TbbPVTNbXiT %"'#CAD4
_PaP[[T[NU^aRTN[^RP[CAD450;B4
_PaP[[T[NX]bcP]RTNVa^d_CAD4
_PaP[[T[NX^NRP_NT]PQ[TS50;B4CAD4
_PaP[[T[N\PgNbTaeTab"! CAD4
_PaP[[T[N\X]N_TaRT]c CAD4
_PaP[[T[N\X]NbTaeTab %50;B4
_PaP[[T[N\X]NcX\TNcWaTbW^[S"50;B4
_PaP[[T[NbTaeTaCAD4CAD4
_PaP[[T[NbTaeTaNX]bcP]RTb!CAD4
_PaP[[T[NbTaeTabNcPaVTc'50;B4
_PaP[[T[NcWaTPSbN_TaNR_d!CAD4
aTR^eTahN_PaP[[T[Xb\ CAD4
'a^fbbT[TRcTS
BHB/4G31 -/_^^[N\T\
0A40<46B
UaTT\T\^ah$%! $
UXgTSNbVP!
[^VNQdUUTa& $
bWPaTS_^^[$$& #
[PaVT_^^[#'
QdUUTaNRPRWT"' '
bd\#('$ %
%a^fbbT[TRcTS
BHB/4G31 -/cPQ[TNbXiT
#(!04%2 -!+).' 0#"3
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
4]cTaeP[dTU^ach_T)
>F=4AB46<4=CN=0<4CH?4C>C0;B8I4N<46B
:B>B:4FC01;4!$%
bd\!$%
a^fbT[TRcTS
BHB/4G31 -/SQPNcPQ[Tb
4]cTaeP[dTU^a^f]Ta):B>
4]cTaeP[dTU^acPQ[TN]P\T)B:4F
>F=4AC01;4N=0<4BC0CDB;0BCN0=0;=D<NA>FB1;>2:B346A44
:B>B:4FE0;83 ' B4? "! # %!!(#
a^fbT[TRcTS
3O THE BUFFER CACHE ON THIS INSTANCE IS ABOUT ' THE TABLE IS ABOUT ' 4HE QUERY WE USED IS
SIMPLE AND WILL NOT BENEFIT FROM STORAGE INDEXES AS VIRTUALLY ALL THE RECORDS SATISFY THE SINGLE F74A4
CLAUSE
BHB/4G31 -RPcPeVbZTf b`[
bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfPfWTaTR^[ -
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTf*
2>D=C
"! #
a^fbT[TRcTS
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTffWTaTR^[ - *
2>D=C
"!
a^fbT[TRcTS
BHB/4G31 -bT[TRc"! "! #Ua^\SdP[*
"! "! #
(((((('&$
a^fbT[TRcTS
#(!04%2 -!+).' 0#"3
.OW WELL SHOW SOME STATISTICS BEFORE AND AFTER EXECUTING THE STATEMENT 9OULL BE ABLE TO SEE THAT
WHILE THE PHYSICAL READS STATISTIC DOES NOT INCREASE THE LOGICAL READS STATISTIC SHOWS A LARGE INCREASE .OTE
THAT WEVE DONE A QUICK CALCULATION OF THE TOTAL NUMBER OF LOGICAL READS BY SELECTING FROM DUAL AND THAT
THE NUMBER OF LOGICAL READS IS VERY CLOSE TO THE NUMBER OF BLOCKS THAT THE TABLE OCCUPIES AS
REPORTED BY 310NC01;4B
BHB/4G31 -/U[dbWN_^^[
BhbcT\P[cTaTS
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)aTPSb
=0<4E0;D4
R^[SaTRhR[TaTPSb
SPcPQ[^RZbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
[^QaTPSb
_WhbXRP[aTPSb#& % &'
_WhbXRP[aTPSbRPRWT"
_WhbXRP[aTPSbRPRWT_aTUTcRW
_WhbXRP[aTPSbSXaTRc#& % #'
_WhbXRP[aTPSbSXaTRc[^Q
_WhbXRP[aTPSbSXaTRccT\_^aPahcPQ[Tb_PRT
_WhbXRP[aTPSbU^aU[PbWQPRZ]Tf
_WhbXRP[aTPSb_aTUTcRWfPa\d_
_WhbXRP[aTPSbaTcahR^aad_c
aTR^eTahPaaPhaTPSb
bTbbX^][^VXRP[aTPSb& $!&&(
caP]bPRcX^]cPQ[TbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
$a^fbbT[TRcTS
BHB/4G31 -bTccX\X]V^]
BHB/4G31 -/PeVbZTf
0E6?:N2>;
% ("&$ '
a^fbT[TRcTS
4[P_bTS) ) ) &!
BHB/4G31 -bTccX\X]V^UU
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)aTPSb
=0<4E0;D4
R^[SaTRhR[TaTPSb
SPcPQ[^RZbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
[^QaTPSb
#(!04%2 -!+).' 0#"3
_WhbXRP[aTPSb#& % &'
_WhbXRP[aTPSbRPRWT"
_WhbXRP[aTPSbRPRWT_aTUTcRW
_WhbXRP[aTPSbSXaTRc#& % #'
_WhbXRP[aTPSbSXaTRc[^Q
_WhbXRP[aTPSbSXaTRccT\_^aPahcPQ[Tb_PRT
_WhbXRP[aTPSbU^aU[PbWQPRZ]Tf
_WhbXRP[aTPSb_aTUTcRWfPa\d_
_WhbXRP[aTPSbaTcahR^aad_c
aTR^eTahPaaPhaTPSb
bTbbX^][^VXRP[aTPSb&" $#(#
caP]bPRcX^]cPQ[TbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
$a^fbbT[TRcTS
BHB/4G31 -bT[TRc&" $#(# & $!&&(Ua^\SdP[*
&" $#(# & $!&&(
%!& $
a^fbT[TRcTS
BHB/4G31 -/S_[P]
4]cTaeP[dTU^ab`[NXS) $R`!WQ a"&ca
4]cTaeP[dTU^aRWX[SN]^)
?;0=NC01;4N>DC?DC
B@;N83 $R`!WQ a"&caRWX[S]d\QTa
bT[TRcPeV_ZNR^[Ua^\Zb^ bZTfPfWTaTR^[ -
?[P]WPbWeP[dT)$&'"%% &
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckkkk% "( kk
k kB>AC066A460C4kk k kkk
k!k?G2>>A38=0C>Akkkkkk
k"k?GB4=3@2A0=3><k)C@ k k kkk
k#kB>AC066A460C4kk k kkk
k$k?G1;>2:8C4A0C>Akk"!<k""$<k% "( k ) ) #k
k%kC01;40224BBBC>A0645D;;kB:4Fk"!<k""$<k% "( k ) ) #k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
% bc^aPVT)I-,)I0=3)I+,)I0=32>; -
UX[cTa2>; -
#(!04%2 -!+).' 0#"3
=^cT
Pdc^\PcXR3>?)2^\_dcTS3TVaTT^U?PaP[[T[Xb\Xb'
BHB/4G31 -/Ubg!
4]cTaeP[dTU^ab`[NcTgc)bZTf
4]cTaeP[dTU^ab`[NXS)
B@;N830E6N4C8<4?G>55;>038>NB0E43B@;NC4GC
$R`!WQ a"&ca #&'=^ bT[TRcPeV_ZNR^[Ua^\
Zb^ bZTfPfWTaTR^[ -
a^fbbT[TRcTS
3O AS YOU CAN SEE THE STATEMENT WAS EXECUTED IN PARALLEL WITH EIGHT SLAVES THE 08 COLUMN IN THE
Ubg b`[ SCRIPT OUTPUT .OTE ALSO THAT THE Ubg b`[ SCRIPT REPORTS AN ESTIMATED VALUE FOR THE 0E6N4C8<4
COLUMN WHEN PARALLEL SLAVES ARE USED 4HIS OCCURS BECAUSE eb`[ REPORTS ELAPSED TIME AS THE SUM OF THE
ALL THE ELAPSED TIMES OF THE SLAVE PROCESSES $IVIDING THIS NUMBER BY THE NUMBER OF SLAVES GIVES AN
ESTIMATE BUT THAT WILL NOT BE TOTALLY ACCURATE AS SLAVES CAN VARY GREATLY IN THEIR ELAPSED TIMES )F YOU LOOK
BACK IN THE LISTING YOULL SEE THAT THE ACTUAL ELAPSED TIME OF PeVbZTf b`[ WAS SECONDS INSTEAD OF THE
ESTIMATED AVERAGE PROVIDED BY THE Ubg b`[ SCRIPT SECONDS .EVERTHELESS IT IS A USEFUL METRIC WHEN
COMPARING BETWEEN TWO STATEMENTS
,ETS NOW COMPARE OUR )N
MEMORY 0ARALLEL %XECUTION TO HOW OUR SYSTEM BEHAVES WHEN )N
MEMORY
0ARALLEL %XECUTION IS NOT IN PLAY 7E CAN DISABLE THIS FEATURE IN A COUPLE OF WAYS 4HE DOCUMENTED WAY IS
TO SET THE ?0A0;;4;N346A44N?>;82H PARAMETER TO <0=D0; (OWEVER THIS ALSO DISABLES !UTO $/0 AND
0ARALLEL 3TATEMENT 1UEUING 4HE OTHER WAY IS TO SET THE HIDDEN PARAMETER
N?0A0;;4;N2;DBC4AN20274N?>;82H TO 030?C8E4
BHB/4G31 -P[cTabhbcT\bTcN_PaP[[T[NR[dbcTaNRPRWTN_^[XRh,PSP_cXeT*
BhbcT\P[cTaTS
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)aTPSb
=0<4E0;D4
R^[SaTRhR[TaTPSb
SPcPQ[^RZbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
[^QaTPSb
_WhbXRP[aTPSb$ (" %
_WhbXRP[aTPSbRPRWT"
_WhbXRP[aTPSbRPRWT_aTUTcRW
_WhbXRP[aTPSbSXaTRc$ (" "
_WhbXRP[aTPSbSXaTRc[^Q
_WhbXRP[aTPSbSXaTRccT\_^aPahcPQ[Tb_PRT
_WhbXRP[aTPSbU^aU[PbWQPRZ]Tf
_WhbXRP[aTPSb_aTUTcRWfPa\d_
_WhbXRP[aTPSbaTcahR^aad_c
aTR^eTahPaaPhaTPSb
bTbbX^][^VXRP[aTPSb' ""'"$
#(!04%2 -!+).' 0#"3
caP]bPRcX^]cPQ[TbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
$a^fbbT[TRcTS
BHB/4G31 -bTccX\X]V^]
BHB/4G31 -/PeVbZTf
0E6?:N2>;
% ("&$ '
a^fbT[TRcTS
4[P_bTS) ) ) &(
BHB/4G31 -bTccX\X]V^UU
BHB/4G31 -/\hbcPcb
4]cTaeP[dTU^a]P\T)aTPSb
=0<4E0;D4
R^[SaTRhR[TaTPSb
SPcPQ[^RZbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
[^QaTPSb
_WhbXRP[aTPSb$"$$"$$
_WhbXRP[aTPSbRPRWT"
_WhbXRP[aTPSbRPRWT_aTUTcRW
_WhbXRP[aTPSbSXaTRc$"$$"!#
_WhbXRP[aTPSbSXaTRc[^Q
_WhbXRP[aTPSbSXaTRccT\_^aPahcPQ[Tb_PRT
_WhbXRP[aTPSbU^aU[PbWQPRZ]Tf
_WhbXRP[aTPSb_aTUTcRWfPa\d_
_WhbXRP[aTPSbaTcahR^aad_c
aTR^eTahPaaPhaTPSb
bTbbX^][^VXRP[aTPSb'!(%"#!
caP]bPRcX^]cPQ[TbR^]bXbcT]caTPSb d]S^aTR^aSbP__[XTS
$a^fbbT[TRcTS
BHB/4G31 -bT[TRc$"$$"$$ $ (" % Ua^\SdP[*
$"$$"$$ $ (" %
%!!(#
a^fbT[TRcTS
BHB/4G31 -/Ubg!
4]cTaeP[dTU^ab`[NcTgc)bZTf
4]cTaeP[dTU^ab`[NXS)
#(!04%2 -!+).' 0#"3
B@;N830E6N4C8<4?G>55;>038>NB0E43B@;NC4GC
$R`!WQ a"&ca #&'=^ bT[TRcPeV_ZNR^[Ua^\
Zb^ bZTfPfWTaTR^[ -
$R`!WQ a"&ca $''HTb$( '!bT[TRcPeV_ZNR^[Ua^\
Zb^ bZTfPfWTaTR^[ -
!a^fbbT[TRcTS
.OTICE THAT WITH )N
MEMORY 0ARALLEL %XECUTION DISABLED THE STATISTICS SHOW THAT THE NUMBER OF
PHYSICAL READS INCREASED BY ROUGHLY THE NUMBER OF BLOCKS IN THE TABLE AS EXPECTED .OTICE ALSO THAT THE
Ubg b`[ SCRIPT SHOWS THAT THERE IS NOW A NEW CURSOR IN THE SHARED POOL THAT WAS EXECUTED WITH EIGHT
PARALLEL SLAVES BUT THAT WAS OFFLOADED TO THE STORAGE TIER 4HIS IS AN IMPORTANT POINT )N
MEMORY 0ARALLEL
%XECUTION DISABLES THE OPTIMIZATIONS THAT %XADATA PROVIDES VIA 3MART 3CANS 4HAT SHOULD BE OBVIOUS
SINCE THE DISK )/ WAS ELIMINATED BY THE )N
MEMORY 0ARALLEL %XECUTION FEATURE BUT IT IS THE MAIN REASON
THAT WE BELIEVE THIS FEATURE WILL NOT BE AS USEFUL ON THE %XADATA PLATFORM AS ON OTHER PLATFORMS
!S A FINAL DEMONSTRATION WELL SHOW A VERY SELECTIVE QUERY THAT BENEFITS MORE FROM /FFLOADING THAN
IT DOES FROM )N
MEMORY 0ARALLEL %XECUTION
BHB/4G31 -/_Pa\b
4]cTaeP[dTU^a_PaP\TcTa)_PaPRPRWT
4]cTaeP[dTU^aXbbTc)
4]cTaeP[dTU^abW^fNWXSST])H
=0<4E0;D48B3450D;
N_PaP[[T[NR[dbcTaNRPRWTN_Rc' CAD4
N_PaP[[T[NR[dbcTaNRPRWTN_^[XRh030?C8E4CAD4
!a^fbbT[TRcTS
BHB/4G31 - 8] \T\^ah?PaPT[[T[4gTRdcX^]Xb^UU
BHB/4G31 -
BHB/4G31 -/U[dbWN_^^[
BhbcT\P[cTaTS
BHB/4G31 -bTccX\X]V^]
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTffWTaTR^[ Xb]d[[*
2>D=C
#
a^fbT[TRcTS
4[P_bTS) ) ) !%
BHB/4G31 -P[cTabhbcT\bTcN_PaP[[T[NR[dbcTaNRPRWTN_^[XRh,RPRWTS*
BhbcT\P[cTaTS
4[P_bTS) ) )
#(!04%2 -!+).' 0#"3
BHB/4G31 -bT[TRcR^d]cUa^\Zb^ bZTffWTaTR^[ Xb]d[[*
2>D=C
#
a^fbT[TRcTS
4[P_bTS) ) ) '#
BHB/4G31 -bTccX\X]V^UU
BHB/4G31 -/Ubg!
4]cTaeP[dTU^ab`[NcTgc)bZTf
4]cTaeP[dTU^ab`[NXS)
B@;N830E6N4C8<4?G>55;>038>NB0E43B@;NC4GC
#ai VYdfa(Yh` #'HTb bT[TRcR^d]cUa^\Zb^ bZTf
fWTaTR^[ Xb]d[[
#ai VYdfa(Yh` &!'=^ bT[TRcR^d]cUa^\Zb^ bZTf
fWTaTR^[ Xb]d[[
!a^fbbT[TRcTS
)N THIS CASE /FFLOADING WAS MORE EFFECTIVE FASTER THAN USING )N MEMORY 0ARALLEL %XECUTION
PRIMARILY BECAUSE 3TORAGE )NDEXES WERE ABLE TO ELIMINATE AN EXTREMELY LARGE PERCENTAGE OF THE DISK )/
3O THE COMPARISON CAME DOWN TO READING ABOUT 'IGABYTES FROM THE BUFFER CACHE ON THE DATABASE
SERVER VS READING THE 3TORAGE )NDEXES AND A HANDFUL OF BLOCKS OFF DISK 'RANTED THIS IS A VERY SIMPLE
CONTRIVED EXAMPLE BUT IT ILLUSTRATES THE POINT THAT THERE MAY BE CASES WHERE 3MART 3CAN IS FASTER THAN )N
MEMORY 0ARALLEL 1UERY
Summary
0ARALLEL EXECUTION OF STATEMENTS IS IMPORTANT FOR MAXIMIZING THROUGHPUT ON THE %XADATA PLATFORM
/RACLE DATABASE G 2ELEASE INCLUDES SEVERAL NEW FEATURES THAT MAKE THE PARALLEL EXECUTION A MORE
CONTROLLABLE FEATURE 4HIS IS ESPECIALLY IMPORTANT WHEN USING THE PLATFORM WITH MIXED WORKLOADS 4HE
NEW !UTO $/0 FEATURE IS DESIGNED TO ALLOW INTELLIGENT DECISIONS ABOUT $/0 TO BE MADE AUTOMATICALLY
BASED ON INDIVIDUAL STATEMENTS )N
MEMORY 0ARALLEL %XECUTION MAY NOT BE AS USEFUL ON %XADATA
PLATFORMS AS IT IS ON NON
%XADATA PLATFORMS BECAUSE IT DISABLES THE OPTIMIZATIONS THAT COME ALONG WITH
3MART 3CANS /F THE THREE NEW FEATURES 0ARALLEL 3TATEMENT 1UEUING IS THE MOST USEFUL AS IT ALLOWS A
MIXTURE OF THROUGHPUT
ORIENTED WORK TO CO
EXIST WITH RESPONSE
TIMEnSENSITIVE WORK )NTEGRATION WITH
/RACLE 2ESOURCE -ANAGER FURTHER ENHANCES THE FEATURE BY PROVIDING A GREAT DEAL OF ADDITIONAL CONTROL
OVER THE QUEUING
CHAPTER 7
Resource Management
)F RESOURCES WERE UNLIMITED THERE WOULD BE NO NEED TO MANAGE THEM 7E SEE THIS IN ALL ASPECTS OF OUR
DAILY LIVES )F YOURS WAS THE ONLY CAR ON THE ROAD TRAFFIC SIGNALS WOULDNT BE NECESSARY )F YOU WERE THE
ONLY CUSTOMER AT THE BANK THERE WOULD BE NO NEED FOR THE WINDING ROPES THAT FORM ORDERLY LINES "UT AS
WE ALL KNOW THIS IS RARELY THE CASE )T IS THE SAME FOR DATABASE SERVERS 7HEN THE LOAD ON THE SYSTEM IS
LIGHT THERE IS VERY LITTLE NEED FOR RESOURCE MANAGEMENT 0ROCESSES COMPLETE IN A FAIRLY CONSISTENT PERIOD
OF TIME "UT WHEN THE SYSTEM GETS BUSY AND RESOURCES BECOME SCARCE WE CAN FIND OURSELVES WITH AN
ANGRY MOB ON OUR HANDS
&OR A NUMBER OF YEARS NOW /RACLES $ATABASE 2ESOURCE -ANAGER $"2- HAS PROVIDED AN EFFECTIVE
WAY TO MANAGE THE ALLOCATION OF CRITICAL RESOURCES WITHIN THE DATABASE 7ITHOUT $"2- ALL DATABASE
CONNECTIONS ARE TREATED WITH EQUAL PRIORITY AND THEY ARE SERVICED IN A SORT OF ROUND
ROBIN FASHION BY THE
OPERATING SYSTEM SCHEDULER 7HEN THE SYSTEM IS UNDER HEAVY LOAD ALL SESSIONS ARE IMPACTED EQUALLY
,OW
PRIORITY APPLICATIONS RECEIVE JUST AS HIGH A PRIORITY AS BUSINESS
CRITICAL APPLICATIONS )T IS NOT
UNCOMMON TO SEE A FEW POORLY WRITTEN AD
HOC QUERIES DEGRADE THE PERFORMANCE OF MISSION
CRITICAL
APPLICATIONS )F YOUVE BEEN A $"! LONG ENOUGH ESPECIALLY IN DATA WAREHOUSE ENVIRONMENTS YOURE
PROBABLY FAMILIAR WITH THE 5NIX aT]XRT COMMAND )T IS A ROOT
LEVEL COMMAND THAT ALLOWS YOU TO
INFLUENCE THE #05 PRIORITY OF A PROCESS AT THE OPERATING
SYSTEM LEVEL ! NUMBER OF YEARS AGO WE WORKED
IN A $"! GROUP SUPPORTING A PARTICULARLY HEAVILY LOADED DATA WAREHOUSE 4HE aT]XRT COMMAND WAS USED
FREQUENTLY TO THROTTLE BACK #05 PRIORITY FOR DATABASE SESSIONS THAT WERE DOMINATING THE SYSTEM 4HERE
WERE A COUPLE OF OBVIOUS PROBLEMS WITH THIS APPROACH &IRST OF ALL IT WAS A PRIVILEGED COMMAND AVAILABLE
ONLY TO THE ROOT USER AND SYSTEM ADMINISTRATORS WERE RELUCTANT TO GRANT $"!S ACCESS TO IT 4HE SECOND
PROBLEM WAS THAT AUTOMATING IT TO MANAGE #05 RESOURCES WAS DIFFICULT AT BEST /RACLES $ATABASE
2ESOURCE -ANAGER IS A MUCH MORE ELEGANT SOLUTION TO THE PROBLEM )T ALLOWS $"!S TO ADDRESS RESOURCE
ALLOCATION WITHIN THE DOMAIN OF THE DATABASE ITSELF )T IS A WELL
ORGANIZED FRAMEWORK THAT IS AUTOMATED
BY DESIGN )T ENSURES THAT CRITICAL SYSTEM RESOURCES LIKE #05 AND )/ WILL BE AVAILABLE TO YOUR IMPORTANT
APPLICATIONS WHENEVER THEY ARE NEEDED EVEN WHEN THE SYSTEM IS UNDER A HEAVY WORKLOAD 4HIS IS DONE
BY CREATING RESOURCE ALLOCATION SCHEMES THAT DEFINE PRIORITIES BASED ON THE NEEDS OF THE BUSINESS
!NOTHER CASE FOR RESOURCE MANAGEMENT IS CONSOLIDATION )T WAS INEVITABLE THAT A PLATFORM WITH THE
PERFORMANCE CAPACITY AND SCALABILITY OF %XADATA WOULD BE VIEWED BY MANY AS AN IDEAL CONSOLIDATION
PLATFORM "UT CONSOLIDATING DATABASES IS A CHALLENGE MAINLY BECAUSE OF THE DIFFICULTY OF MANAGING
RESOURCES ACROSS DATABASES 7EVE WORKED WITH A NUMBER OF CLIENTS WHO HAVE USED %XADATA TO
CONSOLIDATE MULTIPLE DATABASE SERVERS ONTO THE %XADATA PLATFORM /NE SUCH CLIENT CONSOLIDATED
DATABASES FROM DATABASE SERVERS ONTO TWO %XADATA FULL RACKS .EEDLESS TO SAY WITHOUT /RACLES
RESOURCE MANAGEMENT CAPABILITIES IT WOULD BE EXTREMELY DIFFICULT IF NOT IMPOSSIBLE TO BALANCE SYSTEM
RESOURCES AMONG SO MANY DATABASE ENVIRONMENTS 5NTIL RECENTLY THERE WAS REALLY NO WAY TO PRIORITIZE
)/ ACROSS DATABASES 7ITH %XADATA 6 /RACLE INTRODUCED )/ 2ESOURCE -ANAGER )/2- AND FOR THE
FIRST TIME WE CAN VIRTUALLY GUARANTEE )/ SERVICE LEVELS WITHIN AND AMONG DATABASES
3O WHETHER YOU ARE CONSOLIDATING MULTIPLE DATABASES ONTO YOUR %XADATA PLATFORM OR HANDLING
RESOURCE INTENSIVE APPLICATIONS WITHIN A SINGLE DATABASE EFFECTIVE RESOURCE MANAGEMENT WILL PLAY AN
#(!04%2 2%3/52#% -!.!'%-%.4
IMPORTANT ROLE IN YOUR SUCCESS )N THIS CHAPTER WE WILL REVIEW AND DEMONSTRATE THE MAIN COMPONENTS OF
$"2- AND HOW IT IS USED TO MANAGE AND ALLOCATE #05 RESOURCES EFFECTIVELY WITHIN A DATABASE 7ELL ALSO
TAKE A LOOK AT instance caging AND HOW IT CAN BE USED TO SET LIMITS ON THE AMOUNT OF #05 A DATABASE MAY
USE IN ORDER TO PROVIDE PREDICTABLE SERVICE LEVELS FOR MULTI
TENANT DATABASE ENVIRONMENTS )N THE LAST HALF
OF THE CHAPTER WELL COVER THE NEW %XADATA
SPECIFIC FEATURE CALLED )/ 2ESOURCE -ANAGER WHICH ALLOCATES
AND PRIORITIZES DISK )/ AT THE STORAGE CELL
/RACLE 2ESOURCE -ANAGER FOR ALL ITS BENEFITS HAS BEEN AN INFREQUENTLY USED FEATURE OF THE DATABASE
4HIS IS LARGELY DUE TO ITS COMPLEXITY AND A GENERAL LACK OF UNDERSTANDING AMONG THE $"! COMMUNITY
"EYOND INTRODUCING THE NEW %XADATA
SPECIFIC FEATURES OF 2ESOURCE -ANAGER OUR GOALS IN PRESENTING THIS
MATERIAL ARE TWOFOLD &IRST WE WANT TO PROVIDE ENOUGH DETAIL TO DEMYSTIFY /RACLE 2ESOURCE -ANAGER
WITHOUT OVERWHELMING THE READER 3ECOND WE INTEND TO DEMONSTRATE HOW TO BUILD A FULLY FUNCTIONAL
RESOURCE MANAGEMENT MODEL 4HESE GOALS PRESENT A UNIQUE CHALLENGE 0ROVIDE TOO LITTLE INFORMATION
AND THE READER WILL ONLY BE ABLE TO SET UP VERY SIMPLE CONFIGURATIONS 4OO MUCH DETAIL AND WE RISK
CONVOLUTING THE TOPIC AND LOSING THE AUDIENCE 4HE MOST DIFFICULT PART OF WRITING THIS CHAPTER HAS BEEN
STRIKING A BALANCE BETWEEN THE TWO !S YOU READ THROUGH THE EXAMPLES YOU WILL NOTICE THAT WE USED
MULTI
LEVEL RESOURCE PLANS 4HIS IS NOT TO SUGGEST THAT IN ORDER TO BE EFFECTIVE YOU MUST USE COMPLEX
MULTI
LEVEL PLANS )N FACT SIMPLE SINGLE
LEVEL RESOURCE PLANS WILL SOLVE A VAST MAJORITY OF THE RESOURCE
MANAGEMENT PROBLEMS WE SEE IN THE REAL WORLD -OREOVER MULTI
LEVEL RESOURCE PLANS CAN BE DIFFICULT TO
DESIGN AND TEST )N THIS CHAPTER WE DEMONSTRATE MULTI
LEVEL PLANS BECAUSE IT IS IMPORTANT TO UNDERSTAND
HOW THEY WORK "UT IF YOU ARE CONSIDERING USING /RACLE 2ESOURCE -ANAGER THE BEST APPROACH IS TO KEEP
IT SIMPLE AND ADD FEATURES ONLY AS THEY ARE NEEDED
Database Resource Manager
$ATABASE 2ESOURCE -ANAGER $"2- HAS BEEN AROUND FOR A NUMBER OF YEARS AND IS BASICALLY GEARED
TOWARD MANAGING #05 RESOURCES AND )/ INDIRECTLY AT THE DATABASE TIER %XADATA 6 INTRODUCED A NEW
FEATURE CALLED )/ 2ESOURCE -ANAGER )/2- WHICH AS YOU MIGHT EXPECT IS GEARED TOWARD MANAGING
AND PRIORITIZING )/ AT THE STORAGE CELL 7HEN DATABASES ON %XADATA REQUEST )/ FROM THE STORAGE CELLS
THEY SEND ADDITIONAL INFORMATION ALONG WITH THE REQUEST THAT IDENTIFIES THE DATABASE MAKING THE REQUEST
AS WELL AS THE CONSUMER GROUP MAKING THE REQUEST 4HE SOFTWARE ON THE STORAGE CELLS #ELLSERV OR RT[[bae
KNOWS ABOUT THE PRIORITIES YOU ESTABLISH INSIDE THE DATABASE $"2- ANDOR AT THE 3TORAGE #ELL )/2-
AND IT MANAGES HOW )/ IS SCHEDULED $"2- AND )/2- ARE TIGHTLY KNIT TOGETHER SO IT IS IMPORTANT TO
HAVE A SOLID UNDERSTANDING OF $"2- BEFORE )/2- IS GOING TO MAKE ANY SENSE TO YOU .OW $ATABASE
2ESOURCE -ANAGER IS A LENGTHY TOPIC AND COULD EASILY JUSTIFY A BOOK ALL BY ITSELF 3O HERE WELL FOCUS ON
THE BASIC CONSTRUCTS THAT WE WILL NEED FOR CONSTRUCTING AN EFFECTIVE )/2- 2ESOURCE 0LAN )F YOU ALREADY
HAVE EXPERIENCE WITH $"2- YOU MAY BE ABLE TO SKIP OVER THIS TOPIC BUT BE AWARE THAT THE EXAMPLES IN
THIS SECTION WILL BE USED AS WE DISCUSS )/2- IN THE LAST HALF OF THE CHAPTER
"EFORE WE BEGIN LETS REVIEW THE TERMINOLOGY THAT WILL BE USED IN THIS TOPIC 4ABLE DESCRIBES THE
VARIOUS COMPONENTS OF $ATABASE 2ESOURCE -ANAGER 7ELL DISCUSS THESE IN MORE DETAIL AS WE GO ALONG
#(!04%2 2%3/52#% -!.!'%-%.4
Table 7-1. Resource Manager Components
Name Description
2ESOURCE CONSUMER GROUP 4HESE ARE THE VARIOUS NAMES BY WHICH YOU MAY SEE RESOURCE
CONSUMER GROUPS REFERRED 2ESOURCE -ANAGER ALLOCATES
#ONSUMER GROUP RESOURCES TO CONSUMER GROUPS RATHER THAN USER SESSIONS !
CONSUMER GROUP IS A SET OF DATABASE SESSIONS THAT MAY BE
GROUPED TOGETHER BASED ON THEIR PRIORITY ANDOR RESOURCE
REQUIREMENTS
2ESOURCE PLAN DIRECTIVE 4HESE ARE THE NAMES BY WHICH YOU MAY SEE RESOURCE PLAN
DIRECTIVES REFERRED 2ESOURCE ALLOCATIONS ARE NOT ASSIGNED
0LAN DIRECTIVE DIRECTLY TO CONSUMER GROUPS 4HEY ARE DEFINED IN A RESOURCE
$IRECTIVE PLAN DIRECTIVE ! CONSUMER GROUP IS THEN ASSIGNED TO THE PLAN
DIRECTIVE SO THAT RESOURCE ALLOCATIONS MAY BE ENFORCED
2ESOURCE PLAN 2ESOURCE PLANS ARE SOMETIMES REFERRED TO SIMPLY AS hPLANSv OR
hTHE PLANv 0LAN DIRECTIVES ARE GROUPED TOGETHER TO CREATE A
0LAN RESOURCE PLAN THUS DEFINING THE OVERARCHING ALLOCATION OF
RESOURCES TO ALL SESSIONS WITHIN THE DATABASE
!S SHOWN IN THE TABLE $"2- CONSISTS OF THREE MAIN COMPONENTS RESOURCE CONSUMER GROUPS
CONSUMER GROUPS RESOURCE PLAN DIRECTIVES PLAN DIRECTIVES AND RESOURCE PLANS
Consumer groups: ! CONSUMER GROUP CAN REPRESENT A SINGLE DATABASE SESSION OR A
GROUP OF SESSIONS 'ENERALLY SPEAKING CONSUMER GROUPS CONSIST OF END USERS OR
APPLICATION GROUPS THAT SHARE A COMMON BUSINESS PRIORITY 'ROUPING SESSIONS
TOGETHER IN THIS MANNER ALLOW RESOURCES TO BE ASSIGNED AND MANAGED COLLECTIVELY
&OR EXAMPLE IN A MIXED
WORKLOAD DATABASE ENVIRONMENT CONSUMER GROUP
ASSIGNMENTS ALLOW YOU TO COLLECTIVELY ASSIGN MORE #05 AND )/ RESOURCES TO YOUR
HIGH
PRIORITY BUSINESS APPLICATIONS WHILE REDUCING THE RESOURCES ALLOCATED TO LOW
PRIORITY APPLICATIONS
Plan directives: 0LAN DIRECTIVES ARE WHERE YOU DEFINE YOUR RESOURCE ALLOCATIONS !
PLAN DIRECTIVE IS CREATED USING THE 2A40C4N?;0=N38A42C8E4 PROCEDURE )N ADDITION TO
DEFINING THE RESOURCE ALLOCATION ITSELF PERCENTAGE OF #05 FOR EXAMPLE THE
PROCEDURE ALSO REQUIRES YOU TO NAME ONE RESOURCE PLAN AND ONE CONSUMER GROUP
)N THIS WAY A PLAN DIRECTIVE hLINKS v OR ASSIGNS A CONSUMER GROUP TO A RESOURCE PLAN
/NLY ONE CONSUMER GROUP MAY BE ASSIGNED TO A DIRECTIVE AND RESOURCE PLANS
TYPICALLY CONSIST OF MULTIPLE DIRECTIVES 9OU CAN SAY THAT A CONSUMER GROUP IS
ASSIGNED TO A RESOURCE PLAN THROUGH THE CREATION OF A PLAN DIRECTIVE
Resource plan: 4HE RESOURCE PLAN IS THE COLLECTION OF DIRECTIVES THAT DETERMINE HOW
AND WHERE RESOURCES ARE ALLOCATED /NLY ONE PLAN MAY BE ACTIVATED IN THE DATABASE
INSTANCE AT ANY GIVEN TIME 4HE RESOURCE PLAN IS ACTIVATED BY ASSIGNING IT TO THE
A4B>DA24N<0=064AN?;0= INSTANCE PARAMETER &OR EXAMPLE THE FOLLOWING P[cTa
bhbcT\ COMMAND ACTIVATES THE RESOURCE PLAN @\h_[P]
#(!04%2 2%3/52#% -!.!'%-%.4
B@;-P[cTabhbcT\bTcaTb^daRTN\P]PVTaN_[P],z\h_[P]z*
! DATABASES RESOURCE PLAN MAY BE CHANGED AT ANY TIME ALLOWING YOU TO REALLOCATE
RESOURCES AT VARIOUS TIMES OF THE DAY WEEK OR MONTH IN ORDER TO MEET THE VARYING
WORKLOAD REQUIREMENTS OF YOUR BUSINESS 7HEN THE RESOURCE PLAN IS ACTIVATED NO
RESOURCES ARE ALLOCATED TO INDIVIDUAL USER SESSIONS )NSTEAD RESOURCES ARE ALLOCATED
TO THE CONSUMER GROUPS ACCORDING TO THE DIRECTIVES IN THE RESOURCE PLAN
#ONSUMER 'ROUPS
7HEN A RESOURCE PLAN IS ACTIVATED $"2- EXAMINES KEY ATTRIBUTES OF ALL SESSIONS IN THE DATABASE AND
ASSIGNS THEM TO CONSUMER GROUPS 3ESSIONS ARE ASSIGNED TO CONSUMER GROUPS BY MEANS OF MAPPING RULES
THAT YOU DEFINE &OR EXAMPLE A SET OF USER ACCOUNTS CAN BE MAPPED TO A CONSUMER GROUP BASED ON THEIR
USER NAME OR THE MACHINE FROM WHICH THEY ARE LOGGING IN ! USER MAY BELONG TO MANY DIFFERENT
CONSUMER GROUPS AND MAY BE DYNAMICALLY REASSIGNED FROM ONE CONSUMER GROUP TO ANOTHER EVEN IN THE
MIDDLE OF EXECUTING A 31, STATEMENT OR QUERY 3INCE DATABASE RESOURCES ARE ALLOCATED ONLY TO CONSUMER
GROUPS REASSIGNING A SESSION TO ANOTHER GROUP IMMEDIATELY CHANGES ITS RESOURCE ALLOCATION !LL SESSIONS
IN A CONSUMER GROUP SHARE THE RESOURCES OF THAT GROUP &OR EXAMPLE IF THE 0??B CONSUMER GROUP IS
ALLOCATED OF THE TOTAL #05 ON THE SERVER ALL SESSIONS BELONGING TO THAT CONSUMER GROUP WILL EQUALLY
SHARE THE ALLOCATION OF #05 4HERE ARE TWO BUILT
IN CONSUMER GROUPS IN EVERY DATABASE BHBN6A>D?
AND >C74AN6A>D?B 4HESE GROUPS CANNOT BE MODIFIED OR DROPPED
BHBN6A>D?: 4HIS IS THE DEFAULT CONSUMER GROUP FOR THE 393 AND 3934%- USER
ACCOUNTS 4HESE ACCOUNTS MAY BE ASSIGNED TO ANOTHER CONSUMER GROUP USING
MAPPING RULES WELL DISCUSS IN THE NEXT SECTION
>C74AN6A>D?B: 4HIS IS THE DEFAULT CONSUMER GROUP !NY SESSIONS THAT BELONG TO A
CONSUMER GROUP THAT IS NOT DEFINED IN THE CURRENTLY ACTIVE RESOURCE PLAN WILL
AUTOMATICALLY BE MAPPED TO THIS CONSUMER GROUP 4HIS IS /RACLES WAY OF MAKING
SURE ALL USER SESSIONS ARE ASSIGNED TO A CONSUMER GROUP IN THE ACTIVE RESOURCE PLAN
.OTE 7E WOULD HAVE PREFERRED A NAME LIKE /4(%2 FOR THAT LAST CONSUMER GROUP BUT YES IT REALLY IS NAMED
h/4(%2?'2/503 v NOT /4(%2?'2/50 OR /4(%2 7E MUST HAVE BEEN OUT OF TOWN WHEN ,ARRY CALLED TO ASK OUR
OPINION ON THIS ONE
Consumer Group Mapping Rules
!LL USER SESSIONS EXCEPT 393 AND 3934%- ARE MAPPED BY DEFAULT TO THE CONSUMER GROUP >C74AN6A>D?B
4HIS BEHAVIOR CAN BE MODIFIED USING MAPPING RULES SO THAT SESSIONS ARE AUTOMATICALLY REASSIGNED TO
OTHER CONSUMER GROUPS )F THERE IS NO MAPPING RULE FOR A SESSION OR IF THE MAPPING RULES ASSIGN THE
SESSION TO A CONSUMER GROUP THAT IS NOT NAMED IN THE CURRENTLY ACTIVE PLAN THEN IT WILL BE AUTOMATICALLY
ASSIGNED TO THIS BUILT IN CONSUMER GROUP %VERY RESOURCE PLAN MUST HAVE THE >C74AN6A>D?B RESOURCE
GROUP TO HANDLE THIS CONDITION 4HE FOLLOWING EXAMPLE SHOWS HOW A MAPPING RULE IS CREATED 4HIS
MAPPING RULE CALLS FOR THE C?>34A ACCOUNT TO BE AUTOMATICALLY ASSIGNED TO THE A4?>ACB CONSUMER GROUP
WHILE ANYONE LOGGING IN USING THE ?Pha^[[ TgT APPLICATION WILL BE MAPPED TO THE 0??B CONSUMER GROUP
#(!04%2 2%3/52#% -!.!'%-%.4
1468=
31<BNA4B>DA24N<0=064A B4CN2>=BD<4AN6A>D?N<0??8=6
31<BNA4B>DA24N<0=064A >A02;4NDB4AC?>34AA4?>ACB*
31<BNA4B>DA24N<0=064A B4CN2>=BD<4AN6A>D?N<0??8=6
31<BNA4B>DA24N<0=064A 2;84=CN?A>6A0<_Pha^[[ TgT0??B*
4=3*
4HERE ARE TWO TYPES OF SESSION ATTRIBUTES THAT CAN BE USED TO CREATE MAPPING RULES login ATTRIBUTES
AND runtime ATTRIBUTES ,OGIN ATTRIBUTES ARE SET WHEN THE USER LOGS IN AND DO NOT CHANGE DURING THE LIFE OF
THE SESSION 2ESOURCE -ANAGER USES LOGIN ATTRIBUTES TO DETERMINE WHICH CONSUMER GROUP THE SESSION
SHOULD INITIALLY BE ASSIGNED TO 2UNTIME ATTRIBUTES ARE SET AT RUNTIME AND CAN BE CHANGED AT ANY TIME
DURING THE LIFE OF THE SESSION BY THE CLIENT APPLICATION 4ABLE
DESCRIBES THE SESSION ATTRIBUTES
2ESOURCE -ANAGER CAN USE FOR CREATING SESSION
TO
CONSUMER
GROUP MAPPING RULES
Table 7-2. Consumer Group Mapping Rule Attributes
Session Attribute Type Description
4G?;828C NA 4HIS ATTRIBUTE REFERS TO THE EXPLICIT REQUEST BY A USER TO
SWITCH TO ANOTHER CONSUMER GROUP USING ONE OF THE
FOLLOWING STORED PROCEDURES IN THE 31<BNB4BB8>=
PACKAGE
v BF8C27N2DAA4=CN2>=BD<4AN6A>D?
v BF8C27N2>=BD<4AN6A>D?N5>ANB4BB
v BF8C27N2>=BD<4AN6A>D?N5>ANDB4A
)T IS A COMMON PRACTICE TO SET THE PRIORITY OF THIS
MAPPING ATTRIBUTE TO THE HIGHEST LEVEL
>A02;4NDB4A ,OGIN 4HIS IS THE DB4A=0<4 COLUMN FROM ebTbbX^] )T IS THE
USER NAME THE SESSION USED TO AUTHENTICATE TO THE
DATABASE DURING LOGIN
B4AE824N=0<4 ,OGIN 4HIS IS THE DATABASE SERVICE NAME USED TO CONNECT TO THE
DATABASE )T IS THE B4AE824N=0<4 COLUMN IN THE ebTbbX^]
VIEW
2;84=CN>BNDB4A ,OGIN 4HIS IS THE OPERATING SYSTEM USER ACCOUNT OF THE
MACHINE THE USER IS CONNECTING FROM )T IS THE >BDB4A
COLUMN FROM THE ebTbbX^] VIEW
2;84=CN?A>6A0< ,OGIN 4HIS IS THE EXECUTABLE FILE THE END USER IS USING TO
CONNECT TO THE DATABASE FOR EXAMPLE b`[_[dbf TgT
2ESOURCE -ANAGER EVALUATES THIS VALUE WITHOUT
CONSIDERATION OF CASE UPPER OR LOWER
#(!04%2 2%3/52#% -!.!'%-%.4
Session Attribute Type Description
2;84=CN<0278=4 ,OGIN 4HIS IS THE MACHINE FROM WHICH THE USER IS CONNECTING TO
THE DATABASE )T APPEARS IN THE <0278=4 COLUMN OF THE
ebTbbX^] VIEW
<>3D;4N=0<4 2UNTIME 4HIS IS THE MODULE NAME SET BY THE APPLICATION
CONNECTING TO THE DATABASE )T IS STORED IN THE <>3D;4
COLUMN OF THE ebTbbX^] VIEW AND IS SET BY CALLING THE
31<BN0??;820C8>=N8=5> B4CN<>3D;4 PROCEDURE 4HIS IS
AN OPTIONAL SETTING AND SOME APPLICATIONS DO NOT USE IT
<>3D;4N=0<4N02C8>= 2UNTIME 4HIS IS A CONCATENATION OF THE MODULE AND ACTION IN THE
FORM module.action. 4HE APPLICATION SETS THESE
ATTRIBUTES BY CALLING THE FOLLOWING PROCEDURES
v 31<BN0??;820C8>=N8=5> B4CN<>3D;4
v 31<BN0??;820C8>=N8=5> B4CN02C8>=
B4AE824N<>3D;4 2UNTIME 4HIS ATTRIBUTE IS THE CONCATENATION OF THE SERVICE NAME
USED TO CONNECT TO THE DATABASE AND THE <>3D;4
ATTRIBUTE IN THE FORM service.module
B4AE824N<>3D;4N02C8>= 2UNTIME 4HIS ATTRIBUTE IS THE CONCATENATION OF THE SERVICE NAME
MODULE AND ACTION IN THE FORM service.module.action
>A02;4N5D=2C8>= 2UNTIME 4HIS IS A SPECIAL ATTRIBUTE THAT IS MAINTAINED INTERNALLY
BY THE DATABASE )T IS SET WHEN RUNNING 2-!. OR $ATA
0UMP 4HIS ATTRIBUTE CAN BE SET TO 102:D? TO PERFORM A
QPRZd_qPbQPRZd_bTc OR 2>?H TO PERFORM A QPRZd_q
PbR^_h 7HEN $ATA 0UMP IS USED TO LOAD DATA INTO THE
DATABASE THIS ATTRIBUTE IS SET TO 30C0;>03 4HESE
ATTRIBUTES ARE AUTOMATICALLY MAPPED TO BUILT IN
CONSUMER GROUPS SUCH AS 10C27N6A>D? AND %4,?'2/50
4IP B4AE824N<>3D;4 02C8E4NB4BBN?>>;N? AND @D4D48=6N? ARE NOT COMMONLY USED AND MAY BE
DEPRECATED IN FUTURE RELEASES &OR ATTRIBUTES OTHER THAN >A02;4NDB4AAND B4AE824N=0<4 IN 4ABLE YOU CAN
ALSO USE WILDCARDS SUCH AS ? AND FOR SINGLE AND MULTIPLE CHARACTERS RESPECTIVELY
#ONFLICTS CAN OCCUR BETWEEN MAPPING RULES WHEN A USER ACCOUNT MATCHES MORE THAN ONE RULE
/RACLE RESOLVES THESE CONFLICTS BY ALLOWING YOU TO SPECIFY THE RELATIVE PRIORITY OF EACH ATTRIBUTE 4HIS WAY
/RACLE CAN AUTOMATICALLY DETERMINE WHICH RULE ATTRIBUTE SHOULD TAKE PRECEDENCE WHEN SESSION
ATTRIBUTES SATISFY MULTIPLE MAPPING RULES 4HE DEFAULT PRIORITIES FOR THE >A02;4NDB4AAND
2;84=CN0??;820C8>=ATTRIBUTES ARE AND RESPECTIVELY )N THE FOLLOWING EXAMPLE )VE PROMOTED THE
#(!04%2 2%3/52#% -!.!'%-%.4
2;84=CN?A>6A0<TO POSITION AND THE >A02;4NDB4ATO POSITION .OW THE CLIENT APPLICATION ESTABLISHING
A DATABASE CONNECTION WILL TAKE PRECEDENCE OVER THE CLIENTS DB4A=0<4
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa bTcNR^]bd\TaNVa^d_N\P__X]VN_aX
4G?;828C,-
CLIENT_PROGRAM => 2,
ORACLE_USER => 3,
B4AE824N<>3D;4N02C8>=,-#
B4AE824N<>3D;4,-$
<>3D;4N=0<4N02C8>=,-%
<>3D;4N=0<4,-&
B4AE824N=0<4,-'
2;84=CN<0278=4,-(
2;84=CN>BNDB4A,- *
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
&OR EXAMPLE USING THE MAPPING RULES AND PRIORITIES WEVE CREATED SO FAR LETS SAY C?>34A LOGS IN TO THE
DATABASE !CCORDING TO THE >A02;4NDB4A MAPPING RULE THIS USER WOULD ORDINARILY BE ASSIGNED TO THE
A4?>ACB CONSUMER GROUP "UT IF C?>34A LOGS IN FROM THE COMPANYS 0AYROLL APPLICATION HIS SESSION WILL BE
MAPPED TO THE 0??B CONSUMER GROUP 4HIS IS BECAUSE ACCORDING TO THE RULE PRIORITIES THE
2;84=CN0??;820C8>= MAPPING RULE TAKES PRECEDENCE OVER >A02;4NDB4A 4HE FOLLOWING QUERY SHOWS HOW
TWO DIFFERENT SESSIONS FROM THE SAME USER ACCOUNT GET MAPPED ACCORDING TO THE USERNAME AND CLIENT
PROGRAM MAPPING PRIORITIES .OTICE HOW ONLY THE 0AYROLL APPLICATION _Pha^[[ TgT OVERRIDES THE
A4?>ACB MAPPING RULE
BHB)B2A0C27-bT[TRcb dbTa]P\TDbTa
b _a^VaP\?a^VaP\
b aTb^daRTNR^]bd\TaNVa^d_ATb^daRT6a^d_
5A><ebTbbX^]be_a^RTbb_
F74A4b dbTa]P\T8B=>C=D;;
0=3=E;b ^bdbTag+-BHBC4<
0=3b CH?4,102:6A>D=3
0=3_ PSSa,b _PSSa
0=3b dbTa]P\T,C?>34A
>A34A1Hb aTb^daRTNR^]bd\TaNVa^d_b dbTa]P\T*
DbTa?a^VaP\ATb^daRT6a^d_
C?>34A_Pha^[[ TgT0??B
C?>34Ab`[_[db/T]ZSQ ! T]ZXcTR R^\C=BE E"A4?>ACB
C?>34Ac^PS TgTA4?>ACB
C?>34Ab`[_[dbf TgTA4?>ACB
0LAN $IRECTIVES
$"2- ALLOCATES DATABASE RESOURCES TO CONSUMER GROUPS THROUGH plan directives ! PLAN DIRECTIVE
CONSISTS OF ONE CONSUMER GROUP AND ONE OR MORE MANAGEMENT ATTRIBUTES 4HERE IS A ONE
TO
ONE
#(!04%2 2%3/52#% -!.!'%-%.4
RELATIONSHIP BETWEEN A PLAN DIRECTIVE AND A CONSUMER GROUP AND NO TWO DIRECTIVES MAY BE ASSIGNED TO
THE SAME RESOURCE GROUP WITHIN THE SAME PLAN ! PLAN DIRECTIVE IS MADE UP OF A LIST OF MANAGEMENT
ATTRIBUTES IN A key=value FASHION &OR EXAMPLE THE FOLLOWING LISTING SHOWS HOW A SET OF DIRECTIVES MAY BE
DEFINED IN $"2-
31<BNA4B>DA24N<0=064A 2A40C4N?;0=N38A42C8E4
?;0=,-TgP\_[TN_[P]
6A>D?N>ANBD1?;0=,-0??B
2><<4=C,->;C?0__[XRPcX^]BTbbX^]b
<6<CN? ,-&
<0GNDC8;8I0C8>=N;8<8C,-(
<0GN4BCN4G42NC8<4,-"%
*
MGMT_Pn
%XADATAS )/ 2ESOURCE -ANAGER WORKS WITH $"2- THROUGH PLAN DIRECTIVES USING THE #05 MANAGEMENT
ATTRIBUTES <6<CN?n WHERE n MAY BE n AND <0GNDC8;8I0C8>=N;8<8C SO GOING FORWARD THESE ATTRIBUTES
WILL BE THE FOCUS OUR DISCUSSION #05 IS ALLOCATED IN A LEVEL PERCENTAGE MANNER 5SAGE OF THE <6<CN?n
ATTRIBUTE DETERMINES THE RELATIVE PRIORITY IN WHICH #05 IS ALLOCATED TO CONSUMER GROUPS ACROSS THE
VARIOUS LEVELS WHERE IS THE HIGHEST LEVELPRIORITY 4HE PERCENTAGE ASSIGNED TO THE <6<CN?n ATTRIBUTE
DETERMINES HOW AVAILABLE #05 RESOURCES UNALLOCATED PLUS UNUSED ARE ALLOCATED WITHIN A PARTICULAR
LEVEL 7HATEVER #05 IS UNUSED OR UNALLOCATED FROM LEVEL IS ALLOCATED TO LEVEL 5NUSED AND UNALLOCATED
#05 FROM LEVEL IS THEN PASSED TO THE CONSUMER GROUPS ON LEVEL )F THERE ARE TWO CONSUMER GROUPS ON
LEVEL AND ONE OF THEM DOESNT USE ITS ALLOCATION THE UNUSED #05 IS ALWAYS PASSED TO THE NEXT LEVEL IN
THE 0LAN 4HE OTHER CONSUMER GROUP ON LEVEL CANT UTILIZE IT
&IGURE
SHOWS A SIMPLE RESOURCE PLAN AND ILLUSTRATES HOW THIS LEVEL PERCENTAGE METHOD OF
ALLOCATING #05 RESOURCES WORKS
Management Attributes
MGMT_P1 MGMT_P2 MGMT_P3
Consumer Level 1 CPU Level 2 CPU Level 3 CPU
Group Allocation Allocation Allocation
APPS 70%
Resource Directives
REPORTS 50%
MAINTENANCE 50%
OTHER_GROUPS 50%
Figure 7-1. Resource directives
)N &IGURE
THE 0??B GROUP IS ALLOCATED OF TOTAL #05 AVAILABLE TO THE DATABASE 3ESSIONS IN THE
A4?>ACB GROUP ARE THE NEXT HIGHEST PRIORITY AT LEVEL AND WILL BE ALLOCATED HALF OF THE UNALLOCATED #05
FROM LEVEL 3ESSIONS IN THE RESOURCE GROUPS <08=C4=0=24 AND >C74AN6A>D?B EQUALLY SHARE
UNALLOCATED #05 FROM LEVEL 4HIS CAN BE EXPRESSED IN FORMULA FORM AS FOLLOWS
0??B ¾
A4?>ACB n ¾
#(!04%2 2%3/52#% -!.!'%-%.4
<08=C4=0=24 ¾ ¾
>C74AN6A>D?B ¾ ¾
2ESOURCE -ANAGER IS DESIGNED TO MAXIMIZE #05 UTILIZATION 4HIS IS IMPORTANT TO UNDERSTAND BECAUSE IT
MEANS THAT THERE ARE TIMES WHEN CONSUMER GROUPS MAY ACTUALLY EXCEED THEIR ALLOCATION 7HEN #05
RESOURCES ARE LIMITED PLAN DIRECTIVES DEFINE GUARANTEED SERVICE LEVELS FOR CONSUMER GROUPS "UT WHEN
EXTRA #05 IS AVAILABLE PLAN DIRECTIVES ALSO DETERMINE HOW UNUSED #05 RESOURCES ARE ALLOCATED AMONG
CONSUMER GROUPS &OR EXAMPLE IF #05 UTILIZATION IN THE 0??B GROUP FALLS BELOW HALF OF THE UNUSED
#05 IS REDISTRIBUTED TO THE A4?>ACB GROUP ON LEVEL MGMT?P AND HALF IS DISTRIBUTED TO THE
CONSUMER GROUPS ON LEVEL )F THE A4?>ACB GROUP DOES NOT FULLY UTILIZE ITS ALLOCATION OF #05 THE UNUSED
#05 IS ALSO REDISTRIBUTED TO THE CONSUMER GROUPS ON LEVEL )F YOU NEED TO SET AN ABSOLUTE LIMIT ON #05
FOR A CONSUMER GROUP USE THE <0GNDC8;8I0C8>=N;8<8C DIRECTIVE
2ESOURCE 0LAN
! resource plan IS A COLLECTION OF PLAN DIRECTIVES THAT DETERMINE HOW DATABASE RESOURCES ARE TO BE
ALLOCATED 9OU MAY CREATE ANY NUMBER OF RESOURCE PLANS FOR YOUR DATABASE THAT ALLOW YOU TO MEET THE
SPECIFIC SERVICE LEVELS OF YOUR BUSINESS BUT ONLY ONE MAY BE ACTIVE AT ANY GIVEN TIME 9OU MAY DEACTIVATE
THE CURRENT RESOURCE PLAN AND ACTIVATE ANOTHER PLAN WHENEVER THE NEEDS OF THE BUSINESS CHANGE 7HEN
THE ACTIVE RESOURCE PLAN CHANGES ALL CURRENT AND FUTURE SESSIONS WILL BE ALLOCATED RESOURCES BASED ON
DIRECTIVES IN THE NEW PLAN 3WITCHING BETWEEN VARIOUS RESOURCE PLANS IS COMMONLY DONE TO PROVIDE
SUITABLE ALLOCATIONS FOR PARTICULAR TIMES OF THE DAY WEEK OR MONTH &OR EXAMPLE AN AFTER
HOURS PLAN
MAY BE ACTIVATED IN THE EVENING TO FAVOR DATABASE BACKUPS BATCH JOBS EXTRACTS AND DATA
LOADING
ACTIVITIES /THER APPLICATIONS FOR MAINTAINING MULTIPLE PLANS MAY INCLUDE MONTH
END PROCESSING YEAR
END PROCESSING AND THE LIKE
The Pending Area
2ESOURCE PLANS IN THE DATABASE CANNOT BE DIRECTLY MODIFIED NOR CAN YOU DIRECTLY DEFINE NEW PLAN
DIRECTIVES OR RESOURCE GROUPS /RACLE PROVIDES A WORK SPACE CALLED THE pending area FOR CREATING AND
MODIFYING ALL THE ELEMENTS OF A RESOURCE PLAN 9OU CAN THINK OF IT AS A LOADING ZONE WHERE ALL THE
ELEMENTS OF YOUR RESOURCE PLAN ARE STAGED AND VALIDATED TOGETHER BEFORE THEY ARE SUBMITTED TO $"2-
4HERE MAY BE ONLY ONE PENDING AREA IN THE DATABASE AT ANY GIVEN TIME )F A PENDING AREA IS ALREADY OPEN
WHEN YOU TRY TO CREATE ONE /RACLE WILL DISPLAY THE ERROR MESSAGE h>A0 !("& )_T]SX]VPaTPXbP[aTPSh
PRcXeTv 4HE PENDING AREA IS NOT A PERMANENT FIXTURE IN THE DATABASE 9OU MUST EXPLICITLY CREATE IT BEFORE
YOU CAN CREATE OR MODIFY RESOURCE PLANS 4HE FOLLOWING LISTING SHOWS THE TYPICAL PROCESS OF CREATING A
PENDING AREA VALIDATING YOUR CHANGES AND THEN SUBMITTING IT !FTER THE PENDING AREA IS SUBMITTED IT IS
AUTOMATICALLY REMOVED AND A NEW ONE MUST BE CREATED IF YOU WANT TO PERFORM ANY ADDITIONAL WORK ON
$"2- COMPONENTS 4HE FOLLOWING LISTING SHOWS HOW THE 0ENDING !REA IS CREATED VALIDATED AND
SUBMITTED
1468=
31<BNA4B>DA24N<0=064A 2A40C4N?4=38=6N0A40*Å2aTPcTcWT_T]SX]VPaTP
+RaTPcT\^SXUhST[TcTh^daaTb^daRT_[P]-
31<BNA4B>DA24N<0=064A E0;830C4N?4=38=6N0A40*ÅEP[XSPcTh^daf^aZ
31<BNA4B>DA24N<0=064A BD1<8CN?4=38=6N0A40*Å8]bcP[[h^daf^aZX]c^31A<
4=3*
#(!04%2 2%3/52#% -!.!'%-%.4
2ESOURCE -ANAGER 6IEWS
/RACLE SUPPLIES A NUMBER OF VIEWS THAT REPORT CONFIGURATION HISTORY AND METRICS FOR 2ESOURCE -ANAGER
,ETS TAKE A LOOK AT A FEW OF THE VIEWS THAT ARE USEFUL FOR REVIEWING AND MONITORING RESOURCES IN YOUR
$"2- CONFIGURATION
EABA2N2>=BD<4AN6A>D?: 4HE EABA2N2>=BD<4AN6A>D? VIEW DISPLAYS INFORMATION
ABOUT THE ACTIVE RESOURCE CONSUMER GROUPS )T ALSO CONTAINS PERFORMANCE METRICS
THAT ARE USEFUL FOR TUNING PURPOSES 7ELL TAKE A CLOSER LOOK AT THIS VIEW WHEN WE
TEST A RESOURCE PLAN LATER ON IN THE CHAPTER
EABA2N?;0=: 4HIS VIEW DISPLAYS THE CONFIGURATION OF THE CURRENTLY ACTIVE RESOURCE
PLAN
EABA2N?;0=N78BC>AH: 4HE EABA2N?;0=N78BC>AH VIEW SHOWS HISTORICAL INFORMATION
FOR YOUR RESOURCE PLANS INCLUDING WHEN THEY WERE ACTIVATED AND DEACTIVATED AND
WHETHER THEY WERE ENABLED BY THE DATABASE SCHEDULER OR SCHEDULER WINDOWS
EABA2NB4BB8>=N8=5>: 4HIS VIEW SHOWS PERFORMANCE STATISTICS FOR SESSIONS AND
HOW THEY WERE AFFECTED BY THE 2ESOURCE -ANAGER
EB4BB8>=: 4HE EB4BB8>= VIEW IS NOT SPECIFICALLY A 2ESOURCE -ANAGER VIEW BUT ITS
A4B>DA24N2>=BD<4AN6A>D? FIELD IS USEFUL FOR DETERMINING WHAT RESOURCE GROUP A
SESSION IS ASSIGNED TO
310NABA2N20C46>A84B) 4HIS VIEW DISPLAYS THE RESOURCE CATEGORIES THAT ARE
CONFIGURED IN THE DATABASE #ATEGORIES ARE USED BY THE )/ 2ESOURCE -ANAGER FOR
CONTROLLING STORAGE CELL )/ ALLOCATION WITHIN A DATABASE
310NABA2N2>=BD<4AN6A>D?B) 4HIS VIEW DISPLAYS ALL THE CONSUMER GROUPS DEFINED IN
THE DATABASE
310NABA2N2>=BD<4AN6A>D?N?A8EB) 4HIS VIEW REPORTS USERS AND THE RESOURCE
GROUPS TO WHICH THEY HAVE BEEN GRANTED PERMISSION ! USER MUST HAVE PERMISSION
TO SWITCH TO A CONSUMER GROUP BEFORE THE SESSION
TO
CONSUMER GROUP MAPPING
RULES WILL WORK
310NABA2N6A>D?N<0??8=6B) 4HIS VIEW LISTS ALL THE VARIOUS SESSION
TO
RESOURCE
GROUP MAPPING RULES DEFINED IN THE DATABASE
310NABA2N<0??8=6N?A8>A8CH) 4HIS VIEW REPORTS THE PRIORITY OF SESSION ATTRIBUTES
USED IN RESOLVING OVERLAPS BETWEEN MAPPING RULES
310NABA2N8>N20;81A0C4) 4HIS VIEW DISPLAYS THE )/ PERFORMANCE METRICS $"2-
USES FOR )/ RESOURCE MANAGEMENT -AXIMUM READ RATES ARE CAPTURED FOR )/
OPERATIONS PER SECOND )/03 MEGABYTES PER SECOND -"03 AND LATENCIES FOR
DATA BLOCK READ REQUESTS
310NABA2N?;0=B) 4HIS VIEW LISTS ALL RESOURCE PLANS AND THE NUMBER OF PLAN
DIRECTIVES ASSIGNED TO EACH PLAN IN THE DATABASE
310NABA2N?;0=N38A42C8E4B) 4HIS VIEW LISTS ALL RESOURCE PLAN DIRECTIVES RESOURCE
ALLOCATION PERCENTAGES AND LEVELS DEFINED IN THE DATABASE
#(!04%2 2%3/52#% -!.!'%-%.4
310NDB4AB) 4HIS VIEW IS NOT ACTUALLY A 2ESOURCE -ANAGER VIEW BUT IT DOES DISPLAY
THE USERNAME AND INITIAL RESOURCE GROUP ASSIGNMENT IN ITS
8=8C80;NABA2N2>=BD<4AN6A>D? FIELD
310N78BCNABA2N2>=BD<4AN6A>D?) 4HIS VIEW DISPLAYS HISTORICAL PERFORMANCE
METRICS FOR 2ESOURCE CONSUMER GROUPS )T CONTAINS !72 SNAPSHOTS OF THE
EABA2N2>=BN6A>D?N78BC>AH VIEW
310N78BCNABA2N?;0=: 4HIS IS A SIMPLE VIEW THAT DISPLAYS HISTORICAL INFORMATION
ABOUT RESOURCE PLANS SUCH AS WHEN THEY WERE ACTIVATED AND DEACTIVATED
4HE 7AIT %VENT RESMGR CPU QUANTUM
$"2- ALLOCATES #05 RESOURCES BY MAINTAINING AN EXECUTION QUEUE SIMILAR TO THE WAY THE OPERATING
SYSTEMS SCHEDULER QUEUES PROCESSES FOR THEIR TURN ON THE #05 4HE TIME A SESSION SPENDS WAITING IN THIS
EXECUTION QUEUE IS ASSIGNED THE WAIT EVENT aTb\Va)R_d`dP]cd\ ! #05 QUANTUM IS THE UNIT OF #05
TIME FRACTION OF #05 THAT 2ESOURCE -ANAGER USES FOR ALLOCATING #05 TO CONSUMER GROUPS 4HIS EVENT
OCCURS WHEN 2ESOURCE -ANAGER IS ENABLED AND IS ACTIVELY THROTTLING #05 CONSUMPTION )NCREASING THE
#05 ALLOCATION FOR A SESSIONS CONSUMER GROUP WILL REDUCE THE OCCURRENCE OF THIS WAIT EVENT AND INCREASE
THE AMOUNT OF #05 TIME ALLOCATED TO ALL SESSIONS IN THAT GROUP &OR EXAMPLE THE #05 QUANTUM WAIT
EVENTS MAY BE REDUCED FOR THE 0??B RESOURCE GROUP CURRENTLY AT LEVEL BY INCREASING THE GROUPS
#05 ALLOCATION TO
$"2- %XAMPLE
.OW THAT WEVE DISCUSSED THE KEY COMPONENTS OF $"2- AND HOW IT WORKS LETS TAKE A LOOK AT AN
EXAMPLE OF CREATING AND UTILIZING RESOURCE PLANS )N THIS EXAMPLE WELL CREATE TWO RESOURCE PLANS SIMILAR
TO THE ONE IN &IGURE
/NE ALLOCATES #05 RESOURCES SUITABLY FOR CRITICAL 30HC8<4 PROCESSING AND THE
OTHER FAVORS NIGHT
TIME PROCESSING
Step 1: Create Resource Groups
4HE FIRST THING WELL DO IS CREATE THE RESOURCE GROUPS FOR OUR PLAN 4HE FOLLOWING LISTING CREATES THREE
RESOURCE GROUPS 0??B A4?>ACBAND<08=C4=0=24 /NCE WE HAVE THE RESOURCE GROUPS CREATED WELL BE
ABLE TO MAP USER SESSIONS TO THEM
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTNR^]bd\TaNVa^d_
R^]bd\TaNVa^d_,-0??B
R^\\T]c,-2^]bd\TaVa^d_U^aRaXcXRP[>;C?P__[XRPcX^]b*
SQ\bNaTb^daRTN\P]PVTa RaTPcTNR^]bd\TaNVa^d_
R^]bd\TaNVa^d_,-A4?>ACB
R^\\T]c,-2^]bd\TaVa^d_U^a[^]V ad]]X]VaT_^acb*
SQ\bNaTb^daRTN\P]PVTa RaTPcTNR^]bd\TaNVa^d_
R^]bd\TaNVa^d_,-<08=C4=0=24
R^\\T]c,-2^]bd\TaVa^d_U^a\PX]cT]P]RTY^Qb*
SQ\bNaTb^daRTN\P]PVTa eP[XSPcTN_T]SX]VNPaTP*
#(!04%2 2%3/52#% -!.!'%-%.4
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
Step 2: Create Consumer Group Mapping Rules
/KAY SO THAT TAKES CARE OF OUR RESOURCE GROUPS .OW WELL CREATE OUR SESSION TO RESOURCE GROUP
MAPPINGS 4HE FOLLOWING 0,31, BLOCK CREATES MAPPINGS FOR THREE USER ACCOUNTS :>B1>A=4 C?>34A AND
A9>7=B>= AND JUST FOR GOOD MEASURE WELL CREATE A MAPPING FOR OUR 4/!$ USERS OUT THERE 4HIS WILL ALSO
ALLOW US TO SEE HOW ATTRIBUTE MAPPING PRIORITIES WORK
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa bTcNR^]bd\TaNVa^d_N\P__X]V
PccaXQdcT,-SQ\bNaTb^daRTN\P]PVTa ^aPR[TNdbTa
eP[dT,-:>B1>A=4
R^]bd\TaNVa^d_,-0??B*
SQ\bNaTb^daRTN\P]PVTa bTcNR^]bd\TaNVa^d_N\P__X]V
PccaXQdcT,-SQ\bNaTb^daRTN\P]PVTa ^aPR[TNdbTa
eP[dT,-A9>7=B>=
R^]bd\TaNVa^d_,-A4?>ACB*
SQ\bNaTb^daRTN\P]PVTa bTcNR^]bd\TaNVa^d_N\P__X]V
PccaXQdcT,-SQ\bNaTb^daRTN\P]PVTa ^aPR[TNdbTa
eP[dT,-C?>34A
R^]bd\TaNVa^d_,-<08=C4=0=24*
SQ\bNaTb^daRTN\P]PVTa bTcNR^]bd\TaNVa^d_N\P__X]V
PccaXQdcT,-SQ\bNaTb^daRTN\P]PVTa R[XT]cN_a^VaP\
eP[dT,-c^PS TgT
R^]bd\TaNVa^d_,-A4?>ACB*
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
/NE MORE IMPORTANT STEP IS TO GRANT EACH OF THESE USERS PERMISSION TO SWITCH THEIR SESSION TO THE
CONSUMER GROUP YOU SPECIFIED IN YOUR MAPPING RULES )F YOU DONT THEY WILL NOT BE ABLE TO SWITCH THEIR
SESSION TO THE DESIRED RESOURCE GROUP AND WILL INSTEAD BE ASSIGNED TO THE DEFAULT CONSUMER GROUP
>C74AN6A>D?B 3O IF YOU FIND THAT USER SESSIONS ARE LANDING IN >C74AN6A>D?B INSTEAD OF THE RESOURCE GROUP
SPECIFIED IN YOUR MAPPING RULES YOU PROBABLY FORGOT TO GRANT THE bfXcRWNR^]bd\TaNVa^d_ PRIVILEGE TO THE
USER 2EMEMBER THAT THIS WILL ALSO HAPPEN IF THE MAPPING RULE ASSIGNS A SESSION TO A CONSUMER GROUP
THAT IS NOT IN THE ACTIVE RESOURCE PLAN 4HE 6A0=CN>?8C>= PARAMETER IN THE NEXT LISTING DETERMINES
WHETHER OR NOT THE USER WILL BE ALLOWED TO GRANT OTHERS PERMISSION TO SWITCH TO THE CONSUMER GROUP
1468=
SQ\bNaTb^daRTN\P]PVTaN_aXeb VaP]cNbfXcRWNR^]bd\TaNVa^d_
6A0=C44N=0<4,-A9>7=B>=
2>=BD<4AN6A>D?,-A4?>ACB
6A0=CN>?C8>=,-50;B4*
SQ\bNaTb^daRTN\P]PVTaN_aXeb VaP]cNbfXcRWNR^]bd\TaNVa^d_
6A0=C44N=0<4,-:>B1>A=4
2>=BD<4AN6A>D?,-0??B
6A0=CN>?C8>=,-50;B4*
SQ\bNaTb^daRTN\P]PVTaN_aXeb VaP]cNbfXcRWNR^]bd\TaNVa^d_
6A0=C44N=0<4,-C?>34A
#(!04%2 2%3/52#% -!.!'%-%.4
2>=BD<4AN6A>D?,-<08=C4=0=24
6A0=CN>?C8>=,-50;B4*
4=3*
4IP )F YOU TRUST YOUR USERS AND DEVELOPERS NOT TO SWITCH THEIR OWN SESSION TO A HIGHER
PRIORITY CONSUMER GROUP
YOU CAN GRANT THE bfXcRWNR^]bd\TaNVa^d_ PERMISSION TO THE PUBLIC AND MAKE THINGS A LITTLE EASIER ON YOURSELF
Step 3: Set Resource Group Mapping Priorities
3INCE WE WANT TO USE MORE THAN THE SESSIONS DB4A=0<4 TO MAP SESSIONS TO RESOURCE GROUPS WELL NEED TO
SET PRIORITIES FOR THE MAPPING RULES 4HIS TELLS $"2- WHICH RULES SHOULD TAKE PRECEDENCE WHEN A SESSION
MATCHES MORE THAN ONE RULE 4HE FOLLOWING 0,31, BLOCK SETS A PRIORITY FOR THE CLIENT PROGRAM ATTRIBUTE
HIGHER THAN THAT OF THE DATABASE USER ACCOUNT
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa bTcNR^]bd\TaNVa^d_N\P__X]VN_aX
Tg_[XRXc,-
R[XT]cN_a^VaP\,-!
^aPR[TNdbTa,-"
bTaeXRTN\^Sd[TNPRcX^],-#
bTaeXRTN\^Sd[T,-$
\^Sd[TN]P\TNPRcX^],-%
\^Sd[TN]P\T,-&
bTaeXRTN]P\T,-'
R[XT]cN^bNdbTa,-(
R[XT]cN\PRWX]T,- *
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
Step 4: Create the Resource Plan and Plan Directives
'ENERALLY SPEAKING RESOURCE PLANS ARE CREATED AT THE SAME TIME AS THE PLAN DIRECTIVES 4HIS IS BECAUSE
WE CANNOT CREATE AN EMPTY PLAN ! RESOURCE PLAN MUST HAVE AT LEAST ONE PLAN DIRECTIVE FOR THE
>C74AN6A>D?B RESOURCE GROUP 4HE FOLLOWING LISTING CREATES A RESOURCE PLAN CALLED 30HC8<4 AND DEFINES
DIRECTIVES FOR THE RESOURCE GROUPS 0??B A4?>ACB <08=C4=0=24 AND OF COURSE >C74AN6A>D?B
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]
_[P],-SPhcX\T
R^\\T]c,-ATb^daRT_[P]U^a]^a\P[QdbX]TbbW^dab*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-SPhcX\T
Va^d_N^aNbdQ_[P],-0??B
#(!04%2 2%3/52#% -!.!'%-%.4
R^\\T]c,-7XVW_aX^aXchdbTab P__[XRPcX^]b
\V\cN_ ,-& *
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-SPhcX\T
Va^d_N^aNbdQ_[P],-A4?>ACB
R^\\T]c,-<TSXd\_aX^aXchU^aSPhcX\TaT_^acb_a^RTbbX]V
\V\cN_!,-$ *
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-SPhcX\T
Va^d_N^aNbdQ_[P],-<08=C4=0=24
R^\\T]c,-;^f_aX^aXchU^aSPhcX\T\PX]cT]P]RT
\V\cN_",-$ *
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-SPhcX\T
Va^d_N^aNbdQ_[P],->C74AN6A>D?B
R^\\T]c,-0[[^cWTaVa^d_b]^cTg_[XRXcT[h]P\TSX]cWXb_[P]
\V\cN_",-$ *
SQ\bNaTb^daRTN\P]PVTa eP[XSPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
Step 5: Create the Night-Time Plan
/RGANIZATIONS TYPICALLY HAVE DIFFERENT SCHEDULING PRIORITIES FOR AFTER HOURS WORK 4HE =867CC8<4 PLAN
SHIFTS #05 ALLOCATION AWAY FROM THE 0??B RESOURCE GROUP TO THE <08=C4=0=24 GROUP 4HE NEXT LISTING
CREATES THE =867CC8<4 PLAN WITH PRIORITIES THAT FAVOR MAINTENANCE PROCESSING OVER APPLICATIONS AND
REPORTING %VEN SO OF #05 RESOURCES ARE RESERVED FOR THE 0??B AND A4?>ACB RESOURCE GROUPS TO
ENSURE THAT BUSINESS APPLICATIONS AND REPORTS GET SUFFICIENT #05 DURING OFF PEAK HOURS
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]
_[P],-]XVWccX\T
R^\\T]c,-ATb^daRT_[P]U^a]^a\P[QdbX]TbbW^dab*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-]XVWccX\T
Va^d_N^aNbdQ_[P],-<08=C4=0=24
R^\\T]c,-;^f_aX^aXchU^aSPhcX\T\PX]cT]P]RT
\V\cN_ ,-$ *
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-]XVWccX\T
Va^d_N^aNbdQ_[P],-0??B
R^\\T]c,-7XVW_aX^aXchdbTab P__[XRPcX^]b
\V\cN_!,-$ *
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
_[P],-]XVWccX\T
Va^d_N^aNbdQ_[P],-A4?>ACB
R^\\T]c,-<TSXd\_aX^aXchU^aSPhcX\TaT_^acb_a^RTbbX]V
\V\cN_!,-$ *
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_[P]NSXaTRcXeT
#(!04%2 2%3/52#% -!.!'%-%.4
_[P],-]XVWccX\T
Va^d_N^aNbdQ_[P],->C74AN6A>D?B
R^\\T]c,-0[[^cWTaVa^d_b]^cTg_[XRXcT[h]P\TSX]cWXb_[P]
\V\cN_",- *
SQ\bNaTb^daRTN\P]PVTa eP[XSPcTN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
Step 6: Activate the Resource Plan
/NCE OUR RESOURCE PLANS ARE CREATED ONE OF THEM MUST BE ACTIVATED FOR $"2- TO START MANAGING
RESOURCES 2ESOURCE PLANS ARE ACTIVATED BY SETTING THE INSTANCE PARAMETER A4B>DA24N<0=064AN?;0= USING
THE 0;C4ABHBC4< COMMAND )F THE PLAN DOESNT EXIST THEN $"2- IS NOT ENABLED
0;C4ABHBC4<B4CaTb^daRTN\P]PVTaN_[P],30HC8<4B2>?4,1>C7B83,B2A0C27*
9OU CAN AUTOMATICALLY SET THE ACTIVATE RESOURCE PLAN USING SCHEDULER WINDOWS 4HIS METHOD ENSURES THAT
BUSINESS RULES FOR RESOURCE MANAGEMENT ARE ENFORCED CONSISTENTLY 4HE FOLLOWING LISTING MODIFIES THE
BUILT IN SCHEDULER WINDOW F44:=867CNF8=3>F SO THAT IT ENABLES OUR ]XVWccX\T RESOURCE PLAN 4HE WINDOW
STARTS AT 0- HOUR AND RUNS THROUGH !- MINUTES
1468=
31<BNB2743D;4A B4CN0CCA81DC4
=P\T,-BHB F44:=867CNF8=3>F
0ccaXQdcT,-A4B>DA24N?;0=
EP[dT,-NIGHTTIME*
31<BNB2743D;4A B4CN0CCA81DC4
]P\T,-BHB F44:=867CNF8=3>F
PccaXQdcT,-A4?40CN8=C4AE0;
eP[dT,-5A4@,F44:;H*1H30H,<>=CD4F43C7D5A8*1H7>DA,18*1H<8=DC4, *1HB42>=3, *
31<BNB2743D;4A B4CN0CCA81DC4
]P\T,-BHB F44:=867CNF8=3>F
PccaXQdcT,-3DA0C8>=
eP[dT,-]d\c^SbX]cTaeP[780\X]dcT*
31<BNB2743D;4A 4=01;4]P\T,-BHB F44:=867CNF8=3>F*
4=3*
.OW WELL CREATE A NEW WINDOW THAT COVERS NORMAL BUSINESS HOURS CALLED F44:30HNF8=3>F 4HIS WINDOW
WILL AUTOMATICALLY SWITCH THE ACTIVE RESOURCE PLAN TO OUR 30HC8<4 RESOURCE PLAN 4HE WINDOW STARTS AT
!- HOUR AND RUNS UNTIL 0- MINUTES AT WHICH POINT OUR F44:=867CNF8=3>F BEGINS
#(!04%2 2%3/52#% -!.!'%-%.4
1468=
31<BNB2743D;4A 2A40C4NF8=3>F
fX]S^fN]P\T,-F44:30HNF8=3>F
aTb^daRTN_[P],-DAYTIME
bcPacNSPcT,-bhbcX\TbcP\_PccX\Ti^]T %)
SdaPcX^],-]d\c^SbX]cTaeP[660\X]dcT
aT_TPcNX]cTaeP[,-5A4@,F44:;H*1H30H,<>=CD4F43C7D5A8*1H7>DA,7*1H<8=DC4, *1HB42>=3,
T]SNSPcT,-]d[[
fX]S^fN_aX^aXch,-;>F
R^\\T]cb,-FTTZSPhfX]S^f BTcbcWTPRcXeTaTb^daRT_[P]c^30HC8<4*
31<BNB2743D;4A 4=01;4]P\T,-BHB F44:30HNF8=3>F*
4=3*
4ESTING A 2ESOURCE 0LAN
"EFORE WE FINISH WITH $ATABASE 2ESOURCE -ANAGER LETS TEST ONE OF OUR RESOURCE PLANS TO SEE IF IT WORKS AS
ADVERTISED 6ALIDATING THE PRECISE #05 ALLOCATION TO EACH OF OUR RESOURCE GROUPS IS A VERY COMPLICATED
UNDERTAKING SO WE WONT BE DIGGING INTO IT TOO DEEPLY "UT WE WILL TAKE A LOOK AT THE
EABA2N2>=BD<4AN6A>D? VIEW TO SEE IF WE CAN ACCOUNT FOR HOW THE #05 RESOURCES WERE ALLOCATED AMONG
OUR CONSUMER GROUPS &OR THE TEST WELL USE THE B2A0C27 DATABASE AND THE 30HC8<4 RESOURCE PLAN WE
CREATED EARLIER 4HE RESULTS OF THE TEST WILL
v 6ERIFY THAT SESSIONS MAP PROPERLY TO THEIR 2ESOURCE 'ROUPS
v 3HOW HOW TO IDENTIFY $"2- WAIT EVENTS IN A SESSION TRACE
v 6ERIFY THAT #05 IS ALLOCATED ACCORDING TO OUR RESOURCE PLAN
&IGURE
SHOWS RESOURCE ALLOCATION DIRECTIVES FOR THE 30HC8<4 RESOURCE PLAN WELL BE TESTING
Figure 7-2. DAYTIME resource plan allocation
)F OUR 30HC8<4 RESOURCE PLAN IS WORKING #05 WILL BE ALLOCATED ACCORDING TO THE FOLLOWING FORMULA .OTE
THAT THE AND ALLOCATIONS REFLECT THE PERCENT OF TOTAL #05
,EVEL 0??B ¾
,EVEL A4?>ACB
¾
,EVEL <08=C4=0=24
¾ ¾
,EVEL >C74AN6A>D?B
¾ ¾
#(!04%2 2%3/52#% -!.!'%-%.4
Test Outline
.OW THAT WEgVE CREATED OUR RESOURCE PLAN WE CAN TEST TO SEE HOW IT WORKS &OLLOWING ARE THE STEPS WE
WILL FOLLOW TO TEST OUR RESOURCE PLAN
4URN OFF THE $ATABASE 2ESOURCE -ANAGER
3TART A SESSION USING THE A9>7=B>= ACCOUNT
3TART CONCURRENT #05 INTENSIVE QUERIES FROM EACH OF THE USER ACCOUNTS THAT
MAP TO OUR CONSUMER GROUPS 4HESE USER ACCOUNTS MAP TO RESOURCE GROUPS AS
FOLLOWS
:>B1>A=4 Æ 0??B
A9>7=B>= Æ A4?>ACB
C?>34A Æ <08=C4=0=24
5A43 Æ >C74AN6A>D?B
#HECK THE CONSUMER GROUP ASSIGNMENTS IN THE 6B4BB8>=
A4B>DA24N2>=BD<4AN6A>D? VIEW 4HIS COLUMN SHOULD BE NULL SINCE $"2- IS
INACTIVE
3TART A SESSION TRACE ON AN A9>7=B>= SESSION
2UN A #05 INTENSIVE QUERY FROM THE A9>7=B>= SESSION
4AIL THE SESSION TRACE FILE AND WATCH FOR aTb\Va)R_d`dP]cd\ WAIT EVENTS 4HERE
SHOULDNT BE ANY AT THIS POINT BECAUSE $"2- IS INACTIVE
7HILE THE LOAD TEST IS STILL RUNNING ACTIVATE THE 30HC8<4 RESOURCE PLAN
#HECK THE CONSUMER GROUP ASSIGNMENTS AGAIN .OW THAT $"2- IS ACTIVE
SESSIONS SHOULD BE ASSIGNED TO THEIR RESPECTIVE CONSUMER GROUPS
#HECK THE A9>7=B>= SESSION TRACE FILE AGAIN 7E SHOULD SEE aTb\Va)R_d`dP]cd\
WAIT EVENTS NOW THAT THE 30HC8<4 RESOURCE PLAN IS ACTIVE
2EVIEW THE 2ESOURCE -ANAGER METRICS IN THE EABA2N2>=BD<4AN6A>D? VIEW TO
SEE HOW #05 RESOURCES WERE ALLOCATED DURING THE TEST 7E SHOULD SEE #05
ALLOCATED ACCORDING TO THE DIRECTIVES IN OUR RESOURCE PLAN
3TEP $EACTIVATE $"2-
.OW WELL BEGIN OUR TEST OF THE RESOURCE PLAN 4HE FIRST THING WELL DO IS TURN OFF $"2- BY SETTING THE
INSTANCE DATABASE PARAMETER A4B>DA24N<0=064AN?;0= TO gg AN EMPTY STRING
BHB)B2A0C27-P[cTabhbcT\bTcaTb^daRTN\P]PVTaN_[P],*
3TEP ,OG )N AS 2*/(.3/.
.OW WELL START A 31,
0LUS SESSION LOGGING IN AS A9>7=B>=
#(!04%2 2%3/52#% -!.!'%-%.4
JT]ZSQ !)aY^W]b^])B2A0C27L W^\T aY^W]b^] \hbRaX_cb
-b`[_[dbaY^W]b^] g
q
A9>7=B>=)B2A0C27-
3TEP 3TART ,OAD 4EST
)N FOUR SEPARATE TERMINAL WINDOWS WELL GENERATE A LOAD ON THE SYSTEM BY RUNNING A SHELL SCRIPT THAT
SPINS UP 31,
0LUS SESSIONS FOR EACH USER ACCOUNT %ACH SESSION KICKS OFF THE FOLLOWING QUERY WHICH
CREATES A #ARTESIAN PRODUCT 4HE SKEW TABLE HAS ROWS SO THE JOIN WILL CREATE BILLIONS OF LOGICAL
)/ OPERATIONS 4HAT ALONG WITH THE SUM ON 2>; SHOULD CREATE SUFFICIENT #05 LOAD FOR OUR TESTS 4HE
FOLLOWING LISTING SHOWS THE DEFINITION OF THE B:4F TABLE WITH INDEXES ON THE 2>; AND 2>;! COLUMNS
2A40C4C01;4B:4F
?:N2>;=D<14A
2>; =D<14A
2>;!E0A270A!" 1HC4
2>;"30C4
2>;#E0A270A! 1HC4*
2A40C48=34GB:4FN2>;!>=B:4F2>;!*
2A40C48=34GB:4FN2>;!>=B:4F2>;!*
.OW LETS START THE TEST QUERIES AND TAKE A LOOK AT THE #05 UTILIZATION 4HE FOLLOWING LISTING SHOWS THE
QUERY WELL BE USING FOR THE TEST
CTbc@dTah
bT[TRcP R^[!bd\P R^[
Ua^\aY^W]b^] bZTfP
aY^W]b^] bZTfQ
Va^d_QhP R^[!*
CWTQda]NR_d bW SHELL SCRIPT SHOWN NEXT EXECUTES CONCURRENT COPIES OF THE Qda]NR_d b`[ SCRIPT 7ELL
RUN THIS SCRIPT ONCE FOR EACH OF THE USER ACCOUNTS 5A43 :>B1>A=4 A9>7=B>= AND C?>34A /UR TEST
CONFIGURATION IS A SINGLE DATABASE B2A0C27 ON A QUARTER RACK %XADATA 6 2ECALL THAT THE 6 IS
CONFIGURED WITH TWO QUAD
CORE #05S
QX] QPbW
Tg_^acdbTa,
Tg_^ac_PbbfS,!
Tg_^ac_PaP[[T[,"
Qda]NR_dj
b`[_[db b++4>5
dbTa _PbbfS
/Qda]NR_d b`[
TgXc
4>5
l
#(!04%2 2%3/52#% -!.!'%-%.4
9>1B,
fWX[T)*S^
Qda]NR_d
9>1B,OY^QbkfR [O
fWX[TJ9>1B VT_PaP[[T[L*S^
b[TT_$
9>1B,OY^QbkfR [O
S^]T
S^]T
7ITH $"2- DISABLED OUR TEST SESSIONS PUT A HEAVY LOAD ON THE #05 /UTPUT FROM THE c^_ COMMAND
SHOWS RUNNING PROCESSES AND USER #05 TIME AT
c^_ !!)! ) #d_ SPhb()"' "dbTab[^PSPeTaPVT) " ' !! &"!$ ('
CPbZb) !""c^cP[26 running ! &b[TT_X]V bc^__TS i^\QXT
2_db)80.8%us# #bh ]X # &XS fP WX bX bc
3TEP #HECK #ONSUMER 'ROUP !SSIGNMENTS
,ETS TAKE A LOOK AT OUR SESSION
TO
CONSUMER GROUP MAPPINGS 7HEN $"2- IS INACTIVE SESSIONS WILL
SHOW NO CONSUMER GROUP ASSIGNMENT 4HIS IS ANOTHER WAY TO VERIFY THAT 2ESOURCE -ANAGER IS NOT ACTIVE
BHB)B2A0C27-B4;42Cb dbTa]P\Tb aTb^daRTNR^]bd\TaNVa^d_R^d]c
5A><ebTbbX^]be_a^RTbb_
F74A4b dbTa]P\T8B=>C=D;;
0=3=E;b ^bdbTag+-BHBC4<
0=3b CH?4+-102:6A>D=3
0=3_ PSSa,b _PSSa
0=3b dbTa]P\T]^cX]BHB31B=<?
6A>D?1Hb dbTa]P\Tb aTb^daRTNR^]bd\TaNVa^d_
>A34A1Hb dbTa]P\T*
DB4A=0<4A4B>DA24N2>=BD<4AN6A>D?2>D=C
5A43!
:>B1>A=4!
A9>7=B>=!
C?>34A!
4HE QUERY OUTPUT SHOWS A TOTAL OF SESSIONS CONSISTING OF TWENTY SESSIONS PER USER ACCOUNT PLUS ONE
INTERACTIVE A9>7=B>= SESSION THAT WELL TRACE IN THE NEXT STEP .O SESSIONS ARE CURRENTLY MAPPED TO THE
RESOURCE GROUPS BECAUSE $"2- IS INACTIVE
3TEP 3TART 3ESSION 4RACE FOR THE INTERACTIVE A9>7=B>= SESSION
.OW WELL START A TRACE FOR THE INTERACTIVE A9>7=B>= SESSION SO WE CAN SEE THE 2ESOURCE -ANAGER
WAIT EVENTS THAT WOULD INDICATE $"2- IS ACTIVELY REGULATING #05 FOR THIS SESSION 2EMEMBER THAT $"2-
IS STILL INACTIVE SO WE SHOULDNT SEE ANY 2ESOURCE -ANAGER WAIT EVENTS IN THE TRACE FILE YET
A9>7=B>=)B2A0C27-P[cTabTbbX^]bTccaPRTUX[TNXST]cXUXTa,A9>7=B>=*
A9>7=B>=)B2A0C27-P[cTabTbbX^]bTcTeT]cb #%caPRT]P\TR^]cTgcU^aTeTa[TeT[ !*
#(!04%2 2%3/52#% -!.!'%-%.4
3TEP %XECUTE A 1UERY FROM THE A9>7=B>= 3ESSION
.EXT WELL EXECUTE A LONG
RUNNING #05
INTENSIVE QUERY FROM THE INTERACTIVE A9>7=B>= SESSION 4HIS IS
THE SAME QUERY WE USED FOR THE LOAD TEST IN 3TEP
A9>7=B>=)B2A0C27-bT[TRcP R^[!bd\P R^[
Ua^\aY^W]b^] bZTfP
aY^W]b^] bZTfQ
Va^d_QhP R^[!*
3TEP %XAMINE THE 3ESSION 4RACE &ILE
3INCE OUR RESOURCE PLAN IS NOT ACTIVE YET WE DONT SEE ANY 2ESOURCE -ANAGER WAIT EVENTS IN THE TRACE FILE
AT THIS POINT
JT]ZSQ !)aY^W]b^])B2A0C27L
-cPX[ $ UB2A0C27N^aPN!%( NA9>7=B>= caRkVaT_aTb\Va)R_d`dP]cd\
3TEP !CTIVATE 2ESOURCE -ANAGER
.OW WHILE THE LOAD TEST IS STILL RUNNING LETS ENABLE $"2- BY SETTING THE ACTIVE RESOURCE PLAN TO OUR
30HC8<4 PLAN 7HEN THE RESOURCE PLAN IS ACTIVATED OUR RESOURCE MAPPING RULES SHOULD ENGAGE AND SWITCH
THE RUNNING SESSIONS TO THEIR RESPECTIVE CONSUMER GROUPS
BHB)B2A0C27-P[cTabhbcT\bTcaTb^daRTN\P]PVTaN_[P],30HC8<4*
3TEP #HECK #ONSUMER 'ROUP !SSIGNMENTS
.OW LETS RUN THAT QUERY AGAIN AND SEE WHAT OUR CONSUMER GROUP ASSIGNMENTS LOOK LIKE
BHB)B2A0C27-B4;42Cb dbTa]P\Tb aTb^daRTNR^]bd\TaNVa^d_R^d]c
5A><ebTbbX^]be_a^RTbb_
F74A4b dbTa]P\T8B=>C=D;;
0=3=E;b ^bdbTag+-BHBC4<
0=3b CH?4+-102:6A>D=3
0=3_ PSSa,b _PSSa
0=3b dbTa]P\T]^cX]BHB31B=<?
6A>D?1Hb dbTa]P\Tb aTb^daRTNR^]bd\TaNVa^d_
>A34A1Hb dbTa]P\T*
#(!04%2 2%3/52#% -!.!'%-%.4
DB4A=0<4A4B>DA24N2>=BD<4AN6A>D?2>D=C
5A43>C74AN6A>D?B!
:>B1>A=40??B!
A9>7=B>=A4?>ACB!
C?>34A<08=C4=0=24!
/UR USER SESSIONS ARE MAPPING PERFECTLY 4HE QUERY SHOWS THAT ALL USER SESSIONS HAVE BEEN SWITCHED
TO THEIR CONSUMER GROUP ACCORDING TO THE MAPPING RULES WE DEFINED EARLIER
3TEP %XAMINE THE 3ESSION 4RACE &ILE
.OW LETS TAKE ANOTHER LOOK AT THE SESSION TRACE WE STARTED IN STEP AND WATCH FOR $"2- WAIT EVENTS
aTb\Va)R_d`dP]cd\ 4HE OUTPUT FROM THE TRACE FILE SHOWS THE WAIT EVENTS /RACLE USED TO ACCOUNT FOR
THE TIME OUR INTERACTIVE A9>7=B>= SESSION SPENT IN THE $"2- EXECUTION QUEUE WAITING FOR ITS TURN ON THE
#05
JT]ZSQ !)aY^W]b^])B2A0C27L W^\T aY^W]b^]
-R[TPa*cPX[ $ UB2A0C27N^aPN &" NA9>7=B>= caRkVaT_aTb\Va)R_d`dP]cd\
F08C#&((#''%'#&"%')]P\,aTb\Va)R_d`dP]cd\ela= 120[^RPcX^],!R^]bd\TaVa^d_XS,&'$%',
^QY,&'$&#cX\, !('((""( '$'&%$
F08C#&((#''%'#&"%')]P\,aTb\Va)R_d`dP]cd\ela= 14471[^RPcX^],!R^]bd\TaVa^d_XS,&'$%'
, ^QY,&'$&#cX\, !('((""( '&#&(!
F08C#&((#''%'#&"%')]P\,aTb\Va)R_d`dP]cd\ela= 57357[^RPcX^],!R^]bd\TaVa^d_XS,&'$%'
, ^QY,&'$&#cX\, !('((""( (# $%
F08C#&((#''%'#&"%')]P\,aTb\Va)R_d`dP]cd\ela= 109930[^RPcX^],!R^]bd\TaVa^d_XS,&'$%'
, ^QY,&'$&#cX\, !('((""(! $!!$(
F08C#&((#''%'#&"%')]P\,aTb\Va)R_d`dP]cd\ela= 84908[^RPcX^],!R^]bd\TaVa^d_XS,&'$%'
, ^QY,&'$&#cX\, !('((""(! # ( #
!S YOU CAN SEE THE A9>7=B>= USER SESSION IS BEING GIVEN A LIMITED AMOUNT OF TIME ON THE #05 4HE T[P,
ATTRBUTE IN THE TRACE RECORDS SHOWS THE AMOUNT OF TIME IN MICROSECONDS THAT THE SESSION SPENT IN THE
aTb\Va)R_d`dP]cd\ WAIT EVENT )N THE SNIPPET FROM THE TRACE FILE WE SEE THAT THE A9>7=B>= SESSION WAS
FORCED TO WAIT FOR A TOTAL OF MICROSECONDS OR #05 SECONDS .OTE THAT THE OUTPUT SHOWN HERE
REPRESENTS A VERY SMALL SAMPLE OF THE TRACE FILE 4HERE WERE ACTUALLY THOUSANDS OF OCCURRENCES OF THE WAIT
EVENT IN THE TRACE FILE 4HE SUM OF THE T[P TIME IN THESE WAIT EVENTS REPRESENTS THE AMOUNT OF TIME THE
SESSION WAS FORCED OFF THE #05 IN ORDER TO ENFORCE THE ALLOCATION DIRECTIVES IN THE 30HC8<4 PLAN
3TEP #HECK $"2- -ETRICS
!ND FINALLY IF WE LOOK AT THE EABA2N2>=BD<4AN6A>D? VIEW WE CAN SEE THE VARIOUS METRICS THAT /RACLE
PROVIDES FOR MONITORING $"2- 4HESE COUNTERS ARE RESET WHEN A NEW RESOURCE PLAN IS ACTIVATED 3OME
ACCUMULATE OVER THE LIFE OF THE ACTIVE PLAN WHILE OTHERS ARE EXPRESSED AS A PERCENTAGE AND REPRESENT A
CURRENT READING
#(!04%2 2%3/52#% -!.!'%-%.4
4IP 4HE ENABA2<6A<4CA82 AND ENABA2<6A<4CA82N78BC>AH VIEWS ARE ALSO VERY USEFUL FOR MONITORING THE
EFFECTS THAT YOUR $"2- RESOURCE ALLOCATIONS HAVE ON SESSIONS IN THE DATABASE
4ABLE SHOWS THE DEFINITIONS OF THE #05 RELATED COLUMNS WERE INTERESTED IN
Table 7-3. CPU-Related Columns in the V$RSRC_CONSUMER_GROUP View
Column Description
=0<4 4HE RESOURCE GROUP NAME
02C8E4NB4BB8>=B 4HE NUMBER OF ACTIVE SESSIONS IN THE CONSUMER GROUP
4G42DC8>=NF08C4AB 4HE NUMBER OF ACTIVE SESSIONS WAITING FOR A TIME SLICE IN WHICH THEY
CAN USE THE #05
A4@D4BCB 4HE CUMULATIVE NUMBER OF REQUESTS MADE BY SESSIONS IN THE
CONSUMER GROUP
2?DNF08CNC8<4 4HE CUMULATIVE AMOUNT OF TIME THAT 2ESOURCE -ANAGER MADE
SESSIONS IN THE 2ESOURCE 'ROUP WAIT FOR #05 4HIS WAIT TIME DOES
NOT INCLUDE )/ WAITS DELAYS FROM QUEUE OR LATCH CONTENTION OR THE
LIKE 2?DNF08CNC8<4 IS THE SUM OF THE ELAPSED TIME ALLOCATED TO THE
aTb\Va)R_d QUANTUM WAIT EVENT FOR THE CONSUMER GROUP
2?DNF08CB 4HE CUMULATIVE NUMBER OF TIMES SESSIONS WERE MADE TO WAIT
BECAUSE OF RESOURCE MANAGEMENT
2>=BD<43N2?DNC8<4 4HE TOTAL AMOUNT OF #05 TIME ACCUMULATED IN MILLISECONDS BY
SESSIONS IN THE CONSUMER GROUP
H84;3B 4HE CUMULATIVE NUMBER OF TIMES SESSIONS IN THE CONSUMER GROUP
HAD TO YIELD THE #05 TO OTHER SESSIONS BECAUSE OF RESOURCE
MANAGEMENT
4HE FOLLOWING LISTING IS A REPORT YOU MAY USE TO DISPLAY THE METRICS COLLECTED IN THE
EABA2N2>=BD<4AN6A>D? VIEW 4HESE METRICS ARE A VALUABLE TOOL FOR DETERMINING THE EFFECT OUR RESOURCE
ALLOCATIONS HAD ON THE CONSUMER GROUPS DURING THE TEST
R^[]P\TU^a\PcP !WTPSX]V=P\T
R^[PRcXeTNbTbbX^]bU^a\Pc(((WTPSX]V0RcXeTkBTbbX^]b
R^[TgTRdcX^]NfPXcTabU^a\Pc(((WTPSX]V4gTRdcX^]kFPXcTab
R^[aT`dTbcbU^a\Pc(((((((WTPSX]VAT`dTbcb
R^[R_dNfPXcNcX\TU^a\Pc(((((((((WTPSX]V2?DFPXckCX\T
#(!04%2 2%3/52#% -!.!'%-%.4
R^[R_dNfPXcbU^a\Pc((((((((WTPSX]V2?DkFPXcb
R^[R^]bd\TSNR_dNcX\TU^a\Pc((((((((WTPSX]V2^]bd\TSk2?DCX\T
R^[hXT[SbU^a\Pc(((((((WTPSX]VHXT[Sb
B4;42C342>34]P\TN>A02;4N102:6A>D=3N6A>D?N102:6A>D=3]P\T]P\T
PRcXeTNbTbbX^]bTgTRdcX^]NfPXcTabaT`dTbcb
R_dNfPXcNcX\TR_dNfPXcbR^]bd\TSNR_dNcX\ThXT[Sb
5A><eabaRNR^]bd\TaNVa^d_
>A34A1HR_dNfPXcNcX\T*
0RcXeT4gTRdcX^]2?DFPXc2?D2^]bd\TS
=P\TBTbbX^]bFPXcTabAT`dTbcbCX\TFPXcb2?DCX\THXT[Sb
102:6A>D=3"# &%
0??B" "" '& $&&"( #('!'%#&(%"' ("%$%
A4?>ACB" !&" #$$%%$!#!#&%%$ &""!&#&'($
<08=C4=0=24" !(" $$ '( " !' !&($&%"&%## "%'
>C74AN6A>D?B"#!( " $$#"&& $ !$(&%%$$&%%! # %'
)N THIS REPORT YOU CAN SEE HOW 2ESOURCE -ANAGER ALLOCATED #05 RESOURCES TO THE CONSUMER GROUPS
ACCORDING TO OUR PLAN DIRECTIVES .OTICE THE 102:6A>D=3 RESOURCE GROUP
NAMED?>A02;4N102:6A>D=3N6A>D?N $ATABASE BACKGROUND PROCESSES ARE ASSIGNED TO THIS SPECIAL GROUP
0ROCESSES INCLUDED IN THIS GROUP INCLUDE _\^] b\^] SQf [Vfa AND A HOST OF OTHER FAMILIAR BACKGROUND
PROCESSES THAT MANAGE THE DATABASE !SSIGNING PERFORMANCE
CRITICAL PROCESSES TO THIS GROUP IS THE WAY
2ESOURCE -ANAGER EXCLUDES THEM FROM RESOURCE MANAGEMENT &OR ALL OTHER CONSUMER GROUPS YOU CAN
SEE THAT 2ESOURCE -ANAGER FORCED SESSIONS TO YIELD THE PROCESSOR IN ORDER TO DISTRIBUTE #05 RESOURCES
ACCORDING TO OUR RESOURCE DIRECTIVES 4HE NUMBER OF YIELDS AND #05 WAITS ARE OF INTEREST BUT NOT AS
TELLING AS THE #05 WAIT TIME AND #05 TIME CONSUMED 4HE PERCENTAGES IN &IGURE
SHOW HOW #05 AND
WAIT TIME WERE ALLOCATED AMONG OUR CONSUMER GROUPS
Figure 7-3. DAYTIME resource plan allocation
#(!04%2 2%3/52#% -!.!'%-%.4
!CCORDING TO 2ESOURCE -ANAGER THE 0??B GROUP CONSUMED OF THE TOTAL #05 USED BY FOREGROUND
PROCESSES WHICH IS VERY CLOSE TO THE WE ALLOCATED IT IN OUR RESOURCE PLAN !T THE A4?>ACB
GROUP WAS ALMOST A PERFECT MATCH TO THE OUR PLAN CALLED FOR 4HE <08=C4=0=24 GROUP USED
WHICH WAS A LITTLE HIGH BUT STILL A VERY GOOD FIT WITH THE WE DEFINED FOR IT 4HE >C74AN6A>D?B USED
#05 WHICH AGAIN WAS NEARLY A PERFECT MATCH WITH OUR PLAN DIRECTIVE OF 7E SHOULD MENTION
THAT AT FIRST THE ALLOCATIONS IN THIS REPORT WERE NOT PROPORTIONED VERY CLOSELY TO THE ALLOCATIONS IN OUR
RESOURCE PLAN 7E HAD TO LET THE STRESS TEST RUN FOR SEVERAL MINUTES BEFORE $"2- WAS ABLE TO GET THE
NUMBERS FINE
TUNED TO THE LEVELS WE SEE IN &IGURE
.OTE )N ORDER TO GET #05 UTILIZATION TO LINE UP WITH THE RESOURCE PLAN EACH CONSUMER GROUP MUST BE FULLY
CAPABLE OF UTILIZING ITS ALLOCATION 'ETTING A MATCH BETWEEN #05 UTILIZATION AND CONSUMER GROUP #05 ALLOCATION IS
FURTHER COMPLICATED BY MULTI
LEVEL RESOURCE PLANS AND THE WAY 2ESOURCE -ANAGER REDISTRIBUTES UNCONSUMED #05
TO OTHER CONSUMER GROUPS -ULTI
LEVEL RESOURCE PLANS ARE NOT COMMON IN REAL
WORLD SITUATIONS -OST OF THE TIME
SIMPLE SINGLE
LEVEL RESOURCE PLANS ARE SUFFICIENT AND MUCH EASIER TO MEASURE
)N CONCLUSION EVEN THOUGH THE TEST RESULTS SHOW MINOR VARIANCES BETWEEN #05 ALLOCATED IN OUR PLAN
DIRECTIVES AND #05 UTILIZATION REPORTED YOU CAN SEE THAT $"2- WAS IN FACT MANAGING #05 RESOURCES
ACCORDING TO OUR PLAN 4HE TEST ALSO VERIFIED THAT USER SESSIONS PROPERLY SWITCHED TO THEIR RESPECTIVE
CONSUMER GROUPS ACCORDING TO OUR MAPPING RULES WHEN THE RESOURCE PLAN WAS ACTIVATED
$ATABASE 2ESOURCE -ANAGER HAS BEEN AVAILABLE FOR A NUMBER OF YEARS NOW )T IS A VERY ELEGANT
COMPLEX AND EFFECTIVE TOOL FOR MANAGING THE SERVER RESOURCES THAT ARE THE VERY LIFE BLOOD OF YOUR
DATABASES 5NFORTUNATELY IN OUR EXPERIENCE IT IS RARELY USED 4HERE ARE PROBABLY SEVERAL REASONS FOR THIS
$"!S ARE CONTINUALLY BARRAGED BY COMPLAINTS THAT QUERIES RUN TOO LONG AND APPLICATIONS SEEM SLUGGISH
7E ARE OFTEN RELUCTANT TO IMPLEMENT ANYTHING THAT WILL SLOW ANYONE DOWN 4HIS IS OFTEN COMPOUNDED
WHEN MULTIPLE ORGANIZATIONS WITHIN THE COMPANY SHARE THE SAME DATABASE OR SERVER )T IS A DIFFICULT TASK
TO ADDRESS PRIORITIES WITHIN A COMPANY WHERE IT COMES TO DATABASE PERFORMANCE AND THE DECISION IS
USUALLY OUT OF THE CONTROL OF $"!S WHO ARE RESPONSIBLE FOR SOMEHOW PLEASING EVERYONE 3OUND FAMILIAR
/UR SUGGESTION WOULD BE TO START SMALL 3EPARATE THE MOST OBVIOUS GROUPS WITHIN YOUR DATABASE BY
PRIORITY 0RIORITIZING AD
HOC QUERIES FROM /,40 APPLICATIONS WOULD BE A GOOD PLACE TO START 7ITH EACH
STEP YOU WILL LEARN WHAT WORKS AND DOESNT WORK FOR YOUR BUSINESS 3O START SMALL +EEP IT SIMPLE AND
IMPLEMENT RESOURCE MANAGEMENT IN SMALL INCREMENTAL STEPS
Instance Caging
7HILE 2ESOURCE -ANAGER PLAN DIRECTIVES PROVISION #05 USAGE BY CONSUMER GROUP WITHIN THE DATABASE
instance caging PROVISIONS #05 AT THE DATABASE INSTANCE LEVEL 7ITHOUT INSTANCE CAGING THE OPERATING
SYSTEM TAKES SOLE RESPONSIBILITY FOR SCHEDULING PROCESSES TO RUN ON THE #05S ACCORDING TO ITS OWN
ALGORITHMS &OREGROUND AND BACKGROUND PROCESSES AMONG ALL DATABASES INSTANCES ARE SCHEDULED ON THE
#05S WITHOUT RESPECT TO BUSINESS PRIORITIES 7ITHOUT INSTANCE CAGING SESSIONS FROM ONE DATABASE CAN
MONOPOLIZE #05 RESOURCES DURING PEAK PROCESSING PERIODS AND DEGRADE PERFORMANCE OF OTHER DATABASES
ON THE SERVER #ONVERSELY PROCESSES RUNNING WHEN THE LOAD ON THE SYSTEM IS VERY LIGHT TEND TO PERFORM
DRAMATICALLY BETTER CREATING WIDE SWINGS IN RESPONSE TIME FROM ONE MOMENT TO THE NEXT )NSTANCE
CAGING ALLOWS YOU TO DYNAMICALLY SET AN ABSOLUTE LIMIT ON THE AMOUNT OF #05 A DATABASE MAY USE !ND
BECAUSE INSTANCE CAGING ENFORCES A MAXIMUM LIMIT ON THE #05 AVAILABLE TO THE INSTANCE IT TENDS TO
SMOOTH OUT THOSE WIDE PERFORMANCE SWINGS AND PROVIDE MUCH MORE CONSISTENT RESPONSE TIMES TO END
#(!04%2 2%3/52#% -!.!'%-%.4
USERS 4HIS IS NOT TO SAY THAT INSTANCE CAGING LOCKS THE DATABASE PROCESSES DOWN ON A SPECIFIC SET OF
PHYSICAL #05 CORES A TECHNIQUE CALLED CPU affinity ALL #05 CORES ARE STILL UTILIZED BY ALL DATABASE
BACKGROUND AND FOREGROUND PROCESSES 2ATHER INSTANCE CAGING REGULATES THE AMOUNT OF #05 TIME OF
#05 A DATABASE MAY USE AT ANY GIVEN TIME
)NSTANCE CAGING ALSO SOLVES SEVERAL LESS OBVIOUS PROBLEMS CAUSED BY #05 STARVATION 3OME INSTANCE
PROCESSES ARE CRITICAL TO OVERALL HEALTH AND PERFORMANCE OF THE /RACLE DATABASE &OR EXAMPLE IF THE LOG
WRITER PROCESS ;6FA DOESNT GET ENOUGH TIME ON THE PROCESSOR THE DATABASE CAN SUFFER DRAMATIC
SYSTEM
WIDE BROWNOUTS BECAUSE ALL DATABASE WRITE ACTIVITY COMES TO A SCREECHING A HALT WHILE ;6FA WRITES
CRITICAL RECOVERY INFORMATION TO THE ONLINE REDO LOGS )NSUFFICIENT #05 RESOURCES CAN CAUSE SIGNIFICANT
PERFORMANCE PROBLEMS AND STABILITY ISSUES IF 0ROCESS -ONITOR ?<>= CANNOT GET ENOUGH TIME ON THE
#05 &OR 2!# SYSTEMS THE ,OCK -ANAGEMENT 3ERVER ;<B PROCESS CAN EVEN CAUSE SPORADIC NODE
EVICTIONS DUE TO #05 STARVATION WEVE SEEN THIS ONE A NUMBER OF TIMES
.OTE #LUSTERWARE WAS HEAVILY UPDATED IN VERSION AND RENAMED 'RID )NFRASTRUCTURE !CCORDING TO OUR
/RACLE SOURCES #05 STARVATION LEADING TO NODE EVICTION IS RARELY AN ISSUE ANYMORE THANKS TO CHANGES IN
)NSTANCE CAGING DIRECTLY ADDRESSES #05 PROVISIONING FOR MULTITENANT DATABASE ENVIRONMENTS
MAKING IT A VERY USEFUL TOOL FOR DATABASE CONSOLIDATION EFFORTS &OR EXAMPLE LETS SAY YOU HAVE FOUR
DATABASES EACH RUNNING ON A SEPARATE SERVER 4HESE SERVERS EACH HAVE FOUR OUTDATED #05S SO
CONSOLIDATING THEM ONTO A NEW SERVER WITH BRAND
NEW #05 CORES SHOULD EASILY PROVIDE PERFORMANCE
THAT IS AT LEAST ON PAR WITH WHAT THEY CURRENTLY HAVE 7HEN YOU MIGRATE THE FIRST DATABASE THE END USERS
ARE ECSTATIC 1UERIES THAT USED TO RUN FOR AN HOUR BEGIN COMPLETING IN LESS THAN MINUTES 9OU MOVE
THE SECOND DATABASE AND PERFORMANCE SLOWS DOWN A BIT BUT IS STILL MUCH BETTER THAN IT WAS ON THE OLD
SERVER 4HE QUERIES NOW COMPLETE IN A LITTLE LESS THAN MINUTES !S YOU PROCEED TO MIGRATE THE
REMAINING TWO DATABASES PERFORMANCE DECLINES EVEN FURTHER 4O AGGRAVATE THE SITUATION YOU NOW FIND
YOURSELF WITH MIXED WORKLOADS ALL COMPETING FOR THE SAME #05 RESOURCES DURING PEAK PERIODS OF THE DAY
4HIS IS A COMMON THEME IN DATABASE CONSOLIDATION PROJECTS 0ERFORMANCE STARTS OFF GREAT BUT DECLINES TO
A POINT WHERE YOU WONDER IF YOUVE MADE A BIG MISTAKE BRINGING SEVERAL DATABASES TOGETHER UNDER THE
SAME ROOF !ND EVEN IF OVERALL PERFORMANCE IS BETTER THAN IT WAS BEFORE THE PERCEPTION OF THE FIRST CLIENTS
TO BE MIGRATED IS THAT IT IS ACTUALLY WORSE ESPECIALLY DURING PEAK PERIODS OF THE DAY )F YOU HAD USED
INSTANCE CAGING TO SET THE #05 LIMIT FOR EACH DATABASE TO FOUR CORES WHEN THEY WERE MOVED RESPONSE
TIMES WOULD HAVE BEEN MUCH MORE STABLE
#ONFIGURING AND 4ESTING )NSTANCE #AGING
#ONFIGURING INSTANCE CAGING IS VERY SIMPLE !CTIVATING A RESOURCE PLAN AND SETTING THE NUMBER OF #05
CORES ARE ALL THAT IS REQUIRED 2ECALL THAT THE ACTIVE RESOURCE PLAN IS SET USING THE DATABASE PARAMETER
A4B>DA24N?;0= 4HE NUMBER OF #05S IS SET USING THE 2?DN2>D=C PARAMETER WHICH DETERMINES THE NUMBER
OF #05S THE INSTANCE MAY USE FOR ALL FOREGROUND AND BACKGROUND PROCESSES "OTH PARAMETERS ARE
DYNAMIC SO ADJUSTMENTS CAN BE MADE AT ANY TIME )N FACT SCHEDULING THESE CHANGES TO OCCUR
AUTOMATICALLY IS A VERY USEFUL WAY TO ADJUST DATABASE PRIORITIES AT VARIOUS TIMES OF THE DAY OR WEEK
ACCORDING TO THE NEEDS OF YOUR BUSINESS &OR EXAMPLE MONTH
END AND YEAR
END PROCESSING ARE CRITICAL
TIMES FOR ACCOUNTING SYSTEMS )F YOUR DATABASE SERVER IS BEING SHARED BY MULTIPLE DATABASES ALLOCATING
ADDITIONAL PROCESSING POWER TO YOUR FINANCIAL DATABASE DURING HEAVY PROCESSING CYCLES MIGHT MAKE A LOT
OF SENSE
#(!04%2 2%3/52#% -!.!'%-%.4
.OW LETS TAKE A LOOK AT INSTANCE CAGING IN ACTION &OR THIS EXAMPLE WELL USE OUR B2A0C27 AND B=855
DATABASES TO DEMONSTRATE HOW IT WORKS 4HESE ARE STANDALONE NON
2!# DATABASES RUNNING ON AN
%XADATA 6 DATABASE SERVER WITH TWO QUAD CORE #05S 4HE .EHALEM CHIPSET IS HYPER
THREADED SO THE
DATABASE ACTUALLY hSEESv VIRTUAL CORES OR #05 THREADS AS YOU CAN SEE IN THE FOLLOWING LISTING
BHB)B2A0C27-bW^f_PaP\TcTaR_dNR^d]c
=0<4CH?4E0;D4
R_dNR^d]cX]cTVTa %
.OTE -ANY #05 CHIPSETS TODAY IMPLEMENT HYPER THREADING 7HEN A #05 USES HYPER THREADING EACH #05
THREAD IS SEEN BY THE OPERATING SYSTEM AND SUBSEQUENTLY /RACLE DATABASE INSTANCES AS A SEPARATE #05 4HIS IS
WHY TWO QUAD CORE CHIPS APPEAR AS #05S RATHER THAN THE EXPECTED %XADATA 6 8 AND 8 MODELS FEATURE
CHIPSETS THAT EMPLOY HYPER THREADING SO FOR PURPOSES OF OUR DISCUSSION WE WILL USE THE TERMS CPU core AND CPU
threads SYNONYMOUSLY
7ELL BE USING THE BUILT IN RESOURCE PLAN 3450D;CN?;0= FOR THESE TESTS &IGURE SHOWS THE #05
RESOURCE ALLOCATION FOR THIS PLAN .OTE THAT UNDER THE STUPd[cN_[P] ALL USERS OTHER THAN BHB AND BHBC4<
WILL BE MAPPED TO THE >C74AN6A>D?B RESOURCE GROUP
BHB)B2A0C27-bW^f_PaP\TcTaaTb^daRTN_[P]
=0<4CH?4E0;D4
aTb^daRTN\P]PVTaN_[P]bcaX]V3450D;CN?;0=
Figure 7-4. DEFAULT_PLAN resource allocation
&OR THIS TEST WELL USE THE SAME SCRIPT WE USED FOR TESTING OUR $"2- RESOURCE PLANS IN THE PREVIOUS
SECTION !GAIN THE Qda]NR_d bW SCRIPT WITH A PARAMETER OF WILL SPIN UP CONCURRENT SESSIONS EACH
RUNNING THE TEST QUERY 4HIS SHOULD DRIVE THE #05 UTILIZATION UP TO APPROXIMATELY /NCE THE
SESSIONS ARE RUNNING WELL USE THE c^_ COMMAND TO SEE THE EFFECT INSTANCE CAGING HAS ON THE SERVER #05
LOAD ,ETS START OUT BY GETTING A BASELINE 4O DO THIS WELL RUN THE TEST WITH INSTANCE CAGING AND 2ESOURCE
-ANAGER TURNED OFF 2ECALL THAT THESE TESTS ARE RUNNING ON A QUARTER RACK %XADATA 6 WHICH IS CONFIGURED
WITH TWO QUAD
CORE HYPER
THREADED #05S 3O THE DATABASE INSTANCES SEE A 2?DN2>D=C OF
#(!04%2 2%3/52#% -!.!'%-%.4
-Qda]NR_d bWZ^bQ^a]Tg!
c^_ ')#') d_!SPhb%)$"#dbTab[^PSPeTaPVT) $ ( $ $ ! (
CPbZb)( "c^cP[25 running'&'b[TT_X]V bc^__TS i^\QXT
2_db)82.9%us 'bh ]X15.1%id fP WX !bX bc
!S YOU CAN SEE RUNNING THE Qda]NR_d bW SCRIPT DROVE THE #05 USAGE UP FROM A RELATIVELY IDLE
TO WITH RUNNING PROCESSES .OW LETS SEE WHAT HAPPENS WHEN WE RESET THE R_dNR^d]c TO
WHICH IS OF THE TOTAL #05 ON THE SERVER .OTICE THAT THE NUMBER OF RUNNING PROCESSES HAS DROPPED
FROM TO 4HE #05 TIME IN USER SPACE HAS DROPPED TO JUST OVER HALF OF WHAT IT WAS
BHB)B2A0C27-P[cTabhbcT\bTcR_dNR^d]c,'*
c^_ () $) d_!SPhb&)! #dbTab[^PSPeTaPVT)# '!$ $!' '
CPbZb)''&c^cP[10 running'&&b[TT_X]V bc^__TS i^\QXT
2_db)46.1%us &bh ]X52.3%id 'fP WX bX bc
.OW WELL SET THE 2?DN2>D=C PARAMETER TO 4HAT IS HALF OF THE PREVIOUS SETTING SO WE SHOULD SEE THE
#05 UTILIZATION DROP BY ABOUT !FTER THAT WELL DROP THE 2?DN2>D=C TO TO ILLUSTRATE THE DRAMATIC
EFFECT INSTANCE CAGING HAS ON DATABASE #05 UTILIZATION .OTICE THAT WHEN WE SET THE NUMBER OF #05S TO
OUR UTILIZATION DROPPED FROM TO &INALLY SETTING 2?DN2>D=C TO FURTHER REDUCES #05
UTILIZATION TO
BHB)B2A0C27-P[cTabhbcT\bTcR_dNR^d]c,#*
c^_ () #) "d_!SPhb&) '#dbTab[^PSPeTaPVT)! % $ $%( '
CPbZb)''%c^cP[5 running'' b[TT_X]V bc^__TS i^\QXT
2_db)25.1%us 'bh ]X74.1%id fP WX bX bc
BHB)B2A0C27-P[cTabhbcT\bTcR_dNR^d]c, *
c^_ () ()"!d_!SPhb&)!##dbTab[^PSPeTaPVT)# (&$ (& '
CPbZb)''#c^cP[2 running''!b[TT_X]V bc^__TS i^\QXT
2_db)4.8%us 'bh ]X94.0%id !fP WX bX bc
4HIS TEST ILLUSTRATED THE EFFECT OF INSTANCE CAGING ON A SINGLE DATABASE .OW LETS CONFIGURE TWO
DATABASES AND SEE HOW INSTANCE CAGING CONTROLS #05 RESOURCES WHEN MULTIPLE DATABASES ARE INVOLVED
)N THE NEXT TWO TESTS WELL ADD ANOTHER DATABASE TO THE MIX 4HE B=855 DATABASE IS IDENTICAL TO THE
B2A0C27 DATABASE WE USED IN THE PREVIOUS TEST )N THE FIRST OF THE NEXT TWO TESTS WELL RUN A BASELINE WITH
INSTANCE CAGING TURNED OFF BY SETTING 2?DN2>D=C SET TO IN BOTH DATABASES 4HE BASELINE WILL RUN
CONCURRENT COPIES OF THE TEST QUERY ON EACH DATABASE 7ELL LET IT RUN FOR A FEW MINUTES AND THEN TAKE A
LOOK AT THE #05 UTILIZATION OF THESE DATABASES AS WELL AS THE READINGS FROM THE c^_ COMMAND 4HE ACTIVE
RESOURCE PLAN FOR BOTH DATABASES IS SET TO 3450D;CN?;0= AND 2?DN2>D=C IS SET TO
JT]ZSQ !)B2A0C27L-Qda]NR_d bWZ^bQ^a]Tg %
JT]ZSQ !)B=855L-Qda]NR_d bWZ^bQ^a]Tg %
&IGURE
SHOWS A SUMMARY OF OUR SECOND TEST %ACH LINE REPRESENTING A SESSION FOREGROUND PROCESS
SHOWS THE PERCENTAGE OF ONE #05 CORE 4HIS IS SUMMED AND DIVIDED BY #05 CORES TO GET THE
PERCENTAGE OF TOTAL #05 CONSUMED !S EXPECTED THE DISTRIBUTION OF #05 BETWEEN OUR TWO DATABASES IS
APPROXIMATELY EQUAL AT AND ,OOKING AT THE 4OTAL #ONNECTION #05 WE CAN SEE THAT THE
DATABASES ACCOUNTED FOR ABOUT OF TOTAL #05 TIME FOR THE SERVER
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-5. Test summary: two databases, instance caging turned off
4HE SOURCE FOR THE DATA REFLECTED IN THIS SUMMARY WAS COLLECTED AS FOLLOWS
% of 1 Core: )NDIVIDUAL PROCESS #05 FROM THE 2?D COLUMN FROM THE c^_ COMMAND
% of Total CPU: 2ESULT OF OF #ORE CORES #05 THREADS
User: 2_db)]] ]]dbFROM THE c^_ COMMAND
Kernel: 2_db)]] ]]bh FROM THE c^_ COMMAND
Idle: 2_db)]] ]]XS FROM THE c^_ COMMAND
Total CPU: 3UM OF 5SER +ERNEL AND )DLE
Total Connection CPU: 3UM OF OF 4OTAL #05 FOR EACH DATABASE
Total Other + Idle CPU: 4OTAL #05 n 4OTAL #ONNECTION #05
SCRATCH %: OF 4OTAL #05B2A0C27 4OTAL #ONNECTION #05
SNIFF %: OF 4OTAL #05B=855 4OTAL #ONNECTION #05
4HE @B2A0C27 AND @B=855 NUMBERS ARE WHAT WERE INTERESTED IN 4HEY REPRESENT THE PERCENTAGE
OF TOTAL #05 USED BY ALL SESSIONS IN EACH OF THESE DATABASES !S YOU CAN SEE FROM THE SUMMARY THE
DATABASES WERE SPLIT AT APPROXIMATELY EACH
#(!04%2 2%3/52#% -!.!'%-%.4
.OW LETS RUN THE SAME LOAD TEST WITH INSTANCE CAGING CONFIGURED FOR A SPLIT ON THE NUMBER OF
CORES ASSIGNED TO B2A0C27 AND B=855 RESPECTIVELY )N THIS TEST B2A0C27 GETS #05S OF CORES AND
B=855 GETS #05S OF CORES
BHB)B2A0C27-P[cTabhbcT\bTcR_dNR^d]c, !*
BHB)B=855-P[cTabhbcT\bTcR_dNR^d]c,#*
&IGURE
SHOWS THE RESULTS OF OUR SECOND TEST 4HE SPLIT ISNT PERFECT )T IS CLOSER TO AN SPLIT
.OT CAPTURED IN THESE TESTS WAS THE AMOUNT OF #05 CONSUMED BY ALL THE DATABASE BACKGROUND PROCESSES
SO THAT MAY ACCOUNT FOR THE SOME OF THE DIFFERENCE )T IS ALSO IMPORTANT TO UNDERSTAND THAT /RACLES
2ESOURCE -ANAGER OPERATES IN THE USER SPACE OF THE /3 PROCESS MODEL RATHER THAN THE KERNEL SPACE 3O
IT CANNOT DIRECTLY CONTROL THE AMOUNT OF #05 A PROCESS CONSUMES INSIDE THE KERNEL IT CAN ONLY INFLUENCE
THIS BY THROTTLING PROCESSES IN THE USER SPACE )N OUR SUMMARY WE SEE THAT THE B2A0C27 DATABASE SESSIONS
CONSUMED OF TOTAL SESSION #05 WHILE THE B=855 DATABASE USED OF TOTAL SESSION #05 %VEN
THOUGH THE SPLIT ISNT PERFECT IT DOES SHOW THAT INSTANCE CAGING MADE A SOLID EFFORT TO MANAGE THESE
DATABASES TO THE SPLIT WE DEFINED .OW THIS IS NOT TO SAY THAT INSTANCE CAGING LOCKS THE DATABASE
PROCESSES DOWN ON A SPECIFIC SET OF PHYSICAL #05 CORES !LL #05 CORES ARE STILL UTILIZED BY ALL DATABASE
BACKGROUND AND FOREGROUND PROCESSES 2ATHER INSTANCE CAGING REGULATES THE AMOUNT OF #05 TIME OF
#05 A DATABASE MAY USE AT ANY GIVEN TIME
Figure 7-6. Test summary: two databases, instance caging, 75% / 25% split
#(!04%2 2%3/52#% -!.!'%-%.4
/VER
0ROVISIONING
Over-provisioning REFERS TO THE PRACTICE OF ALLOCATING MORE #05S TO THE DATABASES THAN ARE ACTUALLY
INSTALLED IN THE SERVER 4HIS IS USEFUL WHEN YOUR SERVER HOSTS MULTIPLE DATABASES WITH COMPLEMENTING
WORKLOAD SCHEDULES &OR EXAMPLE IF ALL THE HEAVY PROCESSING FOR THE B2A0C27 DATABASE OCCURS AT NIGHT
AND THE B=855 DATABASE IS ONLY BUSY DURING 30HC8<4 HOURS IT WOULDNT MAKE SENSE TO ARTIFICIALLY LIMIT
THESE DATABASES TO #05 THREADS EACH ¾ ! BETTER #05 ALLOCATION SCHEME MIGHT BE MORE ON THE
ORDER OF ¾ 4HIS WOULD ALLOW EACH DATABASE TO FULLY UTILIZE CORES DURING BUSY PERIODS WHILE STILL
hRESERVINGv CORES FOR OFF
PEAK PROCESSING BY THE OTHER DATABASE $"!S WHO CONSOLIDATE MULTIPLE
DATABASES ONTO A SINGLE SERVER ARE AWARE THAT THEIR DATABASES DONT USE THEIR FULL #05 ALLOCATION
2?DN2>D=C ALL OF THE TIME /VER
PROVISIONING ALLOWS UNUSED #05 TO BE UTILIZED BY OTHER DATABASES RATHER
THAN SITTING IDLE /VER
PROVISIONING HAS BECOME A POPULAR WAY OF MANAGING #05 RESOURCES IN MIXED
WORKLOAD ENVIRONMENTS /BVIOUSLY OVER
PROVISIONING #05 INTRODUCES THE RISK OF SATURATING #05
RESOURCES +EEP THIS IN MIND IF YOU ARE CONSIDERING THIS TECHNIQUE "E SURE YOU UNDERSTAND THE WORKLOAD
SCHEDULES OF EACH DATABASE WHEN DETERMINING THE MOST BENEFICIAL #05 COUNT FOR EACH DATABASE
)NSTANCE CAGING LIMITS THE NUMBER OF #05 CORES A DATABASE MAY USE AT ANY GIVEN TIME ALLOWING
$"!S TO ALLOCATE #05 TO DATABASES BASED ON THE NEEDS AND PRIORITIES OF THE BUSINESS )T DOES THIS THROUGH
THE USE OF THE INSTANCE PARAMETER 2?DN2>D=C /UR PREFERENCE WOULD HAVE BEEN TO ALLOCATE #05 BASED ON
A PERCENTAGE OF #05 RATHER THE NUMBER OF CORES 4HIS WOULD GIVE THE $"! MUCH FINER
GRAINED CONTROL
OVER #05 RESOURCES AND WOULD BE ESPECIALLY USEFUL FOR SERVER ENVIRONMENTS THAT SUPPORT NUMEROUS
DATABASES "UT 2?DN2>D=C IS TIGHTLY COUPLED WITH /RACLES #OST "ASED /PTIMIZER #"/ WHICH USES THE
NUMBER OF #05S FOR ITS INTERNAL COSTING ALGORITHMS )T WOULDNT MAKE SENSE TO ALLOW THE $"! TO SET THE
PERCENTAGE OF PROCESSING POWER WITHOUT MATCHING THAT WITH THE VALUE THE #"/ USES FOR SELECTING OPTIMAL
EXECUTION PLANS )T PROBABLY WOULD HAVE BEEN A MUCH MORE DIFFICULT EFFORT TO IMPLEMENT SUCH A CHANGE
TO THE OPTIMIZER "E THAT AS IT MAY INSTANCE CAGING IS A POWERFUL NEW FEATURE THAT WEVE BEEN WAITING FOR
FOR A LONG TIME AND IS A MAJOR ADVANCEMENT DATABASE RESOURCE MANAGEMENT
I/O Resource Manager
%ARLIER IN THIS CHAPTER WE DISCUSSED /RACLES $ATABASE 2ESOURCE -ANAGER WHICH MANAGES #05
RESOURCES WITHIN A DATABASE THROUGH CONSUMER GROUPS AND PLAN DIRECTIVES 3ESSIONS ARE ASSIGNED TO
RESOURCE GROUPS AND PLAN DIRECTIVES MANAGE THE ALLOCATION OF RESOURCES BY ASSIGNING VALUES SUCH AS #05
PERCENTAGE TO RESOURCE MANAGEMENT ATTRIBUTES SUCH AS <6<CN? ' $"2- HOWEVER IS LIMITED TO
MANAGING RESOURCES WITHIN THE DATABASE
$"2- MANAGES )/ RESOURCES IN A SOMEWHAT INDIRECT MANNER BY LIMITING #05 AND PARALLELISM
AVAILABLE TO USER SESSIONS THROUGH CONSUMER GROUPS 4HIS IS BECAUSE UNTIL %XADATA CAME ALONG /RACLE
HAD NO PRESENCE AT THE STORAGE TIER %XADATA LIFTS )/ 2ESOURCE -ANAGEMENT ABOVE THE DATABASE TIER AND
MANAGES )/ AT THE STORAGE CELL IN A VERY DIRECT WAY $ATABASES INSTALLED ON %XADATA SEND )/ REQUESTS TO
RT[[bae ON THE STORAGE CELLS USING A PROPRIETARY PROTOCOL KNOWN AS )NTELLIGENT $ATABASE PROTOCOL I$"
5SING I$" THE DATABASE PACKS ADDITIONAL ATTRIBUTES IN EVERY )/ CALL TO THE STORAGE CELLS 4HIS ADDITIONAL
INFORMATION IS USED IN A NUMBER OF WAYS &OR EXAMPLE )/2- USES THE TYPE OF FILE REDO UNDO DATAFILE
CONTROL FILE AND SO ON FOR WHICH THE )/ WAS REQUESTED TO DETERMINE WHETHER CACHING THE BLOCKS IN FLASH
CACHE WOULD BE BENEFICIAL OR NOT 4HREE OTHER ATTRIBUTES EMBEDDED IN THE )/ REQUEST IDENTIFY THE
DATABASE THE CONSUMER GROUP AND THE CONSUMER GROUPS CATEGORY 4HESE THREE SMALL BITS OF ADDITIONAL
INFORMATION ARE INVALUABLE TO /RACLES INTELLIGENT STORAGE +NOWING WHICH DATABASE IS MAKING THE
REQUEST ALLOWS )/2- TO PRIORITIZE )/ REQUESTS BY DATABASE #ATEGORIES EXTEND THE CONCEPT OF CONSUMER
GROUPS ON %XADATA PLATFORMS #ATEGORIES ARE ASSIGNED TO CONSUMER GROUPS WITHIN THE DATABASE USING
$ATABASE 2ESOURCE -ANAGER #OMMON CATEGORIES DEFINED IN MULTIPLE DATABASES CAN THEN BE ALLOCATED
A SHARED )/ PRIORITY &OR EXAMPLE YOU MAY HAVE SEVERAL DATABASES THAT MAP USER SESSIONS TO AN
#(!04%2 2%3/52#% -!.!'%-%.4
8=C4A02C8E4 CATEGORY )/ REQUESTS COMING FROM THE 8=C4A02C8E4 CATEGORY MAY NOW BE PRIORITIZED OVER
OTHER CATEGORIES SUCH AS A4?>ACB 10C27 OR <08=C4=0=24
)/2- PROVIDES THREE DISTINCT METHODS FOR )/ RESOURCE MANAGEMENT )NTERDATABASE #ATEGORY AND
)NTRADATABASE 4HESE METHODS MAY BE USED INDIVIDUALLY OR IN COMBINATION &IGURE
ILLUSTRATES THE
RELATIONSHIP OF THESE THREE )/ RESOURCE MANAGEMENT METHODS
Exadata I/O
Resource
Management
Inside
One Across Databases
Database
Intradatabase Interdatabase Category
Resource Resource Resource
Plan Plan Plan
IORM Plan
Figure 7-7. Three methods for I/O resource management
Interdatabase IORM (Interdatabase Resource Plan): )/2- DETERMINES THE PRIORITY
OF AN )/ REQUEST BASED ON THE NAME OF THE DATABASE INITIATING THE REQUEST
)NTERDATABASE )/2- IS USEFUL WHEN %XADATA IS HOSTING MULTIPLE DATABASES AND YOU
NEED TO MANAGE )/ PRIORITIES AMONG THE DATABASES
IORM Categories (Category Resource Plan): )/2- DETERMINES THE PRIORITY OF AN
)/ REQUEST AMONG MULTIPLE DATABASES BY THE CATEGORY THAT INITIATED THE REQUEST
-ANAGING )/ BY CATEGORY IS USEFUL WHEN YOU WANT TO MANAGE )/ PRIORITIES BY
WORKLOAD TYPE &OR EXAMPLE YOU CAN CREATE CATEGORIES LIKE 0??B 10C27 A4?>ACB
<08=C4=0=24 IN EACH OF YOUR DATABASES AND THEN SET AN )/ ALLOCATION FOR THESE
CATEGORIES ACCORDING TO THEIR IMPORTANCE TO YOUR BUSINESS )F THE 0??B CATEGORY IS
ALLOCATED THEN SESSIONS ASSIGNED TO THE 0??B CATEGORY IN ALL DATABASES SHARE
THIS ALLOCATION
Intradatabase IORM (Intradatabase Resource Plan): 5NLIKE )NTERDATABASE AND
#ATEGORY )/2- )NTRADATABASE )/2- IS CONFIGURED AT THE DATABASE TIER USING
$"2- $"2- HAS BEEN ENHANCED TO WORK IN PARTNERSHIP WITH )/2- TO PROVIDE
FINE GRAINED )/ RESOURCE MANAGEMENT AMONG RESOURCE GROUPS WITHIN THE
DATABASE 4HIS IS DONE BY ALLOCATING )/ PERCENTAGE AND PRIORITY TO CONSUMER
GROUPS USING THE SAME MECHANISM USED TO ALLOCATE #05 THE <6<CN?] ATTRIBUTE &OR
EXAMPLE THE B0;4B DATABASE MAY BE ALLOCATED USING )NTERDATABASE )/2- 4HAT
#(!04%2 2%3/52#% -!.!'%-%.4
MAY BE FURTHER DISTRIBUTED TO THE 0??B A4?>ACB 10C27 AND >C74AN6A>D?B
CONSUMER GROUPS WITHIN THE DATABASE 4HIS ENSURES THAT )/ RESOURCES ARE AVAILABLE
FOR CRITICAL APPLICATIONS AND IT PREVENTS MISBEHAVING OR )/ INTENSIVE PROCESSES
FROM STEALING )/ FROM HIGHER PRIORITY SESSIONS INSIDE THE DATABASE
(OW )/2- 7ORKS
)/2- MANAGES )/ AT THE STORAGE CELL BY ORGANIZING INCOMING )/ REQUESTS INTO QUEUES ACCORDING THE
DATABASE NAME CATEGORY OR CONSUMER GROUP THAT INITIATED THE REQUEST )T THEN SERVICES THESE QUEUES
ACCORDING TO THE PRIORITY DEFINED FOR THEM IN THE RESOURCE PLAN )/2- ONLY ACTIVELY MANAGES )/
REQUESTS WHEN NEEDED 7HEN A CELL DISK IS NOT FULLY UTILIZED RT[[bae ISSUES )/ REQUESTS TO IT
IMMEDIATELY "UT WHEN A DISK IS HEAVILY UTILIZED RT[[bae INSTEAD REDIRECTS THE )/ REQUESTS TO THE
APPROPRIATE )/2- QUEUES AND SCHEDULES )/ FROM THERE TO THE CELL DISK QUEUES ACCORDING TO THE POLICIES
DEFINED IN YOUR )/2- PLANS &OR EXAMPLE USING OUR B2A0C27 AND B=855 DATABASES FROM EARLIER IN THIS
CHAPTER WE COULD DEFINE A )/ DIRECTIVE FOR THE B2A0C27 DATABASE AND A )/ DIRECTIVE FOR THE
B=855 DATABASE 7HEN THE STORAGE CELLS HAVE EXCESS CAPACITY AVAILABLE THE )/ QUEUES WILL BE SERVICED IN
A FIRST
IN
FIRST
OUT &)&/ MANNER $URING OFF
PEAK HOURS THE STORAGE CELLS WILL PROVIDE MAXIMUM
THROUGHPUT TO ALL DATABASES IN AN EVEN
HANDED MANNER "UT WHEN THE STORAGE CELL BEGINS TO SATURATE THE
B2A0C27 QUEUE WILL BE SCHEDULED OF THE TIME AND THE B=855 QUEUE WILL BE SCHEDULED OF THE TIME
)/ REQUESTS FROM DATABASE BACKGROUND PROCESSES ARE SCHEDULED ACCORDING TO THEIR RELATIVE PRIORITY TO
THE FOREGROUND PROCESSES CLIENT SESSIONS &OR EXAMPLE WHILE THE DATABASE WRITER PROCESSES 31Fn ARE
GIVEN PRIORITY EQUAL TO THAT OF FOREGROUND PROCESSES PERFORMANCE CRITICAL )/ REQUESTS FROM BACKGROUND
PROCESSES THAT MAINTAIN CONTROL FILES AND REDO LOG FILES ARE GIVEN HIGHER PRIORITY
.OTE )/2- ONLY MANAGES )/ FOR PHYSICAL DISKS )/ REQUESTS FOR OBJECTS IN THE FLASH CACHE OR ON FLASH
BASED
GRID DISKS ARE NOT MANAGED BY )/2-
IORM Architecture
&IGURE
ILLUSTRATES THE ARCHITECTURE OF )/2- &OR EACH CELL DISK RT[[bae #ELLSERV MAINTAINS AN )/2-
QUEUE FOR EACH CONSUMER GROUP AND EACH BACKGROUND PROCESS HIGH MEDIUM AND LOW PRIORITY FOR
EACH DATABASE ACCESSING THE STORAGE CELL "Y MANAGING THE FLOW OF )/ REQUESTS BETWEEN THE )/2-
QUEUES AND THE DISK QUEUES RT[[bae PROVIDES VERY EFFECTIVE )/ PRIORITIZATION AT THE STORAGE CELLS )/
REQUESTS SENT TO THE STORAGE CELL INCLUDE TAGS THAT IDENTIFY THE DATABASE AND THE CONSUMER GROUP ISSUING
THE REQUEST AS WELL AS THE TYPE OF )/ REDO CONTROL FILE AND SO ON &OR DATABASES THAT DO NOT HAVE AN
)NTRADATABASE RESOURCE PLAN DEFINED FOREGROUND PROCESSES ARE AUTOMATICALLY MAPPED TO THE CONSUMER
GROUP >C74AN6A>D?B 4HREE SEPARATE QUEUES ARE MAINTAINED FOR BACKGROUND PROCESSES SO THAT RT[[bae
MAY PRIORITIZE SCHEDULING ACCORDING TO THE TYPE OF )/ REQUEST &OR EXAMPLE REDO AND CONTROL FILE )/
OPERATIONS ARE SENT TO THE HIGH
PRIORITY QUEUE FOR BACKGROUND PROCESSES )/2- SCHEDULES )/ REQUESTS
FROM THE CONSUMER GROUP QUEUES ACCORDING TO THE )/ DIRECTIVES IN YOUR )/2- 0LAN
#(!04%2 2%3/52#% -!.!'%-%.4
SCRATCH - Cell Disk 4 Cellserv Storage Cell
SCRATCH - Cell Disk 3
SCRATCH - Cell Disk 2
SCRATCH - Cell Disk 1 IORM Plan
Foreground Queues
SCRATCH APPS
REPORTS
MAINTENANCE
IO request from
OTHER_GROUPS
the database
Background Queues Disk Queues
High Cell Disk
Medium
IO request tag: Low Cell Disk
o Db Name IORM
o Type SNIF - Cell Disk 4 Cell Disk
o Consumer Group
SNIF - Cell Disk 3
SNIF - Cell Disk 2 Cell Disk
SNIF - Cell Disk 1
IO request from
Foreground Queues
the database
APPS
REPORTS
SNIF Performance
MAINTENANCE
Statistics
OTHER_GROUPS
Background Queues
High
Medium
Low
Figure 7-8. IORM architecture
Limiting Excess I/O Utilization
/RDINARILY WHEN EXCESS )/ RESOURCES ARE AVAILABLE ALLOCATED BUT UNUSED BY OTHER CONSUMER GROUPS
)/2- ALLOWS A CONSUMER GROUP TO USE MORE THAN ITS ALLOCATION &OR EXAMPLE IF THE B2A0C27 DATABASE IS
ALLOCATED AT LEVEL IT MAY CONSUME )/ RESOURCES ABOVE THAT LIMIT IF OTHER DATABASES HAVE NOT FULLY
UTILIZED THEIR ALLOCATION 9OU MAY CHOOSE TO OVERRIDE THIS BEHAVIOR BY SETTING AN ABSOLUTE LIMIT ON THE )/
RESOURCES ALLOCATED TO SPECIFIC DATABASES 4HIS PROVIDES MORE PREDICTABLE )/ PERFORMANCE FOR MULTI
TENANT DATABASE SERVER ENVIRONMENTS 4HE ;8<8C )/2- ATTRIBUTE IS USED TO SET A CAP ON THE )/ RESOURCES
A DATABASE MAY USE EVEN WHEN EXCESS )/ CAPACITY IS AVAILABLE 4HE FOLLOWING LISTING SHOWS AN )/2-
PLAN THAT CAPS THE B2A0C27 DATABASE AT
P[cTaX^a\_[P]SQ?[P],
]P\T,B2A0C27[TeT[, P[[^RPcX^],% [X\Xc,'
]P\T,^cWTa[TeT[,!P[[^RPcX^],
"Y THE WAY MAXIMUM )/ LIMITS MAY ALSO BE DEFINED AT THE CONSUMER GROUP LEVEL BY USING THE
<0GNDC8;8I0C8>=N;8<8C ATTRIBUTE IN YOUR $"2- RESOURCE PLANS
#(!04%2 2%3/52#% -!.!'%-%.4
.OTE )N MOST CASES A SINGLE LEVEL )/ RESOURCE PLAN IS SUFFICIENT !S THEY DO WITH $"2- MULTI LEVEL )/2-
RESOURCE PLANS INCREASE THE COMPLEXITY OF MEASURING THE EFFECTIVENESS OF YOUR ALLOCATION SCHEME
7HEN USING MULTI LEVEL ALLOCATION SCHEMES ITS IMPORTANT TO UNDERSTAND THAT )/ RESOURCES ALLOCATED BUT UNUSED BY
A DATABASE CATEGORY OR CONSUMER GROUP ON LEVEL ARE IMMEDIATELY PASSED TO THE NEXT LEVEL &OR EXAMPLE IF YOU
HAVE DATABASES ! AND " ALLOCATED ON LEVEL AND DATABASE # IS ALLOCATED AT LEVEL THEN IF
DATABASE ! USES ONLY OF ITS ALLOCATION THE REMAINING IS PASSED TO DATABASE # $ATABASE " CANNOT
CAPITALIZE ON )/ RESOURCES ALLOCATED BUT UNUSED BY DATABASE ! BECAUSE ! AND " ARE ON THE SAME LEVEL 4HIS IS A
SUBTLE BUT IMPORTANT DISTINCTION OF MULTI LEVEL PLANS )F YOU ARE NOT CAREFUL YOU CAN FIND YOURSELF UNINTENTIONALLY
GIVING EXCESS )/ RESOURCES TO LESS IMPORTANT DATABASES AT LOWER LEVELS RATHER THAN MAKING THOSE RESOURCES
AVAILABLE TO YOUR HIGHER PRIORITY DATABASES ON LEVEL
Workload Optimization
4O OPTIMIZE )/ PERFORMANCE )/2- DISTINGUISHES BETWEEN SMALL AND LARGE )/ REQUESTS 2EQUESTS LESS
THAN + IN SIZE TYPICALLY ASSOCIATED WITH /,40 TRANSACTIONS ARE CATEGORIZED AS SMALL B< REQUESTS
2EQUESTS + IN SIZE OR GREATER WHICH ARE GENERALLY ASSOCIATED WITH $7 TRANSACTIONS ARE CATEGORIZED AS
LARGE ;6 REQUESTS 4HE DISTINCTION BETWEEN SMALL AND LARGE )/ REQUESTS IS IMPORTANT BECAUSE THE
ALGORITHMS USED FOR OPTIMIZING LOW
LATENCY )/ REQUESTS SMALL REQUESTS AND HIGH
THROUGHPUT )/
REQUESTS LARGE REQUESTS ARE POLAR OPPOSITES &OR /,40 TRANSACTIONS LOW LATENCY IS MOST IMPORTANT &OR
EXAMPLE CONSIDER AN END USER WAITING FOR A QUICK :)0 CODE SEARCH TO POPULATE THE SCREEN 4O PROVIDE
OPTIMIZED PERFORMANCE FOR LOW
LATENCY REQUESTS LARGE )/ REQUESTS MUST BE MANAGED IN SUCH A WAY THAT
THEY DONT FULLY CONSUME THE )/ RESOURCES OF THE STORAGE CELL #ONVERSELY $7 TRANSACTIONS REQUIRE HIGH
THROUGHPUT TO MAXIMIZE PERFORMANCE 4O OPTIMIZE )/ FOR THROUGHPUT THE STORAGE CELL MUST SERVICE
MANY CONCURRENT LARGE )/ REQUESTS SO THAT IT IS FULLY CONSUMED "Y COMPARING THE LARGE ;6 AND SMALL
B< )/ REQUESTS IN THE )/2- METRICS YOU CAN DETERMINE WHETHER YOUR DATABASES LEAN MORE TOWARD A
$7 WORKLOAD OR AN /,40 WORKLOAD 2ECENTLY /RACLE ADDED A NEW ATTRIBUTE TO )/2- KNOWN AS THE
OPTIMIZATION ^QYTRcXeT 4HIS ATTRIBUTE DETERMINES THE OPTIMIZATION MODE )/2- WILL USE WHEN
MANAGING CELL DISK )/ QUEUES 4HE OPTIONAL VALUES FOR THE )/2- ^QYTRcXeT ATTRIBUTE ARE AS FOLLOWS
low_latency: 4HIS SETTING PROVIDES OPTIMIZATION FOR APPLICATIONS THAT ARE EXTREMELY
SENSITIVE TO )/ LATENCY )T PROVIDES THE LOWEST POSSIBLE )/ LATENCIES BY
SIGNIFICANTLY LIMITING DISK UTILIZATION )N OTHER WORDS THROUGHPUT
HUNGRY
APPLICATIONS WILL BE SIGNIFICANTLY NEGATIVELY IMPACTED BY THIS OPTIMIZATION
OBJECTIVE
high_throughput: 4HIS SETTING PROVIDES THE BEST POSSIBLE THROUGHPUT FOR $7
TRANSACTIONS BY ATTEMPTING TO FULLY UTILIZE THE )/ CAPACITY OF THE STORAGE CELLS )T IS
THE OPPOSITE OF [^fN[PcT]Rh AND AS SUCH IT WILL SIGNIFICANTLY NEGATIVELY IMPACT
DISK )/ LATENCY
Balanced: 4HIS SETTING ATTEMPTS TO STRIKE A BALANCE BETWEEN LOW LATENCY AND HIGH
THROUGHPUT 4HIS IS DONE BY LIMITING DISK UTILIZATION FOR LARGE )/ OPERATIONS TO A
LESSER DEGREE THAN THE [^fN[PcT]Rh OBJECTIVE DESCRIBED ABOVE 5SE THIS OBJECTIVE
#(!04%2 2%3/52#% -!.!'%-%.4
WHEN WORKLOADS ARE MIXED AND YOU HAVE NO APPLICATIONS THAT REQUIRE EXTREMELY
LOW LATENCY
Auto: 4HIS SETTING ALLOWS )/2- TO DETERMINE THE BEST OPTIMIZATION OBJECTIVE FOR
YOUR WORKLOAD 2T[[bae CONTINUOUSLY MONITORS THE LARGESMALL )/ REQUESTS AND
APPLIES THE OPTIMIZATION METHOD ON A BEST
FIT BASIS )F OR MORE OF THE )/
OPERATIONS FROM A CONSUMER GROUP ARE SMALL )/ LESS THAN + THEN IT IS
CONSIDERED TO BE A LATENCY
ORIENTED CONSUMER GROUP AND IS MANAGED ACCORDINGLY
4HE DEFAULT SETTING FOR AN )/2- PLAN OBJECTIVE IS @ AN EMPTY STRING 4HE OBJECTIVE MAY BE SET USING THE
2T[[2;8 P[cTaX^a\_[P]^QYTRcXeTobjective COMMAND AS FOLLOWS
2T[[2;8-P[cTaX^a\_[P]^QYTRcXeT,[^fN[PcT]Rh
8>A<?;0=bdRRTbbUd[[hP[cTaTS
2T[[2;8-[XbcX^a\_[P]PccaXQdcTb^QYTRcXeT
[^fN[PcT]Rh
-ETHODS FOR -ANAGING )/ ON %XADATA
!S INTRODUCED EARLIER THERE ARE THREE METHODS FOR MANAGING )/ ON THE %XADATA PLATFORM )NTERDATABASE
)/2- #ATEGORY )/2- AND )NTRADATABASE )/2- !LL THREE ADDRESS SPECIFIC NEEDS FOR MANAGING )/
)NTERDATABASE )/2- ALLOCATES )/ RESOURCES BY DATABASE NAME #ATEGORY )/2- ALLOCATES )/ RESOURCES
BY COMMON CATEGORIES AMONG DATABASES AND )NTRADATABASE )/2- MANAGES )/ WITHIN THE DATABASE
USING $"2- RESOURCE GROUPS 9OU CAN CHOOSE TO USE ANY ONE OF THE METHODS FOR MANAGING YOUR )/
RESOURCES OR YOU COMBINE THEM TO IMPLEMENT A MORE SOPHISTICATED RESOURCE PLAN 7HEN THESE METHODS
ARE USED TOGETHER /RACLE SCHEDULES )/ IN THE FOLLOWING ORDER ALLOCATION AMONG CATEGORIES
ALLOCATION AMONG DATABASES AND ALLOCATION AMONG CONSUMER GROUPS WITHIN EACH CATEGORY WITHIN
EACH DATABASE 7ELL SHOW HOW ALL THIS WORKS A LITTLE LATER
Interdatabase I/O Resource Management
)NTERDATABASE )/ RESOURCE MANAGEMENT REFERS TO THE )/2- FEATURE THAT MANAGES )/ PRIORITY AMONG
MULTIPLE DATABASES BY DATABASE NAME )NTERDATABASE )/2- ALLOCATES )/ RESOURCES ACROSS MULTIPLE
DATABASES BY MEANS OF AN )/2- PLAN CONFIGURED ON EACH STORAGE CELL 4HE )/2- PLAN DEFINES ALLOCATION
PRIORITIES USING )/ DIRECTIVES %ACH STORAGE CELL MANAGES ITS OWN )/ QUEUES BASED ON THE LOCAL )/2-
PLAN DEFINED IN THE CELL )/2- PLANS SHOULD BE IDENTICAL FOR ALL STORAGE CELLS IN THE STORAGE GRID )F YOU
HAVE SPLIT YOUR %XADATA STORAGE CELLS INTO MULTIPLE STORAGE GRIDS THEN )/2- PLANS MAY DIFFER FROM GRID TO
GRID BUT THEY SHOULD STILL BE IDENTICAL AMONG STORAGE CELL MEMBERS WITHIN EACH STORAGE GRID 7E DISCUSS
CONFIGURING MULTIPLE STORAGE GRIDS IN #HAPTER &IGURE ILLUSTRATES HOW )/ ALLOCATION POLICIES ARE
ORGANIZED WITHIN THE )/2- PLAN
#(!04%2 2%3/52#% -!.!'%-%.4
)/ 2ESOURCE 0LAN
)/ 2ESOURCE $IRECTIVES
Level 1 I/O Level 2 I/O Level 3 I/O
Database
Allocation Allocation Allocation
SCRATCH 60%
SNIFF 80%
OTHER 100%
Figure 7-9. Interdatabase IORM plan
)N &IGURE OUR B2A0C27 DATABASE THE HIGHEST PRIORITY DATABASE IS GUARANTEED OF THE TOTAL )/ ON
LEVEL /N LEVEL OUR B=855 DATABASE TAKES ITS SLICE OF THE REMAINING OF UNALLOCATED )/ FROM
LEVEL !NY UNUSED )/ FROM LEVEL ALSO CASCADES DOWN TO LEVEL )N ADDITION TO ANY UNCONSUMED )/
FROM LEVELS AND DATABASES ON LEVEL WILL EQUALLY SHARE THE UNALLOCATED OF TOTAL )/ ¾
.OTE THAT ANY DATABASES THAT DO NOT EXPLICITLY APPEAR IN THE ACTIVE )/ RESOURCE PLAN WILL
AUTOMATICALLY BE ASSIGNED TO THE >C74A GROUP
#ONFIGURING )NTERDATABASE )/2-
)NTERDATABASE )/ 2ESOURCE -ANAGER IS CONFIGURED USING AN )/2- PLAN 4HIS PLAN DETERMINES THE
DATABASE )/ PRIORITIES FOR THE STORAGE CELL )/2- PLANS ARE CREATED USING THE 2T[[2;8 COMMAND 0;C4A
8>A<?;0= 4HERE CAN BE ONLY ONE )/2- PLAN PER STORAGE CELL REGARDLESS OF HOW MANY !3- INSTANCES
CLUSTERED OR SINGLE INSTANCE USE IT FOR STORAGE #REATING AN )NTERDATABASE )/2- PLAN IS FAIRLY SIMPLE 4HE
FIRST STEP IS TO DETERMINE WHAT YOUR ALLOCATION POLICY SHOULD BE FOR EACH DATABASE 9OU WILL USE THESE
ALLOCATION POLICIES TO DEFINE THE DIRECTIVES FOR YOUR )/2- PLAN 4HE ;4E4; ATTRIBUTE SPECIFIES THE PRIORITY A
DATABASE SHOULD BE GIVEN RELATIVE TO OTHER DATABASES IN THE PLAN 4HE 0;;>20C8>= ATTRIBUTE DETERMINES
THE PERCENTAGE OF )/ A DATABASE WILL BE GIVEN OUT OF THE TOTAL )/ AVAILABLE ON ITS LEVEL 4HE FOLLOWING
EXAMPLE DEMONSTRATES HOW YOU TO CREATE AN )/2- PLAN TO IMPLEMENT THE ALLOCATION POLICIES ILLUSTRATED
IN &IGURE
2T[[2;8-0;C4A8>A<?;0=
SQ?[P],
]P\T,B2A0C27[TeT[, P[[^RPcX^],%
]P\T,B=855[TeT[,!P[[^RPcX^],'
]P\T,^cWTa[TeT[,"P[[^RPcX^],
8>A<?;0=bdRRTbbUd[[hP[cTaTS
4HE 2T[[2;8 COMMAND [XbcX^a\_[P]STcPX[ DISPLAYS OUR NEW )/2- PLAN .OTICE THAT THE RPc?[P]
ATTRIBUTE IS EMPTY 4HIS IS A PLACEHOLDER FOR THE #ATEGORY )/2- PLAN WELL BE LOOKING IN THE NEXT TOPIC
2T[[2;8-[XbcX^a\_[P]STcPX[
]P\T)T]ZRT[ N8>A<?;0=
RPc?[P])
SQ?[P])]P\T,B2A0C27[TeT[, P[[^RPcX^],%
]P\T,B=855[TeT[,!P[[^RPcX^],'
]P\T,^cWTa[TeT[,"P[[^RPcX^],
bcPcdb)X]PRcXeT
#(!04%2 2%3/52#% -!.!'%-%.4
4HE AGGREGATE ALLOCATION FOR ALL DATABASES ON A LEVEL MAY NOT EXCEED )F THE SUM OF ALLOCATIONS ON
ANY LEVEL EXCEEDS 2T[[2;8 WILL THROW AN ERROR &OR EXAMPLE THE FOLLOWING LISTING SHOWS THE ERROR
2T[[2;8 PRODUCES WHEN OVER IS ALLOCATED ON LEVEL
2T[[2;8-0;C4A8>A<?;0=
SQ?[P],
]P\T,B2A0C27[TeT[, P[[^RPcX^],%
]P\T,B=855[TeT[, P[[^RPcX^],'
]P\T,^cWTa[TeT[,"P[[^RPcX^],
24;; %)8>A<?;0=R^]cPX]bP]X]eP[XSP[[^RPcX^]c^cP[
"EFORE OUR )NTERDATABASE )/2- 0LAN CAN MANAGE )/ RESOURCES IT MUST BE ACTIVATED USING THE 2T[[2;8
COMMAND 0;C4A8>A<?;0=02C8E4 &OR NOW WELL LEAVE IT TURNED OFF ,ATER IN THE h5NDERSTANDING )/2-
-ETRICSv SECTION OF THE CHAPTER WELL TEST OUR )NTERDATABASE )/2- 0LAN AND DISCUSS HOW THE EFFECTS OF
)/2- CAN BE MONITORED AT THE STORAGE CELLS
Category IORM
4HE )/ 2ESOURCE -ANAGER )/2- EXTENDS THE CONCEPT OF RESOURCE GROUPS WITH A NEW ATTRIBUTE KNOWN
AS A category 7HILE RESOURCE GROUPS ALLOW $"2- TO MANAGE RESOURCES WITHIN A DATABASE CATEGORIES
PROVIDE )/ RESOURCE MANAGEMENT AMONG MULTIPLE DATABASES &OR EXAMPLE SUPPOSE OUR TWO DATABASES
B2A0C27 AND B=855 HAVE SIMILAR WORKLOADS 4HEY BOTH HOST /,40 APPLICATIONS THAT DO SHORT TIME
SENSITIVE TRANSACTIONS $URING BUSINESS HOURS THESE TRANSACTIONS MUST TAKE PRIORITY 4HESE DATABASES
ALSO DO A FAIR AMOUNT OF BATCH PROCESSING SUCH AS RUNNING REPORTS AND MAINTENANCE JOBS 4HE BATCH
PROCESSING TAKES A LOWER PRIORITY DURING BUSINESS HOURS 4HESE TWO WORKLOADS CAN BE MANAGED AND
PRIORITIZED USING )/2- CATEGORIES 4HE CATEGORIES 0??BN20C46>AH AND 10C27N20C46>AH CAN BE DEFINED IN
BOTH DATABASES FOR HIGH
PRIORITY APPLICATIONS AND LONG
RUNNING LOWER
PRIORITY ACTIVITIES RESPECTIVELY )F
0??BN20C46>AH IS ALLOCATED ON LEVEL THEN NO MATTER HOW HEAVILY LOADED THE STORAGE GRID IS SESSIONS
ASSIGNED TO THIS CATEGORY FOR BOTH DATABASES WILL BE GUARANTEED A MINIMUM OF OF ALL )/
#ONFIGURING #ATEGORY )/2-
3ETTING UP #ATEGORY )/2- IS FAIRLY STRAIGHTFORWARD /NCE YOUVE CREATED YOUR $"2- CONSUMER GROUPS
YOU WILL CREATE CATEGORIES IN THE DATABASE AND ASSIGN THEM TO YOUR CONSUMER GROUPS 4HE FINAL STEP IS TO
CREATE AN )/2- PLAN IN THE STORAGE CELLS TO ESTABLISH )/ ALLOCATION AND PRIORITY FOR EACH CATEGORY 9OU
CAN DEFINE AS MANY AS EIGHT LEVELS IN YOUR #ATEGORY )/2- 0LAN &IGURE
ILLUSTRATES THE RELATIONSHIP
BETWEEN CONSUMER GROUPS CATEGORIES AND THE #ATEGORY )/2- 0LAN
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-10. IORM resource management by category
&OR THIS EXERCISE WELL CREATE TWO NEW CATEGORIES 0??BN20C46>AH AND 10C27N20C46>AH AND ASSIGN
THEM TO THE 0??B A4?>ACB AND <08=C4=0=24 RESOURCE GROUPS WE CREATED EARLIER IN THE CHAPTER 4HE
FOLLOWING LISTING CREATES OUR NEW CATEGORIES AND ASSIGNS THEM TO THE RESOURCE GROUPS 2EMEMBER THAT
YOU WILL NEED TO RUN THESE COMMANDS ON ALL DATABASES PARTICIPATING IN THE )/2- #ATEGORY 0LAN &OR
ILLUSTRATION PURPOSES WELL KEEP THE NUMBER OF CATEGORIES TO TWO 4HE A4?>ACB AND <08=C4=0=24 RESOURCE
GROUPS WILL BE ASSIGNED TO THE CATEGORY 10C27N20C46>AH
1468=
SQ\bNaTb^daRTN\P]PVTa R[TPaN_T]SX]VNPaTP*
SQ\bNaTb^daRTN\P]PVTa RaTPcTN_T]SX]VNPaTP*
2aTPcT2PcTV^aXTb
SQ\bNaTb^daRTN\P]PVTa RaTPcTNRPcTV^ah
RPcTV^ah,-0??BN20C46>AH
R^\\T]c,-2PcTV^ahU^a8]cTaPRcXeT0__[XRPcX^]b*
SQ\bNaTb^daRTN\P]PVTa RaTPcTNRPcTV^ah
RPcTV^ah,-10C27N20C46>AH
R^\\T]c,-AT_^acb<PX]cT]P]RT9^Qb*
0bbXV]2^]bd\Ta6a^d_bc^2PcTV^aXTb
SQ\bNaTb^daRTN\P]PVTa d_SPcTNR^]bd\TaNVa^d_
R^]bd\TaNVa^d_,-0??B
]TfNRPcTV^ah,-0??BN20C46>AH*
SQ\bNaTb^daRTN\P]PVTa d_SPcTNR^]bd\TaNVa^d_
R^]bd\TaNVa^d_,-A4?>ACB
]TfNRPcTV^ah,-10C27N20C46>AH*
SQ\bNaTb^daRTN\P]PVTa d_SPcTNR^]bd\TaNVa^d_
R^]bd\TaNVa^d_,-<08=C4=0=24
]TfNRPcTV^ah,-10C27N20C46>AH*
#(!04%2 2%3/52#% -!.!'%-%.4
SQ\bNaTb^daRTN\P]PVTa bdQ\XcN_T]SX]VNPaTP*
4=3*
?; B@;_a^RTSdaTbdRRTbbUd[[hR^\_[TcTS
4O CHECK YOUR RESOURCE
GROUP
TO
CATEGORY MAPPINGS QUERY THE 310NABA2N2>=BD<4AN6A>D?B VIEW AS
FOLLOWS .OTICE THAT THE >C74AN6A>D?B CONSUMER GROUP WAS ASSIGNED TO THE >C74A CATEGORY 4HAT MAPPING
IS CREATED AUTOMATICALLY BY /RACLE AND CANNOT BE ALTERED
BHB)B2A0C27-B4;42CR^]bd\TaNVa^d_RPcTV^ah
5A><310NABA2N2>=BD<4AN6A>D?B
F74A4R^]bd\TaNVa^d_
X]0??BA4?>ACB<08=C4=0=24>C74AN6A>D?B
>A34A1HRPcTV^ah*
2>=BD<4AN6A>D?20C46>AH
0??B0??BN20C46>AH
A4?>ACB10C27N20C46>AH
<08=C4=0=2410C27N20C46>AH
>C74AN6A>D?B>C74A
.OW WE CAN CREATE A NEW #ATEGORY )/2- 0LAN ON THE STORAGE CELLS AND SET )/ LIMITS ON THESE CATEGORIES
"EFORE WE DO THOUGH WELL DROP THE )NTERDATABASE )/2- 0LAN WE CREATED IN THE PREVIOUS EXAMPLE
2EMEMBER THAT EACH STORAGE CELL MAINTAINS ITS OWN )/2- PLAN SO YOU WILL NEED TO RUN THESE COMMANDS
ON EVERY CELL IN YOUR STORAGE GRID
2T[[2;8-P[cTaX^a\_[P]SQ_[P],
8>A<?;0=bdRRTbbUd[[hP[cTaTS
.OW WERE READY TO CREATE OUR #ATEGORY )/2- 0LAN 4HE FOLLOWING COMMAND CREATES A PLAN IN WHICH
0??BN20C46>AH AND 10C27N20C46>AH ARE ALLOCATED AND RESPECTIVELY OF THE TOTAL #ELL )/ AT LEVEL
4HE DEFAULT CATEGORY >C74A IS ALLOCATED ON LEVEL
2T[[2;8-P[cTaX^a\_[P]RPc_[P],
]P\T,0??BN20C46>AH[TeT[, P[[^RPcX^],&
]P\T,10C27N20C46>AH[TeT[, P[[^RPcX^],"
]P\T,>C74A[TeT[,!P[[^RPcX^],
8>A<?;0=bdRRTbbUd[[hP[cTaTS
!GAIN WELL USE THE 2T[[2;8 COMMAND [XbcX^a\STcPX[ AND CONFIRM THAT OUR #ATEGORY )/2- 0LAN
IS CONFIGURED THE WAY WE WANT IT
2T[[2;8-[XbcX^a\_[P]STcPX[
]P\T)T]ZRT[ N8>A<?;0=
RPc?[P])]P\T,0??BN20C46>AH[TeT[, P[[^RPcX^],&
]P\T,10C27N20C46>AH[TeT[, P[[^RPcX^],"
]P\T,>C74A[TeT[,!P[[^RPcX^],
#(!04%2 2%3/52#% -!.!'%-%.4
SQ?[P])
bcPcdb)X]PRcXeT
"ECAUSE WE DROPPED THE )NTERDATABASE PLAN FROM THE PREVIOUS EXERCISE THE SQ?[P] FIELD IS EMPTY
7ELL REINSTALL THAT PLAN AGAIN WHEN WE ARE READY TO TEST OUR RESOURCE PLANS .OTE THAT THE )/2- 0LAN IS
INACTIVE 7ELL LEAVE OUR #ATEGORY 0LAN INACTIVE FOR NOW ,ATER IN THE )/2- METRICS SECTION OF THE
CHAPTER WELL TEST OUR #ATEGORY )/2- 0LAN AND DISCUSS HOW THE EFFECTS OF )/2- CAN BE MONITORED AT THE
STORAGE CELLS
Intradatabase IORM
)N THE CASE OF )NTERDATABASE )/ RESOURCE MANAGEMENT RT[[bae IDENTIFIES )/ REQUESTS BY A TAG
CONTAINING THE DATABASE NAME ISSUING THE REQUEST #ATEGORY )/ RESOURCE MANAGEMENT WORKS IN A
SIMILAR WAY USING A TAG IN THE )/ REQUEST TO IDENTIFY THE CATEGORY ISSUING THE REQUEST 4HE THIRD AND
FINAL TYPE OF )/ RESOURCE MANAGEMENT AVAILABLE ON THE %XADATA PLATFORM IS )NTRADATABASE )/2-
)NTRADATABASE )/ RESOURCE MANAGEMENT IS CONFIGURED IN EACH DATABASE USING $"2-S MANAGEMENT
ATTRIBUTES <6<CN? ' 9OU MAY RECALL FROM EARLIER IN THE CHAPTER THAT THE <6<CN?n ATTRIBUTES ARE WHAT
$"2- USES TO MANAGE #05 RESOURCES $"2- USES #05 ALLOCATION DIRECTIVES TO DEFINE PRIORITIES AMONG
CONSUMER GROUPS WITHIN THE DATABASE "Y TYING )/ ALLOCATIONS TO #05 ALLOCATIONS )/2- CARRIES FORWARD
THESE PRIORITIES TO THE STORAGE LAYER THUS MAINTAINING A CONSISTENT PROVISIONING SCHEME FOR BOTH
#ONFIGURING )NTRADATABASE )/2-
)NTRADATABASE )/2- IS NOT TERRIBLY DIFFICULT TO IMPLEMENT )N FACT IN A MANNER OF SPEAKING WE HAVE
ALREADY DONE SO )N THE $"2- SECTION OF THIS CHAPTER WE CREATED CONSUMER GROUPS RESOURCE PLANS AND
PLAN DIRECTIVES THAT MANAGED #05 RESOURCES WITHIN THE B2A0C27 DATABASE /N THE %XADATA PLATFORM
WHEN A $"2- RESOURCE PLAN IS ACTIVATED THE DATABASE TRANSMITS A DESCRIPTION OF THE PLAN INCLUDING
<6<CN?n DIRECTIVES TO ALL CELLS IN THE STORAGE GRID 4HIS ALSO HAPPENS ANY TIME RT[[bae IS STARTED ON THE
STORAGE CELLS 3O WHETHER YOURE INSTALLING A NEW CELL INTO THE STORAGE GRID REBOOTING A CELL OR BOUNCING
RT[[bae SERVICES YOUR )NTRADATABASE RESOURCE PLAN WILL BE PUSHED TO THE CELL AUTOMATICALLY )F )/2- IS
ACTIVE ON THE CELL IT WILL GENERATE AN )NTRADATABASE 0LAN AND BEGIN MANAGING )/ PRIORITIES ACCORDING TO
YOUR RESOURCE DIRECTIVES &IGURE
ILLUSTRATES THE PROCESS OF CONFIGURING AND ACTIVATING )NTRADATABASE
)/2-
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-11. An Intradatabase I/O resource plan
)N &IGURE
WE SEE THE CALLS TO $"-3?2%3/52#%?-!.!'%2 THAT CREATED OUR RESOURCE PLAN AND
PLAN DIRECTIVES FOR THE B2A0C27 AND B=855 DATABASES 4HESE DIRECTIVES ARE USED TO ASSIGN #05 AND )/
ALLOCATION FOR THE RESOURCE GROUPS 0??B A4?>ACB <08=C4=0=24 AND >C74AN6A>D?B 7HEN THE 30HC8<4 PLAN
IS ACTIVATED ITS DEFINITION IS SENT TO RT[[bae ON THE STORAGE CELLS 7HEN RT[[bae RECEIVES THE RESOURCE
PLAN IT GENERATES AN )NTRADATABASE )/2- 0LAN CONFIGURES )/ QUEUES FOR EACH CONSUMER GROUP AND
BEGINS TO MANAGE )/ RESOURCES
"RINGING )T !LL 4OGETHER
!S DISCUSSED EARLIER EACH OF THE THREE )/2- MANAGEMENT METHODS MAY BE USED INDIVIDUALLY OR IN
COMBINATION ,ETS TAKE A LOOK AT HOW ALL THREE METHODS WORK TOGETHER TO MANAGE )/ .OW THIS IS THE
PART WHERE YOUR COLLEGE ACCOUNTING PROFESSOR WOULD SAY hEACH LESSON BUILDS ON THE LAST SO IF YOU SKIPPED
CLASS LAST WEEK YOU MAY FEEL A LITTLE LOST TODAYv )N THIS EXERCISE WELL BE USING ALL OF THE ELEMENTS OF
$"2- AND )/2- WEVE DISCUSSED IN THIS CHAPTER 4O AVOID CONFUSION AND SAVE YOU THE TROUBLE OF
#(!04%2 2%3/52#% -!.!'%-%.4
THUMBING THROUGH THE PAGES YOUVE ALREADY READ 4ABLE SHOWS A SUMMARY OF HOW WEVE CONFIGURED
/RACLE 2ESOURCE -ANAGER FOR THE B2A0C27 AND B=855 DATABASES IN THIS CHAPTER
Table 7-4. Summary of DBRM / IORM Resource Allocation
I/O Management Type Allocation Assignment
Category IORM
Categories Consumer Groups
0??BN20C46>AH AT LEVEL 0??B
10C27N20C46>AH AT LEVEL A4?>ACB <08=C4=0=24
>C74A AT LEVEL >C74AN6A>D?B
Interdatabase IORM
Databases Databases
B2A0C27 AT LEVEL B2A0C27
B=855 AT LEVEL B=855
>C74A AT LEVEL !LL OTHERS
Intradatabase IORM
Consumer Groups User Accounts
0??B AT LEVEL :>B1>A=4
A4?>ACB AT LEVEL A9>7=B>=
<08=C4=0=24 AT LEVEL C?>34A
>C74AN6A>D?B AT LEVEL !LL OTHER USER ACCOUNTS
4HERE IS A FIXED ORDER OF PRECEDENCE BETWEEN THE THREE )/2- METHODS WEVE DISCUSSED !S )/
REQUESTS TRAVERSE THE LAYERS OF THE )/2- STACK THEY ARE DIVIDED AND ALLOCATED BY THE NEXT METHOD !T THE
TOP OF THE )/2- HIERARCHY OF )/ RESOURCES ARE AVAILABLE BECAUSE THE STORAGE CELL IS NOT SATURATED
BY )/ REQUESTS 4HE NEXT LEVEL OF THE HIERARCHY IS #ATEGORY )/2- WHICH ALLOCATES )/ AT A SPLIT
BETWEEN THE CATEGORIES 0??BN20C46>AH AND 10C27N20C46>AH AT LEVEL !NY UNCONSUMED )/ BETWEEN
THESE TWO GROUPS IS AVAILABLE TO THE >C74A CATEGORY AT LEVEL
.EXT IN THE )/2- HIERARCHY IS THE )NTERDATABASE 0LAN !T THE DATABASE LEVEL THE ALLOCATED TO THE
0??BN20C46>AH CATEGORY IS FURTHER DIVIDED GIVING TO THE B2A0C27 DATABASE ON LEVEL AND TO THE
B=855 DATABASE AT LEVEL !NY OTHER DATABASE RECEIVES OF THE UNALLOCATED )/ FROM LEVEL AS WELL AS
ACCESS TO ANY )/ THAT WAS NOT CONSUMED BY THE B2A0C27 AND B=855 DATABASES ON LEVELS AND
4HE THIRD AND FINAL TIER OF THE )/2- HIERARCHY IS )NTRADATABASE )/2- /UR )NTRADATABASE PLAN
FURTHER DIVIDES )/ ALLOCATION INTO THE FOUR $"2- CONSUMER GROUPS WE CREATED IN THE DATABASES 4HE
0??B CONSUMER GROUP GETS TOP PRIORITY AT ON LEVEL 4HE A4?>ACB GROUP IS ALLOCATED AT LEVEL
<08=C4=0=24 AND >C74AN6A>D?B SHARE A SPLIT OF )/ THAT WAS UNALLOCATED OR UNCONSUMED AT LEVELS
AND
#(!04%2 2%3/52#% -!.!'%-%.4
.OW THIS IS NOT TO IMPLY THAT ALL RESOURCE MANAGEMENT METHODS MUST BE CONFIGURED TOGETHER 9OU
MAY INSTEAD CHOOSE TO USE JUST ONE OF THE METHODS OR ANY COMBINATION OF TWO FOR MANAGING )/ &OR
EXAMPLE IF WE HAD CONFIGURED #ATEGORY )/2- ALONE THEN AS ILLUSTRATED IN &IGURE
NO FURTHER
RESTRICTION OF )/ RESOURCES WOULD BE PERFORMED )/ RESOURCES WOULD BE SPLIT AT BETWEEN THE
CATEGORIES 0??BN20C46>AH AND 10C27N20C46>AH 3IMILARLY YOU MAY CHOOSE TO IMPLEMENT )NTERDATABASE
)/2- BY ITSELF IN WHICH CASE )/ WILL BE ALLOCATED STRICTLY AT A SPLIT BETWEEN SESSIONS IN THE B2A0C27
AND B=855 DATABASES RESPECTIVELY )F YOU HAVE ONLY ONE DATABASE AND NEED TO MANAGE DIFFERING
WORKLOADS THEN )NTRADATABASE )/2- IS YOUR ONLY OPTION 7HEN )NTRADATABASE )/2- IS USED BY ITSELF )/
IS MANAGED ACCORDING TO YOUR $"2- 0LAN DIRECTIVES ALONE &IGURE
SHOWS HOW )/ CALLS ARE ALLOCATED
AS THEY FILTER THROUGH EACH LAYER OF )/2-
No IORM
All User IO
100%
Category APPS_CATEGORY BATCH_CATEGORY
IORM
70% 30%
Interdatabase
IORM
60% 80% 60% 80%
Intradatabase
IORM
70%
50%
50%
70%
50%
50%
70%
50%
50%
70%
50%
!003 2043 -!).4 !003 2043 -!).4 !003 2043 -!).4 !003 2043 50%
-!).4
3#2!4#( 3.)&& 3#2!4#( 3.)&&
Figure 7-12. IORM method heirarchy
)N &IGURE
ALLOCATIONS FOR THE DEFAULT CONSUMER GROUP >C74AN6A>D?B AND THE DEFAULT CATEGORY >C74A
ARE NOT SHOWN BECAUSE OF SPACE CONSTRAINTS )N &IGURES
AND WE SEE THE CALCULATIONS USED TO
ALLOCATE )/ RESOURCES AS SHOWN IN &IGURE
4HE >C74AN6A>D?B CONSUMER GROUP AND >C74A CATEGORY
ARE SHOWN HERE ALONG WITH THEIR ALLOCATION &IGURES
THROUGH
SHOWS A TABULAR REPRESENTATION OF
THE RESOURCE PLANS ALONG WITH THEIR ALLOCATION PERCENTAGE !LSO DISPLAYED IN THESE EXAMPLES ARE THE
FORMULAS USED TO CALCULATE THE ALLOCATION
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-13. Category IORM calculations
Figure 7-14. Interdatabase IORM calculations
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-15. Intradatabase IORM calculations
)/2- -ONITORING AND -ETRICS
)/ PERFORMANCE METRICS ARE COLLECTED AND MAINTAINED FOR )/2- IN THE STORAGE CELL 4HESE METRICS
MAY BE USED TO DETERMINE THE EFFECTS YOUR )/2- PLAN HAS ON THE DATABASES CATEGORIES AND RESOURCE
GROUP RESOURCE DIRECTIVES YOU DEFINED IN YOUR ENVIRONMENT &OR EXAMPLE YOU CAN SEE HOW MUCH )/ A
PARTICULAR DATABASE IS USING COMPARED TO OTHER DATABASES "Y OBSERVING THE ACTUAL )/ DISTRIBUTION FOR
YOUR )/2- CONSUMER GROUPS YOU CAN DETERMINE WHETHER ADJUSTMENTS NEED TO BE MADE TO PROVIDE
ADEQUATE )/ RESOURCES FOR APPLICATIONS USING YOUR DATABASES )N THIS SECTION WELL LOOK AT HOW THESE
METRICS ARE ORGANIZED AND TAP INTO THE VALUABLE INFORMATION STORED THERE
Understanding IORM Metrics
7HEN AN )/2- PLAN IS DEFINED IN THE STORAGE CELL RT[[bae CREATES )/ CONSUMER GROUPS CORRESPONDING
TO THE )/ DIRECTIVES DEFINED IN THE )NTRADATABASE #ATEGORY AND )NTRADATABASE )/2- 0LANS &OR
EXAMPLE THE FOLLOWING )/ CONSUMER GROUPS WERE CREATED IN THE STORAGE CELLS FROM OUR )/2- PLAN FOR
THE B2A0C27 AND B=855 DATABASES
#(!04%2 2%3/52#% -!.!'%-%.4
)NTERDATABASE B2A0C27
B=855
>C74A
#ATEGORY 0??BN20C46>AH
10C27N20C46>AH
>C74A
)NTRADATABASE 0??B
A4?>ACB
<08=C4=0=24
>C74AN6A>D?B
%VERY SECONDS RT[[bae COLLECTS KEY PERFORMANCE METRICS FROM THE )/ QUEUES AND STORES THEM IN
THE <4CA822DAA4=C OBJECT .OTE THAT WE SAID object NOT table OR file 2T[[bae IS A *AVA PROGRAM WRITTEN BY
*AVA DEVELOPERS NOT 31, DEVELOPERS 3O UNFORTUNATELY THESE METRICS ARE NOT STORED IN THE FAMILIAR
TABULAR FORMAT $"!S HAVE COME TO KNOW AND LOVE )NSTEAD OF TABLES COLUMNS AND ROWS THESE METRICS
ARE STORED IN OBJECTS AND ATTRIBUTES 4HE FOLLOWING LISTING SHOWS THE STRUCTURE OF THE <4CA822DAA4=C OBJECT
ALONG WITH A FEW SAMPLE RECORDS
2T[[2;8-STbRaXQT\TcaXRRdaaT]c
]P\T
P[TacBcPcT
R^[[TRcX^]CX\T
\TcaXR>QYTRc=P\T
\TcaXRCh_T
\TcaXREP[dT
^QYTRcCh_T
2T[[2;8-;8BC<4CA822DAA4=C~
F74A4]P\T,2CN8>NA@N;6~
0=3^QYTRcch_T,8>A<N20C46>AH
0CCA81DC4B]P\T^QYTRcch_T\TcaXR>QYTRc=P\T\TcaXREP[dT
=P\T^QYTRcCh_T\TcaXR>QYTRc=P\T\TcaXREP[dT
2CN8>NA@N;68>A<N20C46>AH0??BN20C46>AH 8>aT`dTbcb
2CN8>NA@N;68>A<N20C46>AH10C27N20C46>AH % &(8>aT`dTbcb
2CN8>NA@N;68>A<N20C46>AH>C74A!#('8>aT`dTbcb
#(!04%2 2%3/52#% -!.!'%-%.4
.OTE -ETRICS ARE COLLECTED FOR )/ REQUESTS TO FLASH CACHE FLASH BASED GRID DISKS AND PHYSICAL DISKS GRID
DISKS (OWEVER )/2- ONLY MANAGES )/ RESOURCES FOR PHYSICAL DISKS SO METRICS FOR THE PHYSICAL DISKS WILL BE THE
FOCUS OF THIS DISCUSSION
7HEN A NEW COLLECTION IS RUN THE PREVIOUS SET OF METRICS IS MOVED TO THE <4CA8278BC>AH OBJECT
WHERE IT IS KEPT FOR A PERIOD OF TIME -ETRIC HISTORY IS RETAINED FOR THE NUMBER OF DAYS SPECIFIED IN THE
<4CA8278BC>AH30HB ATTRIBUTE OF CELL THE DEFAULT IS DAYS &OR EXAMPLE THE NEXT LISTING SHOWS THAT
CELLSRV WILL MAINTAIN A ROLLING HISTORY OF DAYS
2T[[2;8-[XbcRT[[PccaXQdcTb]P\T\TcaXR7Xbc^ah3Phb
T]ZRT[ &
)F YOU DETERMINE THAT DAYS IS NOT SUFFICIENT HISTORY FOR YOUR NEEDS YOU CAN MODIFY THE RETENTION PERIOD
WITH THE 2T[[2;8 COMMAND P[cTaRT[[ 4HE FOLLOWING LISTING SHOWS HOW YOU CAN CHANGE THE RETENTION
FOR YOUR STORAGE CELLS TO DAYS
2T[[2;8-P[cTaRT[[\TcaXR7Xbc^ah3Phb, #
2T[[T]ZRT[ bdRRTbbUd[[hP[cTaTS
2T[[2;8-[XbcRT[[PccaXQdcTb]P\T\TcaXR7Xbc^ah3Phb
T]ZRT[ #
7ORKLOAD -ANAGEMENT
)/2- METRICS PROVIDE INSIGHT INTO HOW RT[[bae IS ALLOCATING )/ RESOURCES AMONG THE CONSUMERS IN YOUR
STORAGE GRID 2T[[bae KEEPS TRACK OF )/ REQUESTS BROADLY CATEGORIZED AS hSMALL REQUESTSv AND hLARGE
REQUESTSv "Y COMPARING THE LARGE ;6 AND SMALL B< )/ REQUESTS IN THE )/2- METRICS YOU CAN
DETERMINE WHETHER YOUR DATABASES LEAN MORE TOWARD A $7 WORKLOAD HIGH THROUGHPUT OR AN /,40
WORKLOAD LOW LATENCY /RACLE RECENTLY ADDED A NEW OPTION TO THE )/2- PLAN CALLED ^QYTRcXeT THAT TELLS
)/2- TO OPTIMIZE THE CELL DISK )/ QUEUES FOR LOW LATENCY HIGH THROUGHPUT OR SOMETHING IN BETWEEN
4HE )/2- METRIC 8>A<N<>34 WAS ADDED TO CAPTURE THE OBJECTIVE IN USE DURING THE COLLECTION PERIOD "Y
COMPARING THE 8>A<N<>34 WITH THE ACTUAL WORKLOAD ON YOUR STORAGE CELLS YOU CAN DETERMINE WHETHER THE
CURRENT )/2- OBJECTIVE IS APPROPRIATE OR NOT &OR EXAMPLE IF YOU FIND THAT A MAJORITY OF )/ OPERATIONS
IN THE STORAGE CELLS ARE GREATER THAN + AND THUS LARGE ;6 THEN )/2- SHOULD BE SET TO OPTIMIZE FOR
HIGH THROUGHPUT ^QYTRcXeT,WXVWNcWa^dVW_dc /RACLE USES A THRESHOLD OF LARGE )/ WHEN THE )/2-
OBJECTIVE IS SET TO Pdc^ SO THAT SHOULD GIVE YOU AN IDEA OF WHAT IS MEANT BY hMAJORITYv &OR MORE
INFORMATION ON THE )/2- OBJECTIVE AND POSSIBLE SETTINGS REFER TO THE 7ORKLOAD /PTIMIZATION SECTION
EARLIER IN THIS CHAPTER 4HE 8>A<N<>34 METRIC CAN BE OBSERVED USING THE FOLLOWING 2T[[2;8 COMMAND
2T[[2;8-[Xbc\TcaXRRdaaT]cX^a\N\^ST
8>A<N<>34T]ZRT[
0OSSIBLE SETTINGS FOR THE 8>A<N<>34 ATTRIBUTE ARE AS FOLLOWS
v )/2- OBJECTIVE WAS SET TO [^fN[PcT]Rh
v )/2- OBJECTIVE WAS SET TO QP[P]RTS
v )/2- MODE WAS SET TO WXVWNcWa^dVW_dc
#(!04%2 2%3/52#% -!.!'%-%.4
!N 8>A<N<>34 RANGING BETWEEN AND OR BETWEEN AND INDICATES THE MODE WAS CHANGED DURING
THE POLLING PERIOD )T ALSO INDICATES THE PROXIMITY TO THE )/2- OBJECTIVE 4HIS MIGHT OCCUR FOR EXAMPLE IF
THE OBJECTIVE WAS SET TO Pdc^ AND DURING THE POLLING PERIOD THE WORKLOAD SHIFTED SIGNIFICANTLY FROM SMALL
)/ REQUESTS TO LARGE )/ REQUESTS "Y COMPARING THE 8>A<N<>34 METRIC OVER TIME YOU CAN DETERMINE HOW
VOLATILE YOUR WORKLOAD IS ! CONSTANTLY CHANGING WORKLOAD MAY GIVE INSIGHT INTO WHERE )/ 2ESOURCE
-ANAGEMENT IS NEEDED THE MOST
4HE METRICS WERE INTERESTED IN FOR )/2- MONITORING HAVE AN ^QYTRcCh_T OF 8>A<N30C010B4
8>A<N20C46>AH AND 8>A<N2>=BD<4AN6A>D? 4HESE METRICS ARE FURTHER ORGANIZED BY THE ]P\T ATTRIBUTE IN
THE <4CA822DAA4=C OBJECT 4HE NAME OF THE METRIC IS A CONCATENATION OF ABBREVIATIONS THAT INDICATE THE
TYPE OF )/ CONSUMER GROUP THE TYPE OF STORAGE DEVICE AND A DESCRIPTIVE NAME 4HE ELEMENTS OF THE ]P\T
ATTRIBUTE APPEAR AS FOLLOWS
jconsumer_typelNjdevice typelNjmetricl
7HERE consumer_type REPRESENTS THE )/2- RESOURCE GROUP AND IS ONE OF THESE
31 )NTERDATABASE )/2- 0LAN
2C #ATEGORY )/2- 0LAN
26 )NTRADATABASE )/2- 0LAN
!ND device_type IS THE TYPE OF STORAGE THAT SERVICED THE )/ REQUEST AND IS ONE OF THE FOLLOWING
52 FLASH CACHE
53 &LASH BASED GRID DISK
yz )F NEITHER OF THE ABOVE THEN THE METRIC REPRESENTS )/ TO PHYSICAL DISKS
4HE LAST PART OF THE ATTRIBUTE {metricl IS THE DESCRIPTIVE NAME OF THE METRIC 4HE METRIC NAME MAY BE
FURTHER QUALIFIED BY B<zOR ;6 INDICATING THAT IT REPRESENTS SMALL )/ REQUESTS OR LARGE )/ REQUESTS &OR
EXAMPLE
CG_FC_IO_RQ_LG: 4HE TOTAL NUMBER OF LARGE )/ REQUESTS SERVICED FROM FLASH
CACHE &# FOR $"2- CONSUMER GROUPS
CG_FD_IO_RQ_LG: 4HE TOTAL NUMBER OF LARGE )/ REQUESTS SERVICED FROM FLASH
BASED GRID DISKS &$ FOR $"2- CONSUMER GROUPS
CG_IO_RQ_LG: 4HE TOTAL NUMBER OF LARGE )/ REQUESTS SERVICED FROM PHYSICAL DISKS
GRID DISKS FOR $"2- CONSUMER GROUPS
4ABLE DESCRIBES EACH OF THE )/2- CONSUMER GROUP PERFORMANCE METRICS CAPTURED BY RT[[bae USING
THESE ABBREVIATIONS
#(!04%2 2%3/52#% -!.!'%-%.4
Table 7-5. Description of the Name Attribute of IORM METRICCURRENT
Name Description
jCGCTDBlN8>N1HNB42 -EGABYTES PER SECOND SCHEDULED FOR THIS )/ CONSUMER
jCGCTDBlN8>N;>03 !VERAGE )/ LOAD FOR THIS )/ CONSUMER
jCGCTDBlN8>NA@NjSMLGl 4HE CUMULATIVE NUMBER OF SMALL OR LARGE )/ REQUESTS FROM
THIS )/ CONSUMER
jCGCTDBlN8>NA@NjSMLGlNB42 4HE NUMBER OF SMALL OR LARGE )/ REQUESTS PER SECOND ISSUED
BY THIS )/ CONSUMER
jCGCTDBlN8>NFCNjSMLGl 4HE CUMULATIVE TIME IN MILLISECONDS THAT )/ REQUESTS FROM
THE )/ CONSUMER HAVE SPENT WAITING TO BE SCHEDULED BY
)/2-
jCGCTDBlN8>NFCNjSMLGlNA@ $ERIVED FROM j262C31lN8>NFCNjB<;6l ABOVE )T STORES THE
AVERAGE NUMBER OF WAITS IN MILLISECONDS THAT )/ REQUESTS
HAVE SPENT WAITING TO BE SCHEDULED BY )/2- IN THE PAST
MINUTE ! LARGE NUMBER OF WAITS INDICATES THAT THE )/
WORKLOAD OF THIS )/ CONSUMER IS EXCEEDING ITS ALLOCATION &OR
LOWER
PRIORITY CONSUMERS THIS MAY BE THE DESIRED EFFECT &OR
HIGH
PRIORITY CONSUMERS IT MAY INDICATE THAT MORE )/ SHOULD
BE ALLOCATED TO THE RESOURCE TO MEET THE OBJECTIVES OF YOUR
ORGANIZATION
jCGCTDBlN8>NDC8;NjSMLGl 4HE PERCENTAGE OF TOTAL )/ RESOURCES CONSUMED BY THIS )/
CONSUMER
!LL CUMULATIVE METRICS ABOVE ARE RESET TO WHENEVER CELLSRV IS RESTARTED THE )/2- PLAN IS ENABLED OR THE
)/2- PLAN CHANGES FOR THAT )/ CONSUMER GROUP &OR EXAMPLE IF THE #ATEGORY )/2- PLAN IS CHANGED
THE FOLLOWING CUMULATIVE METRICS WILL BE RESET
2CN8>NA@NB<
2CN8>NA@N;6
2CN8>NFCNB<
2CN8>NFCN;6
4HESE )/2- METRICS ARE FURTHER CATEGORIZED USING THE \TcaXR>QYTRc=P\T ATTRIBUTE )NTERDATABASE
RESOURCE PLAN METRICS FOR THE B2A0C27 AND B=855 DATABASES ARE STORED IN DETAIL RECORDS WHERE
\TcaXR>QYTRc=P\TIS SET TO THE CORRESPONDING DATABASE NAME )N A SIMILAR FASHION METRICS FOR #ATEGORY
)/2- 0LANS ARE IDENTIFIED WITH A \TcaXR>QYTRc=P\T MATCHING THE #ATEGORY NAME )/2- CONSUMER
GROUPS ARE IDENTIFIED BY A CONCATENATION OF THE DATABASE NAME AND THE NAME OF THE $"2- CONSUMER
#(!04%2 2%3/52#% -!.!'%-%.4
GROUP &OR EXAMPLE THE FOLLOWING LISTING SHOWS HOW THE )/ 2ESOURCE 'ROUPS FOR OUR B2A0C27 AND B=855
DATABASES WOULD BE REPRESENTED IN THE <4CA822DAA4=C AND <4CA8278BC>AH OBJECTS
^QYTRcCh_T\TcaXR>QYTRc=P\T
8>A<N30C010B4B2A0C27
8>A<N30C010B4B=855
8>A<N20C46>AH0??BN20C46>AH
8>A<N20C46>AH10C27N20C46>AH
8>A<N20C46>AH>C74A
8>A<N2>=BD<4AN6A>D?B2A0C27 0??B
8>A<N2>=BD<4AN6A>D?B2A0C27 <08=C4=0=24
8>A<N2>=BD<4AN6A>D?B2A0C27 A4?>ACB
8>A<N2>=BD<4AN6A>D?B2A0C27 >C74AN6A>D?B
8>A<N2>=BD<4AN6A>D?B=855 0??B
8>A<N2>=BD<4AN6A>D?B=855 <08=C4=0=24
8>A<N2>=BD<4AN6A>D?B=855 A4?>ACB
8>A<N2>=BD<4AN6A>D?B=855 >C74AN6A>D?B
Background Processes
!S WE DISCUSSED EARLIER DATABASE BACKGROUND PROCESSES ARE AUTOMATICALLY ASSIGNED TO BUILT
IN )/2-
CONSUMER GROUPS ACCORDING TO PRIORITY 4ABLE
SHOWS THESE SPECIAL )/2- CONSUMER GROUPS ALONG
WITH A DESCRIPTION OF WHAT THEY ARE USED FOR
Table 7-6. Built-In Consumer Groups for Background Processes
Consumer Group Description
N0B<N )/ RELATED TO /RACLE !3- VOLUME MANAGEMENT
N>A02;4N16N20C46>AHN (IGH PRIORITY )/ REQUESTS ISSUED FROM THE DATABASE
BACKGROUND PROCESSES
N>A02;4N<43?A816N20C46>AHN -EDIUM PRIORITY )/ REQUESTS ISSUED FROM THE DATABASE
BACKGROUND PROCESSES
N>A02;4N;>F?A816N20C46>AHN ,OW PRIORITY )/ REQUESTS ISSUED FROM THE DATABASE
BACKGROUND PROCESSES
Mining the Metrics
.OW THAT YOU HAVE A PRETTY GOOD IDEA OF WHAT TYPES OF METRICS ARE COLLECTED BY RT[[bae AND HOW TO GET TO
THEM LETS TAKE A LOOK AT HOW THIS INFORMATION CAN BE REPORTED AND INTERPRETED &IRST HOWEVER WE
SHOULD MENTION THAT AS A REPORTING TOOL 2T[[2;8 IS VERY LIMITED &OR EXAMPLE MULTIPLE FILTERS CAN BE
APPLIED TO YOUR SEARCH CRITERIA USING ,)+% OR !.$ BUT THE LACK OF AN /2 VERB AND THE LACK OF
PARENTHESES FOR SETTING PRECEDENCE MAKE IT SOMEWHAT CUMBERSOME IF NOT UTTERLY INADEQUATE FOR
REPORTING !LSO THERE IS NO AGGREGATION FUNCTIONALITY &OR EXAMPLE THERE IS NO WAY TO ROLL UP THE SMALL
#(!04%2 2%3/52#% -!.!'%-%.4
AND LARGE )/ REQUESTS FOR A DATABASE AND PRESENT THEM AS h4OTAL )/ 2EQUESTSv 4HE LONG AND SHORT OF IT
IS YOU ARE BETTER OFF TO JUST DUMP OUT ALL THE ROWS YOU ARE INTERESTED IN LOAD THEM INTO A DATABASE AND
USE 31,
0LUS TO REPORT THEM .OW LETS TAKE A LOOK AT SOME SIMPLE ;8BC COMMANDS FOR EXTRACTING )/2-
METRICS ALONG WITH A SAMPLE OF THE OUTPUT THEY GENERATE
2EPORTING )/2- $ATABASE -ETRICS
4HIS FIRST LISTING REPORTS )/ METRICS FOR OUR $ATABASE )/2- 0LAN ]P\T[XZT31N8>N ) 4HE
^QYTRcCh_T,y8>A<N30C010B4z ISNT REALLY NECESSARY WEVE JUST INCLUDED IT TO ILLUSTRATE THAT THIS IS THE
ONLY ^QYTRcCh_T STORED FOR THIS METRIC
2T[[2;8-;8BC<4CA822DAA4=C
F74A4]P\T;8:431N8>N
0=3^QYTRcCh_T,8>A<N30C010B4
0CCA81DC4B]P\T^QYTRcCh_T\TcaXR>QYTRc=P\T\TcaXREP[dTR^[[TRcX^]CX\T
31N8>N1HNB428>A<N30C010B4B2A0C27#$"<1 bTR! " !(C &) #)#% $)
31N8>N1HNB428>A<N30C010B4B<8C74AB &<1 bTR! " !(C &) #)#% $)
31N8>N1HNB428>A<N30C010B4B=855 <1 bTR! " !(C &) #)#% $)
31N8>N;>038>A<N30C010B4B2A0C27&! " !(C &) #)#% $)
2EPORTING )/2- #ATEGORY -ETRICS
4O REPORT )/ METRICS FOR OUR #ATEGORY 2ESOURCE 0LAN THE ]P\T FILTER IS CHANGED TO 2CN8>N !DDING AN
ADDITIONAL FILTER ON THE >QYTRc=P\T METRIC (]^c[XZTN ), ELIMINATES /RACLES AUTOMATICALLY
MAINTAINED CATEGORIES SUCH AS N0B<N, and N>A02;4N16N20C46>AHN.
2T[[2;8-;8BC<4CA822DAA4=C
F74A4]P\T;8:42CN8>N
0=3\TcaXR>QYTRc=P\T=>C;8:4N
0CCA81DC4B]P\T^QYTRcch_T\TcaXR>QYTRc=P\T\TcaXREP[dTR^[[TRcX^]CX\T
2CN8>NDC8;N;68>A<N20C46>AH0??BN20C46>AH ! " !(C %)"()#$ $)
2CN8>NDC8;N;68>A<N20C46>AH10C27N20C46>AH !! " !(C %)"()#$ $)
2CN8>NDC8;N;68>A<N20C46>AH>C74A "! " !(C %)"()#$ $)
2CN8>NDC8;NB<8>A<N20C46>AH0??BN20C46>AH ! " !(C %)"()#$ $)
2CN8>NDC8;NB<8>A<N20C46>AH10C27N20C46>AH ! " !(C %)"()#$ $)
2CN8>NDC8;NB<8>A<N20C46>AH>C74A ! " !(C %)"()#$ $)
2EPORTING )/2- #ONSUMER 'ROUP -ETRICS
4O REPORT )/ METRICS FOR OUR CONSUMER GROUPS WE REPLACE THE ]P\T FILTER WITH26N8>N WHICH DISPLAYS
)/ METRICS FOR CONSUMER GROUPS 4HE ADDITIONAL FILTERS FOR \TcaXR>QYTRc=P\T REMOVE OTHER DATABASES
FROM THE OUTPUT AS WELL AS CONSUMER GROUPS THAT /RACLE CREATES FOR BACKGROUND PROCESSES SUCH AS
N>A02;4N102:6A>D=3N6A>D?N 4HE R^[[TRcX^]CX\T WAS OMITTED FROM THE SAMPLE OUTPUT BECAUSE OF SPACE
CONSTRAINTS
#(!04%2 2%3/52#% -!.!'%-%.4
;8BC<4CA822DAA4=C
F74A4]P\T[XZT26N8>N
P]S^QYTRcCh_T,8>A<N2>=BD<4AN6A>D?
P]S\TcaXR>QYTRc=P\T[XZTB2A0C27
P]S\TcaXR>QYTRc=P\T]^c[XZTB2A0C27K N
0CCA81DC4B]P\T^QYTRcch_T\TcaXR>QYTRc=P\T\TcaXREP[dTR^[[TRcX^]CX\T
26N8>N;>038>A<N2>=BD<4AN6A>D?B2A0C27 0??B q
26N8>N;>038>A<N2>=BD<4AN6A>D?B2A0C27 <08=C4=0=24#q
26N8>N;>038>A<N2>=BD<4AN6A>D?B2A0C27 >C74AN6A>D?B#q
26N8>N;>038>A<N2>=BD<4AN6A>D?B2A0C27 A4?>ACB q
Testing IORM Plans
%ARLIER IN THE CHAPTER WE POSTPONED TESTING OUR )/2- PLANS .OW THAT YOU UNDERSTAND HOW TO MONITOR
)/2- WE CAN RUN A STRESS TEST AND TAKE A LOOK AT THE METRICS COLLECTED BY RT[[bae TO SEE WHAT EFFECT OUR
)/2- PLANS HAD ON PERFORMANCE &OR THIS TEST WELL GENERATE A LOAD ON THE STORAGE CELLS BY RUNNING
SEVERAL CONCURRENT FULL TABLE SCANS ON OUR B2A0C27 AND B=855 DATABASES 7ELL LET THEM RUN FOR A FEW
MINUTES AND THEN COLLECT THE DATABASE CATEGORY AND CONSUMER GROUP METRICS FOR OUR TEST DATABASES !S
WITH OUR $"2- TESTS EARLIER IN THIS CHAPTER VALIDATING THE PRECISE ALLOCATIONS FOR EACH )/ CONSUMER IS A
VERY COMPLICATED UNDERTAKING BECAUSE ANY )/ RESOURCES THAT ARE ALLOCATED BUT UNUSED ARE PASSED TO
OTHER CONSUMER GROUPS CATEGORIES AND DATABASES 4HIS IS PARTICULARLY TRUE WHEN IMPLEMENTING MULTI
LEVEL RESOURCE PLANS 4HE FOCUS OF THIS TEST WILL BE TO VERIFY THAT RT[[bae IS COLLECTING METRICS FOR EACH
DATABASE CATEGORY AND CONSUMER GROUP AND THAT THE RELATIVE )/ UTILIZATION NUMBERS FOR EACH
CONSUMER GROUP ARE WITHIN A REASONABLE RANGE CONSIDERING THEIR RESOURCE ALLOCATIONS 2EFER BACK TO
&IGURES THROUGH FOR THE )/ ALLOCATIONS FOR EACH OF OUR RESOURCE PLANS
4EST /UTLINE
.OW THAT WEgVE CREATED OUR )/ RESOURCE PLAN WE CAN TEST TO SEE HOW IT WORKS &OLLOWING ARE THE STEPS
WE WILL FOLLOW TO TEST OUR RESOURCE PLAN
1. !CTIVATE THE 30HC8<4 RESOURCE PLAN IN THE B2A0C27 AND B=855 DATABASES !LSO ACTIVATE
THE $ATABASE AND #ATEGORY )/2- PLANS IN EACH STORAGE CELL
2. )N BOTH DATABASES START THREE CONCURRENT FULL TABLE SCANS FROM EACH ACCOUNT THAT
MAPS TO OUR CONSUMER GROUPS 4HESE USER ACCOUNTS MAP TO THE CONSUMER GROUPS AS
FOLLOWS
:>B1>A=4 Æ 0??B
A9>7=B>= Æ A4?>ACB
C?>34A Æ <08=C4=0=24
5A43 Æ >C74AN6A>D?B
#(!04%2 2%3/52#% -!.!'%-%.4
3. !LLOW THE QUERIES TO RUN FOR A FEW MINUTES AND THEN WHILE THEY ARE STILL RUNNING WELL
DUMP THE )/2- METRICS FOR DATABASE CATEGORY AND CONSUMER GROUPS TO A FILE 4HIS
DATA WILL BE CLEANED UP AND LOADED INTO AN /RACLE DATABASE FOR REPORTING
3TEP !CTIVATE 2ESOURCE -ANAGER
"EFORE WE START THE LOAD TEST WE NEED TO VERIFY THAT OUR 30HC8<4 RESOURCE PLAN IS ACTIVATED IN THE
DATABASES
BHB)B2A0C27-bW^f_PaP\TcTaaTb^daRTN\P]PVTaN_[P]
=0<4CH?4E0;D4
aTb^daRTN\P]PVTaN_[P]bcaX]V30HC8<4
BHB)B=855-bW^f_PaP\TcTaaTb^daRTN\P]PVTaN_[P]
=0<4CH?4E0;D4
aTb^daRTN\P]PVTaN_[P]bcaX]V30HC8<4
FTP[b^]TTSc^c^RWTRZ^da8>A<?[P]^]TPRWbc^aPVTRT[[c^T]bdaTcWThPaTPRcXeT
2T[[2;8-[XbcX^a\_[P]STcPX[
]P\T)T]ZRT[ "N8>A<?;0=
RPc?[P])]P\T,0??BN20C46>AH[TeT[, P[[^RPcX^],&
]P\T,10C27N20C46>AH[TeT[, P[[^RPcX^],"
]P\T,>C74A[TeT[,!P[[^RPcX^],
SQ?[P])]P\T,B2A0C27[TeT[, P[[^RPcX^],%
]P\T,B=855[TeT[,!P[[^RPcX^],'
]P\T,^cWTa[TeT[,"P[[^RPcX^],
^QYTRcXeT)Pdc^
bcPcdb)active
3TEP 3TART ,OAD 4EST
.EXT WELL START OUR LOAD TEST BY RUNNING THE Qda]NX^ bW SCRIPT IN EIGHT SEPARATE TERMINAL SESSIONS ONE
FOR EACH USER ACCOUNT IN THE B2A0C27 AND B=855 DATABASES 4HIS SCRIPT IS SIMILAR TO THE Qda]NR_d bW SCRIPT
WE USED TO TEST OUR $"2- RESOURCE PLAN EARLIER 4HE SCRIPT TAKES THREE PARAMETERS DB4A=0<4 ?0BBF>A3
AND ?0A0;;4; 4HE ?0A0;;4; PARAMETER DETERMINES HOW MANY QUERIES WILL BE RUN CONCURRENTLY &OR OUR
TEST WELL RUN THREE SESSIONS FROM EACH USER ACCOUNT 4HE QUERY IS
bT[TRcSXbcX]RcbTV\T]cN]P\TUa^\Z^bQ^a]T QXVcPQ
4HE 31, SCRIPT TURNS OFF OUTPUT FROM THE QUERY bTcC4A<>DC^UU TO REDUCE THE OVERHEAD ON THE
DATABASE SERVER 4HE QXVcPQ TABLE CONTAINS MILLION REPEATING ROWS FROM THE SQPNbTV\T]cb VIEW
JT]ZSQ !)B2A0C27L W^\T aY^W]b^] \hbRaX_cb
-Qda]NX^ bW{kosborne, tpoder, rjohnson, fred}g"
#(!04%2 2%3/52#% -!.!'%-%.4
JT]ZSQ !)B=855L W^\T aY^W]b^] \hbRaX_cb
-Qda]NX^ bW{kosborne, tpoder, rjohnson, fred}g"
3TEP $UMP )/2- -ETRICS TO A &ILE
7ELL ALLOW THE STRESS TEST TO RUN FOR A WHILE AND THEN GATHER THE )/2- METRICS FROM THE STORAGE CELLS 4HE
FOLLOWING LISTING SHOWS THE 2T[[2;8 COMMAND FOR REPORTING )/2- METRICS 4HESE METRICS ARE MAINTAINED
ON EACH CELL SO WELL NEED TO RUN THIS COMMAND ON EACH STORAGE CELL AND MERGE THE RESULTS FOR FURTHER
ANALYSIS
2T[[2;8-b_^^[X^a\NT]ZRT[ N\TcaXRb SPc
2T[[2;8-;8BC<4CA822DAA4=C
F74A4^QYTRcCh_T;8:48>A<N
2T[[2;8-b_^^[b_^^[^UU
!NALYZING THE 2ESULTS
!FTER CLEANING UP THE OUTPUT FROM RT[[bae WE CAN LOAD IT INTO A TABLE IN ORDER TO RUN SOME QUERIES
AGAINST IT 4HE STRUCTURE OF THIS TABLE IS AS FOLLOWS
BHB)B2A0C27-STbRX^a\N\TcaXRb
=P\T=d[[.Ch_T
24;;N83E0A270A!'
=0<4E0A270A! (
>1942CNCH?4E0A270A! (
<4CA82N=0<4E0A270A!#
2>;;42C8>=NC8<4E0A270A!!$
<4CA82NE0;D4=D<14A
8=2E0A270A! '
&OR EASE IN REPORTING WELL CREATE A VIEW FOR EACH OF OUR THREE )/ CONSUMER GROUPS FILTERING OUT ACTIVITY
FROM !3- FLASH CACHE AND FLASH
BASED GRID DISKS AND BACKGROUND PROCESSES
8>A<2PcTV^ahEXTf
2A40C4>AA4?;0245>A24E84FA9>7=B>= 2CN8>A<N<4CA82B0B
B4;42C]P\T
>1942CNCH?4
<4CA82N=0<4
BD<<4CA82NE0;D4\TcaXRNeP[dT
<0G8=2X]R
5A><X^a\N\TcaXRb
F74A4]P\T;8:42CKN4B20?4K Å8>A<2PcTV^ahaTR^aSb
0=3]P\T=>C;8:42CKN52KN4B20?4K Å5[PbW2PRWTaTR^aSb
0=3]P\T=>C;8:42CKN53KN4B20?4K Å5[PbW QPbTSVaXSSXbZb
0=3\TcaXRN]P\T=>C;8:4KN4B20?4K Å0B<QPRZVa^d]S_a^RTbbTbTcR
6A>D?1H]P\T>1942CNCH?4<4CA82N=0<4
>A34A1H]P\T>1942CNCH?4<4CA82N=0<4*
#(!04%2 2%3/52#% -!.!'%-%.4
8>A<3PcPQPbTEXTfX]cTaSPcPQPbT8>A<\TcaXRb
2A40C4>AA4?;0245>A24E84FA9>7=B>= 31N8>A<N<4CA82B0B
B4;42C]P\T
>1942CNCH?4
<4CA82N=0<4
BD<C>N=D<14ACA8<<4CA82NE0;D4\TcaXRNeP[dT
<0G8=2X]R
5A><X^a\N\TcaXRb
F74A4]P\T;8:431KN4B20?4K Å8>A<3PcPQPbTaTR^aSb
0=3]P\T=>C;8:431KN52KN4B20?4K
0=3]P\T=>C;8:431KN53KN4B20?4K
0=3<4CA82N=0<48=B2A0C27B=855
6A>D?1H]P\T>1942CNCH?4<4CA82N=0<4
>A34A1H]P\T>1942CNCH?4<4CA82N=0<4*
8>A<2^]bd\Ta6a^d_EXTfX]caPSPcPQPbT8>A<\TcaXRb
2A40C4>AA4?;0245>A24E84FA9>7=B>= 26N8>A<N<4CA82B0B
B4;42C]P\T
>1942CNCH?4
<4CA82N=0<4
BD<<4CA82NE0;D4\TcaXRNeP[dT
<0G8=2X]R
5A><X^a\N\TcaXRb
F74A4]P\T;8:426KN4B20?4K Å8>A<2^]bd\Ta6a^d_aTR^aSb
0=3]P\T=>C;8:426KN52KN4B20?4K
0=3]P\T=>C;8:426KN53KN4B20?4K
0=3\TcaXRN]P\T;8:4B2A0C27>A\TcaXRN]P\T;8:4B=855
0=3\TcaXRN]P\T=>C;8:4KN4B20?4K
6A>D?1H]P\T>1942CNCH?4<4CA82N=0<4
>A34A1H]P\T>1942CNCH?4<4CA82N=0<4*
9OU MAY HAVE NOTICED THE BD<<4CA82NE0;D4 FUNCTION IN THE VIEW DEFINITIONS 7HILE IT IS IMPORTANT TO
UNDERSTAND THE WORKLOAD MIX OF YOUR DATABASES THE PURPOSE OF THIS EXERCISE IS TO VERIFY THE EFFECTIVENESS
OF OUR )/2- 2ESOURCE 0LAN 4O THAT END WELL SUBTOTAL THE SMALL AND LARGE )/ METRICS SO THAT WE CAN SEE
HOW ALL )/ IS DISTRIBUTED )MPORTING THE )/2- METRICS FROM OUR TABLE INTO A SPREADSHEET WE CAN CREATE
PIE GRAPHS FOR EACH )/2- CONSUMER GROUP 4HE RESULTING GRAPHS SHOULD REFLECT THE PROPORTIONS FOR EACH
)/2- CONSUMER IN OUR )/2- PLAN 4HERE ARE SEVERAL INTERESTING METRICS CAPTURED IN THE <4CA822DAA4=C
OBJECT AT THE STORAGE CELL BUT WELL FOCUS ON 8>N1HNB42 4HIS METRIC REPRESENTS THE )/ REQUESTS THAT WERE
SENT TO THE CELL DISK QUEUES FOR EXECUTION )T SHOULD GIVE US A PRETTY GOOD IDEA OF HOW )/ REQUESTS WERE
PRIORITIZED
/NE MORE THING BEFORE WE TAKE A LOOK AT THE RESULTS OF OUR TEST 7HILE THE FILTERS WE APPLIED TO OUR
VIEWS SIMPLIFIED THE OUTPUT THEY ALSO ELIMINATED SOME )/ FROM THE RESULTS &OR EXAMPLE THE
N>A02;4N102:6A>D=3N6A>D?N METRIC ACCOUNTED FOR A MEASURABLE AMOUNT OF )/ IN THE STORAGE CELLS BUT IS
NOT REFLECTED IN THE ANALYSIS 4HATS ACCEPTABLE THOUGH BECAUSE THE POINT OF THIS EXERCISE IS NOT TO
PRECISELY RECONCILE THE RESULTS OF OUR TEST WITH THE )/2- ALLOCATIONS CALLED FOR IN OUR PLAN DIRECTIVES A
VERY TEDIOUS PROCESS 2ATHER THE PURPOSE IS TO SHOW HOW )/ IS ACTUALLY ALLOCATED IN THE STORAGE CELLS
DURING PEAK PERIODS WHEN )/2- IS ACTIVE 3O WHILE THE RESULTS ARE EXPECTED TO RESEMBLE OUR )/2- PLANS
THEY ARE NOT EXPECTED TO MATCH PRECISELY
#(!04%2 2%3/52#% -!.!'%-%.4
#ATEGORY -ETRICS
)/2- PRIORITIZES )/ IN A HIERARCHICAL MANNER ILLUSTRATED IN &IGURE
/UR #ATEGORY 0LAN IS AT THE TOP
OF THE HIERARCHY SO THATS WHERE WELL START 2ECALL THAT OUR PLAN ALLOCATES )/ FOR THE 0??B AND 10C27
CATEGORIES AT A RATIO ON LEVEL !T LEVEL THE >C74A CATEGORY IS ALLOCATED OF THE UNUSED
)/ FROM THE 0??B AND 10C27 CATEGORIES &IGURE
SHOWS THE RATIO OF )/ REQUESTS SERVICED PER SECOND
IN MEGABYTES ON BEHALF OF EACH CATEGORY 2EMEMBER THAT THE LOAD FROM EACH CATEGORY WAS THE SAME
.OTICE THAT 0??BN20C46>AH RECEIVED APPROXIMATELY OF THE )/ WHILE 10C27N20C46>AH RECEIVED ABOUT
4HE >C74A CATEGORY RECEIVED THE REMAINING OF )/ 9OU MAY BE SURPRISED AT THE AMOUNT OF )/
THAT >C74A RECEIVED 4HIS WAS BECAUSE OF THE ALLOCATED BUT UNUSED )/ RESOURCES IT INHERITED FROM THE
0??B ANDOR 10C27 CATEGORIES AT LEVEL
Figure 7-16. I/O operations scheduled per second, by category
$ATABASE -ETRICS
.EXT IN THE )/2- HIERARCHY IS OUR )NTERDATABASE )/2- 0LAN 4HIS PLAN CALLS FOR THE B2A0C27 DATABASE TO
BE ALLOCATED OF THE )/ AT LEVEL AND THE B=855 DATABASE TO RECEIVE AT LEVEL &IGURE
SHOWS
THE RATIO OF )/ REQUESTS SCHEDULED ON BEHALF OF THE B2A0C27 AND B=855 DATABASES !GAIN THE NUMBER OF
REQUESTS COMING FROM THESE DATABASES WAS THE SAME !CCORDING TO THE METRICS COLLECTED DURING OUR
STRESS TEST )/ WAS ALLOCATED TO THE B2A0C27 AND B=855 DATABASES AT A RATIO OF RESPECTIVELY
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-17. I/O operations scheduled per second, by database
#ONSUMER 'ROUP -ETRICS
4HE FINAL LEVEL IN THE )/2- HIERARCHY IS OUR )NTRADATABASE )/2- 0LAN &IGURE
SHOWS THE ALLOCATION
DIRECTIVES FOR )/2- CONSUMER GROUPS IN OUR )NTRADATABASE )/2- 0LAN 2EMEMBER THAT OUR )/2-
CONSUMER GROUPS ARE TIED TO THE $"2- CONSUMER GROUPS THROUGH THE <6<CN? ' #05 ATTRIBUTES
Management Attributes
MGMT_P1 MGMT_P2 MGMT_P3
Consumer Level 1 CPU Level 2 CPU Level 3 CPU
Group Allocation Allocation Allocation
APPS 70%
Resource Directives
REPORTS 50%
MAINTENANCE 50%
OTHER_GROUPS 50%
Figure 7-18. IORM intradatabase plan
&IGURE
SHOWS THE ACTUAL RATIO OF )/ REQUESTS SCHEDULED FOR OUR )/2- CONSUMER GROUPS DURING
OUR STRESS TEST ! QUICK GLANCE AT THE PIE CHART TELLS US THAT OUR )/2- PLAN WAS DEFINITELY MANAGING )/
RESOURCES AT THE CONSUMER GROUP LEVEL 9OU CAN SEE THAT THE 0??B CONSUMER GROUP DOES IN FACT TAKE
PRIORITY OVER ALL OTHER GROUPS !S EXPECTED )/ IS DISPROPORTIONATELY ALLOCATED TO THE 0??B GROUP IN THE
B2A0C27 DATABASE COMPARED TO THE 0??B GROUP IN THE B=855 DATABASE 4HIS IS BECAUSE OF
THE INFLUENCE OUR )NTERDATABASE 2ESOURCE 0LAN HAS OVER OUR )NTRADATABASE 2ESOURCE 0LAN 2ECALL THAT OUR
)NTERDATABASE 0LAN GRANTS A HIGHER PRIORITY TO THE B2A0C27 DATABASE AT LEVEL THAN THE B=855
DATABASE AT LEVEL 4HE OTHER )/2- CONSUMER GROUPS FOLLOW A SIMILAR PATTERN
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-19. I/O operations scheduled per second, by consumer group
Other Processes Managed by IORM
4HERE ARE A FEW INTERNAL PROCESSES MANAGED BY )/2- THAT DONT QUITE FIT ANYWHERE ELSE IN THIS CHAPTER SO
WELL DISCUSS THEM HERE 4HESE INTERNAL PROCESSES ARE
v &AST &ILE #REATION
v %4, EXTRACT TRANSFORM LOAD
v 2ECOVERY -ANAGER
%XADATA PROVIDES A FEATURE CALLED &AST &ILE #REATION FOR CREATING AND EXTENDING DATABASE DATAFILES )T
PROVIDES EXTREMELY FAST )/ FOR CREATING AND GROWING FILES IN THE STORAGE GRID 'ENERALLY SPEAKING THIS IS A
GOOD THING "UT HOW DOES %XADATA MANAGE THE PRIORITY OF CERTAIN LOW PRIORITY HIGHLY )/ INTENSIVE
PROCESSES 4HE ANSWER IS )/2-
&AST &ILE #REATION
%XADATA PROVIDES A FEATURE CALLED Fast File Creation WHICH BRINGS ALL THE )/ RESOURCES OF THE STORAGE CELLS
TO BEAR IN ORDER TO MAKE QUICK WORK OF CREATING AND EXTENDING TABLESPACE DATABASE FILES "UT AS YOU
MIGHT IMAGINE IF RUN UNCHECKED &AST &ILE #REATION CAN STEAL VALUABLE RESOURCES FROM OTHER DATABASE
CRITICAL AND BUSINESS CRITICAL PROCESSES 4O PREVENT THIS THE 50BC58;42A4 FUNCTION WHICH PERFORMS &AST
&ILE #REATION IS MAPPED TO A HIDDEN )/2- CONSUMER GROUP AND ASSIGNED A PRIORITY BELOW THAT OF ALL
OTHER DATABASE FOREGROUND AND BACKGROUND PROCESSES 7HEN THE STORAGE CELLS ARE BUSY &AST &ILE
#(!04%2 2%3/52#% -!.!'%-%.4
#REATION IS THROTTLED BACK SO IT DOESNT DEGRADE OTHER PROCESSES "UT WHEN THE STORAGE CELLS ARE NOT BUSY
)/2- ALLOWS IT TO CONSUME ALL AVAILABLE )/ RESOURCES
)F YOU WANT TO INCREASE THE PRIORITY OF &AST &ILE #REATION ADD A NEW CONSUMER GROUP MAPPING RULE TO
SWITCH IT TO ANOTHER CONSUMER GROUP WITH HIGHER PRIORITY 4HE MAPPING RULE MUST USE THE MAPPING
ATTRIBUTE 31<BNA4B>DA24N<0=064A >A02;4N5D=2C8>= WITH A VALUE SET TO THE NAME OF THE &AST &ILE #REATION
FUNCTION 50BC58;42A4 &OR EXAMPLE WE CAN REASSIGN THE 50BC58;42A4 FUNCTION TO THE <08=C4=0=24
CONSUMER GROUP WE CREATED EARLIER IN THE CHAPTER AS FOLLOWS
1468=
31<BNA4B>DA24N<0=064A 2A40C4N?4=38=6N0A40*
31<BNA4B>DA24N<0=064A B4CN2>=BD<4AN6A>D?N<0??8=6
31<BNA4B>DA24N<0=064A >A02;4N5D=2C8>=
50BC58;42A4<08=C4=0=24*
31<BNA4B>DA24N<0=064A BD1<8CN?4=38=6N0A40*
4=3*
.OTE 4HE PRIORITY FOR EXTENDING EXISTING DATAFILES CANNOT BE MODIFIED 4HE MAPPING PROCEDURE DESCRIBED HERE
ONLY INCREASES PRIORITY FOR CREATING NEW TABLESPACES OR ADDING DATAFILES TO EXISTING TABLESPACES
%4,
$EPENDING ON THE DATABASE EXTRACT TRANSFORM AND LOAD %4, PROCESSES MAY BE CONSIDERED LOW
PRIORITY OR THEY MAY BE ABSOLUTELY
MISSION CRITICAL TO THE BUSINESS /RACLE PROVIDES TWO BUILT
IN $"2-
RESOURCE PLANS TO MANAGE THE PRIORITY FOR TYPICAL %4, PROCESSES 4HE 4C;N6A>D? IS SET UP WITH LOWER
PRIORITY IN THE BUILT
IN 3FN?;0= RESOURCE PLAN AND AS THE NAME IMPLIES IS GIVEN A MUCH HIGHER PRIORITY IN
THE BUILT
IN 4C;N2A8C820;N?;0= RESOURCE PLAN &IGURES
AND
SHOW THE DEFAULT SETTINGS FOR THESE
TWO RESOURCE PLANS
Figure 7-20. DW_PLAN resource plan
#(!04%2 2%3/52#% -!.!'%-%.4
Figure 7-21. ETL_CRITICAL_PLAN resource plan
$ATA 0UMP RUNS UNDER THE BUILT IN30C0;>03FUNCTION WHICH BY DEFAULT MAPS TO THE 4C;N6A>D?
CONSUMER GROUP /THER PROGRAMS SUCH AS 31, ,OADER MAY ALSO BE MAPPED TO THE 4C;N6A>D? USING
MAPPING RULES WE DISCUSSED EARLIER IN THE CHAPTER &OR EXAMPLE THE 31, ,OADER COMMAND IS MAPPED TO
THE 4C;N6A>D? CONSUMER GROUP AS FOLLOWS
1468=
31<BNA4B>DA24N<0=064A 2A40C4N?4=38=6N0A40*
31<BNA4B>DA24N<0=064A B4CN2>=BD<4AN6A>D?N<0??8=6
31<BNA4B>DA24N<0=064A 2;84=CN?A>6A0<B@;;3A4C;N6A>D?*
31<BNA4B>DA24N<0=064A BD1<8CN?4=38=6N0A40*
4=3*
/
2ECOVERY -ANAGER
%XADATA REQUIRES !3- FOR A DATABASE TO ACCESS THE STORAGE GRID !S SUCH 2ECOVERY -ANAGER A\P] IS THE
REQUIRED TOOL FOR EXECUTING DATABASE BACKUP AND RECOVERY "ACKUPS ARE A VERY )/
INTENSIVE OPERATION
&OR A NUMBER OF YEARS NOW A\P] HAS PROVIDED TWO WAYS TO MANAGE THE LOAD BACKUPS PUT ON THE )/
SUBSYSTEM BACKUP CHANNELS AND aPcT 4HE NUMBER OF BACKUP CHANNELS CAN BE USED TO INCREASE OR
DECREASE PERFORMANCE AND LOAD &INER
GRAINED CONTROL IS PROVIDED THROUGH THE aPcT PARAMETER WHICH
CONTROLS THE RATE IN BYTES PER SECOND THAT A\P] STREAMS TO THE BACKUP CHANNELS %XADATA PROVIDES )/2-
MANAGEMENT FOR A\P] BY MAPPING BACKUP OPERATIONS TO THE 102:D? FUNCTION AND COPY OPERATIONS TO THE
2>?H FUNCTION "Y DEFAULT THE 102:D? AND 2>?H FUNCTIONS MAP TO THE BUILT
IN 10C27N6A>D? CONSUMER
GROUP 3IMILAR TO THE 50BC58;42A4 FUNCTION WE DESCRIBED FOR &AST &ILE #REATION THE 102:D? P]S2>?H
FUNCTIONS MAY BE MAPPED TO ANY OTHER CONSUMER GROUP IN ORDER TO ADJUST THEIR )/2- PRIORITIES &OR
EXAMPLE WE CAN REMAP THE 102:D? FUNCTION TO OUR <08=C4=0=24 CONSUMER GROUP AS FOLLOWS
1468=
31<BNA4B>DA24N<0=064A 2A40C4N?4=38=6N0A40*
31<BNA4B>DA24N<0=064A B4CN2>=BD<4AN6A>D?N<0??8=6
31<BNA4B>DA24N<0=064A >A02;4N5D=2C8>=102:D?<08=C4=0=24*
31<BNA4B>DA24N<0=064A B4CN2>=BD<4AN6A>D?N<0??8=6
31<BNA4B>DA24N<0=064A >A02;4N5D=2C8>=2>?H<08=C4=0=24*
31<BNA4B>DA24N<0=064A BD1<8CN?4=38=6N0A40*
4=3*
#(!04%2 2%3/52#% -!.!'%-%.4
Summary
/NE OF THE BIGGEST CHALLENGES FOR $"!S IS EFFECTIVELY MANAGING SYSTEM RESOURCES TO MEET BUSINESS
OBJECTIVES /VER THE YEARS /RACLE HAS DEVELOPED A RICH SET OF FEATURES THAT MAKE RESOURCE MANAGEMENT A
REALITY 5NFORTUNATELY THESE FEATURES ARE RARELY IMPLEMENTED DUE TO THEIR COMPLEXITY "UT MAKE NO
MISTAKE AS SERVERS BECOME MORE POWERFUL AND EFFICIENT DATABASE CONSOLIDATION IS GOING TO BECOME
INCREASINGLY COMMON 4HIS IS ESPECIALLY TRUE OF THE %XADATA PLATFORM 5NDERSTANDING HOW TO LEVERAGE
DATABASE AND )/ RESOURCE MANAGEMENT IS GOING TO BECOME AN INCREASINGLY IMPORTANT TOOL FOR ENSURING
THAT YOUR DATABASES MEET THE DEMANDS OF YOUR BUSINESS 4HE BEST ADVICE WE CAN OFFER IS TO KEEP THINGS
SIMPLE !TTEMPTING TO MAKE USE OF EVERY BELL AND WHISTLE IN /RACLE 2ESOURCE -ANAGER CAN LEAD TO
CONFUSION AND UNDESIRABLE RESULTS )F YOU DO NOT HAVE A SPECIFIC NEED FOR MULTI
LEVEL RESOURCE PLANS THEN
STICK TO THE SINGLE
LEVEL APPROACH THEY ARE EASY TO IMPLEMENT AND MEASURE AND THEY WORK GREAT IN MOST
SITUATIONS #ATEGORY PLANS ARE ANOTHER RARELY NEEDED FEATURE ! MAJORITY OF THE SITUATIONS YOU WILL FACE
CAN BE RESOLVED BY IMPLEMENTING A SIMPLE SINGLE
LEVEL )NTERDATABASE 2ESOURCE 0LAN 3O AGAIN START
SMALL KEEP IT SIMPLE AND ADD FEATURES AS YOU NEED THEM
CHAPTER 8
Configuring Exadata
/RACLE OFFERS AN OPTIONAL SERVICE THAT HANDLES THE PROCESS OF INSTALLING AND CONFIGURING YOUR %XADATA
$ATABASE -ACHINE FROM START TO FINISH -ANY COMPANIES PURCHASE THIS SERVICE TO SPEED UP THEIR
IMPLEMENTATION TIME AND REDUCE THE COMPLEXITY OF INTEGRATING %XADATA INTO THEIR )4 INFRASTRUCTURE )F
YOURE READING THIS CHAPTER YOU MAY BE CONSIDERING PERFORMING THE CONFIGURATION YOURSELF OR PERHAPS
YOURE JUST INTERESTED IN GAINING A BETTER UNDERSTANDING OF HOW ITS DONE 4HE PROCESS WELL DISCUSS HERE
CLOSELY RESEMBLES THE INSTALLATION PROCESS /RACLE USES LARGELY BECAUSE WE WILL BE USING THE SAME UTILITY
/RACLE USES TO CONFIGURE %XADATA 4HE UTILITY IS CALLED /NE#OMMAND )T TAKES SITE
SPECIFIC INFORMATION
YOU PROVIDE AND PERFORMS THE ENTIRE CONFIGURATION FROM NETWORK TO SOFTWARE TO STORAGE 7HEN THE
PROCESS IS COMPLETE YOUR %XADATA $ATABASE -ACHINE WILL BE FULLY FUNCTIONAL INCLUDING A STARTER
DATABASE
Exadata Network Components
/RACLE DATABASE NETWORK REQUIREMENTS HAVE EVOLVED OVER THE YEARS AND WITH %XADATA YOU WILL NOTICE A
FEW NEW TERMS AS WELL AS THE ADDITION OF A NEW NETWORK 4RADITIONALLY /RACLE DATABASE SERVERS REQUIRED
ONE PUBLIC NETWORK LINK TO PROVIDE ADMINISTRATIVE ACCESS TYPICALLY 33( AND DATABASE ACCESS 31,
.ET
7ITH THE INTRODUCTION OF G2 'RID )NFRASTRUCTURE FORMERLY KNOWN AS /RACLE #LUSTERWARE /RACLE
COINED A NEW TERM FOR THIS NETWORK THE client access network /N THE %XADATA PLATFORM ADMINISTRATIVE
AND DATABASE TRAFFIC HAVE BEEN SEPARATED WITH THE CREATION OF A NEW NETWORK FOR ADMINISTRATIVE ACCESS
4HIS NEW NETWORK IS KNOWN AS THE management network 4HE CLIENT ACCESS NETWORK IS NO LONGER
ACCESSIBLE THROUGH 33( AND IS USED ONLY BY THE /RACLE LISTENER FOR INCOMING 31,
.ET CONNECTIONS )N
%XADATA TERMS AND MOSTLY IN THE CONTEXT OF CONFIGURATION THESE TWO NETWORKS ARE REFERRED TO AS .%4
MANAGEMENT NETWORK AND .%4 CLIENT ACCESS NETWORK
4HE NUMBER AND TYPE OF %THERNET PORTS ON THE COMPUTE NODES AND STORAGE CELLS VARIES BETWEEN THE
6 8 AND 8 MODELS OF %XADATA (ARDWARE SPECIFICATIONS FOR EACH MODEL ARE DETAILED IN THE
Exadata Database Machine Owner’s Guide !T A MINIMUM THOUGH ALL MODELS PROVIDE AT LEAST FOUR
EMBEDDED 'IGABIT %THERNET PORTS /RACLE IDENTIFIES THESE PORTS AS .%4 .%4 .%4 AND .%4 !S
NOTED .%4 IS USED FOR THE MANAGEMENT NETWORK AND .%4 IS USED FOR THE CLIENT ACCESS NETWORK )N
2!# ENVIRONMENTS IT IS A COMMON PRACTICE TO BOND TWO %THERNET DEVICES TOGETHER TO PROVIDE HARDWARE
REDUNDANCY FOR THE CLIENT ACCESS NETWORK /N %XADATA THIS IS DONE BY BONDING THE .%4 AND .%4 PORTS
TOGETHER 4HE .%4 INTERFACE IS UNASSIGNED AND AVAILABLE TO BE CONFIGURED AS AN OPTIONAL NETWORK
4HE -ANAGEMENT .ETWORK
%XADATA $ATABASE -ACHINES HAVE AN %THERNET SWITCH MOUNTED IN THE RACK THAT SERVICES THE MANAGEMENT
NETWORK 4HE MANAGEMENT NETWORK CONSISTS OF THE FOLLOWING LINKS
#(!04%2 #/.&)'52).' %8!$!4!
v /NE UPLINK FROM THE MANAGEMENT SWITCH TO YOUR COMPANYS MANAGEMENT
NETWORK
v /NE LINK TO .%4 ON EACH COMPUTE NODE AND STORAGE CELL
v /NE LINK TO THE ),/- ON EACH COMPUTE NODE AND STORAGE CELL
v /NE LINK FOR THE +6-
v /NE LINK FOR EACH OF THE TWO INTERNAL POWER DISTRIBUTION UNITS 0$5S 4HIS IS
OPTIONAL AND IS ONLY NEEDED IF YOU WANT TO MONITOR ELECTRICAL CURRENT REMOTELY
ILOM
)N ADDITION TO THE MANAGEMENT NETWORK INTERFACE .%4 COMPUTE NODES AND STORAGE CELLS COME
EQUIPPED WITH AN )NTEGRATED ,IGHTS /UT -ANAGER ),/- 4HE ),/- IS AN ADAPTER CARD IN EACH COMPUTE
NODE AND STORAGE CELL THAT OPERATES INDEPENDENTLY OF THE OPERATING SYSTEM 4HE ),/- BOOTS UP AS SOON
AS POWER IS APPLIED TO THE SERVER AND PROVIDES 7EB AND 33( ACCESS THROUGH THE MANAGEMENT NETWORK
4HE ),/- ALLOWS YOU TO PERFORM MANY OF THE TASKS REMOTELY THAT WOULD OTHERWISE REQUIRE PHYSICAL
ACCESS TO THE SERVERS INCLUDING GAINING ACCESS TO THE CONSOLE POWERING THE SYSTEM OFF AND ON AND
REBOOTING OR RESETTING THE SYSTEM !DDITIONALLY THE ),/- MONITORS THE CONFIGURATION AND STATE OF THE
SERVERS INTERNAL HARDWARE COMPONENTS 4HE ),/- IS LINKED VIA %THERNET PORT TO THE MANAGEMENT SWITCH
WITHIN THE %XADATA ENCLOSURE
KVM
%XADATA ALSO FEATURES AN INTERNAL +6- KEYBOARD VIDEO AND MOUSE FOR MANAGING THE SYSTEM !S YOU
MIGHT EXPECT THE +6- IS A SLIDE
OUT TRAY WITH A KEYBOARD TOUCH PAD AND FLIP
UP MONITOR ACCESSIBLE
FROM THE FRONT OF THE RACK 4HE +6- ALSO PROVIDES REMOTE 7EB ACCESS TO THE OPERATING SYSTEM CONSOLE OF
ALL COMPUTE NODES AND STORAGE CELLS ,IKE THE ),/- THE +6- IS LINKED VIA %THERNET PORT TO THE INTERNAL
MANAGEMENT SWITCH
.OTE 4HE 8 MODEL OF %XADATA DOES NOT COME WITH A +6- #OMPUTE NODES IN 6 AND 8 SYSTEMS ARE A 5
FORM FACTOR AND ON AN %XADATA FULL RACK CONFIGURATION TAKE UP A TOTAL OF EIGHT SLOTS 4HE COMPUTE NODES ON THE
8 OCCUPY FIVE SLOTS EACH FOR A TOTAL OF TEN SLOTS IN THE RACK DISPLACING THE 5 +6- SWITCH
4HE #LIENT !CCESS .ETWORK
4HE CLIENT ACCESS NETWORK IS USED BY THE /RACLE LISTENER TO PROVIDE 31,
.ET CONNECTIVITY TO THE
DATABASES 4HIS NETWORK HAS TRADITIONALLY BEEN REFERRED TO IN 2!# TERMINOLOGY AS THE public network
/NE OR TWO LINKS .%4 .%4 FROM EACH DATABASE SERVER COMPUTE NODE CONNECT DIRECTLY TO YOUR
CORPORATE SWITCH /RACLE RECOMMENDS BONDING .%4 AND .%4 TO PROVIDE HARDWARE REDUNDANCY FOR
CLIENT CONNECTIONS )F THESE PORTS ARE BONDED THEN EACH LINK SHOULD TERMINATE AT A SEPARATE SWITCH TO
PROVIDE NETWORK REDUNDANCY
#(!04%2 #/.&)'52).' %8!$!4!
4HE 0RIVATE .ETWORK
4HE PRIVATE NETWORK IS SERVICED BY THE INTERNAL )NFINI"AND )" SWITCHES 4HIS NETWORK MANAGES 2!#
)NTERCONNECT TRAFFIC CACHE FUSION HEARTBEAT AS WELL AS I$" TRAFFIC BETWEEN THE DATABASE GRID AND THE
STORAGE GRID 4HIS NETWORK IS SELF
CONTAINED WITHIN THE )NFINI"AND NETWORK SWITCH FABRIC AND HAS NO
UPLINK TO YOUR CORPORATE NETWORK 4HE NETWORK CONFIGURATION CAN BE FOUND IN THE XURUV XQ AND XURUV
XQ CONFIGURATION FILES IN THE TcR bhbR^]UXV ]Tcf^aZ bRaX_cb DIRECTORY 4HEY ARE CONFIGURED AS BONDED
DEVICES AND THE MASTER DEVICE FILE IS XURUV Q^]SXQ &OR EXAMPLE THE FOLLOWING LISTING SHOWS THE
)NFINIBAND NETWORK CONFIGURATION FILES FROM ONE OF THE DATABASE SERVERS IN OUR LAB .OTICE HOW THE
<0BC4A PARAMETER IS USED TO MAP THESE NETWORK DEVICES TO THE Q^]SXQ DEVICE
TcR bhbR^]UXV ]Tcf^aZ bRaX_cb XURUV Q^]SXQ
34E824,Q^]SXQ
DB4A2C;,]^
1>>C?A>C>,]^]T
>=1>>C,hTb
8?033A, (! %' ! %
=4C<0B:,!$$ !$$ !$$
=4CF>A:, (! %' !
1A>0320BC, (! %' ! !$$
1>=38=6N>?CB,\^ST,PRcXeT QPRZd_\XX\^], S^f]ST[Ph,$ d_ST[Ph,$ ]d\NVaPcNPa_,
8?E%8=8C,]^
<CD,%$$!
TcR bhbR^]UXV ]Tcf^aZ bRaX_cb XURUV XQ
34E824,XQ
DB4A2C;,]^
>=1>>C,hTb
<0BC4A,Q^]SXQ
B;0E4,hTb
1>>C?A>C>,]^]T
7>C?;D6,]^
8?E%8=8C,]^
2>==42C43N<>34,hTb
<CD,%$$!
TcR bhbR^]UXV ]Tcf^aZ bRaX_cb XURUV XQ
34E824,XQ
DB4A2C;,]^
>=1>>C,hTb
<0BC4A,Q^]SXQ
B;0E4,hTb
1>>C?A>C>,]^]T
7>C?;D6,]^
8?E%8=8C,]^
2>==42C43N<>34,hTb
<CD,%$$!
.OTICE THAT THE -45 SIZE FOR THE )" NETWORK DEVICES IS SET TO BYTES -45 STANDS FOR
-AXIMUM 4RANSMISSION 5NIT AND DETERMINES THE MAXIMUM SIZE OF A NETWORK PACKET THAT MAY BE
TRANSMITTED ACROSS THE NETWORK 4YPICAL %THERNET NETWORKS SUPPORT AN -45 SIZE OF UP TO BYTES )N
#(!04%2 #/.&)'52).' %8!$!4!
RECENT YEARS THE *UMBO &RAMES TECHNOLOGY HAS BECOME A POPULAR WAY TO IMPROVE DATABASE
PERFORMANCE BY REDUCING THE NUMBER OF NETWORK ROUND TRIPS REQUIRED FOR CACHE FUSION BETWEEN CLUSTER
NODES IN AN /RACLE 2!# CLUSTER #ONVENTIONALLY *UMBO &RAMES SUPPORTS AN -45 OF UP TO BYTES
"UT SOME IMPLEMENTATIONS MAY SUPPORT AN EVEN LARGER -45 SIZE
.OTE /NLY THE DATABASE SERVERS ARE CONFIGURED WITH A + -45 0RESUMABLY THIS IS TO BENEFIT 4#0)0 TRAFFIC
BETWEEN THE DATABASE SERVERS AND BETWEEN THE DATABASE SERVERS AND ANY EXTERNAL HOST THAT IS LINKED TO THE )"
SWITCH 9OU MAY BE SURPRISED TO KNOW THAT THE )" PORTS ON THE STORAGE CELLS ARE CONFIGURED WITH THE STANDARD
BYTE -45 SIZE 4HE LARGE -45 SIZE IS NOT NECESSARY ON THE STORAGE CELLS BECAUSE )/ BETWEEN THE DATABASE GRID
AND THE STORAGE GRID UTILIZES THE 2$3 PROTOCOL WHICH IS MUCH MORE EFFICIENT FOR DATABASE )/ AND BYPASSES THE
4#0)0 PROTOCOL STACK ALTOGETHER
About the Configuration Process
#ONFIGURING /RACLE DATABASE SERVERS HAS ALWAYS BEEN A MANUAL AND SOMEWHAT ERROR PRONE PROCESS
ESPECIALLY FOR 2!# ENVIRONMENTS %XADATA CAN BE CONFIGURED MANUALLY AS WELL BUT THE COMPLEXITIES OF
THE PLATFORM CAN MAKE THIS A RISKY UNDERTAKING /RACLE HAS GREATLY SIMPLIFIED THE CONFIGURATION PROCESS
BY PROVIDING A UTILITY CALLED /NE#OMMAND 4HIS TOOL USES INPUT PARAMETERS YOU PROVIDE AND IT CARRIES
OUT ALL OF THE LOW LEVEL TASKS FOR YOU %VEN SO GATHERING ALL THE RIGHT INFORMATION REQUIRED BY
/NE#OMMAND WILL LIKELY BE A COLLABORATIVE EFFORT ESPECIALLY WITH THE NETWORKING COMPONENTS &IGURE
ILLUSTRATES THE %XADATA CONFIGURATION PROCESS USING /NE#OMMAND
Transfer Configuration Parameters
Complete the Configuration to the DBM Configurator and Upload Parameter & Deployment
Worksheet Generate Parameter & Deployment Files to Exadata
1 2 Files. 3
Run CheckIP to verify Network Configure the Network Stage Grid Infrastructure and
Readiness using Firstboot Database Software
4 5 6
Run OneCommand
7
Figure 8-1. The configuration process
!S INDICATED IN &IGURE
THE FIRST STEP IN THE PROCESS IS TO FILL OUT THE CONFIGURATION WORKSHEET
INCLUDED IN THE %XADATA DOCUMENTATION SET 4HE PURPOSE OF THIS WORKSHEET IS TO AID YOU IN GATHERING ALL
THE INFORMATION YOU WILL NEED TO CONFIGURE YOUR SYSTEM 7HEN FILLING IN THE NETWORKING COMPONENTS OF
THE WORKSHEET YOU WILL NEED TO ENLIST THE HELP OF YOUR NETWORK ADMINISTRATOR TO RESERVE )0 ADDRESSES AND
#(!04%2 #/.&)'52).' %8!$!4!
REGISTER NEW NETWORK NAMES WITH THE DOMAIN NAME SERVER )TS IMPORTANT TO UNDERSTAND HOW THESE
CONFIGURATION SETTINGS ARE USED SO WELL SPEND A SUBSTANTIAL AMOUNT OF TIME DISCUSSING THEM IN h3TEP
4HE #ONFIGURATION 7ORKSHEETv
/NCE YOU HAVE COMPLETED THE CONFIGURATION WORKSHEET YOU WILL TRANSFER THAT INFORMATION TO THE
$"- #ONFIGURATOR SQ\NR^]UXVdaPc^a g[b 4HIS IS AN %XCEL SPREADSHEET THAT GENERATES ALL THE PARAMETER
AND DEPLOYMENT FILES /NE#OMMAND NEEDS FOR CONFIGURING YOUR SYSTEM )N ADDITION TO PARAMETER AND
DEPLOYMENT FILES THE $"- #ONFIGURATOR GENERATES THE RWTRZX_ bW SCRIPT YOU SHOULD USE TO VERIFY YOUR
NETWORK SETTINGS 4HIS SCRIPT VALIDATES THE FORMAT OF THE )0 ADDRESSES YOU ENTERED CHECKS TO SEE IF ANY OF
THEM ARE ALREADY IN USE AND VERIFIES THAT THE NETWORK NAMES YOU SPECIFIED CAN BE RESOLVED BY YOUR $.3
SERVER /NCE THIS IS DONE YOU ARE READY TO UPLOAD THESE FILES TO %XADATA
"EFORE RUNNING /NE#OMMAND YOU WILL NEED TO STAGE THE INSTALLATION MEDIA FOR THE 'RID
)NFRASTRUCTURE AND THE DATABASE INCLUDING ANY /RACLE
PRESCRIBED PATCHES 4HE FINAL STEP OF THE PROCESS
IS TO EXECUTE /NE#OMMAND )TS OPERATION CONSISTS OF MULTIPLE STEPS THAT CONFIGURE ALL THE COMPONENTS OF
YOUR %XADATA $ATABASE -ACHINE 4HE TOP
LEVEL SCRIPT THAT RUNS /NE#OMMAND IS CALLED ST_[^h ! bW
PRESUMABLY STANDS FOR G2 4HIS SCRIPT CAN BE RUN END
TO
END EXECUTING ALL OF THE STEPS
AUTOMATICALLY OR YOU MAY SPECIFY A STEP TO RUN USING COMMAND
LINE OPTIONS 7E STRONGLY RECOMMEND
RUNNING THESE STEPS ONE AT A TIME $OING SO MAKES IT MUCH EASIER TO TROUBLESHOOT IF A STEP FAILS TO
COMPLETE SUCCESSFULLY
.OTE 2ATHER THAN CREATE AN %XADATA SPECIFIC RELEASE OF THEIR 'RID )NFRASTRUCTURE !3- AND 2$"-3 PRODUCTS
/RACLE CHOSE TO INTEGRATE THE %XADATA SPECIFIC CODE RIGHT INTO THE SAME G2 PRODUCT YOU WOULD INSTALL ON NON
%XADATA PLATFORMS 7HEN THEY DID THIS /RACLE PROVIDED A MECHANISM FOR RELEASING %XADATA SPECIFIC PATCHES THAT
COULD BE MANAGED SEPARATELY FROM THE MAINSTREAM PRODUCT AND COINED A NEW TERM bundle patches "UNDLE
PATCHES ARE /RACLES WAY OF TRACKING AND DISTRIBUTING FIXES FOR THEIR STORAGE CELL SOFTWARE AS WELL AS %XADATA
SPECIFIC FIXES FOR THE 'RID )NFRASTRUCTURE !3- AND DATABASE SOFTWARE
7HEN YOU TAKE DELIVERY OF YOUR %XADATA $ATABASE -ACHINE CERTAIN TASKS WILL HE COMPLETED BY AN
/RACLE HARDWARE TECHNICIAN 4HESE TASKS INCLUDE CONNECTING ALL THE NETWORKED COMPONENTS INSIDE THE
%XADATA RACK AND CONFIGURING )0 ADDRESSES FOR THE PRIVATE NETWORK )" SWITCH 7HEN THIS PROCESS IS
COMPLETE ALL COMPUTE NODES AND STORAGE CELLS WILL BE CONNECTED TOGETHER THROUGH THE )" SWITCH
/NE#OMMAND IS ONLY RUN FROM THE FIRST COMPUTE NODE AND USES THE PRIVATE NETWORK TO EXECUTE
CONFIGURATION COMMANDS ON OTHER SERVERS AND STORAGE CELLS AS WELL AS TO INSTALL THE 'RID )NFRASTRUCTURE
AND 2$"-3 SOFTWARE ON ALL DATABASE SERVERS 4HERE IS NO PLACE IN THE CONFIGURATION WORKSHEET OR THE
$"- #ONFIGURATOR TO SPECIFY THE NETWORK ADDRESS AND THE DEFAULT IS )F THIS NETWORK ADDRESS
IS ALREADY IN USE ELSEWHERE IN YOUR NETWORK INFRASTRUCTURE THIS MAY CREATE ROUTING PROBLEMS LATER "E
SURE TO MENTION THIS TO YOUR /RACLE DELIVERY COORDINATOR EARLY IN THE PROCESS SO THEY CAN NEGOTIATE AN
APPROPRIATE NETWORK ADDRESS FOR YOUR )NFINI"AND NETWORK
Configuring Exadata
4HE FIRST STEP IN CONFIGURING %XADATA IS GENERALLY TO WORK THROUGH THE configuration worksheet )F YOU ARE
COMFORTABLE WITH THE PROCESS YOU CAN OPT OUT OF STEP AND ENTER YOUR PARAMETERS DIRECTLY INTO THE $"-
#ONFIGURATOR AS YOU GATHER THEM "UT DONT UNDERESTIMATE THE VALUE OF THE CONFIGURATION WORKSHEET )T IS
AN INTEGRAL PART OF THE PROCESS /RACLE USES AND FOR GOOD REASON %VEN EXPERIENCED %XADATA TECHNICIANS
#(!04%2 #/.&)'52).' %8!$!4!
CONTINUE TO USE THE CONFIGURATION WORKSHEET )T IS A GOOD COMMUNICATION TOOL AND HELPS TO ENSURE THAT
THE INSTALLATION GOES SMOOTHLY
3TEP 4HE #ONFIGURATION 7ORKSHEET
4HE CONFIGURATION WORKSHEET IS PROVIDED BY /RACLE AS PART OF THE %XADATA DOCUMENTATION SET 4HE
DOCUMENTATION IS NOT DOWNLOADABLE FROM ANY OF THE /RACLE 7EB SITES )NSTEAD IT IS INSTALLED IN THE
dba bWPaT S^R ^aPR[T 4gPSPcP DIRECTORY OF THE STORAGE CELLS 3IMPLY CONNECT TO ONE OF THE CELLS AS ROOT
CELLADMIN OR CELLMONITOR AND DOWNLOAD IT TO YOUR LAPTOP /PEN THE fT[R^\T Wc\[ FILE IN YOUR 7EB
BROWSER AND YOU WILL SEE A COMPLETE LIST OF ALL THE DOCUMENTS IN (4-, AND 0$& FORMAT 4HE ONE YOU ARE
LOOKING FOR IS LABELED %XADATA $ATABASE -ACHINE #ONFIGURATION 7ORKSHEETS. 4HE WORKSHEET IS FAIRLY WELL
DOCUMENTED BUT WELL GO THROUGH IT HERE AND TALK A LITTLE MORE ABOUT SOME OF THE PARAMETERS AND
SETTINGS YOU WILL NEED TO PROVIDE 4HE INFORMATION COLLECTED IN THE WORKSHEET FALLS INTO FOUR GENERAL
CATEGORIES
v %XADATA CONFIGURATION PARAMETERS
v /RACLE ENVIRONMENT PARAMETERS
v .ETWORK SETTINGS
v #ELL ALERT NOTIFICATION PARAMETERS
7ELL BE FOLLOWING A SLIGHTLY DIFFERENT FORMAT HERE BUT WELL DISCUSS ALL OF THE CONFIGURATION
SETTINGS AND PARAMETERS FROM THE /RACLE DOCUMENTATION AND ADDING SOME COMMENTARY ALONG
THE WAY
Exadata Configuration Parameters
%XADATA CONFIGURATION PARAMETERS ARE SYSTEM WIDE SETTINGS SOME OF WHICH ARE USED FOR GENERATING YET
OTHER PARAMETERS USED IN THE CONFIGURATION PROCESS &OR EXAMPLE THE %XADATA $ATABASE -ACHINE .AME
YOU PROVIDE IN THE WORKSHEET IS USED AS A HOST NAME PREFIX FOR ALL NETWORK HOST NAMES IN YOUR %XADATA
SYSTEM INCLUDING COMPUTE NODES STORAGE CELLS NETWORK SWITCHES ),/-S AND THE +6- )T IS ALSO USED
AS A PREFIX FOR THE CLUSTER NAME AND 3#!. NAME REQUIRED BY 'RID )NFRASTRUCTURE
.OTE 7E RECENTLY NOTICED THAT THE NEW VERSION OF THE $"- #ONFIGURATOR NOW APPENDS THE %XADATA $ATABASE
-ACHINE .AME TO THE !3- DISK GROUP NAMES &OR EXAMPLE IF THE MACHINE NAME IS @TgP THEN THE #ONFIGURATOR
GENERATES A DISK GROUP NAME OF 30C0N4G0 RATHER THAN 30C0N36 FOR THE DATA DISK GROUP 4HIS IS THE DEFAULT
BEHAVIOR BUT IT MAY BE OVERRIDDEN BEFORE GENERATING YOUR CONFIGURATION FILES 7ELL TALK MORE ABOUT NAMING YOUR
!3- DISK GROUPS IN h3TEP 4HE $"- #ONFIGURATORv 9OU MAY NOTICE THAT SOME OF THE OTHER CHAPTERS OF THIS BOOK
#HAPTER FOR EXAMPLE USE THE OLD NAMING CONVENTION
4ABLE
SHOWS THE %XADATA CONFIGURATION PARAMETERS YOU WILL ENTER IN THE CONFIGURATION
WORKSHEET
#(!04%2 #/.&)'52).' %8!$!4!
Table 8-1. Exadata Configuration Parameters
Configuration Parameter Description
%XADATA $ATABASE -ACHINE NAME 4HE MACHINE NAME MAX CHARACTERS IS USED
AS A PREFIX FOR ALL NETWORK HOST NAMES IN YOUR
%XADATA CONFIGURATION
4YPE OF 3YSTEM 4HIS IS THE TYPE OF %XADATA CONFIGURATION
#HOOSE ONE OF THE FOLLOWING CONFIGURATIONS
8
&ULL 2ACK 8 &ULL 2ACK 8 (ALF 2ACK OR
8 1UARTER 2ACK
$ISK 4YPE 4HIS IS THE TYPE OF DISK DRIVES YOUR %XADATA
SYSTEM IS CONFIGURED WITH #HOOSE ONE OF THE
FOLLOWING CONFIGURATIONS (IGH 0ERFORMANCE
(IGH #APACITY
#OUNTRY 4HIS IS THE COUNTRY IN WHICH %XADATA WILL
RESIDE FOR EXAMPLE 5NITED 3TATES
4IME :ONE 4HIS IS THE TIME ZONE IN WHICH %XADATA RESIDES
&OR EXAMPLE !MERICA#HICAGO
7ORKLOAD 4YPE 4HIS FIELD IS USED TO DEFINE THE PREDOMINANT
WORKLOAD TYPE FOR DATABASES ON THE SYSTEM
#HOOSE EITHER $7 FOR DATA WAREHOUSE OR
/,40 FOR ONLINE TRANSACTION PROCESSING
"ACKUP -ETHOD 4HE INSTALLATION PROCESS USES THE BACKUP
METHOD TO DETERMINE THE SIZES FOR !3- DISK
GROUPS #HOOSE BETWEEN "ACKUPS )NTERNAL TO
/RACLE AND "ACKUPS %XTERNAL TO /RACLE
0ROTECTION ,EVEL 4HIS FIELD DETERMINES WHAT TYPE OF REDUNDANCY
YOU WISH TO USE FOR YOUR !3- DISK GROUPS
#HOOSE ONE OF THE FOLLOWING (IGH FOR !,,
(IGH FOR $!4! (IGH FOR 2%#/ OR .ORMAL FOR
!,,
%XADATA 3MART &LASH #ACHE !LL FLASH DISKS ARE CONFIGURED AS FLASH CACHE
3OME OF THE PARAMETERS LIKE #OUNTRY AND 4IME :ONE ARE PRETTY STANDARD FARE WHEN CONFIGURING
SERVERS /THER PARAMETERS ARE %XADATA
SPECIFIC AND WILL BE NEW TO YOU /NE THING YOU MAY NOTICE RIGHT
AWAY IS THAT THERE IS NO WAY TO NAME YOUR !3- DISK GROUPS IN THE CONFIGURATION WORKSHEET )F YOU DONT
LIKE THE DEFAULT NAMES OF 30C0N{machine_name} AND A42>N{machine_name} YOU WILL HAVE THE OPPORTUNITY
TO CHANGE THEM LATER IN THE $"- #ONFIGURATOR 4HE %XADATA CONFIGURATION PARAMETERS ARE DESCRIBED AS
FOLLOWS
#(!04%2 #/.&)'52).' %8!$!4!
Exadata Database Machine Name: 4HIS PARAMETER CAN BE n CHARACTERS IN
LENGTH AND IS USED TO GENERATE NETWORK HOST NAMES FOR NETWORK ENDPOINTS IN
YOUR %XADATA FRAME )T IS ALSO USED TO NAME THE 2!# CLUSTER AND THE 3IMPLE
#LIENT !CCESS .AME 3#!. REQUIRED BY 2!# &OR EXAMPLE A MACHINE NAME OF
TgP PRODUCES A CLUSTER NAME OF @TgPNR[dbcTaz AND A 3#!. NAME OF TgP
bRP] ^daR^\_P]h R^\ 4HIS PARAMETER IS CASE SENSITIVE FOR ALL BUT THE DISK GROUP
NAME POSTFIX IF ANY IN WHICH CASE IT IS FORCED TO UPPERCASE 4HE FOLLOWING
LISTING SHOWS THE MANAGEMENT NETWORK HOST NAMES GENERATED FOR THE DATABASE
SERVERS AND STORAGE CELLS IN A QUARTER RACK CONFIGURATION )N STEP WELL TAKE A
LOOK AT A COMPLETE LISTING OF THE NAMES PREFIXED WITH THE MACHINE NAME FOR A
QUARTER RACK CONFIGURATION
<P]PVT\T]c=Tcf^aZ
TgPSQ ^daR^\_P]h R^\TgPSQ
TgPSQ ! ^daR^\_P]h R^\TgPSQ !
TgPRT[ ^daR^\_P]h R^\TgPRT[
TgPRT[ ! ^daR^\_P]h R^\TgPRT[ !
TgPRT[ " ^daR^\_P]h R^\TgPRT[ "
)F THERE IS A CHANCE YOU WILL EVER HAVE MORE THAN ONE %XADATA SYSTEM IN YOUR DATA CENTER
CONSIDER ENUMERATING YOUR MACHINE NAME TO PROVIDE UNIQUENESS &OR EXAMPLE A MACHINE
NAME OF TgP ALLOWS FOR FUTURE SYSTEMS TO BE NAMED TgP! TgP" AND SO ON
Type of System 4HE TYPE OF SYSTEM DETERMINES HOW MANY COMPUTE NODES AND
STORAGE CELLS ARE IN YOUR CONFIGURATION 4HIS FIELD IS PRESENTED AS A DROP DOWN
LIST ILLUSTRATED IN &IGURE
Figure 8-2. The Type of System drop-down
4HIS PARAMETER IS USED TO DETERMINE HOW MANY COMPUTE NODES AND STORAGE
CELLS YOUR SYSTEM HAS WHEN GENERATING )0 ADDRESSES AND HOST NAMES 7ELL TALK
MORE ABOUT THIS LATER IN THE h.ETWORK 3ETTINGSv SECTION BUT /NE#OMMAND
GENERATES SEQUENTIAL )0 ADDRESSES BY STARTING WITH A BASE ADDRESS FOR THE FIRST
HOST AND THEN INCREMENTING FROM THAT ADDRESS FOR EACH HOST &OR EXAMPLE IF
YOUR MANAGEMENT )0S START AT THEN THE FOLLOWING )0 ADDRESSES
WILL BE GENERATED FOR THE COMPUTE NODES AND STORAGE CELLS IN A QUARTER RACK
TgPSQ ^daR^\_P]h R^\
TgPSQ ! ^daR^\_P]h R^\
TgPRT[ ^daR^\_P]h R^\
TgPRT[ ! ^daR^\_P]h R^\
TgPRT[ " ^daR^\_P]h R^\
#(!04%2 #/.&)'52).' %8!$!4!
.OTE 4HE WAY /NE#OMMAND ASSIGNS )0 ADDRESSES SEQUENTIALLY FROM A BASE ADDRESS CAN BE PROBLEMATIC WHEN
ADDING COMPUTE NODES AND STORAGE CELLS TO YOUR CONFIGURATION 7ELL TALK MORE ABOUT THIS LATER IN THE CHAPTER WHEN
WE LOOK AT UPGRADING %XADATA
Disk Type: 4HIS FIELD IS PRESENTED AS A DROP DOWN LIST %XADATA 8 MODELS COME
IN TWO FLAVORS (IGH #APACITY $ISKS AND (IGH 0ERFORMANCE $ISKS (IGH CAPACITY
DISKS HAVE A CAPACITY OF TERABYTES WHILE HIGH PERFORMANCE DISKS HAVE A
CAPACITY OF GIGABYTES 4HIS INFORMATION IS USED TO CALCULATE CELL DISK AND GRID
DISK SIZES FOR THE INSTALLATION
Workload Type: 3EVERAL SETTINGS FROM THE CONFIGURATION WORKSHEET ARE USED TO
CREATE A STARTER DATABASE DURING THE INSTALLATION PROCESS 4HESE PARAMETERS ARE
USED TO CREATE A DATABASE TEMPLATE THAT IS IN TURN USED BY THE $ATABASE
#ONFIGURATION !SSISTANT SQRP AT THE END OF THE INSTALLATION PROCESS 4HE
WORKLOAD TYPE YOU SPECIFY DETERMINES WHETHER YOUR DEFAULT DATABASE IS
OPTIMIZED FOR /,40 OR $7 WORKLOADS )NSTANCE PARAMETERS SUCH AS B60NC0A64C
AND ?60NC0A64C ARE SIZED BASED ON THE 7ORKLOAD 4YPE YOU SPECIFIED FOR YOUR
DATABASE
Backup Method 4HIS SETTING DETERMINES HOW YOUR DISK GROUPS ARE SIZED DURING
THE INSTALLATION PROCESS $ISK SPACE IS ALLOCATED TO THE 30C0 AND A42> DISK GROUPS
AS FOLLOWS
Backups Internal to Exadata: )F YOU PLAN TO USE THE &AST 2ECOVERY !REA
&2! TO STORE ONLINE DATABASE BACKUPS CHOOSE "ACKUPS )NTERNAL TO
/RACLE 7HEN THIS ALLOCATION SCHEME IS SELECTED DISK SPACE FOR THE 30C0
AND A42> DISK GROUPS WILL BE ALLOCATED AS FOLLOWS
30C0
A42>
Backups External to Exadata: )F YOU ARE PLANNING TO BACK UP YOUR
DATABASES DIRECTLY TO TAPE .&3 OR SOME OTHER STORAGE EXTERNAL TO
%XADATA CHOOSE "ACKUPS %XTERNAL TO %XADATA 7HEN THIS ALLOCATION
SCHEME IS CHOSEN DISK SPACE WILL BE ALLOCATED TO THE 30C0 AND A42> DISK
GROUPS AS FOLLOWS
30C0
A42>
Protection Level 4HIS DETERMINES WHAT LEVEL OF REDUNDANCY WILL BE CONFIGURED
FOR YOUR !3- DISK GROUPS 30C0 AND A42> 3IMILAR TO THE 4YPE OF 3YSTEM FIELD
THIS FIELD IS PRESENTED AS A DROP DOWN LIST FROM WHICH YOU CAN CHOOSE THE
REDUNDANCY FOR YOUR DISK GROUPS +EEP IN MIND THAT NORMAL REDUNDANCY HALVES
THE USABLE SPACE IN YOUR DISK GROUPS .ORMAL REDUNDANCY IS THE MINIMUM
ALLOWABLE REDUNDANCY LEVEL (IGH REDUNDANCY PROVIDES THE BEST PROTECTION
FROM DISK AND STORAGE CELL FAILURES BUT IT DOES SO AT THE COST OF REDUCING
AVAILABLE SPACE BY TWO THIRDS 4HE FOLLOWING COMBINATIONS ARE SUPPORTED
#(!04%2 #/.&)'52).' %8!$!4!
High For All: 4HIS CONFIGURATION CAN SIGNIFICANTLY REDUCE AVAILABLE SPACE
IN YOUR DISK GROUPS !CCORDING TO /RACLE IT IS BEST USED WHEN THE BACKUP
METHOD CHOSEN IS "ACKUPS %XTERNAL TO /RACLE
High For DATA: )N THIS CONFIGURATION !3- REDUNDANCY IS SET TO HIGH FOR
30C0 WHILE A42> IS SET TO NORMAL REDUNDANCY 4HIS IS BEST SUITED FOR
SYSTEMS THAT BACK UP THE DATABASES TO EXTERNAL STORAGE SUCH AS TAPE
64, .&3 AND SO ON
High For RECO: )N THIS CONFIGURATION HIGH REDUNDANCY IS USED FOR A42>
AND NORMAL REDUNDANCY IS USED FOR THE 30C0 DISK GROUP 4HIS
CONFIGURATION IS BEST SUITED FOR DATABASES THAT BACK UP TO THE &2!
Normal For ALL: 4HIS CONFIGURATION USES NORMAL REDUNDANCY FOR ALL DISK
GROUPS 7HILE IT PROVIDES QUITE A BIT MORE USABLE SPACE IN THE DISK
GROUPS KEEP IN MIND THAT THE SIMULTANEOUS LOSS OF ONE DISK DRIVE FROM
TWO STORAGE CELLS WILL CAUSE !3- TO DISMOUNT YOUR DISK GROUPS )F THAT
HAPPENS YOUR DATABASES USING THOSE DISK GROUPS WILL ALSO BE OFFLINE
!S YOU CONSIDER WHICH PROTECTION SCHEME IS RIGHT FOR YOU THINK ABOUT YOUR PAIN
TOLERANCE WHERE IT COMES TO SYSTEM OUTAGES .ORMAL REDUNDANCY PROVIDES
MORE STORAGE AND LESS PROTECTION FROM DISKCELL FAILURES 5NLESS YOU CAN AFFORD
FOR YOUR DATABASES TO BE DOWN FOR AN EXTENDED PERIOD OF TIME YOU SHOULD LEAN
TOWARD HIGH REDUNDANCY FOR THE 30C0 DISK GROUP )F YOU CAN AFFORD AN OUTAGE
WHILE TRANSIENT DISKCELL FAILURES ARE RESOLVED OR IN A WORST CASE SCENARIO WAIT
FOR A FULL DATABASE RECOVERY THEN PERHAPS HIGH REDUNDANCY FOR THE A42> DISK
GROUP IS A BETTER FIT )F SPACE IS VERY TIGHT AND YOU CAN TOLERATE THESE TYPES OF
OUTAGES THEN YOU MAY CONSIDER SETTING REDUNDANCY FOR ALL DISK GROUPS TO
NORMAL
4HE REDUNDANCY LEVEL FOR YOUR BHBC4<N36 OR 315BN36 DISK GROUP WHICH STORES
THE /#2 AND 6OTING FILES WILL BE AUTOMATICALLY SET TO THE MAXIMUM LEVEL
SUPPORTED BY YOUR %XADATA RACK CONFIGURATION (IGH REDUNDANCY IS NOT
SUPPORTED ON QUARTER RACK CONFIGURATIONS SO NORMAL REDUNDANCY IS THE BEST
YOU CAN DO FOR THEM &OR HALF RACK AND FULL RACK CONFIGURATIONS REDUNDANCY WILL
ALWAYS BE SET TO HIGH FOR THE BHBC4<N36 DISK GROUP
+EVIN 3AYS 0ROTECTION LEVEL IS A VERY SENSITIVE AND SEEMINGLY POLITICALLY CHARGED TOPIC 4HE MANNER IN WHICH
!3- CHOOSES PARTNER DISKS LEAVES ANY NORMAL REDUNDANCY DISK GROUP AT MUCH GREATER RISK OF DATA LOSS FROM A
DOUBLE
DISK FAILURE THAN WOULD BE THE CASE WITH TRADITIONAL 2!)$
STYLE PROTECTION 7HILE THERE MAY BE HUNDREDS OF
DISKS THAT could FAIL SIMULTANEOUSLY IN A DOUBLE DISK FAILURE ITS JUST A MATTER OF BAD LUCK WHICH ONES FAIL )F THE
hWRONG TWO DISKSv IN A NORMAL REDUNDANCY SITUATION HAPPEN TO FAIL THE ADMINISTRATOR WILL HAVE TO DEAL WITH DATA
LOSS &OR THIS REASON THE AUTHORS ARE CORRECT TO POINT OUT WHICH HIGH
REDUNDANCY SCHEME IS RECOMMENDED BASED
ON THE BACKUP STRATEGY PROTECTING THE SYSTEM
#(!04%2 #/.&)'52).' %8!$!4!
4HE TOPIC OF !3- NORMAL REDUNDANCY PROTECTION FROM DOUBLE DISK FAILURE SCENARIOS IS NOT %XADATA
SPECIFIC
(OWEVER UNTIL THE ADVENT OF %XADATA CUSTOMERS RARELY TRUSTED THEIR DATA TO !3- PROTECTIONOPTING INSTEAD FOR
EXTERNAL REDUNDANCY
Exadata Smart Flash Cache 4HERE IS NO MENTION OF THE FLASH CACHE IN THE
CONFIGURATION WORKSHEET BUT THERE IS IN THE $"- #ONFIGURATOR
Oracle Environment Parameters
4HE NEXT SET OF PARAMETERS IS USED TO DEFINE THE /3 USER AND GROUP ACCOUNTS AS WELL AS THE LOCATION FOR
THE 'RID )NFRASTRUCTURE #LUSTERWARE AND DATABASE SOFTWARE INSTALLATION 4HESE PARAMETERS ARE WELL
KNOWN TO $"!S WHO HAVE BEEN THROUGH AN /RACLE 2!# G2 INSTALLATION OR TWO 4ABLE SHOWS THE
PARAMETERS YOU WILL NEED TO PROVIDE 4HE PARAMETERS IN italics CANNOT BE MODIFIED IN THE CONFIGURATION
WORKSHEET AND ARE INCLUDED FOR INFORMATIONAL PURPOSES ONLY 3OME OF THEM HOWEVER CAN BE CHANGED
WHEN YOU FILL OUT THE $"- #ONFIGURATOR LATER
Table 8-2. Oracle Environment Parameters
Environment Parameter Default Value
'RID )NFRASTRUCTURE USER NAME ORACLE
'RID )NFRASTRUCTURE USER IDENTIFIER
2!# HOME USER NAME ORACLE
2!# HOME IDENTIFIER
/RACLE INVENTORY GROUP NAME OINSTALL
/RACLE INVENTORY GROUP IDENTIFIER
2!# HOME $"! GROUP NAME DBA
2!# HOME $"! GROUP IDENTIFIER
2!# HOME /0%2 GROUP NAME RACOPER
2!# HOME /0%2 GROUP IDENTIFIER
!3- $"! GROUP NAME ASMDBA
!3- $"! GROUP IDENTIFIER
#(!04%2 #/.&)'52).' %8!$!4!
Environment Parameter Default Value
!3- !$-). GROUP NAME ASMADMIN
!3- !$-). GROUP IDENTIFIER
/RACLE SOFTWARE OWNER PASSWORD welcome
'RID )NFRASTRUCTURE #LUSTERWARE HOME /u01/app/{version}/grid
BASE DIRECTORY
7HERE VERSION IS CURRENTLY
/RACLE DATABASE HOME BASE DIRECTORY /u01/app/oracle
>A02;4N10B4
/RACLE DATABASE HOME >A02;4N7><4 /u01/app/oracle/product{version}/db_home1
DIRECTORY
6ERSION IS CURRENTLY
/RACLE )NVENTORY DIRECTORY /u01/app/oraInventory
$ATABASE NAME DBM
$ATABASE CHARACTER SET AL32UTF8
$ATABASE NATIONAL CHARACTER SET AL16UTF16
$ATABASE BLOCK SIZE 8192
!3- DISK GROUPS 30C0 for the default data file location;A42> for
the fast recovery area
Note: Default 30C0 and A42> disk group size
depends on the type of system (quarter, half, or
full rack), the type of disk drives (high capacity
or high performance), and the type of backup.
3TARTING )0 !DDRESS FOR THE )NFINI"AND 192.168.10.1
PRIVATE NETWORK1
3UBNET MASK FOR )NFINI"AND PRIVATE 255.255.252.0
NETWORK
4HE )NFINI"AND PRIVATE NETWORK IS MORE COMMONLY KNOWN TO $"!S AS THE private Interconnect 4HIS FIELD
SHOWS STARTING )0 ADDRESS FOR THE RANGE OF )0S THE 2!# CLUSTER WILL USE FOR CLUSTER )NTERCONNECT TRAFFIC
HEARTBEAT CACHE FUSION AND SO ON )F FOR SOME REASON YOU HAVE CHANGED THE )0 ADDRESSES FOR THE )"
NETWORK YOU WILL NEED TO ALSO MAKE THAT ADJUSTMENT IN THE $"- #ONFIGURATOR WHEN YOU TRANSFER THE
WORKSHEET PARAMETERS LATER
#(!04%2 #/.&)'52).' %8!$!4!
.OTE /NE#OMMAND DOES NOT ALLOW YOU TO MODIFY THE CHARACTER SET PARAMETERS 3O THE DEFAULT DATABASE
CREATED WILL USE THE !,54& DATABASE CHARACTER SET AND !,54& NATIONAL CHARACTER SET !FTER THE
INSTALLATION IS COMPLETE YOU MAY CHOOSE TO DROP THE DEFAULT DATABASE AND CREATE YOUR OWN DATABASES THAT MEET
THE SPECIFIC REQUIREMENTS OF YOUR ENVIRONMENT +EEP IN MIND THAT /RACLE RECOMMENDS USING 5NICODE FOR ALL NEW
DATABASE IMPLEMENTATIONS
Network Settings
4HE FINAL SET OF CONFIGURATION PARAMETERS IS WHERE YOU SPECIFY THE NETWORK SETTINGS FOR %XADATA
/NE#OMMAND SUPPORTS THE CONFIGURATION OF UP TO FOUR NETWORKS IDENTIFIED AS .%4 THROUGH .%4
.%4 AND .%4 ARE REQUIRED AND ARE USED FOR THE MANAGEMENT NETWORK AND CLIENT ACCESS NETWORK
RESPECTIVELY .%4 AND .%4 ARE OPTIONAL NETWORKS YOU MAY CHOOSE TO CONFIGURE IF NEEDED 4ABLE
SHOWS ALL THE NETWORK SETTINGS USED TO CONFIGURE %XADATA
Table 8-3. Network Settings
Network Setting Description
$OMAIN NAME 9OUR NETWORK DOMAIN NAME &OR EXAMPLE ^daR^\_P]h R^\
)0 ADDRESS OF THE NAME SERVERS 4HE )0 ADDRESS OF ONE OR MORE OF YOUR NETWORK NAME SERVERS
)0 ADDRESS OF THE .ETWORK 4IME 4HE )0 ADDRESS OF ONE OR MORE .40 SERVERS FOR SERVER TIME
0ROTOCOL .40 SERVERS SYNCHRONIZATION
-ANAGEMENT .ETWORK .%4 4HE STARTING )0 ADDRESS FOR THE MANAGEMENT NETWORK )0
3TARTING )0 ADDRESS ADDRESSES FOR ALL MANAGEMENT NETWORK END
POINTS WILL DERIVE
SEQUENTIALLY FROM THIS )0 ADDRESS 4HIS NETWORK MUST BE
SEPARATE FROM ALL OTHER NETWORKS ON %XADATA
-ANAGEMENT .ETWORK .%4 4HE SUBNET MASK FOR THE MANAGEMENT NETWORK
3UBNET MASK
-ANAGEMENT .ETWORK .%4 4HE )0 ADDRESS OF THE GATEWAY FOR THE MANAGEMENT NETWORK
'ATEWAY )0 ADDRESS
#LIENT !CCESS .ETWORK .%4 4HE CLIENT ACCESS NETWORK IS CONFIGURED ON THE COMPUTE
3TARTING )0 ADDRESS NODES )T IS THE INTERFACE USED BY THE DATABASE 3#!. LISTENERS
#LIENT APPLICATIONS CONNECT TO THE DATABASE THROUGH THE .%4
INTERFACE 4HIS NETWORK MUST BE SEPARATE FROM ALL OTHER
NETWORKS ON %XADATA
#LIENT !CCESS .ETWORK .%4 4HE SUBNET MASK FOR THE #LIENT !CCESS .ETWORK
3UBNET MASK
#(!04%2 #/.&)'52).' %8!$!4!
Network Setting Description
#LIENT !CCESS .ETWORK .%4 4HE GATEWAY FOR THE #LIENT !CCESS .ETWORK
'ATEWAY )0 ADDRESS
#LIENT !CCESS .ETWORK .%4 /RACLE RECOMMENDS THE #LIENT !CCESS .ETWORK %THERNET
#HANNEL BONDING DEVICES TcW AND TcW! BE BONDED TO PROVIDE REDUNDANCY FOR
CLIENT CONNECTIONS TO THE DATABASE 9ES OR .O IN THIS FIELD
DETERMINES WHETHER OR NOT .%4 WILL BE CONFIGURED FOR
CHANNEL BONDING )F IT IS IT WILL BE BONDED TO WITH THE .%4
INTERFACE
!DDITIONAL .ETWORK .%4 )F CHANNEL BONDING IS NOT USED FOR THE #LIENT !CCESS .ETWORK
3TARTING )0 ADDRESS THEN .%4 IS AVAILABLE AS AN ADDITIONAL NETWORK INTERFACE
TcW! 4HIS FIELD IS THE STARTING )0 ADDRESS FOR THE RANGE OF )0S
ASSIGNED TO .%4 ACROSS THE DATABASE SERVERS )F CONFIGURED
THIS NETWORK MUST BE SEPARATE FROM ALL OTHER NETWORKS ON
%XADATA
!DDITIONAL .ETWORK .%4 4HE SUBNET MASK FOR THE .%4 NETWORK INTERFACE
3UBNET MASK
!DDITIONAL .ETWORK .%4 4HE )0 ADDRESS OF THE GATEWAY USED BY THE .%4 NETWORK
'ATEWAY )0 ADDRESS
!DDITIONAL .ETWORK .%4 4HIS SUFFIX IS APPENDED TO THE HOST NAME ON EACH OF THE
.ETWORK HOST NAME SUFFIX DATABASE SERVERS TO IDENTIFY THE .%4 NETWORK
!DDITIONAL .ETWORK .%4 .%4 IS AN OPTIONAL NETWORK AVAILABLE ON THE DATABASE
3TARTING )0 ADDRESS SERVERS 4HIS IS THE )0 ADDRESS FOR THE SEQUENTIAL RANGE OF )0S
FOR THE .%4 NETWORK 4HIS NETWORK MUST BE SEPARATE FROM ALL
OTHER NETWORKS ON THE SYSTEM
!DDITIONAL .ETWORK .%4 4HE SUBNET MASK FOR THE .%4 NETWORK
3UBNET MASK
!DDITIONAL .ETWORK .%4 4HE )0 ADDRESS OF THE GATEWAY FOR THE .%4 NETWORK
'ATEWAY )0 ADDRESS
!DDITIONAL .ETWORK .%4 4HIS SUFFIX IS APPENDED TO THE HOST NAME ON EACH OF THE
.ETWORK HOST NAME SUFFIX DATABASE SERVERS TO IDENTIFY THE .%4 NETWORK
0$5 ! )0 ADDRESS 4HE POWER DISTRIBUTION UNIT 0$5 HAS AN %THERNET INTERFACE
ETH AND CAN BE CONFIGURED ON THE NETWORK BY ASSIGNING IT AN
)0 ADDRESS 4HIS FIELD DEFINES THE )0 ADDRESS FOR THE 0$5
#(!04%2 #/.&)'52).' %8!$!4!
Network Setting Description
0$5 " )0 ADDRESS %XADATA IS EQUIPPED WITH A REDUNDANT 0$5 THAT CAN ALSO BE
CONFIGURED ON THE NETWORK BY ASSIGNING IT AN )0 ADDRESS 4HIS
FIELD DEFINES THE NETWORK FOR THE 0$5
4HE DOMAIN NAME $.3 SERVER AND .40 SERVER SETTINGS ARE ALL STANDARD NETWORK SETTINGS
4HESE SHOULD BE SPECIFIED AS YOU WOULD FOR ANY OTHER /RACLE DATABASE SERVER ON YOUR
NETWORK 4HE REMAINING NETWORK SETTINGS ARE DESCRIBED AS FOLLOWS
Management Network (NET0) 4HE MANAGEMENT NETWORK IS USED FOR
ADMINISTRATIVE ACCESS TO THE DATABASE SERVERS AND STORAGE CELLS 9OU CAN THINK OF
THIS NETWORK AS YOUR 33( ENTRY POINT FOR LOGGING INTO THE COMPUTE NODES
STORAGE CELLS AND )" SWITCHES 4HE MANAGEMENT NETWORK IS SERVICED BY THE
INTERNAL #ISCO NETWORK SWITCH /NLY ONE NETWORK DROP IS REQUIRED TO UPLINK THE
#ISCO MANAGEMENT SWITCH TO YOUR COMPANY NETWORK 4HE MANAGEMENT
NETWORK ALSO INCLUDES THE ),/- INTERFACES ON THE COMPUTE NODES AND STORAGE
CELLS
Client Access Network (NET1): 4HE CLIENT ACCESS NETWORK PROVIDES 31,
.ET
ACCESS TO THE DATABASE 4HIS IS THE NETWORK USED BY THE 3#!. ,ISTENERS TO
ESTABLISH CLIENT CONNECTIONS TO THE DATABASE /NLY THE DATABASE SERVERS USE THIS
NETWORK STORAGE CELLS ARE NOT ASSIGNED AN )0 ADDRESS ON THIS NETWORK !ND EVEN
THOUGH THE DATABASE SERVERS HAVE AN )0 ON THIS NETWORK YOU CANNOT CONNECT TO
THESE )0 ADDRESSES USING 33( BECAUSE 33( IS NOT CONFIGURED TO LISTEN ON THIS
NETWORK /NE NETWORK DROP MINIMUM IS REQUIRED FOR EACH DATABASE SERVER TO
PROVIDE ACCESS THROUGH YOUR COMPANY NETWORK )F CHANNEL BONDING IS USED TO
PROVIDE HARDWARE REDUNDANCY YOU WILL NEED TWO DROPS PER DATABASE SERVER
4HE CABLE PAIRS FROM THE DATABASE SERVERS SHOULD BE CONNECTED TO REDUNDANT
NETWORK SWITCHES TO PROVIDE FULL NETWORK REDUNDANCY )F CHANNEL BONDING IS
CONFIGURED FOR THE CLIENT ACCESS NETWORK /NE#OMMAND CONFIGURES THESE
DEVICES IN THE XURUV TcW AND XURUV TcW! CONFIGURATION FILES IN THE
TcR bhbR^]UXV ]Tcf^aZ bRaX_cb DIRECTORY 4HEY ARE BONDED TO THE Q^]STcW
DEVICE DEFINED IN THE XURUV Q^]STcW FILE &OR EXAMPLE THE FOLLOWING LISTING
SHOWS HOW THE MASTER BOND DEVICE IS REFERENCED BY THE TcW AND TcW! SLAVE
DEVICES
TVaT_34E824k<0BC4AkB;0E4XURUV Q^]STcW XURUV TcW XURUV TcW!
XURUV Q^]STcW )34E824,Q^]STcW
XURUV TcW )34E824,TcW
XURUV TcW )<0BC4A,Q^]STcW
XURUV TcW )B;0E4,hTb
XURUV TcW!)34E824,TcW!
XURUV TcW!)<0BC4A,Q^]STcW
XURUV TcW!)B;0E4,hTb
Channel Bonding (NET1): /RACLE RECOMMENDS THAT YOU CONFIGURE .)# BONDING
FOR THE CLIENT ACCESS NETWORK TO PROVIDE NETWORK REDUNDANCY )F YOU CHOOSE TO
BOND THE CLIENT ACCESS NETWORK /NE#OMMAND WILL USE .%4 AS THE SECONDARY
NETWORK DEVICE
#(!04%2 #/.&)'52).' %8!$!4!
Additional Network (NET2): )F CHANNEL BONDING IS NOT CONFIGURED FOR THE CLIENT
ACCESS NETWORK .%4 IS MAY BE USED TO CONFIGURE AN ADDITIONAL NETWORK
Additional Network (NET3): 4HE .%4 DEVICE IS AVAILABLE AS AN ADDITIONAL
NETWORK IF NEEDED
4IP )N ADDITION TO THE NETWORK DEVICES WEVE DISCUSSED HERE EACH %XADATA 8 COMPUTE NODE FEATURES TWO
GIGABIT %THERNET G% NETWORK PORTS 4HESE PORTS ARE NOT READY TO BE USED RIGHT OUT OF THE BOX 9OU MUST FIRST
PURCHASE AND INSTALL 3&0 TRANSCEIVER MODULES IN ORDER TO CONNECT THEM TO YOUR NETWORK SWITCH %ACH %XADATA 8
COMPUTE NODE HAS EIGHT OF THESE G% PORTS )F YOUR BACKUP MEDIA SERVER SUPPORTS G% THEN THIS NETWORK COULD
BE USED TO IMPROVE THE PERFORMANCE OF YOUR DATABASE BACKUPS
#ELL !LERT .OTIFICATION 0ARAMETERS
4HE RT[[bae MANAGEMENT SERVICE ON EACH STORAGE CELL MONITORS THE HEALTH OF THE STORAGE CELLS AND IS
CAPABLE OF SENDING OUT 3-40 AND 3.-0 NOTIFICATIONS IF PROBLEMS ARISE #ONFIGURING EMAIL AND 3.-0
ALERT NOTIFICATIONS FOR THE STORAGE CELLS IS OPTIONAL BUT HIGHLY RECOMMENDED 4ABLE
DESCRIBES THE
VARIOUS PARAMETERS USED TO CONFIGURE #ELL !LERT .OTIFICATIONS
Table 8-4. Cell Alert Notification Parameters
Configuration Parameter Description
3-40 3ERVER 4HIS IS THE NAME OF THE 3-40 SERVER THE STORAGE CELLS WILL USE
FOR ROUTING EMAIL ALERT NOTIFICATIONS &OR EXAMPLE
\PX[ R^a_^aPcT R^\
3-40 0ORT 4HIS IS THE 3-40 EMAIL PORT USED FOR ROUTING EMAIL
NOTIFICATIONS &OR EXAMPLE OR
SMTP5SE33, 4HIS FIELD INDICATES WHETHER OR NOT YOUR EMAIL SERVER REQUIRES
SECURE CONNECTIONS USING 33, 6ALID RESPONSES ARE 9ES AND
.O
SMTP4O!DDR 4HIS FIELD SHOULD BE FILLED IN WITH A COMMA DELIMITED LIST OF
EMAIL ADDRESSES FOR DISTRIBUTING EMAIL ALERTS &OR EXAMPLE
SQP/R^a_^aPcT R^\Y^W] S^T/R^a_^aPcT R^\
SMTP&ROM!DDR 4HIS IS THE RETURN ADDRESS FOR EMAIL ALERTS &OR EXAMPLE
TgPSPcP/R^a_^aPcT R^\
SMTP&ROM 4HIS EMAIL ADDRESS APPEARS IN THE h&ROMv FIELD OF EMAIL
ALERTS &OR EXAMPLE 4gPSPcP3PcPQPbT<PRWX]T
#(!04%2 #/.&)'52).' %8!$!4!
Configuration Parameter Description
SMTP5SER 4HIS IS THE USER NAME THE STORAGE CELL WILL USE FOR LOGGING IN
TO THE 3-40 SERVER
SMTP0WD 4HIS PASSWORD IS USED TO ESTABLISH SESSIONS WITH YOUR 3-40
SERVER )F YOUR 3-40 SERVER DOES NOT REQUIRE AUTHENTICATION
THIS FIELD MAY BE LEFT BLANK
SNMP3UBSCRIBER 4HIS IS THE HOST PORT AND COMMUNITY FOR RECEIVING 3.-0
ALERT NOTIFICATIONS 4HE PORT AND COMMUNITY ATTRIBUTES ARE
OPTIONAL 4HE DEFAULT PORT IS AND THE DEFAULT COMMUNITY
IS @PUBLIC &OR EXAMPLE b]\_ R^a_^aPcT R^\ %!_dQ[XR
9OU MAY ENTER ONLY ONE TARGET HERE BUT ADDITIONAL 3.-0
TARGETS CAN BE ADDED AFTER THE INSTALLATION IS COMPLETE
3TEP 4HE $"- #ONFIGURATOR
/NCE YOUVE COMPLETED THE CONFIGURATION WORKSHEET THE INFORMATION YOU GATHERED CAN BE TRANSFERRED TO
THE $"- $ATABASE -ACHINE #ONFIGURATOR 4HE $"- #ONFIGURATOR IS AN %XCEL SPREADSHEET
SQ\NR^]UXVdaPc^a g[b THAT GENERATES ALL THE INPUT FILES /NE#OMMAND WILL NEED TO CARRY OUT THE
INSTALLATION PROCESS 4HE SPREADSHEET IS LOCATED IN THE ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S DIRECTORY
ON EVERY COMPUTE NODE 4HE $"- #ONFIGURATOR IS HEAVILY DOCUMENTED IN THE %XADATA Database Machine
Owner’s Guide SO WE WONT BE DISCUSSING ALL OF THE FIELDS HERE "UT WE WILL TAKE A LOOK AT HOW A FEW OF
THEM ARE USED IN THE INSTALLATION PROCESS 4ABLE
DESCRIBES A FEW OF THE KEY DATA
ENTRY FIELDS FROM THE
SPREADSHEET
Table 8-5. DBM Configurator
Field Description
#USTOMER .AME 4HIS FIELD IS USED WHEN CREATING THE BASE DIRECTORY FOR THE
OUTPUT FILES GENERATED BY THE #ONFIGURATOR &OR EXAMPLE IF THE
CUSTOMER NAME IS h/UR#OMPANYv THEN A DIRECTORY NAMED
2)KSQ\PRWX]TN>da2^\_P]h WILL BE CREATED TO STORE THE OUTPUT
FILES
%XADATA $ATABASE -ACHINE .AME !S WE DISCUSSED EARLIER THE MACHINE NAME IS USED AS A PREFIX
FOR NETWORK HOST NAMES AND AS A SUFFIX FOR !3- DISK GROUP
NAMES !DDITIONALLY IT IS ALSO USED FOR THE TARGET DIRECTORY NAME
INSIDE THE BASE DIRECTORY MENTIONED ABOVE FOR STORING THE
OUTPUT FILES 4HIS ALLOWS THE #ONFIGURATOR TO MANAGE THE
CONFIGURATION OF MULTIPLE $ATABASE -ACHINE CONFIGURATIONS
#(!04%2 #/.&)'52).' %8!$!4!
Field Description
$ATA $ISKGROUP .AME 4HE NAME FOR THE !3- 30C0 DISK GROUP IS GENERATED
AUTOMATICALLY IN THE FORM 30C0_{machine_name} 4HE DEFAULT
NAME CAN BE CHANGED HERE AS NEEDED BEFORE GENERATING YOUR
PARAMETER FILES
2ECO $ISKGROUP .AME 4HE NAME FOR THE !3- A42> DISK GROUP IS GENERATED
AUTOMATICALLY IN THE FORM A42>?[machine_name} 4HE DEFAULT
NAME CAN BE CHANGED HERE BEFORE GENERATING YOUR PARAMETER
FILES
4HE $"- #ONFIGURATOR PROCESS IS AS FOLLOWS
1. %NTER THE CONFIGURATION PARAMETERS FROM THE CONFIGURATION WORKSHEET INTO THE
$"- #ONFIGURATOR
2. #LICK THE 'ENERATE BUTTON ON THE RIGHT SIDE OF THE SPREADSHEET ! BRIEF REPORT
SHOWING THE TOPOLOGY OF YOUR SYSTEM IS GENERATED AT THE BOTTOM OF THE
SPREADSHEET 3CROLL DOWN TO REVIEW YOUR SETTINGS AND MAKE CORRECTIONS TO THE
OUTPUT AS NEEDED 7HEN YOU ARE FINISHED MAKING CHANGES YOU ARE READY TO
CREATE THE OUTPUT FILES
3. 4HE LAST STEP IS TO CLICK THE #REATE #ONFIG &ILES BUTTON 4HIS GENERATES THE
PARAMETER AND DEPLOYMENT FILES USED BY /NE#OMMAND )T ALSO CREATES AN
installation template IN A SEPARATE SPREADSHEET TAB IN THE #ONFIGURATOR 4HE
INFORMATION IN THE TEMPLATE CONSISTS MOSTLY OF THE INPUT PARAMETERS YOU
PROVIDED IN AN EASY
TO
READ FORMAT 2EVIEW THE TEMPLATE CAREFULLY AND MAKE
CHANGES IN THE DATA ENTRY OR TOPOLOGY REPORT AREAS AS NEEDED )F YOU MUST GO
BACK AND MAKE ANY CHANGES YOU SHOULD CLICK THE 'ENERATE AND #REATE #ONFIG
&ILES BUTTONS AGAIN SO THAT YOUR CHANGES ARE WRITTEN TO THE OUTPUT FILES
#AUTION "E CAREFUL WHEN FILLING OUT THE SPREADSHEET )TS VERY EASY FOR A TYPO TO SLIP THROUGH AND CAUSE YOU
PROBLEMS LATER 4HE TOPOLOGY REPORT GENERATED BELOW THE DATA ENTRY FIELDS IS USEFUL FOR CATCHING ERRORS IN THE HOST
NAMES AND )0 ADDRESSES YOU ENTERED 4HIS REPORT IS CREATED WHEN YOU CLICK THE 'ENERATE BUTTON
3TEP 5PLOAD 0ARAMETER AND $EPLOYMENT &ILES
4HE $"- #ONFIGURATOR GENERATES A NUMBER OF FILES THAT WILL BE USED TO CONFIGURE YOUR %XADATA $ATABASE
-ACHINE 4HESE FILES ARE SAVED IN THE 2)KSQ\PRWX]TNjcompany_namelKjmachine_namel DIRECTORY 4HE FILES
ARE PRIMARILY USED BY /NE#OMMAND TO PERFORM THE INSTALLATION AND CONFIGURATION PROCESS "UT A FEW OF
THE FILES ARE USED BY OTHER PROCESSES &OR EXAMPLE THE RWTRZX_ bW SCRIPT USES THE SQ\ SPc FILE TO PERFORM
NETWORK READINESS TESTS PRIOR TO RUNNING /NE#OMMAND 4HESE FILES WILL BE NEEDED FOR VARIOUS TASKS
GOING FORWARD IN THE CONFIGURATION AND THEY MUST BE UPLOADED TO THE FIRST COMPUTE NODE ON YOUR
%XADATA PLATFORM
#(!04%2 #/.&)'52).' %8!$!4!
4HERE ARE A COUPLE OF WAYS TO TRANSFER THE PARAMETER AND DEPLOYMENT FILES TO %XADATA /NE METHOD
INVOLVES SETTING UP TEMPORARY NETWORK ACCESS TO THE FIRST COMPUTE NODE 4HIS NETWORK CONFIGURATION WILL
BE REPLACED WITH PERMANENT NETWORK SETTINGS BY &IRSTBOOT OR !PPLY#ONFIG IN SUBSEQUENT STEPS 4HE
OTHER OPTION IS TO SAVE THE FILES TO A PORTABLE 53" FLASH DRIVE AND THEN USING THE 53" PORT ON THE FRONT
PANEL OF THE FIRST COMPUTE NODE COPY THE FILES TO THE /NE#OMMAND DIRECTORY 7ELL TALK ABOUT WHERE
THAT DIRECTORY IS IN THE FOLLOWING EXAMPLE 4HE 53" APPROACH IS AS FOLLOWS
1. #ONNECT THE 53" FLASH DRIVE TO YOUR LAPTOP AND COPY THE DIRECTORY CONTAINING
YOUR CONFIGURATION FILES TO THE 53" DRIVE &OR EXAMPLE
\ZSXaT)KTgP
R^_h2)KSQ\PRWX]TN>da2^\_P]hKTgPT)KTgP
2)K-R^_hSQ\PRWX]TN>da2^\_P]hKTgPT)KTgP
SQ\PRWX]TN>da2^\_P]hKTgPKP[[NVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKP[[NXQNVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKP[[N]^ST[XbcNVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKRT[[NVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKRT[[NXQNVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKRWTRZX_ bW
SQ\PRWX]TN>da2^\_P]hKTgPKR^]UXV SPc
SQ\PRWX]TN>da2^\_P]hKTgPKSPcPQPbT\PRWX]T g\[
SQ\PRWX]TN>da2^\_P]hKTgPKSQ\ SPc
SQ\PRWX]TN>da2^\_P]hKTgPKSQ\PRWX]T _PaP\b
SQ\PRWX]TN>da2^\_P]hKTgPKSQ<PRWX]TNTgP
SQ\PRWX]TN>da2^\_P]hKTgPKSQbNVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKSQbNXQNVa^d_
SQ\PRWX]TN>da2^\_P]hKTgPKW^bcb bP\_[T
SQ\PRWX]TN>da2^\_P]hKTgPK_aTR^]U ! Rbe
SQ\PRWX]TN>da2^\_P]hKTgPK_aTR^]U ! ! ! Rbe
SQ\PRWX]TN>da2^\_P]hKTgPK_aTR^]U ! ! Rbe
SQ\PRWX]TN>da2^\_P]hKTgPK_aTR^]U Rbe
SQ\PRWX]TN>da2^\_P]hKTgPK_aXeNXQNVa^d_
(UX[TbR^_XTS
2. 2EMOVE THE 53" FLASH DRIVE FROM YOUR LAPTOP AND CONNECT IT TO THE 53" PORT
ON THE FRONT PANEL OF THE FIRST COMPUTE NODE IN YOUR SYSTEM 4HE SERVERS ARE
NUMBERED FROM BOTTOM TO TOP IN THE RACK SO THE BOTTOM
MOST COMPUTE NODE
IS THE ONE YOU ARE LOOKING FOR
3. 5SING THE %XADATA +6- LOG IN TO THE FIRST COMPUTE NODE AS ROOT AND CHECK THE
ePa [^V S\TbV FILE TO DETERMINE THE DEVICE NAME ASSIGNED TO THE 53" FLASH
DRIVE &OR EXAMPLE YOU SHOULD SEE A MESSAGE SOMETHING LIKE THE FOLLOWING
q0ccPRWTSbRbXaT\^ePQ[TSXbZbSQ
9OU CAN CONFIRM THE PARTITION AND DEVICE NAME USING THE USXbZ COMMAND AS
FOLLOWS
USXbZ [ STe bSQ
3XbZ STe bSQ)! "'<1! "'#" &##QhcTb
!$ WTPSb%!bTRc^ab caPRZ!$$Rh[X]STab
D]Xcb,Rh[X]STab^U $$%!$ !,&(%&&##QhcTb
#(!04%2 #/.&)'52).' %8!$!4!
3TeXRT1^^cBcPac4]S1[^RZb8SBhbcT\
STe bSQ !$% (( %!#RF($50C"!;10
4. #REATE A DIRECTORY YOU CAN USE AS A MOUNT POINT FOR THE 53" FLASH DRIVE &OR
EXAMPLE \]c dbQ
5. -OUNT THE 53" DEVICE USING THE FOLLOWING COMMAND
\^d]c ceUPc STe bSQ \]c dbQ
6. #OPY THE FILES TO THE ^]TR^\\P]S DIRECTORY AS FOLLOWS
R_ \]c bSQ TgP ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S
7. 5NMOUNT THE 53" FLASH DRIVE AS FOLLOWS
d\^d]c \]c dbQ
8. 9OU CAN NOW REMOVE THE 53" FLASH DRIVE FROM THE COMPUTE NODE
4ABLE
DESCRIBES WHAT THESE FILES ARE AND HOW THEY ARE USED IN THE INSTALLATION AND CONFIGURATION
PROCESS
Table 8-6. Parameter and Deployment Files
File Name Description
SQbNXQNVa^d_ 4HIS FILE CONTAINS THE PRIVATE NETWORK )0 ADDRESS LONG NETWORK
HOST NAME AND SHORT NETWORK HOST NAME FOR ALL COMPUTE
NODES 4HE FORMAT IS SIMILAR TO WHAT YOU WOULD SEE IN THE
TcR W^bcb FILE IN THE FORM ip_address long_name short_name
RT[[NXQNVa^d_ 4HIS FILE CONTAINS INFORMATION SIMILAR TO THAT OF THE
SQbNXQNVa^d_ FILE BUT THE INFORMATION PERTAINS TO THE STORAGE
CELLS
_aXeNXQNVa^d_ 4HIS FILE IS A COMBINATION OF THE SQbNXQNVa^d_ AND
RT[[NXQNVa^d_ FILES AND IS USED FOR TO POPULATE THE TcR W^bcb
FILE
P[[NVa^d_ 4HIS FILE CONTAINS THE HOST NAMES FOR ALL DATABASE SERVERS AND
STORAGE CELLS IN THE %XADATA $ATABASE -ACHINE &OR EXAMPLE
TgPSQ TgPRT[
P[[NXQNVa^d_ 4HIS FILE CONTAINS THE PRIVATE HOST NAMES FOR ALL SERVERS AND
STORAGE CELLS IN THE CONFIGURATION &OR EXAMPLE TgPSQ _aXe
AND TgPRT[ _aXe
P[[N]^ST[XbcNVa^d_ 4HIS FILE IS A COMBINATION OF THE P[[NVa^d_ AND P[[NXQNVa^d_
FILES
#(!04%2 #/.&)'52).' %8!$!4!
File Name Description
RT[[NVa^d_ 4HIS FILE CONTAINS A LIST OF THE HOST NAMES FOR ALL STORAGE CELLS IN
YOUR CONFIGURATION &OR EXAMPLE TgPRT[ TgPRT[ ! TgPRT[ "
AND SO ON
SQbNVa^d_ 4HIS FILE CONTAINS A LIST OF THE HOST NAMES FOR ALL COMPUTE NODES
IN YOUR CONFIGURATION &OR EXAMPLETgPSQ TgPSQ ! TgPSQ "
AND SO ON
W^bcb bP\_[T 4HIS IS A SAMPLE HOSTS FILE THAT IS INSTALLED IN TcR W^bcb WHEN
THERE IS NO $.3 SERVER AVAILABLE
RWTRZX_ bW 4HIS IS THE NETWORK VALIDATION SCRIPT YOU WILL RUN BEFORE RUNNING
THE /NE#OMMAND UTILITY !MONG OTHER THINGS IT VERIFIES ACCESS
TO THE .40 SERVER CHECKS NAME RESOLUTION USING YOUR $.3
SERVER AND CHECKS TO SEE IF ANY OF THE )0 ADDRESSES IN YOUR
CONFIGURATION ARE ALREADY IN USE 7ELL TALK MORE ABOUT
RWTRZX_ bW IN STEP
SQ\ SPc 4HIS FILE CONTAINS YOUR NETWORK HOST NAMES AND )0 ADDRESSES )T
IS USED BY THE RWTRZX_ bW SCRIPT TO VALIDATE THE NETWORK SETTINGS
YOU SPECIFIED 7HEN YOU ARE RUNNING THE RWTRZX_ bW SCRIPT THIS
FILE MUST BE IN THE SAME DIRECTORY
R^]UXV SPc 4HIS FILE CONTAINS ALL THE CONFIGURATION PARAMETERS AND NETWORK
SETTINGS FROM THE $"- #ONFIGURATOR IN A name=value FORMAT
SQ\PRWX]T _PaP\b 4HIS IS THE INPUT PARAMETER FILE USED BY /NE#OMMAND TO
CONFIGURE YOUR SYSTEM
SQ<PRWX]TNj\PRWX]TN]P\Tl 4HIS FILE IS USED TO GENERATE THE )NSTALLATION 4EMPLATE WHICH
SHOWS UP IN A SEPARATE SPREADSHEET TAB IN THE $"-
#ONFIGURATOR AFTER YOU CLICK THE #REATE #ONFIG &ILES BUTTON
_aTR^]U Rbe 4HIS IS THE &IRSTBOOT CONFIGURATION FILE USED BY RELEASE
AND LATER OF THE CELL SOFTWARE IMAGE
_aTR^]U ! Rbe 4HIS IS THE &IRSTBOOT CONFIGURATION FILE USED BY RELEASES
AND OF THE CELL SOFTWARE IMAGE
_aTR^]U ! ! ! Rbe 4HE &IRSTBOOT CONFIGURATION FILE USED BY RELEASE
_aTR^]U ! ! Rbe 4HIS IS THE &IRSTBOOT CONFIGURATION FILE USED BY RELEASES
AND LATER
1
Firstboot is run before OneCommand and performs the initial configuration of the servers. We’ll talk more
about Firstboot later in this chapter.
#(!04%2 #/.&)'52).' %8!$!4!
)N ADDITION TO THE FILES LISTED IN THIS TABLE /NE#OMMAND ALSO CREATES A DATABASE TEMPLATE
SQ\31CT\_[PcT USED FOR CREATING YOUR STARTER DATABASE 4HE TEMPLATE CONTAINS DATABASE PARAMETERS THAT
SHOULD BE USED WHENEVER CREATING DATABASES ON %XADATA SUCH AS THESE
NUX[TNbXiTNX]RaTPbTNX]RaT\T]c
NZX[[NSXPV]^bcXRbNcX\T^dc
N[\NaReaNWP]VNP[[^fNcX\T
4HE TEMPLATE FILE SQ\31CT\_[PcT SQc IS INSTALLED IN THE >A02;4N7><4 PbbXbcP]cb SQRP cT\_[PcTb
DIRECTORY DURING THE /NE#OMMAND INSTALLATION PROCESS
3TEP #HECK)0 RWTRZX_ bW
/NE OF THE FILES THE $"- #ONFIGURATOR GENERATES IS A SHELL SCRIPT USED TO VALIDATE YOUR NETWORK SETTINGS
4HE RWTRZX_ bW #HECK)0 SCRIPT READS YOUR NETWORK CONFIGURATION SETTINGS FROM ITS PARAMETER FILE
SQ\ SPc ALSO GENERATED BY THE #ONFIGURATOR AND RUNS A VARIETY OF TESTS TO VERIFY NETWORK READINESS
BEFORE DURING AND AFTER THE CONFIGURATION PROCESS #HECK)0 TESTS THE NETWORK TO CONFIRM THAT THE
FOLLOWING CONDITIONS ARE MET
v )0 ADDRESSES THAT SHOULD RESPOND TO A PING DO
v )0 ADDRESSES THAT SHOULD NOT RESPOND TO A PING DO NOT
v (OST NAMES THAT MUST BE REGISTERED IN $.3 CAN BE BOTH FORWARD AND REVERSE
RESOLVED USING THE FIRST $.3 SERVER SPECIFIED IN THE SQ\ SPc FILE
"EFORE RUNNING /NE#OMMAND THE RWTRZX_ bW SCRIPT #HECK)0 SHOULD BE RUN TO VALIDATE THE
READINESS OF YOUR CORPORATE NETWORK 4HE SCRIPT HAS SEVERAL MODES OF OPERATION THAT VALIDATE YOUR
NETWORK CONFIGURATION AT VARIOUS POINTS OF THE CONFIGURATION PROCESS !T THIS POINT YOU SHOULD RUN
#HECK)0 IN _aTNP__[hR^]UXV MODE FROM A SERVER EXTERNAL TO %XADATA PLATFORM 4HE SERVER YOU CHOOSE TO
RUN #HECK)0 FROM MUST HAVE THE SAME NETWORK VISIBILITY AS YOUR %XADATA SYSTEM WILL HAVE &OR EXAMPLE
THE SERVER MUST HAVE ACCESS TO THE SAME $.3 SERVER AND .40 SERVER AND IT MUST BE ABLE TO PING THE )0
ADDRESSES YOU LISTED IN YOUR NETWORK SETTINGS FOR %XADATA )DEALLY THE SERVER YOU CHOOSE SHOULD RUN THE
,INUX OPERATING SYSTEM BUT 5NIX SERVERS MAY WORK AS WELL #HECK)0 USES THE "ASH SHELL WHICH IS
POPULAR ON THE ,INUX PLATFORM BUT ALSO AVAILABLE ON MOST VERSIONS OF 5NIX 4HE /3 COMMANDS REQUIRED
BY #HECK)0 INCLUDE W^bc _X]V ]c_SPcT ca Rdc bTS fR VaT_ ANDWTPS "E AWARE THAT EVEN IF THESE
COMMANDS ARE AVAILABLE ON YOUR 5NIX SERVER THEY MAY PRODUCE INCOMPATIBLE OUTPUT AND CAUSE FALSE
FAILURES IN THE OUTPUT 4HE RWTRZX_ bW SCRIPT AND THE SQ\ SPc PARAMETER FILE ARE CREATED IN ,INUX5NIX
FORMAT SO WHEN YOU UPLOAD THEM FROM YOUR LAPTOP BE SURE TO TRANSFER THEM IN BINARY MODE )F YOU WANT
TO BE SURE YOU CAN USE THE S^b!d]Xg COMMAND AFTER UPLOADING THEM TO ENSURE THAT THE FORMAT IS CORRECT
BEFORE EXECUTING THE SCRIPT 5PLOAD THE RWTRZX_ bW AND SQ\ SPc FILES TO A SERVER ON YOUR NETWORK AND RUN
THE FOLLOWING COMMAND AS THE ROOT USER
RWTRZX_ bW \_aTNP__[hR^]UXV
#HECK)0 WILL PRINT ITS PROGRESS OUT TO THE SCREEN AS WELL AS BUILD A SQ\ ^dc REPORT FILE 4HE FOLLOWING LISTING
SHOWS SAMPLE OUTPUT FROM THE #HECK)0 SCRIPT
Ad]]X]VX]\^ST_aTNP__[hR^]UXV
DbX]V]P\TbTaeTa (! %' $U^d]SX]SQ\ SPcU^aP[[3=B[^^Zd_b
?a^RTbbX]VbTRcX^]
?a^RTbbX]VbTRcX^]
?a^RTbbX]VbTRcX^]3><08=
#(!04%2 #/.&)'52).' %8!$!4!
^daR^\_P]h R^\
q
?a^RTbbX]VbTRcX^]60C4F0H
6>>3) "# (% _X]VbbdRRTbbUd[[h
6>>3) "# (& _X]VbbdRRTbbUd[[h
6>>3) "# (& _X]VbbdRRTbbUd[[h
6>>3) "# (' _X]VbbdRRTbbUd[[h
?a^RTbbX]VbTRcX^]B20=
6>>3)TgP bRP] ^daR^\_P]h R^\aTb^[eTbc^"8?PSSaTbbTb
6>>3)TgP bRP] ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (& "%
6>>3) "# (& "%aTeTabTaTb^[eTbc^Tg bRP] ^daR^\_P]h R^\
6>>3) "# (& "%S^Tb]^c_X]V
6>>3)TgP bRP] h^da ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (& "&
6>>3) "# (& "&aTeTabTaTb^[eTbc^Tg bRP] ^daR^\_P]h R^\
6>>3) "# (& "&S^Tb]^c_X]V
6>>3)TgP bRP] ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (& "'
6>>3) "# (& "'aTeTabTaTb^[eTbc^Tg bRP] ^daR^\_P]h R^\
6>>3) "# (& "'S^Tb]^c_X]V
q
?a^RTbbX]VbTRcX^]2><?DC4
6>>3)TgPSQ ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (% !
6>>3) "# (% ! aTeTabTaTb^[eTbc^TgPSQ ^daR^\_P]h R^\
6>>3) "# (% ! S^Tb]^c_X]V
6>>3)TgPSQ ! ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (% !
6>>3) "# (% ! aTeTabTaTb^[eTbc^TgPSQ ! ^daR^\_P]h R^\
6>>3) "# (% ! S^Tb]^c_X]V
q
?a^RTbbX]VbTRcX^]24;;
6>>3)TgPRT[ ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (% !'
6>>3) "# (% !'aTeTabTaTb^[eTbc^TgPRT[ ^daR^\_P]h R^\
6>>3) "# (% !'S^Tb]^c_X]V
6>>3)TgPRT[ ! ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ "# (% !(
6>>3) "# (% !(aTeTabTaTb^[eTbc^TgPRT[ ! ^daR^\_P]h R^\
6>>3) "# (% !(S^Tb]^c_X]V
q
?a^RTbbX]VbTRcX^]BF8C27
q
?a^RTbbX]VbTRcX^]E8?
q
?a^RTbbX]VbTRcX^]=4C"
q
?a^RTbbX]VbTRcX^]B<C?
4HE OUTPUT REPORT GENERATED IN THE SQ\ ^dc FILE CONTAINS THE SAME INFORMATION AS WE SEE IN THE
DISPLAY )F ANY VALIDATION ERRORS OCCUR THEY ARE PREFIXED WITH %22/2 AND A MESSAGE DESCRIBING THE
FAILURE INDICATES THE PROBLEM ENCOUNTERED AND WHAT THE EXPECTED RESULTS SHOULD BE &OR EXAMPLE
#(!04%2 #/.&)'52).' %8!$!4!
?a^RTbbX]VbTRcX^]B20=
6>>3)TgP bRP] ^daR^\_P]h R^\aTb^[eTbc^"8?PSSaTbbTb
4AA>A)TgP bRP] ^daR^\_P]h R^\U^afPaSaTb^[eTbX]R^aaTRc[hc^ ## && #" '! ## && #" '
## && #" ' Tg_TRcTS ## && #" '&
q
?a^RTbbX]VbTRcX^]2><?DC4
6>>3)TgPSQ ^daR^\_P]h R^\U^afPaSaTb^[eTbc^ ' !"
6>>3) ' !" aTeTabTaTb^[eTbc^TgPSQ ^daR^\_P]h R^\
4AA>A) ' !" _X]Vb
4HE OUTPUT FROM #HECK)0 MUST CONTAIN NO ERRORS )F YOU SEE ANY ERRORS IN THE OUTPUT THEY MUST BE
CORRECTED BEFORE RUNNING /NE#OMMAND #HECK THE SQ\ SPc FILE AND MAKE SURE YOU DIDNT ENTER AN )0
ADDRESS INCORRECTLY OR MISTYPE A HOSTNAME BEFORE OPENING A DISCUSSION WITH YOUR NETWORK
ADMINISTRATOR 3OMETIMES A SIMPLE CORRECTION TO A DATA ENTRY FIELD ON THE $"- #ONFIGURATOR IS ALL THAT IS
NEEDED )F EVERYTHING LOOKS IN ORDER FROM YOUR SIDE THEN SEND THE SQ\ ^dc FILE TO YOUR NETWORK
ADMINISTRATOR FOR REMEDIATION
3TEP &IRSTBOOT
7HEN THE /RACLE HARDWARE ENGINEERS COMPLETE THEIR INSTALLATION YOU ARE READY TO BOOT UP THE COMPUTE
NODES AND STORAGE CELLS FOR THE FIRST TIME 7HEN YOU ARE READY TO BEGIN CONFIGURING %XADATA OPEN UP THE
INTERNAL +6- AND POWER UP ONE COMPUTE NODE OR STORAGE CELL AT A TIME )TS NOT IMPORTANT WHICH ORDER
YOU FOLLOW 4HE &IRSTBOOT PROCESS IS NOT DOCUMENTED IN THE /RACLE MANUALS SO WELL TAKE A MINUTE TO TALK
THROUGH THE BOOT PROCESS AND WHAT HAPPENS THE FIRST TIME YOU BOOT THE SERVERS AND STORAGE CELLS
%VERY TIME A SERVER BOOTS UP THE TcR X]Xc S _aTRT[ SCRIPT IS CALLED AT RUN LEVEL 4HIS SCRIPT CALLS
THE ^_c ^aPR[T RT[[^b RT[[5XabcQ^^c bW &IRSTBOOT SCRIPT &IRSTBOOT DETERMINES WHETHER OR NOT THE
NETWORK SETTINGS HAVE BEEN CONFIGURED 4HIS IS UNDOCUMENTED BUT IT APPEARS THAT IT IS TRIGGERED BY THE
EXISTENCE OF THE ^_c ^aPR[T RT[[^b RT[[ R^]U FILE 4HIS FILE IS CREATED AND MAINTAINED BY THE
^_c ^aPR[T RT[[^b X_R^]U _[bRaX_c )0#ONF AND CONTAINS ALL THE INFORMATION ABOUT YOUR NETWORK
CONFIGURATION )F THE FILE EXISTS IT IS ASSUMED THAT THE SYSTEM IS ALREADY CONFIGURED AND THE BOOT CYCLE
CONTINUES "UT IF THE FILE IS NOT FOUND &IRSTBOOT CALLS )0#ONF AND YOU ARE LEAD INTERACTIVELY THROUGH THE
NETWORK CONFIGURATION PROCESS )0#ONF IS USED TO SET THE FOLLOWING NETWORK SETTINGS FOR YOUR COMPUTE
NODES AND STORAGE CELLS
v .AME 3ERVER $.3
v 4IME 3ERVER .40
v #OUNTRY #ODE
v ,OCAL 4IME :ONE
v (OSTNAME
v )0 ADDRESS NETMASK GATEWAY type AND HOSTNAME FOR ALL NETWORK DEVICES 4HE
type IS REQUIRED AND USED FOR INTERNAL DOCUMENTATION IN THE RT[[ R^]U FILE 6ALID
TYPES ARE 0RIVATE -ANAGEMENT 3#!. AND /THER
&OR EXAMPLE THE FOLLOWING LISTING SHOWS THE PROMPTS FOR CONFIGURING THE MANAGEMENT NETWORK
#(!04%2 #/.&)'52).' %8!$!4!
BT[TRcX]cTaUPRT]P\Tc^R^]UXVdaT^a_aTbb4]cTac^R^]cX]dT)TcW
BT[TRcTSX]cTaUPRT TcW
8?PSSaTbb^a]^]T) (! %' ' ! &
=Tc\PbZ)!$$ !$$ !$!
6PcTfPh8?PSSaTbb^a]^]T^a]^]T) (! %'
BT[TRc]Tcf^aZch_TU^aX]cTaUPRTUa^\cWT[XbcQT[^f
)<P]PVT\T]c
!)B20=
")>cWTa
=Tcf^aZch_T)
5d[[h`dP[XUXTSW^bc]P\T^a]^]T)TgPSQ " ^daR^\_P]h R^\
7HEN YOU HAVE FINISHED ENTERING ALL YOUR NETWORK SETTINGS )0#ONF GENERATES A NEW RT[[ R^]U
FILE AND REBOOTS THE SYSTEM /NCE THE SYSTEM HAS FINISHED REBOOTING IT IS READY FOR THE /3
CONFIGURATION AND SOFTWARE INSTALLATION PERFORMED BY /NE#OMMAND
ApplyConfig
&OR QUARTER RACK AND HALF RACK CONFIGURATIONS ITS A FAIRLY TRIVIAL TASK TO ENTER ALL YOUR NETWORK
SETTINGS USING THE &IRSTBOOT PROCESS &OR FULL RACK CONFIGURATIONS HOWEVER IT CAN BE A TIME
CONSUMING AND ERROR
PRONE PROCESS 4HE P__[hR^]UXV bW SCRIPT !PPLY#ONFIG AUTOMATES THE
PROCESS !PPLY#ONFIG IS FOUND IN THE ^_c ^aPR[T Bd__^acC^^[b UXabcR^]U DIRECTORY OF ALL
COMPUTE NODES AND STORAGE CELLS )T IS BASICALLY A WRAPPER SCRIPT FOR THE )0#ONF SCRIPT WE
DISCUSSED EARLIER FOR &IRSTBOOT )N STEP WE DISCUSSED THE VARIOUS PARAMETER AND
DEPLOYMENT FILES GENERATED BY THE $"- #ONFIGURATOR !MONG THESE FILES WAS THE _aTR^]U Rbe
PARAMETER FILE SEE ALSO _aTR^]U ! Rbe _aTR^]U ! ! ! Rbe AND _aTR^]U !
! Rbe 4HE _aTR^]U Rbe FILE CONTAINS ALL THE NETWORK SETTINGS )0#ONF NEEDS TO CREATE A
RT[[ R^]U FILE FOR EACH COMPUTE NODE AND STORAGE CELL
)0#ONF MAY BE RUN INTERACTIVELY AS WE SAW IN THE &IRSTBOOT EXAMPLE ALLOWING YOU TO ENTER YOUR
NETWORK SETTINGS MANUALLY )0#ONF MAY ALSO BE RUN NON
INTERACTIVELY TAKING ITS INPUT FROM THE
_aTR^]U Rbe PARAMETER FILE 7HEN RUN IN THIS MODE )0#ONF CREATES A FULL SET OF RT[[ R^]U FILES ONE FOR
EACH COMPUTE NODE AND STORAGE CELL &OR EXAMPLE THE FOLLOWING COMMAND CREATES ALL THE REQUIRED
RT[[ R^]U FILES FOR AN %XADATA HALF RACK CONFIGURATION
Ja^^c/TgPSQ a^^cLRS ^_c ^aPR[T Bd__^acC^^[b UXabcR^]U
Ja^^c/TgPSQ UXabcR^]UL ^_c ^aPR[T RT[[^b X_R^]UK
~_aTR^]U ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S jcompany_namel _aTR^]U RbeK
VT]TaPcTP[[K
VT]TaPcT^aSTa ^_c ^aPR[T Bd__^acC^^[b UXabcR^]U WP[U
4HAT LAST PARAMETER WP[U IS A REFERENCE TO A PARAMETER FILE CONTAINING THE FACTORY DEFAULT PRIVATE )0
ADDRESS FOR ALL COMPUTE NODES AND STORAGE CELLS IN AN %XADATA HALF RACK CONFIGURATION 4HE FILES GENERATED
BY )0#ONF ARE SAVED IN THE c\_ X_R^]U DIRECTORY AS FOLLOWS
Ja^^c/TgPSQ UXabcR^]UL[b c\_ X_R^]U
RT[[ R^]U TgPRT[ ^daR^\_P]h R^\
RT[[ R^]U TgPRT[ ! ^daR^\_P]h R^\
RT[[ R^]U TgPRT[ " ^daR^\_P]h R^\
RT[[ R^]U TgPRT[ # ^daR^\_P]h R^\
RT[[ R^]U TgPRT[ $ ^daR^\_P]h R^\
#(!04%2 #/.&)'52).' %8!$!4!
RT[[ R^]U TgPRT[ % ^daR^\_P]h R^\
RT[[ R^]U TgPRT[ & ^daR^\_P]h R^\
RT[[ R^]U TgPSQ ^daR^\_P]h R^\
RT[[ R^]U TgPSQ ! ^daR^\_P]h R^\
RT[[ R^]U TgPSQ " ^daR^\_P]h R^\
RT[[ R^]U TgPSQ # ^daR^\_P]h R^\
!PPLY#ONFIG CALLS )0#ONF TO GENERATE THESE FILES AND INSTALLS THEM AS ^_c ^aPR[T RT[[^b RT[[ R^]U
IN EACH COMPUTE NODE AND STORAGE CELL 4O RUN !PPLY#ONFIG LOG IN AS ROOT TO THE FIRST COMPUTE NODE IN
YOUR SYSTEM AND RUN P__[hR^]UXV bW AS FOLLOWS
Ja^^c/TgPSQ a^^cLRS ^_c ^aPR[T Bd__^acC^^[b UXabcR^]U
Ja^^c/TgPSQ UXabcR^]UL P__[hR^]UXV bWWP[UK
^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S _aTR^]U Rbe
/NCE THE RT[[ R^]U FILES HAVE BEEN INSTALLED ON ALL SERVERS !PPLY#ONFIG USES THE SR[X COMMAND TO
RUN THE R[TP]d_N]NaTQ^^c bW SCRIPT ON ALL SERVERS !S THE NAME IMPLIES THIS SCRIPT PERFORMS CLEANUP
TASKS AND REBOOTS THE SERVERS /NCE THE SERVERS COMPLETE THE BOOT CYCLE %XADATA SHOULD BE READY TO BE
CONFIGURED USING /NE#OMMAND
7HETHER YOU CHOOSE TO CONFIGURE THE %XADATA NETWORK COMPONENTS MANUALLY USING &IRSTBOOT OR BY
RUNNING !PPLY#ONFIG YOU SHOULD VERIFY THE NETWORK CONFIGURATION ONCE AGAIN BEFORE PROCEEDING WITH
THE INSTALLATION PROCESS 2UN THE #HECK)0 POST?APPLYCONFIG PROCESS TO VERIFY YOUR CONFIGURATION AS
FOLLOWS
RS ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S
Ja^^c/TgPSQ "^]TR^\\P]SL RWTRZX_ bW \_^bcNP__[hR^]UXV
4gPSPcP3PcPQPbT<PRWX]T=Tcf^aZETaXUXRPcX^]eTabX^] (
=Tcf^aZeTaXUXRPcX^]\^ST_^bcNP__[hR^]UXVbcPacX]V
BPeX]V^dc_dcUX[TUa^\_aTeX^dbad]PbSQ\ ^dcN'"'$
DbX]V]P\TbTaeTa (! %' (U^d]SX]SQ\ SPcU^aP[[3=B[^^Zd_b
?a^RTbbX]VbTRcX^]3><08=)BD224BB
?a^RTbbX]VbTRcX^]=0<4)BD224BB
?a^RTbbX]VbTRcX^]=C?)BD224BB
?a^RTbbX]VbTRcX^]60C4F0H)BD224BB
?a^RTbbX]VbTRcX^]B20=)BD224BB
?a^RTbbX]VbTRcX^]2><?DC4)BD224BB
?a^RTbbX]VbTRcX^]24;;)BD224BB
?a^RTbbX]VbTRcX^]8;><)BD224BB
?a^RTbbX]VbTRcX^]BF8C27)BD224BB
?a^RTbbX]VbTRcX^]E8?)BD224BB
?a^RTbbX]VbTRcX^]B<C?)B<C?4\PX[BTaeTaBTccX]Vb\PX[ ^daR^\_P]h R^\!$) BD224BB
!GAIN THE OUTPUT FROM ALL TESTS MUST BE BD224BB BEFORE CONTINUING WITH THE INSTALLATION )F ERRORS
OCCUR YOU CAN REVIEW THE SQ\ ^dc FILE FOR THE REASON OF THE FAILURE
3TEP 3TAGING THE )NSTALLATION -EDIA
4HE LAST STEP BEFORE RUNNING /NE#OMMAND IS TO MAKE SURE YOU HAVE THE NECESSARY /RACLE INSTALL MEDIA
PROPERLY STAGED ON THE FIRST COMPUTE NODE /NE#OMMAND WILL HANDLE UNZIPPING THE INSTALLATION FILES TO
A WORKING DIRECTORY AND PERFORM THE ENTIRE INSTALLATION AUTOMATICALLY -Y /RACLE 3UPPORT -/3 NOTE
CONTAINS A COMPLETE LIST OF /RACLE SOFTWARE AND PATCHES SUPPORTED FOR THE %XADATA $ATABASE
#(!04%2 #/.&)'52).' %8!$!4!
-ACHINE (OWEVER THE VERSION OF /NE#OMMAND THAT CAME WITH YOUR %XADATA SYSTEM ONLY SUPPORTS
AUTOMATIC INSTALLATION OF CERTAIN SOFTWARE VERSIONS AND PATCHES
4HE Exadata Owner’s Guide THAT CAME WITH YOUR SYSTEM SPECIFIES THE 2$"-3 AND 'RID
)NFRASTRUCTURE INSTALLATION MEDIA /NE#OMMAND NEEDS FOR THE INSTALLATION 4HE PATCHES AND INSTALL MEDIA
REQUIRED BY /NE#OMMAND TENDS TO CHANGE AS NEW VERSIONS OF /NE#OMMAND ARE RELEASED #URRENTLY
THE FOLLOWING MEDIA IS REQUIRED BY /NE#OMMAND
v /RACLE 2$"-3 'RID )NFRASTRUCTURE PATCH
v _ ('' %N ! ! N;X]dg g'% %#N ^U& iX_
v _ ('' %N ! ! N;X]dg g'% %#N!^U& iX_
v _ ('' %N ! ! N;X]dg g'% %#N"^U& iX_
4HE 2EADME FILE INCLUDED WITHIN THE ^]TR^\\P]S DIRECTORY STATES WHICH BUNDLE PATCHES ARE
SUPPORTED BY THE VERSION OF /NE#OMMAND INSTALLED ON YOUR SYSTEM &OR EXAMPLE WE RECENTLY INSTALLED A
QUARTER RACK 8
SYSTEM IN OUR LAB 4HE FOLLOWING LISTING SHOWS THE PATCHES SUPPORTED BY IT IN THE
2EADME FILE
TVaT_?0C274BkQd]S[T_PcRW ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S A403<4
?0C274B0__[XTSfXcWcWXbeTabX^]
1dV !$!#'& ! !SQ\PRWX]TQd]S[T_PcRW
#HECKING -/3 NOTE WE SEE THAT "UNDLE 0ATCH FOR IS PATCH
v "UNDLE 0ATCH PATCH
v P??,INUX
X
ZIP
4HE INSTALL MEDIA ZIP FILES LISTED ABOVE MAY ALREADY BE INSTALLED ON THE FIRST COMPUTE NODE OF YOUR
%XADATA $ATABASE -ACHINE 3O ITS WORTH A LOOK AND IT MAY SAVE YOU SOME TIME TO CHECK BEFORE
DOWNLOADING THE INSTALLATION FILES )F NECESSARY DOWNLOAD AND STAGE THE INSTALLATION FILES IN THE
^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S DIRECTORY ON THE FIRST COMPUTE NODE
3TEP 2UNNING /NE#OMMAND
!LTHOUGH %XADATA CAN BE CONFIGURED MANUALLY /NE#OMMAND IS THE PREFERRED METHOD /NE#OMMAND IS
AN /RACLE
PROVIDED UTILITY CONSISTING OF SEVERAL CONFIGURATION STEPS AS OF THIS WRITING /NE#OMMAND
PROVIDES TWO VERY IMPORTANT BENEFITS TO %XADATA CUSTOMERS AND /RACLES SUPPORT STAFF &IRST IT CREATES A
LIMITED NUMBER OF STANDARDIZED AND WELL KNOWN CONFIGURATIONS WHICH MAKES THE PLATFORM MUCH
EASIER TO SUPPORT !FTER ALL WHO WANTS TO HEAR hOH )VE NEVER SEEN IT CONFIGURED THAT WAY BEFOREv WHEN
WE FINALLY GET A SUPPORT TECH ON THE PHONE 4HIS IS ONE OF %XADATAS KEY STRENGTHS 3ECOND IT PROVIDES A
SIMPLIFIED AND STRUCTURED MECHANISM FOR CONFIGURING %XADATA FROM START TO FINISH 4HIS MEANS THAT WITH
VERY LITTLE KNOWLEDGE OF %XADATA INTERNALS AN EXPERIENCED TECHNICIAN CAN INSTALL AND CONFIGURE %XADATA
IN A MATTER OF HOURS )TS UNCLEAR WHETHER /RACLE ORIGINALLY INTENDED TO PROVIDE SUPPORT FOR
/NE#OMMAND EXTERNALLY BUT ABOUT THE SAME TIME THE 8 BEGAN SHIPPING /RACLE ADDED IT TO THE
Exadata Owner’s Guide 4HE INSTRUCTIONS IN THE /WNERS 'UIDE ARE NOT VERY EXTENSIVE BUT THE 2EADME
INCLUDED WITH THE UTILITY DOES A FAIRLY GOOD JOB OF EXPLAINING HOW TO RUN IT AND WHAT TO WATCH OUT FOR
/NE#OMMAND COMES PREINSTALLED IN THE ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S DIRECTORY ON YOUR
COMPUTE NODES )F YOU NEED ACCESS TO THE LATEST VERSION OF /NE#OMMAND IT IS AVAILABLE FOR DOWNLOAD
FROM /RACLES 4ECHNOLOGY .ETWORK 4HE DOWNLOAD LINK IS PASSWORD
PROTECTED HOWEVER AND YOU WILL
NEED TO OPEN A SERVICE REQUEST WITH /RACLE 3UPPORT TO REQUEST TEMPORARY ACCESS TO DOWNLOAD IT
#(!04%2 #/.&)'52).' %8!$!4!
/NE#OMMAND IS A MULTIPLE STEP PROCESS THAT IS RUN FROM A SHELL SCRIPT CALLED ST_[^h ! bW 4HESE
STEPS CAN BE RUN END TO END OR ONE AT A TIME 4ABLE SHOWS EACH STEP IN THE /NE#OMMAND PROCESS
ALONG WITH A BRIEF DESCRIPTION OF WHAT THE STEP DOES
Table 8-7. OneCommand Steps
File Name Description
3TEP 6ALIDATES THE COMPUTE NODE FROM WHICH YOU ARE RUNNING THE INSTALLATION )T VERIFIES
THE NETWORK CONFIGURATION OF THE SERVER VERIFIES AND UNZIPS THE INSTALLATION MEDIA
VERIFIES THE RT[[ R^]U FILE AND VERIFIES THAT ALL PARAMETER AND DEPLOYMENT FILES CAN
BE FOUND IN THE ^]TR^\\P]S DIRECTORY
3TEP #REATES 33( KEYS FOR THE ROOT ACCOUNT ON ALL COMPUTE NODES AND STORAGE CELLS FOR
TRUSTED 33( AUTHENTICATION
3TEP 6ALIDATES THE RT[[ R^]U FILE ON ALL COMPUTE NODES AND STORAGE SERVERS AND VERIFIES
THAT ALL STORAGE CELLS ARE INSTALLED WITH THE SAME VERSION OF CELL SOFTWARE
3TEP 5NZIPS THE $"-3 'RID )NFRASTRUCTURE AND BUNDLE PATCH FILES
3TEP 5PDATES THE TcR W^bcb FILES
3TEP #REATES THE RT[[X_ ^aP FILE ON THE COMPUTE NODES AND THE RT[[X]Xc ^aP FILE ON THE
STORAGE CELLS
3TEP 6ALIDATES THE HARDWARE ON ALL SERVERS AND CELLS USING THE 2WTRZ7F]5F?a^UX[T
COMMAND
3TEP 6ERIFIES THE )NFINI"AND NETWORK
3TEP 6ALIDATES THE STORAGE CELLS BY RUNNING VARIOUS #ELL#,) COMMANDS &OR EXAMPLE
RT[[R[X T[Xbc_WhbXRP[SXbZ
3TEP #HECKS 2$3 USING THE dba QX] aSb _X]V COMMAND
3TEP #HECK CELL DISKS USING THE #ELL#,) RP[XQaPcTCOMMAND 4HIS COMMAND TESTS THE
PERFORMANCE CHARACTERISTICS OF YOUR CELL DISKS
3TEP 6ALIDATES THE DATE TIME AND .40 SYNCHRONIZATION ACROSS ALL COMPUTED NODES AND
STORAGE CELLS 4HE .40 SERVICE IS STARTED IF NECESSARY
3TEP 5PDATES THE FOLLOWING FILES ON THE DATABASE SERVERS ADDING SETTINGS TYPICAL FOR AN
/RACLE DATABASE SERVER ENVIRONMENT
TcR bTRdaXch [X\Xcb R^]U
TcR _a^UX[T
#(!04%2 #/.&)'52).' %8!$!4!
File Name Description
3TEP #REATES THE /RACLE ,INUX USER AND GROUP ACCOUNTS &OR EXAMPLE ^aPR[T ^X]bcP[[
SQP AND SO ON
3TEP #ONFIGURES 33( EQUIVALENCY FOR THE USER ACCOUNTS CREATED IN 3TEP
3TEP #REATES THE >A0N2ABN7><4 AND >A02;4N7><4 DIRECTORIES FOR THE 'RID )NFRASTRUCTURE
AND DATABASE SOFTWARE
3TEP #REATES ALL CELL DISKS GRID DISKS AND THE FLASH CACHE IF ENABLED ON THE $"-
#ONFIGURATOR SPREADSHEET
3TEP 0ERFORMS A SILENT INSTALLATION OF 'RID )NFRASTRUCTURE
3TEP 2UNS THE ROOT SCRIPTS a^^c bW FOR POST INSTALLATION OF THE 'RID )NFRASTRUCTURE
7HEN THIS STEP IS COMPLETE YOUR /#2 AND 6OTING DISKS WILL BE CREATED AND THE
CLUSTER SHOULD BE RUNNING
3TEP 0ERFORMS A SILENT INSTALLATION OF THE DATABASE SOFTWARE
3TEP #REATES THE DEFAULT /RACLE ,ISTENER
3TEP 2UNS Pb\RP AND CREATES YOUR !3- DISK GROUPS &OR EXAMPLE 30C0 AND A42>
3TEP 3HUTS DOWN THE CLUSTER AND OPENS PERMISSIONS ON THE >A0N2ABN7><4 USING THE
a^^cRab _[~d][^RZ command.
3TEP )NSTALLS "UNDLE 0ATCH PATCH FOR THE 'RID )NFRASTRUCTURE AND DATABASE
SOFTWARE AND RELINKS THE /RACLE SOFTWARE STACK
3TEP #ONFIGURES THE 2$3 PROTOCOL FOR THE CLUSTER )NTERCONNECT
3TEP ,OCKS DOWN THE PERMISSIONS FOR THE 'RID )NFRASTRUCTURE >A0N2ABN7><4 DIRECTORIES
AND RESTARTS THE CLUSTER STACK
3TEP #ONFIGURES CELL ALERT NOTIFICATION ON THE STORAGE CELLS !T THE END OF THIS STEP THE
#ELL#,) COMMAND P[cTaRT[[eP[XSPcT\PX[ GENERATES TEST ALERT MESSAGES )F YOU
CONFIGURED EMAIL ALERTS THEN YOU SHOULD GET A TEST MESSAGE FROM EACH CELL
3TEP 2UNS SQRP IN SILENT MODE AND CREATES YOUR STARTER DATABASE
3TEP 3ETS UP %NTERPRISE -ANAGER 'RID #ONTROL
3TEP !PPLIES SECURITY FIXES FOR IN THE STARTER DATABASE &OR EXAMPLE IT DROPS UNSECURED
USER ACCOUNTS SUCH AS GB=D;; <6<CNE84F AND 38? 4HIS STEP WILL SHUT DOWN AND
RESTART THE CLUSTER
#(!04%2 #/.&)'52).' %8!$!4!
File Name Description
3TEP 3ECURES THE %XADATA ENVIRONMENT 4HIS STEP IS OPTIONAL )T REMOVES THE 33(
AUTHENTICATION KEYS FOR THE ROOT USER ACCOUNT )F YOU RUN THIS STEP YOU WILL NOT BE
ABLE TO RUN REMOTE COMMANDS ON THE COMPUTE NODES AND STORAGE CELLS FROM THE
ROOT ACCOUNT WITHOUT ENTERING THE PASSWORD
4HE MAIN SCRIPT USED TO RUN /NE#OMMAND IS ST_[^h ! bW $EPLOY 4HE INSTALLATION STEPS
MAY BE LISTED BY RUNNING $EPLOY AS FOLLOWS
Ja^^c/TgPSQ ^]TR^\\P]SL ST_[^h ! bW X~[
CWTbcT_bX]^aSTaPaT
BcT_ ,EP[XSPcTCWXb=^STBTcd_
BcT_ ,BTcd_BB75^aA^^c
BcT_!,EP[XSPcT0[[=^STb
BcT_",D]iX_5X[Tb
BcT_#,D_SPcT4cR7^bcb
BcT_$,2aTPcT2T[[X_]Xc^aP
BcT_%,EP[XSPcT7F
BcT_&,EP[XSPcT81
BcT_',EP[XSPcT2T[[
BcT_(,?X]VASb2WTRZ
BcT_ ,Ad]2P[XQaPcT
BcT_ ,EP[XSPcTCX\T3PcT
BcT_ !,D_SPcT2^]UXV
BcT_ ",2aTPcTDbTa0RR^d]cb
BcT_ #,BTcd_BB75^aDbTab
BcT_ $,2aTPcT>aP7^\Tb
BcT_ %,2aTPcT6aXS3XbZb
BcT_ &,8]bcP[[6aXSB^UcfPaT
BcT_ ',Ad]6aXSA^^cBRaX_cb
BcT_ (,8]bcP[[ !31B^UcfPaT
BcT_! ,2aTPcT !;XbcT]Ta
BcT_! ,Ad]0b\2P
BcT_!!,D][^RZ687^\T
BcT_!",0__[h1?
BcT_!#,AT[X]ZA3B
BcT_!$,;^RZD_68
BcT_!%,BTcd_2T[[4\PX[0[Tacb
BcT_!&,Ad]3QRP
BcT_!',BTcd_4<3Q2^]ca^[
BcT_!(,0__[hBTRdaXch5XgTb
BcT_" ,ATbTRdaT<PRWX]T
#(!04%2 #/.&)'52).' %8!$!4!
.OTE /NE#OMMAND IS CONSTANTLY CHANGING TO IMPROVE THE INSTALLATION PROCESS AND TO SUPPORT ADDITIONAL BUNDLE
PATCHES 4HE NUMBER OF STEPS AND WHAT THEY DO IS VERY LIKELY TO CHANGE WITH EACH VERSION OF %XADATA 3O BE SURE
TO REVIEW THE 2%!$-% FILE FOR INSTRUCTIONS ON HOW TO RUN /NE#OMMAND ON YOUR SYSTEM BEFORE YOU BEGIN
4HERE ARE NUMBER OF WAYS $EPLOY MAY BE USED &OR EXAMPLE THE FOLLOWING COMMAND LINE
OPTIONS PROCESS ALL STEPS OF THE INSTALLATION PROCESS ONLY STOPPING IF A STEP FAILES
Ja^^c/TgPSQ ^]TR^\\P]SL ST_[^h ! bW X a "
%ACH STEP MUST COMPLETE SUCCESSFULLY BEFORE YOU CAN PROCEED TO THE NEXT /RACLE RECOMMENDS
RUNNING THE STEPS ONE AT A TIME REVIEWING THE OUTPUT AT THE END OF EACH BEFORE PROCEEDING ON TO THE
NEXT $EPLOY PROVIDES THIS CAPABILITY WITH THE b COMMAND LINE OPTION &OR EXAMPLE THE INSTALLATION
PROCEDURE WOULD LOOK SOMETHING LIKE THE FOLLOWING
Ja^^c/TgPSQ ^]TR^\\P]SL ST_[^h ! bW X b
#HECK OUTPUT FOR ERRORSx
Ja^^c/TgPSQ ^]TR^\\P]SL ST_[^h ! bW X b
#HECK OUTPUT FOR ERRORSx
Ja^^c/TgPSQ ^]TR^\\P]SL ST_[^h ! bW X b!
#HECK OUTPUT FOR ERRORSx
Ja^^c/TgPSQ ^]TR^\\P]SL ST_[^h ! bW X b"
AND SO ONx
$EPLOY TAKES AS INPUT THE PARAMETERS FROM THE FILES YOU GENERATED EARLIER USING THE $"-
#ONFIGURATOR ,OG FILES ARE CREATED EACH TIME $EPLOY IS CALLED TO EXECUTE A CONFIGURATION STEP AND FOR
SOME STEPS IT DYNAMICALLY GENERATES AND EXECUTES A SHELL SCRIPT THAT CARRIES OUT ALL THE TASKS REQUIRED
2EVIEWING THESE FILES CAN BE VERY USEFUL IN DETERMINING WHY A STEP FAILED 4HE LOG FILES AND DYNAMICALLY
GENERATED SHELL SCRIPTS ARE STORED IN THE ^]TR^\\P]S c\_ DIRECTORY
4HE OUTPUT GENERATED BY THE VARIOUS INSTALLATION STEPS VARIES QUITE A BIT "UT IN GENERAL $EPLOY
DISPLAYS SOME HEADER INFORMATION TELLING YOU WHAT STEP IT IS RUNNING WHERE TO FIND THE LOG FILE AND
WHETHER IT COMPLETED SUCCESSFULLY &OR EXAMPLE THE FOLLOWING OUTPUT WAS GENERATED BY RUNNING STEP
ON THE QUARTER RACK SYSTEM IN OUR LAB
BRaX_cbcPacTSUX[TXb ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S c\_ BC4? TgPSQ
! $ " '# [^V
,,,,,,,,,,, BTcd_BB75^aA^^c1TVX],,,,,,,,,,,,,,,
BTccX]Vd_bbWU^aa^^c^]0;;]^STb
2WTRZX]V]^STbX] ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S P[[NVa^d_
8=5>)FTaTad]]X]V ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S bTcbbW ;X]dg bW b RH W
^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S P[[NVa^d_ _fT[R^\T ]= g
bTcbbW ;X]dg bW3^]T
BD224BB)Ad]]X]VBTcBB7R^\_[TcTSbdRRTbbUd[[h ATcda]BcPcdb) BcT_
TgPSQ )5aX<Ph " ) ()$ 23C!
TgPSQ !)5aX<Ph " ) ()$ 23C!
#(!04%2 #/.&)'52).' %8!$!4!
TgPSQ ")5aX<Ph " ) ()$ 23C!
TgPSQ #)5aX<Ph " ) ()$ 23C!
TgPRT[ )5aX<Ph " ) ()$ 23C!
TgPRT[ !)5aX<Ph " ) ()$ 23C!
TgPRT[ ")5aX<Ph " ) ()$ 23C!
TgPRT[ #)5aX<Ph " ) ()$ 23C!
TgPRT[ $)5aX<Ph " ) ()$ 23C!
TgPRT[ %)5aX<Ph " ) ()$ 23C!
TgPRT[ &)5aX<Ph " ) ()$ 23C!
BD224BB)Ad]]X]VSR[XR^\_[TcTSbdRRTbbUd[[h ATcda]BcPcdb) BcT_
8=5>)2^_hX]VaT`dXaTSUX[Tbc^P[[]^STb _[TPbTfPXc
8=5>)FPXcX]VU^aR^_XTbc^R^\_[TcT
8=5>)3XSBB7bTcd_R^aaTRc[h
CX\Tb_T]cX]bcT_ BTcd_BB75^aA^^c,%(bTR^]Sb
,,,,,,,,,,, BTcd_BB75^aA^^cR^\_[TcTS,,,,,,,,,,,,,,,
Ad]]X]VbX]V[TbcT_PcPcX\T 4gXcX]V]^f
4gXcX]V
BRaX_cS^]TUX[TXb ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S c\_ BC4? TgPSQ ! $ " '# [^V
Upgrading Exadata
7ITH ALL THE COMPANIES OUT THERE ADOPTING THE %XADATA PLATFORM WE EXPECT HARDWARE UPGRADES TO BE AN
INCREASINGLY POPULAR TOPIC IN THE NEAR FUTURE /UR FORAY INTO THE %XADATA SPACE BEGAN OVER A YEAR AGO
WITH AN %XADATA 6 1UARTER 2ACK CONFIGURATION ! COUPLE OF MONTHS AGO WE UPGRADED OUR SYSTEM TO A
HALF RACK /F COURSE THE 6S ARE NO LONGER IN PRODUCTION SO OUR UPGRADE CAME IN THE FORM OF TWO 8
DATABASE SERVERS AND FOUR STORAGE CELLS 4HE CONFIGURATION OPTIONS WE CONSIDERED WERE AS FOLLOWS
v #ONFIGURE THE NEW 8 EQUIPMENT AS A SEPARATE 2!# CLUSTER AND STORAGE GRID
CREATING TWO SOMEWHAT ASYMMETRIC QUARTER RACK CONFIGURATIONS WITHIN THE SAME
%XADATA ENCLOSURE /RACLE REFERS TO THIS AS A hSPLIT RACKv CONFIGURATION
v !DD THE NEW 8 EQUIPMENT TO OUR EXISTING QUARTER RACK CLUSTER EFFECTIVELY
UPGRADING IT TO A HALF RACK
#REATING A .EW 2!# #LUSTER
4HE $"- #ONFIGURATOR DOESNT DIRECTLY SUPPORT UPGRADING A SYSTEM IN THIS MANNER BUT WITH A FEW
ADJUSTMENTS IT CAN BE USED TO GENERATE ALL THE FILES /NE#OMMAND NEEDS TO PERFORM THE INSTALLATION
/NCE THE PARAMETER AND DEPLOYMENT FILES ARE UPLOADED TO %XADATA YOU SHOULD HAVE NO PROBLEM
RUNNING THROUGH ALL OF THE CONFIGURATION STEPS WITHOUT IMPACTING YOUR EXISTING CLUSTER /NE COWORKER
ACTUALLY USED THIS PROCESS TO CREATE A SEPARATE %XADATA CONFIGURATION ON THE NEW EQUIPMENT WHILE
LEAVING THE EXISTING SYSTEM UNTOUCHED
&OR THE MOST PART YOU SIMPLY FILL IN THE $"- #ONFIGURATOR SPREADSHEET AS IF YOU ARE CREATING A
TYPICAL QUARTER RACK CONFIGURATION 4HE TRICKY PART IS MAKING SURE THE SEQUENTIALLY ASSIGNED NETWORK )0
ADDRESSES DONT CONFLICT WITH YOUR CURRENT SYSTEM 4HERE ARE A FEW ITEMS YOU WILL NEED TO CONSIDER WHEN
USING THE $"- #ONFIGURATOR FOR THIS TYPE OF %XADATA UPGRADE
Name Prefixes:n )T IS NOT REQUIRED BUT YOU MAY WANT TO SET YOUR $ATABASE
-ACHINE .AME $ATABASE 3ERVER "ASE .AME AND 3TORAGE 3ERVERS "ASE .AMES
VALUES TO MATCH YOUR EXISTING %XADATA CONFIGURATION 4HAT WAY IF YOU EVER
#(!04%2 #/.&)'52).' %8!$!4!
DECIDE TO MERGE THESE SERVERS INTO YOUR OLD CLUSTER YOU WONT HAVE TO MAKE
CHANGES TO THE HOST NAMES &OR EXAMPLE OUR QUARTER RACK CONFIGURATION HAD
DATABASE HOST NAMES OF T]ZSQ AND T]ZSQ ! !DDING THE NEW SERVERS CONTINUED
WITH THE NAMES T]ZSQ " AND T]ZSQ # ,IKEWISE THE STORAGE CELL HOST NAMES
CONTINUED WITH T]ZRT[ # THROUGH T]ZRT[ &
Client Access SCAN Name: 4HIS PROCEDURE WILL BE CREATING A NEW 2!# CLUSTER
SO YOU WILL NEED A NEW 3#!. NAME FOR IT 9OU WILL ALSO NEED TO SEE THAT IT AND ALL
OF THE OTHER NEW HOST NAMES ARE PROPERLY REGISTERED IN YOUR COMPANYS $.3
SERVER JUST AS YOU DID WHEN YOUR EXISTING %XADATA SYSTEM WAS INSTALLED
Country Code / Time Zone: /F COURSE THESE SETTINGS SHOULD MATCH YOUR
EXISTING %XADATA SYSTEM
NTP and DNS Servers: 4HESE SHOULD ALSO MATCH YOUR EXISTING %XADATA
ENVIRONMENT
Oracle Database Machine Model: 4HIS SETTING DETERMINES HOW MANY COMPUTE
NODES AND STORAGE CELLS THE #ONFIGURATOR WILL USE WHEN CREATING HOST NAMES AND
)0 ADDRESSES 9OU WILL WANT TO SET THIS TO 8
1UARTER 2ACK SINCE IT IS THE
CLOSEST CHOICE TO WHAT YOU ARE ACTUALLY CONFIGURING 2EMEMBER THOUGH THAT
UPGRADING FROM A QUARTER TO HALF RACK ADDS FOUR STORAGE CELLS NOT THREE
Oracle Exadata Storage Server Nodes: 9OU WILL NEED TO ADJUST THIS NUMBER TO
REFLECT THE ACTUAL NUMBER OF STORAGE CELLS IN THE UPGRADE 4HE DEFAULT NUMBER OF
STORAGE CELLS IN A QUARTER RACK CONFIGURATION IS "UT SINCE YOU ARE UPGRADING TO
A HALF RACK YOU WILL NEED TO SET THIS TO
Network IP Addresses: 9OU SHOULD CONTINUE TO USE THE NETWORKS YOU
CONFIGURED FOR THE %XADATA RACK YOU ARE UPGRADING !S YOU ENTER THE STARTING )0
ADDRESSES FOR HOSTS IN THE $"- #ONFIGURATOR TAKE CARE THAT YOU ARE NOT CREATING
ANY ADDRESS CONFLICTS WITH EXISTING HARDWARE ON YOUR NETWORK
O/S User & Group Accounts: )T IS NOT REQUIRED BUT YOU SHOULD USE THE SAME
USERGROUP NAMES AND USERGROUP )$S WHEN CONFIGURING YOUR NEW CLUSTER 4HIS
IS ESPECIALLY TRUE IF THERE IS ANY CHANCE THESE USER ACCOUNTS WILL EVER INTERACT
BETWEEN THE NEW SYSTEM AND THE OLD SYSTEM /NE#OMMAND WILL NOT ESTABLISH
USER EQUIVALENCY BETWEEN THE OLD AND NEW SERVERS FOR YOU 3O THAT MUST BE
DONE MANUALLY AFTER THE UPGRADE IS COMPLETE
7HEN YOU ARE FINISHED ENTERING YOUR SETTINGS CLICK THE 'ENERATE BUTTON ON THE RIGHT SIDE OF THE
SPREADSHEET 4HIS CREATES A NETWORK TOPOLOGY REPORT BELOW THE DATA ENTRY AREA OF THE SPREADSHEET 4HIS
REPORT AREA OF THE SPREADSHEET IS A PLACE WHERE YOU CAN MANUALLY OVERRIDE SOME OF THE SETTINGS GENERATED
IN THE DATA ENTRY AREA OF THE SPREADSHEET 9OU WILL NEED TO SCROLL DOWN AND ADJUST A FEW SETTINGS IN THIS
AREA BEFORE YOU ARE READY TO CREATE YOUR PARAMETER FILES AND UPLOAD THEM TO %XADATA 9OU WILL NOTICE THAT
THE TOPOLOGY REPORT SHOWS INCORRECT HOST NAMES FOR THE COMPUTE NODES AND STORAGE CELLS /F COURSE THIS
IS BECAUSE THE $"- #ONFIGURATOR ASSUMES THIS IS A NEW CONFIGURATION NOT AN UPGRADE 4HE HOST NAMES
FOR THE COMPUTE NODES ARE POSTFIXED WITH n AND THE STORAGE CELLS ARE POSTFIXED WITH n -AKE THE
NECESSARY CHANGES TO THE HOST NAMES IN THE REPORT SO THEY REFLECT COMPUTE NODE NAMES OF n AND
STORAGE CELL HOST NAMES OF n /NCE YOUVE MADE ALL THE NECESSARY CHANGES CLICK THE #REATE #ONFIG
&ILES BUTTON TO GENERATE YOUR PARAMETER AND DEPLOYMENT FILES 4AKE A FEW MINUTES TO REVIEW CONTENTS OF
THE FILES TO BE SURE THEY ARE CORRECT BEFORE YOU UPLOAD THEM TO THE ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S
DIRECTORY OF THE FIRST NEW COMPUTE NODE {machine_name}SQ "
#(!04%2 #/.&)'52).' %8!$!4!
&ROM THIS POINT FORWARD THE PROCESS IS NO DIFFERENT THAN IT IS FOR A FRESH INSTALL &IRST YOU WILL NEED TO
CONFIGURE THE NETWORK COMPONENTS USING &IRSTBOOT OR !PPLY#ONFIG AS WE DISCUSSED EARLIER 4HEN SIMPLY
LOG IN TO THE FIRST NEW COMPUTE NODE AS ROOT AND RUN THROUGH THE $EPLOY CONFIGURATION STEPS 7HEN
YOU ARE FINISHED YOU WILL HAVE A NEW 2!# CLUSTER COMPLETE WITH STARTER DATABASE
5PGRADING THE %XISTING #LUSTER
)F YOURE UPGRADING YOUR %XADATA TO A HALF OR FULL RACK CONFIGURATION AND WANT TO INTEGRATE THE NEW SERVERS
AND CELLS INTO YOUR EXISTING 2!# CLUSTER YOU MUST CONFIGURE THE NEW SERVERS AND CELLS MANUALLY 4HE
Exadata Owner’s Guide HAS A SECTION CALLED h#ONFIGURING THE 2EPLACEMENT $ATABASE 3ERVERv THAT
DISCUSSES THE PROCESS IN DETAIL &IRST WELL TAKE A LOOK AT THE BASIC STEPS FOR CONFIGURING THE NEW DATABASE
4HEN WELL TAKE A LOOK AT HOW YOU CAN ADD THE NEW CELLS TO YOUR EXISTING STORAGE GRID
#AUTION 4HE STEPS IN THIS SECTION ARE NOT INTENDED TO BE A COMPREHENSIVE GUIDE AND ARE SUBJECT TO CHANGE
2EFER TO YOUR %XADATA DOCUMENTATION FOR DETAILS SPECIFIC TO YOUR VERSION OF %XADATA
Configuring Database Servers
4HE PROCESS FOR CONFIGURING THE NEW DATABASE SERVERS ARE AS FOLLOWS
1. 5PGRADE THE FIRMWARE ON YOUR )" 3WITCH TO THE CURRENT RELEASE OR LATEST PATCH
4HE /RACLE HARDWARE TECHNICIAN WHO INSTALLED YOUR NEW HARDWARE CAN DO THIS
FOR YOU OR YOU CAN DOWNLOAD THE LATEST PATCH AND INSTALL IT YOURSELF
2ECOMMENDED FIRMWARE PATCHES CAN BE FOUND IN -/3 NOTE
2. )F POSSIBLE UPDATE THE /RACLE 'RID )NFRASTRUCTURE AND DATABASE SOFTWARE TO THE
MOST CURRENT BUNDLE PATCH FOR THE VERSION OF THE SOFTWARE YOUR EXISTING SYSTEM
IS RUNNING )DEALLY THE SOFTWARE SHOULD BE RUNNING AT THE LATEST RELEASE AND
BUNDLE PATCH
3. 5SE THE $"- #ONFIGURATOR TO GENERATE )0 ADDRESSES AND HOST NAMES FOR THE
NEW COMPUTE NODES AND STORAGE CELLS
4. 2EGISTER THE NEW HOST NAMES AND )0 ADDRESSES IN YOUR $.3 SERVER
5. "OOT YOUR NEW COMPUTE NODES ONE AT A TIME 4HE FIRST TIME THEY ARE BOOTED
)0#ONFIG WILL START AUTOMATICALLY ALLOWING YOU TO ENTER YOUR NETWORK SETTINGS
6. /N THE DATABASE SERVERS COPY THE FOLLOWING FILES FROM ONE OF YOUR EXISTING
DATABASE SERVERS TO THE NEW DATABASE SERVERS
v TcR bTRdaXch [X\Xcb
4HIS FILE IS THE SAME ON ALL COMPUTE NODES
v TcR _a^UX[T
4HIS FILE IS THE SAME ON ALL COMPUTE NODES
v TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X_ ^aP
4HIS FILE IS THE SAME ON ALL COMPUTE NODES
#(!04%2 #/.&)'52).' %8!$!4!
v TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X]Xc ^aP
4HE RT[[X]Xc ^aP FILE SHOULD BE UPDATED WITH THE PRIVATE
)NFINI"AND )0 ADDRESS OF THE COMPUTE NODE WHERE THE FILE IS
INSTALLED %ACH COMPUTE NODE WILL HAVE A UNIQUE VERSION OF THIS
FILE
7. 5PDATE THE TcR W^bcb FILE WITH THE CONTENTS OF THE _aXeNXQNVa^d_ FILE
GENERATED FROM $"- #ONFIGURATOR 5SE THE HOSTS FILE ON ONE OF YOUR OTHER
COMPUTE NODES AS A GUIDE )T IS VERY IMPORTANT NOT TO REMOVE COMMENTS SUCH
AS THE FOLLOWING
1468=6T]TaPcTSQh4gPSPcP 3>=>C<>385H
8. 5PDATE THE ^_c ^aPR[T Bd__^acC^^[b ^]TR^\\P]S Va^d_ FILES ON ALL
COMPUTE NODES ADDING THE NODES TO THE CONFIGURATION
9. #REATE THE /3 USERS AND GROUPS AS THEY ARE DEFINED ON THE OTHER COMPUTE
NODES 4HE UX]VTa COMMAND MAY BE USED TO COMPARE USER ACCOUNT DEFINITIONS
AS FOLLOWS
10. 3ET UP 33( USER EQUIVALENCY BETWEEN THE NEW AND OLD COMPUTE NODES FOR THE
/RACLE SOFTWARE OWNER
11. &OLLOW THE STEPS IN THE /WNERS 'UIDE FOR CLONING THE 'RID )NFRASTRUCTURE AND
DATABASE HOMES TO A REPLACEMENT SERVER 4HAT SECTION OF THE /WNERS 'UIDE IS
DISCUSSING THE REPLACEMENT OF A FAILED DATABASE SERVER /F COURSE YOU WILL NOT
BE REPLACING A SERVER SO YOU CAN SKIP OVER ANY STEPS HAVING TO DO WITH
REMOVING A FAILED SERVER FROM THE CLUSTER
.OTE 4HE %XADATA PLATFORM CONSISTS OF MULTIPLE SOFTWARE AND HARDWARE COMPONENTS THAT REQUIRE PERIODIC
UPDATES 4HESE UPDATES COME IN THREE FORMS 3TORAGE 3ERVER PATCHES $ATABASE 3ERVER PATCHES BUNDLE PATCHES
AND )NFINI"AND 3WITCH UPDATES #ONSIDERING THE COMPLEXITY OF %XADATA IT IS MORE IMPORTANT THAN EVER TO KEEP YOUR
SYSTEM FAIRLY CURRENT WITH THE LATEST SOFTWARE AND FIRMWARE PATCHES ! WORD OF CAUTION THOUGH WE RECOMMEND THAT
YOU WAIT AT LEAST A MONTH AFTER A PATCH IS AVAILABLE BEFORE INSTALLING IT ON A PRODUCTION SYSTEM %VEN FOR TEST AND
DEVELOPMENT SYSTEMS WE RECOMMEND YOU WAIT AT LEAST n WEEKS AFTER A PATCH IS AVAILABLE BEFORE INSTALLING IT
/RACLE MAINTAINS A DOCUMENT ON -Y /RACLE 3UPPORT -/3 CONTAINING A LIST OF ALL SUPPORTED SOFTWARE AND PATCHES
AVAILABLE FOR %XADATA STARTING WITH VERSION 4HE DOCUMENT IS CONTINUALLY UPDATED WITH USEFUL INFORMATION
INSTRUCTIONS AND LINKS TO THE LATEST PATCHES AS THEY BECOME AVAILABLE 4HE DOCUMENT IS -/3 NOTE
h$ATABASE -ACHINE AND %XADATA 3TORAGE 3ERVER 2ELEASE 3UPPORTED 6ERSIONSv 3EE !PPENDIX " FOR A LIST OF
OTHER USEFUL -/3 NOTES RELATING TO THE %XADATA PLATFORM
#(!04%2 #/.&)'52).' %8!$!4!
Configuring Storage Cells
!DDING NEW CELLS TO YOUR EXISTING STORAGE GRID IS A FAIRLY SIMPLE PROCESS 4HERE MAY BE OTHER WAYS TO
SIMPLIFY THE PROCESS BY CLONING AN EXISTING STORAGE CELL BUT WELL TAKE A LOOK AT THE MANUAL PROCESS SO YOU
CAN SEE THE COMMANDS AND FILES INVOLVED 4HE PROCESS IS AS FOLLOWS
1. 9OUR NEW STORAGE CELLS WILL COME WITH THE LATEST VERSION OF THE %XADATA CELL SOFTWARE
INSTALLED "EFORE YOU BEGIN CHECK THE VERSION OF SOFTWARE ON THE NEW CELLS AND
UPGRADE YOUR OLD CELLS TO MATCH 4HE X\PVTX]U^ COMMAND CAN BE USED TO DISPLAY THIS
INFORMATION
2. ,OCATE THE RT[[X]Xc ^aP FILE ON ONE OF YOUR OLD CELLS AND COPY IT OVER TO THE NEW CELLS
-ODIFY THE RT[[X]Xc ^aP FILE ON THE NEW CELLS AND CHANGE THE X_PSSaTbb FIELD TO THE
PRIVATE )NFINI"AND ADDRESS OF THE CELL ,OGIN AS THE ROOT ACCOUNT WHEN CONFIGURING THE
STORAGE CELL 4HE >BB2>=5 ENVIRONMENT VARIABLE SHOULD POINT TO THE LOCATION OF THE
CORRECT RT[[X]Xc ^aP FILE
3. 5PDATE THE TcR W^bcb FILE WITH THE CONTENTS OF THE _aXeNXQNVa^d_ FILE GENERATED FROM
$"- #ONFIGURATOR 5SE THE HOSTS FILE ON ONE OF YOUR OTHER STORAGE CELLS AS A GUIDE )T
IS VERY IMPORTANT NOT TO REMOVE COMMENTS SUCH AS THE FOLLOWING
1468=6T]TaPcTSQh4gPSPcP 3>=>C<>385H
4. 2EBOOT THE NEW CELLS AND VERIFY THAT THEY RESTART PROPERLY
5. 6ERIFY THAT THE CELL SERVICES ARE RUNNING USING THE #ELL#,) COMMAND [XbcRT[[ )F THE
CELL SERVICES ARE DOWN YOU MAY SEE AN ERROR SUCH AS THIS
24;; $ #)2^]]TRc4aa^a ETaXUhcWPc<P]PVT\T]cBTaeTaXb[XbcT]X]V
PccWTb_TRXUXTS7CC?_^ac)''''
2T[[BTaeXRTb\dbcQTbc^__TSP]SbcPacTSUa^\cWTa^^c^aRT[[PS\X]
dbTaPRR^d]cb CWTU^[[^fX]VR^\\P]Sb\PhQTdbTSc^\P]dP[[hbWdcS^f]
P]SbcPacd_cWTbTaeXRTb)
2T[[2;8-P[cTaRT[[bWdcS^f]bTaeXRTbP[[*
2T[[2;8-P[cTaRT[[bcPacd_bTaeXRTbP[[*
6. #ONFIGURING THE STORAGE CELL IS DONE USING THE 0;C4A24;; COMMAND &OR EXAMPLE
STORAGE CELL ALERT NOTIFICATION CAN BE CONFIGURED USING THE FOLLOWING COMMAND
0;C4A24;;b\c_BTaeTa,\PX[ ^daR^\_P]h R^\
b\c_5a^\0SSa,4gPSPcP/^daR^\_P]h R^\
b\c_5a^\,4gPSPcP
b\c_C^0SSa,P[[ SQP/^daR^\_P]h R^\P[[ bP/^daR^\_P]h R^\
]^cXUXRPcX^]?^[XRh,RaXcXRP[fPa]X]VR[TPa
]^cXUXRPcX^]<TcW^S,\PX[b]\_
7. 9OUR CURRENT CELL CONFIGURATION MAY BE DISPLAYED USING THE ;8BC24;;34C08;
COMMAND /NCE YOU ARE FINISHED CONFIGURING THE CELL STOP AND RESTART THE CELL
SERVICES TO PICK UP THE NEW SETTINGS
8. #ONFIGURE THE CELL SMART FLASH CACHE USING THE 2A40C45;0B7202740;; COMMAND
9. #ONFIGURE ALL CELL DISKS USING THE 2A40C424;;38B:0;; COMMAND
#(!04%2 #/.&)'52).' %8!$!4!
10. 5SING ONE OF YOUR OLD STORAGE CELLS FOR REFERENCE CREATE YOUR GRID DISKS USING THE
2A40C46A8338B: COMMAND 7E DISCUSS USING THIS COMMAND IN #HAPTER "E SURE
YOU CREATE YOUR GRID DISKS IN THE PROPER ORDER AS THIS WILL IMPACT THE PERFORMANCE OF
THE DISKS )F YOU STILL HAVE THE ORIGINAL CREATE SCRIPTS GENERATED AND RUN BY
/NE#OMMAND ON THE FIRST COMPUTE NODE OF YOUR EXISTING CLUSTER YOU WILL FIND ALL THE
COMMANDS YOU NEED FOR CREATING YOUR GRID DISKS IN THE PROPER ORDER 4HE SCRIPTS ARE
RaTPcT^Rae^cTSV bW AND 2aTPcT6aXS3XbZ bWAND SHOULD BE LOCATED IN THE
^_c ^aPR[T Bd__^acC^^[b c\_ DIRECTORY )F THESE FILES ARE NOT AVAILABLE YOU CAN USE
THE SIZE AND OFFSET ATTRIBUTES OF THE ;8BC6A8338B:34C08; COMMAND TO DETERMINE
THE PROPER SIZE AND CREATION ORDER FOR THE GRID DISKS
11. /NCE YOU ARE FINISHED CONFIGURING THE CELLS AND CREATING YOUR GRID DISKS YOU CAN ADD
THE CELL TO THE STORAGE GRID 4HIS IS DONE ON THE COMPUTE NODES BY ADDING THE CELLS
PRIVATE )NFINI"AND NETWORK )0 ADDRESS TO THE TcR ^aPR[T RT[[ ]Tcf^aZ
R^]UXV RT[[X_ ^aP FILE ON EACH DATABASE SERVER
Summary
#ONFIGURING %XADATA IS A VERY DETAILED PROCESS AND SOME THINGS TEND TO CHANGE SOMEWHAT AS NEW
VERSIONS OF THE SOFTWARE BECOME AVAILABLE 4HIS CHAPTER DISCUSSED SOME OF THE MAIN POINTS OF
CONFIGURING %XADATA COMPUTE NODES AND STORAGE CELLS BUT IT IS NOT INTENDED TO BE A SUBSTITUTE FOR THE
OFFICIAL /RACLE DOCUMENTATION /RACLE HAS DONE AN EXCELLENT JOB OF DOCUMENTING THE PLATFORM AND YOU
WILL FIND THE Owner’s Guide AND User’s Guide TO BE INVALUABLE ASSETS WHEN LEARNING THE INS AND OUTS OF
CONFIGURING %XADATA 4HERE IS SOME OVERLAP IN SUBJECT MATTER COVERED IN THIS CHAPTER WITH THE TOPICS
DISCUSSED IN #HAPTERS AND SO YOU MIGHT FIND THEM HELPFUL AS A CROSS REFERENCE FOR SOME OF THE
CONFIGURATION TASKS DISCUSSED HERE
CHAPTER 9
Recovering Exadata
9OU MAY HAVE HEARD THE SAYING hDISK DRIVES SPIN AND THEN THEY DIEv )TS NOT SOMETHING WE LIKE TO THINK
ABOUT BUT FROM THE MOMENT YOU POWER UP A NEW SYSTEM YOUR DISK DRIVES BEGIN AGING $ISK DRIVES HAVE
COME A LONG WAY IN THE PAST YEARS AND TYPICAL LIFE EXPECTANCY HAS IMPROVED DRAMATICALLY !T THE END
OF THE DAY THOUGH ITS A MATTER OF hWHENv A DISK WILL FAIL NOT hIFv !ND WE ALL KNOW THAT MANY DISK DRIVES
FAIL LONG BEFORE THEY SHOULD +NOWING HOW TO DIAGNOSE DISK FAILURES AND WHAT TO DO WHEN THEY OCCUR HAS
GENERALLY BEEN THE RESPONSIBILITY OF THE SYSTEM ADMINISTRATOR OR STORAGE ADMINISTRATOR &OR MANY $"!S
%XADATA IS GOING TO CHANGE THAT -ANY %XADATA SYSTEMS OUT THERE ARE BEING MANAGED ENTIRELY BY THE $"!
STAFF 7HETHER OR NOT THIS IS THE CASE IN YOUR DATA CENTER THE PROCEDURE FOR RECOVERING FROM A DISK FAILURE
ON %XADATA IS GOING TO BE A LITTLE DIFFERENT THAN YOU ARE USED TO
/RACLE DATABASE SERVERS HAVE TRADITIONALLY REQUIRED TWO TYPES OF BACKUP OPERATING SYSTEM BACKUPS
AND DATABASE BACKUPS %XADATA ADDS STORAGE CELLS TO THE MIX AND WITH THAT COMES A WHOLE NEW
SUBSYSTEM THAT MUST BE PROTECTED AND ON OCCASION RESTORED 4HE STORAGE CELL IS A FAIRLY RESILIENT PIECE OF
HARDWARE THAT EMPLOYS ,INUX SOFTWARE 2!)$ TO PROTECT THE OPERATING SYSTEM VOLUMES !S SUCH IT IS
UNLIKELY THAT A DISK FAILURE WOULD NECESSITATE AN OPERATING SYSTEM RESTORE 4HE MORE LIKELY CAUSES WOULD
BE HUMAN ERROR A FAILED PATCH INSTALL OR A BUG 2EMEMBER THAT THESE PHYSICAL VOLUMES ALSO CONTAIN GRID
DISKS DATABASE VOLUMES SO A LOSS ONE OF THESE DISKS WILL MOST LIKELY MEAN A LOSS OF DATABASE STORAGE AS
WELL /RACLE HAS ENGINEERED SEVERAL FEATURES INTO %XADATA TO PROTECT YOUR DATA AND REDUCE THE IMPACT OF
SUCH FAILURES 4HIS CHAPTER WILL DISCUSS SOME OF THE MORE COMMON STORAGE FAILURE SCENARIOS HOW TO
DIAGNOSE THEM AND HOW TO RECOVER WITH MINIMAL DOWNTIME
■ .OTE /NE OF THE MOST CHALLENGING ASPECTS OF WRITING THIS CHAPTER IS THE RAPIDLY CHANGING NATURE OF THE
COMMANDS AND SCRIPTS WE WILL BE DISCUSSING )N MANY CASES RECOVERY TASKS WILL HAVE YOU WORKING VERY CLOSELY
WITH THE HARDWARE LAYER OF %XADATA 3O AS YOU READ THIS CHAPTER KEEP IN MIND THAT WITH EACH NEW VERSION OF
%XADATA HARDWARE AND SOFTWARE THE COMMANDS AND SCRIPTS DISCUSSED IN THIS CHAPTER MAY CHANGE "E SURE TO
CHECK THE /RACLE DOCUMENTATION FOR THE LATEST UPDATES TO THE COMMANDS AND SCRIPTS DISCUSSED HERE
Exadata Diagnostic Tools
%XADATA IS A HIGHLY COMPLEX BLEND OF HARDWARE AND SOFTWARE THAT WORK TOGETHER TO PRODUCE AN INCREDIBLY
RESILIENT DELIVERY PLATFORM 4HE COMPLEXITY OF THE PLATFORM CAN BE A BIT DAUNTING AT FIRST 4HERE ARE SIMPLY
A LOT OF MOVING PARTS THAT ONE MUST UNDERSTAND IN ORDER TO MAINTAIN THE PLATFORM EFFECTIVELY /RACLE
#(!04%2 2%#/6%2).' %8!$!4!
PROVIDES A WEALTH OF DIAGNOSTIC TOOLS THAT CAN BE USED TO VERIFY ANALYZE AND REPORT IMPORTANT
INFORMATION ABOUT THE CONFIGURATION AND HEALTH OF THE SYSTEM )N THIS SECTION WELL DISCUSS SOME OF THOSE
TOOLS AND HOW TO USE THEM
3UN $IAGNOSTICS SUNDIAGSH
)NSTALLED ON EVERY %XADATA DATABASE SERVER AND STORAGE CELL IS THE bd]SXPV bW SCRIPT LOCATED IN THE
^_c ^aPR[T Bd__^acC^^[b DIRECTORY )F FOR SOME REASON YOU DONT FIND IT INSTALLED ON YOUR SYSTEM YOU
CAN DOWNLOAD IT FROM -Y /RACLE 3UPPORT 2EFER TO -/3 $OC )$ 4HIS SCRIPT IS RUN FROM THE ROOT
ACCOUNT AND COLLECTS DIAGNOSTIC INFORMATION NEEDED FOR TROUBLESHOOTING DISK FAILURES 4HE FILES IT COLLECTS
ARE BUNDLED IN THE FAMILIAR cPa FORMAT AND THEN COMPRESSED USING BZIP
sundiag.sh Output
4HE LOG FILES PRODUCED BY bd]SXPV bW ARE NAMED USING THE HOST NAME OF THE SERVER FOLLOWED BY A
DESCRIPTIVE NAME AND POSTFIXED WITH THE DATE AND TIME OF THE REPORT &OR EXAMPLE RUNNING THE SCRIPT ON
OUR LAB SYSTEM PRODUCED AN OUTPUT FILE CREATED BY THE ST\bV COMMAND NAMED AS FOLLOWS
T]ZSQ NS\TbVN! N N!'N (N#! ^dc
.OW LETS TAKE A LOOK AT THE DIAGNOSTIC FILES COLLECTED BY bd]SXPVb bW
messages: 4HIS IS A COPY OF THE ePa [^V \TbbPVTb FILE FROM YOUR SYSTEM 4HE
\TbbPVTb FILE IS ROTATED AND AGED OUT AUTOMATICALLY BY THE OPERATING SYSTEM )F
YOUR SYSTEM HAS BEEN RUNNING FOR A WHILE YOU WILL HAVE SEVERAL OF THESE FILES
ENUMERATED IN ASCENDING ORDER FROM CURRENT \TbbPVTb TO OLDEST \TbbPVTb #
4HIS FILE IS MAINTAINED BY THE bhb[^V DAEMON AND CONTAINS IMPORTANT
INFORMATION ABOUT THE HEALTH AND OPERATION OF THE OPERATING SYSTEM
dmesg: 4HIS FILE IS CREATED BY THE S\TbV COMMAND AND CONTAINS DIAGNOSTIC
KERNEL
LEVEL INFORMATION FROM THE kernel ring buffer 4HE KERNEL RING BUFFER
CONTAINS MESSAGES SENT TO OR RECEIVED FROM EXTERNAL DEVICES CONNECTED TO THE
SYSTEM SUCH AS DISK DRIVES KEYBOARD VIDEO AND SO ON
lspci: 4HIS FILE CONTAINS A LIST OF ALL THE 0#) BUSES ON THE SYSTEM
lsscsi: 4HE [bbRbX FILE CONTAINS A LIST OF ALL THE 3#3) DRIVES ON THE SYSTEM
fdisk-l: 4HE USXbZ [ FILE CONTAINS A LISTING OF ALL DISK DEVICE PARTITIONS IN YOUR
SYSTEM
sel-list: 4HE bT[ [Xbc FILE CONTAINS OUTPUT FROM THE X_\Xc^^[bT[T[Xbc
COMMAND )0-) STANDS FOR )NTELLIGENT 0LATFORM -ANAGEMENT )NTERFACE AND IS
PART OF THE ),/- )NTEGRATED ,IGHTS /UT -ANAGEMENT COMPONENT 4HE
X_\Xc^^[ COMMAND TAPS INTO THE ),/- AND EXTRACTS SENSOR READINGS ON ALL )0-)
ENABLED DEVICES SUCH AS MEMORY AND #05
megacli64: 4HE bd]SXPVb bW SCRIPT RUNS THE <TVP2[X%# COMMAND WITH VARIOUS
OPTIONS THAT INTERROGATE THE -EGA2!)$ CONTROLLER FOR INFORMATION ON THE
CONFIGURATION AND STATUS OF YOUR DISK CONTROLLER AND ATTACHED DISK DRIVES 4HERE
IS A WEALTH OF INFORMATION COLLECTED BY THE -EGA2!)$ CONTROLLER THAT CAN BE
EASILY TAPPED INTO USING THE <TVP2[X%# COMMAND &OR EXAMPLE THE FOLLOWING
#(!04%2 2%#/6%2).' %8!$!4!
LISTING SHOWS A SUMMARY OF THE 2!)$ CONFIGURATION OF THE DISK DRIVES ON OUR LAB
SYSTEM
B[^c 3TeXRT 78C02787 " #B2BD= #%60!0' &509F(4bcPcdbXb)>][X]T
B[^c 3TeXRT 78C02787 " #B2BD= #%60!0' &510H'4bcPcdbXb)>][X]T
B[^c !3TeXRT (78C02787 " #B2BD= #%60!0' &50: 14bcPcdbXb)>][X]T
B[^c "3TeXRT '78C02787 " #B2BD= #%60!0' &51'?74bcPcdbXb)7^cb_PaT
)NFORMATION IN THESE FILES INCLUDES AN EVENT LOG AND A STATUS SUMMARY OF YOUR
CONTROLLER AND DISK DRIVES &OR EXAMPLE THE FOLLOWING LISTING SHOWS A SUMMARY
OF THE STATE OF THE PHYSICAL DISK DRIVES ATTACHED TO ONE OF OUR DATABASE SERVERS
2WTRZX]VA083bcPcdb^]T]ZSQ T]ZXcTR R^\
2^]ca^[[TaP );B8<TVPA083B0B(!% 'X
=^^U?WhbXRP[SXbZb^][X]T)"
3TVaPSTS)
5PX[TS3XbZb)
)T IS HARD TO SAY WHETHER %XADATA USES THE <TVP2[X%# COMMAND TO MONITOR
PREDICTIVE FAILURE FOR DISK DRIVES OR IF THE DEVELOPERS HAVE TAPPED INTO 3-!24
METRICS THROUGH AN !0) BUT THIS INFORMATION IS ALL AVAILABLE TO YOU AT THE
COMMAND LINE 4HERE ISNT A LOT OF INFORMATION ABOUT -EGA#LI OUT THERE BUT
THE bd]SXPV bW SCRIPT IS A GOOD PLACE TO START IF YOU ARE INTERESTED IN PEEKING
UNDER THE HOOD AND GETTING A CLOSER LOOK AT SOME OF THE METRICS %XADATA COLLECTS
TO DETERMINE THE HEALTH OF YOUR DISK SUBSYSTEM
)F YOU RUN THE bd]SXPV bW SCRIPT ON YOUR STORAGE CELLS ADDITIONAL DATA IS COLLECTED ABOUT THE CELL
CONFIGURATION ALERTS AND SPECIAL LOG FILES THAT DO NOT EXIST ON THE DATABASE SERVER 4HE FOLLOWING LIST
DESCRIBES THESE ADDITIONAL LOG FILES COLLECTED BY bd]SXPV bW
cell-detail: 4HE RT[[ STcPX[ FILE CONTAINS DETAILED SITE
SPECIFIC INFORMATION
ABOUT YOUR STORAGE CELL 4HIS IS OUTPUT FROM THE 2T[[2;8 COMMAND ;8BC
24;;38B:34C08;
celldisk-detail: 4HIS FILE CONTAINS A DETAILED REPORT OF YOUR CELL DISKS 4HE
REPORT IS CREATED USING THE 2T[[2;8 COMMAND ;8BC24;;38B:34C08; !MONG
OTHER THINGS IT SHOWS THE STATUS LOGICAL UNIT NUMBER ,5. AND PHYSICAL
DEVICE PARTITION FOR YOUR CELL DISKS
lun-detail: 4HIS REPORT IS GENERATED USING THE 2T[[2;8 COMMAND ;8BC;D=
34C08; )T CONTAINS DETAILED INFORMATION ABOUT THE UNDERLYING ,5.S ON WHICH
YOUR CELL DISKS ARE CONFIGURED )NCLUDED IN THIS REPORT ARE THE NAMES DEVICE
TYPES AND PHYSICAL DEVICE NAMES LIKE STe bSf OF YOUR ,5.S
physicaldisk-detail: 4HE _WhbXRP[SXbZ STcPX[ FILE CONTAINS A DETAILED REPORT
OF ALL PHYSICAL DISKS AND &-/$S USED BY THE STORAGE CELL FOR DATABASE TYPE
STORAGE AND &LASH #ACHE )T IS GENERATED USING THE 2T[[2;8 COMMAND ;8BC
?7HB820;38B:34C08; AND IT INCLUDES IMPORTANT INFORMATION ABOUT THESE
DEVICES SUCH AS THE DEVICE TYPE HARD DISK OR FLASH DISK MAKE AND MODEL SLOT
ADDRESS AND DEVICE STATUS
physicaldisk-fail: 4HIS FILE CONTAINS A LISTING OF ALL PHYSICAL DISKS INCLUDING
FLASH DISKS THAT DO NOT HAVE A STATUS OF .ORMAL 4HIS WOULD INCLUDE DISKS WITH A
STATUS OF .OT 0RESENT WHICH IS A FAILED DISK THAT HAS BEEN REPLACED BUT NOT YET
REMOVED FROM THE CONFIGURATION 7HEN A PHYSICAL DISK IS REPLACED ITS OLD
#(!04%2 2%#/6%2).' %8!$!4!
CONFIGURATION REMAINS IN THE SYSTEM FOR DAYS AFTER WHICH IT IS AUTOMATICALLY
PURGED
griddisk-detail: 4HIS FILE CONTAINS A DETAILED REPORT OF ALL GRID DISKS CONFIGURED
ON THE STORAGE CELL )T IS CREATED USING THE 2T[[2;8 COMMAND ;8BC6A8338B:
34C08; AND INCLUDES AMONG OTHER THINGS THE GRID DISK NAME CELL DISK NAME
SIZE AND STATUS OF ALL GRID DISKS YOU HAVE CONFIGURED ON THE STORAGE CELL
flashcache-detail: 4HIS REPORT CONTAINS THE LIST OF ALL &-/$S THAT MAKE UP THE
CELL FLASH CACHE )T IS THE OUTPUT OF THE 2T[[2;8 COMMAND ;8BC5;0B720274
34C08; AND INCLUDES THE SIZE AND STATUS OF THE FLASH CACHE !LSO FOUND IN THIS
REPORT IS A LIST OF ALL FLASH CACHE CELL DISKS THAT ARE OPERATING IN A DEGRADED MODE
alerthistory: 4HE P[TacWXbc^ah FILE CONTAINS A DETAILED REPORT OF ALL ALERTS THAT
HAVE OCCURRED ON THE STORAGE CELL )T IS CREATED USING THE 2T[[2;8 COMMAND ;8BC
0;4AC78BC>AH
fdom-l: 4HIS REPORT CONTAINS DETAILED INFORMATION ABOUT THE FOUR FLASH CACHE
CARDS AND FLASH CACHE MODULES &$/-S ON YOUR STORAGE CELL )T IS CREATED BY
RUNNING THE dba QX] U[PbWNS^\~[ COMMAND ON THE STORAGE CELL AND INCLUDES
INFORMATION LIKE FIRMWARE VERSION ADDRESS AND DEVICE NAME OF EACH OF THE
&$/-S &OR EXAMPLE
31NNNCCh_TET]S^a?a^SdRcATe>_TaPcX]VBhbcT\3TeXRT=P\T
3XbZ0C0<0AE4;;B3''B0 !3! H STe bSdJ") ) ) L
! 3XbZ0C0<0AE4;;B3''B0 !3! H STe bSeJ") ) ) L
" !3XbZ0C0<0AE4;;B3''B0 !3! H STe bSfJ") )!) L
# "3XbZ0C0<0AE4;;B3''B0 !3! H STe bSgJ") )") L
alert.log: 4HE P[Tac [^V FILE IS WRITTEN TO BY THE RT[[bae PROCESS 3IMILAR TO A
DATABASE OR !3- ALERT LOG FILE THE STORAGE CELL P[Tac [^V CONTAINS IMPORTANT
RUNTIME INFORMATION ABOUT THE STORAGE CELL AND THE STATUS OF ITS DISK DRIVES 4HIS
FILE IS VERY USEFUL IN DIAGNOSING PROBLEMS WITH CELL STORAGE
ms-odl.trc: 4HE \b ^S[ caR CONTAINS DETAILED RUNTIME TRACE
LEVEL INFORMATION
FROM THE CELLS MANAGEMENT SERVER PROCESS
ms-odl.log: 4HIS FILE IS WRITTEN TO BY THE CELLS MANAGEMENT SERVER PROCESS )T IS
NOT INCLUDED IN THE COLLECTION CREATED BY THE bd]SXPV bW SCRIPT BUT WE HAVE
FOUND IT VERY USEFUL IN DIAGNOSING PROBLEMS THAT OCCUR IN THE STORAGE CELL )T ALSO
CONTAINS NORMAL DAY
TO
DAY OPERATIONAL MESSAGES 3TORAGE CELLS MAINTAIN THEIR
LOG FILES BY ROTATING THEM SIMILAR TO THE WAY THE OPERATING SYSTEM ROTATES THE
SYSTEM LOG ePa [^V \TbbPVTb 4HE \b ^S[ [^V FILE RECORDS THESE TASKS AS WELL
AS MORE CRITICAL TASKS LIKE DISK FAILURES
(EALTH#HECK
(EALTH#HECK IS A SET OF SHELL SCRIPTS DEVELOPED BY /RACLE THAT YOU CAN USE TO VERIFY YOUR CONFIGURATION
4HESE SCRIPTS ARE NOT INCLUDED IN YOUR %XADATA INSTALLATION BUT ARE READILY AVAILABLE FOR DOWNLOAD FROM
-Y /RACLE 3UPPORT UNDER !RTICLE )$ h/RACLE $ATABASE -ACHINE (EALTH#HECKv 4HE SCRIPTS
ARE WELL DOCUMENTED AND PRODUCE A STAGGERING AMOUNT OF INFORMATION ABOUT YOUR %XADATA $ATABASE
-ACHINE SO WE WONT GO THROUGH ALL THE GORY DETAILS HERE (EALTH#HECK REPORTS THE CURRENT
CONFIGURATION OF KEY HARDWARE AND SOFTWARE COMPONENTS OF %XADATA /RACLE RECOMMENDS THAT YOU
ROUTINELY RUN (EALTH#HECK AS A NORMAL PART OF MAINTAINING YOUR DATABASE MACHINE 4HE OUTPUT SHOULD
#(!04%2 2%#/6%2).' %8!$!4!
BE COMPARED WITH BULLETINS AND BEST PRACTICES PUBLISHED BY /RACLE #ORPORATION TO ENSURE OPTIMAL
PERFORMANCE AND MANAGEABILITY OF THE PLATFORM (EALTH CHECKS ARE FAIRLY LIGHTWEIGHT AND ALL EXCEPT ONE
CAN BE RUN DURING NORMAL BUSINESS OPERATIONS 4HE EXCEPTION IS THE 2WTRZ7F]5F?a^UX[T TEST 4HIS TEST IS
NOT EXECUTED BY DEFAULT AND SHOULD ONLY BE RUN IMMEDIATELY FOLLOWING A FRESH INSTALLATION OF THE %XADATA
SOFTWARE
(EALTH#HECK IS MADE UP OF TWO MAIN SCRIPTS THE !3- HEALTH CHECK ad]N^bNR^\\P]SbNPbN^aPR[T bW
AND THE /3 HEALTH CHECK ad]N^bNR^\\P]SbNPbNa^^c bW 4HE !3- HEALTH CHECK SHOULD ONLY TAKE A FEW
SECONDS TO EXECUTE WHILE THE /3 HEALTH CHECK WILL RUN FOR AROUND n MINUTES DEPENDING ON YOUR
SPECIFIC %XADATA CONFIGURATION 6 8
8
AND SYSTEM LOAD 4HE OUTPUT FROM THESE TESTS IS STORED IN
FILES NAMED WITH A hDATE TIMEvPATTERN STORED IN THE OUTPUT DIRECTORY UNDER THE (EALTH#HECK HOME
DIRECTORY &OR EXAMPLE AFTER RUNNING (EALTH#HECK ON A COUPLE DIFFERENT OCCASIONS OUR h^dc_dcNUX[Tbv
DIRECTORY SHOWS THE FOLLOWING
JT]ZSQ )^aPR[T)34<> L d P__ 7TP[cW2WTRZ ^dc_dcNUX[Tb
-[b [ca
c^cP["!#
af af a ^aPR[TSQP$%'"3TR" #)$(Pb\N^dc_dcN !' N #$("' [bc
af a a a^^ca^^c #'"# 3TR" %) ^bN^dc_dcN !" N #$(" [bc
af af a ^aPR[TSQP$%'"9P]!' $) Pb\N^dc_dcN !' N $ !! [bc
af a a a^^ca^^c $"! $9P]!' $)##^bN^dc_dcN !' N $#!"% [bc
4HE THING WE REALLY LIKE ABOUT THE (EALTH#HECK IS THAT INSTEAD OF SIMPLY REPORTING WHAT IT FINDS IT GIVES A
SHORT DESCRIPTION OF WHAT THE EXPECTED READINGS SHOULD BE AND WHAT TO DO IF YOUR RESULTS ARE OUTSIDE THE
NORM !NOTHER NICE THING ABOUT THE (EALTH#HECK IS THAT IT IS ALL WRITTEN IN SHELL SCRIPTS WITH CALLOUTS TO
PROGRAMS LIKE b`[_[db AND <TVP2[X%# 4HIS MEANS THAT IN ADDITION TO BEING A VERY USEFUL AUDITING AND
VERIFICATION TOOL IT IS ALSO A VERY GOOD LEARNING TOOL BECAUSE YOU CAN EASILY SEE SOME OF THE COMMANDS
THAT /RACLE SUPPORT ANALYSTS WOULD USE TO DIAGNOSE YOUR %XADATA $ATABASE -ACHINE
#ELL#,)
4HE RT[[bae PROCESS ON EACH STORAGE CELL COLLECTS IMPORTANT INFORMATION ABOUT THE CURRENT STATE OF THE
STORAGE CELL AND COMPONENTS SUCH AS #05 FLASH CACHE MODULES CELL DISKS GRID DISKS AND MORE 4HESE
CELL METRICS ARE IN TURN RECEIVED BY THE -ANAGEMENT 3ERVICE -3 PROCESS /NCE AN HOUR THE -3 PROCESS
WRITES THE CELL METRICS TO ITS !UTOMATIC $IAGNOSTICS 2EPOSITORY !$2 SIMILAR TO THE WAY G DATABASES
RECORD ALERTS TO THE !$2 ON THE DATABASE SERVERS %XADATA MONITORS THE METRICS IT COLLECTS AND TRIGGERS
ALERTS WHEN KEY THRESHOLDS ARE EXCEEDED HARDWARE ERRORS OCCUR OR SOFTWARE ERRORS OCCUR IN THE RT[[bae
PROCESS 4HE 2T[[2;8 PROGRAM IS YOUR PRIMARY INTERFACE FOR CONFIGURING AND MANAGING THE STORAGE CELLS
2T[[2;8 ALSO PROVIDES CONVENIENT ACCESS TO THE METRICS AND ALERTS RECORDED IN THE !$2 4HE INFORMATION
IN THE !$2 CAN ALSO BE ACCESSED WITH THE PSaRX COMMAND JUST AS YOU WOULD ON THE DATABASE SERVERS FOR
DATABASE ALERTS 4HE !$2 HAS BEEN AROUND FOR SEVERAL YEARS AND IS WELL DOCUMENTED SO WE WONT BE
DISCUSSING PSaRX HERE )NSTEAD WELL BE FOCUSING OUR ATTENTION ON 2T[[2;8 AND HOW IT CAN BE USED TO TAP
INTO THE DIAGNOSTIC INFORMATION STORED IN THE !$2
Cell Metrics
#ELL METRICS ARE KEY MEASUREMENTS COLLECTED FROM VARIOUS COMPONENTS IN THE STORAGE CELL 4HESE METRICS
CAN BE RETRIEVED USING THE ;8BC<4CA823458=8C8>= COMMAND &OR EXAMPLE THE FOLLOWING REPORT SHOWS
THE DEFINITION FOR THE 2;NAD=@ METRIC WHICH TRACKS RUN QUEUE INFORMATION FROM THE OPERATING SYSTEM
#(!04%2 2%#/6%2).' %8!$!4!
2T[[2;8-[Xbc\TcaXRSTUX]XcX^]fWTaT]P\T,2;NAD=@STcPX[
]P\T)2;NAD=@
STbRaX_cX^])0eTaPVT]d\QTa^eTacWT_aTRTSX]V\X]dcT^U_a^RTbbTbX]
cWT;X]dgad]`dTdT\PaZTSad]]X]V^ad]X]cTaad_cXQ[TUa^\
_a^R [^PSPeV
\TcaXRCh_T)8]bcP]cP]T^db
^QYTRcCh_T)24;;
d]Xc)=d\QTa
!T THE TIME OF THIS WRITING THE STORAGE CELL MAINTAINS SPECIFIC METRICS 4HE 34B2A814
<4CA822DAA4=C COMMAND DISPLAYS THE ATTRIBUTES OF THE \TcaXRRdaaT]c OBJECT AS SHOWN IN THE FOLLOWING
LISTING
2T[[2;8-STbRaXQT\TcaXRRdaaT]c
]P\T
P[TacBcPcT
R^[[TRcX^]CX\T
\TcaXR>QYTRc=P\T
\TcaXRCh_T
\TcaXREP[dT
^QYTRcCh_T
4HE ;8BC<4CA822DAA4=C COMMAND CAN BE USED TO REPORT CURRENT READINGS OF THESE METRICS 4HE
REPORT GENERATED BY THIS COMMAND IS VERY LENGTHY SO YOU WILL WANT TO APPLY FILTERS TO LIMIT THE REPORT TO
THE COMPONENTS YOU ARE INTERESTED IN 4HE FOLLOWING LIST SHOWS THE OBJECT TYPES STORED IN THE !$2
v 24;;
v 24;;38B:
v 24;;N58;4BHBC4<
v 5;0B720274
v 6A8338B:
v 7>BCN8=C4A2>==42C
v 8>A<N20C46>AH
v 8>A<N2>=BD<4AN6A>D?
v 8>A<N30C010B4
5NDERSTANDING THE NAMING CONVENTION USED FOR THE METRIC NAME WILL HELP YOU FOCUS THE REPORT ON
THE SPECIFIC METRICS YOU ARE INTERESTED IN 4HE METRIC NAME IS A CONCATENATION OF ABBREVIATIONS FOR THE
TYPE OF COMPONENT DELIMITED BY THE UNDERSCORE CHARACTER ? 4HE FIRST PART OF THE METRIC NAME IS AN
ABBREVIATION FOR THE OBJECT TYPE 4HIS CAN BE USED AS SHORTHAND FOR FILTERING THE REPORT OUTPUT 4ABLE
SHOWS THE METRIC NAME PREFIXES AND WHAT EACH ONE REPRESENTS
#(!04%2 2%#/6%2).' %8!$!4!
Table 9-1. Metric Name Prefixes
Name Prefix Description
2;N #ELL LEVEL METRICS
23N #ELL DISK METRICS
63N 'RID DISK METRICS
52N &LASH #ACHE METRICS
31N $ATABASE METRICS
26N )/2- #ONSUMER GROUP METRICS
2CN )/2- #ATEGORIES
=N .ETWORK )NFINI"AND NETWORK INTERCONNECT
&OR )/
RELATED METRICS THE METRIC NAME CONTINUES WITH ONE OF THE VALUES LISTED IN 4ABLE
Table 9-2. I/O Metric Abbreviations
Abbreviation Description
8>NA@ .UMBER OF )/ REQUESTS
8>N1H .UMBER OF MEGABYTES
8>NC< )/ ,ATENCY
8>NFC )/ 7AIT TIME
&OR )/
RELATED METRICS THE NEXT PART OF THE METRIC NAME MAY BE A FOR READS OR F FOR WRITES
FOLLOWED BY B< OR ;6 FOR SMALL OR LARGE READS AND WRITES 4HE LAST ABBREVIATION IN THE METRIC NAME WILL BE
EITHER B42 FOR SECONDS OR A@ FOR REQUESTS &OR EXAMPLE THE FOLLOWING COMMAND SHOWS THE NUMBER OF
SMALL )/ WRITE OPERATIONS PER SECOND FOR ALL GRID DISKS ON THE STORAGE CELL
2T[[2;8-[Xbc\TcaXRRdaaT]cfWTaT]P\T,63N8>NA@NFNB<NB42
63N8>NA@NFNB<NB4230C0N23N NRT[[ '8> bTR
63N8>NA@NFNB<NB4230C0N23N NRT[[ (8> bTR
63N8>NA@NFNB<NB4230C0N23N !NRT[[ #8> bTR
63N8>NA@NFNB<NB4230C0N23N "NRT[[ %8> bTR
...
#(!04%2 2%#/6%2).' %8!$!4!
4HE DEFAULT RETENTION FOR METRIC DETAIL IS DAYS AFTER WHICH IT WILL BE SUMMARIZED BY MIN MAX AND
AVERAGE AT WHICH TIME THE DETAIL RECORDS WILL BE PURGED 4HE 34B2A814<4CA8278BC>AH COMMAND SHOWS
THE STRUCTURE OF THE \TcaXRWXbc^ah OBJECT
2T[[2;8-STbRaXQT\TcaXRWXbc^ah
]P\T
P[TacBcPcT
R^[[TRcX^]CX\T
\TcaXR>QYTRc=P\T
\TcaXRCh_T
\TcaXREP[dT
\TcaXREP[dT0eV
\TcaXREP[dT<Pg
\TcaXREP[dT<X]
^QYTRcCh_T
4HE ;8BC<4CA8278BC>AH COMMAND PROVIDES A SUMMARIZED REPORT OF THE FULL HISTORY OF CELL METRICS
STORED IN THE !$2 5SUALLY YOU WILL WANT TO APPLY FILTERS TO SPECIFY A DATE RANGE AND POSSIBLY A METRIC TYPE
FOR THIS REPORT 4O DO THIS PROVIDE A F74A4 CLAUSE WITH THE METRIC NAME AND A DATE FILTER AS IN THIS
EXAMPLE
2T[[2;8-[Xbc\TcaXRWXbc^ah
fWTaT]P\T[XZT2;N
P]SR^[[TRcX^]cX\T-! C ') ) ')
Cell Alerts
)N ADDITION TO TRACKING METRICS FOR STORAGE CELL COMPONENTS %XADATA ALSO EVALUATES THESE METRICS AND
APPLIES THRESHOLDS TO VERIFY THAT THE COMPONENTS ARE RUNNING WITHIN NORMAL PARAMETERS 7HEN A METRIC
CROSSES ONE OF THESE OPERATIONAL THRESHOLDS AN ALERT IS GENERATED 3OME OF THE ALERTS GENERATED BY
%XADATA INCLUDE CELL TEMPERATURE CELL DISK READWRITE ERRORS AND SOFTWARE FAILURES %XADATA TRACKS OVER
ALERT TYPES IN THE STORAGE CELL !DDITIONAL ALERTS MAY BE DEFINED USING 'RID #ONTROLS MONITORING AND
ALERTING FEATURES !LERT SEVERITIES FALL INTO FOUR CATEGORIES )NFORMATION 7ARNING #RITICAL AND #LEAR
4HESE ARE USED TO MANAGE ALERT NOTIFICATIONS &OR EXAMPLE YOU MAY CHOOSE TO GET AN EMAIL ALERT
NOTIFICATION FOR CRITICAL ALERTS ONLY 4HE #LEAR SEVERITY IS USED TO NOTIFY YOU WHEN A COMPONENT HAS
RETURNED TO .ORMAL STATUS 4HE ;8BC<4CA8278BC>AH34C08; COMMAND CAN BE USED TO GENERATE A DETAILED
REPORT OF THE ALERTS GENERATED BY THE SYSTEM 4HE FOLLOWING LISTING IS AN EXAMPLE OF AN ALERT GENERATED BY
THE STORAGE CELL
]P\T)! (N
P[Tac<TbbPVT)0[[;^VXRP[SaXeTbPaTX]FaXcTCWa^dVWRPRWX]V\^ST
4XcWTaQPccTahXbX]P[TPa]RhR[T^aXc]TTSbc^QT
aT_[PRTS ?[TPbTR^]cPRc>aPR[TBd__^ac
P[TacBT`dT]RT83)! (
P[TacBW^ac=P\T)7PaSfPaT
P[TacCh_T)BcPcTUd[
QTVX]CX\T)! &C #)#!) %)
T]SCX\T)! &C $)$ )!( %)
TgP\X]TS1h)
\TcaXR>QYTRc=P\T);D=N20274NFCN0;;
]^cXUXRPcX^]BcPcT)
bT`dT]RT1TVX]CX\T)! &C #)#!) %)
#(!04%2 2%#/6%2).' %8!$!4!
bTeTaXch)RaXcXRP[
P[Tac0RcX^])1PccTahXbTXcWTaX]P[TPa]RhR[T^aXc]TTSb
aT_[PRT\T]c ?[TPbTR^]cPRc>aPR[TBd__^ac
7HEN THE BATTERY SUBSEQUENTLY RETURNS TO .ORMAL STATUS A FOLLOW
UP ALERT IS GENERATED WITH A
SEVERITY OF #LEAR INDICATING THAT THE COMPONENT HAS RETURNED TO NORMAL OPERATING STATUS
]P\T)! (N!
P[Tac<TbbPVT)1PccTahXbQPRZc^PV^^SbcPcT
...
bTeTaXch)R[TPa
P[Tac0RcX^])1PccTahXbQPRZc^PV^^SbcPcT =^0RcX^]AT`dXaTS
7HEN YOU REVIEW ALERTS YOU SHOULD GET IN THE HABIT OF SETTING THE TgP\X]TS1h ATTRIBUTE OF THE ALERT SO
YOU CAN KEEP TRACK OF WHICH ALERTS ARE ALREADY BEING INVESTIGATED )F YOU SET THE TgP\X]TS1h ATTRIBUTE YOU
CAN USE IT AS A FILTER ON THE ;8BC0;4AC78BC>AH COMMAND TO REPORT ALL ALERTS THAT ARE NOT CURRENTLY BEING
ATTENDED TO "Y ADDING THE SEVERITY FILTER YOU CAN FURTHER REDUCE THE OUTPUT TO JUST CRITICAL ALERTS &OR
EXAMPLE
;8BC0;4AC78BC>AHF74A4bTeTaXch,RaXcXRP[0=3TgP\X]TS1h,34C08;
4O SET THE TgP\X]TS1h ATTRIBUTE OF THE ALERT USE THE 0;C4A0;4AC78BC>AH COMMAND AND SPECIFY THE
NAME OF THE ALERT YOU WISH TO ALTER &OR EXAMPLE WE CAN SET THE TgP\X]TS1h ATTRIBUTE FOR THE "ATTERY ALERT
JUST SHOWN AS FOLLOWS
2T[[2;8-P[cTaP[TacWXbc^ah! (N TgP\X]TS1h,rjohnson
0[Tac! (N bdRRTbbUd[[hP[cTaTS
2T[[2;8-[XbcP[TacWXbc^ahPccaXQdcTb]P\TP[Tac<TbbPVTTgP\X]TSQhfWTaT]P\T,! (N STcPX[
]P\T)! (N
P[Tac<TbbPVT)0[[;^VXRP[SaXeTbPaTX]FaXcTCWa^dVWRPRWX]V\^ST
4XcWTaQPccTahXbX]P[TPa]RhR[T^aXc]TTSbc^QT
aT_[PRTS ?[TPbTR^]cPRc>aPR[TBd__^ac
TgP\X]TS1h)rjohnson
4HERE IS QUITE A BIT MORE TO SAY ABOUT MANAGING REPORTING AND CUSTOMIZING %XADATA ALERTS !N
ENTIRE CHAPTER WOULD BE NEEDED TO COVER THE SUBJECT IN DETAIL )N THIS SECTION WEVE ONLY TOUCHED ON THE
BASICS &ORTUNATELY ONCE YOU GET EMAIL CONFIGURED FOR ALERT NOTIFICATION VERY LITTLE MUST BE DONE TO
MANAGE THESE ALERTS )N MANY ENVIRONMENTS EMAIL NOTIFICATION IS ALL THAT IS USED TO CATCH AND REPORT
CRITICAL ALERTS
%XADATA IS ONE OF THE BEST
INSTRUMENTED AND SELF
DOCUMENTED SYSTEMS WEVE EVER WORKED ON )T IS
ALSO ONE OF THE MOST COMPLEX 4HIS SECTION HAS DISCUSSED THREE VERY HELPFUL TOOLS THAT PROVIDE THE KIND OF
DIAGNOSTIC INFORMATION YOU MUST HAVE IN ORDER TO RESOLVE %XADATA PROBLEMS 3UN $IAGNOSTICS AND
(EALTH#HECK ARE WRAPPER SCRIPTS THAT CALL OTHER COMMANDS AND SCRIPTS TO TAP INTO KEY HARDWARE AND
SOFTWARE COMPONENT METRICS )F YOU ARE BRAVE ENOUGH IT IS WELL WORTH THE TIME TO CRACK THESE SCRIPTS
OPEN AND BECOME FAMILIAR WITH WHAT THEY ARE DOING 2T[[2;8 PROVIDES AN EXCELLENT WAY OF PRODUCING
REPORTS DETAILING ALERTS THAT AFFECT THE NORMAL OPERATIONS OF YOUR SYSTEM 4HERE IS A LOT TO BE LEARNED
ABOUT WHERE CONFIGURATION AND DIAGNOSTIC INFORMATION IS STORED SO YOU CAN QUICKLY GO STRAIGHT TO THE
SOURCE WHEN PROBLEMS ARISE )N THIS SECTION WEVE TRIED TO TOUCH ON SOME OF THE MORE IMPORTANT AREAS
YOU SHOULD BE AWARE OF
#(!04%2 2%#/6%2).' %8!$!4!
Backing Up Exadata
7HEN WE TOOK DELIVERY OF OUR %XADATA SYSTEM ONE OF OUR PRIMARY CONCERNS WAS hHOW CAN WE BACK UP
EVERYTHING SO WE CAN RESTORE IT TO WORKING ORDER IF SOMETHING GOES HORRIBLY WRONG v 7HEN OUR %XADATA
ARRIVED IN -AY OF THE LATEST VERSION OF THE #ELL SOFTWARE WAS !ND AT THE TIME THE ONLY WAY
TO BACK UP A DATABASE SERVER WAS TO USE THIRD
PARTY BACKUP SOFTWARE OR STANDARD ,INUX COMMANDS LIKE
cPa /RACLE IS CONSTANTLY DEVELOPING NEW FEATURES FOR %XADATA AND LESS THAN A YEAR LATER %XADATA 8
DATABASE SERVERS COME CONFIGURED WITH THE NATIVE ,INUX ,OGICAL 6OLUME -ANAGER ,6- 4HIS IS A BIG
STEP FORWARD BECAUSE THE ,6- HAS BUILT
IN SNAPSHOT CAPABILITIES THAT PROVIDE AN EASY METHOD OF TAKING
BACKUPS OF THE OPERATING SYSTEM 3TORAGE CELLS USE A COMPLETELY DIFFERENT PROPRIETARY METHOD FOR
BACKUP AND RECOVERY )N THIS SECTION WELL TAKE A LOOK AT THE VARIOUS METHODS /RACLE RECOMMENDS FOR
BACKING UP %XADATA DATABASE SERVERS AND STORAGE CELLS 7ELL ALSO TAKE A BRIEF LOOK AT 2ECOVERY -ANAGER
2-!. AND SOME OF THE FEATURES %XADATA PROVIDES THAT IMPROVE THE PERFORMANCE OF DATABASE BACKUP
AND RECOVERY !FTER THAT WELL TAKE A LOOK AT WHAT IT TAKES TO RECOVER FROM SOME OF THE MORE COMMON
TYPES OF SYSTEM FAILURE )T MAY SURPRISE YOU BUT THE FOCUS OF THIS CHAPTER IS NOT DATABASE RECOVERY 4HERE
ARE VERY FEW %XADATA
SPECIFIC CONSIDERATIONS FOR DATABASE BACKUP AND RECOVERY ! MAJORITY OF THE
PRODUCT
SPECIFIC BACKUP AND RECOVERY METHODS PERTAIN TO BACKUP AND RECOVERY OF THE SYSTEM VOLUMES
CONTAINING THE OPERATING SYSTEM AND %XADATA SOFTWARE SO WELL SPEND A QUITE A BIT OF TIME DISCUSSING
RECOVERY FROM THE LOSS OF A CELL DISK TO THE LOSS OF A SYSTEM VOLUME ON THE DATABASE SERVERS OR STORAGE
CELLS
"ACKING 5P THE $ATABASE 3ERVERS
2ECENTLY /RACLE BEGAN SHIPPING %XADATA WITH THE ,INUX ,6- CONFIGURED FOR MANAGING FILE SYSTEM
STORAGE ON THE DATABASE SERVERS ,OGICAL VOLUME MANAGERS PROVIDE AN ABSTRACTION LAYER FOR PHYSICAL DISK
PARTITIONS SIMILAR TO THE WAY !3- DOES FOR ITS UNDERLYING PHYSICAL STORAGE DEVICES ,6-S HAVE VOLUME
GROUPS COMPARABLE TO !3- DISK GROUPS 4HESE VOLUME GROUPS ARE MADE UP OF ONE OR MORE PHYSICAL DISKS
OR DISK PARTITIONS AS !3- DISK GROUPS ARE MADE UP OF ONE OR MORE PHYSICAL DISKS OR DISK PARTITIONS
,6- VOLUME GROUPS ARE CARVED UP INTO LOGICAL VOLUMES IN WHICH FILE SYSTEMS CAN BE CREATED )N A SIMILAR
WAY DATABASES UTILIZE !3- DISK GROUPS FOR CREATING TABLESPACES THAT ARE USED FOR STORING TABLES INDEXES
AND OTHER DATABASE OBJECTS !BSTRACTING PHYSICAL STORAGE FROM THE FILE SYSTEMS ALLOWS THE SYSTEM
ADMINISTRATOR TO GROW AND SHRINK THE LOGICAL VOLUMES AND FILE SYSTEMS AS NEEDED 4HERE ARE A NUMBER
OF OTHER ADVANTAGES TO USING THE ,6- TO MANAGE STORAGE FOR THE %XADATA DATABASE SERVERS BUT OUR FOCUS
WILL BE THE NEW BACKUP AND RESTORE CAPABILITIES THE ,INUX ,6- PROVIDES NAMELY ,6- SNAPSHOTS )N
ADDITION TO THEIR CONVENIENCE AND EASE OF USE ,6- SNAPSHOTS ELIMINATE MANY OF THE TYPICAL CHALLENGES
WE FACE WITH SIMPLE BACKUPS USING THE cPa COMMAND OR THIRD
PARTY BACKUP PRODUCTS &OR EXAMPLE
DEPENDING ON THE AMOUNT OF DATA IN THE BACKUP SET FILE SYSTEM BACKUPS CAN TAKE QUITE A WHILE TO
COMPLETE 4HESE BACKUPS ARE NOT CONSISTENT TO A POINT IN TIME MEANING THAT IF YOU MUST RESTORE A FILE
SYSTEM FROM BACKUP THE DATA IN YOUR FILES WILL REPRESENT VARIOUS POINTS IN TIME FROM THE BEGINNING OF THE
BACKUP PROCESS TO ITS END !PPLICATIONS THAT CONTINUE TO RUN DURING THE BACKUP CYCLE CAN HOLD LOCKS ON
FILES CAUSING THEM TO BE SKIPPED NOT BACKED UP !ND ONCE AGAIN OPEN APPLICATIONS WILL INEVITABLY
MAKE CHANGES TO DATA DURING THE BACKUP CYCLE %VEN IF YOU ARE ABLE TO BACK UP THESE OPEN FILES YOU HAVE
NO WAY OF KNOWING IF THEY ARE IN ANY USABLE STATE UNLESS THE APPLICATION IS SHUT DOWN BEFORE THE BACKUP
IS TAKEN ,6- SNAPSHOTS ARE INSTANTANEOUS BECAUSE NO DATA IS ACTUALLY COPIED 9OU CAN THINK OF A
SNAPSHOT AS AN INDEX OF POINTERS TO THE PHYSICAL DATA BLOCKS THAT MAKE UP THE CONTENTS OF YOUR FILE
SYSTEM 7HEN A FILE IS CHANGED OR DELETED THE ORIGINAL BLOCKS OF THE FILE ARE WRITTEN TO THE SNAPSHOT
VOLUME 3O EVEN IF IT TAKES HOURS TO COMPLETE A BACKUP IT WILL STILL BE CONSISTENT WITH THE MOMENT THE
SNAPSHOT WAS CREATED .OW LETS TAKE A LOOK AT HOW ,6- SNAPSHOTS CAN BE USED TO CREATE A CONSISTENT FILE
SYSTEM BACKUP OF THE DATABASE SERVER
#(!04%2 2%#/6%2).' %8!$!4!
System Backup Using LVM Snapshots
#REATING FILE SYSTEM BACKUPS USING ,6- SNAPSHOTS IS A PRETTY SIMPLE PROCESS &IRST YOU NEED TO CREATE A
DESTINATION FOR THE FINAL COPY OF THE BACKUPS 4HIS CAN BE 3!. OR .!3 STORAGE OR SIMPLY AN .&3 FILE
SYSTEM SHARED FROM ANOTHER SERVER )F YOU HAVE ENOUGH FREE SPACE IN THE VOLUME GROUP TO STORE YOUR
BACKUP FILES YOU CAN CREATE A TEMPORARY ,6- PARTITION TO STAGE YOUR BACKUPS BEFORE SENDING THEM OFF TO
TAPE 4HIS CAN BE DONE USING THE [eRaTPcT COMMAND "EFORE CREATING A NEW ,6- PARTITION MAKE SURE
YOU HAVE ENOUGH FREE SPACE IN YOUR VOLUME GROUP USING THE eVSXb_[Ph COMMAND
Ja^^c/T]ZSQ mLeVSXb_[Ph
E^[d\TVa^d_
E6=P\TE64gP3Q
E6BXiT$$% ' 61
?4BXiT# <1
C^cP[?4 #!$#
0[[^R?4 BXiT#$'!# &( 61
5aTT?4 BXiT(%& & "&& ' 61
4HE eVSXb_[Ph COMMAND SHOWS THE SIZE OF OUR VOLUME GROUP PHYSICAL EXTENTS 0% CURRENTLY IN USE
AND THE AMOUNT OF FREE SPACE AVAILABLE IN THE VOLUME GROUP 4HE 5aTT?4 BXiT ATTRIBUTE INDICATES THAT
WE HAVE '" OF FREE SPACE REMAINING IN THE VOLUME GROUP 4HIS IS PLENTY OF ROOM FOR US TO CREATE A
NEW '" ,6- PARTITION TO STORE OUR BACKUP IMAGES
&IRST WELL USE THE [eRaTPcT COMMAND TO CREATE A NEW '" ,6- PARTITION
Ja^^c/T]ZSQ mL[eRaTPcT ;!$6 ] STe E64gP3Q bhbQPRZ
;^VXRP[e^[d\TbhbQPRZRaTPcTS
.OW WE CAN CREATE A NEW FILE SYSTEM ON THE bhbQPRZ PARTITION
Ja^^c/T]ZSQ mL\ZUb Tgc" \ Q# (% STe E64gP3Q bhbQPRZ
\ZT!Ub "(!( <Ph ! %
...
FaXcX]VX]^STcPQ[Tb)S^]T
2aTPcX]VY^da]P["!&%'Q[^RZb)S^]T
FaXcX]Vbd_TaQ[^RZbP]SUX[TbhbcT\PRR^d]cX]VX]U^a\PcX^])S^]T
...
.EXT WELL CREATE A TARGET DIRECTORY AND MOUNT THE NEW FILE SYSTEM
Ja^^c/T]ZSQ mL\ZSXa \]c bhbQPRZ
Ja^^c/T]ZSQ mL\^d]c STe E64gP3Q bhbQPRZ \]c bhbQPRZ
Ja^^c/T]ZSQ mLSU Z \]c bhbQPRZ
4HE USXbZ COMMAND DISPLAYS OUR NEW FILE SYSTEM AND THE LOGICAL VOLUMES WE WANT TO INCLUDE IN OUR
SYSTEM BACKUP E64gP3Q ;E3QBhb ROOT FILE SYSTEM AND E64gP3Q ;E3Q>aP d FILE SYSTEM .OTICE THAT
THE Q^^c FILE SYSTEM DOES NOT USE THE ,6- FOR STORAGE 4HIS FILE SYSTEM MUST BE BACKED UP USING THE cPa
COMMAND 4HIS ISNT A PROBLEM BECAUSE THE Q^^c FILE SYSTEM IS FAIRLY SMALL AND STATIC SO WE ARENT
CONCERNED WITH THESE FILES BEING MODIFIED LOCKED OR OPEN DURING THE BACKUP CYCLE
Ja^^c/T]ZSQ mLSU W
5X[TbhbcT\BXiTDbTS0ePX[DbT<^d]cTS^]
STe \P__Ta E64gP3Q ;E3QBhb " 6( "6 (6""
STe bSP !#< %< !< # Q^^c
#(!04%2 2%#/6%2).' %8!$!4!
STe \P__Ta E64gP3Q ;E3Q>aP ((6 #6' 6 $ d
c\_Ub' 6! <' 6 STe bW\
STe \P__Ta E64gP3Q bhbQPRZ!$6 &"<!$6 \]c bhbQPRZ
.OW THAT WE HAVE A PLACE TO STAGE OUR BACKUP IMAGES AND WE KNOW THE LOGICAL VOLUMES WE WANT TO
BACK UP WERE READY TO CREATE SNAPSHOTS FOR THE AND d FILE SYSTEMS 5SING THE ,INUX cX\T COMMAND
WE CAN SEE THAT IT TOOK JUST OVER SECOND TO CREATE A SNAPSHOT FOR THE ROOT VOLUME
Ja^^c/T]ZSQ mLcX\T[eRaTPcT ; 6 b ]a^^cNb]P_ STe E64gP3Q ;E3QBhb
;^VXRP[e^[d\Ta^^cNb]P_RaTPcTS
aTP[ \ %%b
dbTa \ $b
bhb \ "b
Ja^^c/T]ZSQ mL[eRaTPcT ;$6 b ]d Nb]P_ STe E64gP3Q ;E3Q>aP
;^VXRP[e^[d\Td Nb]P_RaTPcTS
.OTICE THE ~; 6 AND ;$6 OPTIONS WE USED TO CREATE THESE SNAPSHOTS 4HE ~; PARAMETER DETERMINES
THE SIZE OF THE SNAPSHOT VOLUME 7HEN DATA BLOCKS ARE MODIFIED OR DELETED AFTER THE SNAPSHOT IS CREATED
THE ORIGINAL COPY OF THE BLOCK IS WRITTEN TO THE SNAPSHOT )T IS IMPORTANT TO SIZE THE SNAPSHOT SUFFICIENTLY
TO STORE AN ORIGINAL COPY OF ALL CHANGED BLOCKS )F THE SNAPSHOT RUNS OUT OF SPACE IT WILL BE DEACTIVATED
.OW THAT WE HAVE SNAPSHOTS TRACKING CHANGES TO THE AND d FILE SYSTEMS WE ARE READY TO TAKE A
BACKUP 4O PROVE THAT THESE SNAPSHOTS ARE CONSISTENT WELL COPY THE TcR W^bcb FILE TO A TEST FILE IN THE
a^^c DIRECTORY )F SNAPSHOTS WORK AS THEY ARE SUPPOSED TO THIS FILE WILL NOT BE INCLUDED OUR BACKUP
BECAUSE IT WAS CREATED AFTER THE SNAPSHOT WAS CREATED 4HE COMMAND LOOKS LIKE THIS
Ja^^c/T]ZSQ mLR_ TcR W^bcb a^^c cTbcNUX[T cgc
-OUNT THE SNAPSHOTS AS YOU WOULD ANY OTHER FILE SYSTEM &IRST WELL CREATE A DIRECTORY TO USE AS A
MOUNT POINT FOR OUR SNAPSHOTS 4HEN WELL MOUNT THE ROOT AND U SNAPSHOTS
Ja^^c/T]ZSQ mL\ZSXa \]c b]P_
Ja^^c/T]ZSQ mL\^d]c STe E64gP3Q a^^cNb]P_ a^^c \]c b]P_
Ja^^c/T]ZSQ mL\^d]c STe E64gP3Q d Nb]P_ \]c b]P_ d
.OW THAT THE SNAPSHOTS ARE MOUNTED WE CAN BROWSE THEM JUST LIKE ANY OTHER FILE SYSTEM 4HEY LOOK
AND FEEL JUST LIKE THE ORIGINAL FILE SYSTEMS WITH ONE EXCEPTION )F WE LOOK IN THE MOUNTED SNAPSHOT FOR THE
TEST FILE WE CREATED WE DONT SEE IT )TS NOT THERE BECAUSE IT WAS CREATED AFTER THE SNAPSHOTS WERE CREATED
AND STARTED TRACKING CHANGES TO THE FILE SYSTEM
Ja^^c/T]ZSQ mL[b [ a^^c cTbcUX[T
af a a a^^ca^^c !"9P]!" ")" a^^c cTbcNUX[T cgc<- the test file we created
Ja^^c/T]ZSQ mL[b [ \]c b]P_ a^^c cTbcNUX[T cgc
[b) \]c b]P_ a^^c cTbcUX[T)=^bdRWUX[T^aSXaTRc^ah<- no test file in the snapshot
/NCE THE SNAPSHOTS ARE MOUNTED THEY CAN BE BACKED UP USING ANY STANDARD ,INUX BACKUP SOFTWARE
&OR THIS TEST WELL USE THE cPa COMMAND TO CREATE A TARBALL BACKUP OF THE AND d FILE SYSTEMS 3INCE
WE ARE BACKING UP A SNAPSHOT WE DONT HAVE TO WORRY ABOUT FILES THAT ARE OPEN LOCKED OR CHANGED
DURING THE BACKUP .OTICE THAT WEVE ALSO INCLUDED THE Q^^c DIRECTORY IN THIS BACKUP
Ja^^c/T]ZSQ mLRS \]c b]P_
Ja^^c/T]ZSQ b]P_LcPa _YReU \]c bhbQPRZ T]ZSQ NbhbcT\NQPRZd_ cPa Qi! Q^^cK
TgR[dST \]c bhbQPRZ T]ZSQ NbhbcT\NQPRZd_ cPa Qi!K
- c\_ T]ZSQ NbhbcT\NQPRZd_ bcS^dc!- c\_ T]ZSQ NbhbcT\NQPRZd_ bcSTaa
#(!04%2 2%#/6%2).' %8!$!4!
7HEN THE BACKUP IS FINISHED YOU SHOULD CHECK THE ERROR FILE T]ZSQ NbhbcT\NQPRZd_ bcSTaa FOR ANY
ISSUES LOGGED DURING THE BACKUP )F YOU ARE SATISFIED WITH THE BACKUP YOU CAN UNMOUNT AND DROP THE
SNAPSHOTS 9OU WILL CREATE A NEW SET OF SNAPSHOTS EACH TIME YOU RUN A BACKUP !FTER YOU COPY THE BACKUP
IMAGE OFF TO TAPE YOU CAN OPTIONALLY UNMOUNT AND DROP THE TEMPORARY PARTITION YOU CREATED TO STORE IT
Ja^^c/T]ZSQ b]P_LRS
Ja^^c/T]ZSQ Ld\^d]c \]c b]P_ d
Ja^^c/T]ZSQ La\SXa \]c b]P_ d
Ja^^c/T]ZSQ Ld\^d]c \]c b]P_
Ja^^c/T]ZSQ La\SXa \]c b]P_
Ja^^c/T]ZSQ L[eaT\^eT STe E64gP3Q a^^cNb]P_
3^h^daTP[[hfP]cc^aT\^eTPRcXeT[^VXRP[e^[d\Ta^^cNb]P_.Jh ]L)h
;^VXRP[e^[d\Ta^^cNb]P_bdRRTbbUd[[haT\^eTS
a^^c/T]ZSQ L[eaT\^eT STe E64gP3Q d Nb]P_
3^h^daTP[[hfP]cc^aT\^eTPRcXeT[^VXRP[e^[d\Td Nb]P_.Jh ]L)h
;^VXRP[e^[d\Td Nb]P_bdRRTbbUd[[haT\^eTS
Ja^^c/aXRdSQ L[eaT\^eT STe E64gP3Q bhbQPRZ
3^h^daTP[[hfP]cc^aT\^eTPRcXeT[^VXRP[e^[d\TbhbQPRZ.Jh ]L)h
;^VXRP[e^[d\TbhbQPRZbdRRTbbUd[[haT\^eTS
Ja^^c/T]ZSQ Ld\^d]c \]c bhbQPRZ
Ja^^c/T]ZSQ La\SXa \]c bhbQPRZ
Ja^^c/aXRdSQ L[eaT\^eT STe E64gP3Q bhbQPRZ
3^h^daTP[[hfP]cc^aT\^eTPRcXeT[^VXRP[e^[d\TbhbQPRZ.Jh ]L)h
;^VXRP[e^[d\TbhbQPRZbdRRTbbUd[[haT\^eTS
!S DISCUSSED IN THE OPENING PARAGRAPH EARLY MODELS OF %XADATA 6 DID NOT IMPLEMENT ,6- FOR
MANAGING FILE SYSTEM STORAGE 7ITHOUT ,6- SNAPSHOTS GETTING A CLEAN SYSTEM BACKUP WOULD REQUIRE
SHUTTING DOWN THE APPLICATIONS ON THE SERVER INCLUDING THE DATABASES OR PURCHASING THIRD PARTY
BACKUP SOFTWARE %VEN THEN THERE WOULD BE NO WAY TO CREATE A BACKUP IN WHICH ALL FILES ARE CONSISTENT
WITH THE SAME POINT IN TIME ,6- SNAPSHOTS FILL AN IMPORTANT GAP IN THE %XADATA BACKUP AND RECOVERY
ARCHITECTURE AND OFFER A SIMPLE MANAGEABLE STRATEGY FOR BACKING UP THE DATABASE SERVERS ,ATER IN THIS
CHAPTER WELL DISCUSS HOW THESE BACKUPS ARE USED FOR RESTORING THE DATABASE SERVER WHEN FILE SYSTEMS ARE
LOST OR DAMAGED
"ACKING 5P THE 3TORAGE #ELL
4HE FIRST TWO DISKS IN A STORAGE CELL CONTAIN THE ,INUX OPERATING SYSTEM 4HESE ,INUX PARTITIONS ARE
COMMONLY REFERRED TO AS THE system volumes "ACKING UP THE SYSTEM VOLUMES USING INDUSTRY STANDARD
,INUX BACKUP SOFTWARE IS NOT RECOMMENDED 3O HOW DO YOU BACK UP THE SYSTEM VOLUMES 7ELL THE
ANSWER IS THAT YOU DONT %XADATA AUTOMATICALLY DOES THIS FOR YOU THROUGH THE USE OF AN INTERNAL 53"
DRIVE CALLED THE CELLBOOT USB flash drive )F YOU ARE THE CAUTIOUS SORT YOU CAN ALSO CREATE YOUR OWN CELL
RECOVERY IMAGE USING AN EXTERNAL 53" FLASH DRIVE )N ADDITION TO THE #%,,"//4 53" FLASH DRIVE %XADATA
ALSO MAINTAINS ON A SEPARATE SET OF DISK PARTITIONS A FULL COPY OF THE SYSTEM VOLUMES AS THEY WERE BEFORE
THE LAST PATCH WAS INSTALLED 4HESE BACKUP PARTITIONS ARE USED FOR ROLLING BACK A PATCH .OW LETS TAKE A
LOOK AT HOW THESE BACKUP METHODS WORK
#(!04%2 2%#/6%2).' %8!$!4!
CELLBOOT USB Flash Drive
9OU CAN THINK OF THE INTERNAL #%,,"//4 53" FLASH DRIVE AS YOU WOULD ANY EXTERNAL 53" DRIVE YOU WOULD
PLUG INTO YOUR LAPTOP 4HE DEVICE CAN BE SEEN USING THE USXbZ COMMAND AS FOLLOWS
JT]ZRT[ )a^^cL a^^c
-USXbZ [ STe bSPR
3XbZ STe bSPR)# (<1# (&$#%!#QhcTb
!#WTPSb%!bTRc^ab caPRZ 'Rh[X]STab
D]Xcb,Rh[X]STab^U&%''$ !,"("%!$%QhcTb
3TeXRT1^^cBcPac4]S1[^RZb8SBhbcT\
STe bSPR &"( (" &'";X]dg
*UST FOR FUN WE MOUNTED THE INTERNAL 53" FLASH DRIVE TO TAKE A PEEK AT WHAT /RACLE INCLUDED IN THIS
BACKUP 4HE FOLLOWING LISTING SHOWS THE CONTENTS OF THIS DEVICE
JT]ZRT[ )a^^cL a^^c
\^d]c STe bSPR \]c dbQ
JT]ZRT[ )a^^cL \]c dbQ
-[b [
c^cP[#'!
a ga g a^^ca^^c! #'3TR& &)$'Q^^c RPc
a ga g a^^ca^^c %3TR& &)$$Q^^c \bV
Safga ga g!a^^ca^^c# (%3TR & )##RT[[QXcb
Safga ga g!a^^ca^^c# (%3TR & )##VadQ
af a a a^^ca^^c %3TR & )##8NP\N24;;1>>CNdbQ
a ga g a^^ca^^c'$#3TR & ) "X\PVT XS
a ga g a^^ca^^c## 3TR & ) #X\VQ^^c [bc
af af a a^^ca^^c%$&('##3TR& &)$ X]XcaS ! % ' (# " ! T[$ X\V
af a a a^^ca^^c%''' 3TR & )##X]XcaS ! % ' (# " " T[$ X\V
a ga g a^^ca^^c!(&#& '3TR & ) #X]XcaS X\V
a ga g a^^ca^^c %#'3TR& &)$$8B>[X]dg QX]
a ga g a^^ca^^c(&3TR& &)$$8B>[X]dg RUV
af a a a^^ca^^c!"3TR & )##ZTa]T[ eTa
Safga ga g#a^^ca^^c# (%3TR & )#"[Pbc6^^S2^]UXV
Safga ga g!a^^ca^^c# (%3TR & )#$[^V
Safg !a^^ca^^c %"'#3TR & )#![^bcU^d]S
a ga g a^^ca^^c(#% 3TR& &)$$\T\cTbc
a ga g a^^ca^^c&"!%3TR& &)$$b_[PbW [bb
a ga g a^^ca^^c && 3TR& &)$$caP]b cQ[
a ga g a^^ca^^c ($$ 3TR& &)$$e\[X]di
af a a a^^ca^^c ($$ <Ph!&! e\[X]di ! % ' (# " ! T[$
af a a^^ca^^c ($$ 3TR & )!"e\[X]di ! % ' (# " " T[$
)N THIS BACKUP WE SEE THE ,INUX BOOT IMAGES AND ALL THE FILES REQUIRED TO BOOT ,INUX AND RESTORE THE
OPERATING SYSTEM .OTICE THAT YOU ALSO SEE A DIRECTORY CALLED [Pbc6^^S2^]UXV 4HIS DIRECTORY IS A BACKUP
OF THE ^_c ^aPR[T RT[[^b Xb^ DIRECTORY ON OUR STORAGE CELL 4HERE IS ALSO A DIRECTORY CALLED RT[[QXcb
CONTAINING THE #ELL 3ERVER SOFTWARE 3O NOT ONLY DO WE HAVE A COMPLETE COPY OF EVERYTHING NEEDED TO
RECOVER OUR STORAGE CELL TO A BOOTABLE STATE ON THE INTERNAL 53" DRIVE BUT WE ALSO HAVE AN ONLINE BACKUP
OF ALL OF OUR IMPORTANT CELL CONFIGURATION FILES AND #ELL 3ERVER BINARIES
#(!04%2 2%#/6%2).' %8!$!4!
External USB Drive
)N ADDITION TO THE BUILT IN #%,,"//4 53" FLASH DRIVE %XADATA ALSO PROVIDES A WAY TO CREATE YOUR OWN
EXTERNAL BOOTABLE RECOVERY IMAGE USING A COMMON n'" 53" FLASH DRIVE YOU CAN BUY AT THE LOCAL
ELECTRONICS STORE %XADATA WILL CREATE THE RESCUE IMAGE ON THE FIRST EXTERNAL 53" DRIVE IT FINDS SO BEFORE
YOU CREATE THIS RECOVERY IMAGE YOU MUST REMOVE ALL OTHER EXTERNAL 53" DRIVES FROM THE SYSTEM OR THE
SCRIPT WILL THROW A WARNING AND EXIT
2ECALL THAT %XADATA STORAGE CELLS MAINTAIN TWO VERSIONS OF THE OPERATING SYSTEM AND CELL SOFTWARE
ACTIVE AND INACTIVE 4HESE ARE MANAGED AS TWO SEPARATE SETS OF DISK PARTITIONS FOR THE AND ^_c ^aPR[T
FILE SYSTEMS AS CAN BE CONFIRMED USING THE X\PVTX]U^ COMMAND AS FOLLOWS
-X\PVTX]U^kVaT_STeXRT
0RcXeTbhbcT\_PacXcX^]^]STeXRT) STe \S%
0RcXeTb^UcfPaT_PacXcX^]^]STeXRT) STe \S'
8]PRcXeTbhbcT\_PacXcX^]^]STeXRT) STe \S$
8]PRcXeTb^UcfPaT_PacXcX^]^]STeXRT) STe \S&
4HE X\PVTX]U^ COMMAND SHOWS THE CURRENT !CTIVE AND PREVIOUS )NACTIVE SYSTEM VOLUMES ON THE
STORAGE CELL 5SING THE SU COMMAND WE CAN SEE THAT WE ARE INDEED CURRENTLY USING THE !CTIVE PARTITIONS
STe \S% AND STe \S' IDENTIFIED IN THE OUTPUT FROM THE X\PVTX]U^ COMMAND
-SUkTVaT_5X[TbhbcT\k\S%k\S'
5X[TbhbcT\ : Q[^RZbDbTS0ePX[PQ[TDbT<^d]cTS^]
STe \S% " &&$!%%"!'"%" % ' #%'
STe \S'! %"## %$#($% " "%%'"# ^_c ^aPR[T
"Y DEFAULT THE \PZTNRT[[Q^^cNdbQ COMMAND WILL CREATE A RESCUE IMAGE OF YOUR ACTIVE CONFIGURATION
THE ONE YOU ARE CURRENTLY RUNNING 4HE ~X]PRcXeT OPTION ALLOWS YOU TO CREATE A RESCUE IMAGE FROM THE
PREVIOUS CONFIGURATION 4HE INACTIVE PARTITIONS ARE THE SYSTEM VOLUMES THAT WERE ACTIVE WHEN THE LAST
PATCH WAS INSTALLED
4HE \PZTNRT[[Q^^cNdbQ COMMAND IS USED TO CREATE A BOOTABLE RESCUE IMAGE 4O CREATE AN EXTERNAL
RESCUE IMAGE ALL YOU HAVE TO DO IS PLUG A 53" FLASH DRIVE INTO ONE OF THE 53" PORTS ON THE FRONT PANEL OF
THE STORAGE CELL AND RUN THE \PZTNRT[[Q^^cNdbQ COMMAND
■ #AUTION 4HE RESCUE IMAGE WILL BE CREATED ON THE FIRST EXTERNAL 53" DRIVE FOUND ON THE SYSTEM "EFORE CREATING
AN EXTERNAL RESCUE IMAGE REMOVE ALL OTHER EXTERNAL 53" DRIVES FROM THE SYSTEM
&OR EXAMPLE THE FOLLOWING LISTING SHOWS THE PROCESS OF CREATING AN EXTERNAL 53" RESCUE IMAGE 4HE
OUTPUT FROM THE \PZTNRT[[Q^^cNdbQ SCRIPT IS FAIRLY LENGTHY A LITTLE OVER LINES SO WE WONT SHOW ALL OF
IT HERE 3OME OF THE OUTPUT EXCLUDED FROM THE FOLLOWING LISTING INCLUDES OUTPUT FROM THE USXbZ COMMAND
THAT IS USED TO CREATE PARTITIONS ON THE 53" DRIVE FORMATTING OF THE FILE SYSTEMS AND THE MANY FILES THAT
ARE COPIED TO CREATE THE BOOTABLE RESCUE DISK
JT]ZRT[ )a^^cL ^_c ^aPR[T Bd__^acC^^[b
- \PZTNRT[[Q^^cNdbQ
2P]SXSPcTU^acWT>aPR[T4gPSPcP2T[[bcPacd_Q^^cSTeXRT) STe bSPS
#(!04%2 2%#/6%2).' %8!$!4!
?PacXcX^]^]RP]SXSPcTSTeXRT) STe bSPS
CWTRdaaT]c_a^SdRceTabX^]) ! ! %
...
BTccX]VX]cTaeP[QTcfTT]RWTRZbc^ bTR^]Sb
2^_hX]V X\VQ^^c [bcc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V Q^^c RPcc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V Q^^c \bVc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V X\PVT XSc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V X]XcaS X\Vc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V Xb^[X]dg QX]c^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V Xb^[X]dg RUVc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V \T\cTbcc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V b_[PbW [bbc^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V caP]b cQ[c^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V e\[X]dic^ \]c dbQ \PZT RT[[Q^^c
2^_hX]V RT[[QXcb RT[[ QX]c^ \]c dbQ \PZT RT[[Q^^c RT[[QXcb
2^_hX]V RT[[QXcb RT[[Q^^c cQic^ \]c dbQ \PZT RT[[Q^^c RT[[QXcb
2^_hX]V RT[[QXcb RT[[Uf cQic^ \]c dbQ \PZT RT[[Q^^c RT[[QXcb
2^_hX]V RT[[QXcb RT[[a_\b cQic^ \]c dbQ \PZT RT[[Q^^c RT[[QXcb
...
2^_hX]V [Pbc6^^S2^]UXV RT[[ R^]Uc^ \]c dbQ \PZT RT[[Q^^c [Pbc6^^S2^]UXV
2^_hX]V [Pbc6^^S2^]UXV RT[[ RT[[bae ST_[^h R^]UXV RT[[NSXbZNR^]UXV g\[c^
...
2^_hX]V [Pbc6^^S2^]UXV TcR bbW bbWNW^bcNZTh _dQc^
\]c dbQ \PZT RT[[Q^^c [Pbc6^^S2^]UXV TcR bbW
2^_hX]V [Pbc6^^S2^]UXV TcR bbW bbWNW^bcNSbPNZThc^
\]c dbQ \PZT RT[[Q^^c [Pbc6^^S2^]UXV TcR bbW
2^_hX]V [Pbc6^^S2^]UXV a^^cNbbW cQic^ \]c dbQ \PZT RT[[Q^^c [Pbc6^^S2^]UXV
2^_hX]V [Pbc6^^S2^]UXV RT[[\^]Xc^aNbbW cQic^ \]c dbQ \PZT RT[[Q^^c [Pbc6^^S2^]UXV
Ad]]X]VcPa g Y _ 2 \]c dbQ \PZT RT[[Q^^c U
^_c ^aPR[T RT[[^b Xb^ RT[[QXcb RT[[Q^^c cQiVadQ
2^_hX]V ^_c ^aPR[T RT[[^b c\_[ DB1NVadQc^ \]c dbQ \PZT RT[[Q^^c VadQ VadQ R^]U
2^_hX]V ^_c ^aPR[T RT[[^b c\_[ ^aPR[T g_\ Vic^ \]c dbQ \PZT RT[[Q^^c VadQ ^aPR[T g_\ Vi
...
6=D6AD1eTabX^] (&%# :[^fTa " &!:d__Ta\T\^ah
J<X]X\P[10B7 [XZT[X]TTSXcX]VXbbd__^acTS 5^acWTUXabcf^aSC01
[Xbcb_^bbXQ[TR^\\P]SR^\_[TcX^]b 0]hfWTaTT[bTC01[XbcbcWT_^bbXQ[T
R^\_[TcX^]b^UPSTeXRT UX[T]P\T L
VadQ-a^^cWS
5X[TbhbcT\ch_TXbTgc!Ub_PacXcX^]ch_T g'"
VadQ-bTcd_WS
2WTRZX]VXU Q^^c VadQ bcPVT TgXbcb ]^
2WTRZX]VXU VadQ bcPVT TgXbcb hTb
2WTRZX]VXU VadQ bcPVT!TgXbcb hTb
2WTRZX]VXU VadQ T!UbNbcPVT N$TgXbcb hTb
Ad]]X]VT\QTS VadQ T!UbNbcPVT N$WS $bTRc^abPaTT\QTSSTS
bdRRTTSTS
Ad]]X]VX]bcP[[ VadQ bcPVT WS WS $_WS VadQ bcPVT! VadQ VadQ R^]U
bdRRTTSTS
3^]T
VadQ-
#(!04%2 2%#/6%2).' %8!$!4!
(ERE YOU CAN SEE THAT THE \PZTNRT[[Q^^cNdbQ SCRIPT COPIES OVER ALL OF THE STORAGE CELL SOFTWARE
RT[[QXcb AND CONFIGURATION FILES [Pbc6^^S2^]UXV IT NEEDS TO RECOVER THE STORAGE CELL &INALLY YOU SEE
THAT THE 'RUB BOOT LOADER IS INSTALLED ON THE 53" DRIVE SO YOU CAN BOOT THE SYSTEM FROM IT 7HEN THE
SCRIPT COMPLETES YOU CAN REMOVE THE EXTERNAL 53" DISK FROM THE SYSTEM 4HIS RESCUE DISK CAN LATER BE
USED FOR RESTORING YOUR STORAGE CELL TO WORKING CONDITION SHOULD THE NEED ARISE
!CTIVE AND )NACTIVE 3YSTEM 6OLUMES
%XADATA HAS THREE MODES FOR INSTALLING SOFTWARE FIRST TIME INSTALL PATCH OR in-partition patch AND AN
out-of-partition upgrade ! JOURNAL OF YOUR INSTALLATION HISTORY CAN BE SEEN USING THE X\PVTWXbc^ah
COMMAND
-X\PVTWXbc^ah
ETabX^]) ! ! "
8\PVTPRcXePcX^]SPcT)! $ $! ) &) & &
8\PVX]V\^ST)fresh
8\PVX]VbcPcdb)bdRRTbb
ETabX^]) ! ! %
8\PVTPRcXePcX^]SPcT)! % " () ) ' &
8\PVX]V\^ST)patch
8\PVX]VbcPcdb)bdRRTbb
...
ETabX^]) ! ! ! ! % !
8\PVTPRcXePcX^]SPcT)! ! & )#$) & %
8\PVX]V\^ST)out of partition upgrade
8\PVX]VbcPcdb)bdRRTbb
4HE P[[ OPTION CAN BE USED TO SHOW MUCH MORE DETAIL ABOUT YOUR SOFTWARE INSTALLATION HISTORY
In-Partition Patches
!S DISCUSSED EARLIER %XADATA MAINTAINS TWO SETS OF SYSTEM VOLUMES !CTIVE AND )NACTIVE )N
PARTITION
PATCHES ARE RARE "UT WHEN AN IN
PARTITION PATCH IS PERFORMED THE PATCH IS APPLIED TO THE !CTIVE SYSTEM
IMAGE %XADATA SAVES A COPY OF THE FILES AND SETTINGS THAT WERE CHANGED DURING THE INSTALLATION PROCESS SO
YOU CAN BACK OUT THE PATCH LATER IF NEEDED
Out-of-Partition Upgrades
7HEN AN OUT
OF
PARTITION UPGRADE IS PERFORMED %XADATA INSTALLS THE SOFTWARE INTO THE )NACTIVE SYSTEM
VOLUMES )F THE INSTALLATION FAILS THE CELL WILL CONTINUE TO BOOT FROM THE CURRENT !CTIVE SYSTEM IMAGE
4HIS IS HOW %XADATA ENSURES THAT A FAILED UPGRADE DOES NOT CAUSE AN EXTENDED OUTAGE )F THE UPGRADE
COMPLETES SUCCESSFULLY %XADATA SETS THE UPGRADED VOLUMES TO !CTIVE STATUS AND CHANGES THE OLD !CTIVE
VOLUMES TO )NACTIVE )T THEN UPDATES THE Q^^c VadQ VadQ R^]U FILE WITH THE DEVICE NAME OF THE NEWLY
ACTIVATED ROOT FILE SYSTEM AND REBOOTS $URING THE BOOT PROCESS THE 'RUB BOOT LOADER READS THE
VadQ R^]U FILE TO FIND THE DEVICE IT WILL USE FOR MOUNTING THE ROOT FILE SYSTEM ! QUICK LOOK AT THE
VadQ R^]U FILE SHOWS THAT OUR !CTIVE PARTITION FOR THE ROOT FILE SYSTEM IS STe \S%
-VaT_ZTa]T[ Q^^c VadQ VadQ R^]UkRdc R $"kWTPS
ZTa]T[ e\[X]di ! % ' (# " " T[$a^^c, STe \S%
#(!04%2 2%#/6%2).' %8!$!4!
4HE X\PVTX]U^ COMMAND DISPLAYS EVERYTHING WE NEED TO KNOW ABOUT OUR !CTIVE AND )NACTIVE SYSTEM
IMAGES INCLUDING THE PARTITION DEVICE NAMES WHEN THE IMAGE WAS CREATED INSTALLED WHEN IT WAS
ACTIVATED AND WHETHER OR NOT YOU CAN ROLL BACK TO IT SWITCH IT TO THE !CTIVE IMAGE !LSO NOTICE THAT THE
INTERNAL #%,,"//4 53" FLASH DRIVE WE DISCUSSED EARLIER IS LISTED STe bSPR
-X\PVTX]U^ P[[
:Ta]T[eTabX^])! % ' (# " " T[$ B<?CdT0dV" !!)# ) "43C! g'%N%#
2T[[eTabX^])>BBN ! " N;8=DG G%#N ! % !
2T[[a_\eTabX^])RT[[ ! ! ! N;8=DG G%#N ! % !
0RcXeTX\PVTeTabX^]) ! ! ! ! % !
Active image created: 2010-12-07 05:55:51 -0800
Active image activated: 2010-12-17 11:45:17 -0600
0RcXeTX\PVTch_T)_a^SdRcX^]
0RcXeTX\PVTbcPcdb)bdRRTbb
0RcXeTX]cTa]P[eTabX^])
0RcXeTX\PVT[PQT[)>BBN ! " N;8=DG G%#N ! % !
0RcXeT]^STch_T)BC>A064
0RcXeTbhbcT\_PacXcX^]^]STeXRT) STe \S%
0RcXeTb^UcfPaT_PacXcX^]^]STeXRT) STe \S'
In partition rollback: Impossible
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 11.2.2.2.0.101206.2
8]PRcXeTX\PVTeTabX^]) ! ! $
8]PRcXeTX\PVTRaTPcTS)! $ $)$ )! &
8]PRcXeTX\PVTPRcXePcTS)! ' #)#')#( %
8]PRcXeTX\PVTch_T)_a^SdRcX^]
8]PRcXeTX\PVTbcPcdb)bdRRTbb
8]PRcXeTX]cTa]P[eTabX^]) $
8]PRcXeTX\PVT[PQT[)>BBN ! ! N;8=DG G%#N $
8]PRcXeT]^STch_T)BC>A064
8]PRcXeTbhbcT\_PacXcX^]^]STeXRT) STe \S$
8]PRcXeTb^UcfPaT_PacXcX^]^]STeXRT) STe \S&
Boot area has rollback archive for the version: 11.2.2.1.1.101105
Rollback to the inactive partitions: Possible
%XADATA STORAGE CELLS ARE AN ELEGANT BLEND OF ,INUX COMMANDS AND PROPRIETARY PROGRAMS 4HE MORE
WE POKE AROUND THROUGH THEIR SCRIPTS PROCESSES AND DOCUMENTATION THE GREATER OUR APPRECIATION OF THE
ARCHITECTURE AND WHAT /RACLE HAS DONE TO BUILD IN SOFTWARE REDUNDANCY AND RECOVERY 4HE STORAGE CELLS
MAINTAIN A BOOTABLE CELL RECOVERY IMAGE ON AN INTERNAL 53" FLASH DISK AS WELL AS AN ONLINE COPY OF
EVERYTHING %XADATA NEEDS TO CREATE AN EXTERNAL BOOTABLE RECOVERY IMAGE 4HIS SECONDARY COPY OF YOUR
CONFIGURATION PROVIDES A CONVENIENT WAY TO RESTORE INDIVIDUAL FILES IF NEEDED 4HESE ARE ALL IN ADDITION TO
%XADATAS ABILITY TO ROLL BACK PATCHES AND UPGRADES CONVENIENTLY !LL THESE BUILT
IN FEATURES PROVIDE
CONVENIENT RECOVERY FROM VARIOUS USER ERRORS AND PROTECTION FROM ISSUES THAT CAN ARISE WHEN PATCHING OR
UPGRADING THE STORAGE CELL
#(!04%2 2%#/6%2).' %8!$!4!
Backing Up the Database
%XADATA REPRESENTS A LEAP FORWARD IN CAPACITY AND PERFORMANCE *UST A FEW YEARS AGO LARGE DATABASES
WERE DESCRIBED IN TERMS OF GIGABYTES 4ODAY ITS NOT UNCOMMON TO FIND DATABASES MEASURED IN TERABYTES
)T WASNT LONG AGO WHEN A TABLE WAS CONSIDERED HUGE IF IT CONTAINED TENS OF MILLIONS OF ROWS 4ODAY WE
COMMONLY SEE TABLES THAT CONTAIN TENS OF BILLIONS OF ROWS 4HIS TREND MAKES IT CLEAR THAT WE WILL SOON SEE
DATABASES MEASURED IN EXABYTES !S YOU MIGHT IMAGINE THIS CREATES SOME UNIQUE CHALLENGES FOR BACKUP
AND RECOVERY 4HE TOOLS FOR BACKING UP %XADATA DATABASES HAVE NOT FUNDAMENTALLY CHANGED AND THE
NEED TO COMPLETE BACKUPS IN A REASONABLE PERIOD OF TIME IS BECOMING INCREASINGLY DIFFICULT TO ACHIEVE
3OME OF THE STRATEGIES WELL DISCUSS HERE WILL NOT BE NEW "UT WE WILL BE LOOKING AT WAYS TO LEVERAGE THE
SPEED OF THE PLATFORM SO BACKUP PERFORMANCE CAN KEEP PACE WITH THE INCREASING VOLUME OF YOUR
DATABASES
$ISK
"ASED "ACKUPS
/RACLE G INTRODUCED US TO A NEW FEATURE CALLED THE Flash Recovery Area, WHICH EXTENDED 2ECOVERY
-ANAGERS STRUCTURED APPROACH TO MANAGING BACKUPS 2ECENTLY THIS FEATURE HAS BEEN RENAMED TO THE
Fast Recovery Area (FRA) 4HE &2! IS A STORAGE AREA MUCH LIKE ANY OTHER DATABASE STORAGE )T CAN BE
CREATED ON RAW DEVICES BLOCK DEVICES FILE SYSTEMS AND OF COURSE !3- 3INCE THE &2! UTILIZES DISK
BASED
STORAGE IT PROVIDES A VERY FAST STORAGE MEDIUM FOR DATABASE RECOVERY 4HIS IS ESPECIALLY TRUE WHEN USING
%XADATAS HIGH
PERFORMANCE STORAGE ARCHITECTURE %LIMINATING THE NEED TO RETRIEVE BACKUPS FROM TAPE
CAN SHAVE HOURS AND SOMETIMES DAYS OFF THE TIME IT TAKES TO RECOVER YOUR DATABASES !ND SINCE THE &2!
IS AN EXTENSION OF THE DATABASE /RACLE AUTOMATICALLY MANAGES THAT SPACE FOR YOU 7HEN FILES IN THE &2!
ARE BACKED UP TO TAPE THEY ARE NOT IMMEDIATELY DELETED 4HEY ARE INSTEAD KEPT ONLINE AS LONG AS THERE IS
ENOUGH FREE SPACE TO DO SO 7HEN MORE SPACE IS NEEDED THE DATABASE DELETES IN A &)&/ MANNER
ENOUGH OF THESE FILES TO PROVIDE THE NEEDED SPACE
4APE
"ASED "ACKUPS
5SING THE &2! FOR DISK
BASED BACKUPS CAN GREATLY IMPROVE THE TIME IT TAKES TO RECOVER YOUR DATABASES
BUT IT DOES NOT ELIMINATE THE NEED FOR TAPE BACKUPS !S A MATTER OF FACT TAPE
BASED BACKUPS ARE REQUIRED
FOR BACKING UP THE &2! -OVING LARGE QUANTITIES OF BACKUP DATA TO TAPE CAN BE A CHALLENGE AND WITH THE
VOLUME OF DATA THAT CAN BE STORED ON %XADATA THE NEED FOR HIGH
PERFORMANCE TAPE BACKUPS IS CRITICAL
%XADATA 6 COMES EQUIPPED WITH 'IGABIT %THERNET 'IG% PORTS THAT ARE EACH CAPABLE OF DELIVERING
THROUGHPUT UP TO MEGABITS PER SECOND %XADATA 8
COMES WITH TWO 'IGABIT %THERNET PORTS
EACH CAPABLE OF DELIVERING UP TO TIMES THE THROUGHPUT OF THE 'IG% PORTS OF THE 6 4HE PROBLEM IS THAT
EVEN THE 'IG% PORTS BETWEEN %XADATA AND THE TAPE LIBRARYS MEDIA SERVER MAY NOT BE FAST ENOUGH TO
KEEP UP
! COMMON SOLUTION TO THIS PROBLEM IS TO INSTALL A 'BPS 1$2 )NFINI"AND CARD OR TWO INTO THE
MEDIA SERVER ALLOWING IT TO BE LINKED DIRECTLY INTO THE SPARE PORTS ON THE %XADATA )NFINI"AND NETWORK
SWITCH &IGURE
ILLUSTRATES A COMMON BACKUP CONFIGURATION THAT LEVERAGES THE HIGH
SPEED )NFINI"AND
NETWORK INSIDE THE %XADATA RACK TO PROVIDE HIGH
SPEED BACKUPS TO TAPE
#(!04%2 2%#/6%2).' %8!$!4!
Media Server
Infiniband
RMAN 2/4 channels
Tape Library
2/4 Drives
Figure 9-1. Exadata backup architecture
&OR VERY LARGE DATABASES ONE )NFINI"AND CARD MAY NOT PROVIDE THE THROUGHPUT NEEDED TO COMPLETE
BACKUPS IN A REASONABLE TIME &OR /RACLE 2!# DATABASES BACKUPS CAN BE PARALLELIZED AND DISTRIBUTED
ACROSS ANY OR ALL NODES IN THE 2!# CLUSTER &OR %XADATA FULL RACK CONFIGURATIONS THIS MEANS YOU CAN HAVE
UP TO NODES IN A SINGLE RACK PARTICIPATING IN THE BACKUP WORKLOAD )NSTALLING ADDITIONAL )NFINI"AND
CARDS INTO THE MEDIA SERVER ALLOWS YOU TO INCREASE THE THROUGHPUT IN 'BPS INCREMENTS '"
EFFECTIVE UP TO THE LIMITS OF THE MEDIA SERVER !N ADDITIONAL MEDIA SERVER CAN BE ADDED TO THE
CONFIGURATION AND LOAD
BALANCED TO EXTEND PERFORMANCE EVEN FURTHER /RACLES -!! GROUP PUBLISHED A
VERY GOOD WHITE PAPER ENTITLED h"ACKUP AND 2ECOVERY 0ERFORMANCE AND "EST 0RACTICES FOR %XADATA #ELL
AND THE 3UN /RACLE $ATABASE -ACHINE v IN WHICH THEY REPORTED BACKUP RATES OF UP TO -"SEC OR
4"HR FOR TAPE BACKUPS
"ACKUP FROM 3TANDBY $ATABASE
)F YOU ARE PLANNING TO SET UP A DISASTER RECOVERY SITE USING $ATA 'UARD YOU HAVE THE OPTION OF OFFLOADING
YOUR DATABASE BACKUPS TO THE STANDBY DATABASE 4HIS IS NOT IN ANY WAY AN %XADATA FEATURE SO WE WILL
ONLY TOUCH BRIEFLY ON THE SUBJECT 4HE MAIN PURPOSE OF THE STANDBY DATABASE IS TO TAKE OVER THE
PRODUCTION LOAD IN THE EVENT THAT THE PRIMARY DATABASE EXPERIENCES A TOTAL FAILURE (OWEVER USING A
PHYSICAL STANDBY DATABASE ALSO PROVIDES AN ADDITIONAL BACKUP FOR YOUR PRIMARY DATABASE )F A DATAFILE
FROM THE PRIMARY DATABASE IS LOST A REPLACEMENT DATAFILE FROM THE STANDBY DATABASE CAN BE USED TO
REPLACE IT /NCE THE FILE HAS BEEN RESTORED TO THE PRIMARY DATABASE ARCHIVED REDO LOGS ARE USED TO RECOVER
THE DATAFILE UP TO THE CURRENT 3#. OF THE DATABASE 4HE STANDBY DATABASE IS TYPICALLY MOUNTED BUT NOT
OPEN DURING NORMAL OPERATIONS #OLD BACKUPS CAN BE MADE FROM THE STANDBY TO THE &AST 2ECOVERY !REA
&2! AND THEN TO TAPE "ACKUPS FROM THE STANDBY DATABASE CAN BE RESTORED DIRECTLY TO THE PRIMARY
DATABASE 4HIS PROVIDES THREE LEVELS OF RECOVERY TO CHOOSE FROM BEFORE DECIDING WHETHER A FAILOVER TO THE
STANDBY IS NECESSARY
)T IS BEST TO USE AN %XADATA PLATFORM FOR YOUR STANDBY DATABASE 4HIS IS BECAUSE ALTHOUGH TABLES THAT
USE (YBRID #OLUMNAR #OMPRESSION (## WILL REPLICATE TO NON
%XADATA DATABASES JUST FINE YOU WILL NOT
BE ABLE TO READ FROM THEM 4HE DATABASE KERNEL ON NON
%XADATA DATABASES CANNOT READ (##
#(!04%2 2%#/6%2).' %8!$!4!
COMPRESSED DATA &OR EXAMPLE THE FOLLOWING ERROR IS RETURNED WHEN YOU SELECT FROM AN (## TABLE ON A
STANDARD X DATABASE
B@;-bT[TRcSXbcX]RcbTV\T]cN]P\TUa^\QXVcPQNPaRWNWXVW*
bT[TRcSXbcX]RcbTV\T]cN]P\TUa^\b\P[[NcPQ[TNPaRWNWXVW
4AA>APc[X]T )
>A0 %#" &)WhQaXSR^[d\]PaR^\_aTbbX^]Xb^][hbd__^acTSX]cPQ[Tb_PRTbaTbXSX]V^]4gPSPcP
bc^aPVT
9OUR COMPRESSED DATA IS STILL INTACT 9OU JUST CANNOT READ IT UNLESS YOU FIRST UNCOMPRESS IT (##
COMPRESSED TABLES CAN BE UNCOMPRESSED ON NON
%XADATA DATABASES USING THE 0;C4AC01;4<>E4
COMMAND AS FOLLOWS
B@;-P[cTacPQ[T186C01N0A278E4N7867\^eT]^R^\_aTbb*
0ARTITIONED TABLES CAN BE UNCOMPRESSED IN A SIMILAR MANNER AND THE OPERATION CAN BE PARALLELIZED
USING THE _PaP[[T[ OPTION AS YOU CAN SEE IN THE FOLLOWING COMMAND
B@;-P[cTacPQ[T186C01N0A278E4N7867\^eT_PacXcX^]90=N! ]^R^\_aTbb_PaP[[T[*
/NCE THE TABLE IS UNCOMPRESSED IT CAN BE READ FROM A NON
%XADATA DATABASE +EEP IN MIND THAT
WITH THE HIGH DEGREE OF COMPRESSION (## PROVIDES YOU MUST TAKE INTO CONSIDERATION THE ADDITIONAL DISK
STORAGE THAT WILL BE REQUIRED BY THE UNCOMPRESSED TABLE OR PARTITION WHICH CAN BE QUITE SUBSTANTIAL
%XADATA /PTIMIZATIONS FOR 2-!.
7HEN 2-!. PERFORMS AN INCREMENTAL BACKUP ON THE %XADATA PLATFORM RT[[bae FILTERS OUT UNWANTED
BLOCKS AND SENDS BACK ONLY THOSE THAT HAVE CHANGED SINCE THE LAST LEVEL OR LEVEL BACKUP 4HIS
IMPROVES THE PERFORMANCE OF INCREMENTAL BACKUPS AND REDUCES THE WORKLOAD ON THE DATABASE SERVER
"UT EVEN WHEN ONLY A RELATIVELY SMALL NUMBER OF BLOCKS HAVE CHANGED DISCOVERING THEM IS A VERY )/
INTENSIVE PROCESS BECAUSE EVERY BLOCK IN THE DATABASE MUST BE EXAMINED TO DETERMINE WHICH ONES HAVE
CHANGED SINCE THE LAST INCREMENTAL BACKUP 4HIS IS TRUE FOR BOTH %XADATA AND NON
%XADATA DATABASES
4HE ONLY DIFFERENCE IS WHERE THE WORK IS DONE ON THE DATABASE SERVER OR ON THE STORAGE CELLS ! FEW YEARS
AGO /RACLE G INTRODUCED block change tracking "#4 TO ADDRESS THIS PROBLEM /F COURSE THIS WAS LONG
BEFORE %XADATA CAME ONTO THE SCENE 4HIS FEATURE MAINTAINS A BITMAP STRUCTURE IN A FILE CALLED THE BLOCK
CHANGE TRACKING FILE %ACH BIT IN THE "#4 FILE BIT PER + OF DATA REPRESENTS A GROUP OF BLOCKS IN THE
DATABASE 7HEN A DATA BLOCK IS MODIFIED /RACLE FLIPS A BIT IN THE "#4 FILE REPRESENTING THE GROUP OF
BLOCKS IN WHICH THE CHANGED BLOCK RESIDES 7HEN AN INCREMENTAL BACKUP IS TAKEN 2-!. RETRIEVES THE
WHOLE GROUP OF BLOCKS REPRESENTED BY A FLIPPED BIT IN THE "#4 FILE AND EXAMINES THEM TO DETERMINE
WHICH ONE CHANGED "LOCK CHANGE TRACKING INTRODUCES MINIMAL OVERHEAD ON THE DATABASE SERVER AND IS A
VERY EFFICIENT WAY TO TRACK CHANGED BLOCKS !ND SINCE IT GREATLY REDUCES THE NUMBER OF BLOCKS THAT MUST
BE EXAMINED DURING A BACKUP IT IMPROVES BACKUP PERFORMANCE WHILE REDUCING THE WORKLOAD ON THE
DATABASE SERVER AND STORAGE GRID
&OR THE %XADATA PLATFORM YOU MAY CHOOSE TO ALLOW RT[[bae TO DO ALL OF THE BLOCK FILTERING FOR
INCREMENTAL BACKUPS OR USE IT IN TANDEM WITH BLOCK CHANGE TRACKING "LOCK CHANGE TRACKING SEEMS TO
PROVIDE THE MOST BENEFIT WHEN FEWER THAN PERCENT OF THE BLOCKS IN THE DATABASE HAVE CHANGED SINCE
THE LAST LEVEL OR LEVEL BACKUP )F YOUR DATABASE IS CLOSE TO THAT THRESHOLD YOU SHOULD DO SOME TESTING TO
DETERMINE WHETHER OR NOT "#4 IMPROVES INCREMENTAL BACKUP PERFORMANCE 4HE 1;>2:BNB:8??43N8=N24;;
COLUMN OF THE E102:D?N30C058;4 VIEW SHOWS THE NUMBER OF BLOCKS THAT WERE READ AND FILTERED OUT AT THE
STORAGE CELL 4HIS OFFLOADING IS TRANSPARENT AND REQUIRES NO USER INTERVENTION OR SPECIAL PARAMETERS TO BE
SET AHEAD OF TIME
#(!04%2 2%#/6%2).' %8!$!4!
Wait Events
4HERE ARE TWO %XADATA
SPECIFIC WAIT EVENTS THAT ARE TRIGGERED BY DATABASE BACKUP AND RECOVERY
OPERATIONS ON THE %XADATA PLATFORM RT[[b\PacX]RaT\T]cP[QPRZd_ AND RT[[b\PacaTbc^aTUa^\
QPRZd_ 4HESE WAIT EVENTS ARE COVERED IN MORE DETAIL IN #HAPTER
cell smart incremental backup: 4HIS WAIT EVENT OCCURS WHEN %XADATA OFFLOADS
INCREMENTAL BACKUP PROCESSING TO THE STORAGE CELLS 4HE 0 COLUMN OF THE
EB4BB8>=NF08C VIEW CONTAINS THE CELL HASH NUMBER 4HIS HASH VALUE CAN BE
USED TO COMPARE THE RELATIVE BACKUP PERFORMANCE OF EACH STORAGE CELL AND
DETERMINE IF THERE IS A PERFORMANCE PROBLEM ON ANY OF THE CELLS
cell smart restore from backup: 4HIS WAIT EVENT OCCURS DURING RESTORE
OPERATIONS WHEN %XADATA OFFLOADS THE TASK OF INITIALIZING A FILE TO THE STORAGE
CELLS 4HE 0 COLUMN OF EB4BB8>=NF08C CONTAINS THE CELL HASH NUMBER 4HIS
HASH VALUE CAN BE USED TO COMPARE THE RELATIVE RESTORE PERFORMANCE OF EACH
STORAGE CELL AND DETERMINE IF THERE IS A PERFORMANCE PROBLEM ON ANY OF THE CELLS
Recovering Exadata
! BETTER TITLE FOR THIS SECTION MIGHT BE h7HEN 4HINGS 'O 7RONGv !FTER ALL THATS USUALLY ABOUT THE TIME
WE REALIZE HOW LITTLE PRACTICAL EXPERIENCE WE HAVE RECOVERING OUR SYSTEMS !S CORPORATE !MERICA
CONTINUES TO SQUEEZE EVERY DROP OF PRODUCTIVE TIME OUT OF OUR WORK WEEK $"!S AND 3YSTEM
!DMINISTRATORS SPEND MOST IF NOT ALL OF THEIR WAKING HOURS AND SOMETIMES SLEEPING HOURS JUST hKEEPING
THE WHEELS ONv 3O ACTUALLY PRACTICING SYSTEM RECOVERY IS MORE OFTEN THAN NOT TREATED LIKE THE PROVERBIAL
hRED
HEADED STEP CHILDvSELDOM THOUGHT ABOUT AND RARELY ATTENDED TO !ND EVEN IF WE FIND OURSELVES IN
THE ENVIABLE POSITION OF HAVING THE TIME TO PRACTICE SYSTEM RECOVERY ITS RARE TO HAVE THE SPARE
EQUIPMENT TO PRACTICE ON 3O KUDOS TO YOU IF YOU ARE READING THIS AND NOTHING IS ACTUALLY BROKEN )N THIS
SECTION WELL BE DISCUSSING %XADATA SYSTEM RECOVERY USING THE BACKUP METHODS WE COVERED IN THE
h"ACKING 5P %XADATAv SECTION OF THIS CHAPTER
2ESTORING THE $ATABASE 3ERVER
"ACKING UP AND RESTORING THE DATABASE SERVERS CAN BE DONE USING THIRD
PARTY BACKUP SOFTWARE OR
HOMEGROWN SCRIPTS USING FAMILIAR COMMANDS LIKE cPa AND iX_ 2ECENTLY /RACLE BEGAN CONFIGURING THE
%XADATA DATABASE SERVERS WITH THE ,INUX ,OGICAL 6OLUME -ANAGER ,6- WHICH PROVIDES SNAPSHOTS FOR
CREATING POINT
IN
TIME cPa
BASED BACKUP SETS 4HE PROCEDURE FOR RECOVERING %XADATA DATABASE SERVERS IS
A VERY STRUCTURED PROCESS THAT IS SPECIFIC TO %XADATA )N THIS SECTION WELL BE STEPPING THROUGH THIS
PROCEDURE PRESUMING THE BACKUP WAS TAKEN USING THE BACKUP PROCEDURE DISCUSSED EARLIER IN THIS
CHAPTER 3O IF YOU HAVENT READ THROUGH THAT SECTION OF THIS CHAPTER YOU MIGHT WANT TO TAKE A LOOK AT IT
BEFORE CONTINUING
Recovery Using LVM Snapshot-Based Backup Images
2ESTORING THE DATABASE SERVER USING THE ,6- SNAPSHOT BACKUP PROCEDURE WE DISCUSSED EARLIER IN THIS
CHAPTER IS A FAIRLY STRAIGHTFORWARD PROCESS 4HE BACKUP IMAGE WE WILL USE IN THIS PROCEDURE
T]ZSQ NbhbcT\NQPRZd_ cPa Qi! IS THE ONE WE CREATED EARLIER IN THIS CHAPTER AND INCLUDES THE Q^^c
AND d FILE SYSTEMS 4HE FIRST THING YOU NEED TO DO IS STAGE THE BACKUP IMAGE ON AN .&3 FILE SYSTEM THAT
CAN BE MOUNTED BY THE FAILED DATABASE SERVER 4HE SERVER IS THEN BOOTED FROM A SPECIAL DIAGNOSTICS )3/
#(!04%2 2%#/6%2).' %8!$!4!
BOOT IMAGE INCLUDED ON ALL %XADATA SERVERS 7HEN THE SYSTEM BOOTS FROM THE DIAGNOSTICS )3/ YOU WILL BE
PROMPTED STEP BY STEP THROUGH THE RECOVERY PROCESS ,ETS TAKE A LOOK AT THE BASIC STEPS FOR RECOVERING A
FAILED DATABASE SERVER FROM THE ,6- SNAPSHOT
BASED BACKUP WE TOOK EARLIER IN THIS CHAPTER
0LACE THE ,6- SNAPSHOT BACKUP IMAGE ON AN .&3 SHARED FILE SYSTEM THAT IS
ACCESSIBLE TO THE FAILED SERVER BY )0 ADDRESS 4HE FILE WELL BE WORKING WITH IS
NAMED T]ZSQ NbhbcT\NQPRZd_ cPa Qi!
■ .OTE 4HIS RECOVERY PROCEDURE USES A BACKUP IMAGE TAKEN USING ,6- SNAPSHOTS "ACKUPS THAT ARE NOT
SNAPSHOT
BASED CAN ALSO BE USED FOR SYSTEMS THAT ARE NOT CONFIGURED WITH ,6- 7ITHOUT ,6- SNAPSHOTS THE
BACKUP IMAGE CAN BE CREATED BY RUNNING THE FOLLOWING COMMAND FROM THE ROOT DIRECTORY
cPa _YReUT]ZSQ NbhbcT\NQPRZd_ cPa Qi! Q^^c TgR[dSTT]ZSQ NbhbcT\NQPRZd_ cPa Qi!
!TTACH THE ^_c ^aPR[T Bd__^acC^^[b SXPV]^bcXRb Xb^ BOOT IMAGE TO THE
FAILED SERVER THROUGH THE ),/- 4HIS CAN BE DONE FROM ANY OF THE OTHER
DATABASE SERVERS IN THE RACK
2EBOOT THE FAILED SERVER AND SELECT THE #$
2/- AS THE BOOT DEVICE 7HEN THE
SYSTEM BOOTS FROM THE DIAGNOSTICS )3/ IT WILL ENTER A SPECIAL SERVER RECOVERY
PROCESS
&ROM THIS POINT ON THE RECOVERY PROCESS WILL INCLUDE STEP
BY
STEP DIRECTIONS
&OR EXAMPLE THE FOLLOWING PROCESS RECOVERS THE DATABASE SERVER FROM THE
BACKUP IMAGE T]ZSQ NbhbcT\NQPRZd_ cPa Qi! !NSWERS TO THE PROMPTS ARE
SHOWN IN bold_italics
2W^^bTUa^\U^[[^fX]VQhch_X]V[TccTaX])
T]cTaX]cTaPRcXeTSXPV]^bcXRbbWT[[ <dbcdbTRaTST]cXP[bUa^\>aPR[Tbd__^acc^[^VX]
aTQ^^c^a_^fTaRhR[Tc^TgXccWTbWT[[
aTbc^aTbhbcT\Ua^\=5BQPRZd_PaRWXeT
BT[TRc)r
0aTh^dbdaTh ]J]L)y
CWTQPRZd_UX[TR^d[SQTRaTPcTSTXcWTaUa^\;E<^a]^] ;E<QPbTSR^\_dcT]^ST ETabX^]bQT[^f
! " P]S ! ! ^aWXVWTaS^]^cbd__^ac;E<QPbTS_PacXcX^]X]V DbT;E<QPbTS
bRWT\Th ])y
4]cTa_PcWc^cWTQPRZd_UX[T^]cWT=5BbTaeTaX]U^a\Pc)
+X_NPSSaTbbN^UNcWTN=5BNbWPaT-) +_PcW- +PaRWXeTNUX[T-
5^aTgP\_[T ) Tg_^ac ^_TaPcX]VNbhbcT\ cPa Qi!
=5B[X]T)]UbNX_)/export/enkdb01_system_backup.tar.bz2
8?0SSaTbb^UcWXbW^bc)192.168.10.62
#(!04%2 2%#/6%2).' %8!$!4!
=Tc\PbZ^UcWXbW^bc)255.255.255.0
3TUPd[cVPcTfPh)192.168.15.1
7HEN ALL THE ABOVE INFORMATION IS ENTERED %XADATA WILL PROCEED TO MOUNT THE
BACKUP IMAGE ACROSS THE NETWORK AND RECOVER THE SYSTEM 7HEN THE RECOVERY
IS FINISHED YOU WILL BE PROMPTED TO LOG IN ,OG IN AS ROOT USING THE PASSWORD
PROVIDED IN THE /RACLE DOCUMENTATION
$ETACH THE DIAGNOSTICS )3/ FROM THE ),/-
2EBOOT THE SYSTEM USING THE aTQ^^c COMMAND 4HE FAILED SERVER SHOULD BE
COMPLETELY RESTORED AT THIS POINT
7HEN THE SYSTEM FINISHES BOOTING YOU CAN VERIFY THE RECOVERY USING THE X\PVTWXbc^ah COMMAND
4HE FOLLOWING LISTING SHOWS THAT THE IMAGE WAS CREATED AS A aTbc^aTUa^\]UbQPRZd_ AND WAS COMPLETED
SUCCESSFULLY
JT]ZSQ )^aPR[T)4G31 L W^\T ^aPR[T
-bd
?Pbbf^aS)
JT]ZSQ )a^^cL a^^c
-X\PVTWXbc^ah
ETabX^]) ! ! "
8\PVTPRcXePcX^]SPcT)! $ $ $)$')$% &
8\PVX]V\^ST)UaTbW
8\PVX]VbcPcdb)bdRRTbb
...
ETabX^]) ! ! ! ! % !
8\PVTPRcXePcX^]SPcT)! ! & )$ )$" %
8\PVX]V\^ST)_PcRW
8\PVX]VbcPcdb)bdRRTbb
ETabX^]) ! ! ! ! % !
8\PVTPRcXePcX^]SPcT)! !" $)!") $ %
8\PVX]V\^ST)restore from nfs backup
8\PVX]VbcPcdb)success
'ENERALLY SPEAKING ITS A GOOD IDEA NOT TO GET TOO CREATIVE WHEN IT COMES TO CUSTOMIZING YOUR
%XADATA DATABASE SERVER /RACLE PERMITS YOU TO CREATE NEW ,6- PARTITIONS AND ADD FILE SYSTEMS TO YOUR
DATABASE SERVERS BUT IF YOU DO SO YOUR RECOVERY WILL REQUIRE SOME ADDITIONAL STEPS 4HEY ARENT TERRIBLY
DIFFICULT BUT IF YOU CHOOSE TO CUSTOMIZE YOUR ,6- PARTITIONS BE SURE TO DOCUMENT THE CHANGES
SOMEWHERE AND FAMILIARIZE YOURSELF WITH THE RECOVERY PROCEDURES FOR CUSTOMIZED SYSTEMS IN THE /RACLE
DOCUMENTATION
Reimaging a Database Server
)F A DATABASE SERVER MUST BE REPLACED OR REBUILT FROM SCRATCH AND THERE IS NO BACKUP IMAGE TO RECOVER
FROM AN IMAGE CAN BE CREATED FROM AN INSTALL IMAGE PROVIDED BY /RACLE 3UPPORT )T IS A LENGTHY AND
HIGHLY COMPLICATED PROCESS BUT WELL HIT THE HIGHLIGHTS HERE SO YOU GET A GENERAL IDEA OF WHAT THIS
PROCESS INVOLVES
#(!04%2 2%#/6%2).' %8!$!4!
"EFORE THE SERVER CAN BE REIMAGED IT MUST BE REMOVED FROM THE 2!# CLUSTER 4HIS IS THE STANDARD
PROCEDURE FOR DELETING A NODE FROM ANY G2 2!# CLUSTER &IRST THE LISTENER ON THE FAILED SERVER MUST BE
SHUT DOWN AND DISABLED 4HEN THE >A02;4N7><4 FOR THE DATABASE BINARIES IS REMOVED FROM THE /RACLE
INVENTORY 4HE 6)0 IS THEN STOPPED AND REMOVED FROM THE CLUSTER CONFIGURATION AND THE NODE DELETED
FROM THE CLUSTER &INALLY THE >A02;4N7><4 FOR THE 'RID )NFRASTRUCTURE IS REMOVED FROM THE /RACLE
INVENTORY
/RACLE 3UPPORT WILL PROVIDE YOU WITH A R^\_dcT8\PVT<PZTa FILE THAT IS USED FOR CREATING AN INSTALL
IMAGE FROM ONE OF THE SURVIVING DATABASE SERVERS 4HIS IMAGE MAKER FILE IS SPECIFIC TO THE VERSION AND
PLATFORM OF YOUR %XADATA SYSTEM AND WILL BE NAMED AS FOLLOWS
R^\_dcT8\PVT<PZTaN{exadata_release}N;8=DG G%#N{release_date} {platform} cPa iX_
!N EXTERNAL 53" FLASH DRIVE IS USED TO BOOT THE RECOVERY IMAGE ON THE FAILED SERVER 4HE 53" DRIVE
DOESNT NEED TO BE VERY BIG A n'" THUMB DRIVE CAN BE USED 4HE NEXT STEP IS TO UNZIP THE IMAGE MAKER
FILE YOU DOWNLOADED FROM /RACLE 3UPPORT ON ONE OF THE OTHER %XADATA DATABASE SERVERS IN YOUR RACK !
SIMILAR RECOVERY PROCESSES FOR STORAGE CELLS USES THE FIRST 53" DRIVE FOUND ON THE SYSTEM SO BEFORE
PROCEEDING YOU SHOULD REMOVE ALL OTHER EXTERNAL 53" DEVICES FROM THE SYSTEM 4O CREATE A BOOTABLE
SYSTEM IMAGE FOR RECOVERING THE FAILED DATABASE SERVER YOU WILL RUN THE \PZT8\PVT<TSXP bW SCRIPT IT
PROMPTS YOU FOR SYSTEM
SPECIFIC SETTINGS IT WILL NEED TO APPLY TO THE FAILED SERVER DURING THE REIMAGING
PROCESS 7HEN THE \PZT8\PVT<TSXP bW SCRIPT COMPLETES YOU ARE READY TO INSTALL THE IMAGE ON YOUR FAILED
SERVER 2EMOVE THE 53" DRIVE FROM THE GOOD SERVER AND PLUG IT INTO THE FAILED SERVER ,OGIN TO THE ),/-
ON THE FAILED SERVER AND REBOOT IT 7HEN THE SERVER BOOTS UP IT WILL AUTOMATICALLY FIND THE BOOTABLE
RECOVERY IMAGE ON THE EXTERNAL 53" DRIVE AND BEGIN THE REIMAGING PROCESS &ROM THIS POINT THE PROCESS
IS AUTOMATED &IRST IT WILL CHECK THE FIRMWARE AND ")/3 VERSIONS ON THE SERVER AND UPDATE THEM AS
NEEDED TO MATCH THEM WITH YOUR OTHER DATABASE SERVERS $ONT EXPECT THIS TO DO ANYTHING IF YOU ARE
REIMAGING A SERVER THAT WAS ALREADY PART OF YOUR %XADATA SYSTEM BUT IT IS NECESSARY IF THE DAMAGED SERVER
HAS BEEN REPLACED WITH NEW EQUIPMENT /NCE THE HARDWARE COMPONENTS ARE UP TO DATE A NEW IMAGE
WILL BE INSTALLED 7HEN THE REIMAGING PROCESS IS COMPLETE YOU CAN UNPLUG THE EXTERNAL 53" DRIVE AND
POWER CYCLE THE SERVER TO BOOT UP THE NEW SYSTEM IMAGE
7HEN THE REIMAGING PROCESS IS COMPLETE AND THE DATABASE SERVER IS BACK ONLINE IT WILL BE SET TO
FACTORY DEFAULTS $EPENDING ON THE PATCH LEVEL OF THE INSTALL IMAGE YOU DOWNLOADED FROM /RACLE 3UPPORT
YOU MAY NEED TO REINSTALL SOFTWARE PATCHES ON THE SERVER TO MAKE IT CURRENT WITH THE REST OF YOUR %XADATA
SYSTEM 9OU WILL ALSO NEED TO RECONFIGURE ALL OTHER SITE
SPECIFIC SETTINGS SUCH AS HOST NAME )0 ADDRESSES
.40 SERVER $.3 AND USER AND GROUP ACCOUNTS &OR ALL INTENTS AND PURPOSES YOU SHOULD THINK OF THE
REIMAGED SERVER AS A BRAND
NEW SERVER /RACLE HAS DONE AN EXCELLENT JOB OF IMPROVING THE PRODUCT WITH
EVERY RELEASE 0ERHAPS IN THE NEAR FUTURE WE CAN LOOK FORWARD TO A REIMAGING PROCESS THAT AUTOMATICALLY
CONFIGURES MOST OF THE SITE
SPECIFIC SETTINGS FOR YOU /NCE THE OPERATING SYSTEM IS CONFIGURED YOU WILL
NEED TO REINSTALL THE 'RID )NFRASTRUCTURE AND DATABASE SOFTWARE AND ADD THE NODE BACK INTO THE CLUSTER
4HIS IS A WELL DOCUMENTED PROCESS THAT MANY 2!# $"!S REFER TO AS THE hADD NODEv PROCEDURE )F YOURE
NOT FAMILIAR WITH THE PROCESS LET US REASSURE YOU )TS NOT NEARLY AS DAUNTING OR TIME
CONSUMING AS YOU
MIGHT THINK /NCE YOU HAVE THE OPERATING SYSTEM PREPARED FOR THE INSTALL MUCH OF THE HEAVY LIFTING IS
DONE FOR YOU BY THE /RACLE )NSTALLER 4HE %XADATA /WNERS 'UIDE DOES AN EXCELLENT JOB OF WALKING YOU
THROUGH EACH STEP OF THE PROCESS
Recovering the Storage Cell
3TORAGE CELL RECOVERY IS A VERY BROAD SUBJECT )T CAN BE AS SIMPLE AS REPLACING AN UNDERPERFORMING OR
FAILED DATA DISK AND AS COMPLEX AS RESPONDING TO A TOTAL SYSTEM FAILURE SUCH AS A MALFUNCTIONING CHIP ON
THE MOTHERBOARD )N THIS SECTION WELL BE DISCUSSING VARIOUS TYPES OF CELL RECOVERY INCLUDING REMOVING
AND REPLACING PHYSICAL DISKS FAILED FLASH CACHE MODULES AND WHAT TO DO IF AN ENTIRE STORAGE CELL DIES AND
MUST BE REPLACED
#(!04%2 2%#/6%2).' %8!$!4!
System Volume Failure
2ECALL THAT THE FIRST TWO DISKS IN THE STORAGE CELL CONTAIN THE ,INUX OPERATING SYSTEM AND ARE COMMONLY
REFERRED TO AS THE hSYSTEM VOLUMESv %XADATA PROTECTS THESE VOLUMES USING SOFTWARE MIRRORING THROUGH
THE /3 %VEN SO CERTAIN SITUATIONS MAY REQUIRE YOU TO RECOVER THESE DISKS FROM BACKUP 3OME REASONS
FOR PERFORMING CELL RECOVERY WOULD INCLUDE
v 3YSTEM VOLUMES DISKS AND FAIL SIMULTANEOUSLY
v 4HE BOOT PARTITION IS DAMAGED BEYOND REPAIR
v &ILE SYSTEMS BECOME CORRUPTED
v ! PATCH INSTALLATION OR UPGRADE FAILS
)F YOU FIND YOURSELF IN ANY OF THESE SITUATIONS IT MAY BE NECESSARY OR AT LEAST MORE EXPEDIENT TO
RECOVER THE SYSTEM VOLUMES FROM BACKUP !S DISCUSSED EARLIER %XADATA AUTOMATICALLY MAINTAINS A
BACKUP OF THE LAST GOOD BOOT CONFIGURATION USING A '" INTERNAL 53" FLASH DRIVE CALLED THE #%,,"//4
53" FLASH DRIVE 2ECOVERING THE SYSTEM VOLUMES USING THIS INTERNAL 53" FLASH DISK IS COMMONLY REFERRED
TO AS THE storage cell rescue procedure 4HE STEPS FOR PERFORMING THE CELL RESCUE PROCEDURE BASICALLY
INVOLVE BOOTING FROM THE INTERNAL 53" DRIVE AND FOLLOWING THE PROMPTS FOR THE TYPE OF RESCUE YOU WANT TO
PERFORM "Y THE WAY SINCE %XADATA COMES EQUIPPED WITH AN ),/- )NTEGRATED ,IGHTS /UT -ANAGEMENT
MODULE YOU CAN PERFORM ALL CELL RECOVERY OPERATIONS REMOTELY ACROSS THE NETWORK 4HERE IS NO NEED TO
STAND IN FRONT OF THE RACK TO PERFORM A FULL CELL RECOVERY FROM THE INTERNAL 53" FLASH DISK
!S YOU MIGHT IMAGINE THIS TYPE OF RECOVERY SHOULD NOT BE DONE WITHOUT THE ASSISTANCE OF /RACLE
3UPPORT 4HIS SECTION IS NOT INTENDED TO BE A STEP BY STEP GUIDE TO CELL RECOVERY SO WERE NOT GOING TO GO
INTO ALL THE DETAILS OF CELL RECOVERY FROM THE #%,,"//4 53" FLASH DISK HERE 4HE /RACLE DOCUMENTATION
SHOULD BE USED FOR THAT BUT WE WILL TAKE A LOOK AT WHAT TO CONSIDER BEFORE STARTING SUCH A RECOVERY
Cell Disks and Grid Disks 4HE RESCUE PROCEDURE RESTORES THE ,INUX SYSTEM
VOLUMES ONLY #ELL DISKS AND THEIR CONTENTS ARE NOT RESTORED BY THE RESCUE
PROCEDURE )F THESE PARTITIONS ARE DAMAGED THEY MUST BE DROPPED AND RE
CREATED /NCE THE GRID DISKS ARE ONLINE THEY CAN BE ADDED BACK TO THE !3- DISK
GROUP AND A SUBSEQUENT REBALANCE WILL RESTORE THE DATA
ASM Redundancy: 2ECOVERING A STORAGE CELL FROM 53" BACKUP CAN POTENTIALLY
CAUSE THE LOSS OF ALL DATA ON THE SYSTEM VOLUMES 4HIS INCLUDES YOUR DATABASE
DATA IN THE GRID DISKS ON THESE DISK DRIVES )F YOUR !3- DISK GROUPS USE .ORMAL
REDUNDANCY WE STRONGLY RECOMMEND MAKING A DATABASE BACKUP BEFORE
PERFORMING CELL RECOVERY FROM 53" DISK 7ITH !3- (IGH REDUNDANCY YOU HAVE
TWO MIRROR COPIES OF ALL YOUR DATA SO IT IS SAFE TO PERFORM CELL RECOVERY WITHOUT
TAKING DATABASE BACKUPS %VEN SO WED STILL TAKE A BACKUP IF AT ALL POSSIBLE 4HE
RECOVERY PROCESS DOES NOT DESTROY DATA VOLUMES CELLGRID DISKS UNLESS YOU
EXPLICITLY CHOOSE TO DO SO WHEN PROMPTED BY THE RESCUE PROCEDURE
Software and Patches: !S OF VERSION OF THE STORAGE CELL SOFTWARE THE RESCUE
PROCEDURE WILL RESTORE THE CELL TO ITS FORMER STATE PATCHES INCLUDED WHEN THE
BACKUP WAS TAKEN !LSO INCLUDED IN THE RESTORE ARE THE NETWORK SETTINGS AND
33( KEYS FOR THE ROOT CELLADMIN AND CELLMONITOR ACCOUNTS 4HE ALERT SETTINGS
AND 3-40 CONFIGURATION INCLUDING EMAIL ADDRESSES FOR ALERT NOTIFICATION WILL
NOT BE RESTORED 2ECONFIGURING THESE SETTINGS IS A TRIVIAL TASK AND CAN BE DONE
USING ANOTHER CELL TO DETERMINE THE CORRECT SETTINGS TO APPLY
#(!04%2 2%#/6%2).' %8!$!4!
#ELL 2ESCUE /PTIONS
4HERE ARE TWO MAIN RESCUE OPTIONS Partial Reconstruction AND Full Recovery !S YOU MIGHT GUESS A
PARTIAL RECONSTRUCTION ATTEMPTS TO PRESERVE AS MUCH OF YOUR CURRENT CONFIGURATION AS POSSIBLE WHILE A
FULL RECOVERY WIPES YOUR SYSTEM AND RESETS IT BACK TO THE LAST GOOD CONFIGURATION IT KNOWS ABOUT
Partial Reconstruction: )F THIS OPTION IS CHOSEN THE RESCUE PROCEDURE WILL REPAIR
THE PARTITIONS WITHOUT DAMAGING THE FILE SYSTEMS )T SCANS FOR FILE SYSTEMS ON
THESE PARTITIONS AND IF THEY ARE FOUND WILL TRY TO REBOOT USING THEM )F THE CELL
REBOOTS SUCCESSFULLY YOU CAN USE THE 2T[[2;8 COMMAND TO CHECK THE INTEGRITY
OF THE CELL CONFIGURATION )F THE CELL FAILS TO REBOOT A FULL RECOVERY MUST BE
PERFORMED TO REPAIR THE DAMAGED SYSTEM VOLUMES
Full Recovery: ! FULL RECOVERY DESTROYS AND RE
CREATES ALL SYSTEM PARTITIONS AND
RE
CREATES THE FILE SYSTEMS FROM SCRATCH )T THEN RESTORES THE SYSTEM VOLUMES
FROM BACKUP 4HERE ARE TWO FILES THAT ARE TOO BIG TO FIT ON THE INTERNAL 53" FLASH
DRIVE THE ZTa]T[ STQdVX]U^ AND ZTa]T[ STQdVX]U^ R^\\^] 20-S 4HESE FILES
MUST BE REINSTALLED BEFORE YOU CAN USE CRASH KERNEL SUPPORT )F FULL RECOVERY IS
PERFORMED YOU WILL NEED TO RECONFIGURE THE CELL ALERT AND 3-40 CONFIGURATION
)T IS IMPORTANT SO )LL REPEAT IT #ELL AND GRID DISKS ARE NOT OVERWRITTEN BY THIS
PROCESS 9OU WILL HOWEVER NEED TO RE
IMPORT THE CELL DISKS BEFORE THEY CAN BE
USED
3O WHAT HAPPENS IF FOR SOME REASON THE INTERNAL #%,,"//4 53" FLASH DISK CANNOT BE USED FOR THE
RESCUE PROCEDURE )F THIS HAPPENS YOU CAN BOOT THE STORAGE CELL FROM AN EXTERNAL 53" DRIVE CONTAINING A
BACKUP COPY OF THE #%,,"//4 53" FLASH DISK 4HE PROCESS IS THE SAME AS IT IS FOR PERFORMING CELL RESCUE
USING THE INTERNAL #%,,"//4 53" FLASH DISK
Rolling Back a Failed Patch
%XADATA STORAGE CELLS COME EQUIPPED WITH WHAT /RACLE CALLS A validation framework 4HIS FRAMEWORK IS A
SET OF TESTS THAT ARE RUN AT THE END OF THE BOOT PROCESS 4HE VALIDATION SCRIPT IS CALLED e[Sad] AND IS CALLED
FROM TcR aR [^RP[ AS CAN BE SEEN IN THE FOLLOWING LISTING
-RPc TcR aR S aR >aPR[T 4gPSPcP
...
?TaU^a\eP[XSPcX^]bbcT_
^_c ^aPR[T RT[[^b e[Sad] P[[
...
)N ADDITION TO THE REGULAR VALIDATIONS THAT ARE RUN EVERY TIME THE SYSTEM BOOTS OTHER VALIDATIONS ARE
RUN UNDER CERTAIN CIRCUMSTANCES &OR EXAMPLE AFTER THE CELL IS PATCHED OR UPGRADED OR THE SYSTEM IS
RECOVERED USING THE STORAGE CELL RESCUE PROCEDURE VALIDATIONS ARE RUN TO ENSURE THAT THE CONFIGURATION
AND SOFTWARE ARE ALL INTACT 4HE VALIDATION FRAMEWORK WRITES ITS OUTPUT TO THE
ePa [^V RT[[^b eP[XSPcX^]b DIRECTORY AS YOU CAN SEE IN THE FOLLOWING LISTING
-[b [ ePa [^V RT[[^b eP[XSPcX^]b
c^cP[ '#
af a a^^ca^^c%'$3TR & )# QTVX]UXabcQ^^c [^V
af a a^^ca^^c"('!3TR" !)"&QX^bQ^^c^aSTa [^V
af a a^^ca^^c!$"&3TR & )!%RT[[R^_h\TcPSPcP [^V
af a a^^ca^^c!%'3TR" !)"&RT[[SbcPcdb [^V
#(!04%2 2%#/6%2).' %8!$!4!
af a a^^ca^^c"$$$9P]! ! ) &RWTRZR^]UXVb [^V
af a a^^ca^^c % !3TR" !)"&RWTRZSTeTPRWQ^^c [^V
af a a^^ca^^c!$&3TR & )# RaTPcTRT[[ [^V
af a a^^ca^^c !""3TR" !)"&X_\XbTccX]Vb [^V
af a a^^ca^^c"&''3TR" !)"&\XbRTPRWQ^^c [^V
af a a^^ca^^c "" 3TR & )!%\XbRiTa^Q^^c [^V
af a a^^ca^^c!!'3TR" !)"&^bfPcRWTa [^V
af a a^^ca^^c"##$"3TR & )!%_^bcX]bcP[[ [^V
af a a^^ca^^c# "!3TR" !)"'bPeTR^]UXV [^V
af a a^^ca^^c%$9P]! !)#(b^baT_^ac [^V
af a a^^ca^^c&$3TR & )!%bhbRWTRZ [^V
af a a^^ca^^c& '&"3TR & )##d_VaPSTRQdbQ [^V
!S DISCUSSED IN THE h!CTIVE AND )NACTIVE 3YSTEM 6OLUMESv SECTION OF THIS CHAPTER %XADATA STORAGE
CELLS MAINTAIN TWO SETS OF SYSTEM VOLUMES WHICH CONTAIN THE ,INUX OPERATING SYSTEM AND STORAGE CELL
SOFTWARE "Y MAINTAINING SEPARATE !CTIVE AND )NACTIVE SYSTEM IMAGES %XADATA ENSURES THAT FAILED OUT OF
PARTITION UPGRADES DO NOT CAUSE AN OUTAGE TO THE DATABASES )F THE VALIDATION TESTS DETECT A PROBLEM AFTER
AN OUT OF PARTITION UPGRADE %XADATA WILL AUTOMATICALLY FAIL BACK TO THE LAST GOOD CONFIGURATION BY
SWITCHING THE !CTIVE AND )NACTIVE SYSTEM VOLUMES &OR IN PARTITION PATCHES %XADATA WILL REAPPLY ALL THE
SETTINGS AND FILES CHANGED BY THE PATCH FROM ONLINE BACKUPS &OLLOWING THE FIRST BOOT UP AFTER INSTALLING A
PATCH OR UPGRADE THE VALIDATION RESULTS CAN BE FOUND IN THE LOG FILE
ePa [^V RT[[^b e[Sad] UXabcNQ^^c [^V 6ALIDATION TESTS WILL BE LOGGED TO THE
ePa [^V RT[[^b eP[XSPcX^]b [^V FILE FOR ALL SUBSEQUENT REBOOTS 4HE PATCH ROLLBACK PROCEDURE CAN BE
PERFORMED MANUALLY BUT THERE IS NO MENTION OF IT IN THE DOCUMENTATION SO IT IS PROBABLY NOT SOMETHING
/RACLE EXPECTS ADMINISTRATORS TO RUN WITHOUT THE HELP OF /RACLE 3UPPORT
Cell Disk Failure
!3- HANDLES THE TEMPORARY OR PERMANENT LOSS OF A CELL DISK THROUGH ITS REDUNDANT FAILURE GROUP
TECHNOLOGY 3O THE LOSS OF A CELL DISK SHOULD NOT CAUSE ANY INTERRUPTION TO THE DATABASES AS LONG AS THE
DISK GROUP IS DEFINED WITH .ORMAL REDUNDANCY )F (IGH REDUNDANCY IS USED THE DISK GROUP CAN SUFFER THE
SIMULTANEOUS LOSS OF TWO CELL DISKS WITHIN THE SAME FAILURE GROUP 2ECALL THAT ON %XADATA EACH STORAGE
CELL CONSTITUTES A SEPARATE FAILURE GROUP 4HIS MEANS THAT WITH .ORMAL REDUNDANCY YOU CAN LOSE AN
ENTIRE STORAGE CELL CELL DISKS WITHOUT IMPACT TO YOUR DATABASES 7ITH (IGH REDUNDANCY YOU CAN LOSE
TWO STORAGE CELLS SIMULTANEOUSLY AND YOUR DATABASES WILL CONTINUE TO SERVICE YOUR CLIENTS WITHOUT
INTERRUPTION 4HATS PRETTY IMPRESSIVE 2EDUNDANCY ISNT CHEAP THOUGH &OR EXAMPLE CONSIDER A DISK
GROUP WITH TERABYTES OF USABLE SPACE CONFIGURED FOR %XTERNAL REDUNDANCY 7ITH .ORMAL REDUNDANCY
THAT TERABYTES BECOMES TERABYTES OF USABLE SPACE 7ITH (IGH REDUNDANCY IT BECOMES TERABYTES
OF USABLE STORAGE !LSO KEEP IN MIND THAT THE DATABASE WILL ALWAYS READ THE PRIMARY COPY OF YOUR DATA
UNLESS IT IS UNAVAILABLE .ORMAL AND (IGH REDUNDANCY PROVIDE NO PERFORMANCE BENEFITS 4HEY ARE USED
STRICTLY FOR FAULT TOLERANCE 4HE KEY IS TO CHOOSE A REDUNDANCY LEVEL THAT STRIKES A BALANCE BETWEEN
RESILIENCY AND BUDGET
3IMULATED $ISK &AILURE
)N THIS SECTION WERE GOING TO TEST WHAT HAPPENS WHEN A CELL DISK FAILS 4HE SYSTEM USED FOR THESE TESTS
WAS A QUARTER RACK %XADATA 6 7EVE CREATED A DISK GROUP CALLED B2A0C27N36 DEFINED AS FOLLOWS
#(!04%2 2%#/6%2).' %8!$!4!
BHB)0B<!-2A40C438B:6A>D?B2A0C27N36=>A<0;A43D=30=2H
508;6A>D?24;; 38B:^ (! %' ! " B2A0C27N36N23N $NRT[[
508;6A>D?24;; !38B:^ (! %' ! # B2A0C27N36N23N $NRT[[ !
508;6A>D?24;; "38B:^ (! %' ! $ B2A0C27N36N23N $NRT[[ "
PccaXQdcTR^\_PcXQ[T aSQ\b, !
R^\_PcXQ[T Pb\, !
PdNbXiT,#<
RT[[ b\PacNbRP]NRP_PQ[T,cadT*
.OTICE THAT THIS DISK GROUP IS CREATED USING THREE GRID DISKS &OLLOWING %XADATA BEST PRACTICES WEVE
USED ONE GRID DISK FROM EACH STORAGE CELL )TS INTERESTING TO NOTE THAT EVEN IF WE HADNT SPECIFIED THREE
FAILURE GROUPS WITH ONE DISK IN EACH !3- WOULD HAVE DONE SO AUTOMATICALLY 7E THEN CREATED A SMALL
SINGLE
INSTANCE DATABASE CALLED B2A0C27 USING THIS DISK GROUP 4HE DISK GROUP IS CONFIGURED WITH NORMAL
REDUNDANCY TWO MIRROR COPIES FOR EACH BLOCK OF DATA WHICH MEANS OUR DATABASE SHOULD BE ABLE TO
SUFFER THE LOSS OF ONE CELL DISK WITHOUT LOSING ACCESS TO DATA OR CAUSING A CRASH 3INCE EACH GRID DISK
RESIDES ON A SEPARATE STORAGE CELL WE COULD EVEN SUFFER THE LOSS OF AN ENTIRE STORAGE CELL WITHOUT LOSING
DATA 7ELL DISCUSS WHAT HAPPENS WHEN A STORAGE CELL FAILS LATER IN THE CHAPTER
)N A MOMENT WELL TAKE A LOOK AT WHAT HAPPENS WHEN A GRID DISK IS REMOVED FROM THE STORAGE CELL A
SIMULATED DISK FAILURE "UT BEFORE WE DO THERE ARE A FEW THINGS WE NEED TO CHECK
v 6ERIFY THAT NO REBALANCE OR OTHER VOLUME MANAGEMENT OPERATIONS ARE RUNNING
v %NSURE THAT ALL GRID DISKS FOR THE B2A0C27N36 DISK GROUP ARE ONLINE
v 6ERIFY THAT TAKING A DISK OFFLINE WILL NOT IMPACT DATABASE OPERATIONS
v #HECK THE DISK REPAIR TIMER TO ENSURE THE DISK IS NOT AUTOMATICALLY DROPPED BEFORE
WE CAN BRING IT BACK ONLINE AGAIN
4HERE ARE A COUPLE OF WAYS TO VERIFY THAT VOLUME MANAGEMENT ACTIVITY IS NOT GOING ON &IRST LETS
CHECK THE CURRENT STATE OF THE DISK GROUPS USING Pb\R\S 4HE [b~[ COMMAND SHOWS THE DISK GROUPS THE
TYPE OF REDUNDANCY AND WHETHER OR NOT A REBALANCE OPERATION IS CURRENTLY UNDER WAY "Y THE WAY YOU
COULD ALSO GET THIS INFORMATION USING THE [bSV COMMAND WHICH ALSO INCLUDES OTHER INTERESTING
INFORMATION LIKE SPACE UTILIZATION ONLINEOFFLINE STATUS AND MORE 4HE ATQP[ COLUMN IN THE FOLLOWING
LISTING INDICATES THAT NO REBALANCE OPERATIONS ARE EXECUTING AT THE MOMENT
-Pb\R\S _
0B<2<3JL-[b [
BcPcTCh_TATQP[=P\T
<>D=C43=>A<0;=30C0N36
<>D=C43=>A<0;=A42>N36
<>D=C43=>A<0;=B2A0C27N36
<>D=C43=>A<0;=BC064N36
<>D=C43=>A<0;=BHBC4<N36
.OTICE THAT NOT ALL VOLUME MANAGEMENT OPERATIONS ARE SHOWN IN THE Pb\R\S COMMANDS )F A GRID
DISK HAS BEEN OFFLINE FOR A PERIOD OF TIME THERE MAY BE A CONSIDERABLE AMOUNT OF BACKLOGGED DATA THAT
MUST BE COPIED TO IT IN ORDER TO BRING IT UP TO DATE $EPENDING ON THE VOLUME OF DATA IT MAY TAKE SEVERAL
MINUTES TO FINISH RESYNCHRONIZING A DISK !LTHOUGH THIS OPERATION IS DIRECTLY RELATED TO MAINTAINING
BALANCE ACROSS ALL DISKS IT IS NOT TECHNICALLY A hREBALANCEv OPERATION !S SUCH IT WILL NOT APPEAR IN THE
LISTING &OR EXAMPLE EVEN THOUGH THE [b~[ COMMAND IN THE PREVIOUS LISTING SHOWED A STATUS OF = FOR
REBALANCE OPERATIONS YOU CAN CLEARLY SEE THAT A DISK IS CURRENTLY BEING BROUGHT ONLINE BY RUNNING THE
NEXT QUERY
#(!04%2 2%#/6%2).' %8!$!4!
BHB)0B<!-bT[TRcSV ]P\T^_Ta ^_TaPcX^]^_Ta bcPcT
Ua^\VePb\N^_TaPcX^]^_Ta
VePb\NSXbZVa^d_SV
fWTaTSV Va^d_N]d\QTa,^_Ta Va^d_N]d\QTa
P]SSV X]bcNXS,^_Ta X]bcNXS*
=0<4>?4A0C8>=
B2A0C27N36>=;8=
#HECKING FOR THE ONLINEOFFLINE STATE OF A DISK IS A SIMPLE MATTER OF RUNNING THE FOLLOWING QUERY FROM
31,
0LUS )N THE LISTING BELOW YOU CAN SEE THAT THE B2A0C27N23N $N24;; DISK IS OFFLINE BY ITS
<>D=CNBC0C4 OF <8BB8=6 AND 74034ANBC0CDB OF D=:=>F=
BHB)0B<!-bT[TRcS ]P\TS <>D=CNBC0CDBS 74034ANBC0CDBS BC0C4
Ua^\ePb\NSXbZS
fWTaTS ]P\T[XZTB2A0C27
^aSTaQh *
=0<4<>D=CNB74034ANBC0CDBC0C4
B2A0C27N23N $N24;; <8BB8=6D=:=>F==>A<0;
B2A0C27N23N $N24;; !202743<4<14A=>A<0;
B2A0C27N23N $N24;; "202743<4<14A=>A<0;
3TILL PERHAPS A BETTER WAY OF CHECKING THE STATUS OF ALL DISKS IN THE B2A0C27N36 DISK GROUP WOULD BE TO
CHECK THE \^STNbcPcdb IN E0B<N38B:NBC0C 4HE FOLLOWING LISTING SHOWS THAT ALL GRID DISKS IN THE
B2A0C27N36 DISK GROUP ARE ONLINE
BHB)0B<!-bT[TRc]P\T\^STNbcPcdbUa^\ePb\NSXbZNbcPcfWTaT]P\T[XZTB2A0C27*
=0<4<>34NBC
B2A0C27N23N $N24;; ">=;8=4
B2A0C27N23N $N24;; >=;8=4
B2A0C27N23N $N24;; !>=;8=4
4HE NEXT THING WELL LOOK AT IS THE DISK REPAIR TIMER 2ECALL THAT THE DISK GROUP ATTRIBUTE
SXbZNaT_PXaNcX\T DETERMINES THE AMOUNT OF TIME !3- WILL WAIT BEFORE IT PERMANENTLY REMOVES A DISK
FROM THE DISK GROUP AND REBALANCES THE DATA TO THE SURVIVING GRID DISKS WHEN READWRITE ERRORS OCCUR
"EFORE TAKING A DISK OFFLINE WE SHOULD CHECK TO SEE THAT THIS TIMER IS GOING TO GIVE US ENOUGH TIME TO BRING
THE DISK BACK ONLINE BEFORE !3- AUTOMATICALLY DROPS IT 4HIS ATTRIBUTE CAN BE DISPLAYED USING 31,
0LUS
AND RUNNING THE FOLLOWING QUERY "Y THE WAY THE E0B< VIEWS ARE VISIBLE WHETHER YOU ARE CONNECTED TO
AN !3- INSTANCE OR A DATABASE INSTANCE
BHB)0B<!-bT[TRcSV ]P\T3XbZ6^d_
Pcca ]P\T
Pcca eP[dT
Ua^\ePb\NSXbZVa^d_SV
ePb\NPccaXQdcTPcca
fWTaTSV Va^d_N]d\QTa,Pcca Va^d_N]d\QTa
P]SPcca ]P\T,SXbZNaT_PXaNcX\T*
#(!04%2 2%#/6%2).' %8!$!4!
3XbZ6^d_=0<4E0;D4
30C0N36SXbZNaT_PXaNcX\T&!W
A42>N36SXbZNaT_PXaNcX\T&!W
B2A0C27N36SXbZNaT_PXaNcX\T' $W
BC064N36SXbZNaT_PXaNcX\T&!W
BHBC4<N36SXbZNaT_PXaNcX\T&!W
4HE DEFAULT VALUE FOR THE DISK REPAIR TIMER IS HOURS )N PRODUCTION SYSTEMS YOU WILL WANT TO SET
THE TIMER HIGH ENOUGH FOR YOU TO RESOLVE TRANSIENT DISK ERRORS 4HESE ERRORS OCCUR WHEN A STORAGE CELL IS
REBOOTED OR WHEN A DISK IS TEMPORARILY TAKEN OFFLINE BUT ON RARE OCCASION THEY CAN ALSO OCCUR
SPONTANEOUSLY 3OMETIMES SIMPLY PULLING A DISK OUT OF THE CHASSIS AND REINSERTING IT WILL CLEAR
UNEXPECTED TRANSIENT ERRORS !NY DATA THAT WOULD NORMALLY BE WRITTEN TO THE FAILED DISK WILL QUEUE UP
UNTIL THE DISK IS BROUGHT BACK ONLINE OR THE DISK REPAIR TIME EXPIRES )F !3- DROPS A DISK IT CAN BE
MANUALLY ADDED BACK INTO THE DISK GROUP BUT IT WILL REQUIRE A FULL REBALANCE WHICH CAN BE A LENGTHY
PROCESS 4HE FOLLOWING COMMAND WAS USED TO SET THE DISK REPAIR TIMER TO HOURS FOR THE B2A0C27N36
DISK GROUP
BHB)0B<!-P[cTaSXbZVa^d_B2A0C27N36bTcPccaXQdcTSXbZNaT_PXaNcX\T,' $W*
.OW LETS VERIFY WHETHER TAKING A CELL DISK OFFLINE WILL AFFECT THE AVAILABILITY OF THE DISK GROUP 7E CAN
DO THAT BY CHECKING THE Pb\STPRcXePcX^]^dcR^\T AND Pb\\^STbcPcdb ATTRIBUTES OF OUR GRID DISKS &OR
EXAMPLE THE FOLLOWING LISTING SHOWS THE OUTPUT FROM THE ;8BC6A8338B: COMMAND WHEN A GRID DISK IN A
NORMAL REDUNDANCY DISK GROUP IS TAKEN OFFLINE )N THIS EXAMPLE WE HAVE A B2A0C27N36 DISK GROUP
CONSISTING OF ONE GRID DISK FROM THREE FAILURE GROUPS T]ZRT[ T]ZRT[ ! AND T]ZRT[ " &IRST WELL CHECK
THE STATUS OF THE GRID DISKS WHEN ALL DISKS ARE ACTIVE
JT]ZSQ !)a^^cL a^^c
-SR[X VRT[[NVa^d_ [a^^cbd RT[[PS\X] RKRT[[R[X T[XbcVaXSSXbZK
PccaXQdcTb]P\TPb\STPRcXePcX^]^dcR^\TPb\\^STbcPcdbKkVaT_B2A0C27
T]ZRT[ )B2A0C27N36N23N $NRT[[ HTb>=;8=4
T]ZRT[ !)B2A0C27N36N23N $NRT[[ !HTb>=;8=4
T]ZRT[ ")B2A0C27N36N23N $NRT[[ "HTb>=;8=4
.OW WELL DEACTIVATE ONE OF THESE THE GRID DISKS AT THE STORAGE CELL AND RUN THE COMMAND AGAIN
2T[[2;8-P[cTaVaXSSXbZB2A0C27N36N23N $NRT[[ X]PRcXeT
6aXS3XbZB2A0C27N36N23N $NRT[[ bdRRTbbUd[[hP[cTaTS
JT]ZSQ !)a^^cL a^^c
-SR[X VRT[[NVa^d_ [a^^cbd RT[[PS\X] RKRT[[R[X T[XbcVaXSSXbZK
PccaXQdcTb]P\TPb\STPRcXePcX^]^dcR^\TPb\\^STbcPcdbKkVaT_B2A0C27
T]ZRT[ )B2A0C27N36N23N $NRT[[ HTb>55;8=4
T]ZRT[ !)B2A0C27N36N23N $NRT[[ !2P]]^cST PRcXePcTSdTc^^cWTa^UU[X]TSXbZbX]cWT
SXbZVa^d_>=;8=4
T]ZRT[ ")B2A0C27N36N23N $NRT[[ "2P]]^cST PRcXePcTSdTc^^cWTa^UU[X]TSXbZbX]cWT
SXbZVa^d_>=;8=4
!S YOU CAN SEE THE Pb\\^STbcPcdb ATTRIBUTE OF THE OFFLINED GRID DISK IS NOW SET TO >55;8=4 AND THE
Pb\STPRcXePcX^]^dcR^\T ATTRIBUTE OF THE OTHER TWO DISKS IN THE DISK GROUP WARNS US THAT THESE GRID DISKS
CANNOT BE TAKEN OFFLINE $OING SO WOULD CAUSE !3- TO DISMOUNT THE B2A0C27N36 DISK GROUP DISK GROUP
#(!04%2 2%#/6%2).' %8!$!4!
■ .OTE .OTICE THAT WE USE THE SR[X COMMAND TO RUN THE 2T[[2;8 COMMAND ;8BC6A8338B:0CCA81DC4B ON
EACH CELL IN THE STORAGE GRID "ASICALLY SR[X ALLOWS US TO RUN A COMMAND CONCURRENTLY ON MULTIPLE NODES 4HE
RT[[NVa^d_ PARAMETER IS A FILE CONTAINING A LIST OF ALL OF OUR STORAGE CELLS
)F THE OUTPUT FROM THE ;8BC6A8338B: COMMAND INDICATES IT IS SAFE TO DO SO WE CAN TEST WHAT
HAPPENS WHEN WE TAKE ONE OF THE GRID DISKS FOR OUR B2A0C27N36 DISK GROUP OFFLINE &OR THIS TEST WE WILL
PHYSICALLY REMOVE THE DISK DRIVE FROM THE STORAGE CELL CHASSIS 4HE TEST CONFIGURATION WILL BE AS FOLLOWS
v &OR THIS TEST WE WILL CREATE A NEW TABLESPACE WITH ONE DATAFILE 4HE DATAFILE IS SET TO
Pdc^TgcT]S SO IT WILL GROW INTO THE DISK GROUP AS DATA IS LOADED
v .EXT WELL GENERATE A CONSIDERABLE AMOUNT OF DATA IN THE TABLESPACE BY CREATING A
LARGE TABLE A COUPLE OF BILLION ROWS FROM 310NB46<4=CB SHOULD DO IT
v 7HILE DATA IS BEING LOADED INTO THE LARGE TABLE WE WILL PHYSICALLY REMOVE THE DISK
FROM THE CELL CHASSIS
v /NCE THE DATA IS FINISHED LOADING WE WILL REINSTALL THE DISK AND OBSERVE %XADATAS
AUTOMATED DISK RECOVERY IN ACTION
4HE FIRST ORDER OF BUSINESS IS TO IDENTIFY THE LOCATION OF THE DISK DRIVE WITHIN THE STORAGE CELL 4O DO
THIS WE WILL USE THE GRID DISK NAME TO FIND THE CELL DISK IT RESIDES ON THEN WELL USE THE CELL DISK NAME TO
FIND THE SLOT ADDRESS OF THE DISK DRIVE WITHIN THE STORAGE CELL /NCE WE HAVE THE SLOT ADDRESS WE WILL TURN
ON THE SERVICE ,%$ ON THE FRONT PANEL SO WE KNOW WHICH DISK TO REMOVE
&ROM STORAGE CELL WE CAN USE THE ;8BC6A8338B: COMMAND TO FIND THE NAME OF THE CELL DISK WE ARE
LOOKING FOR
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TRT[[SXbZfWTaT]P\T[XZTB2A0C27 STcPX[
]P\T)B2A0C27N36N23N $NRT[[ "
RT[[3XbZ)23N $NRT[[ "
.OW THAT WE HAVE THE CELL DISK NAME WE CAN USE THE ;8BC;D= COMMAND TO FIND THE SLOT ADDRESS OF
THE PHYSICAL DISK WE WANT TO REMOVE )N THE FOLLOWING LISTING WE SEE THE SLOT ADDRESS WERE LOOKING FOR
2T[[2;8-[Xbc;D=PccaXQdcTbRT[[SXbZ_WhbXRP[SaXeTbfWTaTRT[[SXbZ,23N $NRT[[ "STcPX[
RT[[3XbZ)23N $NRT[[ "
_WhbXRP[3aXeTb) %)$
7ITH THE SLOT ADDRESS WE CAN USE THE <TVP2[X%# COMMAND TO ACTIVATE THE DRIVES SERVICE ,%$ ON THE
FRONT PANEL OF THE STORAGE CELL .OTE THAT THE K CHARACTERS IN THE <TVP2[X%# COMMAND BELOW ARE USED TO
PREVENT THE "ASH SHELL FROM INTERPRETING THE BRACKETS ;= AROUND THE PHYSICAL DRIVE ADDRESS 3INGLE
QUOTES WORK AS WELL BY THE WAY
^_c <TVPA083 <TVP2[X <TVP2[X%# _S[^RPcT _WhbSaeKJ %)$KL P
4HE AMBER ,%$ ON THE FRONT OF THE DISK DRIVE SHOULD BE FLASHING AS CAN BE SEEN IN &IGURE
#(!04%2 2%#/6%2).' %8!$!4!
Amber LED indicator flashes to
help you locate the right disk
Figure 9-2. Disk drive front panel
!ND IN CASE YOU WERE WONDERING THE SERVICE ,%$ CAN BE TURNED OFF AGAIN USING THE bc^_ OPTION OF
THE <TVP2[X%# COMMAND LIKE THIS
^_c <TVPA083 <TVP2[X <TVP2[X%# _S[^RPcT–stop _WhbSaeKJ %)$KL P
.OW THAT WEVE LOCATED THE RIGHT DISK WE CAN REMOVE IT FROM THE STORAGE CELL BY PRESSING THE RELEASE
BUTTON AND GENTLY PULLING THE LEVER ON THE FRONT OF THE DISK AS YOU CAN SEE IN &IGURE
Figure 9-3. Ejected disk drive
■ .OTE !LL DISK DRIVES IN THE STORAGE CELL ARE HOT
PLUGGABLE AND MAY BE REPLACED WITHOUT POWERING DOWN THE
STORAGE CELL
#HECKING THE GRID DISK STATUS IN 2T[[2;8 WE SEE THAT IT HAS BEEN CHANGED FROM !CTIVE TO )NACTIVE
4HIS MAKES THE GRID DISK UNAVAILABLE TO THE !3- STORAGE CLUSTER
#(!04%2 2%#/6%2).' %8!$!4!
2T[[2;8-[XbcVaXSSXbZfWTaT]P\T,B2A0C27N23N $NRT[[ "*
B2A0C27N23N $NRT[[ "X]PRcXeT
!3- IMMEDIATELY NOTICES THE LOSS OF THE DISK TAKES IT OFFLINE AND STARTS THE DISK REPAIR TIMER 4HE
!3- ALERT LOG P[TacN0B<! [^V SHOWS THAT WE HAVE ABOUT HOURS TO BRING THE DISK BACK
ONLINE BEFORE !3- PERMANENTLY DROPS IT FROM THE DISK GROUP
P[TacN0B< [^V
CdT3TR!' ')# )$#!
6<>=RWTRZX]VSXbZ\^STbU^aVa^d_$Pc ! U^a_XS$!^bXS!(!(!
4aa^abX]UX[T d P__ ^aPR[T SXPV Pb\ Pb\ 0B<! caPRT 0B<!NV\^]N$( ! caR)
>A0 !&% ")2T[[bc^aPVT8 >Taa^a8 >UPX[TS^]SXbZ^ (! %' ! $ B2A0C27N23N $NRT[[ "Pc
^UUbTc# ('# U^aSPcP[T]VcW# (%
>A0 !&%!%)4gPSPcPTaa^a)! 6T]TaXR8 >Taa^a
F0A=8=6)FaXcT5PX[TS Va^d_)$SXbZ)"0D) ^UUbTc)# (%bXiT)# (%
...
F0A=8=6)3XbZB2A0C27N36N23N $N24;; "X]\^ST g&UXb]^fQTX]V^UU[X]TS
F0A=8=6)3XbZB2A0C27N36N23N $N24;; "X]\^ST g&UXb]^fQTX]VcPZT]^UU[X]T
...
CdT3TR!' ')#")! !
F0A=8=6)3XbZB2A0C27N36N23N $N24;; "will be dropped in: (30596)bTRb^]0B<X]bc)!
CdT3TR!' ')#")!"!
4HE STATUS OF THE DISK IN !3- CAN BE SEEN USING THE FOLLOWING QUERY FROM ONE OF THE !3- INSTANCES
.OTICE THAT THE B2A0C27 DISK GROUP IS STILL MOUNTED ONLINE
BHB)0B<!-bT[TRcSV ]P\TS ]P\TSV bcPcTS \^d]cNbcPcdbS WTPSTaNbcPcdbS bcPcT
Ua^\ePb\NSXbZS
ePb\NSXbZVa^d_SV
fWTaTSV ]P\T,B2A0C27N36
P]SSV Va^d_N]d\QTa,S Va^d_N]d\QTa
^aSTaQh !*
=0<4=0<4BC0C4<>D=CNB74034ANBC0CDBC0C4
B2A0C27B2A0C27N36N23N $N24;; <>D=C43202743<4<14A=>A<0;
B2A0C27B2A0C27N36N23N $N24;; !<>D=C43202743<4<14A=>A<0;
B2A0C27B2A0C27N36N23N $N24;; "<>D=C43<8BB8=6D=:=>F==>A<0;
7HILE THE DISK IS OFFLINE !3- CONTINUES TO POLL ITS STATUS TO SEE IF THE DISK IS AVAILABLE 7E SEE THE
FOLLOWING QUERY REPEATING IN THE !3- ALERT LOG
P[TacN0B< [^V
F0A=8=6)4gPSPcP0dc^<P]PVT\T]c)>B?83)$( '>_TaPcX^]83)" $)X]SXbZVa^d_5PX[TS
B@;) 4gPSPcP0dc^<V\c)BT[TRcSXbZbX]36cWPcPaT]^c>=;8=4
bT[TRc]P\TUa^\ePb\NSXbZNbcPc
fWTaT
\^STNbcPcdb,>55;8=4
P]S
Va^d_N]d\QTaX]
bT[TRcVa^d_N]d\QTaUa^\ePb\NSXbZVa^d_NbcPc
#(!04%2 2%#/6%2).' %8!$!4!
fWTaT
]P\T,B2A0C27N36
P]S
bcPcT,<>D=C43
/UR TEST DATABASE ALSO DETECTED THE LOSS OF THE GRID DISK AS CAN BE SEEN IN THE DATABASE ALERT LOG
P[TacNB2A0C27 [^V
CdT3TR!' ')# )$#!
4aa^abX]UX[T d P__ ^aPR[T SXPV aSQ\b bRaPcRW B2A0C27 caPRT B2A0C27NRZ_cN!!$!( caR)
>A0 !&% ")2T[[bc^aPVT8 >Taa^a8 >UPX[TS^]SXbZ^ (! %' ! $ B2A0C27N23N $NRT[[ "Pc
^UUbTc!%"% ! & !#U^aSPcP[T]VcW %"'#
>A0 !&%!%)4gPSPcPTaa^a)! 6T]TaXR8 >Taa^a
F0A=8=6)ATPS5PX[TS Va^d_)$SXbZ)"0D)%!'$^UUbTc) %"'#bXiT) %"'#
F0A=8=6)UPX[TSc^aTPS\Xaa^abXST ^UeXacdP[TgcT]c [^VXRP[TgcT]c ^UUX[T!% X]
Va^d_J$ % '#&#"&LUa^\SXbZB2A0C27N23N $N24;; "P[[^RPcX^]d]Xc%!'$aTPb^]Taa^a*XU
_^bbXQ[TfX[[cahP]^cWTa\Xaa^abXST
=>C4)bdRRTbbUd[[haTPS\Xaa^abXST!^UeXacdP[TgcT]c [^VXRP[TgcT]c ^UUX[T!% X]
Va^d_J$ % '#&#"&LUa^\SXbZB2A0C27N23N $N24;; !P[[^RPcX^]d]Xc!!#
...
CdT3TR!' ')# )$#!
=>C4)SXbZ"B2A0C27N23N $N24;; "X]Va^d_$B2A0C27Xb^UU[X]TU^aaTPSb
=>C4)SXbZ"B2A0C27N23N $N24;; "X]Va^d_$B2A0C27Xb^UU[X]TU^afaXcTb
.OTICE THAT THE DATABASE AUTOMATICALLY SWITCHES TO THE MIRROR COPY FOR DATA IT CAN NO LONGER READ
FROM THE FAILED GRID DISK 4HIS IS !3- NORMAL REDUNDANCY IN ACTION
7HEN WE REINSERT THE DISK DRIVE THE STORAGE CELL RETURNS THE GRID DISK TO A STATE OF !CTIVE AND !3-
BRINGS THE DISK BACK ONLINE AGAIN 7E CAN SEE THAT THE GRID DISK HAS RETURNED TO A STATE OF 202743 AND A
74034ANBC0CDB OF =>A<0; IN THE FOLLOWING QUERY
BHB)0B<!-bT[TRcSV ]P\TS ]P\TSV bcPcTS \^d]cNbcPcdbS WTPSTaNbcPcdbS bcPcT
Ua^\ePb\NSXbZS
ePb\NSXbZVa^d_SV
fWTaTSV ]P\T,B2A0C27
P]SSV Va^d_N]d\QTa,S Va^d_N]d\QTa
^aSTaQh !*
=0<4=0<4BC0C4<>D=CNB74034ANBC0CDBC0C4
B2A0C27B2A0C27N23N $N24;; <>D=C43202743<4<14A=>A<0;
B2A0C27B2A0C27N23N $N24;; !<>D=C43202743<4<14A=>A<0;
B2A0C27B2A0C27N23N $N24;; "<>D=C43202743<4<14A=>A<0;Å
)T IS LIKELY THAT THE DISK GROUP WILL NEED TO CATCH UP ON WRITING DATA THAT QUEUED UP WHILE THE DISK WAS
OFFLINE )F THE DURATION OF THE OUTAGE WAS SHORT AND THE WRITE ACTIVITY WAS LIGHT THEN THE REBALANCE WONT
TAKE LONG )T MAY EVEN GO UNNOTICED "UT IF THE DISK HAS BEEN OFFLINE FOR SEVERAL HOURS OR DAYS OR IF THERE
WAS A LOT OF WRITE ACTIVITY IN THE DISK GROUP DURING THE OUTAGE THIS COULD TAKE A WHILE 'ENERALLY SPEAKING
THE DELAY IS NOT A PROBLEM BECAUSE IT ALL HAPPENS IN THE BACKGROUND $URING THE RESILVERING PROCESS
!3- REDUNDANCY ALLOWS OUR DATABASES TO CONTINUE WITH NO INTERRUPTION TO SERVICE
)F THIS HAD BEEN AN ACTUAL DISK FAILURE AND WE ACTUALLY REPLACED THE DISK DRIVE WE WOULD NEED TO WAIT
FOR THE 2!)$ CONTROLLER TO ACKNOWLEDGE THE NEW DISK BEFORE IT COULD BE USED 4HIS DOESNT TAKE LONG BUT
#(!04%2 2%#/6%2).' %8!$!4!
YOU SHOULD CHECK THE STATUS OF THE DISK TO ENSURE THAT ITS STATUS IS .ORMAL BEFORE USING IT 4HE DISK STATUS
MAY BE VERIFIED USING THE 2T[[2;8 COMMAND ;8BC?7HB820;38B: AS SHOWN HERE
2T[[2;8-[Xbc_WhbXRP[SXbZfWTaTSXbZCh_T,7PaS3XbZ0=3bcPcdb,RaXcXRP[STcPX[
7HEN A DISK IS REPLACED THE STORAGE CELL PERFORMS THE FOLLOWING TASKS AUTOMATICALLY
v 4HE DISK FIRMWARE IS UPDATED TO MATCH THE OTHER DISK DRIVES IN THE STORAGE CELL
v 4HE CELL DISK IS RE
CREATED TO MATCH THAT OF THE DISK IT REPLACED
v 4HE REPLACEMENT CELL DISK IS BROUGHT ONLINE STATUS SET TO .ORMAL
v 4HE GRID DISK OR GRID DISKS THAT WERE ON THE FAILED DISK ARE RE
CREATED
v 4HE GRID DISK STATUS IS SET TO !CTIVE
/NCE THE REPLACEMENT GRID DISKS ARE SET TO !CTIVE !3- AUTOMATICALLY OPENS THE DISK AND BEGINS THE
RESILVERING PROCESS !LL THESE TASKS ARE HANDLED AUTOMATICALLY BY %XADATA MAKING DISK REPLACEMENT A
FAIRLY PAINLESS PROCESS
7HEN TO 2EPLACE A #ELL $ISK
$ISK FAILURE CAN OCCUR ABRUPTLY CAUSING THE DISK TO GO OFFLINE IMMEDIATELY OR IT CAN OCCUR GRADUALLY
MANIFESTING POOR )/ PERFORMANCE 3TORAGE CELLS ARE CONSTANTLY MONITORING THE DISK DRIVES 4HIS
MONITORING INCLUDES DRIVE PERFORMANCE IN TERMS OF BOTH )/ AND THROUGHPUT AND 3-!24 METRICS SUCH
AS TEMPERATURE SPEED AND READWRITE ERRORS 4HE GOAL IS TO PROVIDE EARLY WARNING FOR DISKS THAT ARE LIKELY
TO FAIL BEFORE THEY ACTUALLY DO 7HEN %XADATA DETECTS A PROBLEM AN ALERT IS GENERATED WITH SPECIFIC
INSTRUCTIONS ON HOW TO REPLACE THE DISK )F THE SYSTEM HAS BEEN CONFIGURED FOR EMAIL NOTIFICATION THESE
ALERTS WILL BE EMAILED TO YOU AUTOMATICALLY
)N THE PREVIOUS SECTION WE WALKED THROUGH A SIMULATED DRIVE FAILURE (AD THIS BEEN AN ACTUAL DISK
FAILURE THE PROCEDURE FOR REPLACING THE DISK WOULD FOLLOW THE SAME STEPS WE USED FOR THE SIMULATION "UT
WHAT HAPPENS WHEN %XADATAS EARLY WARNING SYSTEM DETERMINES THAT A DRIVE IS LIKELY TO FAIL SOON 7HEN
%XADATA DETECTS DRIVE PROBLEMS IT SETS THE PHYSICAL DISK STATUS ATTRIBUTE ACCORDINGLY 4HE FOLLOWING
2T[[2;8 COMMAND DISPLAYS THE STATUS OF ALL DISKS IN THE STORAGE CELL
2T[[2;8-[Xbc_WhbXRP[SXbZPccaXQdcTb]P\TbcPcdbfWTaTSXbZch_T,7PaS3XbZ
"$) ]^a\P[
"$) ]^a\P[
...
"$) ]^a\P[
4ABLE
SHOWS THE VARIOUS DISK STATUS VALUES AND WHAT THEY MEAN
#(!04%2 2%#/6%2).' %8!$!4!
Table 9-3. Disk Status Definitions
Status Description
.ORMAL 4HE DRIVE IS HEALTHY
0REDICTIVE &AILURE 4HE DISK IS STILL WORKING BUT LIKELY TO FAIL SOON AND SHOULD BE REPLACED AS SOON AS
POSSIBLE
0OOR 0ERFORMANCE 4HE DISK IS EXHIBITING EXTREMELY POOR PERFORMANCE AND SHOULD BE REPLACED
0REDICTIVE &AILURE
)F A DISK STATUS SHOWS 0REDICTIVE &AILURE !3- WILL AUTOMATICALLY DROP THE GRID DISKS FROM THE DRIVE AND
REBALANCE DATA TO OTHER DISKS IN THE DISK GROUP ACCORDING TO THE REDUNDANCY POLICY OF THE AFFECTED DISK
GROUPS THAT USE THE DRIVE /NCE !3- HAS FINISHED REBALANCING AND COMPLETED THE DROP OPERATION YOU
CAN REPLACE THE DISK DRIVE 4HE FOLLOWING LISTING CAN BE USED TO TRACK THE STATUS OF THE !3- DISK ! STATUS
OF /FFLINE INDICATES THAT !3- HAS NOT YET FINISHED REBALANCING THE DISK GROUP /NCE THE REBALANCE IS
COMPLETE THE DISK WILL NO LONGER APPEAR IN THE LISTING "Y THE WAY TAILING THE !3- ALERT LOG IS ALSO AN
EXCELLENT WAY OF CHECKING THE PROGRESS OF THE DROP
BHB)0B<!-bT[TRc]P\T\^STNbcPcdb
Ua^\ePb\NSXbZNbcPc
fWTaT]P\T[XZTB2A0C27
^aSTaQh *
=0<4<>34NBC
B2A0C27N23N $N24;; >=;8=4
B2A0C27N23N $N24;; !>=;8=4
B2A0C27N23N $N24;; ">55;8=4
■ #AUTION 4HE FIRST TWO PHYSICAL DISKS IN THE STORAGE CELL ALSO CONTAIN THE ,INUX OPERATING SYSTEM 4HE /3
PARTITIONS ON THESE TWO DISKS ARE CONFIGURED AS MIRRORS OF ONE ANOTHER )F ONE OF THESE DISKS FAILS THE DATA MUST BE
IN SYNC WITH THE MIRROR DISK BEFORE YOU REMOVE IT 5SE THE #ELL#,) COMMAND P[cTaRT[[eP[XSPcT
R^]UXVdaPcX^] TO VERIFY THAT NO \PS\ ERRORS EXIST BEFORE REPLACING THE DISK
4HE 2T[[2;8 COMMAND E0;830C42>=586DA0C8>= PERFORMS THIS VERIFICATION FOR YOU
2T[[2;8-0;C4A24;;E0;830C42>=586DA0C8>=
2T[[T]ZRT[ bdRRTbbUd[[hP[cTaTS
#(!04%2 2%#/6%2).' %8!$!4!
0OOR 0ERFORMANCE
)F A DISK EXHIBITS POOR PERFORMANCE IT SHOULD BE REPLACED ! SINGLE POORLY PERFORMING CELL DISK CAN
IMPACT THE PERFORMANCE OF OTHER HEALTHY DISKS 7HEN A DISK BEGINS PERFORMING EXTREMELY BADLY ITS
STATUS WILL BE SET TO 0OOR 0ERFORMANCE !S IS THE CASE WITH 0REDICTIVE &AILURE STATUS !3- WILL
AUTOMATICALLY DROP ALL GRID DISKS ON THIS CELL DISK FROM THE DISK GROUPS AND BEGIN A REBALANCE OPERATION
/NCE THE REBALANCE IS COMPLETE YOU CAN REMOVE AND REPLACE THE FAILING DISK DRIVE 9OU CAN USE THE
2T[[2;8 COMMAND 20;81A0C4 TO MANUALLY CHECK THE PERFORMANCE OF ALL DISKS IN THE STORAGE CELL 4HE
FOLLOWING LISTING SHOWS THE OUTPUT OF THE 20;81A0C4 COMMAND /RDINARILY RT[[bae SHOULD BE SHUT DOWN
BEFORE RUNNING 20;81A0C4 BECAUSE IT CAN SIGNIFICANTLY IMPACT )/ PERFORMANCE FOR DATABASES USING THE
STORAGE CELL )F YOU CANNOT SHUT DOWN RT[[bae FOR THE TEST YOU CAN RUN 20;81A0C4 USING THE 5>A24 OPTION
!S DAUNTING AS THAT SOUNDS 5>A24 SIMPLY OVERRIDES THE SAFETY SWITCH AND ALLOWS YOU TO RUN 20;81A0C4
WHILE RT[[bae IS UP AND APPLICATIONS ARE USING THE CELL DISKS 4HE FOLLOWING LISTING SHOWS THE OUTPUT FROM
THE 20;81A0C4 COMMAND RUN ON A HEALTHY SET OF CELL DISKS
2T[[2;8-RP[XQaPcT
2P[XQaPcX^]fX[[cPZTPUTf\X]dcTb
0VVaTVPcTaP]S^\aTPScWa^dVW_dcPRa^bbP[[WPaSSXbZ[d]b) !$<1?B
0VVaTVPcTaP]S^\aTPScWa^dVW_dcPRa^bbP[[U[PbWSXbZ[d]b)"'(( '<1?B
0VVaTVPcTaP]S^\aTPS8>b_TabTR^]S8>?BPRa^bbP[[WPaSSXbZ[d]b) & &
0VVaTVPcTaP]S^\aTPS8>b_TabTR^]S8>?BPRa^bbP[[U[PbWSXbZ[d]b) #$ $'
2^]ca^[[TaaTPScWa^dVW_dc) !#$ '<1?B
2P[XQaPcX]VWPaSSXbZbaTPS^][h
;d] N ^]SaXeTJ"$) LaP]S^\aTPScWa^dVW_dc)'" ' <1?BP]S ##8>?B
;d] N ^]SaXeTJ"$) LaP]S^\aTPScWa^dVW_dc)'& '$<1?BP]S #$8>?B
;d] N ^]SaXeTJ"$) LaP]S^\aTPScWa^dVW_dc)( !"<1?BP]S #$8>?B
;d] N ^]SaXeTJ"$) LaP]S^\aTPScWa^dVW_dc)( "!<1?BP]S $ 8>?B
;d] N!^]SaXeTJ"$)!LaP]S^\aTPScWa^dVW_dc)(! &<1?BP]S #&8>?B
;d] N"^]SaXeTJ"$)"LaP]S^\aTPScWa^dVW_dc)( $(<1?BP]S $ 8>?B
;d] N#^]SaXeTJ"$)#LaP]S^\aTPScWa^dVW_dc)(" & <1?BP]S #(8>?B
;d] N$^]SaXeTJ"$)$LaP]S^\aTPScWa^dVW_dc)'( '%<1?BP]S #&8>?B
;d] N%^]SaXeTJ"$)%LaP]S^\aTPScWa^dVW_dc)( %$<1?BP]S #'8>?B
;d] N&^]SaXeTJ"$)&LaP]S^\aTPScWa^dVW_dc)'' %<1?BP]S #$8>?B
;d] N'^]SaXeTJ"$)'LaP]S^\aTPScWa^dVW_dc)( <1?BP]S #(8>?B
;d] N(^]SaXeTJ"$)(LaP]S^\aTPScWa^dVW_dc)( ' <1?BP]S #(8>?B
2P[XQaPcX]VU[PbWSXbZbaTPS^][h]^cTcWPcfaXcTbfX[[QTbXV]XUXRP]c[hb[^fTa
;d] N ^]SaXeTJJ!) ) ) LLaP]S^\aTPScWa^dVW_dc)!#! $ <1?BP]S ( "$8>?B
;d] N ^]SaXeTJJ!) ) ) LLaP]S^\aTPScWa^dVW_dc)!# '(<1?BP]S ( '8>?B
;d] N!^]SaXeTJJ!) )!) LLaP]S^\aTPScWa^dVW_dc)!# ( <1?BP]S ( 8>?B
;d] N"^]SaXeTJJ!) )") LLaP]S^\aTPScWa^dVW_dc)!# %!<1?BP]S '(!"8>?B
;d]!N ^]SaXeTJJ#) ) ) LLaP]S^\aTPScWa^dVW_dc)!#! '<1?BP]S ( %%8>?B
;d]!N ^]SaXeTJJ#) ) ) LLaP]S^\aTPScWa^dVW_dc)!$ <1?BP]S (!%"8>?B
;d]!N!^]SaXeTJJ#) )!) LLaP]S^\aTPScWa^dVW_dc)!#$ %$<1?BP]S ( (8>?B
;d]!N"^]SaXeTJJ#) )") LLaP]S^\aTPScWa^dVW_dc)!#& '"<1?BP]S ( "(8>?B
;d]#N ^]SaXeTJJ ) ) ) LLaP]S^\aTPScWa^dVW_dc)!$ "%<1?BP]S ( "%8>?B
;d]#N ^]SaXeTJJ ) ) ) LLaP]S^\aTPScWa^dVW_dc)!#( '!<1?BP]S ( %8>?B
;d]#N!^]SaXeTJJ ) )!) LLaP]S^\aTPScWa^dVW_dc)!$! '!<1?BP]S ( # 8>?B
;d]#N"^]SaXeTJJ ) )") LLaP]S^\aTPScWa^dVW_dc)!$ ' <1?BP]S '(&(8>?B
;d]$N ^]SaXeTJJ") ) ) LLaP]S^\aTPScWa^dVW_dc)!$" ! <1?BP]S ( %&8>?B
;d]$N ^]SaXeTJJ") ) ) LLaP]S^\aTPScWa^dVW_dc)!$! <1?BP]S ( $!8>?B
;d]$N!^]SaXeTJJ") )!) LLaP]S^\aTPScWa^dVW_dc)!$ '%<1?BP]S ( % 8>?B
;d]$N"^]SaXeTJJ") )") LLaP]S^\aTPScWa^dVW_dc)!## !#<1?BP]S '(!$8>?B
#(!04%2 2%#/6%2).' %8!$!4!
20;81A0C4aTbd[cbPaTfXcWX]P]PRRT_cPQ[TaP]VT
20;81A0C4bcaTbbcTbcXb]^fad]]X]V
2P[XQaPcX^]WPbUX]XbWTS
Cell Flash Cache Failure
%XADATA STORAGE CELLS COME EQUIPPED WITH FOUR & 0#)E FLASH CACHE CARDS %ACH CARD HAS FOUR FLASH
CACHE DISKS &$/-S FOR A TOTAL OF FLASH DISKS 4HESE FLASH CACHE CARDS OCCUPY SLOTS AND INSIDE
THE STORAGE CELL )F A FLASH CACHE MODULE FAILS PERFORMANCE OF THE STORAGE CELL WILL BE DEGRADED AND IT
SHOULD BE REPLACED AT YOUR EARLIEST OPPORTUNITY )F YOU ARE USING SOME OF YOUR FLASH CACHE FOR FLASH DISK
BASED GRID DISKS YOUR DISK GROUP REDUNDANCY WILL BE AFFECTED AS WELL 4HESE FLASH CACHE CARDS ARE NOT
HOT PLUGGABLE SO REPLACING THEM WILL REQUIRE YOU TO POWER OFF THE AFFECTED CELL
)F A FLASH DISK FAILS %XADATA WILL SEND YOU AN EMAIL NOTIFYING YOU OF THE FAILURE 4HE EMAIL WILL INCLUDE
THE SLOT ADDRESS OF THE CARD AND IF A SPECIFIC &$/- HAS FAILED IT WILL INCLUDE THE ADDRESS OF THE &$/- ON
THE CARD OR 4HE FAILED FLASH CACHE CARD CAN BE SEEN USING THE 2T[[2;8 COMMAND ;8BC
?7HB820;38B: AS FOLLOWS
2T[[2;8-[Xbc_WhbXRP[SXbZfWTaTSXbZch_T,U[PbWSXbZP]SbcPcdb,]^a\P[
]P\T)J#) )") L
SXbZCh_T)5[PbW3XbZ
[d]b)!N"
\PZT<^ST[)<0AE4;;B3''B0 !
_WhbXRP[5Xa\fPaT)3! H
_WhbXRP[8]bTacCX\T)! & !'C! ) ()#" $)
_WhbXRP[8]cTaUPRT)bPb
_WhbXRP[BTaXP[)$ ' ! R&S% 5<>3"
_WhbXRP[BXiT)!! ''' % $!"#"&$6
b[^c=d\QTa)?28B[^c)!*53><)"
bcPcdb)critical
4HE b[^c=d\QTa ATTRIBUTE HERE SHOWS YOU WHERE THE CARD AND &$/- ARE INSTALLED )N OUR CASE THE
CARD IS INSTALLED IN 0#)E SLOT /NCE YOU HAVE THIS INFORMATION YOU CAN SHUT DOWN AND POWER OFF THE
STORAGE CELL AND REPLACE THE DEFECTIVE PART +EEP IN MIND THAT WHEN THE CELL IS OFFLINE !3- WILL NO LONGER
HAVE ACCESS TO THE GRID DISKS 3O BEFORE YOU SHUT DOWN THE CELL MAKE SURE THAT SHUTTING IT DOWN WILL NOT
IMPACT THE AVAILABILITY OF THE DISK GROUPS IT SUPPORTS 4HIS IS THE SAME PROCEDURE WE DESCRIBED IN THE
h#ELL $ISK &AILUREv SECTION OF THIS CHAPTER /NCE THE PART IS REPLACED AND THE CELL REBOOTS THE STORAGE CELL
WILL AUTOMATICALLY CONFIGURE THE CELL DISK ON THE REPLACEMENT CARD AND IF IT WAS USED FOR FLASH CACHE YOU
WILL SEE YOUR FLASH CACHE RETURN TO ITS FORMER SIZE
Cell Failure
4HERE ARE TWO MAIN TYPES OF CELL FAILURE TEMPORARY AND PERMANENT 4EMPORARY CELL FAILURES CAN BE AS
HARMLESS AS A CELL REBOOT OR A POWER FAILURE %XTENDED CELL FAILURES CAN ALSO BE TEMPORARY IN NATURE &OR
EXAMPLE IF A PATCH INSTALLATION FAILS OR A COMPONENT MUST BE REPLACED IT COULD TAKE THE CELL OFFLINE FOR
HOURS OR EVEN DAYS 0ERMANENT CELL FAILURES ARE MORE SEVERE IN NATURE AND REQUIRE THE ENTIRE CELL CHASSIS
TO BE REPLACED )N EITHER CASE IF YOUR SYSTEM IS CONFIGURED PROPERLY THERE WILL BE NO INTERRUPTION TO !3-
OR YOUR DATABASES )N THIS SECTION WELL TAKE A LOOK AT WHAT HAPPENS WHEN A CELL IS TEMPORARILY OFFLINE AND
WHAT TO DO IF YOU EVER HAVE TO REPLACE ONE
#(!04%2 2%#/6%2).' %8!$!4!
4EMPORARY #ELL &AILURE
!S DISCUSSED IN #HAPTER %XADATA STORAGE CELLS ARE 3UN SERVERS WITH INTERNAL DISK DRIVES RUNNING /RACLE
%NTERPRISE ,INUX )F A STORAGE CELL GOES OFFLINE ALL THE DISKS ON THAT CELL BECOME UNAVAILABLE TO THE
DATABASE SERVERS 4HIS MEANS THAT ALL DISK GROUPS CONTAINING DATABASE DATA AS WELL AS /#2 AND 6OTING
FILES ON THAT STORAGE CELL ARE OFFLINE FOR THE DURATION OF THE OUTAGE !3- FAILURE GROUPS PROVIDE
REDUNDANCY THAT ALLOWS YOUR CLUSTER AND DATABASES TO CONTINUE TO RUN DURING THE OUTAGE ALBEIT WITH
REDUCED )/ PERFORMANCE 7HEN GRID DISKS ARE CREATED IN A STORAGE CELL THEY ARE ASSIGNED TO A FAILURE
GROUP %ACH CELL CONSTITUTES A FAILURE GROUP AS CAN BE SEEN IN THE FOLLOWING LISTING
BHB)0B<!-bT[TRcSV ]P\TSXbZVa^d_S ]P\TSXbZS UPX[Va^d_
Ua^\ePb\NSXbZVa^d_SV
ePb\NSXbZS
fWTaTSV Va^d_N]d\QTa,S Va^d_N]d\QTa
P]SSV ]P\T[XZTB2A0C27
^aSTaQh !"*
38B:6A>D?38B:508;6A>D?
B2A0C27N36B2A0C27N36N23N $N24;; 24;;
B2A0C27N36B2A0C27N36N23N $N24;; !24;; !
B2A0C27N36B2A0C27N36N23N $N24;; "24;; "
"ECAUSE B2A0C27N36 WAS CREATED USING .ORMAL REDUNDANCY OUR B2A0C27 DATABASE SHOULD BE ABLE TO
CONTINUE EVEN IF AN ENTIRE STORAGE CELL DIES )N THIS SECTION WELL BE TESTING WHAT HAPPENS WHEN A STORAGE
CELL GOES DARK 7ELL USE THE SAME DISK GROUP CONFIGURATION WE USED FOR THE DISK FAILURE SIMULATION EARLIER
IN THIS CHAPTER 4O SIMULATE A CELL FAILURE WELL LOG IN TO THE ),/- ON STORAGE CELL AND POWER IT OFF
"ECAUSE EACH STORAGE CELL CONSTITUTES AN !3- FAILURE GROUP THIS SCENARIO IS VERY SIMILAR TO LOSING A SINGLE
CELL DISK )/ PERFORMANCE NOTWITHSTANDING 4HE DIFFERENCE OF COURSE IS THAT WE ARE LOSING AN ENTIRE
FAILURE GROUP *UST AS WE DID IN OUR CELL DISK FAILURE TESTS WELL GENERATE DATA IN THE B2A0C27 DATABASE
DURING THE FAILURE TO VERIFY THAT THE DATABASE CONTINUES TO SERVICE CLIENT REQUESTS DURING THE CELL OUTAGE
4O GENERATE )/ FOR THE TESTS WELL BE REPEATEDLY INSERTING ROWS FROM THE 186C01 TABLE INTO
THE QXVcPQ! TABLE
A9>7=B>=)B2A0C27-X]bTac P__T]S X]c^QXVcPQ!]^[^VVX]VbT[TRcUa^\QXVcPQ*
A9>7=B>=)B2A0C27-R^\\Xc*
7HILE THE ABOVE INSERTS ARE RUNNING LETS POWER OFF CELL AND TAKE A LOOK AT THE DATABASE ALERT LOG
!S YOU CAN SEE THE DATABASE THROWS AN ERROR WHEN READING FROM A DISK ON #ELL hFAILED TO READ MIRROR
SIDE v ! COUPLE OF LINES FURTHER DOWN IN THE LOG YOU SEE THE DATABASE SUCCESSFULLY READING THE MIRROR
COPY OF THE EXTENT hSUCCESSFULLY READ MIRROR SIDE v
P[TacNB2A0C27 [^V
5aX9P] #! ) ()#$!
4aa^abX]UX[T d P__ ^aPR[T SXPV aSQ\b bRaPcRW B2A0C27 caPRT B2A0C27N\\^]N" %&" caR)
>A0 !&% ")2T[[bc^aPVT8 >Taa^a8 >UPX[TS^]SXbZ^ (! %' ! $ B2A0C27N23N $NRT[[ "Pc
^UUbTc!#'" ##"$!U^aSPcP[T]VcW %"'#
>A0 !&%!%)4gPSPcPTaa^a) !=Tcf^aZTaa^a
...
F0A=8=6)ATPS5PX[TS Va^d_)"SXbZ)!0D)$(!^UUbTc) %"'#bXiT) %"'#
#(!04%2 2%#/6%2).' %8!$!4!
F0A=8=6)UPX[TSc^aTPS\Xaa^abXST ^UeXacdP[TgcT]c![^VXRP[TgcT]c ^UUX[T!% X]
Va^d_J" %'(#&&%" LUa^\SXbZB2A0C27N23N $N24;; "P[[^RPcX^]d]Xc$(!aTPb^]Taa^a*XU
_^bbXQ[TfX[[cahP]^cWTa\Xaa^abXST
=>C4)bdRRTbbUd[[haTPS\Xaa^abXST!^UeXacdP[TgcT]c![^VXRP[TgcT]c ^UUX[T!% X]
Va^d_J" %'(#&&%" LUa^\SXbZB2A0C27N23N $N24;; P[[^RPcX^]d]Xc$'(
4URNING TO THE !3- ALERT LOG WE SEE THAT !3- ALSO NOTICED THE ISSUE WITH #ELL AND RESPONDS BY
TAKING GRID DISK B2A0C27N23N $N24;; " OFFLINE .OTICE FURTHER ON THAT !3- IS IN THE PROCESS OF TAKING
OTHER GRID DISKS OFFLINE AS WELL 4HIS CONTINUES UNTIL ALL GRID DISKS ON #ELL ARE OFFLINE
P[TacN0B<! [^V
CTbc2T[[ "5PX[daT
5aX9P] #! ) ()#$!
=>C4)_a^RTbb!"##$X]XcXPcX]V^UU[X]T^USXbZ! "( $(""&'#B2A0C27N23N $N24;; "fXcW\PbZ
g&TX]Va^d_"
...
F0A=8=6)3XbZB2A0C27N23N $N24;; "X]\^ST g&UXb]^fQTX]V^UU[X]TS
5aX9P] #! ) ()#&!
=>C4)_a^RTbb (&$"X]XcXPcX]V^UU[X]T^USXbZ "( $(""%" A42>N23N N24;; "fXcW\PbZ
g&TX]Va^d_!
#HECKING THE EB4BB8>= AND EB@; VIEWS WE CAN SEE THAT THE INSERT IS STILL RUNNING
B83?A>6B@;N83B@;NC4GC
"b`[_[db/T](]RRic(`RV \'X]bTac P__T]S X]c^QXVcPQ!]^[^VVX
3O OUR DATABASES CONTINUE TO SERVICE CLIENT REQUESTS EVEN WHEN OF ALL STORAGE IS LOST 4HATS PRETTY
AMAZING ,ETS POWER UP #ELL AGAIN AND OBSERVE WHAT HAPPENS WHEN THIS STORAGE IS AVAILABLE AGAIN
,OOKING AT #ELLS ALERT LOG WE SEE RT[[bae BRING OUR GRID DISKS BACK ONLINE AGAIN 4HE LAST THING WE
SEE IN #ELLS ALERT LOG IS IT REJOINING THE STORAGE GRID BY ESTABLISHING A HEARTBEAT WITH THE SXbZ\^] DISK
MONITOR PROCESS ON THE DATABASE SERVERS
2T[[ "0[Tac[^V
Bc^aPVT8]STg0[[^RPcX^]U^a6aXS3XbZB2A0C27N36N23N $NRT[[ "bdRRTbbUd[JR^ST) L
2T[[3XbZe $]P\T,23N $NRT[[ "bcPcdb,=>A<0;VdXS,TSR$U% T %P% #'R( P#P% $'R# "P'%P&RU^d]S
^]STe, STe bSU
6aXSSXbZB2A0C27N36N23N $NRT[[ " ]d\QTaXb(%
Bc^aPVT8]STg0[[^RPcX^]U^a6aXS3XbZA42>N23N %NRT[[ "bdRRTbbUd[JR^ST) L
Bc^aPVT8]STg0[[^RPcX^]U^a6aXS3XbZBHBC4<N23N %NRT[[ "bdRRTbbUd[JR^ST) L
Bc^aPVT8]STg0[[^RPcX^]U^a6aXS3XbZBC064N23N %NRT[[ "bdRRTbbUd[JR^ST) L
Bc^aPVT8]STg0[[^RPcX^]U^a6aXS3XbZ30C0N23N %NRT[[ "bdRRTbbUd[JR^ST) L
2T[[3XbZe $]P\T,23N %NRT[[ "bcPcdb,=>A<0;VdXS, !' T Q %S"% U^d]S
^]STe, STe bSV
6aXSSXbZA42>N23N %NRT[[ " ]d\QTaXb
6aXSSXbZBHBC4<N23N %NRT[[ " ]d\QTaXb #
6aXSSXbZBC064N23N %NRT[[ " ]d\QTaXb '
6aXSSXbZ30C0N23N %NRT[[ " ]d\QTaXb !
...
5aX9P] #!!)$ )" !
7TPacQTPcfXcWSXbZ\^]bcPacTS^]T]ZSQ ! T]ZXcTR R^\
7TPacQTPcfXcWSXbZ\^]bcPacTS^]T]ZSQ T]ZXcTR R^\
#(!04%2 2%#/6%2).' %8!$!4!
5aX9P] #!!)$ )# !
...
0ERMANENT #ELL &AILURE
3O FAR WEVE DISCUSSED WHAT HAPPENS WHEN A CELL DISK IS LOST AND WHAT HAPPENS WHEN A STORAGE CELL IS
TEMPORARILY OFFLINE "UT WHAT DO YOU DO WHEN A STORAGE CELL COMPLETELY FAILS AND THERE IS NO BRINGING IT
BACK AGAIN %XADATA PROVIDES A MECHANISM FOR REMOVING YOUR GOOD DISK DRIVES FROM A FAILED STORAGE CELL
AND MOVING THEM TO A REPLACEMENT CELL
"ELIEVE IT OR NOT REPLACING A STORAGE CELL IS NOT A TERRIBLY COMPLICATED PROCESS *UST ABOUT EVERYTHING
YOU NEED IS CONTAINED ON THOSE SPINNING DISKS INSIDE THE CELL CHASSIS !S LONG AS YOUR OPERATING SYSTEM IS
INTACT ON THE SYSTEM VOLUMES YOU HAVE JUST ABOUT EVERYTHING YOU NEED TO MAKE THE MOVE +EEP IN MIND
THAT THE DISK REPAIR TIMER MIGHT NEED TO BE READJUSTED TO ALLOW US TIME FOR AN EXTENDED OUTAGE )T WILL
AFTER ALL TAKE TIME TO SHIP A NEW STORAGE CELL TO YOUR DATA CENTER 4HE PROCESS LOOKS SOMETHING LIKE THIS
1. )F POSSIBLE YOU SHOULD TAKE COPIES OF THE CONFIGURATION FILES FROM THE FAILED
STORAGE CELL 4HEY MAY COME IN HANDY LATER ON
v TcR W^bcb
v TcR \^S_a^QT R^]U
v TcR bhbR^]UXV ]Tcf^aZ
v TcR bhbR^]UXV ]Tcf^aZ bRaX_cb
2. 6ERIFY THAT THE DISK REPAIR TIMER WILL GIVE YOU ENOUGH TIME TO COMPLETE THE
PROCEDURE
BHB)0B<!-bT[TRcSV ]P\T3XbZ6^d_Pcca ]P\TPcca eP[dT
Ua^\ePb\NSXbZVa^d_SVePb\NPccaXQdcTPcca
fWTaTSV Va^d_N]d\QTa,Pcca Va^d_N]d\QTa
P]SPcca ]P\T,SXbZNaT_PXaNcX\T*
3XbZ6^d_=0<4E0;D4
30C0N36SXbZNaT_PXaNcX\T'W
A42>N36SXbZNaT_PXaNcX\T'W
B2A0C27N36SXbZNaT_PXaNcX\T'W
BC064N36SXbZNaT_PXaNcX\T'W
BHBC4<N36SXbZNaT_PXaNcX\T'W
)F YOU NEED TO YOU CAN ADJUST THE DISK REPAIR TIMER FOR YOUR DISK GROUPS BY
LOGGING IN TO THE !3- INSTANCE AND USING THE0;C4A38B:6A>D? COMMAND &OR
EXAMPLE
JT]ZSQ !)^aPR[T)0B<!L W^\T ^aPR[T
-b`[_[db PbbhbPb\
q
BHB)0B<!-P[cTaSXbZVa^d_B2A0C27N36bTcPccaXQdcTSXbZNaT_PXaNcX\T,&!W*
3. $EACTIVATE THE GRID DISKS ON THE FAILED STORAGE CELL AND THEN SHUT IT DOWN 4HIS
CAN BE DONE USING THE 2T[[2;8 AND ,INUX COMMANDS AS FOLLOWS
#(!04%2 2%#/6%2).' %8!$!4!
24;;2[X-P[cTaVaXSSXbZP[[X]PRcXeT
JT]ZRT[ ")a^^cL a^^c
-bWdcS^f]~W]^f
4. /NCE THE SYSTEM IS SHUT DOWN YOU CAN POWER IT OFF USING THE POWER BUTTON ON
THE FRONT PANEL
5. /PEN THE CASES OF BOTH THE FAILED CELL AND THE REPLACEMENT CELL AND MOVE THE
FOLLOWING COMPONENTS FROM THE OLD CELL TO THE NEW CELL
a. &LASH #ACHE CARDS
b. $ISK CONTROLLER CARD
c. 53" FLASH DRIVE
d. $ISK DRIVES
■ #AUTION )T IS IMPORTANT THAT YOU PLACE THE FIRST TWO DISKS IN THE SAME LOCATION ON THE NEW STORAGE CELL 4HESE ARE
THE SYSTEM VOLUMES THAT CONTAIN THE ,INUX OPERATING SYSTEM )F YOU DONT THE CELL MAY NOT BOOT !LSO MAKE SURE
YOU INSTALL THE FLASH CACHE MODULES IN THE SAME 0#) SLOTS AS THEY WERE IN THE FAILED CELL )N GENERAL IT IS A GOOD
PRACTICE TO SIMPLY REMOVE EACH DEVICE ONE AT A TIME AND REINSTALL THEM INTO THE SAME LOCATION ON THE NEW STORAGE
CELL
6. 0OWER ON THE NEW CELL
7. ,OG IN TO THE NEW CELL AS ROOT USING EITHER THE +6- OR ),/- INTERFACE
8. #HECK THE CONFIGURATION FILES IN THE FOLLOWING DIRECTORIES )F THEY ARE DAMAGED
OR OTHERWISE CORRUPTED REPLACE THEM WITH THE BACKUP COPIES YOU CREATED IN
STEP
9. 4HE NETWORK SETTINGS FOR YOUR %THERNET DEVICES WILL REMAIN THE SAME WITH THE
EXCEPTION OF THE HARDWARE ADDRESS OR -!# ADDRESS &IND THE NEW -!#
ADDRESSES OF THE NETWORK DEVICES ETH ETH ETH AND ETH 4HIS CAN BE DONE
USING THE XUR^]UXV COMMAND AS FOLLOWS
JT]ZRT[ ")a^^cL a^^c
-XUR^]UXVTcW
TcW ;X]ZT]RP_)4cWTa]Tc7FPSSa )! )!')'4)01)3'
X]TcPSSa) (! %' ' ! $1RPbc) (! %' !$$<PbZ)!$$ !$$ !$!
X]Tc%PSSa)UT' ))!! )!'UU)UT'T)PQS' %#BR^_T);X]Z
D?1A>0320BCAD==8=6<D;C820BC<CD) $ <TcaXR)
AG_PRZTcb)% !"'Taa^ab) Sa^__TS) ^eTaad]b) UaP\T)
CG_PRZTcb)!!"!%Taa^ab) Sa^__TS) ^eTaad]b) RPaaXTa)
R^[[XbX^]b) cg`dTdT[T])
#(!04%2 2%#/6%2).' %8!$!4!
AGQhcTb)$' " &#$ $<X1CGQhcTb)#"' # (# <X1
<T\^ah)U'R% U'R'
10. 5SING THE -!# ADDRESSES FROM STEP EDIT THE TcR bhbR^]UXV ]Tcf^aZ
bRaX_cb XURUV TcWj "l FILES AND REPLACE THE OLD -!# ADDRESSES WITH THE NEW
ONES (ERE IS AN EXAMPLE USING THE XURUV TcW FILE
JT]ZRT[ ")a^^cL TcR bhbR^]UXV ]Tcf^aZ bRaX_cb
-RPcXURUV TcW
3>=>CA4<>E4C74B4;8=4B
64=4A0C431H24;;
34E824,TcW
1>>C?A>C>,bcPcXR
>=1>>C,hTb
8?033A, (! %' ' ! $
=4C<0B:,!$$ !$$ !$!
=4CF>A:, (! %' '
1A>0320BC, (! %' !$$
60C4F0H, (! %'
7>C?;D6,]^
8?E%8=8C,]^
7F033A,00:21:28:8e:ab:d8Å Replace this MAC address with the new address
11. 2EBOOT THE NEW STORAGE CELL TO PICK UP THE NEW -!# ADDRESSES
12. !FTER REBOOTING THE CELL YOU SHOULD BE ABLE TO LOG IN TO IT USING bbW JUST AS YOU
NORMALLY WOULD ,OG IN AND REACTIVATE THE GRID DISKS USING THE 2T[[2;8
COMMAND 0;C4A6A8338B: COMMAND AS FOLLOWS
2T[[2;8-P[cTaVaXSSXbZP[[PRcXeT
13. /NCE THE GRID DISKS ARE ACTIVATED !3- WILL SET THEM TO A STATE OF ONLINE AND
BEGIN USING THEM ONCE AGAIN 4HIS IS ONLY TRUE IF THE DISK REPAIR TIMER HAS NOT
EXPIRED
14. 4HE LAST THING YOU WILL NEED TO DO IS CONFIGURE THE ),/- FOR !32 #ONFIGURING
THE ),/- IS A LITTLE OFF TOPIC AND IS COVERED VERY WELL IN THE /RACLE
DOCUMENTATION SO WE WONT BE DISCUSSING IT HERE
Summary
%XADATA IS A HIGHLY REDUNDANT PLATFORM WITH A LOT OF MOVING PARTS "USINESSES DONT TYPICALLY INVEST IN
SUCH A PLATFORM WITHOUT EXPECTATIONS OF MINIMAL DOWNTIME !S SUCH %XADATA IS COMMONLY USED FOR
HOSTING MISSION
CRITICAL BUSINESS APPLICATIONS WITH VERY STRINGENT UPTIME REQUIREMENTS +NOWING WHAT
TO DO WHEN THINGS GO WRONG IS CRITICAL TO MEETING THESE UPTIME REQUIREMENTS )N THIS CHAPTER WEVE
DISCUSSED THE PROPER PROCEDURES FOR PROTECTING YOUR APPLICATIONS AND CUSTOMERS FROM COMPONENT AND
SYSTEM FAILURES "EFORE YOUR SYSTEM IS ROLLED INTO PRODUCTION MAKE IT A PRIORITY TO PRACTICE BACKING UP
AND RESTORING SYSTEM VOLUMES REMOVING AND REPLACING DISK DRIVES AND REBOOTING STORAGE CELLS AND
BECOME FAMILIAR WITH WHAT HAPPENS TO YOUR DATABASES 2UN THE DIAGNOSTIC TOOLS WEVE DISCUSSED IN THIS
CHAPTER AND MAKE SURE YOU UNDERSTAND HOW TO INTERPRET THE OUTPUT )F YOU ARE GOING TO BE RESPONSIBLE
FOR MAINTAINING %XADATA FOR YOUR COMPANY NOW IS THE TIME TO GET COMFORTABLE WITH THE TOPICS DISCUSSED
IN THIS CHAPTER
C H A P T E R 10
Exadata Wait Events
4HE /RACLE DATABASE IS A VERY WELL INSTRUMENTED PIECE OF CODE AND IT HAS BEEN SO FOR QUITE A WHILE )T
KEEPS TRACK OF THE AMOUNT OF TIME SPENT IN DISCRETE OPERATIONS VIA THE USE OF WAIT EVENTS 7HILE THE
DATABASE SOFTWARE IS QUITE COMPLEX WAIT EVENT ANALYSIS ALLOWS PERFORMANCE ANALYSTS TO DETERMINE
PRECISELY WHERE THE DATABASE IS SPENDING ITS TIME -ANY DIFFICULT PERFORMANCE PROBLEMS CAN BE RESOLVED
BY ANALYZING THESE WAIT EVENTS 4HE INTRODUCTION OF %XADATA HAS RESULTED IN THE CREATION OF SEVERAL NEW
WAIT EVENTS TO SUPPORT THE UNIQUE OPERATIONS THAT ARE PERFORMED ON THE PLATFORM 4HIS CHAPTER WILL FOCUS
ON DESCRIBING THESE NEW EVENTS AND HOW THEY RELATE TO THE ACTIVITIES ACTUALLY BEING PERFORMED WHILE
CONTRASTING THEM WITH THE WAIT EVENTS USED BY THE DATABASE ON NON
%XADATA PLATFORMS )T WILL ALSO
DESCRIBE A FEW WAIT EVENTS THAT ARENT SPECIFIC TO %XADATA BUT PLAY AN IMPORTANT ROLE ON %XADATA
PLATFORMS
! wait event IS ACTUALLY A SECTION OF CODE THAT IS TIMED AND ASSIGNED A NAME 4HE PIECES OF CODE
COVERED BY THESE EVENTS ARE QUITE OFTEN DISCRETE OPERATING SYSTEM CALLS SUCH AS )/ REQUESTS BUT SOME
WAIT EVENTS COVER CONSIDERABLY LARGER PORTIONS OF CODE 4HE EVENTS MAY EVEN INCLUDE SECTIONS OF CODE
THAT CONTAIN OTHER WAIT EVENTS 4HE NAMING OF WAIT EVENTS HAS BEEN SOMEWHAT INCONSISTENT OVER THE
YEARS AND MANY EVENTS HAVE SOMEWHAT MISLEADING NAMES %VEN THOUGH SOME OF THE EVENT NAMES ARE
ACKNOWLEDGED TO BE POTENTIALLY MISLEADING /RACLE HAS BEEN UNDERSTANDABLY RELUCTANT TO CHANGE THEM
4HE %XADATA PLATFORM HAS PROVIDED AN EXCUSE TO RENAME SOME OF THE )/
RELATED EVENTS AND AS YOULL SEE
SHORTLY THE DEVELOPERS TOOK THE OPPORTUNITY TO DO JUST THAT
Events Specific to Exadata
4HERE ARE ACTUALLY NO EVENTS THAT EXIST EXCLUSIVELY ON THE %XADATA PLATFORM 7AIT EVENTS ARE BUILT INTO THE
DATABASE CODE AND SINCE THE COMPUTE NODES ON %XADATA RUN STANDARD /RACLE $ATABASE SOFTWARE ALL THE
WAIT EVENTS THAT ARE USED WHEN INVOKING %XADATA SPECIFIC FEATURES ARE AVAILABLE IN DATABASES THAT ARE
RUNNING ON NON %XADATA PLATFORMS AS WELL "UT BECAUSE THE %XADATA FEATURES ARE ONLY AVAILABLE ON THE
%XADATA PLATFORM NO TIME IS EVER ALLOCATED TO THOSE EVENTS ON OTHER PLATFORMS "Y WAY OF PROOF CONSIDER
THE FOLLOWING EXAMPLE WHICH COMPARES THE EVENTS FROM E4E4=CN=0<4 WHICH EXPOSES THE VALID WAIT
EVENTS FIRST ON AN %XADATA $ATABASE -ACHINE AND THEN ON A STANDARD G2 DATABASE ON A NON %XADATA
PLATFORM
BHB/4G31-bT[TRcR^d]cUa^\eTeT]cN]P\T*
2>D=C
'
#(!04%2 %8!$!4! 7!)4 %6%.43
BHB/4G31-bT[TRcR^d]cUa^\eTeT]cN]P\T/;01 !*
2>D=C
'
BHB/4G31-bT[TRc]P\TTeT]cUa^\eTeT]cN]P\T
!\X]db
"bT[TRc]P\TUa^\eTeT]cN]P\T/;01 !*
]^a^fbbT[TRcTS
3O THERE ARE NO DIFFERENCES IN THE EVENTS 4HIS DOES MAKE IT A LITTLE DIFFICULT TO COME UP WITH A LIST OF
h%XADATA /NLYv EVENTS
4HE hCELLv %VENTS
4HE Oracle Exadata Storage Server Software User’s Guide 11g Release 2 PROVIDES A TABLE OF WAIT EVENTS !LL
OF THEM START WITH THE WORD hCELLv 4HE MANUAL LISTS NINE SUCH EVENTS /NE OF THEM RT[[X]cTaR^]]TRc
aTcaP]b\XcSdaX]V_WhbXRP[aTPS DOESNT ACTUALLY EXIST
4HERE ARE ALSO ANOTHER NINE EVENTS HAVING THE WORD hCELLv IN THEIR NAMES THAT ARE NOT INCLUDED IN THE
DOCUMENTATION 3O WELL USE THE COMPLETE LIST OF hCELLv EVENTS AS A STARTING POINT 9OU CAN QUERY
E4E4=CN=0<4 FOR THAT LIST AND YOULL GET THE FOLLOWING RESULTS .OTICE THAT MOST OF THE EVENTS ARE IN ONE OF
THE )/ CLASSES 4HERE ARE ONLY ROWS BECAUSE ONE OF THE DOCUMENTED EVENTS DOES NOT EXIST IN THE
VERSION OF /RACLE THAT WERE RUNNING
BHB/?>2!-/eP[XSNTeT]cb
4]cTaeP[dTU^aTeT]cN]P\T)RT[[
4E4=C2;0BB
RT[[[Xbc^UQ[^RZb_WhbXRP[aTPSDbTa8 >
RT[[\d[cXQ[^RZ_WhbXRP[aTPSDbTa8 >
RT[[bX]V[TQ[^RZ_WhbXRP[aTPSDbTa8 >
RT[[b\PacUX[TRaTPcX^]DbTa8 >
RT[[b\PacX]STgbRP]DbTa8 >
RT[[b\PaccPQ[TbRP]DbTa8 >
RT[[bcPcXbcXRbVPcWTaDbTa8 >
RT[[\P]PVTaR[^bX]VRT[[BhbcT\8 >
RT[[\P]PVTaSXbR^eTaX]VSXbZbBhbcT\8 >
RT[[\P]PVTa^_T]X]VRT[[BhbcT\8 >
RT[[b\PacX]RaT\T]cP[QPRZd_BhbcT\8 >
RT[[b\PacaTbc^aTUa^\QPRZd_BhbcT\8 >
RT[[\P]PVTaRP]RT[f^aZaT`dTbc>cWTa
RT[[b\PacU[PbWd]ZTT_>cWTa
RT[[f^aZTa^][X]TR^\_[TcX^]>cWTa
RT[[f^aZTaaTcah>cWTa
RT[[f^aZTaXS[T8S[T
&a^fbbT[TRcTS
#(!04%2 %8!$!4! 7!)4 %6%.43
4HE FOLLOWING SECTIONS WILL COVER ALL OF THESE EVENTS ALONG WITH A COUPLE OF ADDITIONAL EVENTS THAT
HAVE SPECIAL APPLICABILITY TO %XADATA
0LAN 3TEPS 4HAT 4RIGGER %VENTS
&IRST THOUGH IT MIGHT BE INTERESTING TO SEE WHAT OPERATIONS PLAN STEPS CAUSE THE hCELLv WAIT EVENTS TO
OCCUR (ERE IS A QUERY AGAINST 310N78BCN02C8E4NB4BBN78BC>AH ON AN ACTIVE PRODUCTION SYSTEM RUNNING
ON AN %XADATA 6 THAT SHOWS CELL EVENTS AND THE OPERATIONS THAT CAUSED THEM
BHB/4G31 -bT[TRcTeT]c^_TaPcX^]R^d]cUa^\
!bT[TRcb`[NXSTeT]cb`[N_[P]N^_TaPcX^]kkkkb`[N_[P]N^_cX^]b^_TaPcX^]
"Ua^\310N78BCN02C8E4NB4BBN78BC>AH
#fWTaTTeT]c[XZTRT[[
$Va^d_Qh^_TaPcX^]TeT]c
%^aSTaQh !"
&
4E4=C>?4A0C8>=2>D=C
RT[[[Xbc^UQ[^RZb_WhbXRP[aTPS%!
33;BC0C4<4=C!
8=34G50BC5D;;B20=
8=34GA0=64B20=" %
8=34GBC>A06450BC5D;;B20=&
8=34GBC>A064B0<?;450BC5D;;B20=
8=34GD=8@D4B20= $'
8=B4ACBC0C4<4=C%
C01;40224BB1H6;>10;8=34GA>F83 $
C01;40224BB1H8=34GA>F83$#$'
C01;40224BB1H;>20;8=34GA>F83 "
C01;40224BBBC>A0645D;; '"
C01;40224BBBC>A064B0<?;4!
C01;40224BBBC>A064B0<?;41HA>F83A0=
RT[[\d[cXQ[^RZ_WhbXRP[aTPS"!!
33;BC0C4<4=C $&
8=34G50BC5D;;B20=(#
8=34GA0=64B20=!
8=34GBC>A06450BC5D;;B20=%""#
8=34GBC>A064B0<?;450BC5D;;B20=#!(
D=8@D4B20=!
E84F0224BBBC>A0645D;;%"#
<0CNE84F0224BBBC>A064B0<?;4$%
C01;40224BB1H6;>10;8=34GA>F83$
C01;40224BB1H8=34GA>F83#'#
C01;40224BB1H;>20;8=34GA>F83"
C01;40224BBBC>A0645D;;# $$(
C01;40224BBBC>A064B0<?;4 &%"
C01;40224BBBC>A064B0<?;41HA>F83A0=&'
D?30C4#
#(!04%2 %8!$!4! 7!)4 %6%.43
RT[[bX]V[TQ[^RZ_WhbXRP[aTPS ' '%
1D554AB>AC
2A40C4C01;4BC0C4<4=C%&
33;BC0C4<4=C('$
34;4C4 ! #
34;4C4BC0C4<4=C%
58G43C01;458G438=34G"$!
5>AD?30C4!&
70B76A>D?1H"
70B79>8= #
70B79>8=A867C>DC4A
8=34G1D8;3=>=D=8@D4'
8=34G1D8;3D=8@D4%
8=34G50BC5D;;B20=(
8=34G5D;;B20=
8=34GA0=64B20= &$(&
8=34GA0=64B20=<8= <0G
8=34GA0=64B20=34B24=38=6%
8=34GB:8?B20=%(
8=34GBC>A06450BC5D;;B20=" "
8=34GBC>A064B0<?;450BC5D;;B20=&!
8=34GD=8@D4B20=" (
8=B4ACBC0C4<4=C$ &#
;>030BB4;42C !
;>03C01;42>=E4=C8>=0;$'!&
<0CNE84F0224BBBC>A0645D;;"
<0CNE84F0224BBBC>A064B0<?;4
<4A64 !
?G2>>A38=0C>A
B4;42CBC0C4<4=C(&'
B>AC2A40C48=34G
B>AC6A>D?1H
B>AC9>8=$
B>AC>A34A1H!
C01;40224BB1H6;>10;8=34GA>F83$' !
C01;40224BB1H8=34GA>F83%$&((
C01;40224BB1H;>20;8=34GA>F83#$(
C01;40224BB1HDB4AA>F83#%#
C01;40224BB2;DBC4A$&
C01;40224BBBC>A0645D;;& %'
C01;40224BBBC>A064B0<?;4! $
C01;40224BBBC>A064B0<?;41HA>F83A0=!#
D=8>= 0;;&
D?30C4'("$"
D?30C4BC0C4<4=C"%&
F8=3>F278;3?DB743A0=:!
F8=3>FB>AC
F8=3>FB>AC?DB743A0=:
RT[[b\PacUX[TRaTPcX^]"$
34;4C4"
8=34G1D8;3=>=D=8@D4$
;>030BB4;42C"
#(!04%2 %8!$!4! 7!)4 %6%.43
;>03C01;42>=E4=C8>=0;
D?30C4
RT[[b\PacX]RaT\T]cP[QPRZd_& #
RT[[b\PacX]STgbRP] #
8=34GBC>A06450BC5D;;B20=#!
8=34GBC>A064B0<?;450BC5D;;B20="!
RT[[b\PaccPQ[TbRP] %"
<0CNE84F0224BBBC>A0645D;;
C01;40224BBBC>A0645D;; !$ #
/BVIOUSLY THIS OUTPUT DOES NOT SHOW ALL THE POSSIBLE COMBINATIONS BUT IT SHOULD GIVE YOU AN IDEA OF
THE RELATIVE FREQUENCY OF EVENTS AND WHICH OPERATIONS GENERALLY MOTIVATE THEM
Exadata Wait Events in the User I/O Class
4HE 5SER )/ #LASS IS FAR AND AWAY THE MOST IMPORTANT FOR %XADATA 4HE MOST INTERESTING EVENTS IN THIS
CATEGORY ARE OF COURSE THE TWO 3MART 3CAN EVENTS RT[[b\PaccPQ[TbRP] AND RT[[b\PacX]STgbRP]
4HESE ARE THE EVENTS THAT RECORD TIME FOR THE PRIMARY /FFLOADING OPTIMIZATIONS PROVIDED BY %XADATA
WHICH INCLUDE PREDICATE FILTERING COLUMN PROJECTION AND STORAGE INDEX USAGE 4HE 5SER )/ #LASS ALSO
CONTAINS THREE EVENTS DESCRIBED AS physical I/O EVENTS 4HESE THREE EVENTS ACTUALLY MEASURE TIME FOR
PHYSICAL )/ USING THE MORE FAMILIAR MULTI BLOCK AND SINGLE BLOCK READ MECHANISMS WERE USED TO SEEING
ON NON %XADATA PLATFORMS ALTHOUGH THEIR NAMES HAVE BEEN CHANGED TO SOMETHING A LITTLE MORE
MEANINGFUL &INALLY THERE ARE TWO EVENTS THAT DONT REALLY SEEM TO BELONG IN THE 5SER )/ CATEGORY AT ALL
/NE HAS TO DO WITH INITIALIZATION OF BLOCKS WHEN FILE SPACE IS ALLOCATED 4HE OTHER HAS TO DO WITH GATHERING
STATISTICAL INFORMATION FROM THE STORAGE CELLS 7ELL COVER EACH OF THESE WAIT EVENTS IN TURN IN THE NEXT
SEVERAL SECTIONS STARTING WITH THE 3MART 3CAN EVENTS
CELL SMART TABLE SCAN
4HE RT[[b\PaccPQ[TbRP] EVENT IS WHAT /RACLE USES TO ACCOUNT FOR TIME SPENT WAITING FOR &ULL 4ABLE
3CANS THAT ARE /FFLOADED )T IS THE MOST IMPORTANT NEW EVENT ON THE %XADATA PLATFORM )TS PRESENCE OR
ABSENCE CAN BE USED TO VERIFY WHETHER A STATEMENT BENEFITED FROM /FFLOADING OR NOT !S DISCUSSED IN
#HAPTER /FFLOADING ONLY OCCURS WHEN /RACLE IS ABLE TO DO DIRECT PATH READS #ONSEQUENTLY THIS EVENT
REPLACES THE SXaTRc_PcWaTPS EVENT IN MOST CASES ON %XADATA !S WITH NORMAL DIRECT PATH READS DATA IS
RETURNED DIRECTLY TO THE 0'! OF THE REQUESTING PROCESS ON THE DATABASE SERVER EITHER THE USERS SHADOW
PROCESS OR A PARALLEL SLAVE PROCESS "LOCKS ARE NOT RETURNED TO THE BUFFER CACHE
Event Meaning
!LTHOUGH THE MECHANISM FOR PERFORMING READS ACROSS THE )NFINI"AND NETWORK IS VERY DIFFERENT FROM THAT
FOR NORMAL READS ON NON
%XADATA PLATFORMS THE CODE PATH DRIVING THE SMART SCANS IS ACTUALLY VERY SIMILAR
TO A DIRECT PATH READ ON A NON
%XADATA PLATFORM 4HE MAIN DIFFERENCE LIES IN THE FACT THAT EACH REQUEST TO
A STORAGE CELL CONTAINS A REFERENCE TO THE METADATA OF THE STATEMENT WHICH IN THE CASE OF %XADATA
INCLUDES THE PREDICATES AND THE LIST OF COLUMNS TO BE RETURNED 3INCE THE STORAGE CELLS HAVE ACCESS TO THIS
INFORMATION THEY CAN APPLY THE FILTERS AND DO THE COLUMN PROJECTION BEFORE RETURNING THE DATA TO THE
REQUESTING PROCESS 4HESE OPTIMIZATIONS ARE APPLIED TO EACH SET OF BLOCKS AS THEY ARE REQUESTED 4HE
PROCESSES ON THE DATABASE SERVERS THAT REQUEST THE DATA HAVE ACCESS TO THE !3- EXTENT MAP AND CAN
THEREFORE REQUEST THE REQUIRED ALLOCATION UNITS !5S FROM EACH STORAGE CELL 4HE STORAGE CELLS READ THE
REQUESTED !5 AND APPLY THE PREDICATE FILTERS AND IF ANY ROWS SATISFY THE FILTERS RETURN THE PROJECTED
#(!04%2 %8!$!4! 7!)4 %6%.43
COLUMNS TO THE REQUESTING PROCESS 4HE PROCESS THEN REQUESTS THE NEXT !5 AND THE ENTIRE ROUTINE IS
REPEATED UNTIL ALL THE DATA HAS BEEN SCANNED 3O THIS EVENT WILL OCCUR REPEATEDLY IN A LARGE SCAN
■ .OTE #OLUMN 0ROJECTION IS ONE OF THE MAJOR OPTIMIZATIONS PROVIDED BY 3MART 3CANS 4HE FEATURE IS SLIGHTLY
MISUNDERSTOOD )T DOES NOT PASS ONLY COLUMNS IN THE SELECT LIST BACK TO THE DATABASE SERVERS IT ALSO PASSES BACK
SOME COLUMNS FROM THE F74A4 CLAUSE /LDER VERSIONS OF RT[[bae PASSED ALL COLUMNS SPECIFIED IN A F74A4 CLAUSE
BACK TO THE DATABASE TIER ,ATER VERSIONS HAVE CORRECTED THIS BEHAVIOR TO INCLUDE ONLY COLUMNS THAT ARE INVOLVED IN
JOIN PREDICATES
(ERES A PORTION OF A TRACE FILE SHOWING SOME OF THE EVENTS 4HE QUERY GENERATING THIS TRACE FILE
DID A FULL TABLE SCAN WHICH WAS PERFORMED VIA THE DIRECT READ MECHANISM .OTE THAT WEVE SHORTENED
SOME OF THE LINES TO GET THEM TO FIT PRIMARILY BY STRIPPING OFF THE TIME VALUES
?0AB8=68=2DAB>A![T],"!ST_, dXS, ^Rc,"[XS, We, !"PS, (%b`[XS, %!fY]efhhQW]
bT[TRcbd\_ZNR^[Ua^\Zb^ bZTf
4=3>5BC<C
?0AB4!)R,! T,!#!#_, Ra, Rd, \Xb, a, ST_, ^V, _[W,$%'"!!"&%
4G42!)R, T,"#_, Ra, Rd, \Xb, a, ST_, ^V, _[W,$%'"!!"&%
F08C!)]P\,B@;=Tc\TbbPVTc^R[XT]cT[P,"SaXeTaXS, %$ ' $!"!QhcTb, _", ^QY,
F08C!)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P,!_^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C!)]P\,:92)FPXcU^a\bVbT]Sbc^R^\_[TcTT[P, \bV,%%&##$ "%'STbckaRea,%$$"%
F08C!)]P\,aT[XPQ[T\TbbPVTT[P, &RWP]]T[R^]cTgc,%& !!& '&!RWP]]T[WP]S[T,%%(%&((
F08C!)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P, _^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C!)]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, #]P\Tk\^ST, !%!,%$$&$ , ^QY,
F08C!)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P, _^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C!)]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, "]P\Tk\^ST, !%!,%$$&$ ,!^QY,
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P, %!RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,!##RT[[WPbW,"&(""(($'_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P, ' RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P, !'$RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P, "!&RT[[WPbW,"&(""(($'_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P, " RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P, (&$$RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,"(RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,!#RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,'#&RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,##RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,%&#RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,"!RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,"(RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,(!'RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,"&RT[[WPbW," &%$(## (_!, _", ^QY,%%'#(
F08C!)]P\,RT[[b\PaccPQ[TbRP]T[P,(&RT[[WPbW,"&(""(($'_!, _", ^QY,%%'#(
4HIS PORTION OF THE TRACE FILE ALSO SHOWS THE T]`):> UPbc^QYTRcRWTRZ_^X]c EVENT WHICH IS USED
TO ENSURE THAT ANY DIRTY BLOCKS FOR THE SCANNED OBJECT ARE FLUSHED TO DISK PRIOR TO BEGINNING THE SCAN "Y
#(!04%2 %8!$!4! 7!)4 %6%.43
THE WAY THE SXaTRc_PcWaTPS EVENT IS NOT COMPLETELY ELIMINATED ON %XADATA PLATFORMS )N FACT WE CAN
USE A HINT TO DISABLE /FFLOADING AND SEE HOW THE SAME STATEMENT BEHAVES WITHOUT /FFLOADING
?0AB8=68=2DAB>A![T],'#ST_, dXS, ^Rc,"[XS, We, !"PS, (%b`[XS,RRii%h%aci#Q]
bT[TRc >?CN?0A0<RT[[N^UU[^PSN_a^RTbbX]VUP[bT bd\_ZNR^[Ua^\Zb^ bZTf
4=3>5BC<C
?0AB4!)R,!%(($T,""$'!_, Ra, Rd, \Xb, a, ST_, ^V, _[W,$%'"!!"&%
4G42!)R, T,! _, Ra, Rd, \Xb, a, ST_, ^V, _[W,$%'"!!"&%
F08C!)]P\,B@;=Tc\TbbPVTc^R[XT]cT[P,"SaXeTaXS, %$ ' $!"!QhcTb, _", ^QY,
F08C!)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P,"_^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C!)]P\,:92)FPXcU^a\bVbT]Sbc^R^\_[TcTT[P, "\bV,%%STbckaRea,%$$"%\ch_T, !
F08C!)]P\,aT[XPQ[T\TbbPVTT[P, "$$RWP]]T[R^]cTgc,%& !!& '&!RWP]]T[
WP]S[T,%%(%&($! %Qa^PSRPbc\TbbPVT,%&# ""$'$%^QY, cX\, !'$#$$%&%!%# '
F08C!)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P,!_^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C!)]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, &!]P\Tk\^ST, !%!,%$$&" , ^QY,
F08C!)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P,!_^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C!)]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P,!! ]P\Tk\^ST, !%!,%$$&" ,!^QY,
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P, #R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,SXaTRc_PcWaTPST[P,!"&#%UX[T]d\QTa,&UXabcSQP,&%"!!Q[^RZR]c,(#
^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,!'R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,! R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P, R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,%R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,SXaTRc_PcWaTPST[P, '$%%UX[T]d\QTa,&UXabcSQP,&%' Q[^RZR]c, !'
^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P, (R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,! R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,%R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,SXaTRc_PcWaTPST[P,"$ '"UX[T]d\QTa,&UXabcSQP,&& '#Q[^RZR]c, !'
^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P, (R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,!!R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,&R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
F08C!)]P\,ZUZ)Pbh]RSXbZ8>T[P,$R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'#(
.OTE THAT WE STILL HAVE THE T]`):>~UPbc^QYTRcRWTRZ_^X]c EVENTS FOR FLUSHING DIRTY BLOCKS 3O IT
IS CLEAR THAT THE RT[[b\PaccPQ[TbRP] EVENT REPLACES THIS EVENT
Parameters
4HE PARAMETERS FOR THIS EVENT ARE NOT PARTICULARLY INFORMATIVE /NLY THE OBJECT )$ OF THE TABLE BEING
SCANNED AND THE CELL HASH NUMBER ARE PROVIDED
?
#ELL HASH NUMBER
?!
.OT USED
?"
.OT USED
^QY
4HE OBJECT NUMBER OF THE TABLE BEING SCANNED
#(!04%2 %8!$!4! 7!)4 %6%.43
9OULL NOTICE THAT THE SXaTRc_PcWaTPS EVENT WHICH RT[[b\PaccPQ[TbRP] REPLACES PROVIDES
ADDITIONAL INFORMATION INCLUDING THE FILE NUMBER THE OFFSET INTO THE FILE UXabcSQP AND THE NUMBER OF
CONTIGUOUS BLOCKS READ Q[^RZR]c /N THE OTHER HAND WITH THE SXaTRc_PcWaTPS EVENT THERE IS NO
INDICATION HOW THE READ REQUESTS ARE ROUTED TO THE INDIVIDUAL CELLS
■ .OTE 4HE #ELL HASH NUMBER REPORTED WITH MANY OF THE WAIT EVENTS CAN BE FOUND IN THE E24;; VIEW 4HIS VIEW
HAS ONLY TWO COLUMNS 24;;N?0C7 AND 24;;N70B7E0; 4HE 24;;N?0C7 COLUMN ACTUALLY CONTAINS THE )0 ADDRESS OF
THE STORAGE CELL
CELL SMART INDEX SCAN
4IME IS CLOCKED TO THE RT[[b\PacX]STgbRP] EVENT WHEN FAST FULL INDEX SCANS ARE PERFORMED THAT ARE
/FFLOADED 4HIS EVENT IS ANALOGOUS TO RT[[b\PaccPQ[TbRP] EXCEPT THAT THE OBJECT BEING SCANNED IS AN
INDEX )T REPLACES THE SXaTRc_PcWaTPS EVENT AND RETURNS DATA DIRECTLY TO THE 0'! OF THE REQUESTING
PROCESS AS OPPOSED TO THE BUFFER CACHE
Event Meaning
4HIS EVENT DOES NOT SHOW UP VERY OFTEN ON THE SYSTEMS WE HAVE OBSERVED PROBABLY FOR SEVERAL REASONS
v %XADATA IS QUITE GOOD AT DOING FULL TABLE SCANS AND SO THE TENDENCY IS TO ELIMINATE A
LOT OF INDEXES WHEN MOVING TO THE PLATFORM
v $IRECT PATH READS ARE NOT DONE AS OFTEN ON INDEX SCANS AS THEY ARE ON TABLE SCANS
/NE OF THE IMPORTANT CHANGES TO /RACLE IS THE AGGRESSIVENESS WITH WHICH IT
DOES DIRECT PATH READS ON SERIAL TABLE SCANS 4HIS ENHANCEMENT WAS PROBABLY
PUSHED FORWARD SPECIFICALLY IN ORDER TO ALLOW %XADATA TO DO MORE 3MART &ULL 4ABLE
3CANS BUT REGARDLESS OF THAT WITHOUT THIS FEATURE ONLY PARALLEL TABLE SCANS WOULD
BE ABLE TO TAKE ADVANTAGE OF 3MART 3CANS 4HE SAME ENHANCEMENT APPLIES TO INDEX
FAST FULL SCANS 4HAT IS THEY CAN ALSO BE DONE VIA SERIAL DIRECT PATH READS (OWEVER
THE ALGORITHM CONTROLLING WHEN THEY HAPPEN APPEARS TO BE LESS LIKELY TO USE THIS
TECHNIQUE WITH INDEXES PROBABLY BECAUSE THE INDEXES ARE USUALLY MUCH SMALLER
THAN TABLES
v )N ADDITION ONLY FAST FULL SCANS OF INDEXES ARE ELIGIBLE FOR 3MART 3CANS RANGE SCANS
AND FULL SCANS ARE NOT ELIGIBLE
#(!04%2 %8!$!4! 7!)4 %6%.43
■ +EVIN SAYS ! 3MART3CAN ON AN INDEX IS POSSIBLE WITHOUT PARALLELISM BUT THE RULES ARE STRICT )F YOU ALTER THE
SESSION TO DISABLE PARALLEL QUERY AND SET NbTaXP[NSXaTRcNaTPS,CAD4 IN THE SESSION AND THEN RUN YOUR TEST YOU
SHOULD GET A SMART SERIAL index fast full scan )&&3 )F NOT THEN YOURE LOOKING AT A BUG )N SHORT TO GET A SERIAL
)&&3 THE OBJECT NEEDS TO BE MORE THAN FIVE TIMES THE SIZE OF THE 3'!
'LUING RANGE SCANS TO SMART INDEX SCANS CAN BE DONE BUT ITS VERY MESSY 4HERE ISNT LIKELY MUCH TO BE GAINED
THERE IN TERMS OF COMPLETING A QUERY
!S A RESULT OF THESE ISSUES RT[[b\PacX]STgbRP]b ARE FAIRLY RARE COMPARED TO RT[[b\PaccPQ[T
bRP]b )T IS OF COURSE POSSIBLE TO ENCOURAGE THE FEATURE WITH HINTS SUCH AS _PaP[[T[NX]STg OR BY
DECORATING SPECIFIC INDEXES WITH A PARALLEL DEGREE SETTING OF GREATER THAN (ERES AN EXCERPT FROM A
TRACE FILE SHOWING THE EVENT
?0AB8=68=2DAB>A [T], !ST_, dXS, ^Rc,"[XS, We, !"PS, (&b`[XS,S]\d\WZ($((_&
bT[TRc _PaP[[T[NX]STgPbZTf!NP[['X]STgNUUbPbZTf!NP[[ bd\_ZNR^[Ua^\
Zb^ bZTf!PfWTaTR^[ -
4=3>5BC<C
?0AB4 )R, T,'(_, Ra, Rd, \Xb, a, ST_, ^V, _[W,!' ' & cX\, !'$"&!##'& $ (!
F08C )]P\,?G3T`)4gTRdcX^]<bVT[P," !#b[TT_cX\T bT]STaXS,!%'_PbbTb, _", ^QY,
F08C )]P\,RT[[b\PacX]STgbRP]T[P, ""RT[[WPbW,"&(""(($'_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,$!!RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,!&&RT[[WPbW," &%$(## (_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P, ( RT[[WPbW,"&(""(($'_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,"" RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,"#'RT[[WPbW," &%$(## (_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,#$ %!RT[[WPbW," &%$(## (_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P, "RT[[WPbW," &%$(## (_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,! &RT[[WPbW,"&(""(($'_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P," RT[[WPbW,"&(""(($'_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P,&&(RT[[WPbW," &%$(## (_!, _", ^QY,%%'$%
F08C )]P\,RT[[b\PacX]STgbRP]T[P," RT[[WPbW,! ""#$(#'"_!, _", ^QY,%%'$%
.OTE THAT THIS TRACE FILE WAS PRODUCED BY ONE OF THE PARALLEL SLAVE PROCESSES AND NOT THE REQUESTING
PROCESS 4HE TRACE PRODUCED FOR THE SAME STATEMENT WHEN /FFLOADING IS DISABLED SHOULD LOOK FAMILIAR
(ERES AN EXCERPT
?0AB8=68=2DAB>A [T], $'ST_, dXS, ^Rc,"[XS, We,""'PS, (%b`[XS,V!(%%SfP!gU\'
bT[TRc >?CN?0A0<RT[[N^UU[^PSN_a^RTbbX]VUP[bT_PaP[[T[NX]STgPbZTf!NP[['
X]STgNUUbPbZTf!NP[[ bd\_ZNR^[Ua^\Zb^ bZTf!PfWTaTR^[ -
4=3>5BC<C
?0AB4 )R, T,(#_, Ra, Rd, \Xb, a, ST_, ^V, _[W,!' ' & cX\, !'$$#%(( # $&##
F08C )]P\,?G3T`)4gTRdcX^]<bVT[P,!"&&b[TT_cX\T bT]STaXS,!%'$%%$!&_PbbTb, _",
F08C )]P\,[PcRW)RPRWTQdUUTabRWPX]bT[P,##PSSaTbb,%%"%%"(" #]d\QTa, $ caXTb,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P, !R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,SXaTRc_PcWaTPST[P,!'($&UX[T]d\QTa,%UXabcSQP, "Q[^RZR]c, !#^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,"!R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!#R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
#(!04%2 %8!$!4! 7!)4 %6%.43
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!#R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,'R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,SXaTRc_PcWaTPST[P, # UX[T]d\QTa,%UXabcSQP, " # %#Q[^RZR]c, !'
^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!#R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,! R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!#R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!"R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!"R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!"R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,"!R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!'R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,! R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!$R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P," R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,&R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,SXaTRc_PcWaTPST[P, %&!$UX[T]d\QTa,%UXabcSQP, " " & Q[^RZR]c, !$
^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!#R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,!&R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,'R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,%%'$%
!GAIN WE SEE THE REPEATING PATTERN OF A SINGLE SXaTRc_PcWaTPS EVENT FOLLOWED BY SEVERAL ZUZ)
Pbh]RSXbZ8> EVENTS 4HE RT[[b\PacX]STgbRP] EVENT REPLACES BOTH OF THESE EVENTS 4HE T]`):>~
UPbc^QYTRcRWTRZ_^X]c EVENTS FOR FLUSHING DIRTY BLOCKS PRIOR TO STARTING THE DIRECT PATH READS ARE STILL
PRESENT ALTHOUGH NOT SHOWN IN THIS EXCERPT BECAUSE THEY OCCUR IN THE QUERY COORDINATOR PROCESS NOT IN
THE PARALLEL SLAVE PROCESSES
Parameters
*UST AS WITH THE RT[[b\PaccPQ[TbRP] EVENT THE PARAMETERS FOR RT[[b\PacX]STgbRP] DO NOT CONTAIN
A LOT OF DETAILS 4HE CELL HASH NUMBER AND THE OBJECT )$ OF THE SEGMENT BEING SCANNED ARE THE ONLY
INFORMATION PROVIDED
?
#ELL HASH NUMBER
?!
.OT USED
?"
.OT USED
^QY
4HE OBJECT NUMBER OF THE INDEX BEING SCANNED
CELL SINGLE BLOCK PHYSICAL READ
4HIS EVENT IS EQUIVALENT TO THE SQUX[TbT`dT]cXP[aTPS EVENT USED ON NON
%XADATA PLATFORMS 3INGLE
BLOCK READS ARE USED MOST OFTEN FOR INDEX ACCESS PATHS BOTH THE INDEX BLOCK READS AND THE TABLE BLOCK
READS VIA ROWIDS FROM THE INDEX LOOKUPS 4HEY CAN ALSO BE USED FOR A WIDE VARIETY OF OTHER OPERATIONS
WHERE IT MAKES SENSE TO READ A SINGLE BLOCK
#(!04%2 %8!$!4! 7!)4 %6%.43
Event Meaning
(ERE IS THE OUTPUT OF A QUERY THAT SHOWS THE OPERATIONS THAT RESULTED IN THE RT[[bX]V[TQ[^RZ_WhbXRP[
aTPS WAIT EVENT ON AN ACTIVE PRODUCTION SYSTEM
BHB/4G31 -bT[TRcTeT]c^_TaPcX^]R^d]cUa^\
!bT[TRcb`[NXSTeT]cb`[N_[P]N^_TaPcX^]kkkkb`[N_[P]N^_cX^]b^_TaPcX^]
"Ua^\310N78BCN02C8E4NB4BBN78BC>AH
#fWTaTTeT]c[XZTRT[[bX]V[T
$Va^d_Qh^_TaPcX^]TeT]c
%^aSTaQh !"
&
4E4=C>?4A0C8>=2>D=C
RT[[bX]V[TQ[^RZ_WhbXRP[aTPS ""!
2A40C4C01;4BC0C4<4=C"$
33;BC0C4<4=C '
34;4C4!%(
58G43C01;458G438=34G"
5>AD?30C4!
70B79>8=#
70B79>8=A867C>DC4A'
8=34G5D;;B20=(!'"
8=34G5D;;B20=<8= <0G
8=34GA0=64B20=!&%"
8=34GBC>A06450BC5D;;B20=%
8=34GBC>A064B0<?;450BC5D;;B20= "
8=34GD=8@D4B20= %&%
8=B4ACBC0C4<4=C '
;>030BB4;42C%
;>03C01;42>=E4=C8>=0;(!
<4A64 %
B4;42CBC0C4<4=C#
B>AC>A34A1H%
C01;40224BB1H6;>10;8=34GA>F83 %"'
C01;40224BB1H8=34GA>F83'& #
C01;40224BB1H;>20;8=34GA>F83 ##%
C01;40224BB2;DBC4A !
C01;40224BBBC>A0645D;;&&%
C01;40224BBBC>A064B0<?;4#
D?30C4' %
!S YOU CAN SEE ROW ACCESS VIA AN INDEX IS THE MOST COMMON OPERATION THAT GENERATES THIS EVENT
9OU SHOULD ALSO BE AWARE THAT %XADATA PROVIDES A LARGE AMOUNT OF FLASH CACHE ' ON EACH STORAGE
CELL &OR THAT REASON PHYSICAL READS BOTH MULTI
BLOCK AND SINGLE
BLOCK ARE CONSIDERABLY FASTER THAN ON
MOST DISK
BASED STORAGE SYSTEMS (ERE IS AN EXCERPT FROM AN !72 REPORT SHOWING A HISTOGRAM OF SINGLE
BLOCK READS FOR THE INSTANCE
^UFPXcb
C^cP[
#(!04%2 %8!$!4! 7!)4 %6%.43
4eT]cFPXcb+ \b+!\b+#\b+'\b+ %\b+"!\b+, b- b
RT[[bX]V[TQ[^RZ_WhbXRP[!(# :(# #" ! " % ( $ !
.OTICE THAT ABOUT OF THE RT[[bX]V[TQ[^RZ_WhbXRP[aTPS EVENTS TAKE LESS THAN MS 4HIS IS
FAIRLY REPRESENTATIVE OF SEVERAL PRODUCTION SYSTEMS THAT WEVE OBSERVED
Parameters
4HE RT[[bX]V[TQ[^RZ_WhbXRP[aTPS EVENT PROVIDES MORE INFORMATION THAN MOST CELL EVENTS 4HE
PARAMETERS ALLOW YOU TO TELL EXACTLY WHICH OBJECT WAS READ ALONG WITH PROVIDING THE DISK AND CELL WHERE
THE BLOCK WAS STORED
?
#ELL HASH NUMBER
?!
$ISK HASH NUMBER
?"
4OTAL BYTES PASSED DURING READ OPERATION ALWAYS ASSUMING + BLOCK
SIZE
^QY
4HE OBJECT NUMBER OF THE OBJECT BEING READ
CELL MULTIBLOCK PHYSICAL READ
4HIS IS ANOTHER RENAMED EVENT )T IS EQUIVALENT TO THE LESS CLEARLY NAMED SQUX[TbRPccTaTSaTPS EVENT
/N NON
%XADATA PLATFORMS /RACLE $ATABASE G2 STILL USES THE SQUX[TbRPccTaTSaTPS EVENT
WHENEVER IT ISSUES A CONTIGUOUS MULTI
BLOCK READ TO THE OPERATING SYSTEM
Event Meaning
4HIS EVENT IS GENERALLY USED WITH &ULL 4ABLE 3CANS AND &AST &ULL )NDEX SCANS ALTHOUGH IT CAN BE USED WITH
MANY OTHER OPERATIONS 4HE NEW NAME ON THE %XADATA PLATFORM IS MUCH MORE DESCRIPTIVE THAN THE OLDER
NAME 4HIS WAIT EVENT IS NOT NEARLY AS PREVALENT ON %XADATA PLATFORMS AS ON NON
%XADATA PLATFORMS
BECAUSE %XADATA HANDLES MANY FULL SCAN OPERATIONS WITH 3MART 3CANS THAT HAVE THEIR OWN WAIT EVENTS
RT[[b\PaccPQ[TbRP] AND RT[[b\PacX]STgbRP] 4HE RT[[\d[cXQ[^RZ_WhbXRP[aTPS EVENT ON
%XADATA PLATFORMS IS USED FOR SERIAL &ULL 3CAN OPERATIONS ON TABLES THAT ARE BELOW THE THRESHOLD FOR SERIAL
DIRECT PATH READS 4HAT IS TO SAY YOU WILL SEE THIS EVENT USED MOST OFTEN ON &ULL 3CANS OF RELATIVELY SMALL
TABLES )T IS ALSO USED FOR &AST &ULL )NDEX 3CANS THAT ARE NOT EXECUTED WITH DIRECT PATH READS (ERE IS THE
OUTPUT OF A QUERY THAT SHOWS THE OPERATIONS THAT RESULTED IN THE RT[[\d[cXQ[^RZ_WhbXRP[aTPS WAIT
EVENT ON AN ACTIVE PRODUCTION SYSTEM
4E4=C>?4A0C8>=2>D=C
RT[[\d[cXQ[^RZ_WhbXRP[aTPS&%#
33;BC0C4<4=C!'
8=34G50BC5D;;B20=!
8=34GBC>A06450BC5D;;B20=%$&
8=34GBC>A064B0<?;450BC5D;;B20= ""
C01;40224BB1H8=34GA>F83&#
#(!04%2 %8!$!4! 7!)4 %6%.43
C01;40224BB1H;>20;8=34GA>F83 #!'
C01;40224BBBC>A0645D;;$ #%
C01;40224BBBC>A064B0<?;4( %
Parameters
4HE RT[[\d[cXQ[^RZ_WhbXRP[aTPS EVENT ALSO PROVIDES MORE INFORMATION THAN MOST CELL EVENTS 4HE
PARAMETERS IN THE FOLLOWING LIST ALLOW YOU TO TELL WHICH OBJECT WAS READ AND IDENTIFIES THE DISK AND CELL
WHERE THE BLOCKS WERE STORED 4HE TOTAL BYTES PASSED SHOULD BE A MULTIPLE OF THE BLOCK SIZE
?
#ELL HASH NUMBER
?!
$ISK HASH NUMBER
?"
4OTAL BYTES PASSED DURING READ OPERATION
^QY
4HE OBJECT NUMBER OF THE OBJECT BEING READ
CELL LIST OF BLOCKS PHYSICAL READ
4HIS EVENT IS A REPLACEMENT FOR THE SQUX[T_PaP[[T[aTPS EVENT )T APPEARS THAT THE DEVELOPERS TOOK THE
OPPORTUNITY TO RENAME SOME OF THE EVENTS THAT ARE RELATED TO DISK OPERATIONS AND THIS IS ONE OF THOSE
EVENTS 4HE NEW NAME IS ACTUALLY MUCH MORE DESCRIPTIVE THAN THE PREVIOUS NAME SINCE THE WAIT EVENT
HAS NOTHING WHATSOEVER TO DO WITH PARALLEL QUERY OR PARALLEL $-,
Event Meaning
4HIS EVENT IS USED FOR A MULTI
BLOCK READ OF NONCONTIGUOUS BLOCKS 4HIS IS MORE EFFECTIVE WITH ASYNC )/
WHICH IS ENABLED ON %XADATA BY DEFAULT 4HIS EVENT CAN BE PROVOKED BY SEVERAL OPERATIONS 4HE MOST
COMMON ARE )NDEX 2ANGE 3CANS )NDEX 5NIQUE 3CANS AND 4ABLE !CCESS "Y )NDEX 2OWID 4HE MOST
COMMON REASON FOR THE EVENT IS INDEX PRE
FETCHING (ERE IS THE OUTPUT OF A QUERY THAT SHOWS THE
OPERATIONS THAT RESULTED IN THIS WAIT EVENT ON AN ACTIVE PRODUCTION SYSTEM
BHB/4G31 -bT[TRcTeT]c^_TaPcX^]R^d]cUa^\
!bT[TRcb`[NXSTeT]cb`[N_[P]N^_TaPcX^]kkkkb`[N_[P]N^_cX^]b^_TaPcX^]
"Ua^\310N78BCN02C8E4NB4BBN78BC>AH
#fWTaTTeT]c[XZTRT[[[Xbc
$Va^d_Qh^_TaPcX^]TeT]c
%^aSTaQh !"
&
4E4=C>?4A0C8>=2>D=C
RT[[[Xbc^UQ[^RZb_WhbXRP[aTPS!
8=34GA0=64B20= $%
8=34GBC>A06450BC5D;;B20=
8=34GD=8@D4B20=%%
C01;40224BB1H6;>10;8=34GA>F83(
C01;40224BB1H8=34GA>F83 !&"
C01;40224BB1H;>20;8=34GA>F83!$("
#(!04%2 %8!$!4! 7!)4 %6%.43
C01;40224BBBC>A0645D;;!
C01;40224BBBC>A064B0<?;4
!S YOU CAN SEE THE VAST MAJORITY OF THESE EVENTS WERE MOTIVATED BY INDEX ACCESS PATHS "Y THE WAY
ON NON
%XADATA PLATFORMS NONCONTIGUOUS MULTI
BLOCK READS STILL CLOCK TIME TO THE OLD SQUX[T_PaP[[T[
aTPS EVENT )TS ALSO POSSIBLE FOR THIS OLDER WAIT EVENT NAME TO SHOW UP ON AN %XADATA PLATFORM FOR SOME
OPERATIONS
Parameters
4HE RT[[[Xbc^UQ[^RZb_WhbXRP[aTPS EVENT PROVIDES MORE INFORMATION THAN MOST CELL EVENTS 4HE
FOLLOWING PARAMETERS ALLOW YOU TO TELL EXACTLY WHICH OBJECT WAS READ ALONG WITH IDENTIFYING THE DISK AND
CELL WHERE THE BLOCK WAS STORED
?
#ELL HASH NUMBER
?!
$ISK HASH NUMBER
?"
.UMBER OF BLOCKS READ
^QY
4HE OBJECT NUMBER OF THE OBJECT BEING READ
CELL SMART FILE CREATION
%XADATA HAS AN OPTIMIZATION TECHNIQUE THAT ALLOWS THE STORAGE CELLS TO DO THE INITIALIZATION OF BLOCKS
WHEN A DATA FILE IS CREATED OR EXTENDED 4HIS OCCURS WHEN A TABLESPACE IS CREATED OR A DATA FILE IS
MANUALLY ADDED TO A TABLESPACE (OWEVER IT CAN ALSO OCCUR WHEN A DATA FILE IS AUTOMATICALLY EXTENDED
DURING $-, OPERATIONS
Event Meaning
7E PREVIOUSLY STATED THAT THIS EVENT SEEMED OUT OF PLACE IN THE 5SER )/ CLASS (OWEVER IF IT OCCURS
BECAUSE OF $-, OPERATIONS IT MAKES SENSE TO HAVE IT IN THIS CATEGORY !T ANY RATE OFFLOADING THE BLOCK
FORMATTING ELIMINATES #05 USAGE AND )/ FROM THE DATABASE SERVERS AND MOVES IT TO THE STORAGE TIER
7HEN THIS OCCURS TIME IS COLLECTED IN THE RT[[b\PacUX[TRaTPcX^] EVENT 4HIS EVENT REPLACES THE 3PcP
UX[TX]XcfaXcT EVENT THAT IS STILL USED ON NON
%XADATA PLATFORMS (ERE IS THE OUTPUT OF A QUERY FROM A
BUSY PRODUCTION SYSTEM SHOWING OPERATIONS THAT GENERATED THIS EVENT
BHB/4G31 -bT[TRcTeT]c^_TaPcX^]R^d]cUa^\
!bT[TRcb`[NXSTeT]cb`[N_[P]N^_TaPcX^]kkkkb`[N_[P]N^_cX^]b^_TaPcX^]
"Ua^\310N78BCN02C8E4NB4BBN78BC>AH
#fWTaTTeT]c[XZTRT[[b\PacUX[T
$Va^d_Qh^_TaPcX^]TeT]c
%^aSTaQh !"
&
4E4=C>?4A0C8>=2>D=C
RT[[b\PacUX[TRaTPcX^]"$
34;4C4"
8=34G1D8;3=>=D=8@D4$
#(!04%2 %8!$!4! 7!)4 %6%.43
;>030BB4;42C"
;>03C01;42>=E4=C8>=0;
D?30C4
9OULL NOTICE THAT ON THIS PARTICULAR SYSTEM THE RT[[b\PacUX[TRaTPcX^] EVENT WAS OCCASIONALLY
GENERATED BY A 34;4C4 STATEMENT 4HE FACT THAT A 34;4C4 COULD CAUSE THIS EVENT MIGHT BE A LITTLE
SURPRISING "UT REMEMBER THAT THIS EVENT IS ACTUALLY TIMING A SECTION OF CODE THAT DOES BLOCK FORMATTING
NOT FILE CREATION
Parameters
4HE ONLY PARAMETER OF INTEREST FOR THIS EVENT IS ? WHICH SHOWS WHICH CELL WAS BEING ACCESSED WHEN THIS
EVENT WAS GENERATED
?
#ELL HASH NUMBER
?!
.OT USED
?"
.OT USED
CELL STATISTICS GATHER
4HE RT[[bcPcXbcXRbVPcWTa EVENT RECORDS TIME SPENT READING FROM VARIOUS E AND G TABLES !LTHOUGH
THE EVENT IS GROUPED IN THE 5SER )/ CATEGORY IT DOES NOT REFER TO )/ IN THE SENSE OF READING AND WRITING
TO AND FROM DISK
Event Meaning
4IME IS CLOCKED TO THIS EVENT WHEN A SESSION IS READING FROM E24;; E24;;NC7A403N78BC>AH
G:258B>BB= E24;;NA4@D4BCNC>C0;BG:258B>BBC AND A FEW OTHER G TABLES IN THE SAME FAMILY 4HE
EVENT IS MISCATEGORIZED IN OUR OPINION AND DOES NOT REALLY BELONG IN THE 5SER )/ CATEGORY
Parameters
4HE PARAMETERS FOR THIS EVENT PROVIDE NO ADDITIONAL INFORMATION )N FACT VALUES ARE NOT EVEN SET FOR THE
PARAMETERS IN THIS EVENT (ERE ARE THE PARAMETER DEFINITIONS
?
#ELL HASH NUMBER ALWAYS
?! .OT USED
?" .OT USED
Exadata Wait Events in the System I/O Class
4HE %XADATA WAIT EVENTS THAT ARE ASSIGNED TO THE 3YSTEM )/ CLASS ARE OF LESS IMPORTANCE AND DO NOT
GENERALLY SHOW UP AS MAJOR CONSUMERS OF TIME 4HE BACKUP EVENTS ARE THE MOST INTERESTING AS THEY
RECORD TIME FOR SECTIONS OF CODE THAT HAVE BEEN OPTIMIZED ON THE %XADATA PLATFORM 4HE OTHERS ARE
#(!04%2 %8!$!4! 7!)4 %6%.43
SIMPLY HOUSEKEEPING EVENTS 4HE NON BACKUP EVENTS ARE LISTED IN 4ABLE WHILE THE BACKUP EVENTS
ARE DETAILED IN THE FOLLOWING SECTIONS
Table 10-1. Miscellaneous System I/O Class Events
Event Description
RT[[\P]PVTaR[^bX]VRT[[ 4HIS IS A SHUTDOWN
RELATED EVENT 4HE CELL HASH NUMBER IS CONTAINED IN
THE ? COLUMN OF THE ebTbbX^]NfPXc VIEW FOR THIS EVENT 4HE ?! AND ?"
COLUMNS ARE UNUSED
RT[[\P]PVTaSXbR^eTaX]VSXbZb 4HIS IS A STARTUP RELATED EVENT 4HE CELL HASH NUMBER IS CONTAINED IN THE
? COLUMN OF THE ebTbbX^]NfPXc VIEW FOR THIS EVENT 4HE ?! AND ?"
COLUMNS ARE UNUSED
RT[[\P]PVTa^_T]X]VRT[[ 4HIS IS A STARTUP RELATED EVENT 4HE CELL HASH NUMBER IS CONTAINED IN THE
? COLUMN OF THE ebTbbX^]NfPXc VIEW FOR THIS EVENT 4HE ?! AND ?"
COLUMNS ARE UNUSED
CELL SMART INCREMENTAL BACKUP
4HIS EVENT IS USED TO MEASURE TIME SPENT WAITING ON 2-!. WHEN DOING AN )NCREMENTAL ,EVEL BACKUP
%XADATA OPTIMIZES INCREMENTAL BACKUPS BY OFFLOADING MUCH OF THE PROCESSING TO THE STORAGE TIER 4HIS
NEW WAIT EVENT WAS ADDED TO ACCOUNT FOR TIME SPENT WAITING ON THE OPTIMIZED INCREMENTAL BACKUP
PROCESSING THAT IS OFFLOADED TO THE STORAGE CELLS
Event Meaning
)NTERESTINGLY AN )NCREMENTAL ,EVEL BACKUP DOES NOT RESULT IN THIS WAIT EVENT EVEN THOUGH THE WORD
hINCREMENTALv IS IN THE 2-!. COMMAND 4HAT IS BECAUSE THE ,EVEL BACKUP DOESNT ACTUALLY DO AN
INCREMENTAL BACKUP AT ALL )T GENERATES A FULL BACKUP THAT IS FLAGGED AS A BASELINE FOR FUTURE INCREMENTAL
BACKUPS (ERES AN EXCERPT FROM A TRACE FILE GENERATED ON A PROCESS THAT WAS DOING AN INCREMENTAL
,EVEL BACKUP
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P,!&RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P,$#RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P, (RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P,% "RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P, ! RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P,"&'UX[T, Q[^RZ, Q[^RZb, ^QY,
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P,"&'UX[T, Q[^RZ, Q[^RZb, ^QY,
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P,!$#UX[T, Q[^RZ, $Q[^RZb, ^QY,
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P,""!UX[T, Q[^RZ, &Q[^RZb, ^QY,
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P,!%'UX[T, Q[^RZ," (Q[^RZb, ^QY,
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P, $&(#UX[T, Q[^RZ," &Q[^RZb, ^QY,
F08C )]P\,R^]ca^[UX[T_PaP[[T[faXcTT[P,"$ UX[Tb,!Q[^RZ," aT`dTbcb,!^QY,
F08C )]P\,R^]ca^[UX[T_PaP[[T[faXcTT[P,# #UX[Tb,!Q[^RZ, 'aT`dTbcb,!^QY,
F08C )]P\,R^]ca^[UX[T_PaP[[T[faXcTT[P,! UX[Tb,!Q[^RZ, %aT`dTbcb,!^QY,
F08C )]P\,R^]ca^[UX[T_PaP[[T[faXcTT[P,!(#UX[Tb,!Q[^RZ, aT`dTbcb,!^QY,
#(!04%2 %8!$!4! 7!)4 %6%.43
F08C )]P\,R^]ca^[UX[TbT`dT]cXP[aTPST[P,(##UX[T, Q[^RZ, Q[^RZb, ^QY,
F08C )]P\,:BE\PbcTafPXcT[P, ""$!_ , _!, _", ^QY, cX\, !'$$ '$'%##!&&$
F08C )]P\,0B<UX[T\TcPSPcP^_TaPcX^]T[P,"!\bV^_,# [^R], _", ^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,"""R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,3XbZUX[T^_TaPcX^]b8 >T[P,%&5X[T>_TaPcX^],"UX[T]^, UX[Tch_T, ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P,"(!RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P,($RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P, %(RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P, #RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P, !'RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P, $RT[[WPbW," &%$(## (_!, _", ^QY,
F08C )]P\,RT[[b\PacX]RaT\T]cP[QPRZd_T[P,&!!RT[[WPbW," &%$(## (_!, _", ^QY,
Parameters
4HE ONLY PARAMETER USED FOR THIS EVENT IS ? WHICH SHOWS WHICH CELL WAS RESPONSIBLE FOR GENERATING THE
EVENT
?
#ELL HASH NUMBER
?!
.OT USED
?"
.OT USED
^QY
.OT USED
■ .OTE 4HE OBJ FIELD IS A PART OF MANY WAIT EVENTS EVEN SOME THAT ARE NOT SPECIFICALLY RELATED TO AN INDIVIDUAL
OBJECT "E AWARE THAT IN SOME CASES THE VALUE MAY BE SET BY ONE EVENT AND THEN NOT CLEARED APPROPRIATELY WHEN
THE WAIT ENDS RESULTING IN MEANINGLESS VALUES LEFT IN PLACE FOR THE NEXT WAIT EVENT )N THE PREVIOUS EXAMPLE THE
OBJ WAS CLEARED SET TO A VALUE OF
CELL SMART RESTORE FROM BACKUP
4HIS EVENT IS USED TO MEASURE TIME SPENT WAITING ON 2-!. WHEN DOING A RESTORE %XADATA OPTIMIZES
2-!. RESTORES BY OFFLOADING PROCESSING TO THE STORAGE CELLS
Event Meaning
4HE EVENT ACTUALLY RECORDS TIME RELATED TO FILE INITIALIZATION DURING A RESTORE (ERES AN EXCERPT FROM A
TRACE FILE TAKEN WHILE A RESTORE WAS IN PROGRESS
CWTX]_dcQPRZd__XTRT
A42> cTbc QPRZd_bTc ! N (N!% ]]]S] NcPV! (!%c 'N !%" &" &!$ &(XbX]10B82 !
R^\_aTbbTSU^a\Pc
#(!04%2 %8!$!4! 7!)4 %6%.43
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,('R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,&$&R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,(#R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P,! R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P,!!R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P, $R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P, R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P, #R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P, "%#R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P, %R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P," R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P,#&'"R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,$R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,A<0=QPRZd_aTR^eTah8 >T[P,!($!R^d]c, X]ca,!$%cX\T^dc,! #&#'"%#&
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P, #R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,""R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,%!R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P,(!R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,ZUZ)Pbh]RSXbZ8>T[P, !R^d]c, X]ca, cX\T^dc,#!(#(%&!($^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P,#"!#(RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, ' # RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P,#" RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P,"&'RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, $RT[[WPbW,! ""#$(#'"_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, "$$RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P,&%%RT[[WPbW,! ""#$(#'"_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, %&RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, ((RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, (RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P,!!%RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, !&RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, RT[[WPbW,! ""#$(#'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, &&RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C )]P\,RT[[b\PacaTbc^aTUa^\QPRZd_T[P, % RT[[WPbW,! ""#$(#'_!, _", ^QY,
Parameters
4HE ONLY PARAMETER USED FOR THIS EVENT IS 0 WHICH SHOWS WHICH CELL WAS RESPONSIBLE FOR GENERATING THE
EVENT
?
#ELL HASH NUMBER
?!
.OT USED
?"
.OT USED
#(!04%2 %8!$!4! 7!)4 %6%.43
Exadata Wait Events in the Other and Idle Classes
4HESE ARE RELATIVELY MINOR EVENTS THAT OCCUR PRIMARILY DURING STARTUP AND SHUT DOWN OF STORAGE CELLS AND
FAULT CONDITIONS 9OU WILL PROBABLY NOT SEE THEM ON NORMALLY FUNCTIONING SYSTEMS 4HERE IS ONE
EXCEPTION TO THIS THE RT[[b\PacU[PbWd]ZTT_ EVENT 4ABLE LISTS THE hCELLv WAIT EVENTS IN THE /THER
CLASS ALONG WITH THEIR PARAMETERS ! SEPARATE SECTION WILL COVER RT[[b\PacU[PbWd]ZTT_
Table 10-2. Miscelaneous Other and Idle Class Events
Event Description
RT[[\P]PVTaRP]RT[f^aZaT`dTbc 4HIS EVENT IS NOT VERY INFORMATIVE AS ALL THREE OF THE PARAMETERS
? ?! ?" FROM THE ebTbbX^]NfPXc VIEW ARE UNUSED
RT[[f^aZTa^][X]TR^\_[TcX^] 4HIS APPEARS TO BE A STARTUP EVENT 4HE CELL HASH NUMBER IS
CONTAINED IN THE ? COLUMN OF THE ebTbbX^]NfPXcVIEW FOR THIS
EVENT 4HE ?! AND ?" COLUMNS ARE UNUSED
RT[[f^aZTaaTcah 4HE CELL HASH NUMBER IS CONTAINED IN THE ? COLUMN OF THE
ebTbbX^]NfPXc VIEW FOR THIS EVENT 4HE ?! AND ?" COLUMNS ARE
UNUSED
RT[[f^aZTaXS[T 4HE ? 0 AND 0 COLUMNS FROM THE ebTbbX^]NfPXc VIEW ARE
UNUSED IN THIS IDLE EVENT
CELL SMART FLASH UNKEEP
4HIS EVENT RECORDS THE TIME SPENT WAITING WHEN /RACLE MUST FLUSH BLOCKS OUT OF %XADATA 3MART &LASH
#ACHE 4HIS CAN OCCUR WHEN A TABLE THAT HAS A STORAGE CLAUSE DESIGNATING THAT IT BE PINNED IN %XADATA
3MART &LASH #ACHE IS TRUNCATED OR DROPPED
Event Meaning
4RUNCATING AND DROPPING TABLES DOESNT GENERALLY HAPPEN VERY OFTEN IN MOST PRODUCTION SYSTEMS WITH
SOME NOTABLE EXCEPTIONS AND THEREFORE THIS WAIT EVENT WILL PROBABLY NOT BE SEEN VERY OFTEN (ERES AN
EXCERPT FROM A TRACE FILE SHOWING THE EVENT
F08C#)]P\,T]`)A> UPbc^QYTRcaTdbTT[P,"("]P\Tk\^ST, "' ( ( %!!,%$$' , ^QY,
F08C#)]P\,aT[XPQ[T\TbbPVTT[P, $#'RWP]]T[R^]cTgc,%& !!& '&!RWP]]T[WP]S[T,%%(%' &'$%
Qa^PSRPbc\TbbPVT,%&# "#!('#^QY, cX\, !'%! '(!%#! ( %
F08C#)]P\,T]`)A> UPbc^QYTRcaTdbTT[P,! !&]P\Tk\^ST, "' ( (!,%$$' , ^QY,
F08C#)]P\,VTb\TbbPVTQdUUTaP[[^RPcX^]T[P,!_^^[, aT`dTbc, P[[^RPcTS, ^QY,
F08C#)]P\,T]`)A> UPbc^QYTRcaTdbTT[P, $(]P\Tk\^ST, "' ( ( $&!,%$$' ,!^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P,""%RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P,!( RT[[WPbW,! ""#$(#'"_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P," (RT[[WPbW," &%$(## (_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P,# 'RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P, RT[[WPbW,! ""#$(#'"_!, _", ^QY,
#(!04%2 %8!$!4! 7!)4 %6%.43
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P,! %RT[[WPbW," &%$(## (_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P,!!!RT[[WPbW,"&(""(($'_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P, (%RT[[WPbW,! ""#$(#'"_!, _", ^QY,
F08C#)]P\,RT[[b\PacU[PbWd]ZTT_T[P,! %RT[[WPbW," &%$(## (_!, _", ^QY,
.OTE THAT THE RT[[b\PacU[PbWd]ZTT_ EVENT IS PRECEDED BY A HANDFUL OF T]`)A>~UPbc^QYTRc
aTdbT EVENTS WHICH ARE USED TO MARK TIME ASSOCIATED WITH CLEANING UP THE BUFFER CACHE AFTER A DROP OR
TRUNCATE 4HE RT[[b\PacU[PbWd]ZTT_ IS BASICALLY AN EXTENSION OF THAT EVENT TO CLEAN UP THE %XADATA
3MART FLASH CACHE ON THE STORAGE SERVER AS WELL
Parameters
4HE ONLY PARAMETER USED FOR THIS EVENT IS ? WHICH SHOWS WHICH CELL WAS RESPONSIBLE FOR GENERATING THE
EVENT
?
#ELL HASH NUMBER
?!
.OT USED
?"
.OT USED
Old Events
)N ADDITION TO THE NEW CELL EVENTS THERE ARE ALSO A FEW NON
%XADATA
SPECIFIC WAIT EVENTS THAT YOU SHOULD
BE AWARE OF 4HESE ARE EVENTS THAT YOU MAY ALREADY BE FAMILIAR WITH FROM MANAGING /RACLE ON OTHER
PLATFORMS 4HEY HAPPEN TO ALSO BE IMPORTANT IN AN %XADATA ENVIRONMENT SO THEY REPRESENT CASES IN
WHICH YOUR EXISTING KNOWLEDGE AND SKILL CAN CARRY OVER AND STAND YOU IN GOOD STEAD AS YOU MOVE IN TO
MANAGING %XADATA
DIRECT PATH READ
$IRECT PATH READS ARE USED BY /RACLE TO READ DATA DIRECTLY INTO 0'! MEMORY INSTEAD OF INTO THE BUFFER
CACHE 4HEY ARE AN INTEGRAL PART OF %XADATA /FFLOADING BECAUSE 31, PROCESSING CAN ONLY BE OFFLOADED TO
THE STORAGE CELLS WHEN THE DIRECT PATH READ MECHANISM IS USED 4HE SXaTRc_PcWaTPS WAIT EVENT IS
ACTUALLY REPLACED BY THE RT[[b\PaccPQ[TbRP] AND RT[[b\PacX]STgbRP] WAIT EVENTS WHEN A QUERY IS
OFFLOADED (OWEVER THE DIRECT PATH READ MECHANISM IS STILL USED BY THE CODE COVERED BY THOSE NEW WAIT
EVENTS 4HAT IS EITHER THE PLAN MUST INCLUDE A PARALLEL SCAN OR /RACLE MUST DECIDE TO USE THE SERIAL DIRECT
PATH READ MECHANISM
Event Meaning
4HIS EVENT RECORDS TIME THAT /RACLE SPENDS WAITING ON A DIRECT PATH READ TO COMPLETE 9OU SHOULD KNOW
THAT THE DIRECT PATH READ WAIT EVENT CAN BE VERY MISLEADING "OTH THE NUMBER OF EVENTS RECORDED AND THE
TIMINGS ASSOCIATED WITH THEM CAN APPEAR TO BE INACCURATE 4HIS IS DUE TO THE FACT THAT DIRECT PATH READS
ARE DONE IN AN ASYNCHRONOUS AND OVERLAPPING FASHION 3EE -/3 NOTE FOR FURTHER DETAILS ABOUT
WHY TIME RECORDED BY THIS EVENT CAN BE MISLEADING )T ALSO BEARS MENTIONING THAT G2 CONTAINS AN
ENHANCEMENT THAT CAUSES SERIAL DIRECT PATH READS TO OCCUR MORE FREQUENTLY THAN IN PREVIOUS RELEASES 3EE
-/3 .OTE WHICH BRIEFLY MENTIONS THIS CHANGE
#(!04%2 %8!$!4! 7!)4 %6%.43
"Y THE WAY THE SXaTRc_PcWaTPS WAIT EVENT DOES STILL SHOW UP ON THE %XADATA PLATFORM FOR VARIOUS
OPERATIONS BUT GENERALLY NOT FOR FULL TABLE SCANS UNLESS THE TABLE OR PARTITION IS RELATIVELY SMALL
Parameters
4HE PARAMETERS FOR THIS EVENT SHOW YOU EXACTLY WHICH SEGMENT OBJ IS SCANNED AND WHICH SET OF BLOCKS
WERE SCANNED DURING THIS EVENT
?
&ILE NUMBER
?!
&IRST $"!
?"
"LOCK COUNT
^QY
4HE OBJECT NUMBER OF THE TABLE BEING SCANNED
!S MENTIONED IN THE #ELL 3MART 4ABLE 3CAN SECTION THE PARAMETERS CONTAIN SPECIFIC INFORMATION
ABOUT WHICH FILE AND OBJECT ARE BEING ACCESSED 4HE OFFSET INTO THE FILE IS ALSO PROVIDED IN THE 0
PARAMETER ALONG WITH THE NUMBER OF CONTIGUOUS BLOCKS READ IN THE 0 PARAMETER
ENQ +/FAST OBJECT CHECKPOINT
4HE T]`):> EVENT HAS A VERY STRANGE NAME $ONT BE PUT OFF BY THAT 4HE EVENT IS ESSENTIALLY AN ^QYTRc
RWTRZ_^X]c EVENT 4HE E;>2:NCH?4 VIEW DESCRIBES THE +/ LOCK AS FOLLOWS
BHB/4G31 -bT[TRcch_T]P\TSTbRaX_cX^]Ua^\e[^RZNch_T
!fWTaTch_T,:>*
CH?4=0<434B2A8?C8>=
:><d[cX_[T>QYTRc2WTRZ_^X]c2^^aSX]PcTbRWTRZ_^X]cX]V^U\d[cX_[T^QYTRcb
Event Meaning
4HIS EVENT IS USED WHEN A SESSION IS WAITING FOR ALL DIRTY BLOCKS TO BE FLUSHED FROM THE BUFFER CACHE FOR AN
OBJECT PRIOR TO STARTING A SXaTRc_PcWaTPS OR CT[[b\PaccPQ[TbRP] OR RT[[b\PacX]STgbRP] 4HIS
EVENT IS IMPORTANT BECAUSE THE TIME REQUIRED TO DO THE CHECKPOINT MAY OUTWEIGH THE BENEFIT OF THE
DIRECT PATH READS 4HIS IS UNLIKELY ON %XADATA STORAGE THOUGH WHERE THE ADDITIONAL 3MART 3CAN BENEFITS
ARE ONLY ENABLED BY THE DIRECT PATH READ MECHANISM
Parameters
4HE PARAMETERS FOR THIS EVENT ARE NOT OVERLY HELPFUL BUT THE EVENT DOES SHOW WHICH OBJECT IS SCANNED
(ERE ARE THE PARAMETER DEFINITIONS
?
.AME -ODE
?!
.OT USED
?"
.OT USED
^QY
4HE OBJECT NUMBER OF THE OBJECT BEING CHECKPOINTED
#(!04%2 %8!$!4! 7!)4 %6%.43
RELIABLE MESSAGE
4HE aT[XPQ[T\TbbPVT EVENT IS USED TO RECORD TIME SPENT COMMUNICATING WITH BACKGROUND PROCESSES
LIKE THE CHECKPOINT PROCESS #+04 7E HAVE INCLUDED IT HERE BECAUSE OF ITS CLOSE ASSOCIATION WITH THE
T]`):>~UPbc^QYTRcRWTRZ_^X]c EVENT
Event Meaning
4HIS EVENT IS THE PRECURSOR TO THE T]`):>~UPbc^QYTRcRWTRZ_^X]c EVENT AMONG OTHERS 4HE
COMMUNICATION IS DONE USING AN INTER
PROCESS COMMUNICATION CHANNEL RATHER THAN A MORE NORMAL POST
MECHANISM 4HIS COMMUNICATION METHOD ALLOWS THE SENDER TO REQUEST AN !#+ BEFORE IT CONTINUES THUS
THE REASON IT IS CALLED A RELIABLE MESSAGE )T IS GENERALLY A VERY SHORT DURATION EVENT AS IT ONLY RECORDS TIME
FOR COMMUNICATING BETWEEN PROCESSES "OTH THE USERS FOREGROUND PROCESS AND THE CHKPT PROCESS WILL
WAIT ON THIS EVENT AS THEY COMMUNICATE WITH EACH OTHER (ERES AN EXCERPT OF A TRACE FILE SHOWING A
COMPLETE RELIABLE MESSAGE EVENT
?0AB8=68=2DAB>A#%(%"(' ("%&##[T],$ ST_, dXS, ^Rc,"[XS, cX\, " ''$!! &#"$!'
We," "!%!%$##PS,!RU%&$UU b`[XS,&h (SchdR#SQW
bT[TRcPeV_ZNR^[Ua^\Zb^ bZTf!PfWTaTR^[ -
4=3>5BC<C
?0AB4
#%(%"(' ("%&##)R, (((T,! !!_, Ra, Rd, \Xb, a, ST_, ^V, _[W,! &' &( cX\, " ''$!!
&#"$!&
F08C#%(%"(' ("%&##)]P\,reliable messageT[P, "% RWP]]T[R^]cTgc, '''"# &'#RWP]]T[
WP]S[T, ! ''$'$'(%Qa^PSRPbc\TbbPVT, ! '( "&! %^QY,&$&$(cX\, " ''$!! &#$"(&
F08C#%()]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, '" ^QY,&$&$(cX\, "
F08C#%()]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, ## ^QY,&$&$(cX\, "
F08C#%()]P\,T]`)?B R^]cT]cX^]T[P,! ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P,$'"""_ , _!, _", ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P, & $_ , _!, _", ^QY,&$&$(cX\, "
F08C#%()]P\,T]`)?B R^]cT]cX^]T[P,!&% ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P, !(''_ , _!, _", ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P, "#($_ , _!, _", ^QY,&$&$(cX\, "
F08C#%()]P\,T]`)?B R^]cT]cX^]T[P,!"& ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P, !%$$_ , _!, _", ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P, !"!(_ , _!, _", ^QY,&$&$(cX\, "
F08C#%()]P\,T]`)?B R^]cT]cX^]T[P," " ^QY,&$&$(cX\, "
F08C#%()]P\,^bcWaTPSbcPacd_T[P, !%&"_ , _!, _", ^QY,&$&$(cX\, "
Parameters
(ERE ARE THE PARAMETERS FOR THE RELIABLE MESSAGE EVENT
?
CHANNEL CONTEXT
?!
CHANNEL HANDLE
?"
BROADCAST MESSAGE
^QY
4HE OBJECT NUMBER OF THE OBJECT OF INTEREST NOT ALWAYS SET
#(!04%2 %8!$!4! 7!)4 %6%.43
Resource Manager Events
&INALLY THERE ARE A FEW 2ESOURCE -ANAGER EVENTS THAT YOU SHOULD BE AWARE OF 7HILE THESE ARE NOT
SPECIFIC TO %XADATA 2ESOURCE -ANAGER PROVIDES KEY FUNCTIONALITY FOR COMBINING MIXED WORKLOADS ON
%XADATA 4HERE ARE ACTUALLY EIGHT SEPARATE EVENTS AS OF RELEASE 4HE FOLLOWING QUERY AGAINST
6%6%.4?.!-% SHOWS THESE EVENTS AND THEIR PARAMETERS
BHB/B0=31>G -/eP[XSNTeT]cb
4]cTaeP[dTU^aTeT]cN]P\T)aTb\Va
4E4=C2;0BB? ?!?"
aTb\Va)X]cTa]P[bcPcTRWP]VT2^]RdaaT]Rh[^RPcX^]
aTb\Va)bTbbX^]bc^TgXc[^RPcX^]
aTb\Va)X]cTa]P[bcPcTR[TP]d_>cWTa[^RPcX^]
aTb\Va)QTR^\TPRcXeTBRWTSd[Ta[^RPcX^]
aTb\Va)R_d`dP]cd\[^RPcX^]R^]bd\TaVa^d_XS
aTb\Va)[PaVT8 >`dTdTS[^RPcX^]
aTb\Va)_``dTdTS[^RPcX^]
aTb\Va)b\P[[8 >`dTdTS[^RPcX^]
'a^fbbT[TRcTS
4HERE ARE ONLY TWO OF THESE EVENTS THAT ARE OF INTEREST
RESMGRCPU QUANTUM
4HIS EVENT IS USED TO RECORD FORCED IDLE TIME IMPOSED BY $ATABASE 2ESOURCE -ANAGER $"2- DUE TO
COMPETITION WITH HIGHER PRIORITY WORK 3AID ANOTHER WAY IT IS THE TIME A PROCESS SPENT WAITING FOR $"2-
TO ALLOCATE IT A TIME SLICE
Event Meaning
$"2- BEHAVES IN AN ANALOGOUS MANNER TO #05 SCHEDULING ALGORITHMS IN THAT IT DIVIDES TIME INTO UNITS
QUANTUM AND EITHER ALLOWS A PROCESS TO RUN OR NOT DEPENDING ON OTHER WORKLOAD ON THE SYSTEM 5NLIKE
#05 SCHEDULING ALGORITHMS THOUGH $"2- THROTTLING IS INTERJECTED AT KEY LOCATIONS IN THE /RACLE CODE TO
ELIMINATE THE POSSIBILITY OF A PROCESS BEING KICKED OFF OF THE #05 WHEN IT IS HOLDING A SHARED RESOURCE
SUCH AS A LATCH 4HIS PREVENTS SOME NASTY BEHAVIOR THAT MAY OCCUR ON HEAVILY LOADED SYSTEMS SUCH AS
PRIORITY INVERSION PROBLEMS )N EFFECT THE PROCESSES VOLUNTARILY GO TO SLEEP WHEN THEY ARE NOT HOLDING
THESE SHARED RESOURCES 4HERE ARE MULTIPLE LOCATIONS IN THE CODE WHERE THESE CHECKS ARE IMPLEMENTED
(ERES AN EXCERPT OF A TRACE FILE SHOWING THE aTb\Va)R_d`dP]cd\ EVENT
#(!04%2 %8!$!4! 7!)4 %6%.43
?0AB8=68=2DAB>A#& #% &" ## [T],"#ST_, dXS,'$^Rc,"[XS,'$cX\, " !&$"(""!"# #
We,"" '(% !"'PS,Q"R(UPU'b`[XS,R#Yb $i!\_UVU
bT[TRcR^d]c
Ua^\Zb^ bZTf
4=3>5BC<C
?0AB4#& #% )R, T,&&_, Ra, Rd, \Xb, a, ST_, ^V, _[W,("&%'& # cX\, "
4G42#& #% )R, T,#(_, Ra, Rd, \Xb, a, ST_, ^V, _[W,("&%'& # cX\, "
F08C#& #% )]P\,B@;=Tc\TbbPVTc^R[XT]cT[P,#SaXeTaXS, %$ ' $!"!QhcTb,
F08C#& #% )]P\,aTb\Va)R_d`dP]cd\T[P,"#$&[^RPcX^],"R^]bd\TaVa^d_XS,&$$!$
F08C#& #% )]P\,aTb\Va)R_d`dP]cd\T[P,%' !%[^RPcX^],"R^]bd\TaVa^d_XS,&$$!$
F08C#& #% )]P\,aTb\Va)R_d`dP]cd\T[P,%' #%[^RPcX^],"R^]bd\TaVa^d_XS,&$$!$
F08C#& #% )]P\,aTb\Va)R_d`dP]cd\T[P, '(# [^RPcX^],"R^]bd\TaVa^d_XS,&$$!$
F08C#& #% )]P\,aTb\Va)R_d`dP]cd\T[P,"'!"#[^RPcX^],"R^]bd\TaVa^d_XS,&$$!$
F08C#& #% )]P\,aTb\Va)R_d`dP]cd\T[P, "!'![^RPcX^],"R^]bd\TaVa^d_XS,&$$!$
Parameters
(ERE ARE THE PARAMETERS FOR THIS EVENT .OTE THAT THE OBJ PARAMETER EXISTS BUT IS NOT USED
?
,OCATION
?!
#ONSUMER GROUP ID
?"
.OT USED
^QY
.!
4HE LOCATION PARAMETER IS A NUMERIC VALUE THAT MOST LIKELY REFERS TO A LOCATION FUNCTION IN THE /RACLE
CODE 4HERE ARE AT LEAST DISTINCT LOCATIONS THAT WE HAVE OBSERVED 5NFORTUNATELY /RACLE DOES NOT
PUBLICLY DOCUMENT WHERE IN THE /RACLE KERNEL THESE CHECKS ARE PERFORMED
4HE CONSUMER GROUP NUMBER IN THE 0 PARAMETER IS PRETTY SELF
EXPLANATORY )T MAPS TO THE
#/.35-%2?'2/50?)$ COLUMN IN THE $"!?232#?#/.35-%2?'2/503 VIEW 4HIS PARAMETER ALLOWS
YOU TO TELL WHAT CONSUMER GROUP A PROCESS WAS ASSIGNED TO WHEN ITS #05 USAGE WAS CURTAILED
RESMGRPQ QUEUED
4HIS EVENT IS USED TO RECORD TIME SPENT WAITING IN THE PARALLEL QUERY QUEUE
Event Meaning
4HE PARALLEL STATEMENT QUEUING FEATURE COMES WITH ITS OWN WAIT EVENT 3TATEMENTS THAT ARE QUEUED DUE
TO INSUFFICIENT PARALLEL SERVER PROCESSES OR OTHER DIRECTIVES CLOCK TIME TO THIS EVENT (ERES AN EXCERPT OF A
TRACE FILE SHOWING THE aTb\Va)_``dTdTS EVENT
?0AB8=68=2DAB>A#&'('#"% ! [T],&"ST_, dXS, ^Rc,"[XS, cX\, " (%% &!""!%(#
We," '%#' % &PS,!RS!S RQ'b`[XS,QVR\\Rhhhe_V(
bT[TRc _PaP[[T[P % PeV_ZNR^[Ua^\Zb^ bZTf"PfWTaTR^[ -
4=3>5BC<C
?0AB4#&')R,! T, $&!_, Ra, Rd, \Xb, a, ST_, ^V, _[W, # #$' & cX\, "
F08C#&')]P\,'resmgr:pq queued'T[P,%$ ! #&[^RPcX^], , , ^QY,$!"cX\, "
F08C#&')]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P,!$' ^QY,$!"
#(!04%2 %8!$!4! 7!)4 %6%.43
F08C#&')]P\,aT[XPQ[T\TbbPVTT[P,! $RWP]]T[R^]cTgc, 'RWP]]T[WP]S[T, !
^QY,$!"
F08C#&')]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, $% ^QY,$!"
F08C#&')]P\,?G3T`)9^X]02:T[P,&&!b[TT_cX\T bT]STaXS,!%'_PbbTb, _", ! ^QY,$!"
F08C#&')]P\,?G3T`)9^X]02:T[P,("!b[TT_cX\T bT]STaXS,!%'_PbbTb,'_", ! ^QY,$!"
F08C#&')]P\,?G3T`)9^X]02:T[P,!b[TT_cX\T bT]STaXS, _PbbTb, _", ^QY,$!"
Parameters
(ERE ARE THE PARAMETERS FOR THE aTb\Va)_``dTdTS EVENT
?
,OCATION
?!
.OT USED
?"
.OT USED
^QY
.!
4HE LOCATION PARAMETER IS A NUMERIC VALUE THAT MOST LIKELY REFERS TO A LOCATION FUNCTION IN THE /RACLE
CODE AS DESCRIBED ABOVE IN THE aTb\Va)R_d`dP]cd\ EVENT
Summary
4HE WAIT INTERFACE HAS BEEN EXPANDED TO COVER SEVERAL %XADATA
SPECIFIC FEATURES )N THIS CHAPTER WEVE
COVERED THE NEW WAIT EVENTS "Y FAR THE MOST INTERESTING OF THE NEW EVENTS ARE RT[[b\PaccPQ[TbRP]
AND RT[[b\PacX]STgbRP] 4HESE EVENTS COVER THE TIME SPENT WAITING ON AN OFFLOADABLE )/ REQUEST TO
A STORAGE CELL 4HERE IS A LOT OF PROCESSING THAT OCCURS AT THE STORAGE LAYER THAT IS LUMPED TOGETHER UNDER
THESE EVENTS )TS IMPORTANT TO UNDERSTAND THAT THESE EVENTS REPLACE THE SXaTRc_PcWaTPS EVENT AND THAT
THE MECHANISM OF RETURNING THE DATA DIRECTLY TO THE PROCESS 0'! EMPLOYED BY THE 3MART 3CAN EVENTS IS
ANALOGOUS TO THE WAY IT IS HANDLED BY SXaTRc_PcWaTPS
C H A P T E R 11
Understanding Exadata
Performance Metrics
/RACLE %XADATA IS A BIG STEP FORWARD FROM THE TRADITIONAL DATABASE SERVER ARCHITECTURE HOWEVER IT IS STILL
RUNNING THE STANDARD /RACLE $ATABASE SOFTWARE -OST OF THE USUAL DATABASE PERFORMANCE RULES STILL
APPLY WITH THE ADDITION OF SOME THAT RECOGNIZE THE ADVANTAGE OF %XADATA FUNCTIONALITY LIKE 3MART 3CANS
CELL JOIN FILTERING AND THE FLASH CACHE )N THIS CHAPTER WE WILL COVER THE %XADATA
SPECIFIC AND
RELATED
PERFORMANCE TOPICS METRICS AND SOME RELEVANT INTERNALS
4HANKFULLY /RACLE BOTH AT THE DATABASE LAYER AND IN CELLS PROVIDES LOTS OF PERFORMANCE METRICS FOR
OUR USE (OWEVER WHEN LOOKING INTO ANY METRIC YOU SHOULD KNOW why YOU ARE MONITORING THIS AND
WHAT NUMBERS ARE YOU LOOKING FOR )N OTHER WORDS HOW DO YOU KNOW WHEN EVERYTHING IS /+ AND NO
ACTION IS NEEDED AND WHEN THINGS ARE BAD AND ACTION IS NEEDED !ND IN ORDER TO AVOID WASTING EFFORT ON
FIXING THE WRONG PROBLEM WE REALLY NEED TO MEASURE what matters &OR DATABASE PERFORMANCE NOTHING
MATTERS MORE THAN response time THE ACTUAL TIME THE END USER OR CONNECTED SYSTEM HAS TO WAIT FOR THE
RESPONSE 3O IF WE WANT TO MAKE SOMETHING FASTER IN THE DATABASE WE SHOULD FOCUS ON MEASURING AND
THEN OPTIMIZING THE RESPONSE TIME !LL THE OTHER METRICS AND INDICATORS LIKE THE NUMBER OF )/S OR CACHE
HITS ARE SECONDARY %ND USERS WHO ARE WAITING FOR THEIR REPORT CARE ABOUT THE TIME THEY HAVE TO WAIT
ONLY NOT SECONDARY METRICS LIKE #05 UTILIZATION OR )/ RATE .EVERTHELESS OFTEN THESE SECONDARY METRICS
BECOME VERY USEFUL FOR UNDERSTANDING AND EXPLAINING PERFORMANCE ISSUES
4HE KEY METRICS FOR BREAKING DOWN DATABASE response time ARE THE /RACLE WAIT INTERFACES WAIT
EVENTS 4HE WAIT EVENTS ARE DISCUSSED IN #HAPTER AND WE WILL LOOK INTO PERFORMANCE MONITORING TOOLS
THAT MAKE USE OF THEM IN THE NEXT CHAPTER (OWEVER THERE ARE ADDITIONAL USEFUL METRICS %XADATA
PROVIDES SUCH AS THE NUMBER OF BYTES OF DATA RETURNED BY SMART SCANS AND THE ACTUAL AMOUNT OF )/
AVOIDED THANKS TO STORAGE INDEXES AND SO ON 3UCH METRICS GIVE VERY IMPORTANT ADDITIONAL INFO ABOUT
WHATS HAPPENING IN THE DATABASE AND STORAGE CELLS DURING 31, EXECUTION )N THIS CHAPTER WE WILL
EXAMINE THESE METRICS AND YOULL LEARN HOW TO GET THEM AND WHAT THEY MEAN %VEN IF YOU ARE NOT
INTERESTED IN KNOWING WHAT EACH METRIC MEANS WE STILL RECOMMEND YOU READ THIS CHAPTER AS IT EXPLAINS
SOME IMPORTANT INTERNALS AND DESIGN DECISIONS BEHIND %XADATA
4HE INFORMATION IN THIS CHAPTER SHOULD GIVE YOU A GOOD UNDERSTANDING ABOUT SOME KEY INTERNAL
WORKINGS OF %XADATA DATABASES AND CELLS AND PREPARE YOU FOR THE NEXT CHAPTER WHERE WE WILL PUT THIS
KNOWLEDGE TO USE WHEN MONITORING AND TROUBLESHOOTING %XADATA PERFORMANCE
Measuring Exadata’s Performance Metrics
"EFORE WE START LOOKING AT %XADATA SPECIFIC PERFORMANCE METRICS LETS EXAMINE SOME INTERNALS AND
REVIEW SOME KEY ELEMENTS OF %XADATA SPECIFIC FEATURES AND METRICS /NE MUST UNDERSTAND WHAT THE
PERFORMANCE NUMBERS ACTUALLY STAND FOR BEFORE TRYING TO MONITOR OR OPTIMIZE ANYTHING WITH THIS INFO
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
"Y NOW YOU KNOW THAT %XADATA DATABASE NODES DONT DO PHYSICAL DISK )/ THEMSELVES BUT ASK THE
CELLS TO DO THE )/ FOR THEM )N THE CASE OF SMART SCANS THE CELL SERVERS WILL ALSO PROCESS THE BLOCKS READ
EXTRACT THEIR CONTENTS FILTER ROWS AND SO ON 3O CONCEPTUALLY THE CELLS ARE KIND OF A BLACK BOX WHEN
VIEWED FROM DATABASE LAYER SIDE 4HE DATABASE LAYER JUST REQUESTS SOME BLOCKS OF DATA AND THE CELLS DO
THE PHYSICAL )/ WORK UNDER THE HOOD AND RETURN THE REQUESTED COLUMNS OF MATCHING ROWS ,UCKILY
/RACLE %XADATA ARCHITECTS AND DEVELOPERS HAVE PUT DECENT INSTRUMENTATION INTO CELLS SO THE CELLS KEEP
TRACK OF HOW MUCH WORK THEY HAVE DONE AND IMPORTANTLY THEY CAN ALSO send the metrics back to the
database layer ALONG THE RESULTS 4HIS ALLOWS US$"!S DEVELOPERS AND TROUBLESHOOTERSTO HAVE AN
END
TO
END OVERVIEW OF WHAT HAPPENED IN THE DATABASE and THE CELLS WHEN SERVICING A USERS REQUEST OR
RUNNING A QUERY
&OR EXAMPLE WHEN A QUERY IS EXECUTED VIA 3MART 3CAN YOU WILL STILL SEE STATISTICS LIKE physical_reads
WHEN YOU QUERY EB@;BC0CB IN THE DATABASE LAYER EVEN THOUGH THE DATABASE LAYER ITSELF DIDNT DO ANY
PHYSICAL READS DIRECTLY !NOTHER EXAMPLE IS THE cell physical IO bytes saved by storage index STATISTIC
WHICH IS COUNTED IN THE CELL LEVEL AND NOT IN THE DATABASE 4HESE NUMBERS ARE VISIBLE IN THE DATABASE
LAYER THANKS TO CELLS SENDING BACK USEFUL PERFORMANCE METRICS IN ADDITION TO THE DATA QUERIED
)N ADDITION TO THE STANDARD /RACLE PERFORMANCE TOOLS WELL USE TWO CUSTOM
BUILT TOOLS IN THIS AND
THE NEXT CHAPTER 4HEY ARE MORE SUITABLE FOR FLEXIBLE AND ADVANCED PERFORMANCE ANALYSIS AND ALLOW YOU
TO GO BEYOND THE STANDARD WAIT EVENTS AND 31, STATEMENT LEVEL STATISTICS 4HE FIRST TOOL /RACLE 3ESSION
3NAPPER IS A SCRIPT CONTAINING JUST AN ANONYMOUS 0,31, BLOCK WHICH MEASURES DETAILED PERFORMANCE
METRICS FROM EB4BB8>= EB4BB8>=N4E4=C EB4BBNC8<4N<>34; EB4BBC0C AND SO ON 4HE LAST
PERFORMANCE VIEW EB4BBC0C IS ESPECIALLY IMPORTANT FOR ADVANCED PERFORMANCE ANALYSISIT CONTAINS
HUNDREDS OF DYNAMIC PERFORMANCE COUNTERS OVER IN /RACLE FOR EACH SESSION IN THE
INSTANCE )N ADDITION TO THE USUAL MONITORING USING WAIT EVENTS DIVING INTO EB4BBC0C GIVES US A MUCH
BETTER IDEA OF WHAT KIND OF WORK /RACLE SESSIONS ARE DOING SUCH AS HOW MANY )/ REQUESTS PER SECOND
THEY ARE DOING HOW MANY FULL SEGMENT SCANS PER SECOND HOW MANY MIGRATEDCHAINED ROWS HAD TO BE
FETCHED DURING A SCAN AND SO ON
Revisiting the Prerequisites for Exadata Smart Scans
)N THIS SECTION WE WILL LOOK AT THE VARIOUS METRICS /RACLE $ATABASE KERNELS INSTRUMENTATION PROVIDES US
7E WILL NOT GO INTO THE DETAILS OF %XADATA WAIT EVENTS HERE AS THESE ARE ALREADY EXPLAINED #HAPTER 7E
WILL REVIEW HOW TO USE THESE WAIT EVENTS FOR UNDERSTANDING DATABASE PERFORMANCE AND YOU MAY FIND
SOME OF THE OFFLOADING AND 3MART 3CAN MATERIAL ALREADY FAMILIAR FROM #HAPTER BUT ITS IMPORTANT TO
REVIEW SOME OF THE CONCEPTS HERE IN THE CONTEXT OF MONITORING AND TROUBLESHOOTING %XADATA
PERFORMANCE
"ECAUSE THE PRIMARY PERFORMANCE BOOSTER FOR DATA WAREHOUSING AND REPORTING WORKLOADS IS THE
%XADATA 3MART 3CAN AND FOR /,40 WORKLOADS IT IS THE USE OF %XADATA 3MART &LASH #ACHE WE WILL SEE FIRST
HOW TO MEASURE WHETHER YOUR WORKLOAD IS BENEFITTING FROM THESE FEATURES 7E WILL USE THIS KNOWLEDGE AS
BUILDING BLOCKS FOR LATER DATABASE AND QUERY PERFORMANCE MONITORING
%XADATA 3MART 3CAN 0ERFORMANCE
,ETS START WITH 3MART 3CAN METRICS "EFORE WE TALK ABOUT ANY METRICS LETS REVIEW HOW THE DECISION TO DO
A 3MART 3CAN IS MADE IN /RACLE .OTE THAT A 3MART 3CAN CAN BE USED ON REGULAR TABLE SEGMENTS AND ALSO
ON MATERIALIZED VIEW SEGMENTS n WHICH ARE PHYSICALLY NO DIFFERENT FROM REGULAR TABLES 3MART 3CANS CAN
ALSO BE USED FOR FULL SCANNING THROUGH "
4REE INDEX SEGMENTS INDEX FAST FULL SCAN AND ALSO BITMAP INDEX
SEGMENTS BITMAP INDEX FAST FULL SCAN 3CANNING THROUGH INDEX SEGMENTS USING THE hBRUTE FORCEv
MULTIBLOCK READS APPROACH IS VERY SIMILAR TO FULL TABLE SCANS 4HE MAJOR DIFFERENCE IS THAT INSIDE INDEX
SEGMENTS THERE ARE ALSO INDEX BRANCH BLOCKS WHICH HAVE TO BE SKIPPED AND IGNORED IN ADDITION TO !33-
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
BITMAP BLOCKS WHICH HAVE TO BE SKIPPED BOTH IN TABLE AND INDEX SCANS 3MART SCANS ON PARTITIONS AND
SUBPARTITIONS OF TABLES AND INDEXES ARE INTERNALLY NO DIFFERENT FROM SCANS ON NONPARTITIONED OBJECTS A
PARTITIONED SEGMENT IS JUST A BUNCH OF SMALLER SEGMENTS GROUPED UNDER THEIR LOGICAL PARENT TABLE OR
INDEX OBJECT 2EMEMBER THAT THE 3MART 3CANS CAN SCAN A VARIETY OF SEGMENTS
2EGARDLESS OF SEGMENT TYPE A 3MART 3CAN ALWAYS REQUIRES DIRECT PATH READS TO BE CHOSEN BY THE 31,
EXECUTION ENGINE DURING RUNTIME THIS IS not AN OPTIMIZER DECISION 3O WHEN TROUBLESHOOTING WHY A
3MART 3CAN WAS NOT USED YOU WILL HAVE TO FIRST CHECK WHETHER DIRECT PATH READS WERE USED OR NOT 9OU
SHOULD CHECK THE EXECUTION PLAN FIRST TO SEE WHETHER A FULL SCAN IS REPORTED THERE AT ALL (ERE YOU SEE
SIMPLE EXAMPLES SHOWING FULL SEGMENT SCANS HAPPENING ON DIFFERENT SEGMENT TYPES
B4;42C0E6[X]T5A><cF74A4^f]Ta;8:4B
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckkkk!($: kk
k kB>AC066A460C4kk k kkk
k!kC01;40224BBBC>A0645D;;kCk$&#":k% <k!($: k )$() k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! bc^aPVT>F=4A;8:4B
UX[cTa>F=4A;8:4B
.OTE THAT THE hBC>A064v IN THE C01;40224BBBC>A0645D;; LINE HERE DOES not MEAN THAT /RACLE IS
ATTEMPTING TO DO A 3MART 3CAN 4HIS WORD MERELY SAYS THAT /RACLE KNOWS THAT THIS SEGMENT IS RESIDING ON
A STORAGE CELL NOT SOMETHING ELSE LIKE AN .&3 MOUNT OR I3#3) DEVICE AND IS USING THE TABLE SCAN
CODEPATH capable of performing A 3MART 3CAN7HETHER A 3MART 3CAN IS ACTUALLY USED DEPENDS ON
MULTIPLE OTHER FACTORS WHICH WERE DISCUSSED IN #HAPTER AND WILL BE ALSO DEMONSTRATED HERE !LSO
Smart Scan IS A BROAD TERM COVERING FILTRATION WHICH MAY BE ABLE TO TAKE ADVANTAGE OF STORAGE INDEXES
COLUMN PROJECTION DECOMPRESSION OF (##
COMPRESSED COMPRESSION UNITS AND HASH JOIN EARLY
ELIMINATION WITH BLOOM FILTERS ALL DONE IN THE CELLS %VEN IF YOU ARE NOT TAKING ADVANTAGE OF THE FILTER
PREDICATE OFFLOADING TO STORAGE FILTRATION THE CELLS MAY BE ABLE TO REDUCE THE AMOUNT OF DATA RETURNED
TO THE DATABASE )F 3MART 3CAN IS USED THEN THE COLUMN PROJECTION IS DONE IN THE CELLS AND THEY RETURN
ONLY THE REQUIRED COLUMNS INSTEAD OF ENTIRE BLOCKS CONTAINING THE FULL
LENGTH ROWS
)N ADDITION IT IS IMPORTANT TO CHECK WHETHER A bc^aPVT PREDICATE IS SHOWN IN THE h0REDICATE
)NFORMATIONv SECTION BELOW THE EXECUTION PLAN 4HIS IS A GOOD INDICATOR OF WHETHER THE EXECUTION PLAN is
capable of DOING A 3MART 3CANS PREDICATE OFFLOAD SMART FILTRATION 5NFORTUNATELY THIS DOESNT MEAN
THAT A 3MART 3CAN PREDICATE OFFLOAD WAS ACTUALLY ATTEMPTED 4HERE ARE CASES WHERE EVEN THE PRESENCE OF A
bc^aPVT PREDICATE DOESNT GUARANTEE THAT A PREDICATE OFFLOAD WILL TAKE PLACE 4HIS IS WHERE /RACLE
PERFORMANCE METRICS WILL BE HELPFUL BUT MORE ABOUT THEM SHORTLY )N SUMMARY A FULL SEGMENT SCAN
ACCESS PATH WITH THE BC>A064 KEYWORD and THE bc^aPVT PREDICATE MUST BE PRESENT IN THE EXECUTION PLAN
IN ORDER FOR IT TO be capable of DOING A 3MART 3CANS PREDICATE OFFLOAD AT ALL )F YOU DO SEE THE BC>A064 LINES
IN EXECUTION PLAN BUT NO bc^aPVT PREDICATES UNDER IT THEN PREDICATE OFFLOAD WONT BE EVEN ATTEMPTED
BUT YOU STILL MAY BENEFIT FROM COLUMN PROJECTION OFFLOAD IN WHICH ONLY THE REQUIRED COLUMNS ARE
RETURNED BY CELLS 9OU WILL PROBABLY SEE MULTIPLE STORAGE PREDICATES AND FULL SCAN OPERATIONS IN REAL
LIFE
QUERY PLANS AS YOULL BE DOING MULTI
TABLE JOINS
)N SUMMARY IF YOU SEE THE BC>A064 OPTION AND bc^aPVT PREDICATE IN AN EXECUTION PLAN IT IS
CAPABLE OF DOING A 3MART 3CAN AND ITS PREDICATE OFFLOAD IN PRINCIPLE BUT ITS NOT GUARANTEED THAT A 3MART
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
3CAN HAPPENS EVERY TIME YOU RUN THE QUERY /N THE OTHER HAND IF YOU DO NOT SEE ABC>A064 KEYWORD IN
THE EXECUTION PLAN THEN THERES NO WAY A 3MART 3CAN COULD HAPPEN ON A CORRESPONDING SEGMENT IN A
GIVEN EXECUTION PLAN STEP 7HEN THERES NO BC>A064 KEYWORD IN THE EXECUTION PLAN LINE IT MEANS THAT
THIS ROWSOURCE OPERATOR IS NOT CAPABLE OF USING THE SMART FEATURES OF THE STORAGE CELLS AND SO WONT BE
ABLE TO PUSH ANY STOaPVT PREDICATES INTO THE CELL EITHER
3MART 3CANS CAN BE DONE ON MATERIALIZED VIEW SEGMENTS TOO A MATERIALIZED VIEW SEGMENT IS
PHYSICALLY EXACTLY THE SAME AS ANY REGULAR TABLE
bT[TRcR^d]cUa^\\e fWTaT^f]Ta[XZTB
k8Sk>_TaPcX^]k=P\Tk4 A^fbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkk "(: k
k kB>AC066A460C4kk kk
k!k<0CNE84F0224BBBC>A0645D;;k<E k! '(:k "(: k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! bc^aPVT>F=4A;8:4B
UX[cTa>F=4A;8:4B
&OLLOWING ARE TWO EXAMPLES WHERE 3MART 3CANS CAN BE ATTEMPTED 4HE FIRST IS WHEN SCANNING
THROUGH A REGULAR "
4REE INDEX SEGMENT
B4;42C 8=34GN55Bc! 0E6;4=6C7^f]Ta5A><c!F74A4^f]Ta;8:4B
k8Sk>_TaPcX^]k=P\Tk4 A^fbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkk$ %$ k
k kB>AC066A460C4kk kk
k!k8=34GBC>A06450BC5D;;B20=kC!N8 k$(&:k$ %$!k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! bc^aPVT>F=4A;8:4B
UX[cTa>F=4A;8:4B
4HE SECOND SCANS A BITMAP INDEX SEGMENT
B4;42C 8=34GN55Bc 0E6;4=6C7^f]Ta5A><c F74A4^f]Ta;8:4B
?[P]WPbWeP[dT)" & $%$!&
k8Sk>_TaPcX^]k=P\Tk4 A^fbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkk$ $ k
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
k kB>AC066A460C4kk kk
k!k18C<0?2>=E4AB8>=C>A>F83Bkk$(&:k$ $ k
k"k18C<0?8=34GBC>A06450BC5D;;B20=k18NC N>F=4Akkk
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
" bc^aPVT>F=4A;8:4B
UX[cTa>F=4A;8:4B
)N BOTH CASES YOU SEE THAT THE SEGMENTS WERE SCANNED USING A fast full scan WHICH IS JUST LIKE A FULL TABLE
SCAN ON THE INDEX SEGMENTS AND THE PRESENCE OF THE BC>A064 OPTION AND bc^aPVT PREDICATES ON THE FULL
SCAN OPERATIONS SHOWS THAT A 3MART 3CAN PREDICATE OFFLOAD CAN BE ATTEMPTED
9OU MAY WONDER WHY WE KEEP SAYING h3MART 3CAN PREDICATE OFFLOAD CAN BE attempted.v )TS BECAUSE
THERE ARE CASES WHERE THE 3MART 3CAN EITHER DOESNT REALLY GET USED OR IS STARTED BUT DOES NOT COMPLETE
DURING RUNTIME 9ES THE EXECUTION PLAN STRUCTURE IS CAPABLE OF USING A 3MART 3CAN BUT WHETHER THE
3MART 3CAN IS ACTUALLY EXECUTED DEPENDS FIRST ON WHETHER A DIRECT PATH READ IS CHOSEN TO SCAN THE
SEGMENT OR NOT 7EVE TALKED ABOUT THIS EARLIER IN THIS CHAPTER AND ALSO IN #HAPTER ON 3MART 3CAN AND
#HAPTER ON PARALLEL EXECUTION -OREOVER EVEN IF A DIRECT PATH READ is CHOSEN and A 3MART 3CAN IS
EXECUTED THEN SOMEWHERE DURING OR AT THE BEGINNING OF 3MART 3CAN EXECUTION A DIFFERENT DECISION
MAY BE MADE 4HIS DEPENDS ON MULTIPLE FACTORS AND WE WILL COVER SOME OF THEM HERE ,UCKILY IN
ADDITION TO EXAMINING THE EXECUTION PLAN WE CAN LOOK AT ADDITIONAL METRICS TO SEE WHATS REALLY
HAPPENING UNDER THE HOOD
7E CAN TRY TO ACHIEVE A 3MART 3CAN ON AN )NDEX /RGANIZED 4ABLE )/4 SEGMENT FIRST TO SEE HOW
VALUABLE THE ADDITIONAL /RACLE METRICS ARE .OTE THAT AS OF THE CURRENT /RACLE VERSION $ATABASE
#ELL /RACLE HAS not IMPLEMENTED 3MART 3CAN FUNCTIONALITY ON )NDEX /RGANIZED 4ABLE
SEGMENTS YET SO THATS WHY THIS IS A GOOD EXAMPLE FOR PRACTICING USING THE 3MART 3CAN RELATED METRICS
#HECK THE EXECUTION PLAN SHOWN HERE IT IS FROM A QUERY USING AN index fast full scan ON AN )NDEX
/RGANIZED 4ABLES INDEX SEGMENT
B4;42C0E6;4=6C7^f]Ta5A><cNX^cF74A4^f]Ta;8:4B
?[P]WPbWeP[dT) &!!## &%(
k8Sk>_TaPcX^]k=P\Tk4 A^fbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkk! k
k kB>AC066A460C4kk kk
k!k8=34G50BC5D;;B20=k?:NCN8>Ck" #("k! k
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
! UX[cTa>F=4A;8:4B
5NLIKE IN PLANS LISTED EARLIER THERE IS NO BC>A064 OPTION LISTED IN THE EXECUTION PLAN ROW SOURCE LINE
AND THERE IS NO bc^aPVT PREDICATE INDICATING AN OPPORTUNITY TO PUSH A FILTER PREDICATE TO THE STORAGE
CELLS 4HIS PLAN IS NOT CAPABLE OF USING ANY 3MART 3CAN FUNCTIONALITY IT WILL DO GOOD OLD BLOCK )/
,OOKING AT ADDITIONAL %XADATA AND CELL METRICS ISNT REALLY NECESSARY AS THE EXECUTION PLAN ITSELF SHOWS
THAT IT DOESNT EVEN USE A 3MART 3CAN
COMPATIBLE ROW SOURCE CODEPATH
5NDERSTANDING %XADATA 3MART 3CAN -ETRICS
AND 0ERFORMANCE #OUNTERS
/NCE YOU HAVE MADE AN /RACLE EXECUTION PLAN USE THE STORAGE
AWARE ROW SOURCES YOU STILL CANNOT BE
SURE ABOUT WHETHER A 3MART 3CAN REALLY IS ATTEMPTED AND WORKS AS YOU EXPECT
/NE WAY TO DETERMINE WHATS HAPPENING IS JUST LOOKING AT THE WAIT EVENTS YOUR SESSION IS WAITING FOR
v CPU usage only: 4HIS SEEMS TO MEAN THAT A BUFFERED DATA ACCESS IS USED NOT DIRECT
PATH AS YOU WONT SEE )/ WAIT EVENTS IF YOU READ THROUGH BUFFER CACHE AND ALL THE
DATA HAPPENS TO BE CACHED
v cell multiblock physical read !PPARENTLY MULTIBLOCK READS ARE USED LOOKS LIKE A
FULL SEGMENT SCAN BUT MULTIBLOCK READ WAITS CAN BE REPORTED ALSO FOR ,/" AND
3ECURE&ILE READ OPERATIONS WHERE IN CASE OF ,/"S THE ,/" CHUNK SIZE IS BIGGER
THAN THE BLOCK SIZE /THERWISE SINGLE
BLOCK READS WOULD BE REPORTED FOR ,/"
ACCESS
v cell single block physical read:– !PPARENTLY SINGLE BLOCK READS ARE USED )F THESE
ARE THE ONLY )/ WAIT EVENTS YOU SEE AND NOT TOGETHER WITH MULTIBLOCK READS THEN
IT APPEARS YOU ARE NOT USING A FULL SEGMENT SCAN AT ALL /F COURSE SOMETIMES SINGLE
BLOCK READS SHOW UP DUE TO OTHER OPERATIONS IN THE EXECUTION PLAN LIKE SOME
INDEX RANGE SCAN OR DUE TO CHAINED ROWS IN DATA BLOCKS
)F YOU SEE REGULAR hCELL MULTIBLOCK PHYSICAL READv WAIT EVENTS INCREMENTED IN YOUR SESSION THEN
DIRECT PATH READS WERE not USED 4HIS MAY HAPPEN MAINLY FOR SERIALLY EXECUTED OPERATIONS AS IF YOU ARE
USING _PaP[[T[NSTVaTTN_^[XRh,<0=D0; OR ;8<8C43 0ARALLEL %XECUTION SLAVES WILL ALWAYS DO DIRECT PATH
READ SCANS WHICH WILL THEN BE OFFLOADED AND EXECUTED AS 3MART 3CANS .EVERTHELESS WHEN YOU ARE USING
THE NEW AUTOMATIC PARALLEL DEGREE POLICY _PaP[[T[NSTVaTTN_^[XRh,0DC> /RACLE MAY DECIDE TO USE
READS THROUGH BUFFER CACHE EVEN FOR PARALLEL OPERATIONS
)N ADDITION TO THESE ISSUES THERE ARE MORE REASONS AND SPECIAL CASES WHERE 3MART 3CANS JUST SILENTLY
ARENT USED OR FALL BACK TO REGULAR BLOCK )/ MODE POTENTIALLY MAKING YOUR QUERIES AND WORKLOAD
SLOWER THAN YOUD EXPECT ,UCKILY /RACLE IS VERY WELL INSTRUMENTED AND WE CAN MEASURE WHAT IS REALLY
HAPPENING IN THE DATABASE AND ALSO THE STORAGE CELLS
Exadata Dynamic Performance Counters
7HILE THE /RACLE WAIT INTERFACES WAIT EVENTS PROVIDE US CRUCIAL INFORMATION ABOUT WHERE THE DATABASE
RESPONSE TIME IS SPENT THE %XADATA DYNAMIC PERFORMANCE COUNTERS DO TAKE US ONE STEP FURTHER AND
EXPLAIN WHAT KIND OF OPERATIONS OR TASKS THE /RACLE KERNEL IS PERFORMING AND HOW MANY OF THEM 7AIT
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
EVENTS AND PERFORMANCE COUNTERS COMPLEMENT EACH OTHER AND SHOULDNT REALLY BE USED ALONE /RACLE
DYNAMIC PERFORMANCE COUNTERS ARE ALSO KNOWN AS EB4BBC0C OREBHBBC0C STATISTICS OR COUNTERS AS
THESE VIEWS ARE USED FOR ACCESSING THEM
7HEN AND (OW TO 5SE 0ERFORMANCE #OUNTERS
7HEN TROUBLESHOOTING PERFORMANCE ONE SHOULD ALWAYS START FROM WAIT EVENTS AND 31,?)$
LEVEL ACTIVITY
MEASUREMENTS AS THEY KEEP TRACK OF THE time, WHICH END USERS CARE ABOUT AND IF ADDITIONAL DETAIL IS
NEEDED THEN PROCEED TO PERFORMANCE COUNTERS )F THESE STANDARD METRICS ARE NOT ENOUGH THE
PERFORMANCE COUNTERS PROVIDE A VERY DETAILED INSIGHT INTO WHAT /RACLE SESSIONS ARE DOING &OR EXAMPLE
IF YOUR SESSION SEEMS TO BE BURNING LOTS OF #05 YOU CAN SEE WHETHER THE session logical reads OR parse
count (hard) COUNTERS INCREASE FOR A SESSION MORE THAN NORMALLY /R IF YOU SEE SOME UNEXPECTED SINGLE
BLOCK READS DURING A FULL TABLE SCAN YOU CAN CHECK WHETHER THE table fetch continued row OR SOME
STATISTIC LIKE data blocks consistent reads – undo records applied INCREASES WHICH INDICATE EITHER A
CHAINEDMIGRATED ROW OR CONSISTENT READ #2 BUFFER CLONING PLUS ROLLBACK OVERHEAD !NOTHER USEFUL
METRIC IS user commits WHICH GIVES YOU AN UNDERSTANDING OF HOW MANY DATABASE TRANSACTIONS ARE DONE
INSIDE AN INSTANCE OR IN CHOSEN SESSIONS 3O THE NEXT TIME A SESSION SEEMS TO BE WAITING FOR A log file
sync WAIT EVENT YOU CAN CHECK ITS user commits COUNTER VALUE FROM EB4BBC0C TO SEE HOW FREQUENTLY THIS
SESSION IS COMMITTING ITS WORK 4HE FOLLOWING EXCERPT IS JUST A SMALL EXAMPLE OF THE KIND OF STATISTICS
/RACLE PROVIDES
B@;-B4;42C]P\TeP[dT5A><ebhbbcPcF74A4]P\T[XZT_PabT*
=0<4E0;D4
_PabTcX\TR_d $#(%
_PabTcX\TT[P_bTS (% (#
_PabTR^d]cc^cP[ $" '
_PabTR^d]cWPaS"
_PabTR^d]cUPX[daTb($"%
_PabTR^d]cSTbRaXQT# %
$YNAMIC PERFORMANCE COUNTERS PROVIDE IMPORTANT CLUES WHICH ALLOW US TO DIRECT OUR
TROUBLESHOOTING EFFORTS BETTER .OTE THAT TOOLS LIKE 3TATSPACK AND !72 REPORTS RELY HEAVILY ON EBHBBC0C
COUNTERS 4HEY JUST STORE VALUES FROM THESE EVER
INCREASING NUMBERS SINCE INSTANCE START IN THEIR
REPOSITORY TABLES 3O WHENEVER YOU RUN A 3TATSPACK!72 REPORT JUST DELTAS BETWEEN VALUES IN CHOSEN
SNAPSHOTS ARE REPORTED 3TATSPACK AND !72 REPORTS ARE ALL ABOUT SHOWING YOU DELTAS BETWEEN EBHBBC0C
AND OTHER VIEWS NUMBERS FROM DIFFERENT SNAPSHOTS OF TIME
/RACLE ADDS NEW COUNTERS INTO EVERY NEW DATABASE VERSION 3OMETIMES NEW DEBUG COUNTERS ARE
INTRODUCED EVEN WITH NEW PATCH SETS /RACLE HAS DIFFERENT PERFORMANCE COUNTERS IN
EBHBBC0C
B@;-B4;42C2>D=C5A><ebhbbcPc*
2>D=C
%!'
4HATS A LOT OF INFORMATION
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
7HILE THE EBHBBC0C VIEW IS FINE FOR MONITORING AND TROUBLESHOOTING INSTANCE
WIDE PERFORMANCE
LIKE !72 AND 3TATSPACK REPORTS DO ITS PROBLEM IS THAT YOU CANT TROUBLESHOOT A SINGLE SESSIONS
PROBLEM WITH SYSTEM
WIDE STATISTICS WHICH AGGREGATE ALL YOUR POTENTIALLY THOUSANDS OF SESSIONS
METRICS TOGETHER INTO ONE SET OF COUNTERS 4HATS WHY /RACLE ALSO HAS EB4BBC0C WHICH KEEPS TRACK OF ALL
THESE COUNTERS FOR EACH SESSION SEPARATELY %VERY SINGLE SESSION IN THE INSTANCE HAS ITS OWN
PERFORMANCE COUNTERS KEEPING TRACK OF ONLY ITS ACTIVITY 4HIS IS A GOLDMINE AS WHEN ONLY A FEW SESSIONS
OR USERS HAVE A PROBLEM IN THE INSTANCE WE CAN MONITOR ONLY THEIR ACTIVITY AND WONT BE DISTRACTED BY
ALL THE OTHER USERS NOISE IN THE DATABASE
!S SAID EARLIER EBHBBC0C ACCUMULATES INSTANCE
WIDE PERFORMANCE COUNTERS THEY START FROM ZERO
AND ONLY INCREASE THROUGHOUT THE INSTANCE LIFETIME -OST OF THE EB4BBC0C COUNTERS ALWAYS INCREASE
CUMULATIVE STATISTICS WITH SOME EXCEPTIONS LIKE logons current AND session pga/uga memory )N ANY
CASE WHEN EXAMINING THE COUNTER VALUES YOU SHOULD NOT JUST LOOK AT THE current VALUE OF A COUNTER
ESPECIALLY IF YOUR SESSION HAS BEEN LOGGED ON FOR A WHILE 4HE PROBLEM IS THAT EVEN IF YOU SEE A BIG
LOOKING NUMBER FOR SOME COUNTER IN EB4BBC0C OF A LONG
RUNNING CONNECTION POOLS SESSION HOW DO YOU
KNOW WHAT PORTION OF THAT WAS INCREMENTED OR ADDED TODAY RIGHT NOW WHEN YOU HAVE THE PROBLEM AS
OPPOSED TO A FEW WEEKS AGO WHEN THAT SESSION LOGGED ON )N OTHER WORDS WHEN TROUBLESHOOTING A
PROBLEM HAPPENING RIGHT NOW YOU SHOULD LOOK AT PERFORMANCE FOR RIGHT NOW ONLY AND A SIMILAR RULE
APPLIES WHEN TROUBLESHOOTING ISSUES OF THE PAST
4HIS IS WHY COAUTHOR 4ANEL 0ODER HAS WRITTEN A LITTLE HELPER TOOL CALLED /RACLE 3ESSION 3NAPPER
WHICH ALLOWS ITS USER TO EASILY DISPLAY THE SESSIONS CURRENT ACTIVITY FROM EB4BBC0C AND VARIOUS OTHER
SESSION LEVEL PERFORMANCE VIEWS !N IMPORTANT THING ABOUT THIS TOOL IS THAT ITS JUST AN ANONYMOUS
0,31, BLOCK PARSED ON THE FLY IT DOESNT REQUIRE ANY INSTALLATION NOR $$, PRIVILEGES WHATSOEVER 4HE
CURRENT 3NAPPER VERSION IS AVAILABLE ONLINE AT 4g_Tac>aPR[T4gPSPcP R^\ &OLLOWING IS ONE EXAMPLE OF
HOW TO RUN 3NAPPER TO MEASURE 3)$ S ACTIVITY FOR A SINGLE
SECOND INTERVAL 2EAD THE 3NAPPER
HEADER FOR INSTRUCTIONS AND DETAILED DOCUMENTATION
B@;-/b]P__TaPbWbcPcb$$ ## "
BP\_[X]VB83# "fXcWX]cTaeP[$bTR^]SbcPZX]V b]P_bW^cb
BTbbX^]B]P__Tae" $!QhCP]T[?^STa/4!B=Wcc_) cTRW T!b] R^\
B83DB4ACH?4BC0C8BC82734;C0
# "C0=4;BC0Cb
bTbbX^][^VXRP[aTPSb "#<
# "C0=4;BC0CdbTa8 >fPXccX\T#
# "C0=4;BC0C]^] XS[TfPXccX\T#
# "C0=4;BC0C]^] XS[TfPXcR^d]c' Z
# "C0=4;BC0C__WhbXRP[aTPSc^cP[8>aT`dTbcb $"Z
# "C0=4;BC0C_WhbXRP[aTPSc^cP[\d[cXQ[^RZaT`dTbcb $"Z
# "C0=4;BC0C__WhbXRP[aTPSc^cP[QhcTb "6
# "C0=4;BC0CRT[[_WhbXRP[8>X]cTaR^]]TRcQhcTb %<
# "C0=4;BC0CR^]bXbcT]cVTcb "$<
# "C0=4;BC0CR^]bXbcT]cVTcbSXaTRc "$<
# "C0=4;BC0C_WhbXRP[aTPSb "$<
# "C0=4;BC0C_WhbXRP[aTPSbSXaTRc "$<
# "C0=4;BC0C_WhbXRP[aTPS8>aT`dTbcb $!Z
# "C0=4;BC0C_WhbXRP[aTPSQhcTb !6
# "C0=4;BC0CUX[TX^fPXccX\T$" &"Z
# "C0=4;BC0CRT[[_WhbXRP[8>QhcTbT[XVXQ[TU^a_aTSXRPcT^UU[^PS "6
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
# "C0=4;BC0CRT[[_WhbXRP[8>X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP] %<
# "C0=4;BC0CRT[[Q[^RZb_a^RTbbTSQhRPRWT[PhTa "$<
# "C0=4;BC0CRT[[Q[^RZb_a^RTbbTSQhcg][PhTa "$<
# "C0=4;BC0CRT[[Q[^RZb_a^RTbbTSQhSPcP[PhTa "$<
# "C0=4;BC0CRT[[Q[^RZbWT[_TSQh\X]bR]^_cX\XiPcX^] "$<
# "C0=4;BC0CRT[[8>d]R^\_aTbbTSQhcTb $6
# "C0=4;F08CRT[[b\PaccPQ[TbRP]# &b
4]S^UBcPcbb]P_ T]S,! # " () &)"%bTR^]Sb,$
0RcXeTkB@;N83k4E4=CkF08CN2;0BB
(
($kQ&i UcW PbQQckRT[[b\PaccPQ[TbRP]kDbTa8 >
$kQ&i UcW PbQQck>=2?Dk>=2?D
4]S^U0B7b]P_ T]S,! # " () &)"%bTR^]Sb,$$bP\_[TbNcPZT],#!
4HIS OUTPUT SHOWS THAT DURING THE
SECOND MONITORING TIME THE SESSION DID MILLION LOGICAL
READS AND IT READ '" OF DATA USING ABOUT MULTIBLOCK READ )/ REQUESTS LARGE NUMBERS ARE
CONVERTED INTO HUMANFRIENDLY APPROXIMATIONS HERE BUT 3NAPPER CAN ALSO PRINT OUT EXACT FIGURES
4HE NEXT EXAMPLE SHOWS HOW YOU CAN USE 3NAPPER ON A PARALLEL QUERY .OTICE THE qc=403 OPTION
WHICH TELLS 3NAPPER TO CAPTURE STATS FOR ALL 08 SLAVES OF 1UERY #OORDINATOR 3)$ !LSO NOTICE THE
bX]R[dST OPTION WHICH ALLOWS YOU TO LIST ONLY THE METRICS OF INTEREST USING A REGULAR EXPRESSION
B@;-/b]P__TaPbWbcPcbVPcWTa,bbbX]R[dST,_WhbXRP[ c^cP[ QhcTbkRT[[ bRP]$ ``R,# "
BP\_[X]VB83`R,# "fXcWX]cTaeP[$bTR^]SbcPZX]V b]P_bW^cb
BTbbX^]B]P__Tae" $!QhCP]T[?^STa/4!B=Wcc_) cTRW T!b] R^\
B83DB4ACH?4BC0C8BC82734;C0
!C0=4;BC0C__WhbXRP[aTPSc^cP[QhcTb!! '%6
!C0=4;BC0CRT[[_WhbXRP[8>X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP]# %Z
!C0=4;BC0CRT[[bRP]b!
$$$C0=4;BC0C_WhbXRP[aTPSc^cP[QhcTb" $6
$$C0=4;BC0CRRT[[_WhbXRP[8>X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP]###" $#Z
$$C0=4;BC0CRT[[bRP]b"
"C0=4;BC0C_WhbXRP[aTPSc^cP[QhcTb! (&6
"C0=4;BC0CRT[[_WhbXRP[8 >X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP]#"! !(Z
"C0=4;BC0CR RT[[bRP]b"
$(C0=4;BC0C_WhbXRP[aTPSc^cP[QhcTb! %"6
$(C0=4;BC0CRT[[_WhbXRP[8 >X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP]"' "&Z
$(C0=4;BC0CRT[[bRP]b!
4]S^UBcPcbb]P_ T]S,! # " ()$&)"$bTR^]Sb,%
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
0RcXeTkB@;N83k4E4=CkF08CN2;0BB
"("kQ#ZeW"b`bccZfkRT[[b\PaccPQ[TbRP]kDbTa8 >
&kQ#ZeW"b`bccZfk>=2?Dk>=2?D
4]S^U0B7b]P_ T]S,! # " ()$&)"$bTR^]Sb,$bP\_[TbNcPZT],##
4HIS OUTPUT SHOWS THAT THERE ARE APPARENTLY FOUR DIFFERENT 08 SLAVE SESSIONS NOTICE THE 3)$ COLUMN THAT
HAVE DONE PHYSICAL READS DURING THE
SECOND 3NAPPER RUN 4HE physical read total bytes TELLS US THAT
EACH SLAVE HAS READ BETWEEN AND '" WORTH OF DATA BLOCKS FROM DATAFILES DURING THE
SECOND
3NAPPER RUN USING CELL SMART SCAN BECAUSE THE cell scans STATISTIC HAS INCREASED FOR EACH SLAVE
-OREOVER THE FACT THAT cell physical I/O interconnect bytes returned by smart scan IS ALSO NON
ZERO MEANS
THAT A CELL SMART SCAN WAS USED AND RT[[bae PROCESSES RETURNED SOME DATA TO THE DATABASE LAYER .OTE
THAT ONLY AROUND n+" OF DATA WAS RETURNED TO THE DATABASE LAYER WHILE CELLS SCANNED THROUGH
GIGABYTES OF BLOCKS FOR THEIR hMASTERv 08 SLAVE SESSIONS 4HIS IS THE 3MART 3CAN AT ITS BESTPARALLEL BRUTE
FORCE SCANNING IN CELLS AT EXTREME SPEEDS AND RETURN ONLY A SMALL PROPORTION OF MATCHING DATA TO THE
DATABASE LAYER
4HE -EANING AND %XPLANATION OF %XADATA 0ERFORMANCE #OUNTERS
.OW ITS TIME TO EXPLORE THE meaning OF PERFORMANCE COUNTERS .O MATTER HOW PRETTY THE CHARTS OR
PICTURES A PERFORMANCE TOOL DRAWS USING THESE METRICS IF YOU DONT KNOW WHAT THEY ACTUALLY MEAN THEY
WILL BE USELESS FOR TROUBLESHOOTING )N THIS BOOK WE WILL COVER MOSTLY %XADATA
SPECIFIC STATISTICS AND A FEW
CLOSELY RELATED ONES 9OU CAN FIND MORE INFO ABOUT SOME COMMON PERFORMANCE COUNTERS IN !PPENDIX %
h3TATISTICS $ESCRIPTIONS v IN Oracle Database Reference guide 11g Release 2 (11.2) CURRENTLY LOCATED AT
Wcc_) S^f][^PS ^aPR[T R^\ S^Rb RS 4 # &!N bTaeTa ! T '! bcPcb Wc\
(ERES A SCRIPT THAT LISTS ALL STATISTICS RELATED TO STORAGE CELLS FROM ebcPc]P\T WITH THE STATISTIC CLASS
WHICH INDICATES THE PURPOSES FOR WHICH /RACLE KERNEL ENGINEERS HAVE EXPECTED TO USE THESE COUNTERS
B@;-B4;42C
!]P\T
"CA8<
#20B4F74=18C0=3R[Pbb , C74=DB4A4=3kk
$20B4F74=18C0=3R[Pbb!,!C74=A43>4=3kk
%20B4F74=18C0=3R[Pbb#,#C74=4=@4=3kk
&20B4F74=18C0=3R[Pbb','C74=202744=3kk
'20B4F74=18C0=3R[Pbb %, %C74=>B34?4=3kk
(20B4F74=18C0=3R[Pbb"!,"!C74=?0AG4=3kk
20B4F74=18C0=3R[Pbb%#,%#C74=B@;C4=3kk
20B4F74=18C0=3R[Pbb !', !'C74=341D64=3
!R[PbbN]P\T
"5A><
#e
ebcPc]P\T
$F74A4
%]P\T;8:4RT[[
&>A34A1H
']P\T
B@;-
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
=0<42;0BBN=0<4
RT[[2Db_a^RTbbTSU^aR^\_aTbbTSB@;C
RT[[2Db_a^RTbbTSU^ad]R^\_aTbbTSB@;C
RT[[2DbbT]cR^\_aTbbTSB@;C
RT[[2DbbT]cWTPS_XTRTB@;C
RT[[2DbbT]cd]R^\_aTbbTSB@;C
RT[[8>d]R^\_aTbbTSQhcTbB@;C
RT[[Q[^RZbWT[_TSQhR^\\XcRPRWTB@;C
RT[[Q[^RZbWT[_TSQh\X]bR]^_cX\XiPcX^]B@;C
RT[[Q[^RZb_a^RTbbTSQhRPRWT[PhTa341D6
RT[[Q[^RZb_a^RTbbTSQhSPcP[PhTa341D6
RT[[Q[^RZb_a^RTbbTSQhX]STg[PhTa341D6
RT[[Q[^RZb_a^RTbbTSQhcg][PhTa341D6
RT[[R^\\XcRPRWT`dTaXTbB@;C
RT[[U[PbWRPRWTaTPSWXcb20274
RT[[X]STgbRP]bB@;C
RT[[]d\PRcXeTb\Pac8>bTbbX^]bB@;C
RT[[]d\UPbcaTb_^]bTbTbbX^]bB@;C
RT[[]d\UPbcaTb_^]bTbTbbX^]bR^]cX]dX]Vc^b\PacbRP]B@;C
RT[[]d\b\Pac8>bTbbX^]bX]aSQ\bQ[^RZ8>SdTc^QXV_Ph[^PSB@;C
RT[[]d\b\Pac8>bTbbX^]bX]aSQ\bQ[^RZ8>SdTc^]^RT[[\T\B@;C
RT[[]d\b\Pac8>bTbbX^]bX]aSQ\bQ[^RZ8>SdTc^dbTaB@;C
RT[[]d\b\Pac8>bTbbX^]bdbX]V_PbbcWad\^STSdTc^RT[[baeB@;C
RT[[]d\b\Pac8>bTbbX^]bdbX]V_PbbcWad\^STSdTc^cX\Ti^]TB@;C
RT[[]d\b\Pac8>bTbbX^]bdbX]V_PbbcWad\^STSdTc^dbTaB@;C
RT[[]d\b\PacUX[TRaTPcX^]bTbbX^]bdbX]VaSQ\bQ[^RZ8>\^STB@;C
RT[[_WhbXRP[8>QhcTbT[XVXQ[TU^a_aTSXRPcT^UU[^PSB@;C
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg20274
RT[[_WhbXRP[8>QhcTbbPeTSSdaX]V^_cX\XiTSA<0=UX[TaTbc^aTB@;C
RT[[_WhbXRP[8>QhcTbbPeTSSdaX]V^_cX\XiTSUX[TRaTPcX^]B@;C
RT[[_WhbXRP[8>X]cTaR^]]TRcQhcTbB@;C
RT[[_WhbXRP[8>X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP]B@;C
RT[[bRP]bB@;C
RT[[bX\d[PcTS_WhbXRP[8>QhcTbT[XVXQ[TU^a_aTSXRPcT^UU[^PSB@;C341D6
RT[[bX\d[PcTS_WhbXRP[8>QhcTbaTcda]TSQh_aTSXRPcT^UU[^PSB@;C341D6
RT[[b\Pac8>P[[^RPcTS\T\^ahQhcTbB@;C
RT[[b\Pac8>\T\^ahQhcTbWf\B@;C
RT[[b\Pac8>bTbbX^]RPRWTWPaS\XbbTbB@;C
RT[[b\Pac8>bTbbX^]RPRWTWXcbB@;C
RT[[b\Pac8>bTbbX^]RPRWTWf\B@;C
RT[[b\Pac8>bTbbX^]RPRWT[^^Zd_bB@;C
RT[[b\Pac8>bTbbX^]RPRWTb^Uc\XbbTbB@;C
RT[[b\Pac8>bTbbX^]bWf\B@;C
RT[[caP]bPRcX^]bU^d]SX]R^\\XcRPRWTB@;C
RWPX]TSa^fb_a^RTbbTSQhRT[[B@;C
RWPX]TSa^fbaTYTRcTSQhRT[[B@;C
RWPX]TSa^fbbZX__TSQhRT[[B@;C
7ITH A SIMILAR QUERY WE CAN LIST ALL THE %XADATA (YBRID #OLUMNAR #OMPRESSION %(## STATISTICS BUT
WE WILL NOT COVER THOSE IN THIS CHAPTER
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
=0<42;0BBN=0<4
47220]P[hiT2Db3TR^\_aTbbTS341D6
47220]P[hiTa2P[[b341D6
47220aRWXeT2Db2^\_aTbbTS341D6
47220aRWXeT2Db3TR^\_aTbbTS341D6
47222DA^f?XTRTb2^\_aTbbTS341D6
47222Db2^\_aTbbTS341D6
47222Db3TR^\_aTbbTS341D6
47222WTRZ2Db3TR^\_aTbbTS341D6
47222^[d\]b3TR^\_aTbbTS341D6
47222^\_aTbbTS;T]VcW2^\_aTbbTS341D6
47222^\_aTbbTS;T]VcW3TR^\_aTbbTS341D6
47223<;2Db3TR^\_aTbbTS341D6
47223TR^\_aTbbTS;T]VcW2^\_aTbbTS341D6
47223TR^\_aTbbTS;T]VcW3TR^\_aTbbTS341D6
47223d\_2Db3TR^\_aTbbTS341D6
4722=^a\P[BRP]2Db3TR^\_aTbbTS341D6
4722?XTRTb1dUUTaTSU^a3TR^\_aTbbX^]341D6
4722@dTah7XVW2Db2^\_aTbbTS341D6
4722@dTah7XVW2Db3TR^\_aTbbTS341D6
4722@dTah;^f2Db2^\_aTbbTS341D6
4722@dTah;^f2Db3TR^\_aTbbTS341D6
4722A^fXS2Db3TR^\_aTbbTS341D6
4722A^fb2^\_aTbbTS341D6
4722A^fb=^c2^\_aTbbTS341D6
4722C^cP[2^[d\]bU^a3TR^\_aTbbX^]341D6
4722C^cP[?XTRTbU^a3TR^\_aTbbX^]341D6
4722C^cP[A^fbU^a3TR^\_aTbbX^]341D6
4722CdaQ^BRP]2Db3TR^\_aTbbTS341D6
!LL THE STATISTICS STARTING FROM cell ARE AS THE NAME SAYS RELATED TO STORAGE CELLS 4HESE STATS ARE MEASURED
AND MAINTAINED BY CELLS THEMSELVES AND THEN SENT BACK TO THE DATABASE SESSIONS DURING ANY INTERACTION
OVER )$" PROTOCOL %VERY DATABASE SESSION RECEIVES THE CELL STATISTICS ALONG WITH THE REPLIES FROM THEIR
CORRESPONDING CELL SESSIONS AND THEN UPDATES THE RELEVANT DATABASE 6 VIEWS WITH IT 4HIS IS HOW THE
/RACLE DATABASE LAYER HAS INSIGHT INTO WHATS GOING ON IN THE hBLACK BOXv OF A CELL LIKE THE REAL NUMBER OF
)/ OPERATIONS DONE THE NUMBER OF CELL FLASH CACHE HITS AND SO ON .OTE THAT THERE ARE A FEW chained
rows […] cell STATISTICS WHICH APPARENTLY USE A DIFFERENT NAMING CONVENTION HAVING THE hCELLv IN THE END
OF THE STATISTIC NAME
%XADATA 0ERFORMANCE #OUNTER 2EFERENCE
4HIS SECTION EXPLAINS SOME OF THE MORE IMPORTANT AND INTERESTING STATISTICS IN THIS CHAPTER ALTHOUGH IF
YOU ARE LUCKY YOU WONT HAVE TO DIG DOWN THIS DEEP IN YOUR TROUBLESHOOTING WAIT EVENTS AND THE 31,
MONITORING FEATURE ARE GOOD ENOUGH FOR MOST BASIC TROUBLESHOOTING AND PERFORMANCE OPTIMIZATION
.EVERTHELESS UNDERSTANDING WHATS BEHIND THESE STATISTICS AND BEHAVIOR WILL GIVE YOU FURTHER INSIGHT
INTO %XADATA INTERNALS AND ENABLE YOU TO TROUBLESHOOT UNUSUAL PERFORMANCE PROBLEMS MORE EFFECTIVELY
7ELL EXPLAIN THE STATS IN ALPHABETICAL ORDER TO MAKE IT EASIER TO USE THIS SECTION AS A REFERENCE
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
cell blocks helped by commit cache
$URING A 3MART 3CAN IN THE STORAGE CELL THE NORMAL DATA CONSISTENCY RULES STILL NEED TO BE APPLIED
SOMETIMES WITH HELP OF UNDO DATA 9ES CONSISTENT READ GUARANTEES MUST WORK ALSO FOR 3MART 3CANS "UT
THE 3MART 3CAN WORKS ENTIRELY IN THE STORAGE CELLS WHERE IT DOES NOT HAVE ACCESS TO ANY UNDO DATA IN THE
DATABASE INSTANCES BUFFER CACHES !ND REMEMBER THAT A CELL NEVER TALKS TO ANY OTHER CELLS BY DESIGN SO IT
WOULD BE UNABLE TO READ THE UNDO DATA FROM UNDO SEGMENTS STRIPED ACROSS MANY CELLS TOO 3O CONSISTENT
READ BUFFER CLONING AND ROLLBACKS WHENEVER NEEDED WOULD HAVE TO BE DONE INSIDE THE DATABASE LAYER
7HENEVER THE 3MART 3CAN HITS A ROW WHICH STILL HAS ITS LOCK BYTE SET THE ROWBLOCK HASNT BEEN CLEANED
OUT IT WOULD TEMPORARILY HAVE TO SWITCH INTO BLOCK )/ MODE AND SEND THE ENTIRE DATA BLOCK BACK TO THE
DATABASE LAYER FOR NORMAL CONSISTENT READ PROCESSING WITH THE HELP OF UNDO DATA AVAILABLE THERE
.OTE THAT WHEN A BLOCK IS CLEANED OUT CORRECTLY LOCK BYTES CLEARED AND ITS CLEANOUT 3#. IN THE
BLOCK HEADER IS FROM AN EARLIER 3#. THAN THE QUERY START TIME 3#. snapshot SCN THEN THE CELL KNOWS
THAT A ROLLBACK OF THAT BLOCK WOULD NOT BE NEEDED )F THE LATEST CHANGE TO THAT BLOCK HAPPENED EARLIER
THAN THE QUERY START THEN THE BLOCK IMAGE IN THE CELL IS VALID A GOOD ENOUGH BLOCK TO SATISFY THE QUERY
WITH THE GIVEN 3#. (OW DOES THE CELL KNOW THE STARTING 3#. OF A QUERY EXECUTED IN THE DATABASE LAYER
4HAT IS THE TASK OF THE STORAGE AWARE ROW SOURCES IN THE EXECUTION PLANS WHICH COMMUNICATE THE 3#. TO
CELLS OVER I$" WHEN SETTING UP 3MART 3CAN SESSIONS FOR THEMSELVES
.OW WHEN SOME OF THE ROWS DO HAVE NONZERO LOCK BYTES IN THE BLOCK OR WHEN THE CLEANOUT 3#. IN A
BLOCK HEADER HAPPENS TO BE HIGHER THAN THE QUERYS SNAPSHOT 3#. THEN THE CELLS CANNOT DETERMINE THE
VALIDITY OF THE BLOCKDATA VERSION THEMSELVES AND WOULD NEED TO SHIP THE BLOCK BACK TO THE DATABASE
LAYER FOR REGULAR NON 3MART 3CAN PROCESSING 4HIS WOULD CONSIDERABLY SLOW DOWN THE 3MART 3CAN
PROCESSING IF SUCH CHECK HAD TO BE DONE FOR MANY LOCKED ROWS AND NOT CLEANED OUT BLOCKS
(OWEVER THERE IS AN OPTIMIZATION THAT HELPS TO REDUCE THE NUMBER OF FALLBACKS TO BLOCK )/
PROCESSING IN THE DATABASE LAYER 7HENEVER A 3MART 3CAN FINDS A LOCKED ROW DURING A SEGMENT SCAN IT
WILL CHECK WHICH TRANSACTION LOCKED THAT ROW 4HIS CAN EASILY BE DONE BY READING THE TRANSACTIONS )4,
ENTRY IN THE CURRENT DATA BLOCK HEADER WHERE THE LOCKED ROWS LOCK BYTE POINTS .OTE THAT BITMAP INDEX
SEGMENT BLOCKS AND (## COMPRESSED BLOCKS DONT HAVE A LOCK BYTE FOR EACH SINGLE ROW IN THE BLOCK BUT
THE IDEA REMAINS THE SAME /RACLE IS ABLE TO FIND OUT THE TRANSACTION )$ OF THE LOCKED ROWS FROM THE
BLOCK AT HAND ITSELF .OW THE 3MART 3CAN CAN SEND THAT TRANSACTION )$ BACK TO THE DATABASE LAYER TO THE
DATABASE SESSION OWNING THE CELL 3MART 3CAN SESSION AND ASK THE DATABASE SESSION TO CHECK WHETHER THAT
TRANSACTION HAS ALREADY COMMITTED OR NOT )F THIS LOCKING TRANSACTION HAS NOT COMMITTED YET THE 3MART
3CAN FALLS BACK TO THE BLOCK )/ MODE FOR THAT BLOCK AND THE DATABASE LAYER WILL HAVE TO GO THROUGH THE
NORMAL CONSISTENT READ BUFFER CLONINGROLLBACK MECHANISM AND THERES NO WORKAROUND FOR THAT )F THE
TRANSACTION has ALREADY COMMITTED BUT HAS LEFT THE LOCK BYTES NOT CLEANED OUT IN SOME BLOCKS THIS DOES
USUALLY HAPPEN FOR LARGE $-,S THEN THE 3MART 3CAN DOES NOT HAVE TO FALL BACK TO BLOCK )/ AND AN IN
DATABASE CONSISTENT READ MECHANISM )T KNOWS THAT THIS ROW IS NOT REALLY LOCKED ANYMORE AS THE LOCKING
TRANSACTION HAS COMMITTED ALREADY EVEN THOUGH THE LOCK BYTE IS STILL IN PLACE
!LL OF THAT TO READ A SINGLE LOCKED ROW 7HEN 3MART 3CANNING YOU ARE PROBABLY SCANNING THROUGH
MILLIONS OR BILLIONS OF ROWS AND MANY OF THESE ROWS MAY STILL HAVE THEIR LOCK BYTES SET 3O YOU DONT WANT
THE 3MART 3CAN TO COMMUNICATE WITH THE DATABASE LAYER EVERY SINGLE TIME IT HITS ANOTHER LOCKED ROW 3O
/RACLE CELLS CAN cache THE INFORMATION ABOUT TRANSACTION STATUSES IN AN IN MEMORY STRUCTURE CALLED
hCOMMIT CACHEv )TS PROBABLY JUST AN IN MEMORY HASH TABLE ORGANIZED BY TRANSACTION )$ AND IT KEEPS
TRACK OF WHICH TRANSACTIONS ARE COMMITTED AND WHICH ARE NOT 7HEN THE CELL 3MART 3CAN SEES A LOCKED
ROW IT WILL EXTRACT THE TRANSACTION )$ FROM THE )4, SECTION OF THE DATA BLOCK AND CHECKS WHETHER THERES
ANY INFO ABOUT THAT TRANSACTION IN THE COMMIT CACHE 4HIS CHECK WILL INCREMENT THE STATISTIC cell commit
cache queries BY ONE )F THERE IS NO SUCH TRANSACTION IN THE COMMIT CACHE THEN THE 3MART 3CAN hASKSv THE
STATUS OF THIS TRANSACTION FROM THE DATABASE LAYER AND STORES CACHES THE RESULT IN THE COMMIT CACHE
)DEALLY EVERY TRANSACTION STATUS IS FETCHED INTO THE CACHE ONLY ONCE AND ALL SUBSEQUENT CHECKS WILL BE
SATISFIED FROM THE COMMIT CACHE 3O EVERY TIME THE 3MART 3CAN OPENS A NEW BLOCK AND REALIZES THAT THERE
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
ARE SOME ROWS LOCKED BUT MANAGES TO AVOID HAVING TO TALK TO THE DATABASE LAYER THANKS TO THE COMMIT
CACHE OUR STATISTIC AT HAND cell blocks helped by commit cache WOULD BE UPDATED BY ONE
3EEING THIS STATISTIC GROWING WHEN YOUR SESSIONS ARE DOING A 3MART 3CAN INDICATES THAT THE CELLS
HAVE SOME OVERHEAD DUE TO CHECKING WHETHER LOCKED ROWS ARE REALLY STILL LOCKED BUT THANKFULLY THE
COMMIT CACHE ELIMINATES THE NEED FOR COMMUNICATING WITH THE DATABASE LAYER REPEATEDLY 7ITHOUT THIS
OPTIMIZATION THE WHOLE 3MART 3CAN WOULD GET SLOWER AS IT MUST REPEATEDLY INTERACT WITH THE DATABASE
LAYER 9OU WOULD ALSO SEE MORE LOGICAL )/ BEING DONE AT THE DATABASE LAYER SEE THE STATISTICS STARTING
WITH consistent gets from cache AS OPPOSED TO ONLY A SINGLE ,)/ PER SEGMENT FOR READING THE EXTENT
LOCATIONS AND NUMBER OF BLOCKS UNDER THE (IGH 7ATER -ARK OUT FROM THE SEGMENT HEADER
.OTE ! SIMILAR OPTIMIZATION ACTUALLY EXISTS ALSO IN THE DATABASE LAYER AND IN NON %XADATA DATABASES /RACLE
CAN CACHE THE COMMITTED TRANSACTION INFO IN THE DATABASE SESSIONS PRIVATE MEMORY SO IT WONT HAVE TO PERFORM
BUFFER GETS ON THE UNDO SEGMENT HEADER WHEN HITTING MANY LOCKED ROWS 7HENEVER A SESSION CACHES A COMMITTED
TRANSACTIONS STATE IN MEMORY IT INCREMENTS THE Commit SCN cached STATISTIC IN ITS EB4BBC0C ARRAY 7HENEVER IT
DOES A LOOKUP FROM THAT CACHE IT INCREMENTS THE Cached Commit SCN referenced STATISTIC
)F YOU ARE NOT FAMILIAR WITH THE DELAYED BLOCK CLEANOUT MECHANISM IN /RACLE YOU MIGHT BE
WONDERING HOW THERE CAN BE ROWS WITH THEIR LOCK BYTES STILL SET WHEN THE TRANSACTION HAS ALREADY
COMMITTED 4HIS IS HOW /RACLE IS DIFFERENT FROM MOST OTHER MAINSTREAM COMMERCIAL 2$"-3 PRODUCTS
/RACLE DOESNT HAVE TO KEEP THE BLOCKS WITH NONCOMMITTED ROWS CACHED IN MEMORY $"72 IS FREE TO
WRITE THEM OUT TO DISK AND RELEASE THE MEMORY FOR OTHER DATA BLOCKS .OW WHEN COMMITTING THE
TRANSACTION IT WOULDNT BE GOOD TO READ ALL THE TRANSACTIONS BACK FROM THE DISK JUST TO CLEAN UP THE LOCK
BYTES )F THERE ARE MANY SUCH BLOCKS THEN YOUR COMMIT MIGHT TAKE AGES )NSTEAD /RACLE JUST MARKS THE
TRANSACTION COMPLETE IN ITS UNDO SEGMENT HEADER SLOT !NY FUTURE BLOCK READERS CAN JUST CHECK WHETHER
THE TRANSACTION IN THAT UNDO SEGMENT HEADER SLOT IS STILL ALIVE OR NOT )F YOU PERFORM BLOCK )/ TO READ THE
DATA BLOCKS TO THE DATABASE LAYER LATER ON THEN THE READING SESSION WOULD CLEAN UP THE BLOCK CLEAR THE
LOCK BYTES OF ROWS MODIFIED BY COMMITTED TRANSACTIONS SO NO FURTHER TRANSACTION STATUS CHECKS WOULD
BE NEEDED IN FUTURE READS (OWEVER STORAGE CELLS DO NOT PERFORM BLOCK CLEANOUTS AS CELLS DONT CHANGE
DATA BLOCKS 4HIS IS BECAUSE DATABASE BLOCK MODIFICATIONS REQUIRE WRITING OF REDO OPERATIONS BUT HOW
WOULD A CELL WRITE TO A REDO LOG FILE THAT IS MANAGED AND STRIPED OVER MANY CELLS AT THE DATABASE LAYER
ALREADY
.OTE THAT FOR SMALL TRANSACTIONS WHICH HAVENT MODIFIED TOO MANY BLOCKS AND WHERE THE BLOCKS ARE
STILL IN BUFFER CACHE /RACLE CAN PERFORM BLOCK CLEANOUT RIGHT DURING THE COMMIT TIME !LSO THE ISSUES
JUST DISCUSSED DO NOT APPLY TO SUCH DATABASES DATA WAREHOUSES USUALLY WHERE TABLES ARE LOADED WITH
DIRECT PATH LOAD INSERTS AND INDEX PARTITIONS ARE BUILT AFTER THE TABLE PARTITIONS ARE LOADED BECAUSE IN
THE CASE OF DIRECT LOADS THE TABLE ROWS ARE NOT LOCKED IN THE NEWLY FORMATTED TABLE BLOCKS THE SAME
APPLIES TO INDEX ENTRIES IN LEAF BLOCKS IF AN INDEX IS CREATED AFTER THE DATA LOAD
cell blocks helped by minscn optimization
%XADATA CELL SERVER HAS ANOTHER OPTIMIZATION DESIGNED TO IMPROVE CONSISTENT READ EFFICIENCY EVEN MORE
)TS CALLED THE -INIMUM !CTIVE 3#. OPTIMIZATION AND IT KEEPS TRACK OF THE LOWEST 3#. OF ANY STILL ACTIVE
UNCOMMITTED TRANSACTION IN THE DATABASE 4HIS ALLOWS US TO EASILY COMPARE THE 3#. IN THE )4, ENTRIES
OF THE LOCKING TRANSACTIONS WITH THE LOWEST 3#. OF THE hOLDESTv ACTIVE TRANSACTION IN THE DATABASE !S THE
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
/RACLE DATABASE IS ABLE TO SEND THIS -IN3#. INFO TO THE CELL WHEN STARTING A 3MART 3CAN SESSION THE CELLS
CAN AVOID HAVING TO TALK TO THE DATABASE LAYER WHENEVER THE KNOWN MINIMUM ACTIVE 3#. PASSED TO THE
CELL IS HIGHER THAN THE 3#. IN THE TRANSACTIONS )4, ENTRY IN THE BLOCK 7HENEVER THE 3MART 3CAN
PROCESSES A BLOCK AND FINDS A LOCKED ROW WITH ACTIVE TRANSACTION IN THE )4, SLOT IN IT BUT REALIZES THAT THE
TRANSACTION MUST BE COMMITTED THANKS TO THE -IN3#. PASSED IN BY THE DATABASE SESSION THE cell blocks
helped by minscn optimization STATISTIC IS INCREMENTED ONCE FOR EACH BLOCK
7ITHOUT THIS OPTIMIZATION /RACLE WOULD HAVE TO CHECK THE COMMIT CACHE DESCRIBED IN THE cell
blocks helped by commit cache STATISTIC SECTION AND IF IT FINDS NO INFO ABOUT THIS TRANSACTION IN THE
COMMIT CACHE THEN INTERACT WITH THE DATABASE LAYER TO FIND OUT WHETHER THE LOCKING TRANSACTION HAS
ALREADY COMMITTED OR NOT 4HIS OPTIMIZATION IS 2!#
AWARE IN FACT THE -INIMUM 3#. IS CALLED 'LOBAL
-INIMUM 3#. AND THE --/. PROCESSES IN EACH INSTANCE WILL KEEP TRACK OF THE -IN3#. AND KEEP IT
SYNCED IN AN IN
MEMORY STRUCTURE IN EACH NODES 3'! 9OU CAN QUERY THE CURRENT KNOWN GLOBAL
-INIMUM !CTIVE 3#. FROM THE gZcd\PbR] FIXED TABLE AS SHOWN HERE AS 393
B@;-2>;\X]NPRcNbR]5>A(((((((((((((((((
B@;-
B@;-B4;42C\X]NPRcNbR]5A><gZcd\PbR]*
<8=N02CNB2=
((! '( '' '$(
4HIS cell blocks helped by minscn optimization STATISTIC IS ALSO SOMETHING YOU SHOULDNT BE WORRIED
ABOUT BUT IT CAN COME IN HANDY WHEN TROUBLESHOOTING ADVANCED 3MART 3CAN ISSUES OR EVEN BUGS WHERE
3MART 3CANS SEEM TO GET INTERRUPTED BECAUSE THEY HAVE TO FALL BACK TO BLOCK )/ AND TALK TO THE DATABASE
TOO MUCH
cell blocks processed by cache layer
4HE cell blocks processed by … layer STATISTICS ARE GOOD INDICATORS OF THE depth OF OFFLOAD PROCESS IN THE
CELLS 4HE MAIN POINT AND ADVANTAGE OF THE %XADATA STORAGE SERVERS IS THAT PART OF THE /RACLE KERNEL CODE
HAS BEEN PORTED INTO THE RT[[bae EXECUTABLE RUNNING IN THE STORAGE CELLS 4HIS IS WHAT ALLOWS THE /RACLE
DATABASE LAYER TO OFFLOAD THE DATA SCANNING FILTERING AND PROJECTION WORK INTO THE CELLS )N ORDER TO DO
THAT THE CELLS MUST BE ABLE TO READ AND UNDERSTAND /RACLE DATA BLOCK AND ROW CONTENTS JUST AS THE
DATABASE DOES 4HE cell blocks processed by cache layer STATISTIC INDICATES HOW MANY DATA BLOCKS THE cells
HAVE PROCESSED OPENED READ AND USED FOR 3MART 3CAN AS OPPOSED TO JUST PASSING THE BLOCKS READ UP
THE DATABASE LAYER
7HEN A CELL JUST PASSES THE BLOCKS BACK TO THE DATABASE IN BLOCK )/ MODE THIS STATISTIC IS NOT
UPDATED "UT WHEN THE CELL ITSELF USES THESE BLOCKS FOR 3MART 3CAN THEN ONE OF THE FIRST THINGS THAT IS
DONE WHEN OPENING A BLOCK CONSISTENT OR CURRENT READ IS TO CHECK THE BLOCK CACHE LAYER HEADER 4HIS IS
TO MAKE SURE IT IS THE CORRECT BLOCK IS NOT CORRUPT AND IS VALID AND COHERENT 4HESE TESTS ARE DONE BY
CACHE LAYER FUNCTIONS +#" FOR +ERNEL #ACHE "UFFER MANAGEMENT AND REPORTED BACK TO THE DATABASE AS
cell blocks processed by cache layer
)N THE DATABASE LAYER WITH REGULAR BLOCK )/ THE CORRESPONDING STATISTICS ARE consistent gets from
cache AND consistent gets from cache (fastpath) DEPENDING ON WHICH BUFFER PINNING CODE PATH IS USED FOR
THE CONSISTENT BUFFER GET .OTE THAT RT[[bae DOES ONLY CONSISTENT MODE BUFFER GETS #2 READS AND NO
CURRENT MODE BLOCK GETS 3O ALL THE CURRENT MODE GETS YOU SEE IN STATS ARE DONE IN THE DATABASE LAYER AND
ARE REPORTED AS db block gets from cache OR db block gets from cache (fastpath) 4HIS STATISTIC IS A USEFUL
AND SIMPLE MEASURE OF HOW MANY LOGICAL READS THE RT[[bae DOES FOR YOUR SESSIONS
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
.OTE THAT IT IS /+ TO SEE SOME DATABASE LAYER )/ PROCESSING DURING A 31, PLAN EXECUTION AS THE
PLAN IS PROBABLY ACCESSING MULTIPLE TABLES AND JOINING THEM 3O WHEN DOING A
TABLE JOIN BETWEEN A
LARGE FACT AND NINE DIMENSION TABLES YOU MAY WELL SEE THAT ALL OF THE DIMENSIONS ARE SCANNED USING
REGULAR CACHED BLOCK )/ AND USING SOME INDEX IF PRESENT AND ONLY THE LARGE FACT TABLE ACCESS PATH WILL
TAKE ADVANTAGE OF THE 3MART 3CAN
cell blocks processed by data layer
7HILE THE PREVIOUS STATISTIC COUNTS ALL THE BLOCK GETS DONE BY THE CACHE LAYER +#" THIS STATISTIC IS
SIMILAR BUT COUNTS THE BLOCKS PROCESSED IN THE CELL BY THE DATA LAYER 4HIS STATISTIC APPLIES SPECIFICALLY TO
READING TABLE BLOCKS OR MATERIALIZED VIEW BLOCKS WHICH ARE PHYSICALLY JUST LIKE TABLE BLOCKS )NFORMATION
IS COLLECTED USING A DATA LAYER MODULE CALLED +$3 FOR +ERNEL $ATA 3CAN WHICH CAN EXTRACT ROWS AND
COLUMNS OUT OF TABLE BLOCKS AND PASS THEM ON TO VARIOUS EVALUATION FUNCTIONS FOR FILTERING AND PREDICATE
CHECKS )F THE CELL 3MART 3CAN CAN DO ALL OF ITS PROCESSING IN THE CELL WITHOUT HAVING TO FALL BACK TO
DATABASE BLOCK )/ MODE THEN THIS hPROCESSED BY DATA LAYERv STATISTIC PLUS THE PROCESSED BY hINDEX
LAYERv STATISTIC SHOULD ADD UP TO THE hPROCESSED BY CACHE LAYERv VALUE 4HIS MEANS THAT EVERY BLOCK
ACTUALLY OPENED MADE ITS WAY THROUGH THE CACHE AND TRANSACTION LAYER CHECKS AND WAS PASSED TO THE DATA
OR INDEX LAYER FOR ROW AND COLUMN EXTRACTION )F THE hPROCESSED BY DATA LAYERv PLUS hPROCESSED BY INDEX
LAYERv STATISTICS ADD UP TO A SMALLER VALUE THAN THE hPROCESSED BY CACHE LAYERv STATISTIC IT MEANS THAT THE
REST OF THE BLOCKS WERE NOT FULLY PROCESSED BY THE CELL AND HAD TO BE SENT BACK TO THE DATABASE FOR REGULAR
BLOCK )/ PROCESSING
cell blocks processed by index layer
4HIS STATISTIC IS JUST LIKE THE PRECEDING cell blocks processed by data layer BUT IT IS INCREMENTED WHEN
3MART 3CANNING THROUGH "
4REE OR BITMAP INDEX SEGMENT BLOCKS 4HE CODE PATH FOR EXTRACTING ROWS OUT
OF INDEX BLOCKS IS DIFFERENT FROM 4HE cell blocks processed by index layer COUNTS HOW MANY INDEX SEGMENT
BLOCKS WERE PROCESSED BY A 3MART 3CAN
cell blocks processed by txn layer
4HIS STATISTIC SHOWS HOW MANY BLOCKS WERE PROCESSED IN THE CELL BY THE TRANSACTION LAYER (ERE IS A
SIMPLIFIED EXPLANATION OF THE SEQUENCE OF ACTIONS DURING A CONSISTENT READ FOR 3MART 3CAN IN A STORAGE
CELL
4HE CACHE LAYER +#" OPENS THE DATA BLOCK AND CHECKS ITS HEADER LAST
MODIFICATION 3#. AND CLEANOUT STATUS
)F THE BLOCK IN THE CELL HAS NOT BEEN MODIFIED AFTER THE SNAPSHOT 3#. OF THE
QUERY RUNNING THE CURRENT 3MART 3CAN THIS BLOCK CAN BE PASSED TO THE
TRANSACTION LAYER FOR PROCESSING (OWEVER IF THE BLOCK IMAGE ON DISK CELL HAS
BEEN MODIFIED AFTER THE QUERYS SNAPSHOT 3#. THEN THE CACHE LAYER ALREADY
KNOWS THAT THIS BLOCK HAS TO BE ROLLED BACK FOR CONSISTENT READ )N THIS CASE THE
BLOCK IS NOT PASSED INTO THE CELL TRANSACTION LAYER AT ALL BUT THE CELL FALLS BACK TO
BLOCK )/ AND PASSES THAT BLOCK TO THE DATABASE LAYER FOR NORMAL #2
PROCESSING
)F THE BLOCK IS PASSED TO THE TRANSACTION LAYER +42 BY THE CACHE LAYER THEN THE
TRANSACTION LAYER CAN USE THE COMMIT CACHE AND -IN!CTIVE3#. OPTIMIZATION
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
TO AVOID FALLING BACK TO REDUCE THE AMOUNT OF COMMUNICATION WITH THE
DATABASE LAYER IF IT HITS LOCKED ROWS AND NOT CLEANED OUT BLOCKS OF COMMITTED
TRANSACTIONS 7HEN THERE IS NO NEED TO FALL BACK TO BLOCK )/ AND DATABASE
LAYER CONSISTENT READS THEN THE CONSISTENT READS WILL BE PERFORMED BY THE DATA
LAYER OR INDEX LAYER CODE INSIDE THE STORAGE CELL (OWEVER IF THE CONSISTENT READ
CANNOT BE COMPLETED WITHIN THE CELL THEN THE ENTIRE DATA BLOCK AT HAND MUST
BE TRANSPORTED BACK TO THE DATABASE LAYER AND THE CONSISTENT READ WILL BE
PERFORMED THERE
4HE POINT OF THIS EXPLANATION IS THAT IF THE 3MART 3CANS WORK OPTIMALLY THEY DO NOT HAVE TO INTERRUPT
THEIR WORK AND TALK TO THE DATABASE LAYER DURING THE 3MART 3CAN PROCESSING IDEALLY ALL THE SCANNING WORK
IS DONE IN THE STORAGE CELL AND ONCE ENOUGH ROWS ARE READY TO BE RETURNED THEYRE SENT TO THE DATABASE
IN A BATCH )F THIS IS THE CASE THEN THE cell blocks processed by data layer (OR index layer) STATISTIC WILL BE AS
HIGH AS THE cell blocks processed by cache layer AND txn layer SHOWING THAT ALL THE BLOCKS COULD BE FULLY
PROCESSED IN THE CELL AND ROWS EXTRACTED FROM THEM WITHOUT HAVING TO FALL BACK TO DATABASE
LAYER BLOCK
)/ AND CONSISTENT READS
2EMEMBER THAT ALL THIS COMPLEXITY RELATED TO CONSISTENT READS IN STORAGE CELLS MATTERS ONLY WHEN
DOING A 3MART 3CAN 7HEN DOING REGULAR BLOCK )/ CELLS JUST PASS THE BLOCKS READ DIRECTLY BACK TO THE
DATABASE LAYER AND THE CONSISTENT READ LOGIC IS EXECUTED IN THE DATABASE LAYER AS USUAL !ND YOU
SHOULDNT REALLY WORRY ABOUT THESE METRICS UNLESS YOU SEE THAT YOUR 3MART 3CAN WAIT EVENTS TEND TO BE
INTERSPERSED WITH cell single block physical reads CONSUMING A SIGNIFICANT PART OF YOUR QUERY RESPONSE
TIME
cell commit cache queries
4HIS IS THE NUMBER OF TIMES THE CELL 3MART 3CAN LOOKED UP A TRANSACTION STATUS FROM THE CELL COMMIT
CACHE HASH TABLE ! LOOKUP FROM COMMIT CACHE IS NORMALLY DONE ONCE PER UNCOMMITTED TRANSACTION
FOUND PER BLOCK SCANNED BY 3MART 3CAN WHERE THE -IN!CTIVE3#. OPTIMIZATION HASNT ALREADY KICKED
IN AND ELIMINATED THE NEED TO CHECK FOR INDIVIDUAL TRANSACTION STATUSES 4HIS IS CLOSELY RELATED TO THE cell
blocks helped by commit cache STATISTIC
cell flash cache read hits
4HIS STATISTIC SHOWS HOW MANY )/ REQUESTS WERE SATISFIED FROM THE CELL FLASH CACHE SO THAT A HARD DISK
READ WAS NOT NECESSARY .OTE THAT WE SAID hHARD DISK READ v NOT JUST PHYSICAL READ 4HATS BECAUSE THE
READS FROM FLASH CARDS ALSO REQUIRE PHYSICAL READS SYSTEM CALLS RESULTING IN FLASH CARD )/ 7HEN YOU SEE
THIS NUMBER IT MEANS THAT THE REQUIRED BLOCKS WERE NOT IN THE DATABASE LAYER BUFFER CACHE OR THE ACCESS
PATH CHOSE TO USE A DIRECT PATH READ BUT LUCKILY ALL THE BLOCKS REQUIRED BY AN )/ REQUEST WERE IN CELL
FLASH CACHE THE OFFICIAL TERM IS %XADATA 3MART &LASH #ACHE .OTE THAT THIS NUMBER SHOWS THE NUMBER OF
)/ REQUESTS NOT THE NUMBER OF BLOCKS READ FROM CELL FLASH CACHE 2EMEMBER THAT CELL FLASH CACHE IS
USABLE BOTH BY REGULAR BLOCK READS AND CELL 3MART 3CANS %SPECIALLY IF YOU RUN AN /,40 DATABASE ON
%XADATA FOR BEST PERFORMANCE YOU SHOULD ATTEMPT TO SATISFY MOST SINGLE BLOCK READS FROM EITHER THE
DATABASE BUFFER CACHE OR FAILING THAT THE CELL FLASH CACHE 9OU CAN READ MORE ABOUT THE %XADATA &LASH
#ACHE IN #HAPTER
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
cell index scans
4HIS STATISTIC IS INCREMENTED EVERY TIME A 3MART 3CAN IS STARTED ON A "
4REE OR BITMAP INDEX SEGMENT
.OTE THAT IN ORDER TO USE 3MART 3CANS ON INDEX SEGMENTS THE index fast full scan EXECUTION PLAN ROW
SOURCE OPERATOR MUST BE USED TOGETHER WITH DIRECT PATH READS 4HIS STATISTIC IS UPDATED AT THE START OF A
3MART 3CAN SESSION SO IF YOU MONITOR ITS VALUE FOR A SESSION THAT HAS BEEN EXECUTING A LONG RUNNING
QUERY FOR A WHILE THEN YOU MIGHT NOT SEE THIS STATISTIC INCREMENTING FOR YOUR SESSION
7HEN RUNNING JUST A SERIAL SESSION WITH 3MART 3CAN ON A NONPARTITIONED INDEX SEGMENT THIS STATISTIC
WOULD BE INCREMENTED BY ONE (OWEVER WHEN RUNNING A 3MART 3CAN ON A PARTITIONED INDEX SEGMENT
THEN THE cell index scans STATISTIC WOULD BE INCREMENTED FOR EACH PARTITION SCANNED USING 3MART 3CAN
4HIS IS BECAUSE THE 3MART 3CAN IS A SEGMENT LEVEL DECISION MADE AT RUNTIME FOR EACH SEGMENT TABLE
INDEX OR PARTITION "ECAUSE THE 3MART 3CAN KICKS IN ONLY WHEN DOING DIRECT PATH READS TO 0'! AND THE
DIRECT PATH READS DECISION IS MADE BASED ON THE SCANNED SEGMENT SIZE AMONG OTHER THINGS DIFFERENT
PARTITIONS OF THE SAME TABLE ACCESSED MAY BE SCANNED USING DIFFERENT METHODS 9OU MIGHT FIND THAT SOME
PARTITIONS IN YOUR MULTIPARTITION TABLE OR INDEX ARE NOT SCANNED WITH 3MART 3CANDIRECT PATH READS AS
/RACLE HAS DECIDED TO USE BUFFERED READS FOR THEM THANKS TO THEIR SMALLER SIZE )N THIS CASE THE cell index
scans STATISTIC WOULD NOT INCREMENT AS MUCH AND YOU WOULD SEE THE cell multiblock physical read WAIT
EVENT POP UP AT THE TABLEINDEX SCAN ROW SOURCE PATH IN AN !3( OR 31, -ONITORING REPORT
cell IO uncompressed bytes
4HIS STATISTIC SHOWS THE uncompressed SIZE OF THE DATA SCANNED IN THE CELL 3O IF YOU SCAN THROUGH A '"
COMPRESSED SEGMENT physical read total bytes STATISTIC WOULD INCREASE BY '" BUT THE cell I/O
uncompressed bytes MAY WELL INCREASE BY '" IF THATS THE TOTAL UNCOMPRESSED SIZE OF THE DATA 4HIS
STATISTIC IS INCREMENTED ONLY WHEN PERFORMING A 3MART 3CAN COMPRESSION OFFLOADING NOT WHEN YOU READ
THE COMPRESSED BLOCKS DIRECTLY TO THE DATABASE LAYER WITH BLOCK )/
cell num fast response sessions
4HIS STATISTIC SHOWS HOW MANY TIMES /RACLE STARTED THE 3MART 3CAN CODE BUT THEN CHOSE NOT TO SET UP THE
3MART 3CAN SESSION IMMEDIATELY BUT INSTEAD TO DO A FEW BLOCK )/ OPERATIONS FIRST IN HOPE TO FIND
ENOUGH ROWS TO SATISFY THE DATABASE SESSION 4HIS OPTIMIZATION IS USED U^a58ABCA>FB EXECUTION PLAN
OPTIONS EITHER WHEN USING A 58ABCNA>FBNg HINT OR EQUIVALENT X]Xc ^aP PARAMETER OR A F74A4a^f]d\+
G CONDITION WHICH MAY ALSO ENABLE THE FIRST ROWS OPTION IN EXECUTION PLANS 4HE IDEA IS THAT IF FETCHING
ONLY A FEW ROWS /RACLE HOPES TO AVOID THE OVERHEAD OF SETTING UP A CELL 3MART 3CAN SESSION WITH ALL THE
CELLS THANKS TO !3- STRIPING BUT IT WILL DO A FEW REGULAR BLOCK )/ OPERATIONS FIRST &OLLOWING IS AN
EXAMPLE OF A FIRST ROWS OPTIMIZATION THANKS TO THE A>F=D< PREDICATE
B4;42C5A><c"F74A4^f]Ta[XZTB0=3aa^f]d\+,
?[P]WPbWeP[dT)" !'%&" &#
k8Sk>_TaPcX^]k=P\Tk4 A^fbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkk# k
k k2>D=CBC>?:4Hkkkk
k!kC01;40224BBBC>A0645D;;
58ABCAA>FBkC"k k# k
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
UX[cTaA>F=D<+,
! bc^aPVT>F=4A;8:4B
UX[cTa>F=4A;8:4B
7HEN EXAMINING EB4BB8>= STATISTICS OF A 3MART 3CAN OF SUCH AN EXECUTION PLAN YOU ARE LIKELY TO SEE THE
cell num fast response sessions INCREMENTED AS /RACLE HAS TRIED TO AVOID THE 3MART 3CAN SESSION SETUP
OVERHEAD
B@;-B4;42C5A><C01;4TgPcTbc SXUURT[[]d\UPbc *
=0<4E0;D4
RT[[]d\UPbcaTb_^]bTbTbbX^]b
RT[[]d\UPbcaTb_^]bTbTbbX^]bR^]cX]dX]Vc^b\PacbRP]
4HE CELL FAST RESPONSE FEATURE IS CONTROLLED BY THE _kcfis_fast_response_enabled PARAMETER AND ENABLED
BY DEFAULT
cell num fast response sessions continuing to smart scan
4HIS STATISTIC SHOWS HOW MANY TIMES THE CELL 3MART 3CAN FAST RESPONSE SESSION WAS STARTED BUT /RACLE
HAD TO SWITCH TO THE REAL 3MART 3CAN SESSION AS IT DIDNT FIND ENOUGH MATCHING ROWS WITH THE FIRST FEW )/
OPERATIONS 4HE NEXT EXAMPLE QUERY HAS THE FIRST ROWS OPTIMIZATION IN THE EXECUTION PLAN BUT THE
ADDITIONAL PREDICATE ON THE ^QYTRcN]P\T COLUMN ENSURES THAT NO MATCHING ROWS ARE FOUND DURING FIRST
FEW )/S FROM THE TABLE
B4;42C5A><c"F74A4^f]Ta[XZTB00=3^QYTRcN]P\T;8:4]^] TgXbcT]c
0=3a^f]d\+,
?[P]WPbWeP[dT)" !'%&" &#
k8Sk>_TaPcX^]k=P\Tk4 A^fbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkk( k
k k2>D=CBC>?:4Hkkkk
k!kC01;40224BBBC>A0645D;;5
58ABCA>FBkC"k k( k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
UX[cTaA>F=D<+,
! bc^aPVT>1942CN=0<4;8:4]^] TgXbcT]c0=3>F=4A;8:4
B0=3>1942CN=0<48B=>C=D;;
UX[cTa>1942CN=0<4;8:4]^] TgXbcT]c0=3>F=4A;8:4
B0=3>1942CN=0<48B=>C=D;;
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
B@;-B4;42C5A><C01;4TgPcTbc SXUURT[[]d\UPbc *
=0<4E0;D4
RT[[]d\UPbcaTb_^]bTbTbbX^]b
RT[[]d\UPbcaTb_^]bTbTbbX^]bR^]cX]dX]Vc^b\PacbRP]
4HESE STATS SHOW THAT A FAST RESPONSE SESSION WAS INVOKED BUT IT HAD TO SWITCH TO 3MART 3CAN LATER ON
)NTERESTINGLY THIS OPTIMIZATION CAUSES THE /RACLE WAIT INTERFACE TO REPORT SOME db file sequential read
WAIT EVENTS WHICH WE THOUGHT WERE IMPOSSIBLE ON %XADATA CELL STORAGE BECAUSE REGULAR CELL SINGLE BLOCK
READS ARE REPORTED AS cell single block physical read WAIT EVENTS
?0AB8=68=2DAB>A#&#!$!&&(( "%[T],("ST_, dXS,("^Rc,"[XS,("cX\, " !% ! %$#!&'!'We
B4;42C5A><c"F74A4^f]Ta[XZTB0=3^QYTRcN]P\T;8:4]^] TgXbcT]c0=3a^f]d\
+,
4=3>5BC<C
?0AB4#&#!$!&&(( "%)R,'(((T,(('"_,!Ra,"Rd, \Xb, a, ST_, ^V, _[W," !'%&" &#
4G42#&#!$!&&(( "%)R, T, (_, Ra, Rd, \Xb, a, ST_, ^V, _[W," !'%&" &#
F08C#&#!$!&&(( "%)]P\,B@;=Tc\TbbPVTc^R[XT]cT[P,!SaXeTaXS, # "%(&$"%QhcTb,
F08C#&#!$!&&(( "%)]P\,B@;=Tc\^aTSPcPc^R[XT]cT[P, &SaXeTaXS, # "%(&$"%
F08C#&#!$!&&(( "%)]P\,RT[[bX]V[TQ[^RZ_WhbXRP[aTPST[P,$ $RT[[WPbW,! ""#$(#'"
F08C#&#!$!&&(( "%)]P\,aT[XPQ[T\TbbPVTT[P, #&"RWP]]T[R^]cTgc, '''"# &'#
F08C#&#!$!&&(( "%)]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P,!(']P\Tk\^ST, !%"#%'$$
F08C#&#!$!&&(( "%)]P\,T]`):> UPbc^QYTRcRWTRZ_^X]cT[P, $#]P\Tk\^ST, !%"#%'$#$
F08C#&#!$!&&(( "%)]P\,Pbh]RWSTbRaX_c^aaTbXiTT[P,!^dcbcP]SX]VPX^,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,#UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P, $(!UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P, UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,#!&UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P, UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,"$ UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,!UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,!!'$&UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,!UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,!"%"%UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,"UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,SQUX[TbT`dT]cXP[aTPST[P,'''' UX[T, Q[^RZ, Q[^RZb,
F08C#&#!$!&&(( "%)]P\,RT[[b\PaccPQ[TbRP]T[P, " RT[[WPbW,! ""#$(#'"_!, _",
F08C#&#!$!&&(( "%)]P\,RT[[b\PaccPQ[TbRP]T[P, !%RT[[WPbW,"&(""(($'_!, _",
F08C#&#!$!&&(( "%)]P\,RT[[b\PaccPQ[TbRP]T[P, !"RT[[WPbW," &%$(## (_!, _",
cell num smart IO sessions using passthru mode due to reason
4HESE THREE STATISTICS WHERE reason CAN BE EITHER dbTa RT[[bae OR cX\Ti^]T INDICATE HOW MANY TIMES
/RACLE DATABASE INITIATED A 3MART 3CAN BUT THE RT[[bae DIDNT START A 3MART 3CAN AND FELL BACK ENTIRELY TO
BLOCK )/ MODE 4HE BLOCKS READ ARE JUST PASSED THROUGH RT[[bae TO DATABASE INSTEAD OF PROCESSING
THEM WITHIN THE CELL 4HIS MEANS THAT WHILE YOU STILL SEE THE cell Smart Scan WAIT EVENTS AND cell physical
IO interconnect bytes returned by smart scan INCREASING WHICH INDICATES THAT A 3MART 3CAN IS HAPPENING
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
THE FULL POWER OF 3MART 3CAN IS NOT UTILIZED AS THE CELLS JUST READ DATA BLOCKS AND RETURN THE BLOCKS BACK
TO THE DATABASE LAYER )N OTHER WORDS IN PASSTHROUGH MODE THE CELLS DO NOT OPEN DATABLOCKS AND EXTRACT
ONLY THE REQUIRED COLUMNS OF MATCHING ROWS BUT RETURN ALL THE PHYSICAL BLOCKS OF THE SEGMENT AS THEY
ARE .OTE THAT THE STORAGE INDEXES DYNAMICALLY CREATED IN RT[[bae MEMORY CAN BE USED TO ELIMINATE )/
IN THE PASSTHROUGH MODE TOO BUT THESE INDEXES MUST BE POPULATED BY A REGULAR 3MART 3CAN WHICH DOES
ACTUALLY OPEN AND PROCESS THE DATA BLOCKS IN THE CELL
9OU SHOULD NOT SEE ANY PASSTHROUGH 3MART 3CANS HAPPENING ON THE LATEST DATABASE AND %XADATA CELL
VERSIONS UNLESS YOU HAVE PROBLEMS LIKE CELLS RUNNING OUT OF MEMORY AND THE LIKE 9OU CAN TEST WHAT
HAPPENS IN A TEST
ENVIRONMENT BY SETTING _kcfis_cell_passthru_enabled TO 425% AND RUNNING A 3MART
3CAN 9OU WILL STILL SEE cell smart scan WAIT EVENT FOR YOUR 3MART 3CANS BUT THEY ARE SLOWER BECAUSE THEY
ARE RETURNING ALL THE BLOCKS TO THE DATABASE FOR PROCESSING
cell physical IO bytes eligible for predicate offload
4HIS PERFORMANCE COUNTER HOLDS ONE OF THE MOST IMPORTANT STATISTICS FOR UNDERSTANDING CELL 3MART 3CAN
7HEN YOU ARE 3MART 3CANNING THROUGH A SEGMENT THIS STATISTIC SHOWS HOW MANY BYTES OF THAT SEGMENT
THE 3MART 3CAN WOULD GO THROUGH %SSENTIALLY ALL THE BYTES FROM THE BEGINNING OF THE SEGMENT ALL THE WAY
TO ITS HIGH WATER MARK WILL BE REPORTED AS THE SCANNING PROGRESSES THROUGH THE ENTIRE SEGMENT 4HE
CATCH IS THAT THIS IS THE THEORETICAL MAXIMUM NUMBER OF BYTES TO SCAN THROUGH BUT SOMETIMES THE CELL
STORAGE INDEX OPTIMIZATION KICKS IN ALLOWING THE SCAN TO SKIP READING SOME BLOCK RANGES %VEN IF THE
STORAGE INDEX ALLOWS YOU TO AVOID SCANNING PERCENT OF A '" SEGMENT REDUCING THE ACTUAL )/
AMOUNT TO ONLY '" THIS STATISTIC STILL SHOWS THE total SIZE OF THE SEGMENT SCANNED REGARDLESS OF ANY
OPTIMIZATIONS )N OTHER WORDS THIS STATISTIC SHOWS THE TOTAL BYTES THAT WOULD HAVE BEEN SCANNED THROUGH
IF ANY CELL LEVEL OPTIMIZATIONS DIDNT KICK IN AND OFTEN THIS IS THE CASE .OTE THAT THIS STATISTIC SIMPLY
COUNTS THE PHYSICAL SIZE OF THE SEGMENT IN THE DATA BLOCKS IN DATA FILES AND NOT THE hEVENTUALv DATA SIZE
AFTER ANY DECOMPRESSION FILTERING OR PROJECTION
)F THIS NUMBER DOES NOT INCREASE FOR YOUR SESSIONS EB4BBC0C OR 3TATSPACK!72 DATA WHEN LOOKING
AT THE WHOLE INSTANCE THEN ITS ANOTHER INDICATOR THAT 3MART 3CANS ARENT BEING USED !NY BLOCK RANGES
SCANNED THROUGH OR EVEN SKIPPED THANKS TO STORAGE INDEX OPTIMIZATIONS BY A 3MART 3CAN SESSION
SHOULD INCREMENT THIS STATISTIC ACCORDINGLY !NOTHER THING TO KNOW IS THAT WHEN THE 3MART 3CAN FALLS BACK
TO PASSTHROUGH FULL BLOCK SHIPPING MODE DESCRIBED EARLIER THE cell physical IO bytes eligible for
predicate offload STATISTIC STILL GETS INCREMENTED ALTHOUGH THERES NO PREDICATE OFFLOADING AND 3MART 3CAN
FILTERING DONE IN THE CELL IN PASSTHROUGH MODE
cell physical IO bytes pushed back due to excessive CPU on cell
4HIS STATISTIC IS RELATED TO THE SELECTIVE FALLBACK TO PASSTHROUGH MODE AS EXPLAINED IN +EVINS
ACCOMPANYING NOTE 4HE BIGGER THIS NUMBER IS COMPARED TO THE cell physical I/O bytes eligible for
predicate offload STATISTIC THE MORE THE CELL HAD TO SEND BACK COMPRESSED #5S FOR DECOMPRESSION AND
PROCESSING IN THE DATABASE LAYER 4HIS HAPPENS WHEN THE CELLS #05 UTILIZATION REACHES A THRESHOLD AND
THERE IS ENOUGH SPARE #05 CAPACITY AVAILABLE IN THE DATABASE LAYER ALLOWING YOU TO UTILIZE THE #05S OF
YOUR WHOLE %XADATA CLUSTER BETTER
.OTE THAT THIS PERFORMANCE COUNTER APPEARED AS PART OF THE CELL SOFTWARE VERSION AND
DATABASE "UNDLE 0ATCH )T WAS APPARENTLY ADDED AS A TEMPORARY ONE OFF FIX AS THIS STATISTIC REPLACED THE
AD HOC “spare statistic 1” COUNTER IN THE 6 VIEWS )N FACT IN /RACLE AND %XADATA "UNDLE 0ATCH
THIS STATISTIC IS RENAMED TO “cell physical IO bytes sent directly to DB node to balance CPU usage.”
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
+EVIN 3AYS h2EGARDING PASSTHROUGH FUNCTIONALITY THE AUTHORS ARE CORRECT IN ASSERTING THAT PASSTHROUGH IS
GENERALLY A SIGN OF PROBLEMS (OWEVER LATE RELEASES OF %XADATA 3TORAGE 3ERVER SOFTWARE HAVE A FEATURE THAT TRIES
TO MITIGATE THE IMBALANCES BETWEEN PROCESSOR UTILIZATION IN THE STORAGE CELLS AND THAT OF DATABASE HOSTS UNDER
STORAGE
INTENSIVE PROCESSING 4HE DEFINITION OF hA PROBLEMv IS SUBJECTIVE )N SITUATIONS WHERE 3MART 3CAN OFFLOAD
PROCESSING IS SATURATING THE STORAGE PROCESSORSTHUS CREATING A BOTTLENECK FOR DATA FLOWQUERY COMPLETION TIME
IS SEVERELY IMPACTED 4HAT IS A PROBLEM 3OLUTIONS CAN SOMETIMES BE PROBLEMS AS WELL
)N THESE MORE RECENT RELEASES OF RT[[bae THERE IS A HINT ABOUT DATABASE GRID PROCESSOR UTILIZATION IN THE METADATA
OF EACH 3MART 3CAN )/ REQUEST PASSED OVER I$" TO STORAGE )F STORAGE IS SIGNIFICANTLY BUSIER THAN THE HOSTS #05
WISE RT[[bae WILL SEND DATA TO THE FOREGROUND PROCESSES AFTER NO OR VERY LITTLE OFFLOAD PROCESSINGDEPENDING
ON WHETHER THERE IS (## INVOLVED OR NOT 4HE TERM passthrough REFERS TO THOSE TIMES WHEN 3MART 3CAN chooses TO
SKIP OFFLOAD PROCESSING !S OF THE PUBLICATION OF THIS BOOK THE ONLY REASON 3MART 3CAN chooses TO SKIP OFFLOAD
PROCESSING IS FOR REASONS OF PROCESSOR SATURATION 4HERE MAY BE OTHER TRIGGERS IN THE FUTURE 4HE INFRASTRUCTURE IS IN
PLACE
4HE AUTHORS AND ) TRY TO IMPART THE CONCEPTS ABOUT WHEN A 3MART 3CAN WILL OCCUR AND WHEN A 3MART 3CAN WILL
need TO REVERT TO BLOCK MODE PROCESSING (ERE ON THE OTHER HAND WE HAVE A CASE WHERE 3MART 3CAN SIMPLY
chooses TO CEASE OFFLOAD PROCESSING FOR A SIGNIFICANT PERCENTAGE OF THE DATA BEING READ FROM DISK 4HIS MIDDLE
GROUND HOWEVER IS NOT A FALL
BACK TO BLOCK MODE PROCESSING BECAUSE THE BUFFERING IS STILL 0'! DIRECT PATH
READS 4HE TERM /RACLE HAS CHOSEN FOR THIS MIDDLE
GROUND SUMS IT UP THE BEST passthrough 4O THAT END WE
COULD SAY THERE IS NOW ANOTHER TYPE OF SCAN 0ARTIALLY 3MART 3CAN
3O HOW MUCH 3MART 3CAN VALUE REMAINS WHEN IN PASSTHROUGH MODE 7ELL IT IS STILL A 3MART 3CAN SO STORAGE
INDEXES CAN STILL BE OF BENEFIT (OWEVER FILTRATION WILL NOT OCCUR IN STORAGE 0ROJECTION ON THE OTHER HAND WILL OCCUR
IN STORAGE ONLY IF THE DATA BEING SCANNED IS %(## 9OU SEE PROJECTING %(## TUPLES IS VERY INEXPENSIVE AND AS WE
KNOW REDUCING PAYLOAD IS THE CRITICAL SERVICE 3MART 3CAN AIMS TO DELIVER 3O IF PAYLOAD REDUCTION CAN OCCUR THAT IS
A GOOD THING
3O WHAT DOES ALL THIS MEAN 7ELL IDLE PROCESSORS ARE A VERY BAD THING WITH $7") WORKLOADS SO THIS NEW BEHAVIOR
COULD BE A GOOD THING 4HE ONLY ASPECT OF THE FEATURE THAT MAKES ME WARY IS THAT FACTORS EXTERNAL TO THE QUERY CAN
CHANGE THE RUNTIME DYNAMIC #ONSIDER FOR INSTANCE A QUERY WITH A GOOD EXECUTION PLAN THAT TAKES SECONDS TO
COMPLETE WHEN PROCESSING WITHOUT OTHER CONCURRENT QUERIES 3UPPOSE FURTHER THAT THIS QUERY BURDENS THE STORAGE
PROCESSORS TO SOME PERCENT 7HEN EXECUTED CONCURRENTLY WITH ANOTHER QUERY THAT UTILIZES THE REMAINING
PERCENT STORAGE PROCESSOR BANDWIDTH 3MART 3CAN WILL STOP OFFLOAD PROCESSING FOR some AMOUNT OF THE DATA READ
FROM DISKFOR BOTH OF THESE QUERIES 4HIS IS A CASE OF HYSTERESIS
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
)F THIS NEW FEATURE STANDS THE TEST OF TIME IT WILL ADD MANY VARIABLES IN MULTIPLE DIMENSIONS TO PERFORMANCE
ANALYSIS )N THE END HOWEVER THE ONLY THING THAT REALLY MATTERS IS WHETHER THE USERS RESPONSIVENESS FROM THE
SYSTEM IS ACCEPTABLE ) CHOOSE TO REMAIN DUBIOUS HOWEVER BECAUSE OF THE ADDED COMPLEXITY )T HAS TAKEN MANY
CHAPTERS OF THIS WELL WRITTEN BOOK TO EXPLAIN WHEN 3MART 3CAN AND THUS OFFLOAD PROCESSING WILL OCCUR .OW WE
HAVE A 3MART 3CAN SITUATION THAT DOESNT PERFORM OFFLOAD PROCESSING FOR VARYING PERCENTAGES OF THE DATA STREAMING
OFF OF STORAGE )T HAS BEEN SAID MANY TIMES THAT OFFLOAD PROCESSING IS THE hSECRET SAUCEv OF %XADATA AND OF COURSE
3MART 3CAN IS THE MAIN AGENT FOR DELIVERY )T IS YET TO BE SEEN HOW COMPLEX WORKLOADS BEHAVE WHEN 3MART 3CAN
SOMETIMES CHOOSES NOT TO DO ITS JOB
cell physical IO bytes saved by storage index
4HIS IS ANOTHER IMPORTANT STATISTIC WHICH SHOWS HOW MANY BYTES THE 3MART 3CAN SESSIONS COULD SIMPLY
SKIP READING FROM DISK THANKS TO THE IN MEMORY STORAGE INDEX IN RT[[bae )F THIS STATISTIC cell physical IO
bytes saved by storage index IS CLOSE TO cell physical IO bytes eligible for predicate offload IT INDICATES THAT
3MART 3CANS ARE GREATLY BENEFITING FROM STORAGE INDEXES AND HAVE AVOIDED DISK READS THANKS TO THAT
cell physical IO bytes saved during optimized file creation
4HIS STATISTIC SHOWS HOW MANY BYTES WORTH OF DATA FILE CREATION AND FILE EXTENSION WORK WAS DONE IN THE
CELL 7ITHOUT THIS OPTIMIZATION /RACLE WOULD FORMAT THE NEW DATA BLOCKS IN THE DATABASE SESSIONS 0'!
AND SEND PHYSICAL BLOCK WRITES TO THE CELL OVER INTERCONNECT )N %XADATA THE FILE BLOCK FORMATTING AND
WRITES CAN BE OFFLOADED TO THE CELL AND THE DATABASE DOES NOT HAVE DO PHYSICAL WRITES ITSELF )N %XADATA
YOU WOULD SEE A cell smart file creation WAIT EVENT WHEN CELLS ARE FORMATTING BLOCKS FOR NEW FILES OR WHEN
EXTENDING FILES 7ITHOUT THIS OPTIMIZATION YOU WOULD SEE THE Data file init write WAIT EVENTS
cell physical IO bytes saved during optimized RMAN file restore
4HIS STATISTIC SHOWS HOW MANY BYTES WORTH OF DATAFILES WERE RESTORED WITHIN CELLS WITHOUT HAVING TO
INVOLVE THE DATABASE LAYER THANKS TO OFFLOADING THE 2-!. RESTORE TO CELLS 9OU SHOULD SEE YOUR RECOVERY
SESSIONS WAITING FOR cell smart restore from backup WAIT EVENT WHEN RESTORING ON %XADATA RMAN backup
& recovery I/O WAIT EVENT
cell physical IO bytes sent directly to DB node to balance CPU usage
4HIS STATISTIC IS THE SAME AS THE hCELL PHYSICAL )/ BYTES PUSHED BACK DUE TO EXCESSIVE #05 ON CELLv STATISTIC
EARLIER WHICH IS USED IN /RACLE WITH "UNDLE 0ATCH BUT IN NEWER /RACLE VERSIONS AND PATCH
LEVELS THE CURRENT STATISTIC NAME IS USED INSTEAD
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
cell physical IO interconnect bytes
4HIS IS A SIMPLE BUT FUNDAMENTAL STATISTIC WHICH SHOWS HOW MANY BYTES WORTH OF ANY DATA HAVE BEEN
TRANSMITTED BETWEEN THE STORAGE CELLS AND YOUR DATABASE SESSIONS 4HIS INCLUDES ALL DATA BOTH SENT AND
RECEIVED BY THE DATABASE THE 3MART 3CAN RESULT SETS FULL BLOCKS READ FROM THE CELLS TEMPORARY )/ READS
AND WRITES LOG WRITES IF ,'72 ANY SUPPLEMENTARY I$" TRAFFIC AND SO ON 3O THIS STATISTIC SHOWS all
traffic bytes REGARDLESS OF ITS DIRECTION CONTENTS OR NATURE
7HEN MEASURING THE WRITE )/ METRICS DONT BE SURPRISED WHEN YOU SEE THAT THE cell physical IO
interconnect bytes STATISTIC IS TWO OR THREE TIMES HIGHER THAN THE physical write total bytes STATISTIC 4HIS IS
BECAUSE THE LATTER STATISTIC IS MEASURED AT THE /RACLE DATABASE LEVEL BUT THE cell physical IO interconnect
bytes IS MEASURED AT THE CELL LEVEL AFTER !3- MIRRORING HAS BEEN DONE 3O IF FOR EXAMPLE ,'72 WRITES
-" TO AN !3- DISK GROUP WITH HIGH REDUNDANCY TRIPLE MIRRORING A TOTAL OF -" OF DATA WOULD BE
SENT OVER THE INTERCONNECT .OTE THAT THERE IS A BUG IN /RACLE THAT INCREMENTS THIS
STATISTIC EVEN ON NON %XADATA DATABASES SO YOU CAN JUST IGNORE IT ON NON %XADATA DATABASES
cell physical IO interconnect bytes returned by smart scan
4HIS STATISTIC IS AN IMPORTANT ONE AND SHOWS HOW MANY BYTES OF DATA WERE RETURNED TO THE DATABASE LAYER
BY THE 3MART 3CAN 4HE BYTES ACTUALLY RETURNED SHOULD BE MUCH LESS THAN THE BYTES SCANNED READ FROM
DISK 4HIS IS THE MAIN POINT OF THE %XADATA 3MART 3CAN FEATURE THE CELLS MAY READ GIGABYTES OF DATA
EVERY SECOND BUT AS THEY PERFORM EARLY FILTERING THANKS TO PREDICATE OFFLOADING THEY MAY SEND ONLY A
SMALL PART OF THE ROWS BACK TO THE DATABASE LAYER !DDITIONALLY THANKS TO THE PROJECTION OFFLOADING THE
3MART 3CANS ONLY RETURN THE REQUESTED COLUMNS BACK NOT FULL ROWS /F COURSE IF THE APPLICATION USES
B4;42C FOR FETCHING ALL THE COLUMNS OF A TABLE PROJECTION OFFLOADING WOULD NOT HELP BUT THE EARLY
FILTERING THANKS TO PREDICATE OFFLOADING CAN STILL HELP A LOT
4HIS STATISTIC IS A SUBSET OF THE cell physical IO interconnect bytes STATISTIC BUT IT COUNTS ONLY THE BYTES
OF ROW SETS THAT ARE RETURNED BY 3MART 3CAN SESSIONS AND NO OTHER TRAFFIC
cell scans
4HIS STATISTIC IS JUST LIKE THE cell index scans BUT cell scans SHOWS THE NUMBER OF 3MART 3CANS DONE ON
TABLE AND MATERIALIZED VIEW SEGMENTS 7ITH SERIAL EXECUTION THIS STATISTIC IS INCREMENTED ONCE IN THE
BEGINNING OF EVERY SEGMENT SCAN 3O WHEN SCANNING THROUGH A PARTITIONED TABLE IN WHICH EACH PARTITION
IS A SEPARATE SEGMENT THIS STATISTIC WOULD BE INCREMENTED FOR EACH PARTITION 7ITH PARALLEL SCANS THE cell
scans STATISTIC WILL INCREMENT EVEN MORE AS PARALLEL SLAVES PERFORM THEIR SCANS ON BLOCK RANGES 08
GRANULES HANDED OVER TO THEM BY THE QUERY COORDINATOR SO THE SCAN ON EACH BLOCK RANGE IS REPORTED AS
A SEPARATE cell scan. 4HE PRESENCE OF THE table scans (rowid ranges) STATISTIC INDICATES THAT 08 SCANS ON
BLOCK RANGES ARE HAPPENING
&OLLOWING IS AN EXAMPLE OF A PARALLEL FULL TABLE SCAN SESSIONS STATISTICS WITH TWO SLAVES 4HE
'" TABLE WAS APPARENTLY hSPLITv INTO ROWID RANGES AND SCANNED USING 3MART 3CANS AND THANKS
TO THE 3MART 3CANS OFFLOADING FEATURES ONLY '" WORTH OF RESULTING DATA HAD TO BE SENT TO THE
DATABASE
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
B83DB4ACH?4BC0C8BC82734;C0
# C0=4;BC0C_WhbXRP[aTPSc^cP[QhcTb "" '6
# C0=4;BC0CRT[[_WhbXRP[8>X]cTaR^]]TRcQhcTb# $#6
# C0=4;BC0CRT[[b\Pac8>bTbbX^]RPRWT[^^Zd_b "
# C0=4;BC0CRT[[b\Pac8>bTbbX^]RPRWTWXcb "
# $#6
# C0=4;BC0CRT[[_WhbXRP[8>X]cTaR^]]TRcQhcTbaTcda]TSQhb\PacbRP]#
# C0=4;BC0CcPQ[TbRP]b[^]VcPQ[Tb "!
# C0=4;BC0CcPQ[TbRP]ba^fXSaP]VTb "!
# C0=4;BC0CcPQ[TbRP]bSXaTRcaTPS "!
# C0=4;BC0CRT[[bRP]b "!
cell smart IO session cache hits
4HIS STATISTIC SHOWS HOW MANY TIMES A DATABASE SESSION MANAGED TO REUSE A PREVIOUSLY INITIALIZED 3MART
3CAN SESSION IN THE CELL 4HIS STATISTIC SHOWS UP WHEN A SINGLE EXECUTION PLAN SCANS THROUGH MULTIPLE
SEGMENTS LIKE WITH PARTITIONED TABLES OR REVISITS THE SAME SEGMENT DURING A SINGLE EXECUTION
cell smart IO session cache lookups
4HIS STATISTIC IS INCREMENTED EVERY TIME A DATABASE SESSION TRIED TO REUSE A PREVIOUSLY INITIALIZED 3MART
3CAN SESSION IN THE CELL )F THE cell smart IO session cache hits STATISTIC INCREMENTS TOO THEN THE LOOKUP
WAS SUCCESSFUL AND A PREVIOUS SESSION CAN BE REUSED 4HE SMART )/ SESSION CACHING WORKS ONLY WITHIN
AN EXECUTION AND SUBSEQUENT FETCHES OF AN OPEN CURSOR /NCE THE EXECUTION FINISHES THE NEXT
EXECUTIONS EVEN OF THE SAME CURSOR WOULD HAVE TO SET UP NEW 3MART 3CAN SESSIONS AND COMMUNICATE
THE NEW CONSISTENT READ SNAPSHOT 3#. TO THE CELLS TOO
cell transactions found in commit cache
4HIS STATISTIC IS RELATED TO THE CONSISTENT READ MECHANISM /RACLE HAS TO GUARANTEE EVEN ON %XADATA )T
SHOWS HOW MANY TIMES THE 3MART 3CAN SESSIONS CHECKED THE CELL COMMIT CACHE TO DECIDE WHETHER A #2
ROLLBACK IS NEEDED OR NOT AND IT FINDS THE TRANSACTION STATUS INFORMATION IN THE CELL COMMIT CACHE 4HIS
AVOIDS A ROUND TRIP TO THE DATABASE LAYER TO CHECK THAT TRANSACTIONS STATUS USING UNDO DATA AVAILABLE
THERE 4HERES MORE INFORMATION ABOUT HOW THE CONSISTENT READS WORK WITH %XADATA 3MART 3CANS IN THE
cell blocks helped by commit cache STATISTIC SECTION
chained rows processed by cell
"EFORE EXPLAINING WHAT THIS SPECIFIC STATISTIC MEANS LETS LOOK AT HOW 3MART 3CANS DEAL WITH CHAINED
ROWS #HAINED ROWS POSE A PROBLEM FOR 3MART 3CANS 4HE CHAINED ROWS hNEXTv ROW PIECES MAY BE
ANYWHERE IN THE SEGMENT AND THANKS TO THE !3- STRIPING ITS NOT GUARANTEED THAT THE NEXT ROW PIECES OF
A CHAINED ROW ARE IN THE SAME CELL WHERE THE ROW HEAD PIECE IS 3O A CHAINED ROW MAY BE PHYSICALLY SPLIT
ACROSS MULTIPLE DIFFERENT CELLS 'IVEN THAT CELLS NEVER SPEAK TO EACH OTHER FOR SCALABILITY AND SIMPLICITY
HOW WOULD YOU BE ABLE TO CONSTRUCT THE FULL ROW WHEN NEEDED 4HE WAY RT[[bae CURRENTLY SOLVES THIS
PROBLEM IS THAT WHENEVER THE 3MART 3CAN HITS A CHAINED ROW AND REALIZES IT HAS TO FETCH ITS NEXT ROW
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
PIECE RT[[bae FALLS BACK TO REGULAR BLOCK )/ FOR THAT ROW AND SENDS THE BLOCK BACK TO THE DATABASE LAYER
FOR NORMAL PROCESSING 4HE DATABASE LAYER CAN THEN EXTRACT THE DATA BLOCK ADDRESS OF THE NEXT ROW PIECE
FROM THE ROW HEAD PIECE AND ISSUE THE BLOCK READ )/ TO THE APPROPRIATE CELL WHERE THE !3- STRIPING HAS
PHYSICALLY PUT THAT BLOCK 4HE REASONING AND FUNDAMENTAL PROBLEM BEHIND THIS OPTIMIZATION IS SIMILAR TO
WHY CONSISTENT READ ROLLBACKS HAVE TO BE DONE IN THE DATABASE LAYER AS OPPOSED TO A CELLITS BECAUSE
SOME OF THE DATA BLOCKS REQUIRED FOR THIS OPERATION MAY JUST HAPPEN TO BE LOCATED IN ANOTHER CELL AND
CELLS NEVER TALK TO OTHER CELLS
4HIS BEHAVIOR MEANS THAT YOUR 3MART 3CAN PERFORMANCE WILL DROP IF IT HITS A LOT OF CHAINED ROWS AND
HAS TO FETCH THEIR NEXT ROW PIECES )F YOU GET LUCKY AND ACCESS ONLY THE COLUMNS THAT ARE PRESENT IN THE
HEAD PIECE OF THE ROW THEN YOU WONT HAVE TO FALL BACK TO DATABASE BLOCK )/ MODE FOR THESE BLOCKS AND
YOUR 3MART 3CANS WILL BE FAST )F YOU HAVE TO FETCH THE NEXT ROW PIECES HOWEVER THEN YOUR 3MART 3CAN
WILL BE CONSTANTLY INTERRUPTED BY THE FALL
BACK TO BLOCK )/ AND THE DATABASE LAYER STARTS DOING LOGICAL
READS AND POSSIBLY SINGLE
BLOCK PHYSICAL READS AS WELL IF THESE BLOCKS ARE NOT CACHED IN BUFFER CACHE
4HIS MEANS THAT YOUR QUERY ENDS UP WAITING MOST OF ITS TIME FOR RANDOM SINGLE
BLOCK READS AS OPPOSED
TO HIGH PERFORMANCE 3MART 3CANNING &OLLOWING IS THE RESPONSE TIME PROFILE FROM A TEST SESSION HITTING
MANY CHAINED ROWS AND SPENDING PERCENT OF THE RESPONSE TIME ON cell single block physical reads
WAITS 4HANKFULLY THERE IS A STATISTIC table fetch continued row WHICH IS USEFUL ALSO ON NON
%XADATA
DATABASES WHICH COUNTS HOW MANY TIMES WE HAD TO FOLLOW A hNEXTv ROW PIECE OF A CHAINED ROW )N THE
FOLLOWING CASE AROUND ROW PIECES HAD TO BE FETCHED USING SINGLE BLOCK READS
B83DB4ACH?4BC0C8BC82734;C0
%('C0=4;BC0CccPQ[TUTcRWR^]cX]dTSa^f "!Z
%('C0=4;C8<431cX\T" (%b
$ '!\b
%('C0=4;F08CRT[[b\PaccPQ[TbRP]$
%('C0=4;F08CRT[[bX]V[TQ[^RZ_WhbXRP[aTPS" ("b
0RcXeTkB@;N83k4E4=CkF08CN2;0BB
'%kPgU"h$iV%#ebdkRRT[[bX]V[TQ[^RZ_WhbXRP[aTPSkDbTa8 >
'
#kPgU"h$iV%#ebdk>=2?Dk>=2?D
.OW THIS chained rows processed by cell STATISTIC INDICATES THAT A CHAINED ROW WAS HIT BY 3MART 3CAN
and WAS PROCESSED WITHIN THE CELL EXACTLY THE OPPOSITE OF WHAT WE JUST SAID 4HIS STATISTIC GETS
INCREMENTED IN A SPECIAL CASE OF intra-block chaining IN WHICH THE NEXT ROW PIECE IS STORED IN THE SAME
BLOCK AS THE CURRENT HEAD ROW PIECE OF A CHAINED ROW )NTRA
BLOCK CHAINING IS USED FOR STORING ROWS WITH
MORE THAN COLUMNS )N SUCH CASES THE STORAGE CELL NOTICES THAT IT JUST HAS TO FETCH THE REST OF THE ROW
FROM THE SAME BLOCK WHICH IT ALREADY HAS OPENED AND CAN PROCESS THIS CHAINED ROW WITHOUT RESORTING TO
DATABASE BLOCK )/
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
.OTE 4HIS CHAINED ROW PERFORMANCE PROBLEM APPLIES ONLY TO REGULAR DATA BLOCKS AND THOSE COMPRESSED WITH
REGULAR BLOCK
LEVEL COMPRESSION ,/!$ OR /,40 ,UCKILY IT IS NOT A PROBLEM FOR %(##
COMPRESSED TABLES AT ALL AS
IN %(## THE ROWS AND COLUMNS ARE PHYSICALLY ORGANIZED DIFFERENTLY !LSO THIS ISSUE DOES NOT APPLY TO MIGRATED
ROWS WHERE THE ENTIRE ROW HAS MOVED TO ANOTHER BLOCK DUE TO AN UPDATE LEAVING ONLY A STUB HEAD PIECE BEHIND
WHEN FULL
SCANNING THROUGH A SEGMENT 4HE FULL SCAN3MART 3CAN JUST IGNORES THE HEAD PIECES OF MIGRATED ROWS
AS THE ENTIRE ROW IS PHYSICALLY ELSEWHERE .OTE THAT UPDATES CAUSE OTHER EFFICIENCY ISSUES FOR (##
COMPRESSED
TABLES EVEN IF CHAINED ROWS ARE NOT A PROBLEM
!NOTHER INTERESTING CASE OF ROW CHAINING PECULIARITIES IS WHEN YOU HAVE OVER COLUMNS IN A TABLE
%VEN WHEN THE TOTAL ROW SIZE IS SMALL ENOUGH TO FIT INSIDE A SINGLE BLOCK WITH OVER COLUMNS /RACLE
WOULD STILL DO intra-block chaining IN WHICH THE ROW IS CHAINED BUT ALL THE ROW PIECES ARE PHYSICALLY
INSIDE THE SAME BLOCK 4HIS WAS NEEDED BECAUSE /RACLE WANTED TO MAINTAIN BACKWARD COMPATIBILITY
WHEN THEY INCREASED THE COLUMN LIMIT FROM TO IN /RACLE 4HE hCOLUMN COUNTv BYTE IN A ROW
PIECE IS JUST ONE BYTE ALLOWING COLUMNS PER ROW PIECE BUT THANKS TO CHAINING YOU CAN HAVE MORE
COLUMNS IN NEXT ROW PIECES 4HIS HOWEVER CAUSED INITIALLY SOME PERFORMANCE PROBLEMS FOR %XADATA
3MART 3CANS WHERE EVEN INTRA
BLOCK ROW CHAINING RESULTED IN A LOT OF SINGLE BLOCK READS (OWEVER THIS
BUG WAS FIXED IN /RACLE %XADATA 0ATCH "UNDLE "0 AND YOU SHOULDNT SEE IT ANYMORE
)T WAS QUITE AN INTERESTING PROBLEM TO TROUBLESHOOT IF YOU ARE INTERESTED IN SEEING MORE DETAILS
INCLUDING THE TROUBLESHOOTING SEQUENCE YOU CAN READ 4ANELS h4ROUBLESHOOTING %XADATA 3MART 3CANv
ARTICLE ABOUT IT
Wcc_) Q[^V cP]T[_^STa R^\ ^aPR[T TgPSPcP ca^dQ[TbW^^cX]V
chained rows rejected by cell
4HIS STATISTIC SHOWS HOW MANY CHAINED ROWS WERE NOT PROCESSED IN THE CELL 7E DONT KNOW YET THE EXACT
REASON FOR rejection AS OPPOSED TO SKIPPING AS IN THE NEXT STATISTIC chained rows skipped by cell). $URING
TESTING MOST OF THE CHAINED ROWS WILL CAUSE THE NEXT STATISTIC TO BE INCREMENTED 3O FAR WE ASSUME THAT
THIS REJECTION IS JUST A SPECIAL CASE OF NOT PROCESSING THE CHAINED ROW IN THE CELL AND FALLING BACK TO BLOCK
)/ IN DATABASE LAYER
chained rows skipped by cell
4HIS IS ONE OF THE MAIN STATISTICS TO LOOK AT WHEN TROUBLESHOOTING hFORCEDv SINGLE BLOCK READ ISSUES IN
3MART 3CANS CAUSED BY ROW CHAINING &OLLOWING IS A MORE EXTENDED OUTPUT FROM THE PREVIOUS TEST CASE
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
B83DB4ACH?4BC0C8BC82734;C0
%('C0=4;BC0CRR^]bXbcT]cVTcb!" !Z
%('C0=4;BC0CRR^]bXbcT]cVTcbUa^\RPRWT !(Z
%('C0=4;BC0CR^]bXbcT]cVTcbUa^\RPRWTUPbc_PcW" &'Z
%('C0=4;BC0CR^]bXbcT]cVTcbSXaTRc &"Z
%('C0=4;BC0CccPQ[TUTcRWR^]cX]dTSa^f "!Z
%('C0=4;BC0CRRT[[Q[^RZb_a^RTbbTSQhRPRWT[PhTa ! ( Z
%('C0=4;BC0CRT[[Q[^RZb_a^RTbbTSQhcg][PhTa ! ( Z
%('C0=4;BC0CRT[[Q[^RZb_a^RTbbTSQhSPcP[PhTa ##Z
%('C0=4;BC0CRRWPX]TSa^fbbZX__TSQhRT[[% ''Z
%('C0=4;BC0CRWPX]TSa^fbaTYTRcTSQhRT[[!'
%('C0=4;C8<431cX\T" (%b
%('C0=4;F08CRT[[b\PaccPQ[TbRP]$ '!\b
%('C0=4;F08CRT[[bX]V[TQ[^RZ_WhbXRP[aTPS" ("b
0RcXeTkB@;N83k4E4=CkF08CN2;0BB
'%kPgU"h$iV%#ebdkRT[[bX]V[TQ[^RZ_WhbXRP[aTPSkDbTa8 >
#kPgU"h$iV%#ebdk>=2?Dk>=2?D
&ROM THESE STATS YOU CAN SEE HOW /RACLE HAS SKIPPED ABOUT CHAINED ROWS IN THE CELL AND HAS SENT
THESE BLOCKS BACK TO THE DATABASE FOR REGULAR PROCESSING !NOTHER INDICATOR IS THE table fetch continued
row STATISTIC WHICH IS SIGNIFICANTLY HIGHER THAN chained rows skipped by cell 4HIS IS BECAUSE IN THE
EXAMPLE SOME OF THE ROWS WERE SO BIG THAT THEY DIDNT EVEN FIT INTO TWO BLOCKS BUT SPANNED THREE
BLOCKS 4HUS FOR SOME ROWS TWO FETCHES OF THE hNEXTv ROW PIECES HAD TO BE DONE IN THE DATABASE !LSO
NOTE HOW THE consistent gets VALUE WHICH COUNTS ALL #2 GETS DONE BOTH IN THE DATABASE and IN CELL BY
3MART 3CAN IS AROUND !T THE SAME TIME THE SUBCOUNTERS consistent gets from cache #2 GETS DONE
BY THE DATABASE LAYER FROM BUFFER CACHE AND consistent gets direct CONSISTENT GETS DONE IN CELL IN THIS
CASE INDICATE HOW MUCH OF THE #2 WORK WAS DONE AT THE CELL LEVEL AND HOW MUCH WAS ADDITIONALLY DONE
AT THE DATABASE LEVEL
!S A CONCLUSION ABOUT CHAINED ROWS THE NEXT TIME YOU SEE THAT YOUR 3MART 3CAN IS WAITING FOR LOTS OF
cell single block physical reads AND IS DOING LOGICAL )/ INSIDE THE DATABASE LAYER consistent gets from
cache ONE OF THE THINGS TO CHECK IS THE ABOVEMENTIONED STATISTICS TO SEE WHETHER YOU ARE HITTING CHAINED
ROWS AND HAVE TO PROCESS THEM IN THE DATABASE /F COURSE DONT FORGET THAT THE 3MART 3CANS OFFLOAD ONLY
THE LARGE SEGMENT SCAN WORKLOAD BUT IF YOUR QUERY PLAN CONTAINS OTHER ROW SOURCES INDEX RANGE SCANS OR
SMALL CACHED TABLE SCANS THEN SEEING LOGICAL )/S AND SINGLE
BLOCK PHYSICAL READS IS EXPECTED ,UCKILY
YOU CAN USE WAIT INTERFACE DATA IN !3( EB4BB8>= OR 31, 4RACE TO SEE AGAINST WHICH OBJECTS THESE SINGLE
BLOCK READS ACCESSED 4HE RdaaT]cN^QY COLUMN IN !3( AND ^QY FIELD IN RAW 31, TRACE FILE REFER TO THE
^QYTRcNXS OF THE TABLE OR INDEX OR PARTITION THE SESSION IS READING FROM
physical read requests optimized
4HIS STATISTIC SHOWS HOW MANY )/ REQUESTS TO DISK WERE AVOIDED EITHER BY READING THE DATA FROM FLASH
CACHE INSTEAD OF DISKS OR THANKS TO THE STORAGE INDEX )/ ELIMINATION 4HIS STATISTIC IS ALSO PROPAGATED TO
EB@; EB@;BC0CB AND EB46<4=CNBC0C8BC82B VIEWS
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
physical read total bytes optimized
4HIS STATISTIC SHOWS HOW MANY BYTES WORTH OF PHYSICAL DISK DRIVE )/ WAS AVOIDED EITHER BY READING IT
FROM FLASH CACHE OR THANKS TO STORAGE INDEX )/ ELIMINATION 7HEN YOU ALSO SEE THE STATISTIC cell physical
I/O bytes saved by storage index EQUALLY INCREASE THIS MEANS THAT )/ WAS COMPLETELY AVOIDED THANKS TO
STORAGE INDEXES )F THE STORAGE INDEX SAVINGS ARE SMALLER THAN THE TOTAL OPTIMIZED BYTES THEN THE REST OF
THE )/ WAS OPTIMIZED THANKS TO READING IT FROM FLASH CACHE INSTEAD OF THE GOOD OLD SPINNING DISKS BUT
)/ TO FLASH CARDS DID STILL HAPPEN INTERNALLY )NTERESTINGLY THIS STATISTIC IS NOT PROPAGATED TO
EB@;EB@;BC0CB AND EB46<4=CNBC0C8BC82B VIEWS AT LEAST AS OF /RACLE
+EVIN 3AYS h)VE NEVER REALLY AGREED WITH THIS PARTICULAR COUNTER 7HILE IT IS TRUE THAT A READ FROM %XADATA
3MART &LASH CACHE IS OPTIMIZED WHEN COMPARED TO A READ FROM A HARD DISK IT IS IN NO WAY AS OPTIMIZED AS AN )/
THAT DIDNT HAPPEN )N MY MIND THE SAVINGS TRACKED BY THIS STATISTIC ARE JUST TOO DISSIMILAR TO BE ALL THAT HELPFULv
table fetch continued row
4HIS STATISTIC IS NOT %XADATA SPECIFIC BUT IS RELEVANT WHEN TROUBLESHOOTING UNEXPECTED SINGLE BLOCK READS
DONE BY THE DATABASE WHILE A 3MART 3CAN IS USED 4HIS STATISTIC COUNTS HOW MANY TIMES /RACLE HAD TO
FETCH A NEXT ROW PIECE OF A CHAINED ROW USING A REGULAR SINGLE BLOCK READ
table scans (direct read)
4HIS STATISTIC IS NOT %XADATA SPECIFIC IT IS SEEN IN ANY /RACLE DATABASE THAT IS DOING FULL TABLE SCANS ON
TABLE SEGMENTS USING DIRECT PATH READS $URING SERIAL EXECUTION THIS STATISTIC IS INCREMENTED IN THE
BEGINNING OF THE TABLE SCAN BUT WITH PARALLEL EXECUTION IT IS INCREMENTED EACH TIME A SLAVE STARTS
SCANNING A NEW ROWID RANGE DISTRIBUTED TO IT $IRECT PATH READS ARE A PREREQUISITE FOR 3MART 3CANS TO
HAPPEN SO IF YOU DONT GET YOUR QUERY TO USE 3MART 3CANS CHECK WHETHER A DIRECT PATH READ IS USED FOR
SCANNING THE TABLE
table scans (long tables)
4HIS IS A SIMILAR STATISTIC TO THE PREVIOUS ONE BUT IT SHOWS WHETHER THE TABLE SCANNED WAS CONSIDERED TO
BE LARGE OR NOT !CTUALLY /RACLE CONSIDERS THIS SEPARATELY FOR EACH SEGMENT 3O SOME PARTITIONS OF A TABLE
MAY BE CONSIDERED SMALL SOME LARGE )F THE SEGMENT UP TO THE HIGH WATER MARK IS BIGGER THAN
PERCENT OF THE BUFFER CACHE THE TABLE IS CONSIDERED LARGE AND DIRECT PATH READS ARE CONSIDERED EVEN FOR
SERIAL SCANS "UT NOTE THAT THIS DECISION LOGIC TAKES OTHER THINGS INTO ACCOUNT LIKE HOW MANY BLOCKS OF
THAT SEGMENT ALREADY HAPPEN DO BE IN BUFFER CACHE 4HE PERCENT OF THE BUFFER CACHE RULE ACTUALLY
COMES FROM THE Nb\P[[NcPQ[TNcWaTbW^[S PARAMETER 4HIS PARAMETER DEFAULTS TO PERCENT OF BUFFER CACHE
SIZE IN BLOCKS BUT /RACLE USES ¾ Nb\P[[NcPQ[TNcWaTbW^[S AS ITS DIRECT PATH SCAN DECISION THRESHOLD
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
Understanding SQL Statement Performance
4HIS SECTION FOCUSES ON THE 31, STATEMENTS PERFORMANCE METRICS AND UNDERSTANDING WHERE A STATEMENT
IS SPENDING ITS TIME AND WHERE ITS BOTTLENECKS ARE 7E WILL LOOK AT THE METRICS AVAILABLE AND WHEN TO USE
THEM BUT WE WILL COVER THE VARIOUS 31, PERFORMANCE MONITORING TOOLS IN THE NEXT CHAPTER
%XADATA SPECIFIC PERFORMANCE STATISTICS OF INDIVIDUAL 31, STATEMENTS MAY BE MONITORED USING THE
FOLLOWING VIEWS
v EB@;AND EB@;0A40
v EB@;BC0CBAND EB@;BC0CBN?;0=N70B7
v EB@;N<>=8C>A AND EB@;N?;0=N<>=8C>A
v E02C8E4NB4BB8>=N78BC>AH AND THE 310N78BCN02C8E4NB4BBN78BC>AH PERSISTED INTO
!72 REPOSITORY
.OTE THAT ALL THE %XADATA SPECIFIC METRICS YOU SEE IN EB@; VIEWS ARE REALLY THE SAME ONES YOU CAN SEE
FROM EB4BBC0C VIEWS 4HEY COME FROM THE SAME SOURCES BUT ARE JUST ACCUMULATED DIFFERENTLY EB4BBC0C
ACCUMULATES STATS FOR A SESSION REGARDLESS OF WHICH 31, STATEMENT OR COMMAND INCREMENTED THEM
WHILE THE VIEWS WITH A EB@; PREFIX AGGREGATE STATS FOR DIFFERENT 31, STATEMENTS REGARDLESS OF THE SESSIONS
EXECUTING THEM 3O IT IS POSSIBLE TO SEE SOME %XADATA METRICS AGGREGATED BY A 31, STATEMENT
(ERE IS AN EXAMPLE OF THE EB@; VIEWS COLUMNS THERE ARE MANY MORE COLUMNS IN EB@; VIEWS BUT
WELL SHOW THOSE MOST RELEVANT TO OUR TOPIC AT HAND
B@;-34B2eb`[
=P\T=d[[.Ch_T
B@;NC4GCE0A270A!
B@;N83E0A270A! "
4G42DC8>=B=D<14A
?;0=N70B7NE0;D4=D<14A
278;3N=D<14A=D<14A
2?DNC8<4=D<14A
4;0?B43NC8<4=D<14A
8>N24;;N>55;>03N4;8681;4N1HC4B=D<14A
8>N8=C4A2>==42CN1HC4B=D<14A
?7HB820;NA403NA4@D4BCB=D<14A
?7HB820;NA403N1HC4B=D<14A
>?C8<8I43N?7HNA403NA4@D4BCB=D<14A
8>N24;;ND=2><?A4BB43N1HC4B=D<14A
8>N24;;N>55;>03NA4CDA=43N1HC4B=D<14A
4HE BOLD COLUMNS ARE SPECIFIC TO %XADATA ALTHOUGH NON %XADATA DATABASES ALSO HAVE THESE COLUMNS
THANKS TO THE SINGLE DATABASE SOFTWARE USED BOTH FOR %XADATA AND NON %XADATA DATABASES
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
Table 11.1. V$SQL Columns and Their Meanings
Column Name Metric Meaning
8>N24;;N>55;>03N4;8681;4N1HC4B (OW MANY BYTES WORTH OF SEGMENT READS WERE OFFLOADED TO
THE CELLS 4HE CELLS EITHER DID READ THIS DATA OR SKIPPED IT IF
STORAGE INDEXES HELPED TO SKIP BLOCK RANGES 4HIS METRIC
CORRESPONDS TO THE cell physical IO bytes eligible for
predicate offload STATISTIC IN EB4BBC0C
4HE TOTAL TRAFFIC BYTES READ AND WRITE SENT BETWEEN THE
8>N8=C4A2>==42CN1HC4B DATABASE NODE AND CELLS .OTE THAT THANKS TO THE BUG
MENTIONED EARLIER THE 8>N8=C4A2>==42CN1HC4B
COLUMN IS POPULATED ALSO ON NON %XADATA PLATFORMS BUT
YOU CAN IGNORE IT ON THESE SYSTEMS
>?C8<8I43N?7HNA403NA4@D4BCB 4HE NUMBER OF DISK )/ REQUESTS THAT WERE EITHER
COMPLETELY AVOIDED THANKS TO STORAGE INDEXES OR DONE
AGAINST CELL FLASH CACHE CARDS
8>N24;;ND=2><?A4BB43N1HC4B 4HE SIZE OF UNCOMPRESSED DATA THE CELLS HAVE SCANNED
THROUGH DURING A 3MART 3CAN .OTE THAT THE CELLS DO NOT
HAVE TO ACTUALLY UNCOMPRESS ALL THE DATA TO KNOW THE
UNCOMPRESSED LENGTH 4HE %(## COMPRESSION UNIT
HEADERS STORE BOTH THE COMPRESSED AND UNCOMPRESSED #5
LENGTH INFO IN THEM 4HIS METRIC IS USEFUL FOR ESTIMATING THE
)/ REDUCTION FROM %(## COMPRESSION .OTE THAT THIS
METRIC WORKS FOR %(## SEGMENTS ONLY &OR REGULAR BLOCK
LEVEL COMPRESSION THIS METRIC JUST SHOWS THE COMPRESSED
SIZE OF DATA
8>N24;;N>55;>03NA4CDA=43N1HC4B 4HIS METRIC SHOWS HOW MUCH DATA WAS RETURNED AS A RESULT
FROM AN OFFLOADED 3MART 3CAN ACCESS PATH 4HIS IS A MAIN
INDICATOR OF SMART SCAN OFFLOADING EFFICIENCY WHEN
COMPARED WITH 8>N24;;N>55;>03N4;8681;4N1HC4BTO
MEASURE THE )/ REDUCTION BETWEEN CELLS AND DATABASE OR
8>N24;;ND=2><?A4BB43N1HC4B WHEN MEASURING THE TOTAL )/
REDUCTION THANKS TO OFFLOADING AND COMPRESSION
(ERE IS EXAMPLE OUTPUT FROM A QUERY ON AN %(##
COMPRESSED TABLE WHERE THE TABLE SCAN WAS OFFLOADED
TO THE CELL 4HE EB@; TABLE OUTPUT IS PIVOTED FOR BETTER READABILITY
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
B@;-B4;42CBD<^QYTRcNXS5A><c#NWRRN`dTahNWXVWF74A4^f]Ta;8:4B*
BD<>1942CN83
" (&# 4 !
B@;-B4;42C5A><eb`[F74A4b`[NXS, ae "&\Vf_]Zg*
(output pivoted and edited for better readability)
8>N24;;N>55;>03N4;8681;4N1HC4B)% ''(!#(!'
8>N8=C4A2>==42CN1HC4B)&! !""" !
?7HB820;NA403NA4@D4BCB)%$#&
?7HB820;NA403N1HC4B)% ''('!!&!
?7HB820;NFA8C4NA4@D4BCB)
?7HB820;NFA8C4N1HC4B)
>?C8<8I43N?7HNA403NA4@D4BCB)&
;>2:43NC>C0;)
?8==43NC>C0;)!
8>N24;;ND=2><?A4BB43N1HC4B) !&# $'$ &
8>N24;;N>55;>03NA4CDA=43N1HC4B)&! &$(%'
)N THIS CASE THE 8>N24;;N>55;>03NA4CDA=43 BYTES IS MUCH SMALLER THAN THE 8>N24;;N>55;>03N4;8681;4
BYTES THUS THE 3MART 3CAN DEFINITELY DID HELP TO REDUCE THE DATA FLOW BETWEEN THE CELLS AND THE
DATABASE &URTHERMORE THE 8>N24;;ND=2><?A4BB43N1HC4B IS WAY BIGGER THAN THE ?7HB820;NA403N1HC4B
WHICH INDICATES THAT THE %(## HELPED TO REDUCE THE NUMBER OF BYTES THAT HAD TO BE READ FROM DISK BY
THE CELLS THANKS TO COMPRESSION .OTE THAT THE 8>N8=C4A2>==42CN1HC4B IS NOT MUCH GREATER THAN THE
8>N24;;N>55;>03NA4CDA=43N1HC4B WHICH INDICATES THAT FOR THIS 31, ALMOST ALL THE TRAFFIC WAS DUE TO THE
DATA RETURNED BY 3MART 3CANS 4HERE WAS NO EXTRA TRAFFIC DUE TO OTHER REASONS LIKE 4EMP TABLE SPACE
READSWRITES THANKS TO NON
OPTIMAL SORTS AND THERE WERE NO HASH JOINS OR OTHER WORK
AREA OPERATIONS OR
DATABASE BLOCK )/S THANKS TO CHAINED ROWS OR IN
DATABASE CONSISTENT READ PROCESSING
.OTE 3MART 3CANNING MAKES DATA retrieval FROM SEGMENTS FASTER BUT IT DOESNT DO ANYTHING TO SPEED UP
JOINING SORTING AND AGGREGATE OPERATIONS WHICH ARE APPLIED AFTER THE DATA HAS BEEN RETRIEVED FROM THE SEGMENTS
! NOTABLE EXCEPTION IS THE "LOOM FILTER PUSHDOWN TO CELLS WHICH ALLOWS THE CELLS TO FILTER THE DATA FROM THE PROBE
TABLE USING A HASH BITMAP BUILT BASED ON THE DRIVING ROW SOURCES DATA IN THE HASH JOIN
7HILE THIS EXAMPLE USED THE EB@; VIEW WHICH SHOWS 31, CHILD CURSOR LEVEL STATISTICS YOU COULD ALSO
USE EB@;N?;0=N<>=8C>A THE COLUMNS ?;0=N;8=4N83 ?;0=N>?4A0C8>= AND SO ON TO MEASURE THESE
METRICS FOR EACH EXECUTION PLAN LINE 4HIS IS USEFUL BECAUSE A SINGLE EXECUTION PLAN USUALLY ACCESSES AND
JOINS MULTIPLE TABLES AND DIFFERENT TABLES MAY BENEFIT FROM THE 3MART 3CAN OFFLOADING DIFFERENTLY 7E WILL
LOOK AT SOME SCRIPTS AND TOOLS THAT USE THIS DATA IN THE NEXT CHAPTER
#(!04%2 5.$%234!.$).' %8!$!4! 0%2&/2-!.#% -%42)#3
Summary
4HE EMPHASIS OF THIS CHAPTER WAS ON UNDERSTANDING %XADATA PERFORMANCE AND THE VARIOUS RELATED
METRICS /RACLE OFFERS FOR US )T IS IMPORTANT TO REMEMBER THAT THE %XADATA 3MART 3CAN POTENTIALLY MAKES
YOUR DATA RETRIEVAL SCANNING FASTER BUT 3MART 3CANS HAPPEN ONLY WHEN DIRECT PATH READS AND FULL
SEGMENT SCANS ARE USED !LSO REMEMBER THAT IT IS NOT POSSIBLE TO DETERMINE WHETHER A 3MART 3CAN
ACTUALLY KICKED IN JUST BY LOOKING AT THE EXECUTION PLAN ALONE 9OU SHOULD ALWAYS CHECK ADDITIONAL
METRICS LIKE WHETHER YOU SEE THE cell smart table/index scan WAIT EVENTS IN YOUR SESSION AND WHETHER THE
IO_CELL_OFFLOAD_ELIGIBLE_BYTES IN EB@; OR cell physical I/O bytes eligible for predicate offload
STATISTIC IN EB4BBC0C INCREASES WHILE YOU RUN YOUR 31, -ANY OF THE OTHER METRICS EXPLAINED HERE WILL
HOPEFULLY BE HELPFUL FOR UNDERSTANDING AND TROUBLESHOOTING ADVANCED PERFORMANCE ISSUES SUCH AS WHEN
A 3MART 3CAN KICKS IN BUT IS THROTTLED BY A MULTITUDE OF SPECIAL CONDITIONS LIKE CHAINED ROWS CONSISTENT
READ ROLLBACKS OR JUST RUNNING OUT OF CELL SERVER RESOURCES )N THE NEXT CHAPTER WE WILL SEE HOW TO USE THIS
KNOWLEDGE IN MONITORING AND TROUBLESHOOTING %XADATA PERFORMANCE AND WE WILL LOOK DEEPER INTO THE
CELL
LEVEL PERFORMANCE METRICS FROM RT[[bae AND THE OPERATING SYSTEM TOO
C H A P T E R 12
Monitoring Exadata Performance
"Y NOW YOU HAVE LEARNED ABOUT THE KEY %XADATA PERFORMANCE FEATURES AND THE RELATED PERFORMANCE
METRICS ,ETS SEE HOW WE CAN USE THESE FOR EVERYDAY TASKS 7E WILL LOOK AT STANDARD TOOLS AVAILABLE FOR
DATABASE
LAYER AND CELL PERFORMANCE MONITORING AND HOW TO INTERPRET THEIR OUTPUT
/RACLE $ATABASE AND THE %XADATA CELLS PROVIDE A HUGE VARIETY OF DIFFERENT METRICS BUT BEFORE
MONITORING ANY METRICS YOU SHOULD ASK YOURSELF WHY YOU ARE MONITORING THEM !DDITIONALLY YOU SHOULD
KNOW WHAT YOUR ACTION WOULD BE IF A METRIC CROSSES SOME THRESHOLD 4HIS LEADS TO THE FOLLOW
UP
QUESTION WHICH EXACT THRESHOLD SHOULD PROMPT SOME ACTION FROM YOUAND WHY )N OTHER WORDS YOU
SHOULD KNOW WHAT ARE YOU TRYING TO ACHIEVE GOOD RESPONSE TIMES FOR USERS AND HOW PERFORMANCE
METRICS RELATE TO THAT
4HE MONITORING TASKS COVERED HERE CAN BE DIVIDED INTO THE FOLLOWING CATEGORIES
v 31, STATEMENT RESPONSE TIME MONITORING
v $ATABASE LAYER UTILIZATION AND EFFICIENCY MONITORING
v 3TORAGE CELL LAYER UTILIZATION AND EFFICIENCY MONITORING
v !DVANCED METRICS AND MONITORING FOR %XADATA PERFORMANCE TROUBLESHOOTING
.OTE THAT WE WILL FOCUS ON THE %XADATA
SPECIFIC PERFORMANCE TOPICS HERE AND NOT THE WHOLE WIDE
RANGE OF OTHER /RACLE PERFORMANCE TOPICS LIKE LOCK CONTENTION OR GENERAL 31, PERFORMANCE ISSUES
A Systematic Approach
7HATEVER METRICS YOU MONITOR YOU SHOULD HAVE A PURPOSE IN MIND )N OTHER WORDS DONT COLLECT AND
DISPLAY METRICS JUST BECAUSE THEY ARE AVAILABLE THIS WILL LEAD YOU NOWHERE OR POTENTIALLY EVEN MISLEAD
YOU TO FIXING THE WRONG PROBLEM .OTE THAT THE TERM hPERFORMANCEv IS VAGUEDIFFERENT PEOPLE MAY
MEAN DIFFERENT THINGS WHEN THEY USE IT &ROM AN )4 SYSTEM USERS PERSPECTIVE PERFORMANCE IS ULTIMATELY
ONLY ABOUT ONE THING RESPONSE TIME !ND NOT SOME RESPONSE TIMES OF INDIVIDUAL WAIT EVENTS MEASURED
AT A LOW LEVEL THE END USERS DONT CARE ABOUT THAT 4HEY DO CARE ABOUT HOW MUCH they HAVE TO WAIT FOR
THEIR business task TO COMPLETE LIKE THE TIME IT TAKES FROM THE REPORT SUBMISSION TO ACTUALLY SEEING ITS
OUTPUT 4HIS TIME IS MEASURED IN REGULAR WALL
CLOCK TIME ITS AS SIMPLE AS THAT
)F YOUR PURPOSE IN MONITORING IS TO ENSURE GOOD RESPONSE TIMES FOR YOUR APPLICATION USERS THEN YOU
SHOULD MEASURE WHAT MATTERSRESPONSE TIME AS YOUR END
USER EXPERIENCES IT 4HIS WOULD BE THE IDEAL
ENTRY POINT TO PERFORMANCE MONITORING )N ADDITION TO THIS ENTRY POINT YOU SHOULD MEASURE MORE
DETAILED LOWER
LEVEL METRICS TO BREAK THE END USER RESPONSE TIME DOWN INTO INDIVIDUAL COMPONENTS LIKE
TIME SPENT IN AN APPLICATION SERVER AND DATABASE TIME 9OUR APPLICATION INSTRUMENTATION AND
MONITORING TOOLS SHOULD KEEP TRACK OF WHICH DATABASE SESSIONS WERE USED FOR WHICH END
USER TASK SO
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
YOU CAN REPORT WHAT THESE EXACT DATABASE SESSIONS WERE DOING WHEN A USER EXPERIENCED UNACCEPTABLE
RESPONSE TIMES
.OTE 7E DELIBERATELY SAID unacceptable response times HERE INSTEAD OF JUST hUSER EXPERIENCED PERFORMANCE
PROBLEMSv 7HENEVER ANYONE COMPLAINS ABOUT A PERFORMANCE PROBLEM YOU SHOULD CLARIFY WHAT THEY ACTUALLY MEAN
BY THAT AND HOW IT WAS MEASURED $OES ANY USER ACTUALLY EXPERIENCE FAR TOO LONG RESPONSE TIMES IN THEIR
APPLICATION OR DID SOME MONITORING SYSTEM MERELY RAISE AN ALARM ABOUT hTOO HIGHv #05 UTILIZATION OR ANY OTHER
SECONDARY METRIC LIKE THAT 9OUR SUBSEQUENT ACTIONS WOULD DEPEND ON THE PROBLEM YOURE TRYING TO SOLVE )DEALLY
YOU SHOULD NOT USE A PERFORMANCE TOOL OR /RACLE METRIC FOR DETERMINING whether YOU HAVE A PERFORMANCE PROBLEM
9OUR STARTING POINT SHOULD BE THE USERS WHO REPORT A PROBLEM OR APPLICATION LEVEL METRICS WHICH SEE THE DATABASE
RESPONSE TIME FROM THE APPLICATION PERSPECTIVE .O MATTER HOW GOOD THE DATABASE METRICS LOOK IF THE APPLICATION
WAITS FOR THE REPORT COMPLETION FOR AGES YOU HAVE A PERFORMANCE PROBLEM TO DRILL DOWN INTO #ONVERSELY NO MATTER
HOW hBADv YOUR DATABASE METRICS SEEM TO BE IF YOUR APPLICATION RESPONSE TIMES ARE SATISFACTORY YOU DONT HAVE
AN ACUTE NEED TO START FIXING ANYTHING
7HEN EXAMINING PERFORMANCE METRICS BECAUSE OF AN ONGOING PROBLEM OF TOO LONG RESPONSE TIMES YOU
SHOULD START BY IDENTIFYING THE SESSIONS SERVICING THIS SLOW APPLICATION OR JOB AND THEN DRILLING DOWN INTO
THAT RESPONSE TIME )T WOULD BE MORE CORRECT TO CALL THIS PERFORMANCE TROUBLESHOOTING NOT JUST
MONITORING
.OTE THAT THERE ARE OTHER KINDS OF PERFORMANCE MONITORING TASKS WHICH YOU MAY WANT TO DOFOR
EXAMPLE PROACTIVE UTILIZATION AND EFFICIENCY MONITORING 0ERFORMING THESE TASKS ALLOWS YOU TO KEEP AN
EYE ON THE UTILIZATION HEADROOM LEFT IN THE SERVERS AND DETECT ANY ANOMALIES AND SUDDEN SPIKES IN
SYSTEM UTILIZATION AND LOW
LEVEL RESPONSE TIMES POSSIBLY EVEN BEFORE USERS NOTICE A RESPONSE TIME
DIFFERENCE 9ET ANOTHER REASON FOR COLLECTING AND MONITORING PERFORMANCE AND UTILIZATION DATA IS FOR
CAPACITY PLANNINGBUT CAPACITY PLANNING IS OUTSIDE THE SCOPE OF THIS BOOK !LSO BECAUSE THIS IS A
DATABASE BOOK WE WONT DIVE INTO ANY END
TO
END PERFORMANCE MEASUREMENT TOPICS WHICH WOULD
INVOLVE IDENTIFYING TIME SPENT IN APPLICATION SERVERS ON THE NETWORK AND SO ON
7E WILL START BY LOOKING AT HOW TO IDENTIFY WHERE A LONG
RUNNING QUERY IS SPENDING MOST OF ITS TIME
7ELL ALSO LOOK AT HOW TO TELL WHETHER A QUERY IS TAKING FULL ADVANTAGE OF %XADATAS PERFORMANCE FEATURES
Monitoring SQL Statement Response Time
4HE BEST TOOL FOR MONITORING LONG RUNNING QUERIES IS /RACLE G@S 31, 2EAL 4IME MONITORING PAGE )T IS
ABLE TO GATHER ALL THE KEY PERFORMANCE INFORMATION ONTO A SINGLE PAGE EVEN IN THE CASE OF PARALLEL
EXECUTION ACROSS MULTIPLE 2!# INSTANCES
4HE 31, -ONITORING FEATURE REQUIRES YOU TO HAVE A $IAGNOSTICS AND 4UNING 0ACK LICENSE 31,
-ONITORING KICKS IN AUTOMATICALLY IF YOU RUN YOUR QUERY WITH PARALLEL EXECUTION OR WHEN A SERIAL QUERY
CONSUMES MORE THAN SECONDS OF )/ AND #05 TIME IN TOTAL !DDITIONALLY YOU CAN CONTROL THE
MONITORING FEATURE WITH <>=8C>A AND =>N<>=8C>A HINTS )F YOU WANT TO MONITOR YOUR FREQUENTLY EXECUTED
SHORT RUNNING QUERIES THE BEST TOOL FOR THIS WOULD BE TO USE !3( DATA AND LIST THE TOP WAIT EVENTS AND TOP
ROWSOURCES FROM THERE USING THE B@;N?;0=N;8=4 COLUMNS IN THE E02C8E4NB4BB8>=N78BC>AH VIEW
)F YOU ARE ALREADY AWARE OF A PERFORMANCE PROBLEM PERHAPS YOUR USERS ARE ALREADY COMPLAINING OF
TOO LONG RESPONSE TIMES THEN YOU SHOULD USE A TOP DOWN APPROACH FOR MONITORING 9OU SHOULD IDENTIFY
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
THE SESSIONS OF THE PROBLEMATIC USERS APPLICATIONS OR REPORTS AND DRILL DOWN INTO WHAT THESE SESSIONS
ARE DOING WITH !3( WHICH GIVES YOU THE 31,?)$S OF THE TOP 31, STATEMENTS FOR THESE SESSIONS AND
WHEN NEEDED THEN DRILL DOWN FURTHER INTO THE TOP STATEMENTS WITH 31, -ONITORING REPORTS
-ONITORING 31, 3TATEMENTS WITH 2EAL
4IME 31,
-ONITORING 2EPORTS
7HEN YOU CLICK THE 31, -ONITORING LINK IN THE %NTERPRISE -ANAGER PERFORMANCE PAGE YOU WILL SEE THE
LATEST MONITORED QUERIES 4HE 31, -ONITORING REPORTS ARE PRESENT IN 'RID #ONTROL G 2 OR G 2
)F YOU ARE NOT USING 'RID #ONTROL THEN YOU CAN EITHER USE THE BUILT
IN %NTERPRISE -ANAGER
$ATABASE #ONSOLE OR RUN THE 31, -ONITORING REPORTS MANUALLY FROM 31,
0LUS AS EXPLAINED SHORTLY .OTE
THAT THE 31, -ONITORING FEATURE REQUIRES $IAGNOSTICS AND 4UNING 0ACK LICENSES
&IGURE
SHOWS THE ENTRY PAGE TO THE 31, -ONITORING REPORTS 9OU CAN GET THERE BY CLICKING THE
31, -ONITORING LINK AT THE BOTTOM OF THE 0ERFORMANCE PAGE IN 'RID #ONTROL 4HE 31, -ONITORING PAGE
AS SEEN IN &IGURE
LISTS THE CURRENTLY RUNNING QUEUED AND RECENTLY COMPLETED MONITORED 31,
EXECUTIONS WITH SOME KEY PERFORMANCE METRICS AND DETAILS SUCH AS THE DEGREE OF PARALLELISM
Figure 12-1. Enterprise Manager’s overview of Monitored SQL Executions
4HE 3TATUS COLUMN SHOWS AN ICON WITH ONE OF FOUR STATUSESRUNNING DONE ERROR OR QUEUED 7HEN YOU
CLICK THE STATUS ICON YOU WILL BE TAKEN TO THE 31, STATEMENTS DETAILED MONITORING PAGE /NE OF THE MOST
IMPORTANT PIECES OF INFORMATION IS THE Duration COLUMN SHOWING HOW LONG A STATEMENT HAS BEEN ACTIVE
4HE DURATION IS THE WALL CLOCK TIME FROM THE START OF THE STATEMENT EXECUTION THROUGH THE FINISH OR TO THE
CURRENT TIME IN THE EVENT THE STATEMENT IS STILL EXECUTING &IGURE ILLUSTRATES THE DIFFERENCE BETWEEN
DURATION WALL CLOCK AND #05 TIME IN STATEMENT EXECUTION
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-2. Statement Duration compared to Database Time in the SQL Monitoring page
4HE $URATION COLUMN SHOWS WHAT USERS CARE ABOUT WHICH IS THE RESPONSE TIME OF A QUERY SINCE THE 31,
EXECUTION START 4HIS IS THE TIME they HAVE HAD TO WAIT FOR THE 31, TO COMPLETE /F COURSE THE END USERS
MAY HAVE HAD TO WAIT FOR MUCH LONGER THAN THE DURATION OF THE QUERY AS THE PAGE IN &IGURE
SHOWS
ONLY THE DATABASE RESPONSE TIME 4IME MAY HAVE ALSO BEEN SPENT IN THE APPLICATION LAYER OR THE NETWORK
CONNECTION IN BETWEEN THE USER AND THE APPLICATION SERVER
)T IS IMPORTANT TO KNOW THAT THE DURATION MEASURES TIME FROM 31, EXECUTION START ALL THE WAY UNTIL
THE CURSOR IS CLOSED OR CANCELLED FOR EXAMPLE WHEN ALL DATA IS FETCHED FROM IT 4HIS MEANS THAT IF YOUR
DATABASE CAN PROCESS THE QUERY IN SECONDS BUT THEN MILLIONS OF ROWS ARE FETCHED A FEW AT A TIME YOUR
QUERY WILL TAKE A LONG TIME AS FAR AS THE APPLICATION IS CONCERNED THANKS TO THE NETWORK PING
PONG BUT
IN FACT ONLY A LITTLE TIME IS SPENT PROCESSING WITHIN THE DATABASE 4HE $URATION COLUMN STILL SHOWS LONG
QUERY hRUNTIME v AS THE CURSOR IS STILL KEPT OPEN FOR FETCHING OF THE DATA 2EMEMBER THE DURATION
MEASURES TIME FROM WHEN THE CURSOR IS EXECUTED ALL THE WAY TO WHEN IT IS FINALLY CLOSED AFTER ALL THE
FETCHES ARE DONE OR THE APPLICATION HAS FETCHED ENOUGH
4HIS LEADS US TO THE NEXT IMPORTANT METRIC$ATABASE 4IME SEEN IN THE SIXTH COLUMN IN &IGURE
4HE $ATABASE 4IME METRIC SHOWS THE TOTAL TIME YOUR QUERY SPENT EXECUTING in the database 3O IF YOU
RUN A SERIAL $-, THAT RUNS FOR SECONDS AND SPENDS ALL OF THE TIME EXECUTING inside THE DATABASE YOU
WILL END UP SEEING SECONDS OF DATABASE TIME TOO (OWEVER IF YOU ARE RUNNING SOME B4;42C STATEMENT
AND ARE FETCHING A LOT OF ROWS CAUSING YOUR SESSION TO SPEND LETS SAY OF ITS TIME EXECUTING IN THE
DATABASE AND ANOTHER WAITING FOR THE NEXT FETCH COMMAND ONCE IT HAS SENT AN ARRAY OF ROWS BACK TO
THE APPLICATION YOU WOULD SEE ONLY HALF OF THAT TOTAL
SECOND DURATION AS DATABASE TIME )N OTHER
WORDS YOU WOULD SEE SECONDS OF DATABASE TIME AS THE DATABASE SESSION HAS BEEN SERVICING YOUR
REQUEST ONLY FOR SECONDS AND THE REST OF THE TIME IT WAS IDLE
,OOKING AT THE SECOND ENTRY IN &IGURE
YOU SEE THAT THE duration OF THE QUERY THE TIME FROM
WHEN THE EXECUTION PHASE STARTED IS MINUTES THE QUERY HAS EXECUTED IN SERIAL THE FIFTH COLUMN
SHOWS THAT AND IT HAS CONSUMED ONLY MINUTES OF database time 3O THIS INDICATES THAT THE EXECUTING
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
SESSION HAS BEEN DOING SOMETHING ELSE FOR MINUTES )T WAS EITHER IDLE PROBABLY WAITING FOR THE NEXT
FETCH REQUEST OR EXECUTING SOME OTHER STATEMENT WHILE THE FIRST STATEMENTS CURSOR WAS STILL OPEN
4HE EXAMPLE WE JUST DISCUSSED WAS ABOUT A SINGLE SERIAL QUERY 7HEN YOU RUN A PARALLEL QUERY YOU
HAVE MULTIPLE SESSIONS EXECUTING PIECES OF THE SAME QUERY FOR YOU 4HEN THE DATABASE TIME MIGHT END UP
BEING MUCH HIGHER THAN THE DURATION RESPONSE TIME OF YOUR QUERY )F YOU LOOK AT THE FIRST ENTRY IN &IGURE
YOU SEE A STATEMENT WITH DURATION OF SECONDS BUT THE TOTAL TIME SPENT IN THE DATABASE IS
MINUTES 7HEN YOU LOOK AT THE parallel COLUMN YOU SEE THAT THE SESSION WAS EXECUTED WITH PARALLEL
DEGREE WHICH MEANS THAT YOU HAD MULTIPLE SESSIONS ACTIVELY WORKING IN THE DATABASE FOR YOUR QUERY
!LL OF THESE PARALLEL SESSIONS DATABASE TIME PLUS THE QUERY COORDINATORS TIME IS ADDED INTO THE $ATABASE
4IME COLUMN 4HIS DATABASE TIME GIVES AN IDEA OF HOW MUCH WORK WAS DONE IN THE DATABASE "UT
BECAUSE THE STATEMENT WAS PARALLELIZED YOU DONT HAVE TO WAIT FOR THAT LONG )F YOU RAN THAT SAME QUERY
IN SERIAL MODE THEN THE DATABASE TIME WOULD BE IN THE BALLPARK OF HOW MUCH TIME YOU MIGHT HAVE TO
WAIT 0LEASE TAKE THAT LAST STATEMENT WITH A GRAIN OF SALT AS IN PRACTICE MANY OTHER THINGS MAY HAPPEN IN
31, EXECUTION WHEN SWITCHING FROM PARALLEL EXECUTION TO SERIAL SO YOU MIGHT HAVE SOME PLEASANT OR
UNPLEASANT SURPRISES IN THE RESPONSE TIME
.OTE THAT THESE ENTRIES IN THE 31, -ONITORING OVERVIEW PAGE ARE NOT 31, statement-SPECIFIC BUT
31, STATEMENT execution
SPECIFIC 3O IF TWO USER SESSIONS WERE RUNNING THE SAME STATEMENT YOU WOULD
SEE TWO SEPARATE ENTRIES IN THIS LIST 4HIS ALLOWS YOU TO EXAMINE EXACTLY WHAT THE PROBLEM IS WITH A
SPECIFIC USER WHO IS COMPLAINING AS OPPOSED TO LOOKING AT STATEMENT
LEVEL AGGREGATE METRICS LIKE THOSE
EB@; PROVIDES AND TRYING TO FIGURE OUT your USERS PROBLEM FROM THERE
Real-Time Monitoring of Single Long-Running Query
/NCE YOU HAVE IDENTIFIED YOUR QUERY OF INTEREST FROM THE LIST OF ALL LONG RUNNING QUERIES YOU CAN CLICK ON
ITS hRUNNINGv ICON IN THE LEFT SIDE OF THE LIST AND YOU WILL BE TAKEN TO THE -ONITORED 31, %XECUTION $ETAILS
PAGE SHOWN IN &IGURE 4HIS PAGE HAS A NUMBER OF SUB TABS SO FEEL FREE TO EXPLORE AND SEE ALL THE
INFORMATION THATS AVAILABLE THERE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-3. Monitored SQL statement execution details
4HE -ONITORED 31, %XECUTION $ETAILS PAGE HAS A LOT OF INFORMATION IN IT 4HIS PAGE IS PRETTY SELF
EXPLANATORY IF YOU HAVE READ 31, EXECUTION PLANS BEFORE SO WE WONT GO THROUGH EVERY SINGLE DETAIL HERE
BUT WILL FOCUS ON THE MOST IMPORTANT METRICS #OMPARED TO OLD
FASHIONED 31<BNG?;0= OUTPUT THE 31,
-ONITORING PAGE IS VERY INTERACTIVE SO MAKE SURE THAT YOU HOVER YOUR MOUSE OVER AND CLICK ON ALMOST
EVERYTHING ON THE SCREEN TO SEE THE FULL ARRAY OF FUNCTIONALITY AVAILABLE TO YOU
,ETS START FROM THE TOP SECTION ON THE PAGE !T THE TOP OF THE SCREEN RIGHT AFTER THE -ONITORED 31,
%XECUTION $ETAILS HEADER YOU SEE A LITTLE ICON WHICH SHOWS THE STATUS OF THE SELECTED 31, STATEMENTS
EXECUTION 4HE LITTLE CIRCLE WILL MEAN THAT THE STATEMENT IS STILL BEING EXECUTED THE CURSOR HAS NOT BEEN
CLOSED NOR CANCELLED
4HE /VERVIEW SECTION ILLUSTRATED IN &IGURE
WILL SHOW SOME BASIC DETAILS LIKE WHICH 31, )$ IS
BEING EXECUTED BY WHOM AND SO ON 9OU CAN SEE THE 31, TEXT WHEN YOU CLICK ON THE LITTLE (i) ICON NEXT TO
THE 31, )$ 9OU WILL SEE THE 31, STATEMENT TEXT AND BEGINNING WITH VERSION YOU WILL ALSO SEE THE
BIND VARIABLE VALUES USED FOR THE EXECUTION &IGURE
SHOWS THE DISPLAY THAT YOU GET BY CLICKING THAT (i)
ICON
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-4. SQL Monitoring overivew section
Figure 12-5. Bind variable values in the SQL Monitoring detail page
/F COURSE WHETHER YOU SHOULD USE BIND VARIABLES FOR YOUR LONG RUNNING REPORTS AND $7 QUERIES IS AN
ENTIRELY SEPARATE QUESTION 9OU SHOULDNT USE BINDS FOR LONG RUNNING QUERIES IN YOUR $7 YOUD WANT TO
SACRIFICE SOME RESPONSE TIME FOR HARD PARSING A NEW QUERY PLAN FOR EACH COMBINATION OF LITERAL VALUES
AND POSSIBLY GET AN OPTIMAL PLAN FOR EACH VARIABLE SET .EVERTHELESS MONITORING BIND VARIABLE VALUES OF
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
AN ALREADY RUNNING QUERY IS EASY WITH /RACLE AND LATER BECAUSE YOU NO LONGER MUST RESORT TO THE
>A0341D64AA>ABC02: COMMAND
4HE 4IME 7AIT 3TATISTICS SECTION IN &IGURE
SHOWS THE FAMILIAR METRICS OF $URATION AND
$ATABASE 4IME OF A STATEMENT -OVE YOUR MOUSE OVER THE DIFFERENT DATABASE TIME COMPONENTS TO SEE
HOW MUCH TIME WAS SPENT WAITING INSIDE THE DATABASE COMPARED TO RUNNING ON #05 4HE 7AIT !CTIVITY
BAR SHOWS THE BREAKDOWN OF WAIT EVENTS .OTE THAT THE IN THIS BAR MEANS OF THE wait time
COMPONENT OF DATABASE TIME NOT OF THE ENTIRE DATABASE TIME WHICH ALSO INCLUDES #05 TIME
Figure 12-6. Time and Wait Statistics in the SQL Monitoring detail page
4HE )/ 3TATISTICS SECTION IN &IGURE
A SHOWS SOME KEY )/ STATISTICS FOR STATEMENT EXECUTION
Figure 12-7a. I/O Statistics in the SQL Monitoring detail page
4HE "UFFER 'ETS BAR SHOWS THE TOTAL NUMBER OF LOGICAL )/S DONE IN BOTH THE DATABASE LAYER AND THE
CELLS IF 3MART 3CAN KICKED IN 4HE )/ 2EQUESTS AND )/ "YTES STATISTICS ARE SELF
EXPLANATORY BUT NOTE THAT
THOSE METRICS SHOW ALL )/ DONE FOR THE GIVEN STATEMENT WHICH INCLUDES THE 3MART 3CAN )/ ANY REGULAR
BLOCK )/ AND ALSO 4%-0 TABLESPACE )/ DONE FOR SORTING HASH JOINS AND FOR ANY OTHER WORK AREA
OPERATIONS THAT DIDNT FIT INTO ALLOWED 0'! MEMORY .OW YOU SHOULD BEGIN TO UNDERSTAND WHY THE #ELL
/FFLOAD %FFICIENCY IS NEGATIVE FOR THE STATEMENT EXECUTION IN THE FIGURE 4HIS METRIC SHOULD IDEALLY SHOW
THE PERCENTAGE OF DISK
TO
DATABASE
HOST INTERCONNECT TRAFFIC THAT WAS AVOIDED THANKS TO THE 3MART 3CANS
PERFORMING FILTERING AND PROJECTION EARLY IN THE CELLS AND RETURNING ONLY A SUBSET OF DATA (OWEVER IN A
COMPLEX EXECUTION PLAN THERE IS MUCH MORE GOING ON THAN JUST A 3MART 3CAN AGAINST A SINGLE TABLE 9OU
HAVE JOINS AGGREGATE OPERATIONS SORTING AND DIRECT PATH DATA LOADS WHICH ALL USE EXTRA )/ BANDWIDTH
DRIVING THE OFFLOAD EFFICIENCY PERCENTAGE DOWN 4HIS IS A GOOD EXAMPLE OF WHY FOCUSING ON ONLY A SINGLE
RATIO IS NOT A GOOD IDEA 4HE SINGLE PERCENTAGE VALUE FOR OFFLOAD EFFICIENCY HIDES A LOT OF INFORMATION SUCH
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
AS WHERE THE PERCENTAGE TAKEN WAS FROM AND THE real VALUES BEHIND IT 9OU CAN MOVE YOUR MOUSE OVER
THE PERCENTAGE AND SEE THE UNDERLYING VALUES USED FOR THE CALCULATION
3O WHEN MOVING YOUR MOUSE OVER THE RATIO SEE &IGURE
B YOULL SEE THAT THE CELLS DID READ
'" FROM DISKS BUT MUCH MORE DATA '" WAS ACTUALLY SENT BACK AND FORTH OVER THE INTERCONNECT
4HE METRIC EXPLANATIONS IN &IGURE
B ARE ACTUALLY INCORRECT 4HE "YTES 2EAD &ROM $ISKS ACTUALLY
MEANS TOTAL BYTES OF READS and WRITES THAT /RACLE $ATABASE HAS ISSUED NOT JUST FOR 3MART 3CANS BUT FOR
ANY REASON !ND THE "YTES 2ETURNED &ROM %XADATA ACTUALLY MEANS THE TOTAL INTERCONNECT TRAFFIC BETWEEN
THE DATABASE AND THE CELLS CAUSED BY ANY REASON SUCH AS BLOCK READS AND WRITES AND ARRAYS OF RETURNED
ROWS BY 3MART 3CANS
Figure 12-7b. Cell Offload Efficiency ratio in the SQL Monitoring detail page
)F YOU ARE WONDERING WHY THE )/ INTERCONNECT BYTES IS '" WHEN THE ACTUAL DATABASE
ISSUED )/
IS ONLY '" THE EXPLANATION IS IN HOW THESE METRICS ARE MEASURED 4HE DATABASE )/ METRIC THE "YTES
2EAD &ROM $ISK IN &IGURE
IS MEASURED BY THE DATABASE LAYER WHILE THE )/ INTERCONNECT BYTES IS
MEASURED BY THE LOW
LEVEL INTERCONNECT/RACLE NETWORKING LAYER !ND ONE OF THE LAYERS IN BETWEEN IS THE
!3- LAYER WHICH MANAGES THE SOFTWARE MIRRORING AMONG OTHER THINGS 3O THE INTERCONNECT )/ TRAFFIC IN
THIS CASE IS HIGHER THAN THE DATABASE TRAFFIC THANKS TO THE WRITE OPERATIONS DONE BY THE 31, STATEMENT
WHICH HAD TO BE MIRRORED BY THE !3- LAYER 3O EVERY -" WRITTEN BY DATABASE WHETHER BECAUSE OF DIRECT
PATH LOAD OR SOME WORK AREA OPERATION SPILLING TO TEMPORARY TABLESPACE RESULTED IN THE WRITING OF -"
OF DATA THANKS TO !3- NORMAL REDUNDANCY MIRRORING
4HERE ARE OTHER REASONS WHY THE INTERCONNECT TRAFFIC MAY BE HIGHER THAN THE ACTUAL AMOUNT OF DATA
READ /NE EXAMPLE IS (## COMPRESSED TABLES )F YOU HAVE COMPRESSED YOUR '" PARTITION DOWN TO '"
THEN YOU WILL HAVE TO DO '" WORTH OF )/ TO READ IT "UT NOW IF THE 3MART 3CAN UNCOMPRESSES THIS DATA IN
THE CELLS ON THE FLY AND RETURNS ALL THAT '" OF uncompressed DATA BACK OVER THE INTERCONNECT ASSUMING
NO PROJECTION OR FILTRATION IN THE CELL WAS DONE THE )/ INTERCONNECT BYTES WILL BE MUCH HIGHER THAN THE
AMOUNT OF DATA READ FROM DISKS 4HIS WOULD DRIVE THE #ELL /FFLOAD %FFICIENCY DOWN TO FOR THIS
EXAMPLE 3MART 3CAN !LL THIS IS YET ANOTHER REASON WHY YOU SHOULDNT FOCUS SOLELY ON IMPROVING JUST THE
#ELL /FFLOAD %FFICIENCY RATIO BUT SHOULD RATHER LOOK INTO WHERE RESPONSE TIME IS SPENT INSTEAD 4IME IS
WHAT END USERS CARE ABOUT
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Execution Plan Rowsource Details
.OW THAT YOU HAVE CHECKED THE KEY METRICS OF YOUR STATEMENT EXECUTION DURATION DATABASE TIME
PARALLELISM USED AND HOW MUCH OF THE DATABASE TIME IS SPENT RUNNING ON #05 VS WAITING ITS TIME TO
DRILL DOWN INTO THE DETAILS OF THE EXECUTION PLAN AT THE ROW SOURCE LEVEL 4HESE ARE SHOWN IN &IGURE
Figure 12-8. Execution plan rowsource activity monitoring in the SQL Monitoring detail page
,ETS FOCUS ON THE RIGHT
HAND COLUMNS OF THE EXECUTION PLAN OUTPUT FIRST 4HE #05 !CTIVITY COLUMN SHOWS
A BREAKDOWN OF TOTAL #05 USAGE BY THE STATEMENT SO FAR 4HE LONGEST BAR AT THE B>AC>A34A1H LINE IN THE
EXECUTION PLAN SHOWS THAT THIS ROWSOURCE CONSUMED OF TOTAL #05 USAGE OF THAT STATEMENT EXECUTION
SO FAR .OTE THAT THIS IS OF TOTAL #05 CONSUMPTION NOT OF TOTAL DURATION OR DATABASE TIME OF THE
STATEMENT EXECUTION 9OU SHOULD EXAMINE THE 4IME AND 7AIT 3TATISTICS SECTION SHOWN EARLIER IN &IGURE
TO SEE HOW MUCH OF THE TOTAL DATABASE TIME WAS CONSUMED BY #05 USAGE AND HOW MUCH BY WAITS
4HE 7AIT !CTIVITY COLUMN SHOWS A BREAKDOWN OF WHICH WAIT EVENTS THIS STATEMENT EXECUTION WAITED
FOR !GAIN THE B>AC>A34A1H ROWSOURCE HAS EXPERIENCED THE MOST WAITS OF TOTAL WAIT TIME OF THAT
STATEMENT EXECUTION .OTE THAT THIS BAR CONSISTS OF TWO DIFFERENT COLOR BARS )F YOU MOVE YOUR MOUSE
POINTER OVER THESE BARS YOULL SEE THE NAMES OF THESE WAIT EVENTS )N THIS CASE THESE WAITS WERE direct
path read temp AND direct path write temp EVENTS WHICH YOU SEE WITH MULTI
PASS WORK AREA OPERATIONS
SUCH AS SORTS AND HASH JOINS 4HIS COLUMN ALSO SHOWS PERCENTAGES OF total wait time OF THIS STATEMENT
EXECUTION NOT THE PERCENTAGE OF TOTAL RUNTIME DURATION NOR TOTAL DATABASE TIME
4HE )/ "YTES COLUMN SHOWS HOW MANY )/ OPERATIONS OR BYTES WERE READ OR WRITTEN BY A ROWSOURCE
9OU CAN RIGHT
CLICK ON THE CHART TO TOGGLE THE DISPLAY OF BYTES OR )/ OPERATIONS )N &IGURE
YOU SEE
THAT THE B>AC>A34A1H ROWSOURCE HAS DONE A TOTAL OF '" WORTH OF )/ 7HEN YOU MOVE YOUR MOUSE
OVER THE DIFFERENT COLOR BARS YOULL SEE THE DETAILS ABOUT HOW MUCH WAS WRITTEN AND HOW MUCH WAS READ
ROUGHLY AS THE CHART SEEMS TO SHOW 4HE EXECUTION PLAN STEP ;>030BB4;42C ABOVE IT HAS WRITTEN
ABOUT '" OF DATA TO DISK .OTE THAT THIS '" OF DATA IS MEASURED AT THE /RACLE DATABASE LEVEL BUT
THE LOWER
LEVEL !3- LAYER WILL LIKELY MIRROR OR TRIPLE
MIRROR DEPENDING ON YOUR CONFIGURATION THESE
WRITES SO ACTUALLY TWICE AS MANY BYTES WERE WRITTEN PHYSICALLY TO DISKS 9OU CAN CHECK THE #ELL 0HYSICAL
)/ )NTERCONNECT "YTES SESSION STATISTIC TO SEE HOW MUCH DATA WAS REALLY SENT AND RECEIVED OVER THE
)NFINI"AND NETWORK THIS METRIC IS AWARE OF THE !3- MIRROR WRITE OVERHEAD AND ANY OTHER LOW
LEVEL TRAFFIC
TOO
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
,ETS LOOK AT A FEW MORE COLUMNS OF THE 2EAL 4IME 31, -ONITORING PAGE 4HE 4IMELINE COLUMN YOU
SEE IN &IGURE
IS ONE OF THE MOST IMPORTANT COLUMNS FOR UNDERSTANDING WHERE THE 31, STATEMENT
SPENDS ITS TIME
Figure 12-9. Row-source activity timeline in the SQL Monitoring detail page
4HE 4IMELINE COLUMN IN THE FIGURE SHOWS A VISUAL TIMELINE OF INDIVIDUAL EXECUTION PLAN ROWSOURCES
ACTIVITY )T IS BASED ON !3( SAMPLES !3( COLLECTS 31, EXECUTION PLAN LINE
LEVEL DETAILS STARTING FROM
/RACLE G2 ,OOK INTO THE BRACKETS IN THE 4IMELINE COLUMN HEADER 4HIS SHOULD SHOW YOU THE TOTAL
RUNTIME DURATION OF THIS 31, EXECUTION SECONDS IN THIS CASE AROUND MINUTES 3O THE TOTAL
WIDTH OF THE TIMELINE COLUMN MEANS ABOUT MINUTES OF WALL
CLOCK RUNTIME .OW ITS EASY TO VISUALLY
INTERPRET THE LENGTH AND POSITION OF THESE BARS IN EACH ROWSOURCE IN THE EXECUTION PLAN 7HEN YOU LOOK
AT THE C01;40224BB5D;; 4 ROWSOURCE AT THE BOTTOM OF THE PLAN YOU SEE THAT THIS TABLE SCAN WAS
ACTIVE FROM THE BEGINNING OF THE EXECUTION AND CONTINUED UP TO AROUND TWO
THIRDS OF THE TOTAL 4IMELINE
TO AROUND SECONDS AND THEN IT COMPLETED 3O IT LOOKS LIKE THIS TABLE SCAN WAS ACTIVE DURING MOST OF
THE EXECUTION TIME
/RACLE EXECUTION PLANS ARE TREES OF ROWSOURCE FUNCTIONS WITH STRICT HIERARCHY ENFORCED SO A CHILD
ROWSOURCE CAN PASS ITS RESULTS ONLY TO ITS DIRECT PARENT )N OUR CASE THE C01;40224BB5D;;AGAINST 4
FETCHED SOME ROWS AND PASSED THEM BACK TO ITS PARENT OPERATION ?G1;>2:8C4A0C>A WHICH THEN SENT
THE ROWS BACK TO THE ?GB4=3A0=64OPERATOR WHICH KNOWS HOW TO SEND RESULTS TO OTHER SLAVES VIA THE 08
4ABLE 1UEUE BUFFERS IN THE 3'! !NOTHER SET OF SLAVES RECEIVED THESE ROWS USING THE ?GA4248E4
ROWSOURCE AND PASSED THESE ROWS TO THEIR PARENT THE B>AC>A34A1HROWSOURCE WHICH PERFORMED THE
SORTING OF THESE ROWS 4HEN cWTB>AC>A34A1HROWSOURCE JUST REPEATED ITS LOOP AND ASKED FOR MORE ROWS
FROM ITS CHILD OPERATOR ?GA4248E4 WHICH THEN READ CONSUMED THE TABLE QUEUE BUFFERS FOR MORE ROWS
PUT THERE BY THE FIRST SET OF SLAVES PRODUCERS 7E ARE NOT GOING DEEPER INTO 31, EXECUTION ENGINE
INTERNALS HERE BUT HOPEFULLY THIS EXAMPLE HAS ILLUSTRATED THE HIERARCHICAL NATURE OF THE 31, EXECUTION
AND HOW THE DATA hFLOWSv UPWARD THROUGH THE EXECUTION PLAN TREE TOWARD THE ROOT OF THE TREE 2A40C4
C01;4BC0C4<4=C IN THIS CASE
7HEN YOU LOOK AT THE TIMELINE BARS YOU SEE THAT ALL THE BARS STARTING FROM ?GA4248E4 AND BELOW IT
STARTED ROUGHLY AT THE SAME TIME AND ENDED ROUGHLY AT THE SAME TIME TOO 7E SAY ROUGHLY BECAUSE THIS
DATA COMES FROM !3( SAMPLES SAMPLED ONCE PER SECOND ONLY SO WE DONT HAVE MICROSECOND ACCURACY
HERE !ND WE DONT NEED IT ANYWAY AS OUR QUERY RUNS FOR MUCH LONGER THAN A SECOND "UT ANYWAY THE
TIMELINE BARS INDICATE THAT THE TABLE SCANNING PART OF THE QUERY INCLUDING THESE hDATA TRANSPORTATIONv
STEPS FOR THAT DATA ?GA4248E4 ?GB4=3A0=64 AND ?G1;>2:8C4A0C>A TOOK AROUND OF THE TOTAL
QUERY RESPONSE TIME )F WE WANT TO MAKE OUR QUERY FASTER WE SHOULD PROBABLY FOCUS ON THAT PART WE
SHOULD EITHER SCAN LESS DATA FILTER MORE IN CELLS HOPEFULLY ACCESS LESS PARTITIONS OR JUST INCREASE THE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
SCANNING THROUGHPUT BY INCREASING PARALLELISM IN THIS CASE BECAUSE THIS C01;40224BB ROWSOURCE WASNT
WAITING TOO MUCH FOR )/ AS SHOWN IN &IGURE
ITS NOT )/
BOUND YET AND THE DATABASE NODES HAD
PLENTY OF #05 RESOURCE LEFT
7HEN YOU LOOK AT THE UPPER ROWSOURCES ;>030BB4;42C AND HIGHER YOU SEE THAT THEY HAVE ONLY
BEEN REPORTED ACTIVE IN THE LAST THIRD OF THE QUERY RESPONSE TIME EXTENDING ALL THE WAY TO THE END OF
EXECUTION 3O IF WE SOMEHOW MADE ONLY THIS DATA LOADING PART FASTER WE WOULD ONLY WIN UP TO OF
THE RESPONSE TIME BACK EVEN IF WE COULD MAKE THE LOADING PART INFINITELY FAST 4HE TABLE ACCESS AND
SORTING WOULD STILL TAKE THE MAJORITY OF THE RESPONSE TIME 4HE TIMELINE BARS ARE VERY VALUABLE FOR QUICKLY
UNDERSTANDING IN WHICH EXECUTION PLAN TREE PART BRANCH MOST OF THE RESPONSE TIME IS SPENT
4HE TIMELINE BARS ARE JUST THE FIRST THING TO CHECK 4HERE ARE LOTS OF OTHER USEFUL DETAILS IN THE 31,
-ONITORING REPORT TOO &OR EXAMPLE IF YOU LOOK AT THE B>AC>A34A1H LINE IN &IGURE
YOU SEE THAT THE
SORT OPERATION HAS BEEN ACTIVE THROUGHOUT THE WHOLE EXECUTION OF THIS STATEMENT !ND IF YOU KNOW THE
BASICS OF SORTING IT SHOULD MAKE SENSETHE SORTING ROWSOURCE WAS ACTIVE FOR THE FIRST TWO
THIRDS OF THE
TIME BECAUSE IT WAS FETCHING DATA INTO SORT BUFFERS AND SORTING IT AND SPILLING SOME DATA TO TEMPORARY
TABLESPACE !FTER THE SORT ITSELF WAS COMPLETE THE CHILD ROWSOURCE RETURNED AN hEND OF DATAv CONDITION
THE SORTING ROWSOURCE FUNCTION HAD STILL HAD TO BE INVOKED )T IS BY INVOKING THAT FUNCTION THAT THE SORT
OPERATION RETURNS THE SORTED ROWS TO ITS PARENT OPERATION 4HIS IS WHY THE PARENT ROWSOURCES BECAME
ACTIVE ONLY TOWARD THE END OF THE EXECUTION TIMELINEBECAUSE all THE ROWS HAD TO BE FETCHED AND SORTED
FIRST BEFORE ANYTHING COULD HAVE BEEN RETURNED FOR FURTHER PROCESSING !ND ONCE ALL THE ROWS were
FETCHED AND SORTED THERE WAS NO REASON FOR THE B>AC>A34A1H ROWSOURCE TO VISIT ITS CHILD ROWSOURCES
AGAIN 4HIS IS WHY THE TIMELINE BAR FOR C01;40224BB ENDED AT AROUND TWO
THIRDS OF THE TOTAL EXECUTION
DURATION TIMELINE
Manually Querying Real-Time SQL Monitoring Data
!LL THE PRETTY CHARTS IN 'RID #ONTROL ARE BASED ON SOME E OR 310N VIEWS INTERNALLY SO IF YOU DONT HAPPEN
TO HAVE ACCESS TO 'RID #ONTROL OR %NTERPRISE -ANAGER $ATABASE #ONSOLE OR IT IS JUST BROKEN YOU CAN GET
WHAT YOU WANT FROM THE E VIEWS DIRECTLY 9OU PROBABLY DONT HAVE TO ACCESS THE E VIEWS FOR YOUR
EVERYDAY MONITORING AND TUNING TASKS BUT NEVERTHELESS ITS USEFUL TO KNOW WHERE THIS INFORMATION
COMES FROM AS IT MAY BECOME HANDY FOR CUSTOM MONITORING AND ADVANCED PROBLEM TROUBLESHOOTING
&OLLOWING ARE SOME KEY VIEWS TO BE AWARE OF
v 4HE 6EB@;N<>=8C>A VIEW CONTAINS THE STATEMENT EXECUTION LEVEL MONITORING DATA
7HEN MULTIPLE SESSIONS ARE RUNNING THE SAME STATEMENT YOU WILL HAVE MULTIPLE
ENTRIES IN THIS VIEW 3O MAKE SURE THAT YOU QUERY THE RIGHT EXECUTION BY USING THE
RIGHT SEARCH FILTERS &OR EXAMPLE YOU SHOULD PAY ATTENTION TO WHICH B83 AND
8=BCN83 YOU ARE REALLY LOOKING FOR OR ?GN@2B83 ?GN@28=BCN83 IF MONITORING A
PARALLEL QUERY AND WHETHER THE BC0CDB COLUMN STILL SHOWS 4G42DC8=6 IF YOU ARE
TRYING TO TROUBLESHOOT A CURRENTLY RUNNING QUERY
v 4HE 6EB@;N?;0=N<>=8C>A VIEW CONTAINS EXECUTION PLAN LINE LEVEL METRICS
MONITORED AND UPDATED IN REAL TIME &OR EXAMPLE YOU CAN QUERY THE
8>N8=C4A2>==42CN1HC4B AND COMPARE IT TO ?7HB820;NA403N1HC4B AND
?7HB820;NFA8C4N1HC4B TO DETERMINE THE OFFLOADING EFFICIENCY BY EACH INDIVIDUAL
EXECUTION PLAN LINE INSTEAD OF THE WHOLE QUERY EFFICIENCY .OTE THAT INCREASING THE
OFFLOADING EFFICIENCY PERCENTAGE SHOULD NOT BE YOUR PRIMARY GOAL OF MONITORING
AND TUNING n WHERE YOU SPEND YOUR RESPONSE time MATTERS
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
v 4HE 6E02C8E4NB4BB8>=N78BC>AH VIEW CONTAINS COLUMNS LIKE B@;N?;0=N;8=4N83
B@;N?;0=N>?4A0C8>= AND B@;N?;0=N>?C8>=B STARTING FROM /RACLE G2 9OU CAN
QUERY THESE COLUMNS IN ADDITION TO B@;N83 TO FIND THE TOP ROWSOURCES OF AN 31,
EXECUTION PLAN TOO INSTEAD OF JUST LISTING THE TOP 31, STATEMENT
Reporting Real-Time SQL Monitoring Data with DBMS_SQLTUNE
)F YOU DO NOT HAVE ACCESS TO 'RID #ONTROL OR %NTERPRISE -ANAGER $ATABASE #ONSOLE FOR SOME REASON YOU
CAN ALSO EXTRACT THE 31, MONITORING DETAILS USING THE 31<BNB@;CD=4 A4?>ACNB@;N<>=8C>A PACKAGE
FUNCTION 9OU CAN USE THE FOLLOWING SYNTAX BUT MAKE SURE YOU READ THE RELATED DOCUMENTATION TO SEE THE
FULL POWER OF THIS FEATURE .OTE THAT THE TEXT BELOW IS EDITED AS THIS FUNCTION GENERATES VERY WIDE OUTPUT
WHICH WOULDNT FIT THE BOOK PAGES
B@;-B4;42C
31<BNB@;CD=4 A4?>ACNB@;N<>=8C>A
bTbbX^]NXS,-bXS
aT_^acN[TeT[,-0;;
ch_T,-C4GCPbaT_^ac
5A><SdP[
B@;-
4]cTaeP[dTU^abXS)$#"
A4?>AC
B@;<^]Xc^aX]VAT_^ac
B@;CTgc
B4;42C _PaP[[T["! 2>D=C5A><C#0
6[^QP[8]U^a\PcX^]
BcPcdb)4G42DC8=6
8]bcP]RT83)!
BTbbX^])C0=4;$#")#%#
B@;83) SdPc(&gc# &Z
B@;4gTRdcX^]83)""$$$$%
4gTRdcX^]BcPacTS) $ !& ! %)$()!"
5XabcATUaTbWCX\T) $ !& ! %)$()!"
;PbcATUaTbWCX\T) $ !& ! %)$()$(
3daPcX^])"%b
<^Sd[T 0RcX^])B@;?[db
BTaeXRT)B0=31>G
?a^VaP\)b`[_[db/\PR [P]C=BE E"
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
6[^QP[BcPcb
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
k4[P_bTSk@dTdX]Vk2_dk8>k1dUUTakATPSkATPSk2T[[k
kCX\TbkCX\TbkCX\TbkFPXcbbk6TcbkAT`bk1hcTbk>UU[^PSk
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
k ($k k &k &(k <k & :k'"61k(% % k
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
B@;?[P]<^]Xc^aX]V3TcPX[b?[P]7PbWEP[dT,"$"%$%&%
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
k8Sk>_TaPcX^]k=P\TkATPSk2T[[k0RcXeXchk
kkkk1hcTbk>UU[^PSkk
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
k kB4;42CBC0C4<4=Ckkkkk
k kB>AC066A460C4kkkkk
k!k?G2>>A38=0C>Akkkkk
k"k?GB4=3@2A0=3><k)C@ kkkk
k -#kB>AC066A460C4kkkk (k
k -$k?G1;>2:8C4A0C>Akkkkk
k -%kC01;40224BBBC>A0645D;;kC#k'&61k(% % k(( ( k
kkkkkkk
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
31<BNB@;CD=4 A4?>ACNB@;N<>=8C>A CAN ALSO TAKE (4-, AS A VALUE FOR THE CH?4 PARAMETER INSTEAD OF C4GC
IN WHICH CASE THE OUTPUT IS GENERATED AS (4-, )F YOU SPOOL THIS OUTPUT INTO AN (4-, FILE AND OPEN IT IN
THE BROWSER YOU WILL SEE MUCH NICER OUTPUT THAN JUST TEXT !ND STARTING WITH /RACLE YOU CAN ALSO
USE 02C8E4 AS A PARAMETER AND SPOOL THAT OUTPUT INTO AN (4-, FILE .OW IF YOU OPEN THIS FILE IN THE
BROWSER YOU WILL SEE THE 31, -ONITORING PAGE ALMOST EXACTLY AS IT LOOKS IN THE 'RID #ONTROL !ND ALL THE
DATA REQUIRED FOR DISPLAYING THAT REPORT IS SELF
CONTAINED IN THE SPOOLED (4-, FILE NO DATABASE ACCESS
NEEDED WHEN OPENING IT 3O THIS IS VERY USEFUL IF YOU WANT TO SEND A DETAILED SELF
CONTAINED REPORT WITH
SOME 31, EXECUTION PROBLEM TO SOMEONE OVER THE EMAIL
Controlling SQL Monitoring
31, -ONITORING KICKS IN IMMEDIATELY FOR ALL STATEMENTS EXECUTED WITH PARALLEL EXECUTION NO MATTER HOW
LONG THEY RUN &OR SERIALLY EXECUTED STATEMENTS THE 31, -ONITORING DOES NOT KICK IN IMMEDIATELY AS ITS
NOT DESIGNED TO MONITOR TYPICAL FAST /,40 QUERIES WHICH ARE EXECUTED MANY TIMES PER SECOND
.EVERTHELESS IF A SERIAL QUERY HAS CONSUMED MORE THAN SECONDS OF TOTAL #05 ANDOR )/ WAIT TIME ITS
CONSIDERED AS A LONG RUNNING QUERY AND THE 31, -ONITORING IS ENABLED FOR THAT STATEMENT EXECUTION
4HIS HAPPENS SEAMLESSLY ON THE FLY AND STATEMENT RESTART IS NOT NEEDED
9OU CAN ALSO USE<>=8C>A AND =>N<>=8C>A HINTS TO CONTROL THE 31, MONITORING FOR A STATEMENT
EB@;N78=CSHOWS ALL THE HINTS AVAILABLE FOR USE AND THE VERSION WHEN THEY WERE INTRODUCED &OR
EXAMPLE
B@;-B4;42C]P\TX]eTabTeTabX^]R[Pbbb`[NUTPcdaT
!5A><eb`[NWX]cF74A4]P\T;8:4<>=8C>A*
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
=0<48=E4AB4E4AB8>=
=>N<>=8C>A8=6'
<>=8C>A=>N<>=8C>A %
=>N<>=8C>A<>=8C>A %
.OTE THAT THE =>N<>=8C>A8=6HINT IS SOMETHING COMPLETELY DIFFERENT DESPITE THE SIMILAR NAME 4HE
=>N<>=8C>A8=6 HINT ALLOWS YOU TO DISABLE THE PREDICATE USAGE MONITORING ON TABLE COLUMNS
bhb R^[NdbPVT AND IT HAS NOTHING TO DO WITH THE 2EAL
4IME 31, -ONITORING OPTION INTRODUCED IN
/RACLE G
-ONITORING 31, 3TATEMENTS USING 631, AND 631,34!43
4HE PERFORMANCE MONITORING IN THE hOLDDAYSv BEFORE
/RACLE G !3( WAS USUALLY DONE USING VARIOUS E
VIEWS WHICH SHOWED AGGREGATED INSTANCE
WIDE METRICS &OR EXAMPLE THE 3TATSPACKS 4/0
WAIT EVENTS
REPORT SECTION WAS JUST A DELTA BETWEEN TWO EBHBC4<N4E4=C VIEW SNAPSHOTS 4HE 4/0 31, REPORTS WERE
BASED ON EB@; SNAPSHOTS WHICH EXTERNALIZE THE EXECUTION STATISTICS AND RESOURCE CONSUMPTION FOR EACH
CHILD CURSOR STILL IN LIBRARY CACHE (OWEVER IN A LARGE DATABASE SYSTEM THINK E
"USINESS 3UITE OR 3!0
YOU CAN HAVE TENS OF THOUSANDS OF CURSORS IN THE LIBRARY CACHE SO GATHERING AND STORING DELTAS OF ALL OF
THEIR STATS IS NOT FEASIBLE 4HATS WHY TOOLS LIKE 3TATSPACK AND !72 STORE DELTAS OF ONLY SOME TOP
RESOURCE
CONSUMING STATEMENTS AND IGNORE THE INSIGNIFICANT ONES 2EMEMBER THAT AS THESE TOOLS GATHER
INSTANCE
WIDE DATA THEY MAY END UP IGNORING A LONG
RUNNING STATEMENT IF THERE ARE ONLY A FEW SESSIONS
EXECUTING IT ! SINGLE SESSION RUNNING A BAD 31, STATEMENT MAY NOT BE hHEARDv IN THE NOISE OF ALL THE
OTHER SESSIONS IN THE INSTANCEPOTENTIALLY THOUSANDS OF THEM 4HIS INSTANCE
WIDE SCOPE PERFORMANCE
DATA ANALYSIS IS NOT AS POWERFUL AS THE SESSION
LEVEL !3( DATA SLICING AND DICING 7ITH !3( YOU CAN DRILL
DOWN INTO ANY SINGLE SESSION REGARDLESS OF HOW MANY SESSIONS IN TOTAL YOU HAVE MAKING NOISE IN THE
INSTANCE
)F YOU USE %XADATA YOU MUST BE RUNNING AT LEAST /RACLE G 2 ON IT SO ALL THESE SUPERIOR TOOLS ARE
AVAILABLE ASSUMING THAT YOU HAVE THE $IAGNOSTICS AND 4UNING 0ACK LICENSES FOR YOUR %XADATA CLUSTER "Y
THE WAY WE HAVENT SEEN AN %XADATA
USING CUSTOMER WITHOUT $IAGNOSTICS AND 4UNING 0ACK LICENSES YET
SO IT LOOKS LIKE THE VAST MAJORITY OF %XADATA USERS DO NOT HAVE TO RESORT TO OLD TOOLS LIKE 3TATSPACK OR CREATE
A CUSTOM !3(
STYLE REPOSITORY THEMSELVES ALTHOUGH ITS EASILY DOABLE WITH A FEW LINES OF 0,31, CODE
POLLING EB4BB8>= OR ITS UNDERLYING G:BDB4 VIEW IN EACH INSTANCE
4HE EB@; AND EB@;BC0CB VIEWS STILL DO HAVE SOME ADVANTAGE OVER 31, -ONITORING AND !3(
STYLE
SAMPLED DATA IN A FEW CASES &OR EXAMPLE IF YOU WANT TO MEASURE METRICS LIKE THE NUMBER OF EXECUTIONS
BUFFER GETS PARSE CALLS FETCHES OR ROWS RETURNED BY THE 31, CHILD CURSOR YOU CAN GET THIS DATA FROM BOTH
2EAL
TIME 31, -ONITORING EB@;N<>=8C>A OR THE EB@;EB@;BC0CB VIEWS BUT NOT !3( "UT THE PROBLEM
WITH 31, -ONITORING IS THAT IT DOESNT MONITOR SHORT
RUNNING QUERIES AT ALL THEREFORE MAKING IT UNUSABLE
FOR KEEPING TRACK OF /,40
STYLE SMALL QUERIES EXECUTED MANY TIMES PER SECOND %VEN ADDING A <>=8C>A
HINT INTO EVERY QUERY OF YOUR APPLICATION WOULD NOT HELP AS THE MAXIMUM NUMBER OF MONITORED PLANS IS
LIMITED CONTROLLED BY THE Nb`[\^]N\PgN_[P] PARAMETER WHICH DEFAULTS TO PLANS PER #05 AND YOU
WOULD LIKELY END UP WITH real-time plan statistics latch CONTENTION AS WELL 4HE 31, -ONITORING FEATURE IS
NOT MEANT TO MONITOR SHORT
RUNNING QUERIES EXECUTED MANY TIMES PER SECOND
!ND THIS LEAVES US WITH EB@; AND EB@;BC0CB 4HEY BOTH MAINTAIN SIMILAR DATA BUT THEY ARE
INTERNALLY DIFFERENT 7HENEVER YOU QUERY EB@; WITHOUT SPECIFYING AN EXACT B@;N83 /RACLE HAS TO TRAVERSE
THROUGH EVERY SINGLE LIBRARY CACHE HASH BUCKET AND ALL CURSORS UNDER IT 4HIS MAY CONTRIBUTE TO LIBRARY
CACHE MUTEX CONTENTION IF YOU HAVE A BUSY DATABASE AND LOTS OF CURSORS IN THE LIBRARY CACHE BECAUSE
WHEN YOU TRAVERSE THE LIBRARY CACHE STRUCTURE AND READ ITS OBJECTS CONTENTS YOULL HAVE TO HOLD A MUTEX
ON THE OBJECT .OTE THAT STARTING FROM /RACLE G ALL LIBRARY CACHE LATCHES ARE GONE AND ARE REPLACED BY
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
MUTEXES 4HESE SAME MUTEXES ARE USED FOR PARSING LOOKING UP AND PINNING CURSORS FOR EXECUTION SO IF
YOUR MONITORING QUERIES POLL EB@; FREQUENTLY THEY MAY END UP CAUSING WAITS FOR OTHER SESSIONS
4HE EB@;BC0CB VIEW WHICH WAS INTRODUCED IN /RACLE G2 DOESNT HAVE THIS PROBLEM 3TARTING
FROM /RACLE G2 /RACLE ACTUALLY MAINTAINS 31, EXECUTION STATISTICS IN TWO PLACES INSIDE THE CHILD
CURSORS THEMSELVES EB@; AND IN A SEPARATE CURSOR STATISTICS ARRAY STORED IN A DIFFERENT LOCATION IN THE
SHARED POOL 4HIS SEPARATION GIVES THE BENEFIT THAT EVEN IF A CURSOR IS FLUSHED OUT FROM THE SHARED POOL ITS
STATS IN THIS SEPARATE ARRAY MAY REMAIN AVAILABLE FOR LONGER !LSO WHEN MONITORING TOOLS QUERY
EB@;BC0CB THEY DONT HAVE TO SCAN THROUGH THE ENTIRE LIBRARY CACHE THANKS TO THE SEPARATE ARRAY WHERE
STATS ARE STORED 4HIS MEANS THAT YOUR MONITORING TOOLS WONT CAUSE ADDITIONAL LIBRARY CACHE LATCH OR
MUTEX CONTENTION WHEN THEY USE EB@;BC0CB INSTEAD OF EB@; "OTH 3TATSPACK AND !72 DO USE
EB@;BC0CB TO COLLECT DATA FOR THEIR TOP 31, REPORTS
,ETS LOOK INTO EB@;BC0CB THE EB@; VIEW HAS PRETTY MUCH THE SAME COLUMNS BY THE WAY 3OME OF
THE OUTPUT HERE IS REMOVED TO SAVE SPACE
B@;-/STbReb`[bcPcb
=P\T=d[[.Ch_T
B@;NC4GCE0A270A!
!B@;N5D;;C4GC2;>1
"B@;N83E0A270A! "
%?;0=N70B7NE0;D4=D<14A
! 2?DNC8<4=D<14A
! 4;0?B43NC8<4=D<14A
!%DB4AN8>NF08CNC8<4=D<14A
""8>N24;;N>55;>03N4;8681;4N1HC4B=D<14A
"#8>N8=C4A2>==42CN1HC4B=D<14A
"$?7HB820;NA403NA4@D4BCB=D<14A
"%?7HB820;NA403N1HC4B=D<14A
"&?7HB820;NFA8C4NA4@D4BCB=D<14A
"'?7HB820;NFA8C4N1HC4B=D<14A
# 8>N24;;ND=2><?A4BB43N1HC4B=D<14A
#!8>N24;;N>55;>03NA4CDA=43N1HC4B=D<14A
4HE HIGHLIGHTED ROWS STARTING WITH 8>N24;; ARE METRICS RELATED TO %XADATA STORAGE CELLS ALTHOUGH THE
8>N8=C4A2>==42CN1HC4B IS POPULATED ON NON %XADATA DATABASES AS WELL THANKS TO A BUG 9OU WOULD
WANT TO COMPARE THESE CELL METRICS TO DATABASE METRICS LIKE _WhbXRP[NaTPSNQhcTb TO UNDERSTAND
WHETHER THIS 31, STATEMENT IS BENEFITTING FROM %XADATA 3MART 3CAN OFFLOADING .OTE THAT THESE METRICS
WHICH MERELY COUNT BYTES SHOULD NOT BE USED AS THE PRIMARY METRIC OF performance AGAIN THE PRIMARY
METRIC THE STARTING POINT SHOULD ALWAYS BE RESPONSE TIME WHICH YOU CAN THEN BREAK DOWN INTO
INDIVIDUAL WAIT EVENTS OR INTO EXECUTION PLAN ROWSOURCE ACTIVITY WITH THE 31, -ONITORING REPORT OR
!3( 9OU CAN LEARN MORE ABOUT THE MEANING OF THE METRICS SHOWN HERE IN #HAPTER
.OTE THAT EVEN THOUGH THE EB@;BC0CB VIEW ALSO CONTAINS THE ?;0=N70B7NE0;D4 COLUMN IT ACTUALLY
DOES NOT STORE SEPARATE STATS FOR THE SAME 31, )$ WITH DIFFERENT PLAN HASH VALUES )T AGGREGATES ALL THE
STATS GENERATED BY DIFFERENT PLANS OF THE SAME 31, )$ UNDER A SINGLE BUCKET 3O YOU DONT REALLY KNOW
WHICH PLAN VERSION CONSUMED THE MOST RESOURCES FROM THIS VIEW ,UCKILY IN /RACLE THERE IS A NEW
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
VIEW NAMED EB@;BC0CBN?;0=N70B7 WHICH YOU SHOULD QUERY INSTEAD )T ORGANIZES AND REPORTS THE STATS
BROKEN DOWN BY 31, )$ and PLAN HASH VALUE INSTEAD OF JUST 31, )$ AS EB@;BC0CB DOES
Monitoring the Storage Cell Layer
,ETS LOOK AT HOW TO MONITOR THE STORAGE CELL LAYER FOR UTILIZATION AND EFFICIENCY !S YOU ALREADY KNOW THE
STORAGE CELLS ARE REGULAR COMMODITY SERVERS WITH SOME DISKS ATTACHED TO THEM AND RUNNING ,INUX 9OU
CAN USE REGULAR ,INUX /3 MONITORING COMMANDS AND TOOLS TO KEEP AN EYE ON /3 METRICS LIKE #05 USAGE
OR DISK )/ ACTIVITY 4HE ONLY CATCH IS THAT /RACLE DOES NOT ALLOW YOU TO INSTALL ANY ADDITIONAL
MONITORING DAEMONS AND SOFTWARE ON THE CELLS IF YOU STILL WANT TO HAVE A SUPPORTED CONFIGURATION
4HE GOOD NEWS IS THAT /RACLE CELL SERVER SOFTWARE AND ADDITIONAL /3 7ATCHER SCRIPTS DO A GOOD JOB OF
COLLECTING DETAILED PERFORMANCE METRICS IN EACH CELL 4HE FOLLOWING SECTIONS SHOW HOW TO EXTRACT DISPLAY
AND USE SOME OF THESE METRICS
!CCESSING #ELL -ETRICS IN THE #ELL ,AYER 5SING #ELL#,)
4HE CELL
COLLECTED METRICS CAN BE ACCESSED USING THE #ELL#,) UTILITY 4HATS IN FACT HOW THE %XADATA
STORAGE CELL PLUG
IN FOR 'RID #ONTROL RETRIEVES ITS DATA 9OU HAVE SEEN #ELL#,) USED IN EARLIER CHAPTERS
&OLLOWING ARE A FEW EXAMPLES OF USING #ELL#,) FOR RETRIEVING PERFORMANCE METRICS 4HE COMMAND NAMES
SHOULD BE FAIRLY SELF
EXPLANATORY
WE WILL COVER THE #ELL#,) COMMANDS IN MORE DETAIL IN !PPENDIX !
RT[[R[X
R
2T[[2;8)AT[TPbT ! ! " ?a^SdRcX^]^]<^]<Ph % ()!!) (23C!
2^_haXVWcR! &! (>aPR[T 0[[aXVWcbaTbTaeTS
2T[[4UUXRXT]RhAPcX^) %"
2T[[2;8-;;8BC<4CA823458=8C8>= Å show metric short names
23N8>N1HNAN;6
23N8>N1HNAN;6NB42
23N8>N1HNANB<
23N8>N1HNANB<NB42
23N8>N1HNFN;6
23N8>N1HNFN;6NB42
23N8>N1HNFNB<
23N8>N1HNFNB<NB42
... lots of output removed ...
=N<1NA4248E43
=N<1NA4248E43NB42
=N<1NA4B4=C
=N<1NA4B4=CNB42
=N<1NB4=C
=N<1NB4=CNB42
=N=82N:1NA2ENB42
=N=82N:1NCA0=BNB42
=N=82N=F
=NA3<0NA4CAHNC<
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
2T[[2;8-;8BC<4CA823458=8C8>=2;N2?DC334C08;*Å show metric description and details
]P\T)2;N2?DC
STbRaX_cX^])?TaRT]cPVT^UcX\T^eTacWT_aTeX^db\X]dcTcWPccWTbhbcT\
2?DbfTaT]^cXS[T
\TcaXRCh_T)8]bcP]cP]T^db
^QYTRcCh_T)24;;
d]Xc)
<4CA822DAA4=C2;N2?DC*Å list the latest metric snapshot value
2T[[2;8-;8BC<
2;N2?DCT]ZRT[ $ "
<4CA822DAA4=C2;N2?DC34C08;*Å list latest snapshot in detail
2T[[2;8-;8BC<
]P\T)2;N2?DC
P[TacBcPcT)]^a\P[
R^[[TRcX^]CX\T)! $ %C ()" )$" $)
\TcaXR>QYTRc=P\T)T]ZRT[
\TcaXRCh_T)8]bcP]cP]T^db
\TcaXREP[dT)$ "
^QYTRcCh_T)24;;
2T[[2;8-;8BC<<4CA8278BC>AH2;N2?DC*Å show historical metric snapshots
2;N2?DCT]ZRT[ ! !! $ %C !) )#% $)
2;N2?DCT]ZRT[ ! #! $ %C !) )#( $)
2;N2?DCT]ZRT[ ! ! $ %C ") ) $ $)
2;N2?DCT]ZRT[ "! $ %C ") ) $ $)
2;N2?DCT]ZRT[ #! $ %C ") !) $ $)
2;N2?DCT]ZRT[ "! $ %C ") ") $ $)
2;N2?DCT]ZRT[ !! $ %C ") #) $ $)
2;N2?DCT]ZRT[ !! $ %C ") $) $ $)
!CCESSING #ELL -ETRICS 5SING THE 'RID #ONTROL %XADATA
3TORAGE 3ERVER 0LUG
)N
)F YOU HAVE CONFIGURED THE 3YSTEM -ONITORING PLUG
IN FOR %XADATA 3TORAGE 3ERVER YOU WILL BE ABLE TO SEE
SOME CELL
LEVEL METRICS REPORTS AND CHARTS IN 'RID #ONTROL 4HE PLUG
IN INSTALLATION IS OUT OF THE SCOPE OF
THIS BOOK BUT ITS WELL DOCUMENTED IN THE PLUG
INS INSTALLATION GUIDE
#URRENTLY THE %XADATA 3TORAGE 3ERVER MONITORING PLUG
IN GIVES ONLY SOME BASIC METRICS TO MONITOR
AND IS NOT VERY SOPHISTICATED FOR SYSTEMATIC CELL PERFORMANCE TROUBLESHOOTING OR MONITORING
.EVERTHELESS IT WILL GIVE YOU A MORE CONVENIENT WAY FOR MONITORING CELL STATISTICS CHANGING OVER TIME
COMPARED TO RUNNING #ELL#,)S [Xbc\TcaXRWXbc^ah COMMANDS MANUALLY 7HEN YOU INITIALLY NAVIGATE TO
A STORAGE CELL TARGET OR GROUP OF CELLS IN 'RID #ONTROL YOU MAY NOT SEE ANY CHARTS DEFINED THERE YET )N
&IGURE
WE HAVE CREATED A GROUP CALLED hCELLSv FROM THE %XADATA 3TORAGE 3ERVER TARGETS AND
NAVIGATED TO THE %DIT 'ROUP PAGE 4HIS IS WHERE YOU CAN ADD NEW CHARTS FOR COLLECTED CELL
LEVEL METRICS
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-10. Adding storage cell metrics into Grid Control monitoring charts
!LL THE CHART DEFINITIONS IN &IGURE
HAVE BEEN MANUALLY ADDED FOR THE CELL GROUP 9OU CAN ADD CHARTS
YOURSELF BY CLICKING THE !DD BUTTON WHICH WILL BRING YOU TO THE !DD #HART PAGE SHOWN IN &IGURE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-11. Selecting a “highest-average” chart for monitoring cells with highest read I/O requests
4O SEE THE CELL METRICS AVAILABLE YOULL HAVE TO SELECT /RACLE %XADATA 3TORAGE 3ERVER AS THE CHART TYPE AND
PICK ONE OF THE METRICS TO SHOW 9OU PROBABLY WANT TO USE THE 'ROUP 3TATISTICS FOR MONITORING THE WHOLE
CELL GROUP AS A STARTING POINT INSTEAD OF MONITORING ALL STORAGE CELLS IN A FULL RACK SEPARATELY 4HE
GROUP STATISTICS ALLOW YOU TO SUM TOGETHER STATS FROM ALL THE CELLS IN THE GROUP WHEN MONITORING TOTAL
DISK )/ THROUGHPUT FOR EXAMPLE OR SHOW MINIMUM MAXIMUM AVERAGES AND STANDARD DEVIATION
ACROSS ALL CELLS WHEN MONITORING AVERAGE DISK )/ RESPONSE TIMES FOR EXAMPLE 0AY ATTENTION TO WHAT
THE METRIC MEANS AND WHICH AGGREGATION OPERATIONS MAKE SENSE FOR A GIVEN METRIC &OR EXAMPLE IT
DOESNT MAKE MUCH SENSE TO DISPLAY THE sum OF ALL AVERAGE DISK )/ RESPONSE TIMES IN A CHART 4HE
METRICS STARTING WITH !VG SHOW AVERAGE VALUES OVER MULTIPLE OBJECTS MONITORED &OR EXAMPLE THE !VG
#ELLDISK 2EADS -" MONITORS AVERAGE MEGABYTES READ per celldisk. /N THE OTHER HAND THE 4OTAL #ELLDISK
2EADS -" METRIC SUMS TOGETHER THE -" READ BY ALL CELL DISKS IN A CELL !ND NOW IF YOU USE THE 'ROUP
3TATISTICS OPTION TO MONITOR MULTIPLE CELLS YOU SHOULD USE 3UM TO SEE THE TOTAL -" READ FROM CELL DISKS
ACROSS ALL CELLS AND !VERAGE OR -IN-AX TO SEE THE AVERAGE per cell OR A CELL WITH THE SMALLEST OR BIGGEST
-" READ CELL DISK THROUGHPUT
)F YOU DO WANT TO MONITOR INDIVIDUAL CELLS FOR EXAMPLE THE TWO STORAGE CELLS WITH THE WORST OFFLOAD
EFFICIENCYTHEN YOU CAN EITHER DEFINE A SEPARATE OFFLOADING EFFICIENCY CHART FOR EACH CELL AND END UP
WITH CHARTS IN A FULL RACK CONFIGURATION OR YOU CAN SELECT 4ARGETS 7ITH ,OWEST !VERAGE AS THE CHART
TYPE AS SHOWN IN &IGURE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-12. Selecting a “lowest average” chart for monitoring cells with worst offload efficiency
9OU CAN READ MORE ABOUT THE MEANING OF VARIOUS METRICS IN THE /RACLE DOCUMENT “Enterprise Manager
System Monitoring Plug-In Metric Reference Manual for Oracle Exadata Storage Server.”
.OW YOU CAN REORDER THE CHARTS ON THE CELL GROUP DASHBOARD BY CLICKING THE 2EORDER BUTTON AS
SHOWN IN &IGURE
EARLIER 4HIS WILL SHOW YOU A LIST OF DEFINED CHARTS WHICH YOU CAN THEN REORDER ON
THE SCREEN AS SEEN IN &IGURE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-13. Ordering the storage cell metric charts in Grid Control
.OW YOU ARE READY TO VIEW YOUR CHARTS $EPENDING ON HOW MANY OF THEM YOU CONFIGURED YOU SHOULD SEE
A SCREEN LAYOUT LIKE THAT IN &IGURE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Figure 12-14. Multiple cell-level metrics displayed in Grid Control
.OTE THAT WHEN YOU FIRST CONFIGURE THE CHARTS THEY WILL BE EMPTY )F YOU WAIT FOR A FEW MINUTES YOU
SHOULD SEE THE FIRST DATA POINTS APPEARING AND AFTER A DAY HAS PASSED THE CHARTS SHOULD BE FULLY
POPULATED IF ,AST (OURS IS SELECTED IN THE 6IEW $ATA DROP
DOWN 9OU CAN CLICK #USTOMIZE #HARTS TO
MODIFY THE CHART METRICS AGAIN
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
.OW LETS LOOK AT SOME OF THE INDIVIDUAL METRICS WE HAVE PUT ONTO THE PAGE AS SHOWN IN &IGURE
Figure 12-15. Various celldisk average metrics charts displayed in Grid Control
)N THE !VG #ELLDISK 3MALL 2EAD ,ATENCY MSEC CHART HERE WE DO SEE AN ISSUE !N HOUR BEFORE 0- THE
AVERAGE SMALL BLOCK )/ UNDER K" LATENCY HAS JUMPED UP PROBABLY BECAUSE MORE LARGE )/
OPERATIONS WERE DONE IN THE CELL 7E CAN CONCLUDE THIS FROM THE OTHER TWO METRICS IN THE TOP ROW THE
!VG #ELLIDISK 2EAD 2EQUESTS HAS SIGNIFICANTLY DROPPED WHILE THE !VG #ELLDISK 2EADS (MB) HAS REMAINED
THE SAME OR ACTUALLY EVEN INCREASED SLIGHTLY 3O THERE ARE MORE LARGE )/S BEING DONE PERHAPS DUE TO
MORE QUERIES USING A 3MART 3CAN 4HE SMALL )/ LATENCY HAS SUFFERED BECAUSE OF THIS 9OUR /,40
DATABASE USERS RUNNING ON THE SAME CLUSTER MAY NOTICE THAT
4HIS IS A PLACE WHERE YOU SHOULD FIND OUT WHAT CAUSES ALL THIS EXTRA )/ 7E WILL LOOK AT IT LATER IN THIS
CHAPTER )F YOU WANT TO AVOID SITUATIONS WHERE LARGE 3MART 3CANS IMPACT SMALL SINGLE BLOCK )/S YOU
SHOULD CONSIDER ENABLING THE )/ 2ESOURCE -ANAGER )/2- WHICH ALLOWS YOU TO SET THE )/2- GOAL TO
OPTIMIZE RANDOM )/ LATENCY AS OPPOSED TO FULL SCANNING THROUGHPUT !DDITIONALLY YOU COULD MAKE SURE
THAT YOUR HOT SEGMENTS ACCESSED WITH SINGLE BLOCK READS ARE CACHED IN FLASH CACHE 3EE #HAPTER FOR
MORE ABOUT )/2- AND RESOURCE MANAGEMENT
&IGURE SHOWS AN EXAMPLE OF THE TOTAL CELL DISK )/ THROUGHPUT CHART .OTE THAT THE METRIC USED
STARTS WITH h4OTAL v INDICATING THAT THE THROUGHPUT -" VALUES ARE SUMMED OVER ALL TWELVE DISKS IN THE CELL
AS OPPOSED TO THE !VERAGE METRICS WHICH SHOW AVERAGES ACROSS THE DISKS 4HIS IS A VERY IMPORTANT
POINT )F YOU ARE SUMMING TOGETHER average METRICS LIKE IN THE !VG #ELLDISK 2EADS SECTION TOP RIGHT
CORNER OF &IGURE THEN YOU WILL STILL END UP WITH A SUM OF THE AVERAGE CELL DISK READ RATES NOT THE
TOTAL AGGREGATE THROUGHPUT OF ALL DISKS IN THE CELL 4HE CELL DISKS average METRICS ARE GOOD FOR CHARTING
THINGS LIKE THE MIN MAX AND CELL LEVEL AVERAGE )/ LATENCIES OR DISK UTILIZATION FIGURES BUT NOT VALID FOR
MONITORING THINGS LIKE )/ THROUGHPUT OF ALL CELLS
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
)T IS VALID TO sum TOGETHER 4OTAL #ELLDISK METRICS TO GET THE TOTAL AGGREGATE THROUGHPUT SO FROM
&IGURE
WE SEE THAT THE AGGREGATE DISK READ THROUGHPUT ACROSS ALL THE CELLS IS AROUND
-"SECOND FOR THE GIVEN WORKLOAD
Figure 12-16. Celldisk-level Total metrics summed together for monitoring total I/O throughput
3OMETIMES ESPECIALLY WHEN THE MONITORING 5) CHARTS DONT SEEM TO MAKE SENSE YOU MIGHT ASK WHERE
THESE CHARTS TAKE THEIR DATA FROM 4HIS IS EASY TO FIND OUT AS THE 'RID #ONTROL METRICS PLUG INS ARE JUST
0ERL SCRIPTS WHICH THEN RUN COMMAND LINE UTILITIES LIKE #ELL#,) OR PLAIN 33( CLIENTS TO FETCH THEIR
METRICS 9OU CAN CHECK THE DIRECTORY WHERE YOU HAVE INSTALLED THE %XADATA 3TORAGE #ELL 3ERVER
MONITORING PLUG IN YOURSELF
[b d P__ ^aPR[T _a^SdRc \XSS[TfPaT PVT]c V bbhb\P] PS\X] bRaX_cb T\g ^aPR[TNRT[[
^aPR[TNRT[[NRPcTV^ahN\TcaXRb _[^aPR[TNRT[[NVaXSSXbZNR^]UXV _[
^aPR[TNRT[[NRT[[SXbZNR^]UXV _[^aPR[TNRT[[NVaXSSXbZN\TcaXRb _[
^aPR[TNRT[[NRT[[SXbZN\TcaXRb _[^aPR[TNRT[[NX]cTaR^]]TRcN\TcaXRb _[
^aPR[TNRT[[NR^\\^] _\^aPR[TNRT[[NX^a\Q^^bcN\TcaXRb _[
^aPR[TNRT[[NR^]UXV _[^aPR[TNRT[[NX^a\NR^]UXV _[
^aPR[TNRT[[NR^]bd\TaVa^d_N\TcaXRb _[^aPR[TNRT[[N[d]NR^]UXV _[
^aPR[TNRT[[NSPcPQPbTN\TcaXRb _[^aPR[TNRT[[N[d]N\TcaXRb _[
^aPR[TNRT[[NSXbZNR^]UXV _[^aPR[TNRT[[N\TcaXRb _[
^aPR[TNRT[[NSXbZN\TcaXRb _[^aPR[TNRT[[N^UU[^PSN\TcaXRb _[
^aPR[TNRT[[NU[PbWRPRWTN\TcaXRb _[^aPR[TNRT[[NaTb_ _[
^aPR[TNRT[[NUbN\TcaXRb _[
,ETS TAKE A QUICK LOOK AT ONE OF THESE 0ERL SCRIPTS ^aPR[TNRT[[NRT[[SXbZN\TcaXRb _[ TO SEE WHERE IT GETS
ITS DATA
...beginning of the script skipped...
\hR^\\P]S,bbW [RT[[\^]Xc^aX_PSSaTbbRT[[R[X
[Xbc\TcaXRRdaaT]cfWTaT^QYTRcCh_T, K24;;38B:K *
g\[ T[
\hg\[^dc,OR^\\P]SO*
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
\h_PabTa,G<;))?PabTa -]Tf7P]S[Tab,-j
BcPac,-KWP]S[TNbcPac
4]S,-KWP]S[TNT]S
2WPa,-KRWPaPRcTa3PcP
l*
...rest of the script skipped...
!S YOU CAN SEE THE %XADATA #ELL 'RID #ONTROL PLUG
IN IS MERELY A BUNCH OF 0ERL SCRIPTS WHICH THEN LOG
INTO TARGET CELLS VIA 33( AND RUN REGULAR #ELL#,) COMMANDS WITH 8-, OUTPUT MODE 4HE OUTPUT IS THEN
PARSED AND RETURNED TO THE 'RID #ONTROL AGENT
.OW THAT WE KNOW WHICH ACTUAL COMMAND THE 'RID #ONTROL PLUG
IN USES FOR PULLING DATA WE CAN
RUN IT OURSELVES AND SEE WHICH DATA IT INCLUDES .OTE THAT THERES AN ADDITIONAL FILTER CONDITION SHOWING
ONLY ONE METRIC FOR BREVITY
2T[[2;8-;
;8BC<4CA822DAA4=CF74A4^QYTRcCh_T,24;;38B:0=3]P\T,23N8>NC<NANB<NA@
23N8>NC<NANB<NA@23N NRT[[ "!$"( db aT`dTbc
23N8>NC<NANB<NA@23N NRT[[ " %""db aT`dTbc
23N8>NC<NANB<NA@23N !NRT[[ "% %&db aT`dTbc
23N8>NC<NANB<NA@23N "NRT[[ " # !db aT`dTbc
23N8>NC<NANB<NA@23N #NRT[[ "! # db aT`dTbc
23N8>NC<NANB<NA@23N $NRT[[ "!'& db aT`dTbc
23N8>NC<NANB<NA@23N %NRT[[ "" #' db aT`dTbc
23N8>NC<NANB<NA@23N &NRT[[ "!!&( db aT`dTbc
23N8>NC<NANB<NA@23N 'NRT[[ "!" "db aT`dTbc
23N8>NC<NANB<NA@23N (NRT[[ " ! (db aT`dTbc
23N8>NC<NANB<NA@23N NRT[[ "! db aT`dTbc
23N8>NC<NANB<NA@23N NRT[[ " $ $%db aT`dTbc
23N8>NC<NANB<NA@553N NRT[[ "#%#db aT`dTbc
23N8>NC<NANB<NA@553N NRT[[ "#& db aT`dTbc
23N8>NC<NANB<NA@553N !NRT[[ "#&%db aT`dTbc
23N8>NC<NANB<NA@553N "NRT[[ "#&"db aT`dTbc
23N8>NC<NANB<NA@553N #NRT[[ "#$ db aT`dTbc
23N8>NC<NANB<NA@553N $NRT[[ "#%&db aT`dTbc
23N8>NC<NANB<NA@553N %NRT[[ "#%(db aT`dTbc
23N8>NC<NANB<NA@553N &NRT[[ "#&#db aT`dTbc
23N8>NC<NANB<NA@553N 'NRT[[ "#$%db aT`dTbc
23N8>NC<NANB<NA@553N (NRT[[ "#$(db aT`dTbc
23N8>NC<NANB<NA@553N NRT[[ "#% db aT`dTbc
23N8>NC<NANB<NA@553N NRT[[ "#$$db aT`dTbc
23N8>NC<NANB<NA@553N !NRT[[ "#%&db aT`dTbc
23N8>NC<NANB<NA@553N "NRT[[ "#' db aT`dTbc
23N8>NC<NANB<NA@553N #NRT[[ "#'"db aT`dTbc
23N8>NC<NANB<NA@553N $NRT[[ "#& db aT`dTbc
4HE GRID DISK NAMES STARTING WITH 53N INDICATE THAT THESE LINES ARE REPORTED FOR FLASH DISKS EACH CELL HAS
¾ '" FLASH CARDS BUT EACH FLASH CARD PRESENTS FOUR FLASH DISKS TO THE OPERATING SYSTEM TOTALING FLASH
DISKS PER CELL 3O USING THIS EXERCISE WE HAVE FOUND OUT WHETHER THE CELL
DISK METRICS INCLUDE FLASH DISK
METRICS IN ADDITION TO REGULAR DISK DRIVES AND YES THEY DO 4HIS MAKES SOME OF THE CHARTS LIE AS THE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
FLASH READ TIMES THAT ARE AN ORDER OF MAGNITUDE FASTER WILL MAKE THE AVERAGE REGULAR DISK READ TIMES LOOK
BETTER AS WELL )N FACT SOME OF THE CELL DISKS HAVE A PRETTY BAD SMALL READ )/ LATENCY &OR EXAMPLE UP TO
MILLISECONDS PER REQUEST ON AVERAGE WHICH IS MUCH HIGHER THAN THE AVERAGE SEEK ROTATIONAL DATA
TRANSFER LATENCY SHOULD BE FOR A MODERN 20- DISK EVEN IF IT IS BUSY WITH RANDOM )/
REQUESTS 4HIS PROBABLY INDICATES )/ QUEUEING SOMEWHERE LIKELY IN THE /3 )/ QUEUE WHICH YOU CAN
CONFIRM BY LOOKING AT THE PfPXc AND beRc\ COLUMNS IN X^bcPc OUTPUT MORE ABOUT X^bcPc LATER IN THIS
CHAPTER BUT \P]X^bcPc IS YOUR FRIEND
3O IDEALLY YOU SHOULD MONITOR THE FLASH DISKS AND GOOD OLD SPINNING DISK DRIVES SEPARATELY BY EITHER
POST
PROCESSING THE RETURNED DATA OR JUST RUNNING TWO SEPARATE #ELL#,) COMMANDS IN YOUR MONITORING
TOOLS #ELL#,) ALLOWS YOU TO FILTER ITS RESULTS FURTHER USING A REGULAR EXPRESSION AS SHOWN HERE
2T[[2;8-;8BC<4CA822DAA4=CF74A4^QYTRcCh_T,24;;38B:
0=3]P\T,23N8>NC<NANB<NA@0=3\TcaXR>QYTRc=P\T;8:453N *
23N8>NC<NANB<NA@53N NRT[[ "#%#db aT`dTbc
23N8>NC<NANB<NA@53N NRT[[ "#& db aT`dTbc
23N8>NC<NANB<NA@53N !NRT[[ "#&%db aT`dTbc
23N8>NC<NANB<NA@53N "NRT[[ "#&"db aT`dTbc
23N8>NC<NANB<NA@53N #NRT[[ "#$ db aT`dTbc
23N8>NC<NANB<NA@53N $NRT[[ "#%&db aT`dTbc
23N8>NC<NANB<NA@53N %NRT[[ "#%(db aT`dTbc
23N8>NC<NANB<NA@53N &NRT[[ "#&#db aT`dTbc
23N8>NC<NANB<NA@53N 'NRT[[ "#$%db aT`dTbc
23N8>NC<NANB<NA@53N (NRT[[ "#$(db aT`dTbc
23N8>NC<NANB<NA@53N NRT[[ "#% db aT`dTbc
23N8>NC<NANB<NA@53N NRT[[ "#$$db aT`dTbc
23N8>NC<NANB<NA@53N !NRT[[ "#%&db aT`dTbc
23N8>NC<NANB<NA@53N "NRT[[ "#' db aT`dTbc
23N8>NC<NANB<NA@53N #NRT[[ "#'"db aT`dTbc
23N8>NC<NANB<NA@53N $NRT[[ "#& db aT`dTbc
.OW WE SEE ONLY THE FLASH DRIVES 9OU CAN READ MORE ABOUT THE #ELL#,) USAGE AND SYNTAX EXAMPLES IN
!PPENDIX ! OF THIS BOOK AND ALSO IN THE OFFICIAL DOCUMENTATION Oracle Exadata Storage Server Software
User’s Guide
7HICH #ELL -ETRICS TO 5SE
7HICH CELL METRICS TO USE IS AN IMPORTANT QUESTION /RACLE PROVIDES A HUGE VARIETY OF DIFFERENT METRICS
FROM THE DATABASE LEVEL THE HOST SERVER LEVEL AND THE %XADATA CELL LEVEL )T IS VERY EASY TO GET LOST IN THEM
OR EVEN WORSE GET MISLED AND TROUBLESHOOT A WRONG OR NON
EXISTENT PROBLEM )N PERFORMANCE
TROUBLESHOOTING WE SHOULD START FROM WHAT MATTERSUSER RESPONSE TIMEAND DRILL DOWN FROM THERE AS
EXPLAINED IN THE PREVIOUS SECTIONS OF THIS CHAPTER "UT GENERAL EFFICIENCY MONITORING UTILIZATION
OVERVIEW AND CAPACITY PLANNING ARE SOMETIMES GOOD REASONS FOR EXAMINING CELL
LEVEL AGGREGATED
METRICS
4HIS MAY COME AS A SURPRISE IF YOU HAVE EXPECTED MORE BUT THERE IS NO LONG LIST OF SECRET AND SPECIAL
%XADATA METRICS YOU SHOULD MONITOR IN YOUR CELLS ! CELL IS JUST A SERVER WITH 2!- #05S AND DISKS
SENDING DATA OUT OVER A NETWORK LINK 3O THE FUNDAMENTAL METRICS YOU WOULD WANT TO MONITOR AT THE CELL
LEVEL ARE THE SAME YOUD USE IN ANY SERVER #05 UTILIZATION SYSTEM LOAD MEMORY UTILIZATION AND DISK
UTILIZATION AND )/ THROUGHPUT AND LATENCY 4HE MOST IMPORTANT METRICS ARE AVAILABLE IN THE 'RID
#ONTROL 3TORAGE #ELL MONITORING PLUG
IN AND THE REST CAN BE EXTRACTED FROM /3 7ATCHER LOGFILES 4HE
ADVANCED METRICS COME INTO PLAY ONLY WHEN DEALING WITH BUGS OR SOME RARE VERY SPECIFIC PERFORMANCE
ISSUES SO THEY SHOULDNT REALLY BE ACTIVELY MONITORED UNLESS THERE IS A PROBLEM
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
/NE TOPIC DESERVES SOME ELABORATION THE #05 USAGE MONITORING BOTH IN THE DATABASE AND CELLS
4HE DATA WAREHOUSES AND REPORTING SYSTEMS UNLIKE /,40 DATABASES DONT USUALLY REQUIRE VERY QUICK
RESPONSE TIMES FOR USER QUERIES /F COURSE THE FASTER A QUERY COMPLETES THE BETTER BUT IN A $7 PEOPLE
DONT REALLY NOTICE IF A QUERY RAN IN SECONDS INSTEAD OF SECONDS "UT A TYPICAL /,40 USER WOULD
DEFINITELY NOTICE IF THEIR
SECOND QUERY TOOK SECONDS OCCASIONALLY 4HATS ONE OF THE REASONS WHY IN
/,40 SERVERS YOU WOULD NOT WANT TO CONSTANTLY RUN AT #05 UTILIZATION 9OU CANNOT DO THAT AND ALSO
MAINTAIN STABLE PERFORMANCE )N AN /,40 SYSTEM YOU MUST LEAVE SOME HEADROOM )N $7 SERVERS
HOWEVER THE SMALL FLUCTUATIONS IN PERFORMANCE WOULD NOT BE NOTICED AND YOU CAN AFFORD TO RUN AT
OF #05 UTILIZATION IN ORDER TO GET THE MOST OUT OF YOUR INVESTMENT
(OWEVER %XADATA COMPLICATES THINGS )N ADDITION TO HAVING MULTIPLE DATABASE NODES YOU ALSO HAVE
ANOTHER WHOLE LAYER OF SERVERS THE CELLS 4HINGS GET INTERESTING ESPECIALLY WHEN RUNNING 3MART 3CANS
WITH HIGH PARALLELISM AGAINST %(## TABLES 4HATS BECAUSE OFFLOADED DECOMPRESSION REQUIRES A LOT OF
#05 CYCLES IN THE CELLS 4HUS IT IS POSSIBLE THAT FOR SOME WORKLOADS YOUR CELLS #05S WILL BE BUSY
AND UNABLE TO FEED DATA BACK TO THE DATABASE LAYER FAST ENOUGH 4HE DATABASE LAYER #05S MAY BE HALF
IDLE WHILE CELLS COULD REALLY USE SOME EXTRA #05 CAPACITY
4HE RISK FROM CELL UTILIZATION REACHING IS THE REASON /RACLE MADE RT[[bae ABLE TO SKIP OFFLOAD
PROCESSING FOR SOME DATABLOCKS AND PASS THESE BLOCKS STRAIGHT BACK TO THE DATABASE STARTING IN RT[[bae
4HE CELL CHECKS WHETHER ITS #05 UTILIZATION IS OVER AND WHETHER THE DATABASE #05
UTILIZATION ITS SENT IN BASED ON RESOURCE MANAGER STATS FROM THE DATABASE IS LOWER THAN THAT !ND IF SO
SOME BLOCKS ARE NOT PROCESSED IN THE CELLS BUT PASSED THROUGH TO THE DATABASE DIRECTLY 4HE DATABASE
THEN WILL DECRYPT IF NEEDED AND DECOMPRESS THE BLOCKS AND PERFORM PROJECTION AND FILTERING IN THE
DATABASE LAYER 4HIS ALLOWS YOU TO FULLY UTILIZE ALL YOUR #05 CAPACITY IN BOTH LAYERS OF THE %XADATA CLUSTER
(OWEVER THIS AUTOMATICALLY MEANS THAT IF SOME BLOCKS ARE SUDDENLY PROCESSED IN THE DATABASE LAYER
INSTEAD OF BEING OFFLOADED TO CELLS THEN YOU MAY SEE UNEXPECTED #05 UTILIZATION SPIKES IN THE DATABASE
LAYER WHEN CELLS ARE TOO BUSY 4HIS SHOULDNT BE A PROBLEM ON MOST $7 SYSTEMS ESPECIALLY WITH
PROPERLY CONFIGURED RESOURCE MANAGER BUT YOU WOULD WANT TO WATCH OUT FOR THIS WHEN RUNNING /,40 OR
OTHER LOW
LATENCY SYSTEMS ON %XADATA
!S USUAL /RACLE PROVIDES GOOD METRICS ABOUT THIS PASS
THROUGH FEATURE 7HENEVER THE OFFLOAD
PROCESSING IS SKIPPED FOR SOME BLOCKS DURING A 3MART 3CAN AND THESE BLOCKS ARE SENT BACK TO THE
DATABASE LAYER FOR PROCESSING THE STATISTIC hCELL PHYSICAL )/ BYTES PUSHED BACK DUE TO EXCESSIVE #05 ON
CELLv GETS INCREMENTED IN EB4BBC0CEBHBBC0C AND !72 REPORTS 2EAD MORE ABOUT THIS FEATURE AND
STATISTIC IN #HAPTER
-ONITORING %XADATA 3TORAGE #ELL /3
LEVEL -ETRICS
/RACLE $ATABASE AND THE RT[[bae SOFTWARE DO A GOOD JOB OF GATHERING VARIOUS PERFORMANCE METRICS BUT
THERE ARE STILL CASES WHERE YOU WOULD WANT TO USE AN /3 TOOL INSTEAD /NE OF THE REASONS IS THAT USUALLY
THE E VIEWS IN /RACLE TELL YOU WHAT /RACLE thinks ITS DOING BUT THIS MAY NOT NECESSARILY BE WHATS REALLY
HAPPENING IF YOU HIT A BUG OR SOME OTHER LIMITATION OF /RACLES BUILT
IN INSTRUMENTATION /NE OF THE
LIMITATIONS IS LOW
LEVEL )/ MEASUREMENT
Monitoring the Storage Cell Server’s I/O Metrics with iostat
"OTH /RACLE $ATABASE AND THE STORAGE CELLS DO MEASURE THE )/ #OMPLETION 4IME IN OTHER WORDS THE
RESPONSE TIME OF )/S 7ITH SYNCHRONOUS )/ OPERATIONS /RACLES )/ WAIT TIME IS MERELY THE SYSTEM CALL
LIKE _aTPS COMPLETION TIME 7ITH ASYNCHRONOUS )/ THE RESPONSE TIME MEASUREMENT IS TRICKIER AS AN
ASYNCHRONOUS )/ SUBMIT SYSTEM CALL WILL NOT BLOCK AND WAIT IT WILL RETURN IMMEDIATELY IN MICROSECONDS
AND SOME )/ REAPING SYSTEM CALL WILL BE EXECUTED LATER WHICH WILL MARK THE )/ OPERATION AS COMPLETE
4HE %XADATA STORAGE CELLS KEEP TRACK OF EACH ASYNCHRONOUS )/ REQUEST FOR EXAMPLE WHEN IT WAS
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
SUBMITTED AND ONCE THE )/ IS SUCCESSFULLY REAPED THEY WILL CHECK THE REAPING TIMESTAMP AGAINST THAT
)/S SUBMIT TIMESTAMP AND KNOW ITS DURATION
2EGARDLESS OF THIS EXTRA RT[[bae LEVEL )/ MONITORING THE CELLS STILL DONT BREAK THE )/ RESPONSE
TIME INTO THE TWO IMPORTANT COMPONENTS HOW MUCH TIME THIS )/ REQUEST SPENT USELESSLY waiting IN THE
/3 )/ QUEUE BEFORE IT WAS EVEN SENT OUT TO THE 3#3) DEVICE AND HOW LONG THE ACTUAL HARDWARE service
time WAS ONCE THE )/ REQUEST WAS SENT OUT TO THE HARDWARE #OMPARING THE )/ SERVICE TIME TO WAITING
TIME GIVES AN IMPORTANT CLUE ABOUT WHETHER THE STORAGE HARDWARE ITSELF RESPONDS SLOWLY IN WHICH CASE
THE SERVICE TIME IS HIGHER THAN YOU WOULD EXPECT FROM A MODERN DISK DRIVE
.OTE 3O WHAT IS THIS NORMAL )/ SERVICE TIME TO EXPECT FROM DISKS 4HAT DEPENDS ON WHICH DISKS YOU HAVE
WHAT THEIR SEEK LATENCY IS HOW FAR THE DISK READWRITE HEAD HAS TO SEEK FROM ITS PREVIOUS LOCATION WHAT THE
ROTATIONAL LATENCY OF THE DISK IS HOW FAST IT SPINS 20- AND HOW FAST IT CAN TRANSFER THE BITS JUST READ OVER THE
WIRES OR FIBER 4HERE IS NO MAGIC IN CALCULATING WHAT A GOOD SERVICE TIME SHOULD BE FOR A DISK ITS ALL BASED ON THE
PHYSICAL CAPABILITIES OF THE DISK AND THESE CAPABILITIES ARE DOCUMENTED IN THE VENDORS DISK SPECS 9OU CAN READ
MORE ABOUT THE VARIOUS LATENCIES INVOLVED FROM 7IKIPEDIA
Wcc_) T] fXZX_TSXP ^aV fXZX 7PaSNSXbZNSaXeT0RRTbbNcX\T
/F COURSE THERE MAY BE OTHER OVERHEAD LIKE 3!. STORAGE NETWORK ROUNDTRIP TIMES AND ANY EXTRA
LATENCY CAUSED BY STORAGE CONTROLLERS SWITCHES AND SO ON "UT REMEMBER %XADATA IS NOT CONNECTED TO
3!. STORAGE THE DISKS ARE ATTACHED DIRECTLY TO THE STORAGE CELLS %ACH STORAGE CELL HAS A LITTLE ,3)
-EGA2AID CONTROLLER IN IT AND THE DISKS ARE ATTACHED TO THAT CONTROLLER 4HERE ARE NO COMPLEX NETWORK
COMPONENTS BETWEEN THE DISKS AND STORAGE CELLS WHICH MIGHT DRIVE UP LATENCY OR FAIL -ORE IMPORTANTLY
THERE ARE NO hOTHERv DATABASES OR APPLICATIONS CONNECTED TO THE STORAGE WHICH YOU HAVE NO CONTROL OVER
BUT WHICH COULD SUDDENLY START SATURATING THE DISKS WITHOUT A WARNING !LL YOU WOULD SEE IS THAT THE DISK
)/ SERVICE TIMES SUDDENLY GO UP AS THE STORAGE ARRAY GETS OVERLOADED
!NYWAY A SMALL )/ REQUEST UP TO +" AGAINST A MODERN 20- DISK DRIVE SHOULD IDEALLY
HAVE AVERAGE SERVICE TIME OF n MILLISECONDS PER )/ REQUEST 4HIS COMES FROM A MS AVERAGE SEEK
LATENCY MS AVERAGE ROTATIONAL LATENCY PLUS MS FOR TRANSFERRING THE K" OVER 3!3 CHANNELS )N
THEORY THIS AVERAGE SEEK LATENCY IS A PESSIMISTIC FIGURE ASSUMING THAT THE WHOLE DISK IS FULL AND THE DISK
READWRITE HEADS HAVE TO DO RANDOM SEEKS ACROSS TRACKS FROM END TO END OF THE DISK CYLINDERS "UT YOU
PROBABLY HAVE CONFIGURED YOUR DISKS INTO HOTTER AND COLDER REGIONS THE 2%#/ DISK GROUP FOR EXAMPLE
AND THE COLDER REGIONS ARE VISITED MUCH LESS FREQUENTLY DRIVING THE REAL AVERAGE SEEK TIMES SHORTER 3O
WHILE YOU MAY GET THESE LOW nMS SERVICE TIMES WHEN YOUR DISK HEADS DONT HAVE TO SEEK TOO FAR IN
PRACTICE MS AVERAGE )/ SERVICE TIMES ARE COMPLETELY /+ ONCE YOU RUN REAL PRODUCTION WORKLOADS
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
+EVIN 3AYS )M OFTEN SURPRISED AT THE LACK OF FOCUS PAID TO DATA PLACEMENT ON PHYSICAL DISKS 4HE AUTHORS ARE
CORRECT TO POINT OUT THAT SO
CALLED SHORT
STROKED )/ REQUESTS HAVE SOME IF NOT ALL SEEK TIME REMOVED FROM THE
SERVICE TIME OF THE REQUEST (OWEVER )D LIKE TO ELABORATE ON THIS POINT FURTHER 7HEN DISCUSSING %XADATA WE OFTEN
CONCENTRATE ON DATA CAPACITY OF ENORMOUS PROPORTION (OWEVER IT IS SELDOM THE CASE THAT ALL DATA IS ACTIVE IN ALL
QUERIES -OST DATA WAREHOUSES HAVE AN ACTIVE SUBSET OF THE TOTAL DATABASE /NE CAN EASILY ENVISION THE ACTIVE
PORTION OF A 4" DATA WAREHOUSE BEING AS SMALL AS 4" *UST ONE PERCENT (OW CAN THIS BE !LWAYS REMEMBER
THAT /RACLE $ATABASE COMBINED WITH %XADATA OFFERS MANY hLINES OF DEFENSEv IN FRONT OF THE DREADED PHYSICAL )/
"EFORE PHYSICAL )/ THERE IS A RESULTS CACHE MISS 3'! BUFFER CACHE MISS YES IT HAS PURPOSE EVEN WHEN %XADATA IS
INVOLVED PARTITION ELIMINATION AND FINALLY A STORAGE INDEX MISS /NCE THROUGH THESE hLINES OF DEFENSE v THERE IS A
PHYSICAL )/ FOR DATA THAT IS POTENTIALLY COMPRESSED WITH A COMPRESSION RATIO OF SAY 4O THAT END THE 4"
ACTIVE PORTION IS ACTUALLY 4"A SIZABLE AMOUNT OF DATA 3EEK TIMES ARE THE LIONS SHARE OF PHYSICAL )/ SERVICE
TIME )N FACT FROM A CURRENT COST PERSPECTIVE THE OPTIMAL DATA WAREHOUSE ARCHITECTURE WOULD CONSIST OF SEEK
FREE
ROTATIONAL HARD DISK DRIVES 7HILE THAT MAY SEEM ABSURD ALLOW ME TO POINT OUT THAT 4" IS EXACTLY PERCENT OF THE
AGGREGATE SURFACE AREA OF THE '" 3!3 DRIVES OFFERED IN A FULL RACK 8 MODEL WITH HIGH
PERFORMANCE
DRIVES ,ONG SEEKS ARE NOT A FIXED COST IN MODERN DATA WAREHOUSESESPECIALLY WHEN THE ACTIVE PORTION IS SUCH A
SMALL PERCENTAGE OF THE WHOLE )F THE ACTIVE PORTION OF THE DATA WAREHOUSE REMAINS A FIXED PERCENTAGE YET SERVICE
TIMES INCREASE OVER TIME THE PROBLEM IS LIKELY FRAGMENTATION $ATA SCATTERED OVER THE ENTIRETY OF ROUND SPINNING
MAGNETIC MEDIA IS NOT OPTIMAL REGARDLESS OF WHETHER /RACLE $ATABASE IS CONNECTED TO CONVENTIONAL STORAGE OR
%XADATA %VEN WITH %XADATA THE FUNDAMENTALS STILL APPLY
4HE WHOLE POINT OF THIS EXPLANATION SO FAR IS REALLY THAT IT is POSSIBLE TO KNOW WHAT THE IDEAL DISK
service TIMES SHOULD BE FOR %XADATA CELLS DISK DRIVES AND IF THE SERVICE TIMES ARE CONSTANTLY MUCH HIGHER
THAN THAT THEN THERE IS SOME PROBLEM WITH THE STORAGE HARDWARE DISKS OR CONTROLLERS 7ITH 3!.
STORAGE HIGH /3
LEVEL SERVICE TIMES COULD ALSO MEAN THAT THERE IS SOME QUEUING GOING ON INSIDE THE 3!.
NETWORK OR THE STORAGE ARRAY FOR EXAMPLE IF A THOUSAND OTHER SERVERS ARE HAMMERING THE SAME STORAGE
ARRAY WITH )/ "UT AS SAID ABOVE %XADATA STORAGE CELLS HAVE DEDICATED STORAGE IN THEM ONLY THE CURRENT
CELL /3 CAN ACCESS THIS STORAGE AND ALL )/S ARE VISIBLE IN X^bcPc
3O WHAT IF THE SERVICE TIME IS /+ BUT /RACLE RT[[bae STILL SEES BAD )/ PERFORMANCE 7ELL THIS MAY
MEAN THERE IS STILL queuing GOING ON INSIDE THE CELL ,INUX SERVERSAND X^bcPc CAN SHOW THIS INFORMATION
TOO 9OU CAN HAVE ONLY A LIMITED NUMBER OF OUTSTANDING )/S IN THE hON
THE
FLYv STATE AGAINST YOUR
STORAGE CONTROLLER ,5.S 4HE STORAGE CONTROLLER NEEDS TO KEEP TRACK OF EACH OUTSTANDING )/ FOR
EXAMPLE IT HAS TO REMEMBER WHERE IN THE HOST 2!- TO WRITE THE BLOCK ONCE IT IS READ FROM DISK AND
ARRIVES AT THE CONTROLLER AND THESE )/ SLOTS ARE LIMITED 3O THE ,INUX KERNEL DOES NOT ALLOW SENDING OUT
MORE )/S THAN THE STORAGE CONTROLLER CAN HANDLE OTHERWISE A 3#3) RESET WOULD OCCUR 4HESE THROTTLED
)/S WILL HAVE TO WAIT IN THE /3 DISK DEVICE )/ QUEUE AND THEY ARE uselessly WAITING THERE THEY ARENT
EVEN SENT OUT TO THE STORAGE CONTROLLER YET /NLY WHEN SOME PREVIOUSLY OUTSTANDING )/ OPERATION
COMPLETES WILL THE FIRST ITEM IN THE QUEUE BE SENT TO THE DISKS ASSUMING THAT THE )/ LATENCY DEADLINE
HASNT BEEN REACHED FOR SOME REQUEST RT[[bae USES THE ,INUX hDEADLINEv )/ SCHEDULER
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
)F YOU DO HAVE )/ WAITING QUEUEING TIME GOING ON THEN THE PfPXc AVERAGE )/ COMPLETION TIME
COLUMN IN X^bcPc WILL BE SIGNIFICANTLY HIGHER THAN THE beRc\ ESTIMATED )/ SERVICE TIME COLUMN .OTE
THAT THE NAME hAWAITv IS SOMEWHAT MISLEADING AS IT DOES NOT SHOW ONLY THE WAIT QUEUING TIME IT
SHOWS THE TOTAL WAIT PLUS SERVICE TIME 3IMILARLY THE PeV`d bi COLUMN DOES NOT SHOW JUST THE AVERAGE )/
WAIT QUEUE LENGTH BUT RATHER THE AVERAGE total NUMBER OF NOT COMPLETED )/S IN THE )/ REQUEST QUEUE
REGARDLESS OF WHETHER THEY HAVE ALREADY BEEN SENT OUT TO STORAGE HARDWARE ARE ALREADY BEING SERVICED
OR STILL WAITING IN THE )/ QUEUE NOT YET SERVICED
,INUX X^bcPc SHOWS STATISTICS FOR THE DISK PARTITION AND THE CELL SOFTWARE 2!)$ DEVICE USED FOR CELL
/3 PARTITION MIRRORING 7E CAN FILTER OUT THOSE LINES SO THAT ONLY THE PHYSICAL DISK STATS WOULD BE LISTED
&OR EXAMPLE
XX^bcPc g\$kTVaT_ ebS J (LkM\S
;X]dg! % ' (# " # T[$T]ZRT[ " T]ZXcTR R^\ $ !! !
PeV R_d)dbTa]XRTbhbcT\X^fPXcbcTP[XS[T
&" &$ (& $
3TeXRT)a bf ba<1 bf<1 bPeVa` biPeV`d biPPfPXcbbeRc\dcX[
bSP ! $%& ""( %" !% ' ## "(& "" ((& (#
bSQ' %#' &$" (# !##( (# '&#( (#" %$ #(
bSR$ (( &$" '& # %! "# %"' '%$ (# !
bSS$ #! '(" &$ $ $ #% '& $ '#
bST '#" !' % ! #'' " (#%% % # &'% &#
bSU %! '" $ $%! ' " $ (! "!
bSV% %!! !% !# $ $ $ $" #($& "!$ (($ &
bSW# $%! $ " '% $ %$ ! #!$( & $ %"" (&
bSX $" ## $' &%' (" %%#% "(# # % "
bSY# "( %%" '& ! "$ "& $ '! $$% $ " (#
bSZ '&" ($ ( ! #%" %% ''$$ (## %#& ""
bS[ % % $ # ( '% $(" % &%"# %'" ! & #
bS\ !' #& (" $( "' #
bS]! '& " " !# #! $$ "#
bS^! '# " " !# %" $& "#
bS_! '! " " !# #( $' "#
bS`! &( #$ " !$ $ $' ""
bSa! '( !' " !# ## % "#
bSb! (# "! " !# "$ $% "#
bSc! '$ " " !# #( $# "#
bSd! '! !( " !# %# $% "#
bSe! '% #( " !$ $$ ""
bSf! (" $' " !$ ! $" ""
bSg! '& "$ " !# "& $& ""
bSh! '# % " !$ !' $! "!
bSi! ' !( " !# %" $& "#
bSPP! '% "! " !# # % "#
bSPQ! '( "" " !# "& $' "#
bSPR! ( !( " !# "% $' "#
7AIT A MINUTE 3HOULDNT WE SEE FEWER DISKS AS EACH CELL HAS ONLY HARD DRIVES IN IT 7E SEE SO MANY
DISKS BECAUSE IN ADDITION TO THE HARD DISKS CELLS ALSO HAVE THEIR FLASH CARDS PRESENTED AS SEPARATE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
DISKS &INALLY THERES ALSO ONE 53" DISK 9OU CAN USE THE [bbRbX COMMAND IN THE CELL TO SEE ALL DISKS
DETECTED BY THE /3
[
[bbRbX
J )!) ) LSXbZ;;B8<A(!% 'X! ! STe bSP
J )!) ) LSXbZ;B8<A(!% 'X! ! STe bSQ
J )!)!) LSXbZ;B8<A(!% 'X! ! STe bSR
J )!)") LSXbZ;B8<A(!% 'X! ! STe bSS
J )!)#) LSXbZ;B8<A(!% 'X! ! STe bST
J )!)$) LSXbZ;B8<A(!% 'X! ! STe bSU
J )!)%) LSXbZ;B8<A(!% 'X! ! STe bSV
J )!)&) LSXbZ;B8<A(!% 'X! ! STe bSW
J )!)') LSXbZ;B8<A(!% 'X! ! STe bSX
J )!)() LSXbZ;B8<A(!% 'X! ! STe bSY
J )!) ) LSXbZ;B8<A(!% 'X! ! STe bSZ
J )!) ) LSXbZ;B8<A(!% 'X! ! STe bS[
J ) ) ) LSXbZDD]XVT]?B0# STe bS\
J') ) ) LSXbZ00C0<<0AE4;;B3''B0 !3! H STe bS]
J') ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bS^
J') )!) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bS_
J') )") LSXbZ0C0<0AE4;;B3''B0 !3! H STe bS`
J() ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSa
J() ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSb
J() )!) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSc
J() )") LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSd
J ) ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSe
J ) ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSf
J ) )!) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSg
J ) )") LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSh
J ) ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSi
J ) ) ) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSPP
J ) )!) LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSPQ
J ) )") LSXbZ0C0<0AE4;;B3''B0 !3! H STe bSPR
4HE ,3) DISKS HERE ARE THE HARD DISKS PRESENTED TO THE HOST BY THE ,3) 3#3) 2!)$ CONTROLLER FROM DEVICES
bSP TO bS[ THE 5NIGEN 03! IS THE 53" DISK AND THE -!26%,, !4! DISKS ARE THE FLASH CARDS )F YOU
COMPARE THE AVERAGE )/ SERVICE TIMES beRc\ IN X^bcPc YOULL SEE THAT THE DEVICES BELONGING TO FLASH
CARDS HAVE A SERVICE TIME AN ORDER OF MAGNITUDE LOWER THAN HARD DISK DEVICES 3O IF YOU WANT TO MONITOR
ONLY THE HARD
DISK DEVICES YOU CAN FILTER X^bcPc OUTPUT THIS WAY
X^bcPc g\$kTVaT_3TeXRTkMbSJP [L
X
3TeXRT)a bf ba<1 bf<1 bPeVa` biPeV`d biPfPXcbeRc\dcX[
bSP ($ ! !!' %! "''#& $( % %% !%" &%( (
bSQ( &( " %!# # "%#' $" "#' !" " & &#
bSR& %% %## & !%&(% %"#% $" %$$
bSS% %& '(" (# !&&%$ ! '#% "'" &#$ !!
bST #$' ! ' ( " ! '( '%#% !" &(& &
bSU % &( $ #$$ "' ! '# " "$
bSV& $% ("$ ( !' ## # $""& &%# !!$ (#
bSW$ % & #'# % !''"$ !% #("& &'" '$$ #
bSX ! '(' # %" " $&( ( & "" ''" $ & "#
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
bSY$ #"% $$# & !"( % '( $%#% #'# $# '$
bSZ "#' '%( &# "# !! & '"# '" ' & &
bS[ % # !"' ' !%&&" (% &$!' !&! '"& $"
Advanced Cell Monitoring With OS Watcher
7HILE THE DATABASE LEVEL WAIT PROFILE AND 31, -ONITORING SHOULD BE USED AS A STARTING POINT FOR
PERFORMANCE MONITORING AND TROUBLESHOOTING SOMETIMES THESE APPROACHES ARE NOT ENOUGH AND YOU
WANT TO DRILL DEEPER 4HIS SECTION EXPLAINS ONE ADDITIONAL DATA SOURCE FOR THAT %XADATA STORAGE CELLS COME
WITH A PREINSTALLED TOOL CALLED /3 7ATCHER LOCATED IN THE ^_c ^aPR[T ^bfPcRWTa ^bf DIRECTORY 4HIS TOOL
IS JUST A SET OF SHELL SCRIPTS WHICH THEN RUN STANDARD /3 TOOLS LIKE e\bcPc X^bcPc AND ]TcbcPc TO COLLECT
THEIR DATA /3 7ATCHERS BENEFIT IS THAT IT RUNS AT /3 LEVEL NOT INSIDE A DATABASE SO IT IS NOT AFFECTED BY
DATABASE HANGS AND PERFORMANCE ISSUES OR CASES WHERE THE DATABASES E VIEWS DONT SHOW THE TRUTH OR
HAVE ENOUGH DETAIL !DDITIONALLY THE HIGH FREQUENCY /3 7ATCHER COLLECTORS SAMPLE DATA EVERY SECOND OR
FEW SECONDS ALLOWING IT TO DETECT SHORT hHICCUPSv OR BURSTS OF ACTIVITY
9OU CAN CHECK WHETHER THE /3 7ATCHER IS RUNNING BY SIMPLY SEARCHING FOR ANY PROCESSES WITH
h/37v IN THEIR NAME BE SURE TO SEARCH FOR BOTH UPPER AND LOWER CASE
_
_VaT_ [U>BFk^bf
$#" QX] ZbW ^bfbdQ bW7XVW5aT` 4gPSPcPNRT[[baebcPc bW
!%# ! QX] ZbW >BFPcRWTa bW $ %'QiX_!
!''!& QX] ZbW >BFPcRWTa5< bW %'
!''#% QX] ZbW ^bfbdQ bW7XVW5aT` 4gPSPcPNe\bcPc bW
!''#& QX] ZbW ^bfbdQ bW7XVW5aT` 4gPSPcPN\_bcPc bW
!''#' QX] ZbW ^bfbdQ bW7XVW5aT` 4gPSPcPN]TcbcPc bW
!''#( QX] ZbW ^bfbdQ bW7XVW5aT` 4gPSPcPNX^bcPc bW
!''$" QX] ZbW ^bfbdQ bW7XVW5aT` 4gPSPcPNc^_ bW
!''&& QX] ZbW ^bfbdQ bW7XVW5aT` ^_c ^aPR[T ^bfPcRWTa ^bf 4gPSPcPASb8]U^ bW
!'( QX] QPbW ^_c ^aPR[T ^bfPcRWTa ^bf 4gPSPcPASb8]U^ bW7XVW5aT`
!PPARENTLY THE /3 7ATCHER IS RUNNING IN THIS STORAGE CELL AND THERE ARE A NUMBER OF %XADATA SPECIFIC
COLLECTORS ENABLED TOO )F YOU WANT TO SEE THE HIERARCHY OF THE /3 7ATCHER PROCESS DAEMONS THEN ON
,INUX YOU CAN USE EITHER THE _b 7 COMMAND SHOWN IN THE FIRST CHAPTER OR THE _bcaTTCOMMAND AS
SHOWN HERE
_bcaTT P0W[d_O_VaT_>BFPcRWTa bWO
_
>BFPcRWTa bW!%# ! >BFPcRWTa bW $ %'QiX_!
k 4gPSPcP3XPV2^[[!''%% ^_c ^aPR[T RT[[^b 4gPSPcP3XPV2^[[TRc^a bW
kO b[TT_!$& '"
k >BFPcRWTa5< bW!''!& >BFPcRWTa5< bW %'
k ^bfbdQ bW $#" ^bfbdQ bW7XVW5aT` 4gPSPcPNRT[[baebcPc bW
kO 4gPSPcPNRT[[bae $#& 4gPSPcPNRT[[baebcPc bW7XVW5aT`
kk QiX_! $( bcS^dc
kO RT[[baebcPc $( X]cTaeP[,$ R^d]c,&!
k ^bfbdQ bW!''#% ^bfbdQ bW7XVW5aT` 4gPSPcPNe\bcPc bW
kO 4gPSPcPNe\bcPc !''$ 4gPSPcPNe\bcPc bW7XVW5aT`
kk QiX_! ! !% bcS^dc
kO e\bcPc ! !$$&!
k ^bfbdQ bW!''#& ^bfbdQ bW7XVW5aT` 4gPSPcPN\_bcPc bW
kO 4gPSPcPN\_bcPc !''$! 4gPSPcPN\_bcPc bW7XVW5aT`
kk QiX_! !%!& bcS^dc
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
kO \_bcPc !%!%$&!
k ^bfbdQ bW!''#' ^bfbdQ bW7XVW5aT` 4gPSPcPN]TcbcPc bW
kO 4gPSPcPN]TcbcPc!''$# 4gPSPcPN]TcbcPc bW7XVW5aT`
kO b[TT_!$& & $
k ^bfbdQ bW!''#( ^bfbdQ bW7XVW5aT` 4gPSPcPNX^bcPc bW
kO 4gPSPcPNX^bcPc !''$% 4gPSPcPNX^bcPc bW7XVW5aT`
kk QiX_! ! $ bcS^dc
kO X^bcPc ! # c g$&!
k ^bfbdQ bW!''$" ^bfbdQ bW7XVW5aT` 4gPSPcPNc^_ bW
kO 4gPSPcPNc^_ bW!''%# 4gPSPcPNc^_ bW7XVW5aT`
kk QiX_! "#$" bcS^dc
kO c^_ "#$! Q R S$ ]&!
O ^bfbdQ bW!''&& ^bfbdQ bW7XVW5aT` ^_c ^aPR[T ^bfPcRWTa ^bf 4gPSPcPASb8]U^ bW
O 4gPSPcPASb8]U^ !'( ^_c ^aPR[T ^bfPcRWTa ^bf 4gPSPcPASb8]U^ bW7XVW5aT`
O b[TT_!$&
4HE /3 7ATCHER DAEMONS STORE THEIR COLLECTED DATA IN THE ^_c ^aPR[T ^bfPcRWTa ^bf PaRWXeT
DIRECTORY )T DOESNT USE ANY SPECIAL FORMAT FOR STORING THE DATA IT JUST STORES THE TEXT OUTPUT OF THE
STANDARD /3 TOOLS IT RUNS 4HIS MAKES IT EASY TO USE REGULAR TEXT PROCESSING UTILITIES LIKE VaT_ !7+ OR
0ERLPYTHON SCRIPTS TO EXTRACT AND PRESENT THE INFORMATION YOU NEED (ERES AN EXAMPLE
[b [ ^_c ^aPR[T ^bfPcRWTa ^bf PaRWXeT
c^cP[!'
Safga b !a^^cRT[[dbTab!'%&!<Ph!$ #) 4gPSPcP3XPV2^[[TRc
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) 4gPSPcP>BF
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) 4gPSPcPA3B
Safga b !a^^cRT[[dbTab#$ $%<Ph!$ #) ^bfRT[[baebcPc
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bfX^bcPc
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bf\T\X]U^
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bf\_bcPc
Safga b !a^^cRT[[dbTab!#$&%<Ph!$ #) ^bf]TcbcPc
Safga b !a^^cRT[[dbTab# (%<Ph% !)"$^bf_aec]Tc
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bf_b
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bfb[PQX]U^
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bfc^_
Safga b !a^^cRT[[dbTab! #' <Ph!$ #) ^bfe\bcPc
%ACH DIRECTORY CONTAINS ARCHIVE FILES WITH CORRESPONDING COMMAND OUTPUT
^bfX^bcPc
RS ^_c ^aPR[T ^bfPcRWTa ^bf PaRWXeT ^
[b cakcPX[
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ $ SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ % SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ & SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ ' SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ ( SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ ! SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ # SPc Qi!
T]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ " SPc Qi!
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
4HERE IS A SEPARATE BZIPPED FILE SAVED FOR EACH HOUR MAKING IT EASY TO MANUALLY INVESTIGATE PAST
PERFORMANCE DATA OR WRITE A QUICK !7+ VaT_ OR 0ERL SCRIPT WHICH EXTRACTS ONLY THE DATA OF INTEREST 4HE
EXAMPLE BELOW EXTRACTS X^bcPc )/ STATISTICS FROM A BZIPPED FILE FROM !- ON -AY
QQiRPcT]ZRT[ " T]ZXcTR R^\NX^bcPcN $ !$ ( SPc Qi!kWTPS $
iiiFTS<<Ph!$ () )#"23C! BP\_[TX]cTaeP[)$bTRR^]Sb
;X]dg! % ' (# " # T[$T]ZRT[ " T]ZXcTR R^\ $ !$
CX\T) () )#"
PeV R_d)dbTa]XRTbhbcT\X^fPXcbcTP[XS[T
&% &' "& (& (
3TeXRT)a bf babTR bfbTR bPeVa` biPeV`d biPfPXcbeRc\dcX[
bSP ! " ! %% &(%' "$" # " '# " !&$ $" %'( '
bSP #$ # " ! & $&
bSP! ! ! %! %
bSP" ##& # &'&( '%!'%( '$ % $% ! %$ (# &%' '#
bSP# # ' # ' #
bSP$ !$# $$ # "" % !'!& ! #' ! "" !
bSP% !( "' % " ! !& $" (( "
4HE /3 7ATCHER OUTPUT SAMPLES ARE PREFIXED BY A TIMESTAMP LINE WHICH STARTS WITH hZZZv AS YOU SEE
ABOVE 4HIS SHOWS YOU THE EXACT TIME UP TO THE SECOND WHEN THE /3 COMMAND WAS EXECUTED AND WHAT
THE SAMPLE INTERVAL WAS .OTE THAT THIS hZZZv LINE WILL ONLY SHOW YOU WHEN THE MONITORING COMMAND WAS
EXECUTED 3OME COMMANDS ARE EXECUTED ONLY ONCE PER HOUR BUT THEY KEEP COLLECTING AND DUMPING DATA
SAMPLES THROUGHOUT THAT HOUR )N SUCH CASE THERE ARE MONITORING COMMAND
SPECIFIC TIMESTAMPS PRINTED
FOR EACH DATA SAMPLE 4HE FOLLOWING EXAMPLE SHOWS HOW THE RT[[baebcPc COMMAND USES ,,,2daaT]c
CX\T,,, TO INDICATE THE EXACT TIME THE PERFORMANCE DATA BELOW IT IS FROM 9OU CAN USE THIS PREFIX IN YOUR
CUSTOM SCRIPTS (ERES AN EXAMPLE
QiRPcT]ZRT[ " T]ZXcTR R^\NRT[[baebcPcN $ !$ " SPc Qi!kWTPS "
iiiFTS<Ph!$ ") ) 23C! BP\_[TX]cTaeP[)$bTRR^]Sb
,,,2daaT]cCX\T,,,FFTS<Ph!$ ") ) !
,,8]_dc >dc_dcaT[PcTSbcPcb,,
=d\QTa^UWPaSSXbZQ[^RZ8>aTPSaT`dTbcb %&('&###
=d\QTa^UWPaSSXbZQ[^RZ8>faXcTaT`dTbcb $ #(%'!(
7PaSSXbZQ[^RZ8>aTPSb:1 '%"##!&#
7PaSSXbZQ[^RZ8>faXcTb:1 !$!!# % $!
=d\QTa^UU[PbWSXbZQ[^RZ8>aTPSaT`dTbcb %&!" ""!
=d\QTa^UU[PbWSXbZQ[^RZ8>faXcTaT`dTbcb $$!'&(#
5[PbWSXbZQ[^RZ8>aTPSb:1 '& ' %' '
5[PbWSXbZQ[^RZ8>faXcTb:1 "#&""&#!#
=d\QTa^USXbZ8>Taa^ab
=d\QTa^UaTPSbUa^\U[PbWRPRWT $&'( ((!
=d\QTa^UfaXcTbc^U[PbWRPRWT " (&
5[PbWRPRWTaTPSb:1 '& # %
5[PbWRPRWTfaXcTb:1 "#&!"% $%
=d\QTa^UU[PbWRPRWT8>Taa^ab
BXiT^UTeXRcX^]Ua^\U[PbWRPRWT:1
=d\QTa^U^dcbcP]SX]V[PaVTU[PbW8>b
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbSdaX]VY^Q "#
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbQhRWTRZTa
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^ab\Pac8>
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb %%$%'
2daaT]caTPSQ[^RZ8>c^QTXbbdTS:1
C^cP[aTPSQ[^RZ8>c^QTXbbdTS:1 ( %%%!"' !
2daaT]cfaXcTQ[^RZ8>c^QTXbbdTS:1
C^cP[faXcTQ[^RZ8>c^QTXbbdTS:1 !$ $#%(%" &
2daaT]caTPSQ[^RZbX]8>:1
C^cP[aTPSQ[^RZ8>XbbdTS:1 ( %%%!"' !
... a lot of output removed ....
4HIS OUTPUT IS JUST THE FIRST OF EACH
SECOND SAMPLES DUMPED BY RT[[baebcPc EVERY HOUR 4HE FIRST
COLUMN SHOWS THE CELL METRIC NAME AND THE SECOND COLUMN ALL ZEROS SHOWS THE METRIC VALUE IN THE
current metric interval SNAPSHOT "ECAUSE THE COMMAND WAS JUST EXECUTED AND TOOK ITS FIRST SAMPLE IT
SHOWS ZERO FOR EACH METRIC AS THE DELTA COMPUTATION STARTS ONLY AFTER THE SECOND SNAPSHOT IS TAKEN )F
YOU NAVIGATE DOWNWARD IN THE /3 7ATCHER DUMPFILE YOU WILL SEE NONZERO VALUES FOR MANY METRICS
STARTING FROM THE SECOND SNAPSHOT 4HE LAST COLUMN SHOWS THE CUMULATIVE VALUE SINCE THE RT[[bae
PROCESS STARTED FOR EACH METRIC 9OU PROBABLY SHOULD JUST IGNORE THIS CUMULATIVE VALUE AS IT CONTAINS
INFORMATION FROM SINCE THE RT[[bae START WHICH MAY BE MONTHS AGO ,OOKING AT A SINGLE CUMULATIVE
VALUE ACCUMULATING INFORMATION FROM SUCH A LONG TIME WOULD NOT TELL YOU MUCH ABOUT WHATS
HAPPENING RIGHT NOW OR WHAT WAS HAPPENING LAST &RIDAY AT !- -ETRIC DELTAS OVER SHORTER TIME PERIODS
ARE THE WAY TO GO AND THIS IS WHAT YOU SEE IN THE SECOND COLUMN THE CURRENT INTERVAL VALUE AS OF
SAMPLING THE METRICS .OW LETS SAY WE ARE INTERESTED ONLY IN A SINGLE RT[[bae METRIC .UMBER OF ,ATENCY
4HRESHOLD FOR 2EDO ,OG 7RITES, AS THE DATABASE LAYER EXPERIENCED SOME log file parallel write DELAYS AS
SEEN IN E4E4=CN78BC>6A0< FOR EXAMPLE EARLIER ,ETS SAY THIS HAPPENED ON -AY EARLY IN THE
MORNING !S THE /3 7ATCHER LOGFILES ARE NAMED ACCORDING THE DATE AND TIME 99--$$ AND HOUR IN
HOUR FORMAT YOU CAN SEARCH FOR SPECIFIC FILES ONLY BY USING REGULAR 5NIX FILESYSTEM WILDCARDS )N THE
FOLLOWING EXAMPLE WERE SEARCHING FOR ALL HOURS OF -AY AND USING TVaT_ TO LIST BOTH THE hCURRENT
TIMEv AND THE METRIC LINES FROM THE MATCHING FILES
QiRPcT]ZRT[ " T]ZXcTR R^\NRT[[baebcPcN $ !$ SPc Qi!kK
TVaT_,,,2daaT]cCX\T,,,k=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTbkK
WTPS
,,,2daaT]cCX\T,,,FTS<Ph!$ ) ) "!
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb %$%#
,,,2daaT]cCX\T,,,FTS<Ph!$ ) ) '!
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb %$%#
,,,2daaT]cCX\T,,,FTS<Ph!$ ) ) "!
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb %$%#
,,,2daaT]cCX\T,,,FTS<Ph!$ ) ) '!
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb %$%#
,,,2daaT]cCX\T,,,FTS<Ph!$ ) )!"!
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb!%$%%
!S ITS HARD TO READ THE ABOVE OUTPUT HERES A LITTLE SCRIPT WHICH FORMATS THE OUTPUT BETTER AND IS MORE
CONVENIENT TO USE
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
QX] QPbW
=P\T)^bfTgcaPRc bW
?da_^bT)4gcaPRcPb_TRXUXR\TcaXRUa^\>BFPcRWTaRT[[baebcPcPaRWXeTb
DbPVT)
^bfTcaPRc bWRT[[bae\TcaXRc^VaT_U^a^bfNPaRWXeTNUX[TbN^UNX]cTaTbc SPc Qi!
4gP\_[T)
^bfTgcaPRc bW=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTbK
RT[[ TgP\_[T R^\NRT[[baebcPcN $ !$ SPc Qi!
<4CA82,
bWXUc
QiRPc `k
TVaT_2daaT]cCX\Tk<4CA82k
PfZ
1468=
j_aX]cU ! b! b! bK]C8<42DAA4=CNE0;2D<D;0C8E4NE0;l
2daaT]c
j_aX]cUbbbb"#$%&l
=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTb
j_aX]cU! S! SK] l
T]S^UbRaX_c
7HILE THIS SCRIPT IS MEANT TO MAKE EXTRACTING THE DESIRED METRIC HISTORY OUT OF THE /3 7ATCHER LOGS EASIER
THE MAIN PURPOSE OF LISTING THE SCRIPT IN THIS BOOK IS TO SHOW HOW EASY EXTRACTING HISTORICAL /3
LEVEL AND
OTHER METRICS CAN BE )NVESTING A DAY INTO LEARNING BASIC !7+ OR 0ERL TEXT PROCESSING WILL LIKELY PAY OFF
AND SAVE TIME WHEN YOU NEED TO DO UNCONVENTIONAL MONITORING OR TROUBLESHOOTING TASKS ,ETS SEE WHAT
IT DOES
^bfTgcaPRc bW=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTbK
T]ZRT[ " T]ZXcTR R^\NRT[[baebcPcN $ !$ SPc Qi!kWTPS
C8<42DAA4=CNE0;2D<D;0C8E4NE0;
FTS<Ph!$ ) ) " %$%#
FTS<Ph!$ ) ) ' %$%#
FTS<Ph!$ ) ) " %$%#
FTS<Ph!$ ) ) ' %$%#
FTS<Ph!$ ) )!"!%$%%
FTS<Ph!$ ) )!' %$%%
FTS<Ph!$ ) )"" %$%%
FTS<Ph!$ ) )"' %$%%
FTS<Ph!$ ) )#" %$%%
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
4HE SCRIPT GOES THROUGH ALL THE FILENAMES WHICH MATCH THE SECOND EXPRESSION ON THE COMMAND LINE AND
VaT_S FOR WHATEVER METRIC IS SPECIFIED AS THE FIRST PARAMETER IN DOUBLE QUOTES )N THIS CASE WE ARE
LOOKING FOR .UMBER OF ,ATENCY 4HRESHOLD 7ARNINGS FOR 2EDO ,OG 7RITES WHICH IS INCREMENTED WHEN )/
OPERATIONS TAGGED AS REDOLOG FILE WRITES TAKE TOO LONG TO COMPLETE IN THE CELL !PPARENTLY THERE WERE TWO
)/ OPERATIONS AROUND THE TIME RANGE BETWEEN AND WHICH HAD RESPONSE TIMES HIGH
ENOUGH TO BE REPORTED
5SING THE PREVIOUS EXAMPLE YOU CAN ALREADY GENERATE FORMATTED DATA GOOD ENOUGH FOR VISUAL
INSPECTION OR LOADING INTO %XCEL OR SOME CHARTING TOOL )F YOU ARE LOOKING ONLY FOR SOME RANDOM PROBLEM
TIMES THEN YOU CAN DO POST
PROCESSING SUCH AS SHOWING JUST THE LINES WHERE THE METRIC VALUE IS NOT
ZERO
^bfTgcaPRc bW=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTbK
T]ZRT[ " T]ZXcTR R^\NRT[[baebcPcN $ SPc Qi!kVVaT_ e
C8<42DAA4=CNE0;2D<D;0C8E4NE0;
BPc<Ph& !) ) !!!
BPc<Ph& %) ) " !
BPc<Ph& ') ) !$ &
BPc<Ph&! )!') $ '
BPc<Ph&!!) ) ( (
BPc<Ph&!!) )!(#!"
BPc<Ph&!!) )"##!&
Bd]<Ph' !) ) " "&
Bd]<Ph' !) ) ('#$
Bd]<Ph' #) ) "##(
Bd]<Ph' %) ) '!
Bd]<Ph' ') ) $%''
<^]<Ph( !) ) % #
<^]<Ph( #) ) !# '
!LTERNATIVELY YOU CAN SORT THE OUTPUT BY THE METRIC VALUE IN THE 2DAA4=CNE0; COLUMN TO SEE THE WORST
PROBLEM TIMES FIRST 4HE NEXT EXAMPLE USES b^ac ]aZ$ AS WERE SORTING THE FIFTH COLUMN COLUMNS n
ARE THE DATE COLUMN COMPONENTS AS FAR AS THE b^ac UTILITY SEES
^bfTgcaPRc bW=d\QTa^U[PcT]RhcWaTbW^[SfPa]X]VbU^aaTS^[^VfaXcTbK
T]ZRT[ " T]ZXcTR R^\NRT[[baebcPcN $ SPc Qi!kK
VaT_ e kbb^ac ]aZ$kWTPS
BPc<Ph # %) ) "" %"
5aX<Ph " ) ) (!$#"!
CdT<Ph !) ) (!! &#
BPc<Ph # ) ) !!$#(
FTS<Ph !) ) ' &! $
<^]<Ph( !) ) % #
CdT<Ph !!) ) ( ! (
Bd]<Ph' %) ) '!
<^]<Ph(!!) ) & !%
Bd]<Ph' !) ) " "&
!PPARENTLY THE MOST LOG FILE WRITE LATENCY PROBLEMS WERE DETECTED AT !- -AY 7HEN YOU LOOK
AT THE OTHER OUTPUT LINES YOU PROBABLY SEE AN INTERESTING PATTERN !LL THESE PROBLEMS ARE REPORTED FROM A
FEW SECONDS AFTER A FULL HOUR 7ELL THIS IS JUST BECAUSE IN THIS PARTICULAR %XADATA TESTING CLUSTER WE HAVE
OVER TEN SEPARATE DATABASES INSTALLED BUT THEY ONLY RUN SOME REPORTS IN PARALLEL AND USUALLY THERE IS NO
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
/,40 ACTIVITY DOING COMMITS "UT IS IT THEN DOING THESE COMMITS JUST AFTER THE HOUR STARTS 7ELL THESE ARE
DEFAULT /RACLE $ATABASE INSTALLATIONS WITH !72 SNAPSHOTS TAKING PLACE AT THE FULL HOUR !ND THERE ARE
OVER DATABASES IN THE CLUSTER WITH AN !72 SNAPSHOT TAKEN AND WRITTEN IN EACH CLUSTER DATABASE
INSTANCE %ACH !72 SNAPSHOT WRITE INVOLVES MULTIPLE COMMITS AND APPARENTLY THE RESULTING LOG FILE
WRITES HAVE OCCASIONALLY TAKEN LONGER THAN EXPECTED 4HE NEXT STEP FOR TROUBLESHOOTING SUCH LOW
LEVEL
)/ ISSUES IN A CELL WOULD BE TO LOOK AT THE /3
LEVEL METRICS LIKE X^bcPc 4HE CELLS DO PERFORM REGULAR
BLOCK )/ AGAINST THEIR DISKS AFTER ALL SO ALL THE /3
LEVEL )/ MONITORING TOOLS STILL DO APPLY
)N THE FOLLOWING EXTRACT FROM THE X^bcPc ARCHIVE YOU SEE A FEW INTERESTING THINGS 4HE PfPXcTIME
SHOWS THAT ON AVERAGE )/ COMPLETION TIME QUEUING PLUS SERVICE TIME IS OVER MS FOR ALMOST ALL THE
DISKS &OR A COUPLE OF DISKS IT IS AS HIGH AS MS AND MS ON AVERAGE 4HE DISKS ESTIMATED SERVICE
TIME beRc\ SHOWS GOOD )/ SERVICE TIMES BELOW MS SO THE HARD DISKS AND LOWER HALF OF THE )/
SUBSYSTEM SEEMS TO WORK /+ 4HE HIGH RESPONSE TIMES MUST HAVE COME FROM )/ QUEUING WAITS WHICH
HAPPENS WHEN MORE )/ REQUESTS ARE ISSUED TO /3 THAN THE STORAGE HARDWARE AND DEVICE DRIVER
CONFIGURATION CAN ACCEPT
CX\T) %) ) $
3TeXRT)a bf babTR bfbTR bPeVa` biPeV`d biPPfPXcbeRc\dcX[
bSP'# % $ ! %!!"$ ! "' # %"% % "$&&"' #(( # (" '#
bSQ%' % % ' !#& % ' "&$ ! #&% #(! ( !$" ('' "&& $
bSR%$ " ( ' # ! # " # & '!' (#$' !
bSS% ! ! "$' # !& % (%& (( &# %& $!' $%$! "'
bST% # ' !!&$! !#$ % '(' # &! &"' "!$" (!
bSU % ! $ ! !' ' !' $& & & !
bSV$' '!&! ! "( &( $ &" &%' $"#( #%
bSW$# % ! # ( ! #' # (#% %&&& ($ $#( "#( '$ &'
bSX!# ! # ' ""!' !#! ' "$ % $ '! !$ $ !#%& %!
bSY%# ! # " $%% # " # ! ! ! & & #"' %'$%
bSZ% ! ' !#$ $ ! !$ % ! ' &! ! "!! ! $(( "!$& &'
bS[&' # ! % " (% ' ! ! % ( (' ! # $" '& $$% #
4HIS CASE IS AN EXAMPLE OF HOW LOW
LATENCY /,40 )/S LIKE LOG FILE WRITES FOR COMMITS DONT GO WELL
TOGETHER WITH LARGE MASSIVELY PARALLEL ASYNCHRONOUS )/ WORKLOADS WHICH DRIVE THE DISK )/ QUEUE
LENGTHS UP POSSIBLY MAKING THE LOW
LATENCY )/S SUFFER 4HIS IS WHY YOU SHOULD KEEP THE )/ 2ESOURCE
-ANAGER ENABLED IN %XADATA CELLS WHEN RUNNING MIXED WORKLOADS AS )/2- IS ABLE TO THROTTLE THE 3MART
3CAN )/ AND PRIORITIZE IMPORTANT BACKGROUND PROCESS )/ SUCH AS LOG FILE WRITES AUTOMATICALLY
9OU CAN READ MORE ABOUT THE /3 7ATCHER TOOL IN -Y /RACLE 3UPPORT ARTICLE h/3 7ATCHER 5SER
'UIDEv $OC )$ )N FACT YOU CAN DOWNLOAD THE LATEST /3 7ATCHER FROM THERE AND USE IT FOR
MONITORING YOUR OTHER SERVERS TOO 4HE -Y /RACLE 3UPPORT VERSION OF /3 7ATCHER DOES NOT INCLUDE THE
%XADATA CELL
SPECIFIC MONITORING SCRIPTS BUT YOU DONT NEED THOSE FOR YOUR REGULAR SERVERS ANYWAY &OR
%XADATA STORAGE CELLS YOULL HAVE TO KEEP USING THE ORIGINAL VERSION SHIPPED WITH THE CELL SOFTWARE AS
/RACLE DOESNT AUTHORIZE INSTALLATION OF ANY ADDITIONAL STORAGE INTO STORAGE CELLS IF YOU WANT TO KEEP THE
SUPPORTED CONFIGURATION
)T PAYS OFF TO OPEN UP AND SKIM THROUGH /3 7ATCHER SCRIPTS YOU MIGHT LEARN SOME NEW USEFUL /3
LEVEL MONITORING COMMANDS FROM THERE AND WILL HAVE A BETTER OVERVIEW OF WHICH /3 METRICS ARE
AVAILABLE 9OU SHOULD DRILL DOWN TO THESE LOW
LEVEL METRICS WHEN THE MORE CONVENIENTLY ACCESSIBLE
/RACLE $ATABASE AND #ELLCLI
LEVEL METRICS ARE NOT ENOUGH
#(!04%2 -/.)4/2).' %8!$!4! 0%2&/2-!.#%
Summary
4HERE ARE THOUSANDS OF METRICS THAT /RACLE $ATABASE AND THE CELLS PROVIDE US 7EVE ONLY TOUCHED ON A
SMALL SET OF THEM HERE 4HIS LEADS TO THE OBVIOUS QUESTIONS ARE ALL THESE METRICS IMPORTANT 7HAT SHOULD
BE THEIR hGOODv VALUES 7HICH ONES SHOULD WE ACT ON !ND SO ON 4HE GENERAL ANSWER TO ALL THOSE
QUESTIONS IS THAT NO YOU DO NOT NEED TO LEARN MEMORIZE AND hTUNEv ALL OF THESE METRICS IT IS
IMPOSSIBLE 9OU SHOULD ALWAYS MEASURE WHAT MATTERS n AND ITS USUALLY THE RESPONSE TIME )DEALLY YOU
SHOULD START BY MEASURING THE end user RESPONSE TIME AND DRILL DOWN WHERE NEEDED FROM THERE
SOMETIMES TO THE DATABASE SOMETIMES TO THE APPLICATION SERVER OR NETWORK METRICS )TS NOT ALWAYS THE
DATABASE THAT IS CAUSING THE TROUBLE YOU KNOW (OWEVER THANKS TO THE COMPLEXITY OF MODERN MULTI
TIER
APPLICATIONS THIS END
TO
END DIAGNOSIS MAY NOT BE AVAILABLE AND NOT FEASIBLE TO IMPLEMENT
)N SUCH CASE YOU WOULD TAKE A STEP DOWNWARD AND MONITOR THE RESPONSE TIME OF YOUR DATABASE
QUERIES AND TRANSACTIONS 9OU JUST START THE TOP
DOWN APPROACH FROM A LITTLE LOWER IN THE APPLICATION
STACK KEEPING IN MIND THAT THE PROBLEM MAY STILL ACTUALLY BE HAPPENING SOMEWHERE IN THE APPLICATION
LAYER OR HIGHER 7HEN MONITORING THE RESPONSE TIMES OF YOUR USER REPORTS AND $7 QUERIES YOU SHOULD
PROBABLY START FROM THE 31, -ONITORING PAGE OR !3( DATA AND IDENTIFY THE PROBLEM USERS QUERY FROM
THERE SO YOU CAN DRILL DOWN INTO IT 5SING THE TOP
DOWN APPROACH AND FOLLOWING THE BIGGEST TIME
CONSUMERS IN THE 31, PLAN FROM THERE IS MUCH EASIER THAN THE OPPOSITE BOTTOM
UP APPROACH WHERE YOU
MIGHT LOOK AT SOME DATABASE
WIDE TOP 31, REPORT AND HOPE TO FIGURE OUT WHATS WRONG WITH YOUR SPECIFIC
USERS WORKLOAD
/F COURSE THERE ARE CASES WHERE YOULL BE MONITORING SYSTEM
WIDE AGGREGATED anonymous
PERFORMANCE DATA NOT TIED TO ANY SPECIFIC USER OR SESSION AS IN CAPACITY PLANNING AND UTILIZATION
MONITORING (OWEVER QUESTIONS LIKE h)S MY DATABASE PERFORMING WELL v SHOULD NEVER BE ANSWERED BY
LOOKING AT SOME SYSTEM UTILIZATION REPORT .ONE OF THESE TOOLS CAN TELL YOU WHETHER YOUR DATABASE SYSTEM
PERFORMS WELL ONLY YOUR USERS CAN !ND IF THE USERS ARE UNHAPPY YOU CAN START FROM AN UNHAPPY USER
AND DRILL DOWN INTO THE RESPONSE TIME FROM THERE /RACLE PROVIDES ALL THE METRICS YOU NEED
C H A P T E R 13
Migrating to Exadata
3O THE DAY IS FINALLY HERE 9OUR %XADATA $ATABASE -ACHINE IS INSTALLED CONFIGURED TUNED TWEAKED AND
READY TO GO "Y NOW YOUVE PROBABLY INVESTED MANY MANY HOURS LEARNING ABOUT %XADATA PROVING ITS
VALUE TO THE COMPANY AND PLANNING HOW YOU WILL MAKE THE MOST OF THIS POWERFUL DATABASE PLATFORM .O
DOUBT IT HAS BEEN A LONG ROAD TO TRAVEL BUT YOU ARENT THERE QUITE YET .OW THE REAL WORK BEGINS
MIGRATION
4HIS WAS A MUCH MORE DIFFICULT CHAPTER TO WRITE THAN WE EXPECTED 7E CANT COUNT ALL THE MIGRATIONS
WEVE BEEN THROUGH OVER THE YEARS "UT WHEN WE CONSIDERED ALL THE VARIOUS VERSIONS OF /RACLE THE
MIGRATION TOOLS AVAILABLE AND HOW THEY HAVE CHANGED FROM ONE VERSION TO THE NEXT IT BECAME CLEAR THAT
WE NEEDED TO NARROW THE SCOPE SOMEWHAT 3O TO KEEP THIS INTERESTING AND SAVE A FEW TREES WELL BE
FOCUSING ON VERSION %NTERPRISE %DITION FOR A MAJORITY OF THIS CHAPTER !LONG THE WAY WELL POINT OUT
WAYS TO MAKE THE MOST OF THE FEATURES AVAILABLE IN PREVIOUS VERSIONS OF THE /RACLE DATABASE
4HERE ARE MANY METHODS TOOLS AND TECHNIQUES FOR MIGRATING YOUR DATABASE FROM LEGACY HARDWARE
TO %XADATA BUT GENERALLY SPEAKING THEY FALL INTO TWO BROAD CATEGORIES PHYSICAL MIGRATION AND LOGICAL
MIGRATION 7HILE THERE ARE SEVERAL FACTORS THAT DETERMINE WHICH METHOD IS BEST THE DECISION MAKING
PROCESS IS USUALLY DOMINATED BY ONE FACTOR THE AVAILABLE DOWN TIME TO COMPLETE THE MOVE 4HE GOOD
NEWS IS THAT THERE ARE SEVERAL STRATEGIES TO HELP YOU GET THERE %ACH METHOD COMES WITH ITS OWN PROS AND
CONS )N THIS CHAPTER WERE GOING TO DIG INTO EACH OF THESE METHODS 7ELL TALK ABOUT REASONS YOU MIGHT
USE ONE OVER THE OTHER THE RELATIVE ADVANTAGES AND DISADVANTAGES AND WHAT COMMON PITFALLS YOU SHOULD
WATCH OUT FOR
.OTE -IGRATING YOUR APPLICATIONS TO /RACLE %XADATA FROM NON
/RACLE PLATFORMS IS OUT OF THE SCOPE OF THIS BOOK
SO IT WONT BE COVERED HERE
#(!04%2 -)'2!4).' 4/ %8!$!4!
+EVIN 3AYS $URING 4ECHNICAL 2EVIEW ) APPROACHED THIS CHAPTER WITH SKEPTICISM )N MY THINKING THE IDEA OF AN
ENTIRE CHAPTER DEDICATED TO MIGRATING A DATABASE TO THE %XADATA $ATABASE -ACHINE SEEMED LIKE A WASTE OF SPACE
!LLOW ME TO EXPLAIN
) ROUTINELY REMIND PEOPLE THAT UNLESS %XADATA (YBRID #OLUMNAR #OMPRESSION %(## IS BEING USED THERE IS NO
DIFFERENCE BETWEEN DATA SEGMENTS STORED IN %XADATA 3TORAGE 3ERVER CELLS AND THOSE IN CONVENTIONAL STORAGE 4HE
/RACLE $ATABASE SOFTWARE DEPLOYED IN THE %XADATA $ATABASE -ACHINE IS /RACLE $ATABASE GWITH A VERY SMALL
AMOUNT OF CODE THAT KNOWS HOW TO COMPRESS DATA INTO %(## FORM AND OTHERWISE INTERFACE WITH %XADATA VIA I$" )
CONDUCTED MY FIRST REVIEW PASS OF THIS CHAPTER WITH THE MINDSET OF AN IMAGINARY CUSTOMER WHO FOR WHATEVER
REASON FINDS NO NEED FOR %(## 7ITH THAT MINDSET ) EXPECTED THIS CHAPTER TO BE NOTHING MORE THAN AN OVERVIEW OF
DATABASE MIGRATION CONCEPTS AN OCCASIONAL REFERENCE TO /RACLE PRODUCT DOCUMENTATION AND PERHAPS A PARAGRAPH
OR TWO OF SPECIAL HANDLING CONSIDERATIONS FOR %(## (OWEVER AFTER MY FIRST READING OF THE CHAPTER ) FELT COMPELLED
TO ADD THIS NOTE TO THE READER %VEN IF YOU KNOW EVERYTHING THERE IS TO KNOW ABOUT DATABASE MIGRATION IN THE
CONTEXT OF /RACLE DATABASE ) ENCOURAGE YOU TO READ THIS CHAPTER (AVING SAID THAT ) STILL WISH TO REINFORCE THE
PRINCIPLE THAT THERE IS NO DIFFERENCE BETWEEN A DATABASE STORED IN %XADATA 3TORAGE 3ERVER CELLS AND ONE STORED IN
CONVENTIONAL STORAGEUNLESS (## IS INVOLVED 4O END UP WITH AN /RACLE DATABASE STORED IN %XADATA 3TORAGE
3ERVER CELLS YOU HAVE TO FLOW THE DATA THROUGH THE DATABASE GRID USING THE SAME TOOLS USED FOR ANY OTHER MIGRATION
TO /RACLE (OWEVER ) BELIEVE THAT THE PRINCIPLES CONVEYED IN THIS CHAPTER WILL BE QUITE HELPFUL IN ANY MIGRATION
SCENARIO TO /RACLE ) CONSIDER THAT A BONUS
Migration Strategies
/NCE YOU HAVE A GOOD UNDERSTANDING WHAT %XADATA IS AND HOW IT WORKS YOU ARE READY TO START THINKING
ABOUT HOW YOU ARE GOING TO GET YOUR DATABASE MOVED -IGRATION STRATEGIES FALL INTO TWO GENERAL
CATEGORIES LOGICAL MIGRATION AND PHYSICAL MIGRATION ,OGICAL MIGRATION INVOLVES EXTRACTING THE DATA FROM
ONE DATABASE AND LOADING IT INTO ANOTHER 0HYSICAL MIGRATION REFERS TO LIFTING THE DATABASE BLOCK BY
BLOCK FROM ONE DATABASE SERVER AND MOVING IT TO ANOTHER 4HE DATA ACCESS CHARACTERISTICS OF YOUR
DATABASE ARE A KEY CONSIDERATION WHEN DECIDING WHICH MIGRATION METHOD IS BEST 4HIS IS PRIMARILY
BECAUSE OF THE WAY THE DATA IS ACCESSED ON %XADATA /,40 DATABASES TEND TO USE SINGLE BLOCK READS AND
UPDATE DATA ACROSS ALL TABLES WHEREAS $ATA 7AREHOUSE $7 DATABASES ARE TYPICALLY OPTIMIZED FOR FULL
TABLE SCANS AND ONLY UPDATE CURRENT DATA %XADATA USES &LASH #ACHE ON THE STORAGE CELLS TO OPTIMIZE
SINGLE BLOCK READS AND IMPROVE THE OVERALL PERFORMANCE FOR /,40 DATABASES &OR $7 DATABASES %XADATA
USES 3MART 3CAN TECHNOLOGY TO OPTIMIZE FULL TABLE SCANS 4HE DETAILS OF THESE TWO OPTIMIZATION METHODS
ARE COVERED IN #HAPTERS AND ,OGICAL MIGRATION ALLOWS YOU THE OPPORTUNITY TO MAKE CHANGES TO YOUR
DATABASE TO OPTIMIZE IT FOR THE %XADATA PLATFORM 3UCH CHANGES MIGHT INCLUDE RESIZING EXTENTS
IMPLEMENTING OR REDESIGNING YOUR CURRENT PARTITIONING SCHEMES AND COMPRESSING TABLES USING (##
4HESE ARE ALL VERY IMPORTANT STORAGE CONSIDERATIONS FOR LARGE TABLES AND ESPECIALLY SO FOR $7 DATABASES
"ECAUSE /,40 APPLICATIONS TEND TO UPDATE DATA THROUGHOUT THE DATABASE (## COMPRESSION IS NOT A
#(!04%2 -)'2!4).' 4/ %8!$!4!
GOOD FIT AND WOULD ACTUALLY DEGRADE PERFORMANCE !ND WHILE LARGE EXTENTS -" AND -" ARE
BENEFICIAL FOR $7 DATABASES THEY ARE LESS ADVANTAGEOUS FOR /,40 DATABASES WHICH USE MOSTLY INDEX
BASED ACCESS AND hRANDOMv SINGLE
BLOCK READS 0HYSICAL MIGRATION BY ITS VERY NATURE ALLOWS NO CHANGES
TO BE MADE TO THE STORAGE PARAMETERS FOR TABLES AND INDEXES IN THE DATABASE WHILE LOGICAL MIGRATION
ALLOWS MUCH MORE FLEXIBILITY IN REDEFINING STORAGE COMPRESSION PARTITIONING AND MORE
Logical Migration
2EGARDLESS OF THE TECHNOLOGY USED logical migration CONSISTS OF EXTRACTING OBJECTS FROM THE SOURCE
DATABASE AND RELOADING THEM INTO A TARGET DATABASE %VEN THOUGH LOGICAL MIGRATION STRATEGIES TEND TO BE
MORE COMPLICATED THAN PHYSICAL STRATEGIES THEY ARE USUALLY PREFERABLE BECAUSE OF THE FOLLOWING
ADVANTAGES
Staged Migration :4ABLES AND PARTITIONS THAT ARE NO LONGER TAKING UPDATES CAN
BE MOVED OUTSIDE OF THE MIGRATION WINDOW REDUCING THE VOLUME TO BE MOVED
DURING THE FINAL CUT OVER
Selective Migration: /FTEN TIMES THE SOURCE DATABASE HAS OBSOLETE USER
ACCOUNTS AND DATABASE OBJECTS THAT ARE NO LONGER NEEDED 7ITH THE LOGICAL
METHOD THESE OBJECTS MAY BE SIMPLY OMITTED FROM THE MIGRATION 4HE OLD
DATABASE MAY BE KEPT AROUND FOR AWHILE IN CASE YOU LATER DECIDE YOU NEED
SOMETHING THAT DIDNT GET MIGRATED
Platform Differences: $ATA IS CONVERTED TO TARGET DATABASE BLOCK SIZE
AUTOMATICALLY "IG
ENDIAN TO LITTLE
ENDIAN CONVERSION IS HANDLED AUTOMATICALLY
%XADATA HYBRID COLUMNAR COMPRESSION (## CAN BE CONFIGURED BEFORE DATA IS
MOVED 4HAT IS THE TABLES MAY BE DEFINED WITH (## IN THE %XADATA DATABASE SO
THAT DATA IS COMPRESSED AS IT IS LOADED INTO THE NEW DATABASE
Extent Sizing: 4ARGET TABLES PARTITIONS AND INDEXES MAY BE PRE
CREATED WITH
OPTIMAL EXTENT SIZES MULTIPLES OF -" BEFORE THE DATA IS MOVED
Allows Merging of Databases: 4HIS IS PARTICULARLY IMPORTANT WHEN %XADATA IS
USED AS A CONSOLIDATION PLATFORM )F YOUR %XADATA IS MODEL 6 OR 8
MEMORY
ON THE DATABASE SERVERS MAY BE A SOMEWHAT LIMITING FACTOR 6 DATABASE SERVERS
ARE CONFIGURED WITH ' OF 2!- EACH WHILE 8
COMES WITH ' 2!- PER
SERVER 4HIS IS A LOT OF MEMORY WHEN DEALING WITH OR FEWER MODERATE TO LARGE
SIZED DATABASES "UT IT IS BECOMING FAIRLY COMMON TO SEE OR MORE DATABASES
ON A SERVER &OR EXAMPLE ONE OF US WORKED ON A PROJECT WHERE %XADATA WAS USED
TO HOST 0EOPLE3OFT (2 AND &INANCIALS DATABASES 4HE IMPLEMENTER REQUESTED
DATABASES FOR THIS EFFORT !DD TO THIS THE DATABASES IN THEIR PLAN FOR OTHER
APPLICATIONS AND 3'! MEMORY BECAME A REAL CONCERN 4HE SOLUTION OF COURSE IS
TO MERGE THESE SEPARATE DATABASES TOGETHER ALLOWING THEM TO SHARE MEMORY
MORE EFFICIENTLY 4HIS MAY OR MAY NOT BE A DIFFICULT TASK DEPENDING ON HOW
CONTAINED THE DATABASES ARE AT THE SCHEMA LEVEL
)F USING THE hCREATE TABLE AS SELECTv METHOD OR hINSERT INTO AS SELECTv METHOD
#4!3 OR )!3 OVER A DATABASE LINK THEN THE DATA MAY ALSO BE SORTED AS IT IS
LOADED INTO THE TARGET DATABASE TO IMPROVE INDEX EFFICIENCY OPTIMIZE FOR
%XADATA 3TORAGE )NDEXES AND ACHIEVE BETTER COMPRESSION RATIOS
#(!04%2 -)'2!4).' 4/ %8!$!4!
4HERE ARE BASICALLY TWO APPROACHES FOR LOGICAL MIGRATION /NE INVOLVES EXTRACTING DATA FROM THE
SOURCE DATABASE AND LOADING IT INTO THE TARGET DATABASE 7ELL CALL THIS THE h%XTRACT AND ,OADv METHOD
4OOLS COMMONLY USED IN THIS APPROACH ARE $ATA 0UMP %XPORT)MPORT AND #4!3 OR )!3 THROUGH A
DATABASE LINK 4HE OTHER METHOD IS TO REPLICATE THE SOURCE DATABASE DURING NORMAL BUSINESS OPERATIONS
7HEN THE TIME COMES TO SWITCH TO THE NEW DATABASE REPLICATION IS CANCELLED AND CLIENT APPLICATIONS ARE
REDIRECTED TO THE NEW DATABASE 7ELL REFER TO THIS AS THE h2EPLICATION
"ASEDv METHOD 4OOLS COMMONLY
USED IN THE 2EPLICATION
"ASED METHOD ARE /RACLE 3TREAMS /RACLE $ATA 'UARD ,OGICAL 3TANDBY AND
/RACLE 'OLDEN 'ATE )T IS ALSO POSSIBLE TO USE A COMBINATION OF PHYSICAL AND LOGICAL MIGRATION SUCH AS
COPYING MOSTLY READ ONLY TABLESPACES OVER WELL AHEAD OF THE FINAL CUT
OVER AND APPLYING CHANGES TO
THEM VIA SOME REPLICATION METHOD LIKE 3TREAMS
%XTRACT AND ,OAD
'ENERALLY SPEAKING THE %XTRACT AND ,OAD METHOD REQUIRES THE MOST DOWNTIME OF ALL THE MIGRATION
STRATEGIES 4HIS IS BECAUSE ONCE THE EXTRACT BEGINS AND FOR THE DURATION OF THE MIGRATION ALL $-, ACTIVITY
MUST BE BROUGHT TO A STOP $ATA WAREHOUSE ENVIRONMENTS ARE THE EXCEPTION TO THE RULE BECAUSE DATA IS
TYPICALLY ORGANIZED IN AN hAGE
INAGE
OUTv FASHION 3INCE DATA IS TYPICALLY PARTITIONED BY DATE RANGE
STATIC DATA IS SEPARATED FROM DATA THAT IS STILL UNDERGOING CHANGE 4HIS hREAD ONLYv DATA MAY BE MIGRATED
AHEAD OF TIME OUTSIDE THE FINAL MIGRATION WINDOW PERHAPS EVEN DURING BUSINESS HOURS 4HE BIGGEST
ADVANTAGE OF THE %XTRACT AND ,OAD STRATEGY IS ITS SIMPLICITY -OST $"!S HAVE USED $ATA 0UMP OR #4!3 FOR
ONE REASON OR ANOTHER SO THE TOOL SET IS FAMILIAR !NOTHER BIG ADVANTAGE IS THE CONTROL IT GIVES YOU /NE OF
THE GREAT NEW FEATURES %XADATA BRINGS TO THE TABLE IS (YBRID #OLUMNAR #OMPRESSION (## 3INCE YOU
HAVE COMPLETE CONTROL OVER HOW THE DATA IS LOADED INTO THE TARGET DATABASE IT IS A RELATIVELY SIMPLE TASK
TO EMPLOY (## TO COMPRESS TABLES AS THEY ARE LOADED IN %XTRACT AND ,OAD ALSO ALLOWS YOU TO IMPLEMENT
PARTITIONING OR CHANGE PARTITIONING STRATEGIES ,OADING DATA USING #4!3 ALLOWS YOU TO SORT DATA AS IT IS
LOADED WHICH IMPROVES THE EFFICIENCY OF %XADATAS STORAGE INDEXES /NE COULD ARGUE THAT ALL THESE THINGS
COULD BE DONE POST MIGRATION AND THAT IS TRUE "UT WHY MOVE THE DATA TWICE WHEN IT CAN BE INCORPORATED
INTO THE MIGRATION PROCESS ITSELF )N SOME SITUATIONS IT MAY NOT EVEN BE POSSIBLE TO FIT THE DATA ONTO THE
PLATFORM WITHOUT APPLYING COMPRESSION )N THE NEXT FEW SECTIONS WE WILL COVER SEVERAL APPROACHES FOR
PERFORMING %XTRACT AND ,OAD MIGRATIONS
+EVIN 3AYS 4HE POINT ABOUT ORDERED LOADING OF DATA IS A VERY IMPORTANT TOPIC )T IS TRUE THAT THIS MAY NOT ALWAYS
BE AN OPTION BUT THE BENEFIT CAN GO EVEN FURTHER THAN THE IMPROVED STORAGE INDEX EFFICIENCY ALREADY MENTIONED
/RDERED LOADING CAN INCREASE THE COMPRESSION RATIO OF %XADATA (YBRID #OLUMNAR #OMPRESSION AS WELL )N SPITE OF
THE SOPHISTICATED LOAD
TIME COMPRESSION TECHNIQUES EMPLOYED BY THE (## FEATURE THE FUNDAMENTALS CAN NEVER BE
FORGOTTEN )N THE CASE OF COMPRESSION LIKE
VALUES ALWAYS COMPRESS MORE DEEPLY )T IS WORTH CONSIDERING WHETHER
THIS APPROACH FITS INTO THE WORKFLOW AND OPPORTUNITY WINDOW FOR DATA LOADING
Data Pump
$ATA 0UMP IS AN EXCELLENT TOOL FOR MOVING LARGE QUANTITIES OF DATA BETWEEN DATABASES $ATA 0UMP
CONSISTS OF TWO PROGRAMS Tg_S_ AND X\_S_ 4HE Tg_S_ COMMAND IS USED TO EXTRACT DATABASE OBJECTS OUT
OF THE SOURCE DATABASE )T CAN BE USED TO DUMP THE CONTENTS OF AN ENTIRE DATABASE OR MORE SELECTIVELY BY
SCHEMA OR BY TABLE ,IKE ITS PREDECESSOR %XPORT Tg_ $ATA 0UMP EXTRACTS DATA AND SAVES IT INTO A
#(!04%2 -)'2!4).' 4/ %8!$!4!
PORTABLE DATA FILE 4HIS FILE CAN THEN BE COPIED TO %XADATA AND LOADED INTO THE TARGET DATABASE USING THE
X\_S_ COMMAND $ATA 0UMP MADE ITS FIRST APPEARANCE IN /RACLE G SO IF YOUR DATABASE IS VERSION I OR
EARLIER YOU WILL NEED TO USE THE OLD %XPORT)MPORT Tg_X\_ INSTEAD %XPORT AND )MPORT HAVE BEEN
AROUND SINCE /RACLE AND ALTHOUGH THEY ARE GETTING A LITTLE LONG IN THE TOOTH THEY ARE STILL VERY EFFECTIVE
TOOLS FOR MIGRATING DATA AND OBJECTS FROM ONE DATABASE TO ANOTHER !ND EVEN THOUGH /RACLE HAS BEEN
TALKING ABOUT DROPPING Tg_ AND X\_ FOR YEARS NOW THEY ARE STILL PART OF THE BASE INSTALL &IRST WELL TALK
ABOUT $ATA 0UMP AND HOW IT CAN BE USED TO MIGRATE TO %XADATA !FTER THAT WE CAN TAKE A QUICK LOOK AT
WAYS TO MIGRATE OLDER DATABASES USING %XPORT AND )MPORT +EEP IN MIND THAT NEW FEATURES AND
PARAMETERS ARE ADDED TO $ATA 0UMP WITH EACH MAJOR RELEASE #HECK THE /RACLE DOCUMENTATION FOR
CAPABILITIES AND FEATURES SPECIFIC TO YOUR DATABASE VERSION
&ROM TIME TO TIME IN THIS CHAPTER WELL MAKE REFERENCE TO TESTS AND TIMINGS WE SAW IN OUR LAB 4ABLE
SHOWS SOME OF THE RELEVANT CHARACTERISTICS OF THE SERVERS AND DATABASES WE USED FOR THESE TESTS 4HE
,!" DATABASE IS THE SOURCE DATABASE AND %8$" IS THE TARGET %XADATA DATABASE )T IS AN %XADATA 6
QUARTER RACK CONFIGURATION
Table 13-1. Lab Configuration
Database Db Version Platform Processors CPU Clock
,!" 2ED (AT ,INUX BIT $UAL #ORE )NTEL 8EON '(Z
)NTEL 8EON -0 #05
%8$" /RACLE %NTERPRISE ,INUX BIT 1UAD #ORE )NTEL 8EON '(Z
% .AHALEM
(ERE IS A BREAKDOWN OF THE SEGMENTS IN MY TEST DATABASE
B46<4=CNCH?4<1HC4B
2;DBC4A%"
8=34G " "&
8=34G?0AC8C8>=!"%
;>18=34G#'
;>1B46<4=C!(
C01;4! %%!
C01;4?0AC8C8>= &%'
CH?4!D=3> #!
.OW LETS TAKE A LOOK AT SOME OF THE $ATA 0UMP PARAMETERS YOULL WANT TO KNOW ABOUT (ERE ARE
SOME OF THE KEY PARAMETERS THAT ARE USEFUL FOR MIGRATING DATABASES
2><?A4BB8>= $ATA 0UMP COMPRESSION IS A RELATIVELY NEW FEATURE )N G YOU HAD
THE ABILITY TO COMPRESS METADATA BUT IN G THIS CAPABILITY WAS EXTENDED TO
TABLE DATA AS WELL 6ALID OPTIONS ARE 0;; 30C0N>=;H <4C030C0N>=;H AND =>=4
5SING THE 2><?A4BB8>=,0;; OPTION $ATA 0UMP REDUCED THE SIZE OF OUR EXPORT
FROM ' TO ' A COMPRESSION RATIO OF OVER TIMES 4HATS A PRETTY
SIGNIFICANT SAVINGS IN STORAGE 7HEN WE RAN THE TEST WITH COMPRESSION TURNED
ON WE FULLY EXPECTED IT TO SLOW DOWN THE EXPORT BUT INSTEAD IT ACTUALLY REDUCED
OUR EXPORT TIME FROM MINUTES TO JUST OVER MINUTES 4HIS WONT ALWAYS BE
THE CASE OF COURSE /N OUR TEST SYSTEM THE EXPORT WAS CLEARLY )/ BOUND "UT IT
DOES POINT OUT THAT COMPRESSION CAN SIGNIFICANTLY REDUCE THE STORAGE
#(!04%2 -)'2!4).' 4/ %8!$!4!
REQUIREMENTS FOR EXPORTING YOUR DATABASE WITHOUT NECESSARILY SLOWING DOWN THE
PROCESS 5NFORTUNATELY THE ABILITY TO COMPRESS TABLE DATA ON THE FLY WAS NOT
INTRODUCED UNTIL RELEASE G2 )F YOUR DATABASE IS G AND YOU NEED TO
COMPRESS YOUR DUMPFILES BEFORE TRANSFERRING THEM TO %XADATA YOU WILL NEED TO
DO THAT USING EXTERNAL TOOLS LIKE GZIP ZIP OR COMPRESS .OTE THAT THE USE OF THE
data 2><?A4BB8>= OPTION IN $ATA 0UMP REQUIRES /RACLE !DVANCED #OMPRESSION
LICENSES
5;0B7102:NC8<4, 5;0B7102:NB2= "ELIEVE IT OR NOT BY DEFAULT $ATA 0UMP DOES
NOT GUARANTEE THE READ CONSISTENCY OF YOUR EXPORT 4O EXPORT A READ
CONSISTENT
IMAGE OF YOUR DATABASE YOU MUST USE EITHER THE 5;0B7102:NB2= OR THE
5;0B7102:NC8<4 PARAMETER )F YOU USE 5;0B7102:NC8<4 $ATA 0UMP LOOKS UP THE
NEAREST 3YSTEM #HANGE .UMBER 3#. CORRESPONDING TO THE TIME YOU SPECIFIED
AND EXPORTS ALL DATA AS OF THAT 3#. 5;0B7102:NC8<4 CAN BE PASSED IN TO $ATA
0UMP AS FOLLOWS
5;0B7102:NC8<4,c^NcX\TbcP\_ $ B4? ! ! ) ) 33 <>= HHHH77!#)<8)BB
)F YOU CHOOSE TO USE 5;0B7102:NB2= YOU CAN GET THE CURRENT 3#. OF YOUR
DATABASE BY RUNNING THE FOLLOWING QUERY
B@;-bT[TRcRdaaT]cNbR]Ua^\eSPcPQPbT*
5D;;, B274<0B, C01;4B 4HESE OPTIONS ARE MUTUALLY EXCLUSIVE AND SPECIFY WHETHER
THE EXPORT WILL BE FOR THE FULL DATABASE A SELECTION OF SCHEMAS OR A SELECTION OF
INDIVIDUAL TABLES .OTE THAT CERTAIN SCHEMAS LIKE 393 -$393 #48393 AND
$"3.-0 ARE NEVER EXPORTED WHEN DOING A FULL DATABASE EXPORT
?0A0;;4; 4HE ?0A0;;4; PARAMETER INSTRUCTS $ATA 0UMP TO SPLIT THE WORK UP INTO
MULTIPLE PARTS AND RUN THEM CONCURRENTLY ?0A0;;4; CAN VASTLY IMPROVE THE
PERFORMANCE OF THE EXPORT PROCESS
=4CF>A:N;8=: 4HIS PARAMETER SPECIFIES A DATABASE LINK IN THE TARGET DATABASE TO
BE USED FOR THE EXPORT )T ALLOWS YOU TO EXPORT A DATABASE FROM A REMOTE SERVER
PULL THE DATA DIRECTLY THROUGH THE NETWORK VIA DATABASE LINK IN THE TARGET
DATABASE AND LAND THE FILES ON AN %XADATA FILE SYSTEM 7E SEE THIS AS MORE OF A
CONVENIENCE THAN ANYTHING ELSE AS IT SAVES YOU THE EXTRA STEP OF TRANSPORTING
THE DUMPFILES MANUALLY AT THE END OF THE EXPORT )T IS USED BY 'RID #ONTROL TO
AUTOMATE THE MIGRATION PROCESS USING THE h)MPORT &ROM $ATABASEv PROCESS
5SING THIS METHOD FOR MANUAL MIGRATION DOESNT MAKE MUCH SENSE IF YOU ARE
GOING TO COPY THE DATA OVER A DATABASE LINK ANYWAY WHY NOT LOAD IT TO TARGET
TABLES DIRECTLY USING #4!3 OR DIRECT
PATH INSERT INSTEAD OF DUMPING IT TO DISK
AND RELOADING BACK LATER ON
.OW LETS TURN OUR ATTENTION TO THE IMPORT PROCESS 3CHEMA
LEVEL IMPORT IS USUALLY PREFERABLE WHEN
MIGRATING DATABASES )T ALLOWS YOU TO BREAK THE PROCESS UP INTO SMALLER MORE MANAGEABLE PARTS 4HIS IS
NOT ALWAYS THE CASE AND THERE ARE TIMES WHEN A FULL DATABASE IMPORT IS THE BETTER CHOICE -OST OF THE
TASKS WE WILL TALK ABOUT HERE APPLY TO BOTH SCHEMA
LEVEL AND FULL DATABASE IMPORTS !S WE GO ALONG WELL
NOTE ANY EXCEPTIONS YOU WILL NEED TO BE AWARE OF )F YOU CHOOSE NOT TO DO A FULL DATABASE IMPORT BE
AWARE THAT SYSTEM OBJECTS INCLUDING ROLES PUBLIC SYNONYMS PROFILES PUBLIC DATABASE LINKS SYSTEM
PRIVILEGES AND OTHERS WILL NOT BE IMPORTED 9OU WILL NEED TO EXTRACT THE $$, FOR THESE OBJECTS USING THE
B@;58;4 PARAMETER AND A 5D;;,H IMPORT 9OU CAN THEN EXECUTE THE $$, INTO THE TARGET DATABASE TO CREATE
THEM ,ETS TAKE A LOOK AT SOME OF THE PARAMETERS USEFUL FOR MIGRATING DATABASES
#(!04%2 -)'2!4).' 4/ %8!$!4!
A4<0?NB274<0 !S THE NAME IMPLIES THIS PARAMETER TELLS $ATA 0UMP TO CHANGE
THE OWNERSHIP OF OBJECTS FROM ONE SCHEMA TO ANOTHER DURING THE COURSE OF THE
IMPORT 4HIS IS PARTICULARLY USEFUL FOR RESOLVING SCHEMA CONFLICTS WHEN MERGING
MULTIPLE DATABASES INTO ONE %XADATA DATABASE
A4<0?N30C058;4 $ATAFILES CAN BE RENAMED DYNAMICALLY DURING THE IMPORT
PROCESS USING THIS PARAMETER 4HIS ALLOWS !3- TO AUTOMATICALLY ORGANIZE AND
NAME THE DATAFILES ACCORDING TO /RACLE -ANAGED &ILES /-& RULES
A4<0?NC01;4B?024 4HIS OPTION CHANGES THE TABLESPACE NAME REFERENCE FOR
SEGMENTS FROM ONE TABLESPACE TO ANOTHER )T IS USEFUL WHEN YOU WANT TO
PHYSICALLY RELOCATE TABLES FROM ONE TABLESPACE TO ANOTHER DURING THE IMPORT
B274<0B ,IST OF SCHEMAS TO IMPORT
B@;58;4 )NSTEAD OF IMPORTING ANYTHING INTO THE DATABASE /BJECT DEFINITIONS
$$, ARE WRITTEN TO AN 31, SCRIPT 4HIS CAN BE QUITE USEFUL FOR PRE
BUILDING
OBJECTS IF YOU WANT TO MAKE CHANGES TO THEIR PHYSICAL STRUCTURE SUCH AS
PARTITIONING OR USING (## COMPRESSION
C01;4N4G8BCBN02C8>= 4HE ACTION TO TAKE IF THE IMPORTED OBJECT ALREADY EXISTS
6ALID KEYWORDS ARE 0??4=3 A4?;024 JB:8?L AND CAD=20C4
C01;4B ! LIST OF TABLES TO IMPORT &OR EXAMPLE C01;4B,:B> B:4FA9>7=B>= C4BC
CA0=B5>A< 4HIS PARAMETER ALLOWS YOU TO MAKE CHANGES TO SEGMENT ATTRIBUTES IN
OBJECT
CREATION $$, STATEMENTS LIKE STORAGE ATTRIBUTES 4HIS PROVIDES A
CONVENIENT WAY TO OPTIMIZE EXTENT SIZES FOR TABLES WHEN THEY ARE CREATED IN
%XADATA
"EFORE YOU BEGIN IMPORTING SCHEMAS INTO YOUR %XADATA DATABASE BE AWARE THAT $ATA 0UMP ONLY
CREATES TABLESPACES AUTOMATICALLY WHEN A FULL DATABASE IMPORT IS DONE 3O IF YOU ARE IMPORTING AT THE
SCHEMA OR TABLE LEVEL YOU WILL NEED TO CREATE YOUR TABLESPACES MANUALLY 4O DO THIS GENERATE THE $$, FOR
TABLESPACES USING THE PARAMETERS 5D;;,hTb AND B@;58;4,jh^daNb`[NbRaX_cl 4HIS PRODUCES A SCRIPT WITH
THE $$, FOR ALL OBJECTS IN THE DUMPFILE INCLUDING DATAFILES /NE THING YOU MAY NOTICE ABOUT THE 2A40C4
C01;4B?024 $$, IS THAT THE DATAFILE FILE NAMES ARE FULLY QUALIFIED 4HIS ISNT AT ALL WHAT WE WANT BECAUSE IT
CIRCUMVENTS /-& AND CREATES HARD
CODED FILE NAMES THAT CANNOT BE MANAGED BY THE DATABASE 4HE
A4<0?N30C058;4 PARAMETER ALLOWS YOU TO RENAME YOUR DATAFILES TO REFLECT THE !3- DISK GROUPS IN YOUR
%XADATA DATABASE 4HE SYNTAX LOOKS SOMETHING LIKE THIS
A4<0?N30C058;4, d ! ^aPSPcP ;01 ! TgP\_[T SQU)30C0
/NE FINAL NOTE BEFORE WE MOVE ON TO %XPORT)MPORT #HARACTER SET TRANSLATION BETWEEN THE SOURCE
AND TARGET DATABASES IS DONE AUTOMATICALLY WITH $ATA 0UMP -AKE SURE THE CHARACTER SET OF THE SOURCE
DATABASE IS A SUBSET OF THE TARGET DATABASE OR SOMETHING MAY BE LOST IN TRANSLATION &OR EXAMPLE ITS
OKAY IF YOUR SOURCE DATABASE IS 53!3#)) BIT AND THE TARGET DATABASE IS 7%)3/0 BIT "UT
MIGRATING BETWEEN DIFFERENT
BIT CHARACTER SETS OR GOING FROM BIT TO BIT MAY CAUSE SPECIAL CHARACTERS
TO BE DROPPED
Export and Import
)F THE DATABASE YOU ARE MIGRATING TO %XADATA IS A RELEASE PRIOR TO VERSION G $ATA 0UMP WONT BE AN
OPTION )NSTEAD YOU WILL NEED TO WORK WITH ITS PREDECESSORS %XPORT Tg_ AND )MPORT X\_
%XPORT)MPORT FEATURES HAVENT CHANGED MUCH SINCE /RACLE BUT IF YOU ARE MIGRATING FROM A PREVIOUS
#(!04%2 -)'2!4).' 4/ %8!$!4!
RELEASE YOU WILL NOTICE THAT SOME FEATURES MAY BE MISSING (OPEFULLY YOU ARENT STILL SUPPORTING I
DATABASES OR 'OD FORBID X BUT NOT TO WORRY %VEN THOUGH SOME OPTIONS LIKE 5;0B7102:NB2= AND
?0A0;;4; ARE NOT OPTIONS IN THESE OLDER RELEASES THERE ARE WAYS TO WORK AROUND THESE MISSING FEATURES
?0A0;;4; IS STRICTLY A $ATA 0UMP FEATURE BUT YOU CAN STILL PARALLELIZE DATABASE EXPORTS BY RUNNING
CONCURRENT SCHEMA EXPORTS 4HIS IS A MUCH LESS CONVENIENT WAY OF hPARALLELIZINGv YOUR EXPORT PROCESS )F
YOU HAVE TO PARALLELIZE YOUR EXPORT PROCESS IN THIS WAY YOU WILL HAVE TO DO THE WORK OF FIGURING OUT WHICH
SCHEMAS GROUPED TOGETHER ARE FAIRLY EQUAL IN SIZE TO MINIMIZE THE TIME IT TAKES FOR ALL OF THEM TO
COMPLETE
2><?A4BB8>= IS ANOTHER FEATURE MISSING FROM %XPORT 4HIS HAS NEVER BEEN MUCH OF AN ISSUE FOR $"!S
SUPPORTING 5NIX,INUX PLATFORMS 4HESE SYSTEMS PROVIDE THE ABILITY TO REDIRECT THE OUTPUT FROM %XPORT
THROUGH THE R^\_aTbb OR ViX_ COMMANDS BY MEANS OF A NAMED PIPE SOMETHING LIKE THIS THE SIGN IS THE
SHELL PROMPT OF COURSE
\ZUXU^Tg_ S\_
[b [Tg_ S\_
_af af a aY^W]b^]SQP >Rc! $) &Tg_ S\_
RPcTg_ S\_kViX_ R-\hNR^\_aTbbTSNTg_^ac S\_ Vi
Tg_bhbcT\UX[T,Tg_ S\_^f]Ta,aY^W]b^]R^]bXbcT]c,hR^\_aTbb,]bcPcXbcXRb,]^]T
[^V,\hNR^\_aTbbTSNTg_^ac [^V
[b [\hNR^\_aTbbTSNTg_^ac
af af a aY^W]b^]SQP" "# #' %$>Rc!!!)"!\hNR^\_aTbbTSNTg_^ac S\_ Vi
af af a aY^W]b^]SQP #"!>Rc!!!)"!\hNR^\_aTbbTSNTg_^ac [^V
4HE A4<0?NC01;4B?024 PARAMETER IS NOT AVAILABLE IN %XPORT)MPORT 4O WORK AROUND THIS YOU WILL
HAVE TO GENERATE A 31, FILE USING THE 8=34G58;4 PARAMETER WHICH PRODUCES A 31, SCRIPT LIKE $ATA 0UMPS
B@;58;4 PARAMETER 9OU CAN THEN MODIFY TABLESPACE REFERENCES AND PRE
CREATE SEGMENTS IN THE NEW
TABLESPACE AS NEEDED 5SING THE 86=>A4 PARAMETER WILL ALLOW )MPORT TO SIMPLY PERFORM AN INSERT INTO THE
TABLES YOU MANUALLY CREATED AHEAD OF TIME 4HE A4<0?NB274<0 PARAMETER TAKES ON A SLIGHTLY DIFFERENT
FORM IN )MPORT 4O CHANGE THE NAME OF A SCHEMA DURING IMPORT USE THE 5A><DB4A AND C>DB4A
PARAMETERS
4HERE IS ONE LIMITATION WITH %XPORT)MPORT THAT CANNOT BE ESCAPED )MPORT DOES NOT SUPPORT
%XADATA (YBRID #OLUMNAR #OMPRESSION (## /UR TESTS SHOW THAT WHEN IMPORTING DATA USING )MPORT
THE BEST TABLE COMPRESSION YOU CAN EXPECT TO GET IS ABOUT WHAT YOU WOULD GET WITH TABLES COMPRESSED FOR
/,40 ALSO KNOWN IN G AS h!DVANCED #OMPRESSIONv )T DOESNT MATTER IF A TABLE IS CONFIGURED FOR ANY
ONE OF THE FOUR (## COMPRESSION MODES AVAILABLE ON %XADATA 1UERY ,OW(IGH AND !RCHIVE
,OW(IGH 4HIS IS BECAUSE (## COMPRESSION CAN ONLY OCCUR IF THE DATA IS DIRECT
PATH INSERTED USING
SYNTAX LIKE X]bTac 0??4=3 FOR EXAMPLE !CCORDING THE %XADATA 5SERS 'UIDE h#ONVENTIONAL
INSERTS AND UPDATES ARE SUPPORTED v BUT hRESULT IN A LESS COMPRESSED FORMAT AND REDUCED COMPRESSION
RATIOv 4HIS hREDUCED COMPRESSION RATIOv IS ACTUALLY THE SAME AS THE /,40 COMPRESSION PROVIDED BY THE
!DVANCED #OMPRESSION OPTION WHICH (## FALLS BACK TO FOR NORMAL INSERTS "Y THE WAY )MPORT WILL NOT
COMPLAIN OR ISSUE ANY WARNINGS TO THIS EFFECT )T WILL SIMPLY IMPORT THE DATA AT A MUCH LOWER COMPRESSION
RATE SILENTLY EATING UP FAR MORE STORAGE THAN YOU PLANNED OR EXPECTED 4HERE IS NOTHING YOU CAN DO
ABOUT IT OTHER THAN REBUILD THE AFFECTED TABLES AFTER THE IMPORT IS COMPLETE 4HE IMPORTANT THING TO
UNDERSTAND IS THAT YOU CANNOT EXPLOIT %XADATAS (## COMPRESSION USING %XPORT)MPORT
4HE %XPORT)MPORT APPROACH ALSO DOES NOT SUPPORT 4RANSPARENT $ATA %NCRYPTION 4$% )F YOUR
DATABASE USES 4$% YOU WILL NEED TO USE $ATA 0UMP TO MIGRATE THIS DATA )F YOU ARE IMPORTING AT THE
SCHEMA LEVEL SYSTEM OBJECTS LIKE ROLES PUBLIC SYNONYMS PROFILES PUBLIC DATABASE LINKS SYSTEM
PRIVILEGES AND OTHERS WILL NOT BE IMPORTED 3YSTEM OBJECTS LIKE THESE CAN BE EXTRACTED BY DOING A FULL
#(!04%2 -)'2!4).' 4/ %8!$!4!
DATABASE IMPORT AND WITH THE 8=34G58;4 PARAMETER TO EXTRACT THE $$, TO CREATE THESE OBJECTS 4HIS STEP
IS WHERE THE MOST MISTAKES ARE MADE )T IS A TEDIOUS PROCESS AND CAREFUL ATTENTION MUST BE GIVEN SO THAT
NOTHING FALLS THROUGH THE CRACKS &ORTUNATELY THERE ARE THIRD
PARTY TOOLS THAT DO A VERY GOOD JOB OF
COMPARING TWO DATABASES AND SHOWING YOU WHERE YOUVE MISSED SOMETHING -OST OF THESE TOOLS LIKE
4/!$ AND $" #HANGE -ANAGER FROM %MBARCADERO ALSO PROVIDE A FEATURE TO SYNCHRONIZE THE OBJECT
DEFINITIONS ACROSS TO THE NEW DATABASE
)F YOU ARE STILL THINKING ABOUT USING %XPORT)MPORT NOTE THAT AS THE DATA LOADING WITH )MPORT DOESNT
USE DIRECT
PATH LOAD INSERTS IT WILL HAVE MUCH HIGHER #05 USAGE OVERHEAD DUE TO UNDO AND REDO
GENERATION AND BUFFER CACHE MANAGEMENT 9OU WOULD ALSO HAVE TO USE A PROPER 1D554A PARAMETER FOR
ARRAY INSERTS YOULL WANT TO INSERT THOUSANDS OF ROWS AT A TIME AND USE 2><<8C,H WHICH WILL COMMIT
AFTER EVERY BUFFER INSERT SO YOU WOULDNT FILL UP THE UNDO SEGMENTS WITH ONE HUGE INSERT TRANSACTION
When to Use Data Pump or Export/Import
$ATA 0UMP AND %XPORT)MPORT ARE VOLUME SENSITIVE OPERATIONS 4HAT IS THE TIME IT TAKES TO MOVE YOUR
DATABASE WILL BE DIRECTLY TIED TO ITS SIZE AND THE BANDWIDTH OF YOUR NETWORK &OR /,40 APPLICATIONS THIS IS
DOWNTIME !S SUCH IT IS BETTER SUITED FOR SMALLER /,40 DATABASES )T IS ALSO WELL SUITED FOR MIGRATING LARGE
$7 DATABASES WHERE READ ONLY DATA IS SEPARATED FROM READ WRITE DATA 4AKE A LOOK AT THE DOWNTIME
REQUIREMENTS OF YOUR APPLICATION AND RUN A FEW TESTS TO DETERMINE WHETHER $ATA 0UMP IS A GOOD FIT
!NOTHER BENEFIT OF $ATA 0UMP AND %XPORT)MPORT IS THAT THEY ALLOW YOU TO COPY OVER ALL THE OBJECTS IN
YOUR APPLICATION SCHEMAS EASILY RELIEVING YOU FROM MANUALLY HAVING TO COPY OVER 0,31, PACKAGES
VIEWS SEQUENCE DEFINITIONS AND SO ON )T IS NOT UNUSUAL TO USE %XPORT)MPORT FOR MIGRATING SMALL TABLES
AND ALL OTHER SCHEMA OBJECTS WHILE THE LARGEST TABLES ARE MIGRATED USING A DIFFERENT METHOD
What to Watch Out for when Using Data Pump or Export/Import
#HARACTER SET DIFFERENCES BETWEEN THE SOURCE AND TARGET DATABASES ARE SUPPORTED BUT IF YOU ARE
CONVERTING CHARACTER SETS MAKE SURE THE CHARACTER SET OF THE SOURCE DATABASE IS A SUBSET OF THE TARGET )F
YOU ARE IMPORTING AT THE SCHEMA LEVEL CHECK TO BE SURE YOU ARE NOT LEAVING BEHIND ANY SYSTEM OBJECTS
LIKE ROLES AND PUBLIC SYNONYMS OR DATABASE LINKS 2EMEMBER THAT (## IS ONLY SUPPORTED IN $ATA 0UMP
"E SURE YOU USE THE CONSISTENCY PARAMETERS OF %XPORT OR $ATA 0UMP TO ENSURE THAT YOUR DATA IS EXPORTED
IN A READ CONSISTENT MANNER $ONT FORGET TO TAKE INTO ACCOUNT THE LOAD YOU ARE PUTTING ON THE NETWORK
$ATA 0UMP AND %XPORT)MPORT METHODS ALSO REQUIRE YOU TO HAVE SOME TEMPORARY DISK SPACE BOTH
IN THE SOURCE AND TARGET SERVER FOR HOLDING THE DUMPFILES .OTE THAT USING $ATA 0UMPS TABLE DATA
COMPRESSION OPTION REQUIRES YOU TO HAVE /RACLE !DVANCED #OMPRESSION LICENSES BOTH FOR THE SOURCE
AND TARGET DATABASE ONLY THE \TcPSPcPN^][h COMPRESSION OPTION IS INCLUDED IN THE %NTERPRISE %DITION
LICENSE
#OPYING $ATA OVER A $ATABASE ,INK
7HEN EXTRACTING AND COPYING VERY LARGE AMOUNTS OF DATAMANY TERABYTESBETWEEN DATABASES
DATABASE LINKS MAY BE YOUR BEST OPTION 5NLIKE THE $ATA0UMP OPTION WITH DATABASE LINKS YOU WILL READ
YOUR DATA ONCE FROM THE SOURCE TRANSFER IT IMMEDIATELY OVER THE NETWORK AND WRITE IT ONCE INTO THE
TARGET DATABASE 7ITH $ATA 0UMP /RACLE WOULD HAVE TO READ THE DATA FROM SOURCE THEN WRITE IT TO A
DUMPFILE AND THEN YOULL TRANSFER THE FILE WITH SOME FILE
TRANSFER TOOL OR DO THE NETWORK COPY OPERATION
USING .&3 READ THE DUMPFILE IN THE TARGET DATABASE AND THEN WRITE IT INTO THE TARGET DATABASE TABLES )N
ADDITION TO ALL THE EXTRA DISK )/ DONE FOR WRITING AND READING THE DUMPFILES YOU WOULD NEED EXTRA DISK
SPACE FOR HOLDING THESE DUMPFILES DURING THE MIGRATION .OW YOU MIGHT SAY h(OLD ON $ATA0UMP does
#(!04%2 -)'2!4).' 4/ %8!$!4!
HAVE THE =4CF>A:N;8=: OPTION AND THE ABILITY TO TRANSFER DATA DIRECTLY OVER DATABASE LINKSv 9ES THATS
TRUE BUT UNFORTUNATELY WHEN USING X\_S_ WITH DATABASE LINKS $ATA 0UMP PERFORMS CONVENTIONAL 8=B4AC
0BB4;42CS NOT DIRECT PATH INSERTS !ND THIS MEANS THAT THE INSERTS WILL BE MUCH SLOWER GENERATE LOTS OF
REDO AND UNDO AND POSSIBLY RUN OUT OF UNDO SPACE !ND MORE IMPORTANTLY CONVENTIONAL PATH )!3 DOES
NOT COMPRESS DATA WITH (## COMPRESSION BUT RESORTS TO REGULAR /,40 COMPRESSION INSTEAD IF (## IS
ENABLED FOR THE TABLE 3O THIS MAKES THE $ATA0UMP FILE
LESS IMPORT OVER DATABASE LINKS VIRTUALLY USELESS
FOR LOADING LOTS OF DATA INTO %XADATA FAST 9OU WOULD HAVE TO USE YOUR OWN DIRECT PATH )!3 STATEMENTS
WITH 0??4=3 HINTS TO GET THE BENEFITS OF DIRECT PATH LOADS AND MAXIMUM PERFORMANCE OUT OF FILE
LESS
TRANSFER OVER DATABASE LINKS
+EVIN 3AYS !DD TO THIS LIST OF POSSIBLE DUMPFILE TRANSFER TECHNIQUES THE CAPABILITY KNOWN AS )NJECTING &ILES WITH
$ATABASE &ILE 3YSTEM $"&3 4HE $"&3 CLIENT SQUbNR[XT]c HAS BEEN PORTED TO ALL ,INUX AND 5NIX PLATFORMS AND
CAN BE USED TO COPY FILES INTO THE %XADATA $ATABASE -ACHINE EVEN IF THE $"&3 FILE SYSTEM IS NOT MOUNTED ON THE
HOSTS OF THE %XADATA $ATABASE -ACHINE NOR ON THE SENDING SYSTEM )T IS A MOUNT
FREE APPROACH /RACLE
$OCUMENTATION PROVIDES CLEAR EXAMPLES OF HOW TO USE THE BUILT
IN R^_h COMMAND THAT SQUbNR[XT]c SUPPORTS 4HE
DATA FLOWS FROM THE SQUbNR[XT]c EXECUTABLE OVER 31,
.ET AND IS INSERTED DIRECTLY INTO 3ECURE&ILE ,/"S 4HE
TRANSFER IS MUCH MORE EFFICIENT THAN THE .&3 PROTOCOL AND IS SECURE -OST %XADATA $ATABASE -ACHINE DEPLOYMENTS
MAINTAIN $"&3 FILE SYSTEMS SO THERE IS LESS OF A NEED FOR LOCAL DISK CAPACITY USED AS A STAGING AREA
4HERE ARE SOME CASES WHERE MOVING YOUR DATA THROUGH DATABASE LINKS MAY NOT PERFORM AS WELL AS
THE $ATA0UMP APPROACH )F YOU HAVE A SLOW NETWORK LINK BETWEEN THE SOURCE AND TARGET MIGRATIONS
BETWEEN REMOTE DATA CENTERS PERHAPS THEN YOU MAY BENEFIT MORE FROM COMPRESSING THE DUMPFILES
WHILE DATABASE LINKS OVER /RACLE .ET WONT DO AS AGGRESSIVE COMPRESSION AS FOR EXAMPLE GZIP CAN DO
/RACLE .ET 31,
.ET DOES SIMPLE COMPRESSION BY DE
DUPLICATING COMMON COLUMN VALUES WITHIN AN
ARRAY OF ROWS WHEN SENT OVER 31,
.ET 4HIS IS WHY THE NUMBER OF BYTES TRANSFERRED OVER 31,
.ET MAY
SHOW A SMALLER VALUE THAN THE TOTAL SIZE OF RAW DATA
4RANSFERRING THE DATA OF A SINGLE TABLE OVER A DATABASE LINK IS VERY EASY )N THE TARGET %XADATA
DATABASE YOULL NEED TO CREATE A DATABASE LINK POINTING TO THE SOURCE DATABASE THEN JUST ISSUE EITHER A
#4!3 OR 8=B4ACB4;42C COMMAND OVER THE DATABASE LINK
2A40C430C010B4;8=:sourcedb
2>==42CC>source_user
834=C858431Hsource_password
DB8=6tns_alias*
2A40C4C01;4UPRc0BB4;42C5A><UPRc/b^daRTSQ*
4HIS EXAMPLE WILL CREATE THE TABLE STRUCTURE AND COPY THE DATA BUT IT WONT CREATE ANY OTHER OBJECTS
SUCH AS INDEXES TRIGGERS OR CONSTRAINTS FOR THE TABLE 4HESE OBJECTS MUST BE MANUALLY CREATED LATER EITHER
BY RUNNING THE $$, SCRIPTS OR BY DOING A METADATA
ONLY IMPORT OF THE SCHEMA OBJECTS
#(!04%2 -)'2!4).' 4/ %8!$!4!
.OTE 7HEN CREATING THE DATABASE LINK YOU CAN SPECIFY THE DATABASE LINKS 4.3 CONNECT STRING DIRECTLY WITH A
53).' CLAUSE LIKE THIS
2A40C430C010B4;8=: DB8=634B2A8?C8>=,033A4BB,?A>C>2>;,C2?7> BC,
[^RP[W^bc?>AC, $! 2>==42CN30C0,B4AE4A,343820C43B4AE824N=0<4,>A0 6
4HAT WAY YOU DONT HAVE TO SET UP c]b]P\Tb ^aP ENTRIES IN THE DATABASE SERVER
!NOTHER OPTION IS TO USE 8=B4ACB4;42C FOR LOADING DATA INTO AN EXISTING TABLE STRUCTURE 7E WANT TO
BYPASS THE BUFFER CACHE AND THE UNDO
GENERATION
AND
REDO
LOGGING MECHANISM FOR THIS BULK DATA LOAD
SO WE CAN USE THE 0??4=3 HINT TO MAKE THIS A DIRECT PATH LOAD INSERT
8=B4AC 0??4=3 8=C>UPRcB4;42C5A><UPRc/b^daRTSQ*
2><<8C*
7ERE ASSUMING HERE THAT THE DATABASE IS IN =>0A278E4;>6 MODE DURING THE MIGRATION SO WE HAVENT
SET ANY LOGGING ATTRIBUTES FOR THE TABLE BEING LOADED )N =>0A278E4;>6 MODE ALL BULK OPERATIONS SUCH AS
8=B4AC0??4=3 X]STg A41D8;3 AND 0;C4AC01;4<>E4 ARE AUTOMATICALLY =>;>668=6
)F YOUR DATABASE MUST BE IN 0A278E4;>6 MODE DURING SUCH DATA LOADING BUT YOU STILL WANT TO PERFORM
THE LOADING OF SOME TABLES WITHOUT LOGGING THEN YOU CAN JUST TEMPORARILY TURN OFF LOGGING FOR THOSE
TABLES FOR THE DURATION OF THE LOAD
0;C4AC01;4UPRc=>;>668=6*
8=B4AC 0??4=3 8=C>UPRcB4;42C5A><UPRc/b^daRTSQ*
0;C4AC01;4UPRc;>668=6*
/F COURSE IF YOUR DATABASE OR TABLESPACE CONTAINING THIS TABLE IS MARKED 5>A24;>668=6 THEN LOGGING
WILL STILL OCCUR DESPITE ANY TABLE
LEVEL =>;>668=6 ATTRIBUTES
Achieving High-Throughput CTAS or IAS over a Database Link
7HILE THE PREVIOUS EXAMPLES ARE SIMPLE THEY MAY NOT GIVE YOU THE EXPECTED THROUGHPUT ESPECIALLY
WHEN THE SOURCE DATABASE SERVER ISNT IN THE SAME ,!. AS THE TARGET 4HE DATABASE LINKS AND THE
UNDERLYING 4#0 PROTOCOL MUST BE TUNED FOR HIGH THROUGHPUT DATA TRANSFER 4HE DATA TRANSFER SPEED IS
LIMITED OBVIOUSLY BY YOUR NETWORKING EQUIPMENT THROUGHPUT AND IS ALSO DEPENDENT ON THE NETWORK
ROUND TRIP TIME 244 BETWEEN THE SOURCE AND TARGET DATABASE
7HEN MOVING TENS OF TERABYTES OF DATA IN A SHORT TIME YOU OBVIOUSLY NEED A LOT OF NETWORK
THROUGHPUT CAPACITY 9OU MUST HAVE SUCH CAPACITY FROM END TO END FROM YOUR SOURCE DATABASE TO THE
TARGET %XADATA CLUSTER 4HIS MEANS THAT YOUR SOURCE SERVER MUST BE ABLE TO SEND DATA AS FAST AS YOUR
%XADATA CLUSTER HAS TO RECEIVE IT AND ANY NETWORKING EQUIPMENT SWITCHES ROUTERS IN BETWEEN MUST
ALSO BE ABLE TO HANDLE THAT IN ADDITION TO ALL OTHER TRAFFIC THAT HAS TO FLOW THROUGH THEM $EALING WITH
CORPORATE NETWORK TOPOLOGY AND NETWORK HARDWARE CONFIGURATION IS A VERY WIDE TOPIC AND OUT OF THE
#(!04%2 -)'2!4).' 4/ %8!$!4!
SCOPE OF THIS BOOK BUT WELL TOUCH THE SUBJECT OF THE NETWORK HARDWARE BUILT IN TO %XADATA DATABASE
SERVERS HERE
)N ADDITION TO THE )NFINI"AND PORTS %XADATA CLUSTERS ALSO HAVE BUILT
IN %THERNET PORTS 4ABLE
LISTS ALL THE %THERNET AND )NFINI"AND PORTS
Table 13-2. Exadata Ethernet and InfiniBand ports in each database server
Exadata Version Hardware Ethernet Ports InfiniBand Ports
6 (0 ¾ 'BS ¾ 'BS $$2
6 3UN ¾ 'BS ¾ 'BS 1$2
8 3UN ¾ 'BS ¾ 'BS
¾ 'BS
8 3UN ¾ 'BS ¾ 'BS
¾ 'BS
The database servers and cells each have one more administrative Ethernet port for server management
(ILOM).
.OTE THAT THIS TABLE SHOWS THE NUMBER OF NETWORK PORTS PER DATABASE SERVER 3O WHILE %XADATA 6
DOES NOT HAVE ANY 'B% PORTS IT STILL HAS ¾ 'B% PORTS PER DATABASE SERVER 7ITH DATABASE SERVERS IN
A FULL RACK THIS WOULD ADD UP TO ¾ 'B% PORTS GIVING YOU A MAXIMUM THEORETICAL THROUGHPUT OF
GIGABITS PER SECOND WHEN USING ONLY %THERNET PORTS 7ITH VARIOUS OVERHEADS GIGAbytes PER SECOND OF
TRANSFER SPEED WOULD THEORETICALLY STILL BE ACHIEVABLE IF YOU MANAGE TO PUT ALL OF THE NETWORK PORTS
EQUALLY INTO USE AND THERE ARE NO OTHER BOTTLENECKS 4HIS WOULD MEAN THAT YOU HAVE TO EITHER BOND THE
NETWORK INTERFACES OR ROUTE THE DATA TRANSFER OF DIFFERENT DATASETS VIA DIFFERENT NETWORK INTERFACES
$IFFERENT DBLINKS CONNECTIONS CAN BE ROUTED VIA DIFFERENT )0S OR $ATA0UMP DUMPFILES TRANSFERRED VIA
DIFFERENT ROUTES
4HIS ALREADY SOUNDS COMPLICATED THATS WHY COMPANIES MIGRATING TO %XADATA OFTEN USED THE HIGH
THROUGHPUT BONDED )NFINI"AND LINKS FOR MIGRATING LARGE DATASETS WITH LOW DOWNTIME 5NFORTUNATELY THE
EXISTING DATABASE NETWORKING INFRASTRUCTURE IN MOST COMPANIES DOES NOT INCLUDE )NFINI"AND IN OLD BIG
IRON SERVERS 4HE STANDARD USUALLY IS A NUMBER OF SWITCHED AND BONDED 'B% %THERNET PORTS OR 'B%
PORTS IN SOME CASES 4HATS WHY FOR %XADATA 66 MIGRATIONS YOU WOULD HAVE HAD TO EITHER INSTALL AN
)NFINI"AND CARD INTO YOUR SOURCE SERVER OR USE A SWITCH CAPABLE OF BOTH HANDLING THE SOURCE %THERNET
TRAFFIC AND FLOWING IT ON TO THE TARGET %XADATA )NFINI"AND NETWORK
,UCKILY THE NEW %XADATA 8
AND 8
RELEASES BOTH HAVE 'B% PORTS INCLUDED IN THEM SO YOU
DONT NEED TO GO THROUGH THE HASSLE OF GETTING YOUR OLD SERVERS )NFINI"AND
ENABLED ANYMORE AND CAN
RESORT TO 'B% CONNECTIONS IF YOUR OLD SERVERS OR NETWORK SWITCHES HAVE 'B% %THERNET CARDS IN
PLACE 0ROBABLY BY THE TIME THIS BOOK COMES OUT NOBODY PLANS LARGE
SCALE MIGRATIONS TO %XADATA 6 AND
6 ANYMORE
#(!04%2 -)'2!4).' 4/ %8!$!4!
.OTE 2EMEMBER THAT IF YOU DO not HAVE HUGE DATA TRANSFER REQUIREMENTS WITHIN A VERY LOW DOWNTIME WINDOW
THEN THE ISSUES DESCRIBED HERE MAY NOT BE PROBLEMS FOR YOU AT ALL 9OU WOULD WANT TO PICK THE EASIEST AND
SIMPLEST DATA TRANSFER METHOD THAT GETS THE JOB DONE WITHIN THE REQUIRED TIME WINDOW )TS IMPORTANT TO KNOW THE
AMOUNT OF DATA TO BE TRANSFERRED IN ADVANCE AND TEST THE ACTUAL TRANSFER SPEEDS IN ADVANCE TO SEE WHETHER YOU
WOULD FIT INTO THE PLANNED DOWNTIME )F YOUR DBLINK TRANSFER OR DUMPFILE COPY OPERATION IS TOO SLOW THEN YOU CAN
USE FREE TOOLS LIKE I0ERF Wcc_) X_TaU b^daRTU^aVT ]Tc TO TEST OUT YOUR NETWORK THROUGHPUT BETWEEN SOURCE
AND TARGET SERVERS )F THE DBLINKS OR DATAPUMP DUMPFILE TRANSFER IS SIGNIFICANTLY WORSE THAN I0ERFS RESULTS THERE
MUST BE A CONFIGURATION BOTTLENECK SOMEWHERE
4HIS LEADS US TO SOFTWARE CONFIGURATION TOPICS FOR HIGH NETWORK THROUGHPUT FOR DATABASE MIGRATIONS
4HIS CHAPTER DOES NOT AIM TO BE A NETWORK TUNING REFERENCE BUT WE WOULD LIKE TO EXPLAIN SOME
CHALLENGES WEVE SEEN 'ETTING THE /RACLE DATABASE LINKS THROUGHPUT RIGHT INVOLVES CHANGING MULTIPLE
SETTINGS AND REQUIRES MANUAL PARALLELIZATION (OPEFULLY THIS SECTION WILL HELP YOU AVOID REINVENTING THE
WHEEL WHEN DEALING WITH HUGE DATASETS AND LOW DOWNTIME REQUIREMENTS
)N ADDITION TO THE NEED FOR SUFFICIENT THROUGHPUT CAPACITY AT THE NETWORK HARDWARE LEVEL THERE ARE
THREE MAJOR SOFTWARE CONFIGURATION SETTINGS THAT AFFECT /RACLES DATA TRANSFER SPEED
v &ETCH ARRAY SIZE PaaPhbXiT
v 4#0 SEND AND RECEIVE BUFFER SIZES
v /RACLE .ET 3ESSION $ATA 5NIT 3$5 SIZE
7ITH REGULAR APPLICATION CONNECTIONS THE FETCH ARRAY SIZE HAS TO BE SET TO A HIGH VALUE RANGING FROM
HUNDREDS TO THOUSANDS IF YOU ARE TRANSFERRING LOTS OF ROWS /THERWISE IF /RACLE SENDS TOO FEW ROWS OUT AT
A TIME MOST OF THE TRANSFER TIME MAY END UP BEING SPENT WAITING FOR 31,
.ET PACKET PING
PONG BETWEEN
THE CLIENT AND SERVER
(OWEVER WITH DATABASE LINKS /RACLE IS SMART ENOUGH TO AUTOMATICALLY SET THE FETCH ARRAY SIZE TO THE
MAXIMUMIT TRANSFERS ROWS AT A TIME 3O WE DONT NEED TO TUNE IT OURSELVES
Tuning TCP Buffer Sizes
4HE 4#0 SEND AND RECEIVE BUFFER SIZES ARE CONFIGURED AT THE OPERATING SYSTEM LEVEL SO EVERY /3 HAS
DIFFERENT SETTINGS FOR IT )N ORDER TO ACHIEVE HIGHER THROUGHPUT THE 4#0 BUFFER SIZES HAVE TO BE INCREASED
IN BOTH ENDS OF THE CONNECTION 7ELL COVER A ,INUX AND 3OLARIS EXAMPLE HERE READ YOUR /3 NETWORKING
DOCUMENTATION IF YOU NEED TO DO THIS ON OTHER PLATFORMS 7E OFTEN USE THE 0ITTSBURGH 3UPERCOMPUTING
#ENTERS h%NABLING (IGH 0ERFORMANCE $ATA 4RANSFERSv PAGE FOR REFERENCE
Wcc_) fff _bR TSd ]Tcf^aZX]V _a^YTRcb cR_cd]T &IRST YOULL NEED TO DETERMINE THE MAXIMUM
BUFFER SIZE 4#0 PER CONNECTION IN YOUR SYSTEM
/N THE %XADATA SERVERS JUST KEEP THE SETTINGS FOR 4#0 BUFFER SIZES AS THEY WERE SET DURING STANDARD
%XADATA INSTALL /N %XADATA THE 4#0 STACK HAS ALREADY BEEN CHANGED FROM GENERIC ,INUX DEFAULTS $O not
CONFIGURE %XADATA SERVERS SETTINGS BASED ON GENERIC DATABASE DOCUMENTATION SUCH AS THE h/RACLE
$ATABASE 1UICK )NSTALLATION 'UIDE FOR ,INUXv
#(!04%2 -)'2!4).' 4/ %8!$!4!
(OWEVER ON THE SOURCE SYSTEM WHICH IS ABOUT TO SEND LARGE AMOUNTS OF DATA THE DEFAULT 4#0 BUFFER
SIZES MAY BECOME A BOTTLENECK 9OU SHOULD ADD THESE LINES TO TcR bhbRc[ R^]U IF THEYRE NOT THERE
ALREADY
]Tc R^aT a\T\NSTUPd[c,!%! ##
]Tc R^aT a\T\N\Pg,1048576
]Tc R^aT f\T\NSTUPd[c,!%! ##
]Tc R^aT f\T\N\Pg,4194304
4HEN ISSUE A bhbRc[~_ COMMAND TO APPLY THESE VALUES INTO THE RUNNING KERNEL
4HE MAXIMUM READ AND WRITE BUFFER SIZES PER 4#0 CONNECTION WILL BE SET TO -" AND -"
RESPECTIVELY BUT THE DEFAULT STARTING VALUE FOR BUFFERS IS K" FOR EACH ,INUX KERNELS AND HIGHER
AND AND HIGHER CAN AUTOMATICALLY TUNE THE ACTUAL BUFFER SIZES FROM DEFAULT VALUES UP TO ALLOWED
MAXIMUMS DURING RUNTIME )F THESE PARAMETERS ARE NOT SET ,INUX KERNEL DEFAULTS TO K" BUFFER
SIZES WHICH MAY BECOME A BOTTLENECK WHEN TRANSFERRING LARGE AMOUNTS OF DATA 4HE WRITE BUFFER HAS
BEEN CONFIGURED BIGGER AS THE SOURCE DATA WOULD BE SENDING WRITING THE LARGE AMOUNTS OF DATA THE
AMOUNT OF DATA RECEIVED WILL BE MUCH SMALLER
4HE OPTIMAL BUFFER SIZES ARE DEPENDENT ON YOUR NETWORK ROUNDTRIP TIME 244 AND THE NETWORK LINK
MAXIMUM THROUGHPUT OR DESIRED THROUGHPUT WHICHEVER IS LOWER 4HE OPTIMAL BUFFER SIZE VALUE CAN BE
CALCULATED USING THE "ANDWIDTH
$ELAY PRODUCT "$0 FORMULA ALSO EXPLAINED IN THE h%NABLING (IGH
0ERFORMANCE $ATA 4RANSFERSv DOCUMENT MENTIONED EARLIER IN THIS SECTION .OTE THAT CHANGING THE KERNEL
PARAMETERS SHOWN EARLIER MEANS A GLOBAL CHANGE WITHIN THE SERVER AND IF YOUR DATABASE SERVER HAS A LOT
OF PROCESSES RUNNING THE MEMORY USAGE MAY RISE THANKS TO THE INCREASED BUFFER SIZES 3O YOU MIGHT NOT
WANT TO INCREASE THESE PARAMETERS UNTIL THE ACTUAL MIGRATION HAPPENS
(ERES AN EXAMPLE FROM ANOTHER /3 TYPE /N 3OLARIS AND NEWER VERSIONS THE DEFAULT SEND AND
RECEIVE BUFFER SIZE IS +" 4HIS WOULD RESULT IN EVEN POORER THROUGHPUT COMPARED TO ,INUXS DEFAULT
+" /N 3OLARIS YOU CAN CHECK THE MAX BUFFER SIZE AND MAX 4#0 CONGESTION WINDOW WITH THE FOLLOWING
COMMANDS
]SS STe cR_cR_N\PgNQdU
#'$&%
]SS STe cR_cR_NRf]SN\Pg
#'$&%
4HIS OUTPUT SHOWS THAT THE MAXIMUM SEND OR RECEIVE BUFFER SIZE IN THE 3OLARIS /3 IS -" 4HIS
CAN BE CHANGED WITH THE ]SS~bTc COMMAND BUT YOULL NEED ROOT PRIVILEGES FOR THAT
]SS bTc STe cR_cR_N\PgNQdU# (#" #
]SS bTc STe cR_cR_NRf]SN\Pg# (#" #
4HIS HOWEVER JUST SETS THE maximum 4#0 BUFFER SIZES AND THE 4#0 CONGESTION WINDOW PER
CONNECTION BUT NOT THE DEFAULT BUFFER SIZES WHICH A NEW CONNECTION WOULD ACTUALLY GET 4HE DEFAULT
BUFFER SIZES CAN BE READ THIS WAY
]SS STe cR_cR_Ng\XcNWXfPc
#( $!
]SS STe cR_cR_NaTReNWXfPc
#( $!
"OTH THE DEFAULT SEND BUFFER XMIT MEANS TRANSMIT AND RECEIVE BUFFER SIZES ARE +" 4O CHANGE
THESE DEFAULTS TO -" EACH WE WOULD RUN THESE COMMANDS
]SS bTc STe cR_cR_NaTReNWXfPc# (#" #
]SS bTc STe cR_cR_Ng\XcNWXfPc# (#" #
#(!04%2 -)'2!4).' 4/ %8!$!4!
.OTE THAT THESE SETTINGS ARE NOT PERSISTENT THEY WILL BE LOST AFTER A REBOOT )F YOU WANT TO PERSIST THESE
SETTINGS YOU SHOULD ADD A STARTUP SCRIPT INTO RCD OR SOME RCLOCAL EQUIVALENT WHICH WOULD RE
RUN THE
PREVIOUS COMMANDS !NOTHER OPTION WOULD BE TO PUT THESE VALUES TO TcR bhbcT\ BUT STARTING FROM
3OLARIS THE USE OF THE GLOBAL TcR bhbcT\ SETTINGS IS NOT ENCOURAGED 9ET ANOTHER APPROACH WOULD BE TO
PUT THE VALUES INTO A 3OLARIS 3ERVICE -ANAGEMENT &RAMEWORK 3-& MANIFEST FILE BUT THAT IS OUT OF THE
SCOPE OF THIS BOOK
)F THE SOURCE SERVER IS STILL GOING TO BE ACTIVELY IN USE IN PRODUCTION DURING THE DATA TRANSFER THEN
THINK TWICE BEFORE INCREASING THE DEFAULT BUFFER SIZES GLOBALLY 7HILE YOU COULD POTENTIALLY MAKE THE BULK
DATA TRANSFER MUCH FASTER FOR EVERYBODY WITH LARGER BUFFER SIZES YOUR SERVER MEMORY USAGE WOULD ALSO
GROW 4#0 BUFFERS LIVE IN KERNEL MEMORY AND YOU COULD RUN INTO MEMORY SHORTAGE ISSUES
4HIS IS WHERE A MORE SOPHISTICATED WAY FOR CHANGING BUFFER SIZES BECOMES VERY USEFUL 3OLARIS ALLOWS
YOU TO SET THE DEFAULT SEND AND RECEIVE BUFFER SIZES AT route LEVEL 3O ASSUMING THAT THE TARGET SERVER IS IN
A DIFFERENT SUBNET FROM MOST OTHER PRODUCTION SERVERS YOU CAN CONFIGURE THE BUFFER SIZES FOR A SPECIFIC
ROUTE ONLY
,ETS CHECK THE CURRENT ROUTES FIRST
netstat -rn
A^dcX]VCPQ[T)8?e#
3TbcX]PcX^]6PcTfPh5[PVbATUDbT8]cTaUPRT
STUPd[c (! %' && !D6 T V
STUPd[c &! % ( D6
&! % &! % ( $ D !T V
(! %' && (! %' && !'D "T V
!!# (! %' && !'D T V
!& !& D7 '![^
,ETS ASSUME THAT THAT THE TARGET %XADATA SERVER USES SUBNET AND ONE OF THE SERVERS HAS
)0 9OU CAN USE THE a^dcTVTc COMMAND IN THE 3OLARIS MACHINE TO SEE IF THERE ARE ANY
EXISTING ROUTE
SPECIFIC SETTINGS
a^dcTVTc (! %' && !"
a^dcTc^) (! %' && !"
STbcX]PcX^]) (! %' &&
\PbZ)!$$ !$$ !$$
X]cTaUPRT)T V
U[PVb)+D?3>=4-
recvpipe sendpipebbcWaTbWacc\baccePa\bW^_R^d]c\cdTg_XaT
$
4HE aTRe_X_T AND bT]S_X_T VALUES ARE ZERO THEY USE WHATEVER ARE THE /3 SYSTEM
WIDE DEFAULTS
,ETS CHANGE THE ROUTE SETTINGS NOW AND CHECK THE SETTINGS WITH a^dcTVTc AGAIN
a^dcTRWP]VT ]Tc (! %' && -recvpipe 4194304 -sendpipe 4194304
RWP]VT]Tc (! %' &&
a^dcTVTc (! %' && !"
a^dcTc^) (! %' && !"
STbcX]PcX^]) (! %' &&
\PbZ)!$$ !$$ !$$
X]cTaUPRT)T V
U[PVb)+D?3>=4-
recvpipe sendpipebbcWaTbWacc\baccePa\bW^_R^d]c\cdTg_XaT
4194304 4194304 $
#(!04%2 -)'2!4).' 4/ %8!$!4!
.OW ALL CONNECTIONS BETWEEN THE SOURCE SERVER AND THAT SUBNET WOULD REQUEST BOTH SEND AND
RECEIVE SOCKET BUFFER SIZE -" BUT CONNECTIONS USING OTHER ROUTES WOULD CONTINUE USING DEFAULT VALUES
.OTE THAT THIS ROUTE CHANGE SETTING ISNT PERSISTENT ACROSS REBOOTS SO YOU WOULD NEED TO GET THE
SYSADMIN TO ADD THIS COMMAND IN A 3OLARIS OR 3-& SERVICE STARTUP FILE
"EFORE YOU CHANGE ANY OF THESE SOCKET BUFFER SETTING AT THE /3 LEVEL THERES SOME GOOD NEWS IF YOUR
SOURCE DATABASE IS /RACLE G OR NEWER 3TARTING FROM /RACLE G IT IS POSSIBLE TO MAKE /RACLE REQUEST A
CUSTOM BUFFER SIZE ITSELF UP TO THE cR_N\PgNQdU LIMIT WHEN A NEW PROCESS IS STARTED 9OU CAN DO THIS BY
CHANGING THE [XbcT]Ta ^aP ON THE SERVER SIDE SOURCE DATABASE AND c]b]P\Tb ^aP OR THE RAW 4.3
CONNECT STRING IN DATABASE LINK DEFINITION IN THE TARGET DATABASE SIDE 4HE TARGET DATABASE ACTS AS THE
CLIENT IN THE DATABASE LINK CONNECTION POINTING FROM TARGET TO SOURCE 4HIS IS WELL DOCUMENTED IN THE
/PTIMIZING 0ERFORMANCE SECTION OF THE Oracle Database Net Services Administrator’s Guide SECTION
h#ONFIGURING )/ "UFFER 3IZEv
!DDITIONALLY YOU CAN REDUCE THE NUMBER OF SYSCALLS /RACLE USES FOR SENDING NETWORK DATA BY
INCREASING THE /RACLE .ET 3ESSION $ATA 5NIT 3$5 SIZE 4HIS REQUIRES EITHER A CHANGE IN [XbcT]Ta ^aP OR
SETTING THE DEFAULT 3$5 SIZE IN SERVER
SIDE b`[]Tc ^aP 2EAD THE /RACLE DOCUMENTATION FOR MORE DETAILS
(ERE IS THE SIMPLEST WAY TO ENABLE HIGHER 3$5 AND NETWORK BUFFER IN /RACLE VERSIONS G AND NEWER
!DD THE FOLLOWING LINE TO THE SOURCE DATABASES b`[]Tc ^aP
3450D;CNB3DNB8I4,"!&%&
-AKE SURE THE SOURCE DATABASES [XbcT]Ta ^aP CONTAINS STATEMENTS LIKE THE FOLLOWING EXAMPLE
;8BC4=4A,
34B2A8?C8>=N;8BC,
34B2A8?C8>=,
033A4BB,?A>C>2>;,8?2:4H,4GC?A>2 $!
033A4BB,
?A>C>2>;,C2?
7> BC,b^[PaXb
?>AC, $!
(SEND_BUF_SIZE=4194304)
(RECV_BUF_SIZE=1048576)
4HE TARGET %XADATA SERVERS c]b]P\Tb ^aP WOULD THEN LOOK LIKE THIS EXAMPLE
B>; !,
34B2A8?C8>=,
(SDU=32767)
033A4BB,
?A>C>2>;,C2?
7> BC,b^[PaXb
?>AC, $!
(SEND_BUF_SIZE=1048576)
(RECV_BUF_SIZE=4194304)
2>==42CN30C0,
B4AE4A,343820C43
#(!04%2 -)'2!4).' 4/ %8!$!4!
B4AE824N=0<4,B>; !
7ITH THESE SETTINGS WHEN THE DATABASE LINK CONNECTION IS INITIATED IN THE TARGET %XADATA DATABASE
THE c]b]P\Tb ^aP CONNECTION STRING ADDITIONS WILL MAKE THE TARGET /RACLE DATABASE REQUEST A LARGER 4#0
BUFFER SIZE FOR ITS CONNECTION 4HANKS TO THE 3$5 SETTING IN THE TARGET DATABASES c]b]P\Tb ^aP AND THE
SOURCE DATABASES b`[]Tc ^aP THE TARGET DATABASE WILL NEGOTIATE THE MAXIMUM 3$5 SIZE POSSIBLE
BYTES
.OTE )F YOU HAVE DONE 31,
.ET PERFORMANCE TUNING IN OLD /RACLE VERSIONS YOU MAY REMEMBER ANOTHER
31,
.ET PARAMETER C3D 4RANSMISSION $ATA 5NIT SIZE 4HIS PARAMETER IS OBSOLETE AND IS IGNORED STARTING WITH
/RACLE .ET /RACLE
)T IS POSSIBLE TO ASK FOR DIFFERENT SIZES FOR SEND AND RECEIVE BUFFERS 4HIS IS BECAUSE DURING THE DATA
TRANSFER THE BULK OF DATA WILL MOVE FROM SOURCE TO TARGET DIRECTION /NLY SOME ACKNOWLEDGEMENT AND
hFETCH MOREv PACKETS ARE SENT IN THE OTHER DIRECTION 4HATS WHY WEVE CONFIGURED THE SEND BUFFER LARGER
IN THE SOURCE DATABASE [XbcT]Ta ^aP AS THE SOURCE WILL DO MOSTLY SENDING /N THE TARGET SIDE
c]b]P\Tb ^aP WEVE CONFIGURED THE RECEIVE BUFFER LARGER AS THE TARGET DATABASE WILL DO MOSTLY RECEIVING
.OTE THAT THESE BUFFER SIZES ARE STILL LIMITED BY THE /3
LEVEL MAXIMUM BUFFER SIZE SETTINGS
]Tc R^aT a\T\N\Pg AND ]Tc R^aT f\T\N\Pg PARAMETERS IN TcR bhbRc[ R^]U IN ,INUX AND cR_N\PgNQdU
KERNEL SETTING IN 3OLARIS
Parallelizing Data Load
)F YOU CHOOSE THE EXTRACT LOAD APPROACH FOR YOUR MIGRATION THERES ONE MORE BOTTLENECK TO OVERCOME IN
CASE YOU PLAN TO USE %XADATA (YBRID #OLUMNAR #OMPRESSION %(## 9OU PROBABLY WANT TO USE %(##
TO SAVE THE STORAGE SPACE AND ALSO GET BETTER DATA SCANNING PERFORMANCE COMPRESSED DATA MEANS FEWER
BYTES TO READ FROM DISK .OTE THAT FASTER SCANNING MAY NOT MAKE YOUR QUERIES SIGNIFICANTLY FASTER IF MOST
OF YOUR QUERY EXECUTION TIME IS SPENT IN OPERATIONS OTHER THAN DATA ACCESS LIKE SORTING GROUPING JOINING
AND ANY EXPENSIVE FUNCTIONS CALLED EITHER IN THE B4;42C LIST OR FILTER CONDITIONS (OWEVER %(##
COMPRESSION REQUIRES MANY MORE #05 CYCLES THAN THE CLASSIC BLOCK LEVEL DE DUPLICATION AS THE FINAL
COMPRESSION IN %(## IS PERFORMED WITH HEAVY ALGORITHMS ,:/ :,IB OR ":IP DEPENDING ON THE
COMPRESSION LEVEL !LSO WHILE deCOMPRESSION CAN HAPPEN EITHER IN THE STORAGE CELL OR DATABASE LAYER
THE COMPRESSION OF DATA CAN HAPPEN ONLY IN THE DATABASE LAYER 3O IF YOU LOAD LOTS OF DATA INTO A %(##
COMPRESSED TABLE USING A SINGLE SESSION YOU WILL BE BOTTLENECKED BY THE SINGLE #05 YOURE USING
4HEREFORE YOULL NEED TO PARALLELIZE THE DATA LOAD TO TAKE ADVANTAGE OF ALL THE DATABASE LAYERS #05S TO GET
THE DATA LOAD DONE FASTER
3OUNDS SIMPLEWELL JUST ADD A ?0A0;;4; FLAG TO THE TARGET TABLE OR A ?0A0;;4; HINT INTO THE QUERY
AND WE SHOULD BE ALL SET RIGHT 5NFORTUNATELY THINGS ARE MORE COMPLEX THAN THAT 4HERE ARE A COUPLE OF
ISSUES TO SOLVE ONE OF THEM IS EASY BUT THE OTHER ONE REQUIRES SOME EFFORT
#(!04%2 -)'2!4).' 4/ %8!$!4!
)SSUE -AKING 3URE THE $ATA Load )S 0ERFORMED IN 0ARALLEL
4HE PROBLEM HERE IS THAT WHILE PARALLEL 1UERY AND $$, ARE ENABLED BY DEFAULT FOR ANY SESSION THE
PARALLEL $-, IS NOT 4HEREFORE PARALLEL #4!3 STATEMENTS WILL RUN IN PARALLEL FROM END TO END BUT THE
LOADING PART OF PARALLEL )!3 STATEMENTS WILL BE DONE IN SERIAL 4HE QUERY PART B4;42C WILL BE PERFORMED IN
PARALLEL AS THE SLAVES PASS THE DATA TO THE SINGLE 1UERY #OORDINATOR AND THE 1# IS THE SINGLE PROCESS
WHICH IS DOING THE DATA LOADING INCLUDING THE #05
INTENSIVE COMPRESSION
4HIS PROBLEM IS SIMPLE TO FIX THOUGH YOULL JUST NEED TO ENABLE PARALLEL $-, IN YOUR SESSION ,ETS
CHECK THE PARALLEL EXECUTION FLAGS IN OUR SESSION FIRST
B@;-B4;42C_`NbcPcdb_S\[NbcPcdb_SS[NbcPcdb_S\[NT]PQ[TS
!-5A><ebTbbX^]F74A4bXS,BHBN2>=C4GCdbTaT]ebXS*
?@NBC0CDB?3<;NBC0CDB?33;NBC0CDB?3<;N4=01;43
4=01;43DISABLED4=01;43NO
4HE PARALLEL $-, IS DISABLED IN THE CURRENT SESSION 4HE ?3<;N4=01;43 COLUMN IS THERE FOR BACKWARD
COMPATIBILITY ,ETS ENABLE 0$-,
B@;-0;C4AB4BB8>=4=01;4?0A0;;4;3<;*
BTbbX^]P[cTaTS
B@;-B4;42C_`NbcPcdb_S\[NbcPcdb_SS[NbcPcdb_S\[NT]PQ[TS
!-5A><ebTbbX^]F74A4bXS,BHBN2>=C4GCdbTaT]ebXS*
?@NBC0CDB?3<;NBC0CDB?33;NBC0CDB?3<;N4=01;43
4=01;434=01;434=01;43H4B
!FTER ENABLING PARALLEL $-, THE 8=B4AC0BB4;42CS ARE ABLE TO USE PARALLEL SLAVES FOR THE LOADING
PART OF THE )!3 STATEMENTS
(ERES ONE IMPORTANT THING TO WATCH OUT FOR REGARDING PARALLEL INSERTS )N THE NEXT EXAMPLE WE HAVE
STARTED A NEW SESSION THUS THE 0$-, IS DISABLED IN IT AND WERE ISSUING A PARALLEL INSERT STATEMENT 7E
HAVE ADDED A hSTATEMENT
LEVELv ?0A0;;4; HINT INTO BOTH INSERT AND QUERY BLOCKS AND THE EXPLAINED
EXECUTION PLAN OUTPUT THE 31<BNG?;0= PACKAGE SHOWS US THAT PARALLELISM IS USED (OWEVER THIS
EXECUTION PLAN WOULD BE VERY SLOW LOADING INTO A COMPRESSED TABLE AS THE PARALLELISM IS ENABLED ONLY FOR
THE QUERY B4;42C PART NOT THE DATA LOADING PART
0AY ATTENTION TO WHERE THE ACTUAL DATA LOADING HAPPENSIN THE ;>030BB4;42C OPERATOR IN THE
EXECUTION PLAN TREE 4HIS ;>030BB4;42C HOWEVER RESIDES ABOVE THE ?G2>>A38=0C>A ROW SOURCE THIS IS
THE ROW SOURCE THAT CAN PULL ROWS AND OTHER INFORMATION FROM SLAVES INTO 1# !LSO IN LINE YOU SEE THE
? -B OPERATOR WHICH MEANS THAT ANY ROWS PASSED UP THE EXECUTION PLAN TREE FROM LINE ARE RECEIVED BY
A SERIAL PROCESS 1#
#(!04%2 -)'2!4).' 4/ %8!$!4!
B@;-8=B4AC 0??4=3?0A0;;4; % 8=C>c!B4;42C ?0A0;;4; % 5A><c
k8Sk>_TaPcX^]k=P\TkA^fbkC@k8= >DCk?@3XbcaXQk
k k8=B4ACBC0C4<4=Ckk' :kkkk
k kLOAD AS SELECTkC!kkkkk
k!k?G2>>A38=0C>Akkkkkk
k"k?GB4=3@2A0=3><k)C@ k' :k@ k? -Bk@2A0=3k
k#k?G1;>2:8C4A0C>Akk' :k@ k?2F2kk
k$kC01;40224BBBC>A0645D;;kC k' :k@ k?2F?kk
=^cT
3TVaTT^U?PaP[[T[Xb\Xb %QTRPdbT^UWX]c
%a^fbbT[TRcTS
4HE MESSAGE h$EGREE OF 0ARALLELISM IS BECAUSE OF HINTv MEANS THAT A REQUEST FOR RUNNING SOME
PART OF THE QUERY WITH PARALLEL DEGREE WAS UNDERSTOOD BY /RACLE AND THIS DEGREE WAS USED IN #"/
CALCULATIONS WHEN OPTIMIZING THE EXECUTION PLAN (OWEVER AS EXPLAINED ABOVE THIS DOESNT MEAN THAT
THIS PARALLELISM WAS USED THROUGHOUT THE WHOLE EXECUTION PLAN )TS IMPORTANT TO CHECK WHETHER THE
ACTUAL DATA LOADING WORK ;>030BB4;42C IS DONE BY THE SINGLE 1# OR BY 08 SLAVES
,ETS SEE WHAT HAPPENS WHEN WE ENABLE PARALLEL $-,
B@;-0;C4AB4BB8>=4=01;4?0A0;;4;3<;*
BTbbX^]P[cTaTS
B@;-8=B4AC 0??4=3?0A0;;4; % 8=C>c!B4;42C ?0A0;;4; % 5A><c
k8Sk>_TaPcX^]k=P\TkA^fbkC@k8= >DCk?@3XbcaXQk
k k8=B4ACBC0C4<4=Ckk' :kkkk
k k?G2>>A38=0C>Akkkkkk
k!k?GB4=3@2A0=3><k)C@ k' :k@ k? -Bk@2A0=3k
k"kLOAD AS SELECTkC!kk@ k?2F?kk
k#k?G1;>2:8C4A0C>Akk' :k@ k?2F2kk
k$kC01;40224BBBC>A0645D;;kC k' :k@ k?2F?kk
=^cT
3TVaTT^U?PaP[[T[Xb\Xb %QTRPdbT^UWX]c
.OW COMPARE THIS PLAN TO THE PREVIOUS ONE 4HEY ARE DIFFERENT )N THIS CASE THE ;>030BB4;42C
OPERATOR HAS MOVED DOWN THE EXECUTION PLAN TREE ITS NOT A PARENT OF ?G2>>A38=0C>A ANYMORE (OW YOU
CAN READ THIS SIMPLE EXECUTION PLAN IS THAT THE C01;40224BBBC>A0645D;; SENDS ROWS TO ?G1;>2:
8C4A0C>A WHICH IS THE ROW SOURCE WHO ACTUALLY CALLS THE C01;40224BB AND PASSES IT THE NEXT RANGE OF
DATA BLOCKS TO READ 08 ",/#+ )4%2!4/2 THEN SENDS ROWS BACK TO ;>030BB4;42C WHICH THEN
#(!04%2 -)'2!4).' 4/ %8!$!4!
IMMEDIATELY LOADS THE ROWS TO THE INSERTED TABLE WITHOUT PASSING THEM TO 1# AT ALL !LL THE B4;42C AND
;>03 WORK IS DONE WITHIN THE SAME SLAVE THERES NO INTERPROCESS COMMUNICATION NEEDED (OW DO WE
KNOW THAT )T IS BECAUSE THE 8= >DC COLUMN SAYS ?2F? 0ARALLEL OPERATION #OMBINED 7ITH 0ARENT FOR
BOTH OPERATIONS AND THE C@ VALUE FOR BOTH OF THE OPERATIONS IS THE SAME @ 4HIS INDICATES THAT
PARALLEL EXECUTION SLAVES DO PERFORM ALL THESE STEPS UNDER THE SAME 4ABLE 1UEUE NODE WITHOUT PASSING
THE DATA AROUND BETWEEN SLAVE SETS
4HE PROPER EXECUTION PLAN WHEN READING DATA FROM A DATABASE LINK LOOKS LIKE THIS
B@;-8=B4AC 0??4=3?0A0;;4; % 8=C>c\_
!-B4;42C ?0A0;;4; % 5A><SQPNb^daRT/baRSQ
k8Sk>_TaPcX^]k=P\TkA^fbkC@ 8]bk8= >DCk?@3XbcaXQk
k k8=B4ACBC0C4<4=Ckk! :kkkk
k k?G2>>A38=0C>Akkkkkk
k!k?GB4=3@2A0=3><k)C@ k! :k@ k? -Bk@2A0=3k
k"kLOAD AS SELECTkC<?kk@ k?2F?kk
k#k?GA4248E4kk! :k@ k?2F?kk
k$k?GB4=3A>D=3 A>18=k)C@ k! :kkB -?kA=3 A>18=k
k%kA4<>C4k310NB>DA24k! :kBA231kA -Bkk
AT\^cTB@;8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
% B4;42C >?0@D4NCA0=B5>A<B70A43 %B70A43 %
>F=4A=0<4CH?4;8=4C4GC
5A><310NB>DA24310NB>DA24PRRTbbX]VBA231
=^cT
3TVaTT^U?PaP[[T[Xb\Xb %QTRPdbT^UWX]c
)N THIS EXAMPLE BECAUSE WE ENABLED PARALLEL $-, AT THE SESSION LEVEL THE DATA LOADING IS DONE IN
PARALLEL THE ;>030BB4;42C IS A PARALLEL OPERATION THE 8= >DC COLUMN SHOWS ?2F? EXECUTED WITHIN 08
SLAVES AND NOT THE 1#
.OTE THAT 31<BNG?;0= SHOWS THE 31, STATEMENT FOR SENDING TO THE REMOTE SERVER OVER THE DATABASE
LINK IN THE 2EMOTE 31, )NFORMATION SECTION ABOVE )NSTEAD OF SENDING ?0A0;;4; HINTS TO THE REMOTE
SERVER AN UNDOCUMENTED B70A43 HINT IS SENT WHICH IS AN ALIAS OF THE ?0A0;;4; HINT
4HIS WAS THE EASIER ISSUE TO FIX )F YOUR DATA VOLUMES ARE REALLY BIG THEN THERE IS ANOTHER PROBLEM TO
SOLVE WITH DATABASE LINKS AND ITS EXPLAINED BELOW
)SSUE !CHIEVING &ULLY 0ARALLEL .ETWORK $ATA 4RANSFER
!NOTHER ISSUE WITH DATABASE LINKS AND PARALLEL EXECUTION IS THAT EVEN IF YOU MANAGE TO RUN PARALLEL
EXECUTION ON BOTH ENDS OF THE LINK IT IS THE QUERY COORDINATORS THAT ACTUALLY OPEN THE DATABASE LINK AND
DO THE NETWORK TRANSFER 4HE 08 SLAVES DONT SOMEHOW MAGICALLY OPEN THEIR OWN DATABASE LINK
CONNECTIONS TO THE OTHER DATABASE ALL TRAFFIC FLOWS THROUGH THE SINGLE QUERY COORDINATOR OF A QUERY 3O
YOU CAN RUN YOUR #4!3)!3 STATEMENT WITH HUNDREDS OF 08 SLAVESBUT YOULL STILL HAVE ONLY A SINGLE
DATABASE LINK CONNECTION DOING NETWORK TRANSFER 7HILE YOU CAN OPTIMIZE THE NETWORK THROUGHPUT BY
#(!04%2 -)'2!4).' 4/ %8!$!4!
INCREASING THE 4#0 BUFFER AND 3$5 SIZES THERES STILL A LIMIT OF HOW MUCH DATA THE 1# PROCESS ON A
SINGLE #05 IS ABLE TO INGEST
&IGURE
ILLUSTRATES HOW THE DATA FLOWS THROUGH A SINGLE QUERY COORDINATORS DESPITE ALL THE PARALLEL
EXECUTION
PX PX
PX PX
QC QC
Source DB Exadata
PX PX
PX PX
Figure 13-1. Data flow is single-threaded through query coordinators.
)N ADDITION TO THE SINGLE 1# DATABASE LINK BOTTLENECK SENDING DATA MESSAGES BETWEEN THE 1# AND
08 SLAVES TAKES SOME EXTRA #05 TIME 4HIS IS WHERE FINE TUNING THE _PaP[[T[NTgTRdcX^]N\TbbPVTNbXiT
PARAMETER HAS HELPED A LITTLE IN THE PAST BUT STARTING FROM /RACLE ITS VALUE DEFAULTS TO +" ANYWAY
SO THERE PROBABLY WONT BE ANY SIGNIFICANT BENEFIT IN ADJUSTING THIS FURTHER !ND IF YOU ARE DOING PARALLEL
DATA LOADS ACROSS MULTIPLE NODES IN A 2!# INSTANCE THERE WILL STILL BE A SINGLE 1# PER QUERY WITH A SINGLE
NETWORK CONNECTION 3O IF 1# RUNS IN NODE AND THE PARALLEL SLAVES IN NODE THE 1# WILL HAVE TO SEND
THE DATA IT FETCHES ACROSS THE DBLINK TO THE 08 SLAVES OVER THE 2!# INTERCONNECT AS 08 MESSAGES
3O IF YOU CHOOSE TO USE DATABASE LINKS WITH PARALLEL SLAVES YOU SHOULD RUN multiple separate queries
IN DIFFERENT INSTANCES AND FORCE THE 08 SLAVES TO BE IN THE SAME INSTANCE AS THE 1# USING THE FOLLOWING
COMMAND
B@;-0;C4AB4BB8>=B4C_PaP[[T[NU^aRTN[^RP[,CAD4*
BTbbX^]P[cTaTS
4HAT WAY YOU WILL AVOID AT LEAST THE 2!# INTER INSTANCE TRAFFIC AND REMOTE MESSAGING #05 OVERHEAD
BUT THE 1# TO 08 SLAVE INTRA INSTANCE MESSAGING ROW DISTRIBUTION OVERHEAD STILL REMAINS HERE
.OW DESPITE ALL THESE OPTIMIZATIONS AND MIGRATING DIFFERENT LARGE TABLES USING PARALLEL QUERIES IN
DIFFERENT INSTANCES YOU MAY STILL FIND THAT A SINGLE DATABASE LINK AND QUERY COORDINATOR DOES NOT
PROVIDE ENOUGH THROUGHPUT TO MIGRATE THE LARGEST FACT TABLES OF YOUR DATABASE WITHIN YOUR DOWNTIME !S
STATED EARLIER THE DATABASE LINKS APPROACH IS BEST USED FOR THE FEW HUGE TABLES IN YOUR DATABASE AND ALL
THE REST CAN BE EXPORTEDIMPORTED WITH $ATA 0UMP 0ERHAPS YOU ONLY HAVE COUPLE OF HUGE FACT TABLES
BUT IF YOU HAVE EIGHT 2!# INSTANCES IN THE FULL RACK %XADATA CLUSTER HOW COULD YOU MAKE ALL THE INSTANCES
TRANSFER AND LOAD DATA EFFICIENTLY 9OU WOULD WANT TO HAVE AT LEAST ONE PARALLEL QUERY WITH ITS QUERY
COORDINATOR AND DATABASE LINK PER INSTANCE AND LIKELY MULTIPLE SUCH QUERIES IF A SINGLE 1# PROCESS CANT
PULL AND DISTRIBUTE DATA FAST ENOUGH
4HE OBVIOUS SOLUTION HERE IS TO TAKE ADVANTAGE OF PARTITIONING AS YOUR LARGE MULTI TERABYTE TABLES
ARE LIKELY PARTITIONED IN THE SOURCE DATABASE ANYWAY 3O YOU CAN COPY THE HUGE TABLE IN MULTIPLE SEPARATE
PARTS (OWEVER THERE ARE A COUPLE OF PROBLEMS ASSOCIATED WITH THIS APPROACH
4HE FIRST ISSUE IS THAT WHEN PERFORMING THE USUAL DIRECT PATH LOAD INSERT WHICH IS NEEDED FOR (YBRID
#OLUMNAR #OMPRESSION AND FOR =>;>668=6 LOADS YOUR SESSION WOULD LOCK THE TABLE IT INSERTS INTO
EXCLUSIVELY FOR ITSELF .OBODY ELSE CAN MODIFY NOR INSERT INTO THAT TABLE WHILE THERE IS AN UNCOMMITTED
#(!04%2 -)'2!4).' 4/ %8!$!4!
DIRECT PATH LOAD TRANSACTION ACTIVE AGAINST THAT TABLE .OTE THAT OTHERS CAN STILL READ THAT TABLE AS B4;42C
STATEMENTS DONT TAKE ENQUEUE LOCKS ON TABLES THEY SELECT FROM
3O HOW TO WORK AROUND THIS CONCURRENCY ISSUE ,UCKILY /RACLE 8=B4AC SYNTAX ALLOWS YOU TO SPECIFY
THE EXACT PARTITION OR SUBPARTITION WHERE YOU WANT TO INSERT BY ITS NAME
8=B4AC 0??4=3
8=C>
UPRcPARTITION ( Y20080101 )
B4;42C
5A><
UPRc/b^daRTSQ
F74A4
^aSTaNSPcT-,C>N30C4! ' HHHH<<33
0=3^aSTaNSPcT+C>N30C4! ' !HHHH<<33
7ITH THIS SYNTAX THE DIRECT
PATH INSERT STATEMENT WOULD LOCK ONLY THE SPECIFIED PARTITION AND OTHER
SESSIONS COULD FREELY INSERT INTO OTHER PARTITIONS OF THE SAME TABLE /RACLE WOULD STILL PERFORM PARTITION
KEY CHECKING TO ENSURE THAT DATA WOULDNT BE LOADED INTO WRONG PARTITIONS )F YOU ATTEMPT TO INSERT AN
INVALID PARTITION KEY VALUE INTO A PARTITION /RACLE RETURNS THE ERROR MESSAGE
>A0 ## )X]bTacTS_PacXcX^]ZThXb^dcbXSTb_TRXUXTS_PacXcX^]
.OTE THAT WE DID NOT USE THE ?0AC8C8>=partition_name SYNTAX IN THE B4;42C PART OF THE QUERY
4HE PROBLEM HERE IS THAT THE QUERY GENERATOR UNPARSER WHICH COMPOSES THE 31, STATEMENT TO BE SENT
OVER THE DATABASE LINK DOES NOT SUPPORT THE ?0AC8C8>= SYNTAX )F YOU TRY IT YOU WILL GET AN ERROR
>A0 # )_PacXcX^]TgcT]STScPQ[T]P\TRP]]^caTUTac^PaT\^cT^QYTRc
4HATS WHY WE ARE RELYING ON THE partition pruning ON THE SOURCE DATABASE SIDEWE JUST WRITE THE
FILTER PREDICATES IN THE F74A4 CONDITION SO THAT ONLY THE DATA IN THE PARTITION OF INTEREST WOULD BE
RETURNED )N THE EXAMPLE JUST SHOWN THE SOURCE TABLE IS RANGE
PARTITIONED BY ORDER?DATE COLUMN AND
THANKS TO THE F74A4 CLAUSE PASSED TO THE SOURCE DATABASE THE PARTITION PRUNING OPTIMIZATION IN THAT
DATABASE WILL ONLY SCAN THROUGH THE REQUIRED PARTITION AND NOT THE WHOLE TABLE
.OTE THAT WE ARE NOT USING THE 14CF44= CLAUSE IN THIS EXAMPLE AS IT INCLUDES BOTH VALUES IN THE RANGE
SPECIFIED IN THE F74A4 CLAUSE WHEREAS /RACLE 0ARTITIONING OPTIONS hVALUES LESS THANv CLAUSE EXCLUDES THE
VALUE SPECIFIED IN $$, FROM THE PARTITIONS VALUE RANGE
)T IS ALSO POSSIBLE TO USE SUBPARTITION
SCOPE INSERT SYNTAX TO LOAD INTO A SINGLE SUBPARTITION THUS
LOCKING ONLY A SINGLE SUBPARTITION AT TIME 4HIS IS USEFUL WHEN EVEN A SINGLE PARTITION OF DATA IS TOO LARGE
TO BE LOADED FAST ENOUGH VIA A SINGLE PROCESSDATABASE LINK ALLOWING YOU TO SPLIT YOUR DATA INTO EVEN
SMALLER PIECES
8=B4AC 0??4=3
8=C>
UPRcSUBPARTITION ( Y20080101_SP01 )
B4;42C
5A><
UPRc/b^daRTSQ
F74A4
^aSTaNSPcT-,C>N30C4! ' HHHH<<33
0=3^aSTaNSPcT+C>N30C4! ' !HHHH<<33
0=3ORA_HASH(customer_id, 63, 0) + 1 = 1
#(!04%2 -)'2!4).' 4/ %8!$!4!
)N THIS EXAMPLE THE SOURCE TABLE IS STILL RANGE PARTITIONED BY ORDER?DATE BUT IT IS HASH PARTITIONED TO
SUBPARTITIONS !S ITS NOT POSSIBLE TO SEND THE BD1?0AC8C8>= CLAUSE THROUGH THE DATABASE LINK EITHER
WE HAVE USED THE >A0N70B7 FUNCTION TO FETCH ONLY THE ROWS BELONGING TO THE FIRST HASH SUBPARTITION OF
TOTAL SUBPARTITIONS 4HE >A0N70B7B@; FUNCTION USES THE SAME ZVVWPbW FUNCTION INTERNALLY WHICH IS
USED FOR DISTRIBUTING ROWS TO HASH PARTITIONS AND SUBPARTITIONS )F WE HAD SUBPARTITIONS WE WOULD
CHANGE THE %" IN THE 31, SYNTAX TO !&]~
!S WE WOULD NEED TO TRANSFER ALL THE SUBPARTITIONS WE WOULD COPY OTHER SUBPARTITIONS IN PARALLEL
DEPENDING ON THE SERVER LOAD OF COURSE BY RUNNING SLIGHT VARIATIONS OF THE ABOVE QUERY AND CHANGING
ONLY THE TARGET SUBPARTITION NAME AND THE >A0N70B7 OUTPUT TO CORRESPONDING SUBPARTITION POSITION
8=B4AC 0??4=3
8=C>
UPRcBD1?0AC8C8>=H! ' NSP02
B4;42C
5A><
UPRc/b^daRTSQ
F74A4
^aSTaNSPcT-,C>N30C4! ' HHHH<<33
0=3^aSTaNSPcT+C>N30C4! ' !HHHH<<33
0=3>A0N70B7Rdbc^\TaNXS%" ,2
!ND SO YOULL NEED TO RUN A TOTAL OF VERSIONS OF THIS SCRIPT
8=C>UPRcSUBPARTITION ( Y20080101_SP03 ) F74A4>A0N70B7Rdbc^\TaNXS%" ,3
8=C>UPRcSUBPARTITION ( Y20080101_SP04 ) F74A4>A0N70B7Rdbc^\TaNXS%" ,4
8=C>UPRcSUBPARTITION ( Y20080101_SP64 ) F74A4>A0N70B7Rdbc^\TaNXS%" ,64
)F YOUR TABLES HASH SUBPARTITION NUMBERING SCHEME IN THE SUBPARTITION NAME DOESNT CORRESPOND TO
THE REAL SUBPARTITION POSITION THE >A0N70B7 RETURN VALUE THEN YOULL NEED TO QUERY
310NC01NBD1?0AC8C8>=B AND FIND THE CORRECT SUBPARTITION?NAME USING THE BD1?0AC8C8>=N?> B8C8>=
COLUMN
4HERES ONE MORE CATCH THOUGH 7HILE THE ORDER?DATE PREDICATE WILL BE USED FOR PARTITION PRUNING IN
THE SOURCE DATABASE THE >A0N70B7 FUNCTION WONTTHE QUERY EXECUTION ENGINE JUST DOESNT KNOW HOW TO
USE THE >A0N70B7 PREDICATE FOR SUBPARTITION PRUNING )N OTHER WORDS THE ABOVE QUERY WILL READ ALL
SUBPARTITIONS OF A SPECIFIED RANGE PARTITION THEN THE >A0N70B7 FUNCTION WILL BE APPLIED TO EVERY ROW
FETCHED AND THE ROWS WITH NON MATCHING >A0N70B7 RESULT WILL BE THROWN AWAY 3O IF YOU HAVE
SESSIONS EACH TRYING TO READ ONE SUBPARTITION WITH THE ABOVE METHOD EACH OF THEM WOULD END UP
SCANNING THROUGH ALL SUBPARTITIONS UNDER THIS RANGE PARTITION THIS MEANS ¾ SUBPARTITION
SCANS
7E HAVE WORKED AROUND THIS PROBLEM BY CREATING VIEWS ON THE SOURCE TABLE IN THE SOURCE DATABASE
7E WOULD CREATE A VIEW FOR EACH SUBPARTITION USING A SCRIPT OF COURSE 4HE VIEW NAMES WOULD FOLLOW A
NAMING CONVENTION SUCH AS 6?&!#4?9?30 AND EACH VIEW WOULD CONTAIN THE BD1?0AC8C8>=
ghi CLAUSE IN THE VIEWS B4;42C STATEMENT 2EMEMBER THESE VIEWS WOULD BE CREATED IN THE SOURCE
DATABASE SO THERE WONT BE AN ISSUE WITH DATABASE LINKS SYNTAX RESTRICTION !ND WHEN ITS TIME TO
MIGRATE THE INSERT INTO SUBPARTITION STATEMENTS EXECUTED IN THE TARGET %XADATA DATABASE WOULD
REFERENCE APPROPRIATE VIEWS DEPENDING ON WHICH SUBPARTITION IS REQUIRED 4HIS MEANS THAT SOME LARGE
FACT TABLES WOULD HAVE THOUSANDS OF VIEWS ON THEM 4HE VIEWS MAY BE IN A SEPARATE SCHEMA AS LONG AS
#(!04%2 -)'2!4).' 4/ %8!$!4!
THE SCHEMA OWNER HAS READ RIGHTS ON THE SOURCE TABLE !LSO YOU PROBABLY DONT HAVE TO USE THIS TRICK ON
ALL PARTITIONS OF THE TABLE AS IF YOUR LARGEST TABLES ARE TIME
PARTITIONED BY SOME ORDER?DATE OR SIMILAR
THEN YOU CAN PROBABLY TRANSFER MUCH OF THE OLD PARTITIONS BEFORE THE DOWNTIME WINDOW SO YOU WONT
NEED TO USE SUCH EXTREME MEASURES
4HE TECHNIQUES JUST DISCUSSED MAY SEEM QUITE COMPLICATED AND TIME CONSUMING BUT IF YOU HAVE
TENS OR HUNDREDS OF TERABYTES OF RAW DATA TO EXTRACT TRANSFER AND COMPRESS AND ALL THIS HAS TO HAPPEN
VERY FAST SUCH MEASURES ARE GOING TO BE USEFUL 7E HAVE USED THESE TECHNIQUES FOR MIGRATING 6,$" DATA
WAREHOUSES UP TO 4" IN SIZE COMPRESSED WITH OLD
FASHIONED BLOCK COMPRESSION WITH RAW DATA SETS
EXCEEDING A QUARTER OF A PETABYTE (OWEVER WE NEED TO OPTIMIZE OUR OWN TIME TOO SO READ THE NEXT
SECTION ABOUT WHEN IT IS FEASIBLE TO GO WITH DATABASE LINKS AND WHEN IT IS NOT
When to Use CTAS or IAS over Database Links
)F YOU HAVE ALLOCATED PLENTY OF DOWNTIME AND THE DATABASE TO BE MIGRATED ISNT TOO BIG AND YOU DONT
WANT TO DO MAJOR REORGANIZATION OF YOUR DATABASE SCHEMAS THEN YOU PROBABLY DONT HAVE TO USE DATA
LOAD OVER DATABASE LINKS ! FULL $ATA0UMP EXPORTIMPORT IS MUCH EASIER IF YOU HAVE THE DOWNTIME AND
DISK SPACE AVAILABLE EVERYTHING CAN BE EXPORTED AND IMPORTED WITH A SIMPLE COMMAND
(OWEVER WHEN YOU ARE MIGRATING 6,$"S WITH LOW DOWNTIME WINDOWS DATABASE LINKS CAN PROVIDE
ONE PERFORMANCE ADVANTAGEWITH DATABASE LINKS YOU DONT HAVE TO DUMP THE DATA TO A DISK FILE JUST TO
COPY IT OVER AND RELOAD IT BACK IN THE OTHER SERVER !LSO YOU DONT NEED ANY INTERMEDIATE DISK SPACE FOR
KEEPING DUMPS WHEN USING DATABASE LINKS 7ITH DATABASE LINKS YOU READ THE DATA FROM DISK ONCE IN THE
SOURCE DATABASE TRANSFER IT OVER THE WIRE AND WRITE IT TO DISK ONCE IN THE TARGET
4RANSFERRING LOTS OF SMALL TABLES MAY ACTUALLY BE FASTER WITH THE %XPORT)MPORT OR $ATA 0UMP
METHOD !LSO THE OTHER SCHEMA OBJECTS VIEWS SEQUENCES 0,31, AND SO ON HAVE TO BE SOMEHOW
MIGRATED ANYWAY 3O ITS A GOOD IDEA TO TRANSFER ONLY THE LARGE TABLES OVER DATABASE LINKS AND USE
%XPORT)MPORT OR $ATA 0UMP FOR MIGRATING EVERYTHING ELSE !ND BY THE WAY $ATA 0UMP HAS A NICE
4G2;D34) PARAMETER YOU CAN USE FOR EXCLUDING THE LARGE MANUALLY TRANSFERRED TABLES FROM AN EXPORT JOB
What to Watch Out For When Copying Tables over Database Links
#OPYING TABLES DURING BUSINESS HOURS CAN IMPACT PERFORMANCE ON THE SOURCE DATABASE )T CAN ALSO PUT A
LOAD ON THE NETWORK SOMETIMES EVEN WHEN DEDICATED NETWORK HARDWARE IS INSTALLED FOR REDUCING THE
IMPACT OF HIGH
THROUGHPUT DATA TRANSFER DURING PRODUCTION TIME
4ABLE
SHOWS A SUMMARY OF THE CAPABILITIES OF EACH OF THESE %XTRACT AND ,OAD METHODS
Table 13-3. When to Use Extract and Load
Requirement Data Pump Export/Import Database Link
3CHEMA #HANGES 9ES 9ES 9ES
4ABLE)NDEX 3TORAGE #HANGES 9ES 9ES 9ES
4ABLESPACE #HANGES 9ES .O 9ES
#(!04%2 -)'2!4).' 4/ %8!$!4!
Requirement Data Pump Export/Import Database Link
$ATA )S %NCRYPTED 4$% 9ES .O 9ES
$ATABASE 6ERSION I .O 9ES 9ES
(YBRID #OLUMNAR #OMPRESSION 9ES .O 9ES
$IRECT PATH LOAD 9ES .O 9ES
1
It is possible to change the object creation DDL by running imp with show=y option to extract the DDL
statements, modifying the script and creating the objects manually in Sqlplus. Data Pump has made such
tasks much easier thanks to its TRANSFORM parameter.
2
Oracle doesn’t support SQL*Net connectivity between Oracle 10g (and newer) and pre-8.1.7.4 database
versions. If you attempt such connection, you will receive an error message ORA-03134: Connections to this
server version are no longer supported.
2EPLICATION
"ASED -IGRATION
'ENERALLY SPEAKING replication-based MIGRATION IS DONE BY CREATING A COPY OF THE SOURCE DATABASE AND
THEN KEEPING IT IN SYNC BY APPLYING CHANGES TO THE COPY OR hTARGET DATABASEv 4HERE ARE TWO VERY
DIFFERENT METHODS FOR APPLYING THESE CHANGES Physical replication SHIPS ARCHIVED REDO LOGS TO THE TARGET
WHERE IT IS APPLIED TO THE DATABASE USING ITS INTERNAL RECOVERY MECHANISMS 4HIS IS CALLED h2EDO !PPLYv
7ELL TALK MORE ABOUT HOW THAT WORKS IN THE h0HYSICAL -IGRATIONv SECTION LATER IN THIS CHAPTER 7ITH
logical replication, CHANGES TO THE SOURCE DATABASE ARE EXTRACTED AS 31, STATEMENTS AND EXECUTED ON THE
TARGET DATABASE 4HE TECHNIQUE OF USING 31, STATEMENTS TO REPLICATE CHANGES FROM SOURCE TO TARGET IS
CALLED SQL Apply "ACK IN THE S WHEN /RACLE AND WERE ALL THE RAGE IT WAS COMMON PRACTICE AMONG
$"!S TO USE snapshots TO KEEP TABLES IN A REMOTE DATABASE IN SYNC WITH MASTER TABLES IN THE SOURCE
DATABASE 4HESE SNAPSHOTS USED TRIGGERS TO CAPTURE CHANGES TO THE MASTER TABLE AND EXECUTE THEM ON THE
TARGET 3NAPSHOT LOGS WERE USED TO QUEUE UP THESE CHANGES WHEN THE TARGET TABLES WERE UNAVAILABLE SO
THEY COULD BE EXECUTED AT A LATER TIME )T WAS A SIMPLE FORM OF LOGICAL REPLICATION /F COURSE THIS WAS FINE
FOR A HANDFUL OF TABLES BUT IT BECAME UNWIELDY WHEN REPLICATING GROUPS OF TABLES OR ENTIRE SCHEMAS )N
LATER RELEASES /RACLE WRAPPED SOME MANAGEABILITY FEATURES AROUND THIS TECHNOLOGY AND BRANDED IT
v3IMPLE 2EPLICATIONv %VEN BACK IN THE EARLY DAYS OF DATABASE REPLICATION THERE WERE COMPANIES THAT
FIGURED OUT HOW TO MINE DATABASE REDO LOGS TO CAPTURE $-, MORE EFFICIENTLY AND LESS INTRUSIVELY THAN
TRIGGERS COULD 4ODAY THERE ARE SEVERAL PRODUCTS ON THE MARKET THAT DO A VERY GOOD JOB OF USING THIS hLOG
MININGv TECHNIQUE TO REPLICATE DATABASES 4HE ADVANTAGE LOGICAL REPLICATION HAS OVER PHYSICAL REPLICATION
IS IN ITS FLEXIBILITY &OR EXAMPLE LOGICAL REPLICATION ALLOWS THE TARGET TO BE AVAILABLE FOR READ ACCESS )N
SOME CASES IT ALSO ALLOWS YOU TO IMPLEMENT TABLE COMPRESSION AND PARTITIONING IN THE TARGET DATABASE )N
THE NEXT FEW SECTIONS WELL DISCUSS SEVERAL TOOLS THAT SUPPORT REPLICATION
BASED MIGRATION
Oracle Streams and Golden Gate
/RACLE 3TREAMS IS INCLUDED IN THE BASE 2$"-3 PRODUCT AND WAS INTRODUCED AS A NEW FEATURE IN VERSION
I 'OLDEN 'ATE IS A PRODUCT RECENTLY ACQUIRED BY /RACLE "OTH PRODUCTS REPLICATE DATA IN MUCH THE SAME
WAY ! COPY OF THE SOURCE DATABASE IS CREATED THE TARGET AND STARTED UP 4HIS MAY BE DONE FROM A FULL
DATABASE BACKUP USING 2ECOVERY -ANAGER OR BY USING $ATA 0UMP TO INSTANTIATE B4;42C SCHEMAS 4HEN
CHANGES IN THE SOURCE DATABASE ARE EXTRACTED OR MINED FROM THE REDO LOGS 4HE CHANGES ARE THEN
#(!04%2 -)'2!4).' 4/ %8!$!4!
CONVERTED TO EQUIVALENT $-, AND $$, STATEMENTS AND EXECUTED IN THE TARGET DATABASE /RACLE CALLS
THESE STEPS #APTURE 3TAGE AND !PPLY
2EGARDLESS OF WHICH PRODUCT YOU USE THE TARGET DATABASE REMAINS ONLINE AND AVAILABLE FOR
APPLICATIONS TO USE WHILE REPLICATION IS RUNNING .EW TABLES MAY BE CREATED IN THE TARGET SCHEMA OR IN
OTHER SCHEMAS !NY RESTRICTIONS ON THE TARGET ARE LIMITED TO THE TABLES BEING REPLICATED 4HIS IS
PARTICULARLY USEFUL WHEN YOUR MIGRATION STRATEGY INVOLVES CONSOLIDATIONTAKING SCHEMAS FROM MULTIPLE
SOURCE DATABASES AND CONSOLIDATING THEM INTO ONE DATABASE "ECAUSE OF ITS EXTREMELY HIGH PERFORMANCE
AND SCALABILITY MOST COMPANIES USE %XADATA FOR DATABASE CONSOLIDATION TO AT LEAST SOME DEGREE SO THIS
IS A VERY USEFUL FEATURE )F YOU READ BETWEEN THE LINES YOU REALIZE THAT THIS MEANS YOU CAN MIGRATE
MULTIPLE DATABASES CONCURRENTLY
!NOTHER CAPABILITY OF BOTH 3TREAMS AND 'OLDEN 'ATE IS THE ABILITY TO DO DATA TRANSFORMATION 4HIS IS
NOT SOMETHING WE WOULD NORMALLY ASSOCIATE WITH DATABASE MIGRATION BUT IT IS AVAILABLE SHOULD YOU NEED
IT 3INCE 3TREAMS USES 31, !PPLY TO PROPAGATE DATA CHANGES TO THE TARGET YOU CAN IMPLEMENT CHANGES TO
THE TARGET TABLES TO IMPROVE EFFICIENCIES &OR EXAMPLE YOU CAN CONVERT CONVENTIONAL TARGET TABLES TO
PARTITIONED TABLES 9OU CAN ALSO ADD OR DROP INDEXES AND CHANGE EXTENT SIZES TO OPTIMIZE FOR %XADATA "E
AWARE THAT EVEN THOUGH REPLICATION PROVIDES THIS CAPABILITY IT CAN GET MESSY )F YOU ARE PLANNING A LOT OF
CHANGES TO THE SOURCE TABLES CONSIDER IMPLEMENTING THEM BEFORE YOU BEGIN REPLICATION
4ABLES IN THE TARGET DATABASE MAY BE COMPRESSED USING %XADATA (## "E PREPARED TO TEST
PERFORMANCE WHEN INSERTING DATA AND USING (## COMPRESSION )T IS A #05
INTENSIVE PROCESS MUCH
MORE SO THAN "ASIC AND /,40 COMPRESSION (OWEVER THE REWARDS ARE PRETTY SIGNIFICANT IN TERMS OF
STORAGE SAVINGS AND QUERY PERFORMANCE DEPENDING ON THE NATURE OF THE QUERY OF COURSE )F CONVENTIONAL
INSERTS OR UPDATES ARE EXECUTED ON (##
COMPRESSED TABLES /RACLE SWITCHES TO /,40 COMPRESSION FOR
THE AFFECTED ROWS AND YOUR COMPRESSION RATIO DROPS SIGNIFICANTLY $IRECT PATH INSERTS CAN BE DONE BY
IMPLEMENTING THE INSERT APPEND HINT AS FOLLOWS
X]bTac P__T]S X]c^\hNWRRNcPQ[TbT[TRc *
X]bTac P__T]SNeP[dTb X]c^\hNWRRNcPQ[TeP[dTb<array of rows>*
.OTE THAT THE P__T]SNeP[dTb HINT ONLY WORKS FROM /RACLE G2 ONWARDS
!LSO NOTE THAT AS OF THIS WRITING 'OLDEN 'ATE DOES NOT SUPPORT reading CHANGES OF COMPRESSED
TABLES YET 4HERE ARE THREE BUGENHANCEMENT REQUESTS OUTSTANDING FOR ADDING EXTRACTION SUPPORT FOR
TABLES COMPRESSED WITH REGULAR /,40 AND (## COMPRESSION !S FAR AS writing THE DATA TO THE
REPLICATION TARGET GOES WE RAN SOME TESTS IN OUR LAB AND FOUND THAT WE WERE ABLE TO ACHIEVE THE
hREDUCEDv COMPRESSION RATIO BY USING 'OLDEN 'ATES DIRECT LOAD FEATURE )T DIDNT APPEAR TO SUPPORT THE
HIGHER LEVELS OF COMPRESSION YOU WOULD EXPECT FROM BULK INSERTS 4ABLE
SHOWS THE RESULTS OF OUR
COMPRESSION TESTS
#(!04%2 -)'2!4).' 4/ %8!$!4!
Table 13-4. Golden Gate Table Compression
Method of Table Load Table Size (MB)
5NCOMPRESSED 4ABLE BASELINE
INSERT
APPEND
SELECT
FROM x
'' DIRECT LOAD
'' DIRECT LOAD AND @!PPEND HINT ON THE INSERT INTO THE SOURCE TABLE
* Note that Golden Gate has its own meaning for the term “direct load.” In Golden Gate terminology, it
means that the initial table data copy (instantiation of a table) is done by directly inserting the extracted
rows into the target database with array insert commands. The indirect approach in Golden Gate means
that the initially extracted data is dumped to an extract file first and later loaded with other tools like
Replicat (initial-load mode) or Oracle’s SQL*Loader. This is why we didn’t see ideal compression with
Golden Gate s “direct load” method as array inserts are not as good as INSERT SELECTs for compression.
!S FOR 3TREAMS THE DOCUMENTATION SAYS THAT DIRECT PATH INSERTS ARE NOT SUPPORTED IN G DATABASES
G DOES PROVIDE LIMITED SUPPORT THROUGH THE $IRECT 0ATH !0) /NCE AGAIN MAKE SURE YOU CHECK THE
/RACLE DOCUMENTATION TO SEE WHAT IS SUPPORTED FOR YOUR DATABASE VERSION )F IN DOUBT RUN A FEW TESTS TO
SEE WHETHER YOU DO GET THE LEVEL OF COMPRESSION YOU EXPECT (OWEVER A BIG IMPROVEMENT IN /RACLE
ALLOWS 3TREAMS TO capture DATA FROM TABLES COMPRESSED WITH BASIC TABLE COMPRESSION AND /,40
COMPRESSION &ROM /RACLE PATCHSET ONWARDS 3TREAMS83TREAM CAN ALSO CAPTURE CHANGES TO
(## COMPRESSED TABLES 4HE Oracle Streams Concepts and Administration Guide FOR G 2ELEASE
EXPLAINS THIS IN MORE DETAIL 4HIS FEATURE GIVES 3TREAMS A BIG ADVANTAGE OVER 'OLDEN 'ATE if YOUR SOURCE
TABLES ARE COMPRESSED and THE COMPRESSED DATA IS UPDATED THUS REQUIRING SOME SORT OF CHANGE CAPTURE
UNLESS YOU WANT TO COPY ALL THAT DATA DURING DOWNTIME
4HE LIST OF DATA TYPES 3TREAMS DOES NOT SUPPORT IS FAIRLY SHORT IN VERSION G2 (ERE IS THE LIST FROM
THE /RACLE DOCUMENTATION
v 158;4
v A>F83
v 5SER
DEFINED TYPES INCLUDING OBJECT TYPES A45S VARRAYS AND NESTED TABLES
v G<;Ch_T STORED OBJECT RELATIONALLY OR AS BINARY 8-,
v 4HE FOLLOWING /RACLE
SUPPLIED TYPES 0]h TYPES 52) TYPES SPATIAL TYPES AND
MEDIA TYPES
!ND HERE IS THE LIST OF UNSUPPORTED DATA TYPES IN 'OLDEN 'ATE
v 158;4
v 1;>1
v 258;4
#(!04%2 -)'2!4).' 4/ %8!$!4!
v 2;>1
v =2;>1
v =CH
4HE FOLLOWING IS A QUICK SUMMARY OF THE BENEFITS COMMON TO BOTH /RACLE 3TREAMS AND 'OLDEN 'ATE
v 3UPPORT FOR FAILOVER /RACLE 2!# ENVIRONMENTS
v 3UPPORT FOR MULTI
TENANT DATABASES
v 3UPPORT FOR TABLE PARTITIONING TARGET TABLES
v 3UPPORT FOR (## COMPRESSION TARGET TABLES ONLY FOR 'OLDEN 'ATE ALTHOUGH YOU
MAY NOT GET THE BEST LOADING COMPRESSION RATIO COMPARED TO #4!3 OR )!3
v 3UPPORT FOR STORAGE CHANGES FOR TABLES AND INDEXES TARGET TABLES
v !BILITY TO CHANGE INDEXING STRATEGY TARGET TABLES
v 2EAD ACCESS TO THE TARGET TABLES DURING MIGRATION
v 3UPPORT FOR ENDIAN DIFFERENCES BETWEEN SOURCE AND TARGET
v 3UPPORT FOR CHARACTER SET DIFFERENCES BETWEEN SOURCE AND TARGET
v 3UPPORT FOR DIFFERENT DATABASE BLOCK SIZES BETWEEN SOURCE AND TARGET
v 3UPPORT FOR CHANGE #APTURE ON THE TARGET SERVER REDUCING THE PERFORMANCE
IMPACT ON THE SOURCE SYSTEM
v .OT DIFFICULT TO SETUP
v $ATABASE VERSION
AGNOSTIC
v 6ERY LITTLE DOWNTIME TO SWITCH OVER
$ISADVANTAGES COMMON TO BOTH 3TREAMS AND 'OLDEN 'ATE
v 2EPLICATED TABLES MUST BE KEYED )F THERE IS NO PRIMARY KEY ON THE SOURCE TABLE
YOU MUST PROVIDE A LIST OF COLUMNS THAT PROVIDE UNIQUENESS )F NO SUCH
COMBINATION EXISTS THE TABLE CANNOT BE REPLICATED
v 3OME DATA TYPES ARE NOT SUPPORTED FOR REPLICATION
v ,OG MINING ON THE SOURCE SYSTEM CAN IMPACT PERFORMANCE #ONSIDER MOVING THE
CHANGE #APTURE TO THE TARGET SYSTEM
v )TS MORE COMPLEX THAN OTHER MIGRATION STRATEGIES
$ISADVANTAGES OF 3TREAMS COMPARED TO 'OLDEN 'ATE
v $ATA MINING CAN HEAVILY IMPACT PERFORMANCE ON THE SOURCE DATABASE
v )T IS MORE COMPLEX TO SET UP
v $$, IS SUPPORTED BUT PROBLEMATIC
#(!04%2 -)'2!4).' 4/ %8!$!4!
v )F REPLICATION BREAKS FIXING IT CAN BE MESSY )T IS OFTEN EASIER TO SCRAP THE TARGET AND
REBUILD IT
v 31, !PPLY IS MORE PRONE TO FALLING BEHIND THE SOURCE THAN 'OLDEN 'ATE
v 3TREAMS TENDS TO REQUIRE MUCH MORE #05 RESOURCES THAN 'OLDEN 'ATE
!DVANTAGES OF 3TREAMS OVER 'OLDEN 'ATE
v 3TREAMS IS INCLUDED WITH YOUR DATABASE LICENSE WHILE 'OLDEN 'ATE IS SOLD
SEPARATELY
v )F THE SOURCE DATABASE IS /RACLE OR HIGHER 3TREAMS SUPPORTS CAPTURING
CHANGES TO COMPRESSED SOURCE TABLES BASIC AND /,40 COMPRESSION )F BOTH THE
SOURCE AND TARGET DATABASES ARE RUNNING /RACLE VERSION OR HIGHER THEN THE
3TREAMS SUPPORTS CAPTURING CHANGES TO (## COMPRESSED TABLES TOO
v !LL PROCESS INFORMATION IS STORED IN THE DATABASE PROVIDING CONVENIENT REMOTE
ACCESS
v 4HERE IS INTEGRATION WITH 'RID #ONTROL
3O WHICH SHOULD YOU USE 3TREAMS OR 'OLDEN 'ATE 7EVE SPOKEN WITH CLIENTS AND COLLEAGUES WHO
HAVE USED BOTH PRODUCTS )N EVERY CASE THEY ARE USING THE PRODUCT FOR SIMPLE REPLICATION TO A REMOTE SITE
MUCH LIKE YOU WOULD FOR DATABASE MIGRATION 4HE FEEDBACK WE GET IS OVERWHELMINGLY IN FAVOR OF 'OLDEN
'ATE 4HESE USERS CITE 'OLDEN 'ATES EASE OF USE STABILITY AND PERFORMANCE AS THE MOST IMPORTANT
REASONS )N ADDITION TO THIS 'OLDEN 'ATE SUPPORTS NON
/RACLE DATABASE SOURCES 3TREAMS RUNS INSIDE THE
/RACLE DATABASE AND HAS NOT BEEN PORTED TO ANY OTHER DATABASE PLATFORM !S MENTIONED 3TREAMS IS
INCLUDED IN YOUR /RACLE DATABASE LICENSE 'OLDEN 'ATE IS NOT AND THE COST BASED ON THE TOTAL NUMBER OF
CORES ON THE SOURCE AND TARGET SERVERS IS NOT TRIVIAL )F YOU HAVE NEED FOR DATA REPLICATION BEYOND YOUR
%XADATA MIGRATION THEN THE INVESTMENT MAY BE WELL WORTH IT )F YOU ARE FACED WITH A ONE
TIME MIGRATION
3TREAMS MAY BE A BETTER FIT FOR COST
7HEN TO 5SE 3TREAMS OR 'OLDEN 'ATE
3TREAMS AND 'OLDEN 'ATE BOTH PROVIDE A LOT OF FLEXIBILITY WITH NEAR
ZERO DOWNTIME 4HEY ARE WELL SUITED
FOR MIGRATIONS THAT REQUIRE CHANGES TO THE TARGET TABLES 4HIS INCLUDES RESIZING TABLE AND INDEX EXTENTS
IMPLEMENTING PARTITIONING STRATEGIES AND COMPRESSING TABLES 4HEY ALSO ALLOW YOU TO CHANGE SCHEMAS
TABLESPACES AND DISK GROUPS AMONG OTHER THINGS
7HAT TO 7ATCH /UT FOR WITH 3TREAMS AND 'OLDEN 'ATE
3INCE 3TREAMS AND 'OLDEN 'ATE PROPAGATE DATABASE CHANGES BY MINING THE REDO LOGS YOU MUST ACCOUNT
FOR =>;>668=6 OPERATIONS 4HIS CAN BE DONE AT THE TABLESPACE LEVEL USING P[cTacPQ[Tb_PRTU^aRT
[^VVX]V OR AT THE DATABASE LEVEL USING P[cTaSPcPQPbTU^aRT[^VVX]V
(ERE ARE A FEW EXAMPLES OF =>;>668=6 OPERATIONS
v 2A40C4C01;4]TfcPQ=>;>668=60BbT[TRcUa^\q
v INSERT
APPEND
x FOR TABLES IN NOLOGGING MODE
v $IRECT PATH 31,
,OADER FOR TABLES IN NOLOGGING MODE
#(!04%2 -)'2!4).' 4/ %8!$!4!
.OTE THAT SOME LARGE $7 DATABASES ARE DESIGNED TO USE =>;>668=6 FOR LARGE DATA LOADS TO SAVE #05
AND DISK SPACE FEWER ARCHIVE LOGS GENERATED 4HESE DATABASES RELY ON INCREMENTAL BACKUPS PLUS DATA
RELOADS AS A RECOVERY STRATEGY )N SUCH DATABASES FORCING REDO LOGGING FOR EVERYTHING MAY CAUSE
UNACCEPTABLE PERFORMANCE DEGRADATION AND ARCHIVE LOG GENERATION )N THOSE CASES IT MAKES SENSE TO
CONFIGURE THE %4, ENGINE TO PERFORM LOADS TO BOTH THE OLD AND NEW $7 SYSTEM DURING THE TRANSITION
PERIOD
3EQUENCES ARE NOT REPLICATED 4HEY MUST BE DROPPED FROM THE TARGET AND RECREATED FROM THE SOURCE
DATABASE DURING THE FINAL CUT
OVER 4RIGGERS ON THE TARGET TABLES MUST BE DISABLED DURING REPLICATION AND
RE
ENABLED DURING THE FINAL CUT
OVER AFTER REPLICATION HAS BEEN SHUT OFF
Logical Standby
!NOTHER TYPE OF REPLICATION USEFUL FOR DATABASE MIGRATION IS logical standby ,OGICAL STANDBY IS A BUILT
IN
FEATURE OF THE /RACLE DATABASE )T WAS INTRODUCED IN VERSION I RELEASE AS AN EXTENSION OR ADDITIONAL
FEATURE OF $ATA 'UARD FORMERLY 0HYSICAL 3TANDBY /N THE SURFACE THIS STRATEGY SOUNDS A LOT LIKE 3TREAMS
AND 'OLDEN 'ATE IN THAT IT IS BASICALLY DOING THE SAME THING #HANGES TO THE SOURCE DATABASE ARE MINED
FROM THE REDO OR ARCHIVED REDO AND CONVERTED TO 31, STATEMENTS AND EXECUTED ON THE TARGET DATABASE
!S IN OTHER LOGICAL REPLICATION METHODS THE DATABASE IS OPEN AND AVAILABLE WHILE REPLICATION IS RUNNING
,OGICAL STANDBY IS FAR MORE RESTRICTIVE THAN 3TREAMS OR 'OLDEN 'ATE "ECAUSE THE LOGICAL STANDBY IS
ACTUALLY INSTANTIATED FROM A PHYSICAL BLOCK
FOR
BLOCK COPY OF THE SOURCE DATABASE SEVERAL RULES COME
INTO PLAY 4HE DATABASE VERSION MUST BE THE SAME FOR THE SOURCE AND THE TARGET DATABASES EVEN DOWN TO
THE PATCH LEVEL 7ITH FEW EXCEPTIONS SOURCE AND TARGET PLATFORMS MUST ALSO MATCH AT LEAST WITHIN THE
PROCESSOR ARCHITECTURE &OR EXAMPLE LOGICAL STANDBY DOES SUPPORT A SOURCE DATABASE RUNNING ON AN
!-$ PROCESSOR ARCHITECTURE WHILE THE TARGET DATABASE %XADATA IN OUR CASE RUNS ON AN )NTEL %-4
PROCESSOR 4HE OPERATING SYSTEM FOR BOTH SOURCE AND TARGET MUST BE THE SAME ALTHOUGH DIFFERENT
DISTRIBUTIONS OF ,INUX WILL WORK &OR EXAMPLE YOU COULD BE RUNNING 3USE ,INUX ON THE SOURCE SYSTEM BUT
/RACLE %NTERPRISE ,INUX ON THE TARGET AS LONG AS THE ,INUX KERNEL IS THE SAME
)T MIGHT SURPRISE YOU TO KNOW THAT WHILE INDEXES AND MATERIALIZED VIEWS CAN BE CREATED IN THE TARGET
DATABASE YOU CANNOT USE LOGICAL STANDBY TO IMPLEMENT A PARTITIONING STRATEGY THERE &OR EXAMPLE YOU
CAN CREATE OR DROP INDEXES ON A TARGET TABLE BUT YOU CANNOT PAUSE REPLICATION DROP THE TABLE RECREATE IT
AS A PARTITIONED TABLE RELOAD IT AND RESUME REPLICATION 4ABLE COMPRESSION IS NOT SUPPORTED EITHER 7E
TRIED WORKING AROUND THAT BY COMPRESSING THE TABLE AFTER CONVERTING THE TARGET DATABASE FROM A PHYSICAL
TO A LOGICAL STANDBY BUT THAT DOESNT WORK 2EMEMBER THAT (## COMPRESSION ONLY WORKS WITH DIRECT
PATH LOAD AND LOGICAL STANDBY APPLIES CHANGES USING CONVENTIONAL INSERTS #OMPRESSION AND TABLE
PARTITIONING ARE VERY IMPORTANT PERFORMANCE STRATEGIES FOR %XADATA 4HE LACK OF SUPPORT FOR THESE
FEATURES IS A SIGNIFICANT DOWNSIDE FOR LOGICAL STANDBY
4HE LIST OF UNSUPPORTED DATA TYPES IS THE SAME FOR LOGICAL STANDBY AS IT IS FOR 3TREAMS AND 'OLDEN
'ATE
v 158;4
v A>F83 DA>F83
v 5SER
DEFINED TYPES
v #OLLECTIONS INCLUDING E0AA0HB AND NESTED TABLES
v G<;Ch_T STORED OBJECT RELATIONALLY OR AS BINARY 8-,
v -ULTIMEDIA DATA TYPES INCLUDING 3PATIAL )MAGE AND /RACLE 4EXT
#(!04%2 -)'2!4).' 4/ %8!$!4!
4HERE ARE A LOT OF STEPS IN CONFIGURING AND INSTANTIATING A LOGICAL STANDBY AND WE WONT GO INTO ALL THE
DETAILS HERE "UT TO GIVE YOU AN IDEA OF HOW A LOGICAL STANDBY IS SET UP HERE ARE THE HIGH
LEVEL STEPS
#REATE A PHYSICAL STANDBY DATABASE WELL TALK ABOUT THIS IN THE h0HYSICAL
3TANDBYv SECTION
3TOP REDO APPLY ON THE PHYSICAL STANDBY DATABASE
#ONFIGURE ,OG -INER IN THE 0RIMARY DATABASE
#ONVERT THE PHYSICAL STANDBY DATABASE TO A LOGICAL STANDBY
/PEN THE LOGICAL STANDBY DATABASE AND RESTART REDO APPLY
6ERIFY THAT THE LOGICAL STANDBY DATABASE IS APPLYING CHANGES FROM THE SOURCE
,OGICAL STANDBY USES 31, !PPLY TECHNOLOGY TO REPLICATE CHANGES FROM THE SOURCE DATABASE TO THE
TARGET !S MENTIONED EARLIER CHANGES IN THE SOURCE DATABASE ARE CONVERTED TO 31, STATEMENTS AND
SHIPPED TO THE TARGET FOR EXECUTION !LTHOUGH THIS MAY SOUND FAIRLY STRAIGHTFORWARD THERE ARE SOME
INTERESTING IMPLICATIONS BECAUSE OF HOW IT IS IMPLEMENTED (ERE ARE A FEW THINGS TO KEEP IN MIND
#ONSIDERATIONS FOR $-, 3TATEMENTS
v "ATCH UPDATES ARE EXECUTED ONE ROW AT A TIME
v $IRECT PATH INSERTS ARE PERFORMED IN CONVENTIONAL MANNER
v 0ARALLEL $-, IS NOT EXECUTED IN PARALLEL
#ONSIDERATIONS FOR $$, 3TATEMENTS
v 0ARALLEL $$, IS NOT EXECUTED IN PARALLEL
v #4!3 IS EXECUTED AS CREATE TABLE INSERT INSERT INSERT x
7HEN TO 5SE ,OGICAL 3TANDBY
,OGICAL STANDBY HAS A VERY NARROW LIST OF BENEFITS THAT SET IT APART FROM OTHER MORE LIKELY MIGRATION
METHODS )T PROVIDES FULL DATABASE REPLICATION AND ALLOWS READ ACCESS TO THE STANDBY DATABASE DURING
REPLICATION !S SUCH THE STANDBY MAY BE USED TO OFFLOAD RESOURCE
INTENSIVE REPORTING OPERATIONS FROM
THE PRODUCTION DATABASE UNTIL THE TIME YOU ARE READY TO MAKE THE FINAL SWITCH 4ABLES AND MATERIALIZED
VIEWS MAY BE CREATED ON THE STANDBY DATABASE AND INDEXES MAY BE CREATED OR DROPPED TO IMPROVE
PERFORMANCE
7HAT TO 7ATCH /UT FOR WITH ,OGICAL 3TANDBY
=>;>668=6 OPERATIONS DO NOT GENERATE REDO WHICH MEANS THEY WILL NOT BE REPLICATED TO THE STANDBY
DATABASE "E SURE TO TURN OFF =>;>668=6 OPERATIONS USING THE P[cTaSPcPQPbTU^aRT[^VVX]V COMMAND IN
31,
0LUS #ONSIDERING ITS LACK OF SUPPORT FOR MODIFYING STORAGE OF TARGET TABLES COMPRESSION AND
PARTITIONING YOU MIGHT OPT TO USE THE PHYSICAL STANDBY APPROACH INSTEAD )F YOU DONT REQUIRE FULL
DATABASE REPLICATION 3TREAMS AND 'OLDEN 'ATE ARE EXCELLENT ALTERNATIVES PROVIDING SUPPORT FOR MANY OF
THE CAPABILITIES LACKING IN A LOGICAL STANDBY DATABASE
#(!04%2 -)'2!4).' 4/ %8!$!4!
Parallel Load
4HIS IS A BIT OF hROLL YOUR OWNv METHOD FOR MIGRATION )N THIS STRATEGY DATA CHANGES ARE FED TO THE DATABASE
THROUGH AN IN
HOUSE DEVELOPED APPLICATION !S SUCH IT IS A SMALL TASK TO CONFIGURE THE APPLICATION TO
UPDATE %XADATA TABLES IN PARALLEL WITH THE CURRENT PRODUCTION DATABASE (ISTORICAL DATA IS THEN MIGRATED
OVER TO %XADATA DURING NORMAL BUSINESS HOURS $EPENDING ON THE VOLUME THIS MAY TAKE SEVERAL DAYS TO
SEVERAL WEEKS /NCE ALL TABLES HAVE BEEN MIGRATED TO %XADATA THE FEED TO THE OLD PRODUCTION SYSTEM IS
CUT AND %XADATA BECOMES THE NEW PRODUCTION DATABASE
Non-Oracle Databases
#LEARLY THE LIST OF OPTIONS WILL BE MUCH SMALLER FOR MIGRATING NON
/RACLE DATABASES /NE OF OUR CLIENTS IS
CURRENTLY MIGRATING THIRTY OR SO 31, 3ERVER DATABASES TO %XADATA 4O BEGIN WITH THIS TYPE OF MIGRATION
TAKES US DOWN THE LOGICAL MIGRATION PATH ESSENTIALLY EXTRACTING EVERY OBJECT AND ROW OF DATA FROM THE
SOURCE DATABASE AND INSERTING IT INTO THE TARGET DATABASE )F THE SOURCE DATABASE INCLUDES CODE OBJECTS
SUCH AS STORED PROCEDURES FUNCTIONS AND TRIGGERS THEY WILL NEED TO BE CONVERTED TO 0,31, OR REWRITTEN
COMPLETELY
#HANGING DATABASE VENDORS CAN BE A MESSY AND COMPLICATED PROCESS 4HERE IS A LOT TO CONSIDER
PLAN AND TEST /RACLE AND OTHER VENDORS HAVE DONE A PRETTY GOOD JOB OF PROVIDING TOOLS THAT AUTOMATE
THE PROCESS TO SOME DEGREE -IGRATING FRONT
END APPLICATIONS TO AN /RACLE DATABASE IS A MUCH LONGER
DISCUSSION LARGELY BECAUSE EACH DATABASE VENDOR HAS IMPLEMENTED LOCKING AND CONCURRENCY CONTROL
DIFFERENTLY !N /RACLE DATABASE TENDS TO DO MUCH LESS LOCKING THAN MANY OF ITS COMPETITORS 4HIS IS A
GOOD THING BECAUSE LESS AGGRESSIVE LOCKING MEANS MORE CONCURRENCY AND MORE CONCURRENCY TRANSLATES
TO BETTER SCALABILITY (OWEVER BECAUSE OF THIS TENDENCY TO hUNDER
LOCK v APPLICATIONS WRITTEN FOR NON
/RACLE DATABASES SHOULD BE CLOSELY REVIEWED TO ENSURE THAT THEY ADEQUATELY LOCK ROWS WHERE SERIALIZATION
IS REQUIRED $EVELOPMENT BEST PRACTICES ARE BEYOND THE SCOPE OF THIS BOOK BUT ARE WELL DOCUMENTED
4OM +YTES BOOK Expert Oracle Database Architecture !PRESS IS AN EXCELLENT RESOURCE FOR $"!S
AND DEVELOPERS MAKING THE SWITCH TO /RACLE "EWARE THE CONCEPT OF hDATABASE AGNOSTICv APPLICATIONS IT
IS A MYTH )F YOU ARE MIGRATING A FRONT
END APPLICATION ALONG WITH YOUR DATABASE THE CODE SHOULD BE
REVIEWED AND TESTED THOROUGHLY /RACLES -IGRATION 7ORKBENCH CAN TAKE SOME OF THE PAIN OUT OF THE
PROCESS OF CONVERTING YOUR NON
/RACLE DATABASES TO %XADATA "ETTER YET IT IS FREE AND FULLY SUPPORTED !S
OF THIS WRITING THE DATABASES IT CAN MIGRATE ARE THESE
v -ICROSOFT 31, 3ERVER
v )"- $"
v 3YBASE !DAPTIVE 3ERVER
v 4ERADATA
v -ICROSOFT !CCESS
v )NFORMIX
v -931,
-IGRATION 7ORKBENCH EXTRACTS SOURCE DATABASE OBJECTS STORES THEM IN ITS REPOSITORY AND CONVERTS
THEM TO /RACLE
COMPATIBLE OBJECTS 3UPPORTED OBJECTS ARE TABLES VIEWS CONSTRAINTS PRIMARY KEYS
FOREIGN KEYS INDEXES STORED PROCEDURES FUNCTIONS TRIGGERS 31, 3ERVER IDENTITY COLUMNS TEMPORARY
OBJECTS USERS USER
DEFINED TYPES AND MORE &OR EXAMPLE E0A270A FIELDS WITH A LENGTH GREATER THAN
CHARACTERS ARE CONVERTED TO #,/" IN /RACLE $ATA TYPES THAT /RACLE DOES NOT DIRECTLY SUPPORT ARE
#(!04%2 -)'2!4).' 4/ %8!$!4!
CONVERTED TO DATA TYPES WITH SIMILAR CHARACTERISTICS &OR EXAMPLE THE <4<> DATA TYPE WILL BE CONVERTED TO
AN /RACLE #,/" DATA TYPE /RACLE HANDLES THE NOTORIOUS 834=C8CH DATA TYPE IN 31, 3ERVER BY CONVERTING
IT TO A =D<14A COLUMN ON THE /RACLE SIDE AND CREATING A CORRESPONDING INSERT TRIGGER AND SEQUENCE 3OME
OBJECTS AND SYNTAX CANNOT BE CONVERTED AUTOMATICALLY &OR THOSE OBJECTS -IGRATION 7ORKBENCH ANALYZES
AND REPORTS TASKS THAT ONCE REQUIRED HUMAN INTERVENTION #ONVERTING CODE AND DATA TYPES MANUALLY IS A
TEDIOUS TIME
CONSUMING AND ERROR PRONE TASK "E SURE TO ALLOW ENOUGH TIME AND TEST TEST TEST )F YOUR
DATABASE STORES DATA STRUCTURES ONLY SUCH AS TABLES AND INDEXES OTHER TOOLS LIKE 'OLDEN 'ATE CAN BE
EFFECTIVELY USED FOR MIGRATING YOUR DATABASE TO %XADATA
7HEN TO 5SE -IGRATION 7ORKBENCH
-IGRATION 7ORKBENCH IS A VERY USEFUL TOOL FOR REDUCING THE TIME AND EFFORT INVOLVED IN CROSS
VENDOR
DATABASE MIGRATION 4HIS IS ESPECIALLY TRUE WHEN THERE IS A SUBSTANTIAL AMOUNT OF PROCEDURAL CODE INSIDE
THE DATABASE TO BE MIGRATED )NSTEAD OF SPENDING HOURS IN THE BOOKS TRYING TO FIGURE OUT WHICH /RACLE
DATA TYPES MAP BEST TO THE SOURCE DATABASE -IGRATION 7ORKBENCH GENERATES THESE MAPPINGS
AUTOMATICALLY 7EVE ALSO FOUND IT TO BE A REAL TIME SAVER FOR $"!S AND DEVELOPERS WHO NEED TO LEARN
HOW BUILT
IN DATA TYPES PACKAGES AND FUNCTIONS TRANSLATE TO 0,31,
7HAT TO 7ATCH /UT FOR 7HEN -IGRATING FROM NON
/RACLE $ATABASES
$O NOT UNDERESTIMATE THE DIFFICULTY OF CROSS
VENDOR DATABASE MIGRATION %VEN AMONG THE 31, FAMILY OF
DATABASES ON THE MARKET THERE ARE SIGNIFICANT DIFFERENCES IN IMPLEMENTATION 9OU WILL NEED A THOROUGH
UNDERSTANDING OF HOW THINGS LIKE READ CONSISTENCY AND LOCKING ARE DONE IN YOUR SOURCE DATABASE AS WELL
AS HOW THEY ARE DONE IN /RACLE 4HE TIME AND EFFORT REQUIRED FOR CROSS
VENDOR DATABASE MIGRATION IS
DIRECTLY TIED TO THE COMPLEXITY OF YOUR DATABASE AND THE AMOUNT OF SOURCE CODE FOR PACKAGES PROCEDURES
AND FUNCTIONS THAT MUST BE CONVERTED )F USER
DEFINED DATA TYPES ARE USED IN THE SOURCE DATABASE THEY
WILL NEED TO BE CONVERTED MANUALLY
,OGICAL -IGRATION 7RAP 5P
)N MOST CASES LOGICAL MIGRATION PROVIDES THE MOST FLEXIBILITY FOR RECONFIGURING EXTENT SIZES IMPLEMENTING
OR MODIFYING PARTITIONING STRATEGIES AND COMPRESSING TABLES USING (## 4HIS FLEXIBILITY COMES AT THE COST
OF COMPLEXITY AND RESTRICTIONS THAT SHOULD BE WEIGHED AND MEASURED AGAINST THE SIMPLICITY OF SOME OF
THE PHYSICAL MIGRATION STRATEGIES PRESENTED IN THIS CHAPTER !T THE END OF THE DAY YOU WILL WANT TO
PROPERLY SIZE EXTENTS FOR LARGE TABLES IMPLEMENT A GOOD PARTITIONING STRATEGY AND COMPRESS READ
ONLY
TABLES USING (## COMPRESSION 4HIS CAN BE DONE BEFORE YOU SWITCH OVER TO %XADATA USING MOST OF THE
LOGICAL MIGRATION STRATEGIES WEVE DISCUSSED HERE
Physical Migration
0HYSICAL DATABASE MIGRATION AS THE NAME IMPLIES IS THE PROCESS OF CREATING A BLOCK
FOR
BLOCK COPY OF THE
SOURCE DATABASE OR PARTS OF THE DATABASE AND MOVING IT TO %XADATA 0HYSICAL MIGRATION IS A MUCH
SIMPLER PROCESS THAN SOME OF THE LOGICAL MIGRATION STRATEGIES DISCUSSED EARLIER IN THIS CHAPTER !S YOU
MIGHT EXPECT IT DOES NOT ALLOW FOR ANY CHANGES TO BE MADE TO THE TARGET DATABASE OTHER THAN CHOOSING
NOT TO MIGRATE SOME UNNECESSARY TABLESPACES 4HIS MEANS THAT YOU WILL NOT BE ABLE TO MODIFY EXTENT
SIZES FOR TABLES AND INDEXES ALTER YOUR INDEXING STRATEGY IMPLEMENT PARTITIONING OR APPLY (## TABLE
COMPRESSION !LL THESE TASKS MUST BE DONE POST
MIGRATION (OWEVER PHYSICAL MIGRATION IS HANDS DOWN
THE FASTEST WAY TO MIGRATE YOUR DATABASE TO %XADATA &OR ALL PHYSICAL MIGRATION STRATEGIES EXCEPT
#(!04%2 -)'2!4).' 4/ %8!$!4!
4RANSPORTABLE 4ABLESPACES 443 THE NEW %XADATA DATABASE STARTS OUT AS A SINGLE INSTANCE DATABASE
0OST MIGRATION STEPS ARE NEEDED TO REGISTER THE DATABASE AND ALL ITS INSTANCES WITH #LUSTER 2EADY 3ERVICES
'RID )NFRASTRUCTURE "ECAUSE PHYSICAL MIGRATION CREATES AN EXACT COPY OF THE SOURCE DATABASE THE LIST OF
RESTRICTIONS IS VERY SHORT
v 4HE SOURCE $ATABASE VERSION MUST BE
4HIS ASSUMES THAT YOU WILL BE MIGRATING TO %XADATA 6 OR 8 THE OLD %XADATA 6
SUPPORTS /RACLE AS WELL
/NE OPTION IS TO UPGRADE THE PRODUCTION DATABASE TO VERSION BEFORE COPYING IT
TO %XADATA 5PGRADING IS USUALLY A QUICK PROCESS AND IS NOT DEPENDENT ON THE SIZE
OF THE DATABASE TABLES BUT RATHER ON THE NUMBER OF OBJECTS IN THE DATABASE 4HE
POST UPGRADE RECOMPILATION OF 0,31, PACKAGES AND VIEWS MAY TAKE A WHILE
HOWEVER
v 4HE SOURCE PLATFORM MUST BE CERTIFIED FOR RUNNING /RACLE /%, 2(%,
3,%3 OR OTHER CERTIFIED ,INUX VERSIONS
v 4HE SOURCE PLATFORM MUST BE LITTLE ENDIAN !3- REBALANCE AND CROSS PLATFORM
TRANSPORTABLE DATABASE ONLY 7HEN USING A HYBRID SOLUTION CREATING A NEW
DATABASE ON %XADATA AND USING CROSS PLATFORM TRANSPORTABLE TABLESPACES THE
SOURCE DATABASE CAN BE ON DIFFERENT PLATFORMS WITH DIFFERENT ENDIANNESS
SUPPORTED PLATFORMS ARE LISTED IN ecaP]b_^acPQ[TN_[PcU^a\ VIEW
4HERE ARE THREE STRATEGIES FOR PERFORMING A PHYSICAL MIGRATION BACKUP AND RESTORE PHYSICAL
STANDBY AND !3- REBALANCE )N THIS SECTION WELL DISCUSS THESE STRATEGIES HOW THEY WORK AND WHAT THEY
ARE BEST SUITED FOR
Backup and Restore
4HE BACKUP AND RESTORE STRATEGY USES /RACLES 2ECOVERY -ANAGER 2-!. TO CREATE A FULL BACKUP OF THE
SOURCE DATABASE AND THEN RESTORE IT TO THE %XADATA PLATFORM 5NLESS YOU PLAN TO SHUT DOWN THE DATABASE
DURING THIS PROCESS THE SOURCE DATABASE MUST BE RUNNING IN !RCHIVELOG MODE 4HE BACKUP AND RESTORE
PROCESS CAN BE DONE IN ONE PASS USING A FULL BACKUP 4HIS IS THE BEST WAY TO MOVE SMALLER DATABASES TO
%XADATA SINCE SMALLER DATABASES TAKE MUCH LESS TIME ,ARGER DATABASES MAY TAKE HOURS TO BACKUP AND
RESTORE &OR THESE DATABASES THE PROCESS CAN BE DONE IN TWO PASSES BY TAKING A FULL BACKUP FOLLOWED BY AN
INCREMENTAL BACKUP
&ULL "ACKUP AND 2ESTORE
2-!. IS USED TO CREATE A FULL BACKUP OF THE DATABASE TO BE MIGRATED 4HE BACKUP FILES ARE STAGED IN A FILE
SYSTEM OR TAPE LIBRARY ACCESSIBLE TO %XADATA 4HE BACKUP IS THEN RESTORED TO THE %XADATA PLATFORM )F THE
SOURCE DATABASE IS ON A BIG
ENDIAN PLATFORM THE DATAFILES CAN BE CONVERTED DURING BACKUP OR RESTORE
PROCESS USING 2-!.S R^]eTacSPcPQPbT COMMAND 4HE BASIC STEPS ARE
1. 0ERFORM PRE
MIGRATION TASKS
a. #REATE AN ENTRY IN THE c]b]P\Tb ^aP FILE FOR YOUR DATABASE ON %XADATA
OPTIONAL
b. #OPY THE PASSWORD FILE AND PARAMETER FILE X]Xc ^aP OR SPFILE TO %XADATA
#(!04%2 -)'2!4).' 4/ %8!$!4!
2. 2ESTRICT USER AND APPLICATION ACCESS TO THE DATABASE
3. 4AKE A FULL DATABASE BACKUP OPTIONALLY CONVERT TO LITTLE
ENDIAN PLATFORM
4. #OPY THE FILES TO %XADATA NOT REQUIRED IF BACKUP RESIDES ON A SHARED FILE
SYSTEM OR TAPE
5. /N %XADATA STARTUP THE DATABASE IN NOMOUNT MODE
6. 2ESTORE THE CONTROL FILE OPTIONALLY CONVERT TO ,INUX X
BIT PLATFORM
7. -OUNT THE DATABASE
8. 2ESTORE THE DATABASE OPTIONALLY CONVERT TO ,INUX X
BIT PLATFORM
9. 2ECOVER THE DATABASE .OTE THAT YOU CANNOT recover A DATABASE WITH ARCHIVE
AND REDO LOGS FROM THE SOURCE DATABASE IF THE SOURCE DATABASE IS ON A DIFFERENT
PLATFORM THAN THE TARGET DATABASE 4HIS MEANS THAT YOU CANT REDUCE THE
MIGRATION DOWNTIME BY RESTORING AND ROLLING FORWARD AN OLD PRODUCTION
BACKUP IF THE SOURCE DATABASE IS RUNNING ON A DIFFERENT PLATFORM THAN THE
TARGET )N SUCH A CASE THE WRITABLE SOURCE DATABASE FILES SHOULD BE
RESTOREDCOPIED ONLY WHEN THEY ARE IN THEIR FINAL STATE THAT IS THE SOURCE
DATABASE IS ALREADY SHUT DOWN 2EAD
ONLY TABLESPACE FILES CAN STILL BE
COPIEDRESTORED IN ADVANCE
10. 0ERFORM POST
MIGRATION TASKS
a. #ONVERT THE DATABASE TO 2!# AND CREATE SERVICE NAMES
b. 2ECONFIGURE CLIENT c]b]P\Tb ^aP FILES CONFIGURATION FILES FOR CONNECTING TO
THE %XADATA DATABASE
11. -AKE THE DATABASE AVAILABLE TO USERS AND APPLICATIONS
!S YOU CAN SEE THIS IS A FAIRLY STRAIGHTFORWARD PROCESS !FTER FOLLOWING THESE STEPS YOU HAVE AN EXACT
COPY OF YOUR PRODUCTION DATABASE RUNNING ON %XADATA 4HERE ARE A FEW THINGS TO KEEP IN MIND THOUGH )F
YOUR SOURCE DATABASE USES FILE SYSTEMS FOR DATABASE STORAGE OR IF YOU ARE USING !3- BUT THE DISK GROUP
NAMES ARE DIFFERENT ON %XADATA YOU MAY REDIRECT THE RESTORED FILES TO THE NEW DISK GROUP NAMES BY
CHANGING THE SQNRaTPcTNUX[TNSTbc AND SQNaTR^eTahNUX[TNSTbc PARAMETERS IN THE X]Xc ^aP FILE BEFORE
STARTING THE RESTORE PROCESS 4ABLE
SHOWS HOW THESE PARAMETERS CAN BE USED TO REMAP DISK GROUP
NAMES FROM THE SOURCE DATABASE TO %XADATA
Table 13-5. Remap Disk Groups
Init.ora Parameters Source Database Exadata Database
SQNRaTPcTNUX[TNSTbc30C0N58;4B 30C0
SQNaTR^eTahNUX[TNSTbcA42>E4AHN58;4B A42>
)F YOUR DATABASE USES MULTIPLE !3- DISK GROUPS TO STORE YOUR DATABASE FILES USE THE 2-!.
SQNUX[TN]P\TNR^]eTac COMMAND TO REMAP THE FILE NAMES TO !3- DISK GROUPS &OR EXAMPLE
#(!04%2 -)'2!4).' 4/ %8!$!4!
SQNUX[TN]P\TNR^]eTac,K
30C0N58;4B TgSQ SPcPUX[T bhbcT\ &"& &!(&!"%((30C0
SQNUX[TN]P\TNR^]eTac,K
30C0N58;4B TgSQ SPcPUX[T bhbPdg &#! &!(&!"& 30C0
)NCREMENTAL "ACKUP
)F THE DATABASE YOU NEED TO MIGRATE IS LARGE AND THE TIME TO MIGRATE IS LIMITED YOU MIGHT CONSIDER USING
THE INCREMENTAL BACKUP AND RESTORE STRATEGY TO REDUCE DOWNTIME FOR THE FINAL SWITCH TO %XADATA !LAS THE
INCREMENTAL METHOD does not SUPPORT ENDIAN FORMAT CONVERSION (ERE ARE THE BASIC STEPS
1. 0ERFORM PRE
MIGRATION TASKS
a. #REATE AN ENTRY IN THE c]b]P\Tb ^aP FILE FOR YOUR DATABASE ON %XADATA
OPTIONAL
b. #OPY THE PASSWORD FILE AND PARAMETER FILE X]Xc ^aP OR SPFILE TO %XADATA
2. -AKE A LEVEL BACKUP OF THE DATABASE
3. #OPY THE BACKUP FILES TO %XADATA NOT REQUIRED IF BACKUP RESIDES ON A SHARED
FILE SYSTEM OR TAPE
4. /N %XADATA START UP THE DATABASE IN NOMOUNT MODE
5. 2ESTORE THE CONTROL FILE
6. -OUNT THE DATABASE
7. 2ESTORE THE DATABASE
8. 2ESTRICT USER AND APPLICATION ACCESS TO THE DATABASE
9. -AKE AN INCREMENTAL LEVEL BACKUP OF THE DATABASE
10. #OPY THE INCREMENTAL BACKUP FILES TO %XADATA
11. 2ECOVER THE DATABASE APPLYING INCREMENTAL BACKUP AND ARCHIVED REDO LOGS
12. 0ERFORM POST
MIGRATION TASKS
a. #ONVERT THE DATABASE TO 2!# AND CREATE SERVICE NAMES
b. 2ECONFIGURE CLIENT c]b]P\Tb ^aP FILES
13. -AKE THE DATABASE AVAILABLE TO USERS AND APPLICATIONS
9OU MAY NOTICE THAT THE STEPS FOR THE INCREMENTAL BACKUP METHOD ARE ALMOST IDENTICAL TO THE FULL
BACKUP METHOD 4HE IMPORTANT DIFFERENCE IS THE DOWNTIME REQUIRED 4HE BULK OF THE TIME FOR MIGRATING
THE DATABASE IS IN THE FULL BACKUP ITSELF 4HIS COULD TAKE HOURS TO COMPLETE AND WITH THE FULL BACKUP
METHOD THAT IS ALL DATABASE DOWNTIME 4HE INCREMENTAL BACKUP METHOD USES A LEVEL BACKUP INSTEAD OF
A FULL BACKUP )N ACTUALITY THE LEVEL BACKUP IS THE SAME AS THE FULL BACKUP EXCEPT THAT IT HAS SPECIAL
PROPERTIES THAT ALLOW IT TO BE USED AS A BASELINE TO WHICH INCREMENTAL BACKUPS CAN BE APPLIED 3O USING
THE INCREMENTAL METHOD THE DATABASE REMAINS ONLINE FOR USERS DURING THE LONGEST PART OF THE MIGRATION
"LOCK CHANGE TRACKING MUST BE ACTIVATED ON THE SOURCE DATABASE before THE INCREMENTAL LEVEL IS
TAKEN 7HEN BLOCK CHANGE TRACKING IS TURNED ON /RACLE KEEPS TRACK OF ALL THE BLOCKS THAT HAVE CHANGED
#(!04%2 -)'2!4).' 4/ %8!$!4!
SINCE THE LAST LEVEL BACKUP BY FLIPPING A BIT IN A SMALL BITMAP FILE 4HIS MEANS THAT WHEN YOU TAKE AN
INCREMENTAL LEVEL BACKUP /RACLE DOESNT HAVE TO SCAN EVERY BLOCK IN THE DATABASE TO SEE WHICH ONES
HAVE CHANGED )VE SEEN THIS REDUCE AN INCREMENTAL BACKUP OF A 4 DATA WAREHOUSE FROM HOURS TO JUST
OVER MINUTES 4O SEE IF BLOCK CHANGE TRACKING IS ACTIVE EXECUTE THE FOLLOWING QUERY IN 31,
0LUS
B@;-B4;42CbcPcdb5A><eQ[^RZNRWP]VTNcaPRZX]V*
BC0CDB
38B01;43
9OU DO NOT HAVE TO SHUT DOWN THE DATABASE TO ACTIVATE BLOCK CHANGE TRACKING )T CAN BE DONE AT ANY
TIME 4O TURN ON BLOCK CHANGE TRACKING EXECUTE THE FOLLOWING COMMAND
B@;-0;C4A30C010B44=01;41;>2:270=64CA02:8=6*
3PcPQPbTP[cTaTS
"Y DEFAULT THE BLOCK CHANGE TRACKING FILE IS CREATED IN THE SQNRaTPcTNUX[TNSTbc LOCATION )F THIS
PARAMETER IS NOT SET YOU WILL NEED TO SET IT OR SPECIFY THE FILE NAME FOR THE BLOCK CHANGE TRACKING FILE LIKE
THIS
0;C4A30C010B44=01;41;>2:270=64CA02:8=6
DB8=658;4 d P__ ^aPR[T _a^SdRc ! SQW^\TN SQb C4BCNQRc RcjA4DB4l*
0OST
MIGRATION TASKS ARE NECESSARY TO CONVERT YOUR SINGLE
INSTANCE %XADATA DATABASE TO A MULTI
INSTANCE 2!# DATABASE #ONVERTING A SINGLE
INSTANCE DATABASE TO 2!# WILL BE COVERED AT THE END OF THE
h0HYSICAL -IGRATIONv SECTION OF THIS CHAPTER /NCE THE DATABASE IS RECOVERED AND RUNNING ON %XADATA
YOU SHOULD MAKE A FULL DATABASE BACKUP !SSUMING THE DATABASE IS RUNNING IN !RCHIVELOG MODE YOU CAN
DO THIS AFTER THE DATABASE IS BACK ONLINE AND SERVICING END USERS AND APPLICATIONS
When to Use Backup and Restore
%XECUTING A SIMPLE DATABASE BACKUP AND RECOVERY USING 2-!. IS SOMETHING EVERY $"! SHOULD BE ABLE
TO DO IN THEIR SLEEP 4HIS MAKES THE "ACKUP AND 2ESTORE A VERY ATTRACTIVE STRATEGY )T IS BEST SUITED FOR
/,40 DATABASES THAT DONT REQUIRE PARTITIONING AND (## COMPRESSION )T IS ALSO SUITABLE FOR $7
DATABASES THAT ALREADY RUN ON A LITTLE ENDIAN BIT PLATFORM POST MIGRATION STEPS NOTWITHSTANDING
What to Watch Out for when Considering the Backup and Restore Strategy
)NCREMENTAL BACKUP AND RESTORE DOES NOT SUPPORT PLATFORM CONVERSION )F THIS IS A REQUIREMENT YOU
MIGHT BE BETTER OFF USING THE 443 MIGRATION STRATEGY OR REVISITING SOME OF THE LOGICAL MIGRATION STRATEGIES
WEVE DISCUSSED IN THIS CHAPTER /BJECTS WILL NEED TO BE REBUILT AFTER MIGRATION TO TAKE ADVANTAGE OF
%XADATA STORAGE FEATURES SUCH AS (##
4RANSPORTABLE 4ABLESPACES AND 8443
4RANSPORTABLE TABLESPACES 443 CAN BE USED TO MIGRATE SUBSETS OF THE SOURCE DATABASE TO %XADATA 4O DO
THIS YOU WILL NEED A RUNNING DATABASE ON %XADATA TO HOST THESE SUBSETS OF YOUR DATABASE 7E OFTEN
DESCRIBE 443 AS A SORT OF h0RUNE AND 'RAFTv PROCEDURE 4HIS METHOD ALLOWS A SET OF TABLESPACES TO BE
COPIED FROM A SOURCE DATABASE AND INSTALLED INTO A LIVE TARGET DATABASE 4HE 443 PROCESS IS FAIRLY SIMPLE
BUT THERE ARE A FEW THINGS YOU WILL NEED TO BE AWARE OF BEFORE BEGINNING
#(!04%2 -)'2!4).' 4/ %8!$!4!
2ESTRICTIONS
v 4ABLESPACES MUST BE PUT IN READ
ONLY MODE DURING THE PROCESS
v #HARACTER SETS WILL IDEALLY BE THE SAME FOR THE SOURCE AND TARGET DATABASES 4HERE
ARE EXCEPTIONS TO THIS RULE 3EE THE /RACLE DOCUMENTATION IF YOU PLAN TO CHANGE
CHARACTER SETS DURING MIGRATION
v /BJECTS WITH UNDERLYING DEPENDENCIES LIKE MATERIALIZED VIEWS AND TABLE PARTITIONS
ARE NOT TRANSPORTABLE UNLESS THEY ARE CONTAINED IN THE SAME SET ! TABLESPACE SET IS
A WAY OF MOVING A GROUP OF RELATED TABLESPACES TOGETHER IN ONE OPERATION
v "EFORE TRANSPORTING ENCRYPTED TABLESPACES YOU MUST FIRST COPY THE /RACLE WALLET TO
THE TARGET SYSTEM AND ENABLE IT FOR THE TARGET DATABASE ! $ATABASE CAN ONLY HAVE
ONE /RACLE WALLET )F THE DATABASE YOU ARE MIGRATING TO ALREADY HAS A WALLET YOU WILL
NEED TO USE $ATA 0UMP TO EXPORT AND IMPORT TABLE DATA
v 4ABLESPACES THAT DO NOT USE BLOCK LEVEL ENCRYPTION BUT HAVE TABLES THAT USE
COLUMN ENCRYPTION CANNOT BE TRANSPORTED WITH 443 )N THIS CASE $ATA 0UMP IS
PROBABLY THE BEST ALTERNATIVE
v 4ABLESPACES THAT CONTAIN 8-, DATA TYPES ARE SUPPORTED BY 443 AS OF G2 BUT
YOU WILL HAVE TO USE $ATA 0UMP TO EXPORT THE METADATA 4HERE ARE OTHER
RESTRICTIONS AND CAVEATS TO TRANSPORTING 8-, DATA 2EFER TO THE Oracle XML DB
Developer’s Guide FOR A COMPLETE LISTING 4O LIST TABLESPACES WITH 8-, DATA TYPES
RUN THE FOLLOWING QUERY
B@;-bT[TRcSXbcX]Rc_ cPQ[Tb_PRTN]P\T
Ua^\SQPNcPQ[Tb_PRTb_
SQPNg\[NcPQ[Tbg
SQPNdbTabd
P[[NP[[NcPQ[Tbc
fWTaTc cPQ[TN]P\T,g cPQ[TN]P\T
P]Sc cPQ[Tb_PRTN]P\T,_ cPQ[Tb_PRTN]P\T
P]Sg ^f]Ta,d dbTa]P\T*
v /PAQUE TYPES SUCH AS 2!7 AND "&),% ARE SUPPORTED BY 443 BUT ARE NOT CONVERTED
CROSS
PLATFORM 4HE STRUCTURE OF THESE TYPES IS ONLY KNOWN TO THE APPLICATION AND
ANY DIFFERENCES IN ENDIAN FORMAT MUST BE HANDLED BY THE APPLICATION 4YPES AND
OBJECTS ARE SUBJECT TO THIS LIMITATION WHETHER THEIR USE OF OPAQUE TYPES IS DIRECT OR
INDIRECT
v $ATABASE VERSION DIFFERENCES ARE SUPPORTED AS LONG AS THE TARGET DATABASE IS OF THE
SAME OR HIGHER VERSION THAN THE SOURCE DATABASE
#ROSS 0LATFORM 4RANSPORTABLE 4ABLESPACES 8443 SUPPORTS CONVERSION BETWEEN MOST BUT NOT ALL
PLATFORMS 4O DETERMINE WHETHER YOUR PLATFORM IS SUPPORTED RUN THE FOLLOWING QUERY
#(!04%2 -)'2!4).' 4/ %8!$!4!
BHB)4G31 -B4;42C5A><ECA0=B?>AC01;4N?;0C5>A<>A34A1H?;0C5>A<N=0<4*
?;0C5>A<N83?;0C5>A<N=0<44=380=N5>A<0C
%08G 1PbTSBhbcT\b%# QXc1XV
%0__[T<PR> B1XV
! 0__[T<PR> Bg'% %#;Xcc[T
(7?80>_T]E<B;Xcc[T
$7?>_T]E<B;Xcc[T
$7?Cad%#D=8G;Xcc[T
"7? DG%# QXc1XV
#7? DG80%# QXc1XV
'81<?^fTa1PbTS;X]dg1XV
(81<iBTaXTb1PbTS;X]dg1XV
;X]dg80"! QXc;Xcc[T
;X]dg80%# QXc;Xcc[T
";X]dgg'%%# QXc;Xcc[T
&<XRa^b^UcFX]S^fb80"! QXc;Xcc[T
'<XRa^b^UcFX]S^fb80%# QXc;Xcc[T
!<XRa^b^UcFX]S^fbg'%%# QXc;Xcc[T
&B^[PaXb>_TaPcX]VBhbcT\g'%;Xcc[T
! B^[PaXb>_TaPcX]VBhbcT\g'% %#;Xcc[T
B^[PaXbJc\L>4"! QXc1XV
!B^[PaXbJc\L>4%# QXc1XV
%XADATA IS LITTLE
ENDIAN SO IF THE SOURCE DATABASE IS ALSO LITTLE
ENDIAN TABLESPACES MAY BE
TRANSPORTED AS IF THE PLATFORM WERE THE SAME )F THE SOURCE PLATFORM IS BIG
ENDIAN THEN AN ADDITIONAL
STEP IS REQUIRED 4O CONVERT A TABLESPACE FROM ONE PLATFORM TO ANOTHER USE THE 2-!. 2>=E4AC
C01;4B?024 OR 2>=E4AC30C058;4 COMMAND
9OU MAY CONVERT THE ENDIAN FORMAT OF FILES DURING THE BACKUP USING THE FOLLOWING COMMAND
A<0=-2>=E4ACC01;4B?024_Pha^[[NSPcP_Pha^[[N\eXTfb
C>?;0C5>A<;X]dgg'%%# QXc
5>A<0C d bWPaTSNUX[Tb D*
)N THIS EXAMPLE 2-!. CONVERTS THE DATAFILES TO AN ENDIAN FORMAT COMPATIBLE WITH %XADATA 4HE
CONVERTED DATAFILES ARE UNIQUELY NAMED AUTOMATICALLY 5 AND SAVED IN THE d bWPaTSNUX[Tb
DIRECTORY 4HIS CONVERSION CAN BE DONE ON THE SOURCE SYSTEM OR THE TARGET %XADATA
4HE FOLLOWING COMMAND CONVERTS THE ENDIAN FORMAT DURING THE RESTORE OPERATION ON %XADATA
A<0=-2>=E4AC30C058;4_Pha^[[NSPcP SQU_Pha^[[N\eXTfb SQU
5A><?;0C5>A<B^[PaXbJc\L>4%# QXc
31N58;4N=0<4N2>=E4AC
d bWPaTSNUX[Tb _Pha^[[NSPcP SQU30C0
d bWPaTSNUX[Tb _Pha^[[N\eXTfb SQU30C0*
! TABLESPACE CAN BE TRANSPORTED INDIVIDUALLY OR AS PART OF A TRANSPORT SET 4RANSPORT SETS ARE MORE
COMMON BECAUSE MORE OFTEN THAN NOT OBJECT DEPENDENCIES EXIST ACROSS TABLESPACES &OR EXAMPLE
THERE MAY BE TABLES IN ONE TABLESPACE AND DEPENDENT MATERIALIZED VIEWS OR INDEXES IN ANOTHER )N ORDER
TO TRANSPORT A TABLESPACE YOU MUST FIRST PUT IT IN READ
ONLY MODE 4ABLESPACE METADATA IS THEN EXPORTED
USING $ATA 0UMP WITH THE caP]b_^acPQ[TNcPQ[Tb_PRTb PARAMETER 9OU SHOULD ALSO SPECIFY THE
CA0=B?>ACN5D;;N2742: PARAMETER TO ENSURE STRICT CONTAINMENT OF THE TABLESPACES BEING TRANSPORTED 4HIS
ENSURES THAT NO DEPENDENT OBJECTS LIKE INDEXES EXIST OUTSIDE OF THE TRANSPORT SET 2-!. IS THEN USED TO
#(!04%2 -)'2!4).' 4/ %8!$!4!
TAKE A BACKUP OF THE TABLESPACES IN THE TRANSPORT SET #ONVERSION BETWEEN ENDIAN FORMATS MAY BE DONE
DURING THE 2-!. BACKUP OR DURING THE RESTORE ON THE TARGET SYSTEM (ERE ARE THE STEPS FOR TRANSPORTING
TABLESPACES TO %XADATA
1. )DENTIFY TABLESPACE OBJECT DEPENDENCIES
2. 3ET TABLESPACES TO READ
ONLY MODE
3. %XPORT METADATA FOR THE TRANSPORT SET USING $ATA 0UMP
4. 4AKE AN 2-!. BACKUP OF THE TABLESPACES IN THE TRANSPORT SET
5. #OPY THE EXPORT FILES ALONG WITH THE DATA FILE BACKUPS TO %XADATA
6. 2ESTORE THE DATA FILES FROM THE 2-!. BACKUP TO YOUR %XADATA DATABASE )F ENDIAN
CONVERSION IS NEEDED USE THE 2>=E4AC30C058;4 COMMAND TO RESTORE AND CONVERT THE
DATAFILES SIMULTANEOUSLY
7. -AKE THE TABLESPACES READWRITE AGAIN
8. 5SING $ATA 0UMP IMPORT THE TABLESPACE METADATA INTO THE %XADATA DATABASE USING
THE caP]b_^acNSPcPUX[Tb PARAMETER 9OU CAN OPTIONALLY REMAP THE SCHEMA OF THE
TABLESPACE CONTENTS USING THE aT\P_NbRWT\P PARAMETER
When to Use Transportable Tablespaces
443 AND 8443 ARE USEFUL FOR MIGRATING PORTIONS OF THE SOURCE DATABASE USING THE SPEED OF 2-!. )F PARTS
OF YOUR DATABASE ARE READY TO MOVE TO %XADATA BUT OTHERS ARE NOT 443 MAY BE A GOOD FIT
What to Watch Out for with the Transportable Tablespace Strategy
#HECK YOUR /RACLE DOCUMENTATION FOR SPECIFIC RESTRICTIONS OR CAVEATS THAT MAY APPLY TO YOUR DATABASE
7ATCH OUT FOR TABLESPACE AND SCHEMA NAME COLLISIONS ON THE TARGET DATABASE 4ABLESPACES MUST BE PUT
IN READ ONLY MODE FOR THE MOVE SO THIS WILL INCUR DOWNTIME 9OU CAN RUN AN 2-!. BACKUP OF THE
TABLESPACES WHILE THEY ARE IN READWRITE MODE TO SEE HOW LONG THIS OPERATION WILL TAKE BEFORE DECIDING
WHETHER 443 IS AN APPROPRIATE METHOD OR NOT FOR YOUR MIGRATION
0HYSICAL 3TANDBY
)N THE PHYSICAL STANDBY STRATEGY THE TARGET DATABASE IS INSTANTIATED FROM A FULL BACKUP OF THE SOURCE
DATABASE 4HE BACKUP IS RESTORED TO %XADATA IN THE SAME WAY YOU WOULD FOR THE h"ACKUP AND 2ESTOREv
STRATEGY /NCE THE DATABASE IS RESTORED TO %XADATA IT IS STARTED IN MOUNT MODE AND KEPT IN A CONTINUAL
STATE OF RECOVERY !S CHANGES OCCUR IN THE SOURCE DATABASE ARCHIVED REDO LOGS ARE GENERATED AND
TRANSMITTED TO THE 3TANDBY DATABASE WHERE THEY ARE APPLIED 2EDO !PPLY 5NLIKE THE h"ACKUP AND
2ESTOREv STRATEGY THIS DATABASE IS KEPT IN RECOVERY MODE FOR A PERIOD OF TIME "ECAUSE ARCHIVED REDO LOGS
ARE CONSTANTLY BEING APPLIED TO THE 3TANDBY DATABASE CONVERSION FROM BIG
ENDIAN TO LITTLE
ENDIAN
FORMAT IS NOT SUPPORTED 3TANDBY DATABASES HAVE BEEN AROUND SINCE VERSION OF /RACLE 4HE CAPABILITY IS
INHERENT IN THE DATABASE ARCHITECTURE /F COURSE BACK THEN YOU HAD TO WRITE SCRIPTS TO MONITOR THE
ARCHIVE LOG DESTINATION ON THE SOURCE SYSTEM AND THEN COPY THEM USUALLY VIA &40 TO THE 3TANDBY
SYSTEM WHERE ANOTHER SCRIPT HANDLED APPLYING THEM TO THE 3TANDBY DATABASE )N VERSION I /RACLE
#(!04%2 -)'2!4).' 4/ %8!$!4!
INTRODUCED A NEW PRODUCT CALLED $ATA 'UARD TO MANAGE AND AUTOMATE MANY OF THOSE TEDIOUS TASKS OF
MANAGING AND MONITORING THE 3TANDBY ENVIRONMENT 4ODAY $ATA 'UARD PROVIDES THE FOLLOWING SERVICES
v 2EDO 4RANSPORT 3ERVICES
v (ANDLES THE TRANSMISSION OF ARCHIVED REDO LOGS TO THE TARGET SYSTEM
v 2ESOLVES GAPS IN ARCHIVE REDO LOGS DUE TO NETWORK FAILURE
v $ETECT AND RESOLVE MISSING OR CORRUPT ARCHIVED REDO LOGS BY RETRANSMITTING
REPLACEMENT LOGS
v !PPLY 3ERVICES
v !UTOMATICALLY APPLIES ARCHIVED REDO LOGS TO THE 3TANDBY DATABASE
v !LLOWS READ
ONLY ACCESS TO THE 3TANDBY DATABASE DURING REDO APPLY
v 0ROVIDES ROLE TRANSITION MANAGEMENT 4HE ROLE OF THE SOURCE AND 3TANDBY
DATABASES MAY BE SWITCHED TEMPORARILY OR PERMANENTLY
v 3WITCHOVER 4EMPORARILY SWITCHES THE ROLES OF THE SOURCE AND 3TANDBY
DATABASES 5SEFUL FOR FAILING BACK A MIGRATION
v &AILOVER 0ERMANENT CHANGE IN ROLES 5SEFUL FOR FINALIZING THE MIGRATION TO
%XADATA 4HE OLD SOURCE DATABASE BECOMES THE STANDBY AND IS KEPT IN SYNC
WITH %XADATA USING 2EDO !PPLY
v $ATA 'UARD "ROKER
v 3IMPLIFIES THE CONFIGURATION AND INSTANTIATION OF THE 3TANDBY DATABASE
v #ENTRALIZED CONSOLE FOR MONITORING AND MANAGING THE 3TANDBY DATABASE
ENVIRONMENT
v 3IMPLIFIES SWITCHOVERFAILOVER OF THE 3TANDBY DATABASE
$ATA 'UARD PROVIDES MODES OF REPLICATION TO THE 3TANDBY DATABASE
Maximum Availability: 4RANSACTIONS IN THE SOURCE DATABASE DONT COMMIT UNTIL
ALL REDO NECESSARY TO RECOVER THE 3TANDBY ARE WRITTEN TO THE ONLINE REDO LOG AND
TO THE STANDBY REDO LOG
Maximum Performance: 4RANSACTIONS DONT COMMIT UNTIL THEY ARE WRITTEN TO
THE SOURCE ONLINE REDO LOGS
Maximum Protection: %NSURES ZERO DATA LOSS AT THE COST OF PERFORMANCE ON THE
SOURCE DATABASE
+EEP IN MIND THAT ALTHOUGH $ATA 'UARD IS AN EXCELLENT TOOL FOR DATABASE MIGRATION THAT IS NOT ITS
ONLY PURPOSE $ATA 'UARDS FORTE IS PROTECTING DATABASES FROM MEDIA CORRUPTION CATASTROPHIC MEDIA
FAILURE AND SITE FAILURE )T IS AN INTEGRAL COMPONENT IN /RACLES -AXIMUM !VAILABILITY !RCHITECTURE 3O ITS
NO SURPRISE THAT SOME OF THE REPLICATION MODES MENTIONED ABOVE MAKE NO SENSE IN THE CONTEXT OF
DATABASE MIGRATION -AXIMUM 0ERFORMANCE IS THE REPLICATION MODE MOST APPROPRIATE FOR MIGRATING
DATABASES BECAUSE IT HAS NO IMPACT ON THE PERFORMANCE OF THE SOURCE DATABASE )N -AXIMUM
#(!04%2 -)'2!4).' 4/ %8!$!4!
0ERFORMANCE MODE THE SOURCE DATABASE CONTINUES TO FUNCTION AS IT ALWAYS HAS 4RANSACTIONS ARE NOT
DELAYED BY NETWORK ISSUES OR DOWNSTREAM REPLICATION PROBLEMS )TS ALSO WORTH MENTIONING THAT $ATA
'UARD FULLY SUPPORTS EITHER A SINGLE
INSTANCE STANDBY OR AN /RACLE 2!# STANDBY !ND AS OF G2 REDO
CAN BE APPLIED WHILE THE 3TANDBY DATABASE IS OPEN IN READ
ONLY MODE 4HERE IS ALSO STRONG INTEGRATION
WITH %NTERPRISE -ANAGER 'RID #ONTROL
When to use Physical Standby
4HE PHYSICAL STANDBY DATABASE DOES NOT ALLOW ANY CHANGES TO THE DATABASE OTHER THAN DATAFILE NAME
CHANGES "ECAUSE OF THIS IT IS BEST SUITED FOR DATABASE MIGRATIONS WHERE NO CHANGES TO THE TARGET
DATABASE ARE REQUIRED )F CHANGES NEED TO BE MADE THEY WILL HAVE TO BE DONE POST MIGRATION 'ENERALLY
SPEAKING THIS IS LESS OF AN ISSUE WITH /,40 DATABASES BECAUSE CHANGES LIKE MIGRATING TO LARGE TABLE
EXTENT SIZES IMPLEMENTING PARTITIONING AND IMPLEMENTING (## ARE NOT AS BENEFICIAL AS THEY ARE FOR
LARGER $7 DATABASES )F GETTING TO %XADATA AS QUICKLY AND SAFELY AS POSSIBLE IS YOUR GOAL THEN PHYSICAL
STANDBY MAY BE A GOOD FIT 7ITH %XADATAS PERFORMANCE AND SCALABILITY POST MIGRATION TASKS MAY TAKE
FAR LESS TIME TO IMPLEMENT THAN YOU MIGHT EXPECT
What to Watch Out for when Considering the Physical Standby Strategy
4HERE ARENT MANY TWISTS OR TURNS WITH THE PHYSICAL STANDBY STRATEGY 9OU SHOULD KEEP AN EYE ON NETWORK
STABILITY AND PERFORMANCE 7HILE IT IS POSSIBLE TO USE /RACLES CROSS
PLATFORM PHYSICAL STANDBY FEATURE FOR
LOW
DOWNTIME MIGRATIONS BETWEEN SOME PLATFORMS YOU HAVE NO OPPORTUNITY TO DO SUCH MIGRATIONS
ACROSS PLATFORMS WITH DIFFERENT BYTE ORDER ENDIAN ORIENTATION 4HERE ARE ALSO SOME /RACLE VERSION
SPECIFIC LIMITATIONS READ MORE FROM -Y /RACLE 3UPPORT NOTE “Data Guard Support for
Heterogeneous Primary and Physical Standbys in Same Data Guard Configuration” )F A LOW
DOWNTIME
MIGRATION BETWEEN INCOMPATIBLE PLATFORMS IS REQUIRED THEN YOU SHOULD CONSIDER THE ,OGICAL 3TANDBY
3TREAMS OR 'OLDEN 'ATE STRATEGIES
!3- 2EBALANCE
4HIS WAS A DIFFICULT TOPIC TO COVER BECAUSE OF THE LACK OF DOCUMENTATION AVAILABLE )T IS ALSO FAR AND AWAY
THE MOST RESTRICTIVE AND POSSIBLY THE SLOWEST METHOD OF MIGRATING TO %XADATA 3O WHY DOES IT MERIT
INCLUSION IN THIS CHAPTER )T IS ONE OF THE MOST INTERESTING TOPICS IN THIS CHAPTER 4HE !3- 2EBALANCE
STRATEGY UTILIZES !3-S BUILT
IN REBALANCING FEATURE TO LITERALLY SHIFT BLOCK
BY
BLOCK OFF OF CURRENT STORAGE
AND ONTO %XADATAS HIGH PERFORMANCE STORAGE CELL HARDWARE WITH ZERO DOWNTIME TO THE DATABASE (OW IS
THIS POSSIBLE "EFORE WE GET TO THAT LETS TAKE A LOOK AT THE RESTRICTIONS FOR THIS MIGRATION STRATEGY
v 4HE PLATFORM MUST BE /%, BIT 2(%, RELEASE LIMITED SUPPORT
v 4HE DATABASE VERSION MUST BE LATEST PATCH RELEASE OF G2 OR HIGHER LATEST G2
IS BEST
v 4HERE MUST BE CURRENT DRIVERS FOR /RACLE 2$3/PEN &ABRICS %NTERPRISE
$ISTRIBUTION /&%$
v )T MUST BE CONNECTED TO THE %XADATA )NFINI"AND SWITCH
/RACLE RECOMMENDS THE FOLLOWING KERNEL NETWORK PARAMETERS FOR 2$3 SUPPORT
v ]Tc R^aT a\T\NSTUPd[c
#(!04%2 -)'2!4).' 4/ %8!$!4!
v ]Tc R^aT f\T\NSTUPd[c
v ]Tc R^aT a\T\N\Pg
v ]Tc R^aT f\T\N\Pg
v ]Tc X_e# X_N[^RP[N_^acNaP]VT
3O HOW DOES IT WORK "ASICALLY ONCE ALL PLATFORM HARDWARE AND SOFTWARE REQUIREMENTS ARE MET THE
FOLLOWING STEPS ALLOW YOUR CURRENT PRODUCTION SYSTEM TO JOIN THE %XADATA STORAGE NETWORK AND GAIN
ACCESS TO THE STORAGE CELLS
1. ! FAMILIAR PRE
MIGRATION TASK IS TO CONFIGURE THE X]Xc ^aP PASSWORD AND ORATAB
FILES ON %XADATA TO ALLOW YOU TO START UP YOUR DATABASE ON %XADATA
2. #ONFIGURE THE )" NETWORK DEVICE GIVING IT AN )0 ADDRESS ON THE %XADATA STORAGE
NETWORK
3. #REATE THE RT[[X]Xc ^aP FILE WITH ONE LINE CONTAINING THE )0 ADDRESS AND
NETWORK MASK FOR SERVER ON THE STORAGE NETWORK
-RPc TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X]Xc ^aP
X_PSSaTbb , (! %' ! !#
4. #REATE A RT[[X_ ^aP FILE CONTAINING THE ADDRESSES FOR EACH OF THE STORAGE CELLS
-RPcRT[[X_ ^aP
RT[[, (! %' ! "
RT[[, (! %' ! #
RT[[, (! %' ! $
5. "RING THE )NFINI"AND DEVICE ONLINE
XUR^]UXVXQ d_
XQ ;X]ZT]RP_)8]UX]X1P]S7FPSSa
' ) ) )#')54)' ) ) ) ) ) ) ) ) ) ) ) ) ) )
X]TcPSSa) (! %' ! 1RPbc) (! %' ! !$$<PbZ)!$$ !$$ !$$
X]Tc%PSSa)UT' ))!! )!' ) "T)TQ#& %#BR^_T);X]Z
+[X]Tb^\XccTS-
6. !T THIS POINT IF EVERYTHING IS WORKING YOU SHOULD BE ABLE TO SEE THE %XADATA
DISKS BY RUNNING THE COMMAND FROM THE ORACLE USER ACCOUNT KFOD DISK ALL
JT]ZSQ )^aPR[T)4G31 L W^\T ^aPR[T
-ZU^SSXbZ,P[[
JT]ZSQ )^aPR[T)4G31 L W^\T ^aPR[T
-ZU^SSXbZ,P[[
3XbZBXiT?PcWDbTa6a^d_
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
) " "% <Q^ (! %' ! " 30C0N23N NRT[[ +d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
!()("#$%<Q^ (! %' ! " A42>N23N NRT[[ +d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
# )#%( "%<Q^ (! %' ! " BC064N23N NRT[[ +d]Z]^f]-+d]Z]^f]-
#(!04%2 -)'2!4).' 4/ %8!$!4!
+[X]Tb^\XccTS-
$")""%<Q^ (! %' ! " BHBC4<N23N NRT[[ +d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
%$) " "% <Q^ (! %' ! # 30C0N23N NRT[[ !+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
(")("#$%<Q^ (! %' ! # A42>N23N NRT[[ !+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
$)#%( "%<Q^ (! %' ! # BC064N23N NRT[[ !+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
&)""%<Q^ (! %' ! # BHBC4<N23N NRT[[ !+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
!() " "% <Q^ (! %' ! $ 30C0N23N NRT[[ "+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
$&)("#$%<Q^ (! %' ! $ A42>N23N NRT[[ "+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
%()#%( "%<Q^ (! %' ! $ BC064N23N NRT[[ "+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
' )""%<Q^ (! %' ! $ BHBC4<N23N NRT[[ "+d]Z]^f]-+d]Z]^f]-
+[X]Tb^\XccTS-
>A02;4NB83>A02;4N7><4
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
0B< d P__ ! VaXS
0B<! d P__ ! VaXS
7. /NCE CONNECTIVITY IS ESTABLISHED BETWEEN THE SERVER AND THE STORAGE CELLS THE
!3- INSTANCE ON THE DATABASE SERVER SHOULD BE ABLE TO ADD %XADATA GRID DISKS
TO ITS CURRENT !3- DISK GROUPDISK GROUPS USING THE P[cTaSXbZVa^d_PSSSXbZ
qA410;0=24?>F4A$]^fPXc COMMAND 7HEN THE GRID DISKS ARE ADDED TO THE
CURRENT DISK GROUPS !3- WILL BEGIN REBALANCING DATA ONTO THE NEW DISKS AT A
RATE DETERMINED BY THE _^fTa] PARAMETER OF THE PSSSXbZ COMMAND ! POWER
RANGE OF n REPRESENTS THE NUMBER OF REBALANCE PROCESSES THAT WILL
PARTICIPATE IN THE OPERATION .OWAIT CAUSES THE COMMAND TO RETURN YOU TO THE
PROMPT RATHER THAN HANGING UNTIL THE OPERATION IS COMPLETE
8. 9OU CAN SEE THE CURRENT STATUS OF THE REBALANCE OPERATION BY QUERYING THE
ePb\N^_TaPcX^] VIEW FROM 31,
0LUS CONNECTED TO THE !3- INSTANCE
9. !FTER THE REBALANCE IS COMPLETE THE OLD DISKS CAN BE REMOVED FROM THE DISK
GROUPS USING THE P[cTaSXbZVa^d_Sa^_SXbZq_^fTa$]^fPXc COMMAND
!NOTHER DISK GROUP REBALANCE WILL BEGIN MOVING THE REMAINING DATA OFF OF THE
OLD DISKS AND ONTO THE %XADATA STORAGE GRID DISKS
10. /NCE ALL DATA HAS BEEN MIGRATED ONTO %XADATA STORAGE THE DATABASE IS SHUT
DOWN AND RESTARTED ON %XADATA "EFORE RESTARTING THE DATABASE USE THIS
OPPORTUNITY TO UPGRADE THE DISK GROUPDISK GROUPS FROM VERSION TO IF
NECESSARY
11. 0OST
MIGRATION TASKS MUST BE PERFORMED TO ADD THE DATABASE TO THE #LUSTER
2EADY 3ERVICES CONFIGURATION
#(!04%2 -)'2!4).' 4/ %8!$!4!
When to Use ASM Rebalance
!3- 2EBALANCE IS AN EXCELLENT WAY OF LEVERAGING !3-S BUILT IN FEATURES TO MIGRATE YOUR DATABASE WITH
NEAR ZERO DOWNTIME )T IS NOT THE FASTEST WAY TO MIGRATE TO %XADATA AND THERE ARE A LOT OF VERY STRICT
CONFIGURATION REQUIREMENTS BUT ONCE THE SYSTEM IS CONFIGURED THE MIGRATION IS FAIRLY AUTOMATED )T IS
PROBABLY THE MOST RELIABLE AND ERROR FREE METHOD FOR MIGRATING YOUR DATABASE TO %XADATA
What to Watch Out for when Using the ASM Rebalance Strategy
4HIS STRATEGY IS NOT WELL USED OR WELL DOCUMENTED !ND IT IS NOT FOR THE FAINT OF HEART AND CERTAINLY NOT FOR
THE INEXPERIENCED SYSTEM ADMINISTRATOR )T REQUIRES STRONG SYSTEM ADMINISTRATION SKILLS AND SOLID ,INUX
SYSTEMS EXPERIENCE 3OME OF THE PRELIMINARY STEPS SUCH AS INSTALLING AND CONFIGURING THE )NFINI"AND
CARD AND PROPER 2$3/&%$ DRIVERS CAN BE DISRUPTIVE TO PRODUCTION UPTIME REQUIREMENTS 4HIS METHOD
DOES NOT CREATE A COPY OF YOUR DATABASE )T MIGRATES YOUR CURRENT DATABASE AT THE STORAGE LEVEL !S SUCH
STORAGE LAYOUT CHANGES LIKE EXTENT SIZING PARTITIONING AND (## COMPRESSION MUST BE DONE POST
MIGRATION ! DOWNSIDE TO THE !3- REBALANCE METHOD IS THAT !3- !LLOCATION 5NITS !5S CANNOT BE
CHANGED AFTER THE DISK GROUP IS CREATED %XADATA IS MORE EFFICIENT AT SCANNING -" !5S SO IF YOUR DISK
GROUPS WERE CONFIGURED WITH -" !5S YOU WOULD EVENTUALLY WANT TO CREATE NEW DISK GROUPS IN %XADATA
WITH -" !5 SIZES AND MIGRATE YOUR DATABASE TO THEM
0OST
-IGRATION 4ASKS
"ECAUSE !3- REBALANCE MAKES NO CHANGES TO THE DATABASE TASKS LIKE RESIZING EXTENTS PARTITIONING LARGE
TABLES AND COMPRESSING TABLES WILL ALL HAVE TO BE DONE POST
MIGRATION 4HIS CAN BE A SUBSTANTIAL TASK THAT
TAKES ANYWHERE FROM SEVERAL DAYS TO SEVERAL WEEKS TO COMPLETE &OR ALL PHYSICAL MIGRATION STRATEGIES
EXCEPT 443 YOU WILL NEED TO PERFORM SOME POST
MIGRATION STEPS TO ADD THE DATABASE INTO THE #LUSTER
2EADY 3ERVICES CONFIGURATION 4HERE IS NOTHING %XADATA
SPECIFIC TO THIS PROCESS OTHER THAN CONNECTING TO
THE STORAGE CELLS IT IS THE SAME AS FOR ANY G2 DATABASE 4HE PROCESS IS WELL DOCUMENTED AND
EXAMPLES ARE EASY TO FIND SO WE WONT GO INTO THE DETAILS HERE BUT THE BASIC STEPS ARE AS FOLLOWS
1. 3ET THE FOLLOWING INSTANCE PARAMETERS IN THE SPFILE
v R[dbcTaNSPcPQPbT,cadT
v R[dbcTaNSPcPQPbTNX]bcP]RTb,!
v =4FN?A>3 X]bcP]RTN]P\T,=4FN?A>3
v =4FN?A>3! X]bcP]RTN]P\T,=4FN?A>3!
v =4FN?A>3 X]bcP]RTN]d\QTa,
v =4FN?A>3! X]bcP]RTN]d\QTa,!
v =4FN?A>3 cWaTPS,
v =4FN?A>3! cWaTPS,!
v =4FN?A>3 d]S^NcPQ[Tb_PRT,d]S^NcQb
v =4FN?A>3! d]S^NcPQ[Tb_PRT,d]S^NcQb!
#(!04%2 -)'2!4).' 4/ %8!$!4!
2. )NSTALL THE 2!# SYSTEM OBJECTS BY RUNNING THE RPcR[dbc b`[ SCRIPT AS THE SYS
USER
3. 2EGISTER THE NEW DATABASE WITH #LUSTER 2EADY 3ERVICES #23
a. 2EGISTER THE DATABASE
b. 2EGISTER THE INSTANCES
c. #REATE 3ERVICE .AMES FOR THE DATABASE
4. #REATE AN ADDITIONAL REDO THREAD FOR EACH INSTANCE .OTE THAT THREAD IS
ALREADY ACTIVE FOR THE FIRST INSTANCE
5. #REATE AN 5NDO TABLESPACE FOR EACH DATABASE INSTANCE .OTE THAT ONE ALREADY
EXISTS FOR THE FIRST INSTANCE
7RAP 5P 0HYSICAL -IGRATION 3ECTION
0HYSICAL MIGRATION MAY PROVE TO BE AN EASY MIGRATION OPTION IF YOUR APPLICATION SCHEMA IS COMPLEX
ENOUGH THAT YOU DONT WANT TO TAKE ANY LOGICAL PATH !LSO PHYSICAL MIGRATION CAN POTENTIALLY BE DONE
WITH VERY LOW DOWNTIME BY RESTORING A PRODUCTION COPY TO THE %XADATA DATABASE AS A physical STANDBY
DATABASE AND APPLYING PRODUCTION ARCHIVELOGS UNTIL ITS TIME TO SWITCH OVER AND MAKE STANDBY THE NEW
PRODUCTION DATABASE (OWEVER THIS APPROACH CANNOT BE USED BETWEEN PLATFORMS WITH DIFFERENT
ENDIANNESS AND THERE ARE A FEW MORE /RACLE VERSION SPECIFIC RESTRICTIONS
Dealing with Old init.ora Parameters
7HEN MIGRATING FROM AN OLDER VERSION OF /RACLE YOU MIGHT BE TEMPTED TO KEEP ALL THE OLD
UNDOCUMENTED X]Xc ^aP PARAMETERS FOR hTUNINGv OR hSTABILITYv 4HE FACT IS THAT /RACLE HAS VERY GOOD
DEFAULT VALUES FOR ITS PARAMETERS SINCE G ESPECIALLY SO IN WHICH LIKELY RUNS ON YOUR %XADATA
CLUSTER 7HATEVER PROBLEMS WERE SOLVED BY SETTING THESE UNDOCUMENTED PARAMETERS YEARS AGO ARE
PROBABLY FIXED IN THE DATABASE CODE ALREADY !LSO MOVING TO %XADATA BRINGS A MUCH BIGGER CHANGE THAN
ANY PARAMETER ADJUSTMENT CAN INTRODUCE SO THE STABILITY POINT IS ALSO MOOT !S SUCH ITS RECOMMENDED
NOT TO CARRY OVER ANY UNDOCUMENTED PARAMETERS FROM THE OLD DATABASES UNLESS YOUR APPLICATION LIKE
/RACLE 3IEBEL 3!0 DOCUMENTATION CLEARLY STATES IT AS A REQUIREMENT
!LSO THERE ARE SOME DOCUMENTED PARAMETERS WHICH SHOULD BE UNSET WHEN MIGRATING TO /RACLE
TO ALLOW /RACLE TO PICK APPROPRIATE VALUES AUTOMATICALLY
SQNUX[TN\d[cXQ[^RZNaTPSNR^d]c) 4HIS PARAMETER SHOULD BE UNSET AS STARTING FROM
/RACLE THE DATABASE CAN PICK THE APPROPRIATE VALUES FOR IT AUTOMATICALLY )N
FACT THIS WAY THERE ACTUALLY TWO DIFFERENT MULTIBLOCK READ COUNT PARAMETERS USED
IN /RACLE ONE FOR COSTING AND ONE FOR EXECUTION 4HIS ARRANGEMENT GENERALLY
WORKS BETTER THAN LEAVING THE PARAMETER SET TO SOME ARBITRARY VALUE WHICH AFFECTS
BOTH THE COSTING AND THE EXECUTION
v _PaP[[T[NTgTRdcX^]N\TbbPVTNbXiT) IF YOU HAVE TUNED THIS PARAMETER IN PAST TO
REDUCE 08 1REF LATCH CONTENTION OR PARALLEL MESSAGING OVERHEAD THEN YOU CAN
UNSET IT AS THE DEFAULT VALUE FOR THIS PARAMETER IS K" STARTING FROM /RACLE
#(!04%2 -)'2!4).' 4/ %8!$!4!
Planning VLDB Migration Downtime
7HEN YOU ARE ESTIMATING THE DOWNTIME OR DATA MIGRATION TIME YOU SHOULD ONLY RELY ON ACTUAL
MEASUREMENTS IN YOUR ENVIRONMENT WITH YOUR DATA YOUR NETWORK CONNECTION DATABASE SETTINGS AND
COMPRESSION OPTIONS 7HILE IT IS POSSIBLE TO LOAD RAW DATA TO A FULL RACK AT 4"HOUR %XADATA 6 OR EVEN
4"HOUR %XADATA 8 YOU PROBABLY WONT GET SUCH LOADING SPEEDS WHEN THE TARGET TABLES ARE
COMPRESSED WITH 0A278E4 OR @D4AH7867 .O MATTER WHICH NUMBERS YOU FIND FROM THIS BOOK OR OFFICIAL
SPECS YOU WILL HAVE TO TEST EVERYTHING OUT YOURSELF END TO END 4HERE ARE SOME TEMPORARY WORKAROUNDS
FOR IMPROVING LOADING RATES FOR EXAMPLE IF YOU HAVE ENOUGH DISK SPACE YOU CAN FIRST LOAD TO NON
COMPRESSED OR 4722@D4AH;>F COMPRESSED TABLES DURING THE DOWNTIME WINDOW AND ONCE IN
PRODUCTION RECOMPRESS INDIVIDUAL PARTITIONS WITH HIGHER COMPRESSION RATES
Summary
)N THIS LENGTHY CHAPTER WE LOOKED INTO THE WIDE RANGE OF TOOLS AVAILABLE FOR MIGRATING YOUR DATA TO
%XADATA 9OU SHOULD CHOOSE THE SIMPLEST APPROACH YOU CAN AS THIS REDUCES RISK AND CAN ALSO SAVE YOUR
OWN TIME 7E RECOMMEND YOU EVALUATE $ATA 0UMP FIRST AS IT CAN TRANSFER THE WHOLE DATABASE OR
SCHEMA !ND IT CAN DO THE JOB FAST AND IS EVEN FLEXIBLE WHEN YOU WANT TO ADJUST THE OBJECT $$, METADATA
IN THE PROCESS /FTEN HOWEVER THIS APPROACH IS NOT FAST ENOUGH OR REQUIRES TOO MUCH TEMPORARY DISK
SPACE )T ALL DEPENDS ON HOW MUCH DATA YOU HAVE TO TRANSFER AND HOW MUCH DOWNTIME YOU ARE ALLOWED
7HEN MOVING 6,$"S THAT ARE TENS OR EVEN HUNDREDS OF TERABYTES IN SIZE YOU MAY HAVE TO GET CREATIVE
AND USE LESS STRAIGHTFORWARD APPROACHES LIKE DATABASE LINKS COPYING READ
ONLY DATA IN ADVANCE OR
PERHAPS DOING A COMPLETELY INCREMENTAL MIGRATION USING ONE OF THE REPLICATION APPROACHES
%VERY ENTERPRISES DATABASE ENVIRONMENTS AND BUSINESS REQUIREMENTS ARE DIFFERENT BUT YOU CAN USE
THE METHODS EXPLAINED IN THIS CHAPTER AS BUILDING BLOCKS 9OU MAY NEED TO COMBINE MULTIPLE TECHNIQUES
IF THE MIGRATED DATABASE IS VERY LARGE AND THE ALLOWED DOWNTIME IS SMALL .O MATTER WHICH TECHNIQUES
YOU USE THE MOST IMPORTANT THING TO REMEMBER IS TO TEST EVERYTHING THE WHOLE MIGRATION PROCESS FROM
END TO END 9OU WILL LIKELY FIND AND FIX MANY PROBLEMS IN ADVANCE THANKS TO SYSTEMATIC TESTING 'OOD
LUCK
C H A P T E R 14
Storage Layout
)N /RACLE G2 /RACLE INTRODUCED !UTOMATIC 3TORAGE -ANAGEMENT !3- AND CHANGED THE WAY WE
THINK OF MANAGING DATABASE STORAGE %XADATA IS TIGHTLY INTEGRATED WITH !3- AND PROVIDES THE UNDERLYING
DISKS THAT HAVE TRADITIONALLY BEEN PRESENTED TO !3- BY THE OPERATING SYSTEM ,OOKING AT ALL THE VARIOUS
INTRICACIES OF CELL STORAGE CAN BE A LITTLE DAUNTING AT FIRST 4HERE ARE SEVERAL LAYERS OF ABSTRACTION BETWEEN
PHYSICAL DISKS AND THE !3- DISK GROUPS MANY $"!S ARE FAMILIAR WITH )F YOUVE NEVER WORKED WITH
/RACLES !3- PRODUCT THERE WILL BE A LOT OF NEW TERMS AND CONCEPTS TO UNDERSTAND THERE AS WELL )N
#HAPTER WE DISCUSSED THE UNDERLYING LAYERS OF %XADATA STORAGE FROM THE PHYSICAL DISKS UP THROUGH THE
CELL DISK LAYER 4HIS CHAPTER WILL PICK UP WHERE #HAPTER LEFT OFF AND DISCUSS HOW CELL DISKS ARE USED TO
CREATE GRID DISKS FOR !3- STORAGE 7ELL BRIEFLY DISCUSS THE UNDERLYING DISK ARCHITECTURE OF THE STORAGE CELL
AND HOW ,INUX PRESENTS PHYSICAL DISKS TO THE APPLICATION LAYER &ROM THERE WELL TAKE A LOOK AT THE
OPTIONS FOR CARVING UP AND PRESENTING %XADATA GRID DISKS TO THE DATABASE TIER 4HE APPROACH /RACLE
RECOMMENDS IS TO CREATE A FEW LARGE hPOOLSv OF DISKS ACROSS ALL STORAGE CELLS 7HILE THIS APPROACH
GENERALLY WORKS WELL FROM A PERFORMANCE STANDPOINT THERE ARE REASONS TO CONSIDER ALTERNATIVE STRATEGIES
3OMETIMES ISOLATING A SET OF STORAGE CELLS TO FORM A SEPARATE STORAGE GRID IS DESIRABLE 4HIS PROVIDES
SEPARATION FROM MORE CRITICAL SYSTEMS WITHIN THE %XADATA ENCLOSURE SO THAT PATCHES MAY BE INSTALLED AND
TESTED BEFORE THEY ARE IMPLEMENTED IN PRODUCTION !LONG THE WAY WELL TAKE A LOOK AT HOW !3- PROVIDES
FAULT RESILIENCY AND STORAGE VIRTUALIZATION TO DATABASES ,ASTLY WELL TAKE A LOOK AT HOW STORAGE SECURITY IS
IMPLEMENTED ON %XADATA 4HE STORAGE CELL IS A HIGHLY PERFORMANT HIGHLY COMPLEX AND HIGHLY
CONFIGURABLE BLEND OF HARDWARE AND SOFTWARE 4HIS CHAPTER WILL TAKE A CLOSE LOOK AT HOW ALL THE VARIOUS
PIECES WORK TOGETHER TO PROVIDE FLEXIBLE HIGH PERFORMANCE STORAGE TO /RACLE DATABASES
Exadata Disk Architecture
7HEN ,INUX BOOTS UP IT RUNS A SCAN TO IDENTIFY DISKS ATTACHED TO THE SERVER 7HEN A DISK IS FOUND THE /3
DETERMINES THE DEVICE DRIVER NEEDED AND CREATES A BLOCK DEVICE CALLED A ,5. FOR APPLICATION ACCESS
7HILE IT IS POSSIBLE FOR APPLICATIONS TO READ AND WRITE DIRECTLY TO THESE BLOCK DEVICES IT IS NOT A COMMON
PRACTICE $OING SO SUBJECTS THE APPLICATION TO CHANGES THAT ARE COMPLICATED TO DEAL WITH &OR EXAMPLE
BECAUSE DEVICE NAMES ARE DYNAMICALLY GENERATED ON BOOTUP ADDING OR REPLACING A DISK CAN CAUSE ALL OF
THE DISK DEVICE NAMES TO CHANGE !3- AND DATABASES NEED FILE PERMISSIONS TO BE SET THAT WILL ALLOW
READWRITE ACCESS TO THESE DEVICES AS WELL )N EARLIER RELEASES OF !3- THIS WAS MANAGED BY THE SYSTEM
ADMINISTRATOR THROUGH THE USE OF NATIVE ,INUX COMMANDS LIKE aPf AND dSTe %XADATA SHIELDS SYSTEM
ADMINISTRATORS AND $"!S FROM THESE COMPLEXITIES THROUGH VARIOUS LAYERS OF ABSTRACTION #ELL DISKS
PROVIDE THE FIRST ABSTRACTION LAYER FOR ,5.S #ELL DISKS ARE USED BY RT[[bae TO MANAGE )/ RESOURCES AT
THE STORAGE CELL 'RID DISKS ARE THE NEXT LAYER OF ABSTRACTION AND ARE THE DISK DEVICES PRESENTED TO THE
DATABASE SERVERS AS !3- $ISKS &IGURE SHOWS HOW CELL DISKS AND GRID DISKS FIT INTO THE OVERALL STORAGE
ARCHITECTURE OF AN %XADATA STORAGE CELL
#(!04%2 34/2!'% ,!9/54
Cell System Cell System
Linux O/S
Area Area
Grid Disk 1
Grid Disk 2
Physical Grid Disk 3
LUN Cell Disk
Disk Grid Disk n
Cell Disk
Figure 14-1. The relationship between physical disks and grid disks
7ITH THE INTRODUCTION OF !3- /RACLE PROVIDED A WAY TO COMBINE MANY PHYSICAL DISKS INTO A SINGLE
STORAGE VOLUME CALLED A disk group $ISK GROUPS ARE THE !3- REPLACEMENT FOR TRADITIONAL FILE SYSTEMS AND
ARE USED TO IMPLEMENT /RACLES 3!-% 3TRIPE AND -IRROR %VERYTHING METHODOLOGY FOR OPTIMIZING DISK
PERFORMANCE !S THE NAME IMPLIES THE GOAL OF 3!-% IS TO SPREAD )/ EVENLY ACROSS ALL PHYSICAL DISKS IN
THE STORAGE ARRAY 6IRTUALIZING STORAGE IN THIS WAY ALLOWS MULTIPLE DATABASES TO SHARE THE SAME PHYSICAL
DISKS )T ALSO ALLOWS PHYSICAL DISKS TO BE ADDED OR REMOVED WITHOUT INTERRUPTING DATABASE OPERATIONS )F A
DISK MUST BE REMOVED !3- AUTOMATICALLY MIGRATES ITS DATA TO THE OTHER DISKS IN THE DISK GROUP BEFORE IT
IS DROPPED 7HEN A DISK IS ADDED TO A DISK GROUP !3- AUTOMATICALLY REBALANCES DATA FROM OTHER DISKS
ONTO THE NEW DISK )N A VERY BASIC !3- CONFIGURATION ,5.S ARE PRESENTED TO !3- AS !3- DISKS !3-
DISKS ARE THEN USED TO CREATE DISK GROUPS WHICH IN TURN ARE USED TO STORE DATABASE FILES SUCH AS DATAFILES
CONTROL FILES AND ONLINE REDO LOGS 4HE ,INUX OPERATING SYSTEM PRESENTS ,5.S TO !3- AS NATIVE BLOCK
DEVICES SUCH AS STe bSP %XADATA VIRTUALIZES PHYSICAL STORAGE THROUGH THE USE OF GRID DISKS AND !3-
DISK GROUPS 'RID DISKS ARE USED FOR CARVING UP CELL DISKS SIMILAR TO THE WAY PARTITIONS ARE USED TO CARVE UP
PHYSICAL DISK DRIVES &IGURE
SHOWS THE RELATIONSHIP BETWEEN CELL DISKS GRID DISKS AND !3- DISK
GROUPS
ASM Disk Group
Grid Disk 1
Grid Disk 2
Grid Disk 3
Grid Disk n
Cell Disk 1
Grid Disks 1-n
Figure 14-2. ASM disk group with its underlying grid disks, and cell disks
#(!04%2 34/2!'% ,!9/54
&AILURE 'ROUPS
"EFORE WE TALK IN MORE DETAIL ABOUT GRID DISKS LETS TAKE A BRIEF DETOUR AND TALK ABOUT HOW DISK
REDUNDANCY IS HANDLED IN THE !3- ARCHITECTURE !3- USES REDUNDANT SETS OF !3- DISKS CALLED failure
groups TO PROVIDE MIRRORING 4RADITIONAL 2!)$ MIRRORING MAINTAINS A BLOCK
FOR
BLOCK DUPLICATE OF THE
ORIGINAL DISK !3- FAILURE GROUPS PROVIDE REDUNDANCY BY ASSIGNING !3- DISKS TO FAILURE GROUPS AND
GUARANTEEING THAT THE ORIGINAL AND MIRROR COPY OF A BLOCK DO NOT RESIDE WITHIN THE SAME FAILURE GROUP )T
IS CRITICALLY IMPORTANT TO SEPARATE PHYSICAL DISKS INTO SEPARATE FAILURE GROUPS %XADATA DOES THIS BY
ASSIGNING THE GRID DISKS FROM EACH STORAGE CELL TO A SEPARATE FAILURE GROUPS &OR EXAMPLE THE FOLLOWING
LISTING SHOWS THE FAIL GROUPS AND GRID DISKS FOR STORAGE CELLS
!S THE NAMES IMPLY THESE FAIL GROUPS
CORRESPOND TO STORAGE CELLS n 4HESE FAIL GROUPS WERE CREATED AND NAMED AUTOMATICALLY BY !3- WHEN
THE GRID DISKS WERE CREATED
BHB)0B<!-bT[TRcUPX[Va^d_]P\TUa^\ePb\NSXbZ^aSTaQh !
508;6A>D?=0<4
24;; 30C0N36N23N N24;;
24;; 30C0N3623N N24;;
24;; 30C0N3623N !N24;;
24;; 30C0N3623N "N24;;
24;; !30C0N3623N N24;; !
24;; !30C0N3623N N24;; !
24;; !30C0N3623N !N24;; !
24;; !30C0N3623N "N24;; !
24;; "30C0N3623N N24;; "
24;; "30C0N3623N N24;; "
24;; "30C0N3623N !N24;; "
24;; "30C0N3623N "N24;; "
&IGURE
SHOWS THE RELATIONSHIP BETWEEN THE 30C0N36 DISK GROUP AND THE FAILURE GROUPS 24;;
24;; ! AND 24;; " .OTE THAT THIS DOES NOT INDICATE WHICH LEVEL OF REDUNDANCY IS BEING USED ONLY THAT
THE 30C0N36 DISK GROUP HAS ITS DATA ALLOCATED ACROSS THREE FAILURE GROUPS
#(!04%2 34/2!'% ,!9/54
DATA_DG
CELL01
CELL03
CELL02
Figure 14-3. ASM failure groups CELL01 – CELL03
4HERE ARE THREE TYPES OF REDUNDANCY IN !3- %XTERNAL .ORMAL AND (IGH
External Redundancy .O REDUNDANCY IS PROVIDED BY !3- )T IS ASSUMED THAT
THE STORAGE ARRAY USUALLY A 3!. IS PROVIDING ADEQUATE REDUNDANCY IN MOST
CASES 2!)$ 2!)$ OR 2!)$ 4HIS HAS BECOME THE MOST COMMON METHOD
WHERE LARGE STORAGE AREA NETWORKS ARE USED FOR !3- STORAGE )N THE %XADATA
STORAGE GRID !3- PROVIDES THE ONLY MECHANISM FOR MIRRORING )F %XTERNAL
2EDUNDANCY WERE USED THE LOSS OF A SINGLE DISK DRIVE WOULD MEAN A
CATASTROPHIC LOSS OF ALL !3- DISKGROUPS USING THAT DISK )T ALSO MEANS THAT EVEN
THE TEMPORARY LOSS OF A STORAGE CELL REBOOT CRASH OR THE LIKE WOULD MAKE ALL
DISK GROUPS USING STORAGE ON THE FAILED CELL UNAVAILABLE FOR THE DURATION OF THE
OUTAGE
Normal Redundancy .ORMAL 2EDUNDANCY MAINTAINS TWO COPIES OF DATA BLOCKS
IN SEPARATE FAILURE GROUPS $ATABASES WILL ALWAYS ATTEMPT TO READ FROM THE
PRIMARY COPY OF A DATA BLOCK FIRST 3ECONDARY COPIES ARE ONLY READ WHEN THE
PRIMARY BLOCKS ARE UNAVAILABLE !T LEAST TWO FAILURE GROUPS ARE REQUIRED FOR
.ORMAL 2EDUNDANCY BUT MANY MORE THAN THAT MAY BE USED &OR EXAMPLE AN
%XADATA FULL RACK CONFIGURATION HAS STORAGE CELLS AND EACH STORAGE CELL
CONSTITUTES A FAILURE GROUP 7HEN DATA IS WRITTEN TO THE DATABASE THE FAILURE
GROUP USED FOR THE PRIMARY COPY OF A BLOCK ROTATES FROM FAILURE GROUP TO FAILURE
GROUP IN A ROUND ROBIN FASHION 4HIS ENSURES THAT DISKS IN ALL FAILURE GROUPS
PARTICIPATE IN READ OPERATIONS
High Redundancy (IGH 2EDUNDANCY IS SIMILAR TO .ORMAL 2EDUNDANCY EXCEPT
THAT THREE COPIES OF DATA BLOCKS ARE MAINTAINED IN SEPARATE FAILURE GROUPS
#(!04%2 34/2!'% ,!9/54
'RID $ISKS
'RID DISKS ARE CREATED WITHIN CELL DISKS WHICH YOU MAY RECALL ARE MADE UP OF PHYSICAL DISKS AND &LASH
#ACHE MODULES )N A SIMPLE CONFIGURATION ONE GRID DISK CAN BE CREATED PER CELL DISK -ORE COMPLEX
CONFIGURATIONS HAVE MULTIPLE GRID DISKS PER CELL DISK 4HE #ELL#,) COMMAND [XbcVaXSSXbZ DISPLAYS THE
VARIOUS CHARACTERISTICS OF GRID DISKS &OR EXAMPLE THE FOLLOWING OUTPUT SHOWS THE RELATIONSHIP BETWEEN
GRID DISKS AND CELL DISKS THE TYPE OF DEVICE ON WHICH THEY ARE CREATED AND THEIR SIZE
JT]ZRT[ ")a^^cLa^^c
-RT[[R[X
2T[[2;8)AT[TPbT ! " ?a^SdRcX^]^]BPc>Rc!" &)!")"!23C!
2^_haXVWcR! &! (>aPR[T 0[[aXVWcbaTbTaeTS
2T[[4UUXRXT]RhAPcX^)! <
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TRT[[SXbZSXbZch_TbXiT
30C0N23N NRT[[ "23N NRT[[ "7PaS3XbZ !'! ' !$6
30C0N23N NRT[[ "23N NRT[[ "7PaS3XbZ !'! ' !$6
q
5;0B7N53N NRT[[ "53N NRT[[ "5[PbW3XbZ# &' !$6
5;0B7N53N NRT[[ "53N NRT[[ "5[PbW3XbZ# &' !$6
q
!3- DOESNT KNOW ANYTHING ABOUT PHYSICAL DISKS OR CELL DISKS 'RID DISKS ARE WHAT THE STORAGE CELL
PRESENTS TO THE DATABASE SERVERS AS !3- DISKS TO BE USED FOR #LUSTERWARE AND DATABASE STORAGE !3-
USES GRID DISKS TO CREATE DISK GROUPS IN THE SAME WAY CONVENTIONAL BLOCK DEVICES ARE USED ON A NON
%XADATA PLATFORM 4O ILLUSTRATE THIS THE FOLLOWING QUERY SHOWS WHAT !3- DISKS LOOK LIKE ON A NON
%XADATA SYSTEM
BHB)0B< -bT[TRc_PcWc^cP[N\QUPX[Va^d_
Ua^\ePb\NSXbZ
^aSTaQhUPX[Va^d_Va^d_N]d\QTa_PcW*
?0C7C>C0;N<1508;6A>D?
STe bSS ###30C0
STe bST ###30C0 !
q
STe bSY "' %A42>
STe bSZ "' %A42> !
q
4HE SAME QUERY ON %XADATA REPORTS GRID DISKS THAT HAVE BEEN CREATED AT THE STORAGE CELL
BHB)0B< -bT[TRc_PcWc^cP[N\QUPX[Va^d_
Ua^\ePb\NSXbZ
^aSTaQhUPX[Va^d_Va^d_N]d\QTa_PcW*
?0C7C>C0;N<1508;6A>D?
^ (! %' ! " 30C0N23N NRT[[ " "% 24;;
^ (! %' ! " 30C0N23N NRT[[ " "% 24;;
^ (! %' ! " 30C0N23N !NRT[[ " "% 24;;
#(!04%2 34/2!'% ,!9/54
q
^ (! %' ! " A42>N23N NRT[[ ("#$%24;;
^ (! %' ! " A42>N23N NRT[[ ("#$%24;;
^ (! %' ! " A42>N23N !NRT[[ !"!%#24;;
q
^ (! %' ! # 30C0N23N NRT[[ ! " "% 24;; !
^ (! %' ! # 30C0N23N NRT[[ ! " "% 24;; !
^ (! %' ! # 30C0N23N !NRT[[ ! " "% 24;; !
q
^ (! %' ! # A42>N23N NRT[[ !("#$%24;; !
^ (! %' ! # A42>N23N NRT[[ !("#$%24;; !
^ (! %' ! # A42>N23N !NRT[[ ! !"!%#24;; !
q
^ (! %' ! $ 30C0N23N NRT[[ " " "% 24;; "
^ (! %' ! $ 30C0N23N NRT[[ " " "% 24;; "
^ (! %' ! $ 30C0N23N !NRT[[ " " "% 24;; "
q
^ (! %' ! $ A42>N23N NRT[[ "("#$%24;; "
^ (! %' ! $ A42>N23N NRT[[ "("#$%24;; "
^ (! %' ! $ A42>N23N !NRT[[ " !"!%#24;; "
q
4YING IT ALL TOGETHER &IGURE
SHOWS HOW THE LAYERS OF STORAGE FIT TOGETHER FROM THE STORAGE CELL TO
THE !3- DISK GROUP .OTE THAT THE ,INUX /3 PARTITIONS ON THE FIRST TWO CELL DISKS IN EACH STORAGE CELL ARE
IDENTIFIED BY A DOTTED LINE 7ELL TALK A LITTLE MORE ABOUT THE /3 PARTITIONS LATER IN THE CHAPTER AND IN
MUCH MORE DETAIL IN #HAPTER
#(!04%2 34/2!'% ,!9/54
!3- $ISK 'ROUPS
DATA_DG RECO_DG SYSTEM_DG
CELL01 CELL03 CELL01 CELL03 CELL01 CELL03
CELL02 CELL02 CELL02
Linux O/S 'RID $ISKS
SYSTEM_DG
Storage Cell 1 DATA_DG
Failure Group: CELL01
RECO_DG
Cell Disk 1 Cell Disk 2 Cell Disk 3 Cell Disk 4 Cell Disk 10 Cell Disk 11 Cell Disk 12
SYSTEM_DG
DATA_DG
Storage Cell 2
Failure Group: CELL02 RECO_DG
Cell Disk 1 Cell Disk 2 Cell Disk 3 Cell Disk 4 Cell Disk 10 Cell Disk 11 Cell Disk 12
SYSTEM_DG
DATA_DG
Storage Cell 3
Failure Group: CELL03 RECO_DG
Cell Disk 1 Cell Disk 2 Cell Disk 3 Cell Disk 4 Cell Disk 10 Cell Disk 11 Cell Disk 12
Figure 14-4. Storage virtualization on Exadata
3TORAGE !LLOCATION
$ISK DRIVES STORE DATA IN CONCENTRIC BANDS CALLED tracks "ECAUSE THE OUTER TRACKS OF A DISK HAVE MORE
SURFACE AREA THEY ARE ABLE TO STORE MORE DATA THAN THE INNER TRACKS !S A RESULT DATA TRANSFER RATES ARE
HIGHER FOR THE OUTER TRACKS AND DECLINE SLIGHTLY AS YOU MOVE TOWARD THE INNERMOST TRACK &IGURE
SHOWS HOW TRACKS ARE LAID OUT ACROSS THE DISK SURFACE FROM FASTEST TO SLOWEST
#(!04%2 34/2!'% ,!9/54
Fastest Track
Slowest Track
Figure 14-5. Disk tracks
%XADATA PROVIDES TWO POLICIES FOR ALLOCATING GRID DISK STORAGE ACROSS THE SURFACE OF DISK DRIVES 4HE
FIRST METHOD IS THE DEFAULT BEHAVIOR FOR ALLOCATING SPACE ON CELL DISKS )T HAS NO OFFICIAL NAME SO FOR
PURPOSES OF THIS DISCUSSION )LL REFER TO IT AS THE default policy 4HE OTHER ALLOCATION POLICY /RACLE CALLS
interleaving 4HESE TWO ALLOCATION POLICIES ARE DETERMINED WHEN THE CELL DISKS ARE CREATED )NTERLEAVING
MUST BE EXPLICITLY ENABLED USING THE X]cTa[TPeX]V PARAMETER OF THE RaTPcTRT[[SXbZ COMMAND &OR A
COMPLETE DISCUSSION ON CREATING CELL DISKS REFER TO CHAPTER
Fastest Available Tracks First
4HE DEFAULT POLICY SIMPLY ALLOCATES SPACE STARTING WITH THE FASTEST AVAILABLE TRACKS FIRST MOVING INWARD AS
SPACE IS CONSUMED 5SING THIS POLICY THE FIRST GRID DISK CREATED ON EACH CELL DISK WILL BE GIVEN THE FASTEST
STORAGE WHILE THE LAST GRID DISK CREATED WILL BE RELEGATED TO THE SLOWER INNER TRACKS OF THE DISK SURFACE
7HEN PLANNING YOUR STORAGE GRID REMEMBER THAT GRID DISKS ARE THE BUILDING BLOCKS FOR !3- DISK GROUPS
4HESE DISK GROUPS WILL IN TURN BE USED TO STORE TABLES INDEXES ONLINE REDO LOGS ARCHIVED REDO LOGS AND
SO ON 4O MAXIMIZE DATABASE PERFORMANCE FREQUENTLY ACCESSED OBJECTS SUCH AS TABLES INDEXES AND
ONLINE REDO LOGS SHOULD BE STORED IN THE HIGHEST PRIORITY GRID DISKS ,OW PRIORITY GRID DISKS SHOULD BE
USED FOR LESS PERFORMANCE SENSITIVE OBJECTS SUCH AS DATABASE BACKUPS ARCHIVED REDO LOGS AND FLASHBACK
LOGS &IGURE SHOWS HOW GRID DISKS ARE ALLOCATED USING THE DEFAULT ALLOCATION POLICY
#(!04%2 34/2!'% ,!9/54
3rd Grid Disk
st (STAGE_DG Disk Group) 2nd Grid Disk
1 Grid Disk (RECO_DG Disk Group)
(DATA_DG Disk group)
Figure 14-6. The default allocation policy
4ABLE
SHOWS THE PERFORMANCE EFFECT ON THE !3- DISK GROUPS FROM THE FIRST TO THE LAST GRID DISK
CREATED WHEN USING THE DEFAULT ALLOCATION POLICY 9OU WONT FIND THE TERM h)/ 0ERFORMANCE 2ATINGv IN
THE /RACLE DOCUMENTATION )TS A TERM )M COINING HERE TO DESCRIBE THE RELATIVE PERFORMANCE CAPABILITIES
OF EACH DISK GROUP DUE TO ITS LOCATION ON THE SURFACE OF THE PHYSICAL DISK DRIVE
Table 14-1. I/O Performance – Default Allocation Policy
ASM Disk Group I/O Performance Rating
30C0N36
A42>N36
BC064N36
&OURTH DISK GROUP
BHBC4<N36
Interleaving
4HE OTHER POLICY INTERLEAVING ATTEMPTS TO EVEN OUT PERFORMANCE OF THE FASTER AND SLOWER TRACKS BY
ALLOCATING SPACE IN AN ALTERNATING FASHION BETWEEN THE SLOWER AND FASTER TRACKS OF THE DISKS 4HIS IS DONE
BY SPLITTING EACH CELL DISK INTO TWO REGIONS AN OUTER REGION AND AN INNER REGION 'RID DISKS ARE SLICES OF
CELL DISKS THAT WILL BE USED TO CREATE !3- DISK GROUPS &OR EXAMPLE THE FOLLOWING COMMAND CREATES
GRID DISKS ONE PER PHYSICAL DISK SEE &IGURE
ON #ELL TO BE USED FOR THE 30C0N36 DISK GROUP
#(!04%2 34/2!'% ,!9/54
2T[[2;8-2A40C46A8338B:0;;70A338B:?A458G,30C0N36bXiT,&## %' "6
4HESE GRID DISKS WERE USED TO CREATE THE FOLLOWING $!4!?$' DISK GROUP .OTICE HOW EACH GRID DISK
WAS CREATED ON A SEPARATE CELL DISK
BHB)0B<!-bT[TRcSV ]P\TSXbZVa^d_
bdQbcaS ]P\T% !RT[[NSXbZ
S ]P\TVaXSNSXbZ
Ua^\ePb\NSXbZVa^d_SV
ePb\NSXbZS
fWTaTSV Va^d_N]d\QTa,S Va^d_N]d\QTa
P]SSV ]P\T,30C0
P]SUPX[Va^d_,24;; "
^aSTaQh !*
38B:6A>D?24;;N38B:6A83N38B:
30C0N3623N N24;; "30C0N36N23N N24;; "
30C0N3623N N24;; "30C0N36N23N N24;; "
30C0N3623N !N24;; "30C0N36N23N !N24;; "
30C0N3623N "N24;; "30C0N36N23N "N24;; "
q
30C0N3623N N24;; "30C0N36N23N N24;; "
30C0N3623N N24;; "30C0N36N23N N24;; "
5SING INTERLEAVING IN THIS EXAMPLE 30C0N36N23N N24;; " THE FIRST GRID DISK IS ALLOCATED TO THE OUTER
MOST TRACKS OF THE OUTER FASTEST REGION OF THE 23N N24;; " CELL DISK 4HE NEXT GRID DISK
30C0N36N23N N24;; " IS CREATED ON THE OUTERMOST TRACKS OF THE SLOWER INNER REGION OF CELL DISK
23N N24;; " 4HIS PATTERN CONTINUES UNTIL ALL GRID DISKS ARE ALLOCATED 7HEN THE NEXT SET OF GRID DISKS
IS CREATED FOR THE A42>N36 DISK GROUP THEY START WITH THE INNER REGION OF CELL DISK AND ALTERNATE FROM
INNER TO OUTER REGION UNTIL ALL GRID DISKS ARE CREATED &IGURE
SHOWS HOW THE INTERLEAVING POLICY
WOULD LOOK IF TWO GRID DISK GROUPS WERE CREATED
Second Disk Group
(RECO_DG)
Cell Disk 1
Outer Region
Cell Disk 2 (faster)
Cell Disk 3
Inner Region
(slower)
First Disk Group
(DATA_DG)
Figure 14-7. The interleaving allocation policy
#(!04%2 34/2!'% ,!9/54
4ABLE
SHOWS THE PERFORMANCE EFFECT ON THE !3- DISK GROUPS FROM THE FIRST TO THE LAST GRID DISK
CREATED WHEN THE INTERLEAVING ALLOCATION POLICY IS USED
Table 14-2. I/O Performance – Interleaving Policy
ASM Disk Group I/O Performance Rating
30C0N36
A42>N36
BHBC4<N36
&OURTH DISK GROUP
&IFTH DISK GROUP
!S YOU CAN SEE THE MAIN DIFFERENCE BETWEEN THE DEFAULT POLICY AND THE INTERLEAVING POLICY IS THAT
DEFAULT PROVIDES FINER
GRAINED CONTROL OVER !3- DISKS 7ITH THE DEFAULT POLICY YOU HAVE THE ABILITY TO
CHOOSE WHICH SET OF GRID DISKS WILL BE GIVEN THE ABSOLUTE FASTEST POSITION ON THE DISK 4HE INTERLEAVING
POLICY HAS THE EFFECT OF EVENING OUT THE PERFORMANCE OF GRID DISKS )N PRACTICE THIS GIVES THE FIRST TWO SETS
OF GRID DISKS FOR 30C0N36 AND A42>N36 THE SAME PERFORMANCE CHARACTERISTICS 4HIS MAY BE USEFUL IF THE
PERFORMANCE DEMANDS OF THE FIRST TWO DISK GROUPS ARE EQUAL )N OUR EXPERIENCE THIS IS RARELY THE CASE
5SUALLY THERE IS A CLEAR WINNER WHEN IT COMES TO THE PERFORMANCE DEMANDS OF A DATABASE ENVIRONMENT
4ABLES INDEXES ONLINE REDO LOGS THE 30C0N36 DISK GROUP HAVE MUCH HIGHER PERFORMANCE REQUIREMENTS
THAN DATABASE BACKUPS ARCHIVED REDO LOGS AND FLASHBACK LOGS WHICH ARE USUALLY STORED IN THE A42> DISK
GROUP 5NLESS THERE ARE SPECIFIC REASONS FOR USING INTERLEAVING WE RECOMMEND USING THE DEFAULT POLICY
Creating Grid Disks
"EFORE WE RUN THROUGH A FEW EXAMPLES OF HOW TO CREATE GRID DISKS LETS TAKE A QUICK LOOK AT SOME OF THEIR
KEY ATTRIBUTES
v -ULTIPLE GRID DISKS MAY BE CREATED ON A SINGLE CELL DISK BUT A GRID DISK MAY NOT SPAN
MULTIPLE CELL DISKS
v 3TORAGE FOR GRID DISKS IS ALLOCATED IN - !LLOCATION 5NITS !5S AND IS ROUNDED DOWN
IF THE SIZE REQUESTED IS NOT A MULTIPLE OF THE !5 SIZE
v 'RID DISKS MAY BE CREATED ONE AT A TIME OR IN GROUPS WITH A COMMON NAME PREFIX
v 'RID DISK NAMES MUST BE UNIQUE WITHIN A STORAGE CELL AND SHOULD BE UNIQUE ACROSS ALL
STORAGE CELLS
v 'RID DISK NAMES SHOULD INCLUDE THE NAME OF THE CELL DISK ON WHICH THEY RESIDE
/NCE A GRID DISK IS CREATED ITS NAME IS VISIBLE FROM !3- IN THE E0B<N38B: VIEW )N OTHER WORDS GRID
DISKS !3- DISKS )T IS VERY IMPORTANT TO NAME GRID DISKS IN SUCH A WAY THAT THEY CAN EASILY BE ASSOCIATED
WITH THE PHYSICAL DISK TO WHICH THEY BELONG IN THE EVENT OF DISK FAILURE 4O FACILITATE THIS GRID DISK NAMES
SHOULD INCLUDE BOTH OF THE FOLLOWING
#(!04%2 34/2!'% ,!9/54
v 4HE NAME OF THE !3- DISK GROUP FOR WHICH IT WILL BE USED
v 4HE CELL DISK NAME WHICH INCLUDES THE NAME OF THE STORAGE CELL
&IGURE
SHOWS THE PROPERLY FORMATTED NAME FOR A GRID DISK BELONGING TO THE C4BCN36 DISK GROUP
CREATED ON CELL DISK 23N NRT[[ "
TEST_DG_CD_00_cell03
Disk Group Cell Disk
Figure 14-8. Grid disk naming
#REATING 'RID $ISKS
4HE #ELL#,) COMMAND RaTPcTVaXSSXbZ IS USED TO CREATE GRID DISKS )T MAY BE USED TO CREATE INDIVIDUAL
GRID DISKS ONE AT A TIME OR IN GROUPS )F GRID DISKS ARE CREATED ONE AT A TIME IT IS UP TO YOU TO PROVIDE THE
COMPLETE GRID DISK NAME 4HE FOLLOWING EXAMPLE CREATES ONE PROPERLY NAMED -" GRID DISK ON CELL
DISK 23N NRT[[ " )F WE HAD OMITTED THE bXiT, < PARAMETER THE RESULTING GRID DISK WOULD HAVE
CONSUMED ALL FREE SPACE ON THE CELL DISK
2T[[2;8-RaTPcTVaXSSXbZC4BCN36N23N NRT[[ "~
RT[[SXbZ,23N NRT[[ "bXiT, <
6aXS3XbZC4BCN36N23N NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TRT[[SXbZbXiT~
fWTaT]P\T,C4BCN36N23N NRT[[ "
C4BCN36N23N NRT[[ "23N NRT[[ "(%<
4HERE ARE DRIVES PER STORAGE CELL AND THE NUMBER OF STORAGE CELLS VARIES FROM FOR A QUARTER RACK
TO FOR A FULL RACK 4HAT MEANS YOU WILL BE CREATING A MINIMUM OF GRID DISKS FOR A QUARTER RACK AND
UP TO GRID DISKS FOR A FULL RACK &ORTUNATELY #ELL#,) PROVIDES A WAY TO CREATE ALL THE GRID DISKS NEEDED
FOR A GIVEN !3- DISK GROUP IN ONE COMMAND &OR EXAMPLE THE FOLLOWING COMMAND CREATES ALL THE GRID
DISKS FOR THE !3- DISK GROUP C4BCN36
2T[[2;8-RaTPcTVaXSSXbZP[[WPaSSXbZ_aTUXg,C4BCbXiT, <
6aXS3XbZC4BCN23N NRT[[ "bdRRTbbUd[[hRaTPcTS
6aXS3XbZC4BCN23N NRT[[ "bdRRTbbUd[[hRaTPcTS
q
6aXS3XbZC4BCN23N NRT[[ "bdRRTbbUd[[hRaTPcTS
6aXS3XbZC4BCN23N NRT[[ "bdRRTbbUd[[hRaTPcTS
7HEN THIS VARIANT OF THE RaTPcTVaXSSXbZ COMMAND IS USED #ELL#,) AUTOMATICALLY CREATES ONE GRID
DISK ON EACH CELL DISK NAMING THEM WITH THE PREFIX YOU PROVIDED IN THE FOLLOWING MANNER
{prefix}N{celldisk_name}
4HE OPTIONAL bXiT PARAMETER SPECIFIES THE SIZE OF EACH INDIVIDUAL GRID DISK )F NO SIZE IS PROVIDED THE
RESULTING GRID DISKS WILL CONSUME ALL REMAINING FREE SPACE OF THEIR RESPECTIVE CELL DISK 4HE P[[WPaSSXbZ
#(!04%2 34/2!'% ,!9/54
PARAMETER INSTRUCTS #ELL#,) TO USE ONLY DISK BASED CELL DISKS *UST IN CASE YOU ARE WONDERING &LASH
#ACHE MODULES ARE ALSO PRESENTED AS CELL DISKS OF TYPE 5[PbW3XbZ AND MAY BE USED FOR CREATING GRID
DISKS AS WELL 7ELL DISCUSS FLASH DISKS LATER ON IN THIS CHAPTER 4HE FOLLOWING COMMAND SHOWS THE GRID
DISKS CREATED
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TRT[[3XbZSXbZCh_TbXiT
fWTaT]P\T[XZTC4BCN
C4BCN23N NRT[[ "53N NRT[[ "7PaS3XbZ(%<
C4BCN23N NRT[[ "53N NRT[[ "7PaS3XbZ(%<
q
C4BCN23N NRT[[ "53N NRT[[ "7PaS3XbZ(%<
C4BCN23N NRT[[ "53N NRT[[ "7PaS3XbZ(%<
'RID $ISK 3IZING
!S WE DISCUSSED EARLIER GRID DISKS ARE EQUIVALENT TO !3- DISKS 4HEY ARE LITERALLY THE BUILDING BLOCKS OF
THE !3- DISK GROUPS YOU WILL CREATE 4HE BHBC4<N36 DISK GROUP IS CREATED WHEN %XADATA IS INSTALLED ON A
SITE )T IS PRIMARILY USED TO STORE THE /#2 AND VOTING FILES USED BY /RACLE CLUSTERWARE 'RID
)NFRASTRUCTURE (OWEVER THERE IS NO REASON BHBC4<N36 CANNOT BE USED TO STORE OTHER OBJECTS SUCH AS
TABLESPACES FOR THE $ATABASE &ILE 3YSTEM $"&3 )N ADDITION TO THE BHBC4<N36 OR 315BN36 DISK GROUP
%XADATA IS ALSO DELIVERED WITH 30C0 AND A42> DISK GROUPS TO BE USED FOR DATABASE FILES AND &AST 2ECOVERY
!REAS "UT THESE DISK GROUPS MAY ACTUALLY BE CREATED WITH WHATEVER NAMES MAKE THE MOST SENSE FOR YOUR
ENVIRONMENT &OR CONSISTENCY THIS CHAPTER USES THE NAMES BHBC4<N36 30C0N36 AND A42>N36 )F YOU ARE
CONSIDERING SOMETHING OTHER THAN THE hFACTORY DEFAULTSv FOR YOUR DISK GROUP CONFIGURATION REMEMBER
THAT A MAIN REASON FOR USING MULTIPLE !3- DISK GROUPS ON %XADATA IS TO PRIORITIZE )/ PERFORMANCE 4HE
FIRST GRID DISKS YOU CREATE WILL BE THE FASTEST RESULTING IN HIGHER PERFORMANCE FOR THE ASSOCIATED !3- DISK
GROUP
■ .OTE 7HEN %XADATA 6 ROLLED OUT BHBC4<N36 WAS THE DISK GROUP USED TO STORE /#2 AND VOTING FILES FOR THE
/RACLE CLUSTERWARE 7HEN %XADATA 8 WAS INTRODUCED THIS DISK GROUP WAS RENAMED TO 315BN36 PRESUMABLY
BECAUSE THERE WAS QUITE A BIT OF USABLE SPACE LEFT OVER THAT MADE FOR A NICE LOCATION FOR A MODERATELY SIZED $"&3
FILE SYSTEM !LSO THE OTHER DEFAULT DISK GROUP NAMES CHANGED SOMEWHAT WHEN 8 CAME OUT 4HE %XADATA
$ATABASE -ACHINE NAME WAS ADDED AS A POSTFIX TO THE 30C0 AND A42> DISK GROUP NAMES &OR EXAMPLE THE
MACHINE NAME FOR ONE OF OUR LAB SYSTEMS IS 4=: 3O 30C0 BECAME 30C0N4=: AND A42> BECAME A42>N4=:
"Y THE WAY /RACLE RECOMMENDS YOU CREATE A SEPARATE DATABASE FOR $"&3 BECAUSE IT REQUIRES
INSTANCE PARAMETER SETTINGS THAT WOULD NOT BE OPTIMAL FOR TYPICAL APPLICATION DATABASES
3OME OF THE MOST COMMON !3- DISK GROUPS ARE
BHBC4<N36: 4HIS DISK GROUP IS THE LOCATION FOR CLUSTERWARES /#2 AND VOTING FILES
)T MAY ALSO BE USED FOR OTHER FILES WITH SIMILAR PERFORMANCE REQUIREMENTS &OR
/#2 AND VOTING FILES NORMAL REDUNDANCY IS THE MINIMUM REQUIREMENT
.ORMAL REDUNDANCY WILL CREATE VOTING FILES AND /#2 FILES 4HE VOTING FILES
MUST BE STORED IS SEPARATE !3- FAILURE GROUPS 2ECALL THAT ON %XADATA EACH
#(!04%2 34/2!'% ,!9/54
STORAGE CELL CONSTITUTES A FAILURE GROUP 4HIS MEANS THAT ONLY NORMAL
REDUNDANCY MAY BE USED FOR AN %XADATA QUARTER RACK CONFIGURATION STORAGE
CELLSFAILURE GROUPS /NLY IN HALF RACK AND FULL RACK CONFIGURATIONS OR
STORAGE CELLSFAILURE GROUPS ARE THERE A SUFFICIENT NUMBER OF FAILURE GROUPS TO
STORE THE REQUIRED NUMBER OF VOTING AND /#2 FILES REQUIRED BY HIGH
REDUNDANCY 4ABLE
SUMMARIZES THE STORAGE REQUIREMENTS FOR /#2 AND
VOTING FILES AT VARIOUS LEVELS OF REDUNDANCY .OTE THAT %XTERNAL 2EDUNDANCY IS
NOT SUPPORTED ON %XADATA 7EVE INCLUDED IT IN THE TABLE FOR REFERENCE ONLY
30C0N36: 4HIS DISK GROUP IS USED FOR STORING FILES ASSOCIATED WITH THE
SQNRaTPcTNUX[TNSTbc DATABASE PARAMETER 4HESE INCLUDE DATAFILES ONLINE REDO
LOGS CONTROL FILES AND SPFILES
A42>N36: 4HIS DISK GROUP IS WHAT USED TO BE CALLED &LASH 2ECOVERY !REA &2!
3OMETIME AFTER G2 /RACLE RENAMED IT TO THE h&AST 2ECOVERY !REAv RUMOR HAS
IT THAT THE OVERUSE OF h&LASHv WAS CAUSING CONFUSION AMONG THE MARKETING TEAM
3O TO CLARIFY THIS DISKGROUP WILL BE USED TO STORE EVERYTHING CORRESPONDING TO
THE SQNaTR^eTahNUX[TNSTbc DATABASE PARAMETER )T INCLUDES ONLINE DATABASE
BACKUPS AND COPIES MIRROR COPIES OF THE ONLINE REDO LOG FILES MIRROR COPIES OF
THE CONTROL FILE ARCHIVED REDO LOGS FLASHBACK LOGS AND $ATA 0UMP EXPORTS
Table 14-3. OCR & Voting File Storage Requirements
Redundancy Min # of Disks OCR Voting Total
%XTERNAL -" -" -"
.ORMAL -" -" '"
(IGH -" '" '"
2ECALL THAT %XADATA STORAGE CELLS ARE ACTUALLY FINELY TUNED ,INUX SERVERS WITH INTERNAL DISK DRIVES
/RACLE COULD HAVE DEDICATED TWO OF THESE INTERNAL DISKS TO RUN THE OPERATING SYSTEM BUT DOING SO WOULD
HAVE WASTED A LOT OF SPACE )NSTEAD THEY CARVED OFF SEVERAL SMALL PARTITIONS ON THE FIRST TWO DISKS IN THE
ENCLOSURE 4HESE PARTITIONS ABOUT '" EACH CREATE A SLIGHT IMBALANCE IN THE SIZE OF THE CELL DISKS 4HIS
IMBALANCE IS SO SMALL IT DOESNT AFFECT PERFORMANCE BUT IT IS SOMETHING THAT YOU WILL HAVE TO WORK WITH
WHEN PLANNING YOUR GRID DISK ALLOCATION STRATEGY &IGURE ILLUSTRATES THIS IMBALANCE
#(!04%2 34/2!'% ,!9/54
Linux Partitions
Cell Disk 3 Cell Disk 11 Cell Disk 12
Cell Disk 1 Cell Disk 2
Figure 14-9. Cell disk layout
4HIS RESERVED SPACE CAN BE SEEN BY RUNNING USXbZ ON ONE OF THE STORAGE CELLS 4HE STe bSP"
PARTITION IN THE LISTING IS THE LOCATION OF THE CELL DISK !LL OTHER PARTITIONS ARE USED BY THE ,INUX OPERATING
SYSTEM
JT]ZRT[ ")a^^cL a^^c
-USXbZ [d STe bSP
q
3TeXRT1^^cBcPac4]S1[^RZb8SBhbcT\
STe bSP %"!# (&# ! #$%US;X]dgaPXSPdc^STcTRc
STe bSP!!# (&$!$& "(' "!'";X]dg
/dev/sda3 257040 3843486989 1921614975 83 Linux
STe bSP#"'#"#'%(( "( #!(" #" # " !UF($4gcS;10
STe bSP$"'#"#'& $""'%##$ ' # #'!"' US;X]dgaPXSPdc^STcTRc
STe bSP%"'%##$ '&'"''$# %%"( #'!"' US;X]dgaPXSPdc^STcTRc
STe bSP&"''$# %& ""''(% (% #! (%#$ US;X]dgaPXSPdc^STcTRc
STe bSP'"''(% (%%'"'("' !$%(! (%#$ US;X]dgaPXSPdc^STcTRc
STe bSP("'("' !%"""'(&(($$"#! (%#$ US;X]dgaPXSPdc^STcTRc
STe bSP "'(&(($$('"'((#!$" (& #'% US;X]dgaPXSPdc^STcTRc
STe bSP "'((#!$"'""( #!(" #!#""' %US;X]dgaPXSPdc^STcTRc
9OU CAN SEE THE SMALLER CELL DISKS IN THE bXiT ATTRIBUTE WHEN YOU RUN THE [XbcRT[[SXbZ COMMAND
2T[[2;8-[XbcRT[[SXbZPccaXQdcTb]P\TSTeXRT?PacXcX^]bXiT~
fWTaTSXbZCh_T,7PaS3XbZ
CD_00_cell03 /dev/sda3 1832.59375G
CD_01_cell03 /dev/sdb3 1832.59375G
23N !NRT[[ " STe bSR '% & " !$6
23N "NRT[[ " STe bSS '% & " !$6
23N #NRT[[ " STe bST '% & " !$6
23N $NRT[[ " STe bSU '% & " !$6
23N %NRT[[ " STe bSV '% & " !$6
23N &NRT[[ " STe bSW '% & " !$6
23N 'NRT[[ " STe bSX '% & " !$6
23N (NRT[[ " STe bSY '% & " !$6
23N NRT[[ " STe bSZ '% & " !$6
23N NRT[[ " STe bS[ '% & " !$6
,ETS TAKE A LOOK AT A FAIRLY TYPICAL CONFIGURATION TO ILLUSTRATE HOW GRID DISKS ARE ALLOCATED IN THE
STORAGE CELL )N THIS EXAMPLE WELL CREATE GRID DISKS TO SUPPORT THREE !3- DISK GROUPS BHBC4<N36 IS FAIRLY
LARGE AND WILL BE USED FOR $"&3 CLUSTERED FILE SYSTEM STORAGE IN ADDITION TO THE /#2 AND 6OTING FILES
#(!04%2 34/2!'% ,!9/54
v 30C0N36
v A42>N36
v BHBC4<N36
4HIS STORAGE CELL IS CONFIGURED WITH 3!4! DISKS SO THE TOTAL SPACE PER STORAGE CELL IS TERABYTES
4ABLE
SHOWS WHAT THE ALLOCATION WOULD LOOK LIKE )N ALL CASES THE PHYSICAL DISK SIZE IS
4HE SUM OF THE DISK SIZES IS THEN
Table 14-4. I/O Grid Disk Space Allocation (all sizes expressed in gigabytes)
Cell Disk OS DATA_DG SYSTEM_DG RECO_DG Total Grid Disk Space
23N NRT[[ "
23N NRT[[ "
23N !NRT[[ "
23N "NRT[[ "
23N #NRT[[ "
23N $NRT[[ "
23N %NRT[[ "
23N &NRT[[ "
23N 'NRT[[ "
23N (NRT[[ "
23N NRT[[ "
23N NRT[[ "
Total Per Cell 58.22 8,936.4 4,468.087500 8,877.73125 22,282.218750
#REATING A CONFIGURATION LIKE THIS IS FAIRLY SIMPLE 4HE FOLLOWING COMMANDS CREATE GRID DISKS
ACCORDING TO THE ALLOCATION IN 4ABLE
2T[[2;8-RaTPcTVaXSSXbZP[[_aTUXg,30C0N36bXiT,&## &6
2T[[2;8-RaTPcTVaXSSXbZP[[_aTUXg,BHBC4<N36bXiT,"&! "# %!$6
2T[[2;8-RaTPcTVaXSSXbZP[[_aTUXg,A42>N36
#(!04%2 34/2!'% ,!9/54
.OTICE THAT NO SIZE WAS SPECIFIED FOR THE A42>N36 GRID DISKS 7HEN SIZE IS NOT SPECIFIED #ELL#,)
AUTOMATICALLY CALCULATES THE SIZE FOR EACH GRID DISK SO THEY CONSUME THE REMAINING FREE SPACE ON THE CELL
DISK &OR EXAMPLE
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TbXiT
30C0N36N23N NRT[[ "&## &6
30C0N36N23N NRT[[ "&## &6
q
A42>N36N23N NRT[[ ""&! "# %!$6
A42>N36N23N NRT[[ ""&! "# %!$6
q
BHBC4<N36N23N NRT[[ "& $ $$6
BHBC4<N36N23N NRT[[ "& $ $$6
BHBC4<N36N23N !NRT[[ "&## %%!$6
BHBC4<N36N23N "NRT[[ "&## %%!$6
q
#REATING &LASH$ISK
"ASED 'RID $ISKS
%XADATA USES OFFLOADING FEATURES LIKE 3MART 3CAN TO PROVIDE STRIKINGLY FAST )/ FOR DIRECT PATH READS
TYPICALLY FOUND IN $33 DATABASES 4HESE FEATURES ARE ONLY ACTIVATED FOR VERY SPECIFIC DATA ACCESS PATHS IN
THE DATABASE 4O SPEED UP )/ PERFORMANCE FOR RANDOM READS %XADATA 6 INTRODUCED SOLID
STATE STORAGE
CALLED Flash Cache %ACH STORAGE CELL COMES CONFIGURED WITH FOUR ' &LASH #ACHE CARDS ' USABLE TO
AUGMENT )/ PERFORMANCE FOR FREQUENTLY ACCESSED DATA 7HEN CONFIGURED AS %XADATA 3MART &LASH #ACHE
THESE DEVICES ACT LIKE A LARGE DATABASE
AWARE DISK CACHE FOR THE STORAGE CELL 7E DISCUSSED THIS IN DETAIL
IN #HAPTER /PTIONALLY SOME SPACE FROM THE &LASH #ACHE MAY BE CARVED OUT AND USED LIKE HIGH
SPEED
SOLID
STATE DISKS &LASH #ACHE IS CONFIGURED AS A CELL DISK OF TYPE 5[PbW3XbZ AND JUST AS GRID DISKS ARE
CREATED ON 7PaS3XbZ CELL DISKS THEY MAY ALSO BE CREATED ON 5[PbW3XbZ CELL DISKS 7HEN &LASH$ISKS ARE
USED FOR DATABASE STORAGE ITS PRIMARILY TO IMPROVE PERFORMANCE FOR HIGHLY WRITE
INTENSIVE WORKLOADS
WHEN DISK
BASED STORAGE CANNOT KEEP UP &LASH$ISK CELL DISKS MAY BE SEEN USING THE #ELL#,) [Xbc
RT[[SXbZ COMMAND AS IN THE FOLLOWING EXAMPLE
2T[[2;8-[XbcRT[[SXbZPccaXQdcTb]P\TSXbZCh_TbXiT
23N NRT[[ "7PaS3XbZ '"! $("&$6
23N NRT[[ "7PaS3XbZ '"! $("&$6
q
53N NRT[[ "5[PbW3XbZ!! '&$6
53N NRT[[ "5[PbW3XbZ!! '&$6
q
53N $NRT[[ "5[PbW3XbZ!! '&$6
&LASH$ISK TYPE CELL DISKS ARE NAMED WITH A PREFIX OF &$ AND A SXbZCh_E OF 5[PbW3XbZ )T IS NOT
RECOMMENDED TO USE ALL OF YOUR &LASH #ACHE FOR GRID DISKS 7HEN CREATING THE &LASH #ACHE USE THE bXiT
PARAMETER TO HOLD BACK SOME SPACE TO BE USED FOR GRID DISKS 4HE FOLLOWING COMMAND CREATES A &LASH
#ACHE OF '" RESERVING '" '" ¾ FOR GRID DISKS
2T[[2;8-RaTPcTU[PbWRPRWTP[[bXiT," V
5[PbWRPRWTRT[[ "N5;0B720274bdRRTbbUd[[hRaTPcTS
.OTE THAT THE RaTPcTU[PbWRPRWT COMMAND USES THE bXiT PARAMETER DIFFERENTLY THAN THE RaTPcT
VaXSSXbZ COMMAND 7HEN CREATING THE FLASH CACHE THE bXiT PARAMETER DETERMINES THE TOTAL SIZE OF THE
CACHE
#(!04%2 34/2!'% ,!9/54
2T[[2;8-[XbcU[PbWRPRWTSTcPX[
]P\T)RT[[ "N5;0B720274
RT[[3XbZ)53N NRT[[ "53N "NRT[[ "53N &NRT[[ "q
q
bXiT)" 6
bcPcdb)]^a\P[
2T[[2;8-[XbcRT[[SXbZPccaXQdcTb]P\TbXiTUaTTb_PRT~
fWTaTSXbZch_T,5[PbW3XbZ
53N NRT[[ "!! '&$6# &' !$6
53N NRT[[ "!! '&$6# &' !$6
q
53N $NRT[[ "!! '&$6# &' !$6
.OW WE CAN CREATE GRID DISKS USING THE REMAINING FREE SPACE ON THE &LASH $ISKS USING THE
FAMILIAR RaTPcTVaXSSXbZ COMMAND 4HIS TIME WELL SPECIFY U[PbWSXbZ FOR THE CELL DISKS TO USE
2T[[2;8-RaTPcTVaXSSXbZP[[U[PbWSXbZ_aTUXg,A0<38B:
6aXS3XbZA0<38B:N53N NRT[[ "bdRRTbbUd[[hRaTPcTS
q
6aXS3XbZA0<38B:N53N #NRT[[ "bdRRTbbUd[[hRaTPcTS
6aXS3XbZA0<38B:N53N $NRT[[ "bdRRTbbUd[[hRaTPcTS
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TSXbZCh_TbXiT~
fWTaTSXbZch_T,5[PbW3XbZ
A0<38B:N53N NRT[[ "5[PbW3XbZ# &' !$6
A0<38B:N53N NRT[[ "5[PbW3XbZ# &' !$6
q
A0<38B:N53N $NRT[[ "5[PbW3XbZ# &' !$6
/NCE THE GRID DISKS HAVE BEEN CREATED THEY MAY BE USED TO CREATE !3- DISK GROUPS USED TO STORE
DATABASE OBJECTS JUST AS YOU WOULD ANY OTHER DISK
BASED DISK GROUP 4HE BEAUTY OF &LASH #ACHE
CONFIGURATION IS THAT ALL THIS MAY BE DONE WHILE THE SYSTEM IS ONLINE AND SERVICING )/ REQUESTS !LL OF THE
COMMANDS WEVE JUST USED TO DROP AND RECONFIGURE THE &LASH #ACHE WERE DONE WITHOUT THE NEED TO
DISABLE OR SHUTDOWN DATABASES OR CELL SERVICES
Storage Strategies
%ACH %XADATA STORAGE CELL IS AN INTELLIGENT MINI
3!. OPERATING SOMEWHAT INDEPENDENTLY OF THE OTHER
CELLS IN THE RACK .OW THIS MAY BE STRETCHING THE DEFINITION OF 3!. A LITTLE BUT WITH THE #ELL 3ERVER
SOFTWARE INTELLIGENTLY CONTROLLING )/ ACCESS WE BELIEVE IT IS APPROPRIATE 3TORAGE CELLS MAY BE CONFIGURED
IN SUCH A WAY THAT ALL #ELLS IN THE RACK PROVIDE STORAGE FOR ALL DATABASES IN THE RACK 4HIS PROVIDES
MAXIMUM )/ PERFORMANCE AND DATA TRANSFER RATES FOR EACH DATABASE IN THE SYSTEM 3TORAGE CELLS MAY
ALSO BE CONFIGURED TO SERVICE SPECIFIC DATABASE SERVERS USING THE RT[[X_ ^aP FILE )N ADDITION CELL SECURITY
MAY BE USED TO RESTRICT ACCESS TO SPECIFIC DATABASES OR !3- INSTANCES THROUGH USE OF STORAGE REALMS )N
THIS SECTION )LL DISCUSS STRATEGIES FOR SEPARATING CELLS INTO GROUPS THAT SERVICE CERTAIN DATABASE SERVERS OR
2!# CLUSTERS 4O BORROW A FAMILIAR TERM FROM THE 3!. WORLD THIS IS WHERE WE WILL TALK ABOUT hZONINGv A
SET OF STORAGE CELLS TO SERVICE DEVELOPMENT TEST AND PRODUCTION ENVIRONMENTS
#(!04%2 34/2!'% ,!9/54
#ONFIGURATION /PTIONS
%XADATA REPRESENTS A SUBSTANTIAL INVESTMENT FOR MOST COMPANIES &OR ONE REASON OR ANOTHER WE FIND
THAT MANY COMPANIES WANT TO BUY A FULL OR HALF RACK FOR CONSOLIDATING SEVERAL DATABASE ENVIRONMENTS
%XADATAS ARCHITECTURE MAKES IT A VERY GOOD CONSOLIDATION PLATFORM 4HESE ARE SOME OF THE MOST
COMMON CONFIGURATIONS WEVE SEEN
v ! FULL RACK SERVICING DEVELOPMENT TEST AND PRODUCTION
v ! FULL RACK SERVICING SEVERAL INDEPENDENT PRODUCTION ENVIRONMENTS
v ! HALF RACK SERVICING DEVELOPMENT AND TEST
v )SOLATING A SCRATCH ENVIRONMENT FOR $"! TESTING AND DEPLOYING SOFTWARE PATCHES
&OR EACH OF THESE CONFIGURATIONS ISOLATING )/ TO SPECIFIC DATABASE SERVERS MAY BE A KEY
CONSIDERATION &OR EXAMPLE YOUR COMPANY MAY BE HOSTING DATABASE ENVIRONMENTS FOR EXTERNAL CLIENTS
THAT REQUIRE SEPARATION FROM OTHER DATABASE SYSTEMS /R YOUR COMPANY MAY HAVE LEGAL REQUIREMENTS TO
SEPARATE SERVER ACCESS TO DATA !NOTHER REASON FOR SEGMENTING STORAGE AT THE CELL LEVEL MAY BE TO PROVIDE
AN ENVIRONMENT FOR $"! TRAINING OR TESTING SOFTWARE PATCHES 4HERE ARE TWO WAYS TO ISOLATE %XADATA
STORAGE CELLS BY NETWORK ACCESS AND BY STORAGE REALM
)SOLATING 3TORAGE #ELL !CCESS
2ECALL THAT !3- GAINS ACCESS TO GRID DISKS THROUGH THE )NFINIBAND NETWORK 4HIS IS CONFIGURED BY ADDING
THE )0 ADDRESS OF STORAGE CELLS IN THE RT[[X_ ^aPFILE &OR EXAMPLE IN A FULL RACK CONFIGURATION ALL
STORAGE CELLS ARE LISTED AS FOLLOWS
JT]ZSQ !)^aPR[T)4G31!L W^\T ^aPR[T
-RPc TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X_ ^aP
RT[[, (! %' ! (
RT[[, (! %' !
RT[[, (! %' !
RT[[, (! %' ! !
RT[[, (! %' ! "
RT[[, (! %' ! #
RT[[, (! %' ! $
RT[[, (! %' ! %
RT[[, (! %' ! &
RT[[, (! %' ! '
RT[[, (! %' ! (
RT[[, (! %' ! !
RT[[, (! %' ! !
RT[[, (! %' ! !!
7HEN !3- STARTS UP IT INTERROGATES THE STORAGE CELLS ON EACH OF THESE )0 ADDRESSES FOR GRID DISKS IT
CAN USE FOR CONFIGURING !3- DISK GROUPS 7E CAN EASILY SEGREGATE STORAGE CELLS TO SERVICE SPECIFIC
DATABASE SERVERS BY REMOVING THE )0 ADDRESS OF CELLS THAT SHOULD NOT BE USED /BVIOUSLY THIS IS NOT
ENFORCED BY ANY KIND OF SECURITY BUT IT IS AN EFFECTIVE SIMPLE WAY OF PAIRING UP DATABASE SERVERS WITH THE
STORAGE CELLS THEY SHOULD USE FOR STORAGE 4ABLE
ILLUSTRATES A CONFIGURATION THAT SPLITS A FULL RACK INTO
TWO DATABASE AND STORAGE GRIDS 0RODUCTION IS CONFIGURED WITH SIX DATABASE SERVERS AND STORAGE CELLS
WHILE 4EST IS CONFIGURED FOR TWO DATABASE SERVERS AND THREE STORAGE CELLS
#(!04%2 34/2!'% ,!9/54
Table 14-5. A Storage Network Configuration
Production Database Servers, 1-6 Production Storage Cells, 1-11
TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X_ ^aP
RT[[, (! %' ! ( S\ RT[ (! %' ! (
RT[[, (! %' ! S\ RT[ ! (! %' !
RT[[, (! %' ! S\ RT[ " (! %' !
RT[[, (! %' ! ! S\ RT[ # (! %' ! !
RT[[, (! %' ! " S\ RT[ $ (! %' ! "
RT[[, (! %' ! # S\ RT[ % (! %' ! #
RT[[, (! %' ! $ S\ RT[ & (! %' ! $
RT[[, (! %' ! % S\ RT[ ' (! %' ! %
RT[[, (! %' ! & S\ RT[ ( (! %' ! &
RT[[, (! %' ! ' S\ RT[ (! %' ! '
RT[[, (! %' ! ( S\ RT[ (! %' ! (
Test Database Servers, 7-8 Test Storage Cells, 12-14
TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X_ ^aP
RT[[, (! %' ! ! S\ RT[ ! (! %' ! !
RT[[, (! %' ! ! S\ RT[ " (! %' ! !
RT[[, (! %' ! !! S\ RT[ # (! %' ! !!
$ATABASE SERVERS AND STORAGE CELLS CAN BE PAIRED IN ANY COMBINATION THAT BEST SUITS YOUR SPECIFIC
NEEDS 2EMEMBER THAT THE MINIMUM REQUIREMENTS FOR /RACLE 2!# ON %XADATA REQUIRES TWO DATABASE
SERVERS AND THREE STORAGE CELLS WHICH IS BASICALLY A QUARTER RACK CONFIGURATION 4ABLE
SHOWS THE
STORAGE AND PERFORMANCE CAPABILITIES OF %XADATA STORAGE CELLS IN QUARTER RACK HALF RACK AND FULL RACK
CONFIGURATIONS
)F SOME OF YOUR ENVIRONMENTS DO NOT REQUIRE /RACLE 2!# THERE IS NO REASON THEY CANNOT BE
CONFIGURED WITH STAND ALONE NON
2!# DATABASE SERVERS )F THIS IS DONE THEN A MINIMUM OF ONE STORAGE
CELL MAY BE USED TO PROVIDE DATABASE STORAGE FOR EACH DATABASE SERVER )N FACT A SINGLE STORAGE CELL MAY
EVEN BE SHARED BY MULTIPLE STANDALONE DATABASE SERVERS /NCE AGAIN %XADATA IS A HIGHLY CONFIGURABLE
#(!04%2 34/2!'% ,!9/54
SYSTEM "UT JUST BECAUSE YOU CAN DO SOMETHING DOESNT MEAN YOU SHOULD 3TORAGE CELLS ARE THE
WORKHORSE OF %XADATA %ACH CELL SUPPORTS A FINITE DATA TRANSFER RATE -"03 AND NUMBER OF )/S PER
SECOND )/03 2EDUCING THE STORAGE CELL FOOTPRINT OF YOUR DATABASE ENVIRONMENT DIRECTLY IMPACTS THE
PERFORMANCE YOUR DATABASE CAN YIELD
Table 14-6. Performance Capabities of Exadata Storage Cells
MBPS & IOPS by Device Type Quarter Rack Half Rack Full Rack
$ISK 4RANSFER "ANDWIDTH 3!3 '"S '"S '"S
3!4! '"S '"S '"S
&LASH $ISK 4RANSFER "ANDWIDTH '"S '"S '"S
$ISK )/03 3!3
3!4!
&LASH $ISK )/03
■ .OTE %XADATA 6 OFFERED TWO CHOICES FOR DISK DRIVES IN THE STORAGE CELLS 3!4! AND 3!3 7ITH %XADATA 8 THE
3!4! OPTION HAS BEEN REPLACED WITH (IGH #APACITY 3!3 DRIVES THAT HAVE STORAGE AND PERFORMANCE CHARACTERISTICS
VERY SIMILAR TO THOSE OF THE 3!4! DRIVES THEY REPLACED 3O NOW WITH 8 YOUR STORAGE OPTIONS ARE (IGH #APACITY
AND (IGH 0ERFORMANCE 3!3 DRIVES
Cell Security
)N ADDITION TO ISOLATING STORAGE CELLS BY THEIR NETWORK ADDRESS %XADATA ALSO PROVIDES A WAY TO SECURE
ACCESS TO SPECIFIC GRID DISKS WITHIN THE STORAGE CELL !N ACCESS CONTROL LIST !#, IS MAINTAINED AT THE
STORAGE CELL AND GRID DISKS ARE DEFINED AS BEING ACCESSIBLE TO SPECIFIC !3- CLUSTERS AND OPTIONALLY
DATABASES WITHIN THE !3- CLUSTER )F YOUVE ALREADY LOGGED SOME TIME WORKING ON YOUR %XADATA SYSTEM
CHANCES ARE YOU HAVENT NOTICED ANY SUCH ACCESS RESTRICTIONS 4HAT IS BECAUSE BY DEFAULT CELL SECURITY IS
OPEN ALLOWING ALL !3- CLUSTERS AND DATABASES IN THE SYSTEM ACCESS TO ALL GRID DISKS #ELL SECURITY CONTROLS
ACCESS TO GRID DISKS AT TWO LEVELS BY !3- CLUSTER AND BY DATABASE
ASM-Scoped Security: !3-
SCOPED SECURITY RESTRICTS ACCESS TO GRID DISKS BY
!3- CLUSTER 4HIS IS THE FIRST LAYER OF CELL SECURITY )T ALLOWS ALL DATABASES IN THE
!3- CLUSTER TO HAVE ACCESS TO ALL GRID DISKS MANAGED BY THE !3- INSTANCE &OR
EXAMPLE AN %XADATA FULL RACK CONFIGURATION CAN BE SPLIT SO THAT FOUR DATABASE
SERVERS AND SEVEN STORAGE CELLS CAN BE USED BY #USTOMER
! AND THE OTHER FOUR
DATABASE SERVERS AND SEVEN STORAGE CELLS CAN BE USED BY #USTOMER
"
Database-Scoped Security: /NCE !3-
SCOPED SECURITY IS CONFIGURED ACCESS TO
GRID DISKS MAY BE FURTHER CONTROLLED AT THE DATABASE LEVEL USING DATABASE
SCOPED SECURITY $ATABASE
SCOPED SECURITY IS MOST APPROPRIATE WHEN DATABASES
WITHIN THE !3- CLUSTER SHOULD HAVE ACCESS TO A SUBSET OF THE GRID DISKS MANAGED
#(!04%2 34/2!'% ,!9/54
BY THE !3- INSTANCE )N THE EARLIER EXAMPLE #USTOMER
!S ENVIRONMENT COULD
USE DATABASE
SCOPED SECURITY TO SEPARATE DATABASE ENVIRONMENTS FROM ONE
ANOTHER WITHIN ITS HALF RACK CONFIGURATION
#ELL 3ECURITY 4ERMINOLOGY
"EFORE WE GET TOO FAR ALONG LETS TAKE A LOOK AT SOME OF THE NEW TERMINOLOGY SPECIFIC TO %XADATAS CELL
SECURITY
Storage Realm 'RID DISKS THAT SHARE A COMMON SECURITY DOMAIN ARE REFERRED TO
AS A storage realm
Security Key ! SECURITY KEY IS USED TO AUTHENTICATE !3- AND DATABASE CLIENTS TO
THE STORAGE REALM )T IS ALSO USED FOR SECURING MESSAGES SENT BETWEEN THE STORAGE
CELLS AND THE !3- AND DATABASE CLIENTS 4HE SECURITY KEY IS CREATED USING THE
#ELL#,) COMMAND RaTPcTZTh 4HE KEY IS THEN ASSIGNED TO GRID DISKS USING THE
#ELL#,) PbbXV]ZTh COMMAND
cellkey.ora: 4HE RT[[ZTh ^aP FILE IS STORED ON THE DATABASE SERVERS /NE
RT[[ZTh ^aP FILE IS CREATED FOR !3-
SCOPED SECURITY AND ANOTHER RT[[ZTh ^aP
FILE IS CREATED FOR EACH DATABASE REQUIRING DATABASE
SCOPED SECURITY 4HE
RT[[ZTh ^aP FILES ARE USED TO IDENTIFY SECURITY KEYS THE STORAGE REALM AND THE
UNIQUE NAME OF THE !3- CLUSTER OR DATABASE
4ABLE
SHOWS THE DEFINITIONS FOR THE FIELDS IN THE RT[[ZTh ^aP FILE
Table 14-7. The Contents of the Cellkey.ora File
Field Description
ZTh 4HIS IS THE SECURITY KEY GENERATED AT THE STORAGE CELL WITH THE RaTPcTZTh COMMAND 4HIS KEY
IS USED TO AUTHENTICATE THE !3- CLUSTER AND DATABASE TO THE STORAGE REALM
&OR !3-
SCOPED SECURITY THIS VALUE MUST MATCH THE KEY ASSIGNED TO THE !3- CLUSTER USING
THE PbbXV]ZTh COMMAND
&OR DATABASE
SCOPED SECURITY THIS VALUE MUST MATCH THE SECURITY KEY ASSIGNED TO THE
DATABASE USING THE PbbXV]ZTh COMMAND
Pb\ 4HIS IS THE UNIQUE NAME OF THE !3- CLUSTER FOUND IN THE 31ND=8@D4N=0<4 PARAMETER OF THE
!3- INSTANCE )T IS USED TO ASSOCIATE THE !3- CLUSTER WITH THE PePX[PQ[TC^ ATTRIBUTE OF THE
GRID DISKS IN THE STORAGE REALM 'RID DISKS ARE ASSIGNED THIS VALUE USING THE #ELL#,) RaTPcT
VaXSSXbZ AND P[cTaVaXSSXbZ COMMANDS
aTP[\ 4HIS FIELD IS OPTIONAL )F USED THE VALUE MUST MATCH THE RTP[\=P\T ATTRIBUTE ASSIGNED TO THE
STORAGE CELLS USING THE #ELL#,) COMMAND P[cTaRT[[aTP[\=P\T
#(!04%2 34/2!'% ,!9/54
#ELL 3ECURITY "EST 0RACTICES
&OLLOWING /RACLES BEST PRACTICES IS AN IMPORTANT PART OF CONFIGURING CELL SECURITY )T WILL HELP YOU AVOID
THOSE ODD SITUATIONS WHERE THINGS SEEM TO WORK SOME OF THE TIME OR ONLY ON CERTAIN STORAGE CELLS
&OLLOWING THESE BEST PRACTICES WILL SAVE YOU A LOT OF TIME AND FRUSTRATION
v )F DATABASE
SCOPED SECURITY IS USED BE SURE TO USE IT FOR ALL DATABASES IN THE !3-
CLUSTER
v -AKE SURE THE !3- RT[[ZTh ^aP FILE IS THE SAME ON ALL SERVERS FOR AN !3- CLUSTER
4HIS INCLUDES CONTENTS OWNERSHIP AND PERMISSIONS
v *UST AS YOU DID FOR THE !3- RT[[ZTh ^aP FILE MAKE SURE CONTENTS OWNERSHIP AND
PERMISSIONS ARE IDENTICAL ACROSS ALL SERVERS FOR THE DATABASE RT[[ZTh ^aP FILE
v %NSURE THE CELL SIDE SECURITY SETTINGS ARE THE SAME FOR ALL GRID DISKS BELONGING TO THE
SAME !3- DISK GROUP
v )T IS VERY IMPORTANT THAT THE RT[[ZTh ^aP FILES AND CELL COMMANDS ARE EXECUTED
CONSISTENTLY ACROSS ALL SERVERS AND CELLS 5SE THE SR[X UTILITY TO DISTRIBUTE THE
RT[[ZTh ^aP FILE AND REDUCE THE LIKELIHOOD OF HUMAN ERROR
#ONFIGURING !3-
3COPED 3ECURITY
7ITH !3-
SCOPED SECURITY THE !3- CLUSTER IS AUTHENTICATED TO THE STORAGE CELL BY ITS 31ND=8@D4N=0<4 AND
A SECURITY KEY 4HE SECURITY KEY IS CREATED AT THE STORAGE CELL AND STORED IN THE RT[[ZTh ^aP FILE ON THE
DATABASE SERVER !N ACCESS CONTROL LIST !#, IS DEFINED ON THE STORAGE CELL THAT IS USED TO VERIFY THE
SECURITY KEY IT RECEIVES FROM !3- 4HE PePX[PQ[TC^ ATTRIBUTE ON EACH GRID DISK DICTATES WHICH !3-
CLUSTERS ARE PERMITTED ACCESS
.OW LETS TAKE A LOOK AT THE STEPS FOR CONFIGURING !3-
SCOPED SECURITY
3HUT DOWN ALL DATABASES AND !3- INSTANCES IN THE !3- CLUSTER
#REATE THE SECURITY KEY USING THE #ELL#,) RaTPcTZTh COMMAND
2T[[2;8-RaTPcTZTh
"%#'T!P" & %( ($Q&((R##U !UTP(
4HIS SIMPLY GENERATES THE KEY WHICH IS NOT AUTOMATICALLY STORED ANYWHERE
4HE RaTPcTZTh COMMAND ONLY NEEDS TO BE RUN ONCE AND CAN BE DONE ON ANY
STORAGE CELL 4HIS SECURITY KEY WILL BE ASSIGNED TO THE !3- CLUSTER IN THE ZTh
FIELD OF THE RT[[ZTh ^aP FILE
.EXT CREATE A RT[[ZTh ^aP FILE AND INSTALL IT IN THE TcR ^aPR[T RT[[ ]Tcf^aZ
R^]UXV DIRECTORY FOR EACH DATABASE SERVER ON WHICH THIS !3- CLUSTER IS
CONFIGURED 3ET THE OWNERSHIP OF THE FILE TO THE USER AND GROUP SPECIFIED DURING
THE !3- SOFTWARE INSTALLATION 0ERMISSIONS SHOULD ALLOW IT TO BE READ BY THE
USER AND GROUP OWNER OF THE FILE &OR EXAMPLE
ZTh,"%#'T!P" & %( ($Q&((R##U !UTP(
Pb\,0B<
aTP[\,Rdbc^\TaN0NaTP[\
-RW^f]^aPR[T)SQPRT[[ZTh ^aP
#(!04%2 34/2!'% ,!9/54
-RW\^S%# RT[[ZTh ^aP
.OTE THAT IF A REALM IS DEFINED IN THIS FILE IT MUST MATCH THE REALM NAME
ASSIGNED TO THE STORAGE CELLS USING THE P[cTaRT[[aTP[\, COMMAND
&IND THE 31ND=8@D4N=0<4 FOR YOUR !3- CLUSTER USING THE bW^f_PaP\TcTa
COMMAND FROM ONE OF THE !3- INSTANCES
BHB)0B< -bW^f_PaP\TcTaSQNd]X`dTN]P\T
=0<4CH?4E0;D4
SQNd]X`dTN]P\TbcaX]V0B<
5SE THE #ELL#,) PbbXV]ZTh COMMAND TO ASSIGN THE SECURITY KEY TO THE !3-
CLUSTER BEING CONFIGURED 4HIS MUST BE DONE ON EACH STORAGE CELL TO WHICH YOU
WANT THE !3- CLUSTER TO HAVE ACCESS
2T[[2;8-0BB86=:4H
5>A0B<,%%T !PSQ((%' $"$'QU'!!$'$'&U$ $
5SING THE #ELL#,) RaTPcTVaXSSXbZ COMMAND SET THE PePX[PQ[TC^ ATTRIBUTE FOR
EACH GRID DISK TO WHICH YOU WANT THIS !3- CLUSTER TO HAVE ACCESS 4HIS CAN BE
DONE FOR ALL GRID DISKS ON THE CELL AS FOLLOWS
2T[[2;8-RaTPcTVaXSSXbZP[[_aTUXg,30C0N36
bXiT, !'! ' !$6PePX[PQ[TC^,0B<
&OR EXISTING GRID DISKS USE THE P[cTaVaXSSXbZ COMMAND TO SET UP SECURITY
2T[[2;8-P[cTaVaXSSXbZP[[_aTUXg,30C0N36
PePX[PQ[TC^,0B<
! SUBSET OF GRID DISKS MAY ALSO BE ASSIGNED AS FOLLOWS
2T[[2;8-P[cTaVaXSSXbZ30C0N23N NRT[[ "
30C0N23N NRT[[ "
30C0N23N !NRT[[ "
q
PePX[PQ[TC^,0B<
4HIS COMPLETES THE CONFIGURATION OF !3-
SCOPED CELL SECURITY 4HE !3- CLUSTER AND ALL DATABASES
CAN NOW BE RESTARTED 7HEN !3- STARTS UP IT WILL CHECK FOR THE RT[[ZTh ^aP FILE AND PASS THE KEY TO THE
STORAGE CELLS IN ORDER TO GAIN ACCESS TO THE GRID DISKS
#ONFIGURING $ATABASE
3COPED 3ECURITY
$ATABASE
SCOPED SECURITY LOCKS DOWN DATABASE ACCESS TO SPECIFIC GRID DISKS WITHIN AN !3- CLUSTER )T IS
USEFUL FOR CONTROLLING ACCESS TO GRID DISKS WHEN MULTIPLE DATABASES SHARE THE SAME !3- CLUSTER "EFORE
DATABASE
SCOPED SECURITY MAY BE IMPLEMENTED !3-
SCOPED SECURITY MUST BE CONFIGURED AND VERIFIED
7HEN USING DATABASE
SCOPED SECURITY THERE WILL BE ONE RT[[ZTh ^aP FILE PER DATABASE PER DATABASE
SERVER AND ONE !#, ENTRY ON THE STORAGE CELL FOR EACH DATABASE 4HE FOLLOWING STEPS MAY BE USED TO
IMPLEMENT SIMPLE DATABASE
SCOPED SECURITY FOR TWO DATABASES CALLED (2 (UMAN 2ESOURCES AND 0!9
0AYROLL
#(!04%2 34/2!'% ,!9/54
1. 3HUT DOWN ALL DATABASES AND !3- INSTANCES IN THE !3- CLUSTER
2. #REATE THE SECURITY KEY USING THE #ELL#,) RaTPcTZTh COMMAND
2T[[2;8-RaTPcTZTh
&$#'P&S PQUUPSUTU($P$" '$PQP T('
2T[[2;8-RaTPcTZTh
'T& $QSQS%PS(UP$"S# &"%P$""Q(Q
4HE RaTPcTZTh COMMAND MUST BE RUN ONCE FOR EACH DATABASE IN THE !3-
CLUSTER )T CAN BE RUN FROM ANY STORAGE CELL /NE SECURITY KEY WILL BE ASSIGNED TO
EACH DATABASE WITHIN THE !3- CLUSTER IN THE ZTh FIELD OF THE DATABASE
RT[[ZTh ^aP FILE
3. &OR EACH DATABASE CREATE A CELLKEYORA FILE USING THE KEYS CREATED IN STEP
)NSTALL THESE CELLKEYORA FILES IN THE
>A02;4N7><4 PS\X] jSQNd]X`dTN]P\Tl _UX[T DIRECTORIES FOR EACH DATABASE
SERVER ON WHICH DATABASE
SCOPED SECURITY WILL BE CONFIGURED *UST AS YOU DID
FOR !3-
SCOPED SECURITY SET THE OWNERSHIP OF THE FILE TO THE USER AND GROUP
SPECIFIED DURING THE !3- SOFTWARE INSTALLATION 0ERMISSIONS SHOULD ALLOW IT TO
BE READ BY THE USER AND GROUP OWNER OF THE FILE &OR EXAMPLE
2T[[ZTh ^aPUX[TU^acWT7ASPcPQPbT
ZTh,&$#'P&S PQUUPSUTU($P$" '$PQP T('
Pb\,0B<
aTP[\,Rdbc^\TaN0NaTP[\
-RW^f]^aPR[T)SQP>A02;4N7><4 PS\X] 7A RT[[ZTh ^aP
-RW\^S%# >A02;4N7><4 PS\X] 7A RT[[ZTh ^aP
2T[[ZTh ^aPUX[TU^acWT?0HSPcPQPbT
ZTh,'T& $QSQS%PS(UP$"S# &"%P$""Q(Q
Pb\,0B<
aTP[\,Rdbc^\TaN0NaTP[\
-RW^f]^aPR[T)SQP>A02;4N7><4 PS\X] ?0H RT[[ZTh ^aP
-RW\^S%# >A02;4N7><4 PS\X] ?0H RT[[ZTh ^aP
.OTE THAT IF A REALM IS DEFINED IN THIS FILE IT MUST MATCH THE REALM NAME
ASSIGNED TO THE STORAGE CELLS USING THE P[cTaRT[[aTP[\, COMMAND
4. 2ETRIEVE THE 31ND=8@D4N=0<4 FOR EACH DATABASE BEING CONFIGURED USING THE bW^f
_PaP\TcTa COMMAND FROM EACH OF THE DATABASES
BHB)7A-bW^f_PaP\TcTaSQNd]X`dTN]P\T
=0<4CH?4E0;D4
SQNd]X`dTN]P\TbcaX]V7A
#(!04%2 34/2!'% ,!9/54
BHB)?0H-bW^f_PaP\TcTaSQNd]X`dTN]P\T
=0<4CH?4E0;D4
SQNd]X`dTN]P\TbcaX]V?0H
5. 5SE THE #ELL#,) PbbXV]ZTh COMMAND TO ASSIGN THE SECURITY KEYS FOR EACH
DATABASE BEING CONFIGURED 4HIS MUST BE DONE ON EACH STORAGE CELL YOU WANT
THE (2 AND 0!9 DATABASES TO HAVE ACCESS TO 4HE FOLLOWING KEYS ARE ASSIGNED TO
THE 31ND=8@D4N=0<4 OF THE (2 AND 0!9 DATABASES
2T[[2;8-0BB86=:4H
5>A7A,&$#'P&S PQUUPSUTU($P$" '$PQP T('~
?0H,'T& $QSQS%PS(UP$"S# &"%P$""Q(Q ~
:ThU^a7AbdRRTbbUd[[hRaTPcTS
:ThU^a?0HbdRRTbbUd[[hRaTPcTS
6. 6ERIFY THAT THE KEYS WERE ASSIGNED PROPERLY
2T[[2;8-[XbcZTh
7AS"#%&(!S%PSTP%& S'U""Q$#R" U ST%
?0HRPT &T'USRT&$ RR !TQ&"&$U$##"P'
7. 5SING THE #ELL#,) RaTPcTSXbZ OR P[cTaVaXSSXbZ COMMAND ASSIGN ACCESS TO
THE GRID DISKS TO EACH DATABASE .OTE THAT THE !3- UNIQUE NAME IS INCLUDED
WITH THE DATABASE UNIQUE NAME IN THIS ASSIGNMENT
2T[[2;8-RaTPcTVaXSSXbZ30C0N23N NRT[[ "
30C0N23N NRT[[ "
bXiT, !'! ' !$6
PePX[PQ[TC^,0B<7A
2T[[2;8-RaTPcTVaXSSXbZ30C0N23N !NRT[[ "
30C0N23N "NRT[[ "
bXiT, !'! ' !$6
PePX[PQ[TC^,0B<?0H
8. 4HE P[cTaVaXSSXbZ COMMAND MAY BE USED TO CHANGE SECURITY ASSIGNMENTS FOR
GRID DISKS &OR EXAMPLE
2T[[2;8-P[cTaVaXSSXbZ30C0N23N $NRT[[ "
30C0N23N %NRT[[ "
PePX[PQ[TC^,0B<7A
2T[[2;8-P[cTaVaXSSXbZ30C0N23N NRT[[ "
30C0N23N !NRT[[ "
PePX[PQ[TC^,0B<?0H
4HIS COMPLETES THE CONFIGURATION OF DATABASE
SCOPED SECURITY FOR THE (2 AND 0!9 DATABASES 4HE
!3- CLUSTER AND DATABASES MAY NOW BE RESTARTED 4HE HUMAN RESOURCES DATABASE NOW HAS ACCESS TO THE
30C0N23N NRT[[ " AND 30C0N23N NRT[[ " GRID DISKS WHILE THE PAYROLL DATABASE HAS ACCESS TO THE
30C0N23N !NRT[[ " AND 30C0N23N "NRT[[ " GRID DISKS
#(!04%2 34/2!'% ,!9/54
2EMOVING #ELL 3ECURITY
/NCE IMPLEMENTED CELL SECURITY MAY BE MODIFIED AS NEEDED BY UPDATING THE !#, LISTS ON THE STORAGE
CELLS AND CHANGING THE AVAILABLE4O ATTRIBUTE OF THE GRID DISKS 2EMOVING CELL SECURITY IS A FAIRLY
STRAIGHTFORWARD PROCESS OF BACKING OUT THE DATABASE SECURITY SETTINGS AND THEN REMOVING THE !3-
SECURITY SETTINGS
4HE FIRST STEP IN REMOVING CELL SECURITY IS TO REMOVE DATABASE
SCOPED SECURITY 4HE FOLLOWING STEPS
WILL REMOVE DATABASE
SCOPED SECURITY FROM THE SYSTEM
1. "EFORE DATABASE SECURITY MAY BE REMOVED THE DATABASES AND !3- CLUSTER
MUST BE SHUT DOWN
2. 2EMOVE THE DATABASES FROM THE AVAILABLE4O ATTRIBUTE OF THE GRID DISKS USING THE
#ELL#,) COMMAND P[cTaVaXSSXbZ 4HIS COMMAND DOESNT SELECTIVELY REMOVE
DATABASES FROM THE LIST )T SIMPLY REDEFINES THE COMPLETE LIST .OTICE THAT WE
WILL JUST BE REMOVING THE DATABASES FROM THE LIST AT THIS POINT 4HE !3- UNIQUE
NAME SHOULD REMAIN IN THE LIST FOR NOW 4HIS MUST BE DONE FOR EACH CELL YOU
WANT TO REMOVE SECURITY FROM
2T[[2;8-P[cTaVaXSSXbZ30C0N23N NRT[[ "
30C0N23N NRT[[ "
PePX[PQ[TC^,0B<
2T[[2;8-P[cTaVaXSSXbZ30C0N23N !NRT[[ "
30C0N23N "NRT[[ "
PePX[PQ[TC^,0B<
/PTIONALLY ALL THE DATABASES MAY BE REMOVED FROM THE SECURED GRID DISKS WITH THE FOLLOWING
COMMAND
2T[[2;8-P[cTaVaXSSXbZ30C0N23N NRT[[ "
30C0N23N NRT[[ "
30C0N23N !NRT[[ "
30C0N23N "NRT[[ "
PePX[PQ[TC^,0B<
!SSUMING THAT THESE DATABASES HAVE NOT BEEN CONFIGURED FOR CELL SECURITY ON ANY OTHER GRID DISKS IN
THE CELL THE SECURITY KEY MAY BE REMOVED FROM THE !#, LIST ON THE STORAGE CELL AS FOLLOWS
2T[[2;8-PbbXV]ZThU^a7A,?0H,
:ThU^a7AbdRRTbbUd[[hSa^__TS
:ThU^a?0HbdRRTbbUd[[hSa^__TS
3. 2EMOVE THE RT[[ZTh ^aP FILE LOCATED IN THE
>A02;4N7><4 PS\X] jSQNd]X`dTN]P\Tl _UX[T DIRECTORY FOR THE DATABASE CLIENT
4. .OW THE RT[[ZTh ^aP FILE FOR THE (2 AND 0!9 DATABASES MAY BE REMOVED FROM
THE DATABASE SERVERS
-a\>A02;4N7><4 PS\X] 7A RT[[ZTh ^aP
-a\>A02;4N7><4 PS\X] ?0H RT[[ZTh ^aP
5. 6ERIFY THAT THE (2 AND 0!9 DATABASES ARE NOT ASSIGNED TO ANY GRID DISKS WITH
THE FOLLOWING #ELL#,) COMMAND
#(!04%2 34/2!'% ,!9/54
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TPePX[PQ[TC^
/NCE DATABASE
SCOPED SECURITY HAS BEEN REMOVED YOU CAN REMOVE !3-
SCOPED SECURITY 4HIS WILL
RETURN THE SYSTEM TO OPEN SECURITY STATUS 4HE FOLLOWING STEPS REMOVE !3-
SCOPED SECURITY /NCE THIS IS
DONE THE GRID DISKS WILL BE AVAILABLE TO ALL !3- CLUSTERS AND DATABASES ON THE STORAGE NETWORK
1. "EFORE CONTINUING WITH THIS PROCEDURE BE SURE THAT DATABASE
SCOPED SECURITY HAS
BEEN COMPLETELY REMOVED 4HE [XbcZTh COMMAND SHOULD DISPLAY THE KEY
ASSIGNMENT FOR THE !3- CLUSTER ONLY .O DATABASES SHOULD BE ASSIGNED KEYS AT THIS
POINT 4HE [XbcVaXSSXbZ COMMAND SHOULD SHOW ALL THE NAMES OF THE GRID DISKS
ASSIGNMENTS FOR THE !3- CLUSTER 0B<
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TPePX[PQ[TC^
2. .EXT REMOVE THE !3- UNIQUE NAME FROM THE AVAILABLE4O ATTRIBUTE ON ALL GRID DISKS
2T[[2;8-[XbcVaXSSXbZPccaXQdcTb]P\TPePX[PQ[TC^
3. .OW REMOVE THE !3- SECURITY FROM THE !#, BY RUNNING THE FOLLOWING COMMAND
2T[[2;8-P[cTaVaXSSXbZP[[PbbXV]C^,
4. 4HE FOLLOWING COMMAND REMOVES THE !3- CLUSTER ASSIGNMENT FOR SELECT GRID DISKS
2T[[2;8-P[cTaVaXSSXbZ30C0N23N NRT[[ "
30C0N23N NRT[[ "
30C0N23N !NRT[[ "
30C0N23N "NRT[[ "
PePX[PQ[TC^,
5. 4HE [XbcVaXSSXbZ COMMAND SHOULD SHOW NO ASSIGNED CLIENTS 6ERIFY THIS BY
RUNNING THE [XbcVaXSSXbZ COMMAND
6. 4HE !3- CLUSTER KEY MAY NOW BE SAFELY REMOVED FROM THE STORAGE CELL USING THE
#ELL#,) PbbXV]ZTh COMMAND
2T[[2;8-[XbcZThSTcPX[
]P\T)0B<
ZTh) (%S&('"P(P""URRPT!&%T!#T&P(U'(
2T[[2PbbXV]ZThU^a0B<,
:ThU^a0B<bdRRTbbUd[[hSa^__TS
7. 2EMOVE THE CELLKEYORA FILE FROM THE ETCORACLECELLNETWORK
CONFIG DIRECTORY ON ALL
DATABASE SERVERS IN THE !3- CLUSTER
4HIS COMPLETES THE REMOVAL OF !3-
SCOPED SECURITY 4HE !3- CLUSTER MAY NOW BE RESTARTED AS WELL
AS ALL THE DATABASES IT SERVICES
#(!04%2 34/2!'% ,!9/54
Summary
5NDERSTANDING ALL THE VARIOUS LAYERS OF THE %XADATA STORAGE ARCHITECTURE AND HOW THEY FIT TOGETHER IS A
KEY COMPONENT TO PROPERLY LAYING OUT STORAGE FOR DATABASES 5NDERSTANDING THE RELATIONSHIP BETWEEN
PHYSICAL DISKS ,5.S CELL DISKS GRID DISKS AND !3- DISK GROUPS IS ABSOLUTELY NECESSARY IF YOU WANT TO
CARVE UP DISK STORAGE FOR MAXIMUM PERFORMANCE )N THIS CHAPTER WEVE DISCUSSED WHAT GRID DISKS ARE
WHAT THEY ARE MADE UP OF AND HOW THEY FIT INTO THE !3- STORAGE GRID 7EVE TAKEN A LOOK AT HOW TO CREATE
DISK GROUPS SO THAT )/ IS PRIORITIZED FOR PERFORMANCE CRITICAL DATAFILES #ARVING UP STORAGE DOESNT END AT
THE DISK SO WE ALSO DISCUSSED METHODS FOR PARTITIONING STORAGE BY CELL AND BY GRID DISK WITHIN THE CELL
C H A P T E R 15
Compute Node Layout
4HE TERM node IS A FAIRLY GENERIC ONE THAT HAS MANY DIFFERENT MEANINGS IN THE )4 INDUSTRY &OR EXAMPLE
NETWORK ENGINEERS CALL ANY ADDRESSABLE DEVICE ATTACHED TO THEIR NETWORK A NODE 5NIX ADMINISTRATORS
COMMONLY USE THE TERM INTERCHANGEABLY WITH host OR server /RACLE $"!S OFTEN REFER TO A DATABASE SERVER
THAT IS A MEMBER OF AN 2!# CLUSTER AS A NODE /RACLES DOCUMENTATION USES THE TERM compute node WHEN
REFERRING TO THE DATABASE SERVER TIER OF THE PLATFORM 4HIS CHAPTER IS ABOUT THE VARIOUS WAYS IN WHICH YOU
CAN CONFIGURE YOUR %XADATA COMPUTE NODES WHETHER THEY ARE MEMBERS OF AN 2!# CLUSTER NODES OR
NONCLUSTERED DATABASE SERVERS
)TS A COMMON MISCONCEPTION THAT AN %XADATA RACK MUST BE CONFIGURED AS A SINGLE /RACLE 2!#
CLUSTER 4HIS COULDNT BE FURTHER FROM THE TRUTH )N ITS SIMPLEST FORM THE %XADATA DATABASE TIER CAN BE
DESCRIBED AS A COLLECTION OF INDEPENDENT DATABASE SERVERS HARDWIRED INTO THE SAME STORAGE AND THE SAME
MANAGEMENT NETWORKS %ACH OF THESE SERVERS CAN BE CONFIGURED TO RUN STANDALONE DATABASES COMPLETELY
INDEPENDENT OF THE OTHERS (OWEVER THIS IS NOT COMMONLY DONE FOR A COUPLE OF REASONSSCALABILITY AND
HIGH AVAILABILITY /RACLE 2!# HAS HISTORICALLY BEEN USED TO PROVIDE NODE REDUNDANCY IN THE EVENT OF NODE
OR INSTANCE FAILURE BUT /RACLE MARKETING HAS MADE IT CLEAR ALL ALONG THAT THE ABILITY TO SCALE
OUT HAS BEEN
AN EQUALLY IMPORTANT GOAL 4RADITIONALLY IF WE NEEDED TO INCREASE DATABASE PERFORMANCE AND CAPACITY
WE DID SO BY UPGRADING SERVER HARDWARE 4HIS METHOD BECAME SO COMMONPLACE THAT THE INDUSTRY COINED
THE PHRASE hHARDWARE REFRESHv TO DESCRIBE IT 4HIS TERM CAN MEAN ANYTHING FROM ADDING #05S MEMORY
OR )/ BANDWIDTH TO A COMPLETE REPLACEMENT OF THE SERVER ITSELF )NCREASING PERFORMANCE AND CAPACITY IN
THIS WAY IS REFERRED TO AS scale-up 7ITH %XADATAS ABILITY TO PROVIDE EXTREME )/ PERFORMANCE TO THE
DATABASE SERVER BUS SPEED IS NOW THE LIMITING FACTOR FOR SCALE
UP 3O WHAT HAPPENS WHEN YOU REACH THE
LIMITS OF SINGLE
SERVER CAPACITY 4HE OBVIOUS ANSWER IS TO ADD MORE SERVERS 4O CONTINUE TO SCALE YOUR
APPLICATION YOU MUST SCALE
OUT USING /RACLE 2!# .ONETHELESS UNDERSTANDING THAT THE DATABASE
SERVERS ARE NOT TIED TOGETHER IN SOME PROPRIETARY FASHION CLARIFIES THE HIGHLY CONFIGURABLE NATURE OF
%XADATA )N #HAPTER WE DISCUSSED VARIOUS STRATEGIES FOR CONFIGURING %XADATAS STORAGE SUBSYSTEMS TO
SERVICE SPECIFIC DATABASE SERVERS )N THIS CHAPTER WELL TAKE A LOOK AT WAYS THE DATABASE TIER MAY BE
CONFIGURED TO CREATE CLUSTERED AND NONCLUSTERED DATABASE ENVIRONMENTS THAT ARE WELL SUITED TO MEET THE
NEEDS OF YOUR BUSINESS
Provisioning Considerations
%XADATA IS AN EXTREMELY CONFIGURABLE PLATFORM $ETERMINING THE BEST CONFIGURATION FOR YOUR BUSINESS WILL
INVOLVE REVIEWING THE PERFORMANCE AND UPTIME DEMANDS OF YOUR APPLICATIONS AS WELL AS ENSURING
ADEQUATE SEPARATION FOR DEVELOPMENT TEST AND PRODUCTION SYSTEMS (ERE ARE A FEW OF THE KEY
CONSIDERATIONS FOR DETERMINING THE MOST SUITABLE COMPUTE NODE LAYOUT TO SUPPORT YOUR DATABASE
ENVIRONMENTS
#(!04%2 #/-054% ./$% ,!9/54
CPU Resources: 7HEN DETERMINING THE OPTIMAL NODE LAYOUT FOR YOUR DATABASES
KEEP IN MIND THAT %XADATA HANDLES THE )/ WORKLOAD VERY DIFFERENTLY FROM
TRADITIONAL DATABASE PLATFORMS /N NON %XADATA PLATFORMS THE DATABASE SERVER
IS RESPONSIBLE FOR RETRIEVING ALL DATA BLOCKS FROM STORAGE TO SATISFY )/ REQUESTS
FROM THE APPLICATIONS %XADATA OFFLOADS A LOT OF THIS WORK TO THE STORAGE CELLS
4HIS CAN SIGNIFICANTLY REDUCE THE #05 REQUIREMENTS OF YOUR DATABASE SERVERS
&IGURING OUT HOW MUCH LESS #05 YOUR DATABASES WILL REQUIRE IS A DIFFICULT TASK
BECAUSE IT DEPENDS IN PART ON HOW MUCH YOUR DATABASE IS UTILIZING PARALLEL
QUERY AND (## COMPRESSION AND HOW SUITABLE YOUR APPLICATION 31, IS TO
OFFLOADING 3OME OF THE 3MART 3CAN OPTIMIZATIONS SUCH AS DECRYPTION
PREDICATE FILTERING AND (## DECOMPRESSION WILL REDUCE #05 REQUIREMENTS
REGARDLESS OF THE TYPE OF APPLICATION 7E COVERED THESE TOPICS IN DETAIL IN
#HAPTERS n
3YSTEMS REQUIRING THOUSANDS OF DEDICATED SERVER CONNECTIONS CAN OVERWHELM
THE RESOURCES OF A SINGLE MACHINE 3PREADING THESE CONNECTIONS ACROSS MULTIPLE
COMPUTE NODES REDUCES THE BURDEN ON THE SYSTEMS PROCESS SCHEDULER AND
ALLOWS THE #05 TO SPEND ITS TIME MORE EFFECTIVELY SERVICING CLIENT REQUESTS ,OAD
BALANCING CONNECTIONS ACROSS MULTIPLE COMPUTE NODES ALSO IMPROVES THE
DATABASES CAPACITY FOR HANDLING CONCURRENT CONNECTION REQUESTS
Memory Resources: 3YSTEMS THAT REQUIRE THOUSANDS OF DEDICATED SERVER
CONNECTIONS CAN ALSO PUT A BURDEN ON MEMORY RESOURCES %ACH DEDICATED
SERVER CONNECTION REQUIRES A SLICE OF MEMORY WHETHER OR NOT THE CONNECTION IS
ACTIVELY BEING USED 3PREADING THESE CONNECTIONS ACROSS MULTIPLE 2!# NODES
ALLOWS THE DATABASE TO HANDLE MORE CONCURRENT CONNECTIONS THAN A SINGLE
COMPUTE NODE CAN MANAGE
I/O Performance and Capacity: %ACH COMPUTE NODE AND STORAGE CELL IS
EQUIPPED WITH ONE 'BPS 1$2 DUAL PORT )NFINI"AND CARD THROUGH WHICH IN
PRACTICALITY EACH COMPUTE NODE CAN TRANSMITRECEIVE A MAXIMUM OF
GIGABYTES PER SECOND )F THIS IS SUFFICIENT BANDWIDTH THEN THE DECISION OF
MOVING TO A MULTI NODE 2!# CONFIGURATION MAY BE MORE OF AN (!
CONSIDERATION )F YOU HAVE )/ HUNGRY APPLICATIONS THAT REQUIRE MORE
THROUGHPUT THAN ONE COMPUTE NODE CAN PROVIDE THEN 2!# MAY BE USED TO
PROVIDE HIGH AVAILABILITY AS WELL AS ADDITIONAL )/ CAPACITY
Patching and Testing: !NOTHER KEY CONSIDERATION IN DESIGNING A STABLE DATABASE
ENVIRONMENT IS PROVIDING A SEPARATE AREA WHERE PATCHES AND NEW FEATURES CAN
BE TESTED BEFORE ROLLING THEM INTO PRODUCTION &OR NON %XADATA PLATFORMS
PATCHING AND UPGRADING GENERALLY INVOLVES /3 PATCHES AND /RACLE 2$"-3
PATCHES %XADATA IS A HIGHLY COMPLEX DATABASE PLATFORM CONSISTING OF SEVERAL
ADDITIONAL HARDWARE AND SOFTWARE LAYERS THAT MUST BE PATCHED PERIODICALLY
SUCH AS #ELL 3ERVER 2T[[2;8 ),/- FIRMWARE )NFINI"AND SWITCH FIRMWARE
)NFINI"AND NETWORK CARD FIRMWARE AND /&%$ DRIVERS !S SUCH IT IS ABSOLUTELY
CRUCIAL THAT A TEST ENVIRONMENT BE ISOLATED FROM CRITICAL SYSTEMS TO BE USED FOR
TESTING PATCHES
#(!04%2 #/-054% ./$% ,!9/54
■ +EVIN 3AYS 2EGARDING )/ 0ERFORMANCE AND #APACITY IT IS ESSENTIAL TO UNDERSTAND THAT A PRODUCERS ABILITY TO
SEND DATA IS METERED BY THE ABILITY OF THE CONSUMER TO INGEST THE DATA 4HERE IS INHERENT FLOW
CONTROL IN THE
RELATIONSHIP BETWEEN THE DATABASE GRID AND THE STORAGE GRID IN %XADATA ARCHITECTURE $URING A 3MART 3CAN EACH
CELL PRODUCER HAS THE CAPACITY TO PRODUCE AN OUTBOUND FLOW OF DATA VIA I$" AT ROUGHLY '"PS WITH THE 8
MODELS "UT THE DATA HAS TO GO SOMEWHERE 3CALING FROM THIS SINGLE
CELL NUMBER IS LIMITED TO THE AGGREGATE INGEST
BANDWIDTH OF THE DATABASE GRID THE CONSUMERS WHICH IS '"PS FOR A FULL RACK CONFIGURATION 4O PUT IT ANOTHER
WAY THE 8 MODELS CAN STREAM DATA UNIMPEDED FROM HARD DISK TO THE DATABASE GRID AT FULL ADVERTISED SCAN RATE
'"PS (OWEVER WHEN SCANNING BOTH FLASH AND HARD DISK ASSETS CONCURRENTLY THE FULL RACK SCAN RATE INCREASES
TO APPROXIMATELY '"PS 7HERE DOES THE EXCESS GO )TS CLEAR TO SEE THAT ONLY ABOUT OF THAT '"PS CAN
FLOW INTO THE DATABASE GRID 4HE NECESSARY PAYLOAD REDUCTION IS THE RESPONSIBILITY OF 3MART 3CAN FILTRATION AND
PROJECTION )F THE QUERIES ARE NOT SELECTIVE ENOUGH TO REDUCE THE PAYLOAD THE RESULT IS THROTTLED SCAN THROUGHPUT
4HAT LEADS TO A RELATED TOPIC EFFECTIVE SCAN RATES AND DATA FLOW
&OR %(## TABLES THE EFFECTIVE SCAN RATE IS EQUAL TO THE PHYSICAL SCAN RATE MULTIPLIED BY THE COMPRESSION RATIO &OR
INSTANCE CONSIDER AN %(## TABLE WITH A COMPRESSION RATIO OF THAT FITS WITHIN THE CAPACITY OF A FULL RACK
%XADATA 3MART &LASH #ACHE ROUGHLY 4" 3MART 3CAN WILL SCAN THIS TABLE AT ROUGHLY '"PS BUT THE EFFECTIVE
SCAN RATE IS ABOUT '"PS 7HILE NOBODY WOULD ARGUE THAT '"PS IS A PHENOMENAL SCAN RATE WITH CURRENT
TECHNOLOGY THE IMPORTANT QUESTION MUST STILL BE ASKED (OW SELECTIVE ARE YOUR QUERIES #ONSIDERING THE AGGREGATE
INGEST BANDWIDTH OF A FULL RACK 8 MODEL ITS CLEAR THAT THE '"PS EFFECTIVE SCAN RATE HAS TO BE FILTERED DOWN BY
OVER '" ¯ '" )F THE FILTRATION IS NOT AGGRESSIVE ENOUGH THE SCAN RATE TAILS OFF BECAUSE THE
CONSUMER THROTTLES THE PRODUCER (EREIN LIES A CONUNDRUMTHE SELECTIVITY OF YOUR QUERIES MUST COMPLEMENT THE
COMPRESSION FACTOR OF YOUR %(## DATA 4HE PROBLEM HOWEVER IS THAT QUERY SELECTIVITY IS A CONSTANT AND YET
SOMETHING MUST GIVE 4HAT SOMETHING IS PHYSICAL )/ #ONSIDER FOR EXAMPLE A QUERY ACCESSING TABLES WITH A
COMBINED FOOTPRINT OF 4" SO AS TO FIT WITHIN THE %XADATA 3MART &LASH CAPACITY OF A FULL RACK ,ETS SAY FURTHER THAT
THIS QUERY CITES VERY FEW COLUMNS AND IS SO SELECTIVE THAT 3MART 3CAN IS ABLE TO WHITTLE THE PAYLOAD DOWN BY
4HAT WOULD BE VERY EFFECTIVE FILTRATION AND AS SUCH SCANNING FLASH AND HARD DISK AT A FULL RACK RATE OF '"PS
WOULD RENDER AN I$" PAYLOAD OF ABOUT '"PSA RATE THAT DOESNT EVEN SATURATE TWO DATABASE HOSTS I$" INGEST
BANDWIDTH !T '"PS THE HYPOTHETICAL QUERY COMPLETES IN ABOUT SECONDS #AN %(## REDUCE THE QUERY TIME
&OR THE SAKE OF EXAMPLE LETS SAY THE DATA IN THE HYPOTHETICAL 4" TABLE COMPRESSES TENFOLD WITH %(## DOWN TO
ROUGHLY '" !T A 3MART 3CAN RATE OF '"PS THE QUERY SHOULD COMPLETE IN ABOUT SECONDS HOWEVER THE
DATA AND QUERY ARE THE SAME SO THE PAYLOAD REDUCTION REMAINS CONSTANT AT %(## DATA IS DECOMPRESSED IN
THE STORAGE GRID AFTER FILTRATION AND PROJECTION SO FOR A GIVEN QUERY THE SAME DATA FLOWS OVER I$" WHETHER
NONCOMPRESSED OR COMPRESSED WITH %(## 7ITH PAYLOAD REDUCTION BEING CONSTANT CAN OUR HYPOTHETICAL QUERY
IMPROVE IN A MANNER COMMENSURATE WITH THE COMPRESSION RATIO FOR EXAMPLE TH THE TIME .O BECAUSE
FILTERING AWAY FROM THE EFFECTIVE SCAN RATE OF '"PS RENDERS AN I$" DEMAND OF '"PSABOUT
#(!04%2 #/-054% ./$% ,!9/54
MORE THAN THE AGGREGATE INGEST BANDWIDTH OF THE DATABASE GRID 4HE SCAN RATE IS THEREFORE THROTTLED 4HE QUERY
SHOULD INDEED RUN FASTER THAN NONCOMPRESSED BUT AS WE CAN SEE THERE IS MORE TO IT THAN SIMPLE MATH 4HE
HYPOTHETICAL QUERY CANNOT COMPLETE IN TH THE TIME )N SUCH A CASE THE PHYSICAL SCAN RATE FOR THE FULL RACK
WOULD BE THROTTLED BACK TO ROUGHLY '"PS AND SO THE QUERY WOULD COMPLETE IN APPROXIMATELY SECONDS
TH THE TIME AS IN THE NONCOMPRESSED CASE 4HIS EXAMPLE DOES NOT TAKE INTO CONSIDERATION ANY OTHER FACTOR THAT
MAY CAUSE CONSUMERS TO THROTTLE PRODUCERS SUCH AS JOIN PROCESSING AND SORTING )T CONSIDERS ONLY THE AGGREGATE
I$" INGEST BANDWIDTH IN THE DATABASE GRID /THER FACTORS SUCH AS DATABASE GRID #05
SATURATION DUE TO HEAVY JOIN
AND SORT PROCESSING CAN FURTHER THROTTLE THE FLOW OF DATA OVER I$" 4HERE IS A DIRECT CORRELATION BETWEEN THE
SELECTIVITY OF A QUERY THE HOST PROCESSOR UTILIZATION LEVEL AND THE PERFORMANCE IMPROVEMENT DELIVERED BY %(## )
OFTEN HAVE TO REMIND PEOPLE THAT THE CONCEPT OF COMPRESSION AS A PERFORMANCE FEATURE IS QUITE NEW AND OFTEN
MISREPRESENTED
Non-RAC Configuration
#OMPUTE NODES MAY BE CONFIGURED IN A NUMBER OF WAYS )F YOUR APPLICATION DOES NOT NEED THE HIGH
AVAILABILITY OR SCALE
OUT FEATURES OF /RACLE 2!# THEN %XADATA PROVIDES AN EXCELLENT PLATFORM FOR
DELIVERING HIGH PERFORMANCE FOR STANDALONE DATABASE SERVERS 9OU CAN MANAGE )/ SERVICE LEVELS
BETWEEN INDEPENDENT DATABASES BY CONFIGURING THE )/2- 3EE #HAPTER FOR MORE INFORMATION ABOUT
)/2- )N A NON
2!# CONFIGURATION EACH COMPUTE NODE WILL HAVE ITS OWN NON
CLUSTERED !3- INSTANCE
THAT PROVIDES STORAGE FOR ALL DATABASES ON THAT SERVER %VEN THOUGH YOUR DATABASE SERVERS MAY BE
INDEPENDENT OF ONE ANOTHER THEY CAN STILL SHARE %XADATA STORAGE CELL DISKS 4HIS ALLOWS EACH DATABASE TO
MAKE USE OF THE FULL )/ BANDWIDTH OF THE %XADATA STORAGE SUBSYSTEM .OTE THAT IN THIS CONFIGURATION
#LUSTERWARE IS NOT INSTALLED AT ALL *UST LIKE ANY OTHER STANDALONE DATABASE SERVER MULTIPLE DATABASES
COEXIST QUITE NICELY WITHIN THE SAME DISK GROUPS &OR EXAMPLE LETS SAY YOU HAVE THREE DATABASES ON YOUR
SERVER CALLED B0;4B 7A AND ?0HA>;; !LL THREE DATABASES CAN SHARE THE SAME DISK GROUPS FOR STORAGE 4O
DO THIS ALL THREE DATABASES WOULD SET THEIR INSTANCE PARAMETERS AS FOLLOWS
SQNRaTPcTNUX[TNSTbc,30C0 N36
SQNaTR^eTahNUX[TNSTbc,A42> N36
)N &IGURE
WE SEE ALL EIGHT COMPUTE NODES IN AN %XADATA FULL RACK CONFIGURATION RUNNING
STANDALONE DATABASES &OR EXAMPLE $" .ODE USES THE 30C0 N36 AND A42> N36 DISK GROUPS WHICH ARE
SERVICED BY THE LOCAL NONCLUSTERED !3- INSTANCE %ACH !3- INSTANCE HAS ITS OWN SET OF !3- DISK
GROUPS WHICH CONSIST OF GRID DISKS FROM ALL STORAGE CELLS !T THE STORAGE CELL THESE INDEPENDENT !3-
INSTANCES CANNOT SHARE GRID DISKS %ACH !3- INSTANCE WILL HAVE ITS OWN PRIVATE SET OF GRID DISKS
#(!04%2 #/-054% ./$% ,!9/54
DB1
DB2
DB3
DB4
Database: DB1
ASM Diskgroups for DB1 DB5
DB6
DB7
DB8
Infiniband Switch
DATA_01_DG Disk Group RECO_01_DG Disk Group
DATA_02_DG Disk Group DATA_02_DG Disk Group
DATA_01_DG DATA_03_DG Disk Group DATA_03_DG Disk Group
Disk Group
DATA_04_DG Disk Group DATA_04_DG Disk Group
Each ASM disk group
DATA_05_DG Disk Group DATA_05_DG Disk Group
consists of 12 grid gisks
from each storage cell.
DATA_06_DG Disk Group DATA_06_DG Disk Group
DATA_07_DG Disk Group DATA_07_DG Disk Group
DATA_08_DG Disk Group DATA_08_DG Disk Group
14 Storage Cells, 168 Cell
disks, 16 ASM Disk Groups
Figure 15-1. Example of a non-RAC Exadata configuration
#(!04%2 #/-054% ./$% ,!9/54
2ECALL FROM #HAPTER THAT GRID DISKS ARE ACTUALLY SLICES OF CELL DISKS 4HESE GRID DISKS ARE IN TURN
USED TO CREATE !3- DISK GROUPS &OR EXAMPLE EACH OF THE FOLLOWING COMMANDS CREATES TWELVE ' GRID
DISKS ON STORAGE CELL ONE PER CELL DISK 4HESE GRID DISKS ARE THEN USED TO CREATE THE 30C0 x30C0' AND
A42>xA42>' DISK GROUPS )F ALL FOURTEEN STORAGE CELLS OF AN %XADATA FULL RACK CONFIGURATION ARE USED IT
WILL YIELD TERABYTES OF STORAGE ASSUMING 4" HIGH
CAPACITY DRIVES FOR EACH DISK GROUP STORAGE
CELLS ¾ GRID DISKS PER CELL ¾ '
2T[[2;8-2A40C46A8338B:0;;70A338B:?A458G,30C0 N36bXiT,! 6
2T[[2;8-2A40C46A8338B:0;;70A338B:?A458G,30C0'N36bXiT,! 6
2T[[2;8-2A40C46A8338B:0;;70A338B:?A458G,A42> N36bXiT,! 6
2T[[2;8-2A40C46A8338B:0;;70A338B:?A458G,A42>'N36bXiT,! 6
4ABLE
IS A SUMMARY OF A STORAGE LAYOUT THAT SUPPORTS EIGHT STANDALONE DATABASES EACH HAVING A
4" 30C0 DISK GROUP AND A 4" A42> DISK GROUP
Table 15-1. Grid Disks
Disk Group Grid Disk Total GB Storage Cell Cell Disk
30C0 30C0 N36N23N n n
30C0!30C0!N36N23N n n
30C0"30C0"N36N23N n n
30C0#30C0#N36N23N n n
30C0$30C0$N36N23N n n
30C0%30C0%N36N23N n n
30C0&30C0&N36N23N n n
30C0'30C0'N36N23N n n
A42> A42> N36N23N n n
A42>!A42>!N36N23N n n
A42>"A42>"N36N23N n n
A42>#A42>#N36N23N n n
A42>$A42>$N36N23N n n
A42>%A42>%N36N23N n n
#(!04%2 #/-054% ./$% ,!9/54
Disk Group Grid Disk Total GB Storage Cell Cell Disk
A42>&A42>&N36N23N n n
A42>'A42>'N36N23N n n
"Y DEFAULT ALL !3- INSTANCES WILL HAVE ACCESS TO ALL GRID DISKS 4HE FOLLOWING 31, QUERY RUN FROM ONE
OF THE !3- INSTANCES ILLUSTRATES THIS .OTICE HOW GRID DISKS FROM ALL STORAGE CELLS ARE FLAGGED AS
CANDIDATES THAT IS AVAILABLE FOR USE
BHB)0B< -bT[TRcbdQbca_PcW %[T]VcW_PcWVaXSSXbZK
UPX[Va^d_WTPSTaNbcPcdb
Ua^\ePb\NSXbZ
fWTaTbdQbca_PcW %[T]VcW_PcW[XZT30C0N36N
^abdQbca_PcW %[T]VcW_PcW[XZTA42>N36N
^aSTaQhUPX[Va^d_bdQbca_PcW %[T]VcW_PcW*
6A8338B:508;6A>D?74034ANBC0CDB
30C0 N23N NRT[[ 24;; 20=3830C4
30C0!N23N NRT[[ 24;; 20=3830C4
30C0"N23N NRT[[ 24;; 20=3830C4
30C0#N23N NRT[[ 24;; 20=3830C4
30C0$N23N NRT[[ 24;; 20=3830C4
30C0%N23N NRT[[ 24;; 20=3830C4
30C0&N23N NRT[[ 24;; 20=3830C4
30C0'N23N NRT[[ 24;; 20=3830C4
30C0 N23N NRT[[ !24;; !20=3830C4
30C0!N23N NRT[[ !24;; !20=3830C4
30C0"N23N NRT[[ !24;; !20=3830C4
30C0&N23N NRT[[ "24;; "20=3830C4
30C0'N23N NRT[[ "24;; "20=3830C4
A42> N23N NRT[[ 24;; 20=3830C4
A42>!N23N NRT[[ 24;; 20=3830C4
A42>"N23N NRT[[ 24;; 20=3830C4
A42>#N23N NRT[[ 24;; 20=3830C4
A42>$N23N NRT[[ 24;; 20=3830C4
A42>%N23N NRT[[ 24;; 20=3830C4
A42>&N23N NRT[[ 24;; 20=3830C4
A42>'N23N NRT[[ 24;; 20=3830C4
A42> N23N NRT[[ !24;; !20=3830C4
A42>!N23N NRT[[ !24;; !20=3830C4
A42>"N23N NRT[[ !24;; !20=3830C4
A42>&N23N NRT[[ "24;; "20=3830C4
A42>'N23N NRT[[ "24;; "20=3830C4
#(!04%2 #/-054% ./$% ,!9/54
7ITH THIS MANY GRID DISKS IN THIS CASE VISIBLE TO ALL !3- INSTANCES ITS EASY TO SEE HOW THEY
CAN BE ACCIDENTALLY MISALLOCATED TO THE WRONG !3- DISK GROUPS 4O PROTECT YOURSELF FROM MISTAKES LIKE
THAT YOU MIGHT WANT TO CONSIDER USING CELL SECURITY TO RESTRICT THE ACCESS OF EACH !3- INSTANCE SO THAT IT
ONLY hSEESv ITS OWN SET OF GRID DISKS &OR DETAILED STEPS ON HOW TO IMPLEMENT CELL SECURITY REFER TO
#HAPTER
RAC Clusters
.OW THAT WEVE DISCUSSED HOW EACH COMPUTE NODE AND STORAGE CELL CAN BE CONFIGURED IN A FULLY
INDEPENDENT FASHION LETS TAKE A LOOK AT HOW THEY CAN BE CLUSTERED TOGETHER TO PROVIDE HIGH AVAILABILITY
AND HORIZONTAL SCALABILITY USING 2!# CLUSTERS "UT BEFORE WE DO THAT WELL TAKE A BRIEF DETOUR AND
ESTABLISH WHAT HIGH AVAILABILITY AND SCALABILITY ARE
(IGH AVAILABILITY (! IS A FAIRLY WELL UNDERSTOOD CONCEPT BUT IT OFTEN GETS CONFUSED WITH FAULT
TOLERANCE )N A TRULY FAULT
TOLERANT SYSTEM EVERY COMPONENT IS REDUNDANT )F ONE COMPONENT FAILS
ANOTHER COMPONENT TAKES OVER WITHOUT ANY INTERRUPTION TO SERVICE (IGH AVAILABILITY ALSO INVOLVES
COMPONENT REDUNDANCY BUT FAILURES MAY CAUSE A BRIEF INTERRUPTION TO SERVICE WHILE THE SYSTEM
RECONFIGURES TO USE THE REDUNDANT COMPONENT 7ORK IN PROGRESS DURING THE INTERRUPTION MUST BE
RESUBMITTED OR CONTINUED ON THE REDUNDANT COMPONENT 4HE TIME IT TAKES TO DETECT A FAILURE
RECONFIGURE AND RESUME WORK VARIES GREATLY IN (! SYSTEMS &OR EXAMPLE ACTIVEPASSIVE 5NIX CLUSTERS
HAVE BEEN USED EXTENSIVELY TO PROVIDE GRACEFUL FAILOVER IN THE EVENT OF A SERVER CRASH .OW YOU MIGHT
CHUCKLE TO YOURSELF WHEN YOU SEE THE WORDS hGRACEFUL FAILOVERv AND hCRASHv USED IN THE SAME SENTENCE
UNLESS YOU WORK IN THE AIRLINE INDUSTRY SO LET ME EXPLAIN 'RACEFUL FAILOVER IN THE CONTEXT OF
ACTIVEPASSIVE CLUSTERS MEANS THAT WHEN A SYSTEM FAILURE OCCURS OR A CRITICAL COMPONENT FAILS THE
RESOURCES THAT MAKE UP THE APPLICATION DATABASE AND INFRASTRUCTURE ARE SHUT DOWN ON THE PRIMARY
SYSTEM AND BROUGHT BACK ONLINE ON THE REDUNDANT SYSTEM AUTOMATICALLY WITH AS LITTLE DOWNTIME AS
POSSIBLE 4HE ALTERNATIVE AND SOMEWHAT LESS GRACEFUL TYPE OF FAILOVER WOULD INVOLVE A PHONE CALL TO YOUR
SUPPORT STAFF AT IN THE MORNING )N ACTIVEPASSIVE CLUSTERS THE DATABASE AND POSSIBLY OTHER
APPLICATIONS ONLY RUN ON ONE NODE AT A TIME &AILOVER USING IN THIS CONFIGURATION CAN TAKE SEVERAL
MINUTES TO COMPLETE DEPENDING ON WHAT RESOURCES AND APPLICATIONS MUST BE MIGRATED /RACLE 2!# USES
AN ACTIVEACTIVE CLUSTER ARCHITECTURE &AILOVER ON AN 2!# SYSTEM COMMONLY TAKES LESS THAN A MINUTE TO
COMPLETE 4RUE FAULT TOLERANCE IS GENERALLY VERY DIFFICULT AND MUCH MORE EXPENSIVE TO IMPLEMENT THAN
HIGH AVAILABILITY 4HE TYPE OF SYSTEM AND IMPACT OR COST OF A FAILURE USUALLY DICTATES WHICH IS MORE
APPROPRIATE #RITICAL SYSTEMS ON AN AIRLINER SPACE STATION OR A LIFE SUPPORT SYSTEM EASILY JUSTIFY A FAULT
TOLERANT ARCHITECTURE "Y CONTRAST A WEB APPLICATION SERVICING THE COMPANYS RETAIL STORE FRONT USUALLY
CANNOT JUSTIFY THE COST AND COMPLEXITY OF A FULLY FAULT
TOLERANT ARCHITECTURE %XADATA IS A HIGH
AVAILABILITY
ARCHITECTURE PROVIDING FULLY REDUNDANT HARDWARE COMPONENTS 7HEN /RACLE 2!# IS USED THIS
REDUNDANCY AND FAST FAILOVER IS EXTENDED TO THE DATABASE TIER
7HEN #05 MEMORY OR )/ RESOURCE LIMITS FOR A SINGLE SERVER ARE REACHED ADDITIONAL SERVERS MUST
BE ADDED TO INCREASE CAPACITY 4HE TERM hSCALABILITYv IS OFTEN USED SYNONYMOUSLY WITH PERFORMANCE
4HAT IS INCREASING CAPACITY EQUALS INCREASING PERFORMANCE "UT THE CORRELATION BETWEEN CAPACITY AND
PERFORMANCE IS NOT A DIRECT ONE 4AKE FOR EXAMPLE A SINGLE
THREADED #05 INTENSIVE PROGRAM THAT TAKES
MINUTES TO COMPLETE ON A TWO
#05 SERVER !SSUMING THE SERVER ISNT #05
BOUND INSTALLING TWO MORE
#05S IS NOT GOING TO MAKE THE PROCESS RUN ANY FASTER )F IT CAN ONLY RUN ON ONE #05 AT A TIME IT WILL ONLY
EXECUTE AS FAST AS ONE #05 CAN PROCESS IT 0ERFORMANCE WILL ONLY IMPROVE IF ADDING MORE #05S ALLOWS A
PROCESS TO HAVE MORE UNINTERRUPTED TIME ON THE PROCESSOR .EITHER WILL IT RUN ANY FASTER IF WE RUN IT ON A
FOUR
NODE CLUSTER !S THE OLD SAYING GOES NINE PREGNANT WOMEN CANNOT MAKE A BABY IN ONE MONTH
(OWEVER SCALING OUT TO FOUR SERVERS COULD MEAN THAT WE CAN RUN FOUR COPIES OF OUR PROGRAM
CONCURRENTLY AND GET ROUGHLY FOUR TIMES THE AMOUNT OF WORK DONE IN THE SAME MINUTES 4O SUM IT UP
SCALING OUT ADDS CAPACITY TO YOUR SYSTEM 7HETHER OR NOT IT IMPROVES PERFORMANCE DEPENDS ON HOW
SCALABLE YOUR APPLICATION IS AND HOW HEAVILY LOADED YOUR CURRENT SYSTEM IS +EEP IN MIND THAT /RACLE
#(!04%2 #/-054% ./$% ,!9/54
2!# SCALES EXTREMELY WELL FOR WELL WRITTEN APPLICATIONS #ONVERSELY POORLY WRITTEN APPLICATIONS TEND TO
SCALE POORLY
%XADATA CAN BE CONFIGURED AS MULTIPLE 2!# CLUSTERS TO PROVIDE ISOLATION BETWEEN ENVIRONMENTS
4HIS ALLOWS THE CLUSTERS TO BE MANAGED PATCHED AND ADMINISTERED INDEPENDENTLY !T THE DATABASE TIER
THIS IS DONE IN THE SAME WAY YOU WOULD CLUSTER ANY ORDINARY SET OF SERVERS USING /RACLE #LUSTERWARE 4O
CONFIGURE STORAGE CELLS TO SERVICE SPECIFIC COMPUTE NODES THE RT[[X_ ^aP FILE ON EACH NODE LISTS THE
STORAGE CELLS IT WILL USE &OR EXAMPLE THE FOLLOWING RT[[X_ ^aP FILE LISTS SEVEN OF THE FOURTEEN STORAGE CELLS
BY THEIR NETWORK ADDRESS
JT]ZSQ )^aPR[T)4G31 L W^\T ^aPR[T
-RPc TcR ^aPR[T RT[[ ]Tcf^aZ R^]UXV RT[[X_ ^aP
RT[[, (! %' ! (
RT[[, (! %' !
RT[[, (! %' !
RT[[, (! %' ! !
RT[[, (! %' ! "
RT[[, (! %' ! #
RT[[, (! %' ! $
7HEN !3- STARTS UP IT SEARCHES THE STORAGE CELLS ON EACH OF THESE )0 ADDRESSES FOR GRID DISKS IT CAN
USE FOR CONFIGURING !3- DISK GROUPS !LTERNATIVELY #ELL 3ECURITY CAN BE USED TO LOCK DOWN ACCESS SO THAT
ONLY CERTAIN STORAGE CELLS ARE AVAILABLE FOR COMPUTE NODES TO USE 4HE RT[[X_ ^aP FILE AND CELL SECURITY ARE
COVERED IN DETAIL IN #HAPTER
4O ILLUSTRATE WHAT A MULTI
2!# %XADATA CONFIGURATION MIGHT LOOK LIKE LETS CONSIDER AN %XADATA 6
FULL RACK CONFIGURATION PARTITIONED INTO THREE /RACLE 2!# CLUSTERS ! FULL RACK GIVES US EIGHT COMPUTE
NODES AND FOURTEEN STORAGE CELLS TO WORK WITH #ONSIDER AN %XADATA FULL RACK CONFIGURED AS FOLLOWS
v /NE 0RODUCTION 2!# CLUSTER WITH FOUR COMPUTE NODES AND SEVEN STORAGE CELLS
v /NE 4EST 2!# CLUSTER WITH TWO COMPUTE NODES AND THREE STORAGE CELLS
v /NE $EVELOPMENT 2!# CLUSTER WITH TWO COMPUTE NODES AND FOUR STORAGE CELLS
4ABLE
SHOWS THE RESOURCE ALLOCATION OF THESE 2!# CLUSTERS EACH WITH ITS OWN STORAGE GRID !S
YOU READ THIS TABLE KEEP IN MIND THAT HARDWARE IS A MOVING TARGET 4HESE FIGURES ARE FROM AN %XADATA 6
)N THIS EXAMPLE WE USED THE HIGH
CAPACITY 4" 3!4! DISK DRIVES
Table 15-2. Cluster Resources
Cluster Db Servers Db Memory Db CPU Storage Cell Cell Disks Raw Storage
02/$?#,534%2 0ROD
0ROD ' ¾ ¾ n 4
4%34?#,534%2 4EST 4EST ' ¾ ¾ n 4
$%6?#,534%2 $EV $EV ' ¾ ¾ n 4
4HESE 2!# ENVIRONMENTS CAN BE PATCHED AND UPGRADED COMPLETELY INDEPENDENTLY OF ONE ANOTHER
4HE ONLY HARDWARE RESOURCES THEY SHARE ARE THE )NFINI"AND SWITCH AND THE +6- SWITCH )F YOU ARE
CONSIDERING A MULTI 2!# CONFIGURATION LIKE THIS KEEP IN MIND THAT PATCHES TO THE )NFINI"AND SWITCH WILL
#(!04%2 #/-054% ./$% ,!9/54
AFFECT ALL STORAGE CELLS AND COMPUTE NODES 4HIS IS ALSO TRUE OF THE +6- SWITCH WHICH IS NOT NEEDED TO
RUN THE CLUSTERS !3- INSTANCES OR DATABASES SO IF AN ISSUE TAKES YOUR +6- SWITCH OFFLINE FOR A FEW DAYS
IT WONT IMPACT SYSTEM PERFORMANCE OR AVAILABILITY &IGURE
ILLUSTRATES WHAT THIS CLUSTER CONFIGURATION
WOULD LOOK LIKE
PROD_CLUSTER
Prod1
Prod2
Prod3
Prod4 Storage Cells 1-7
84 Cell Disks, 168T Raw Storage
DATA_DG Disk Group RECO_DG Disk Group
SYSTEM_DG Disk Group
TEST_CLUSTER
Test1
Test2
Storage Cells 8-10
36 Cell Disks, 72T Raw Storage
DATA_DG Disk Group RECO_DG Disk Group
SYSTEM_DG Disk Groups
DEV_CLUSTER
Dev1
Dev2 Storage Cells 8-10
48 Cell Disks, 96T Raw Storage
DATA_DG Disk Group RECO_DG Disk Group
SYSTEM_DG Disk Group
Figure 15-2. An Exadata full rack configured for three RAC clusters
Typical Exadata Configuration
4HE TWO CONFIGURATION STRATEGIES WEVE DISCUSSED SO FAR ARE FAIRLY EXTREME EXAMPLES 4HE hNON 2!#
$ATABASEv CONFIGURATION ILLUSTRATED HOW %XADATA CAN BE CONFIGURED WITHOUT #LUSTERWARE TO CREATE A
hSHARED NOTHINGv CONSOLIDATION PLATFORM 4HE SECOND EXAMPLE h2!# #LUSTERS v SHOWED HOW
#(!04%2 #/-054% ./$% ,!9/54
#LUSTERWARE CAN BE USED TO CREATE MULTIPLE ISOLATED 2!# CLUSTERS .EITHER OF THESE CONFIGURATIONS IS
TYPICALLY FOUND IN THE REAL WORLD BUT THEY ILLUSTRATE THE CONFIGURATION CAPABILITIES OF %XADATA .OW LETS
TAKE A LOOK AT A CONFIGURATION WE COMMONLY SEE IN THE FIELD &IGURE
SHOWS A TYPICAL SYSTEM WITH TWO
%XADATA HALF RACKS )T CONSISTS OF A PRODUCTION CLUSTER ?A>3N2;DBC4A HOSTING A TWO
NODE PRODUCTION
DATABASE AND A TWO
NODE 5!4 DATABASE 4HE PRODUCTION AND 5!4 DATABASES SHARE THE SAME !3- DISK
GROUPS MADE UP OF ALL GRID DISKS ACROSS ALL STORAGE CELLS )/ RESOURCES ARE REGULATED AND PRIORITIZED
USING %XADATA )/ 2ESOURCE -ANAGER )/2- DISCUSSED IN #HAPTER 4HE PRODUCTION DATABASE USES
!CTIVE $ATA 'UARD TO MAINTAIN A PHYSICAL STANDBY FOR DISASTER RECOVERY AND REPORTING PURPOSES 4HE 5!4
DATABASE IS NOT CONSIDERED BUSINESS
CRITICAL SO IT IS NOT PROTECTED WITH $ATA 'UARD /N THE STANDBY
CLUSTER BC1HN2;DBC4A THE BcQh DATABASE USES FOUR OF THE SEVEN STORAGE CELLS FOR ITS !3- STORAGE /N THE
DEVELOPMENT CLUSTER 34EN2;DBC4A THE 3Te DATABASE USES THE REMAINING THREE CELLS FOR ITS !3- STORAGE
4HE DEVELOPMENT CLUSTER IS USED FOR ONGOING PRODUCT DEVELOPMENT AND PROVIDES A TEST BED FOR INSTALLING
%XADATA PATCHES DATABASE UPGRADES AND NEW FEATURES
RAC Cluster: PROD_CLUSTER RAC Cluster: STBY_CLUSTER
Stby1
Prod1
Stby2
Prod2
DATA_DG
uard
Data G RECO_DG
DATA_DG SYSTEM_DG
RECO_DG
4 Storage Cells,
48 Cell disks, 96T Raw Storage.
SYSTEM_DG 3 ASM Disk Groups
UAT1
RAC Cluster: DEV_CLUSTER
UAT2 Dev1
Dev2
Half Rack with 1 RAC Cluster
7 Storage Cells,
DATA_DG
84 Cell disks, 168T Raw Storage. 3 ASM Disk RECO_DG
Groups Shared by Prod and UAT databases.
SYSTEM_DG
3 Storage Cells,
36 Cell disks, 72T Raw Storage.
3 ASM Disk Groups
Half Rack with 2 RAC Clusters
4, 3 Storage Cells respectively,
Disk Groups not shared between Stby & Dev
Figure 15-3. A typical configuration
Exadata Clusters
%XADATAS ABILITY TO SCALE OUT DOESNT END WHEN THE RACK IS FULL 7HEN ONE %XADATA RACK DOESNT QUITE GET
THE JOB DONE FOR YOU ADDITIONAL RACKS MAY BE ADDED TO THE CLUSTER CREATING A LARGE
SCALE DATABASE GRID
5P TO EIGHT RACKS MAY BE CABLED TOGETHER TO CREATE A MASSIVE DATABASE GRID CONSISTING OF DATABASE
SERVERS AND TERABYTES OF STORAGE !CTUALLY %XADATA WILL SCALE BEYOND EIGHT RACKS BUT ADDITIONAL
)NFINI"AND SWITCHES MUST BE PURCHASED TO DO IT %XADATA LINKS CABINETS TOGETHER USING WHAT /RACLE CALLS A
#(!04%2 #/-054% ./$% ,!9/54
spine switch 4HE SPINE SWITCH IS INCLUDED WITH ALL HALF RACK AND FULL RACK CONFIGURATIONS 1UARTER RACK
CONFIGURATIONS DO NOT HAVE A SPINE SWITCH AND CANNOT BE LINKED WITH OTHER %XADATA RACKS )N A FULL RACK
CONFIGURATION THE PORTS OF A LEAF SWITCH ARE USED AS FOLLOWS
v %IGHT LINKS TO THE DATABASE SERVERS
v &OURTEEN LINKS TO THE STORAGE CELLS
v 3EVEN LINKS TO THE REDUNDANT LEAF SWITCH
v /NE LINK TO THE SPINE SWITCH
v 3IX PORTS OPEN
&IGURE
SHOWS AN %XADATA FULL RACK CONFIGURATION THAT IS NOT LINKED TO ANY OTHER %XADATA RACK )TS
INTERESTING THAT /RACLE CHOSE TO CONNECT THE TWO LEAF SWITCHES TOGETHER USING THE SEVEN SPARE CABLES
0ERHAPS ITS BECAUSE THESE CABLES ARE PRECONFIGURED IN THE FACTORY AND PATCHING THEM INTO THE LEAF
SWITCHES SIMPLY KEEPS THEM OUT OF THE WAY AND MAKES IT EASIER TO RECONFIGURE LATER 4HE LEAF SWITCHES
CERTAINLY DO NOT NEED TO BE LINKED TOGETHER 4HE LINK BETWEEN THE LEAF SWITCHES AND THE SPINE SWITCH
DOESNT SERVE ANY PURPOSE IN THIS CONFIGURATION EITHER )T IS ONLY USED IF TWO OR MORE %XADATA RACKS ARE
LINKED TOGETHER
Spine Switch
Links to 8 Database Links to 8 Database
Servers Servers
7 Inter-switch links
Leaf Switch 1 Leaf Switch 2
Links to 14 Storage Cells Links to 14 Storage Cells
Figure 15-4. An Exadata full rack InfiniBand network
4HE SPINE SWITCH IS JUST LIKE THE OTHER TWO )NFINI"AND SWITCHES THAT SERVICE THE CLUSTER AND STORAGE
NETWORK 4HE DIAGRAM MAY LEAD YOU TO WONDER WHY /RACLE WOULD MAKE EVERY COMPONENT IN THE RACK
REDUNDANT EXCEPT THE SPINE SWITCH 4HE ANSWER IS THAT REDUNDANCY IS PROVIDED BY CONNECTING EACH LEAF
SWITCH TO EVERY SPINE SWITCH IN THE CONFIGURATION FROM TWO TO EIGHT SPINE SWITCHES
4O NETWORK TWO %XADATA RACKS TOGETHER THE SEVEN INTER
SWITCH CABLES SEEN IN &IGURE
ARE
REDISTRIBUTED SO THAT FOUR OF THEM LINK THE LEAF SWITCH WITH ITS INTERNAL SPINE SWITCH AND FOUR OF THEM LINK
THE LEAF SWITCH TO THE SPINE SWITCH IN THE ADJACENT RACK &IGURE
SHOWS THE NETWORK CONFIGURATION FOR
TWO %XADATA RACKS NETWORKED TOGETHER 7HEN EIGHT %XADATA RACKS ARE LINKED TOGETHER THE SEVEN INTER
#(!04%2 #/-054% ./$% ,!9/54
SWITCH CABLES SEEN IN &IGURE ARE REDISTRIBUTED SO THAT EACH LEAF TO SPINE SWITCH LINK USES ONE CABLE
EIGHT CABLES PER LEAF SWITCH 7HEN YOURE LINKING FROM THREE TO SEVEN %XADATA RACKS TOGETHER THE SEVEN
INTER SWITCH CABLES ARE REDISTRIBUTED AS EVENLY AS POSSIBLE ACROSS ALL LEAF TO SPINE SWITCH LINKS ,EAF
SWITCHES ARE NOT LINKED TO OTHER LEAF SWITCHES AND SPINE SWITCHES ARE NOT LINKED TO OTHER SPINE SWITCHES
.O CHANGES ARE EVER NEEDED FOR THE LEAF SWITCH LINKS TO THE COMPUTE NODES AND STORAGE CELLS
Exadata Full Rack #1 Exadata Full Rack #2
Spine Switch Spine Switch
Leaf Switch 1 Leaf Switch 2 Leaf Switch 1 Leaf Switch 2
Figure 15-5. A switch configuration for two Exadata racks, with one database grid
Summary
%XADATA IS A HIGHLY COMPLEX HIGHLY CONFIGURABLE DATABASE PLATFORM )N #HAPTER WE TALKED ABOUT ALL
THE VARIOUS WAYS DISK DRIVES AND STORAGE CELLS CAN BE PROVISIONED SEPARATELY OR IN CONCERT TO DELIVER WELL
BALANCED HIGH
PERFORMANCE )/ TO YOUR /RACLE DATABASES )N THIS CHAPTER WE TURNED OUR ATTENTION TO
PROVISIONING CAPABILITIES AND STRATEGIES AT THE DATABASE TIER %XADATA IS RARELY USED TO HOST STANDALONE
DATABASE SERVERS )N MOST CASES IT IS FAR BETTER SUITED FOR /RACLE 2!# CLUSTERS 5NDERSTANDING THAT EVERY
COMPUTE NODE AND STORAGE CELL IS A FULLY INDEPENDENT COMPONENT IS IMPORTANT SO WE SPENT A
CONSIDERABLE AMOUNT OF TIME SHOWING HOW TO PROVISION EIGHT STANDALONE COMPUTE NODES ON AN %XADATA
FULL RACK CONFIGURATION &ROM THERE WE MOVED ON TO AN /RACLE 2!# PROVISIONING STRATEGY THAT PROVIDED
SEPARATION BETWEEN THREE COMPUTING ENVIRONMENTS !ND FINALLY WE TOUCHED ON HOW %XADATA RACKS MAY
BE NETWORKED TOGETHER TO BUILD MASSIVE DATABASE GRIDS 5NDERSTANDING THE CONCEPTS EXPLORED IN #HAPTER
AND OF THIS BOOK WILL HELP YOU MAKE THE RIGHT CHOICES WHEN THE TIME COMES TO ARCHITECT A
PROVISIONING STRATEGY FOR YOUR %XADATA DATABASE ENVIRONMENT
C H A P T E R 16
Unlearning Some Things
We Thought We Knew
“It ain't what you don't know that gets you into trouble.
It's what you know for sure that just ain't so.”
~ -ARK 4WAIN
/RACLE DOES SOME THINGS VERY DIFFERENTLY WHEN RUNNING ON %XADATA THAN WHEN RUNNING ON NON
%XADATA
PLATFORMS 4HE OPTIMIZATIONS PROVIDED BY %XADATA ARE DESIGNED TO TAKE A DIFFERENT APPROACH THAN /RACLE
HAS TRADITIONALLY FOLLOWED 4HIS CHANGE MEANS THAT WE NEED TO ATTACK SOME PROBLEMS WITH A COMPLETELY
DIFFERENT MINDSET 4HAT IS NOT TO SAY THAT EVERYTHING IS DIFFERENT )N FACT MOST OF THE FUNDAMENTAL
PRINCIPLES REMAIN UNCHANGED !FTER ALL THE SAME DATABASE SOFTWARE RUNS ON %XADATA THAT RUNS ON OTHER
PLATFORMS "UT THERE ARE SOME THINGS THAT ARE JUST FUNDAMENTALLY DIFFERENT
4HIS CHAPTER FOCUSES ON HOW WE SHOULD CHANGE OUR THINKING WHEN RUNNING DATABASES ON %XADATA
A Tale of Two Systems
4HE WAY WE THINK ABOUT SYSTEMS RUNNING ON %XADATA LARGELY DEPENDS ON THE WORKLOAD BEING EXECUTED
/,40 ORIENTED WORKLOADS TEND TO FOCUS US ON USING FLASH CACHE FOR SPEEDING UP SMALL READS "UT FRANKLY
THIS TYPE OF WORKLOAD IS NOT ABLE TO TAKE ADVANTAGE OF MOST OF THE ADVANTAGES PROVIDED BY %XADATA $7
ORIENTED WORKLOADS TEND TO FOCUS US ON MAKING USE OF 3MART 3CANS AT EVERY OPPORTUNITY AND TRYING TO USE
ALL THE AVAILABLE RESOURCES #05 RESOURCES ON THE STORAGE AND DATABASE TIERS FOR EXAMPLE 4HIS IS WHERE
%XADATAS BUILT IN ADVANTAGES CAN HAVE THE MOST IMPACT 5NFORTUNATELY MOST SYSTEMS EXHIBIT
CHARACTERISTICS OF BOTH $7 AND /,40 WORKLOADS 4HESE hMIXED 7ORKLOADSv ARE THE MOST DIFFICULT AND
IRONICALLY THE MOST COMMON 4HEY ARE THE MOST COMMON BECAUSE IT IS RARE TO SEE AN /,40 ORIENTED
SYSTEM THAT DOESNT HAVE SOME REPORTING COMPONENT THAT PRODUCES LONG RUNNING THROUGHPUT SENSITIVE
QUERIES )TS ALSO QUITE COMMON TO SEE $7 ORIENTED WORKLOADS THAT HAVE /,40 LIKE TRICKLE FEEDS 4HESE
COMBINATION SYSTEMS REQUIRE THE MOST DIFFICULT THOUGHT PROCESS BECAUSE DEPENDING ON THE ISSUE AT
HAND YOU WILL HAVE TO BE CONSTANTLY RESETTING YOUR APPROACH TO MANAGING PERFORMANCE 4HEY ARE ALSO
DIFFICULT BECAUSE $7 WORKLOADS ARE GENERALLY CONSTRAINED BY THE DATA FLOW DYNAMICS THAT CONTROL
THROUGHPUT WHILE /,40 SYSTEMS ARE USUALLY CONSTRAINED BY LATENCY ISSUES 3O FOR MIXED WORKLOADS YOU
BASICALLY NEED TO TRAIN YOURSELF TO EVALUATE EACH SCENARIO AND CATEGORIZE IT AS EITHER LATENCY SENSITIVE OR
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
THROUGHPUT
SENSITIVE 4HIS EVALUATION OF WORKLOAD CHARACTERISTICS SHOULD BE DONE PRIOR TO BEGINNING ANY
ANALYSIS
OLTP-Oriented Workloads
4HERE IS LITTLE TO SAY ABOUT RUNNING /,40 WORKLOADS ON %XADATA 3INCE %XADATA RUNS STANDARD /RACLE
DATABASE SOFTWARE YOU SHOULD NOT HAVE TO ADJUST YOUR BASIC APPROACH SIGNIFICANTLY 4HERE ARE ONLY A
HANDFUL OF POINTS TO KEEP IN MIND WITH THIS TYPE OF SYSTEM
%XADATA 3MART &LASH #ACHE %3&#
4HE KEY COMPONENT OF %XADATA WHEN IT COMES TO /,40 WORKLOADS IS %XADATA 3MART &LASH #ACHE %3&#
WHICH CAN SIGNIFICANTLY REDUCE DISK ACCESS TIMES FOR SMALL READS &OR THAT REASON IT IS IMPORTANT TO VERIFY
THAT %3&# IS WORKING CORRECTLY &OR THIS TYPE OF WORKLOAD YOU SHOULD ALSO EXPECT THAT A LARGE PERCENTAGE
OF PHYSICAL )/ OPERATIONS ARE BEING SATISFIED BY %3&# 4HIS CAN BE INFERRED FAIRLY EASILY BY LOOKING AT THE
AVERAGE SINGLE
BLOCK READ TIMES ! SINGLE
BLOCK READ SHOULD TAKE AROUND MS IF IT IS SATISFIED BY FLASH
CACHE "Y CONTRAST SINGLE
BLOCK READS TAKE ON AVERAGE AROUND MS IF THEY ARE SATISFIED BY ACTUAL DISK
READS 3TANDARD !72 REPORTS PROVIDE BOTH AVERAGE VALUES AND A HISTOGRAM OF WAIT EVENTS )F THE AVERAGE
SINGLE
BLOCK READ TIMES ARE WELL ABOVE THE MS RANGE YOU SHOULD BE LOOKING FOR A SYSTEMIC PROBLEM LIKE
FLASH CARDS THAT ARE NOT WORKING OR A CRITICAL TABLE HAS BEEN DEFINED TO NEVER BE CACHED USING THE
24;;N5;0B7N20274=>=4 SYNTAX 4HE HISTOGRAMS SHOULD BE USED AS WELL TO VERIFY THAT THE AVERAGE IS NOT
COVERING UP A SIGNIFICANT NUMBER OF OUTLIERS (ERES THE RT[[R[X SYNTAX TO CHECK THE STATUS OF THE FLASH
CARDS
2T[[2;8-[XbcU[PbWRPRWTSTcPX[
]P\T)S\ RT[ "N5;0B720274
RT[[3XbZ)
53N NS\ RT[ "53N NS\ RT[ "53N !NS\ RT[ "53N "NS\ RT[ "53N #NS\ RT[ "53N $NS\ RT[
"53N %NS\ RT[ "53N &NS\ RT[ "53N 'NS\ RT[ "53N (NS\ RT[ "53N NS\ RT[ "53N NS\ RT
[ "53N !NS\ RT[ "53N "NS\ RT[ "53N #NS\ RT[ "53N $NS\ RT[ "}
RaTPcX^]CX\T)! " !!C &)"()#% $)
XS)'$ QT&'# & #R ###$ ( P' S"R(% PS(!#Q
bXiT)"%$ !$6
bcPcdb)RRaXcXRP[
.OTE THAT THE bcPcdb ATTRIBUTE ON THIS CELL IS RaXcXRP[ !S YOU MIGHT EXPECT THIS IS NOT A GOOD THING
/N THIS PARTICULAR SYSTEM THE FLASH CACHE HAD BASICALLY DISABLED ITSELF 7E NOTICED IT BECAUSE THE SINGLE
BLOCK READ TIMES HAD SLOWED DOWN 4HIS EXAMPLE IS FROM AN EARLY VERSION OF RT[[bae 4HE LATER VERSIONS
INCLUDE A LITTLE MORE INFORMATION (ERE IS AN EXAMPLE FROM RT[[bae
2T[[2;8-[XbcU[PbWRPRWTSTcPX[
]P\T)T]ZRT[ "N5;0B720274
RT[[3XbZ)
53N #NRT[[ "53N 'NRT[[ "53N "NRT[[ "53N NRT[[ "53N !NRT[[ "53N #NRT[[ "53N NRT[[ "53N
NRT[[ "53N $NRT[[ "53N "NRT[[ "53N !NRT[[ "53N NRT[[ "53N $NRT[[ "53N (NRT[[ "53N &N
RT[[ "53N %NRT[[ "
RaTPcX^]CX\T)! " !&C &)$ ) # $)
STVaPSTS2T[[SXbZb)
TUUTRcXeT2PRWTBXiT)"%$ !$6
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
XS)&U% '& T S$U! #(( Q&T$ " R#P!"'ST !
bXiT)"%$ !$6
bcPcdb)]]^a\P[
.OTICE THE NEW ATTRIBUTE STVaPSTS2T[[SXbZb !LSO NOTICE THAT THE FLASH CACHE ON THIS CELL SHOWS A
bcPcdb OF ]^a\P[ -ONITORING STORAGE SOFTWARE BEHAVIOR IS COVERED IN MORE DETAIL IN #HAPTER
3CALABILITY
!NOTHER THING TO KEEP IN MIND WHEN DEALING WITH /,40 WORKLOADS IS THAT THE %XADATA PLATFORM PROVIDES
EXCEPTIONAL SCALABILITY 5PGRADING FROM A HALF RACK TO FULL RACK DOUBLES THE NUMBER OF #05S AT BOTH THE
DATABASE LAYER AND THE STORAGE LAYER 4HE AMOUNT OF %3&# IS ALSO DOUBLED AS IS THE AVAILABLE MEMORY
4HIS ALLOWS %XADATA TO SCALE IN A NEARLY LINEAR FASHION FOR MANY SYSTEMS
+EVIN 3AYS 4HE AUTHORS ARE CORRECT THAT DOUBLING FROM A HALF RACK TO A FULL RACK IS PERFECT HARDWARE SCALE
UP )
URGE CAUTION HOWEVER IN PRESUMING THAT THE SOFTWARESPECIFICALLY 2EAL !PPLICATION #LUSTERS 2!# WILL SCALE
LINEARLY WITH ALL APPLICATIONS 2!# HAS BEEN AROUND FOR OVER TEN YEARS SO LETS NOT THROW AWAY OUR UNDERSTANDING OF
ITS SCALABILITY CHARACTERISTICS WHEN CONSIDERING %XADATA 4HERE IS NOTHING ABOUT %XADATA ARCHITECTURE THAT CHANGES
THE INTRINSIC 2!# SCALABILITY CHARACTERISTICS
7RITE
)NTENSIVE /,40 7ORKLOADS
7RITE
INTENSIVE WORKLOADS ARE A SUBSET OF /,40
ORIENTED SYSTEMS 4HERE ARE SOME SYSTEMS THAT JUST BANG
AWAY AT SINGLE
ROW INSERTS 4HESE SYSTEMS ARE OFTEN LIMITED BY THE SPEED AT WHICH COMMITS CAN BE DONE
WHICH OFTEN DEPENDS ON THE SPEED WITH WHICH WRITES TO THE LOG FILES CAN BE ACCOMPLISHED 4HIS IS ONE
AREA WHERE %XADATA COMPETES WITH OTHER PLATFORMS ON A FAIRLY EVEN PLAYING FIELD 4HERE ARE NO MAJOR
ENHANCEMENTS THAT MAKE %XADATA RUN ORDERS OF MAGNITUDES FASTER FOR SYSTEMS THAT ARE BOTTLENECKED ON
WRITE OPERATIONS 4HIS MEANS THAT THERE IS NO MAGIC BULLET AND TRADITIONAL METHODS OF TUNING SUCH AS
MINIMIZING COMMITS ARE APPROPRIATE FOR THESE TYPES OF SYSTEMS
+EVIN 3AYS 7RITE
INTENSIVE WORKLOADS WARRANT DEEP THOUGHT 4HE AUTHORS POINT OUT THAT THERE ARE NO major
enhancements THAT MAKE %XADATA PERFORM ORDERS OF MAGNITUDE BETTER THAN /RACLE SYSTEMS CONNECTED TO
CONVENTIONAL STORAGE 4HERE ARE IN FACT NO ATTRIBUTES OF %XADATA THAT FAVOR WRITE
INTENSIVE WORKLOADS AS OF THE
PUBLICATION DATE OF THIS BOOK 4HE 6 AND 8 MODELS ARE CAPABLE OF SERVICING OVER MILLION RANDOM SINGLE
BLOCK
READS PER SECONDFROM %XADATA 3MART &LASH #ACHE ON THE FULL RACK CONFIGURATION 2ANDOM WRITES ON THE OTHER
HAND HAVE TO GO TO SPINNING DISKS OF WHICH THERE ARE IN THE FULL RACK CONFIGURATIONS 4HE HIGH
PERFORMANCE
3!3 DRIVES CAN SERVICE ROUGHLY RANDOM WRITE )/03AS LONG AS THE SEEKS ARE SHORT (OWEVER WRITES MUST BE
REDUNDANT AND THUS THE NET RANDOM WRITE BANDWIDTH AVAILABLE TO DATABASES IS ROUGHLY )/03 4O SCALE AN
APPLICATION TO MILLION READ )/03 ON %XADATA THE IMPOSED READWRITE RATIO IS TO 4HAT IS %XADATA IS
FOLD
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
MORE READ CAPABLE THAN WRITE CAPABLE FOR RANDOM )/ !PPLICATIONS THAT LEAN HEAVY ON RANDOM WRITES WILL BE THE
FIRST TO SUFFER THIS INHERENT IMBALANCE 4HIS IMBALANCE IS THE REASON SOME DEPLOYMENTS REQUIRE THE FLASH ASSETS OF
THE CELLS TO BE PROVISIONED AS FLASH GRID DISKS #OMMITTING FLASH CAPACITY IN THE STATIC GRID DISK FASHION HOWEVER
EATS INTO THE VALUE PROPOSITION OF %XADATA %VERY BYTE OF STORAGE COMMITTED TO FLASH GRID DISK IS LESS CAPACITY
AVAILABLE FOR THE DYNAMIC INTELLIGENT FLASH CACHE ALGORITHMS $EPLOYING /RACLE IS OFTEN A BALANCING ACTEVEN IN
THE %XADATA ENVIRONMENT 4HIS BOOK SHOULD EQUIP READERS WITH SUFFICIENT SKILLS TO USE %XADATA ASSETS WISELY
DW-Oriented Workloads
%XADATA WAS DESIGNED TO SPEED UP LONG
RUNNING QUERIES AGAINST LARGE VOLUMES OF DATA 3O IT SHOULD COME
AS NO SURPRISE THAT DATA WAREHOUSING IS WHERE ITS IMPORTANT THAT WE CHANGE SOME OF OUR BASIC THOUGHT
PROCESSES /F COURSE ONE OF THE MAJOR TECHNIQUES IS TO BE CONSTANTLY LOOKING FOR OPPORTUNITIES TO ALLOW
%XADATA OPTIMIZATIONS TO KICK IN 4HIS MEANS MAKING SURE THAT THE APPLICATION CAN TAKE ADVANTAGE OF
3MART 3CANS
%NABLING 3MART 3CANS
4HE MOST IMPORTANT CONCEPT TO KEEP IN MIND WHEN DEALING WITH $7
ORIENTED WORKLOADS IS THAT LONG
RUNNING STATEMENTS SHOULD USUALLY BE OFFLOADED 4HE STEPS TO FOLLOW ARE
$ETERMINE WHETHER 3MART 3CANS ARE BEING USED
)F 3MART 3CANS ARE NOT BEING USED FIX THINGS SO THAT THEY WILL BE USED
4HESE POINTS SEEM SO OBVIOUS THAT WE SHOULDNT NEED TO REPEAT THEM ! LARGE PORTION OF THE
OPTIMIZATIONS BUILT INTO %XADATA WORK ONLY WHEN 3MART 3CANS ARE USED 3O ONE OF THE FIRST CHANGES YOU
NEED TO MAKE IN THE WAY YOU THINK IS TO TRAIN YOURSELF TO BE CONSTANTLY CONSIDERING WHETHER 3MART 3CANS
ARE BEING USED APPROPRIATELY OR NOT 4HIS MEANS YOU NEED TO HAVE A GOOD UNDERSTANDING OF WHICH
STATEMENTS OR PARTS OF STATEMENTS CAN BE OFFLOADED AND BE ABLE TO DETERMINE WHETHER STATEMENTS ARE
BEING OFFLOADED OR NOT 7E HAVE COVERED THE REQUIREMENTS FOR 3MART 3CANS AND SOME OF THE TECHNIQUES
THAT CAN BE USED TO VERIFY WHETHER THEY ARE BEING PERFORMED OR NOT THROUGHOUT THE BOOK "UT AT THE RISK
OF BEING REPETITIVE WELL MENTION THEM AGAIN HERE
4HERE ARE TWO MAIN REQUIREMENTS THAT MUST BE MET IN ORDER FOR 3MART 3CANS TO OCCUR 4HE FIRST IS
THAT THE OPTIMIZER MUST CHOOSE TO DO A FULL SCAN OF A TABLE OR A MATERIALIZED VIEW OR THE OPTIMIZER MUST
CHOOSE TO DO A FAST FULL SCAN OF AN INDEX .OTE THAT 3MART 3CANS ARE NOT LIMITED TO QUERIES OR EVEN TO
SUBQUERIES 4HE OPTIMIZER CAN ALSO CHOOSE TO USE FULL SCANS FOR 34;4C4b ANDD?30C4S WHEN A LARGE
PERCENTAGE OF THE ROWS WILL BE AFFECTED (OWEVER IF YOUR APPLICATION IS DOING THIS YOU MIGHT WANT TO
CONSIDER MODIFYING IT TO DO SOMETHING LIKE A TRUNCATE AND REBUILD
4HE SECOND REQUIREMENT FOR 3MART 3CANS IS THAT THE SCANS MUST BE PERFORMED USING THE DIRECT PATH
READ MECHANISM .OTE THAT WE INTENTIONALLY DIDNT MENTION THE OPTIMIZER IN THE DESCRIPTION OF THE
SECOND REQUIREMENT 4HIS IS BECAUSE THE OPTIMIZER DOES NOT MAKE THE DECISION ABOUT WHETHER TO USE
DIRECT PATH READS OR NOT )T IS A HEURISTIC DECISION THAT IS MADE AFTER THE PLAN HAS BEEN DETERMINED !S
SUCH IT IS NOT DIRECTLY EXPOSED BY ANY OF THE TOOLS LIKE Tg_[PX]_[P] TYPE TOOLS 7HAT THIS MEANS IN
PRACTICE IS THAT ITS EASY TO VERIFY THAT THE FIRST REQUIREMENT HAS BEEN MET BUT MORE CHALLENGING TO VERIFY
THE SECOND REQUIREMENT
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
/N MOST %XADATA IMPLEMENTATIONS A FAIRLY HIGH PERCENTAGE OF LONG RUNNING QUERIES ARE OFFLOADED
9OU CAN CHECK TO SEE WHAT PERCENTAGE OF YOUR LONG RUNNING 31, STATEMENTS HAVE BEEN OFFLOADED BY
SELECTING ALL THE STATEMENTS FROM eb`[ OR FROM THE !72 TABLE 310N78BCNB@;BC0C THAT HAVE AN AVERAGE
RUN TIME OF OVER SOME NUMBER OF SECONDS OR THAT HAVE AN AVERAGE LOGICAL )/ VALUE THAT IS GREATER THAN
SOME REASONABLE VALUE !CTUALLY THE LOGICAL )/ IS A BETTER METRIC TO USE AS SOME OF THE OFFLOADED
STATEMENTS WILL RUN VERY QUICKLY AND MAY NOT MEET YOUR MINIMUM TIME CRITERIA WHICH WILL GIVE YOU A
DISTORTED PERSPECTIVE ON THE PERCENTAGE OF STATEMENTS THAT ARE BEING OFFLOADED (ERES AN EXAMPLE NOTE
THAT THE SCRIPTS ARE INCLUDED IN THE ONLINE CODE REPOSITORY
BHB/B0=31>G -/^UU[^PSN_TaRT]c
4]cTaeP[dTU^ab`[NcTgc)
4]cTaeP[dTU^a\X]NTcX\T)
4]cTaeP[dTU^a\X]NPeVN[X^)$
C>C0;>55;>0343>55;>0343N
" '# %!
BHB/B0=31>G -
4]cTaeP[dTU^ab`[NcTgc)B4;42C
4]cTaeP[dTU^a\X]NTcX\T)
4]cTaeP[dTU^a\X]NPeVN[X^)$
C>C0;>55;>0343>55;>0343N
4HE LISTING MAKES USE OF THE ^UU[^PSN_TaRT]c b`[ SCRIPT WHICH CALCULATES A PERCENTAGE OF
STATEMENTS CURRENTLY IN THE SHARED POOL THAT HAVE BEEN OFFLOADED 7E INITIALLY USED IT TO EVALUATE ALL
STATEMENTS THAT HAD OVER LOGICAL )/S 7E THEN RAN IT A SECOND TIME AND LIMITED OUR INVESTIGATION
TO STATEMENTS THAT BEGIN WITH THE WORD B4;42C )N THE NEXT LISTING WEVE MADE USE OF A SCRIPT Ubg^ b`[
THAT ALLOWS US TO SEE THE ACTUAL STATEMENTS THAT CONTRIBUTE TO THE >55;>03N CALCULATED IN THE PREVIOUS
LISTING
BHB/B0=31>G -/Ubg^
4]cTaeP[dTU^ab`[NcTgc)
4]cTaeP[dTU^ab`[NXS)
4]cTaeP[dTU^a\X]NTcX\T)
4]cTaeP[dTU^a\X]NPeVN[X^)$
4]cTaeP[dTU^a^UU[^PSTS)
B@;N834G42B0E6N4C8<4>55;>038>NB0E43NB@;NC4GC
Qec$i#'c 'Qh HTb bT[TRcR^d]cUa^\bZTf"fWT
YhcUa h YSa (HTb bT[TRcR^d]cUa^\bZTf"fWT
!_Zfc'iYSWQg (HTb bT[TRcR^d]cUa^\bZTf"fWT
!iQc$$$cV !"b!# "&HTb& '$bT[TRc _PaP[[T[P' P
# !]# #hdVWbh (HTb bT[TRcR^d]cUa^\bZTf"fWT
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
$iadR#e%h"!U($$ "=^ 342;0A4Y^Q18=0AHN8=C464A),
%Sg!#&aehZa&! HTb bT[TRcR^d]cUa^\bZTf"fWT
%ddcS\`a&!b\R!"! '"HTb& '$bT[TRc PeVbZTf" b`[ PeV
&h (SchdR#SQW ! '&HTb& '"bT[TRcPeV_ZNR^[Ua^\Zb^ bZ
Q%dbaV'!WfbP"$'" ' =^ RP[[SQ\bNbcPcb VPcWTaNSPcPQPb
Ueg"e f_egefc $HTb(( ((bT[TRcR^d]cUa^\bZTf"fWT
VR`(P$"i&biYc (HTb bT[TRcR^d]cUa^\bZTf"fWT
Vb"$e$c! S(hU ' !HTb(( ((bT[TRcR^d]cUa^\bZTf"fWT
"a^fbbT[TRcTS
4HE Ubg^ b`[ SCRIPT PROVIDES THE SAME LIMITING FACTORS AS THE ^UU[^PSN_TaRT]c b`[ SCRIPT NAMELY A
MINIMUM AVERAGE ELAPSED TIME AND A MINIMUM AVERAGE LOGICAL )/ )T ALSO OPTIONALLY ALLOWS YOU TO LIMIT
THE STATEMENTS TO ONLY THOSE THAT ARE OFFLOADED OR THOSE THAT ARE NOT OFFLOADED 0LEASE REFER TO THE SCRIPTS
FOR FURTHER DETAILS AND KEEP IN MIND THAT THESE TECHNIQUES CAN ALSO BE APPLIED TO THE DATA RECORDED BY
!72 FOR A HISTORICAL PERSPECTIVE
)N THE NEXT SECTION WELL DISCUSS SOME ISSUES THAT CAN COMPLICATE YOUR EFFORTS TO ENABLE 3MART 3CANS
4HINGS THAT #AN #RIPPLE 3MART 3CANS
4HERE ARE SEVERAL COMMON CODING hTECHNIQUESv THAT EITHER DISABLE 3MART 3CANS COMPLETELY OR CAUSE
THEM TO BE MUCH LESS EFFECTIVE THAN THEY COULD BE 3OME OF THE TECHNIQUES ARE JUST BAD PRACTICES
REGARDLESS OF WHETHER YOU ARE ON THE %XADATA PLATFORM OR NOT /THERS DONT CARRY AS SIGNIFICANT A PENALTY
ON NON
%XADATA PLATFORMS BUT WHEN RUN ON %XADATA THEY CAN PREVENT THE STORAGE SOFTWARE FROM DOING
ALL THAT IT COULD DO 7EVE DISCUSSED MANY OF THESE ISSUES THROUGHOUT THIS BOOK (ERE ARE A FEW THAT YOU
SHOULD KEEP IN MIND BECAUSE OF THE FUNDAMENTALLY DIFFERENT BEHAVIOR ON THE %XADATA PLATFORM
Functions in WHERE Clauses
/RACLE PROVIDES A LARGE SET OF FUNCTIONS THAT CAN BE APPLIED DIRECTLY IN 31, STATEMENTS !S WE DISCUSSED
IN #HAPTER NOT ALL OF THOSE FUNCTIONS ARE OFFLOADABLE +NOWING WHICH FUNCTIONS ARE NOT OFFLOADABLE IS
IMPORTANT BECAUSE THE USE OF THOSE FUNCTIONS IN F74A4 CLAUSES DISABLES PREDICATE FILTERING THAT MAY
OTHERWISE PROVIDE A MASSIVE REDUCTION IN THE AMOUNT OF DATA TO BE TRANSFERRED BACK TO THE DATABASE
LAYER /BVIOUSLY CUSTOM WRITTEN 0,31, FUNCTIONS FALL INTO THE CATEGORY OF hNON OFFLOADABLEv FUNCTIONS
AS WELL
4HIS ISSUE IS SOMEWHAT COUNTERINTUITIVE SINCE WE ARE OFTEN DOING FULL TABLE SCANS ANYWAY WITH DATA
WAREHOUSE SYSTEMS /N NON %XADATA PLATFORMS APPLYING A FUNCTION IN A F74A4 CLAUSE OF A STATEMENT THAT
IS EXECUTED VIA A FULL TABLE SCAN DOES NOT IMPOSE MUCH OF A PENALTY WITH REGARD TO THE AMOUNT OF DATA
THAT MUST BE RETURNED BECAUSE THE DATABASE MUST ALREADY RETURN ALL BLOCKS FROM THE TABLE TO THE
DATABASE SERVER 7ITH %XADATA THOUGH APPLYING A FUNCTION THAT CAN DISABLE PREDICATE FILTERING CAN
IMPOSE A HUGE PERFORMANCE PENALTY "Y THE WAY USING CUSTOM 0,31, FUNCTIONS IN A F74A4 CLAUSE IS
GENERALLY ALSO A BAD IDEA ON NON %XADATA PLATFORMS BECAUSE ADDITIONAL #05 WILL BE REQUIRED TO PROCESS
0,31, FOR EACH ROW AS OPPOSED TO THE OPTIMIZED FUNCTIONS BASED ON # CODE PROVIDED BY /RACLE
.OTE 9OU CAN QUERY EB@;5=N<4C030C0 TO SEE WHICH FUNCTIONS ARE OFFLOADABLE
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
&URTHERMORE hOFFLOADABLEv FUNCTIONS CAN ALSO IMPOSE LARGE PERFORMANCE PENALTIES (ERES A VERY
SIMPLE EXAMPLE SHOWING THE NEGATIVE EFFECT OF AN OFFLOADABLE FUNCTION IN A F74A4 CLAUSE
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[!,:B>*
2>D=C
(
4[P_bTS) ) ) "
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTd__TaR^[!,:B>*
2>D=C
(
4[P_bTS) ) ) " %
BHB/B0=31>G -/Ubg#
4]cTaeP[dTU^ab`[NcTgc)bT[TRcR^d]cUa^\Zb^ bZTf"fWTaT:B>
4]cTaeP[dTU^ab`[NXS)
B@;N83278;3>55;>038>NB0E43N0E6N4C8<4B@;NC4GC
d'V]g\\&RQSZ HTb !bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[
"&c&UiZPYi!UR HTb(( (( " %bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTd__
4[P_bTS) ) ) '
3O D??4A IS AN OFFLOADABLE FUNCTION 9ET WHEN WE USED IT IN THE F74A4 CLAUSE OF THIS PARTICULAR
STATEMENT THE RESULT WAS A LARGE DEGRADATION IN PERFORMANCE )F YOUVE BEEN FOLLOWING ALONG ON
SCREEN
YOU MAY ALREADY HAVE A PRETTY GOOD IDEA WHY
BHB/B0=31>G -bT[TRc]P\TeP[dTUa^\e\hbcPcbebcPc]P\T]
!fWTaT] bcPcXbcXR,b bcPcXbcXR
"P]S]P\T[XZTbc^aPVT*
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg
BHB/B0=31>G -bTccX\X]V^]
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTd__TaR^[!,:B>*
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
2>D=C
(
4[P_bTS) ) ) " !(
BHB/B0=31>G -bT[TRc]P\TeP[dTUa^\e\hbcPcbebcPc]P\T]
!fWTaT] bcPcXbcXR,b bcPcXbcXR
"P]S]P\T[XZTbc^aPVT*
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg
4[P_bTS) ) )
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"fWTaTR^[!,:B>*
2>D=C
(
4[P_bTS) ) )
BHB/B0=31>G -bT[TRc]P\TeP[dTUa^\e\hbcPcbebcPc]P\T]
!fWTaT] bcPcXbcXR,b bcPcXbcXR
"P]S]P\T[XZTbc^aPVT*
=0<4E0;D4
RT[[_WhbXRP[8>QhcTbbPeTSQhbc^aPVTX]STg $(('" !##'
4[P_bTS) ) )
3TORAGE INDEXES ARE DISABLED BY FUNCTIONS JUST LIKE REGULAR INDEXES 4HIS IS NOT TOO SURPRISING BUT
AGAIN WHEN WERE ALREADY DOING A FULL SCAN WEVE TRAINED OURSELVES TO NOT WORRY ABOUT FUNCTIONS IN THE
F74A4 CLAUSE THAT COULD DISABLE INDEXES %XADATA IS DIFFERENT
Chained Rows
4HIS IS A VERY BROAD GENERALIZATION BUT BASICALLY ANY /RACLE PROCESSING THAT REQUIRES READING AN EXTRA
BLOCK TO COMPLETE A ROW CAUSES THE %XADATA STORAGE SOFTWARE TO REVERT TO BLOCK SHIPPING OR pass-through
MODE 7E HAVE DISCUSSED THIS IN SEVERAL PLACES IN THE PREVIOUS CHAPTERS ! SIMPLE EXAMPLE IS A CHAINED
ROW BUT THERE ARE OTHER SITUATIONS THAT CAN CAUSE /RACLE TO REVERT TO PASS
THROUGH MODE 7HAT THIS
MEANS IN PRACTICE IS THAT SOME OPERATIONS THAT CAUSE SLIGHT DELAYS ON NON
%XADATA PLATFORMS CAN
SEVERELY IMPACT PERFORMANCE ON %XADATA 4HE PRIMARY DIAGNOSTIC SYMPTOM OF THIS ISSUE IS THE PRESENCE
OF MANY SINGLE
BLOCK
READ WAIT EVENTS IN COMBINATION WITH CELL 3MART 3CAN WAIT EVENTS )N SUCH
SITUATIONS YOU MAY FIND THAT YOU ARE BETTER OFF NOT USING OFFLOADING FOR THE STATEMENTS IN QUESTION (ERE
IS AN EXAMPLE SHOWING WHERE /RACLE SPENDS ITS TIME WHEN SELECTING FROM A TABLE WITH CHAINED ROWS
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
BHB/B0=31>G -bT[TRc]d\Na^fbRWPX]NR]cPeVNa^fN[T]Ua^\SQPNcPQ[Tb
!fWTaTcPQ[TN]P\T,CN2708=43!*
=D<NA>FB2708=N2=C0E6NA>FN;4=
! "&
a^fbT[TRcTS
BHB/B0=31>G-P[cTabhbcT\U[dbWQdUUTaNRPRWT*
BhbcT\P[cTaTS
BHB/B0=31>G-bT[TRcPeV[T]VcWRUa^\cP]T[ cNRWPX]TS!*
0E6;4=6C72
#
a^fbT[TRcTS
4gTRdcX^]?[P]
?[P]WPbWeP[dT) (!& !!'#$
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?DkCX\Tk
k kB4;42CBC0C4<4=Ckk k# k#&$"" k ) ')#"k
k kB>AC066A460C4kk k# kkk
k!kC01;40224BBBC>A0645D;;kCN2708=43!k :k"' <k#&$"" k ) ')#"k
BcPcXbcXRb
aTRdabXeTRP[[b
SQQ[^RZVTcb
""!%''R^]bXbcT]cVTcb
""!%'"_WhbXRP[aTPSb
aTS^bXiT
"# QhcTbbT]ceXPB@;=Tcc^R[XT]c
##'QhcTbaTRTXeTSeXPB@;=TcUa^\R[XT]c
!B@;=Tca^d]ScaX_bc^ Ua^\R[XT]c
b^acb\T\^ah
b^acbSXbZ
a^fb_a^RTbbTS
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
,,,,cZ_a^U>dc_dc,,,,
B@;83)"d_R\P$]b]QRS?[P]7PbW) (!& !!'#$
bT[TRcPeV[T]VcWR
Ua^\
cP]T[ cNRWPX]TS!
RP[[R^d]cR_dT[P_bTSSXbZ`dTahRdaaT]ca^fb
?PabT!
4gTRdcT!
5TcRW#!" "( $%""!%'"%%$"(! !
c^cP['!" "( $%""!%'"%%$"(! !
<XbbTbX][XQaPahRPRWTSdaX]V_PabT)
>_cX\XiTa\^ST)0;;NA>FB
?PabX]VdbTaXS)BHB
=d\QTa^U_[P]bcPcXbcXRbRP_cdaTS)!
A^fb bcA^fbPeVA^fb\PgA^fB^daRT>_TaPcX^]
B>AC066A460C4Ra,""!%(%
C01;40224BBBC>A0645D;;CN2708=43!Ra,""!%(%
4[P_bTScX\TbX]R[dSTfPXcX]V^]U^[[^fX]VTeT]cb)
4eT]cfPXcTS^]CX\Tb<Pg FPXcC^cP[FPXcTS
FPXcTS
B@;=Tc\TbbPVTc^R[XT]c#
B@;=Tc\TbbPVTUa^\R[XT]c# !
RT[[bX]V[TQ[^RZ_WhbXRP[aTPS %%! ' $(
(" !$+,,f^f
aT[XPQ[T\TbbPVT
T]`):> UPbc^QYTRcRWTRZ_^X]c!
RT[[b\PaccPQ[TbRP]!$%
)N THIS EXAMPLE WE RAN A QUERY AGAINST A TABLE IN WHICH EVERY ROW WAS CHAINED ADMITTEDLY A BIT OF
AN EXTREME EXAMPLE 7E GENERATED A TRACE DURING THE EXECUTION OF THE STATEMENT AND THEN RAN
cZa_a^U ON THE TRACE FILE WHICH PRODUCED THE LIST OF WAIT EVENTS JUST SHOWN .OTICE THAT THE STATEMENT
HAD AN ELAPSED TIME OF ALMOST MINUTES SECONDS OF WHICH MORE THAN SECONDS WAS SPENT DOING
SINGLE
BLOCK PHYSICAL READS 4HE STATEMENT USED A FULL TABLE SCAN AND WAS CLEARLY OFFLOADED AS INDICATED
BY THE CELL SMART TABLE SCAN WAIT EVENTS BUT IT SPENT MOST OF ITS TIME DOING SINGLE
BLOCK READS /F COURSE
THE SINGLE
BLOCK READS WERE A RESULT OF THE CHAINED ROWS 4HE NEXT LISTING SHOWS THE DIFFERENCE BETWEEN
RUNNING THE QUERY WITH OFFLOADING AND WITHOUT OFFLOADING
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
,,,,cda]^UUb\PacbRP]b
BHB/B0=31>G -P[cTabTbbX^]bTcNbTaXP[NSXaTRcNaTPS,cadT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,UP[bT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G -P[cTabhbcT\U[dbWQdUUTaNRPRWT*
BhbcT\P[cTaTS
4[P_bTS) ) ) ((
BHB/B0=31>G -bT[TRcPeV[T]VcWRUa^\cP]T[ cNRWPX]TS!*
0E6;4=6C72
#
4[P_bTS) ) ) ' &'+,,RWPX]TSa^fbSXaTRc_PcWaTPSb]^b\PacbRP]
,,,,cda]^UUSXaTRc_PcWaTPSb
BHB/B0=31>G -P[cTabTbbX^]bTcNbTaXP[NSXaTRcNaTPS,UP[bT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G -P[cTabhbcT\U[dbWQdUUTaNRPRWT*
BhbcT\P[cTaTS
4[P_bTS) ) ) #
BHB/B0=31>G -bT[TRcPeV[T]VcWRUa^\cP]T[ cNRWPX]TS!*
0E6;4=6C72
#
4[P_bTS) ) ) ( "$+,,RWPX]TSa^fbbX]V[TQ[^RZaTPSb]^b\PacbRP]
,,,,cda]^]b\PacbRP]b
BHB/B0=31>G -P[cTabTbbX^]bTcNbTaXP[NSXaTRcNaTPS,cadT*
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,cadT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G -P[cTabhbcT\U[dbWQdUUTaNRPRWT*
BhbcT\P[cTaTS
4[P_bTS) ) ) '
BHB/B0=31>G -bT[TRcPeV[T]VcWRUa^\cP]T[ cNRWPX]TS!*
0E6;4=6C72
#
4[P_bTS) ) )$( !#+,,>D27b\PacbRP]Xb]^cV^^SU^a[^cb^URWPX]TSa^fb
BHB/B0=31>G -/Ubg#
4]cTaeP[dTU^ab`[NcTgc)bT[TRcPeV[T]VcWRUa^\cP]T[ cNRWPX]TS!
4]cTaeP[dTU^ab`[NXS)
B@;N83278;3>55;>038>NB0E43N0E6N4C8<4B@;NC4GC
"d_R\P$]b]QRS HTb !& # $( !#bT[TRcPeV[T]VcWRUa^\cP]T[ cNRWPX]
4[P_bTS) ) ) %
.OTICE THAT THE STATEMENT WAS OFFLOADED AND THAT THE 8>NB0E43N CALCULATION PRODUCED A NEGATIVE
NUMBER 4HIS SHOWS THAT MORE BYTES WERE RETURNED BY THE STORAGE CELLS THAN THE TOTAL NUMBER OF BYTES
THAT THE TABLE OCCUPIES )TS CLEAR FROM THIS EXAMPLE THAT CHAINED ROWS CAN BE MUCH MORE DETRIMENTAL TO
PERFORMANCE WHEN ATTEMPTING TO MAKE USE OF %XADATAS 3MART 3CANS THAN TO QUERIES THAT ARE HANDLED IN
NORMAL BLOCK
SHIPPING MODE )TS WORTH NOTING THAT THIS IS AN EXTREME EXAMPLE AS EVERY ROW WAS
CHAINED )N FACT MANY OF THE RECORDS SPANNED THREE BLOCKS )N THIS EXTREME CASE IT IS CLEAR THAT AVOIDING
OFFLOADING ACTUALLY IMPROVES THE PERFORMANCE SIGNIFICANTLY /F COURSE ELIMINATING THE CHAINED ROWS
WOULD BE THE BEST SOLUTION
Very Busy Cells
7HEN A STORAGE CELL BECOMES VERY BUSY WITH #05 OPERATIONS IT IS POSSIBLE FOR THE STORAGE SOFTWARE TO
BEGIN REFUSING TO PERFORM OFFLOAD ACTIVITIES ON SOME PORTION OF THE REQUESTS 4HAT IS IF A PARTICULAR CELL
BECOMES EXTREMELY BUSY AND IS GETTING OFFLOADABLE REQUESTS FROM A DATABASE SERVER THAT IS NOT
PARTICULARLY BUSY IT MAY DECIDE TO SEND SOME DATA BACK TO THE DATABASE SERVER THAT HAS NOT BEEN
PROCESSED OR HAS ONLY PARTIALLY BEEN PROCESSED )N SOME CASES COLUMN PROJECTION MAY BE DONE BUT NOT
FILTERING IN OTHER CASES RT[[bae MAY REVERT TO SHIPPING ENTIRE BLOCKS BACK TO THE DATABASE LAYER 3O WHILE
THIS ISSUE DOES NOT RESULT IN A COMPLETE SHUTDOWN OF OFFLOADING IT CAN REDUCE THE AMOUNT OF WORK THAT IS
DONE ON THE STORAGE TIER
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
4HIS IS AN EXTREMELY COMPLEX TOPIC AND IT IS DIFFICULT TO OBSERVE THE BEHAVIOR DIRECTLY 4HE GOAL OF THE
FEATURE IS TO UTILIZE AVAILABLE #05 RESOURCES REGARDLESS OF WHETHER THEY ARE ON THE DATABASE TIER OR THE
STORAGE TIER 4HIS BEHAVIOR WAS INTRODUCED RELATIVELY RECENTLY IN RT[[bae VERSION WITH
DATABASES RUNNING WITH BUNDLE PATCH OR LATER 4HERE IS A STATISTIC CALLED RT[[_WhbXRP[8>
QhcTb_dbWTSQPRZSdTc^TgRTbbXeT2?D^]RT[[ IN WITH "0 AND LATER THAT SHOWS THIS IS
HAPPENING .OTE THAT THE STATISTIC NAME MAY CHANGE IN A FUTURE VERSION TO SOMETHING REFERRING TO
hBALANCING #05 USAGEv 4HE FEATURE IS DESIGNED TO IMPROVE THROUGHPUT ON VERY BUSY SYSTEMS BUT IT MAY
ALSO CAUSE SOME DEGREE OF INSTABILITY IN THE PERFORMANCE OF CERTAIN STATEMENTS )T IS POSSIBLE TO DISABLE
THIS FEATURE IF YOUR RT[[bae IS ERRONEOUSLY DECIDING IT IS TOO BUSY TO TAKE ON ADDITIONAL WORK BUT IN
GENERAL IF YOU OBSERVE THIS BEHAVIOR YOU ARE PROBABLY GETTING CLOSE TO THE LIMITS OF THE SYSTEM !DDING
ADDITIONAL RESOURCES AT THE STORAGE LAYER MORE STORAGE CELLS MAY BE A VIABLE OPTION
+EVIN 3AYS 4HE AUTHORS ARE CORRECT IN THEIR DESCRIPTION OF THIS NEW hFEATUREv (OWEVER THERE IS ONE POINT )
CANNOT STRESS ENOUGH AND THAT IS THAT THE MOST COMPUTE
INTENSIVE WORK IN QUERY PROCESSING must OCCUR IN THE
DATABASE GRID YET THERE ARE SIGNIFICANTLY FEWER PROCESSORS THERE THAN IN THE STORAGE GRID FOR EXAMPLE IN THE
8
AND IN THE 8
MODELS )N THE END THE ONLY THING THAT MATTERS IS WHETHER THE USERS EXPERIENCE IS
SATISFACTORY (OWEVER DIAGNOSING AN UNSATISFACTORY USER EXPERIENCE MAY PROVE DIFFICULT WHEN THIS FEATURE IS
INVOKED 4HE FEATURE HAS THE PROPENSITY TO PASS AS MUCH AS OF THE PHYSICAL )/ PAYLOAD TO THE CELLS EITHER
COMPLETELY UNPROCESSED OR ONLY LIGHTLY PROCESSED AS IN FOR EXAMPLE %(## PROJECTION 3O THERE ARE SOME
SCENARIOS ) WOULD BE PARTICULARLY CONCERNED ABOUT !LLOW ME TO EXPLAIN
7E ARE NO LONGER LIVING THE hgood lifev OF THE S WHEN 5NIX SYSTEMS WERE PHYSICALLY CONSTRAINED BY CARD SLOTS
FOR MEMORY AND #05 7E HAVE SYSTEMS WITH VAST AMOUNTS OF MEMORY A RAPIDLY INCREASING TREND AND AMPLE #05
7E ARE NO LONGER LIVING THE hgood lifev OF hAMPLE #05v BOTTLENECKED BY FRONT SIDE BUS LIMITATIONS )N SHORT THE
INDUSTRY IS DELIVERING TO US HUGE FAST BALANCED SYSTEMS 4HE SERVERS IN THE %XADATA $ATABASE -ACHINE ARE NO
EXCEPTION 4HE SIGNIFICANT COST OF %XADATA INCLUDES A FEATURE KNOWN AS in-memory parallel query AND OTHER
FEATURES THAT CAN SERVICE USERS NEEDS WITHOUT TOUCHING STORAGE IN THE CELLS )M REFERRING TO FEATURES THAT LEAN
HEAVILY ON HOST TIER RESOURCES AND IN FACT RELY VERY LITTLE ON OFFLOAD PROCESSING )F YOUR %XADATA SYSTEM HOSTS AN
APPLICATION THAT EXHIBITS BURSTS OF DATABASE TIER #05 INTENSIVE PROCESSING INTERMIXED WITH USE CASES THAT DRIVE
CELLS TO PROCESSOR SATURATION ) URGE CLOSE MONITORING OF THIS PUSH BACK AKA PASS THROUGH FEATURE (EURISTICS THAT
GOVERN THE hWHENv AND hHOW MUCHv FOR SUCH A FEATURE ARE VERY DIFFICULT TO GET RIGHT )F THE HEURISTICS ARE TRIGGERING
THIS FEATURE THE PERFORMANCE OF CERTAIN APPLICATIONS MAY SUFFERPROBABLY AT A TIME WHEN YOU REALLY ARENT IN THE
MOOD FOR DEALING WITH UNPREDICTABLE SYSTEM PERFORMANCE (OWEVER AS THE AUTHORS POINT OUT THERE ARE METRICS
AVAILABLE AND THE FEATURE CAN BE TUNEDOR EVEN DISABLED IF NECESSARY
7ITH THIS FEATURE IN THE MIX ) KNOW THE FIRST THING )D CHECK SHOULD USERS COMPLAIN ABOUT RESPONSE TIME BECAUSE
WHEN THIS FEATURE IS TRIGGERED YOU SHOULD EXPECT YOUR DATABASE GRID PROCESSORS TO RATCHET UP TO COMPLETE
SATURATION )F YOUR NORMAL HOST #05 UTILIZATION LEVEL IS NOT WHEN USERS ARE HAPPY THEN PERHAPS A LITTLE
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
DEDUCTIVE REASONING WILL GO A LONG WAY TOWARD FIGURING OUT WHAT HAS GONE WRONG SHOULD THE COMPLAINTS COME IN
(AVING SAID ALL THAT LETS NOT FORGET FOR A MOMENT THAT $7") WORKLOADS must BURN #05 4HAT IS THE NATURE OF THE
BEAST AS IT WERE ,IFE IS A NEVER
ENDING SERIES OF CHOICES
CASE Statements
20B4 STATEMENTS AND THEIR OLDER COUSIN THE 342>34 FUNCTION ARE OFTEN USED TO AVOID REPEATED SCANS OF
THE SAME TABLE 'ENERALLY SPEAKING THIS IS A REASONABLE TECHNIQUE ALTHOUGH ANALYTIC FUNCTIONS NOW
PROVIDE ANOTHER WAY TO DEAL WITH THIS ISSUE .EVERTHELESS IT IS QUITE COMMON TO SEE CODE THAT APPLIES
THIS TRICK 4HE WHOLE IDEA IS TO REPLACE F74A4 CLAUSES WITH 20B4 STATEMENTS 4HIS IS PERFECTLY REASONABLE
WHEN ALL THE BLOCKS MUST BE SHIPPED BACK TO THE DATABASE SERVER TO BE EVALUATED "UT WITH %XADATAS
ABILITY TO DO FILTERING AT THE STORAGE LAYER THIS TECHNIQUE MAY NOT BE THE BEST SOLUTION (ERES A QUICK
EXAMPLE
BHB/B0=31>G -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,UP[bT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G - FXcW^dcB\PacBRP]b
BHB/B0=31>G -bT[TRc Ud[[P R^d]cUa^\Zb^ bZTfPfWTaTR^[ , *
2>D=C
" (((&
4[P_bTS) ) ) #
BHB/B0=31>G -bT[TRc Ud[[P R^d]cUa^\Zb^ bZTfPfWTaTR^[ ,((((((*
2>D=C
"!
4[P_bTS) ) ) $!
BHB/B0=31>G -bT[TRcbd\RPbTfWT]R^[ , cWT] T[bT T]SW^fN\P]hN^]Tb
!bd\RPbTfWT]R^[ ,((((((cWT] T[bT T]SW^fN\P]hN]X]Tb
"Ua^\Zb^ bZTf*
7>FN<0=HN>=4B7>FN<0=HN=8=4B
" (((& "!
4[P_bTS) ) ) ! '"
3O YOU CAN SEE THAT WITHOUT OFFLOADING COMBINING THE STATEMENTS BASICALLY CUTS THE ELAPSED TIME IN
HALF 4HIS IS WHAT WERE USED TO SEEING ON A NON
%XADATA PLATFORM .OW LETS SEE WHAT HAPPENS WHEN WE
ENABLE 3MART 3CANS
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
BHB/B0=31>G -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,cadT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G -P[cTabTbbX^]bTcNbTaXP[NSXaTRcNaTPS,cadT*
BTbbX^]P[cTaTS
4[P_bTS) ) )
BHB/B0=31>G - FXcWB\PacBRP]b
BHB/B0=31>G -bT[TRc Ud[[P R^d]cUa^\Zb^ bZTfPfWTaTR^[ , *
2>D=C
" (((&
4[P_bTS) ) ) %
BHB/B0=31>G -bT[TRc Ud[[P R^d]cUa^\Zb^ bZTfPfWTaTR^[ ,((((((*
2>D=C
"!
4[P_bTS) ) ) '
BHB/B0=31>G -bT[TRcbd\RPbTfWT]R^[ , cWT] T[bT T]SW^fN\P]hN^]Tb
!bd\RPbTfWT]R^[ ,((((((cWT] T[bT T]SW^fN\P]hN]X]Tb
"Ua^\Zb^ bZTf*
7>FN<0=HN>=4B7>FN<0=HN=8=4B
" (((& "!
4[P_bTS) ) ) # ""
/FFLOADING COMPLETELY REVERSES THE RESULTS 4HE TOTAL ELAPSED TIME TO RUN THE TWO INDIVIDUAL
STATEMENTS SECONDS IS ABOUT OF THE TIME REQUIRED TO COMPLETE THE COMBINED STATEMENT
SECONDS .OTICE ALSO THAT THE COMBINED STATEMENT RUNS IN A QUARTER OF THE TIME IT TOOK WITHOUT
OFFLOADING BUT THE INDIVIDUAL STATEMENTS COMPLETED IN ROUGHLY ONE
TENTH THE TIME THAT WAS REQUIRED
WITHOUT OFFLOADING 4HE COMBINED STATEMENT WAS NOT AS EFFECTIVE AS THE INDIVIDUAL STATEMENTS SINCE THE
20B4 STATEMENT CAUSES EACH RECORD IN THE TABLE TO BE RETURNED TO THE DATABASE SERVER FOR EVALUATION 3O
WHILE THE STATEMENT DOES BENEFIT FROM COLUMN PROJECTION IT DOES NOT BENEFIT FROM PREDICATE FILTERING OR
STORAGE INDEXES *UST TO BE CLEAR 20B4 STATEMENTS DONT DISABLE 3MART 3CANS BUT THEY ARE OFTEN USED IN A
WAY THAT REDUCES THEIR EFFECTIVENESS BY ELIMINATING THE OPPORTUNITY TO FILTER ROWS AT THE STORAGE LAYER
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
Hinted Code
Hints ARE VERY USEFUL FOR COERCING THE OPTIMIZER TO DO WHAT YOU WANT IT TO DO 5NFORTUNATELY HINTS ARE NOT
WELL DOCUMENTED AND EVEN LESS WELL UNDERSTOOD )N MANY CASES HINTS ARE USED TO RESOLVE A PROBLEM THAT
IS CAUSED BY SOME MISCONFIGURATION OF THE DATABASE 4HEIR INTENDED PURPOSE IS TO ALLOW HUMANS TO HELP
THE OPTIMIZER MAKE THE RIGHT CHOICES IN SITUATIONS WHERE IT JUST CANT DO THE JOB EFFECTIVELY OR
CONSISTENTLY ON ITS OWN 4HIS HAPPENS IN SITUATIONS WHERE THE OPTIMIZER IS JUST NOT SMART ENOUGH YET TO
ARRIVE AT THE BEST EXECUTION PLAN IN A SPECIFIC SITUATION (OWEVER EVEN WHEN HINTS ARE USED APPROPRIATELY
AND ARE GENERATING THE EXPECTED BEHAVIOR THEY CAN PREVENT %XADATA FROM TAKING ADVANTAGE OF SOME OF ITS
BUILT IN ABILITIES 3O THE BEST APPROACH WHEN MIGRATING TO %XADATA IS TO ALLOW AMPLE TIME FOR TESTING !ND
IF YOUR APPLICATION MAKES USE OF HINTS ONE OF THE IMPORTANT STEPS IN THE TEST PLAN SHOULD BE TO TEST ITS
BEHAVIOR WITHOUT THE HINTS 4HIS CAN EASILY BE ACCOMPLISHED BY SETTING THE HIDDEN PARAMETER
N^_cX\XiTaNXV]^aTNWX]cb TO TRUE
Indexes
4HIS MAY SEEM LIKE A STRANGE TOPIC BUT INDEXES CAN WORK AGAINST 3MART 3CANS AS WELL 4HE OPTIMIZER WILL
TRY TO USE INDEXES IF THEY ARE AVAILABLE )N A TRUE DATA WAREHOUSE ENVIRONMENT INDEXES MAY NOT BE
NECESSARY AT ALL 7ELL HAVE MORE TO SAY ABOUT INDEXES IN THE NEXT SECTION ON MIXED WORKLOADS BUT IT IS
IMPORTANT TO UNDERSTAND THAT INDEX USAGE OFTEN MEANS THAT OFFLOADING IS LESS LIKELY TO OCCUR
Row-at-a-Time Processing
)T IS AMAZING HOW OFTEN WE SEE ROW AT A TIME PROCESSING IN VERY LARGE DATA SETS 4HIS TYPE OF CODING IS
RARELY A GOOD IDEA ON NON
%XADATA PLATFORMS AND IT IS DEFINITELY NOT A GOOD IDEA ON %XADATA )N FACT THE
DIFFERENCES IN PERFORMANCE CAN BE EVEN MORE DRAMATIC ON %XADATA BECAUSE OF THE OPTIMIZATIONS BUILT
INTO THE PLATFORM
/THER 4HINGS TO +EEP IN -IND
4HERE ARE A FEW OTHER THINGS THAT YOU SHOULD KEEP IN MIND WHEN WORKING WITH $7 SYSTEMS THE USE OF
%XADATA 3MART &LASH #ACHE COMPRESSION AND PARTITIONING
Exadata Smart Flash Cache: Keep
%XADATA 3MART &LASH #ACHE %&3# IS THOUGHT OF PRIMARILY AS PROVIDING BENEFIT TO LATENCY SENSITIVE 31,
STATEMENTS )TS IMPORTANT TO REMEMBER THAT IT CAN DRAMATICALLY IMPROVE SCAN PERFORMANCE AS WELL "Y
DEFAULT 3MART 3CANS WILL IGNORE %3&# AND WILL ONLY SCAN DISKS (OWEVER IF AN OBJECT IS DEFINED WITH THE
24;;N5;0B7N20274 ATTRIBUTE SET TO :44? 3MART 3CANS WILL USE BOTH THE DISKS AND THE %3&# FOR SCANNING
/BVIOUSLY OVERRIDING THE DEFAULT CACHING BEHAVIOR FOR A LARGE OBJECT WILL REQUIRE SOME THOUGHT AND
TESTING BUT THIS IS DEFINITELY SOMETHING WORTH CONSIDERING (ERES AN EXAMPLE
BHB/B0=31>G -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,UP[bT*
BTbbX^]P[cTaTS
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
4[P_bTS) ) )$
BHB/B0=31>G -P[cTabTbbX^]bTcRT[[N^UU[^PSN_a^RTbbX]V,cadT*
BTbbX^]P[cTaTS
4[P_bTS) ) ) %
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
4[P_bTS) ) )!% !
4[P_bTS) ) ) #
BHB/B0=31>G -P[cTacPQ[TZb^ bZTf"BC>A06424;;N5;0B7N20274:44?*
CPQ[TP[cTaTS
4[P_bTS) ) ) &
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
4[P_bTS) ) )!' &#
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
4[P_bTS) ) ) ( #%
BHB/B0=31>G -
2>D=C
"'# #'
4[P_bTS) ) ) ' %
BHB/B0=31>G -P[cTacPQ[TZb^ bZTf"BC>A06424;;N5;0B7N202743450D;C*
CPQ[TP[cTaTS
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
4[P_bTS) ) ) !
BHB/B0=31>G -bT[TRcR^d]cUa^\Zb^ bZTf"*
2>D=C
"'# #'
4[P_bTS) ) )!%
)N THIS EXAMPLE WE TURNED OFF 3MART 3CANS AND DID A QUERY WITHOUT A F74A4 CLAUSE 4HE QUERY TOOK
SECONDS 7E THEN ENABLED 3MART 3CANS AND RAN THE QUERY AGAIN 4HIS TIME THE QUERY TOOK ONLY
SECONDS .EXT WE ALTERED THE TABLE TELLING /RACLE THAT WE WANTED %3&# TO BE MORE AGGRESSIVE IN CACHING
THIS TABLE 24;;N5;0B7N20274:44? 7E THEN EXECUTED THE STATEMENT AGAIN THIS DID NOT RESULT IN AN
IMPROVEMENT IN PERFORMANCE BECAUSE THE TABLES BLOCKS WERE NOT STORED IN %3&# AT THAT POINT 4HE
SUBSEQUENT EXECUTIONS THOUGH TOOK ONLY TO SECONDS SHOWING THE BENEFIT OF THE CACHING &INALLY WE
DISABLED THE AGGRESSIVE CACHING AND THE STATEMENT EXECUTION RETURNED TO SECONDS 4HIS SHORT EXAMPLE
SHOWS THAT MAKING USE OF %3&# FOR OFFLOADED QUERIES CAN HAVE A SIGNIFICANT IMPACT ON PERFORMANCE
Compression
%XADATAS (## IS A BIG STEP FORWARD IN ITS ABILITY TO REDUCE THE SIZE OF DATA STORED INSIDE OF /RACLE
DATABASES 4HE COMPRESSION RATIOS THAT ARE ACHIEVABLE WITH (## TURN THE CONCEPT OF INFORMATION
LIFECYCLE MANAGEMENT ON ITS HEAD (## MAKES IT PRACTICAL TO CONSIDER USING COMPRESSION INSTEAD OF
TIERED STORAGE OR ARCHIVING AND PURGING STRATEGIES "ECAUSE PARTITIONS OF A TABLE CAN BE DEFINED TO USE
DIFFERENT COMPRESSION METHODS THE COMBINATION OF PARTITIONING AND COMPRESSION CAN PROVIDE A MUCH
MORE ROBUST SOLUTION FOR hARCHIVINGv DATA THAN ACTUALLY PURGING IT FROM THE DATABASE
9OU SHOULD REMEMBER THOUGH THAT (## IT IS NOT APPROPRIATE FOR DATA THAT IS BEING ACTIVELY UPDATED
! BETTER APPROACH IS TO PARTITION DATA SUCH THAT (## CAN BE APPLIED TO DATA THAT IS NO LONGER BEING
CHANGED 4HIS LEADS US TO THE NEXT TOPIC PARTITIONING
Partitioning
0ARTITIONING IS STILL A VERY KEY COMPONENT FOR DATA WAREHOUSING SYSTEMS 4HE OPTIMIZATIONS PROVIDED BY
%XADATA DO NOT ALLEVIATE THE NEED FOR A WELL
THOUGHT
OUT PARTITIONING STRATEGY /F COURSE DATE
BASED
STRATEGIES ARE VERY USEFUL FROM A MANAGEMENT STANDPOINT "EING ABLE TO USE MORE AGGRESSIVE
COMPRESSION ON OLDER DATA IS OFTEN A GOOD APPROACH "UT PARTITION ELIMINATION IS STILL A TECHNIQUE THAT
YOU WILL WANT TO USE !ND OF COURSE STORAGE INDEXES CAN WORK WELL WITH PARTITIONING PROVIDING BEHAVIOR
COMPARABLE TO PARTITION ELIMINATION ON ADDITIONAL COLUMNS
9OU SHOULD KEEP IN MIND THAT THE SIZES OF PARTITIONS CAN AFFECT /RACLES DECISION TO USE 3MART 3CANS
7HEN PERFORMING A SERIAL SCAN ON A PARTITIONED OBJECT THE DECISION TO DO DIRECT PATH READS IS BASED ON
THE INDIVIDUAL PARTITION SIZE NOT THE OVERALL SIZE OF THE OBJECT 4HIS CAN RESULT IN SITUATIONS WHERE SCANS OF
SOME PARTITIONS ARE OFFLOADED WHILE SCANS OF OTHERS ARE NOT
Mixed Workloads
4HERE IS A THIRD TYPE OF SYSTEM THAT IS A COMBINATION OF THE OTHER TWO )N FACT WE COULD ARGUE THAT THE
OTHER TWO /,40 AND $7 RARELY EXIST IN THE REAL WORLD 4HERE ARE MANY SYSTEMS THAT DONT FALL NEATLY
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
INTO THE TWO MAIN CATEGORIES WEVE ALREADY DESCRIBED )N FACT MOST SYSTEMS DISPLAY CHARACTERISTICS OF
BOTH
#OMBINING LONG
RUNNING THROUGHPUT
SENSITIVE QUERIES WITH FAST LATENCY
SENSITIVE STATEMENTS
DEFINITELY INTRODUCES SOME ADDITIONAL ISSUES THAT MUST BE DEALT WITH /NE OF THE MAIN ISSUES IN SYSTEMS
OF THIS TYPE IS HOW TO DEAL WITH INDEXES
4O )NDEX OR .OT TO )NDEX
/NE OF THE BIGGEST DEBATES WEVE HAD DURING %XADATA IMPLEMENTATIONS IS WHETHER TO DROP INDEXES OR NOT
!CCESS PATHS THAT USE INDEXES ARE GENERALLY NOT ABLE TO TAKE ADVANTAGE OF %XADATA
SPECIFIC OPTIMIZATIONS
7E SAY GENERALLY BECAUSE OFFLOADING CAN OCCUR IN CASES WHERE THE OPTIMIZER CHOOSES TO EXECUTE A &AST
&ULL 3CAN ON AN INDEX BUT THIS IS NOT THE MOST COMMON USAGE PATTERN FOR INDEXES 4HE MORE COMMON
PATTERN IS TO USE THEM FOR RETRIEVING RELATIVELY FEW RECORDS FROM A TABLE USING AN )NDEX 2ANGE 3CAN AND
THIS OPERATION IS NOT CURRENTLY OFFLOADABLE 'ENERALLY SPEAKING WE WANT TO USE INDEX RANGE SCANS ON
SELECTIVE PREDICATES BUT SINCE %XADATA IS SO EFFECTIVE AT SCANNING DISK IN MANY CASES THE INDEX
BASED
ACCESS PATHS ARE NO LONGER FASTER THAN THE SCAN
BASED ACCESS OPERATIONS )TS REALLY A CASE OF GETTING OUR
BEARINGS ALL OVER AGAIN WITH RESPECT TO WHEN WE WANT TO USE INDEXES AND WHEN WE WOULD EXPECT A FULL
SCAN TO PERFORM BETTER
/NE OF THE THINGS WE COMMONLY HEARD WHEN %XADATA WAS FIRST STARTING TO APPEAR AT CUSTOMER SITES
WAS THAT INDEXES WERE NO LONGER NECESSARY AND THAT THEY SHOULD BE DROPPED &OR PURE DATA WAREHOUSE
WORKLOADS THIS MAY ACTUALLY BE PRETTY GOOD ADVICE (OWEVER WE RARELY SEE ANYTHING PURE -OST SYSTEMS
HAVE A MIX OF ACCESS PATTERNS WITH ONE SET OF STATEMENTS HOPING FOR LOW LATENCY AND ANOTHER SET HOPING
FOR HIGH THROUGHPUT )N THESE CASES DROPPING ALL INDEXES JUST WILL NOT WORK 4HIS IS WHY WE SAVED THIS
DISCUSSION FOR THIS SECTION 4HE PROBLEM WITH MIXED WORKLOADS WHERE IT IS NECESSARY TO KEEP INDEXES FOR
SPECIFIC SETS OF STATEMENTS IS THAT THE OPTIMIZER IS NOT AS WELL EQUIPPED TO CHOOSE BETWEEN USING AND
IGNORING THEM AS ONE MIGHT HOPE 7ELL DISCUSS THE OPTIMIZERS LIMITATIONS NEXT
+EVIN 3AYS ) KNOW THE ORIGIN OF THE MISTAKEN NOTION THAT INDEXES ARE NOT NECESSARY WITH %XADATA 9OU SEE
/RACLE $ATABASE ENJOYED MAJORITY MARKET SHARE OF THE WORLD
WIDE DATA WAREHOUSING MARKET AS PER INDUSTRY
ANALYSTS before %XADATA CAME TO MARKET 4HE METHODOLOGY REQUIRED TO COERCE /RACLE $ATABASE INTO DELIVERING
ADEQUATE PERFORMANCE IN THE PRE
%XADATA ERA REQUIRED AN IMMENSE AMOUNT OF hINDEX TRICKERYv /RACLES
COMPETITION MOST NOTABLY .ETEZZA ;NOW )"-= AND 'REENPLUM ;NOW %-#= FOCUSED THEIR MARKETING MESSAGE ON THIS
h!CHILLES HEELv (OWEVER THIS OVER
INDEXING PHENOMENON WITH ALL THE ASSOCIATED COMPLEXITIES WAS NEVER AN
/RACLE DATABASE WEAKNESS PER SE "EFORE THE ORIGINAL RELEASE OF %XADATA IN CUSTOMERS DID NOT HAVE THE
RICHNESS OF SYSTEMS TECHNOLOGY AVAILABLE FOR CONFIGURING A HIGH
BANDWIDTH BALANCED $7") SYSTEM )MBALANCE
AND LOW BANDWIDTH DROVE THE NEED TO AVOID DOING PHYSICAL )/ 4HE ONLY WAY TO AVOID PHYSICAL )/ IN THAT ERA WAS TO
FALL INTO THE hINDEX DEATH
SPIRAL v AS IT BECAME KNOWN IN CERTAIN CLIQUES WITHIN /RACLE 7HILE %XADATA CERTAINLY
ADDRESSES THE )/ BOTTLENECKS THERE REMAINS ONE ALL
ENCOMPASSING TRUTH )T IS ALWAYS FASTER not TO DO SOMETHING
THAN TO DO SOMETHING FAST 4O THAT END %XADATA ADMINISTRATORS NEED TO REMEMBER THAT SOME OF THE TOOLS THEIR
COMPANY HAS PAID FOR INCLUDE ON
DISK INDEXING TECHNOLOGY !LWAYS USE THE RIGHT TOOLS FOR THE JOB AT HAND
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
4O CLOSE THIS TRAIN OF THOUGHT )D LIKE TO PUT SOMETHING INTO PERSPECTIVE 4HE SOFTWARE THAT PUT /RACLE IN THE MARKET
LEADERSHIP POSITION FOR $7") PRIOR TO %XADATA WAS /RACLE $ATABASE G 2ELEASE )F YOU WERE TO DEPLOY /RACLE
$ATABASE G 2ELEASE ON THE SAME NUMBER OF PROCESSOR CORES YOU USED IN THE PRE
%XADATA TIMEFRAME CIRCA
YOU COULD EASILY ENJOY AS MUCH AS A TENFOLD PERFORMANCE INCREASE FOR PROCESS
INTENSIVE USE CASES !ND
SINCE THE METHODOLOGY FOR DEPLOYING /RACLE FOR $7") USE CASES IN THAT TIMEFRAME USED EVERY TRICK IN THE BOOK TO
AVOID PHYSICAL )/ IT IS FAIR TO PRESUME THAT THOSE SYSTEMS WERE GENERALLY #05
BOUND )NDEED SYSTEMS THAT ARE
NEITHER #05
INTENSIVE NOR )/
INTENSIVE ARE PROBABLY NOT CONSIDERED CRITICAL
)N OUR DAY
TO
DAY WORK AS )4 PROFESSIONALS IT IS EASY TO MISS JUST HOW RAPIDLY TECHNOLOGY IS ADVANCING ALL AROUND
USPARTICULARLY WHEN WE ARE NATURALLY CONSERVATIVE ABOUT MIGRATING PRODUCTION APPLICATIONS TO STATE
OF
THE
ART
TECHNOLOGY AND THE STATE OF THE ARTWITH OR WITHOUT %XADATAHAS BEEN ADVANCING AT A PHENOMENAL PACE
4HE /PTIMIZER $OESNT +NOW
7EVE MADE THE POINT SEVERAL TIMES THAT THE OPTIMIZER IS NOT AWARE THAT IT IS RUNNING ON %XADATA YET )N
GENERAL THE PRINCIPLES THAT GUIDE THE OPTIMIZER DECISIONS ARE SOUND REGARDLESS OF THE STORAGE PLATFORM
4HE FACT THAT THE CODE ON THE DATABASE TIER IS IDENTICAL REGARDLESS OF WHETHER ITS RUNNING ON %XADATA OR
NOT MEANS THAT AN APPLICATION WILL BEHAVE SIMILARLY ON %XADATA IN TERMS OF PLAN SELECTION 3O YOU
SHOULDNT EXPECT ANY APPLICATION TO EXPERIENCE A LARGE NUMBER OF CHANGES IN THE PLANS CAUSED SIMPLY BY
MOVING TO %XADATA 4HIS IS A GOOD THING FROM A STABILITY STANDPOINT
4HE DOWNSIDE IS THAT THE OPTIMIZER IS NOT AWARE THAT %XADATA HAS OPTIMIZATIONS THAT CAN CAUSE FULL
SCANS TO PERFORM MUCH BETTER THAN ON OTHER PLATFORMS 3O MIXED
WORKLOAD SYSTEMS THAT HAVE MANY
INDEXES MAKE THE OPTIMIZERS JOB MORE CHALLENGING )N FACT AS YOU MIGHT EXPECT THE OPTIMIZER WILL TEND
TO PICK INDEX
ORIENTED PLANS IN PREFERENCE TO SCAN
BASED PLANS IN SITUATIONS WHERE INDEXES ARE AVAILABLE
DESPITE THE FACT THAT THE SCAN
BASED PLANS ARE OFTEN MUCH FASTER
4HERE ARE SEVERAL WAYS TO DEAL WITH THE OPTIMIZERS TENDENCY TO PREFER INDEX ACCESS OVER FULL TABLE
SCANS 3YSTEM STATISTICS OPTIMIZER PARAMETERS AND HINTS ALL COME TO MIND AS POTENTIAL SOLUTIONS
+EVIN 3AYS !LTHOUGH THE AUTHORS ARE CORRECT TO POINT OUT THE CURRENT LACK OF %XADATA AWARE OPTIMIZER CODE )
DO EXPECT THAT TO CHANGE 4HERE SHOULD CERTAINLY BE %XADATA AWARE OPTIMIZER IMPROVEMENTS SHOWING UP OVER TIME
System Statistics
3YSTEM STATISTICS PROVIDE THE OPTIMIZER WITH ADDITIONAL INFORMATION ABOUT THE hSYSTEM v INCLUDING HOW
LONG IT TAKES TO DO A SINGLE
BLOCK READ TYPICAL OF INDEX LOOKUPS AND HOW LONG IT TAKES TO DO A MULTI
BLOCK
READ TYPICAL OF FULL TABLE SCANS 4HIS MAY APPEAR TO BE AN IDEAL MECHANISM TO MANIPULATE THE OPTIMIZER
BY GIVING IT THE ADDITIONAL INFORMATION IT NEEDS TO MAKE THE RIGHT DECISIONS 5NFORTUNATELY 3MART 3CANS
ARE NOT BASED ON MULTI
BLOCK READS AND IN FACT 3MART 3CANS CAN BE ORDERS OF MAGNITUDE FASTER THAN
MULTI
BLOCK READS 3O MODIFYING 3YSTEM 3TATISTICS IS PROBABLY NOT THE BEST OPTION IN THIS CASE
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
.OTE 4HE ISSUE OF WHETHER TO GATHER SYSTEM STATISTICS ON %XADATA COMES UP FAIRLY OFTEN 3YSTEM STATISTICS ARE
DESIGNED TO PROVIDE ADDITIONAL INFORMATION TO THE OPTIMIZER ABOUT THE BEHAVIOR CHARACTERISTICS OF THE HARDWARE )N
PARTICULAR THEY PROVIDE VALUES FOR AVERAGE SINGLE
BLOCK AND AVERAGE MULTI
BLOCK READ TIMES 3INCE %XADATA OFTEN
AVOIDS ACTUALLY PERFORMING MULTI
BLOCK READS USING 3MART 3CANS INSTEAD IT SEEMS COUNTERINTUITIVE TO PROVIDE
INFORMATION TO THE OPTIMIZER ABOUT AN OPERATION THAT WILL SELDOM BE USED PARTICULARLY WHEN THIS IS ONE OF THE MAIN
INPUTS INTO THE OPTIMIZERS COST CALCULATIONS )F THAT ARGUMENT IS NOT ENOUGH FOR YOU THEN PLEASE REVIEW -/3 .OTE
WHICH SPECIFICALLY STATES
=^cT)8cXbRdaaT]c[h]^c]TRTbbPahc^VPcWTabhbcT\bcPcXbcXRb^]cWTSPcPQPbT\PRWX]T
Optimizer Parameters
4HERE ARE A COUPLE OF INITIALIZATION PARAMETERS THAT CAN PUSH THE OPTIMIZER TOWARD OR AWAY FROM INDEX
USAGE 4HE PARAMETERS >?C8<I4AN8=34GN20278=6 AND >?C8<8I4AN8=34GN2>BCN039 CAN BOTH BE USED FOR THIS
PURPOSE 7HILE THESE ARE BIG KNOBS THAT CAN AFFECT THE CORE FUNCTIONALITY OF THE OPTIMIZER THEY WERE
DESIGNED FOR THE VERY PURPOSE OF MAKING INDEXES MORE OR LESS ATTRACTIVE TO THE OPTIMIZER 5SING THE
PARAMETERS IN A LIMITED WAY SUCH AS WITH AN P[cTabTbbX^] COMMAND BEFORE RUNNING LARGE BATCH
PROCESSES IS A VIABLE APPROACH IN SOME CASES 4HESE PARAMETERS CAN ALSO BE SET AT THE STATEMENT LEVEL
USING THE >?CN?0A0< HINT (ERES A VERY SIMPLE EXAMPLE
BHB/B0=31>G-/Q[^^\NY^X]! b`[
2>;!BD<0 2>;
PbSSbPSPbS $"$('# %
!"#! ##
4[P_bTS) ) ")! "(
BHB/B0=31>G-P[cTabTbbX^]bTc^_cX\XiTaNX]STgNR^bcNPSY, *
BTbbX^]P[cTaTS
4[P_bTS) ) ) &
BHB/B0=31>G-/Q[^^\NY^X]! b`[
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
2>;!BD<0 2>;
PbSSbPSPbS $"$('# %
!"#! ##
4[P_bTS) ) ) ' %#
BHB/B0=31>G -/S_[P]
4]cTaeP[dTU^ab`[NXS) (\%c$`_VZhfg
4]cTaeP[dTU^aRWX[SN]^)
?;0=NC01;4N>DC?DC
B@;N83 (\%c$`_VZhfgRWX[S]d\QTa
bT[TRc Q[^^\Y^X]!dbTNWPbWbZTfcT\_NbZTf P R^[!
bd\P R^[ Ua^\Zb^ bZTf"PZb^ bZTf!QfWTaTP _ZNR^[,Q _ZNR^[P]S
Q R^[ , Va^d_QhP R^[!
?[P]WPbWeP[dT)#%%(#& "&
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkkk"&!"( k
k k?G2>>A38=0C>Akkkkk
k!k?GB4=3@2A0=3><k)C@ !k!k%%k"&!"( k
k"k70B76A>D?1Hkk!k%%k"&!"( k
k#k?GA4248E4kk!k%%k"&!"( k
k$k?GB4=370B7k)C@ k!k%%k"&!"( k
k%k70B76A>D?1Hkk!k%%k"&!"( k
k&k70B79>8=kk & %k$%!('k"&!"' k
k'k1D554AB>ACkkkkk
k(k?GA4248E4kk #!k $%!k " k
k k?GB4=31A>0320BCk)C@ k #!k $%!k " k
k kC01;40224BB1H8=34GA>F83kB:4F!k #!k $%!k " k
k !k8=34GA0=64B20=kB:4F!N2>; k #!kk" k
k "k?G1;>2:8C4A0C>Akk"'#<k' $%<k"& ! k
k #kC01;40224BBBC>A0645D;;kB:4F"k"'#<k' $%<k"& ! k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
& PRRTbb0 ?:N2>;,1 ?:N2>;
! PRRTbb1 2>; ,
# bc^aPVT)I-,)I0=3)I+,)I0=3BHBN>?N1;>><N58;C4A)15 0 ?:N2>;
UX[cTaBHBN>?N1;>><N58;C4A)15 0 ?:N2>;
B@;N83 (\%c$`_VZhfgRWX[S]d\QTa
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
bT[TRc Q[^^\Y^X]!dbTNWPbWbZTfcT\_NbZTf P R^[!
bd\P R^[ Ua^\Zb^ bZTf"PZb^ bZTf!QfWTaTP _ZNR^[,Q _ZNR^[P]S
Q R^[ , Va^d_QhP R^[!
?[P]WPbWeP[dT)!%!'"(! (!
k8Sk>_TaPcX^]k=P\TkA^fbk1hcTbk2^bc2?Dk
k kB4;42CBC0C4<4=Ckkkk#(#%$ k
k k?G2>>A38=0C>Akkkkk
k!k?GB4=3@2A0=3><k)C@ !k!k%%k#(#%$ k
k"k70B76A>D?1Hkk!k%%k#(#%$ k
k#k?GA4248E4kk!k%%k#(#%$ k
k$k?GB4=370B7k)C@ k!k%%k#(#%$ k
k%k70B76A>D?1Hkk!k%%k#(#%$ k
k&k70B79>8=kk & %k$%!('k#(#%# k
k'k?GA4248E4kk #!k $%!k !"$& k
k(k?GB4=31A>0320BCk)C@ k #!k $%!k !"$& k
k k?G1;>2:8C4A0C>Akk #!k $%!k !"$& k
k kC01;40224BBBC>A0645D;;kB:4F!k #!k $%!k !"$& k
k !k?G1;>2:8C4A0C>Akk"'#<k' $%<k"& ! k
k "kC01;40224BBBC>A0645D;;kB:4F"k"'#<k' $%<k"& ! k
?aTSXRPcT8]U^a\PcX^]XST]cXUXTSQh^_TaPcX^]XS)
& PRRTbb0 ?:N2>;,1 ?:N2>;
bc^aPVT)I-,)I0=3)I+,)I0=31 2>; ,
UX[cTa1 2>; ,
" bc^aPVT)I-,)I0=3)I+,)I0=3BHBN>?N1;>><N58;C4A)15 0 ?:N2>;
UX[cTaBHBN>?N1;>><N58;C4A)15 0 ?:N2>;
3O IN THIS SIMPLE EXAMPLE PUSHING THE OPTIMIZER AWAY FROM INDEXES WITH THE P[cTabTbbX^] CAUSED
THE OPTIMIZER TO PICK A PLAN THAT WAS CONSIDERABLY FASTER 4HE PLANS SHOW THAT THE IMPROVEMENT IN
ELAPSED TIME WAS A RESULT OF DOING A FULL TABLE SCAN INSTEAD OF USING THE INDEX
Hints
/F COURSE HINTS CAN ALSO BE USED TO HELP THE OPTIMIZER MAKE THE RIGHT CHOICES BUT THAT IS SOMEWHAT OF A
SLIPPERY SLOPE .EVERTHELESS TELLING /RACLE THAT YOU WOULD PREFER TO DO A HASH JOIN OR IGNORE A SPECIFIC
INDEX IS AN OPTION !S MENTIONED IN THE PREVIOUS SECTION THE >?CN?0A0< HINT CAN ALSO PROVE USEFUL FOR
SETTING SOME INITIALIZATION PARAMETERS THAT CAN INFLUENCE THE OPTIMIZERS DECISIONS
5SING 2ESOURCE -ANAGER
)TS A COMMONLY HELD BELIEF THAT /RACLE DATABASES CANT BE CONFIGURED TO ADEQUATELY HANDLE BOTH $7 AND
/,40 WORKLOADS AT THE SAME TIME !ND IN TRUTH KEEPING THEM ON SEPARATE SYSTEMS DOES MAKE THEM
EASIER TO MANAGE 4HE DOWNSIDE OF THIS APPROACH IS THAT IT IS EXPENSIVE -ANY COMPANIES DEDICATE THE
#(!04%2 5.,%!2.).' 3/-% 4().'3 7% 4(/5'(4 7% +.%7
MAJORITY OF THEIR COMPUTING RESOURCES TO MOVING DATA BETWEEN PLATFORMS 4HE POWER OF %XADATA MAKES IT
TEMPTING TO COMBINE THESE ENVIRONMENTS +EEP IN MIND THAT %XADATA HAS ADDITIONAL CAPABILITIES FOR
DIVIDING RESOURCES BETWEEN MULTIPLE DATABASES THAT ARE NOT AVAILABLE ON OTHER PLATFORMS )/2- CAN
PREVENT LONG
RUNNING $7 QUERIES FROM CRIPPLING LATENCY
SENSITIVE STATEMENTS THAT ARE RUNNING ON THE
SAME SYSTEM (AVING A GOOD UNDERSTANDING OF /RACLES RESOURCE MANAGEMENT CAPABILITIES SHOULD
CHANGE THE WAY YOU THINK ABOUT WHAT IS POSSIBLE IN A MIXED
WORKLOAD OR CONSOLIDATED ENVIRONMENT
2ESOURCE MANAGEMENT IS COVERED IN DEPTH IN #HAPTER
Summary
%XADATA IS DIFFERENT 4O MAKE THE BEST USE OF IT YOULL NEED TO THINK DIFFERENTLY
APPENDIX A
CellCLI and dcli
#ELL#,) IS A COMMAND INTERPRETER THROUGH WHICH YOU CAN MANAGE A STORAGE CELL )T IS TO A CELL WHAT
31,
0LUS IS TO A DATABASE INSTANCE SR[X IS A UTILITY BY WHICH YOU CAN SEND A SINGLE COMMAND TO ALL YOUR
DATABASE SERVERS ANDOR STORAGE CELLS IN ONE GO 7E DESCRIBE BOTH UTILITIES BRIEFLY IN THIS APPENDIX
CellCLI Command Syntax
%XADATA STORAGE SOFTWARE USES THE #ELL#,) UTILITY AS ITS COMMAND LINE INTERFACE 5NFORTUNATELY ALTHOUGH
THE DOCUMENTATION SET THAT COMES WITH %XADATA DOES HAVE MANY EXAMPLES OF #ELL#,) COMMANDS AND
EVEN A CHAPTER DEDICATED TO #ELL#,) IT DOES NOT INCLUDE ANY REFERENCE MATERIAL ON THE SYNTAX ITSELF
PARTICULARLY THE ;8BC COMMAND 3O WE THOUGHT WE WOULD INCLUDE A FEW OF THE THINGS WE LEARNED WHILE
WORKING WITH IT
)TS INTERESTING THAT /RACLE CHOSE TO WRITE AN ENTIRELY NEW COMMAND LINE TOOL FOR MANAGING THE
STORAGE CELL 4HEY COULD HAVE USED 31,
0LUS WHICH HAS BECOME THE MOST WELL KNOWN TOOL FOR MANAGING
DATABASES AND !3- "E THAT AS IT MAY #ELL#,) IS THE TOOL YOU WILL USE FOR MANAGING THE STORAGE CELLS 4HE
SYNTAX IS SOMEWHAT DIFFERENT FROM 31,
0LUS BUT THERE ARE SIMILARITIES PARTICULARLY WITH THE ;8BC
COMMAND ;8BC IS USED TO EXECUTE QUERIES AND IT LOOKS VERY SIMILAR TO THE B4;42C COMMAND THAT $"!S
HAVE BECOME ACCUSTOMED TO ,IKE B4;42C IT HAS F74A4 AND ;8:4 KEYWORDS THAT ALLOW YOU TO FILTER OUT
UNWANTED INFORMATION FROM THE OUTPUT
&OLLOWING IS OUR TOP TEN LIST OF THINGS YOU SHOULD KNOW ABOUT #ELL#,)
#ELL#,) DOES IMPLEMENT A HANDFUL OF 31,
0LUS COMMANDS BC0AC/ B4C
427>>= B?>>; 34B2A814 AND 74;?
B4;42C IS REPLACED BY ;8BC AND IT MUST BE THE FIRST KEYWORD ON THE COMMAND
LINE
4HERE IS NO 5A>< KEYWORD THE ;8BC KEYWORD MUST BE IMMEDIATELY FOLLOWED
BY THE >QYTRcCh_T WHICH IS EQUIVALENT TO A TABLE NAME
4HERE IS A 34B2A814 COMMAND WHICH DISPLAYS THE ATTRIBUTES COLUMNS THAT
MAKE UP AN >QYTRcCh_T TABLE
#OLUMN NAMES ARE SPECIFIED WITH THE 0CCA81DC4B KEYWORD FOLLOWED BY THE
COLUMNS YOU WISH TO BE DISPLAYED
4HERE IS A DEFAULT SET OF COLUMNS FOR EACH ObjectType THAT WILL BE RETURNED IF
THE 0CCA81DC4B KEYWORD IS NOT SPECIFIED
4HERE IS A F74A4 CLAUSE THAT CAN BE APPLIED TO ANY ATTRIBUTE AND MULTIPLE
CONDITIONS CAN BE 0=3ED TOGETHER HOWEVER THERE IS NO SUPPORT FOR >A
!00%.$)8 ! #%,,#,) !.$ $#,)
4HERE IS NO >A34A1H EQUIVALENT
4HE 34C08; KEYWORD CAN BE APPENDED TO ANY ,)34 COMMAND TO CHANGE THE
OUTPUT FROM COLUMN ORIENTED TO ROW ORIENTED
4HE ;8:4 OPERATOR WORKS BUT INSTEAD OF THE STANDARD 31, WILDCARD #ELL#,)
USES REGEX SO BECOMES THE
'ETTING &AMILIAR WITH #ELL#,)
! GOOD WAY TO BEGIN TO GET FAMILIAR WITH #ELL#,) IS TO EXPLORE ITS HELP INTERFACE &OLLOWING IS AN EXAMPLE
OF INVOKING ONLINE HELP
2T[[2;8-WT[_
74;?Jc^_XRL
0ePX[PQ[TC^_XRb)
0;C4A
0;C4A0;4AC78BC>AH
0;C4A24;;
0;C4A24;;38B:
0;C4A6A8338B:
0;C4A8>A<?;0=
0;C4A;D=
0;C4AC7A4B7>;3
0BB86=:4H
20;81A0C4
2A40C4
2A40C424;;
2A40C424;;38B:
2A40C45;0B720274
2A40C46A8338B:
2A40C4:4H
2A40C4C7A4B7>;3
34B2A814
3A>?
3A>?0;4AC78BC>AH
3A>?24;;
3A>?24;;38B:
3A>?5;0B720274
3A>?6A8338B:
3A>?C7A4B7>;3
4G?>AC24;;38B:
8<?>AC24;;38B:
;8BC
;8BC02C8E4A4@D4BC
;8BC0;4AC3458=8C8>=
;8BC0;4AC78BC>AH
;8BC24;;
;8BC24;;38B:
;8BC5;0B720274
!00%.$)8 ! #%,,#,) !.$ $#,)
;8BC5;0B7202742>=C4=C
;8BC6A8338B:
;8BC8>A<?;0=
;8BC:4H
;8BC;D=
;8BC<4CA822DAA4=C
;8BC<4CA823458=8C8>=
;8BC<4CA8278BC>AH
;8BC?7HB820;38B:
;8BCC7A4B7>;3
B4C
B?>>;
BC0AC
2T[[2;8-WT[_[Xbc
4]cTa74;?;8BC+^QYTRcNch_T-U^ab_TRXUXRWT[_bh]cPg
+^QYTRcNch_T-)j02C8E4A4@D4BCk0;4AC78BC>AHk0;4AC3458=8C8>=k24;;
k24;;38B:k5;0B720274k5;0B7202742>=C4=Ck6A8338B:
k8>A<?;0=k:4Hk;D=
k<4CA822DAA4=Ck<4CA823458=8C8>=k<4CA8278BC>AH
k?7HB820;38B:kC7A4B7>;3l
2T[[2;8-WT[_[Xbc5;0B7202742>=C4=C
DbPVT);8BC5;0B7202742>=C4=CJ+UX[cTab-LJ+PccaXQdcTN[Xbc-LJ34C08;L
?da_^bT)3Xb_[Phbb_TRXUXTSPccaXQdcTbU^aU[PbWRPRWTT]caXTb
0aVd\T]cb)
+UX[cTab-)0]Tg_aTbbX^]fWXRWSTcTa\X]TbcWTT]caXTbc^QTSXb_[PhTS
+PccaXQdcTN[Xbc-)CWTPccaXQdcTbcWPcPaTc^QTSXb_[PhTS
0CCA81DC4Bj0;;kPcca JPcca!L l
>_cX^]b)
J34C08;L)5^a\PcbcWTSXb_[PhPbP]PccaXQdcT^]TPRW[X]TfXcW
P]PccaXQdcTSTbRaX_c^a_aTRTSX]VTPRWeP[dT
4gP\_[Tb)
;8BC5;0B7202742>=C4=C34C08;
!S YOU CAN SEE THE HELP SYSTEM ALLOWS YOU TO SEE A BIT OF THE SYNTAX FOR EACH COMMAND 9OU MAY
ALSO HAVE NOTICED A COUPLE OF 31,
0LUS CARRY
OVERS B4C B?>>; AND BC0AC WORK PRETTY MUCH AS EXPECTED
.OTE THAT THE / CHARACTER IS EQUIVALENT TO THE 31,
0LUS BC0AC COMMAND AND THAT THE ONLY THINGS YOU CAN
USE B4C FOR ARE 427> AND 30C45>A<0C .OW HERE ARE A FEW EXAMPLES OF QUERIES USING THE ;8BC COMMAND
2T[[2;8-STbRaXQTU[PbWRPRWTR^]cT]c
RPRWTS:TT_BXiT
RPRWTSBXiT
SQ83
!00%.$)8 ! #%,,#,) !.$ $#,)
SQD]X`dT=P\T
WXc2^d]c
W^dabC^4g_XaPcX^]
\Xbb2^d]c
^QYTRc=d\QTa
cPQ[TB_PRT=d\QTa
2T[[2;8-bTcTRW^^]
2T[[2;8-/URNR^]cT]c
-2T[[2;8-[XbcU[PbWRPRWTR^]cT]c~
fWTaTSQD]X`dT=P\T[XZT4G31~
P]SWXcR^d]c- ~
PccaXQdcTbSQD]X`dT=P\T^QYTRc=d\QTaRPRWTS:TT_BXiTRPRWTSBXiT
WXc2^d]c\Xbb2^d]c
4G31! # (#" #% ! '
4G31# !#!#'"!"&%%
4G31!!# ' !!# $'
4G31!%& #$'&$! !'(
4G31"'" !$#&& ! $&!&
4G31#!" '%&&&% ' #
4G31#& # & #!#$$!'$
4G31#&! !&&($! #!!
4G31#&# "!#%#%#!'%"!%
4G31#&$ $( #%!#$ ( !#
4G31$ " $" '# %%%(#$$
4G31$& "&"$$$!"%"(
4G31%! & "("! % !(
4G31%! " "'#! #'"$( #&
4G31%! % !#$ '# '#!(
4G31%"&" "#' % !!!%
4G31$% '$ # (#" #'!! !(
4G31%%'#( #"'&%"$! !! ""!!
4G31& #(" $%"% (%" ! !&
4G31& #(& "$ %' "! !!
4G31& $&" !&% & # "&
4G31& &&$ ' # !% '"#((##%" $
2T[[2;8-[XbcU[PbWRPRWTR^]cT]cfWTaTSQD]X`dT=P\T[XZT4G . .~
P]SWXcR^d]c-
PccaXQdcTbSQD]X`dT=P\T^QYTRc=d\QTaRPRWTS:TT_BXiTRPRWTSBXiT
4G31! # (#" #
4G31 ' &(%#'
4G31"& %!!$(!
4G31# !#!#'"!
4G31%" $!#!''
4G31 # %'' !'
4G31!!# "# &'&!
4G31!%& #$'&$!
4G31"'" !%& $(!
!00%.$)8 ! #%,,#,) !.$ $#,)
4G31#! $ &"!'
4G31#!" '%&&&%
4G31#!# &! '(%
4G31#& # & #!#
4G31#&! !&&($!
4G31#&" !"$ #
4G31#&# "$&# ##
4G31#&$ $$! # '
4G31$ " $" '# %
4G31$& ! !%! ##
4G31$& ( !# %%#
4G31$& "&"$$$!
4G31%! & "("! %
4G31%! " &!
4G31%! " #!!& &!
4G31%! % !#$ '#
4G31%"&" "$&(( #
4G31$% '$ # (#" #
4G31%%'#( #"'&%"$!
4G31& #(" $%"% (%
4G31& #(& "$ %'
4G31& $&" !' %%#
4G31& &&$ ' # !% '
2T[[2;8-[XbcU[PbWRPRWTR^]cT]c~
fWTaTSQD]X`dT=P\T[XZT4G . .P]SWXcR^d]c-
P]S^QYTRc=d\QTa[XZT &&$
!"$%%"&&#!%& &&$
2T[[2;8-[XbcU[PbWRPRWTR^]cT]c~
fWTaTSQD]X`dT=P\T[XZT G . .~
P]S^QYTRc=d\QTa[XZT &&$STcPX[
RPRWTS:TT_BXiT)
RPRWTSBXiT) ' # !% '
SQ83)!"$%%"&&#!
SQD]X`dT=P\T)4G31
WXc2^d]c)"#((#
\Xbb2^d]c)#%" $
^QYTRc=d\QTa)& &&$
cPQ[TB_PRT=d\QTa)%
2T[[2;8-[XbcU[PbWRPRWTR^]cT]c~
fWTaTSQD]X`dT=P\T[XZT4G . .~
P]SWXcR^d]c-
P]S^QYTRc=d\QTa[XZT &&$
!"$%%"&&#!%& &&$
2T[[2;8-[XbcU[PbWRPRWTR^]cT]c~
PccaXQdcTb^QYTRc=d\QTaWXc2^d]c\Xbb2^d]c~
!00%.$)8 ! #%,,#,) !.$ $#,)
fWTaTSQD]X`dT=P\T[XZT4G . .~
P]SWXcR^d]c- ~
P]S^QYTRc=d\QTa[XZT &&$
& &&$"#((##%" $
4HE 34B2A814 VERB WORKS SIMILARLY TO THE WAY IT DOES IN 31,
0LUS BUT IT MUST BE FULLY SPELLED OUT
YOU CANT USE THE FAMILIAR 34B2 AS AN ABBRECIATION .OTICE THAT THERE ARE NO HEADINGS FOR COLUMN
ORIENTED
OUTPUT !S YOU CAN SEE YOU CAN EXECUTE SCRIPTS THAT CONTAIN #ELL#,) COMMANDS USING THE / CHARACTER
AND USE B4C427>>= TO DISPLAY THE COMMANDS IN THE SCRIPTS THAT YOU EXECUTE -ANY OF THE ;8BC
COMMANDS WERE STRUNG ACROSS MULTIPLE LINES BY USING THE CONTINUATION OPERATOR 4HE ;8BC
COMMANDS LOOK A LOT LIKE 31, EXCEPT FOR ;8BC BEING USED INSTEAD OF B4;42C AND THE REGEX EXPRESSIONS FOR
MATCHING WHEN USING THE ;8:4 KEYWORD !LSO NOTICE THAT IN THE LAST COMMAND A NUMBER WAS MATCHED
WITH A REGEX EXPRESSION IMPLYING A DATA TYPE CONVERSION ALTHOUGH ALL DATA MAY BE TREATED AS TEXT 9OU
CAN SEE THAT THE 0CCA81DC4B AND F74A4 KEYWORDS CAN BE ANYWHERE ON THE COMMAND LINE AFTER THE ;8BC
>QYTRcCh_T KEYWORDS )N OTHER WORDS THESE TWO KEYWORDS ARE NOT POSITIONAL EITHER ONE CAN BE USED FIRST
&INALLY THE 34C08; KEYWORD TURNS THE OUTPUT SIDEWAYS /R AS THE HELP SAYS h&ORMATS THE DISPLAY AS AN
ATTRIBUTE ON EACH LINE WITH AN ATTRIBUTE DESCRIPTOR PRECEDING EACH VALUEv
3ENDING #OMMANDS FROM THE /PERATING 3YSTEM
)N ADDITION TO RUNNING #ELL#,) INTERACTIVELY AS YOUVE SEEN IN THESE EXAMPLES YOU CAN SPECIFY THE T
OPTION TO PASS IN #ELL#,) COMMANDS FROM YOUR OPERATING SYSTEM PROMPT &OR EXAMPLE THE FOLLOWING
LISTING SHOWS HOW THE T OPTION CAN BE USED TO QUERY THE STATUS OF RT[[bae DIRECTLY FROM THE /3
COMMAND LINE
JTgPRT[ $)a^^cL a^^c
-RT[[R[X T[XbcRT[[STcPX[
]P\T)TgPRT[ $
Q\RCh_T)8?<8
RT[[ETabX^])>BBN ! " N;8=DG G%#N $!
q
RT[[baeBcPcdb)ad]]X]V
\bBcPcdb)ad]]X]V
abBcPcdb)ad]]X]V
!MONG OTHER THINGS THE ~T OPTION IS HELPFUL WHEN YOU WANT TO INVOKE #ELL#,) FROM WITHIN AN
OPERATING SYSTEM SHELL SCRIPT
#ONFIGURING AND -ANAGING THE 3TORAGE #ELL
#ELL#,) IS ALSO USED IN A NUMBER OF WAYS FOR CONFIGURING EVERYTHING FROM DISK STORAGE TO CELL ALERTS 9OU
CAN ALSO USE #ELL#,) FOR MANAGEMENT TASKS SUCH AS STARTUP AND SHUTDOWN &OLLOWING ARE A FEW EXAMPLES
OF HOW TO USE #ELL#,) TO CONFIGURE AND MANAGE THE STORAGE CELL
#ELL 3ERVICES CAN BE SHUT DOWN ONE AT A TIME OR ALL AT ONCE 4HE FOLLOWING COMMANDS ARE USED TO
SHUT DOWN CELL SERVICES
BWdcS^f]RT[[bTaeXRTb^]TPcPcX\T
2T[[2;8-P[cTaRT[[bWdcS^f]bTaeXRTbRT[[bae
2T[[2;8-P[cTaRT[[bWdcS^f]bTaeXRTb\b
!00%.$)8 ! #%,,#,) !.$ $#,)
2T[[2;8-P[cTaRT[[bWdcS^f]bTaeXRTbab
BWdcS^f]P[[RT[[bTaeXRTb
2T[[2;8-P[cTaRT[[bWdcS^f]bTaeXRTbP[[
#ELL SERVICES MAY ALSO BE STARTED UP ONE BY ONE OR ALL AT ONCE .OTE THAT THE AB PROCESS MUST BE
STARTED FIRST OR #ELL#,) WILL THROW AN ERROR SUCH AS THE FOLLOWING
2T[[2;8-P[cTaRT[[bcPacd_bTaeXRTbRT[[bae
BcPacX]V24;;BAEbTaeXRTb
24;; $ ()ATbcPacBTaeTaAB]^caTb_^]SX]V
4HE FOLLOWING COMMANDS ARE USED TO START UP CELL SERVICES
BcPacd_RT[[bTaeXRTb^]TPcPcX\T
2T[[2;8-P[cTaRT[[bcPacd_bTaeXRTbab
2T[[2;8-P[cTaRT[[bcPacd_bTaeXRTb\b
2T[[2;8-P[cTaRT[[bcPacd_bTaeXRTbRT[[bae
BcPacd_P[[RT[[bTaeXRTb
2T[[2;8-P[cTaRT[[bcPacd_bTaeXRTbP[[
4HE FOLLOWING COMMANDS SHUT DOWN AND RESTART THE CELL SERVICES
1^d]RTRT[[bTaeXRTb^]TPcPcX\T
P[cTaRT[[aTbcPacbTaeXRTbRT[[bae
P[cTaRT[[aTbcPacbTaeXRTbab
P[cTaRT[[aTbcPacbTaeXRTb\b
1^d]RTP[[RT[[bTaeXRTb
P[cTaRT[[aTbcPacbTaeXRTbP[[
4O SHOW THE CURRENT STATUS OF RT[[bae USE THE ;8BC24;;34C08; COMMAND AS FOLLOWS
2T[[2;8-[XbcRT[[STcPX[
]P\T)TgPRT[ $
Q\RCh_T)8?<8
RT[[ETabX^])>BBN ! " N;8=DG G%#N ! % !
R_d2^d]c)!#
UP]2^d]c) ! !
UP]BcPcdb)]^a\P[
XS) $5<< 9$
X]cTaR^]]TRc2^d]c)"
X]cTaR^]]TRc )Q^]SXQ
X^a\1^^bc)
X_PSSaTbb ) (! %' ! ( !#
ZTa]T[ETabX^])! % ' (# " " T[$
[^RPc^a;43BcPcdb)^UU
\PZT<^ST[)BD=<82A>BHBC4<BBD=58A4G#!& <!B4AE4AB0C0
\TcaXR7Xbc^ah3Phb)&
!00%.$)8 ! #%,,#,) !.$ $#,)
]^cXUXRPcX^]<TcW^S)\PX[
]^cXUXRPcX^]?^[XRh)RaXcXRP[fPa]X]VR[TPa
^UU[^PS4UUXRXT]Rh) "(("$ %
_^fTa2^d]c)! !
_^fTaBcPcdb)]^a\P[
b\c_5a^\)4gPSPcP
b\c_5a^\0SSa)TgPSPcP/^daR^\_P]h R^\
b\c_?^ac)!$
b\c_?fS)
b\c_BTaeTa)b\c_ ^daR^\_P]h R^\
b\c_C^0SSa)P[[NSQP/^daR^\_P]h R^\P[[NbP/^daR^\_P]h R^\
b\c_DbTa)
b\c_DbTBB;)50;B4
bcPcdb)^][X]T
cT\_TaPcdaTATPSX]V)!&
cT\_TaPcdaTBcPcdb)]^a\P[
d_CX\T)"&SPhb ')!!
RT[[baeBcPcdb)ad]]X]V
\bBcPcdb)ad]]X]V
abBcPcdb)ad]]X]V
3EVERAL OF THE SETTINGS YOU SEE IN THIS LISTING CAN BE SET USING THE 0;C4A24;; COMMAND 4HESE
SETTINGS MAY BE CONFIGURED ONE AT A TIME OR TOGETHER BY SEPARATING THEM WITH A COMMA &OR EXAMPLE
2^]UXVdaT]^cXUXRPcX^][TeT[U^aP[Tacb
2T[[2;8-0;C4A24;;]^cXUXRPcX^]?^[XRh,RaXcXRP[fPa]X]VR[TPa
2^]UXVdaTcWTRT[[U^aT\PX[]^cXUXRPcX^]b
2T[[2;8-0;C4A24;;b\c_BTaeTa,b\c_ T]ZXcTR R^\
b\c_5a^\0SSa,4gPSPcP/^daR^\_P]h R^\
b\c_5a^\,4gPSPcP
b\c_C^0SSa,P[[NSQP/^daR^\_P]h R^\P[[NbP/^daR^\_P]h R^\
]^cXUXRPcX^]?^[XRh,RaXcXRP[fPa]X]VR[TPa
]^cXUXRPcX^]<TcW^S,\PX[
"Y THE WAY IF YOU HAVENT ALREADY STUMBLED ACROSS THIS FEATURE #ELL#,) STORES A COMMAND HISTORY
SIMILAR TO THE "ASH SHELL 9OU CAN SCROLL UP AND DOWN THROUGH YOUR HISTORY AND EDIT COMMANDS USING THE
ARROW KEYS !ND REGEX ALSO PROVIDES A VERY POWERFUL PATTERN
MATCHING CAPABILITY 4HE #ELL#,) SYNTAX WILL
BE SOMETHING NEW TO SYSTEM ADMINISTRATORS AND $"!S ALIKE BUT ONCE YOU UNDERSTAND THE SYNTAX IT
REALLY ISNT DIFFICULT TO MASTER
dcli Command Syntax
SR[X IS A TOOL BY WHICH YOU CAN EXECUTE A SINGLE COMMAND ACROSS ALL CELLS (AVING WORKED ON VARIOUS
CLUSTERED SYSTEMS OVER THE YEARS WEVE COME TO APPRECIATE THE IMPORTANCE OF KEEPING SCRIPTS AND SOME
CONFIGURATION FILES IDENTICAL ACROSS ALL NODES )TS ALSO VERY HANDY TO HAVE A FACILITY FOR EXECUTING THE
SAME COMMAND CONSISTENTLY ACROSS ALL NODES OF A CLUSTER /RACLE PROVIDES THE SR[X COMMAND TO DO JUST
THAT !MONG OTHER THINGS THE SR[X COMMAND ALLOWS YOU TO
v #ONFIGURE 33( EQUIVALENCY ACROSS ALL STORAGE CELLS ANDOR DATABASE SERVERS
!00%.$)8 ! #%,,#,) !.$ $#,)
v $ISTRIBUTE A FILE TO THE SAME LOCATION ON ALL SERVERSCELLS IN THE CLUSTER
v $ISTRIBUTE AND EXECUTE A SCRIPT ON SERVERSCELLS IN THE CLUSTER
v %XECUTE COMMANDS AND SCRIPTS ON SERVERSCELLS IN THE CLUSTER
SR[X USES 33( EQUIVALENCY TO AUTHENTICATE YOUR SESSION ON THE REMOTE SERVERS )F YOU DO NOT HAVE
33( EQUIVALENCY ESTABLISHED ACROSS SERVERSCELLS YOU CAN STILL USE IT BUT IT WILL PROMPT YOU FOR A
PASSWORD FOR EACH REMOTE SYSTEM BEFORE EXECUTING THE COMMAND SR[X EXECUTES ALL COMMANDS IN
PARALLEL AGGREGATES THE OUTPUT FROM EACH SERVER INTO A SINGLE LIST AND DISPLAYS THE OUTPUT ON THE LOCAL
MACHINE &OR EXAMPLE THE FOLLOWING LISTING SHOWS THE _\^] PROCESSES RUNNING ON ALL DATABASE SERVERS
JT]ZSQ !)a^^cL a^^c
- dba [^RP[ QX] SR[X [a^^c VSQbNVa^d__b TUkVaT_^aPN_\^]kVaT_ eVaT_
T]ZSQ )^aPR[T#(&" 9d] (. ) ) %^aPN_\^]N34<>
T]ZSQ )^aPR[T(( & 9d] %. ) ) $^aPN_\^]N1; %:
T]ZSQ )^aPR[T((!( 9d] %. ) )! ^aPN_\^]NC4BC
T]ZSQ )^aPR[T "!$ 9d] %. ) ) $^aPN_\^]NB0=31>G
T]ZSQ )^aPR[T"!%" 9d] %. ) ) &^aPN_\^]N315B
T]ZSQ !)^aPR[T& $ 9d] (. ) ) %^aPN_\^]N34<>!
T]ZSQ !)^aPR[T"& ' 9d] %. ) ) $^aPN_\^]N315B!
T]ZSQ !)^aPR[T($" 9d] %. ) )! ^aPN_\^]N4G31!
T]ZSQ !)^aPR[T &! 9d] %. ) ) ^aPN_\^]NB=855
T]ZSQ !)^aPR[T '$ 9d] %. ) )! ^aPN_\^]NC4BC!
T]ZSQ !)^aPR[T '& 9d] %. ) ) "^aPN_\^]N1; %:!
T]ZSQ !)^aPR[T "% 9d] %. ) ) #^aPN_\^]NB0=31>G!
T]ZSQ !)^aPR[T!"#(# 9d] %. ) ) ^aPN_\^]NB2A0C27
SR[X IS PARTICULARLY USEFULL WHEN YOU WANT TO COLLECT INFORMATION FROM ALL STORAGE CELLS USING THE
#ELL#,) COMMANDS 4HE FOLLOWING EXAMPLE SHOWS HOW SR[X AND #ELL#,) COMMANDS CAN BE USED
TOGETHER TO REPORT THE STATUS OF ALL STORAGE CELLS IN A HALF RACK CLUSTER
JT]ZSQ )a^^cL a^^c
-SR[X V a^^c RT[[NVa^d_ [a^^cRT[[R[X T[XbcRT[[
T]ZRT[ )T]ZRT[ ^][X]T
T]ZRT[ !)T]ZRT[ !^][X]T
T]ZRT[ ")T]ZRT[ "^][X]T
T]ZRT[ #)T]ZRT[ #^][X]T
T]ZRT[ $)T]ZRT[ $^][X]T
T]ZRT[ %)T]ZRT[ %^][X]T
T]ZRT[ &)T]ZRT[ &^][X]T
9OU MAY HAVE RECOGNIZED THE a^^c RT[[NVa^d_ PARAMETER IN THIS EXAMPLE 4HIS FILE IS GENERATED
DURING THE %XADATA INSTALLATION PROCEDURE SEE #HAPTER FOR MORE DETAILS 4HERE ARE ACTUALLY SEVERAL OF
THESE hGROUP FILESv THAT ARE USEFUL FOR RUNNING DISTRIBUTED COMMANDS USING SR[X 4HESE FILES MAY BE
DESCRIBED AS FOLLOWS
SQbNVa^d_: 4HIS FILE CONTAINS THE MANAGEMENT HOSTNAMES FOR ALL DATABASE
SERVERS IN YOUR %XADATA CONFIGURATION )T PROVIDES A CONVENTIENT WAY TO EXECUTE
SR[X COMMANDS ON THE DATABASE SERVERS
!00%.$)8 ! #%,,#,) !.$ $#,)
RT[[NVa^d_ 4HIS FILE CONTAINS THE MANAGEMENT HOSTNAMES FOR ALL STORAGE CELLS
IN YOUR %XADATA CONFIGURATION )T PROVIDES A CONVENIENT WAY TO EXECUTE SR[X
COMMANDS LIMITED TO THE STORAGE CELLS
P[[NVa^d_: 4HIS FILE IS A COMBINATION OF THE SQbNVa^d_ AND RT[[NVa^d_ FILES AND
CONTAINS A LIST OF THE MANAGEMENT HOSTNAMES FOR ALL DATABASE SERVERS AND
STORAGE CELLS IN YOUR %XADATA CONFIGURATION 5SING THIS FILE YOU CAN EXECUTE SR[X
COMMANDS ON ALL DATABASE SERVERS AND STORAGE CELLS
!NY OF THE #ELL#,) COMMANDS WEVE DISCUSSED IN THIS APPENDIX MAY BE EXECUTED FROM A CENTRAL
LOCATION USING SR[X )N FACT THE ONLY RESTRICTION IS THAT THE COMMAND CANNOT BE INTERACTIVE REQUIRING USER
INPUT DURING EXECUTION &OR EXAMPLE THE FOLLOWING LISTING ILLUSTRATES COLLECTING ALL THE CURRENT
PERFORMANCE METRICS FROM THE STORAGE CELLS
SR[X [a^^c V a^^c RT[[NVa^d_RT[[R[X T;8BC<4CA822DAA4=C0CCA81DC4B]P\T^QYTRcch_T
\TcaXR>QYTRc=P\T\TcaXREP[dTR^[[TRcX^]CX\T
4HE OUTPUT FROM THIS COMMAND IS MUCH TOO VERBOSE TO SHOW HERE "UT WITH A SIMPLE SCRIPT IT CAN BE
FORMATTED AND DISTRIBUTED AS NEEDED )N FACT WE USED THIS SCRIPT IN #HAPTER TO COLLECT )/2-
PERFORMANCE METRICS AND REPORT ON THEM THROUGH AN EXTERNAL TABLE DEFINITION IN THE DATABASE
3CHEDULING A REPORT SUCH AS THIS TO RUN DAILY WOULD BE A VERY CONVENIENT WAY TO MONITOR )/ AT THE
STORAGE
CELL LEVEL
Summary
4HERE ARE MANY MORE USES FOR SR[X AND #ELL#,) THAN WEVE COVERED HERE 3YSTEM ADMINISTRATORS WILL ALSO
FIND IT USEFUL FOR CREATING NEW USER ACCOUNTS ON THE DATABASE SERVERS USING THE dbTaPSS AND Va^d_PSS
COMMANDS FOR EXAMPLE $"!S WILL FIND SR[X USEFUL FOR DISTRIBUTING SCRIPTS AND OTHER FILES TO OTHER
SERVERS IN THE CLUSTER !ND USING SR[X AND #ELL#,) TOGETHER PROVIDES A CONVENIENT WAY OF MANAGING
EXTRACTING AND REPORTING KEY PERFORMANCE METRICS FROM THE STORAGE CELLS
APPENDIX B
Online Exadata Resources
4HIS APPENDIX DETAILS SOME HELPFUL ONLINE RESOURCES FOR $"!S MANAGING %XADATA /RACLE 3UPPORT CREATES
A GOOD MANY OF WHAT ARE TERMED support notes 7E LIST SOME OF THE MOST HELPFUL AND YOU CAN READ
THEMIF YOU ARE A LICENSED USER BY GOING TO /RACLES SUPPORT SITE )N ADDITION WE LIST A FEW HELPFUL
BLOGS 4HE LIST OF BLOGS IS SMALL BUT GROWING 7E LIST FOUR THAT WE PARTICULARLY LIKE AND ENDORSE
Exadata MOS Support Notes
,ISTED HERE ARE SEVERAL GOOD ONLINE NOTES FOR MANAGING THE %XADATA PLATFORM ON -Y /RACLE 3UPPORT
-/3 3OME OF THESE NOTES ARE LIVING DOCUMENTS MEANING THEY ARE CONTINUALLY UPDATED AS NEW
SOFTWARE VERSIONS AND PATCHES BECOME AVAILABLE -/3 .OTE IS A MUST READ FOR ANYONE
RESPONSIBLE FOR ADMINISTERING THE SYSTEM )T CONTAINS CRITICAL INFORMATION ABOUT SUPPORTED SOFTWARE
RELEASES 3OME OF THE -/3 .OTES LISTED HERE SUCH AS ARE SIMPLY PLACEHOLDERS FOR DOZENS OF
OTHER IMPORTANT DOCUMENTS YOU WILL WANT TO BE AWARE OF /BVIOUSLY THIS IS NOT A COMPREHENSIVE LIST AND
THERE ARE MANY MORE GOOD TECHNICAL DOCUMENTS FOR %XADATA ON -/3 "UT WE HOPE YOU FIND THIS LIST
HELPFUL IN GETTING YOU STARTED OFF IN THE RIGHT DIRECTION
Helpful Exadata MOS Support Notes
&LASHBACK $ATABASE "EST 0RACTICES -/3 .OTE
$ATABASE %NDIAN #ONVERSION -/3 .OTE
2$3/&%$ -/3 .OTE
%XADATA "EST 0RACTICES -/3 .OTE
G2 3#!. %XPLAINED -/3 .OTE
%XADATA 3TORAGE 3ERVER G2 3UPPORTED 6ERSIONS -/3 .OTE
$"&3 -/3 .OTE
%XADATA (EALTHCHECK 3CRIPT -/3 .OTE
%XADATA 6 $IAGNOSABILITY AND 4ROUBLESHOOTING "EST 0RACTICES -/3 .OTE
"ARE -ETAL 2ESTORE 0ROCEDURE FOR #OMPUTE .ODES -/3 .OTE
-ONITORING %XADATA -/3 .OTE
3TEPS TO SHUT DOWN OR REBOOT AN %XADATA STORAGE CELL WITHOUT AFFECTING !3- -/3 .OTE
-ASTER .OTE FOR /RACLE $ATABASE -ACHINE AND %XADATA 3TORAGE 3ERVER -/3 .OTE
%XADATA 0ATCHING /VERVIEW AND 0ATCH 4ESTING 'UIDELINES -/3 .OTE
%XADATA #RITICAL )SSUES -/3 .OTE
%XADATA 8 $IAGNOSABILITY AND 4ROUBLESHOOTING "EST 0RACTICES -/3 .OTE
4ROUBLESHOOTING )NFINI"AND 3WITCH 0ROBLEMS ON %XADATA -/3 .OTE
TO $ATABASE 5PGRADE ON %XADATA $ATABASE -ACHINE -/3 .OTE
!00%.$)8 " /.,).% %8!$!4! 2%3/52#%3
Exadata Bulletins and Blogs
&OLLOWING ARE SOME HELPFUL BLOGS 3OME FOCUS ON %XADATA ALONE /THERS SUCH AS 4ANEL 0ODERS BLOG ARE
MORE WIDE RANGING AND COVER THE GAMUT OF /RACLE $ATABASE TECHNOLOGIES
%XPERT /RACLE %XADATA "LOG
Wcc_) fff 4g_Tac>aPR[T4gPSPcP R^\
+ERRY /SBORNES /RACLE "LOG
Wcc_) ZTaah^bQ^a]T ^aPR[T Vdh R^\
4ANEL 0ODERS "LOG
Wcc_) Q[^V cP]T[_^STa R^\
!NDY #OLVINS "LOG
Wcc_) Q[^V ^aPR[T ]X]YP R^\
APPENDIX C
Diagnostic Scripts
7EVE USED SEVERAL DIAGNOSTIC SCRIPTS IN THIS BOOK 7HILE THE CONTENTS OF MANY OF THEM ARE DISPLAYED IN
THE BODY OF THE BOOK SOME OF THEM ARE LENGTHY ENOUGH THAT WE DECIDED NOT TO PRINT THEIR CONTENTS IN THE
LISTINGS 4HESE SCRIPTS ARE ALL AVAILABLE ONLINE AT fff 4_Tac>aPR[T4gSPcP R^\ 4ABLE #
IN THIS APPENDIX
CONTAINS A LIST OF THE SCRIPTS ALONG WITH A BRIEF DESCRIPTION OF EACH ONE
Table C-1. Diagnostic Scripts Used in This Book
Script Name Description
Pb b`[ !3 IS SHORT FOR !CTIVE 31, 4HIS SCRIPT SHOWS ALL ACTIVE 31, STATEMENTS ON
THE CURRENT INSTANCE AS SHOWN BY EB4BB8>= .OTE THAT YOU MAY NEED TO
EXECUTE IT SEVERAL TIMES TO GET AN IDEA OF WHATS HAPPENING ON A SYSTEM
AS FAST STATEMENTS MAY NOT BE hCAUGHTv BY THIS QUICK AND DIRTY
APPROACH
RP[XQaPcTNX^ b`[ 4HIS SCRIPT PROVIDES A SIMPLE WRAPPER FOR THE
31<BNA4B>DA24N<0=064A 20;81A0C4N8> PROCEDURE 4HE PROCEDURE MUST
BE RUN BEFORE /RACLE WILL ALLOW YOU TO ENABLE !UTO $/0
RWTRZN_g b`[ 4HIS SCRIPT CONTAINS A SIMPLE QUERY OF E?GN?A>24BBNBHBBC0C TO SHOW
HOW MANY PARALLEL SERVER PROCESSES ARE CURRENTLY IN USE
R^\_NaPcX^ b`[ 4HIS IS A SIMPLE SCRIPT THAT COMPUTES A COMPRESSION RATIO BASED ON AN
INPUT VALUE THE ORIGINAL TABLE SIZE
RaTPcTNSXb_[PhNaPf b`[ 4HIS SCRIPT CREATES THE SXb_[PhNaPf FUNCTION WHICH TRANSLATES RAW
DATA TYPE VALUES INTO VARIOUS OTHER DATA TYPES ORIGINALLY WRITTEN BY
'REG 2AHN
SQPNcPQ[Tb b`[ 4HIS IS A SIMPLE SCRIPT TO QUERY 310NC01;4B )T SHOWS THE NUMBER OF ROWS
NUMBER OF BLOCKS AND DEFAULT DEGREE OF PARALLELISM
STbR b`[ 4HIS SCRIPT PRODUCES OUTPUT SIMILAR TO THE 31,
0LUS 34B2A814 COMMAND
!00%.$)8 # $)!'./34)# 3#2)043
Script Name Description
SXb_[PhNaPf b`[ 4HIS IS A SIMPLE SCRIPT TO TRANSLATE A RAW VALUE INTO A SPECIFIED DATA TYPE
FORMAT SUCH AS =D<14A OR E0A270A! )T DEPENDS ON THE SXb_[PhNaPf
FUNCTION CREATED BY THE RaTPcTNSXb_[PhNaPf b`[ SCRIPT
S_[P] b`[ 4HIS SCRIPT SHOWS THE ACTUAL EXECUTION PLAN FOR A 31, STATEMENT IN THE
SHARED POOL 4HIS IS A VERY SIMPLE SCRIPT THAT PROMPTS FOR A B@;N83 AND
278;3N=> AND THEN CALLS SQ\bNg_[P] SXb_[PhNRdab^a
Sd\_NQ[^RZ b`[ 4HIS SCRIPT DUMPS A DATA BLOCK TO A TRACE FILE USING 0;C4ABHBC4<3D<?
30C058;4 )T PROMPTS FOR FILENO AND BLOCKNO
TbURNWXcNaPcX^ b`[ 5SELESS SCRIPT FOR CALCULATING AN INCORRECT %DXADATA 3MART &LASH #ACHE
HIT RATIO
TbURNZTT_NcPQ[Tb b`[ 4HIS SCRIPT DISPLAYS OBJECTS THAT HAVE THE 24;;N5;0B7N20274 ATTRIBUTE SET
TO :44?
U[dbWN_^^[ b`[ 4HIS SCRIPT USES 0;C4ABHBC4<5;DB7B70A43N?>>; TO FLUSH ALL 31,
STATEMENTS FROM THE SHARED POOL
U[dbWNb`[ b`[ 4HIS SCRIPT USES 31<BNB70A43N?>>; ?DA64 TO FLUSH A SINGLE 31,
STATETEMENT FROM THE SHARED POOL )T ONLY WORKS WITH AND LATER
Ub b`[ 4HIS SCRIPT ALLOWS YOU TO SEARCH THROUGH EB@; USING A BIT OF 31, TEXT OR A
B@;N83 &3 IS SHORT FOR &IND 31, 4HE SCRIPT REPORTS SOME STATISTICAL
INFORMATION SUCH AS AVERAGE %LAPSED 4IME AND AVERAGE ,)/S
Ubg b`[ &38 STANDS FOR &IND 31, E8ADATA 4HIS SCRIPT SEARCHES THE SHARED POOL
EB@; BASED ON THE 31, STATEMENT TEXT OR A SPECIFIC B@;N83 AND REPORTS
WHETHER STATEMENTS WERE OFFLOADED OR NOT AND IF OFFLOADED WHAT
PERCENTAGE OF )/ WAS SAVED .OTE THAT THERE ARE SEVERAL ALTERNATE
VERSIONS OF THIS SCRIPT USED IN THE BOOK Ubg! b`[ Ubg" b`[ AND
Ubg# b`[ 4HESE VERSIONS REDUCE THE WIDTH OF THE OUTPUT TO SOMETHING
MORE EASILY PRINTED IN THE LIMITS IMPOSED BY THE PRINTED BOOK FORMAT
Ubg^ b`[ 4HIS SCRIPT IS SIMILAR TO THE Ubg b`[ SCRIPT BUT CAN BE USED TO IDENTIFY
LONG
RUNNING STATEMENTS AND REPORT ON WHETHER THEY HAVE BEEN
OFFLOADED OR NOT )T CAN BE USED IN CONJUNCTION WITH THE
^UU[^PSN_TaRT]c b`[ SCRIPT TO DRILL INTO THE INDIVIDUAL STATEMENTS
CONTRIBUTING TO ITS CALCUALTED OFFLOAD PERCENTAGE
VPcWTaNcPQ[TNbcPcb b`[ 4HIS IS A SIMPLE SCRIPT TO GATHER TABLE STATISTICS USING THE
31<BNBC0CB 60C74ANC01;4NBC0CB PROCEDURE
!00%.$)8 # $)!'./34)# 3#2)043
Script Name Description
VTcNR^\_aTbbX^]NaPcX^ b`[ 4HIS SCRIPT IS A WRAPPER FOR THE BUILT IN COMPRESSION ADVISOR
FUNCTIONALITY 31<BN2><?A4BB8>= 64CN2><?A4BB8>=NA0C8> )T PROMPTS FOR
A TABLE NAME AND A COMPRESSION TYPE AND THEN ESTIMATES THE EXPECTED
COMPRESSION RATIO BY ACTUALLY COMPRESSING A SUBSET OF THE TABLES ROWS
VTcNR^\_aTbbX^]Nch_T b`[ 4HIS SCRIPT PROVIDES A WRAPPER FOR THE
31<BN2><?A4BB8>= 64CN2><?A4BB8>=NCH?4 PROCEDURE )T CAN BE USED TO
IDENTIFIY THE ACTUAL COMPRESSION TYPE USED FOR A SPECIFIC ROW )T PROMPTS
FOR A TABLE NAME AND A ROWID AND RETURNS THE ACTUAL COMPRESSION TYPE
FOR THAT ROW AS OPPOSED TO THE COMPRESSION TYPE ASSIGNED TO THE TABLE
\hbcPcb b`[ 4HIS IS A SIMPLE SCRIPT FOR QUERYING E<HBC0CB
^[SNa^fXS b`[ 4HIS SCRIPT CREATES THE ^[SNa^fXS FUNCTION 4HE ^[SNa^fXS FUNCTION
ACCEPTS A a^fXS AND RETURNS THE UX[T]^ Q[^RZ]^ AND a^f]^ THE OLD a^fXS
FORMAT
^QYNQhNWTg b`[ 4HIS SCRIPT TRANSLATES AN ^QYTRcNXS IN HEX FORMAT INTO AN OBJECT NAME
4HE HEX VALUE IS CONTAINED IN BLOCK DUMPS
^UU[^PSN_TaRT]c b`[ 4HIS SCRIPT CAN BE USED TO PROVIDE A QUICK CHECK ON WHETHER STATEMENTS
ARE BEING OFFLOADED OR NOT ON %XADATA PLATFORMS )T ALLOWS ALL STATEMENTS
OVER A MINIMUM TIME OR A MINIMUM NUMBER OF ,)/S TO BE EVALUATED
AND CALCULATES A PERCENTAGE OF STATEMENTS THAT HAVE BEEN OFFLOADED
_Pa\b b`[ 4HIS SCRIPT DISPLAYS DATABASE PARAMETERS AND THEIR CURRENT VALUES
)NCLUDES A SWITCH TO SHOW OR SUPPRESS DISPLAY OF HIDDEN PARAMETERS
_Pa\bS b`[ 4HIS SCRIPT DISPLAYS DATABASE PARAMETERS AND THEIR DESCRIPTIONS
)NCLUDES A SWITCH TO SHOW OR SUPPRESS DISPLAY OF HIDDEN PARAMETERS
_PacNbXiT! b`[ 4HIS SCRIPT SHOWS THE SIZES OF PARTITIONS AS REPORTED BY 310NB46<4=CB
_^^[N\T\ b`[ 4HIS SCRIPT PROVIDES A SIMPLE QUERY AGAINST EB60BC0C SHOWING MEMORY
ASSIGNED TO VARIOUS hPOOLSv
`dTdTSNb`[ b`[ 4HIS SIMPLE SCRIPT QUERIES EB@;N<>=8C>A FOR STATEMENTS THAT ARE QUEUED
BY THE PARALLEL STATEMENT QUEUING FEATURE
aT_^acNb`[N\^]Xc^a b`[ 4HIS IS A SCRIPT TO CALL 31<BNB@;CD=4 A4?>ACNB@;N<>=8C>A
bX b`[ 4HIS SCRIPT DISPLAYS THE CURRENT VALUE FOR THE STATISTIC 2T[[?WhbXRP[8>
1hcTbBPeTSQhBc^aPVT8]STg FROM E<HBC0CB )T PROVIDES A QUICK WAY
TO CHECK STORAGE INDEX USAGE
!00%.$)8 # $)!'./34)# 3#2)043
Script Name Description
b]P__Ta b`[ 4HIS IS FAR AND AWAY THE MOST ROBUST SCRIPT USED IN THE BOOK )T IS REALLY
MORE LIKE A MONITORING PROGRAM THAT CAN REPORT ON AN EXTREMELY WIDE
RANGE OF INFORMATION ABOUT ACTIVE SESSIONS $OCUMENTATION CAN BE
FOUND ON 4ANELS BLOG
bbN^UU b`[ 4HIS SCRIPT TURNS OFF 3MART 3CANS VIA P[cTabTbbX^] THAT IS IT SETS
24;;N>55;>03N?A>24BB8=6,50;B4
bbN^] b`[ 4HIS SCRIPT 4URNS ON 3MART 3CANS VIA P[cTabTbbX^] THAT IS IT SETS
24;;N>55;>03N?A>24BB8=6,CAD4
cPQ[TNbXiT b`[ 4HIS SCRIPT SHOWS SIZES OF OBJECTS AS REPORTED BY 310NB46<4=CB 4HERE IS
ANOTHER VERSION cPQ[TNbXiT! b`[ THAT IS BASICALLY THE SAME SCRIPT WITH
A REDUCED NUMBER OF OUTPUT COLUMNS
eP[XSNTeT]cb b`[ 4HIS SCRIPT DISPLAYS A LIST OF WAIT EVENTS THAT MATCH A TEXT STRING
fW^P\X b`[ 4HIS SCRIPT DISPLAYS CURRENT SESSION INFORMATION INCLUDING 3)$ 3ERIAL
0REVIOUS 31, (ASH 6ALUE AND /3 3HADOW 0ROCESS )$
Index
3!4! DISKS
A 3934%-?$' DISK GROUP
INTERLEAVING
!CCESS CONTROL LIST !#, )/ PERFORMANCE RATING
!LLOCATION UNITS !5S DEFAULT ALLOCATION POLICY
!LTER GRIDDISK COMMAND INTERLEAVING POLICY
!PPLY#ONFIG ISOLATING STORAGE CELL ACCESS
!UTOMATIC DEGREE OF PARALLELISM !UTO $/0 STORAGE ALLOCATION
)/ CALIBRATION STORAGE VIRTUALIZATION
OPERATION AND CONFIGURATION n !VG #ELLDISK 2EADS (MB)
SINGLE $/0 VALUE !VG #ELLDISK 3MALL 2EAD ,ATENCY MSEC CHART
!UTOMATIC STORAGE MANAGEMENT !3-
CELL SECURITY
!3-
SCOPED SECURITY
BEST PRACTICES B
CELLKEYORA FILE
DATABASE
SCOPED SECURITY n "ACKUP AND RESTORE STRATEGY
REMOVING CELL SECURITY BLOCK CHANGE TRACKING FILE
SECURITY KEY FULL BACKUP AND RESTORE
STORAGE REALM INCREMENTAL BACKUP
CONFIGURATION OPTIONS "ALANCED $7") SYSTEM
DEFAULT POLICY "ASIC COMPRESSION METHOD
%XADATA DISK ARCHITECTURE "LOCK CHANGE TRACKING "#4
FAILURE GROUPS "ULLETINS AND "LOGS
FAULT RESILIENCY %XPERT /RACLE %XADATA "LOG
GRID DISKS +ERRY /SBORNES /RACLE "LOG
#ELL#,) COMMAND 4ANEL 0ODERS "LOG
CELL DISKS
CREATION C
$!4! DISK GROUPS
DEVSDA PARTITION #%,,"//4 53" FLASH DRIVE
%XADATA REPORTS #ELL#,)
&LASH$ISK
BASED GRID DISKS ASSIGN KEY COMMAND
I/O space allocation CELL ALERTS
KEY ATTRIBUTES ALERT SEVERITIES
,INUX /3 PARTITIONS CELL DISK READWRITE ERRORS
LIST CELLDISK COMMAND CELL TEMPERATURE
naming #LEAR SEVERITY
NON
%XADATA SYSTEM EXAMINED"Y ATTRIBUTE
PHYSICAL DISKS AND FLASH CACHE MODULES ,)34 -%42)#()34/29 $%4!),
COMMAND
2%#/ DISK GROUPS SOFTWARE FAILURES
).$%8
CELL METRICS 2!# CLUSTERS
$%3#2)"% -%42)##522%.4 COMMAND CELLIPORA FILE
FULL RACK CONFIGURATION
$%3#2)"% -%42)#()34/29 COMMAND GRACE FAILOVER
HIGH AVAILABILITY
)/ METRIC ABBREVIATIONS RESOURCE ALLOCATION
,)34 -%42)##522%.4 COMMAND SCALABILITY
,)34 -%42)#$%&).)4)/. COMMAND 5!4 DATABASE
#ONSUMER GROUPS
,)34 -%42)#()34/29 COMMAND !003
METRIC NAME PREFIXES HIGH
PRIORITY BUSINESS APPLICATIONS
CELL SERVICES
!,4%2 #%,, COMMAND MAPPING RULES
,)34 #%,, $%4!), COMMAND #,)%.4?!00,)#!4)/.
SHUT DOWN AND RESTART CELL SERVICES login ATTRIBUTES
SHUT DOWN CELL SERVICES /2!#,%?53%2
START UP CELL SERVICES 2%0/243
E OPTION RUNTIME ATTRIBUTES
FLASHCACHECONTENT DETAIL SESSION
TO
CONSUMER
GROUP
HELP INTERFACE
KEYWORDS 40/$%2
,)34 COMMANDS /4(%2?'2/503
STORAGE CELL CONFIGURING AND MANAGING 31, STATEMENTQUERY
#ELL DISK FAILURE 393?'2/50
DISK REPAIR TIMER #OST "ASED /PTIMIZER #"/
DISK REPLACEMENT #05
INTENSIVE PROCESSING
GRID DISKS REMOVAL CREATE GRIDDISK COMMAND
,)34 '2)$$)3+ COMMAND
POOR PERFORMANCE
PREDICTIVE FAILURE D
3#2!4#(?$' $ATABASE BACKUP
CELLIPORA FILE BACKUP ARCHITECTURE
CELLKEYORA FILES DISK
BASED BACKUPS
#ELL /FFLOAD %FFICIENCY RATIO 2-!.
CHECKIPSH SCRIPT #HECK)0 BLOCK CHANGE TRACKING
Client access network INCREMENTAL BACKUPS
#OMPRESSION !DVISOR WAIT EVENTS
#OMPUTE NODE STANDBY DATABASE
CONSIDERATIONS TAPE
BASED BACKUPS
#05 RESOURCES $ATABASE &ILE 3YSTEM $"&3
)/ PERFORMANCE AND CAPACITY $ATABASE -ACHINE CONFIGURATION
MEMORY RESOURCES !PPLY#ONFIG
PATCHING AND TESTING #HECK)0
%XADATA CLUSTERS CONFIGURATION PROCESS
FULL
RACK CONFIGURATION CONFIGURATION WORKSHEET
LEAF SWITCH PORTS %XADATA CONFIGURATION PARAMETERS
NETWORK CONFIGURATION NETWORK SETTINGS
NON
2!# CONFIGURATION /RACLE ENVIRONMENT PARAMETERS
ARCHITECTURE $"- #ONFIGURATOR
GRID DISKS &IRSTBOOT PROCESS
PRODUCTION CLUSTER
).$%8
NETWORK COMPONENTS $ATA 0UMP
CLIENT ACCESS NETWORK DATABASE MIGRATING PARAMETERS
MANAGEMENT NETWORK COMPRESSION
PRIVATE NETWORK &,!3("!#+?3#.
/NE#OMMAND &,!3("!#+?4)-%
STAGING &5,, 3#(%-!3 4!",%3
STORAGE CELL CONFIGURATION .%47/2+?,).+
UPGRADES 0!2!,,%, PARAMETER
UPGRADING %XADATA 2%-!0?$!4!&),%
CREATING NEW 2!# CLUSTER 2%-!0?3#(%-!
UPGRADING THE EXISTING CLUSTER 2%-!0?4!",%30!#%
UPLOAD PARAMETERS AND DEPLOYMENT FILES 3#(%-!3
$ATABASE 2ESOURCE -ANAGER $"2- 31,&),%
AD
HOC QUERIES 4!",%?%8)343?!#4)/.
COMPONENTS 4!",%3
CONSUMER GROUPS see #ONSUMER GROUPS 42!.3&/2-
$"! EXPDP COMMAND
$"!?()34?232#?#/.35-%2?'2/50 IMPDP COMMAND
$"!?()34?232#?0,!. LAB CONFIGURATION
$"!?232#?#!4%'/2)%3 /RACLE !DVANCED #OMPRESSION
$"!?232#?#/.35-%2?'2/50?02)63 USAGE TIME
$ATA RECOVERY
$"!?232#?#/.35-%2?'2/503 DIAGNOSTIC TOOL see $IAGNOSTIC TOOLS
$"!?232#?'2/50?-!00).'3 DATABASE BACKUP see $ATABASE BACKUP
$"!?232#?)/?#!,)"2!4% SYSTEM BACKUP see 3YSTEM BACKUP
$"!?232#?-!00).'?02)/2)49 SYSTEM RECOVERY see 3YSTEM RECOVERY
$"!?232#?0,!.?$)2%#4)6%3 $ATA WAREHOUSE $7
$"!?232#?0,!.3 APPLIANCE
$"!?53%23 DATABASE
)/2- SYSTEMS
/,40 APPLICATIONS $"- #ONFIGURATOR
PLAN DIRECTIVES see 0LAN DIRECTIVES CONFIGURATION PROCESS
RESOURCE PLANS see 2ESOURCE PLAN DATA
ENTRY FIELDS
STORAGE CELLS %XADATA UPGRADE
5NIX RENICE COMMAND $"-3?31,45.%2%0/24?31,?-/.)4/2
6232#?#/.35-%2?'2/50 PACKAGE
6232#?0,!. DB?RECOVERY?FILE?DEST DATABASE
6232#?0,!.?()34/29 DCLI COMMAND
6232#?3%33)/.?).&/ ALL?GROUP
63%33)/. CELL?GROUP
WAIT EVENT RESMGR CPU QUANTUM #ELL#,) COMMANDS
$ATABASE 3MART &LASH #ACHE $"&# DBS?GROUP
$ATABASE TEMPLATE DBM$"4EMPLATE PMON PROCESSES
$ATABASE 7RITER $"72 33( EQUIVALENCY
$!4!?#$??CELL GRID DISKS $%#/$% FUNCTION
$!4!?#$??CELL GRID DISKS $EFAULT ALLOCATION POLICY
$ATA MIGRATION STRATEGY $EPLOY
INITORA PARAMETERS COMMAND
LINE OPTIONS PROCESS
LOGICAL MIGRATION see ,OGICAL MIGRATION $"- #ONFIGURATOR
PHYSICAL MIGRATION see 0HYSICAL MIGRATION INSTALLATION PROCEDURE
6,$" MIGRATION DOWNTIME $IAGNOSTICS AND 4UNING 0ACK LICENSES
).$%8
$IAGNOSTIC SCRIPTS CHAINED ROWS
ASSQL BLOCK SHIPPING
CALIBRATE?IOSQL EXAMPLE n
CHECK?PXSQL )/?3!6%$? CALCULATION
COMP?RATIOSQL pass-through MODE
CREATE?DISPLAY?RAWSQL QUERY WITH OFFLOADING VS WITHOUT
DBA?TABLESSQL OFFLOADING
DESCSQL TKRPROF
DISPLAY?RAWSQL COMPRESSION
DPLANSQL DATA FLOW DYNAMICS
DUMP?BLOCKSQL %&3#
ESFC?HIT?RATIOSQL %3&#
ESFC?KEEP?TABLESSQL HINTED CODE
FLUSH?POOLSQL INDEXES
FLUSH?SQLSQL PARTITIONING
FSSQL ROW AT A TIME PROCESSING
FSXSQL 3MART 3CANS
FSXOSQL CODING TECHNIQUES
GATHER?TABLE?STATSSQL DIRECT PATH READ MECHANISM
GET?COMPRESSION?RATIOSQL %80,!). 0,!. TYPE TOOL
GET?COMPRESSION?TYPESQL FSXOSQL SCRIPT
MYSTATSSQL LONG RUNNING STATEMENTS
OBJ?BY?HEXSQL OFFLOAD?PERCENTSQL SCRIPT
OFFLOAD?PERCENTSQL ONLINE CODE REPOSITORY
PARMSDSQL OPTIMIZER
PARMSSQL VERY BUSY CELLS
PART?SIZESQL 7(%2% CLAUSES
POOL?MEMSQL CUSTOM WRITTEN 0,31, FUNCTIONS
QUEUED?SQLSQL OFFLOADABLE FUNCTION
REPORT?SQL?MONITORSQL STORAGE INDEXES
SISQL $YNAMIC PERFORMANCE COUNTERS
SNAPPERSQL CELL BLOCKAGE
SS?OFFSQL COMMIT CACHE
SS?ONSQL -IN3#. OPTIMIZATION
TABLE?SIZESQL CELL BLOCKS PROCESSING
VALID?EVENTSSQL CACHE LAYER
WHOAMISQL DATA LAYER
$IAGNOSTIC TOOLS INDEX LAYER
#ELL#,) see #ELL#,) TXN LAYER
(EALTHCHECK CELL COMMIT CACHE QUERIES
!3- HEALTH CHECK CELL FLASH CACHE
/3 HEALTH CHECK CELL )/ UNCOMPRESSED BYTES
3UN $IAGNOSTICS see SUNDIAGSH OUTPUT CELL INDEX SCANS
$ISK BASED BACKUPS CELL NUM FAST RESPONSE SESSIONS
$ISK DRIVE FRONT PANEL SWITCH TO SMART SCAN SESSION
$)3+-/. USING PASSTHRU MODE
$ISK STATUS CELL PHYSICAL )/ BYTES
$ISK TRACKS EXCESSIVE #05 ON CELL
$7") WORKLOADS INTERCONNECT BYTES
$7 ORIENTED WORKLOADS OPTIMIZED FILE CREATION
#!3% STATEMENTS OPTIMIZED 2-!. FILE RESTORE
).$%8
PREDICATE OFFLOAD )NFINI"AND NETWORK
SENT DIRECTLY TO $" NODE TO BALANCE #05 /,40 MACHINE
USAGE OPERATING SYSTEMS
STORAGE INDEX SOFTWARE COMPONENTS
CELL SCANS DATABASE SERVERS
CELL SMART )/ SESSION STORAGE SERVER
CACHE HITS STORAGE CELLS
CACHE LOOKUPS STORAGE LAYER
CELL TRANSACTIONS IN COMMIT CACHE STORAGE SERVERS
CHAINED ROWS 8 DATABASE SERVERS
PROCESSED BY CELL 8 DATABASE SERVERS
REJECTED BY CELL %XADATA CONFIGURATION PARAMETERS
SKIPPED BY CELL BACKUP METHOD
MEANING AND EXPLANATION EXTERNAL TO %XADATA
/RACLE COUNTERS INTERNAL TO %XADATA
/RACLE 3ESSION 3NAPPER DATABASE MACHINE NAME
PHYSICAL READ REQUESTS OPTIMIZED DISK TYPE
PHYSICAL READ TOTAL BYTES OPTIMIZED PROTECTION LEVEL
08 SLAVE SESSIONS 3MART &LASH #ACHE
3TATSPACK!72 REPORTS TYPE OF SYSTEM
TABLE FETCH CONTINUED ROW WORKLOAD TYPE
TABLE SCANS %XADATA DATABASE MACHINE
DIRECT READ 8
LONG TABLES 8
TROUBLESHOOTING PERFORMANCE %XADATA (YBRID #OLUMNAR #OMPRESSION %(##
%XADATA 3MART &LASH #ACHE %3&#
E $"&#
%JECTED DISK DRIVE FLASH CACHE CREATION
%NERGY STORAGE MODULE %3- FLASH DISK
%XADATA FLASH MEMORY n
ARCHITECTURE HARDWARE
!3- MONITORING
$"2- vs )/2- DATABASE LAYER
$)3+-/. STORAGE LAYER
,)"#%,, PERFORMANCE
2$3 PROTOCOL USAGE CONTROL
3YSTEM 'LOBAL !REA %XADATA 6
BITS AND PIECES %XPERT /RACLE %XADATA "LOG
CELL SECURITY %XPORT AND )MPORT METHOD
CONSOLIDATION PLATFORM "5&&%2 PARAMETER
DATABASE LAYER #/-02%33)/.
DATABASE MACHINE DIRECT PATH LOAD INSERTS
8 (## COMPRESSION
8 )'./2% PARAMETER
DISKS ).$%8&),% PARAMETER
$7 APPLIANCE /RACLE !DVANCED #OMPRESSION
FLASH CACHE PARALLELIZING PROCESS
HARDWARE COMPONENTS 2%-!0?3#(%-!
HISTORY 4RANSPARENT $ATA %NCRYPTION
USAGE TIME
).$%8
%XTERNAL 53" DRIVE RECOVERY IMAGE LOAD PERFORMANCE
IMAGEINFO COMMAND QUERY PERFORMANCE
MAKE?CELLBOOT?USB COMMAND
SOFTWARE INSTALLATION
IN PARTITION PATCHES I, J
OUT OF PARTITION UPGRADES )NDEX FAST FULL SCAN )&&3
%XTRACT TRANSFORM AND LOAD %4, PROCESS )NDEX /RGANIZED 4ABLE )/4 SEGMENT
)NDEX TRICKERY
)NFINI"AND PRIVATE NETWORK
F )N MEMORY 0ARALLEL %XECUTION
CLUSTER AWARE FEATURE
&AST RECOVERY AREA &2! DISABLING FEATURE n
&IRSTBOOT PROCESS FSXSQL SCRIPT OUTPUT
&IRST IN FIRST OUT &)&/ MEMORY ACCESS
&LASH #ACHE PARAMETER PERTINENT INFORMATION n
&LASH GRID DISKS In-memory parallel query
&LASH RECOVERY AREA &2! )NSTANCE CAGING
&UNCTION SHIPPING ARCHITECTURE CONFIGURING AND TESTING
BURN?CPUSH SCRIPT
#05?#/5.4 PARAMETER
G DEFAULT_PLAN resource allocation
G2 PARALLEL FEATURES /4(%2?'2/503 RESOURCE GROUP
2%3/52#%?0,!. DATABASE PARAMETER
H 3#2!4#( AND 3.)&& DATABASES
(## TEST SUMMARY
(4-, FILE CPU affinity
(UMAN RESOURCES (2 DATABASE FOREGROUND AND BACKGROUND PROCESSES
(YBRID #OLUMNAR #OMPRESSION ,'72
CHALLENGES MULTITENANT DATABASE ENVIRONMENTS
DISABLING SERIAL DIRECT PATH READS OVER PROVISIONING
LOCKING ISSUES 0-/.
MOVING DATA TO A NON %XADATA PLATFORM )NTEGRATED ,IGHTS /UT -ANAGER ),/-
)NTELLIGENT $ATABASE PROTOCOL I$"
SINGLE ROW ACCESS Interleaving allocation policy
COMPRESSION RATIOS )/ PAYLOAD
#OMPRESSION !DVISOR )/ 2ESOURCE -ANAGER )/2-
CUSTOM APPLICATION DATA !003?#!4%'/29 AND "!4#(?#!4%'/29
FINANCIAL DATA
REAL WORLD APPLICATIONS ARCHITECTURE
RETAIL SALES DATA BACKGROUND PROCESSES
TELECOM CALL DETAIL DATA CELLSRV ISSUES
COMPRESSION TYPES CONFIGURING CATEGORY
!2#()6% ()'( CALCULATIONS
!2#()6% ,/7 #ELL#,) COMMAND LIST
15%29 ()'( $"!?232#?#/.35-%2?'2/503 VIEW
15%29 ,/7
COMPRESSION UNITS #5S DB0LAN FIELD
$-, PERFORMANCE
),- STRATEGIES
).$%8
/4(%2?'2/503 CONSUMER GROUP IORMPLAN OBJECTIVE COMMAND
LOW?LATENCY
2%0/243 AND -!).4%.!.#% RESOURCE SMALL AND LARGE )/ REQUESTS
GROUPS )/2- See )/ 2ESOURCE -ANAGER )/2-
STORAGE CELLS )/S PER SECOND )/03
%4, PROCESS )0#ONF
%XADATA
FAST FILE CREATION
&)&/ K
HIERARCHY +ERRY /SBORNES /RACLE "LOG
I$" +EYBOARD VIDEO AND MOUSE +6-
INTERDATABASE
CONFIGURING INTERDATABASE
HIGHEST PRIORITY DATABASE L
)/ PRIORITIES
)/ REQUEST ,IMITED PARALLEL DEGREE POLICY
STORAGE CELL AND GRID ,INUX OPERATING SYSTEM
INTRADATABASE ,OGICAL MIGRATION
CALCULATIONS DATA COPYING OVER DATABASE LINK
CONFIGURING INTRADATABASE !00%.$ HINT
-'-4?0n ATTRIBUTES #4!3 USAGE
)/ ALLOCATION %XADATA %THERNET AND )NFINI"AND PORTS
METRICS
CATEGORY ).3%24 3%,%#4 COMMAND
#ELL#,) COMMAND /RACLE .ET 31,
.ET
CELLSRV PARALLELIZING DATA LOAD see 0ARALLELIZING
CONSUMER GROUP DATA LOAD
DATABASE SOFTWARE CONFIGURATION SETTINGS
-%42)##522%.4 OBJECT TABLE COPYING
-%42)#()34/29$!93 ATTRIBUTE 4#0 BUFFER SIZES n
-%42)#()34/29 OBJECT EXTRACT AND LOAD METHOD
3#2!4#( AND 3.)&& DATABASES ADVANTAGE
STORAGE CELLS $ATA 0UMP see $ATA 0UMP
WORKLOAD MANAGEMENT see 7ORKLOAD %XPORT AND )MPORT METHOD see %XPORT
MANAGEMENT AND )MPORT METHOD
MONITORING PLATFORM DIFFERENCES
/,40 APPLICATIONS DATABASE MERGING
PLAN TESTING EXTENT SIZING
ACTIVATE RESOURCE MANAGER (##
CELLSRV REPLICATION BASED METHOD see 2EPLICATION
DUMP METRICS TO FILE "ASED MIGRATION
3#2!4#( AND 3.)&& DATABASES SELECTIVE MIGRATION
START LOAD TEST STAGED MIGRATION
RESOURCE ALLOCATION ,OG 7RITER ,'72 PROCESS
2MAN
UTILIZATION
WORKLOAD OPTIMIZATION
M
!UTO Management network
"ALANCED ),/-
$33/,40 WORKLOAD +6-
HIGH?THROUGHPUT -ANAGEMENT 3ERVER -3
).$%8
-ANUAL PARALLEL DEGREE POLICY DATABASE
-INIMUM !CTIVE 3#. OPTIMIZATION MACHINE
-IXED WORKLOADS /RACLE
HINTS COMPRESSION MECHANISMS
INDEX "!3)# COMPRESSION
OPTIMIZER (## COMPRESSION
INDEX ORIENTED PLANS /,40 COMPRESSION
PARAMETERS n DATABASE
SCAN BASED PLANS DATA STORAGE
STORAGE PLATFORM BLOCK FORMAT
2ESOURCE -ANAGER 0#4&2%%
SYSTEM STATISTICS ROW FORMAT
-/3 SUPPORT NOTES ROW MIGRATION
-Y /RACLE 3UPPORT -/3 NOTE ENVIRONMENT PARAMETERS
/RACLE #LUSTERWARE
/RACLE G2
/RACLE G2
N /RACLE 3ESSION 3NAPPER
.ETWORK SETTINGS
CELL ALERT NOTIFICATION PARAMETERS
CHANNEL BONDING .%4
P, Q
CLIENT ACCESS NETWORK .%4 0!2!,,%,?#,534%2?#!#(%?0/,)#9
MANAGEMENT NETWORK .%4 PARAMETER
.ON /RACLE DATABASES 0!2!,,%,?$%'2%%?0/,)#9
.ORMAL REDUNDANCY 0ARALLELIZING DATA LOAD
FULLY PARALLEL NETWORK DATA TRANSFER
O PARALLEL DATA LOAD n
/#2 AND VOTING FILE STORAGE 0ARALLEL OPERATIONS
/FFLOADING OPTIMIZATIONS AUTO $/0 see !UTOMATIC DEGREE OF
/,40 ORIENTED WORKLOADS PARALLELISM
%3&# PARALLEL QUEUING FUNCTIONALITY
FLASH CACHE #05 UTILIZATION
LATENCY ISSUES MIXED WORKLOAD
SCALABILITY /RACLES PARALLEL CAPABILITIES
WRITE INTENSIVE /,40 WORKLOADS PARALLEL ADAPTIVE MULTIUSER
/N DISK INDEXING TECHNOLOGY 0!2!,,%,?3%26%23?4!2'%4
/NE#OMMAND STORAGE LAYER
CHECKIPSH SCRIPT 0ARALLEL QUEUING FUNCTIONALITY
CONFIGURATION STEPS CONTROLLING MECHANISM
DATABASE TEMPLATE ./?34!4%-%.4?15%5).' HINT n
DEPLOYSH
-Y /RACLE 3UPPORT -/3 NOTE /RACLES $ATABASE 2ESOURCE -ANAGER
NETWORK SETTINGS $"2- n
SIMPLIFIED AND STRUCTURED MECHANISM 0!2!,,%,?3%26%23?4!2'%4
STANDARDIZED CONFIGURATIONS PARAMETER
/NLINE TRANSACTION PROCESSING /,40 0!2!,,%,?34!4%-%.4?15%5).'
COMPRESSION METHOD PARAMETER n
CHARACTERISTICS 34!4%-%.4?15%5).' HINT
INDIRECT PATH LOADS
).$%8
#05 UTILIZATION 2EDO !PPLY
MIXED WORKLOAD 2ELIABLE $ATAGRAM 3OCKETS 2$3 PROTOCOL
/RACLE 2EPLICATION BASED MIGRATION
PARALLEL ADAPTIVE MULTIUSER LOGICAL REPLICATION
0!2!,,%,?3%26%23?4!2'%4 LOGICAL STANDBY
0ASSTHROUGH NON /RACLE DATABASE
0AYROLL 0!9 DATABASE /RACLE 3TREAMS AND 'OLDEN 'ATE n
0#4&2%% PARALLEL LOAD
BLOCK FREE SPACE CONTROL PHYSICAL REPLICATION
PURPOSES 2ESOURCE MANAGEMENT
ROW MIGRATION CONSOLIDATION
0ERFORMANCE METRICS $"2- see $ATABASE 2ESOURCE -ANAGER
DYNAMIC PERFORMANCE COUNTERS INSTANCE CAGING see )NSTANCE CAGING
INTERNALS )/2- see )/ 2ESOURCE -ANAGER
/RACLE 3ESSION 3NAPPER 2ESOURCE PLANS
3MART 3CAN ACTIVATION
METRICS AND PERFORMANCE COUNTERS AFTER HOURS PLAN
PERFORMANCE CONSUMER GROUP MAPPING RULES
63%334!4 MYPLAN
0HYSICAL MIGRATION .)'(44)-% PLAN
!3- REBALANCE pending area
BACKUP AND RESTORE STRATEGY AND PLAN DIRECTIVES
BLOCK CHANGE TRACKING FILE RESOURCE GROUP MAPPING PRIORITIES
FULL BACKUP AND RESTORE RESOURCE GROUPS CREATION
INCREMENTAL BACKUP 2%3/52#%?-!.!'%2?0,!. INSTANCE
PHYSICAL STANDBY STRATEGY PARAMETER
POST MIGRATION TASKS TESTING
SOURCE DATABASE VERSION ACTIVATE RESOURCE MANAGER
443 see 4RANSPORTABLE TABLESPACES CHECK CONSUMER GROUP ASSIGNMENTS
0LAN DIRECTIVES
#2%!4%?0,!.?$)2%#4)6% PROCEDURE CHECK $"2- METRICS
$!94)-% RESOURCE PLAN ALLOCATION
MANAGEMENT ATTRIBUTES DEACTIVATE $"2-
-'-4?0n EXAMINE SESSION TRACE FILE
0RIVATE NETWORK EXECUTE A QUERY
0ROCESS MONITOR 0-/. LOG IN AS 2*/(.3/.
SESSION TRACE
START LOAD TEST
R 6232#?#/.35-%2?'2/50 VIEW
2EAL TIME 31, MONITORING See also 31,
MONITORING 2ESTART 3ERVER 23
$"-3?31,45.% runtime ATTRIBUTES
EXECUTION PLAN ROWSOURCE
ACTIVITY MONITORING
ACTIVITY TIMELINE
S
/RACLE EXECUTION PLANS 3#2!4#(?$'
08 2%#%)6% ROWSOURCE 3+%7
TABLE ACCESS AND SORTING (## LOAD TIMES vs COMPRESSION RATIOS
MANUAL QUERY statistics gathering timing
2ECOVERY MANAGER 2MAN UNQUALIFIED QUERY TIMING
).$%8
3MART 3CAN LSSCSI COMMAND
METRICS AND PERFORMANCE COUNTERS /RACLE CELLSRV
#05 USAGE 3!. STORAGE NETWORK
MULTIBLOCK READS SMALL )/ REQUEST
SINGLE BLOCK READS SYNCHRONOUS AND ASYNCHRONOUS )/
PERFORMANCE /3 7ATCHER
BITMAP INDEX SEGMENTS ARCHIVE FILES OUTPUT
"
4REE INDEX SEGMENTS BENEFITS
COLUMN PROJECTION BZIPPED FILE
FILTRATION CELLSRVSTAT COMMAND
FULL SEGMENT SCANS COLLECTED DATA STORAGE
INDEX FAST FULL SCAN IOSTAT ARCHIVE
)NDEX /RGANISED 4ABLE )/4 SEGMENT LITTLE SCRIPT
LOGFILES
MATERIALIZED VIEW SEGMENTS LOG FILE WRITE LATENCY PROBLEM
PREDICATE OFFLOAD -Y /RACLE 3UPPORT VERSION
STORAGE PREDICATE NAME SEARCH
3PINE SWITCH OUTPUT SORTING
31, MONITORING 0ERL TEXT PROCESSING
CONTROL POST PROCESSING
DATABASE TIME PSTREE COMMAND
DURATION VS #05 TIME 3TORAGE CELLS CONFIGURATION
%NTERPRISE -ANAGER PERFORMANCE 3TORAGE )NDEXES
'RID #ONTROL ?#%,,?34/2)$8?-/$% PARAMETER
31, EXECUTIONS ??+#&)3?34/2!'%)$8?$)!'?-/$%
FEATURES PARAMETER
PARALLEL QUERY ?KCFIS?STORAGEIDX?DISABLED PARAMETER
REAL TIME MONITORING SINGLE LONG RUNNING BIND VARIABLES
QUERY CELLINITORA 3TORAGE )NDEX PARAMETERS
BIND VARIABLE VALUES ENCRYPTION
#ELL /FFLOAD %FFICIENCY RATIO FEATURES AND SYNTAX PREVENTION
)/ 3TATISTICS (##
-ONITORED 31, %XECUTION $ETAILS JOINS
OVERVIEW SECTION AT LEAST ONE PREDICATE
4IME AND 7AIT 3TATISTICS LIMITATIONS
631, AND 631,34!43 MONITORING
31, STATEMENTS PERFORMANCE METRICS DATABASE STATISTICS
%XADATA SPECIFIC PERFORMANCE STATISTICS TRACING MECHANISM
631, COLUMNS AND MEANINGS WRAP UP
3TORAGE CELL LAYER MONITORING MULTI COLUMN PREDICATES
CELL METRICS ASSESSMENT NULL OPTIMIZATION
CELL LAYER CELL#,) OPERATORS
#05 UTILIZATION PARALLEL QUERY
GRID CONTROL %XADATA 3TORAGE 3ERVER PARTITIONED OBJECTS
/RACLE PERFORMANCE BENEFITS
)/ METRICS IOSTAT POTENTIAL ISSUES
DISK SERVICE TIME INCOMPATIBLE CODING TECHNIQUES
)/ RESPONSE TIME INCORRECT RESULTS
)/ WAITING QUEUEING TIME MOVING TARGETS
,INUX IOSTAT PARTITION SIZE
,3) DISKS 3MART 3CAN
).$%8
STRUCTURE STORAGE CELL
SUB QUERY DISK FAILURE see #ELL DISK FAILURE
Storage network configuration FLASH CACHE FAILURE
3TRIPE AND MIRROR EVERYTHING 3!-% PERMANENT CELL FAILURE
METHODOLOGY SYSTEM VOLUME FAILURE
SUNDIAGSH OUTPUT TEMPORARY CELL FAILURE
DMESG VALIDATION FRAMEWORK
FDISK L
LOG FILES
ALERTLOG T
ALERTHISTORY 4ANEL 0ODERS "LOG
CELL DETAIL 4APE BASED BACKUPS
CELLDISK DETAIL 4EMPORARY CELL FAILURE
FDOM L 4RANSPORTABLE TABLESPACES 443
FLASHCACHE DETAIL CROSS PLATFORM 443
GRIDDISK DETAIL ENDIAN FORMAT CONVERSION
LUN DETAIL RESTRICTIONS
MS ODLLOG 42!.30/24?&5,,?#(%#+ PARAMETER
MS ODLTRC
PHYSICALDISK DETAIL
PHYSICALDISK FAIL U
LSPCI
LSSCSI 5NBREAKABLE %NTERPRISE +ERNEL 5%+
MEGACLI 5SER )/ CLASS
MESSAGES CELL LIST OF BLOCKS PHYSICAL READ EVENT
SEL LIST CELL MULTIBLOCK PHYSICAL READ EVENT
3UN & 0#)E &LASH #ARD CELL SINGLE BLOCK PHYSICAL READ EVENT
3YSTEM BACKUP ACTIVE PRODUCTION SYSTEM
DATABASE SERVER CELL HASH NUMBER
,6- SNAPSHOTS DISK BASED STORAGE SYSTEMS
STORAGE CELL DISK HASH NUMBER
#%,,"//4 53" FLASH DRIVE FLASH CACHE
EXTERNAL 53" DRIVE HISTOGRAM
3YSTEM )/ CLASS INDEX ACCESS PATHS
CELL SMART INCREMENTAL BACKUP CELL SMART FILE CREATION
CELL HASH NUMBER CELL SMART INDEX SCAN EVENT
)NCREMENTAL ,EVEL BACKUP AGGRESSIVENESS
)NCREMENTAL ,EVEL BACKUP CELL HASH NUMBER
2-!. COMMAND FAST FULL INDEX SCANS
TRACE FILE OBJECT )$
CELL SMART RESTORE FROM BACKUP SERIAL?DIRECT?READ 425%
CELL HASH NUMBER TRACE FILE
FILE INITIALIZATION CELL SMART TABLE SCAN EVENT
2-!. RESTORES ALLOCATION UNITS
TRACE FILE CELL HASH NUMBER
MISCELLANEOUS 3YSTEM )/ CLASS EVENTS &ULL 4ABLE 3CANS
3YSTEM RECOVERY )NFINI"AND NETWORK
DATABASE SERVERS OBJECT )$
,6- SNAPSHOT BASED BACKUP TRACE FILE
REIMAGING CELL STATISTICS GATHER
MULTI BLOCK READ MECHANISMS
).$%8
5SER )/ CLASS cont RESOURCE MANAGER EVENTS
PHYSICAL )/ EVENTS RESMGRCPU QUANTUM EVENT
SINGLE
BLOCK READ MECHANISMS RESMGRPQ QUEUED EVENT
6%6%.4?.!-%
STEPS PLANNING TRIGGER EVENTS
V 3YSTEM )/ CLASS see 3YSTEM )/ CLASS
631, AND 631,34!43 VIEWS 5SER )/ CLASS see 5SER )/ CLASS
VEVENT?NAME
7ORKLOAD MANAGEMENT
W, X, Y, Z #'?&#?)/?21?,'
#'?&$?)/?21?,'
7AIT EVENTS #'?)/?21?,'
CELL EVENTS consumer_type attribute
CELL SMART INCREMENTAL BACKUPS device_type attribute
CELL SMART RESTORE FROM BACKUPS $33 WORKLOAD
DISCRETE OPERATING SYSTEM CALLS )/2-?-/$%
G2 CODE -%42)##522%.4 AND -%42)#()34/29
)DLE #LASS EVENTS OBJECTS
OLD EVENTS METRIC/BJECT.AME ATTRIBUTE
DIRECT PATH READ NAME ATTRIBUTE OF )/2- -%42)##522%.4
ENQ+/FAST OBJECT CHECKPOINT
RELIABLE MESSAGE /,40 WORKLOAD
/RACLE DATABASE STORAGE CELLS
You might also like
- Perform basic tasks in Oracle MultitenantDocument18 pagesPerform basic tasks in Oracle MultitenantVinod Kumar KannieboinaNo ratings yet
- Databricks Apache Spark Certified Developer Master Cheat SheetDocument29 pagesDatabricks Apache Spark Certified Developer Master Cheat SheetrajikareNo ratings yet
- Oracle Database 12c R2: Administration Workshop Ed 3: DurationDocument6 pagesOracle Database 12c R2: Administration Workshop Ed 3: DurationBugz Binny100% (1)
- Oracle® Database 2 Day Performance Tuning GuideDocument240 pagesOracle® Database 2 Day Performance Tuning Guidepepet1000No ratings yet
- Oracle9i Database - Advanced Backup and Recovery Using RMAN - Student GuideDocument328 pagesOracle9i Database - Advanced Backup and Recovery Using RMAN - Student Guideacsabo_14521769No ratings yet
- Oracle: SceneDocument56 pagesOracle: Sceneelcaso34No ratings yet
- 11g SQL Tuning Workshop Volume 2 PDFDocument470 pages11g SQL Tuning Workshop Volume 2 PDFRupam ChowdhuryNo ratings yet
- Oracle QsDocument6 pagesOracle QshellopavaniNo ratings yet
- Kanna Technologies - DBA ContentDocument3 pagesKanna Technologies - DBA ContentJ.B.BABUNo ratings yet
- Use Only Oracle Internal & Oracle Academy Use Only: Oracle SOA Suite 11g: Essential ConceptsDocument404 pagesUse Only Oracle Internal & Oracle Academy Use Only: Oracle SOA Suite 11g: Essential ConceptssureshghugeNo ratings yet
- Oracle 12c MultitenantDocument101 pagesOracle 12c Multitenantnarendrabalasa8633100% (2)
- Installing Oracle Application Server 11g R1 On CentOSDocument13 pagesInstalling Oracle Application Server 11g R1 On CentOSNguyen Quoc Huy100% (3)
- PySpark Reference GuideDocument2 pagesPySpark Reference GuideTarun SinghNo ratings yet
- Pro Oracle SQL Development: Best Practices for Writing Advanced QueriesFrom EverandPro Oracle SQL Development: Best Practices for Writing Advanced QueriesNo ratings yet
- Database Vault Administration 102Document318 pagesDatabase Vault Administration 102Aide ArceNo ratings yet
- Getting Started With Spark Redis PDFDocument9 pagesGetting Started With Spark Redis PDFAdam Sánchez Ayte0% (1)
- Oracle Data Integrator 12cDocument9 pagesOracle Data Integrator 12cNarayana AnkireddypalliNo ratings yet
- Performance Tuning - Expert Panel PDFDocument135 pagesPerformance Tuning - Expert Panel PDFAmit VermaNo ratings yet
- Getting Started With Amazon DocumentdbDocument28 pagesGetting Started With Amazon DocumentdbSpit Fire100% (1)
- Best Practices for Data Prep on Snowflake's Cloud Data WarehouseDocument8 pagesBest Practices for Data Prep on Snowflake's Cloud Data WarehouseTrue callerNo ratings yet
- Gridlink RacDocument21 pagesGridlink RacAngel Freire RamirezNo ratings yet
- Mongodb: Ravikumer MaddiDocument48 pagesMongodb: Ravikumer MaddiRavikumar MaddiNo ratings yet
- 0396 Oracle Goldengate 12c TutorialDocument28 pages0396 Oracle Goldengate 12c TutorialPrinceNo ratings yet
- Oracle Database 11g SQL FundamentalsDocument546 pagesOracle Database 11g SQL FundamentalsAvinash SinghNo ratings yet
- Exadata Awr 5100655 PDFDocument23 pagesExadata Awr 5100655 PDFRam RamNo ratings yet
- High Availability Case StudiesDocument2 pagesHigh Availability Case StudiesAlberto Hernandez HernandezNo ratings yet
- E49321-16 - Manual Data Base Oracle 12cDocument100 pagesE49321-16 - Manual Data Base Oracle 12cCharlesEduardoNo ratings yet
- Multitenant Administrators GuideDocument627 pagesMultitenant Administrators GuidePaohua ChuangNo ratings yet
- D56497GC20 - 1001 - US - Oracle Data Modeling and Relational Database DesignDocument5 pagesD56497GC20 - 1001 - US - Oracle Data Modeling and Relational Database DesignMNo ratings yet
- Exploring Oracle Internals PDFDocument2 pagesExploring Oracle Internals PDFTimothy0% (1)
- Understanding Oracle SCN for RecoveryDocument9 pagesUnderstanding Oracle SCN for RecoveryRahul KumarNo ratings yet
- 52492-rc071 Postgresql PDFDocument11 pages52492-rc071 Postgresql PDFJedt3DNo ratings yet
- Obiee 12CDocument6 pagesObiee 12CDilip Kumar AluguNo ratings yet
- ORACLE 12C New FeaturesDocument51 pagesORACLE 12C New FeaturesThangavelu Agathian100% (1)
- FY-Linux Lab ManualDocument16 pagesFY-Linux Lab ManualDeepak SimaiyaNo ratings yet
- Oracle Database Clusterware and Services Startup and Shutdown CommandsDocument2 pagesOracle Database Clusterware and Services Startup and Shutdown CommandskarthikdhadalaNo ratings yet
- Administering Oracle Database Classic Cloud Service PDFDocument425 pagesAdministering Oracle Database Classic Cloud Service PDFGanesh KaralkarNo ratings yet
- Girishpote DBA KUWAITDocument7 pagesGirishpote DBA KUWAITMusa TahirliNo ratings yet
- Oracle Database 12cR2 New FeaturesDocument48 pagesOracle Database 12cR2 New FeaturesConstantin CaiaNo ratings yet
- Oracle PL/SQL Course - Learn Database ProgrammingDocument26 pagesOracle PL/SQL Course - Learn Database ProgrammingchensuriyanNo ratings yet
- D56261GC10 Appendix A DWDocument130 pagesD56261GC10 Appendix A DWHishamAlruainiNo ratings yet
- All About TransactionScope - CodeProjectDocument16 pagesAll About TransactionScope - CodeProjectGrace Patiño PerezNo ratings yet
- Informix AdminDocument471 pagesInformix AdminabidouNo ratings yet
- Challenges of Distributing Postgres: A Citus Story: Ozgun ErdoganDocument47 pagesChallenges of Distributing Postgres: A Citus Story: Ozgun ErdoganNguyen N. PhanNo ratings yet
- Oracle 9i - Database Supporting Recovery Manager - RMAN - Student GuideDocument330 pagesOracle 9i - Database Supporting Recovery Manager - RMAN - Student Guidesrav2010No ratings yet
- 1z0-066.exam: Number: 1z0-066 Passing Score: 800 Time Limit: 120 Min File Version: 1Document64 pages1z0-066.exam: Number: 1z0-066 Passing Score: 800 Time Limit: 120 Min File Version: 1Rosa JiménezNo ratings yet
- Oracle DataGuard Step by Step Installation With RMAN Duplicate Method-1Document4 pagesOracle DataGuard Step by Step Installation With RMAN Duplicate Method-1knugroho1982No ratings yet
- Installing EBS R12Document41 pagesInstalling EBS R12VenkateshNo ratings yet
- Oracle Database 11g - Administration Workshop IIDocument5 pagesOracle Database 11g - Administration Workshop IIsonicefuNo ratings yet
- Oracle Database 12c Global Data Services PDFDocument50 pagesOracle Database 12c Global Data Services PDFSrinivas EllendulaNo ratings yet
- Oracle Rac & GridDocument6 pagesOracle Rac & GridDheeraj SinghNo ratings yet
- Oracle RAC DBA Training :DBA TechnologiesDocument29 pagesOracle RAC DBA Training :DBA TechnologiesDBA Technologies0% (1)
- Essbase 11 Bootcamp - Table of ContentsDocument12 pagesEssbase 11 Bootcamp - Table of ContentsBalaji VarathriNo ratings yet
- Oracle EBS (E-Business Suite 12.2.6) Step by Step Installation - 4 Final - IT TutorialDocument1 pageOracle EBS (E-Business Suite 12.2.6) Step by Step Installation - 4 Final - IT TutorialCarlos HernandezNo ratings yet
- MV Refresh ParallelDocument37 pagesMV Refresh ParallelBiplab ParidaNo ratings yet
- Release 12 Apps DBA 101: John Stouffer Independent ConsultantDocument61 pagesRelease 12 Apps DBA 101: John Stouffer Independent Consultantjagatheeshr@gmail.comNo ratings yet
- 12c DB Upgrade v1.5Document71 pages12c DB Upgrade v1.5gloprestiNo ratings yet
- (Ebook - PDF) Oracle Enterprise Dba Part 1B - Backup and Recovery Workshop, Volume 2Document326 pages(Ebook - PDF) Oracle Enterprise Dba Part 1B - Backup and Recovery Workshop, Volume 2jonaswpNo ratings yet
- Tuning The Redolog Buffer Cache and Resolving Redo Latch ContentionDocument5 pagesTuning The Redolog Buffer Cache and Resolving Redo Latch ContentionMelissa MillerNo ratings yet
- OBIEE - Aggregate TablesDocument3 pagesOBIEE - Aggregate Tablesvenkatesh.gollaNo ratings yet
- ORACLE 12C NEW FEATURES GUIDEDocument240 pagesORACLE 12C NEW FEATURES GUIDEvbpanchalNo ratings yet
- Oracle GGDocument34 pagesOracle GGvbpanchalNo ratings yet
- A Technical Overview of New Features For Automatic Storage Management in Oracle Database 12cDocument14 pagesA Technical Overview of New Features For Automatic Storage Management in Oracle Database 12cvbpanchal100% (2)
- ITIL Foundation Study NotesDocument25 pagesITIL Foundation Study NotesThought Rock92% (12)
- Rac For BeginnersDocument37 pagesRac For BeginnersvbpanchalNo ratings yet
- Oracle 11g NF 1.0Document120 pagesOracle 11g NF 1.0Ramu SplNo ratings yet
- Lect 18 - Binary Addition and SubtractionDocument18 pagesLect 18 - Binary Addition and SubtractionvbpanchalNo ratings yet
- Installation Guide Cloud Control 12c On OEL5U7Document19 pagesInstallation Guide Cloud Control 12c On OEL5U7vbpanchalNo ratings yet
- GCOUG GoldengateDocument29 pagesGCOUG GoldengatevbpanchalNo ratings yet
- GG - SQL 2008Document22 pagesGG - SQL 2008vbpanchalNo ratings yet
- Oracle Databases On VMware - Best Practices GuideDocument26 pagesOracle Databases On VMware - Best Practices GuidevbpanchalNo ratings yet
- TocDocument20 pagesTocrajscribd12345No ratings yet
- Home Loan EMICalculatorDocument1 pageHome Loan EMICalculatorvbpanchalNo ratings yet
- Inst 10g On Rhel4Document65 pagesInst 10g On Rhel4kappi4uNo ratings yet
- Sorting AlgorithmsDocument14 pagesSorting AlgorithmsNonstop MasterNo ratings yet
- Ankit PYTHON LABDocument17 pagesAnkit PYTHON LABMic CavNo ratings yet
- Survey On: Multimedia Content Protection Using Cloud: A. M. ChavanDocument3 pagesSurvey On: Multimedia Content Protection Using Cloud: A. M. ChavanAnonymous lPvvgiQjRNo ratings yet
- Abinash ResumeDocument3 pagesAbinash ResumeengineeringwatchNo ratings yet
- Mechanical Engineering ResumeDocument2 pagesMechanical Engineering ResumeZohaib KhanNo ratings yet
- High Performance Grid Computing IntroductionDocument37 pagesHigh Performance Grid Computing IntroductionPrathameshNo ratings yet
- Letter of Motivation - Computer ScienceDocument2 pagesLetter of Motivation - Computer ScienceHasibul Kabir EmonNo ratings yet
- Security Advisory: TBOX-SA-2021-0007Document5 pagesSecurity Advisory: TBOX-SA-2021-0007edisonpcNo ratings yet
- US5GMe Carrier Network SynchronizationDocument3 pagesUS5GMe Carrier Network SynchronizationMohammed El-AwamryNo ratings yet
- HMC Ibm Z PDFDocument372 pagesHMC Ibm Z PDFrithwikNo ratings yet
- ShootingsmDocument6 pagesShootingsmapi-398062839No ratings yet
- Xplore 1.3, 1.4, 1.5Document8 pagesXplore 1.3, 1.4, 1.5subhakarlokeshoffersNo ratings yet
- (4.0) 3609.9705.22 - DB5400 - 6179.2985.10 - VCMS - and - RRMC - Server - Specification - v0100Document16 pages(4.0) 3609.9705.22 - DB5400 - 6179.2985.10 - VCMS - and - RRMC - Server - Specification - v0100Nguyen Xuan NhuNo ratings yet
- Tcs CDocument5 pagesTcs CtinaNo ratings yet
- User Request Form V4 - 2Document1 pageUser Request Form V4 - 2Thofan ArisNo ratings yet
- Command-Line Basics for Deep Security AutomationDocument12 pagesCommand-Line Basics for Deep Security AutomationaforabadNo ratings yet
- Hikvision Installation Manual of Networsjhsu9dkkso Camera V3.0.0Document23 pagesHikvision Installation Manual of Networsjhsu9dkkso Camera V3.0.0Hd1998No ratings yet
- AWS Networking Course SlidesDocument177 pagesAWS Networking Course SlidesShan AnandNo ratings yet
- Avionic Computer FV-4000Document2 pagesAvionic Computer FV-4000Thomas KrauseNo ratings yet
- Erased Log by SosDocument2 pagesErased Log by SosStefanNo ratings yet
- S2A6620 User GuideV1.2Document164 pagesS2A6620 User GuideV1.2sanajy15880No ratings yet
- Exchange On-Premises Modern AuthenticationDocument50 pagesExchange On-Premises Modern AuthenticationAymenNo ratings yet
- Service CatalogueDocument9 pagesService CatalogueHenry Mauricio Castro GerardinoNo ratings yet
- MadgeTech RF ManualDocument32 pagesMadgeTech RF ManualIsrael_itaNo ratings yet
- Applications of Artificial IntelligenceDocument6 pagesApplications of Artificial IntelligencedjwakodeNo ratings yet
- Trabajo Semana 13 Ingles - InglesDocument3 pagesTrabajo Semana 13 Ingles - InglesJuan Pako RamNo ratings yet
- Az 104 PDFDocument10 pagesAz 104 PDFwilfred maswaya100% (2)
- MRK - Spring 2022 - CS302 - 1 - BC210405737Document5 pagesMRK - Spring 2022 - CS302 - 1 - BC210405737Sahil AliNo ratings yet
- "A Full Five To Six Years Before Bricklin and Visicalc": Copy of 1974 ManualDocument40 pages"A Full Five To Six Years Before Bricklin and Visicalc": Copy of 1974 Manualkennethdakin1777100% (6)
- Application Notes BDL3245E/BDL4245E/BDL4645E: Remote Control Protocol For RJ-45 InterfaceDocument16 pagesApplication Notes BDL3245E/BDL4245E/BDL4645E: Remote Control Protocol For RJ-45 Interfacezebigbos22No ratings yet
