Oracle Database Acceleration Discovery Day

Oracle on Solaris: Tips, Tricks & Best Practices
Andreas Neuhold
Commercial Products Sales Lead Sun Microsystems
1

Agenda
> Orac > Orac > Orac > Orac

le & S o laris : # n Solar

1

le D B o

le DB i n

is ers

le in C o ntainer Cluster > Res s ources

Contain

2

Source: Oracle + Sun Strategy Webcast

3

2H FY2008. 2H FY2009 8000 7000 6000 5000 4000 3000 2000 1000 0 Solaris SPARC 64 Solaris x64 HP-UX ItaniumHP-UX PA-RISC AIX 64 2H2007 2H2008 2H2009 4 .Number of Unique Customers Entering Tars for Mainstream Unix Platforms – 2H FY2007.

Percentage of UNIX Customers for mainstream UNIX platforms per Service Request – 2HFY2009 Solaris SPARC Solaris x64 27% 36% HP-UX Itanium 14% 10% 13% HP-UX PARISC AIX 5L 64-bit • Total Percentage of Customers per UNIX O/S • 46% Solaris • 27% HP-UX • 27% AIX 5 .

Agenda > Orac > Orac > Orac > Orac le & S o laris : # n Solar 1 le D B o le DB i n is ers le in C o ntainer Cluster > Res s ources Contain 6 .

San Francisco.com/download/attachments/128484865/OOW09 Optimal Oracle DB Perf 1_1.pdf 7 . CA http://wikis.sun.• Oracle DB on Solaris – based on > “Best Practices for Optimal Configuration of Oracle Databases on Sun Hardware” by Allan Packer and Bob Sneed > Oracle Openworld 2009.

Oracle on Solaris • Solaris 10 is the #1 UNIX platform for Oracle deployments • Solaris benefits from many years of performance enhancements designed to benefit Oracle • Solaris offers outstanding > > > > > Scalability.. Security.g.. @ no additional cost) File System technology (ZFS – no limits) Robustness. to hundreds of CPUs Observability (e. Application support.. Dtrace) Virtualization (container / zones. 8 .

sun. 2005) • FX – Fixed Priority Class (2002) • Direct IO with concurrent writes (2002) • Sun Cluster optimizations for Oracle RAC (multiple) http://wikis.com/display/Performance/Key+Sun+technologies+for+running+Oracle 9 .More than 15 years of joint Solaris/Oracle DBMS Optimizations • Intimate Shared Memory (1993) / Dynamic Intimate Shared Memory (2002) • Scheduler control optimizations (1997) • 64-bit memory space (1998) • /dev/poll (1998) • Reconfiguration Coordination Manager (2002) • Locality Groups (2002) • Large Page Support (2002.

where you waste a further 10%-15% • This problem applies to typical multi-core chips > Designed with sophisticated out-of-order cores and aggressive speculation Source: “Database Servers on Chip Multiprocessors: Limitations and Opportunities” September 2006. Carnegie Mellon University 10 .The Dark Underbelly of DB Workloads: Data Stalls • Data stalls dominate execution time > 46%-64% of execution time wasted on data stalls > True for both OLTP and DSS database workloads > Even more execution time wasted on instruction stalls > Especially for OLTP.

provided you keep the CPUs busy Source: “Database Servers on Chip Multiprocessors: Limitations and Opportunities” September 2006.Everything Changes with Sun's CoolThreads CMT Systems • Data stalls drop to 13% > > > > 76%-80% of execution time spent on usefull computation True for both OLTP and DSS database workloads Instruction stalls are negligible END RESULT: 70% higher throughput than traditional systems • BUT: not good for unsaturated workloads > Single-threaded workloads perform 41% worse > So CMT is a win. Carnegie Mellon University 11 .

A Simple Analogy: Cars are Good! 12 .

Buses are good. too 13 .

Business Logic > Divide and Conquer > Increase Concurrency and Parallelism 14 .Oracle Performance Expectations • Most OLTP environments run very well with CMT > Understand response time components > DB CPU usually around 20% of response time > IO. Network. Indexes. and application tier makes up the rest • Bad code is a bottleneck on any server > CPU-bound bad code bites you sooner on CMT servers • Serial Batch jobs run slower with CMT > Check the usual suspects > Execution Plans.

Oracle on SPARC Servers – Summary • Sun's CMT Server Advantages > Low power consumption > Maximum rack computational density > Logical Domains (LDoms) powerful and bundled (by use of Solaris 10) • Sun's M-Series Server Advantages > Outstanding scalability > Enterprise-class RAS features > Flexible domaining • Solaris Containers run on both CMT and M-Series > Secure virtual OS instances. negligible perf overhead > Capped Containers recognized for licensing purposes 15 .

or application-specific settings (some not widely-known) • In general. frameworks.Best Practices Categories • Avoiding known performance bugs > Some are surprisingly high-impact > Architecturally-dependent factors are not widely-known • Copying proven patterns. being disciplined about performance > Holistically. tools. interdisciplinary and interdepartmental alignment. Both may be needed to meet system performance objectives. processes NOTE: While there is no clear objective line between “Best Practices” and “tuning” . solution architectures.the distinction is much like how “just do it” relates to “optimization”. backup and recovery. business continuance > Methods. avoiding known antipatterns > Filesystem and network choices > “Big knob” tunables > Platform. operational plans. 16 . including the application code. metrics.

with these fixes (among others) ...4 has point patches available for these . filesystemio_options=setall was broken > Bug #6814520 .4 is a Good Place to be.1 > Patch #8199533 .0.For enabling direct I/O on certain filesytems. see Metalink #759565.Excessive close() calls on new session creation > Scalability/consolidation issue: kstat overhead excessive with many instances running > Patch #8531434 – "KSTAT CALLS BY MMNL/CJQ0 INCUR HIGH SYS .Disables NUMA features as new default 17 ... > Excessive client attach latency & high cost of session initiation > Bug #8737425 .Compute efficiency of block checksum code on SPARC was • 10.Oracle Performance Bugs & Patches • 10.2. very poor > Bug #5752399 .2..0.” > RE: Oracle NUMA features..

sun.com/bobs 18 . concurrent I/O stack and consistent contiguous ondisk data placement > Solutions with these characteristics perform similarly > RAW > UFS direct I/O > QFS with samaio or qwrites > ASM > VxFS with ODM.Filesystem/Feature Selection • Performance Best Practice: use an unbuffered. or direct with 'cio' (VxFS has new 'cio' option. ODM no longer an extra fee) > NFS with direct I/O (Of course. performance depends greatly with NFS server factors and network factors) • Each solution has its own footnotes > See: http://blogs. QIO.

even when FSS is in use > Otherwise. but LGWR is uniquely critical path to all transactions • This is especially important on systems with high aggregate CPU utilization or systems with slower single-thread speeds 19 .Control LGWR Priority on Purpose • Run LGWR at Fixed Priority (FX) 60. TS scheduling will reduce its priority when it gets busy - exactly opposite of what one would want! • Other background processes may also benefit from special scheduling.

and hardware interrupts Avoids scaling challenges of IP packet reassembly Radically improves scaling of RAC on CMT CPUs • Implementation notes > Oracle specifies dedicated interconnects for RAC.sun.RAC: Use Jumbo Frames! • Jumbo frames delivers multiple benefits > > > > Reduces GC latency Reduces CPU utilization.com/glennf/entry/jumbo_frames_with_oracle_rac 20 . jumbo frames should not be an 'infrastructure project' • See also > http://blogs. %sys.

and Software > For Oracle on Sun Performance information in particular http://wikis. Storage.com/display/performance 21 .sun.For more information Check out the Sun Performance Wiki! > For Sun Performance information generally > Servers.

Agenda > Orac > Orac > Orac > Orac le & S o laris : # n Solar 1 le D B o le DB i n is ers le in C o ntainer Cluster > Res s ources Contain 22 .

Whitepaper 23 .

filesystems. e. devices.g. 24 . max CPUs • Most resources must be mapped into a container explicitely > e.Solaris Containers • Provide a virtual OS environment for applications.g. ideal for consolidation. dedicated CPUs. etc. Oracle Dbs • Secure execution environment with priviliges granted from the global zone > No way of reaching out of the zone except via the network • Many controls for resource management > CPU sets.

Srv.. 25 . disks. memory. software-defined boundaries OS > “Can't see outside zone” > “Can't affect activity outside zone” • Simple isolation Server > Network.) What is a Solaris 10 Container ?? It is NOT an additional OS installation. environment • Each Container has it's own identity (/etc/. providing multiple separated database instances. Container > One single OS installation > Multiple runtime environments. CPU. But it is a separated runtime environment. within an existing Solaris 10 OS installation. appear as “regular” Solaris OS to applications and users • Flexible. processes.Solaris 10 ContainerOS Virtualization • Database Container System Container App Appl. Analogy with Databases: One single RDBMS installation..

..Solaris Container – Technical view (Whole-root Zone) cpu-shares = 30 memory_target = 3G cpu-shares = 20 memory_target = 2G cpu-shares = 50 em/initial_size_MB = 4096 abap/heap_area_total = 2048 Global Zone / /etc /lib /opt /platform /usr /var Local Zone Database / /etc /lib /opt /platform /usr /var /u01/ora/product/. Solaris 10 Kernel 26 .... /u01/ora/fra/CRM2 /u02/ora/data Local Zone ERP System / /etc /lib /opt /platform /usr /var /usr/sap/P01 /sapmnt/P01 /oracle/P01 /dev → lofs mount . /u01/ora/fra/CRM1 /u02/ora/data Local Zone Database / /etc /lib /opt /platform /usr /var /u01/ora/product/..

Online and dynamically. 27 . > "Security island" (EAL4+ certification). Memory. Clone. > Resource Management.Solaris Containers A convenient virtual platform for Oracle Solaris 10 Containers have unique features. • Consolidation > Isolate applications software stacks and services. No additional license fees or support costs. Thus providing not only a server consolidation but also an OS consolidation. • Standardization & Administration > Reuse one quality-assured Solaris OS implementation for the system landscape. IO bandwidth) for Containers. > Only one OS installation for multiple separated runtime environments. Limit or assure resources (CPU. No additional (3rd party) software installation. Migrate. Reboot. no new OS Installation !). • Built-in > Contained in Solaris 10. valuable especially in Oracle environments !! • Flexibility > Fast creation (takes only couple of minutes.

• Proven > Many customer references 28 . • Oracle Support > Oracle does support “single instance” in Solaris 10 Containers.25% for 4 Containers). AMD.Solaris Containers A convenient virtual platform for Oracle Part2 Solaris 10 Containers have unique features.. Fujitsu/Siemens and more . (Status: New QA). (SAP Benchmark: 1. valuable especially in Oracle environments !! • Scalable > Up to 8192 Container. • Platform Support > Available for all Solaris 10 platforms: Intel. > Container (CPU pool or CPU cap) is accepted as Oracle license limitation.. IBM. Nearly no performance degradation. HP. Dell. > New: Oracle will certify and support RAC in Solaris 10 Containers. UltraSPARC on Sun.

Highly Available Solaris Containers • Containers can be moved between Servers • Containers can be made HA using Oracle Solaris Cluster > HA Container agent > Blackbox approach > Zone nodes > Switch Applications between “static” containers > Container Clusters > Full virtual cluster 29 .

Agenda > Orac > Orac > Orac > Orac le & S o laris : # n Solar 1 le D B o le DB i n is ers le in C o ntainer Cluster > Res s ources Contain 30 .

Blueprint 31 .

why • A logical step forward – a virtual cluster for a virtual OS • Delegation of cluster administration • Encapsulated cluster > Global components only manageable from the global zone • Primary target application: > Oracle Real Application Cluster 32 .Container Clusters .

Oracle Solaris Container Cluster Best Environment For Multi-tier Deployments Node 1 Finance Node 2 Finance Node 3 Finance Container Cluster with RAC 10g Sales Container Cluster with RAC 10g Engineering Container Cluster with RAC 11g Sales Node 4 Sales Application fault isolation Security isolation Consolidation use cases • Multiple Oracle RAC databases • Test & Development databases • Multiple data center tier / multiple application Engineering Engineering Engineering Engineering Finance Database Engineering Database Sales Database 33 .

Agenda > Orac > Orac > Orac > Orac le & S o laris : # n Solar 1 le D B o le DB i n is ers le in C o ntainer Cluster > Res s ources Contain 34 .

com/display/performance > > > > > > • CoolThreads Experts Q&A Portal • Performance Wiki • Blogroll Glenn Fawcett .Resources • Cool Tools • Blueprints > http://cooltools.sun.com/deniss Tanel Poder .tanelpoder.com/allanp 35 .http://blogs.sun.sun.http://structureddata.sun.http://blogs.http://blog.com Greg Rahn .sun.net/ > http://wikis.sunsource.com/secure/servers/coolthreads/tnb/contact.com/bobs (paused) Allan Packer .sun.org Bob Sneed .sun.com/display/BluePrints/Main > http://www.http://blogs.jsp > http://wikis.http://blogs.com/glennf Denis Sheahan .

neuhold@sun.com 36 .Oracle Database Acceleration Discovery Day Oracle on Solaris: Tips. Tricks & Best Practices Andreas Neuhold andreas.