PL/SQL Enhancements in 11g

Enhanced SQL – PIVOT and UNPIVOT
PROD Shoes Shoes Jeans Jeans Jeans Jeans QUARTER Q1 Q2 Q1 Q3 Q3 Q4 AMOUNT 2000 1000 1000 500 100 Shoes 1000 Jeans 1000 Null 600 1000 2000 1000 Null Null PROD Q1 Q2 Q3 Q4

SELECT * FROM sales PIVOT (sum(amount) FOR quarter in ‘Q1’,’Q2’,’Q3’,’Q4’);

Enhanced performance of ETL in your warehouse

PL/SQL Enhancements in 11g
Fine Grained Dependency Tracking

The challenge create table t(a number) / create view v as select a from t / alter table t add(Unheard_Of number) / select status from User_Objects where Object_Name = 'V' / • View v ends up invalid in 10.2 because we know only that its dependency parent has changed – at the granularity of the whole object .

1 we track dependencies at the level of element within unit • so we know that these changes have no consequence .Fine Grained Dependency Tracking create table t(a number) / create view v as select a from t / alter table t add(Unheard_Of number) / select status from User_Objects where Object_Name = 'V' / • In 11.

PL/SQL Enhancements in 11g Real Native Compilation .

The challenge • Through 10.2. PL/SQL compiled to a native DLL is significantly faster than PL/SQL compiled for interpretation by the PVM • Oracle translates PL/SQL source to C code and leaves the last step to a 3rd party C compiler • BUT… some customers’ religion forbids a C compiler on a production box! • AND… other customers’ religion forbids paying to license a C compiler when they’ve already paid to license Oracle Database! .

Oracle does the linking and loading so that the filesystem directories are no longer needed • So PL/SQL native compilation will work out of the box – and without compromising religion • Only one parameter remains: the on/off switch. PLSQL_Code_Type .1. Oracle translates PL/SQL source directly to the DLL for the current hardware • Moreover.Real native compilation • In 11.

Real native compilation • As a bonus. it’s faster! • Compilation with real native is twice as fast as with C native • The Whetstone benchmark is 2.5x faster with real native than with C native at run-time • Contrived tests have shown 20x run-time speedup .

PL/SQL Enhancements in 11g Sequence in a PL/SQL expression .

The challenge create or replace trigger Trg before insert on My_Table for each row declare s number.Annoying locution select My_Seq. end. :New.PK := s.Nextval into s from Dual. begin -. / .

Nextval. end.Sequence in a PL/SQL expression create or replace trigger Trg before insert on My_Table for each row begin :New. / .ID := My_Seq.

PL/SQL Enhancements in 11g Tools support .

got lost following reference after reference Impact Analysis: how do you determine who references a particular variable? Solution PL/Scope Usage alter session set PLScope_Settings = 'identifiers:all' create or replace.PL/Scope Challenge Inherit code. find a problem in one area that links elsewhere. UI expected in SQL Developer 2.0 . from … User_Identifiers where Object_Name = . Select ….....

START_PROFILING('PLSHPROF_DIR'.trc'). 'test.PL/SQL Hierarchical Profiler Challenge Where is time really being spent in PL/SQL code? Solution PL/SQL Hierarchical Profiler reports dynamic execution profile organised by subprogram accounts for SQL and PL/SQL separately generates hypertexted HTML reports Usage > EXECUTE DBMS_HPROF.0 . % plshprof UI expected in SQL Developer 2.

for Oracle Database 11g Release 1. June 2007 . An Oracle White Paper. better and easier See:.XML DB Enhancements in 11g Fast.New Features in Oracle XML DB.

1 In Place Schema Evolution allows simple changes to registered XML schemas with zero down-time • no data copy required • schema change takes a few seconds regardless of amount of data. • changes to the XML must not invalidate existing documents • add optional elements and attributes • adding new values to enumerations • increase in length .2 copyEvolve() unloads and reloads the data when the XML Schema changes • very flexible but expensive: time to evolve schema is proportionate to the amount of data being managed • 11.In Place Schema Evolution • 10.

1 REF partitioning • nested Tables are partitioned using the same key as the top level XMLType table • supports “partition maintenance” operations • supports “partition pruning” .XMLType Partitioning Challenge <PurchaseOrder> <Date>…</Date> <Ref>…</Ref> <LineItems> <LineItem>… <LineItem>… <LineItem> <LineItems> • repeating elements typify XML documents and are managed using nested tables Solution • leverage 11.

inserting a purchase order into “approved” folder triggers ordering process . • e.XML Misc Improvements • Significant performance improvements • Large node handling eliminates current 64K limit on size of a text-node.g. • Stream based replication support for LOB-based XML storage • Events: trigger support for the XML DB repository.

1 binary XML optimized for schema-less and flexible schema-based XML • Post Parse compact representation of XML • Flexible XML Schema Support • High performance fragment access and extraction with new XML Index .2 schema-less XML not well optimized Solution • 11.Binary XML Challenge • 10.

“in-memory” and “on-the-wire” Reduced CPU and Memory overhead • single format avoids parse and serialize issue between tiers Reduced Network Overhead • post-parse format used when XML moves between tiers Database App Server Binary XML Binary XML Web Cache Binary XML Client Oracle Binary XML .Binary XML • • • Single format for “on-disk”.

XML Index • New universal index for Binary and LOB based XMLType • All possible paths are indexed by default • XML index optimization • asynchronous operation possible • path sub-setting controls what is indexed • partial re-indexing (binary XML and securefiles only) • Repository integration with XML Index on XML DB repository .

html .oracle.Java Enhancements in 11g Doing it inside like you do outside See:.Java Developer Center on OTN http://www.com/technology/tech/java/index.

JDBC Performance Saving 1 Million Roundtrips per Day Prefetch in First Roundtrip (JDBC-Thin) • saves 1 database roundtrip per SELECT Statement • benefit for ad-hoc queries New Native AQ protocol (JDBC-Thin) • AQ/OJMS operations 40%-300% Faster • reduced server-side CPU Utilization .

Java Standards
JDBC 4.0
• • • • • • SQLXML RowId NChar & Nclob Wrapper Interface SQLException Hierarchy conn.createBlob, createClob, etc

Disconnect Java VM upgrade from database
• Java 5 (i.e. JDK 1.5) in base 11.1 release • Java 6 (i.e. JDK 1.6) and newer JDKs, in patch sets

From Black Box to White Box Monitoring
• JMX Instrumentation • Support standard JMX-enabled tools(i.e., JConsole)

Java in the Database Mass Adoption
Challenge
• Performance

Solution
• New JIT compiler compiles Java in the database natively and transparently (on the fly) without the need of a C compiler • Enabled by default, Out-of-the-Box • Up to an Order of magnitude (~x 10) speed-up for pure Java, 10g versus 11g, out-of-the-box

Challenge
• Ease of Use

Solution
• JDK-Like interface, with code on filesystem, using familiar notation, e.g. –{classpath}, -D system properties • Eases the migration for developers from JDK VM world

Enhancements for PHP developers
Accessing the database will never be the same
See:- PHP Developer Center on OTN http://www.oracle.com/technology/tech/php/index.html

PHP users have asked for Database Connection Pooling • Web apps issue hundreds connects per second • Connection establishment is expensive • PHP does not allow real multi-threading • persistent vs non persistent connections • Existing client-side connection pooling not possible • Options: • apache process can repeatedly connect/disconnect • apache process can hold onto private connection • Results in inefficient use of the database resources .

Database Resident Connection Pool No Connection Pooling 11g Database Resident Connection Pooling .

no other software installation and no coding Secure: HTTP authentication and easily integrate with WS Security Products High performance : C-based engine.e ut rat A e en G WSDL PL/SQL SOAP ? DB WS SOA SQL XQuery Easy: a few steps.Native Database Web Service DBA o. natively build in to the Oracle DB .

Oracle Scheduler enhancements in Oracle Database 11g Schedule Jobs • Local OS jobs • run as any OS user 11g Execute local OS job A as OS user X & job B as OS user Y • Remote OS jobs • no Oracle database required on node • run as any OS user Agent Agent Execute remote Execute remote OS job as OS user X OS Job as OS user Z .

Business Intelligence • OLAP • Data Mining See:Oracle Database 11g for Data Warehousing and Business Intelligence An Oracle White Paper July 2007 .

• transparent SQL interface to OLAP Cubes and dimensions • cube-organized materialized views • OLAP metadata in the Oracle Data Dictionary .What is Oracle OLAP Option? • A multidimensional calculation and aggregation ‘engine’ embedded in the database • multidimensional data types (Cubes and Dimensions) • OLAP API for cube definition and multidimensional query and coming in Oracle Database 11g ….

Oracle OLAP 11g New Features in Oracle Database 11g • Data accessibility: Enhanced SQL access to OLAP cubes • SQL Optimizer awareness of OLAP cubes • Cube Organized Materialized Views • System-maintained generation of cube and dimension views • Feature accessibility: Shift the focus from the skilled OLAP specialist to the database generalist • Cube and dimensions in the Oracle Data Dictionary • Cost-based aggregation • Dimensionally aware SQL-like calculation syntax .

Cube-Organized Materialized Views • Materialized view with data stored in an OLAP cube • • • Automatic query write of summary queries into the cube for improving performance of summary queries Materialized view managed refresh of cubes and dimensions Similar to materialized view on pre-built table. data is stored in cube and MV is a meta data object .

Cube-Organized Materialized Views • SQL Query • • Tables Query Rewrite Cube MV Cube Organized MV representing summary data • Detail data managed in relational tables • Database-managed Databaserefresh Summary data managed in cubes Detail data is stored in relational tables Application queries relational tables with SQL Summary data is managed by OLAP cube Access to summary data occurs via automatic query rewrite to the cube Application is unchanged. but updates and queries are faster .

FAMILY_DSC. a13. a13. a12.ITEM_ID = a13. sum(a11.Query Rewrite to Cube select a13.CALENDAR_YEAR_ID CALENDAR_YEAR_ID.FAMILY_DSC FAMILY_DSC.FAMILY_ID FAMILY_ID.CALENDAR_YEAR_ID. SUM(SALES) Select from relational tables GROUP BY .FAMILY_ID.SALES) WJXBFS1 from UNITS_FACT a11 join TIME_DIM a12 on (a11. a12.MONTH_ID) join PRODUCT_DIM a13 on (a11. a12.ITEM_ID) group by a13.CALENDAR_YEAR_DSC.CALENDAR_YEAR_DSC CALENDAR_YEAR_DSC. a12.MONTH_ID = a12.

Query Rewrite to Cube OPERATION --------------------------------------SELECT STATEMENT HASH HASH JOIN HASH JOIN CUBE SCAN CB$UNITS_CUBE VIEW HASH TABLE ACCESS PRODUCT_DIM VIEW HASH TABLE ACCESS TIME_DIM Cube Access .

guided data mining • Spreadsheet Add-In for Predictive Analytics • “1-click data mining” from a spreadsheet • PL/SQL API & Java (JDM) API • develop advanced analytical applications .What is Oracle Data Mining? • In-Database Mining Engine • wide range of algorithms • • • • • • • • anomaly detection attribute importance association rules clustering classification & regression nonnegative matrix factorization structured & unstructured data (text mining) BLAST (life sciences similarity search algorithm) • Oracle Data Miner (GUI) • simplified.

“What is the impact on SUVs sales if the price of gas goes up by a $1.Oracle Data Mining 11g New Features in Oracle Database 11g • Simplified development and deployment of models • supermodels: data preparation combined with mining model • tighter database integration – DMSYS schema eliminated. improved sharing of objects. Indicate the impact of different attributes on a key performance indicator.00?” . manageability • New algorithms: “General Linear Models” • multivariate linear regression. logistic regression • basic tool for microeconomics analysis and business decision-making. security.

5 million records scored in 6 seconds on a single CPU system • Real-time scoring: • 100 models on a single CPU: 0.Why In-Database Analytics ? • Fewer moving parts : data remains in the database at all times … with appropriate access security control mechanisms • Straightforward inclusion within interesting and arbitrarily complex queries • Real-world scalability and performance • Fast scoring: • 2.085 seconds .

.Integration with Oracle BI EE ODM provides likelihood of fraud and other important questions.

Oracle Database 11g Release 1 Transparent Solutions for Security and Compliance An Oracle White Paper June 2007 .Security • • • • User Management Access Control Data Protection Monitoring See:.

• Foreign Exchange Instruments and Exchange Law (J-SOX) • EU Privacy Directives. COBIT • Separation of duty. Proof of compliance.Key Drivers for Data Security Regulatory Compliance • Sarbanes-Oxley (SOX). • Payment Card Industry (PCI) • Adequate IT controls. COSO. Risk Assessment and Monitoring Insider Threats • Large percentage of threats go undetected • Outsourcing and off-shoring trend • Customers want to monitor insider/DBA . CA SB 1386….

Oracle Database Security Products User Management • Oracle Identity Management • Enterprise User Security Access Control • Oracle Database Vault • Oracle Label Security Core Platform Security Monitoring • Oracle Audit Vault • EM Configuration Pack Data Protection • Oracle Advanced Security • Oracle Secure Backup .

Oracle Database 11g Enterprise User Security • User Management for Compliance • Centralized User Management • Consolidate database accounts with shared database schemas • Centrally managed DBAs • Validated with Oracle Virtual Directory HR Database Financial Database Customer Database EUS EUS • Enterprise Strong Authentication • • • • Kerberos (MSFT.1.509v3) Password SYSDBA Strong Authentication • Database Enterprise Edition Feature • Requires Oracle Identity Management • Available since Oracle 8. MIT) PKI (x.6 Oracle Identity Management .

Oracle Database Security Products User Management • Oracle Identity Management • Enterprise User Security Access Control • Oracle Database Vault • Oracle Label Security Core Platform Security Monitoring • Oracle Audit Vault • EM Configuration Pack Data Protection • Oracle Advanced Security • Oracle Secure Backup .

schemas.Oracle Label Security Manageability • Policy based model • Multiple policies supported • ACME. Legal • Policies are umbrellas applying to one or more tables. HR. users • Web based management • Integrated with Oracle Identity Management .

Oracle Database Security Products User Management • Oracle Identity Management • Enterprise User Security Access Control • Oracle Database Vault • Oracle Label Security Core Platform Security Monitoring • Oracle Audit Vault • EM Configuration Pack Data Protection • Oracle Advanced Security • Oracle Secure Backup .

un-encrypted tablespaces • however.Oracle Advanced Security Tablespace Encryption • Define a new tablespace as ‘encrypted’ • cannot convert existing. same wallet used as for column based Transparent Data Encryption • No storage overhead (!) . content can be moved into encrypted tablespaces • Always salted for higher security • Overcomes limitation of column-based TDE: • supports indexes other than b-tree • supports foreign keys • No additional management overhead • integrated into TDE key management.

Oracle Advanced Security Master key stored in HSM device • Store the Master key in an external hardware device • Master key never leaves the device • Standard PKCS #11 API allows customers to choose from a wide range of HSM vendors • Encryption and decryption done on the database server • Simplifies key management in distributed environments (data guard. RAC) .

Oracle Advanced Security SECUREFILE LOB Encryption • SECUREFILE LOB encryption • All SECUREFILE LOBs in an encrypted column are encrypted • In-line (in table) and out-of-line (in tablespace) are both encrypted • BFILEs are not encrypted • Always salted for higher security .

Oracle Database Security Products User Management • Oracle Identity Management • Enterprise User Security Access Control • Oracle Database Vault • Oracle Label Security Core Platform Security Monitoring • Oracle Audit Vault • EM Configuration Pack Data Protection • Oracle Advanced Security • Oracle Secure Backup .

Oracle Database 11g Security Manageability • Integrated with EM • • • • Label Security Virtual Private Database Application Context Enterprise Security Manager • Transparent Data Encryption .

Oracle Database 11g: Manageability Overview An Oracle White Paper August 2007 .11g Database Manageability • • • • • Memory Management AWR Baselines Automatic SQL Tuning Advisors Upgrade Considerations See:.

11g Database Manageability Automatic Memory Management .

Memory Management in 10g/11g 11g Automatic Memory Management • MEMORY_TARGET • MEMORY_MAX_TARGET • Improves memory utilization • Eases memory management 10g Automatic Shared Memory Management (ASMM) • SGA_TARGET • Key shared memory components tuned automatically • Single parameter controls all shared memory 10g Automatic PGA Memory Management • PGA_AGGREGATE_TARGET • Various PGA sort areas tuned automatically • Single parameter controls all PGA memory .

Automatic Memory Management in 11g O/S Memory O/S Memory SGA SGA PGA PGA • Unifies system (SGA) and process (PGA) memory management • Dynamic parameters for all database memory: MEMORY_TARGET MEMORY_MAX_TARGET • Automatically adapts to workload changes • Maximizes memory utilization • Available on: • • • • • Linux Windows Solaris HPUX AIX .

Memory Target Benefits • Balances dynamic memory needs of SGA and PGA • • • • Previously either or both could end up undersized PGA can grow but not to the detriment of SGA performance Best-effort memory transfers Tunable PGA can help relieve 4031 from SGA • Built-in safety nets: • • • • Decides victim components Paced memory donation avoids thrashing Run-away workload cannot starve resources Component-specific settings still honored .

11g Database Manageability AWR Baseline and Adaptive Thresholds .

Performance Monitoring and Baselines • Performance Page shows certain metrics against 99th percentile over selected baseline • Adaptive Thresholds over select system metrics • Enabled by System Moving Window baseline • AWR Compare Period Report • Use Baseline as one of the periods .

Thanksgiving • System Moving Window Baseline • Always available • Defined by AWR retention or user specified • Adaptive thresholds computed over this baseline .Other Baselines Features • Baseline Templates enable future Baseline creation • Pre-specify known interesting times. e.g.

Creating Single AWR Baseline .

Quick Configure Adaptive Thresholds .

Adaptive Thresholds Adaptive threshold values follow baseline values AWR Baseline .

11g Database Manageability Automatic SQL Tuning .

Manual SQL Tuning Challenges • Complex .requires expertise in several domains • SQL optimization: adjust the execution plan • Access design: provide fast data access • SQL design: use appropriate SQL constructs • Time consuming • Each SQL statement is unique • Potentially large number of statements to tune • Never ending task • SQL workload always evolving • Plan regressions can happen .

but the DBA is still required .SQL Tuning in Oracle Database 10g Implement Evaluate Recommendations Workload DBA DBA Generate Recommendations one hour DBA ADDM AWR SQL Tuning Candidates Invoke Advisor SQL Tuning Advisor Some meaningful automation.

SQL Tuning Automation in 11g • Fully automatic SQL Tuning task • Runs automatically under Autotask framework: • Maintenance Window. ad-hoc (infrequent) • Tests and (optionally) implements SQL profiles • Only implements significantly improved plans (3x) • Time budgeted to avoid run-away plans . on/off switch • Identifies. recursive. DML/DDL. CPU resource controlled. ranks and tunes candidate SQL • Leverages SQL Tuning Advisor • Candidate SQL automatically chosen • Excluded: parallel queries.

Automatic SQL Tuning in Oracle Database 11g Implement SQL Profiles Test SQL Profiles Workload Generate Recommendations Choose Candidate SQL one week SQL Tuning Candidates It’s Automatic! AWR DBA View Reports / Control Process .

Result Summary .

Result Details .

Result Recommendations .

11g Database Manageability SQL Access Advisor .

SQL Access Advisor: Partition Advice SQL Access Advisor SQL cache Hypothetical STS Filter Options Complete Workload Hash partitions? Interval partitions? Indexes Materialized views Materialized Partitioned views log objects .

SQL Access Advisor Results .

Access Advisor Recommendations

Editable implementation script

11g Database Manageability
Fault Diagnostic Automation

Oracle Database 11g R1 Fault Diagnostic Automation Goal: Reduce Time to Problem Resolution Realistic Testing and Automatic Health Checks Automatic Diagnostic Workflow Intelligent Resolution Proactive Patching Diagnostic Solution Delivery Prevention Resolution .

Automatic Diagnostic Workflow Critical Error Automatic Diagnostic Repository DBA Auto incident creation First failure capture Alert DBA Targeted health checks Assisted SR filling 1 2 No DBA Known bug? Yes 4 EM Support Workbench: Package incident info Data Repair EM Support Workbench: Apply patch / Data Repair 3 DBA .

g by patch • They are assigned a signature-based key value • An incident is a single occurrence of a problem • They happen at point(s) in time and thus have timestamps • They induce diagnostic actions like dumps and traces • They are managed by ADR and packaged by IPS .Problems and Incidents • Problems are fundamental code or configuration issues that can cause execution failures • They exist until they are corrected. e.

xml alert_SID.Automatic Diagnostic Repository DIAGNOSTIC_DEST BACKGROUND_DUMP_DEST CORE_DUMP_DEST USER_DUMP_DEST $ORACLE_HOME/log ADR Base diag Support Workbench $ORACLE_BASE rdbms DB Name ADR Home SID metadata alert cdump incpkg incident hm trace (others) incdir_1 … incdir_n ADRCI log.log V$DIAG_INFO .

Important points about ADR • Location of traces and dumps has changed • See V$DIAG_INFO for new locations • Symbolic links can provide work around for script compatibility • Format of alert.log is changed • New XML format more parse-able • Old style still available for backward compatibility • Space management • Incident flood control minimizes number of traces • ADR auto-purging reduces disk footprint .

error code • Recommends further diagnostic actions for DBA • For example build SQL test case • DBAs can explicitly add/edit or remove files before packaging • Including non-ADR managed files .Incident Packaging Service (IPS) • IPS automates packaging of diagnostic data for given problem • Creates zip files from incident package contents for sending to Oracle Support • Intelligently correlates and includes dumps and traces from related incidents • Correlation rules based on time proximity. client id.

EM Support Workbench Overview • Primary customer-facing interface into IPS and ADR • Task-oriented interface over problems and incidents • • • • View problem and incident details Create and manage IPS packages Run health checks or generate additional diagnostic data Run advisors to help resolve problems • Linkages to Oracle Support • Generate and send incident package files using OCM • Create and track Metalink service requests from problem • Close problem SR once resolved .

EM Support Workbench .

.

Package details and IPS in EM

EM Support Workbench Quick Package
• Simple wizard workflow from problem through package and send to support
• Single-problem package • Good for rapid first fault reporting

• Limitations
• Package zip must be 2GB or less • Cannot scrub traces or customize package

Prevention: Health Monitoring
• Proactive Health Checks
• E.g. Dictionary Validation

• Reactive Checks
• Trigger on error via Intelligent dumping Rules • E.g. Checks blocks around corrupted block

• All Checks can be Activated on Demand • Incident Meter for EM

Intelligent Repair Advisors • Data Recovery Advisor • Guided expert data recovery system using diagnostic data and Health Check output • SQL Test Case Builder • Automatically retrieves exact environment information from ADR to build SQL test cases and replicate SQL issues • SQL Repair Advisor • Analyzes failing SQL statements to isolate bug • May recommend SQL Patch as work around .

Data Recovery Advisor .

Data Recovery Advisor • Oracle Database tool that automatically diagnoses data failures. and executes repairs at the user's request • Determines failures based on symptoms • E.g. presents repair options.dbf and f003.dbf are missing • Failure Information recorded in diagnostic repository (ADR) • Flags problems before user discovers them. an “open failed” because datafiles f045. via automated health monitoring • Intelligently determines recovery strategies • Aggregates failures for efficient recovery • Presents only feasible recovery options • Indicates any data loss for each option • Can automatically perform selected recovery steps Reduces downtime by eliminating confusion .

Data Recovery Advisor Enterprise Manager Support .

SQL Test Case Builder .

SQL Test Case Builder Business Requirement • Bug resolution • Test case required for fast bug resolution • Not always easy to provide a test case • What information should be provided? • How much data is need? • Getting the test case to Oracle can be tricky Solution • Oracle automatically creates a test case • Collects necessary information relating to a SQL incident • Collected data is packaged to be sent to Oracle • Collected data allows a developer to reproduce the problem .

SQL Repair Advisor .

sends necessary diagnostic information to Oracle .SQL Repair Advisor Business Requirement The most common types of SQL problems . are hard to diagnose A lot of time is spent trying to reproduce the problem If a workaround is found it has to be applied to entire system Solution • Advisor • • • • Investigates the incident locally Automatically determines the root cause Provides a workaround (SQL Patch) for just the effected SQL If not.exception.. performance regression etc.

SQL Repair Advisor Flow SQL statement Execute Statement crashes Generate incident in ADR automatically Trace files SQL Repair Advisor investigates DBA run SQL Repair Advisor DBA gets alerted Statement executes successfully again Execute DBA DBA accept SQL patch SQL patch generated SQL statement patched .

DB Management Pack Enhancements • Diagnostic Pack • ADDM for RAC • AWR Baselines and Adaptive Thresholds • Transportable AWR • Tuning Pack • • • • Automatic SQL Tuning Advisor Partition Advisor Real-time SQL Monitoring SQL Plan Management and Plan Evolution .

Upgrade Considerations .

GMON Maintains disk membership in ASM disk groups ASM GTX0-j Provides transparent support for XA global transactions in a RAC environment. this per-instance process (Atomic Controlfile to Memory Service) is an agent which contributes to ensuring a distributed SGA memory update is either globally committed on success or globally aborted in the event of a failure. ALL DBRM The Resource Manager process is responsible for setting Resource Plans and other Resource Manager related tasks. ALL.New Background Processes in 11g Process Name ACMS Description In an RAC environment. ALL ALL with any registered event activity (optional non-fatal) EMNC / e0xx FBDA The process archives historical rows for tracked tables into flashback data archives and manages the flashback archives. The database autotunes the number of these processes based on the workload of XA global transactions. ALL DIAG Performs diagnostic dumps and executes global oradebug commands. RAC only . Continuous Query Notifications and Fast Application Notifications. The Event Monitor Coordinator coordinates the event management and notification activity in the database which includes Streams Event Notifications. Found RAC DIA0 Responsible for hang detection and deadlock resolution.

PSP0 Spawns Oracle processes. The space management coordinator process coordinates the execution of various space management related tasks. ALL RMSn The RAC Management Processes perform manageability tasks for RAC.g. e. It dynamically spawns slave processes (Wnnn) to implement the task. Found ASM. ? RAC only RMSN RAC only SMCO / Wnnn ALL (optional nonfatal) VKTM ALL . spawned on demand. creation of RAC related resources when new instances are added to the clusters In a RACenvironment. MARK Marks ASM Allocation Units as stale following a missed write to an offline disk.New Background Processes in 11g Process Name KATE Description Performs proxy I/O to an ASM metafile when a disk becomes offline. ASM. this process manages background slave process creation and communication on remote instances. spawned on demand. These background slave processes perform tasks on behalf of a coordinating process running in another instance. such as proactive space allocation and space reclamation. The Virtual Keeper of TiMe is responsible for providing a Wall-Clock time (updated every second) and Reference-Time Counter (updated every 20ms and available only when running at elevated priority).

1 .0.0.1 • 8.0.0.4->9.2.8 -> 11.1 or higher.Oracle Database 11g Release 1 Upgrade Paths • Direct upgrade to 11g is supported from 9. • If you are not at one of these versions you need to perform a “double-hop” upgrade • For example: • 7.2 or higher.2.2.7.0.3.1.2. and 10. 10.1.4 or higher.8->11.4 -> 9.

run the upgrade scripts and analyze the upgrade results • Export-Import • Use Data Pump or original Export/Import • CREATE TABLE AS SQL statement .Choose an Upgrade Method • Database Upgrade Assistant (DBUA) • Automated GUI tool that interactively steps the user through the upgrade process and configures the database to run with Oracle Database 11g Release 1 • Manual Upgrade • Use SQL*Plus to perform any necessary actions to prepare for the upgrade.

Oracle Database 11g Installation Changes • Addition of new products to the install • SQL Developer • Movement of APEX from companion CD to main CD • Warehouse Builder (server-side pieces) • Oracle Configuration Management (OCM) • New Transparent Gateways • Removal of certain products and features from the installation: • OEM Java Console • Oracle Data Mining Scoring Engine • Oracle Workflow • iSQL*Plus .

Fine Grained Access Control for Utl_TCP and its cousins Challenge • Oracle Database provides packaged APIs for PL/SQL subprograms to access machines (specified by host and port) using bare TCP/IP and other protocols built on it (SMTP and HTTP) • Utl_TCP. Utl_HTTP… • If you have Execute on the package. Utl_SMTP. you can access ANY host-port Solution • an Access Control List (ACL) specifies a set of users and roles • you assign an ACL to a host and port range • you may need to explicitly grant this access in 11.1 .

Case Sensitive Password • By default: • Default password profile is enabled • Account is locked after 10 failed login attempts • In upgrade: • Passwords are case insensitive until changed • Passwords become case sensitive by ALTER USER • On creation: • Passwords are case sensitive • Review: • Scripts • Database links with stored passwords • Consider backward compatibility parameter. SEC_CASE_SENSITIVE_LOGON .

xml adrci> show alert –tail $ORACLE_BASE/diag/rdbms/<dbname>/trace/alert_SID.log • • • • xml format $ORACLE_BASE/diag/rdbms/orcl/orcl/alert/log.Log files changes Automatic Diagnostic Repository • $ORACLE_BASE/diag alert.log .

but even move tools to ease the move • SQL performance analyzer • SQL plan management .2. same behavior as 10.Moving On RBO • Still there.

Moving On These deprecated features / privileges are still available for backward compatibility but the recommendation is to move on • • • • Stored Outline SQL Plan Management Ultrasearch Secure Enterprise Search CTXPath Index XML Index SYSDBA for ASM administration SYSASM No longer supported • Workflow BPEL .

Oracle Database 11g: Real Application Testing & Manageability Overview An Oracle White Paper June 2007 .Real Application Testing • Real Application Testing • Enterprise Manager Management Packs Enhancements See:.

What is Real Application Testing? • New database option available with EE only • Includes two new features • Database Replay (DB Replay) • SQL Performance Analyzer (SPA) • SQL Tuning Set (STS) functionality is shared between Real Application Testing option and EM Tuning Pack • Advanced functionality requires additional packs • Diagnostic Pack for DB Replay • Tuning Pack for SPA • License • Requires licenses for both capture and replay system CPUs .

Real Application Testing • Value • Rapid technology adoption • Higher testing quality Deploy Test • Business Benefit • Lower cost • Lower risk Change Remediate Solution for the Agile Business .

Real Application Testing Database Replay .

The Need for Database Replay • • • Businesses want to adopt new technology that adds value Extensive testing and validation is expensive in time and cost Despite expensive testing success rate low • • Many issues go undetected System availability and performance negatively impacted Existing tools provide inadequate testing • Simulate synthetic workload instead of replaying actual production workload • Provide partial workflow coverage • Cause of low success rate • Database Replay makes real-world testing possible .

Database Replay • • • Replay actual production database workload in test environment Identify. analyze and fix potential instabilities before making changes to production Capture Workload in Production • • Capture full production workload with real load. timing & concurrency characteristics Move the captured workload to test system Make the desired changes in test system Replay workload with full production characteristics Honor commit ordering Errors Data divergence Performance divergence • Replay Workload in Test • • • • Analyze & Report • • • Analysis & Reporting .

Interconnect •OS Platforms.Database Replay: Supported Changes Client Client Client … Changes Unsupported Middle Tier Changes Supported •Database Upgrades. OS Upgrades •CPU. Patches •Schema. Parameters •RAC nodes. Storage Recording of External Client Requests . Memory •Storage •Etc.

Comparison of LoadRunner & DB Replay Testing e-Business Suite
80
80

Time Taken (Days)

60 40 20

24 4 0
Understand Application Usage

20 0
Identify Key Transactions

20 5 2
Generate Workload LoadRunner DB Replay

5
Run Test

0 Install & Setup

Total Testing Time DB Replay: 2 weeks LoadRunner: 30 weeks

Why DB Replay?

From:
Artificial workloads Partial workflows Months of development Manual intensive 150 Days High risk

To:
Production workloads Complete workflows Days of development Automated Low risk 10 Days

Database Replay Workflow
Production (10.2.0.4)
Clients

Test (11.1)
Replay Driver

Mid-Tier

Storage

Storage

Capture

Process

Replay

Analysis & Reporting

internal activity excluded Minimal performance overhead for capture For RAC.g. e. Can capture on 10.2. etc. month-end processing. shared and local file system supported Specify interesting time period for capture. peak workload.4 and replay on 11g Storage … File System Middle Tier File 1 File 2 … File n • .Step 1: Workload Capture • • • • • All external client requests captured in binary files Client Client Production System Client System background..0.

Capture Options • Workload can be filtered to customize what is captured • Filter Types • Inclusion Filters: Specifies which sessions should be captured • Exclusion Filters: Specifies which sessions should NOT be captured • Filter Attributes: Workload capture can be filtered using any of the following session attributes • User • Program • Module • Action • Service • Session ID • Workload capture can be run on-demand or scheduled to run at later time .

workload can be replayed many times • For RAC copy all capture files to single location for processing … File n … File n Metadata Capture Files Replay Files .Step 2: Process Workload Files • Setup test system • Test DB is at same point in time as before production capture • Use RMAN to physically restore production db from backup • Use Snapshot standby • Use imp/exp. Data Pump. etc. File 1 File 2 File 2 Test System File 1 • Processing transforms captured data into replayable format • Once processed.

Step 3: Replay Workload • Replays workload preserving timing. concurrency and dependencies of the capture system Replay Driver is a special client program that consumes processed workload and sends requests to the replay system Replay Driver consists of one or more clients. it may be necessary to start multiple clients to drive workload Test System Replay Driver • File 1 File 2 • … File n Metadata Replay Files . For workloads with high concurrency.

Replay Options • Synchronized Replay (Default) • • • • Workload is replayed in full synchronized mode Exact same concurrency and timing as production workload Transaction commit order is honored Ensures minimal data divergence Workload can be replayed in unsynchronized mode Useful for load/stress testing High data divergence Three (3) parameters provided to control degree of synchronization • Think time synchronization • Connect (logon) time synchronization • Commit order synchronization • Unsynchronized Replay • • • • .

Replay Options • Number of Replay Clients • Configurable by user • Client Calibration Advisor recommends number of replay clients needed for specific workload • Replay clients are multithreaded clients that can drive multiple workload sessions each .

ASH Report: Facilitates comparative or skew analysis • .Analysis & Reporting • • Comprehensive reports are provided for analysis purposes There (3) types of divergences are reported • • Data Divergence: Number of rows returned by each call are compared and divergences reported Error Divergence: For each call error divergence is reported • New: Error encountered during replay not seen during capture • Not Found: Error encountered during capture not seen during replay • Mutated: Different error produced in replay than during capture Performance Divergence • Capture and Replay Report: Provides high-level performance information • ADDM Report: Provides in-depth performance analysis • AWR.

Database Replay Summary Report .

Performance Page – Database Replay .

Top Activity Page: Database Replay .

Performance Page – Database Replay .

Performance Page – Database Replay .

Current Restrictions • Database Replay does not support the following features in the current release • • • • • • Direct path load. import/export OCI based object navigation (ADTs) and REF binds Streams. remote describe/commit operations Flashback Shared Server . non-PL/SQL based AQ Distributed txns.

use shared file system Ensure data in test is identical to production as of capture start time to minimize data divergence during replay Use RMAN backup/restore or Snapshot Standby to setup test system For performance analysis test system capacity should be similar to production Reset system clock to same time as production if application logic involves SYSDATE usage Processing workload has performance overhead and can possibly take a long time Process workload on test system instead of production Use Client Calibration Advisor to identify number of replay clients needed to replay workload properly • Test System Setup • • • • • Process Workload • • • Replay • .Best Practices • Capture • • • Provide adequate disk space for captured workload (binary files) Database restart (Optional): Recommended to minimize divergence For RAC.

Real Application Testing SQL Performance Analyzer (SPA) .

The Need for SQL Performance Analyzer (SPA) • Businesses want systems that are performant and meet SLA’s • SQL performance regressions are #1 cause of poor system performance • Solution for proactively detecting all SQL regressions resulting from changes not available • DBA’s use ineffective and time-consuming manual process to identify problems SPA automates identification of all SQL performance regressions resulting from changes .

Why SQL Performance Analyzer? From: Manual workload capture Synthetic workload Months of manual analysis Partial workload High risk To: Automated workload capture Production workload Automated analysis in minutes Complete workload Low risk .

SQL Performance Analyzer • • • • Test impact of change on SQL query performance Capture SQL workload in production including statistics & bind variables Re-execute SQL queries in test environment Analyze performance changes – improvements and regressions Client Client … Client Production Test Re-execute SQL Queries Middle Tier Capture SQL Oracle DB … … Use SQL Tuning Advisor to tune regression Storage .

etc.SPA Benefits • Enables identification of SQL performance regressions before end-users can be impacted SPA can help with any change that impacts SQL execution plan • • • DB upgrades Optimizer statistics refresh New indexes. Materialized Views. • • Automates SQL performance tracking of hundreds of thousands of SQL statements – impossible to do manually Captures SQL workload with low overhead Integrated with SQL Tuning Advisor and SQL Plan Baselines for regression remediation • • . Partitions.

SQL Performance Analyzer Workflow Production (10.1) Mid-Tier Storage Storage Capture SQL Transport SQL Execute SQL Pre-change Execute SQL Post-change Compare Perf .2) … Clients Test (11.

1 and higher can be used for SPA tasks in 11g • SQL Tuning Set • Production Database .0.2.Step 1: Capture SQL Workload • • SQL Tuning Set (STS) used to store SQL workload STS includes: • • Cursor Cache SQL Text Bind variables Execution plans Execution statistics • • Incremental Capture • Incremental capture used to populate STS from cursor cache over a time period SQL tuning set’s filtering and ranking capabilities filters out undesirable SQL SQL workload captured in 10.

db link.) Copy SQL tuning set from staging table (“unpack”) . etc.Step 2: Move SQL Workload to Test System Cursor Cache SQL Tuning Set Export/Import SQL Tuning Set Production Database Test Database • • • Copy SQL tuning set to staging table (“pack”) Transport staging table to test system (datapump.

Step 3: Execute SQL Before Making Change SQL Tuning Set Fetch Next SQL • Establishes SQL workload performance baseline SQL execution plan and statistics captured SQL executed serially (no concurrency) Each SQL executed only once DDL/DML skipped Option to do Explain Plan only analysis • Test Execute • • • • Execution Plan & Statistics Save Result e Chang efore B SQL Performance Analyzer .

.g. e.Step 4: Execute SQL After Making Change SQL Tuning Set Fetch Next SQL • Test Execute Manually implement the planned change • Database upgrade. patches • Optimizer statistics refresh • Schema changes • Database parameter changes • Tuning actions. SQL Profile creation Execution Plan & Statistics Save Result Completed e Chang efore B e Chang After • Re-execute SQL after change • Gathers new SQL execution plans and statistics SQL Performance Analyzer .

Step 5: Compare & Analyze Performance Completed nge e Cha Befor Completed • e Chang After Compare performance using different metrics.g. e. • • • • Elapsed Time CPU Time Optimizer Cost Buffer Gets Compare SQL Performance • SPA Report shows impact of change for each SQL • • • Improved SQL Regressed SQL Unchanged SQL Analysis Report • SQL Performance Analyzer Fix regressed SQL using SQL Tuning Advisor or SQL Plan Baselines ..

SPA Report .

SPA Report – Regressed SQL Statements .

Best Practices • Workload Capture • Use incremental STS capture • Peak or representative workloads can be more accurately captured with this mechanism • Test system • Run SPA on test system instead of production as analysis can be resource intensive • Ensure test system has similar configuration and comparable optimizer statistics as production • Performance comparison • Use several different metrics..g. e. CPU time..and post-change performance for reliable results • Regression remediation • Use SQL Tuning Advisor and SQL Plan Baselines to fix regressions . to compare pre. etc. elapsed time.

Real Application Testing Summary • Provides cost-effective and easy-to-use solution of assessing impact of change on production systems • Holistic and actual workload testing results in lower risk • Brings down testing cycle from months down to days • Lowers hardware costs by eliminating need for mid-tier and application setup on test system • Maximizes ROI by leveraging Diagnostics and Tuning Pack to remediate regressions • With Real Application Testing businesses can • Stay competitive • Improve profitability • Be compliant .

Q U E S T I O N S A N S W E R S .

Thank you for your attention ! .

Sign up to vote on this title
UsefulNot useful