Professional Documents
Culture Documents
Agenda
Adaptive Query Optimization Optimizer Statistics enhancements SQL Plan Management enhancements
Adaptive Statistics
Discovered at compile time Discovered at run time
Adaptive Plans
Adaptive Statistics
Join Methods
At compile time
At run time
Rows coming out of order_items table are buffered up to a point If row count is less than the threshold use nested Loops otherwise switch to hash join
Statistics Collector
Statistics collector disabled after decision is made and becomes a pass through operation
NESTED LOOPS
HASH JOIN
to see the final plan selected Example shows that hash join picked at execution time Again the statistics collector is not visible in the plan
Full adaptive plan displayed when format parameter +adaptive is set Example shows both the nested loops and hash join in the plan
10
Additional information displayed on why operations are inactive can be seen with format parameter +report
11
Indicates statement had an adaptive plan which was resolved on first execution Resolved plan is used for subsequent executions Statistics collectors and buffering is disabled
12 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Reporting mode shows what would have happened during execution in the plan
13
Possible sub-plans are nested loop joins or hash joins and vice versa
14 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
User
Parallel servers communicate among themselves & the QC using messages that are passed via memory buffers in the shared pool How the data is distributed among the processes is based on the operation being done & number of rows expected
15
Distribution method decision based on expected number of rows New adaptive distribution method HYBRID-HASH
Statistic collectors inserted in front of PX process on the left hand side of the join
If actual number of rows less than threshold, switch from HASH to Broadcast Threshold number of total rows < 2x DOP
Enabled by default
16
P1
P2
P5 P6
DEPT
P7 P8
EMP
17
18
Hybrid hash join between EMP and DEPT Distribution method based on runtime stats If DEPT uses Broadcast - EMP uses Round-Robin DOP used is 6 Number of rows returned is 4 Broadcast used because 24 rows distributed (6 X 4)
19
Hybrid hash join between EMP and DEPT Distribution method based on runtime stats If DEPT uses Broadcast - EMP uses Round-Robin If DEPT used HASH - EMP uses HASH DOP used is 2 Hash used because only 4 rows distributed
20 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Hash/Hash
Adaptive Statistics
Discovered at compile time Discovered at run time
Adaptive Plans
Adaptive Statistics
Join Methods
At compile time
At run time
21
Dynamic Statistics
During compilation optimizer decides if statistics are sufficient to
generate a good plan or not Dynamic statistics are used to compensate for missing, stale, or incomplete statistics They can be used for table scans, index access, joins and group bys One type of dynamic statistics is dynamic sampling
22
Dynamic Statistics
Dynamic Sampling
Dynamic sampling has a new level 11(AUTO) Decision to use dynamic sampling depends on the complexity of predicate, existing statistics and total execution time Dynamic statistics shared among queries
23
Dynamic Statistics
Select * From product_information Where list_price-min_price=29 And category_id not in (11,22) And prod_name like Smart%;
SGA
SHARED DYNAMIC STATISTICS
Table cardinality estimate for prod_info
SHARED_POOL
Cursor 0: Select * from product_information
Dynamic sampling occurs on small number of blocks from table Resulting dynamic statistics are stored in cache
DATA DICTIONARY
24
Dynamic Statistics
Select supplier_id, prod_name From product_information Where list_price-min_price=29 And category_id not in (11,22) And prod_name like Smart%;
SGA
SHARED DYNAMIC STATISTICS
Table cardinality estimate for prod_info
10 Optimizer determines
plan using dynamic statistics
SHARED_POOL
Cursor 0: Select supplier_id, prod_name
DATA DICTIONARY
25
Adaptive Statistics
Re-optimization
During execution optimizer estimates are compared to execution statistics
26
Adaptive Statistics
Cardinality Feedback pre 12c
Statistics gathered about data volume and data type seen during execution
the next execution using the execution statistics instead Statements are only monitored once if they dont show significant differences initially they wont change in the future Only individual table cardinalities and group by estimates examined Information is stored in the cursor only and is lost if cursor ages out
27
Adaptive Statistics
Cardinality Feedback pre 12c
28
Adaptive Statistics
Cardinality Feedback pre 12c
Execution Plan after the second execution
Execution statistics used to reparse the statement on the second execution New plan shows correct cardinality estimates and a new join order Information learnt is stored in the cursor only and is lost if cursor ages out
29
Adaptive Statistics
Re-optimization in 12c
Join statistics are also monitored
30
Adaptive Statistics
Re-optimization indicator in V$SQL
31
Adaptive Statistics
Adaptive Plans & Re-optimization working together
Adapt join method example showed the join change from nested loops to hash join But cursor is also marked IS_REOPTIMIZABLE
Why?
Re-optimization will occur on second execution because cardinality estimates off
32 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Adaptive Statistics
Adaptive Plans & Re-optimization working together
On second execution statement is re-parsed using execution statistics New child cursor created with new plan New plan changes the join order
33
Adaptive Statistics
SQL Plan Directives
Directives are additional information used during optimization to
34
SGA
DIRECTIVE CACHE
Directive: Use DS for customers table when column city, country are used
2
SQL statement is submitted
SHARED_POOL
Cursor 0: Select * from customers .. IS_REOPTIMIZABLE = Y
Sysaux Tablespace
35
SGA
DIRECTIVE CACHE
Directive: Use DS for customer table when column city, country are used
SHARED_POOL
Cursor 0: Select * from customers .. IS_REOPTIMIZABLE = Y Cursor 1: Select * from customers .. IS_REOPTIMIZABLE = N
Sysaux Tablespace
Extension Binds Stats Sqlid
36
SGA
DIRECTIVE CACHE
SHARED_POOL
Cursor 0: Select email from customers
Optimizer adds column group creation for state & country columns to next statistics gather on customer table
10
Optimizer checks for directives on customers table and finds one on the columns state and country
Sysaux Tablespace
Extension Binds Stats Sqlid
SQL Directives
37
38
39
40
42
43
Adaptive Statistics
Discovered at compile time Discovered at run time
Adaptive Plans
Adaptive Statistics
Join Methods
At compile time
At run time
44
Summary
Optimizer begins to learn from its mistakes
Adaptive Plans Adapt join methods Adapt parallel distribution methods Adaptive Statistics Dynamic statistics Re-optimization SQL Plan Directives
Agenda
Adaptive Query Optimization Optimizer Statistics enhancements SQL Plan Management enhancements
46
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
Histograms
Histograms tell Optimizer about the data distribution in a Column
WHERE clause or GROUP BY of a statement AND has a data skew Relies on column usage information gathered at compilation time and stored in SYS.COL_USAGE$ Four types of histograms
Frequency Top-Frequency
48 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Height balanced
Hybrid
Oracle Confidential
Histograms
Frequency Histograms (FREQUENCY)
A frequency histogram is only created if the number of distinct values in
Frequency histogram
49 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Oracle Confidential
Histograms
Top Frequency (TOP-FREQUENCY)
Traditionally a frequency histogram is only created if NDV < 254
But if a small number of values occupies most of the rows (>99% rows)
Creating a frequency histograms on that small set of values is very
popular values Built using the same technique used for frequency histograms Only created with AUTO_SAMPLE_SIZE
50
51
Histograms
Height Balanced Histograms (HEIGHT BALANCED)
A height balanced histogram is created if the number of distinct values
Histograms
Hybrid Histograms (HYBRID)
A height balanced histogram is created if the number of distinct values
Hybrid histogram
53 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Histograms
Hybrid Histograms (HYBRID)
Similar to height balanced histogram as created if the NDV >254
54
TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :
With a traditional height based histogram an even number of rows goes in each bucket Multiple buckets can have the same endpoint There are 960 rows in the sales table Automatically created histograms have 254 buckets That is 3.77 rows per bucket
TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :
BUCKET INFORMATION
Bucket 0 has end point 02-JAN-98
Multiple buckets have the same endpoint Max number of popular values that could have been recorded 96
58
TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :
As with traditional height based histogram want an even number of rows in each bucket But no two buckets have the same endpoint There are 960 rows in the sales table Automatically created histograms have 254 buckets That is 3.77 rows per bucket
TIME_ID
02-JAN-98 03-JAN-98 05-JAN-98 05-JAN-98 06-JAN-98 09-JAN-98 09-JAN-98 09-JAN-98 10-JAN-98 10-JAN-98 :
BUCKET INFORMATION
Bucket 1 has end point 02-JAN-98
FREQUENCY OF ENDPOINT
1
Frequency of endpoint values recorded in new column called endpoint repeat count
Potential to recorded 254 popular values :
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
Statistics available directly after load No additional table scan required to gather statistics
All internal maintenance operations that use CTAS benefit from this
Note only occurs on IAS if table is empty
64
65
66
Histogram & index statistics gathered using new GATHER AUTO option without regathering base column statistics
67
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
69
70
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
generated from partition level statistics NDV statistics can now be accurately aggregated by the introduction of the synopsis The synopses are stored in the Sysaux tablespace In 12c reduced the space required to store synopses on disk
72
NDV statistics can now be accurately aggregated The synopses are stored in the Sysaux tablespace 12c reduces the space required to store synopses on disk
Sysaux Tablespace
73 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
DBA
1. Create external table for flat files
INCREMENTAL_LEVEL
to TABLE
May 23rd 2012
4. Gather Statistics
5. Alter table Sales exchange partition May_24_2012 with table tmp_sales
74 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Sysaux Tablespace
Sales Table
May 18th 2012
When set to
USE_STALE_PERCENT DML on
less than 10% of rows in older partitions will not trigger regather
75 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Sysaux Tablespace
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
Statistics Enhancements
Concurrent Statistics Gathering Original introduced in 11.2
Gather statistics on multiple objects at the same time
job_queue_processes parameter
Each gather operation can still operate in parallel
77
Level 2 contain statistics gathering jobs for each partition in the partitioned tables
78 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
The number of concurrent gathers is controlled by the parameter job_queue_processes In this example it is set to 3
Sales Table
Partition 1 May 18th 2012
Job1
Partition 2
Remember each concurrent gather operates in parallel In this example parallel degree is 4
Job2
Job3
79
Statistics Enhancements
Multiple partitioned table support in 12c
over head Auto statistics gather job can now use concurrency Cap to limit the resource available to the job via resource manager
80
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
Extended Statistics
Two types of Extended Statistics
Column groups statistics
Column group statistics useful when multiple column from the same table are
determined
82 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
1.
84
Actual number of rows returned by this query is 145 Optimizer over-estimates the cardinality as it is not aware of the real-world relations between state & country
85
COLUMN USAGE REPORT FOR SH.CUSTOMERS FILTER means columns used together 1. COUNTRY_ID : EQ as filter predicates rather than join etc. 2. CUST_CITY : EQ Comes from query 1 3. CUST_STATE_PROVINCE : EQ 4. (CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID) : FILTER 5. (CUST_STATE_PROVINCE, COUNTRY_ID) : GROUP_BY
GROUP_BY columns used in group by expression in query 2
86
2.
Column group statistics will now be automatically maintained every time you gather statistics on this table
87 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Statistics Enhancements
New types of histograms Online statistics gathering Session level statistics for GGTs Enhanced incremental statistics
DBMS_STATS.REPORT_STATS_OPERATIONS
89
DBMS_STATS.REPORT_GATHER_SCHEMA_STATS
90
Agenda
Adaptive Query Optimization Optimizer Statistics enhancements SQL Plan Management enhancements
91
Plan Acceptable
Execute
Users
HJ HJ
Actual execution plans stored in SQL plan baseline in Oracle Database 12c
92
Users
NL NL
GB NL NL GB HJ HJ
Note you do not need to be in autocapture mode to have a new plan added to an existing SQL plan baseline
Note additional fields such as fetches, row processed etc. are not populated because new plan never executed
93
Plan Acceptable
Execute
Users
HJ HJ
GB NL NL GB HJ HJ
94
statement is active
95
Plan baseline
GB NL NL HJ GB HJ
It then test execute to check if new plan is as good as or better than old plan
2
Plans which dont perform as good as the original plan stay in the L plan history & are marked unaccepted but last_verified updated
96 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
HJ NL HJ
Plans which perform better than original plan are recommended to be added to the plan baseline
97
New recommendation section that clearly states how to implement the recommendations proposed
98
Summary
Optimizer begins to learn from its experiences
More Information
White papers
Optimizer with Oracle Database 12c
Optimizer Blog
http://blogs.oracle.com/optimizer
Oracle.com
http://www.oracle.com/technetwork/database/focus-areas/bi-
datawarehousing/dbbi-tech-info-optmztn-092214.html
101 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
102