Professional Documents
Culture Documents
AN EYE ON PERFORMANCE
110516346.doc
Author:
Brendan Furey
Creation Date:
2 May 2011
Version:
1.3
Last Updated:
25 September 2012
Page 1 of 52
Table of Contents
Introduction.......................................................................................................4
Hardware/Software Summary.......................................................................4
Overview...........................................................................................................5
Problem Description.....................................................................................5
Test Data Sets..............................................................................................6
Pivot and Prune Query Strategies.................................................................6
Pivot Strategies.....................................................................................................6
Prune Strategies....................................................................................................7
Strategy Modifiers.................................................................................................7
Test Combinations................................................................................................8
Testing Strategy...........................................................................................8
Output Files..................................................................................................9
The Queries.....................................................................................................10
JNSQ: Join, Subquery..................................................................................10
Query Text...........................................................................................................10
Query Diagram....................................................................................................11
Execution Plan Example (W64-D16)...................................................................11
Results.................................................................................................................12
110516346.doc
Page 2 of 52
Analysis of Results...........................................................................................37
Slice Analysis.............................................................................................37
Narrow Slice........................................................................................................37
Wide Slice............................................................................................................38
Shallow Slice.......................................................................................................40
Deep Slice...........................................................................................................41
Summary Analysis......................................................................................43
Tables and Graphs of Relative Performance ......................................................43
Which Query is Best?..........................................................................................44
Program Logic............................................................................................49
Example Output.........................................................................................50
References.......................................................................................................52
Change Record
Date
Author
Version
Change Reference
02-May-2011
03-May-2011
20-Jun-2011
25-Sep-2011
BPF
BPF
BPF
BPF
1.0
1.1
1.2
1.3
Initial
Typos and references
More typos and reference to new article
References now hyperlinks, and minor tidying
110516346.doc
Page 3 of 52
Introduction
It is a very common requirement in SQL to join two record sets where there is a one to many relationship
between the two sets, but where the cardinality of the result set is the same as that of the set on the 'one'
side. The obvious case is for standard grouping and aggregation querying, such as simply counting the
number of records in the 'many' set for each record in the 'one' set. There are also some slightly less
obvious cases where there may be various SQL techniques available, with varying performance and
complexity characteristics. This article looks at two such cases: the first where one wishes to join multiple
subtypes of a given entity this is generally referred to as pivoting from rows to columns; and the second
where one wishes to join just one record from the 'many' set, but does not have a pure join condition to
identify the record and so must use an ordering condition instead I will call this pruning.
This work attempts to find the best SQL techniques for such queries in Oracle 11g, mainly in terms of
performance. It does this by running a variety of queries within the context of an outbound interface
against a deliberately simple data model across a two-dimensional range of data sizes. A simple generic
PL/SQL package has been written to perform the testing efficiently, and it uses a previously described
object type for timing (a more recent version is described here: Code Timing and Object Orientation and
Zombies). Visio diagrams are provided for query structures, based on a similar approach previously
described (A Structured Approach to SQL Query Design), and Microsoft Excel graphs are used to display
comparative performances. The results reveal some interesting features of the behaviour of the Cost
Based Optimiser in Oracle 11g.
The output file is provided here.
Hardware/Software Summary
Component
Database
Diagrammer
Grapher
Operating System
Computer
110516346.doc
Description
Oracle Database 11g Express Edition Release 11.2.0.2.0 - Beta
Microsoft Visio 2003 (11.3216.5606)
Microsoft Excel 2007
Microsoft Windows 7 Home Premium (32 bit)
Samsung X120, 3GB memory, Intel U4100 @ 1.3GHz x 2 (XE only uses 1 though)
Page 4 of 52
Overview
Problem Description
The SQL problem is to obtain a single output record for each master record, to include a single best value
of a field in the detail record for each of several subtypes.
We consider a simple example scenario where the master table is employees in Oracles demo HR
schema, and the detail table is a new table, phone_numbers, a simplified version of hz_contact_points in
Oracle Applications. The phone_numbers table has records of four subtypes: HOME, WORK, MOBILE,
FAX, and a valid from date, where the most recent number is preferred for a given type, with the maximum
number used as a tie-breaker. Well assume that an employee should always be returned even if he has
no phone numbers.
In the real world, this kind of query might be used in an outbound interface that supplies the latest contact
details on a companys customers to a third-party supplier for marketing or other purposes. In that kind of
case, there would typically be a small number of detail records across a relative large table. This would be
the wide-shallow scenario discussed below. Another kind of real world example might be a CRM
110516346.doc
Page 5 of 52
(Customer Relations Management) system in which one wants a report on the most recent customer
interactions, and this could fall into the wide/narrow-deep scenarios.
W1
W2
W4
W8
W16
W32
W64
W128
107
214
428
856
1,712
3,424
6,848
13,696
423
851
1,707
3,419
6,843
13,691
27,387
54,779
D2
846
1,702
3,414
6,838
13,686
27,382
54,774
109,558
D4
16
1,692
3,404
6,828
13,676
27,372
54,764
109,548
219,116
D8
32
3,384
6,808
13,656
27,352
54,744
109,528
219,096
438,232
D16
64
6,768
13,616
27,312
54,704
109,488
219,056
438,192
876,464
D32
128
13,536
27,232
54,624
109,408
218,976
438,112
876,384
1,752,928
D64
256
27,072
54,464
109,248
218,816
437,952
876,224
1,752,768
3,505,856
D128
512
54,144
108,928
218,496
437,632
875,904
1,752,448
3,505,536
D256
1,024
108,288
217,856
436,992
875,264
1,751,808
3,504,896
7,011,072
D512
2,048
216,576
435,712
873,984
1,750,528
3,503,616
7,009,792
D102
4
4,096
433,152
871,424
1,747,968
3,501,056
7,007,232
14,019,58
4
7,011,712
14,023,42
4
28,046,84
8
56,093,69
6
14,022,14
4
28,044,28
8
Pivot
Strateg
y
Pivot
Join
Select
Scalar
Subquer
y
110516346.doc
Notes
Oracle 11g explicit PIVOT syntax.
Repeat the tables and joins for each subtype. This is quite a commonly
used strategy but might be expected to be inefficient due to additional
table reads. Efficiency can be improved by combining with the Oracle
10g feature of subquery factoring, using the WITH keyword.
Obtain each subtype column using a separate scalar subquery on the
detail table in the Select clause, correlated to the master record. The
detail table then does not appear in the main query. This might also be
expected to be inefficient since multiple queries are effectively repeated
for every row; however, there may be scenarios where this effect is
Page 6 of 52
Databas
e
function
Prune Strategies
Oracle Database version 9.0.1 introduced an explicit SQL syntax for pruning rows via aggregate
functions, using the keyword KEEP, with FIRST or LAST. Prior to this, pruning could be achieved by
combining various non-purpose-built techniques, and it seems that these alternatives continue to be
widely used today.
Prune
Strateg
y
Keep
Analytic
s
Select
Scalar
Subquer
y
Join
Scalar
Subquer
y
Order
By
Max
Notes
The maximum function can be applied to an expression on the detail
record with the KEEP clause specifying that the maximum is over only
those records ranking first (or last) in a given ordering. This is Oracles
explicit pruning syntax and so might be expected to perform well. From
the SQL manual (REF-1):
When you need a value from the first or last row of a sorted group, but
the needed value is not the sort key, the FIRST and LAST functions
eliminate the need for self-joins or views and enable better performance.
The detail table can be placed within an inline view where the analytic
function Row_Number can be selected to rank the records; a condition is
then placed outside the view that the rank equals 1.
The same approach as mentioned above for pivoting can also be used for
pruning, where the pruning takes place within its own subquery in the
Select clause; there are various possible subquery implementations
Select an expression defining the preferred record in a subquery
correlated to the master record, and join the main record. Aside from the
obvious performance disadvantage of additional reads of the detail table,
applying this strategy directly prevents outer joining; however, when
used in conjunction with subquery factoring, outer joining to the factor is
possible
The detail records are selected within an inline view in the preferred
order, then a condition is applied outside the view that ROWNUM = 1.
This has to be used within a database function: we tried to use it within a
select scalar subquery but the SQL is invalid.
Within a select scalar subquery a maximum is taken of an expression
that prepends the ordering fields to the desired select field, using
appropriate type casting (e.g. taking the Julian format for a date)
Strategy Modifiers
There are some general SQL techniques that can be used in conjunction with the above pivot and prune
strategies in some cases, and that may affect performance.
Strateg
y
Modifier
Subquer
y
Factorin
g
110516346.doc
Notes
This feature was introduced in Oracle 10g, and, using the WITH keyword,
allows a subquery to be defined once and subsequently referenced
several times by its alias. Where appropriate Oracle will run the subquery
in advance and store the results in a temporary table to improve
performance. Unlike with subqueries in the where clause of a query,
outer joins can be used.
Page 7 of 52
Inline
View
Test Combinations
The table below lists the queries tested with the strategies employed. Note that not all combinations of
strategies make sense, but we have tried to cover all that do.
Code
Pivot
Prune
Modifier
Notes
Main query set tested on both wide and deep data sets, except JNSQ wide only
Join Scalar
JNSQ
Join
Subquery
Join Scalar
Subquery
WJSQ
Join
Subquery
Factoring
Subquery
WJAN
Join
Analytic s
Factoring
Subquery
WJKP
Join
Keep
Factoring
PVAN
Pivot
Analytic s
Pruning is done
PVANIV
Pivot
Analytic s
Inline View
within the inline
view
PVKP
Pivot
Keep
Pruning is done
PVKPIV
Pivot
Keep
Inline View
within the inline
view
Database
FNSC
Order By
Inline View
Function
Additional queries tested on W1-D1024 only - we tried to match the performance
characteristics of FNSC on the narrow-deep data set without using a database
function
Select Scalar
SSSM
Max
Subquery
Select Scalar
SSKP
Keep
Subquery
Same pruning
strategy as
FNSC but within
query. The
Select Scalar
query is invalid
SSOB
Order By
Inline View
Subquery
owing to the
need to
correlate
through two
levels
Testing Strategy
In order to provide a realistic scenario, the queries are executed within the context of a simple outbound
interface that writes each record to a file as a comma-separated string. A small PL/SQL package has been
written to automate the testing. The program loops over width and depth dimensions, and for each data
set point makes a call to a separate package to set up the test data and have the CBO statistics gathered;
it then loops over a set of queries defined in the same separate package as strings that are executed by
the main package. Detailed timings are made using the timing objects mentioned earlier and written to a
generic log table. The total CPU and elapsed times for the query and data set point are written to a special
run statistics table, from which they can be output in a form easily loaded into Microsoft Excel. The times
can be viewed as a function of the 2-dimensional width-depth domain, and displayed as a 3-dimensional
surface graph for each query.
110516346.doc
Page 8 of 52
The execution plan is obtained in each case, using an Oracle API, and is written to the generic log. The
query string includes a random number that guarantees a hard-parse and thus recalculation of the
execution plans at each data set point. Further details of the packages and supporting tables are given in
a later section.
The testing strategy here seemed to me to provide much greater insight than the more conventional
approach of testing on a single large data set, and was later used again in Forming Range-Based Break
Groups with Advanced SQL, and other articles.
Output Files
The complete output files are provided in SQL Pivot and Prune Queries Output.
LST File
Test_Phone_8-7.LST
Test_Phone_411.LST
Test_Phone_111.LST
Test_Phone_111_3.LST
110516346.doc
Notes
Wide data set first 9 queries
Deep data set - first 9 queries, except JNSQ
W1-D1024, SSKP, SSSM, FNSC testing select scalar
subqueries
W1-D1024, PVKP, PVKPG, PVKPIV CBO grouping problem
Page 9 of 52
The Queries
For each query subsection, the following are provided (for the main queries, the last three were added
later and have a bit less):
The query text in static form for readability; the program in fact uses dynamic SQL and
consequently the random number at the end of the CSV string ensures that the query is hardparsed for each data set point
A query diagram. The diagrams use extended versions of notation first described in A Structured
Approach to SQL Query Design
The last Execution Plan output on the wide data set (usually)
A results section, split into wide and deep data sets, with tables of CPU times and 3-d graphs
generated from Microsoft Excel. Analysis is reserved for a later section
110516346.doc
Page 10 of 52
Query Diagram
Page 11 of 52
|
1 | SORT ORDER BY
|
|
1 |
220 | 1777
(2)| 00:00:22 |
|
2 |
NESTED LOOPS
|
|
1 |
220 | 1776
(1)| 00:00:22 |
|
3 |
NESTED LOOPS
|
|
1 |
201 | 1710
(1)| 00:00:21 |
|
4 |
NESTED LOOPS
|
|
1 |
182 | 1644
(2)| 00:00:20 |
|
5 |
NESTED LOOPS
|
|
1 |
155 | 1642
(2)| 00:00:20 |
|
6 |
NESTED LOOPS
|
|
1 |
128 | 1577
(2)| 00:00:19 |
|
7 |
NESTED LOOPS
|
|
3 |
327 | 1379
(2)| 00:00:17 |
|
8 |
NESTED LOOPS
|
|
1 |
82 | 1314
(2)| 00:00:16 |
|* 9 |
HASH JOIN
|
|
1 |
55 | 1313
(2)| 00:00:16 |
| 10 |
VIEW
| VW_SQ_1
| 4840 |
99K|
658
(2)| 00:00:08 |
| 11 |
HASH GROUP BY
|
| 4840 |
122K|
658
(2)| 00:00:08 |
|* 12 |
TABLE ACCESS FULL
| PHONE_NUMBERS
|
109K| 2774K|
653
(1)| 00:00:08 |
|* 13 |
TABLE ACCESS FULL
| PHONE_NUMBERS
|
109K| 3628K|
654
(1)| 00:00:08 |
| 14 |
TABLE ACCESS BY INDEX ROWID| EMPLOYEES
|
1 |
27 |
1
(0)| 00:00:01 |
|* 15 |
INDEX UNIQUE SCAN
| EMP_EMP_ID_PK
|
1 |
|
0
(0)|
|
|* 16 |
TABLE ACCESS BY INDEX ROWID | PHONE_NUMBERS
|
16 |
432 |
65
(0)| 00:00:01 |
|* 17 |
INDEX RANGE SCAN
| PHONE_NUMBERS_N2 |
64 |
|
2
(0)| 00:00:01 |
|* 18 |
VIEW PUSHED PREDICATE
| VW_SQ_4
|
1 |
19 |
66
(0)| 00:00:01 |
| 19 |
SORT GROUP BY
|
|
1 |
26 |
66
(0)| 00:00:01 |
|* 20 |
TABLE ACCESS BY INDEX ROWID| PHONE_NUMBERS
|
16 |
416 |
66
(0)| 00:00:01 |
|* 21 |
INDEX RANGE SCAN
| PHONE_NUMBERS_N2 |
64 |
|
3
(0)| 00:00:01 |
|* 22 |
TABLE ACCESS BY INDEX ROWID
| PHONE_NUMBERS
|
16 |
432 |
65
(0)| 00:00:01 |
|* 23 |
INDEX RANGE SCAN
| PHONE_NUMBERS_N2 |
64 |
|
2
(0)| 00:00:01 |
|* 24 |
TABLE ACCESS BY INDEX ROWID
| PHONE_NUMBERS
|
16 |
432 |
2
(0)| 00:00:01 |
|* 25 |
INDEX RANGE SCAN
| PHONE_NUMBERS_N2 |
64 |
|
2
(0)| 00:00:01 |
|* 26 |
VIEW PUSHED PREDICATE
| VW_SQ_3
|
1 |
19 |
66
(0)| 00:00:01 |
| 27 |
SORT GROUP BY
|
|
1 |
26 |
66
(0)| 00:00:01 |
|* 28 |
TABLE ACCESS BY INDEX ROWID
| PHONE_NUMBERS
|
16 |
416 |
66
(0)| 00:00:01 |
|* 29 |
INDEX RANGE SCAN
| PHONE_NUMBERS_N2 |
64 |
|
3
(0)| 00:00:01 |
|* 30 |
VIEW PUSHED PREDICATE
| VW_SQ_2
|
1 |
19 |
66
(0)| 00:00:01 |
| 31 |
SORT GROUP BY
|
|
1 |
26 |
66
(0)| 00:00:01 |
|* 32 |
TABLE ACCESS BY INDEX ROWID
| PHONE_NUMBERS
|
16 |
416 |
66
(0)| 00:00:01 |
|* 33 |
INDEX RANGE SCAN
| PHONE_NUMBERS_N2 |
64 |
|
3
(0)| 00:00:01 |
-------------------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------9 - access("VW_COL_1"=TO_CHAR(INTERNAL_FUNCTION("VALID_FROM"),'j')||"PHONE_NUMBER" AND
"ITEM_1"="PHO_F"."EMPLOYEE_ID" AND "ITEM_2"="PHO_F"."PHONE_TYPE")
12 - filter("SBQ_F"."PHONE_TYPE"='FAX')
13 - filter("PHO_F"."PHONE_TYPE"='FAX')
15 - access("PHO_F"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
16 - filter("PHO_H"."PHONE_TYPE"='HOME')
17 - access("PHO_H"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
18 - filter("VW_COL_1"="PHO_H"."SYS_NC00007$")
20 - filter("SBQ_H"."PHONE_TYPE"='HOME')
21 - access("SBQ_H"."EMPLOYEE_ID"="PHO_H"."EMPLOYEE_ID")
22 - filter("PHO_M"."PHONE_TYPE"='MOBILE')
23 - access("PHO_M"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
24 - filter("PHO_W"."PHONE_TYPE"='WORK')
25 - access("PHO_W"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
26 - filter("VW_COL_1"="PHO_W"."SYS_NC00007$")
28 - filter("SBQ_W"."PHONE_TYPE"='WORK')
29 - access("SBQ_W"."EMPLOYEE_ID"="PHO_W"."EMPLOYEE_ID")
30 - filter("VW_COL_1"="PHO_M"."SYS_NC00007$")
32 - filter("SBQ_M"."PHONE_TYPE"='MOBILE')
33 - access("SBQ_M"."EMPLOYEE_ID"="PHO_M"."EMPLOYEE_ID")
Results
Wide Data Set
Table of CPU Times
Note that the testing program iterates over the depth dimension within the width dimension
loop, and stops the inner iteration once a CPU time of 300 seconds has been exceeded, hence
the gaps.
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
1.35
1.73
2
2.15
2.34
4.01
7.3
13.53
1.29
1.94
2.71
3.49
5.37
10.26
20.02
34.59
1.51
2.39
4.02
8.99
11.63
21.29
34.43
67.66
1.89
4.28
7.47
24.91
15.56
28.7 154.86 298.62
2.95
7.83 163.68 320.74
608
85.89 878.53 324.23
5.98
16.54
55.29
213.4
14.59
59.48 159.65
292.08
Graph
110516346.doc
Page 12 of 52
110516346.doc
Page 13 of 52
Query Diagram
Page 14 of 52
|
1 | TEMP TABLE TRANSFORMATION |
|
|
|
|
|
|
|
2 |
LOAD AS SELECT
|
|
|
|
|
|
|
|* 3 |
HASH JOIN
|
|
1 |
64 | 1592K| 27704
(2)| 00:05:33 |
|
4 |
VIEW
| VW_SQ_1
| 38739 | 1134K|
| 14699
(2)| 00:02:57 |
|
5 |
SORT GROUP BY
|
| 38739 |
983K|
121M| 14699
(2)| 00:02:57 |
|* 6 |
TABLE ACCESS FULL
| PHONE_NUMBERS
| 3509K|
87M|
| 5285
(2)| 00:01:04 |
|
7 |
TABLE ACCESS FULL
| PHONE_NUMBERS
| 3509K|
113M|
| 5265
(1)| 00:01:04 |
|
8 |
SORT ORDER BY
|
| 13583 | 1313K| 1520K|
386
(2)| 00:00:05 |
|* 9 |
HASH JOIN RIGHT OUTER
|
| 13583 | 1313K|
|
78
(3)| 00:00:01 |
|* 10 |
VIEW
|
|
1 |
18 |
|
2
(0)| 00:00:01 |
| 11 |
TABLE ACCESS FULL
| SYS_TEMP_0FD9D66F|
1 |
19 |
|
2
(0)| 00:00:01 |
|* 12 |
HASH JOIN RIGHT OUTER |
| 13583 | 1074K|
|
76
(3)| 00:00:01 |
|* 13 |
VIEW
|
|
1 |
18 |
|
2
(0)| 00:00:01 |
| 14 |
TABLE ACCESS FULL
| SYS_TEMP_0FD9D66F|
1 |
19 |
|
2
(0)| 00:00:01 |
|* 15 |
HASH JOIN RIGHT OUTER |
| 13583 |
835K|
|
73
(2)| 00:00:01 |
|* 16 |
VIEW
|
|
1 |
18 |
|
2
(0)| 00:00:01 |
| 17 |
TABLE ACCESS FULL
| SYS_TEMP_0FD9D66F|
1 |
19 |
|
2
(0)| 00:00:01 |
|* 18 |
HASH JOIN RIGHT OUTER|
| 13583 |
596K|
|
71
(2)| 00:00:01 |
|* 19 |
VIEW
|
|
1 |
18 |
|
2
(0)| 00:00:01 |
| 20 |
TABLE ACCESS FULL | SYS_TEMP_0FD9D66F|
1 |
19 |
|
2
(0)| 00:00:01 |
| 21 |
TABLE ACCESS FULL
| EMPLOYEES
| 13583 |
358K|
|
68
(0)| 00:00:01 |
------------------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------3 - access("VW_COL_1"=TO_CHAR(INTERNAL_FUNCTION("VALID_FROM"),'j')||"PHONE_NUMBER" AND
"ITEM_0"="PHO"."EMPLOYEE_ID" AND "ITEM_1"="PHO"."PHONE_TYPE")
6 - filter(("SBQ"."PHONE_TYPE"='FAX' OR "SBQ"."PHONE_TYPE"='HOME' OR "SBQ"."PHONE_TYPE"='MOBILE' OR
"SBQ"."PHONE_TYPE"='WORK'))
9 - access("WIT_F"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
10 - filter("WIT_F"."PHONE_TYPE"='FAX')
12 - access("WIT_M"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
13 - filter("WIT_M"."PHONE_TYPE"='MOBILE')
15 - access("WIT_W"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
16 - filter("WIT_W"."PHONE_TYPE"='WORK')
18 - access("WIT_H"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
19 - filter("WIT_H"."PHONE_TYPE"='HOME')
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.13
0.22
0.24
0.37
0.67
1.17
2.16
4.44
0.14
0.19
0.28
0.41
0.68
1.29
2.26
4.62
0.15
0.18
0.28
0.45
0.75
1.39
2.75
5.05
0.16
0.24
0.31
0.56
0.93
1.67
3.3
6.14
0.16
0.28
0.38
0.63
1.17
2.19
4.34
8.7
0.22
0.3
0.5
0.93
1.73
3.4
6.47
13.22
0.3
0.42
0.79
1.42
2.73
5.7
11.6
22.62
Graph
110516346.doc
Page 15 of 52
Query
D1
D2
D4
D8
D16
D32
D64
D128
D256
D512
D1024
W1
W2
W4
W8
0.19
0.21
0.23
0.38
0.19
0.23
0.25
0.41
0.14
0.2
0.26
0.42
0.15
0.21
0.29
0.48
0.17
0.22
0.39
0.61
0.22
0.3
0.54
0.88
0.28
0.47
0.73
1.42
0.44
0.67
1.28
2.5
0.62
1.19
2.38
4.59
1.11
2.15
4.45
9.54
1.43
2.76
5.74
12.59
Graph
Page 16 of 52
ORDER BY 1
Query Diagram
Page 17 of 52
|* 8 |
VIEW
|
| 3509K|
60M|
| 5063
(1)| 00:01:01 |
|
9 |
TABLE ACCESS FULL
| SYS_TEMP_0FD9D6| 3509K|
63M|
| 5063
(1)| 00:01:01 |
|* 10 |
HASH JOIN RIGHT OUTER |
|
242G|
17T|
100M| 4830K (21)| 16:06:11 |
|* 11 |
VIEW
|
| 3509K|
60M|
| 5063
(1)| 00:01:01 |
| 12 |
TABLE ACCESS FULL
| SYS_TEMP_0FD9D6| 3509K|
63M|
| 5063
(1)| 00:01:01 |
|* 13 |
HASH JOIN RIGHT OUTER |
|
928M|
54G|
100M| 30789 (13)| 00:06:10 |
|* 14 |
VIEW
|
| 3509K|
60M|
| 5063
(1)| 00:01:01 |
| 15 |
TABLE ACCESS FULL
| SYS_TEMP_0FD9D6| 3509K|
63M|
| 5063
(1)| 00:01:01 |
|* 16 |
HASH JOIN OUTER
|
| 3552K|
152M|
| 5146
(1)| 00:01:02 |
| 17 |
TABLE ACCESS FULL
| EMPLOYEES
| 13583 |
358K|
|
68
(0)| 00:00:01 |
|* 18 |
VIEW
|
| 3509K|
60M|
| 5063
(1)| 00:01:01 |
| 19 |
TABLE ACCESS FULL | SYS_TEMP_0FD9D6| 3509K|
63M|
| 5063
(1)| 00:01:01 |
----------------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------3 - filter("ILV"."IND"=1)
4 - filter(ROW_NUMBER() OVER ( PARTITION BY "PHO"."EMPLOYEE_ID","PHO"."PHONE_TYPE" ORDER BY
INTERNAL_FUNCTION("PHO"."VALID_FROM") DESC ,"PHO"."PHONE_NUMBER")<=1)
7 - access("WIT_F"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
8 - filter("WIT_F"."PHONE_TYPE"='FAX')
10 - access("WIT_M"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
11 - filter("WIT_M"."PHONE_TYPE"='MOBILE')
13 - access("WIT_W"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
14 - filter("WIT_W"."PHONE_TYPE"='WORK')
16 - access("WIT_H"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
18 - filter("WIT_H"."PHONE_TYPE"='HOME')
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.14
0.14
0.24
0.34
0.64
1.06
2.11
4.31
0.14
0.16
0.27
0.44
0.58
1.11
2.25
4.39
0.14
0.19
0.28
0.45
0.67
1.25
2.39
4.97
0.14
0.2
0.28
0.47
0.83
1.53
3.07
6.19
0.16
0.23
0.39
0.69
1.13
2.13
4.42
8.72
0.17
0.32
0.52
0.97
1.74
3.48
7.05
12.49
0.34
0.45
0.85
1.65
3.41
6.71
10.42
21.31
Graph
Query
D1
D2
D4
D8
110516346.doc
W1
W2
W4
W8
0.15
0.2
0.27
0.32
0.14
0.19
0.26
0.39
0.14
0.19
0.27
0.42
0.14
0.19
0.27
0.44
Page 18 of 52
D16
D32
D64
D128
D256
D512
D1024
0.17
0.18
0.24
0.42
0.8
1.56
3.36
0.23
0.25
0.44
0.78
1.56
3.17
6.94
0.32
0.47
0.86
1.62
3.18
6.68
9.86
0.61
0.89
1.58
3.09
6.44
9.89
19.2
Graph
110516346.doc
Page 19 of 52
Query Diagram
Page 20 of 52
access("WIT_F"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
filter("WIT_F"."PHONE_TYPE"='FAX')
access("WIT_M"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
filter("WIT_M"."PHONE_TYPE"='MOBILE')
access("WIT_W"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
filter("WIT_W"."PHONE_TYPE"='WORK')
access("WIT_H"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
filter("WIT_H"."PHONE_TYPE"='HOME')
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.15
0.18
0.23
0.36
0.62
1.09
2.07
4.16
0.15
0.17
0.26
0.32
0.66
1.08
2.09
4.04
0.14
0.17
0.25
0.34
0.69
1.14
2.28
4.4
0.14
0.17
0.25
0.38
0.72
1.28
2.45
4.93
0.15
0.2
0.28
0.44
0.81
1.52
3.07
6.07
0.15
0.21
0.34
0.56
1.05
2
4.17
8.08
0.2
0.3
0.48
0.78
1.53
3.04
6.05
12.29
Graph
Query
D1
D2
D4
D8
D16
D32
D64
D128
D256
D512
D1024
W1
W2
W4
W8
0.14
0.14
0.21
0.38
0.14
0.16
0.23
0.36
0.14
0.17
0.25
0.36
0.16
0.17
0.25
0.39
0.14
0.19
0.28
0.5
0.15
0.22
0.34
0.53
0.2
0.3
0.41
0.8
0.23
0.33
0.64
1.21
0.36
0.53
1.02
2
0.48
0.9
1.86
4.18
0.82
1.7
3.51
7.31
Graph
110516346.doc
Page 21 of 52
Query Diagram
Page 22 of 52
|
1 | SORT ORDER BY
|
| 3480K|
169M|
| 51794
(1)| 00:10:22 |
|
2 |
HASH GROUP BY PIVOT
|
| 3480K|
169M|
| 51794
(1)| 00:10:22 |
|* 3 |
VIEW
|
| 3480K|
169M|
| 51510
(1)| 00:10:19 |
|* 4 |
WINDOW SORT PUSHED RANK|
| 3480K|
179M|
226M| 51510
(1)| 00:10:19 |
|* 5 |
HASH JOIN OUTER
|
| 3480K|
179M|
| 5348
(2)| 00:01:05 |
|
6 |
TABLE ACCESS FULL
| EMPLOYEES
| 13583 |
358K|
|
68
(0)| 00:00:01 |
|
7 |
TABLE ACCESS FULL
| PHONE_NUMBERS | 3509K|
90M|
| 5265
(1)| 00:01:04 |
---------------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------3 - filter("from$_subquery$_001"."IND"=1)
4 - filter(ROW_NUMBER() OVER ( PARTITION BY "EMP"."EMPLOYEE_ID","PHO"."PHONE_TYPE" ORDER
BY INTERNAL_FUNCTION("PHO"."VALID_FROM") DESC ,"PHO"."PHONE_NUMBER")<=1)
5 - access("PHO"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.15
0.15
0.19
0.32
0.56
1.08
2.07
3.9
0.11
0.16
0.22
0.35
0.62
1.17
2.29
4.28
0.13
0.14
0.21
0.39
0.69
1.31
2.53
4.98
0.14
0.17
0.28
0.44
0.86
1.67
3.26
6.52
0.15
0.2
0.33
0.64
1.26
2.44
4.92
9.41
0.19
0.3
0.55
1.04
2.08
4.13
7.71
15.79
0.29
0.47
0.97
1.9
3.84
6.88
13.67
29.13
Graph
Query
D1
D2
D4
D8
D16
D32
D64
D128
D256
D512
D1024
W1
W2
W4
W8
0.14
0.14
0.19
0.33
0.12
0.13
0.21
0.34
0.11
0.14
0.23
0.37
0.12
0.17
0.28
0.47
0.15
0.2
0.34
0.67
0.17
0.29
0.53
1
0.27
0.49
0.97
1.94
0.46
0.9
1.85
3.65
0.9
1.88
3.91
6.03
1.89
3.84
6.26
11.58
3.88
6.55
11.65
22.13
Graph
110516346.doc
Page 23 of 52
Query Diagram
110516346.doc
Page 24 of 52
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.11
0.12
0.21
0.33
0.61
1.06
2.02
3.81
0.11
0.15
0.23
0.39
0.61
1.12
2.04
4.17
0.11
0.16
0.22
0.39
0.67
1.23
2.35
4.7
0.13
0.19
0.28
0.45
0.78
1.49
2.86
5.91
0.14
0.19
0.33
0.62
1.12
2.19
4.11
8.19
0.21
0.26
0.45
0.91
1.82
3.56
6.96
13.03
0.23
0.46
0.78
1.63
3.2
6.8
10.69
22.97
Graph
Query
D1
D2
D4
D8
D16
D32
110516346.doc
W1
W2
W4
W8
0.11
0.14
0.19
0.33
0.11
0.14
0.22
0.36
0.11
0.15
0.21
0.36
0.14
0.16
0.25
0.42
0.12
0.18
0.31
0.54
0.18
0.26
0.46
0.78
Page 25 of 52
D64
D128
D256
D512
D1024
0.3
0.39
0.77
1.5
3.32
0.42
0.78
1.53
3.18
6.81
0.78
1.54
3.17
6.66
10.02
1.57
3.04
6.3
10.01
18.25
Graph
Query Diagram
110516346.doc
Page 26 of 52
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.14
0.17
0.2
0.31
0.58
0.98
2
3.84
0.14
0.15
0.22
0.32
0.61
1.04
2.14
4.16
0.12
0.14
0.22
0.35
0.63
1.17
2.4
4.74
0.09
0.15
0.24
0.4
0.7
1.44
2.83
5.66
0.14
0.17
0.3
0.5
1
1.87
3.8
7.84
0.14
0.21
0.41
0.71
1.34
2.8
5.76
12
0.2
0.33
0.54
1.11
2.24
4.77
9.56
20.19
Graph
Query
D1
D2
D4
D8
D16
D32
110516346.doc
W1
W2
W4
W8
0.12
0.14
0.19
0.38
0.13
0.12
0.2
0.32
0.12
0.14
0.25
0.38
0.12
0.15
0.22
0.39
0.14
0.19
0.3
0.49
0.14
0.24
0.37
0.68
Page 27 of 52
D64
D128
D256
D512
D1024
0.2
0.28
0.47
0.77
1.45
0.32
0.49
0.81
1.58
2.86
0.56
0.92
1.64
3.12
6.2
1.09
1.88
3.44
6.85
13.06
Graph
/* PVKPIV*/
first_name || ' ' || last_name || '","' || h || '","' || w || '","' || m || '","' || f || '599"'
employees
emp
JOIN (
SELECT pho.employee_id,
pho.phone_type,
MAX (pho.phone_number) KEEP (DENSE_RANK LAST ORDER BY pho.valid_from) phone_number_last
FROM phone_numbers pho
GROUP BY pho.employee_id, pho.phone_type
)
ilv
ON ilv.employee_id
= emp.employee_id
PIVOT (MAX(phone_number_last) FOR phone_type IN ('HOME' AS h, 'WORK' AS w, 'MOBILE' AS m, 'FAX' AS f))
ORDER BY 1
Query Diagram
110516346.doc
Page 28 of 52
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.13
0.16
0.21
0.35
0.57
1.01
1.95
3.83
0.14
0.16
0.22
0.33
0.6
1.03
2.01
3.97
0.11
0.15
0.21
0.35
0.62
1.07
2.14
4.23
0.14
0.17
0.23
0.37
0.65
1.21
2.42
4.79
0.13
0.19
0.27
0.49
0.76
1.43
2.87
5.86
0.1
0.17
0.35
0.6
1.01
1.97
3.94
7.79
0.16
0.24
0.4
0.73
1.47
3.14
5.97
11.99
Graph
Query
D1
D2
D4
D8
D16
D32
D64
110516346.doc
W1
W2
W4
W8
0.11
0.14
0.18
0.32
0.11
0.15
0.2
0.33
0.11
0.18
0.2
0.34
0.12
0.15
0.22
0.39
0.13
0.18
0.25
0.42
0.12
0.19
0.29
0.53
0.18
0.25
0.42
0.73
Page 29 of 52
D128
D256
D512
D1024
0.19
0.29
0.45
0.83
0.32
0.52
0.9
1.69
0.61
1.01
1.84
3.45
1.17
2.05
4.04
7.26
Graph
110516346.doc
Page 30 of 52
Query/Function Diagram
Page 31 of 52
|* 4 |
INDEX RANGE SCAN DESCENDING| PHONE_NUMBERS_N1 |
8 |
|
3
(0)| 00:00:01 |
--------------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------1 - filter(ROWNUM=1)
3 - filter("PHO"."PHONE_TYPE"=:B2)
4 - access("PHO"."EMPLOYEE_ID"=:B1)
Results
Wide Data Set
Table of CPU Times
Query
D1
D2
D4
D8
D16
D32
D64
W1
W2
W4
W8
W16
W32
W64
W128
0.35
0.56
0.87
1.49
2.85
5.6
12.02
21.98
0.36
0.64
0.83
1.53
2.9
5.52
11.87
22.13
0.41
0.89
0.86
1.61
2.82
5.61
11.57
22.75
0.46
0.45
0.9
1.51
2.87
5.77
11.7
22.68
0.28
0.44
0.9
1.56
2.9
5.78
11.45
22.9
0.28
0.45
0.87
1.52
3.03
5.72
11.44
23.59
0.31
0.5
0.78
1.53
3.04
5.88
11.72
28.61
Graph
Query
D1
D2
D4
D8
D16
D32
D64
D128
D256
D512
D1024
W1
0.3
0.34
0.38
0.47
0.28
0.27
0.25
0.25
0.25
0.26
0.28
W2
W4
W8
0.58
0.81
1.44
0.58
0.77
1.48
0.78
0.84
1.46
0.44
0.83
1.45
0.47
0.83
1.46
0.48
0.78
1.47
0.44
0.84
1.48
0.42
0.78
1.45
0.43
0.76
1.47
0.42
0.81
1.74
0.43
0.78
2.11
Graph
110516346.doc
Page 32 of 52
110516346.doc
|| '","' ||
BY valid_from)
BY valid_from)
BY valid_from)
BY valid_from)
Page 33 of 52
Query Diagram
Results
The query ran in 44 seconds CPU time. Compare with FNSC, which took 0.3 seconds for this data set
point.
Page 34 of 52
AND employee_id
) ||'133"'
FROM employees emp
ORDER BY 1
= emp.employee_id
Query Diagram
Results
The query ran in 46 seconds CPU time. Compare with FNSC, which took 0.3 seconds for this data set
point (but 0.7s in the last run in Test_Phone_1-11.LST).
Page 35 of 52
FROM
(SELECT pho.phone_number
FROM phone_numbers
pho
WHERE pho.phone_type
= 'WORK'
AND pho.employee_id
= emp.employee_id
ORDER BY pho.valid_from DESC)
WHERE ROWNUM = 1
) || '","' ||
(SELECT pho.phone_number
FROM
(SELECT pho.phone_number
FROM phone_numbers
pho
WHERE pho.phone_type
= 'MOBILE'
AND pho.employee_id
= emp.employee_id
ORDER BY pho.valid_from DESC)
WHERE ROWNUM = 1
) || '","' ||
(SELECT pho.phone_number
FROM
(SELECT pho.phone_number
FROM phone_numbers
pho
WHERE pho.phone_type
= 'FAX'
AND pho.employee_id
= emp.employee_id
ORDER BY pho.valid_from DESC)
WHERE ROWNUM = 1
) ||'999"'
FROM employees
emp
ORDER BY 1
Query Diagram
Results
The query fails with error:
ORA-00904: "EMP"."EMPLOYEE_ID": invalid identifier
110516346.doc
Page 36 of 52
Analysis of Results
Slice Analysis
We would like to like to compare the performances of the queries graphically, but as it would be difficult to
do this using the 3-dimensional graphs above, we will instead consider the four slices obtained by fixing
one of the width and depth dimensions at its minimum and maximum with the other displayed across its
range. For each slice and data set the CPU times are tabulated across all queries and a line graph is
given, then the relative performances are assessed in an analysis subsection.
Narrow Slice
Wide Data Set
Table of CPU Times
Query
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
JNSQ
D1
D2
D4
D8
D16
D32
D64
0.35
0.36
0.41
0.46
0.28
0.28
0.31
0.15
0.11
0.13
0.14
0.15
0.19
0.29
0.11
0.11
0.11
0.13
0.14
0.21
0.23
0.14
0.14
0.12
0.09
0.14
0.14
0.2
0.13
0.14
0.11
0.14
0.13
0.1
0.16
0.14
0.14
0.14
0.14
0.16
0.17
0.34
0.15
0.15
0.14
0.14
0.15
0.15
0.2
0.13
0.14
0.15
0.16
0.16
0.22
0.3
1.35
1.29
1.51
1.89
2.95
5.98
14.59
0.3
0.1
4
0.1
1
0.1
2
0.1
1
0.1
D2
0.3
4
0.1
2
0.1
1
0.1
3
0.1
1
0.1
D4
0.3
8
0.1
1
0.1
1
0.1
2
0.1
1
0.1
D8
0.4
7
0.1
2
0.1
4
0.1
2
0.1
2
0.1
D16
0.2
8
0.1
5
0.1
2
0.1
4
0.1
3
0.1
D32
0.2
7
0.1
7
0.1
8
0.1
4
0.1
2
0.1
D64
0.2
5
0.2
7
D128
D256
D512
D1024
0.25
0.25
0.26
0.28
0.46
0.9
1.89
3.88
0.3
0.39
0.77
1.5
3.32
0.2
0.1
8
0.2
0.28
0.47
0.77
1.45
0.19
0.42
0.29
0.8
0.45
1.56
0.83
3.36
Page 37 of 52
5
0.1
4
0.1
9
WJKP
WJSQ
Graph of CPU Times
4
0.1
4
0.1
9
4
0.1
4
0.1
4
4
0.1
6
0.1
5
7
0.1
4
0.1
7
8
0.1
5
0.2
2
4
0.2
0.2
8
0.23
0.36
0.48
0.82
0.44
0.62
1.11
1.43
Analysis
Wide Data Set
This is a subset of the deep data set for the narrow slice (other than having wider date range), but was the
only set that JNSQ was tested on (to save time).
FNSC is worst up to depth of 32 but best from 256, taking 34% of the next best time at maximum
depth
FNSC shows essentially constant time; this is presumably because the main query scans the
unchanging employees table, while the function query uses the index and is coded to stop after
the first row and so may be able to obtain the rowid in the same small number of reads.
PVPKIV and WJKP show similar performance as next best at 2.9 x best
PVKP and WJSQ show similar performance as next best at 5.1 x best
PVANIV AND WJAN show similar performance as next best at 11.9 x best, with PVAN just behind
at 13.9
Wide Slice
Wide Data Set W128
Table of CPU Times
Query
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
JNSQ
D1
D2
D4
D8
D16
D32
D64
21.98
22.13
22.75
22.68
22.9
23.59
28.61
3.9
4.28
4.98
6.52
9.41
15.79
29.13
3.81
4.17
4.7
5.91
8.19
13.03
22.97
3.84
4.16
4.74
5.66
7.84
12
20.19
3.83
3.97
4.23
4.79
5.86
7.79
11.99
4.31
4.39
4.97
6.19
8.72
12.49
21.31
4.16
4.04
4.4
4.93
6.07
8.08
12.29
4.44
4.62
5.05
6.14
8.7
13.22
22.62
13.53
34.59
67.66 298.62 324.23
Page 38 of 52
1.4
4
0.3
3
0.3
3
0.3
8
0.3
2
0.3
2
0.3
8
0.3
8
WJSQ
Graph of CPU Times
D2
1.4
8
0.3
4
0.3
6
0.3
2
0.3
3
0.3
9
0.3
6
0.4
1
D4
1.4
6
0.3
7
0.3
6
0.3
8
0.3
4
0.4
2
0.3
6
0.4
2
D8
1.4
5
0.4
7
0.4
2
0.3
9
0.3
9
0.4
4
0.3
9
0.4
8
D16
1.4
6
0.6
7
0.5
4
0.4
9
0.4
2
0.6
1
0.5
0.6
1
D32
1.4
7
1
0.7
8
0.6
8
0.5
3
0.8
9
0.5
3
0.8
8
D64
1.4
8
1.9
4
1.5
7
1.0
9
0.7
3
1.5
8
0.8
1.4
2
D128
D256
D512
D1024
1.45
1.47
3.65
6.03
3.04
6.3
1.74
11.5
8
10.0
1
1.88
3.44
6.85
2.11
22.1
3
18.2
5
13.0
6
1.17
2.05
4.04
7.26
3.09
6.44
9.89
19.2
1.21
4.18
2.5
4.59
9.54
7.31
12.5
9
Analysis
Wide Data Set
110516346.doc
Page 39 of 52
This is a subset of the deep data set for the wide slice (other than having wider date range), but was the
only set that JNSQ was tested on (to save time).
JNSQ took 55 x best time at the maximum depth it was run at (D16)
FNSC is worst up to depth of 32 but best from 256, taking 29% of the next best time at maximum
depth
FNSC shows essentially constant time for the first 9 depths; this is presumably because the main
query scans the unchanging employees table, while the function query uses the index and is
coded to stop after the first row and so may be able to obtain the rowid in the same small number
of reads. The last two depths show increases of 18% and a further 21%; possibly the greater
sizes of the index have increased the index depth and hence the numbers of blocks read, but I
have not verified this.
PVPKIV and WJKP show similar performance as next best at 3.5 x best at maximum depth
PVKP and WJSQ show similar performance as next best at 6.2 x best
PVANIV AND WJAN show similar performance as next best at 8.9 x best, with PVAN just behind
at 10.5
Apart from FNSC, the relative performances remain largely similar after divergence appears from
about D8
Shallow Slice
Wide Data Set
Table of CPU Times
Query
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
JNSQ
W1
W2
W4
W8
W16
W32
W64
W128
0.35
0.56
0.87
1.49
2.85
5.6
12.02
21.98
0.15
0.15
0.19
0.32
0.56
1.08
2.07
3.9
0.11
0.12
0.21
0.33
0.61
1.06
2.02
3.81
0.14
0.17
0.2
0.31
0.58
0.98
2
3.84
0.13
0.16
0.21
0.35
0.57
1.01
1.95
3.83
0.14
0.14
0.24
0.34
0.64
1.06
2.11
4.31
0.15
0.18
0.23
0.36
0.62
1.09
2.07
4.16
0.13
0.22
0.24
0.37
0.67
1.17
2.16
4.44
1.35
1.73
2
2.15
2.34
4.01
7.3
13.53
Query
FNSC
110516346.doc
W1
W2
W4
W8
0.3
0.58
0.81
1.44
Page 40 of 52
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
0.14
0.11
0.12
0.11
0.15
0.14
0.19
0.14
0.14
0.14
0.14
0.2
0.14
0.21
0.19
0.19
0.19
0.18
0.27
0.21
0.23
0.33
0.33
0.38
0.32
0.32
0.38
0.38
Analysis
Wide Data Set
JNSQ is worst up to depth of 8, then second worst to FNSC, with JNSQ and FNSC taking 3.6 and
5.8 times the best time at maximum width
PVAN, PVANIV, PVKP and PVPKIV are best, taking about 90% of the times of the next
group at maximum width
WJAN, WJKP and WJSQ take about 1.1 x the times of the best group
Query
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
JNSQ
W1
W2
W4
W8
W16
W32
W64
W128
0.31
0.5
0.78
1.53
3.04
5.88
11.72
28.61
0.29
0.47
0.97
1.9
3.84
6.88
13.67
29.13
0.23
0.46
0.78
1.63
3.2
6.8
10.69
22.97
0.2
0.33
0.54
1.11
2.24
4.77
9.56
20.19
0.16
0.24
0.4
0.73
1.47
3.14
5.97
11.99
0.34
0.45
0.85
1.65
3.41
6.71
10.42
21.31
0.2
0.3
0.48
0.78
1.53
3.04
6.05
12.29
0.3
0.42
0.79
1.42
2.73
5.7
11.6
22.62
14.59
59.48 159.65
292.08
110516346.doc
Page 41 of 52
Query
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
W1
W2
W4
W8
0.28
0.43
0.78
2.11
3.88
6.55
11.65
22.13
3.32
6.81
10.02
18.25
1.45
2.86
6.2
13.06
0.83
1.69
3.45
7.26
3.36
6.94
9.86
19.2
0.82
1.7
3.51
7.31
1.43
2.76
5.74
12.59
Analysis
Wide Data Set
110516346.doc
JNSQ is worst in all cases where it was run, having 96 times the best time at the maximum width
it was run at (W32)
PVPKIV and WJKP are best, taking about 57% of the times of the next group at
maximum width
PVANIV, PVKP, WJAN, and WJSQ take about 1.8 x the times of the best group
FNSC and PVAN take about 2.4 x the times of the best group
Page 42 of 52
Summary Analysis
In this section, the relative rankings and performance factors are tabulated across the query sets and
slices for both data sets.
Tables and Graphs of Relative Performance
Table of Rankings
DataSet
>
Slice ->
End ->
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
JNSQ
Deep
Wide
Shallow
Narro
Wide
w
8
9
6
1
1
1
4
1
2
1
4
5
6
5
2
6
9
8
Deep
Narro
Wide
w
6
7
5
7
4
5
2
3
1
1
8
4
2
1
6
5
9
9
Narrow
Shallo
Deep
w
8
1
4
8
1
6
3
4
1
2
6
6
4
2
7
4
NA
NA
Wide
Shallo
Deep
w
8
1
1
8
1
6
5
4
1
2
1
6
5
2
5
4
NA
NA
DataSet
>
Slice ->
End ->
FNSC
PVAN
PVANIV
PVKP
PVKPIV
WJAN
WJKP
WJSQ
JNSQ
Deep
Wide
Shallow
Narro
Wide
w
3.2
5.8
1.4
1
1
1
1.3
1
1.2
1
1.3
1.1
1.4
1.1
1.2
1.2
12.3
3.6
Deep
Narro
Wide
w
1.9
2.4
1.8
2.4
1.4
1.9
1.3
1.7
1
1
2.1
1.8
1.3
1
1.9
1.9
91.2
0
Narrow
Shallo
Deep
w
2.7
1
1.3
13.9
1
11.9
1.1
5.2
1
3
1.4
12
1.3
2.9
1.7
5.1
NA
NA
Wide
Shallo
Deep
w
4.5
1
1
10.5
1
8.6
1.2
6.2
1
3.4
1
9.1
1.2
3.5
1.2
6
NA
NA
Page 43 of 52
It is not surprising that the query performing best overall (PVKPIV) uses Oracles native syntax for
both pivoting and pruning, being better in particular than those pruning by use of the more general
technique of analytic functions. It is very surprising though that the inline view modifier is needed
to obtain the best performance, as is the result that using a database function outperforms all
others in some scenarios. These points are discussed in the next section.
WJKP shows a very similar performance profile to PVKPIV, but slightly worse; it seems that the
execution plans, although superficially different, may amount to much the same but with the
overhead of writing to the temporary table likely explaining the differences.
By experiment, I found that replacing the employee names expression with its components led to the CBO
obtaining an accurate estimate.
Note that the two forms are equivalent, as Oracle clearly knows at the syntax-checking level, or it would
raise the error: ORA-00979: not a GROUP BY expression
The two forms were tested on the W1-D1024 data set point, along with PVKPIV (see Test_Phone_111_3.LST for full listing), and the two explain plans were:
110516346.doc
Page 44 of 52
Original
-----------------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes |TempSpc| Cost (%CPU)| Time
|
-----------------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
|
|
|
| 2573 (100)|
|
|
1 | SORT ORDER BY
|
| 18565 |
688K|
| 2573
(2)| 00:00:31 |
|
2 |
HASH GROUP BY PIVOT |
| 18565 |
688K|
| 2573
(2)| 00:00:31 |
|
3 |
VIEW
|
| 18565 |
688K|
| 2570
(1)| 00:00:31 |
|
4 |
SORT GROUP BY
|
| 18565 |
815K|
24M| 2570
(1)| 00:00:31 |
|* 5 |
HASH JOIN OUTER
|
|
433K|
18M|
|
690
(2)| 00:00:09 |
|
6 |
TABLE ACCESS FULL| EMPLOYEES
|
107 | 2033 |
|
68
(0)| 00:00:01 |
|
7 |
TABLE ACCESS FULL| PHONE_NUMBERS |
433K|
10M|
|
619
(1)| 00:00:08 |
-----------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------5 - access("PHO"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
Modified
---------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost (%CPU)| Time
|
---------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
|
|
|
708 (100)|
|
|
1 | SORT ORDER BY
|
|
303 | 11514 |
708
(4)| 00:00:09 |
|
2 |
HASH GROUP BY PIVOT |
|
303 | 11514 |
708
(4)| 00:00:09 |
|
3 |
VIEW
|
|
303 | 11514 |
706
(4)| 00:00:09 |
|
4 |
SORT GROUP BY
|
|
303 | 13635 |
706
(4)| 00:00:09 |
|* 5 |
HASH JOIN OUTER
|
|
433K|
18M|
690
(2)| 00:00:09 |
|
6 |
TABLE ACCESS FULL| EMPLOYEES
|
107 | 2033 |
68
(0)| 00:00:01 |
|
7 |
TABLE ACCESS FULL| PHONE_NUMBERS |
433K|
10M|
619
(1)| 00:00:08 |
---------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------5 - access("PHO"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
1.43
PVKPG (modified)
1.31
PVKPIV
.84
Although the plans appear the same, the modified query appeared to be consistently slightly faster (Is the
plan output hiding some differences under the covers? Incidentally, if the reader has not yet read Under
the Net, the first, maybe best, book by the late English writer Iris Murdoch, its highly recommended). The
change thus made little difference here, but in another query it might.
Inline View Modifier
Using an inline view to enclose the pruning operation is always at least as good as not, and up to 21%
faster for the analytics strategy, and 59% faster for the keep strategy. The view guides the CBO to reduce
the number of records being processed before joining them to the master records. This is extremely
surprising because one would expect the CBO to work this out for itself, especially on our deliberately
simple queries.
Subquery Factoring Modifier
Applying the subquery factor modifier proved extremely effective for the join subquery strategy, both in
performance and in allowing outer joining.
It is interesting to note from the detailed timings in the output files (SQL Pivot and Prune Queries Output)
that, while Oracle normally defers execution of a query until the first fetch shows its needed, in all cases
with a subquery factor execution occurs straight away on cursor opening.
Performance of JNSQ
One of the benefits of testing the queries across a 2 dimensional domain is that one can compare
performances across data set points, and one sees immediately that the CBO has performed very poorly
on some of these for JNSQ. In one case, doubling the number of master records reduced the CPU time by
a factor of 7!
110516346.doc
Page 45 of 52
110516346.doc
Page 46 of 52
110516346.doc
Page 47 of 52
Table Structures
Generic Tables
RUN_CONTROL
Column Name
id*
description
status
message
point_wide_max
point_deep_max
cpu_time
elapsed_time
creation_date
Type
Number
Char(500)
Char(1)
Char(4000)
Number
Number
Number
Number
Date
Notes
Sequence generated primary key
Description of run
S success, F - failure
Error message if any
Maximum width point
Maximum depth point
Total CPU time
Total elapsed time
Creation date
Type
Number
Char(60)
Number
Number
Number
Number
Date
Char(1)
Char(4000)
Notes
Foreign key to RUN_CONTROL table
Query code
Width point
Depth point
CPU time
Elapsed time
Creation date
S success, F - failure
Error message if any
RUN_STATISTICS
Column Name
run_control_id*
run_type*
point_wide*
point_deep*
cpu_time
elapsed_time
creation_date
status
message
110516346.doc
Page 48 of 52
OUTPUT_LOG
Column Name
line_ind
line_text
id
creation_date
Type
Integer
Char(4000)
Char(30)
Date
Notes
Line number
Line text
Identifier code
Creation date
Type
Number
Char(20)
Char(25)
phone_number
hire_date
job_id
salary
commission_pct
manager_id
department_id
Number
Date
Number
Number
Number
Number
Number
Char(20)
Notes
Sequence generated primary key
First name copy number appended when copying
Last name
Email address copy number appended when copying for
uniqueness, but not referenced in queries
Index EMP_EMP_ID_PK
employee_id
Type
Number
Number
Char(10)
Date
Date
Char(9)
Notes
Sequence generated primary key
Foreign key to EMPLOYEES
Phone number type (HOME, WORK, MOBILE, FAX)
Valid from date (set randomly within a range)
Valid to date (not set)
Phone number (set randomly within a range)
Index PHONE_NUMBERS_N1
employee_id
valid_from
Index PHONE_NUMBERS_N2
employee_id
The idea here is to ensure that the queries have reasonable indexes to use if the CBO deems it advisable,
but to avoid including all fields in an index, for the purpose of this testing. The function-based index is
included to provide a usable index for the subquery strategies.
Program Logic
110516346.doc
Setup test data, and gather CBO statistics, using Query_Test_Set package
Call interface
End loop
End loop
End loop
Example Output
The output is included in SQL Pivot and Prune Queries Output. Here is an extract, with indentation
removed, and the first 100 characters shown thereafter per line (the version of code included also writes
the number of lines written, but that was added after the example below was run):
Setup data: 7-1024 Phone numbers truncated
3424 numbers added, (iteration 1 of 1024)
.
.
.
3424 numbers added, (iteration 1024 of 1024)
3506176 numbers added in total
4096 numbers deleted for emp 121
1024 numbers deleted for emp 196, type FAX
79014 duplicate dates
527835 duplicate numbers
856 total employees
3501056 total numbers
Timings
Elapsed (CPU)
Delete/Add employees
0.00 (0.00)
Add numbers
5,166.15 (637.35)
Gather stats
55.51 (20.32)
Count duplicate dates
20.94 (12.53)
Count duplicate numbers
9.94 (6.27)
Count totals
4.46 (0.98)
Total times:
5,257.00 (677.45)
(Other):
0.00 (0.00)
PVKP
SELECT /* PVKP*/ '"' || emp_name || '","' || h || '","' || w || '","' || m || '","' || f || '8943"'
ame emp_name, pho.phone_type, Max (pho.phone_number) KEEP (DENSE_RANK LAST ORDER BY pho.valid_from)
ne_numbers pho ON pho.employee_id = emp.employee_id GROUP BY emp.first_name || ' ' || emp.last_name,
R phone_type IN ('HOME' AS h, 'WORK' AS w, 'MOBILE' AS m, 'FAX' AS f)) ORDER BY 1
SQL_ID 9xt2by4jcdg1x, child number 0
------------------------------------SELECT /* PVKP*/ '"' || emp_name || '","' || h || '","' || w || '","'
|| m || '","' || f || '8943"' FROM ( SELECT emp.first_name || ' ' ||
emp.last_name emp_name, pho.phone_type, Max (pho.phone_number) KEEP
(DENSE_RANK LAST ORDER BY pho.valid_from) phone_number_last FROM
employees emp LEFT JOIN phone_numbers pho ON pho.employee_id =
emp.employee_id GROUP BY emp.first_name || ' ' || emp.last_name,
pho.phone_type ) PIVOT (Max(phone_number_last) FOR phone_type IN
('HOME' AS h, 'WORK' AS w, 'MOBILE' AS m, 'FAX' AS f)) ORDER BY 1
Plan hash value: 3462645326
-----------------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes |TempSpc| Cost (%CPU)| Time
|
-----------------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
|
|
|
| 22768 (100)|
|
110516346.doc
Page 50 of 52
|
1 | SORT ORDER BY
|
|
148K| 5511K|
| 22768
(1)| 00:04:34 |
|
2 |
HASH GROUP BY PIVOT |
|
148K| 5511K|
| 22768
(1)| 00:04:34 |
|
3 |
VIEW
|
|
148K| 5511K|
| 22757
(1)| 00:04:34 |
|
4 |
SORT GROUP BY
|
|
148K| 7686K|
227M| 22757
(1)| 00:04:34 |
|* 5 |
HASH JOIN OUTER
|
| 3495K|
176M|
| 5348
(2)| 00:01:05 |
|
6 |
TABLE ACCESS FULL| EMPLOYEES
|
856 | 22256 |
|
68
(0)| 00:00:01 |
|
7 |
TABLE ACCESS FULL| PHONE_NUMBERS | 3495K|
90M|
| 5265
(1)| 00:01:04 |
-----------------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------5 - access("PHO"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
Timings
Elapsed (CPU)
Open cursor
0.04 (0.03)
First fetch
13.34 (12.78)
Remaining fetches
0.10 (0.04)
Write to file
0.08 (0.08)
Write plan
0.12 (0.05)
Total times:
13.71 (13.06)
(Other):
0.03 (0.08)
Summary for 8 * 107 = 856 employees with 1024 * 4 = 4096 numbers per employee
Timings
Elapsed (CPU)
Run_One - Total
13.72 (13.06)
Total times:
13.72 (13.06)
(Other):
0.00 (0.00)
PVAN
.
.
.
110516346.doc
Page 51 of 52
References
REF
REF-4
REF-5
REF-6
Document
Oracle Database SQL Language Reference 11g Release 2
(11.2)
Oracle Database Concepts 11g Release 2 (11.2)
Oracle Database PL/SQL Packages and Types Reference
11g Release 2 (11.2)
A Structured Approach to SQL Query Design
Code Timing and Object Orientation and Zombies
SQL Pivot and Prune Queries Output
REF-7
REF-8
REF-9
REF-1
REF-2
REF-3
110516346.doc
Details
http://www.oracle.com/pls/db112
Page 52 of 52