Customer Submitted Case Studies

Case Study: Plan Stability using oracle stored outlines

Author: Rahul Gupta , ASE Email : ecengineer84@gmail.com http://onlineerpdba.blogspot.com/ Skill Level Rating for this Case Study: Intermediate

About Oracle Customer Submitted Case Studies
Oracle Customer Submitted Case Studies are intended as learning tools and for sharing information or knowledge related to a complex event, process, procedure, or to a series of related events. Each case study is written based upon the experience that the writer/s encountered. Customers should not interpret or use information in these case studies as solutions or recommendations without first contacting Oracle Support. Each Case Study contains a skill level rating. The rating provides an indication of what skill level the reader should have as it relates to the information in the case study. Ratings are:
• • •

Expert: significant experience with the subject matter Intermediate: some experience with the subject matter Beginner: little experience with the subject matter

Case Study Abstract
This case study is discussed that how we can stabilize a query plan using outlines. It might happen that, for a SQL statement due to some reason optimizer is opting for a wrong plan instead of the optimize plan. Hints can be used to guide optimizer optimizer, but in some cases it is not possible to edit application SQL to add index hint. In this case stored outline can be used to select a desired plan. But text of the SQL should be same while using stored outline which will not be case after adding index hint.

Case History

This case study is based on a real issue running on an oracle 9.2.0.5 database. This is about how plan change after gathering the statistics and how can be stabilize them using outline. Concurrent " EOTH daily " a customized one was running proper earlier taking around 2-3 hours . But user started to complain that it's not getting completed even in whole day. Moreover, COLD backup was scheduled on daily basis. So Concurrent gets restarted after the COLD backup. After analyzing we found the plan of the query has been changed because of monthly scheduled stats gather activity. As this Concurrent was very critical thus we’re not allowed to change any logic in the program source code. So we decided to create the outline for the same.

Pre-Analysis Work (optional)
Summary We ask the user to unable the trace for the problematic concurrent program. In Oracle APPS environment trace can be enable through the front end using. System Administrator Concurrent Program Define

Here search for the program name and then click on the Enable Trace ( there at the left – bottom )

But it won’t work for as the program was not getting completed within the 1 day time and thus it was not possible to us to find the problematic query using trace. So we decided to monitor at runtime and thus we used the query mentioned below to find the sid of the concurrent program with the help of its request id.
Page 2

select s.sid from v$session S ,fnd_concurrent_requests R where r.oracle_session_id=S.audsid and r.request_id=&request_id So with the help of above mentioned query we find the sid for the respective concurrent program. select s.sid from v$session S ,fnd_concurrent_requests R where r.oracle_session_id=S.audsid and r.request_id=118030209

We got the sid as 135 and thus we used the v$sqlarea & v$session to find the problematic sql . We found that one query with hash_value 2925922762 was running for a very long time and didn’t complete

Select a.sql_text , a.hash_value from v$sqlarea a , v$session b where a.hash_value = b.sql_hash_value and b.sid=135 [ S u mSql_text Hash_value m a SELECT DISTINCT X.INVENTORY_ITEM_ID 2925922762 r FROM i EINV_ONHAND_TXN_HISTORY_DAILY X, z EINV_PRICE_LIST_GT Y e WHERE X.INVENTORY_ITEM_ID = Y.INVENTORY_ITEM_ID t AND X.CREATION_DAY =NVL(:B1,TRUNC(SYSDATE)-1) h e . Detail Now we got our problematic query. Thus we used $ORACLE_HOME/rdbms/admin/sprepsql.sql to find the plan stability of the query with hash value 2925922762 . Metalink Doc : 209197.1 can be used to find more information about the sprep.sql

Page 3

STATSPACK SQL report for Hash Value: 2925922762

Module: EOTHDAILY

DB Name DB Id Instance Inst Num Release Cluster Host ------------ ----------- ------------ -------- ----------- ------- -----------ILPROD 2027196492 ILPROD 1 9.2.0.5.0 NO ilerpdb01 Start Id Start Time End Id End Time Duration(mins) --------- ------------------- --------- ------------------- -------------11955 25-May-08 06:00:02 11968 25-May-08 19:00:03 780.02

SQL Statistics ~~~~~~~~~~~~~~ -> CPU and Elapsed Time are in seconds (s) for Statement Total and in milliseconds (ms) for Per Execute % Snap Statement Total Per Execute Total --------------- --------------- -----Buffer Gets: 29,099,706 29,099,706.0 .05 Disk Reads: 1,548,458 1,548,458.0 .20 Rows processed: 4,481 4,481.0 CPU Time(s/ms): 460 459,520.0 Elapsed Time(s/ms): 19,992 19,991,502.4 Sorts: 1 1.0 Parse Calls: 1 1.0 Invalidations: 0 Version count: 1 Sharable Mem(K): 55 Executions: 1 SQL Text ~~~~~~~~ SELECT DISTINCT X.INVENTORY_ITEM_ID FROM EINV_ONHAND_TXN_HISTORY _DAILY X,EINV_PRICE_LIST_GT Y WHERE X.INVENTORY_ITEM_ID = Y.INVE NTORY_ITEM_ID AND X.CREATION_DAY =NVL(:B1,TRUNC(SYSDATE)-1)

Plan Hash Value Snap Id Cost Optimizer ------------ -------- ---------- -------------------4209813261 8152 20 CHOOSE 1573558166 9713 19 CHOOSE 501312428 11598 20 CHOOSE 3745587098 11706 22 CHOOSE

-------------------------------------------------------------------------------| Operation | PHV/Object Name | Rows | Bytes| Cost | -------------------------------------------------------------------------------|SELECT STATEMENT |----- 3745587098 ----| | | 22 | |SORT UNIQUE | | 1 | 27 | 22 | | CONCATENATION | | | | | | FILTER | | | | | | TABLE ACCESS BY INDEX ROWID |EINV_ONHAND_TXN_HIST | 1 | 14 | 2 | | NESTED LOOPS | | 1 | 27 | 3 | | INDEX FULL SCAN |EINV_PRICE_LIST_GT_N | 1 | 13 | | | INDEX RANGE SCAN |EINV_ONHAND_ITEM_ID_ | 1K| | 2 | | FILTER | | | | | | TABLE ACCESS BY INDEX ROWID |EINV_ONHAND_TXN_HIST | 1 | 14 | 2 | | NESTED LOOPS | | 1 | 27 | 3 | | INDEX FULL SCAN |EINV_PRICE_LIST_GT_N | 1 | 13 | | | INDEX RANGE SCAN |EINV_ONHAND_ITEM_ID_ | 1K| | 2 | --------------------------------------------------------------------------------

Page 4

From the sprep.sql we found the the plan was continuously changing and it’s not good. So we ask the user to run the same program on test environment with enable trace and on test it got completed within 2 hours . From there we got the optimize plan for the problematic query .

Optimize Plan ( On test environment )
12:09:00 SQL > explain plan for 12:09:08 2 SELECT DISTINCT X.INVENTORY_ITEM_ID FROM EINV_ONHAND_TXN_HISTORY_DAILY X,EINV_PRICE_LIST_GT Y WHERE X.INVENTORY_ITEM_ID = Y.INVENTORY_ITEM_ID AND X.CREATION_DAY =NVL(:B1,TRUNC(SYSDATE)-1)12:09:14 3 ; Explained. 12:09:15 SQL > @?/rdbms/admin/utlxpls PLAN_TABLE_OUTPUT ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost | -----------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 27 | 48 | | 1 | SORT UNIQUE | | 1 | 27 | 48 | | 2 | NESTED LOOPS | | 1 | 27 | 32 | | 3 | TABLE ACCESS BY INDEX ROWID| EINV_ONHAND_TXN_HISTORY_DAILY | 1362 | 19068 | 32 | |* 4 | INDEX RANGE SCAN | EINV_ONHAND_TXN_HIST_DAILY_UK | 1362 | | 7 | |* 5 | INDEX RANGE SCAN | EINV_PRICE_LIST_GT_N1 | 1 | 13 | | ------------------------------------------------------------------------------------------------

Plan on Production environment i.e., unturned plan or changed plan

Untuned Plan ( on Production environment )
12:05:18 SQL > explain plan for 12:06:59 2 SELECT DISTINCT X.INVENTORY_ITEM_ID FROM EINV_ONHAND_TXN_HISTORY_DAILY X,EINV_PRICE_LIST_GT Y WHERE X.INVENTORY_ITEM_ID = Y.INVENTORY_ITEM_ID AND X.CREATION_DAY =NVL(:B1,TRUNC(SYSDATE)-1)12:07:00 3 ; Explained. 12:07:01 SQL > @?/rdbms/admin/utlxpls PLAN_TABLE_OUTPUT --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost | ----------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 27 | 19 | | 1 | SORT UNIQUE | | 1 | 27 | 19 | |* 2 | TABLE ACCESS BY INDEX ROWID| EINV_ONHAND_TXN_HISTORY_DAILY | 1 | 14 | 2 | | 3 | NESTED LOOPS | | 1 | 27 | 3 | | 4 | INDEX FULL SCAN | EINV_PRICE_LIST_GT_N1 | 1 | 13 | | |* 5 | INDEX RANGE SCAN | EINV_ONHAND_ITEM_ID_IDX | 1153 | | 2 | -----------------------------------------------------------------------------------------------

Page 5

Analysis
After analyzing the test and production environment. We noticed that we have gathered the stats for the respective tables. Since only one program was showing the unwanted behavior. Thus it’s not possible for us to revert the stats of the respective tables as it might cause problem for other programs also. As CBO calculation are based on the tables stats. To know more about CBO and its limitation refer to the metalink doc: 212809.1 . So we decided to create the outline for the same. A) Create outline for the query on Test environment . Create outline <Ouitline Name > for the category <Categorn Name> on <Query >; i.e., create outline EOTHDAILY for category e_dly on SELECT DISTINCT X.INVENTORY_ITEM_ID FROM EINV_ONHAND_TXN_HISTORY_DAILY X,EINV_PRICE_LIST_GT Y WHERE X.INVENTORY_ITEM_ID = Y.INVENTORY_ITEM_ID AND X.CREATION_DAY =NVL(:B1,TRUNC(SYSDATE)-1);

B) Transfer Stored Outlines from Test to Production Now take the export of the outline. Outline regarding information’s are contained in the tables ol$ , ol$hints . To know more about how to transfer the outline from one database to another refer to doc 728647.1

exp parfile=outln_eoth.par [ILTEST:oratest] /home/users/oratest> more outln_eoth.par userid=outln/outln file=outl_eoth.dmp log=outl_eoth.log tables=ol$,ol$hints query="WHERE ol_name='EOTHDAILY'"

Now , transfer the dump outl_eoth.dmp from Test to Production and import into the production environment . imp outln/outln file=outl_eoth.dmp log=imp_outl_eoth.log tables='ol$','ol$hints' ignore=y

Page 6

C ) Force the session to pick the outline Put the line mentioned below in the starting of the program code to force the session to pick the outline. Remember here we’re not making any changes in the code .We’re just executing a session level command at the beginning of program . execute immediate 'ALTER SESSION SET use_stored_outlines=E_DLY';

After that we ask the user to run the concurrent and it got completed within 2 hours. Thus it made him and me very happy. As a DBA I believe that we should be thankful to the developers or users. Because it’s because of them we tend to learn new new things.

Results, Conclusion and Learnings
I wrote this because I’ve seen that outline is a great tool. But generally we don’t use it, instead we use hints etc. According to me it’s a great tool in case you know. What’s your best plan is and would like to make that mandatory for your query.

References
209197.1
212809.1 728647.1

DISCLAIMER: The information in this document is the opinion of the author, not of Oracle Corporation. Any content, materials, information or software downloaded or otherwise obtained through the use of the site is done at your own discretion and risk. Oracle shall have no responsibility for any damage to your computer system or loss of data that results form the download of any content, materials, information or software.

Page 7

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.