Issue..........................................................................................................................................................2 SQL to check database before Shutdown database ................................................

..................................2 Oracle.Metalink.com document ...............................................................................................................2 1076161.6:-Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX Recovery ......................................................................................................................................................3 375935.1:- What To Do and Not To Do When 'shutdown immediate' Hangs.............................5 Applies to: ...............................................................................................................................6 Goal..........................................................................................................................................6 Solution....................................................................................................................................6 References................................................................................................................................8 Errors........................................................................................................................................8 Bug:5057695: Shutdown Immediate Very Slow To Close Database...........................................8 Applies to: ...............................................................................................................................8 Symptoms................................................................................................................................8 Changes....................................................................................................................................8 Cause........................................................................................................................................9 Solution....................................................................................................................................9 61997.1: SMON - Temporary Segment Cleanup and Free Space Coalescing...........................10 100054.1: Transaction Rollback after a failed operation or during Database Shutdown..........15 Conversation between me and Oracle.Metalink.com document ...........................................................16 More information....................................................................................................................................45

1

Issue
TO SHUTDOWN DATABASE IT IS TAKING LONG TIME, SOMETIME MORE THAN 2 HOURS. Frequently we are facing this problem. Some time to take shutdown it takes 10 min .One days we have got one error that is " ORA-01013: user requested cancel of current operation" time of shutdown database

SQL to check database before Shutdown database
To make database shutdown faster please run SQL before shutdown database.and see the status of database 1. All active session ====================== If Active session is exist then, try to find out what is doing in the database by this session. Active session make shutdown slower select sid, serial#, username, status, schemaname, logon_time from v$session where status='ACTIVE' and username is not null 2.Temporary Tablespace Recover ================================ Sea any session used Temporary tablespace for sorting

. For this also may shutdown slower

Select f.R "Recovered", u.nr "Need Recovered" from (select count(block#) R , 1 ch from sys.fet$ ) f,(select count(block#) NR, 1 ch from sys.uet$) u where f.ch=u.ch 3. Long Running Query ======================= select * from v$session_longops where time_remaining>0 order by username 4. Large Transactions ======================== select sum(used_ublk) from v$transaction 5. Progress of the transactions that Oracle is recovering. ======================================================== select * from v$fast_start_transactions 6. Parallel transaction recovery ================================== select * from v$fast_start_servers

Oracle.Metalink.com document
1 2 3
1076161.6:-Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX Recovery Note 375935.1 - What To Do and Not To Do When 'shutdown immediate' Hangs Note 428688.1 ­ Bug 5057695: Shutdown Immediate Very Slow  To Close Database.

2

4 5

Note 61997.1 SMON ­ Temporary Segment Cleanup and Free  Space Coalescing  100054.1 Subject: Transaction Rollback after a failed operation or during  Database Shutdown

1076161.6:-Shutdown

Normal or Shutdown Immediate Hangs. SMON disabling TX Recovery
Subject: Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX Recovery Doc ID: Note:1076161.6 Type: BULLETIN Last Revision Date: Status: 21-DEC-2007 PUBLISHED
Description =========== SHUTDOWN NORMAL or SHUTDOWN IMMEDIATE hangs. In the alert.log, you see only  the following: Shutting down instance (immediate) License high water mark = 12 Thu Dec  8 18:43:16 1994 alter database  close normal Thu Dec  8 18:43:17 1994 SMON: disabling tx recovery SMON: disabling cache recovery or         waiting for smon to disable tx recovery There are no ORA errors or trace files.

Scope & Application =================== Informational During a SHUTDOWN IMMEDIATE and SHUTDOWN NORMAL, SMON is cleaning up extents  which are no longer needed and marking them as freed. Either wait for SMON to clean up the free extents in the database as it shuts  down or perform a SHUTDOWN ABORT to shutdown the instance. A SHUTDOWN ABORT  will not perform a clean shutdown. Verify that temporary segments are decreasing ­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ To verify that the temporary segments are decreasing have an active session  available in Server Manager or SQLPLUS during the SHUTDOWN IMMEDIATE. Issue 

3

the following query to ensure the database is not hanging, but is actually  perform extent  cleanup:     SVRMGR/SQL> select count(block#) from fet$;     COUNT(BLOC     ­­­­­­­­­­              7     SVRMGR/SQL> select count(block#) from uet$;     COUNT(BLOC     ­­­­­­­­­­            402   After some time has elapsed, reissue the query and see that the values for  fet$ have increased while the values or uet$ have decreased:     SVRMGR/SQL> select count(block#) from fet$;     COUNT(BLOC     ­­­­­­­­­­             10     SVRMGR/SQL> select count(block#) from uet$;     COUNT(BLOC     ­­­­­­­­­­            399 select f.R "Recovered", u.nr "Need Recovered" from (select count(block#) R  ,  1 ch  from fet$ ) f, (select  count(block#) NR, 1 ch  from uet$) u where f.ch=u.ch During shutdown the SMON process is cleaning up extents and updating the data  dictionary tables with the marked free extents. As the extents are marked as  freed, they are removed from the table for used extents, UET$ and placed on  the table for free extents, FET$. How to Avoid creating many Temporary Extents ­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ Once the database has shutdown cleanly, to avoid creating many temporary  extents change the initial and next extent sizes on temporary tablespaces to a  more appropriate size:     ALTER TABLESPACE <temp> DEFAULT STORAGE (INITIAL <size>M/K NEXT  <size>M/K);   Note: If the temporary tablespace is of type TEMPORARY, then this change will  only affect temporary segments created after issuing the above command. Any  existing temporary segments already in the TEMPORARY tablespace will not be  affected till the instance is restarted. On shutdown, existing temporary  segments are dropped. If the TEMPORARY TABLESPACE is of type PERMANENT, then  cleanup is performed by SMON after completion of the process using it.

4

Increasing the initial and next extent size will decrease the number of  extents that are allocated to temporary segments. Since there are fewer  extents to deallocate, the database should shutdown more speedily. Take the following scenario: A database was subject to large sorts with the following sort parameter in  the "init.ora" file:             ­ sort_area_size=1000000    The temporary tablespaces for this database were all created with initial and  next extents sized at 50k and the total database size was about 300mb.     Database sorts will utilize memory as much as possible based on the "init.ora"  parameter "sort_area_size".  Once this memory­based sort area is filled, the  database will utilize the temporary table space associated with the database  user to complete the sort operation.  During a shutdown normal, the database  will attempt to clean up the temporary tablespaces.      If a small extent size is used, then a large number of extents will be created  for a large sort.  The cleanup of the temporary tablespace takes much longer  with a large number of extents. Note: ===== You have to do a shutdown abort and then bring the database back up to run the  suggested queries.   For other reasons for slow/hung shutdown see also these notes: Note 375935.1 ­ What To Do and Not To Do When 'shutdown immediate' Hangs Note 428688.1 ­ Bug 5057695: Shutdown Immediate Very Slow To Close Database.     References: =========== Note 61997.1 SMON ­ Temporary Segment Cleanup and Free Space Coalescing                         Search Words: ============= hanging shutdown

375935.1:- What To Do and Not To Do When 'shutdown immediate' Hangs
Subject: What To Do and Not To Do When 'shutdown immediate' Hangs Doc ID: Note:375935.1 Type: HOWTO 5

Last Revision 02-JUN-2007 Date:

Status:

MODERATED

This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) Rapid Visibility (RaV) process, and therefore has not been subject to an independent technical review.

Applies to:
Oracle Server - Enterprise Edition - Version: 8.1.7 to 10.2 Information in this document applies to any platform.

Goal
What to do when shutdown immediate appears to hang. Sometimes, the message 'Waiting for smon to disable tx recovery' is posted in the alert log. This note only addresses situations when the apparent hang occurs when the database is going from OPEN to MOUNT, which is actually the most common situation. If the apparent hang occurs at a different step, then this note does not apply.

Solution
The big problem in these situations is that it is noticed only after the shutdown immediate has been issued. This kind of situation is mostly caused by 2 things: 1. a large query running at the shutdown moment. 2. a large transaction running at the shutdown moment. Both have to complete in order for the database to be brought down when shutdown immediate is issued. Actually, the files cannot be closed consistently because of one of the 2 possibilities above and, as such, the transition from OPEN to MOUNT is postponed until the files are closed, which means that either the large query completes or the large transaction is rolled back. This is not a hang, it is the expected behavior. So, before issuing the shutdown immediate, it would be recommended to check the following views, especially when the database needs to be brought down for a very short period of time: 1. for large queries:
select count(*) from v$session_longops where time_remaining>0;

2. for large transactions:

select sum(used_ublk) from v$transaction;

A result greater than 0 for the first query and a large value returned for the second one would mean a relatively long time to wait until the shutdown immediate completes. For the second situation, please also check step 9 in Note 117316.1 to "guestimate" the time to rollback the transactions.

----------------------------------------------------------Step 9 from Note 117316.1 Check for rollback:

6

        select used_ublk from v$transaction where ADDR=<value from         TADDR in v$session>;         If there is a value there, this is the number of undo blocks used by  the transaction.  Wait one minute and again select "used_ublk" from  "v$transaction" where ADDR=<value from TADDR in v$session>; .        Note the  value.  If it is decreasing, a rollback is occuring and based on the  difference between these values, you can "guesstimate" the time required to  complete the rollback.  For example, if the first query returns a value of  80000 and the second one returns 70000, it took 1 minute to rollback 10000  blocks.  Based on this number,you can guestimate the time to complete the  rollback.  In this case, it would be 7 minutes.

--------------------------------------------------------------------------

1. For the large queries situation, when the shutdown immediate is hanging, you can just bring down the database using: shutdown abort, as the database could be easily brought to a consistent state by: startup restrict followed by shutdown immediate. One should take the backup and/or do whatever else need to be done after the shutdown immediate. 2. For the second situation, the workaround cannot be applied, especially when it's needed to take a cold backup. The database must be closed in a consistent state in order to do this and the consistent state cannot be achieved until all the transactions have completed one way or another (commit/rollback). As such, it's up to the local personnel to decide what to do, depending on the local needs. It is very important to realize that: BY SHUTTING DOWN A DATABASE YOU DO NOT SOLVE A PERFORMANCE PROBLEM CAUSED BY A LARGE TRANSACTION. You are only making things worse. There are situations when the database is brought down even when a large transaction/large recovery is taking place. Then it's brought up again and a new shutodwn is tried. Again, the shutdown immediate is hanging, for a very simple reason - the large recovery is still going on. At this moment, the v$transaction view is not displaying anything. Hoever, it is still possible to check the recovery operation by checking the:
select * from v$fast_start_transactions; and/or select * from v$fast_start_servers;

views. They are the ones that display the recovery status. As such, when a large transaction is taking place, do not try successive shutdown aborts, startups and shutdown immediate. The hang will reoccur. The database must be consistent when the database is dismounted - performing successive shutdowns/startups is not helping at all, it's only making the recovery even more lengthy. You should prevent these situations by notifying the users a shutdown will be done and no large operations should be started. If a large operation has already started at the moment when you want to shutdown immedate,

7

assess what would be faster - rollback the current situation or allow it to complete.

References
Note 117316.1 - ORA-0054: When Dropping or Truncating Table, When Creating or Rebuilding Index

Errors
ORA-54 "resource busy and acquire with NOWAIT specified"

Bug:5057695: Shutdown Immediate Very Slow To Close Database.
Subject: Bug:5057695: Shutdown Immediate Very Slow To Close Database. Doc ID: Note:428688.1 Type: PROBLEM Last Revision Date:24-DEC-2007 Status: PUBLISHED

Applies to:
Oracle Server - Enterprise Edition - Version: 9.2.0.8 to 10.2.0.3 This problem can occur on any platform. Oracle Database Server

Symptoms
Recently upgraded the database to versions between 9.2.0.7 to 10.2.0.3 Database with 150 or so connections take a very long time to shutdown, compared to the same system when using Oracle 9i (9.2.0.6 and prior) where shutdown was very fast. Given the fact there is no activity on the connections in question, so rollbacks aren't an issue. There are no trace files and no indication of a problem in the alert log.

Changes
Recently upgraded the database to versions between 9.2.0.7 to 10.2.0.3

8

Cause
SHUTDOWN IMMEDIATE can be slower in 9.2.0.8 and 10.2.0.1/10.2.0.2/10.2.0.3 than in earlier releases due to a change in how processes are terminated introduced in these releases. Previously the shutdown immediate, the ksukia function would kill each individual user session and check in intervals of 5 seconds that it had died before progressing to the next user. With fix for bug 5080775 (Unpublished) this time was reduced so that instead of waiting for 5 seconds it now waited 0.05 seconds and this would be repeated 40 times at intervals of 0.05 seconds then it would increase to checking every 5 seconds. With large numbers of users this could involve a lengthy wait for shutdown that needed to be reduced. Note this code is relatively new. It was introduced in 9.2.0.7 and 10.2. It was a design change where previously it issues a kill (skgpkill) and then proceeds onto the next process.

Solution
Workaround : Wait for the shutdown to finish. Solution : Apply Patch 5057695 available on Metalink on top of 9.2.0.8, 10.2.0.2 or 10.2.0.3 Fix included with 10.2.0.4 Patchset One-off patches are available for most platforms, If a patch is not available for your platform, please contact Oracle Support.

9

61997.1: SMON - Temporary Segment Cleanup and Free Space Coalescing
Subject: SMON - Temporary Segment Cleanup and Free Space Coalescing Doc ID: Note:61997.1 Type: FAQ Last Revision Date:22-APR-2004 Status: PUBLISHED
SMON ­ Temporary Segment Cleanup and Free Space Coalescing in Oracle 7.3 and  Higher PURPOSE ~~~~~~~   Since the introduction of the unlimited extents feature in Oracle 7.3, it is  possible for SMON to have to either clean up a large number of temporary  extents, or to coalesce a large number of free extents. This can manifest  itself by SMON appearing to spin, consuming a high percentage of CPU for long  periods. This article explains what is happening, and what (if anything) can  be done.   The discussion concentrates mainly on non­TEMPORARY type tablespaces. There   is however a section at the end of the article which discusses possible   issues with tablespaces of type TEMPORARY. SCOPE & APPLICATION ~~~~~~~~~~~~~~~~~~~   This article is intended to assist DBAs encountering SMON appearing to spin    and consume high percentages of CPU by providing an understanding of the   issues. RELATED DOCUMENTS ~~~~~~~~~~~~~~~~~   Note 35513.1 Removing `stray` TEMPORARY Segments   Note 50592.1 Extent Sizes for Sort, Direct Load and Parallel Operations   Note 65973.1 Temporary Tablespaces and the Sort Extent Pool   Note 68836.1 How to efficiently drop a table with many extents   Note 47400.1 EVENT: DROP_SEGMENTS ­ Forcing cleanup of TEMPORARY segments What to look for ~~~~~~~~~~~~~~~~   The most common indicator is the SMON process consuming large amounts   of CPU for a long period. UNIX O/S utilities sar or vmstat will show how  busy   CPU(s) are; ps will show which process is using the CPU. What is SMON doing ~~~~~~~~~~~~~~~~~~   Once you have identified that SMON is using lots of CPU, you need to   identify whether it is performing temporary segment (extent) cleanup, or   free space coalescing.

10

Free space coalescing ~~~~~~~~~~~~~~~~~~~~~   When does SMON coalesce?     o. SMON wakes itself every 5 minutes and checks for tablespaces with        default pctincrease != 0.   How to identify whether SMON is coalescing     o. Check whether there are a large number of free extents that might        be being coalesced by running the following query a few times:          SELECT COUNT(*) FROM DBA_FREE_SPACE;        If the count returned is dropping while SMON is working, it is        likely that SMON is coalescing free space.   What are the effects on the database?     o. Because SMON acquires the Space Transaction (ST) enqueue in         exclusive mode, other processes requiring the enqueue will be        blocked. This is typically manifested by multiple <oerr:ORA­1575>         errors.     o. SMON sits in a very tight loop while coalescing, and consumes close        to 100% CPU. If the system is CPU­bound, the run queue will increase        as other processes try to get onto CPU.   Can anything be done to stop SMON grabbing CPU?     o. If there is no CPU contention, and no processes being blocked because        of failure to acquire the ST enqueue, DO NOT DO ANYTHING. Leave SMON        to complete the coalescing.        THE DATABASE CAN BE SHUTDOWN CLEANLY WITH UN­COALESCED EXTENTS. If SMON        is performing the coalesce, a shutdown will NOT undo the work completed        so far.     o. Use the 'alter tablespace <tbs name> coalesce' command. This is quicker        than SMON, and the work is performed in in fewer space transactions,  and        therefore makes fewer enqeueue acquisitions. HOWEVER, IF THE COMMAND IS        INTERRUPTED, ALL ITS COALESCING WORK WILL BE LOST.     o. It is possible to force a user session to coalesce free extents. See        Note 35513.1 "Removing `stray` TEMPORARY Segments" for details. Again,         quicker than SMON. HOWEVER, IF THIS OPERATION IS INTERRUPTED, ALL IT'S         COALESCING WORK WILL BE LOST.     o. Offlining the tablespace/datafiles containing the extents to be        coalesced has NO effect. Temporary segment cleanup

11

~~~~~~~~~~~~~~~~~~~~~~~~~   When does SMON cleanup temporary segments?     o. Typically a user process allocates a temporary segment (multiple        extents) and then dies before cleaning them up, or the user process        receives an error causing the statement to fail. SMON is posted to do        the cleanup. SMON also might get tied up cleaning uncommitted         transactions though, and be too busy to process requests to grow an           existing sort segment. Starting with Oracle 8i, playing around with         fast_start_parallel_rollback might workaround that.        In addition, actions like CREATE INDEX create a temporary segment for         the index, and only convert it to permanent once the index has been         created. Also, DROP <object> converts the segment to temporary and then        cleans up the temporary segment.     o. During normal operations, user processes that create temporary segments        are responsible for cleanup.   How to identify whether SMON is cleaning up temporary extents     o. Check whether there are a large number of temporary extents that might        be being cleaned up by running the following query a few times:          SELECT COUNT(*) FROM DBA_EXTENTS WHERE SEGMENT_TYPE='TEMPORARY';        If the count returned by the above query is dropping while SMON is        working, it is likely that SMON is performing temp segment cleanup.        See section 'Tablespaces of type TEMPORARY' for more details on        this.   What are the effects on the database?     o. Again, SMON will continually acquire and then release the ST enqueue        in exclusive mode. This can cause contention with other processes and         lead to <oerr:ORA­1575> errors.      o. CPU utilization is not exceptionally high. During tests, SMON        consumed between 10% and 20% CPU during cleanup, and so this operation        has less impact than coalescing, as far as SMON is concerned.        Furthermore, SMON performed the cleanup in 'chunks', cleaning up a        subset of the extents at a time.          Can anything be done to stop SMON grabbing CPU?     o. Not a great deal. As with coalescing, if there is no CPU contention,        and no processes being blocked because of failure to acquire the ST        enqueue, DO NOT DO ANYTHING. However because SMON does not work as hard        cleaning up temporary extents, it should not be a big issue.        Note: If you are using TEMPORARY type temporary tablespaces then              SMONs cleanup of the segment can be a problem as it will not

12

             service sort segment requests while performing cleanup.              See below (TEMPORARY tablespaces) for more information.        It should be noted that a normal/immediate shutdown will not complete         until all temporary segments have been cleaned up. Shutdown will        'kick' SMON to complete cleanup.     o. Offlining the tablespace/datafiles in which the extents reside has NO        effect.    o. DROP_SEGMENTS event could be set set to force the cleanup of        temporary segments, see Note 47400.1 "EVENT: DROP_SEGMENTS ­ Forcing        cleanup of TEMPORARY segments". Avoidance ~~~~~~~~~    With a little forethought and care, the above situations can be avoided:    o. Do not create temporary tablespaces with small initial and next default       storage parameters. Also beware of unlimited maxextents on temporary       tablespaces.       Note, TEMPORARY type tablespaces set maxextents unlimited automatically.       Furthermore, the NEXT AND INITIAL extent sizes are determined from       the default NEXT size (default INITIAL is ignored). For more details on       temporary extent sizes, see Note 50592.1 "Extent Sizes for Sort, Direct        Load and Parallel Operations (PCTAS & PDML)".    o. Use tablespaces of type TEMPORARY. Sort segments in these tablespaces       are not cleaned up. This reduces contention on the ST enqueue and also       reduces CPU usage by SMON **UNLESS** the database is shutdown and       restarted. If TEMPORARY type tablespaces are in use then SMON will       clean up its segments after startup following a shutdown. In this case        large numbers of extents can be a severe problem as SMON will not        service user "sort segment requests" until the cleanup is complete.        If the cleanup is to take a long time users will not be able to perform        sort operations.        In this scenario you can point users at a PERMANENT temporary tablespace        while SMON cleans up the TEMPORARY temporary tablespace. This is likely  to         cause ST enqueue contention but will allow users sessions to sort on  disk        when necessary rather then them just blocking.       Eg:    If SMON is busy cleaning up a TEMP segment containing a lot    of extents it cannot service 'sort segment requests' from other    sessions. Pointing the users at a PERMANENT tablespace as    their temporary tablespace can help keep the system running             until SMON is free again:    CREATE TABLESPACE NEWTEMP .... (your own specification here)

13

   (DO NOT CREATE IT AS TYPE TEMPORARY)            Move the users over to this:    select username from dba_users   where temporary_tablespace='TEMP';    For each user in this list: alter user XXXXX temporary tablespace NEWTEMP;    Once SMON has cleaned up the extents reset the storage clause    on each tablespace to sensible values and you can then point    users back at a TEMPORARY temp tablespace.       Starting with Oracle8i, rather than reverting back to a PERMANENT        tablespace if SMON is cleaning up an old sort segment at startup,        you can potentially drop and recreate the tempfiles of the existing        TEMPORARY tablespace. The cleanup should be faster anyway since by rule       a TEMPORARY tablespace made of tempfiles need to be LOCALLY MANAGED.       You can remove tempfiles from TEMPORARY tablespaces and keep the logical        structure empty.    o. Beware of creating large objects with inappropriate (small) extents. If       the creation of the object fails, SMON cleans up. Also, dropping such an       object will create a lot of cleanup work for the user process.       Oracle8 ONLY. Make use of the tablespace MINIMUM EXTENT size to help       minimise the risk of mistakes in scripts causing small extent sizes.       This parameter ensures that every used and/or free extent size in a       tablespace is at least as large as, and is a multiple of, this value.       Oracle8i ONLY: It is worth considering the use of a locally managed        temporary tablespace. This has the benefit of faster temporary segment        cleanup after the instance has been aborted.        Note, locally managed temporary tablespaces must be created using        tempfile(s). Any attempt to create a locally managed temporary        tablespace using a datafile will result in the error:          ORA­25144: invalid option for CREATE TABLESPACE with TEMPORARY contents 

Tablespaces of type TEMPORARY ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~    TEMPORARY­type tablespaces were introduced in Oracle 7.3 (see Note 65973.1    "Temporary Tablespace, the Sort Extent Pool, and OPS"). In summary:    o. The first disk sort (after instance startup) creates a sort segment in        the TEMPORARY tablespace.    o. Free extents in the sort segment are re­used as required by sessions.

14

          o. The sort segment grows to a steady­state.    o. Sort extents are not de­allocated whilst the instance is running.    o. Permanent objects cannot be created in TEMPORARY tablespaces.    o. There is a maximum of one sort segment per TEMPORARY tablespace.    Thus, contention on the ST enqueue is reduced as user sessions are  allocating    and de­allocating fewer extents. Even if a user session dies, SMON will not    de­allocate extents that the session was using for sorting.    SMON actually de­allocates the sort segment after the instance has been     started and the database has been opened. Thus, after the database has been     opened, SMON may be seen to consume large amounts of CPU as it first     de­allocates the (extents from the) temporary segment, and then when it is     requested to perform free space coalescing of the free extents created by     the temporary segment cleanup. Again, this behaviour will be exaggerated if     the TEMPORARY tablespace in which the sort segment resides has  inappropriate     (small) default NEXT storage parameters (see 'Avoidance' above). Permanent object cleanup ~~~~~~~~~~~~~~~~~~~~~~~~    If a permanent object is made up of many extents, and the object is to be    dropped, the user process dropping the object will consume large amounts    of CPU in the same way as SMON does when cleaning up a temporary segment.    Please see Note 68836.1 "How To Efficiently Drop A Table With Many Extents"    for a discussion of this. References ~~~~~~~~~~

100054.1: Transaction Rollback after a failed operation or during Database Shutdown
Subject: Transaction Rollback after a failed operation or during Database Shutdown Doc ID: Note:100054.1 Type: BULLETIN Last Revision Status: 17-JAN-2006 PUBLISHED Date:
Problem Description ­­­­­­­­­­­­­­­­­­­ To explain what happens to uncommitted transactions after a failed or aborted operation (i.e. long running transaction fails) or caused by a Database  Shutdown command being issued.

15

  Problem Explanation ­­­­­­­­­­­­­­­­­­­   If a long running transaction (indeed any transaction) is cancelled or fails,  then Oracle needs to cleanup the uncommitted work that was done by this  transaction so that other transactions can progress.  This cleanup involves rolling back the uncommitted work. If a database shutdown is issued with outstanding transactions to be rolled back, then the database will not be shutdown until these transactions have  completed or have been rolled back.  For example on a shutdown immediate we send a message to all the shadow  processes asking them to complete this rollback and cleanup.  We will not shutdown until  all of them have completed this cleanup.  This means that because we have  shutdown cleanly no redo needs to be applied on startup. If the database is aborted with outstanding transactions then these  transactions must be rolled back after the database has been restarted.  Post 7.3 this  transaction recovery is applied by SMON after the database has been opened.  If another transaction needs to access blocks that were modified by the  transaction that is rolling back then the new session will rollback the blocks that it  requires itself.  This can cause these transactions to perform slower than  normal. The activity of SMON has been designed so as to minimize the impact on other processes by only rolling back a small proportion of rollback entries at a  time. References ­­­­­­­­­­ Bug 517917 SHUTDOWN IMMEDIATE HANGS ­ SMON DOING TX RECOVERY Note 61997.1 SMON ­ Temporary Segment Cleanup and Free Space Coalescing

Conversation between me and Oracle.Metalink.com document

16

SR Number 6779932.992 Support Identifier 15272898 Severity 3 Product Oracle Server - Enterprise Edition Platform AIX5L Based Systems (64-bit) SR Reference n/a

Open Date Name Last Update Product Version

19-FEB-08 04:30:01 mamun seraji 17-APR-08 07:02:57 9.2.0.8.0

Detailed Status Soft Close BUG Reference n/a

Abstract
TO SHUTDOWN DATABASE IT IS TAKING LONG TIME, SOMETIME MORE THAN 2 HOURS.

Resolution History
19-FEB-08 04:30:01 GMT This SR was saved as a Draft SR on: 18-FEB-2008 20:19:00 ### Introduction to the Database Performance Issues iTAR Questions ### N/A ### Is your system completely hung or is just performing slow? ### Slow ### Please describe any recent change in your environment? ### During night so many batch job running in our database. That time we have checked alert log file and found "ORA-01555: caused by SQL statement below (Query Duration=8000)". Then we have changed undo_rete ntion parameter value. The old value was 6500 and the new value is 8000. ### When was the last time you performed a tuning process to your database? ### Frequently we are facing this problem. some time to take shutdown it take 10 min. ### When did you start to notice slow performance? ### From beginning we are facing this problem. ### Can you consistently reproduce the problem? ### No ### Is the problem related to heavy load hours only? ### No ### Are you getting any kind of error messages? ### One day we have got one error that is " ORA-01013: user requested cancel of current operation"

17

### Please list all files that you plan to upload: ### if need we will send Can you easily recover from, bypass or work around the problem? Yes Does your system or application continue normally after the problem occurs? Yes Are the standard features of the system or application still available; is the loss of service minor? No Error : ORA-01013 Contact me via : MetaLink

19-FEB-08 09:47:01 GMT UPDATE ====== Hi Mamun, Thank you for using MetaLink. We are currently reviewing/researching the situation and will update the Service Request (SR) or call you as soon as we have relevant information. Thank you for your patience. Best Regards, Bharath Global Customer Services STATUS ======= @WIP -- Work In Progress

19-FEB-08 09:51:14 GMT . UPDATE ======= Hi Mamun,

18

Can you please upload the Alert_log file which contains the last 5-6 shutdown information logged 1) Were there any transactions that were cancelled by doing shutdown immediate. Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

19-FEB-08 09:51:26 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 25-FEB-08 05:29:02 GMT The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink: Alert_log.txt

25-FEB-08 05:34:01 GMT New info : ruhul.amin@bracbank.com Dear Mr Bharath Sunku Sorry for delay. i was out of office. There was no transaction canceled from back-end at the time of sh utdown. Thanks and regards Md Ruhul Amin

25-FEB-08 12:18:38 GMT DATA COLLECTED

19

============ Fri Feb 15 01:50:39 2008 Shutting down instance (immediate) License high water mark = 1327 Fri Feb 15 01:55:04 2008 ALTER DATABASE CLOSE NORMAL Fri Feb 15 01:55:04 2008 SMON: disabling tx recovery SMON: disabling cache recovery Sun Feb 17 02:06:04 2008 Shutting down instance (immediate) License high water mark = 858 Sun Feb 17 02:07:39 2008 ALTER DATABASE CLOSE NORMAL Sun Feb 17 02:07:39 2008 SMON: disabling tx recovery SMON: disabling cache recovery Sun Feb 17 02:07:39 2008 Shutting down archive processes Mon Feb 18 00:56:29 2008 Shutting down instance: further logons disabled Shutting down instance (immediate) License high water mark = 1762 Mon Feb 18 02:52:10 2008 ALTER DATABASE CLOSE NORMAL Mon Feb 18 02:52:10 2008 SMON: disabling tx recovery SMON: disabling cache recovery Mon Feb 18 02:52:13 2008 Shutting down archive processes Shutting down instance (immediate) License high water mark = 2430 Tue Feb 19 01:53:47 2008 ALTER DATABASE CLOSE NORMAL Tue Feb 19 01:53:48 2008 SMON: disabling tx recovery SMON: disabling cache recovery Tue Feb 19 01:53:52 2008 Shutting down archive processes Archiving is disabled Tue Feb 19 01:53:52 2008

20

Wed Feb 20 00:33:51 2008 Shutting down instance (immediate) License high water mark = 1905 Wed Feb 20 00:59:31 2008 ALTER DATABASE CLOSE NORMAL Wed Feb 20 00:59:31 2008 SMON: disabling tx recovery SMON: disabling cache recovery Wed Feb 20 00:59:33 2008 Shutting down archive processes Archiving is disabled Thu Feb 21 01:04:52 2008 Shutting down instance (immediate) License high water mark = 2062 Thu Feb 21 01:07:22 2008 ALTER DATABASE CLOSE NORMAL Thu Feb 21 01:07:22 2008 SMON: disabling tx recovery SMON: disabling cache recovery Thu Feb 21 01:07:24 2008 Shutting down archive processes Archiving is disabled Thu Feb 21 17:45:07 2008 Shutting down instance: further logons disabled Shutting down instance (immediate) License high water mark = 376 Thu Feb 21 17:46:18 2008 ALTER DATABASE CLOSE NORMAL Thu Feb 21 17:46:18 2008 SMON: disabling tx recovery SMON: disabling cache recovery Thu Feb 21 17:46:18 2008 Shutting down archive processes Completed checkpoint up to RBA [0x6aea.2.10], SCN: 0x0002.831b5524 Sun Feb 24 00:44:21 2008 Shutting down instance (immediate) License high water mark = 618 Sun Feb 24 00:46:37 2008 ALTER DATABASE CLOSE NORMAL Sun Feb 24 00:46:37 2008 SMON: disabling tx recovery SMON: disabling cache recovery

21

Sun Feb 24 00:46:37 2008 Shutting down archive processes Archiving is disabled . UPDATE ======== Hi Mamun, Thanks for the Update We have checked the Alert_log file and found the below 1) Shutdown immediate is not taking 2 hours everytime its executed 2) In all the instances of shutdown slow, only Transaction recovery is counsuming 99% of the time. As you have also specified that there are some Batch Jobs which are running in the Night and we suspect they might have got cancelled due to "Shut immediate" which is consuming long time for transaction recovery Please let me know if you need any other clarifications on this issue Regards Bharath Sunku

25-FEB-08 12:18:49 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 25-FEB-08 12:56:02 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku Thank you for fast reply I need two clarification 1) How you came to know that Transaction recovery is consuming 99% of the time ? 2) is there any way to shutdown database at short time ? Regards

22

Md Ruhul Amin

25-FEB-08 13:11:55 GMT . UPDATE ======= Hi Ruhul, Thanks for the Update 1) Please see the below ~~~~~~~~~~~~~~~~~~~~~~~ Mon Feb 18 00:56:29 2008 Shutting down instance: further logons disabled Shutting down instance (immediate) License high water mark = 1762 Mon Feb 18 02:52:10 2008 ALTER DATABASE CLOSE NORMAL Mon Feb 18 02:52:10 2008 SMON: disabling tx recovery ~~~~~~~~~~~~~~~~~~~~~~~~ From the Above we see that the Shutdown was performed at 00:56 and 02:52 tx recovery was completed.. Please check my Data Collected Tag.. you will find that the 99% of the time is consumed at this step and after that its hardly taking 3-5 minutes to shutdown the databa se 2) Is the Shutdown completed in short time? ~~~~~~~ Thu Feb 21 17:45:07 2008 Shutting down instance: further logons disabled Shutting down instance (immediate) License high water mark = 376 Thu Feb 21 17:46:18 2008 ALTER DATABASE CLOSE NORMAL Thu Feb 21 17:46:18 2008 SMON: disabling tx recovery SMON: disabling cache recovery Thu Feb 21 17:46:18 2008

23

Shutting down archive processes ~~~~~~~~~~~~ Shutdown was executed at 17:45 and 17:46 is the tx recovery completed and immediatly database got shutdown Please let me know if you need any other clarifications on this issue Regards Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

25-FEB-08 13:12:06 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 25-FEB-08 13:33:06 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku Thank you for your nice reply. i am very much happy for this can u please tell me how can i came to know that today shutdown will take more time. Please arrange sometime to talk over ch at. regards Md Ruhul Amin

25-FEB-08 14:23:21 GMT .

24

UPDATE ======= Hi Ruhul, Time to shutdown depends on the operations being performed on the database before you do a Shutdown If there are any transaction happening on the database and if the SHUT IMMEDIATE is performed which is going to terminate the transactions and then do the Rollback, in this case depending on t he time of transaction recovery your shoudown will be depended Please let me know if you need any other clarifications on this issue My Shift time is from 1:30 AM GMT to 10:00 AM GMT Please let me at what time you want me to start an Web-Conference.. If you want this issue to be addressed after my Shift time.. Please call your Local HUB from the below Link so that This SR will be rassaigned to a n available engineer and he will help you with the issue http://www.oracle.com/corporate/contact/index.html Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

25-FEB-08 14:23:27 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 25-FEB-08 14:26:06 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 25-FEB-08 14:58:04 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku i will be available from 4.30 am GMT to 10.30 am GMT

25

Regards md Ruhul Amin

25-FEB-08 15:04:12 GMT . UPDATE ======= Hi Ruhul, Will update the SR / Call you so that we can further progress the issue Thanks, Bharath STATUS ======= @CUS -- Waiting for requested information

25-FEB-08 15:04:18 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 25-FEB-08 16:06:18 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku Any un-commited transaction will be rollback during shutdown database. if there is huge uncommited transa ction exists than shut down will take much time. so my concern is there any way to find how many transaction is un-commited now, which need to be roll back at the time of shutdown database. Thanks and regards md Ruhul Amin

26-FEB-08 05:19:40 GMT

26

. UPDATE ======= Hi Ruhul, Please read the below note to understand more on this issue Article-ID: Note 1076161.6 Title: Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX Recovery Thanks, Bharath STATUS ======= @CUS -- Waiting for requested information

26-FEB-08 05:19:51 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 26-FEB-08 07:41:05 GMT New info : ruhul.amin@bracbank.com Dear Mr Bharath Sunku Thank you. Please read this document and give me advise Subject: Bug:5057695: Shutdown Immediate Very Slow To Close Database. Doc ID: Note:428688.1 Thanks Md Ruhul Amin

26-FEB-08 10:06:45 GMT . ISSUE CLARIFICATION ==================== During night so many batch job running in our database. That time we have checked alert log

27

file and found "ORA-01555: caused by SQL statement below (Query Duration=8000)". Then we have ch anged undo_retention parameter value. The old value was 6500 and the new value is 8000. . ISSUE VERIFICATION =================== Verified by seeing the Alert_log file . CAUSE DETERMINATION ==================== Tx recovery is taking lot of time

CAUSE JUSTIFICATION ==================== Bug:5057695: Shutdown Immediate Very Slow To Close Database. . POTENTIAL SOLUTION(S) ====================== Apply the patch POTENTIAL SOLUTION JUSTIFICATION(S) ==================================== same as above . SOLUTION / ACTION PLAN ======================= Hi Mamun, Thanks for the Update Yes.. this also could be an issue in your case.. What the BUG says is that there is an change in the Functionality in which the shutdown

28

happened before 9.2.0.8 If there are any active sessions in your database which has to be killed while "shutdown immediate" of the database, th en oracle is waiting for 5 seconds on each session and this is consuming more ti me when there are many databases.. With the BUG FIX applied on the database.. Or acle is going to wait only for .05 seconds so that the Killing Active sessions w ill be completed farstly.. Since this is an Generic BUG and implies to all the 9.2.0.8 databases on all the platforms.. we recommand you to apply the patch and see whether you are still having the same issue Regards Bharath Sunku Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

26-FEB-08 10:06:51 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 26-FEB-08 10:25:03 GMT New info : ruhul.amin@bracbank.com Dear Mr Bharath Sunku Thank you for advise Patch 5057695 is no available for IBM AIX 5.3 and database is 9.2.0.8.0 one clarification is need how to find whether this patch is applied in our database or n ot please check Regards md Ruhul Amin

29

26-FEB-08 10:29:21 GMT . UPDATE ======= Hi Ruhul, TO find whether the patch is applied or not.. 1) Go to Oracle_Home opatch lsinventory -details ++ This contains the information about the patches applied on the database With regarding to the one-off patch, please answer the below question so that we can goahead and request for a one-off patch ~~~~~~~~~~ 1) Platform and the database version uname -a ++ Uplaod the below file Unix: $ORACLE_HOME/inventory/ContentsXML Windows: $ORACLE_HOME\inventory\ContentsXML SQL:> select * from V$version; 2) List of one-off patches applied since applying the current patch set. opatch lsinventory -detail 3) Release Status of Customer system affected 4) Does this affect a major Milestone? If yes, Milestone Date 5) Milestone Name 6) Date by which the fix must be installed on customers system 7) Technical Impact of this bug 8) Business Impact: Please let me know if you need any further clarifications ~~~~~~~~~~~~~~~

30

Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

26-FEB-08 10:29:27 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 26-FEB-08 13:41:02 GMT The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink: Config.zip

26-FEB-08 13:42:06 GMT New info : ruhul.amin@bracbank.com Dear Mr Bharath Sunku Here is the date [][DB][FINBRAC]/oracle/app/product/9.2.0/inventory/ContentsXML> uname -a AIX DBServer 3 5 00C6F01D4C00 SQL> select * from V$version; BANNER ---------------------------------------------------------------Oracle9i Enterprise Edition Release 9.2.0.8.0 - 64bit Production PL/SQL Release 9.2.0.8.0 - Production CORE 9.2.0.8.0 Production TNS for IBM/AIX RISC System/6000: Version 9.2.0.8.0 - Production NLSRTL Version 9.2.0.8.0 - Production Regards Md Ruhul Amin

31

27-FEB-08 04:33:03 GMT working on the issue 27-FEB-08 04:33:14 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 27-FEB-08 04:35:23 GMT . UPDATE ======= Hi Mamun, Can you please answer the rest of the questions so that we can goahead and raise the backport request Answers to all teh questions are mandatory since based on that the Backport request priority will be decided Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

27-FEB-08 04:35:29 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 27-FEB-08 07:25:02 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku Daily night we are taking cold backup. when shutdown taking more time, it is not possible take cold backup . if it is possible need to solve this issue by today

32

Regards md Ruhul Amin

27-FEB-08 09:17:22 GMT . UPDATE ======= Hi Mamun, Database should be shutdown fully to take cold backup Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

27-FEB-08 09:17:28 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 27-FEB-08 12:35:03 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku Thank you. Please guide me how to solve this issue. Regards md Ruhul Amin

27-FEB-08 12:41:02 GMT . UPDATE ======= Hi Mamun,

33

Since the problem is intermitant so you can goahead and shutdown the database to take the cold backups. Meanwhile we will raise an backport request for the BUG and will try to get the one-off patch as soon as pos sible so that the issue can be resolved.. For this please provide me the Answers for rest of the questions so that we can goahead with raising the backport req uest 3) Release Status of Customer system affected 4) Does this affect a major Milestone? If yes, Milestone Date 5) Milestone Name 6) Date by which the fix must be installed on customers system 7) Technical Impact of this bug 8) Business Impact: Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

27-FEB-08 12:41:11 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 02-MAR-08 05:45:02 GMT New info : ruhul.amin@bracbank.com Dear Bharath Sunku i need some additional information for answer your following questions 3 q. Release Status of Customer system affected ======================================== What are you trying to mean by the "Release Status of Customer system affected"? Is it the circular or support request from the user who are currently operating the core system? please describe in details

34

4 q. Does this affect a major Milestone? ======================================== What are you trying to mean by the "major Milestone"? please describe in details Regards Md Ruhul Amin

03-MAR-08 02:53:56 GMT . UPDATE ======= Hi Mamun, "Major Milestone" means is there any specific Activity on your database ( Ex migration , Upgradation.. etc ) which is getting blocke d because of this BUG.. "Release status" means is it an production environmet or DR ( Disaster Recovery Site ) Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

03-MAR-08 02:54:18 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 05-MAR-08 04:04:00 GMT . FOLLOW UP ========== Hi Mamun, Please update the SR as soon as possible with the current status and/or requested information so we can continue working the i

35

ssue. Thank You, Bharath STATUS ======= @CUS -- Waiting for Customer Update

05-MAR-08 04:04:05 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 05-MAR-08 09:04:04 GMT New info : ruhul.amin@bracbank.com Dear Mr Bharath 3) Release Status of Customer system affected ================================================== It is affecting our Production environment 4 and 5 Does this affect a major Milestone? ========================================== Yes. And it is effecting our database backup 6) Date by which the fix must be installed on customers system ============================================================== as soon as possible 7) Technical Impact of this bug ================================ no 8) Business Impact: ===================== It is a every day process and there is a time limit for it. we have to open our business activities before 8.00 am. Regards, Md Ruhul Amin

36

05-MAR-08 09:37:07 GMT . UPDATE ======= Hi Ruhul, One off patch ( Patch Patch 5057695 ) for your BUG is ready and can be downloaded from metalink.. Please download the patch and apply on the database abnd let us know if the issue is resolved or not Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

05-MAR-08 09:37:14 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 05-MAR-08 11:04:05 GMT New info : ruhul.amin@bracbank.com Dear Mr Bharath Thank you for your reply. one clarification i need about patch. from 2 weeks we are not facing problem about database shutdown issue, it is now taking 4-5 minutes. Before one hour of shutdo wn of database we are checking and killing sessions which sessions running repo rt related Long SQL. we do not want to kill session manually. Is your patch will make shutdown faster if we are not following our manual process( kill sessio n)?. Regards Md Ruhul Amin

37

06-MAR-08 02:26:14 GMT . UPDATE ======= Hi Ruhul, Firstly the Patch that we have requeste dyou to download is for an identified BUG in 9.2.0.8.0 which is causing problem while Shutdo wn Immediate of the database As specified in the BUG, it is also closing the Active sessions bit more faster than before patchset and version releases and so this patch should solve your problem.. We resommand you to apply the patch and monitor the database for a week so that we can progress the issue further Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

06-MAR-08 02:26:56 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 17-MAR-08 11:01:34 GMT TAR has been in CUS status for 11 days -- Sending First Close Alert email. 18-MAR-08 02:27:29 GMT . FOLLOW UP ========== Hi Mamun, Please update the SR as soon as possible with the current status and/or requested information so we can continue working the i

38

ssue. Thank You, Preetham STATUS ======= @CUS -- Waiting for Customer Update 18-MAR-08 02:27:35 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 19-MAR-08 04:32:00 GMT . FOLLOW UP ========== Hi Mamun, Please update the SR as soon as possible with the current status and/or requested information so we can continue working the i ssue. Thank You, Preetham STATUS ======= @CUS -- Waiting for Customer Update 19-MAR-08 04:32:05 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 30-MAR-08 11:01:46 GMT TAR has been in CUS status for 11 days -- Sending First Close Alert email. 02-APR-08 11:02:06 GMT TAR has been in CUS status for 14 days -- Setting status to ACL -- Sending Second Close Alert email.

39

09-APR-08 17:59:25 GMT . FOLLOW UP ========== Hi Mamun, Please update the SR as soon as possible with the current status and/or requested information so we can continue working the i ssue. Thank You, Preetham STATUS ======= @CUS -- Waiting for Customer Update 09-APR-08 17:59:35 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 15-APR-08 04:17:06 GMT CALL SUMMARY ============= Called Rahul and spoke about the issue Rahul told that he will start working on this issue from tommarow Rahul asked me steps for applying the patch.. Told that Readme file has all the instructions and asked him to follow and if problem asked him to update the SR 15-APR-08 04:17:14 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 16-APR-08 06:32:13 GMT Call Summary ==========

40

Called Rahul and spoke about the issue Explained Rahul on how to apply the patch Found that Opatch is not installed in the Ho,e.. instruncted on how to download and install the opatch and asked him to apply the pat ch Told Rahul to update the SR if he needs any clarifications on how to apply the patch

16-APR-08 08:28:04 GMT New info : ruhul.amin@bracbank.com Dear I have installed OPatch 10.2.0.1.6. Install the Opatch: --------------------------I have unzip file p4898608_10204_GENERIC.zip, in ORACLE home directory.

Again i heve tried to apply patch, but again error found. here is error information ================================================================== [FINBRAC]/home/oracle/5057695> ls -lart total 24 drwxr-xr-x 3 oracle orainv 256 Jun 16 2007 files drwxr-xr-x 4 oracle orainv 256 Jun 16 2007 etc -rw-rw-rw- 1 oracle orainv 4968 Jun 16 2007 README.txt drwxr-xr-x 4 oracle orainv 256 Jun 16 2007 . drwxr-xr-x 8 oracle orainv 4096 Apr 16 14:00 .. [FINBRAC]/home/oracle/5057695> /oracle/oraInventory/OPatch/opatch apply Invoking OPatch 10.2.0.1.6 Oracle Interim Patch Installer version 10.2.0.1.6 Copyright (c) 2007, Oracle Corporation. All rights reserved. Oracle Home : /oracle/oraInventory Central Inventory : /oracle from : /etc/oraInst.loc OPatch version : 10.2.0.1.6 OUI version : 10.1.0.5.0 OUI location : /oracle/oraInventory/oui Log file location : /oracle/oraInventory/cfgtoollogs/opatch/opatch2008-04-16_14-18-43PM.log

41

ApplySession failed: -------------------------------------------------------------------------------The Oracle Home does not meet OUI version requirement. This OPatch (version 10.2.0.1.6) detects OUI version 10.1.0.5.0 in the home. It requires OUI version 10.2 or above. System intact, OPatch will not attempt to restore the system OPatch failed with error code 73 [fin7013][FINBRAC]/home/oracle/5057695> =============================================== The log file with attachment. regards Md Ruhul Amin

16-APR-08 08:30:02 GMT The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink: C:\Documents_and_Settings\User\Desktop\opatch2008-04-16_14-16-17PM.log.txt

16-APR-08 09:18:39 GMT . UPDATE ======= Hi Rahul, Looks like there is a problem installing opatch Will create a spin off SR with INSTALL team on your behalf to progress the issue fur ther Regards, Bharath Sunku STATUS

42

======= @CUS -- Waiting for requested information

16-APR-08 12:36:05 GMT The customer : ruhul.amin@bracbank.com : has uploaded the following file via MetaLink: C:\Documents_and_Settings\User\Desktop\opatch2008-04-16_14-16-17PM.log.txt

17-APR-08 02:30:12 GMT . UPDATE ======= Hi Rahul, We have created a Spin-off ( SR:18780616.6 ) with t h einstall team to resolve opatch errors that you are facing. Please upload the latest log files that you have uploaded in this SR to the New SR so that the Ins tall Analyst can progress the issue further Regards Bharath Sunku

17-APR-08 02:30:18 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 17-APR-08 05:44:01 GMT New info : ruhul.amin@bracbank.com Dear Bharath i have apply patch through ($opatch apply) command. Thanks a lot to help me to apply patch. is there any way to see patch install is correct or not. regards md Ruhul Amin

43

17-APR-08 06:10:28 GMT . UPDATE ======= Hi Rahul, + Run the below command opatch lsinventory -details ++ The above command will provide the list of patches installed on your database after the patchset installed on your database Please let us know whether we can keep the SR in the Soft Closure ( Monitoring ) status for 2 weeks so that yo u can monitor the database for few days to check whether the same issue is reocc urring or not Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information

17-APR-08 06:10:41 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com. 17-APR-08 06:30:05 GMT New info : ruhul.amin@bracbank.com Dear Bharath Than you for your help.Please keep the SR in the Soft Closure status for 2 weeks to check whether the same iss ue is reoccurring or not Regards,

44

Ruhul

17-APR-08 06:32:24 GMT . UPDATE ======= Hi Rahul, Thanks for the Update As per your update we shall keep the SR in the SCL status for next 2 weeks This means that the TAR will remain in inactive mode(extended sleep mode) for another 14 days, during which period you can directly update the TAR via Metalink to re-activate it and I will be gl ad to assist you further. Otherwise, no update is necessary and the TAR will aut omatically close after two weeks. For further new issues/errors, if any, please open a new SR so that it gets best handled by appropriate Team Regards, Bharath Sunku STATUS ======= @CUS -- Waiting for requested information 17-APR-08 07:02:57 GMT Email Update button has been pressed: Sending email to ruhul.amin@bracbank.com.

More information
To get more information, login to this web side www.dbasolve.com

45