FRAGMENTATION & DEFRAGMENTATION OF A TABLE

Note:

This document is intended for the ASGs to give some basic concepts on fragmentation and defragmentation on table

Version 1.0

HCL Technologies Pvt. Ltd.

VERSION HISTORY

VERSION 1.0 2.0

DATE 21-05-2009 22-05-2009

DESCRIPTION First Copy Second Copy

AUTHOR Bhavani Shankar G Bhavani Shankar G

REVIEWED BY SIDE DBA Team. Balaji C.S

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 2

TABLE OF CONTENTS

WHEN THE TABLE WILL BE FRAGMENTED?....................................................4 WHAT ARE THE IMPACTS OF FRAGMENTED TABLE?.......................................4 HOW DOES DEFRAGMENTATION/REORGANIZING CHANGE THE TABLE?.......4 WHAT ARE THE BENEFITS ON THE DEFRAGGED TABLE?................................4 WHAT IS THE IDEAL TIME FOR DEFRAGMENTATION?...................................5 HOW TO IDENTIFY TABLE THAT NEEDS TO BE DEFRAGGED/REORGANIZED? 5 WHY TO GATHER STATISTICS FOR TABLE?...................................................5 WHY TO BACKUP OLD VERSION OF STATISTICS?..........................................6 HOW TO BACKUP OLD VERSION OF STATISTICS?.........................................6 HOW TO GATHER STATISTICS FOR A TABLE?................................................7 IS IT ADVISABLE TO GET CONFIRMATION FROM DEVELOPMENT TEAM?.......7 HOW TO IDENTIFY THE INDEXES ASSOCIATED WITH THE TABLE?...............7 HOW TO CHECK THE SPACE WASTED IN A FRAGMENTED TABLE?..................8 WHAT ARE THE DIFFERENT WAYS TO REORGANIZE THE TABLE?..................8 HOW TO USE ALTER TABLE .. MOVE?.............................................................9 HOW TO USE SHRINK METHOD?....................................................................9

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 3

When the Table will be fragmented? Fragmentation occurs in a table when we perform DML (Data Manipulation Language like INSERT/UPDATE/DELETE) operations in table. When rows are not stored contiguously, or if rows are split onto more than one block, performance decreases because these rows require additional block accesses. When a lot of DML operations are applied on a table, the table will become fragmented because DML does not release free space from the table below the High water Mark (HWM) High Water Mark (HWM) is an indicator of USED BLOCKS in the database. Blocks below the high water mark (used blocks) have at least once contained data. When a query is executed, Oracle will scan the entire table to throw the result. This is referred as Full Table Scan (FTS). While performing full table scan, Oracle will always read the data up to HWM. And if there is lot of free space with-in HWM, that is read too, and hence degrading the performance of FTS. What are the Impacts of Fragmented Table? A fragmented table will have the following impacts: a) Slower response time (from that table b) It will have high number of chained (actually migrated) rows. c) Table has grown many folds and the old space is not getting reused. This will increase the table size even if the occupied data is less. For Eg if the Table size is 50 GB and the size of the actual data in the table is 15 GB, then the wasted space in the table is 35 GB. How does defragmentation/reorganizing change the Table? The Chained rows will be coalesced and this will reset the HWM so that usable blocks for the table are increased. What are the benefits on the defragged table? It can be difficult to determine when reorganization will actually create performance gains. However, the performance gains that can be accrued are tremendous when fragmentation and disorganization exist. The defragged table will have the following benefits 1) Since the HWM is reset, any full table scan will read the blocks that contain the data and hence increased response time for a query 2) Since the used blocks are coalesced, the wasted space in the table is eliminated. Thus more tablespace available for the database. Note: Index based queries may not get that much benefited by reorganization as compared to queries which does Full table scan.

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 4

What is the ideal time for defragmentation? Reorganization/Defragmentation will lock the table till the process gets completed. No DML activity cannot be done on the table however, SELECT query is allowable. Hence the activity should be started during the idle time of the table. In this way reorganization can be costly in terms of downtime (outage) and computing resources. How to identify table that needs to be defragged/reorganized? Any table that is frequently purged for any reason, subjected to DML operation would be in fragmented state. An ASG with his business knowledge should be in a position to identify tables. Alternatively the following query can be executed to identify the tables that are fragmented: SQL> SELECT TABLE_NAME FROM USER_TABLES WHERE CHAIN_CNT > 0; Note: 1) The above query will give result only if the statistics for the table is gathered. The above query might throw several tables in the schema. The User has to determine the critical tables that are involved in the application. 2) The User has to note down the response time before the defragmentation to check the impact after the activity. For Instance consider a table that has 5 million rows and a simple query like SELECT COUNT(1) FROM TABLE takes 10 minutes to throw the result. If 3 million rows are later deleted, then the same query will take the same time even though there is reduction in the number of rows. After the defragmentation activity the response time would be lesser than 10 minutes.

Why to gather statistics for table? Whenever a valid SQL statement is processed Oracle has to decide how to retrieve the necessary data. This decision can be made using one of two methods: Rule Based Optimizer (RBO) - This method is used if the server has no internal statistics relating to the objects referenced by the statement. This method is no longer favored by Oracle and will be desupported in future releases. Cost Based Optimizer (CBO) - This method is used if internal statistics are present. The CBO checks several possible execution plans and selects the one with the lowest cost, where cost relates to system resources. Because the cost-based approach relies on statistics, you should generate statistics for all tables and clusters and all indexes accessed by your SQL statements before using the cost-based approach. If the size and data distribution of the tables change

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 5

frequently, then regenerate these statistics regularly to ensure the statistics accurately represent the data in the tables Note: Make sure that you backup old version of statistics before you start collecting new statistics for a table Why to backup old version of statistics? If you don't backup old version of statistics and if optimizer (CBO) behaves differently because of new statistics, then you are in trouble. If you have backup of old version of statistics, you can restore them for the optimizer to respond properly with the old statistics. How to backup old version of statistics? You can do this by the following way: Step 1. Create a new statistics table. The new statistics table can be created by the following query: SQL> EXEC DBMS_STATS.CREATE_STAT_TABLE(ownname =>'SCHEMA_NAME' ,stat_tab => 'STATS_TABLE' , tblspace => 'STATS_TABLESPACE'); Example: SQL > EXEC DBMS_STATS.CREATE_STAT_TABLE(ownname =>'SYSTEM',stat_tab => 'STATS_TABLE'); Step 2. Export statistics to statistics table The existing statistics can be exported to new statistics table by the following query: SQL > EXEC DBMS_STATS.EXPORT_SCHEMA_STATS('ORIGINAL_SCHEMA' ,'STATS_TABLE',NULL,' SYSTEM');

8.

How to restore old version of statistics? Step 1. Import statistics into the data dictionary. The exported statistics can be imported by the following query: SQL > EXEC DBMS_STATS.IMPORT_SCHEMA_STATS('NEW_SCHEMA', 'STATS_TABLE',NULL,'SYSTEM'); Step 2. Drop the statistics table. The backed up statistics if not required can be dropped by the following query: SQL > EXEC DBMS_STATS.DROP_STAT_TABLE('SYSTEM','STATS_TABLE');

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 6

How to gather statistics for a table? The statistics for the table can be gathered by any one of the following methods: 1) ANALYZE TABLE 2) DBMS_STATS.GATHER_STATS ANALYZE TABLE The following is the syntax to gather the statistics of the table/index using ANALYZE TABLE method: SQL> ANALYZE TABLE <TABLE_NAME> COMPUTE STATISTICS; SQL> ANALYZE INDEX <INDEX_NAME> COMPUTE STATISTICS; DBMS_STATS.GATHER_STATS The following is the syntax to gather the statistics of the table/index using GATHER_STATS SQL> EXEC DBMS_STATS.GATHER_STATS('<SCHEMA>','<TABLE_NAME>'); SQL> EXEC DBMS_STATS.GATHER_STATS('<SCHEMA>','<INDEX_NAME>'); The following query can be used to check whether the table has been analyzed or not SQL> SELECT TABLE_NAME, LAST_ANALYZED FROM USER_TABLES WHERE TABLE_NAME=<TABLE_NAME>; Note: Gathering Statistics can be very resource intensive. Hence avoid gathering statistics during peak hours.

Is it advisable to get confirmation from Development Team? There could be situation where the application code is configured to particular EXECUTITION FASHION. Defragmentation/Analysis might change the EXECUTION Plan and might affect the application performance. Hence it is better to get the confirmation from the Development Team/DBA Support Team on the approach chosen How to identify the indexes associated with the table? The following query can be used to identify the indexes associated with the table SQL> SELECT INDEX_NAME FROM USER_INDEXES WHERE TABLE_NAME=<TABLE NAME>

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 7

How to check the Space wasted in a Fragmented Table? The following query can be used to check the size of the table: SQL> SELECT ROUND ((BLOCKS*DBBLK),2) AS SIZE1 FROM USER_TABLES,
(SELECT VALUE AS DBBLK FROM V$PARAMETER WHERE NAME='DB_BLOCK_SIZE') WHERE TABLE_NAME = <'TABLE-NAME'>

The following query can be executed to estimate the actual data in the table SQL> SELECT
USER_TABLES ROUND((NUM_ROWS*AVG_ROW_LEN/1024),2) WHERE TABLE_NAME = '<TABLE-NAME'> as SIZE2 FROM

The difference between the SIZE1 and SIZE2 is the wasted space in the table.

What are the different ways to reorganize the table? There are various ways to reorganize the table. Each has its own merits and its limitations. The conventional ways are: 1. 2. 3. 4. 5. ALTER TABLE ... MOVE + REBUILD INDEXES EXPORT / TRUNCATE / IMPORT CREATE TABLE AS SELECT ( CTAS) DBMS_REDEFINITION SHRINK (from Oracle 10g onwards)

Of all the methods, the ALTER TABLE MOVE is the option generally used for defragmentation. About ALTER TABLE MOVE

1. It is the fastest way to defrag the table as it utilizes more resources. BT also
acknowledges this as the ideal way to defrag the table. 2. ALTER TABLE MOVE will make the indexes associated with the table invalid. Once the table is moved, its associated its indexes have to be Rebuild. The indexes can be rebuilt online. 3. In this method additional space is required in the tablespace. Oracle will create an image of the table with same size of the original table and will drop the table after the completion of the defragmentation activity. For instance if the size of the fragmented table is 10 GB , additional 10GB is required in the tablespace

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 8

How to use ALTER TABLE .. MOVE? The following steps have to be followed by every table and its associated indexes. Step1: The Table and its associated indexes have to be put into NOLOGGING mode to avoid excessive REDO generation by the following query: SQL> ALTER TABLE <TABLE NAME> NOLOGGING; SQL> ALTER INDEX <INDEX1> NOLOGGGING; SQL> ALTER INDEX <INDEX2> NOLOGGGING; Step2: The chosen table can be defragged by the following query: SQL> ALTER TABLE <TABLE NAME> MOVE; Step3: The associated indexes of the table will be invalid position. Hence it has to be rebuilt and the following query can be used: SQL> ALTER INDEX <INDEX1> REBUILD ONLINE; SQL> ALTER INDEX <INDEX2> REBUILD ONLINE; Step4: Once the table is defragged and its associated indexes are rebuilt, they can be put back to LOGGING Mode by the following query: SQL> ALTER TABLE <TABLE NAME> LOGGING; SQL> ALTER INDEX <INDEX1> LOGGING; SQL> ALTER INDEX <INDEX2> LOGGING; Step5: Gather the statistics of the table and its indexes How to use SHRINK Method? The SHRINK Method is applicable for tables only in TABLESPACE with Auto Segment Space Management. As mentioned earlier it is applicable from Oracle 10g onwards. The steps to be followed are as under: Step1: The row movement should be enabled and the following can be used: SQL> ALTER TABLE <TABLE NAME> ENABLE ROW MOVEMENT; Step 2 (a) In first part rearrange rows and in second part reset the HWM. To rearrange the rows, the following query can be used: SQL> ALTER TABLE <TABLE NAME> SHRINK SPACE COMPACT; (All DML's can happen during this time)

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 9

To reset the HWM, the following query can be used: SQL> ALTER TABLE <TABLE NAME> SHRINK SPACE; (No DML can happen. but this is fairly quick, infact goes unnoticed.) Step3: Do it in one go: SQL> ALTER TABLE <TABLE NAME> SHRINK SPACE; Few advantages over the conventional methods 1. Unlike "ALTER TABLE.. MOVE", indexes are not in UNUSABLE state. After shrink command, indexes are updated also. 2. It is an online operation and hence no downtime to do this reorganization 3. It does not require any extra space for the process to complete.

16.

How to check the Impact after Defragmentation? 1) Gather the statistics of the table 2) Check the response time by executing the query on the defragged table earlier chosen to check the reduction in response time(Refer 6.0) 3) Check for the wasted space in the table. Just to recap the following queries can be used to check the wasted space in a table SQL> SELECT ROUND ((BLOCKS*DBBLK),2) AS SIZE1 FROM USER_TABLES,
(SELECT VALUE AS DBBLK FROM V$PARAMETER WHERE NAME='DB_BLOCK_SIZE') WHERE TABLE_NAME = <'TABLE-NAME'>

The following query can be executed to estimate the actual data in the table SQL> SELECT
USER_TABLES ROUND((NUM_ROWS*AVG_ROW_LEN/1024),2) WHERE TABLE_NAME = '<TABLE-NAME'> as SIZE2 FROM

The difference between the SIZE1 and SIZE2 is the wasted space in the table.

------------------------------------------------------------------------------------------------------Version 2.0 HCL Technologies Pvt. Ltd. 10