Paul Tsien/Ashish Ray

Product Manager Oracle Corporation

LogMiner , Flashback Query, Online Redefinition

Power Tools for DBAs

Oracle LogMiner

Manageability, Serviceability & Auditing

Critical issues for an enterprise class database:

Key data changes User errors that may cause system downtime Right tools
Analyze, resolve and rectify these situations as needed Should have minimal impact on the production system

LogMiner is a powerful tool that meets these manageability, serviceability and auditing requirements for Oracle databases

What is LogMiner?
Comprehensive and flexible tool that enables administrators and auditors to track changes made to the Oracle database Ability to undo the changes (after auditing logs) No overhead normally associated with auditing large systems Underlying technology for a powerful set of management and analysis tools Integrated feature of both the Standard and Enterprise editions of the database

LogMiner Capabilities
Track specific sets of changes

Based on transaction, user, table, time, value, etc. Determine who modified a database object Determine the states previous to and following the data changes

Track database objects e.g. tables, as they undergo structural changes, e.g. after applying DDLs on them

Effectively and accurately monitor schema evolution

Pinpoint when an incorrect modification was made Logs provide supplemental information for tuning and capacity planning Retrieve critical information for debugging complex applications

How Does LogMiner Work?

Uses Oracle redo log files

Wealth of useful information about the database activities Provides a flexible infrastructure to read, analyze and interpret redo log files through a standard SQL-based interface Only Oracle-endorsed method of accessing the redo stream Translate internal object identifiers and data types to meaningful external object names and data formats Each row represents complete information on a database operation Can be easily accessed via standard SQL, or programmatically from any supported language or interface

Uses database dictionary

Log files mapped to a dynamic view named V$LOGMNR_CONTENTS

V$LOGMNR_CONTENTS
Type of change (insert, update, delete, DDL) System Change Number (SCN) at which the change occurred The transaction identification containing the change SCN at which the particular transaction committed Table and schema name of the modified object User who issued the DML or DDL statements Reconstructed SQL statements (SQL_REDO column) showing the "equivalent" SQL that generated the redo records Reconstructed SQL statements (SQL_UNDO column) providing the SQL required to undo the change Other information

Steps to Use LogMiner

Initial setup turn on Supplemental Logging:
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Extract dictionary
SQL> EXECUTE DBMS_LOGMNR_D.BUILD ( 2 OPTIONS=>DBMS_LOGMNR_D.STORE_IN_REDO_LOGS);

Specify redo log files for analysis:

SQL> EXECUTE DBMS_LOGMNR.ADD_LOGFILE( 2 LOGFILENAME => '/oracle/logs/log1.f', 3 OPTIONS => DBMS_LOGMNR.NEW); SQL> EXECUTE DBMS_LOGMNR.ADD_LOGFILE( 2 LOGFILENAME => '/oracle/logs/log2.f', 3 OPTIONS => DBMS_LOGMNR.ADDFILE);

Steps to Use LogMiner (contd.)

Start LogMiner session:
SQL> EXECUTE DBMS_LOGMNR.START_LOGMNR(OPTIONS => 2 DBMS_LOGMNR.DICT_FROM_REDO_LOGS + 3 DBMS_LOGMNR.COMMITTED_DATA_ONLY, 4 STARTTIME => TO_DATE('01-Nov-2002 08:00:00', 'DD-MON-YYYY HH:MI:SS'), 5 ENDTIME => TO_DATE('01-Nov-2002 17:00:00', 'DD-MON-YYYY HH:MI:SS'));

Query V$LOGMNR_CONTENTS:
SQL> SELECT SQL_REDO, SQL_UNDO FROM V$LOGMNR_CONTENTS 2 WHERE USERNAME = 'joesmith' AND SEG_NAME = employee';

End LogMiner session:

SQL> EXECUTE DBMS_LOGMNR.END_LOGMNR;

LogMiner Example
Employee Joe Smith suspected to have gained access to the corporate database and altered his personnel record DBA must find out what he did LogMiner comes to the rescue

LogMiner Example (contd.)

DBA starts a LogMiner session LogMiner extracts redo contents into V$LOGMNR_CONTENTS DBA executes the following query to determine the changes Joe Smith made to the EMPLOYEE table:
SELECT SQL_REDO, SQL_UNDO FROM V$LOGMNR_CONTENTS WHERE USERNAME = joe smith AND SEG_NAME = EMPLOYEE;

LogMiner Example (contd.)

Query returns the following output:

SQL_REDO
DELETE * FROM EMPLOYEE WHERE EMPNO = 12345 AND ROWID =AAABOOAABAAEPCABA; INSERT INTO EMPLOYEE (NAME,EMPNO,SALARY) VALUES (joesmith, 12345, 90000);

SQL_UNDO
INSERT INTO EMPLOYEE (NAME,EMPNO,SALARY) VALUES (joesmith, 12345, 75000); DELETE * FROM EMPLOYEE WHERE EMPNO = 12345 AND ROWID =AAABOOAABAAEPCABA;

Joe Smith deleted himself and recreated himself with a higher salary!

Tracking DDLs
LogMiner uses a Multi-Versioned Dictionary Tracks objects as they undergo DDL-related structural changes (e.g. adding/dropping columns) Example:
SQL> select username,operation,sql_redo from v$logmnr_contents 2 where username = 'HRand operation = 'DDL; USERNAME OPERATION SQL_REDO -------- --------- --------------------------HR DDL alter table job_history add final_sal number(8,2)

Can accurately show DML operations performed on these objects previous to, or after, their structural changes

Example Capacity Planning

Assume you manage a direct marketing database Determine how productive the customer contacts have been in generating revenue for a two week period in August Steps:

Start LogMiner and specify a range of times:

SQL> EXECUTE DBMS_LOGMNR.START_LOGMNR( 2 STARTTIME => TO_DATE('01-Aug-2002 08:30:00','DD-MON-YYYY HH:MI:SS'), 3 ENDTIME => TO_DATE(15-Aug-2002 08:45:00','DD-MON-YYYY HH:MI:SS'), 4 DICTFILENAME => '/usr/local/dict.ora');

Query V$LOGMNR_CONTENTS for tables modified in the time range specified:

SQL> SELECT SEG_OWNER, SEG_NAME, COUNT(*) AS Hits FROM 2 V$LOGMNR_CONTENTS WHERE SEG_NAME NOT LIKE '%$' GROUP BY 3 SEG_OWNER, SEG_NAME;

Example Capacity Planning (contd.)

Steps (contd.):

The following data is displayed:

SEG_OWNER SEG_NAME --------- -------CUST SCOTT SYS UNIV UNIV UNIV ACCOUNT EMP DONOR DONOR EXECDONOR MEGADONOR Hits ---384 12 12 234 325 32

Hits column shows the number of times each table had an insert, delete, or update operation performed on it in time range specified

LogMiner Viewer
GUI-based interface improves LogMiner usability:

LogMiner - Summary
Powerful data audit tool

Insulates clients from log file format changes Common and familiar interface to log data Eases development of tools that use log info No additional data collection overhead

Enables sophisticated analysis:

Tracking changes through a series of statements and transactions Determining how triggers and application updates interact Determining the rate at which a table is being updated over time to do capacity planning and tuning

Oracle Flashback Query

Human Error Challenge

Critical issues for an enterprise class database:

Human error is one of the major causes of system failures

Flashback Query is a powerful tool that empowers end-users to correct their own mistakes without involving the DBA and without an
expensive recovery operation

What is Flashback Query?

Versatile tool that provides a simple and powerful mechanism for recovering from human errors Allows users to view state of data at a point in time in the past without requiring any structure changes to the database based on

Wall clock time System Change Number (SCN)

Allows users to query, compare, and recover past data without involving the DBA and without an expensive recovery operation

Oracles Time Machine

Flashback Query

Query the database at a time of your choosing Set the time and run any SQL query Leverages Oracles unique multiversioning read consistency architecture Use standard SQL operations to make corrections

Oracle Invents Time Machine

Create table old_emp as select * from emp AS OF yesterday;

Flashback Query Capabilities

Standard SQL interface simplifies deployment Self-service means faster, cheaper, and easier Flashback Query is a fast operation to enable self service

Speed is relative to size of queried data, not total database workload No overhead for change tracking

How Does Flashback Query Work?

Leverages Oracles unique multi-version read consistency architecture

The data image is saved in the undo tablespace (or Rollback Segments) before being modified Flashback Query uses the data saved in the undo tablespace to recreate an image of the data as it existed at a time in the past.

Oracle9is Automatic Undo Management feature allows administrators to specify how long they wish to retain the undo data

DBAs can control how far back a Flashback Query can go

How Does Flashback Query Work? (contd.)

DBA can easily determine the estimated undo space requirement using the Oracle Enterprise Manager

Flashback Query Example

A set of bank accounts was accidentally deleted from the ACCOUNTS table at 11 AM, and the error was discovered at 2 PM DBA needs to fix this problem

Rolling back the database prior to 11 AM means lost transactions Systems are unavailable during the recovery

Flashback Query comes to the rescue

Flashback Query Example (contd.)

The following query retrieves the deleted accounts by comparing the current data in the ACCOUNTS table against what it was just prior to 11 AM and, inserts them back into the table.
INSERT INTO accounts (SELECT * FROM accounts AS OF TIMESTAMP TO_TIMESTAMP(13-MAR-02 10:59:58,DD-MON-YY HH24:MI:SS) MINUS SELECT * FROM accounts);

Flashback Query Example Tracking Data Change

Flashback Query can also be used to generate a daily report of all new accounts created on any given day by creating a view as shown below
CREATE VIEW accounts_created_today AS SELECT * FROM accounts MINUS SELECT * FROM accounts AS OF TIMESTAMP TRUNC(SYSDATE); SELECT * FROM accounts_created_today;

Flashback Query - Summary

Powerful tool for a DBA and users

Empowers end-users to correct their own mistakes

without involving the DBA Avoids expensive recovery operation Simplifies daily operations such as tracking incremental data changes and querying the past state of data

Reduces system downtime due to human errors

LogMiner vs. Flashback Query

LogMiner

Powerful data auditing tool Provides extensive information on transaction, user, table, time, value, etc. Past information limited only by the availability of archive logs Can be used for tuning, capacity planning and application debugging Powerful tool that enable users to correct their own mistakes Correct human errors up to many days in the past Simple, quick, and easy to use Applications can use Flashback Query to undo user errors

Flashback Query

Oracle Online Redefinition

Continuous Operation
Critical issues for an enterprise class database:

Data needs to evolve

Competition moves at web speed Data volume growth Routine software and hardware maintenance and upgrades

Data reorganization must not interrupt service

Oracle9i has the most complete online data reorganization capability in the industry Online Redefinition is a powerful tool that allows an Oracle DBA to perform data reorganizations online

What is Online Redefinition?

New architecture to reorganize and redefine tables online

Table contents are copied to a new table

Eliminates fragmentation Allows changing location, table type, partitioning, etc.

Contents can be transformed as they are copied

Use SQL select statement to specify result table format Allows changes to columns, type, sizes, etc.

Updates and queries can continue uninterrupted

Online Redefinition Capabilities

An Oracle DBA can use Online Redefinition to

Modify the physical attributes or storage parameter of a table Move a heap table or index-organized-table (IOT) to a different tablespace Add support for parallel queries Add or drop partitioning support Recreate a heap table or IOT to reduce fragmentation Change a heap table to IOT and vice versa Add, drop, or rename columns in a table Transform data in a table Improve the application upgrade process

How Does Online Redefinition Work?

Source Table Copy Table Transform Result Table GUI interface to make it Simple Update Tracking
Store Updates

Transform Updates

Steps to Use Online Redefinition

Create an empty interim table with all the desired attributes Start the online redefinition process by calling
SQL> EXECUTE DBMS_REDEFINITION.START_REDEF_TABLE()

Create any triggers, indexes, grants and constraints on the interim table Optionally synchronize and validate data in the interim table periodically Complete the online redefinition process by calling
SQL> EXECUTE DBMS_REDEFINITION.FINISH_REDEF_TABLE()

Online Redefinition Example

DBA needs to redefine a non-partitioned heap table in the u1 schema

The column salary is multiplied by a factor of 1.10 and renamed as sal The column phone is dropped A new column deptno with default value of 10 is added The redefined table is partitioned by range on empno

Online Redefinition Example (contd.)

DBA creates the interim table, int_emp:
CREATE TABLE int_emp (empno NUMBER PRIMARY KEY, name VARCHAR2(100), sal NUMBER, deptno NUMBER DEFAULT 10) PARTITION BY RANGE(empno) (PARTITION emp1000 VALUES LESS THAN (1000) TABLESPACE tbs_1,PARTITION emp2000 VALUES LESS THAN (2000) TABLESPACE tbs_2);

Online Redefinition Example (contd.)

DBA starts the redefinition process
DBMS_REDEFINITION.START_REDEF_TABLE('u1', 'emp', 'int_emp', 'empno empno, name name, salary*1.10 sal');

DBA creates any triggers, indexes, and constraints on the interim table, int_emp

Online Redefinition Example (contd.)

DBA optionally synchronizes the interim table
DBMS_REDEFINITION.SYNC_INTERIM_TABLE('u1', 'emp', 'int_emp');

DBA completes the redefinition process

DBMS_REDEFINITION.FINISH_REDEF_TABLE('u1', 'emp', 'int_emp');

Oracle Enterprise Manager

Oracle Enterprise Manager (OEM) GUI-based interface improves Online Redefinition usability:

Online Redefiniton - Summary

Powerful tool for a DBA to

Modify the tables physical attributes online Change the table structure online Transform the table data online

Online Redefiniton enables a DBA to

Improve database performance and response time Improve disk space utilization Improve application upgrade process

Enables 24x7 availability for an enterprise

Sessions by Oracle Database Development

Monday RAC: The Present, The Future, but not Science Fiction
Mon, 1pm -- Moscone Room 103

Next Steps

Tuesday Breaking All the Rules with The Unbreakable Database

Tue, 11am -- Moscone Room 103

Running Your Applications on Oracle Real Application Clusters

Mon, 11am -- Moscone Room 134

Oracles Recipe For Building An Unbreakable System

Tue, 1pm -- Moscone Room 134

Real Customers, Real Application Clusters, Real Results

Mon, 4pm -- Moscone Room 134

Bullet-Proof Data Protection with Oracle Data Guard

Tue, 4pm -- Moscone Room 134

Deploying A Highly Manageable Oracle Real Application Clusters Database

Mon, 5:30pm -- Moscone Room 134

For More Info On Oracle HA Go To http://otn.oracle.com/deploy/availability/

Sessions by Oracle Database Development

Wednesday Getting Under The Hood With Data Guard SQL Apply
Wed, 8:30am -- Moscone Room 134

Next Steps

Database HA Demos All Four Days In The Oracle Demo Campground Real Application Clusters Data Guard Backup & Recovery with Recovery Manager LogMiner, Flashback Query and Online Redefinition

LogMiner, Flashback Query and Online Redefinition: Power Tools For DBAs
Wed, 11am -- Moscone Room 134

Are You Using The Best To Protect Your Enterprise Data?

Wed, 4pm -- Moscone Room 252

Oracle LogMiner - Not Just An Error Recovery Tool

Wed, 5:30pm -- Moscone Room 102

For More Info On Oracle HA Go To http://otn.oracle.com/deploy/availability/

Sessions by Oracle Database Development

Showcase Presentation/Demo Monday 11:00 AM -- Database High Availability: Data Guard 11:30 AM -- Database High Availability: Backup & Recovery and Recovery Manager 12:00 PM -- Database High Availability: Online Reorg, Flashback Query and LogMiner Tuesday 11:00 AM -- Real Application Clusters: Scalability 11:30 AM -- Real Application Clusters: High Availability 12:00 PM -- Real Application Clusters: CFS on Linux Wednesday 11:00 AM -- Real Application Clusters: Scalability 11:30 AM -- Real Application Clusters: High Availability 12:30 PM -- Database High Availability: Data Guard

Next Steps

For More Info On Oracle HA Go To http://otn.oracle.com/deploy/availability/