• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • 5
    CommentGo Back
Download
 
S
TEP
-
BY
-
STEP
S
TREAMS
Lewis R Cunningham AKA LewisC, PricewaterhouseCoopers, LLP 
Introduction
Recently, I had a project to migrate data from an Oracle 10g EE database to an Oracle 9i EE database. The data to bemigrated was real-time and required fairly large data transformation. The basic use of the migration was to keep an older  production 9i database in sync with the newer 10g database for several months while applications dependant on this datawere remediated to use the new database. This paper and the accompanying presentation are based on the proof of concept(POC) we performed to prove this functionality.The POC would use Oracle streams, Change Data Capture (CDC) specifically, to capture the data. That data was thenavailable to other 10g databases but not to 9i databases. To enable us to send the data to 9i, we enqueued the captured datainto an AQ queue that was then propagated to a 9i database. Once the data was in 9i, it was a simple process to dequeue thedata and apply it.Below is the short list for setting up Change Data Capture using Oracle Streams. These steps are mostly from the docs with afew tweaks I have added. Following the setup of CDC is a discussion of using AQ to move the data from 10g to 9i. The paper concludes with a detailed discussion of creating your own manual Logical Change Record (LCR).
Change Data Capture
First the set up: we will use the HR account's Employee table. We'll capture all changes to the Employee table and insert theminto an audit table. I'm not necessarily saying this is the way you should audit your database but it makes a nice example.I'll also add a monitoring piece to capture process. I want to be able to see exactly what is being captured when it is beingcaptured.You will need to have sysdba access to follow along with me. Your database must also be in archivelog mode. The changesare picked up from the redo log.So, away we go!The first step is to create our streams administrator. I will follow the guidelines from the oracle docs exactly for this:Connect as sysdba:
sqlplus / as sysdba
Create the streams tablespace (change the name and/or location to suit):
create tablespace streams_tbs datafile 'c:\temp\stream_tbs.dbf' size 25M reuse autoextend on maxsize unlimited;
Create our streams administrator:
create user strmadmin identified by strmadmindefault tablespace streams_tbsquota unlimited on streams_tbs;
I haven't quite figured out why, but we need to grant our administrator DBA privs. I think this is a bad thing. There is probably a work around where I could do some direct grants instead but I haven't had time to track those down.
grant dba to strmadmin;
We also want to grant streams admin privs to the user.
BEGINDBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE(grantee => 'strmadmin',
www.odtug.com1ODTUG Kaleidoscope 2007
 
Step-By-Step Streams Cunningham
grant_privileges => true);END;/
The next steps we'll run as the HR user.
conn hr/hr
Grant all access to the employee table to the streams admin:
grant all on hr.employees to strmadmin;
We also need to create the employee_audit table. Note that I am adding three columns in this table that do not exist in theemployee table.
CREATE TABLE employee_audit(employee_id NUMBER(6),first_name VARCHAR2(20),last_name VARCHAR2(25),email VARCHAR2(25), phone_number VARCHAR2(20),hire_date DATE,job_id VARCHAR2(10),salary NUMBER(8,2),commission_pct NUMBER(2,2), manager_id NUMBER(6),department_id NUMBER(4),upd_date DATE,user_name VARCHAR2(30),action VARCHAR2(30));
Grant all access to the audit table to the streams admin user:
grant all on hr.employee_audit to strmadmin;
We connect as the streams admin user:
conn strmadmin/strmadmin
We can create a logging table. You would NOT want to do this in a high-volume  production system. I am doing this to illustrate user defined monitoring and show how you can get inside the capture process.
CREATE TABLE streams_monitor (date_and_time TIMESTAMP(6) DEFAULT systimestamp,txt_msg CLOB );
Here we create the queue. Unlike AQ, where you have to create a separate table, this step creates the queue and theunderlying ANYDATA table.
BEGINDBMS_STREAMS_ADM.SET_UP_QUEUE(queue_table => 'strmadmin.streams_queue_table',queue_name => 'strmadmin.streams_queue');END;/
This just defines that we want to capture DML and not DDL.
BEGINDBMS_STREAMS_ADM.ADD_TABLE_RULES(table_name => 'hr.employees',streams_type => 'capture',streams_name => 'capture_emp',queue_name => 'strmadmin.streams_queue',include_dml => true,include_ddl => false,inclusion_rule => true);
www.odtug.com2ODTUG Kaleidoscope 2007
 
Step-By-Step Streams Cunningham
END;/
Tell the capture process that we want to know who made the change:
BEGINDBMS_CAPTURE_ADM.INCLUDE_EXTRA_ATTRIBUTE(capture_name => 'capture_emp',attribute_name => 'username',include => true);END;/
We also need to tell Oracle where to start our capture. Change the source_database_name to match your database.
DECLAREiscn NUMBER;BEGINiscn := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER();DBMS_APPLY_ADM.SET_TABLE_INSTANTIATION_SCN(source_object_name => 'hr.employees',source_database_name => 'ORCL',instantiation_scn => iscn);END;/
And the fun part! This is where we define our capture procedure. I'm taking this right from the docs but I'm adding a couplesteps.
CREATE OR REPLACE PROCEDURE emp_dml_handler(in_any IN ANYDATA) ISlcr SYS.LCR$_ROW_RECORD;rc PLS_INTEGER;command VARCHAR2(30);old_values SYS.LCR$_ROW_LIST;BEGIN-- Access the LCR rc := in_any.GETOBJECT(lcr);-- Get the object command typecommand := lcr.GET_COMMAND_TYPE();-- I am inserting the XML equivalent of the LCR into the monitoring table.insert into streams_monitor (txt_msg) values (command ||DBMS_STREAMS.CONVERT_LCR_TO_XML(in_any) );-- Set the command_type in the row LCR to INSERTlcr.SET_COMMAND_TYPE('INSERT');-- Set the object_name in the row LCR to EMP_DELlcr.SET_OBJECT_NAME('EMPLOYEE_AUDIT');-- Set the new values to the old values for update and deleteIF command IN ('DELETE', 'UPDATE') THEN-- Get the old values in the row LCR old_values := lcr.GET_VALUES('old');-- Set the old values in the row LCR to the new values in the row LCR lcr.SET_VALUES('new', old_values);-- Set the old values in the row LCR to NULLlcr.SET_VALUES('old', NULL);END IF;-- Add a SYSDATE for upd_datelcr.ADD_COLUMN('new', 'UPD_DATE', ANYDATA.ConvertDate(SYSDATE));-- Add a user columnlcr.ADD_COLUMN('new', 'user_name',lcr.GET_EXTRA_ATTRIBUTE('USERNAME') );-- Add an action columnlcr.ADD_COLUMN('new', 'ACTION', ANYDATA.ConvertVarChar2(command));
www.odtug.com3ODTUG Kaleidoscope 2007
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
gpattni7772

Thanks, do you have a 11g step by step guide?

reply05 / 04 / 2012
td1947

Very useful, thanks for taking the time to document this.

reply05 / 06 / 2010
holyindian

Anyone implemented Oracle Streams using this impressive looking document. I have a major project coming up with streams, and i am looking upto this manual.

reply12 / 17 / 2008
jhaque

Excellent work! Keep up good work.

reply11 / 25 / 2008
You must be to leave a comment.
Submit
Characters: ...