40621435 123217 Oracle Streams Step by Step PPT | Oracle Database | Databases

Oracle Streams: Step by Step

What is Oracle Streams? What is Advanced Queues (AQ)? What is Change Data Capture (CDC)? What is a Logical Change Record (LCR)? How Streams Works How do you configure Streams, AQ and CDC? How do you use Streams in your programs?
± ± ±

Adding records automatically Adding record programmatically Pulling records off the queue

Who am I
Lewis R Cunningham Oracle ACE Certified PL/SQL Developer Author Blogger Database Geek Member of SOUG, IOUG, ODTUG, ACM

What is Oracle Streams?
Streams enables data and event movement Movement may be within or between databases Can implement replication Data and events can be captured
± ± ± ±

explicitly: programatically Implicitly: redo log Log mining reduces overhead Downstreams mining reduces it even more

Performant

cont'd Streams Features ± ± ± ± ± ± ± ± Distinct capture/propagation/apply services Message Queueing (via AQ) Publish Subscribe Rules based Capture/Transform/Apply Database integration Easy configuration Flexible Inline transformations .What is Oracle Streams?.

What is Oracle AQ? Oracle's messaging solution Queue based ± ± Persistent queues protect data Buffered. non-persistent queues offer maximum performance Oracle provides all of the maintenance code Secure ± Queue Level Access Control Messages Types ± ± ± ± Raw XML Object Anydata .

OCCI (C++) Java JMS PL/SQL OO40 (COM) Internet Data Access (IDAP) ODBC via Heterogeneous Gateways . cont'd Publish/Subscribe ± ± Multi-consumer Multi-message types RAC integration API support ± ± ± ± ± ± OCI (C).What is Oracle AQ?.

What is Change Data Capture? CDC efficiently identifies changed data and make it available for downstrean use CDC enables efficient warehouse loading ± ± No flat files Direct data movement Changes are queued Multiple subscribers are allowed Subscribers can receive changed data Data may be transformed Changes can include DDL and DML .

the record is a database object For DML.What is an LCR? LCR = Logical Change Record Committed records generate an LCR to the redo log An LCR shows the difference between an existing record and the new record For DDL. the record is table data .

What is an LCR? .

How it works Sourc ¡ t s T r t t s T   £¢ l 0 .

Data Changes Sourc ¥¤ t s T r t t s T l og Writ r ¦ t i ul tion 11 .

DB Writes to OS File Sourc ¨§ t s T rget t se T le Log Writer Writes To LOG 12 .

Streams Takes Over Source © t se T rget t se T le Log Writer Capture rocess eads edo and tracts t e Logical Change ecord (LCR) Streams apture O LOG 13 .

Streams Queues The LCR Source atabase Table Target atabase Streams ueue Log Writer REDO LOG Streams Capture Streams Capture rocess rites the LCR to a local OR REMOTE Streams ueue Table 14 .

LCR Propagates Source Database Table Target Database Streams ueue Streams ueue Log Writer REDO LOG Streams Capture Streams Capture rocess rites the LCR to a local OR REMOTE Streams ueue Table 15 .

Changes Are Applied Source Database Table Target Database Streams ueue Streams ueue Log Writer REDO LOG Streams Capture Streams ppl Streams ppl can also transform for one record to many. add columns. etc. Streams Capture rocess rites the LCR to a local OR REMOTE Streams ueue Table 16 .

Target Database Source Database Table Target Database Streams ueue Table Streams ueue Log Writer Table REDO LOG Streams Capture Streams pply Table Streams Capture rocess rites the LCR to a local OR REMOTE Streams ueue Table Oracle Streams Overview .US IFS 17 .

Configuring Streams The Scenario ± ± ± ± Capture changes to the Employee table in 10g ORCL instance) Send changes to Employee_audit in 9i (SECOND instance) We will start with capture and apply in 10g Once the capture and apply is working. we will modify the stream to send the data to 9i .

Configuring Streams 10g Create Streams Admin Setup CDC Queue Create Capture Rule Instantiate SCN Create Apply Rule Add Supplemental Logging Create DML Procedure Create DML Handlers Set Parameters Start Apply Start Capture .

Configuring Streams 9i Create AQ Table Create AQ Queue Start AQ Queue Create Propagate on 10g Create Dequeue Procedure Dequeue Transactions .

queue_name END. include_dml include_ddl > 3 3 4 "  ! &%   &% 5     #   $   . > true. BEGIN B S _STREAMS_AUTH. streams_t pe > ' apture'. BEGIN DBMS_STREAMS_ADM.ADD_TAB E _RULES( table_name > ' es_ods.ent2'. queue_name > 'streams.  2 CDC Setup CDC ueue 3 3 CDC Create Capture Rule 2    3 3 3 3 3 3 3 7 6  im Setup Create Streams mi Create Streams mi BEGIN DBMS_STREAMS_ADM.streams_ent_queue'. > 'streams.Configuring Streams 2 0) '   1 ) (   St fi Sour r t m l CREATE S ER Streams.streams_ent_queue_table'.streams_ent_queue' ). RANT A TO Streams. streams_name > ' apture_ent2'. rant_pri ileges > true).SET_UP_QUEUE( ueue_table > 'streams.GRANT_ADMIN_PRIVI E GE( rantee > 'streams'. END.

include = true). END.world'. BEGIN l_scn := DBMS_FLASHBACK.ADD_TABLE_RULES( table_name = 'ces_ods. instantiation_scn = l_scn DECLARE entity_rule_name_dml ARCHAR2(30). entity_rule_name_ddl ARCHAR2(30). BEGIN DBMS_CAPTURE_ADM. attribute_name = 'username'.SET_TABLE_INSTANTIATION_SCN ( source_object_name = 'ces_ods.ent2'. DBMS_APPLY_ADM. source_database_name = 'lbces.INCLUDE_EXTRA_ATTRIBUTE( capture_name = 'capture_ent2'. streams_type = 'apply'.GET_SYSTEM_CHANGE_NUMBER ().Configuring Streams DECLARE l_scn number. BEGIN DBMS_STREAMS_ADM.ent2'. / 8 8 8 9 9 8 8 8 8 8 8 4 CDC Instantiate SCN 5 CDC Create Apply Rule 6 CDC Add Any Sumplemental Logging . streams_name = 'apply_ent2'.

± .LCR$_ROW_LIST. old_values SYS. BEGIN -.LCR$_ROW_RECORD. insert into streams_monitor (txt_msg) values (command || DBMS_STREAMS.Access the LCR rc := in_any. command VARCHAR2(30). -.I am inserting the XML equivalent of the LCR into the monitoring table.GET_COMMAND_TYPE().Configuring Streams Create DML Handler Proc ± CREATE OR REPLACE PROCEDURE emp_dml_handler(in_any IN ANYDATA) IS lcr SYS. -. rc PLS_INTEGER.Get the object command type command := lcr.GETOBJECT(lcr).CONVERT_LCR_TO_XML(in_any) ).

GET_VALUES('old'). ± .Set the object_name in the row LCR to EMP_DEL lcr. END IF.Set the old values in the row LCR to the new values in the row LCR lcr. NULL).Set the new values to the old values for update and delete IF command IN ('DELETE'.Set the old values in the row LCR to NULL lcr.Configuring Streams Create DML Handler (cont'd) ± -.Get the old values in the row LCR old_values := lcr.Set the command_type in the row LCR to INSERT lcr.SET_VALUES('new'. -.SET_VALUES('old'. -. -.SET_COMMAND_TYPE('INSERT'). 'UPDATE') THEN -.SET_OBJECT_NAME('EMPLOYEE_AUDIT'). old_values). -.

Add an action column lcr.GET_EXTRA_ATTRIBUTE('USERNAME') ).ConvertDate(SYSDATE)). -.Make the changes lcr.ADD_COLUMN('new'. / . END.Add a SYSDATE for upd_date lcr.ADD_COLUMN('new'.ConvertVarChar2(command)). 'UPD_DATE'.Add a user column lcr. 'user_name'. -.EXECUTE(true).ADD_COLUMN('new'. lcr. commit. -.Configuring Streams Create DML Handler (cont'd) -. ANYDATA. 'ACTION'. ANYDATA.

ent ' object_type => 'TA LE' operation_na e => 'INSERT' error_handler => fal e. value => 'n' . E IN D M S_CAPTURE_ADM. END. E IN D M S_APPLY_ADM.START_CAPTURE( capture_na e => 'capture_ent ' . u er_procedure => ' trea . END.Configuring Streams E IN D M S_APPLY_ADM.SET_DML_HANDLER( object_na e => 'ce _od . END.ce _d l_handler'. apply_d E IN D M S_APPLY_ADM. para eter => 'di able_on_error'.START_APPLY( apply_na e => 'apply_ent ' . CDC Start Capture F D C B A A F D C B A A C E D @ E C @ @ E D @ A @ @ @ F C C C C @ B B A A A A 8 CDC Create DML handler CDC Start Apply C 9 CDC Set Any Additional Para eter @ @ GG G H .SET_PARAMETER( apply_na e => 'apply_ent '.

queue_payload_type > 'sys. queue_table > 'streams. 13 AQ Create AQ Q Q Create AQ Q Q BEGIN DBMS_AQADM.lrc_src_ent3_mc_q@lbindp.lrc_src_ent3_mc_t'). 14 AQ Start AQ Q Start AQ Q BEGIN DBMS_STREAMS_ADM. END.lrc_src_ent3_mc_q'.Configuring Streams BEGIN DBMS_AQADM.ADD_TABLE_PROPAGATION_RULES( table_name > 'ces_ods. streams_name > 'ces_to_gems2'.lrc_src_ent3_mc_q'.lrc_src_ent3_mc_q'). mltype'.ent2'. queue_name END. compatible > '8.1'). multiple_consumers > TRUE. source_queue_name > 'streams. orld'.START_QUEUE ( > 'streams.CREATE_QUEUE_TABLE( queue_table > 'lrc_src_ent3_mc_t'.CREATE_QUEUE( queue_name > 'streams. 15 AQ Create Prop Schedule Q I I I I I I I P I I I I . destination_queue_name > 'streams. END. 12 AQ Ceate AQ Q Table Create AQ Q Table BEGIN DBMS_AQADM.

row_lcr SYS.Configuring Streams Create a DEQUEUE proc in 9i ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± CREATE OR REPLACE PROCEDURE emp_dq (consumer IN VARCHAR2) AS msg ANYDATA. . more_messages BOOLEAN := true. num_var pls_integer.LCR$_ROW_RECORD. navigation VARCHAR2(30).NO_WAIT). streams_name => consumer. wait => DBMS_STREAMS_MESSAGING.PUT_LINE('Looping thru messages'). WHILE (more_messages) LOOP DBMS_OUTPUT.streams_queue'. payload => msg. BEGIN navigation := 'FIRST MESSAGE'. navigation => navigation.DEQUEUE( queue_name => 'strmadmin. BEGIN DBMS_STREAMS_MESSAGING.

navigation := 'NEXT MESSAGE'.PUT_LINE(row_lcr. WHEN DBMS_STREAMS_MESSAGING.GET_COMMAND_TYPE || ' row LCR dequeued').GETTYPENAME() = 'SYS.Configuring Streams Create a DEQUEUE proc in 9i ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± IF msg. DBMS_OUTPUT.DBMS_STREAMS_MESSAGING.NOMOREMSGS THEN more_messages := false.PUT_LINE('No more messages. WHEN OTHERS THEN RAISE. END LOOP.ENDOFCURTRANS THEN navigation := 'NEXT TRANSACTION'. COMMIT. END. / .LCR$_ROW_RECORD' THEN num_var := msg. END.'). END IF. DBMS_OUTPUT.GetObject(row_lcr). EXCEPTION WHEN SYS.

Summary Streams only looks complex It's conceptually simple and is implemented via a few procedures Questions? Thank you for coming .

Sign up to vote on this title
UsefulNot useful