<Insert Picture Here>

Oracle PTS GoldenGate Workshop for partners

GoldenGate is Strategic to Oracle
• Best-in-class leader in real-time data solutions • Complements existing Oracle products • Continuous Availability for heterogeneous systems • Real-time data access for Reporting • Real-time data for BI, EPM, Exadata • Zero-downtime migrations / upgrades to Oracle Database and Applications • Over 400 customers with 4,000+ implementations across Fortune 500 companies: Financial Services, Communications, Healthcare, Public Sector, Retail & Utilities industries • Top 3 of 5 largest commercial banks • Top 3 of 3 busiest ATM networks • Top 7 of 10 financial data services companies • Top 4 of 5 telecommunications providers • Top 3 of 5 largest food & drug stores


Industry Proof Points

“Oracle will become more competitive in data integration tools and may enhance its high-availability products by buying GoldenGate.” Ted Friedman, Gartner

“This is a critical requirement to enable high volume data extraction to support real-time data warehousing and BI, as well as master data management and other operational use cases.” Rob Karel, Forrester

“It's quite a catch for Oracle in that it's a significant addition to its data warehousing and BI portfolio.” James Kobielus, Forrester

and any ODBC compatible databases ETL products JMS message queues or topics O/S and Platforms Windows 2000. 2003. HP Neoview. XP Linux Sun Solaris HP NonStop HP-UX HP TRU64 HP OpenVMS IBM AIX IBM z/OS .Oracle GoldenGate: Databases and Platforms Databases Capture: Oracle IBM DB2 Microsoft SQL Server Sybase ASE Ingres Teradata Enscribe SQL/MP SQL/MX Delivery: All listed above MySQL. Netezza.

Local Apply Efficient IO and Bandwidth Usage Bidirectional Group Transactions Bulk Operations Compression One-to-Many. SQL Formats Queue Interface Stored Procedures User Exits ETL Integration Java/JMS Integration . Resolution Dynamic Rollback Incremental TDM Initial Data Load GUI-based Monitoring and Configuration Proactive Alerts Encryption Real-Time Deferred or Batch Event Markers Integration Heterogeneous Data Sources Mapping Transformation Enrichment Decoupled Architecture Table.Oracle GoldenGate Advantages Movement Speed – Subsecond Latency Volume – Thousands of TPS Log-based Capture Native. ASCII. Many-to-One Cascade Management Transaction Integrity Transparent Capture Guaranteed Delivery Conflict Detection. Column Filtering XML. Row.

Oracle Products for Data Movement Comparing How They Work Disaster Recovery & Data Protection Redo in Memory Buffer Direct Memory Access Active Data Guard Direct Write to Logs HETEROGENEOUS Real Time Data Integration & High Availability Read On-Disk Logs GoldenGate Fast SQL Data Integration for Data Warehouse & SOA SQL Query Data Integrator Set-based. Complex SQL 6 .

Customers Demand Continuous Real Time Data While Lowering IT Costs and Gaining Operational Efficiencies Real-Time Business Operations Continuous Availability of Critical Applications and Data • Continuous Uptime in event of disaster • No downtime during planned outage • Load balancing & Data distribution Real-Time Information Access for Intelligence & Decisioning • Instantly available Data for Reporting • Access to Key Information for Analysis • Across Heterogeneous Systems Lower IT Spend & Improve Efficiencies • Choose best system for need • Modernize Legacy Systems • Offload for reporting .

Real-time Business Delivers Value & ROI 90% reduced recovery time for mission-critical transactional application Bank of America World’s Busiest ATM Network Remains Continuously Available VocaLINK Processing > 1 Billion Transactions each year. and zero outages Swedbank 100% paperless environment drives efficiency and improves patient care across hospital Montefiore 100% availability during phased upgrade for 24x7 critical CRM application HSN 6X productivity gains in manufacturing plant with real-time data AMD .

Resilient against interruptions and failures Reliable 9 9 .Oracle GoldenGate Technology Differentiators Oracle GoldenGate provides low-impact capture. Low Impact. Sub-second latency Extensible & Flexible Open. transformation.Heterogeneous sources & Targets Maintains Transactional Integrity . modular architecture . routing. and delivery of database transactions across heterogeneous environments in real-time Key Differentiators: Performance Non-intrusive.

RealTime Change Data Capture • Standardize on Single Technology for Multiple Needs • Deploy for Continuous Availability and Real-time Data Access for Reporting / BI Operational Reporting OGG Reporting Database ETL ODS ETL EDW Heterogeneous Source Systems Real-time BI EDW Query Offloading • Highly Flexible • Fast Deployments • Lower TCO & Improved ROI Data Distribution .Oracle GoldenGate Use Cases Enterprise-wide Solution for Real Time Data Needs Disaster Recovery. Data Protection Standby (Open & Active) Zero Downtime Migration and Upgrades Log Based.

current data for reporting demands • Unlimited scalability as user demands and data volumes grow Real-Time and Lower Total Cost of Ownership .Data Integration: Live Reporting When you need: • Solution for reporting’s performance impact on production application • Real-time.

legacy or big iron) on Oracle with RAC on Linux. • Sync replica in real-time and route all read-only traffic OGG Non-Oracle or Legacy Platform Oracle Platform Benefits • Reduce/Eliminate query load on transactional system • Directly reduce cost of legacy. non-oracle Move query processing to standard.g.Query Offloading Infrastructure Cost Reduction Query Offloading Cost Reduction OLTP Transactions Read Only Users Solution • Oracle GoldenGate • Create a real-time replica of existing expensive operational system (e. lower cost DB & Hardware infrastructure with no disruption to legacy critical transaction processing systems 12 . mainframe.

Real-Time Data Integration Enables Real-time BI Real Time Data Integration Solution • Oracle GoldenGate with Oracle Data Integrator Real-time Integration + Bulk Data Transformation ODS ODI • Populate an operational data store for operational BI with real-time data • Run ELT from the ODS to load data warehouse with transformed data OGG Enterprise Data Warehouse Oracle Database. Oracle Exadata Data Mart Benefit • Improves business insight • Combines intelligence on current business status with historical context • Eliminates load on source systems • Reduces latencies and batch windows Better business insight with the most current data with no impact to your critical systems 13 .

heterogeneity. topologies) – Easier to implement and maintain .Data Integration: Real-Time Data Warehousing When you need: • Real-time data for Business Intelligence • Eliminate batch window dependency • Data integration that is: – Lower impact & overhead – More flexible (transformations and user exits.

Continuous Availability Zero Down-time Migration and Upgrade Solution Continuous Availability Previous Version New Version • Oracle GoldenGate • Migrate from any heterogeneous database to Oracle11g with minimal downtime • Migrate hardware or OS platforms with no downtime • Upgrade Siebel CRM with no downtime • Allow active users to move to new system gradually while both systems are live OGG Benefit Old DB Version New DB Version • Standardize your data center and modernize platforms • Zero down-time for mission critical applications • Safely migrate users from one system to the next. while keeping both systems running. • Lower risk by enabling failback Upgrade critical applications or migrate your database without any down time or impacting critical business processes 15 .

handle large & growing data volumes • Access to consistent data across geographically dispersed data centers Keep up with the demands of a 24x7 global business.Continuous Data Availability Data Distribution / Synchronization Solution Oracle 9i Oracle 11g Continuous Availability OGG • Oracle GoldenGate • Replicate data for distributed applications to drive global operations and improve operational efficiency • Distribute data in real-time across Oracle and Non-Oracle environments • Active-Active configuration keeps multiple data centers in sync Oracle 10g SQL Server Benefit • High Availability for mission critical data • Ability to scale. continuously 16 . While all business units access the same. up-to-data information at any time.

Oracle® GoldenGate Architecture .

Trail files: Stages and queues data for routing. Capture Source Trail Network (TCP/IP) Target Trail Delivery Source Database(s) Target Delivery Trail Bi-directional Source Trail Capture Target Database(s) . transforming the data as required. encrypted for routing to targets.How Oracle GoldenGate Works: Modular “Building Capture: Blocks” Committed changes are captured (and can be filtered) as they occur by reading the transaction logs. Delivery: Applies data with transaction integrity. Route: Data is compressed.

Oracle GoldenGate Data Capture and Delivery Oracle GoldenGate Transactional Data Management: • Primarily used for change data capture and delivery from database transaction logs • Can optionally be used for initial load directly from database tables – Especially useful for synchronizing heterogeneous databases – Database-specific methods may be preferable for homogeneous configurations .

Change Data Capture & Delivery Source Database Network (TCP/IP) Extract Transaction Log Server Collector Replicat Target Database Trail Manager Manager .

Change Data Capture & Delivery using a Data Pump Source Database Network (TCP/IP) Extract Transactio n Log Local Trail Data Pump Server Collector Replicat Target Database Remote Trail Manager Manager .

or GGSCI processes – Error and lag reporting – GoldenGate trail management • Parameter file – mgr.prm file in GGS .Manager .Overview • Performs system management and monitoring tasks – Starting GoldenGate processes – Starting dynamic Server Collector.log file ./dirprm directory • Event information written to ggserr. Replicat.

Change Capture .Extract Overview Extract can be configured to: • Capture changed data from database logs • Distribute data from local trails to remote systems (data pump) • Capture data directly from source tables for initial data load .

Change Capture . add a local trail and a data pump Extract (reading from the local trail) with an associated parameter file Add a remote trail Start the Extract(s) • • • .Tasks On the source system: • Add a primary Extract (reading from source transaction logs) with an associated parameter file Optionally.

Overview • Data is stored in a local trail on the source system • A second Extract. the data pump: – Reads this trail and sends it to one or more targets – Manipulates the data or passes it through without change • Reasons for using – – – – A safeguard against network and target failures To break complex data filtering and transformation into phases To consolidate data from many sources To synchronize one source with multiple targets .Data Pumps .

Data Pumps – One to Many Trails Trail Primary Extract Trail Data Pump Trail Trail .

Data Pumps – One to Many Target Systems Data Pump 1 Trail Primary Extract Trail Data Pump 2 Trail Data Pump 3 Trail .

row.Replicat Overview Replicat can: • Read data out of GoldenGate trails • Perform data filtering – Table.Change Delivery . operation • Perform data transformation • Perform database operations just as your application performed them .

Change Delivery .Tasks On the target system: – Create a checkpoint table in the target database (best practice) • DBLOGIN • ADD CHECKPOINTTABLE Create a parameter file for Replicat • EDIT PARAMS Add your initial Replicat checkpoint into GoldenGate trails • ADD REPLICAT – Start the Replicat process • START REPLICAT – – .

Extract Trails and Files . with variable length records – I/O performed using large block writes • Extract writes checkpoints for trails during change capture: – Guarantees no data lost during restart – Multiple Replicat processes may process the same trail • Extract does not write checkpoints for files 30 .Overview • Extract writes data to any of: – – – – Remote trail (RMTTRAIL) Remote file (RMTFILE) Local trail (EXTTRAIL) Local file (EXTFILE) • Extract trails and files are unstructured.

Extract Trails and Files . then distribute over IP with a data pump to remote trails To multiple trails • For distribution multiple systems/disk storage devices • For parallel processing by downstream processes • Trails and files can be transported online using TCP/IP or sent in batch using any file transfer method 31 .Distribution – Extract can write: To local trails.

in the order they were applied • By default.Extract Trails and Files .Contents – Each record in the trail contains an operation that has been committed in the source database • Transactions are output in commit order – Operations in a transaction are grouped together. only the primary key and changed columns are recorded – Flags indicate the first and last records in each transaction .

Cleanup • Trail files can be purged once consumed – Temporary storage requirement is small if processes keep pace – Configure Manager to purge used trail data (best practice) .Extract Trails and Files .

Initial Load Source Database Tables Extract Network (TCP/IP) Target Database Replicat Or DB Bulk Load Utility Server Collector Files Manager Manager GoldenGate initial load methods: Direct Load (Extract sends data directly to Replicat to apply using SQL) Direct Bulk Load (Replicat uses Oracle SQL*Loader API) File to Replicat (Extract writes to a file that Replicat applies using SQL) File to database utility (Extract writes to a file formatted for a DB bulk load utility) .

Initial Load: File to Replicat Manager Source Database Extract Files Replicat Target Database ADD EXTRACT <name> Extract parameters: SOURCEISTABLE RMTTRAIL<name> .

Initial Load: File to Database Utility Manager Source Database Extract File SSIS File SQL* Loader Target Database File BCP ADD EXTRACT <name> Extract parameters: SOURCEISTABLE RMTFILE <name> FORMATASCII BCP or SQLLOADER .

GROUP <name> ADD REPLICAT <name>. SOURCEISTABLE Extract parameters: RMTTASK REPLICAT. SPECIALRUN .Initial Load: Direct Load Manager Source Database Extract Replicat Manager Target Database ADD EXTRACT <name>.

Initial Load: Direct Bulk Load (to Oracle) Manager Source Database Source Database Extract Manager Oracle Target Replicat SQL*Loader API ADD EXTRACT <name>. GROUP <name> ADD REPLICAT <name>. SOURCEISTABLE Extract parameters: RMTTASK REPLICAT. SPECIALRUN Replicat parameters: BULKLOAD .

. • Initial load is always a special run (batch run).Online versus Batch • Change data capture & delivery can be run either continuously (online) or as a special run (batch run) to capture changes for a specific period of time.

TX 2 Commit. TX 3 Current Write Position Current Read Position Source Database Capture Commit Ordered Source Trail . TX 3 Delete. TX 3 Insert. TX 1 Insert. TX 3 Commit. TX 2 Update. TX 2 Begin. TX 3 Insert. Pump.Checkpointing Capture. TX 4 Begin. TX 2 Begin. TX 1 Insert. TX 1 Begin. and Delivery save positions to a checkpoint file so they can recover in case of failure Begin. TX 2 Start of Oldest Open (Uncommitted) Transaction Capture Checkpoint Commit. TX 4 Commit. TX 3 Begin. TX 2 Insert.

TX 2 Start of Oldest Open (Uncommitted) Transaction Pump Checkpoint Begin. TX 4 Commit. TX 3 Delete. TX 4 Begin. TX 2 Insert. TX 3 Current Read Position Current Write Position Current Write Position Current Read Position Commit. TX 2 Begin. Pump. and Delivery save positions to a checkpoint file so they can recover in case of failure Begin. TX 1 Begin. TX 2 Insert. TX 1 Insert. TX 2 Commit. TX 3 Insert.Checkpointing Capture. TX 2 Capture Checkpoint Begin. TX 3 Insert. TX 1 Insert. TX 2 Commit. TX 3 Source Database Capture Commit Ordered Source Trail Pump Commit Ordered Target Trail . TX 2 Commit. TX 2 Update. TX 3 Begin.

TX 1 Begin. TX 2 Insert. TX 3 Source Database Capture Commit Ordered Source Trail Pump Commit Ordered Target Trail Delivery Target Database . TX 2 Commit. TX 2 Commit. TX 2 Commit.Checkpointing Capture. TX 3 Delete. TX 3 Insert. TX 3 Insert. TX 2 Update. TX 2 Begin. TX 3 Begin. TX 3 Current Read Position Current Write Position Current Read Position Current Write Position Current Read Position Commit. and Delivery save positions to a checkpoint file so they can recover in case of failure Begin. Pump. TX 4 Commit. TX 4 Begin. TX 2 Insert. TX 2 Start of Oldest Open (Uncommitted) Transaction Pump Checkpoint Delivery Checkpoint Begin. TX 2 Capture Checkpoint Begin. TX 1 Insert. TX 1 Insert.

2010 .Configuring Oracle® GoldenGate September 8.

Prepare the Environment 2. Change Delivery . Initial Load 4. Change Capture 3.Configuring Oracle GoldenGate Oracle GoldenGate can be deployed quickly and easily in four steps: 1.

Configuring Oracle GoldenGate 1. Change Delivery . Change Capture Data Pump Remote Trail Replicat 4. Prepare the Environment Source Database 3. Initial Load (various methods) Target Database Transaction Log Extract Local Trail 2.

check process status. • GGSCI commands also add trails. . • Processes are added and started using the GoldenGate Software Command Interface (GGSCI) with the group name. etc. Process Groups and Commands • GoldenGate processes are configured by ASCII parameter files.Parameters. • A process group consists of: – An Extract or Replicat process – Associated parameter file – Associated checkpoint file – Any other files associated with that process Each process group on a system must have a unique group name.

2010 .Configuring Oracle GoldenGate: Step 1. Prepare the Environment September 8.

Step 1. Prepare the Environment
1. Prepare the Environment Source Database 3. Initial Load (various methods)

Target Database

Transaction Log Extract Local Trail 2. Change Capture Data Pump Remote Trail Replicat 4. Change Delivery

Step 1. Prepare the Environment
Set up each system: • • Install Oracle GoldenGate software on source and target Configure and start GoldenGate Manager on source and target If heterogeneous source/target, generate source definitions and copy to target Prepare the database. For example:
– – Ensure database access by GoldenGate Enable transaction logging

Prepare the Environment:

Install Oracle GoldenGate Software

com Prepare Environment: Installation – Access the Media Pack • Identify the proper release of GoldenGate for your source and target environments – Database and version – Operating system and version .oracle.• Access the product media pack (software and documentation) at edelivery.

Linux or z/OS • • • • Download .gz file to /ggs gzip –d {filename}.Prepare Environment: Installation .tar GGSCI> CREATE SUBDIRS .UNIX.gz tar -xvof {filename}.tar.

Prepare Environment: Installation .Windows • • • Download .zip file to C:\GGS Unzip .zip file into C:\GGS folder Configure a Windows Service Name for Manager process in a GLOBALS parameter file (required only if multiple Managers on the server) C:\GGS> INSTALL ADDSERVICE ADDEVENTS GGSCI> CREATE SUBDIRS • • .

Prepare Environment: Installation – Windows INSTALL Program • On Windows. an INSTALL program performs the following functions: – Installs GoldenGate event messages into the system registry – Installs the Manager as a Windows service • Syntax: INSTALL <item> [<item> …] • Example: C:\GGS> INSTALL ADDEVENTS ADDSERVICE Note: The uninstall command is: INSTALL DELETESERVICE DELETEEVENTS .

you can specify the name – Create a GLOBALS parameter file for each Manager – Specify the one-word name of the service using the MGRSERVNAME <name> parameter • INSTALL ADDSERVICE – Reads the GLOBALS MGRSERVNAME for the service name – If no GLOBALS setting.Prepare Environment: Installation . uses default service name GGSMGR . or – GoldenGate with a Veridata C Agent (which uses a Manager) • Each Manager service must be assigned a unique name • Before installing the service.Multiple Manager Services • GoldenGate supports running multiple Manager services on Windows – For two or more GoldenGate instances.

GoldenGate Directories Contents GoldenGate checkpoint files GoldenGate trail and extract files Data definitions produced by DEFGEN and used to translate heterogeneous data Process status files Parameter files Process report files SQL scripts Temporary storage for transactions that exceed allocated memory .Directory dirchk dirdat dirdef dirpcs dirprm dirrpt dirsql dirtmp Prepare Environment: Installation .

GGSCI Examples • Start a Manager process GGSCI> START MGR • Add an Extract group GGSCI> ADD EXTRACT myext. EXTRACT myext • Start an Extract group GGSCI> START EXTRACT myext . TRANLOG. BEGIN NOW • Add a local trail GGSCI> ADD EXTTRAIL /ggs/dirdat/rt.

Editing Parameter Files • Edit parameter files to configure GoldenGate processes • The GLOBALS parameter file is identified by its file path GGSCI> EDIT PARAMS ./GLOBALS • Manager and utility parameter files are identified by keywords GGSCI> EDIT PARAMS MGR GGSCI> EDIT PARAMS DEFGEN • Extract and Replicat parameter files are identified by the process group name GGSCI> EDIT PARAMS <group name> .

Extract. Server Collector.prm – Most apply to all tables processed but some can be specified at the table level . Utilities) – Set when the process starts – Override GLOBALS settings – Reside by default in the dirprm directory in files named <processname>. Replicat.GLOBALS versus Process Parameters • GLOBALS parameters apply to all processes – Set when Manager starts – Reside in <GoldenGate install directory>/GLOBALS • Process parameters apply to a specific process (Manager.

Data Transformation • GoldenGate provides the capability to transform columns by using a set of built-in functions • Transformation functions can be applied either for Extract or Replicat • If you require more. you also have the ability to call your own logic through user exits .Functions .

Overview • Using column conversion functions.Functions . strings. you can: – – – – – Perform string and number conversion Extract portions of strings or concatenate columns Compare strings or numbers Perform a variety of date mappings Use single or nested IF statements to evaluate numbers. and other column values to determine the appropriate value and format for target columns • Functions are identified with the @ prefix .

Prepare Environment: Source Definitions .Overview • The problem – Understanding source and target layouts across disparate systems and databases • The solution – the DEFGEN utility program – – DEFGEN produces a file containing layout definitions of the source files and tables This source definition file is used to interpret layouts for data stored in GoldenGate trails • At start up Replicat reads the definition file specified with the SOURCEDEFS parameter • Server Collector uses the –d argument to specify which definition file to read at startup • Can also capture target definitions on target system and copy to source system for Extract to use .

Sign up to vote on this title
UsefulNot useful