Integrated Replicat Abends As Trail Files Were Missing - Page On-Call DBA

5/9/23, 10:26 AM Integrated Replicat abends as trail files were missing | Page On-Call DBA
Page On-Call DBA
anywhere…anytime
Integrated Replicat abends as trail files

were missing
Here is scenario where trail files were missing on target when Integrated Replicat was processing so
recovery involved identifying last source CSN when replicat was hung or abended while processing due
to trail missing on target and using lodump lets dig extract trail to find the corresponding transaction in
source OGG trails and then reposition pump to write to remote trail.
Checking the status of Replicat which was hung as remote trail being read was
missing.
GGSCI > info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
JAGENT RUNNING
REPLICAT RUNNING REP1 00:15:01 00:01:38
REPLICAT RUNNING REP2 00:00:03 00:00:02
GGSCI > send rep1 status
Sending STATUS request to REPLICAT rep1 ...

Current status: At EOF
Sequence #: 24
RBA: 1023999809
9627 records in current transaction
https://raviarra.wordpress.com/2015/09/04/integrated-replicat-abends-as-trail-files-were-missing/ 1/13
Checking if the remote trails exists on target server.
GGSCI > sh ls –ltr /ggapp_src01/src01_dbfs01/1212/dirdat/rr000025
ls: –ltr: No such file or directory

ls: /ggapp_src01/src01_dbfs01/1212/dirdat/rr000025: No such file or directory
NOTE: Since remote trails from 25 are missing on target lets re-point the Pump
based on the CSN value.
Stop Replicat process if it’s not already abended.
GGSCI > stop rep1
NOTE: Use forcestop if stop command hangs.
Identifying LOG_CSN until which replicat Replicat has processed the transactions.
SQL > select server_name, applied_low_position, applied_high_position, logbsn from

DBA_GG_INBOUND_PROGRESS;
SERVER_NAME APPLIED_LOW_POSITION APPLIED_HIGH_POSITIO LOGBSN

------------------------------ -------------------- -------------------- ----------
----------
OGG$REP2 8647072103546 8647072106473
OGG$REP1 8542700698930 8542700698930
Note: We use applied_low_position SCN value ato search local trail that has this
SCN information.
Stopping Pump process if not already stopped.
GGSCI > info all
GGSCI > stop pmp1
GGSCI > alter pmp1 etrollover
Using LOGDUMP utility find local trail containing the LOG_CSN value.The below
logdump process involves locating the replicat log_csn from the source local trail.
GGSTOKEN DETAIL will display log_csn info and using FILTER LOGCSN to search the
record with this CSN value in trail file.
Logdump 1 >open ./dirdat/ei000020

Current LogTrail is /opt/oracle/app/gg_home/dirdat/ei000020
Logdump 2 >ggstoken detail
Logdump 3 >filter logcsn 8542700698930
Logdump 4 >pos 0
Reading forward from RBA 0
Logdump 5 >n
Scanned 10000 records, RBA 2150766, 2015/08/30 16:48:52.000.000






2015/08/30 16:53:11.000.000 Insert Len 119 RBA 676253784

Name: RARRA.TEST_TABLE1
After Image: Partition 4 G b
0000 000b 0000 0007 3133 3130 3030 3100 0100 0b00 | ........1310001.....
0000 0731 3331 3030 3031 0002 000f 0000 000b 6475 | ...1310001........du
705f 7465 7374 5f30 3100 0300 1f00 0032 3031 352d | p_test_01......2015-
3036 2d30 393a 3136 3a35 333a 3130 2e30 3030 3030 | 08-30:16:53:10.00000
3030 3030 0004 001f 0000 3230 3135 2d30 362d 3039 | 0000......2015-08-30
3a31 363a 3533 3a31 302e 3030 3030 3030 3030 30 | :16:53:10.000000000
GGS tokens:
TokenID x52 'R' ORAROWID Info x00 Length 20
4141 416d 544e 4141 4541 4141 436e 7941 4276 0001 | AAAmTNAAEAAACnyABv..
TokenID x4c 'L' LOGCSN Info x00 Length 13
3835 3432 3730 3036 3938 3933 30 | 8542700698930
TokenID x36 '6' TRANID Info x00 Length 10
372e 382e 3130 3831 3138 | 7.8.108118
TokenID x69 'i' ORATHREADID Info x01 Length 2
0001 | ..
Repositioning PUMP to start from localtrail where the LOG_CSN exists.
GGSCI > alter pmp1 extseqno 20 extrba 0

EXTRACT altered.
GGSCI > info pmp1
EXTRACT pmp1 Initialized 2015-08-30 18:44 Status STOPPED

Checkpoint Lag 00:00:00 (updated 00:00:03 ago)
Log Read Checkpoint File /opt/oracle/app/gg_home/dirdat/ei000020
First Record RBA 0
NOTE: We can start from RBA 0 as Integarted Replicat will automatically filter out
transactions already done so there is no need to locate the RBA for the SCN record.
GGSCI > start pmp1
Sending START request to MANAGER ...

EXTRACT pmp1 starting
GGSCI > send pmp1 status
Sending STATUS request to EXTRACT pmp1 ...
EXTRACT pmp1 (PID 27610)

Current status: Recovery complete: Processing data
Current read position:

Sequence #: 20
RBA: 863509253
Timestamp: 2015-08-30 16:54:15.000000
Extract Trail: /opt/oracle/app/gg_home/dirdat/ei
Current write position:

Sequence #: 26
RBA: 513750786
Timestamp: 2015-08-30 18:46:50.147940
Extract Trail: /ggapp_src01/src01_dbfs01/1212/dirdat/rr
Repositioning Replicat as ETROLLOVER was performed.
GGSCI > alter rep1 extseqno 26 extrba 0

REPLICAT altered.
GGSCI > info rep1
REPLICAT rep1 Initialized 2015-08-30 11:47 Status STOPPED

INTEGRATED
Checkpoint Lag 00:00:00 (updated 00:00:04 ago)
Log Read Checkpoint File /ggapp_src01/src01_dbfs01/1212/dirdat/rr000026
First Record RBA 0
GGSCI > start replicat rep1 , aftercsn 8542700698930
NOTE: This is the CSN value based on which on which we filtered local trail.
Checking if all the trail files are applied without any issues.
GGSCI > send rep1 status
Sending STATUS request to REPLICAT rep1 ...

Current status: At EOF
Sequence #: 17
RBA: 983223400
0 records in current transaction
GGSCI > send rep1 getlag
Sending GETLAG request to REPLICAT rep1 ...

Last record lag 11,223 seconds.
At EOF, no more records to process.
Running stats to verify if any records are discarded.Unique constarint (-0001)

errors can be ignored or HANDLECOLLISIONS can to used(assuming primary key exists)
to suppress these.
GGSCI > stats rep1 totalsonly *.*
From the above case we have recovered the trail files until the point where replicat processed last and
once recovered these were applied without compromising data integrity.
Posted in Uncategorized on September 4, 2015 by Ravi Arra. Leave a comment
BLOG AT WORDPRESS.COM.

Integrated Replicat Abends As Trail Files Were Missing - Page On-Call DBA

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Integrated Replicat Abends As Trail Files Were Missing - Page On-Call DBA

Uploaded by

Copyright:

Available Formats

5/9/23, 10:26 AM Integrated Replicat abends as trail files were missing | Page On-Call DBA

Page On-Call DBA

Integrated Replicat abends as trail files

GGSCI > info all

Program Status Group Lag at Chkpt Time Since Chkpt

GGSCI > send rep1 status

Sending STATUS request to REPLICAT rep1 ...

Checking if the remote trails exists on target server.

GGSCI > sh ls –ltr /ggapp_src01/src01_dbfs01/1212/dirdat/rr000025

ls: –ltr: No such file or directory

Stop Replicat process if it’s not already abended.

GGSCI > stop rep1

NOTE: Use forcestop if stop command hangs.

SQL > select server_name, applied_low_position, applied_high_position, logbsn from

SERVER_NAME APPLIED_LOW_POSITION APPLIED_HIGH_POSITIO LOGBSN

Stopping Pump process if not already stopped.

GGSCI > info all

GGSCI > stop pmp1

GGSCI > alter pmp1 etrollover

Logdump 1 >open ./dirdat/ei000020

Scanned 350000 records, RBA 75251034, 2015/08/30 16:49:17.000.000

Scanned 840000 records, RBA 180601382, 2015/08/30 16:49:49.000.000

Scanned 1330000 records, RBA 285951769, 2015/08/30 16:50:26.000.000

Scanned 1820000 records, RBA 391302151, 2015/08/30 16:51:05.000.000

Scanned 2310000 records, RBA 495503072, 2015/08/30 16:52:09.000.000

Scanned 2800000 records, RBA 599874948, 2015/08/30 16:52:45.000.000

2015/08/30 16:53:11.000.000 Insert Len 119 RBA 676253784

Repositioning PUMP to start from localtrail where the LOG_CSN exists.

GGSCI > alter pmp1 extseqno 20 extrba 0

GGSCI > info pmp1

EXTRACT pmp1 Initialized 2015-08-30 18:44 Status STOPPED

GGSCI > start pmp1

Sending START request to MANAGER ...

GGSCI > send pmp1 status

Sending STATUS request to EXTRACT pmp1 ...

EXTRACT pmp1 (PID 27610)

Current read position:

Current write position:

Repositioning Replicat as ETROLLOVER was performed.

GGSCI > alter rep1 extseqno 26 extrba 0

GGSCI > info rep1

REPLICAT rep1 Initialized 2015-08-30 11:47 Status STOPPED

GGSCI > start replicat rep1 , aftercsn 8542700698930

GGSCI > send rep1 status

Sending STATUS request to REPLICAT rep1 ...

GGSCI > send rep1 getlag

Sending GETLAG request to REPLICAT rep1 ...

Running stats to verify if any records are discarded.Unique constarint (-0001)

GGSCI > stats rep1 totalsonly *.*

You might also like

GGSCI > stats rep1 totalsonly .