This document describes about consolidated list of all mes

<lakshmikantha.gundala@cgi.c
Author om>
Scope CAE & SNCL
Created On 27-01-2020
Updated On 27-01-2020
onsolidated list of all messgages that we received during oncall support.

Reviewed by Balasubramanian, Chockalingam <

Review on
 Message Type
CPU Idle %
Error 30754889 SLI FTP And FTPS Transfer Program
WARNING: inbound connection timed out (ORA-3136)
TYPE TIME_MINS action USER_NAME programme
/cifs_mnt/primary/financial >= 90
ORA-00060: Deadlock detected. More info in file
/u01/app/oraoem/diag/rdbms/oemprd/oemprd/trace/oemprd_ora_6350.trc. OR

ORA-1652: unable to extend temp segment by 128 in tablespace TEMP1 ORA-1652: unable to extend temp

ORA-1653: unable to extend table SYS.AUD$ by 5314958 in tablespace SYSTEM ORA-1653: unable to exten
ORA-16957: SQL Analyze time limit interrupt ORA-16957: SQL Analyze time limit interrupt
ORA-12012: error on auto execute of job "SYSTEM"."DBA_KILL_USERS" ORA-00030: User session ID does n
exist. ORA-0651
ORA-12012: error on auto execute of job 11762 ORA-06502: PL/SQL: numeric or value error ORA-06512: a
"DBSNMP.BSLN_IN
ORA-00600: internal error code, arguments: [kgasra_2], [], [], [], [], [], [], [], [], [], [], []

ORA-609 : opiodr aborting process unknown ospid (22979_140444635862752) TNS-12547: TNS:lost contac
ORA-1100 signalled during: alter database mount...
ORA-3214 signalled during: alter database datafile '+DATA_PROD/ebsprd/datafile/apps_ts_media_10.dbf' re
22...

ORA-15082: ASM failed to communicate with client

ORA-02291: integrity constraint (FDM.FK_TLOGPROCESS_TPOVPERIOD) violated - parent key not found OR

31684:

ORA-01555 caused by SQL statement below (SQL ID: dsphq1qsnymv8, SCN: 0x0599.50652aaa):

ORA-01555 caused by SQL statement below (SQL ID: d809xy950587v, Query Duration=3523 sec, SCN:
0x059a.0b97e3d2):
/u01/scripts/plw/cgi_bkp_dpx.ksh 1>/u01/scripts/prod/crontab/log/cgi_bkp_dpx.log
2>/u01/scripts/prod/crontab/error/cgi_bkp_dpx.err
/u01/scripts/bd/cgi_ebs_chk_wfmailer_enhanced.ksh
>/u01/scripts/PROD1/crontab/log/cgi_ebs_chk_wfmailer_enhanced.log 2>/u01/sc
/u01/scripts/bd/cgi_db_chk_wfmailer.ksh >/u01/scripts/PROD2/crontab/log/cgi_db_chk_wfmailer.log
2>/u01/scripts/PROD2/crontab/error/cgi_db
/u01/scripts/apps/cgi_ebs_check_cm.ksh bounce >/u01/scripts/PROD2/crontab/log/cgi_ebs_check_cm.log
2>/u01/scripts/PROD2/crontab/er
/u01/scripts/apps/cgi_soa_clean_logs.ksh >/u01/scripts/SOAPROD/crontab/log/cgi_soa_clean_logs.log
2>/u01/scripts/SOAPROD/crontab/error/cgi_soa
/u01/planisware/PROD/PROD_satellite/bin/run_eac_ids_batch.sh
1>/u01/scripts/prod/crontab/log/run_eac_ids_batch.log 2>/u01/scripts/prod/cron
/u01/planisware/PROD/PROD_satellite/bin/run_budget_ids_batch.sh
1>/u01/scripts/prod/crontab/log/run_budget_ids_batch.log 2>/u01/scripts/pro
/u01/app/applmgr/EBSPRD/inst >= 90

/tmp >= 93
File system /u01 is 93% full. There is XXXXXXX k of available space.
File system /common is 93% full. There is XXXXXX k of available space.
File system /home is 94% full. There is XXXXXXX k of available space.
PAGER Error 30122055 Validate Application Accounting Definitions
PAGER Error 30060129 Report Set 01-NOV 06:00 01-NOV 19:27
PAGER Error 30476552 Report Set 05-DEC 23:00 06-DEC 02:56
PAGER Error 30463154 Report Set 04-DEC 23:00 05-DEC 02:57
PAGER Error 30444917 Report Set 03-DEC 23:00 04-DEC 04:57
PAGER Error 30431409 Report Set 02-DEC 23:00 03-DEC 03:27
PAGER Error 30423338 Report Set 01-DEC 23:00 02-DEC 03:53
PAGER Error 30589779 SLI Positive Pay - BMO - Candu
PAGER Error 30832155 SLI Positive Pay File (Create and send)
DAC Notification: 'Failed' status for the process 'XX_NEW_PLM_EXTRACTION: ETL Run - 2020-01-09
19:15:00.026 ETL Process Id : 26310256ETL Name : XX_NE
DAC Notification: 'Failed' status for the process 'SDE_ORA_APTransactionFact_Distributions_Full: ETL Run -
2019-11-03 11:00:00.026 ETL Process Id : 2
DAC Notification: 'Aborted' status for the process 'SDE_ORA_APTransactionFact_Distributions_Full: ETL Run
2019-11-03 11:00:00.026 ETL Process Id :
DAC Notification: 'Failed' status for the process 'CAE_FIN_PROC_PROJ Oracle 11.5.10 Production: ETL Run -
2019-11-03 20:30:00.308 ETL Process Id : 26
DAC Notification: 'Failed' status for the process 'XX_AUDIT_TRIAL_ASCP: ETL Run - 2019-11-11 08:00:00.0
ETL Process Id : 26224621ETL Name : XX_AUDI
DAC Notification: 'Aborted' status for the process 'SCM007 - Sanity Check: ETL Run - 2019-11-11 04:00:00.
ETL Process Id : 26224381ETL Name : SCM0
OPMN process down
CAE Apache down !!!
Some MWA Server are missing on caeerpapp02
Some MWA Server are missing on caeerpapp01
Form Server Down
Missing PLW PROD processes
MWA Dispatcher not found on caeerpapp01
MWA Dispatcher not found on caeerpapp02
WF mailer check:
DB_CHECK: Locks detected in PROD2
DB_CHECK: Locks detected in PROD1

Check ORA- Error in db alert log # Check ORA- Error in db alert log # Check ORA- Error in db alert log 9999
DB_CHECK: Database is Down in PROD1
DB_CHECK: Database is Down in PROD2
DB_CHECK: Database is Down in SOAPROD1
DB_CHECK: Locks detected in SOAPROD1
DB_CHECK: Database is Down in hypp2
DB_CHECK: Database is Down in hypp1
DB_CHECK: Locks detected in hypp1
Deadlock detected. More info in file /u01/app/oraoem/diag/rdbms/oemprd/oemprd/trace/oemprd_ora_2019
OR
Bouncing MWA Sun Jan 26 00:00:02 EST 2020 Shutdown output: ================
====================================== No more process, restarting MWA Star
Error 30422524 SLI Report 001 and PCAE Storage Process 01-DEC 01:10 01-DEC 03:08
Error 30475546 PRC: Generate Cross Charge Accounting Events 04-DEC 22:54 04-DEC 22:55
Below PRC Create Accounting programs are running for more than 3 hours. Please check the job if it is
processing or not, from backend
Disk Free Space Alert (92%) on caeplwapp01
Disk Free Space Alert (92%) on caeplwapp06
Disk Free Space Alert (92%) on caeplwapp03
Disk Free Space Alert (92%) on caeplwapp04
Disk Free Space Alert (92%) on caeplwapp02
Disk Free Space Alert (92%) on caeplwapp05
TNS-12537: TNS:connection closed TNS-00507: Connection closed TNS-12537: TNS:connection closed TNS
00507: Connection
Tablespace Alert
ASM DiskGroup Alert
Alert Content Managed Server adf_server1 is not working ... Please check!!! Managed Server soa_server1 is
working properly! the current state is FORCE_S
Alert Content Managed Server adf_server2 is not working ... Please check!!! Managed Server adf_server2 is
working properly! the current state is UNKNOW
CTABLESPACE_NAME USED_PERCENT
 Action tobe taken Comments
No Action required
Arun SNC
No Action required
Locks
gowtham

gowtham

tilak table space

tilak table space

No Action required

No Action required

No Action required
No Action required

No Action required
No Action required

No Action required
if we received more times then
gowtham action required

No Action required

No Action required

No Action required

Space reached threshold Go through respective solution

value document
 Cause

You tried to execute a statement, but your session was deadlocked because another session had the same r
locked. The statement(s) that you tried to execute have been rolled back.
a failure in allocating the extent for the temp segment in the tablespace. The temp segment refers to a temp
tablespace used internally by the Oracle database for the function of certain operations

The Oracle Database Administrator (DBA) USERS table space quota is full.
This is an internal error code used indicate that SQL analyze has reached its time limit.

An error was caught while doing an automatic execution of a job

An error was caught while doing an automatic execution of a job

The Oracle Database uses ORA-00600 when it encounters an unexpected internal condition.
 This happens when the 60-second default timeout passes or is triggered when the database session is abor
the operating system prompt.
restore or recovery error

File Size specified for add/resize datafile/tempfile does not allow for the minimum required of one allocation
There was a failure or time out when ASM tried to communicate with a connected RDBMS or Oracle ASM Dyn
Volume Manager (Oracle ADVM) client
This means that you attempted to execute a reference to a certain table using a primary key. However, in th
of doing so, the columns that you specified failed to match the primary key. The error can also be triggered
referencing a primary key that does not exist for the table in question.
an happen when a query is unable to access enough undo to build a copy of the data at the time the query s
The error is a legitimate issue with getting to an undo block that has been overwritten due to the
undo retention period has passed
an happen when a query is unable to access enough undo to build a copy of the data at the time the query s
The error is a legitimate issue with getting to an undo block that has been overwritten due to the
undo retention period has passed