__________________________________________ ____

EMPTORIS ETL ERROR and AUDIT LOG PROCESS USING INFORMATICA

__________________________________________ ____

Table of Contents
1 DOCUMENT PURPOSE..........................................................................................................3 2 ERROR LOG TABLE DETAILS ..........................................................................................4

2.1.1 PMERR_DATA...............................................................................................4 2.1.2 PMERR_MSG.................................................................................................6 2.1.3 PMERR_SESS.................................................................................................8 2.1.4 PMERR_TRANS..............................................................................................9

3 DATA ERROR LOG TABLE DETAILS .............................................................................10

3.1.1 ETL Data Error Table..................................................................................10

4 AUDITING DETAILS -...........................................................................................................11

4.1.1 Auditing Log Table.......................................................................................11

1 Document Purpose The Purpose of this document is to understand the Error Handling & Auditing process for EMPTORIS using Informatica. There are four a) b) c) d) types of errors : Source Error (Reader Error) Transformation Error Target Error (Load error / Writer Error) Data Error

Source Error (Reader Error) Examples: 1) Source Database not Available, 2) Table or view does not exist 3) SQL Override Query not correct Transformation Error Examples: 1) Look Up error- Having multiple Values 2) Records getting rejected for some reasons. 3) Data Type mismatch between 2 transformations. Load / Writer Error Examples: 1) Target Table does not exist 2) Data Type is different in Informatica and Target Data Error Examples: 1) Invalid Data 2) Erroneous Data 3) Null data

2 Error Log Table Details

Understanding the Error Log Tables
PMERR_DATA. Stores data and metadata about a transformation row error and
its corresponding source row.

PMERR_MSG. Stores metadata about an error and the error message. PMERR_SESS. Stores metadata about the session. PMERR_TRANS. Stores metadata about the source and transformation ports,
such as name and datatype, when a transformation error occurs.

2.1.1 PMERR_DATA
When the Integration Service encounters a row error, it inserts an entry into the PMERR_DATA table. This table stores data and metadata about a transformation row error and its corresponding source row. The following table describes the structure of the PMERR_DATA table: Column Name REPOSITORY_GID Description Unique identifier for the repository.

WORKFLOW_RUN_ID Unique identifier for the workflow. WORKLET_RUN_ID Unique identifier for the worklet. If a session is not part of a worklet, this value is 0. Unique identifier for the session.

SESS_INST_ID

TRANS_MAPPLET_IN Name of the mapplet where an error occurred. ST TRANS_NAME Name of the transformation where an error occurred. Name of the input group or output group where an error occurred. Defaults to either input or output if the transformation does not have a group. Specifies the partition number of the transformation where an error occurred.

TRANS_GROUP

TRANS_PART_INDEX

Column Name TRANS_ROW_ID

Description Specifies the row ID generated by the last active source. Delimited string containing all column data, including the column indicator. Column indicators are: D - valid N - null T - truncated B - binary U - data unavailable The fixed delimiter between column data and column indicator is colon ( : ). The delimiter between the columns is pipe ( | ). You can override the column delimiter in the error handling settings. The Integration Service converts all column data to text string in the error table. For binary data, the Integration Service uses only the column indicator. This value can span multiple rows. When the data exceeds 2000 bytes, the Integration Service creates a new row. The line number for each row error entry is stored in the LINE_NO column.

TRANS_ROW_DATA

SOURCE_ROW_ID

Value that the source qualifier assigns to each row it reads. If the Integration Service cannot identify the row, the value is -1. Row indicator that tells whether the row was marked for insert, update, delete, or reject. 0 - Insert 1 - Update 2 - Delete 3 - Reject

SOURCE_ROW_TYPE

SOURCE_ROW_DATA

Delimited string containing all column data, including the column indicator. Column indicators

Column Name are: D - valid O - overflow N - null T - truncated B - binary

Description

U - data unavailable The fixed delimiter between column data and column indicator is colon ( : ). The delimiter between the columns is pipe ( | ). You can override the column delimiter in the error handling settings. The Integration Service converts all column data to text string in the error table or error file. For binary data, the Integration Service uses only the column indicator. This value can span multiple rows. When the data exceeds 2000 bytes, the Integration Service creates a new row. The line number for each row error entry is stored in the LINE_NO column. LINE_NO Specifies the line number for each row error entry in SOURCE_ROW_DATA and TRANS_ROW_DATA that spans multiple rows.

2.1.2 PMERR_MSG
When the Integration Service encounters a row error, it inserts an entry into the PMERR_MSG table. This table stores metadata about the error and the error message. The following table describes the structure of the PMERR_MSG table: Column Name REPOSITORY_GID Description Unique identifier for the repository.

WORKFLOW_RUN_ID Unique identifier for the workflow. WORKLET_RUN_ID Unique identifier for the worklet. If a session is not

Column Name

Description part of a worklet, this value is 0.

SESS_INST_ID MAPPLET_INST_NAME

Unique identifier for the session. Mapplet to which the transformation belongs. If the transformation is not part of a mapplet, this value is n/a. Name of the transformation where an error occurred. Name of the input group or output group where an error occurred. Defaults to either input or output if the transformation does not have a group. Specifies the partition number of the transformation where an error occurred. Specifies the row ID generated by the last active source. Counter for the number of errors per row in each transformation group. If a session has multiple partitions, the Integration Service maintains this counter for each partition. For example, if a transformation generates three errors in partition 1 and two errors in partition 2, ERROR_SEQ_NUM generates the values 1, 2, and 3 for partition 1, and values 1 and 2 for partition 2.

TRANS_NAME

TRANS_GROUP

TRANS_PART_INDEX

TRANS_ROW_ID

ERROR_SEQ_NUM

ERROR_TIMESTAMP

Timestamp of the Integration Service when the error occurred. Coordinated Universal Time, called Greenwich Mean Time, of when an error occurred. Error code that the error generates. Error message, which can span multiple rows. When the data exceeds 2000 bytes, the Integration Service creates a new row. The line number for each row error entry is stored in the LINE_NO column.

ERROR_UTC_TIME

ERROR_CODE ERROR_MSG

Column Name ERROR_TYPE

Description Type of error that occurred. The Integration Service uses the following values: 1 - Reader error 2 - Writer error 3 - Transformation error

LINE_NO

Specifies the line number for each row error entry in ERROR_MSG that spans multiple rows.

2.1.3 PMERR_SESS
When you choose relational database error logging, the Integration Service inserts entries into the PMERR_SESS table. This table stores metadata about the session where an error occurred. The following table describes the structure of the PMERR_SESS table: Column Name REPOSITORY_GID Description Unique identifier for the repository.

WORKFLOW_RUN_ID Unique identifier for the workflow. WORKLET_RUN_ID Unique identifier for the worklet. If a session is not part of a worklet, this value is 0. Unique identifier for the session. Timestamp of the Integration Service when a session starts. Coordinated Universal Time, called Greenwich Mean Time, of when the session starts. Repository name where sessions are stored. Specifies the folder where the mapping and session are located. Specifies the workflow that runs the session being logged. Fully qualified session name that can span

SESS_INST_ID SESS_START_TIME

SESS_START_UTC_TIM E REPOSITORY_NAME FOLDER_NAME

WORKFLOW_NAME

TASK_INST_PATH

Column Name

Description multiple rows. The Integration Service creates a new line for the session name. The Integration Service also creates a new line for each worklet in the qualified session name. For example, you have a session named WL1.WL2.S1. Each component of the name appears on a new line: WL1 WL2 S1 The Integration Service writes the line number in the LINE_NO column.

MAPPING_NAME LINE_NO

Specifies the mapping that the session uses. Specifies the line number for each row error entry in TASK_INST_PATH that spans multiple rows.

2.1.4 PMERR_TRANS
When the Integration Service encounters a transformation error, it inserts an entry into the PMERR_TRANS table. This table stores metadata, such as the name and datatype of the source and transformation ports. The following table describes the structure of the PMERR_TRANS table: Column Name REPOSITORY_GID Description Unique identifier for the repository.

WORKFLOW_RUN_ID Unique identifier for the workflow. WORKLET_RUN_ID Unique identifier for the worklet. If a session is not part of a worklet, this value is 0. Unique identifier for the session.

SESS_INST_ID

TRANS_MAPPLET_INST Specifies the instance of a mapplet. TRANS_NAME Name of the transformation where an error occurred. Name of the input group or output group where an

TRANS_GROUP

Column Name

Description error occurred. Defaults to either input or output if the transformation does not have a group.

TRANS_ATTR

Lists the port names and datatypes of the input or output group where the error occurred. Port name and datatype pairs are separated by commas, for example: portname1:datatype, portname2:datatype. This value can span multiple rows. When the data exceeds 2000 bytes, the Integration Service creates a new row for the transformation attributes and writes the line number in the LINE_NO column.

SOURCE_MAPPLET_INS Name of the mapplet in which the source resides. T SOURCE_NAME Name of the source qualifier. n/a appears when a row error occurs downstream of an active source that is not a source qualifier or a non pass-through partition point with more than one partition. Lists the connected field(s) in the source qualifier where an error occurred. When an error occurs in multiple fields, each field name is entered on a new line. Writes the line number in the LINE_NO column. Specifies the line number for each row error entry in TRANS_ATTR and SOURCE_ATTR that spans multiple rows.

SOURCE_ATTR

LINE_NO

3 Data Error Log Table Details 3.1.1 ETL Data Error Table
Table Description

Error details at the attribute level captured during Informatica ETL execution.

Column Name
ERROR_TABLE_PK SESSION_NAME SOURCE_PRIMARY_KEY SOURCE_FIELD_VALUE SOURCE_TABLE_NAME SOURCE_FIELD_NAME TARGET_TABLE_NAME TARGET_FIELD_NAME MAPPING_NAME TARGET_FIELD_VALUE LOOKUP_TABLE_NAME ERROR_TYPE ERROR_DETAILS ETL_LOAD_DATE

Column Description
The Primary Key of the Table. The Session name Primary Key of the Actual Source Data Source Field Value. Source Table Name Source Field Name Target Table name Target Table Value Mapping name Target Field Value Look Up Table Name The Type of Error The Error Details occurred. Error date

4 Auditing Details 4.1.1 Auditing Log Table

Table Description

Status information captured for a particular informatica ETL execution. Column Name
Audit_Id WF_name Entity_Name Session_Name Mapping_Name Start_Dtm End_Dtm Status Created_By Created_Dtm Updated_By Updated_Dtm

Column Description
the unique sequential number for the Table Workflow name Table Name Session Name of the Audit. Mapping Name Start Date Time End Date Time Holds the Status (Success, Failure, etc). Date & Time Created. System User id of the user who updated the Table. System Date and Time when the Table was updated.

The Above Table will be populated during the Session Execution. Procedure 1- Will Insert a New Record when the Session Starts.

Procedure 2 Will Update, The Inserted Record for the Session. For Sequence :
create sequence audit_id Increment by 1 Start with 1 ;