INFORMATICA

Monday, 14 November 2011

Process Control / Audit of Workflows in Informatica

1. Process Control – Definition
Process control or Auditing of a workflow in an Informatica is capturing the job information like
start time, end time, read count, insert count, update count and delete count. This information
is captured and written into table as the workflow executes
2. Structure of Process Control/Audit table
The table structure of process control table is given below,
Process Control structure

A unique number used to identify a

PROCESS_RUN_ID Number(p,s) 11 specific process run.
The name of the process (this column
will be populated with the names of
PROCESS_NME Varchar2 120 the Informatica mappings.)
The date/time when the process
START_TMST Date 19 started.
The date/time when the process
END_TMST Date 19 ended.
The number of rows read by the
ROW_READ_CNT Number(p,s) 16 process.
The number of rows inserted by the
ROW_INSERT_CNT Number(p,s) 16 process.
The number of rows updated by the
ROW_UPDATE_CNT Number(p,s) 16 process.
The number of rows deleted by the
ROW_DELETE_CNT Number(p,s) 16 process
The number of rows rejected by the
ROW_REJECT_CNT Number(p,s) 16 process.
The etl user identifier associated with
USER_ID Varchar2 32 the process.

3. Mapping Logic and Build Steps

The process control flow has two data flows, one is an insert flow and the other is an update
flow. The insert flow runs before the main mapping and update flows runs after the main
mapping, this option is chosen in “Target Load Plan”. The source for both the flows could be a
dummy source which will return one record as output, for example select ‘process’ from dual or
select count(1) from above Table. The following list of mapping variable is to be created,

Mapping Parameter and variables

$$PROCESS_ID
$$PROCESS_NAME
$$INSERT_COUNT
$$UPDATE_COUNT
$$DELETE_COUNT
$$REJECT_COUNT
Steps to create Insert flow:
 1. Have “select ‘process’ from dual” as Sequel in source qualifier
 2. Have a sequence generator to create running process_run_Id ’s
 3. In an expression SetVariable ($$PROCESS_RUN_ID,NEXTVAL), $$PROCESS_NAME to
o_process_name, a output only field
 4. In an expression assign $$SessionStarttime to o_Starttime, an output only field
 5. In an expression accept the sequence id from sequence generator
 6. Insert into target’ process control table’ with all the above three values
Process Control Image after Insert flow

PROCESS_RUN_ID 1
PROCESS_NME VENDOR_DIM_LOAD
START_TMST 8/23/2009 12:23
END_TMST
ROW_READ_CNT
ROW_INSERT_CNT
ROW_UPDATE_CNT
ROW_DELETE_CNT
ROW_REJECT_CNT
USER_ID INFA8USER

Steps in main mapping,

 1. After the source qualifier, increment the read count in a variable (v_read_count) for
each record been read in an expression and SetMaxVariable
($$READ_COUNT,v_read_count)
 2. Before the update strategy of target instances, do the same for Insert/Update/Delete
counts; all the variables are now set with all their respective counts

Steps to create Update flow:

 1. Have “select ‘process’ from dual” as Sequel in source qualifier
 2. Use SetMaxvariable to get the process_run_id created in insert flow
 3. In an expression assign $$INSERT_COUNT to an o_insert_count, a output only field,
assign all the counts in the same way
 4. In an expression assign $$SessionEndtime to o_Endtime, an output only field
 5. Update the target ‘Process Control Table’ with all the above three values where
process_run_id equals the process_run_id generated in Insert flow
Process Control Image after Update flow

PROCESS_RUN_ID 1
PROCESS_NME VENDOR_DIM_LOAD
START_TMST 8/23/2009 12:23
END_TMST 8/23/2009 12:30
ROW_READ_CNT 1000
ROW_INSERT_CNT 900
ROW_UPDATE_CNT 60
ROW_DELETE_CNT 40
ROW_REJECT_CNT 0
USER_ID INFA8USER

4. Merits over Informatica Metadata

This information is also available in Informatica metadata, however maintaining this within our
system has following benefits,
 Need not write complex query to bring in the data from metadata tables
 Job names need not be mapping names and can be user friendly names
 Insert/Delete/Update counts of all as well as individual target can be audited
 This audit information can be maintained outside the metadata security level and can be
used by other mappings in their transformations
 Can be used by mappings that build parameter files
 Can be used by mappings that govern data volume
 Can be used by Production support to find out the quick status of load
You are welcome to share your thoughts/suggestions to inflate any “Infa Tip” -
Posted by BI- Informatica at 06:49
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
1 comment:
1.
Naveen23 July 2012 at 01:12

Hi,

Thank you the detailed workflow.

But how do i use this in my mapping.

i.,e If I have 10 workflows, how will i use this, will I be creating the above
mapping once and how do I call it.

Thanks,
Naveen
Reply
Newer PostOlder PostHome
Subscribe to: Post Comments (Atom)
 Cracks


TELUGUEBOOKS.TK
Navya Weekly 2018 - upto March2018 - 2018మార్చి : 07 * 14 * 21 * 28 2018ఫిబ్రవర్చ : 07 * 14 * 21 * 28 2018జనవర్చ : 03 * 10 * 17 * 24 * 31
1 year ago

SharePoint 2010, SharePoint, C-sharp, ASP.Net, JQuery, SQL Server Solutions
New SharePoint CSOM update for Office 365 - Wow. This is such a great build. Loving the new additions added in the new SharePoint CSOM
submitting the feedback and wa...
4 years ago

Complete reference to Informatica
Informatica Version Upgrade - *Informatica Upgrade Process: * *Stages across upgrade can be categorized as below: * 1. *Upgrading the doma
Informatica...
7 years ago

Radhakrishna Sarma
SQL Overrides in Lookups, Source Qualifier etc - I have seen in many instances, you would want to know the SQL overrides written in a mappi
Lookup SQL override or Pre-...
9 years ago

ETL-Developer.com
-

ORACLE-BASE - Oracle DBA and development articles, scripts, HOWTOs and forums (8i, 9i, 10g, 11g, 12
-

S2K - Serials 2000
-

Oracle SQL & PL/SQL: SQL Interview Questions
-

system mechanic serial | system mechanic crack | system mechanic keygen | system mechanic key
-

Business Intelligence
-

Online Magazines - Read Top eMagazines (Online Magazines)
-

Latest Editor Reviews
-

Followers

Blog Archive

 ► 2012 (3)
 ▼ 2011 (19)
o ► December (1)
o ▼ November (3)
 Process Control / Audit of Workflows in Informatic...
 crontab -e
 Data Integration Challenge – Capturing Changes
o ► October (6)
o ► September (9)
About Me

BI- Informatica
View my complete profile
Follow by Email

Submit

Rajasekhar's Blog, All rights reserved.. Ethereal theme. Powered by Blogger.