Siebel EIM Best Practice

Siebel EIM Best Practice

IN CONFIDENCE
1 Overview
1.1 Purpose

This document is intended to be used as a guide for the design and deployment of EIM
interfaces to support the delivery of Siebel based projects at BT. It will be of particular relevance
to the delivery of OneSiebel (aka Account Managed Platform, AMP) and OneView (aka Volume
Siebel Release 6).

This document is split into two main sections as follows:

1. Siebel EIM Best Practice
•
This section details techniques to be adopted to ensure that EIM interfaces and their
configurations adhere to Siebel and BT Best Practices.
2. OneView Release 4 data load Lessons Learned
•

This section details the key lessons learned from the OneView (aka Volume Siebel, Big Data, CCC) bulk data
load of Billing Accounts, Assets and Consumer Updates into the OneView Siebel instance.

This document has been assembled from experience from BT’s eVolve and Volume platforms.
These applications are fed data via ODM controlled data loads. Where processes other than
ODM will drive EIM interfaces (e.g. Converge to OneSiebel migration jobs) then some ODM
related content will still be relevant and so all sections should be read.

1.2 What’s not included

•

This document is intended to share Best Practice and Lessons Learned and is neither a
High Level nor a Low Level Design. This document should be used to feed into the above
mentioned designs.

Details of Oracle-Level tuning is not included although some techniques may be

mentioned in the context of other EIM specific parameters or configurations.
•
This is not a ‘One Stop Shop’ from which to learn EIM. Instead, this document is
intended to communicate EIM Best Practice.
Siebel EIM Best Practice
IN CONFIDENCE

1.3 Document Conventions

Key points within this document have been included in call-out boxes as follows:
 This indicates a convention that should be followed by Development Teams within
BT.

A warning sign indicates that specific care should be taken, that a piece of work is
required to determine the best configuration or that a convention or parameter
should not be followed.

An information symbol is used to indicate additional points of interest, not

necessarily relevant to specific implementations at BT.
These Document Conventions appear for most key point (in the EIM Best Practice section) and
so the document’s recommendations could be summarised by reading just the illustrated points.
2 EIM Best Practice
2.1 Introduction

Siebel’s Enterprise Integration Manager (EIM) operation is determined by a configuration file.

This file is called the IFB file and, coupled with loading data into a Siebel EIM/Interface table (a
staging table that Siebel uses so that it can build the logical data relationships as defined within
Siebel’s Repository) determines what operations EIM carries out.

As a generic, out-of-the-box interface mechanism, EIM’s performance can be streamlined by

reducing the amount of work it has to do. This is done by imposing strict IFB guidelines.

Not all parameters are discussed in detail.

A single Interface Table maps to multiple Base Tables (to closely follow the nature of Siebel

Business Objects and Business Components).

All tables and related foreign keys are processed by default unless parameters are specified in
the ifb file.

This document describes how to alter EIM’s default operation by specifying parameters in the ifb
file. Some of these changes make functional impacts (i.e. change the way in whichhow EIM
processes the data) whereas others can improve performance (normally by limiting the amount
ofwork EIM and the underlying database has to do).

2.2 EIM Configuration

2.2.1 Overview
EIM supports the following 4 modes of operation:

o Import
o Delete
o Merge
o Export

The EIM Import job is actually an Upsert operation and so this document will refer to both
Import (Insert) and Update EIM operations.
There are a number of generic ifb parameters that are valid for all modes of operation as well as
those that are operation specific.

This document will explore Import, Delete and Merge jobs in more detail. EIM Exports will not
be covered, as they are not (generally) used within BT – instead, PL/SQL (often controlled by Ab
Initio graphs) is used to query the Siebel data tables directly in order to export information.
 So that some of the terminology within this document is clear, here’s an example IFB file (…
indicates lines removed for clarity):

[Siebel Interface Manager]

LOG TRANSACTIONS TO FILE = FALSE
USING SYNONYMS = FALSE

…[Update_Volume_Bill_Accnt]

TYPE = IMPORT
BATCH = $BatchRange
TABLE = EIM_ACCOUNT
ONLY BASE TABLES = S_PARTY, S_ORG_EXT
INSERT ROWS = S_PARTY, FALSE
UPDATE ROWS = S_PARTY, FALSE
INSERT ROWS = S_ORG_EXT, FALSE
NET CHANGE = FALSE
ONLY BASE COLUMNS = S_PARTY.PARTY_UID, \

S_PARTY.PARTY_TYPE_CD, \
S_ORG_EXT.NAME, \
S_ORG_EXT.LOC, \
S_ORG_EXT.BU_ID, \

…
Note: a backslash (‘\’) should be used to escape new lines for readability.
Many parameters can be specified in the Header section, the Process section or both. The rules
of inheritance are as follows:
•

If a parameter is set in the Header section, it is included in all Process sections.

•

If a parameter is used in a Shell process, it is included in all Shell Sub processes

(detailed later).
•

If a parameter is in the Header section and also in the Process section, then the value in
the Process section over-rides that of the Header section. The same is true for Shell
process and Shell Sub process.

If a parameter exists only in a Process section and not in the Header section, then it is
only used in that Process.

Siebel EIM Best Practice

IN CONFIDENCE
2.2.2 IFB Generic Parameters
The parameters here apply to all EIM job types, with process specific (i.e. Import, Delete or
Merge) parameters and techniques being specified in subsequent sections.
2.2.2.1 Only Base Tables
This is a Process section parameter.
Usage:
ONLY BASE TABLES = BASE_TABLE1, BASE_TABLE2
…where BASE_TABLE1 & BASE_TABLE2 are tables mapped to by the EIM table in question.
By only including the Base Table relationships that you need to resolve, EIM’s work is reduced
and so Performance can be improved.

Example: In OneView, a Consumer Update job incorrectly carried out updates to S_PARTY,
S_ORG_EXT, S_ACCNT_POSTN and S_ORG_BU despite the fact that the data in
S_ACCNT_POSTN and S_ORG_BU was being defaulted by the Consumer Adapter (Ab Initio)
each time. By removing these tables from the ONLY BASE TABLES parameter, the performance
of the Volume Siebel Release 4 Consumer Update job was greatly improved. So that
performance gains such as these can be made, detailed functional analysis of the EIM interface
must be carried out in order to fully understand the data source.

➾ This parameter should be used – ONLY BASE TABLES

2.2.2.2 Ignore Base Tables
This is a Process section parameter.
Usage:
INGORE BASE TABLES = BASE_TABLE1, BASE_TABLE2
…where BASE_TABLE1 & BASE_TABLE2 are tables mapped to by the EIM table in question.
As an alternative to ONLY BASE TABLES, this parameter can be used to detail specific tables
that should be ignored by the EIM process.

For clarity, the only instance in which this parameter would be used instead of ONLY BASE
TABLES is where the majority of the tables must be included. By using this parameter, it will be
clear which tables will not be used.

➽ This parameter will generally not be used – IGNORE BASE TABLES

2.2.2.6 Use Index Hints, Use Essential Index Hints

These are Process section ifb file parameter.
Usage:
USE INDEX HINTS = TRUE|FALSE
USE ESSENTIAL INDEX HINTS =TRUE|FALSE

The default values of these parameters are indicated inbold, meaning that EIM adds index
hints to certain SQL statements. There are two levels of granularity available. By specifying USE
INDEX HINTS = TRUE, then additional index hints are used from the default. If no hints are
desired, then set both parameters to FALSE.

EIM does not always suggest the most appropriate execution path and so to determine whether the (hard coded)
hints improve performance or not, EIM jobs should be performance tested with and without this set toTrue in
order to gauge performance characteristics. Within Volume most interfaces (but not all) have been found to
perform better with no index hints in place when the Oracle Cost Based Optimizer (CBO) is used.
The actual index hints themselves cannot be specified – just an ability to turn on|off index hints
is provided.

In Volume Release 4 this parameter was used in conjunction with the SESSION SQL = "Alter session set
optimizer_mode = all_rows" although as OneSiebel have chosen the Siebel 7.7 version, then the database will
already be running in Cost Based mode. It is important that OneSiebel performance testing explores this area.

As Siebel 7.7’s now supports (in fact, mandates) the Oracle Cost Based Optimizer, then the USE INDEX
HINTS, USE ESSENTIAL INDEX HINTS must be thoroughly examined by Performance Testing.

On Volume it has been observed that Siebel often adds inappropriate index hints such
as hinting the use of S_ORG_EXT_U2 to resolve foreign keys to S_ORG_EXT. As this
index is led by BU_ID which is non-selective in the Volume application (all Accounts
belong to the ‘BT Volume’ organisation), this is an inappropriate index to use. By
turning hints off, the more appropriate S_ORG_EXT_U1 index (or better still a custom
index) may be chosen by Oracle.

Siebel EIM Best Practice

IN CONFIDENCE
2.2.2.7 Session SQL
This parameter has already been introduced in the section on USE INDEX HINTS, USE
ESSENTIAL INDEX HINTS.
Usage:
SESSION SQL = “sql statement to be executed”
Note that no terminating semi-colon (;) should be used eg.
SESSION SQL = "Alter session set optimizer_mode = all_rows"
This parameter can be used to execute custom SQL at the beginning of EIM’s session (EIM is a
single threaded, single database connection component). Only a single statement can be issued.
Session SQL should NOT be used to modify data in the Siebel schema directly as Siebel does
not support direct SQL (except for querying).
Session SQL should NOT be used to modify data in the Siebel schema directly as Siebel
does not support direct SQL (except for querying).

As Siebel 7.7’s now supports (in fact, mandates) the Oracle Cost Based Optimizer, then
setting of optimiser mode via SESSION SQL may no longer be appropriate. The
Performance Team will investigate this area.

2.2.2.8 Using Synonyms

This is an EIM Process section parameter.
Usage:
USING SYNONYMS = TRUE|FALSE
EIM’s default operation is to look up Account Synonyms in order to resolve foreign keys to
S_ORG_EXT. This requires that EIM does an additional lookup and so setting this parameter to
False can reduce overall run time. The default is True.

Although OneSiebel may have a requirement to support Synonyms (Trading As names), ODM
will supply Siebel User Keys of the Account as defined in S_ORG_EXT and so setting this
parameter to FALSE will not impact this functionality.

➾ This parameter should be used – USING SYNONYMS = FALSE

2.2.2.10Update After Failed Insert

If single commit is not implemented (and it should generally not be used for EIM Import
processes) in OneView, ODM’s TrueDiff process should check all tables in a multi-table entity to
ensure that the Update after failed Insert operation performs properly.

EIM Import jobs intended to Insert-only into the Base Tables are not generally configured with
UPDATE ROWS = FALSE, which may otherwise be expected (to avoid a Data Quality Issue
where an anomaly has occurred). This is so that if the initial insert fails, then sending the record
again will result in an update to a partial record. In Volume at present, this is not implemented
properly because the TrueDiff process (the ODM process that identifies which records to insert,
update or delete from Siebel based on comparing the data within Siebel with that within ODM’s
datastores) does not compare columns in all tables for every entity.

➾ As per Siebel Alert 290, these settings should be set to perform a single commit
(ROLLBACK ON ERROR = TRUE, COMMIT EACH PASS = FALSE, COMMIT
EACH TABLE =
FALSE) for all Merge and Delete Operations.
If single commit is not used (and it should not be), ODM’s TrueDiff process should be
capable of including all columns in all Base tables involved in each EIM process.

2.2.3 IFB Import Process Specific Parameters

This section details those parameters that are specific to EIM jobs of type ‘Import’ (both Insert
and Update operations).
As well as the parameters detailed within this section, those parameters in the Generic section
can also be applied.
2.2.3.1 Only Base Columns
This is a Process section parameter.
Usage:

IGNORE BASE COLUMNS = BASE_TABLE1.BASE_COLUMN1,\

BASE_TABLE1.BASE_COLUMN2,\ BASE_TABLE2.BASE_COLUMN1

…where BASE_TABLE1 and BASE_TABLE2 are mapped to by the EIM table in question and
BASE_COUMN1 & BASE_COLUMN2 are columns on each Base Table.
Whereas the generic parameter ‘Ignore Base Columns’ instructs EIM to ignore specific columns,
‘Only Base Columns’ instructs EIM to only process those columns specifically listed.
It does not make sense to use both ‘Ignore Base Columns’ and ‘Only Base Columns’ in the same

job. Either specify one or the other.

ONLY BASE COLUMNS should be used for every interface to achieve best performance

You must include Foreign Key columns (Insert and Update), required fields (Insert operation)

and any data columns to be loaded.

➾
ONLY BASE COLUMNS should be used for every interface to achieve best performance.
2.2.3.2 Insert Rows, Update Rows
These two Process section parameters impact upon both Data Quality and Performance.
Usage:
INSERT ROWS = BASE_TABLEN, TRUE|FALSE
UPDATE ROWS = BASE_TABLEN, TRUE|FALSE

As has already been stated, an EIM table maps to multiple base tables. The use of the above parameters
controls what type of operation is allowed on each specified table. The default for both parameters isTrue.

As an EIM Import job is actually anUpsert operation. i.e. if the User Keys populated in the EIM table
matches those of an existing row in the Base Tables, then the operation is an Update. If the User Keys
populated in the EIM table do not match those of an existing row in the Base Tables, then the operation is
an Insert operation.

BT has invested heavily in the ODM layer’s capability of creating separate Insert and Update
transactions, based around theunload functionality which is necessary due tomu lti-mastering.
As such, ODM determines whether the operationshould be an insert or an update to Siebel’s
data. Therefore, the author believes that every EIM job should specifically use INSERT ROWS