UCM

CDI Integrating Application

Tata Consultancy Services ltd. December 1, 2010 1

 COURSE AGENDA

Day 1
Module 1 : Review of CDI Architecture
Module 2 : Overview of Application Integration
Module 3 : Initial Data Loads into UCM
Module 4 : The UCM Batch Workflow and SDH tables
Day2
Module 5 : UCM Messaging and CRMML
Module 6 : UCM Server Party Package Workflows
Module 7 : The UCM Batch Management Workflows
Module 8 : Publish and Subscribe
Module 9: Wrap-up

2
Module 1: Review of Siebel CDI
Architecture

3
 The CDI Solution
The Customer Data integration(CDI) solution consists of three
products

CDI

UCM SDQ UAN

4
 UCM = Universal Customer Master
What is UCM?
A Siebel horizontal application which is the central master
database/application for storing any and all customer information .
What it does?
1. Acts as a central repository of customer data.
2. Keeps customer information in synch with other systems in the
enterprise, thus giving a 360 real-time true picture of the
customer.
In a typical large enterprise, you have hundreds of various
proprietary and legacy applications which deal with different parts
of the customer information , some deal with service data, some
with sales data, some others purely with demographics and contact
data. UCM has the capability to synchronize and bring together all
this data to give a single unified view of the customer as a whole.

5
 SDH = Source Data and History
tables
The central components of the CDI solution are the Source Data
and History (SDH) tables associated with UCM
1. Provide a mechanism to maintain audit-trace of changes
happening to the customer record.
2. Provide a staging area for data quality.
External systems first integrate with these SDH tables when
sending data to UCM, UCM will then in turn use its data quality
rules to sort out the
good updates from the bad ones and apply the good ones to
the base tables while storing the bad ones too for audit
purposes.
E.g. In an enterprise, suppose you have a .NET based sales
application and a legacy service application, both storing the
customers contact number. If the enterprise feels that its service
department is more likely to have a more current contact number
of the customer, you can define the service application with a
higher confidence level for updating this field. In cases where both
applications are sending simultaneous updates to this field, both
updates will be stored in the SDH
6
tables, but the one from the
 UCM Data Model
Is an extension of Siebel Data model
Source, Data and history tables are added.

List
UL Column/ Applet View
Screen
Application
Control

.
BL
Field BC BO

DL SDH SDH
Column table

7
 Source Data and History types
SDH data contains several types of records distinguished by a
Type value
Each record is one of the following types:
1. History
2. Source
3. Merged
4. Cleansed
5. Queued
6. Unmerge
7. Batch

A single transaction typically results in the creation of multiple

Source data and history records.

8
 Source Data and History : Data
schema
Provide the infrastructure necessary to support child structures
Example : addresses for contacts

Base table
S_ORG_EXT S_CONTACT

Source data
History Table

S_UCM_ORG_EXT S_UCM_CONTACT

S_CIF_EXT_SYST

External System
S_UCM_ORG_CHLD S_UCM_CON_CHLD
information table

Source Data History

Child table

9
 UCM Application Integration
Revolves around integrating to these SDH tables
First basic CDI configuration must be performed
1. Systems registered
2. Data quality configured
3. Survivorship configured

10
 CDI Configuration
External System Registration

Data Quality Configuration

Survivorship Rules

11
 External System Registration
Every operational application that connects to UCM must register
with UCM.
After registration, a systems privileges and accessibility can be
defined
1. Query
2. Insert
3. Update
4. Delete

Defines grant access to Siebel UCM

Privileges are granted at the BO layer

12
 How to register an external application
with UCM
Use the system registration view
1. Select Administration Universal Customer Master > System
registrations
2. Click New
3. Enter a system name, id and protocol type(HTTP,MQSeries)
4. Below System registrations applet Select New and assign BO
privileges in theSystem Privileges.

13
Registering an External Application

14
 External System referencing: Cross-
Referencing
Cross-referencing stores source identification information for data
in the UCM
UCM stores identification information about external systems
that have been registered.
Cross-referencing provides a one-to-many mapping of customer
records across
multiple systems
Allows UCM to keep track of the same customer record across
multiple systems
A single UCM record maps to one or many records in any
registered application
UCM supports cross-referencing for the following objects:
1. Contacts
2. Accounts
3. Households
15
 Cross-Referencing
Cross-reference entries are created automatically with the creation
of every new account, contact, or household record
External systems send requests to insert a record
If there is an existing cross-reference entry, the entry is
updated
If there is no existing cross-reference entry, a new record is
inserted
The system ID is the value assigned when the system was
registered and system name is the associated name
There are multiple possible external Ids to account for
situations where multiple records in an operational application
map to a single record in UCM
Only the first is stored in the ID tag in a message request,
by default

16
 External System Referencing:
Universal Unique ID (UUID)
Another mechanism to track the same record across multiple
applications
Instead of keeping track of the system IDs., UCM publishes a
UUID to the operational application

The UUID is not exposed in the user interface

Is stored in a column of the S_PARTY table
Can be exposed using Siebel Tools

Many customers use cross-referencing and the UUID at the same

time

17
 Data Quality Options
Data quality consists of
1. Data Cleansing
2. Data Matching(De-Duplication)
Data Cleansing
1. Address validation and standardization
2. Capitalization
3. Abbreviations
Data Matching identifies possible duplicate records for
account,contact and prospect records
Identify duplicates and merge them based on certain key
attributes.
E.g. from our previous case, if we decide that the key
attributes of a unique customer are : first and last names and
say the passport number. So, assuming we have the passport
number stored in both the .NET and J2EE application, data
matching in UCM will be able 18
to identify that customer # 1-
 How to configure Data Quality
To configure data quality :
1. Enable the data quality component group
2. Set data quality parameters
3. Specify data quality settings
4. Specify data quality settings: Site Map > Administration
Data quality > Data quality settings
During integration cycle disable deduplication to simplify the
environment.

19
 What to Do with Duplicate Records
Once suspect duplicate have been identified, the records can be
evaluated
and:
Merged with existing records
Introduced as new UCM records (promoted)

For records that are merged, UCM:

Stores the duplicate record, with Merged as the type, and the
surviving record, with History as the type, in the Source Data
History table
Merges parent records using survivorship rules to generate a
new merged surviving parent record
Updates the parent surviving record with the new merged
surviving record
Deletes the duplicate record

20
 Survivorship Rules
Customer data originates in multiple systems
Conflicts arise when different sources have different data for the
same attribute
For eg: two systems have different addresses for customer
The role of UCM is to resolve these conflicts, which ensures that
the best version record survives

Survivorship rules are an automated mechanism for consolidating

customer data to produce a best-version record
Evaluates updates from each registered system
Applies rules to determine which is the best value for each
attribute
Can be constructed by a system integrator or a CDI data
steward

Three comparison methods

Recent 21
 Survivorship Rules and Match Scores
UCM allows two thresholds to be set depending on the match
scores from the matching process
Records with high match scores can be auto-merged, applying
survivorship rules

High Score - 90

Records with match scores in the middle will be brought to the

attention of the data steward for manual review and resolution

Low Score - 70

Records with low match scores will be left as separate records

22
 Setting Match Thresholds
Auto-threshold: Determines the high matching score threshold
Account Auto Threshold
Contact Auto Threshold

Manual threshold: Determines the low matching score threshold

Account Manual Threshold
Contact Manual Threshold
Can be constructed by system integrator or CDI data steward

Can be set by a system integrator or a CDI steward

23
 Survivorship Rule Structure
Employ comparison criteria to determine if a new value for a given
attribute coming from any source is better than the existing value
in the best-version record
Three comparison methods:
Recent
History
Source
Multiple rules can be constructed for each object (account or
contact) but only one rule can be active at a time
Use attribute groups
A group of fields that survive or get overwritten together
Makes survivorship rule creation less cumbersome
Each field within an object can only be included in a single attribute
group
To avoid survivorship rule conflicts
24
 Building Survivorship Rule
Three primary steps:
Create an attribute group for the survivorship rule
Create the survivorship rule logic
Activate the survivorship rule

25
Setting Survivorship Rules

26
 Unmerge
For records that have been merged
Surviving record--- record that remains after the merge
Victim record ----- record that is merged and ceases to exist
If merged records are unmerged after changes have been made to
be the merged (surviving) record
All changes and updates since the merge will be applied to the
surviving record
The victim record will be resurrected from the Source Data
history table Merge Unmerge
A
A2

A1 A2

B B

Changed to merged record

27
 Outbound Data Flow: Publish and
Subscribe
Publish and subscribe allows records inserted or updated in UCM to
be published to external systems
Publication can be targeted to any system registered with UCM

The mechanism for publication is a Siebel Workflow

Workflow policies track changes to UCM records and flag them
for production
Then Workflow processes collect the appropriate information
and guarantee publication of changes to specified (subscribed)
systems

Publish and subscribe requires that publication parameters be

specified
Can be real-time or daily-batch

28
Module 2: Overview Of Application
Integration

29
 Application Integration and the UCM
SDH tables
UCM integration extends the EAI toolset by transferring data to
SDH tables prior to placing data into Siebel base tables
SDH tables provide a queuing station for processing incoming
records
Records can be analyzed for future processing
Data must be processed at this layer before entering standard
Siebel base
tables
This is the source of integration process differences between UCM
and
the standard Siebel CRM Enterprise product

30
 Data Transfers to SDH tables
Data transfers to SDH tables use traditional methods
Data layer data loads via the EIM
Business layer data loads via standard integration application
procedures
Data transfers paths to SDH tables analogous to standard CRM
case
No data cleansing or matching
Business Layer Data
Loads
SDH Tables Base Tables

Data Layer Data Loads

31
 Data Transfers to Siebel Base Tables
Are independent of how data is placed on SDH tables
Data layer
Business logic layer
During this transfer(insert) data quality is activated
Cleansing and matching
Application of survivorship rules
Implemented as a UCM workflow process The UCM Batch Manager

32
 UCM Service Requests
External Systems request services from a UCM application
Insert
Query
Update
Delete

UCM must determine which external system is requesting the

service
To validate privileges

The System ID must be supplied in addition to the traditional inputs

even if the UCM application is not physically connected to an
external operational system

33
 UCM External System Parameters
Request for service can only be accepted from a known
source
Registered system
System ID
System Name

The records to be operated on also have UCM specific

information associated with them
Unique external record ID
Unique user ID
Transfer code describing data transfer mechanism

34
 Inbound Data flow for the business
layer
Request for UCM application services
Submitted through a (CRMML) input file
Security service checks privileges of requesting external
system
Transaction service executes operations
To SDH
tables
Siebel Adapter
Security and
Through
Transaction Services
Dispatch Service

Outbound
Input data Data
(CRMML)

35
 CRMML Input Files
Contain
Information about external system --- system ID
Service being requested
Methods and arguments to be used

User does not have to build the integration objects

Internal integration objects provided by the UCM application
The CRMML file directs the UCM application on which the
integration object to use

User does not have to specify UCM specific record information

The UCM application will make these assignments when placing
records on SDH tables

36
 Security and Transaction Services
The security service checks the system ID specified in the input
(CRMML) file for Insert ,Update, Query and Delete privileges
If the system is not registered data will not be processed

Transaction manager will perform the requested operations

Uses an internal UCM integration object to obtain the data
necessary
to process a request

37
 Implementing Inbound Data Flow at
the Business Layer
Inbound Requests are processed by a workflow process
HTTP
Queue Based
File Transport

Protocol specified when system is registered

To SDH
tables
Siebel Adapter
Security and
Through Dispatch
Transaction Services
Service

Input Data
Outbound
(CRMML) Data
Server Party Package

38
 Inbound/Outbound Processing
UCM has automated much of the inbound processing --- a user
must:
Prepare input(CRMML) file
Run (appropriate) Server Party Package workflow to place records
of SDH tables
Run UCM Batch workflow to activate data quality and place data on
Siebel base tables

The UCM application supports an outbound publish and subscribe

facility
Insert and Update records can be published to external system
subscribers
Individually or as a group
In batch or real time mode

39
Module 3: Initial Data Loads into UCM

40
 Populating UCM Data
Use business layer imports to enter , update or delete small
amounts of data in the base tables
Use the EIM server task to populate the SDH tables ( through
interface tables)

Use the UCM batch manager to transfer data from SDH tables to
Siebel Base tables
Cleanse andNon-Siebel
match UCM Database
loading tool

External data Interface UCM SDH Siebel Base

may be table tables tables
cleansed

41
 Staging Tables
The UCM application uses two sets of staging tables before making
data available to the Business Logic layer through Siebel Base
tables

The EIM staging tables: Data is staged to be transformed into a

Siebel standard format
The Source ,Data and history tables: Data is staged for cleansing
and matching

42
 Data Quality Options for Initial Loads
Data may be cleansed and matched while being transformed from
the source , data and history tables or preprocessed before loading
into EIM
tables

Cleansing and matching during data import adds loading latency

----
time overhead
If the Siebel Universal Connector is being used for cleansing and/or
matching , it will need to be integrated into the application

43
 Siebel Base Tables
User data is stored in one or more Siebel base tables
Relationships between base tables (referential integrity) rely upon
primary and foreign keys based on ROW_IDs
ROW_ID is the system primary key for every base table
ROW_ID is a system-generated value

PK
PK
FK FK

S_BU S_ORG_TXT S_ADDR_ORG

44
 User Keys
Based on multiple columns, user keys are used to uniquely identify
a row for EIM
Primary and foreign keys based on ROW_ID are used for system-
wide referential integrity
PK User Key FK

ROW_ID Name LOC BU_ID

S_ORG_EXT S_UCM_BU

45
 UCM Data Loads
Loading Data into the EIM Tables
Loading Data from the EIM Table to the SDH Tables
Loading Data from the SDH Table to the Base Tables

46
 Interface Tables
Staging area for data to be imported, updated, or merged into
Source Data and History (SDH)
Staging area for data imported by EIM
EIM tables do not have ROW_Ids assigned
S_UCM_ORG_EXT

External EIM_UCM_ORG
Database

Interface Table S_UCM_CONTACT

External Data
SDH Tables

47
 Populate Interface Tables
Populate source data into interface tables using a non-Siebel
process
Use database vendor utilities to load data from flat files
Use SELECT INTO to pull data from other SQL databases
Write programs or stored procedures to cleanse or preprocess
and insert rows
Use direct SQL insert, update, or delete statements to update
columns in EIM tables
External Data
EIM_UCM_ORG

Non-Siebel
process

48
 Required Interface Table Columns
In addition to standard non-nullable columns
IF_ROW_BATCH_NUM ---- Batch number
IF_ROW_STAT --- Status
ROW_ID Row ID
The UCM application also requires a non-null inputs
UCM_UID A unique UCM record identifier
UCM_TYPE_CD --- SDH type (example : batch for initial loads)
EXT_SYSTEM_NUM External System ID
UCM_EXT_ID --- A unique external record identifier (usually
assigned by the external system)

49
Source History Data (SDH) and EIM
Schema
UCM Source Data History and EIM Tables

S_UCM_ORG_EXT Source and History table for accounts

S_UCM_CONTACT Source and History table for contacts
S_UCM_ORG_CHLD Source and History table for child objects
of account
S_UCM_CON_CHLD Source and History table for child objects
of contact
EIM_UCM_ORG EIM interface table for S_UCM_ORG_EXT
EIM_UCM_CON EIM interface table for S_UCM_CONTACT

EIM_UCM_ORG_CHLD EIM interface table for S_UCM_ORG_CHLD

EIM_UCM_CON_CHLD EIM interface table for S_UCM_CON_CHLD

50
 Referential Integrity
User key column mappings in interface tables are used to resolve
ROW_Ids for SDH tables, maintaining referential integrity

PK(ROW_ID)
User key

Acme 1-8D

Interface Table
FK(ROW_ID)
1-8D

UCM SDH Tables

51
 Enterprise Integration Manager (EIM)
Server task that manages the exchange of data between Interface
tables and Source Data History tables

EIM_UCM_ORG S_UCM_ORG_EXT

S_UCM_CONTACT

External Data

52
 Running the Enterprise Integration
Manager
The EIM server task is run as a workflow from Administration
Server > Jobs

53
 Run as EIM Task with Parameter
Settings
Creates an operating system (OS) process that writes log entries,
reads an .ifb file, loads a dictionary and executes EIM steps

54
 Configuration (.ifb) file
Role for initial data loads
Specifies the batch , order and type of processing desired
Should be placed in c:\Susrvr\siebsrvr\ADMIN
For imports
[Siebel Interface Manager]
PROCESS = IMPORT EIM_UCM_CON

[IMPORT EIM_UCM_CON]
TYPE = IMPORT
TABLE = EIM_UCM_CON
BATCH = 100
ONLY BASE TABLE = S_UCM_CONTACT

55
Use Log Information to Troubleshoot
Navigate to Administration Server > Tasks
Error information in Task view or log file

56
 Verify Data Load into SDH Tables
Navigate to Administration Universal Customer Master > Contact
(Account) > Source Data and inspect the Source Data table

57
 UCM Batch Manager
Transfers data from SDH tables to Siebel Base tables
Implemented by separate workflows for Accounts and Contacts
UCM Account Batch Data Management workflow
UCM Contact Batch Data Management workflow

S_UCM_ORG_EXT S_ORG_EXT

SDH
table Base
Table

SDH Base
table column

58
 Running the UCM Batch Manager
The UCM Batch Manager is run as a workflow from Administration-
Server > Jobs

59
Verify the Data Load into Siebel Base
Tables
Navigate to Site Map > Contacts (Companies) >
Contacts(Accounts) List and inspect the list

60
 Determining Data Mapping
Need to analyze and determine :
Which Siebel base table columns will store external source data
Which interface (EIM) tables and Source Data History (SDH)
tables will be used to import from source to destination

Source
(External

Where data
ends up
How to
get there
Destination
(base)

Interface SDH
(EIM)

61
 How to Get There
After verifying and documenting the source-to-base data mappings,
determine EIM-to-SDH and SDH-base mappings
Source Source EIM Table SDH Base Table Base Column
Entity Attribute and Table and
Column Column
Customer Name S_ORG_EXT NAME

Customer Phone S_ORG_EXT MAIN_PH_NUM

Address S_ADDR_ORG ADDR

S_ADDR_ORG CITY

S_ADDR_ORG ZIPCODE

Interface
(EIM) SDH
(UCM)

62
 Table Column Mappings
Can be determined using Siebel Tools

EIM Interface tables Source Data History Tables

Source Data History Tables Base tables

63
Module 4: The UCM Batch Workflow
and SDH Tables

64
 The UCM Batch Manager and SDH
Tables
This module will discuss two aspects of UCM Batch Manager
operations on SDH tables.
Data transfer to Siebel Base Tables
Data transfer for child objects

65
 The UCM Batch Manager
Batch processes insert requests queued on the Source, Data ,and
History (SDH) tables
Initial data loads (legacy systems)
Incremental inserts
Moves records from SDH tables to Siebel base tables
Example: S_UCM_CONTACT S_CONTACT
Two flavors of batch workflows for accounts and contacts
UCM Account Batch Data Management Workflow
UCM Contact Batch Data Management Workflow

66
 UCM Account Batch Data
Management Workflow

Start

Get Error Process

Query by Id Error?
Code NoMatch

CDM CDM Process

Match id? Error? Converter
Cleanse Match AutoMatch

Process PubSub
Review

End

67
 UCM Batch Workflow Steps
Query By Id
CDM Cleanse
CDM Match
Process NoMatch
Process AutoMatch
Process Review

68
 Query By ID
Invokes the Query method of the EAI Siebel Adapter business
service
Reads an object to be inserted from a SDH table
Outputs a Siebel message

Workflow Process
Props

OutputintObjectName
SDH int Obj Name

PrimaryRowId
Object Id
Query By Id
SiebeMessage
Siebel Message

69
 CDM Cleanse
Invokes the Cleanse method of the UCM Data Quality Manager
business service
Reads a Siebel Message containing object to be inserted from
SDH table
Outputs a cleansed XML representation of the original Siebel
Message
Workflow Process
Props
IntObjectName
Base int Obj Name

SiebeMessage
Siebel Message CDM Cleanse

XML Hierarchy
Cleansed Prop

70
 CDM Match
Invokes the Match method of the UCM Data Quality Manager
business service
Reads an XML representation of a Siebel Message containing an
object to be inserted from SDH table
Output indicates if record is a duplicate and the auto-merge
level
Workflow Process
Props
XML Hierarchy
Cleansed Prop

NumAutoMatch
Number of AutoMatch CDM Match

MatchingrowId
Cleansed Prop

71
 To Merge or Not to Merge
The workflow splits the output of the match process into three
cases (CaseType)
No match ---- insert record
Potential match --- activate survivorship rule to determine if
merge is appropriate
Transfer record for manual resolution

72
 Process NoMatch
Is activated in the event that the incoming record is not scored as a
match
Record is inserted
Invokes the HandleMatchCase method of the UCM Batch
Manager business service
Workflow Process
Props
CaseType
Matching Id
ObjectType
Object Type
TurnOnSE
Turn On Survivorship Process NoMatch
XML Hierarchy
Cleansed Prop
XML Hierarchy
Processed Prop

73
 Process AutoMatch
Is activated in the event that the incoming record is scored as a
match
Record evaluated by survivorship rule --- is either merged or
inserted
Invokes the HandleMatchCase method of the UCM Batch
Manager business service
Workflow Process
Props CaseType
MatchingRowId
Matching Id
ObjectType
Object Type
TurnOnSE Process AutoMatch
Turn On Survivorship
XML Hierarchy
Cleansed Prop
XML Hierarchy
Processed Prop

74
 Process Review
Is activated in the event that the incoming record is scored as a
match but not resolved by survivorship rule
Record is posted in Duplicate (Accounts) view
Invokes the HandleMatchCase method of the UCM Batch
Manager business service
Workflow Process
Props
CaseType

ObjectType
Object Type
XML Hierarchy Process Review
Cleansed Prop

XML Hierarchy
Processed Prop

75
 Post Process Activities
In the event that the record is resolved through an insert or a
merge
A real-time message can be published
In the event record is queued for manual resolution
The workflow is terminated and no information is published(in
real--time mode)

76
 Address for Contacts
Contact business objects come preconfigured with address child
busniess components

77
 The UCM Data Model and Child
components
All UCM child components for each business object (Account and
Contact) are stored in one table
The explicit component object type is configured in each UCM
business component to distinguish the child components

Busniess
Layer Field BC BO

SDH SDH
Data
Layer Column table

78
 Address for Contacts
Contact business objects come preconfigured with address child
business components
Primary
UCM Contact SDH UCM Contact SDH
Business
Component

UCM Contact Address SDH

Business Component Business Object

S_UCM_CONTACT

S_UCM_CON_CHILD

Table

79
 Source Data and History : Data
schema
Provide the infrastructure necessary to support child structures
Example : addresses for contacts
Base table S_CONTACT
S_ORG_EXT

Source data
S_UCM_ORG_EXT History Table S_UCM_CONTACT

S_CIF_EXT_SYST

S_UCM_ORG_CHLD External System S_UCM_CON_CHLD

information table

Source Data History

Child table

80
 Creating UCM SHD Child Business
Components
The UCM data model can easily be extended to include custom SDH
components

81
 Example: Address for Contacts
The UCM SDH Contact Address business component is UCM Contact
Address Source Data and History

82
 Custom Child Business Components
Name Business Components and Tables

Assign Data Fields

Configure UCM Fields

Configure UCM Batch Workflow

83
 Name Business Components and
Tables
Copy the Siebel base business component and rename it
Siebel suggested standard child business component names:
UCM + <Parent BC Name> + <Child BC Name> + SDH

Copy of UCM Contact Address Source

Data and History

84
 Assign Data Fields
Change the predefined column names (ATTRIB_N) for the business
component to the UCM SDH table Name field

85
 Configure UCM Fields
There are five UCM specific fields that must be configured
Component Column Predefault
Field Name Value
Object Name OBJECT_NAME Party Object
name
(Account/Conta
ct)
Object Table OBJECT_TABLE UCM Party base
table
Object Type OBJECT_TYPE Equal to child
name
Par Rpw Id PAR_ROW_ID Parent Row Id
UCM UId UCM_UID Field identifier

86
 Configure UCM Batch Workflow
The OBJECT_TYPE is used for the search specification in the UCM
Batch Workflow parameter list
Search Specification = Object Type
Example for Addresses: [Object Type] = address

87
Module 5: UCM Messaging and
CRMML

88
 CRMML
An XML based markup language for data exchange and remote
business process invocation
Used for communication with UCM
Inbound requests (insert, query, update, delete)
Responses
Publishing messages

Request
Back-Office
UCM
Application Response
Publication

These exchanges use

CRMML messages

89
 Example: Insert Contact Profile
<CRMML>
System requesting the
<CRMHeader> info
<FromSystem>
<SystemInfo> <SystemId>System123</SystemId>
<SystemName>External App</SystemName>
</SystemInfo>
</FromSystem>
<ToSystem> </ToSystem>
<MessageDescriptor> </ MessageDescriptor>
</CRMHeader> Requested transaction,
<CIFPartyPackage> or method
<InsertContactProfile>
<Contact> Method argument
<FirstName>Test</FirstName>
<LastName>Contact</LastName>
.
</Contact>
<InsertContactProfile>
</CIFPartyPackage>
</CRMML>
90
 CRMML and Integration Objects
Integration Objects
Define the structure of the CRMML message
Are used as templates to create memory-resident instances
of data for processing by business services
UCM provides integration objects for different parts of the
CRMML message
Integration Object Description
Type
Envelope Integration Represents CRMML
Object header structure
Dispatcher map Contains rule sets for
integration object mapping from method to
integration objects
UCM integration objects Represent the method
argument
(Contact, Account, etc)
91
 CRMML Header
Contains
Information identifying sending and receiving systems
Used by Universal Customer Master for security
verification
Message descriptor identifying the message

<CRMHeader>
<FromSystem>
<SystemInfo> <SystemId>System123</SystemId>
<SystemName>External
App</SystemName>
</SystemInfo>
</FromSystem>
<ToSystem> </ToSystem>
<MessageDescriptor> </ MessageDescriptor>
92
 CRMML Header : System Info
Used to describe both the sending and receiving systems
<FromSystem> - sender
<ToSystem> - receiver
For inbound messages, FromSystem SystemId is matched against
registered systems as part of UCMs security check
<Systeminfo>
SystemId used for <SystemId>123-456-789</SystemId>
security <SystemRole>Insurance App</SystemRole>
verification
<SystemName>Client System</SystemName>
</Systeminfo>

93
 CRMML Header : Message Descriptor
Identifies the message
MessageId : Unique ID for the message
Timestamp: when message was issued
TimeToLive: how long (after timestamp) message is valid
TransactionScope : reserved for future use
<MessageDescriptor>
<MessageId>1T3D2005083115:45:32</MessageId>
<TransactionScope>any</TransactionScope>
<Timestamp>Sunday</Timestamp>
<TimeToLive>OneYear</TimeToLive>
</MessageDescriptor>

94
 CRMML Header : Summary
CRMHeader element has the following structure
<FromSystem> <SystemInfo>

<CRMMLHeader> <ToSystem> <SystemInfo>

<CRMML> <MessageDescriptor>

<Body>

95
 CRMML Header: Integration Object
The CIF Envelope Integration Object is used to process the CRMML
Header

96
 CRMML Body
Contains one or more message elements representing:
Inbound requests from an external applications to
insert, query, update, or delete a database object
Outbound response to a requested transaction
Outbound publication messages to be sent to an
external subscribing application
Each message element includes:
Software package must be CIFPartyPackage
Method --- the requested action
Argument the argument to the action

<SoftwarePackage> <CIFPartyPackage>
<Method> <InsertContact>
<Argument> <Contact>

97
 Mapping to Integration Objects
UCM methods and arguments are mapped to integration objects
according to a set of mapping rules
These mapping rules define:
What methods are allowed
What arguments those methods take

Methods contain Integration Objects contain

Arguments,which contain Integration Components, which contain
Fields Integration Component Fields

<CIFPartyPackage> CIFContact
<InsertContact>
<Contact> Contact
<LastName> FirstName
</LastName> LastName
<FirstName>
</FirstName>

98
Mapping to Integration Objects Continued

The mapping rules are defined in user properties of a mapping

integration object
The default mapping object is CIFDispMap

Envelope Package Method

CRMML/CIFPartyPackage/InsertContact ->
CRMML/CIFPartyPackage/InsertContact;a;b;CIFContactInsert;CIFContactInsertRs;
IXML_Operation_ADD

Integration Object Integration Object

for inbound for outbound
requests responses

99
 CRMML Structure: Summary
Structure of a base CRMML message

<FromSystem> <SystemInfo>

<CRMHeader> <ToSystem> <SystemInfo>

<CRMML> <MessageDescriptor>

<CIFPartyPackage> <InsertContact> <Contact>

Package Method Argument

100
 Complex CRMML Messages
Arguments may contain sub-arguments
For example: Contacts contain addresses
These sub-arguments are represented by sub-elements in the
CRMML message <Contact_
INSPersonalAddress>

Method Argument Method <Contact_

<InsertContactAddress> <Contact> <InsertContactAddress> INSPersonalAddress>

<Contact_
INSPersonalAddress>

101
 Example: Insert Contact Address
<CRMML>
<CRMHeader> .</CRMHeader>
<CIFPartyPackage>
<InsertContactAddress>
<Contact>
<FirstName>Test</FirstName>
<LastName>Contact</LastName>
<ListOfContact_INSPersonalAddress>
<Contact_INSPersonalAddress IsPrimaryMVG=Y>
<INSPersonalStreetAddress>100 Main St.</ INSPersonalStreetAddress>
<INSPersonalCity>York</INSPersonalCity>
<INSPersonalState>AL</INSPersonalState>
<INSPersonalPostalCode>78221</INSPersonalPostalCode>
<INSPersonalCountry>USA</INSPersonalCountry>
</Contact_INSPersonalAddress>
<Contact_INSPersonalAddress IsPrimaryMVG=N>
<INSPersonalStreetAddress>50 Center St.</INSPersonalStreetAddress>
.
</ListOfContact_INSPersonalAddress>
</Contact>
</InsertContactAddress>
</CIFPartyPackage>
</CRMML> 102
Module 6: UCM Server Party Package
Workflows

103
 Party Package Workflows
Handle incoming requests
Check source system privileges
Create memory-resident instances of data for processing by
business services, based on integration object definitions
Execute requested operation
Return response to requesting system
Publish (real-time) to subscribing systems

104
 Party Package Workflows
continued

Include

Workflow Process Name Description

UCM Server Party Package Reads CRMML message from a
Workflow (File Transport) file and writes response to a file
(for testing)
UCM Server Party Package Used for HTTP and MQSeries
Workflow (MQSeries Server message transports.
Transport)
UCM Server Party Package Also for HTTP and MQSeries
Workflow(MQSeries Server transports, Models error
Transport Error Handling) handling.

105
 UCM Server Party Package Workflow
(File Transport)
Workflow is composed of the following steps:

Read Convert to XML Dispatch Security

Start
From File Hierarchy Message Check

Convert
Request In

Transaction
Manager
PubSub

Convert
Realtime
Request
Pub/Sub
Out

End Write To Convert to

File Hierarchy

106
 Read From File
Invokes the Receive method of the EAI File Transport business
service
Reads contents of a file into string
Input file should contain a valid CRMML message
WorkFlow Start
Process Props

File name
Input File Name
Read From File:
EAI File Transport
Receive
<value>
XML

Convert to XML
Hierarchy

107
 Convert to XML Hierarchy
Invokes the XMLDocToXMLHier method of the XML Hierarchy
Converter business service
Converts an XML string into an XML hierarchy object

WorkFlow Read From File

Process Props

<value>
XML
Convert to XML Hierarchy:
XML Hierarchy Converter
XMLDocToXMLHier
XML Hierarchy
Core Req Prop

Dispatch
message

108
 Dispatch Message
Invokes the DispatchMessage method of the UCM Dispatcher
business service
Verifies structure of CRMML message
Acts information to message components that is used by
downstream business services Convert to XML
WorkFlow
Hierarchy
Process Props

XML Hierarchy
Core Req Prop
Dispatch Message:
UCM Dispatcher
DispatchMessage
XML Hierarchy
Disp Req Prop

Security Check

109
 Dispatch Message Continued

Uses two integration objects, specified as user properties of the

business service, to represent the message

Integration object
User Property Value used to process body
Name
DispatcherMapNa CIFDispMap Integration object
used to process
me header
XMLEnvIntObjectN CIF Envelope
ame

110
 CIF Envelope Integration Object
Defines the allowed structure of the CRMML message header.

111
 Dispatcher Map Integration Object
Contains a set of rules to map methods to integration objects used
to process those methods
Each rule is a user property of the mapping integration object
Name identifies the message content that must be matched
Value specifies the information that should be added to the
method when a mapping is found
Content to match in
order to trigger this
rule

Location where CRMML/CIFPartyPackage/InsertContact ->

information gets CRMML/CIFPartyPackage/InsertContact;
inserted a;b;CIFContactInsert;CIFContactInsertRs;IXML_Operation_ADD

Integration Object Integration Object

Reserved for for inbound Operation for the
future use for outbound business process
requests responses

112
 Dispatch Message Continued

Sample message before Dispatcher additions

<CRMML>
<CRMHeader>
.
</CRMHeader>

<CIFPartyPackage>
<InsertContactProfile>
<Contact>
.
</Contact>
</InsertContactProfile>
</CIFPartyPackage>
</CRMML

113
 Dispatch Message Continued
Sample message after Dispatcher additions

<CRMML SiebelFINSObjType=Envelope>
<CRMHeader SiebelFINSObjType=Header>
..
</CRMHeader> Additional
content added
by Dispatcher
<CIFPartyPackage SiebelFINSObjType=Body>
<InsertContactProfile
Content SiebelFINSOperation=IXMLOperation_ADD
added SiebelFINSIntObjName=CIFContactProfileInsert
here

SiebelFINSRespIntObjName=CIFContactProfileInsertRs
SiebelFINSMapOut=b SiebelFINSMapin=a>
Dispatcher adds
these attribute
<Contact> values which come
.. from the triggered
mapping rule
</Contact>
</InsertContactProfile>
</CIFPartyPackage>
<CRMML>
114
 Security Check
Invokes the SecurityCheck method of the UCM Security
Manager business service
Verify the SystemId in the message header against system
registrations
Return an error if the system is not recognized
For each method in the request, verify that the system has
the privilege required to execute the operation
Failed operations are removed from the request and
putProcess
WorkFlow into the Fail Security Req Prop property set
Dispatch
Message
Props

XML Hierarchy
Disp Req Prop
Security Check:
Fail Security FailSecurityHierarchy UCM Security Manager
Req Prop Security Check
XML Hierarchy
Security Req Prop

Security Check

115
 Security Check Continued

User properties map operation (inserted by the Dispatcher) to

privileges (assigned to systems in system registration)
Operation Name Privilege
IXMLOperation_ADD Insert
IXMLOperation_DELETE Delete
IXMLOperation_QUERY Query
IXMLOperation_UPDATE Update

In order to perform
this operation Requesting system must be
registered with this privilege

116
 Convert Request In
Invokes the XMLPropSetToPropSet method of the UCM Converter
business service
Each method is converted into the format required by the
Transaction Manager
Requesting System information is extracted from the header to
make it available to downstream business services
UsesProcess
WorkFlow integration object, CIF Envelope
Security Check
Props

XML Hierarchy
Security Req Prop
Convert Request In:
CIF System prop CIFSystemContainer UCM Converter
XMLPropSetToPropSet
Tran Req Prop XML Hierarchy

Transaction
Manager

117
 Convert Request In Continued

Sample converted method

Before After

<CIFPartyPackage.> <CIFPartyPackage.>
<InsertContactProfile.> <ListOfCIFContactProfileInsert.>
<Contact> <Contact>
<FirstName>Test</FirstName> First_spcName=Test
<LastName>Contact</LastName> Last_spcName=Contact
<Status>Active</Status> Status=Active
.. ..
</Contact> />
</InsertContactProfile> </ListOfCIFContactProfileInsert>
</CIFPartyPackage> </CIFPartyPackage>

118
 Transaction Manager
Invokes the Execute method of the UCM Transaction Manager
business service
Performs operations in message by invoking services
configured in its user properties
Convert Request
In
WorkFlow Process
Props

Tran Req Prop XML Hierarchy

Turn On Data Mgmt TurnOnDM
Turn On Survivorship TurnOnSE
StatusObject Transaction Manager:
Status Object
UCM Transaction Manager
NotificationHierarchy
Execute
Notification Req Prop
XML Hierarchy
Trans Resp Prop

Convert Request
Out

119
 Transaction Manager Operations
Supported Operations

Operation Description
Insert Insert into Source Data history tables with
status,Queued.
Update Insert into Source Data History tables with
status,Update.
Invoke Survivorship and Data Quality
Manager, if enabled.
Insert updated record directly into base
tables.
Query Query base tables.
Delete Locate record to delete based on user key
in message.
Store delete request in Source Data
History table.
Store historical
120
best version.
 Transaction Manager Configuration
Configuration Transaction Manager inputs in the workflow to
Enable data management and survivorship
TurnOnDM enable data management (default is false)
TurnOnSE--- enable survivorship (default is false)
Include a representation of the affected object in the response
to the requesting system
StatusObject (default is true)

121
 Convert Request Out
Invokes the PropSetToXMLPropSet method of the UCM Converter
business service
Generates a response that combines information from its inputs
Response will be converted to an XML hierarchy and then sent
back to the requesting system Transaction
Manager
WorkFlow Process
Props

Tran Resp Prop XML Hierarchy

Notification Req Prop NotificationHierarchy

CIFSystemContainer Transaction Manager:

CIF System Prop
UCM Transaction Manager
FailSecurityHierarchy Execute
Fail Securtiy Req Prop
XML Hierarchy
Conv Resp Prop

Realtim
e
Pub/Su
b
122
 Considerations
Error Handling
Configuring Inbound Server Communication

123
 Error Handling
The UCM Server Party Package Workflow (MQSeries Server
Transport Error Handling) demonstrates how error handling could
be added to a workflow.
Convert to
XML Hierarchy

Dispatch
Message

Security Error Handler: Error Handler:

Check UCM Converter: UCM Converter:
GenerateErrorPropSet GenerateErrorPropSet

Convert
Request In

Transaction
Manager End

124
 Error Handling Continued

The UCM Converter service includes a method to aid with error

handling

WorkFlow Process
Props

Error Code ErrorCode

Error Message ErrorMessageText

XML Hierarchy UCM Converter:

Disp Req Prop
GenerateErrorPropSet

XML Hierarchy
Error Hierarchy Prop

125
 Configuring Inbound Server
Communication
Your UCM environment must be configured for inbound server
communication
Configuration procedure depends on the system transport protocol
MQSeries Transport
HTTP Transport

126
Module 7: UCM Batch Data
Management Workflows

127
 Batch Data Management Workflows
Process data queued in Source Data and History tables
During initial data load
As a result of an insert request processed by Transaction
Manager
Separate workflows for contacts and accounts
UCM Contact Batch Data Management Workflow
UCM Account Batch Data Management Workflow
Each workflow performs data cleansing, matching, and real-time
publication (if enabled)

128
 UCM Contact Batch Data Management
Workflow
Performs the following actions on a single input record
Cleansing
Start
Matching
Real-Time publication
Query by ID

CDM Cleanse

Error match
Get Error Code CDM Match Auto

Matching
Match
?
Yes No

Process Process Process

Error? NoMatch AutoMatch Review

Realtim True Converter

Publishing
ePub/S
ub
False
End PubSub

129
 Data Cleansing
Performed by Cleanse method of the UCM Data Quality Manager
business service
Invokes third-party data cleansing software (if enabled)
Output is cleansed record
Start

Retrieve record data

Query by ID for processing

CDM Cleanse Perform data

cleansing

130
 Data Matching (Deduplication)
Performed by Match method of the UCM Data Quality Manager
business service
Three possible outcomes:
No match found
Match found that must be reviewed
Match found that can be processed automatically
Error match
Get Error Code CDM Match Auto
Match
?
Yes No

Process Process Process

Error? NoMatch AutoMatch Review

131
 Data Matching
Continued

All three match cases are processed by the same business service
method:
UCM Batch Manager HandleMatchCase

Process Process Process

NoMatch AutoMatch Review

132
 Data Matching
Continued

Thresshold settings determine how match is handled

Process Records with high match scores can be auto-

NoMatch merged,applying survivorship rules.

High Score - 90

Process Records with match scores in the middle will be

AutoMatch brought to the attention of the data steward for manual
review and resolution.

Low Score - 70

Process Records with low match scores will be left as seperate

Review records.

133
 Adjusting Threshold Settings
Configure user properties of the UCM Batch Manager business
service to adjust merging thresholds
User Property Default Value

Account Auto Threshold 90

Account Manual Threshold 70

Contact Auto Threshold 90

Contact Manual Threshold 70

134
 Publishing
Data changes can be published to subscribing systems
Result of auto merging of records
Result of insert of record (no match condition)
Publishing is addressed in the next module
Process Process
NoMatch AutoMatch

Convert data
Realtim True Converter to format for
ePub/S publishing
ub
False
PubSub Publish
Transaction

End

135
 Running Batch Data Management
Workflows
Create a Job

Configure Job Parameters

Run the Job

136
 1. Create a Job
In the Administration Server Management > Jobs view. Create a
new job to run the Workflow Process Batch Manager component/job

Select this
component/job

If job is repeating
specify repeat info in
job details

137
 2. Configure Job Parameters
Include
Search Specification: Identifies items to process
Workflow Process Name : Identifies workflow to invoke on each
item

138
 3. Run the Job
Once the job parameters are set, run job
Click start button to start
the job

139
Module 8: Publish and Subscribe

140
 Publish and Subscribe
Allows records inserted or updated in UCM to be published to
external systems
Distributes the best version customer profile
Systems registered as subscribers receive publication
Update: Contact123
messages Subscriber 1

UCM Update: Contact123

Subscriber 2

Update: Contact123
Subscriber 3

141
 Publish/Subscribe Modes
Publish and Subscribe framework supports two operative modes:

Real-time
Transaction are published as they happen
Each individual transaction is published to real-time
subscribers
Daily Batch Mode
Transactions are published once per day
Best version of record is published, individual transactions
are not

Systems can mix and match modes

Contact updates received in real-time
Account updates received from daily batch job
142
 Publication Messages
Publication messages are CRMML messages
FromSystem is UCM
ToSystem is subscriber

Message body content is constructed using integration objects

Real-time publishing uses the response integration object
associated with the particular UCM transaction
Batch publishing uses the integration object type specified as
input to the both Publish/Subscribe business service

143
 Workflows and Real-Time Publishing
UCM Workflows include steps to perform near real-time publishing
Decision step invokes publishing method if publishing is
enabled Process
NoMatch
Transaction Manager
response is published Result of match
is published
Process
PubSub AutoMatch

Realtim
PubSub
ePub/S
ub
Realtim Convert
ePub/S Request Converter
ub Out

Convert to PubSub
Hierarchy

End

UCM Server Party Package Workflow UCM Contact/Account Batch Data

Management Workflow
144
 Real-Time Publishing of Insert
Operations
Insert operations are processed in two steps:
Queue insert in SDH tables
Process insert, performing any necessary data cleansing and
matching
Both steps will generate a publish message
The subscriber system should only process the second message
Subscriber should ignore publish messages reporting that an
insert operation has been queued.

145
 Workflows and Batch Publishing
UCM Daily Publish workflow supports regular batch publishing

Start UCM Daily

End
Publish

146
 UCM Publish/Subscribe Business
Service
Workflows invoke this business service to perform publishing
Method
PublishMethod
Inputs
XML Hierarchy
Contains data to be published
Integration Object
Used to generate the publication message (batch
publishing only)

147
 Configuring Publishing: Real-Time
Set workflow user property to enable publishing

User Property names differ! Set to true

to enable publishing

148
Configuring Publishing: Real-Time
Continued

User Property names differ! Set to true

to enable publishing

149
 Configuring Publishing: Daily Batch
Configure a job to run the UCM Batch Publish Subscribe component
This job will run the Daily Batch workflow

Integration object
to use

Number of records to
processed in one
batch

150
 Subscribers
Registered systems can be configured as subscribers to receive
publication messages in real-time or batch
URL where publication
messages are sent

Required for batch

publishing

Updated each time a message

is published to this subscriber

151
Module 9: Wrap-Up

152
 Course Goal
To enable participants to integrate Siebel Universal Customer
Master (UCM) with external applications
Legacy systems
External databases
Was achieved by covering the following topics
Initial data loads into UCM at the data layer
Incremental data loads into UCM at the business logic layer
Managing outbound data flows via publish and subscribe
The CRMML messaging protocol for requesting UCM services

153
 Initial Data Loads into UCM at the
Data Layer
Batched data layer loads into UCM are a two-stage process
Loading from Siebel Interface table to Source Data and History
(SDH) tables using the normal Siebel EIM process
Loading from SDH tables to Siebel base tables using UCM Batch
Manager

Interface tables
Are loaded using non-Siebel software
Operated on by the Enterprise Integration Manager to transfer
data to SHD tables

The UCM Batch Manager is used to move data from SDH tables to
Siebel base tables
With data cleaning and matching to produce best version
data
Data is then accessible from the Siebel applications standard
business logic displays 154
 Data Loads into UCM at the Business
Layer
Data loads at the business logic layer employ two distinct process
managers:
The Server Party Package Manager
The UCM Batch Manager
The Server Party Package Manager:
Reads input data
Identifies the data source
Dispatches the incoming Insert message
Queues the data on the SDH tables
The UCM Batch Manager is used to move data from SDH tables to
Siebel base tables
With data cleansing and matching to produce best version
data
Data is then accessible from the Siebel applications standard
business logic displays
155
 Managing Outbound Data Flows
Records that have been inserted or updated can be selectively
published to subscribers
Making best version record available throughout the data
ecosystem
Real-time or batch mode
Outbound publication service is performed by
The Server Party Package Manager for Query, Update, and
Delete operations
The UCM Batch Manager for Insert Operations

156
 The CRMML Messaging Protocol
XML based mark-up language used by UCM
Inbound server requests
Insert
Query
Update
Delete
Outbound server features
Publish and Subscribe

Formatted to provide
Requesting external system ID
Service(s) being requested
Pointers to predefined UCM integration objects

157
 Some Guidelines
Initial data for UCM should be cleaned outside (Before
loading into UCM)

EIM may be more effective to load data into UCM from

external systems for performance reasons

Batch publish method is more effective when send data

from UCM to external systems

UCM is not complete Customer data store, UCM can not

store complete customer information. UCM Contact table
has limited columns only; this data is primarily for
referencing customer data between multiple external
systems

UUID is not numeric and big in size (about 20+ characters

string), so it may not be useful as general purpose contact
Id
158
Thank You

159