This action might not be possible to undo. Are you sure you want to continue?
© Tata Consultancy Services ltd.
December 1, 2010
Day 1 Module 1 : Review of CDI Architecture Module 2 : Overview of Application Integration Module 3 : Initial Data Loads into UCM Module 4 : The UCM Batch Workflow and SDH tables Day2 Module 5 : UCM Messaging and CRMML Module 6 : UCM Server Party Package Workflows Module 7 : The UCM Batch Management Workflows Module 8 : Publish and Subscribe Module 9: Wrap-up
Module 1: Review of Siebel CDI Architecture
The CDI Solution The Customer Data integration(CDI) solution consists of three products CDI UCM SDQ UAN 4 .
What it does? 1. Acts as a central repository of customer data. Keeps customer information in synch with other systems in the enterprise. some with sales data. 2. some others purely with demographics and contact data. thus giving a “360° real-time true picture” of the customer.UCM = Universal Customer Master What is UCM? A Siebel horizontal application which is the central master database/application for storing any and all customer information . you have hundreds of various proprietary and legacy applications which deal with different parts of the customer information . In a typical large enterprise. some deal with service data. UCM has the capability to synchronize and bring together all this data to give a single unified view of the customer as a whole. 5 .
E. Provide a staging area for data quality. both storing the customer’s contact number. In an enterprise.SDH = Source Data and History tables The central components of the CDI solution are the Source Data and History (SDH) tables associated with UCM 1. UCM will then in turn use it’s data quality rules to sort out the “good” updates from the “bad” ones and apply the “good” ones to the base tables while storing the “bad” ones too for audit purposes. Provide a mechanism to maintain audit-trace of changes happening to the customer record. suppose you have a .NET based sales application and a legacy service application. but the one from the 6 . External systems first integrate with these SDH tables when sending data to UCM. In cases where both applications are sending simultaneous updates to this field. 2.g. both updates will be stored in the SDH tables. you can define the service application with a higher confidence level for updating this field. If the enterprise feels that it’s service department is more likely to have a more current contact number of the customer.
Data and history tables are added.UCM Data Model Is an extension of Siebel Data model Source. Field BC BO DL SDH Column SDH table 7 . UL List Column/ Control Applet View Screen Application BL .
Merged 4. Cleansed 5. Queued 6. Unmerge 7. History 2.Source Data and History types SDH data contains several types of records distinguished by a Type value— Each record is one of the following types: 1. 8 . Source 3. Batch A single transaction typically results in the creation of multiple Source data and history records.
Source Data and History : Data schema Provide the infrastructure necessary to support child structures Example : addresses for contacts S_ORG_EXT Base table Source data History Table S_UCM_ORG_EXT S_UCM_CONTACT S_CONTACT S_CIF_EXT_SYST S_UCM_ORG_CHLD External System information table S_UCM_CON_CHLD Source Data History Child table 9 .
Systems registered 2. Survivorship configured 10 .UCM Application Integration Revolves around integrating to these SDH tables First basic CDI configuration must be performed 1. Data quality configured 3.
CDI Configuration External System Registration Data Quality Configuration Survivorship Rules 11 .
Query 2. a system’s privileges and accessibility can be defined 1. Insert 3.External System Registration Every operational application that connects to UCM must register with UCM. Delete Defines grant access to Siebel UCM Privileges are granted at the BO layer 12 . After registration. Update 4.
How to register an external application with UCM Use the system registration view 1. Below System registrations applet Select New and assign BO privileges in theSystem Privileges.MQSeries) 4. Click New 3. Enter a system name. id and protocol type(HTTP. Select Administration –Universal Customer Master > System registrations 2. 13 .
Registering an External Application 14 .
External System referencing: CrossReferencing
Cross-referencing stores source identification information for data in the UCM UCM stores identification information about external systems that have been registered. Cross-referencing provides a one-to-many mapping of customer records across multiple systems Allows UCM to keep track of the same customer record across multiple systems A single UCM record maps to one or many records in any registered application UCM supports cross-referencing for the following objects: 1. Contacts 2. Accounts 3. Households
Cross-reference entries are created automatically with the creation of every new account, contact, or household record External systems send requests to insert a record If there is an existing cross-reference entry, the entry is updated If there is no existing cross-reference entry, a new record is inserted The system ID is the value assigned when the system was registered and system name is the associated name There are multiple possible external Ids to account for situations where multiple records in an operational application map to a single record in UCM Only the first is stored in the ID tag in a message request, by default
External System Referencing: Universal Unique ID (UUID)
Another mechanism to track the same record across multiple applications Instead of keeping track of the system IDs., UCM publishes a UUID to the operational application The UUID is not exposed in the user interface Is stored in a column of the S_PARTY table Can be exposed using Siebel Tools Many customers use cross-referencing and the UUID at the same time
Address validation and standardization 2. Data Cleansing 2. E. from our previous case. assuming we have the passport number stored in both the . if we decide that the key attributes of a unique customer are : first and last names and say the passport number.Data Quality Options Data quality consists of 1. Data Matching(De-Duplication) Data Cleansing 1.NET and J2EE application. So. data matching in UCM will be able to identify that customer # 118 .contact and prospect records Identify duplicates and merge them based on certain key attributes. Abbreviations Data Matching identifies possible duplicate records for account. Capitalization 3.g.
Specify data quality settings 4.How to configure Data Quality To configure data quality : 1. 19 . Enable the data quality component group 2. Set data quality parameters 3. Specify data quality settings: Site Map > Administration —Data quality > Data quality settings During integration cycle disable deduplication to simplify the environment.
and the surviving record. with History as the type. the records can be evaluated and: Merged with existing records Introduced as new UCM records (promoted) For records that are merged.What to Do with Duplicate Records Once suspect duplicate have been identified. in the Source Data History table Merges parent records using survivorship rules to generate a new merged surviving parent record Updates the parent surviving record with the new merged surviving record Deletes the duplicate record 20 . UCM: Stores the duplicate record. with Merged as the type.
Survivorship Rules Customer data originates in multiple systems Conflicts arise when different sources have different data for the same attribute For eg: two systems have different addresses for customer The role of UCM is to resolve these conflicts. which ensures that the best version record survives Survivorship rules are an automated mechanism for consolidating customer data to produce a best-version record Evaluates updates from each registered system Applies rules to determine which is the best value for each attribute Can be constructed by a system integrator or a CDI data steward Three comparison methods Recent 21 .
Survivorship Rules and Match Scores UCM allows two thresholds to be set depending on the match scores from the matching process Records with high match scores can be auto-merged.90 Records with match scores in the middle will be brought to the attention of the data steward for manual review and resolution Low Score .70 Records with low match scores will be left as separate records 22 . applying survivorship rules High Score .
Setting Match Thresholds Auto-threshold: Determines the high matching score threshold Account Auto Threshold Contact Auto Threshold Manual threshold: Determines the low matching score threshold Account Manual Threshold Contact Manual Threshold Can be constructed by system integrator or CDI data steward Can be set by a system integrator or a CDI steward 23 .
Survivorship Rule Structure Employ comparison criteria to determine if a new value for a given attribute coming from any source is better than the existing value in the best-version record Three comparison methods: Recent History Source Multiple rules can be constructed for each object (account or contact) but only one rule can be active at a time Use attribute groups A group of fields that survive or get overwritten together Makes survivorship rule creation less cumbersome Each field within an object can only be included in a single attribute group To avoid survivorship rule conflicts 24 .
Building Survivorship Rule Three primary steps: Create an attribute group for the survivorship rule Create the survivorship rule logic Activate the survivorship rule 25 .
Setting Survivorship Rules 26 .
Unmerge For records that have been merged Surviving record--.record that is merged and ceases to exist If merged records are unmerged after changes have been made to be the merged (surviving) record All changes and updates since the merge will be applied to the surviving record The victim record will be resurrected from the Source Data history table Merge Unmerge A A2 A1 B A2 B Changed to merged record 27 .record that remains after the merge Victim record ----.
Outbound Data Flow: Publish and Subscribe Publish and subscribe allows records inserted or updated in UCM to be published to external systems Publication can be targeted to any system registered with UCM The mechanism for publication is a Siebel Workflow Workflow policies track changes to UCM records and flag them for production Then Workflow processes collect the appropriate information and guarantee publication of changes to specified (subscribed) systems Publish and subscribe requires that publication parameters be specified Can be real-time or daily-batch 28 .
Module 2: Overview Of Application Integration 29 .
Application Integration and the UCM SDH tables UCM integration extends the EAI toolset by transferring data to SDH tables prior to placing data into Siebel base tables SDH tables provide a queuing station for processing incoming records Records can be analyzed for future processing Data must be processed at this layer before entering standard Siebel base tables This is the source of integration process differences between UCM and the standard Siebel CRM Enterprise product 30 .
Data Transfers to SDH tables Data transfers to SDH tables use traditional methods Data layer data loads via the EIM Business layer data loads via standard integration application procedures Data transfers paths to SDH tables analogous to standard CRM case No data cleansing or matching Business Layer Data Loads SDH Tables Base Tables Data Layer Data Loads 31 .
Data Transfers to Siebel Base Tables Are independent of how data is placed on SDH tables Data layer Business logic layer During this transfer(insert) data quality is activated Cleansing and matching Application of survivorship rules Implemented as a UCM workflow process– The UCM Batch Manager 32 .
UCM Service Requests External Systems request services from a UCM application Insert Query Update Delete UCM must determine which external system is requesting the service To validate privileges The System ID must be supplied in addition to the traditional inputs even if the UCM application is not physically connected to an external operational system 33 .
UCM External System Parameters Request for service can only be accepted from a known source Registered system System ID System Name The records to be operated on also have UCM specific information associated with them Unique external record ID Unique user ID Transfer code describing data transfer mechanism 34 .
Inbound Data flow for the business layer Request for UCM application services Submitted through a (CRMML) input file Security service checks privileges of requesting external system Transaction service executes operations Siebel Adapter Through Dispatch Service To SDH tables Security and Transaction Services Input data (CRMML) Outbound Data 35 .
CRMML Input Files Contain Information about external system --.system ID Service being requested Methods and arguments to be used User does not have to build the integration objects Internal integration objects provided by the UCM application The CRMML file directs the UCM application on which the integration object to use User does not have to specify UCM specific record information The UCM application will make these assignments when placing records on SDH tables 36 .
Update. Query and Delete privileges If the system is not registered data will not be processed Transaction manager will perform the requested operations Uses an internal UCM integration object to obtain the data necessary to process a request 37 .Security and Transaction Services The security service checks the system ID specified in the input (CRMML) file for Insert .
Implementing Inbound Data Flow at the Business Layer Inbound Requests are processed by a workflow process HTTP Queue Based File Transport Protocol specified when system is registered Siebel Adapter Through Dispatch Service To SDH tables Security and Transaction Services Input Data (CRMML) Server Party Package 38 Outbound Data .
a user must: Prepare input(CRMML) file Run (appropriate) Server Party Package workflow to place records of SDH tables Run UCM Batch workflow to activate data quality and place data on Siebel base tables The UCM application supports an outbound publish and subscribe facility Insert and Update records can be published to external system subscribers Individually or as a group In batch or real time mode 39 .Inbound/Outbound Processing UCM has automated much of the inbound processing --.
Module 3: Initial Data Loads into UCM 40 .
Populating UCM Data Use business layer imports to enter . update or delete small amounts of data in the base tables Use the EIM server task to populate the SDH tables ( through interface tables) Use the UCM batch manager to transfer data from SDH tables to Siebel Base tables Non-Siebel Cleanse and match UCM Database loading tool External data – may be cleansed Interface table 41 UCM SDH tables Siebel Base tables .
Staging Tables The UCM application uses two sets of staging tables before making data available to the Business Logic layer through Siebel Base tables The EIM staging tables: Data is staged to be transformed into a Siebel standard format The Source .Data and history tables: Data is staged for cleansing and matching 42 .
data and history tables or preprocessed before loading into EIM tables Cleansing and matching during data import adds loading latency ---time overhead If the Siebel Universal Connector is being used for cleansing and/or matching .Data Quality Options for Initial Loads Data may be cleansed and matched while being transformed from the source . it will need to be integrated into the application 43 .
Siebel Base Tables User data is stored in one or more Siebel base tables Relationships between base tables (referential integrity) rely upon primary and foreign keys based on ROW_IDs ROW_ID is the system primary key for every base table ROW_ID is a system-generated value PK PK FK FK S_BU S_ORG_TXT S_ADDR_ORG 44 .
user keys are used to uniquely identify a row for EIM Primary and foreign keys based on ROW_ID are used for systemwide referential integrity PK ROW_ID User Key FK BU_ID Name LOC S_ORG_EXT S_UCM_BU 45 .User Keys Based on multiple columns.
UCM Data Loads Loading Data into the EIM Tables Loading Data from the EIM Table to the SDH Tables Loading Data from the SDH Table to the Base Tables 46 .
or merged into Source Data and History (SDH) Staging area for data imported by EIM EIM tables do not have ROW_Ids assigned External Database S_UCM_ORG_EXT EIM_UCM_ORG Interface Table External Data SDH Tables 47 S_UCM_CONTACT .Interface Tables Staging area for data to be imported. updated.
update. or delete statements to update columns in EIM tables External Data EIM_UCM_ORG Non-Siebel process 48 .Populate Interface Tables Populate source data into interface tables using a non-Siebel process Use database vendor utilities to load data from flat files Use SELECT INTO to pull data from other SQL databases Write programs or stored procedures to cleanse or preprocess and insert rows Use direct SQL insert.
Batch number IF_ROW_STAT --.Required Interface Table Columns In addition to standard non-nullable columns IF_ROW_BATCH_NUM ---.SDH type (example : batch for initial loads) EXT_SYSTEM_NUM – External System ID UCM_EXT_ID --.A unique external record identifier (usually assigned by the external system) 49 .Status ROW_ID – Row ID The UCM application also requires a non-null inputs UCM_UID – A unique UCM record identifier UCM_TYPE_CD --.
Source History Data (SDH) and EIM Schema UCM Source Data History and EIM Tables S_UCM_ORG_EXT S_UCM_CONTACT S_UCM_ORG_CHLD S_UCM_CON_CHLD EIM_UCM_ORG EIM_UCM_CON EIM_UCM_ORG_CHLD EIM_UCM_CON_CHLD Source and History table for accounts Source and History table for contacts Source and History table for child objects of account Source and History table for child objects of contact EIM interface table for S_UCM_ORG_EXT EIM interface table for S_UCM_CONTACT EIM interface table for S_UCM_ORG_CHLD EIM interface table for S_UCM_CON_CHLD 50 .
Referential Integrity User key column mappings in interface tables are used to resolve ROW_Ids for SDH tables. maintaining referential integrity User key PK(ROW_ID) Acme 1-8D Interface Table 1-8D FK(ROW_ID) UCM SDH Tables 51 .
Enterprise Integration Manager (EIM) Server task that manages the exchange of data between Interface tables and Source Data History tables S_UCM_ORG_EXT EIM_UCM_ORG S_UCM_CONTACT External Data 52 .
Running the Enterprise Integration Manager The EIM server task is run as a workflow from Administration – Server > Jobs 53 .
loads a dictionary and executes EIM steps 54 .ifb file. reads an .Run as EIM Task with Parameter Settings Creates an operating system (OS) process that writes log entries.
Configuration (. order and type of processing desired Should be placed in c:\Susrvr\siebsrvr\ADMIN For imports [Siebel Interface Manager] PROCESS = IMPORT EIM_UCM_CON [IMPORT EIM_UCM_CON] TYPE = IMPORT TABLE = EIM_UCM_CON BATCH = 100 ONLY BASE TABLE = S_UCM_CONTACT 55 .ifb) file Role for initial data loads Specifies the batch .
Use Log Information to Troubleshoot Navigate to Administration – Server > Tasks Error information in Task view or log file 56 .
Verify Data Load into SDH Tables Navigate to Administration – Universal Customer Master > Contact (Account) > Source Data and inspect the Source Data table 57 .
UCM Batch Manager Transfers data from SDH tables to Siebel Base tables Implemented by separate workflows for Accounts and Contacts UCM Account Batch Data Management workflow UCM Contact Batch Data Management workflow S_UCM_ORG_EXT S_ORG_EXT SDH table Base Table SDH table Base column 58 .
Running the UCM Batch Manager The UCM Batch Manager is run as a workflow from AdministrationServer > Jobs 59 .
Verify the Data Load into Siebel Base Tables Navigate to Site Map > Contacts (Companies) > Contacts(Accounts) List and inspect the list 60 .
Determining Data Mapping Need to analyze and determine : Which Siebel base table columns will store external source data Which interface (EIM) tables and Source Data History (SDH) tables will be used to import from source to destination Source (External Where data ends up How to get there Destination (base) Interface (EIM) SDH 61 .
determine EIM-to-SDH and SDH-base mappings Source Entity Customer Customer Source Attribute Name Phone Address EIM Table and Column SDH Base Table Table and Column S_ORG_EXT S_ORG_EXT S_ADDR_ORG S_ADDR_ORG S_ADDR_ORG Base Column NAME MAIN_PH_NUM ADDR CITY ZIPCODE Interface (EIM) 62 SDH (UCM) .How to Get There After verifying and documenting the source-to-base data mappings.
Table Column Mappings Can be determined using Siebel Tools EIM Interface tables Source Data History Tables Source Data History Tables Base tables 63 .
Module 4: The UCM Batch Workflow and SDH Tables 64 .
Data transfer to Siebel Base Tables Data transfer for child objects 65 .The UCM Batch Manager and SDH Tables This module will discuss two aspects of UCM Batch Manager operations on SDH tables.
The UCM Batch Manager Batch processes insert requests queued on the Source. Data .and History (SDH) tables Initial data loads (legacy systems) Incremental inserts Moves records from SDH tables to Siebel base tables Example: S_UCM_CONTACT S_CONTACT Two flavors of batch workflows for accounts and contacts UCM Account Batch Data Management Workflow UCM Contact Batch Data Management Workflow 66 .
UCM Account Batch Data Management Workflow Start Query by Id Get Error Code Error? Process NoMatch CDM Cleanse CDM Match Match id? Process AutoMatch Process Review Error? Converter PubSub End 67 .
UCM Batch Workflow Steps Query By Id CDM Cleanse CDM Match Process NoMatch Process AutoMatch Process Review 68 .
Query By ID Invokes the Query method of the EAI Siebel Adapter business service Reads an object to be inserted from a SDH table Outputs a Siebel message Workflow Process Props SDH int Obj Name OutputintObjectName PrimaryRowId Object Id SiebeMessage Siebel Message Query By Id 69 .
CDM Cleanse Invokes the Cleanse method of the UCM Data Quality Manager business service Reads a Siebel Message containing object to be inserted from SDH table Outputs a cleansed XML representation of the original Siebel Workflow Process Message Props Base int Obj Name IntObjectName SiebeMessage Siebel Message XML Hierarchy Cleansed Prop CDM Cleanse 70 .
CDM Match Invokes the Match method of the UCM Data Quality Manager business service Reads an XML representation of a Siebel Message containing an object to be inserted from SDH table Output indicates if record is a duplicate and the auto-merge Workflow level Process Props Cleansed Prop XML Hierarchy Number of AutoMatch NumAutoMatch CDM Match MatchingrowId Cleansed Prop 71 .
insert record Potential match --.To Merge or Not to Merge The workflow splits the output of the match process into three cases (CaseType) No match ---.activate survivorship rule to determine if merge is appropriate Transfer record for manual resolution 72 .
Process NoMatch Is activated in the event that the incoming record is not scored as a match Record is inserted Invokes the HandleMatchCase method of the UCM Batch Manager business service Workflow Process Props Matching Id Object Type Turn On Survivorship Cleansed Prop Processed Prop XML Hierarchy XML Hierarchy CaseType ObjectType TurnOnSE Process NoMatch 73 .
is either merged or inserted Invokes the HandleMatchCase method of the UCM Batch Workflow Process Manager business service Props CaseType Matching Id Object Type Turn On Survivorship Cleansed Prop Processed Prop MatchingRowId ObjectType TurnOnSE XML Hierarchy XML Hierarchy Process AutoMatch 74 .Process AutoMatch Is activated in the event that the incoming record is scored as a match Record evaluated by survivorship rule --.
Process Review Is activated in the event that the incoming record is scored as a match but not resolved by survivorship rule Record is posted in Duplicate (Accounts) view Invokes the HandleMatchCase method of the UCM Batch Manager business service Workflow Process Props CaseType Object Type Cleansed Prop ObjectType XML Hierarchy Process Review Processed Prop XML Hierarchy 75 .
Post Process Activities In the event that the record is resolved through an insert or a merge A real-time message can be published In the event record is queued for manual resolution The workflow is terminated and no information is published(in real--time mode) 76 .
Address for Contacts Contact business objects come preconfigured with address child busniess components 77 .
The UCM Data Model and Child components All UCM child components for each business object (Account and Contact) are stored in one table The explicit component object type is configured in each UCM business component to distinguish the child components Busniess Layer Field BC BO Data Layer SDH Column SDH table 78 .
Address for Contacts Contact business objects come preconfigured with address child business components Primary Business Component UCM Contact SDH UCM Contact SDH UCM Contact Address SDH Business Object Business Component S_UCM_CONTACT S_UCM_CON_CHILD Table 79 .
Source Data and History : Data schema Provide the infrastructure necessary to support child structures Example : addresses for contacts S_ORG_EXT Base table S_CONTACT S_UCM_ORG_EXT Source data History Table S_UCM_CONTACT S_CIF_EXT_SYST S_UCM_ORG_CHLD External System information table S_UCM_CON_CHLD Source Data History Child table 80 .
Creating UCM SHD Child Business Components The UCM data model can easily be extended to include custom SDH components 81 .
Example: Address for Contacts The UCM SDH Contact Address business component is UCM Contact Address Source Data and History 82 .
Custom Child Business Components Name Business Components and Tables Assign Data Fields Configure UCM Fields Configure UCM Batch Workflow 83 .
Name Business Components and Tables Copy the Siebel base business component and rename it Siebel suggested standard child business component names: “UCM” + <Parent BC Name> + <Child BC Name> + “SDH” Copy of UCM Contact Address Source Data and History 84 .
Assign Data Fields Change the predefined column names (‘ATTRIB_N’) for the business component to the UCM SDH table Name field 85 .
Configure UCM Fields There are five UCM specific fields that must be configured Component Field Name Object Name Column OBJECT_NAME Predefault Value Party Object name (Account/Conta ct) UCM Party base table Equal to child name Parent Row Id Field identifier Object Table Object Type Par Rpw Id UCM UId OBJECT_TABLE OBJECT_TYPE PAR_ROW_ID UCM_UID 86 .
Configure UCM Batch Workflow The OBJECT_TYPE is used for the search specification in the UCM Batch Workflow parameter list Search Specification = Object Type Example for Addresses: [Object Type] = ‘address’ 87 .
Module 5: UCM Messaging and CRMML 88 .
update.CRMML An XML based markup language for data exchange and remote business process invocation Used for communication with UCM Inbound requests (insert. query. delete) Responses Publishing messages Request Back-Office Application UCM Response Publication These exchanges use CRMML messages 89 .
Example: Insert Contact Profile <CRMML> ‘System’ requesting the <CRMHeader> info <FromSystem> <SystemInfo> <SystemId>System123</SystemId> <SystemName>External App</SystemName> </SystemInfo> </FromSystem> <ToSystem> … </ToSystem> <MessageDescriptor> … </ MessageDescriptor> </CRMHeader> Requested transaction. <CIFPartyPackage> or ‘method’ <InsertContactProfile> <Contact> Method “argument” <FirstName>Test</FirstName> <LastName>Contact</LastName> …. </Contact> <InsertContactProfile> </CIFPartyPackage> </CRMML> 90 .
CRMML and Integration Objects Integration Objects Define the structure of the CRMML message Are used as templates to create memory-resident instances of data for processing by business services UCM provides integration objects for different parts of the CRMML message Integration Object Description Type Envelope Integration Object Dispatcher map integration object UCM integration objects Represents CRMML header structure Contains rule sets for mapping from method to integration objects Represent the method argument (Contact. Account. etc) 91 .
CRMML Header Contains Information identifying sending and receiving systems Used by Universal Customer Master for security verification Message descriptor identifying the message <CRMHeader> <FromSystem> <SystemInfo> <SystemId>System123</SystemId> <SystemName>External App</SystemName> </SystemInfo> </FromSystem> <ToSystem> … </ToSystem> <MessageDescriptor> … </ MessageDescriptor> 92 .
FromSystem SystemId is matched against registered systems as part of UCM’s security check SystemId used for security verification <Systeminfo> <SystemId>123-456-789</SystemId> <SystemRole>Insurance App</SystemRole> <SystemName>Client System</SystemName> </Systeminfo> 93 .CRMML Header : System Info Used to describe both the sending and receiving systems <FromSystem> .receiver For inbound messages.sender <ToSystem> .
CRMML Header : Message Descriptor Identifies the message MessageId : Unique ID for the message Timestamp: when message was issued TimeToLive: how long (after timestamp) message is valid TransactionScope : reserved for future use <MessageDescriptor> <MessageId>1T3D2005083115:45:32</MessageId> <TransactionScope>any</TransactionScope> <Timestamp>Sunday</Timestamp> <TimeToLive>OneYear</TimeToLive> </MessageDescriptor> 94 .
CRMML Header : Summary CRMHeader element has the following structure <FromSystem> <SystemInfo> <CRMMLHeader> <ToSystem> <SystemInfo> <CRMML> <MessageDescriptor> <Body> 95 .
CRMML Header: Integration Object The CIF Envelope Integration Object is used to process the CRMML Header 96 .
or delete a database object Outbound response to a requested transaction Outbound publication messages to be sent to an external subscribing application Each message element includes: Software package – must be CIFPartyPackage Method --.CRMML Body Contains one or more message elements representing: Inbound requests from an external applications to insert. query. update.the requested action Argument – the argument to the action <SoftwarePackage> <Method> <Argument> <CIFPartyPackage> <InsertContact> <Contact> 97 .
which contain Fields <CIFPartyPackage> <InsertContact> <Contact> <LastName> </LastName> <FirstName> </FirstName> ……. Integration Objects contain Integration Components.Mapping to Integration Objects UCM methods and arguments are mapped to integration objects according to a set of mapping rules These mapping rules define: What methods are allowed What arguments those methods take Methods contain Arguments. which contain Integration Component Fields CIFContact Contact FirstName LastName 98 .
Mapping to Integration Objects
The mapping rules are defined in user properties of a mapping integration object The default mapping object is CIFDispMap
CRMML/CIFPartyPackage/InsertContact -> CRMML/CIFPartyPackage/InsertContact;a;b;CIFContactInsert;CIFContactInsertRs; IXML_Operation_ADD Integration Object for inbound requests Integration Object for outbound responses
CRMML Structure: Summary
Structure of a base CRMML message
Complex CRMML Messages
Arguments may contain sub-arguments For example: Contacts contain addresses These sub-arguments are represented by sub-elements in the CRMML message <Contact_
</ListOfContact_INSPersonalAddress> </Contact> </InsertContactAddress> </CIFPartyPackage> </CRMML> 102 .</ INSPersonalStreetAddress> <INSPersonalCity>York</INSPersonalCity> <INSPersonalState>AL</INSPersonalState> <INSPersonalPostalCode>78221</INSPersonalPostalCode> <INSPersonalCountry>USA</INSPersonalCountry> </Contact_INSPersonalAddress> <Contact_INSPersonalAddress IsPrimaryMVG=“N”> <INSPersonalStreetAddress>50 Center St.</INSPersonalStreetAddress> …….Example: Insert Contact Address <CRMML> <CRMHeader> ….</CRMHeader> <CIFPartyPackage> <InsertContactAddress> <Contact> <FirstName>Test</FirstName> <LastName>Contact</LastName> <ListOfContact_INSPersonalAddress> <Contact_INSPersonalAddress IsPrimaryMVG=“Y”> <INSPersonalStreetAddress>100 Main St.
Module 6: UCM Server Party Package Workflows 103 .
based on integration object definitions Execute requested operation Return response to requesting system Publish (real-time) to subscribing systems 104 .Party Package Workflows Handle incoming requests Check source system privileges Create memory-resident instances of data for processing by business services.
Party Package Workflows continued Include Workflow Process Name UCM Server Party Package Workflow (File Transport) UCM Server Party Package Workflow (MQSeries Server Transport) UCM Server Party Package Workflow(MQSeries Server Transport Error Handling) Description Reads CRMML message from a file and writes response to a file (for testing) Used for HTTP and MQSeries message transports. Models error handling. 105 . Also for HTTP and MQSeries transports.
UCM Server Party Package Workflow (File Transport) Workflow is composed of the following steps: Start Read From File Convert to XML Hierarchy Dispatch Message Security Check Convert Request In PubSub Transaction Manager Realtime Pub/Sub Convert Request Out End Write To File Convert to Hierarchy 106 .
Read From File Invokes the Receive method of the EAI File Transport business service Reads contents of a file into string Input file should contain a valid CRMML message WorkFlow Process Props Input File Name File name Read From File: EAI File Transport Receive Start XML <value> Convert to XML Hierarchy 107 .
Convert to XML Hierarchy Invokes the XMLDocToXMLHier method of the XML Hierarchy Converter business service Converts an XML string into an XML hierarchy object WorkFlow Process Props XML <value> Convert to XML Hierarchy: XML Hierarchy Converter XMLDocToXMLHier Read From File Core Req Prop XML Hierarchy Dispatch message 108 .
Dispatch Message Invokes the DispatchMessage method of the UCM Dispatcher business service Verifies structure of CRMML message Acts information to message components that is used by downstream business services Convert to XML WorkFlow Process Props Hierarchy Core Req Prop XML Hierarchy Dispatch Message: UCM Dispatcher DispatchMessage Disp Req Prop XML Hierarchy Security Check 109 .
Dispatch Message Continued Uses two integration objects. to represent the message Integration object used to process body Integration object used to process header User Property Name DispatcherMapNa me Value CIFDispMap XMLEnvIntObjectN CIF Envelope ame 110 . specified as user properties of the business service.
CIF Envelope Integration Object Defines the allowed structure of the CRMML message header. 111 .
CIFContactInsertRs.CIFContactInsert. a.Dispatcher Map Integration Object Contains a set of rules to map methods to integration objects used to process those methods Each rule is a user property of the mapping integration object Name identifies the message content that must be matched Value specifies the information that should be added to the method when a mapping is found Content to match in order to trigger this rule Location where information gets inserted CRMML/CIFPartyPackage/InsertContact -> CRMML/CIFPartyPackage/InsertContact.IXML_Operation_ADD Reserved for future use Integration Object for inbound requests Integration Object for outbound responses Operation for the business process 112 .b.
Dispatch Message Sample message before Dispatcher additions Continued <CRMML> <CRMHeader> ……. </Contact> </InsertContactProfile> </CIFPartyPackage> </CRMML 113 . </CRMHeader> <CIFPartyPackage> <InsertContactProfile> <Contact> ….
Dispatch Message Sample message after Dispatcher additions Continued <CRMML SiebelFINSObjType=“Envelope”> <CRMHeader SiebelFINSObjType=“Header”> …. mapping rule </Contact> </InsertContactProfile> </CIFPartyPackage> <CRMML> 114 ... </CRMHeader> <CIFPartyPackage SiebelFINSObjType=“Body”> <InsertContactProfile SiebelFINSOperation=“IXMLOperation_ADD” Content added SiebelFINSIntObjName=“CIFContactProfileInsert” here Additional content added by Dispatcher SiebelFINSRespIntObjName=“CIFContactProfileInsertRs” Dispatcher SiebelFINSMapOut=“b” SiebelFINSMapin=“a”> adds these attribute <Contact> values which come from the triggered ….
verify that the system has the privilege required to execute the operation Failed operations are removed from the request and Dispatch put into WorkFlow Process the ‘Fail Security Req Prop’ property set Props Message Disp Req Prop Fail Security Req Prop Security Req Prop XML Hierarchy FailSecurityHierarchy XML Hierarchy Security Check: UCM Security Manager Security Check Security Check 115 .Security Check Invokes the SecurityCheck method of the UCM Security Manager business service Verify the SystemId in the message header against system registrations Return an error if the system is not recognized For each method in the request.
Security Check Operation Name IXMLOperation_ADD IXMLOperation_DELETE IXMLOperation_QUERY IXMLOperation_UPDATE Privilege Insert Delete Query Update Continued User properties map operation (inserted by the Dispatcher) to privileges (assigned to systems in system registration) In order to perform this operation Requesting system must be registered with this privilege 116 .
Convert Request In Invokes the XMLPropSetToPropSet method of the UCM Converter business service Each method is converted into the format required by the Transaction Manager Requesting System information is extracted from the header to make it available to downstream business services Security WorkFlow Process Uses integration object. CIF Envelope Check Props Security Req Prop CIF System prop Tran Req Prop XML Hierarchy CIFSystemContainer XML Hierarchy Convert Request In: UCM Converter XMLPropSetToPropSet Transaction Manager 117 .
</Contact> </InsertContactProfile> </CIFPartyPackage> Continued After <CIFPartyPackage….Convert Request In Sample converted method Before <CIFPartyPackage….> <Contact> First_spcName=“Test” Last_spcName=“Contact” Status=“Active” …...> <Contact> <FirstName>Test</FirstName> <LastName>Contact</LastName> <Status>Active</Status> …. /> </ListOfCIFContactProfileInsert> </CIFPartyPackage> 118 .> <InsertContactProfile….> <ListOfCIFContactProfileInsert….
Transaction Manager Invokes the Execute method of the UCM Transaction Manager business service Performs operations in message by invoking services configured in its user properties WorkFlow Process Props Tran Req Prop Turn On Data Mgmt Turn On Survivorship Status Object Notification Req Prop Trans Resp Prop XML Hierarchy XML Hierarchy TurnOnDM TurnOnSE StatusObject NotificationHierarchy Convert Request In Transaction Manager: UCM Transaction Manager Execute Convert Request Out 119 .
Insert updated record directly into base tables.Update. Locate record to delete based on user key in message. Store historical best version. Invoke Survivorship and Data Quality Manager. 120 Query Delete .Queued. Query base tables. if enabled. Store delete request in Source Data History table. Insert into Source Data History tables with status.Transaction Manager Operations Supported Operations Operation Insert Update Description Insert into Source Data history tables with status.
Transaction Manager Configuration Configuration Transaction Manager inputs in the workflow to Enable data management and survivorship TurnOnDM– enable data management (default is false) TurnOnSE--.enable survivorship (default is false) Include a representation of the affected object in the response to the requesting system StatusObject (default is true) 121 .
Convert Request Out Invokes the PropSetToXMLPropSet method of the UCM Converter business service Generates a response that combines information from its inputs Response will be converted to an XML hierarchy and then sent Transaction back to the requesting system WorkFlow Process Props Tran Resp Prop Notification Req Prop CIF System Prop Fail Securtiy Req Prop Conv Resp Prop XML Hierarchy NotificationHierarchy CIFSystemContainer FailSecurityHierarchy XML Hierarchy Manager Transaction Manager: UCM Transaction Manager Execute Realtim e Pub/Su b 122 .
Considerations Error Handling Configuring Inbound Server Communication 123 .
Error Handling The UCM Server Party Package Workflow (MQSeries Server Transport Error Handling) demonstrates how error handling could be added to a workflow. Convert to XML Hierarchy Dispatch Message Security Check Convert Request In Transaction Manager Error Handler: UCM Converter: GenerateErrorPropSet Error Handler: UCM Converter: GenerateErrorPropSet End 124 .
Error Handling Continued The UCM Converter service includes a method to aid with error handling WorkFlow Process Props Error Code Error Message Disp Req Prop ErrorCode ErrorMessageText XML Hierarchy UCM Converter: GenerateErrorPropSet Error Hierarchy Prop XML Hierarchy 125 .
Configuring Inbound Server Communication Your UCM environment must be configured for inbound server communication Configuration procedure depends on the system transport protocol MQSeries Transport HTTP Transport 126 .
Module 7: UCM Batch Data Management Workflows 127 .
Batch Data Management Workflows Process data queued in Source Data and History tables During initial data load As a result of an insert request processed by Transaction Manager Separate workflows for contacts and accounts UCM Contact Batch Data Management Workflow UCM Account Batch Data Management Workflow Each workflow performs data cleansing. matching. and real-time publication (if enabled) 128 .
UCM Contact Batch Data Management Workflow Performs the following actions on a single input record Cleansing Start Matching Real-Time publication Query by ID CDM Cleanse Get Error Code Error CDM Match match Auto Match ? Yes No Process Review Matching Error? Process NoMatch Process AutoMatch End True Realtim ePub/S ub False Converter Publishing PubSub 129 .
Data Cleansing Performed by Cleanse method of the UCM Data Quality Manager business service Invokes third-party data cleansing software (if enabled) Output is cleansed record Start Retrieve record data for processing Query by ID CDM Cleanse Perform data cleansing 130 .
Data Matching (Deduplication) Performed by Match method of the UCM Data Quality Manager business service Three possible outcomes: No match found Match found that must be reviewed Match found that can be processed automatically Error match Get Error Code CDM Match Yes Auto Match ? No Process Review Error? Process NoMatch Process AutoMatch 131 .
Data Matching Continued All three match cases are processed by the same business service method: UCM Batch Manager HandleMatchCase Process NoMatch Process AutoMatch Process Review 132 .
Records with high match scores can be automerged.70 Process Review Records with low match scores will be left as seperate records. 133 . Low Score .90 Process AutoMatch Records with match scores in the middle will be brought to the attention of the data steward for manual review and resolution.applying survivorship rules.Data Matching Continued Thresshold settings determine how match is handled Process NoMatch High Score .
Adjusting Threshold Settings Configure user properties of the UCM Batch Manager business service to adjust merging thresholds User Property Default Value Account Auto Threshold Account Manual Threshold Contact Auto Threshold Contact Manual Threshold 90 70 90 70 134 .
Publishing Data changes can be published to subscribing systems Result of auto merging of records Result of insert of record (no match condition) Publishing is addressed in the next module Process NoMatch Process AutoMatch Realtim ePub/S ub False True Converter Convert data to format for publishing Publish Transaction PubSub End 135 .
Running Batch Data Management Workflows Create a Job Configure Job Parameters Run the Job 136 .
Create a new job to run the Workflow Process Batch Manager component/job Select this component/job If job is repeating specify repeat info in job details 137 . Create a Job In the Administration – Server Management > Jobs view.1.
2. Configure Job Parameters Include Search Specification: Identifies items to process Workflow Process Name : Identifies workflow to invoke on each item 138 .
Run the Job Once the job parameters are set. run job Click start button to start the job 139 .3.
Module 8: Publish and Subscribe 140 .
Publish and Subscribe Allows records inserted or updated in UCM to be published to external systems Distributes the best version customer profile Systems registered as subscribers receive publication Update: Contact123 Subscriber 1 messages UCM Update: Contact123 Subscriber 2 Update: Contact123 Subscriber 3 141 .
individual transactions are not Systems can mix and match modes Contact updates received in real-time Account updates received from daily batch job 142 .Publish/Subscribe Modes Publish and Subscribe framework supports two operative modes: Real-time Transaction are published as they happen Each individual transaction is published to real-time subscribers Daily Batch Mode Transactions are published once per day Best version of record is published.
Publication Messages Publication messages are CRMML messages FromSystem is UCM ToSystem is subscriber Message body content is constructed using integration objects Real-time publishing uses the response integration object associated with the particular UCM transaction Batch publishing uses the integration object type specified as input to the both Publish/Subscribe business service 143 .
Workflows and Real-Time Publishing
UCM Workflows include steps to perform near real-time publishing Decision step invokes publishing method if publishing is enabled Process
Transaction Manager response is published PubSub PubSub Realtim ePub/S ub Convert to Hierarchy End UCM Server Party Package Workflow
NoMatch Process AutoMatch Realtim ePub/S ub Convert Request Out Converter Result of match is published
UCM Contact/Account Batch Data Management Workflow
Real-Time Publishing of Insert Operations
Insert operations are processed in two steps: Queue insert in SDH tables Process insert, performing any necessary data cleansing and matching Both steps will generate a publish message The subscriber system should only process the second message Subscriber should ignore publish messages reporting that an insert operation has been queued.
Workflows and Batch Publishing
UCM Daily Publish workflow supports regular batch publishing
UCM Daily Publish
UCM Publish/Subscribe Business Service Workflows invoke this business service to perform publishing Method PublishMethod Inputs XML Hierarchy – Contains data to be published Integration Object – Used to generate the publication message (batch publishing only) 147 .
Configuring Publishing: Real-Time Set workflow user property to enable publishing User Property names differ! Set to true to enable publishing 148 .
Configuring Publishing: Real-Time Continued User Property names differ! Set to true to enable publishing 149 .
Configuring Publishing: Daily Batch Configure a job to run the UCM Batch Publish Subscribe component This job will run the Daily Batch workflow Integration object to use Number of records to processed in one batch 150 .
Subscribers Registered systems can be configured as subscribers to receive publication messages in real-time or batch URL where publication messages are sent Required for batch publishing Updated each time a message is published to this subscriber 151 .
Module 9: Wrap-Up
To enable participants to integrate Siebel Universal Customer Master (UCM) with external applications Legacy systems External databases Was achieved by covering the following topics Initial data loads into UCM at the data layer Incremental data loads into UCM at the business logic layer Managing outbound data flows via publish and subscribe The CRMML messaging protocol for requesting UCM services
Initial Data Loads into UCM at the Data Layer
Batched data layer loads into UCM are a two-stage process Loading from Siebel Interface table to Source Data and History (SDH) tables using the normal Siebel EIM process Loading from SDH tables to Siebel base tables using UCM Batch Manager Interface tables Are loaded using non-Siebel software Operated on by the Enterprise Integration Manager to transfer data to SHD tables The UCM Batch Manager is used to move data from SDH tables to Siebel base tables With data cleaning and matching to produce “best” version data Data is then accessible from the Siebel application’s standard business logic displays 154
Data Loads into UCM at the Business Layer Data loads at the business logic layer employ two distinct process managers: The Server Party Package Manager The UCM Batch Manager The Server Party Package Manager: Reads input data Identifies the data source Dispatches the incoming Insert message Queues the data on the SDH tables The UCM Batch Manager is used to move data from SDH tables to Siebel base tables With data cleansing and matching to produce “best” version data Data is then accessible from the Siebel application’s standard business logic displays 155 .
Update. and Delete operations The UCM Batch Manager for Insert Operations 156 .Managing Outbound Data Flows Records that have been inserted or updated can be selectively published to subscribers Making “best” version record available throughout the data ecosystem Real-time or batch mode Outbound publication service is performed by The Server Party Package Manager for Query.
The CRMML Messaging Protocol XML based mark-up language used by UCM Inbound server requests Insert Query Update Delete Outbound server features Publish and Subscribe Formatted to provide Requesting external system ID Service(s) being requested Pointers to predefined UCM integration objects 157 .
so it may not be useful as general purpose contact Id 158 . UCM Contact table has limited columns only. this data is primarily for referencing customer data between multiple external systems UUID is not numeric and big in size (about 20+ characters string). UCM can not store complete customer information.Some Guidelines Initial data for UCM should be cleaned outside (Before loading into UCM) EIM may be more effective to load data into UCM from external systems for performance reasons Batch publish method is more effective when send data from UCM to external systems UCM is not complete Customer data store.
Thank You 159 .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.