ERP Integration Tips and Best

Practices

Copyright © 2018 Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing, and pricing of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Agenda

1 Integration Anti Patterns

2 Know your Connectivity Agent

3 File Handling Best Practices

Copyright
Copyright
©©2018,
2019,
Oracle
Oracle
and/or
and/or
itsits
affiliates.
affiliates.
AllAll
rights
rights
reserved.
reserved.| Confidential – Oracle Internal/Restricted/Highly Restricted 3
 Don’t Starve
Scheduled Jobs
Susceptible to
timeouts
Stressed too much on Periodic Analysis of Scheduled
Anti
Synchronous Pattern
I Skipped the Spring
Flows Cleaning
Patterns
Don’t Hold
Don’t keep the target
system under Duress
Resources Read moreHacked
than what I
My
could chew
Consolidate Integration
FlowsDelete Flows No
Longer Needed

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

 Integration which talked too much

• Use Case
– Sync up records in a file or large dataset with external system
(E.g. Synchronizing journal transactions, Upload employee records into HCM)

• Anti-pattern
– Use invoke activity within a looping construct to call external APIs for every record

• Why?
– Downstream applications receiving large number of atomic requests – putting the
entire system under duress
– Usage based pricing model translates to high costs
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
 Integration which talked too much

• Best Practice
– Leverage applications’ capabilities to accept multiple records in a single request
• Salesforce: 200 records, OSC / ERP Cloud: 100 records, RightNow: 1000 records
– Leverage Adapters’ capabilities to send large data set as attachments / files
• Salesforce Adapter : 10000 records / 10 MB files, ERP Adapter support FBDI Files
– Use Stage File Action for append to file - send file to the destination at the end

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

 Bulk Import Pattern (FBDI Compliant)
 On-premise / PaaS based
applications require to import
recurring billing transactions into
ERP Cloud OIC ERP CLOUD
Flow 1
 On-premise insurance claim
processing applications creates Retrieve
Payables Invoices for handling File
Schedule or
payments Polling for new IMPORT BULK DATA
file
Enrich Upload
Transform To UCM

Initiate Load &

Import Import

Flow 2
Staged
Source File Get Status

Callback upon
Send completion
Notification

On-Prem

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Schedule Job Which never gave up
• Use case
– Process records within a set of files with a tight SLA (e.g. Synchronize
employee records into Active Directory from HCM, Send Benefits Info)

• Antipattern
– Scheduled integration looks for all files to process and loops over all of
them to sequentially process till no files left
• Why?
– If large number of files exist, one run of a scheduled job executes for a long
time - starves other jobs, may get killed by framework
– Processing pinned to single server - does not leverage multiple nodes in
a cluster

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Schedule Job Which never gave up
• Best Practice
– Cap the number of files to process in a single scheduled run
• Limit results from list file
– Use “schedule parameters" to remember last processed file for
the next run
– Invoke "run now" from the same integration to trigger
processing of next file if waiting for next scheduled run not
feasible

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

SEND BENEFITS ENROLLMENT TO PROVIDER
BENEFITS PROVIDER OIC Schedule HCM CLOUD
Write File
to Target
Retrieve
File
EXPORT BULK DATA
Generate ESS Schedule
Decrypt Extract File
File Decrypt
File
Compress &
Encrypt File
Stage File Generate
Benefits File
Load to
UCM
Compress &
Encrypt File
Purge

Write File
to SFTP

Send Purge
Notification UCM

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Look, I Hacked my Integration!
• Use case
– Need to leverage advanced XSL constructs that may not be available in OIC mapper

• Antipattern
– Updating the IAR file externally and then importing into OIC

• Why?
– Can lead to metadata inconsistency -> Validation failures
– Activation failures may happen

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Look, I Hacked my Integration!
• Best Practice
– Use "import map" feature in OIC
• Keeps metadata consistent and leverages validation
• JET mapper can show imported map and in future can allow edit
– Submit enhancement request via Oracle Connect
• https://tinyurl.com/oicideas

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Synchronous flow which tried to do too much
• Use case
– Request triggers a complex processing involving enrichment and updates across
multiple system

• Antipattern
– Gigantic synchronous flows modeling a large number of invokes / conditional logic
– Synchronous flow with invokes within a loop with large number of iterations

• Why?
– Susceptible to timeouts - any marginal slowdown adds up
– Blocking call – holds resources, starves other integration flows

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Synchronous flow which tried to do too much

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Synchronous flow which tried to do too much
• Best Practice
– Explore moving completely to asynchronous flow - fire and forget, async response
• Integration platform provides acknowledgement to clients upon receiving the message
• Platform ensures guaranteed processing and also support resubmission of failures
– Split into Synchronous integration containing mandatory processing before sending
out response and triggering separate asynch fire and forget flows for other
processing logic
– Optimize synchronous processing with a coarse grained external api to replace
multiple chatty calls

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Too many keys for same door!
• Use case
– As developers create integration, they define their own connections
pointing to same application, leading to bunch of duplicate connections

• Antipattern
– Every developer creates their own connection using different set of
configuration/credentials
• Why?
– High number of connections makes manageability painful
• Especially when you need to update endpoint, credentials, configuration etc.
– Complicates impact analysis when there is an app upgrade or metadata/coordinate
change
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Too many keys for same door!
• Best Practices
– Have a custodian to create needed connections and make sure duplicate connection
of same types are not created
• Build a best practice for naming conventions and maintaining set of configuration
– Use Configurator Tool to edit/replace connection in integration (coming soon!)

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Read more than what I could chew
• Use Case
– Read a file with large number of records and process individual records

• Antipattern
– Reading the whole file in memory using "read file" and processing record
by record

• Why?
– Consumes large amount of memory impacting other processing in system
– Sequential processing does not leverage built-in "map reduce" capabilities

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Read more than what I could chew
• Best Practice
– Download the file to stage location using "download file"
• Stage files are secured and deleted at the completion of instance
– Use "read file with segments"
• Developer can still model record level processing inside the "segment processing"
• Platform automatically processes segments in parallel
• Platform brings in only portions of file into to memory as needed

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

I Skipped the Spring Cleaning ...
• Use case
– Integration Flows / Schedules created during initial implementation continue to run
even though business requirements have changed over period of time
• Antipattern
– Integration flows and scheduled jobs once created are never re-evaluated against
changing business needs
– Proliferation of integrations without looking at existing flows
• Why?
– Unnecessary runs of jobs that pick up no work
– Unoptimized invocations of multiple flows with similar functionality
– Clutter with dead integration flows – LCM overheads, confusion for developers
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
I Skipped the Spring Cleaning ...
• Best Practice
– Periodic analysis of existing flows / schedules against current business needs
• Look at monitoring data for historical runs
– Consolidate the flows that are very similar
• Flow A & B subscribing ‘Customer Created Event in OSC‘ – Flow A for invoking endpoint A; Flow B for
invoking endpoint B
– Deactivate the flows that are no longer relevant
• Integration flow for generating a file meant for partner consumption becomes irrelevant when
partner no longer needs it
– Adjust the schedules for flows that need lesser frequency, delete schedules no longer
required
• Files generated monthly have schedules run every few minutes
• Changes in business needs could make certain schedules completely unnecessary

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Understanding FTP Adapter
• Use case:
– To pick up any file from SFTP server of pattern *.zip
• Catch:
– SFTP Read File does not have a feature to read the file using file type. It can only
read the specific file by filename.
• Best Practice:
– Start the orchestration with a Scheduler
– Before ReadFile have another step in your orchestration to List the Files from a SFTP
location using FTP Adapter. In list file use the file type as ‘.zip’ .
– After list file run a loop on the List File response (if you need to process multiple file
from that location) and pass the file name as a parameter of your Read File activity.

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Agenda
1 • Antipatterns
2 • Know your Connectivity Agent
• File Handling Best practices
3

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Know your Connectivity Agent
• It shouldn't be required to open inbound ports to communicate with On-Premises
applications. This means that there isn't a need to create firewall rules to provide
access. Because of this no open ports can be abused.
• It is not required to expose a private SOAP-based web service using a (reverse) proxy,
for example, API Gateway or Oracle HTTP Server (OHS).
• No On-Premises assets have to be installed in the De-Militarized Zone (DMZ). The
agent is installed in the local network where the backend systems are accessible.
• The Agent monitoring is done through OIC Console
• When a new version of the agent is available, your host is automatically upgraded with
the latest version (Refer) (Exception Windows OS)
• Any lib dependencies ex: MySQL / SAP jars should be copied to lib folder

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Know your Agent (contd…)
• Backup and Recovery best practices in case of crash or agent unavailable
– Backup
• Perform regular backups of the agent directory located under AGENT_INSTALL_DIR/agenthome.
• Ensure that the regular backups are not on the same physical host on which the agent is currently
running.
• Ensure that backups are available in the event of an agent host crash.
– Recovery
• Download new connectivity agent
• Create a directory as before with the same name
• Unzip the agent installer
• Replace the agenthome directory with the backup directory
• Restart the connectivity agent

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

What is Agent 240 Seconds Timeout?
• For Invoke messages, Integration Cloud waits for a maximum of 240s after it has
posted the request for the agent. This timeout is not configurable
• The agent needs to pull this work, execute against the On-Premises application and
return the response within this timeout period
• Some Causes for Agent Timeout
– Exhaustion of Agent Worker threads (40 threads available) because of slow performance of DB Query
response
– Resource contention among the worker threads can cause invoke requests to Queue
• Best Practice
– Leverage Agent HA Topology wherein multiple agents subscribe to the same Agent group which helps
in load balancing and scaling for performance (Currently feature flag enabled)
– Agent should be co-located in the same network as the end system

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Agenda
1 • Antipatterns
2 • Know your Connectivity Agent
• File Handling Best practices
3

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Quick Recap of specific adapters in the context of File Handling
FTP Adapter

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

File Adapter

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

• SOAP Adapter
–SOAP adapter supports MTOM attachments and inline attachments
(base64encoded string). These are often used in conjunction with File
contents in integration flows.
–SOAP adapter allows upto 10MB for inline attachments and upto 1GB
for MTOM attachments.
• REST Adapter
–REST adapter supports multipart message attachments and
unstructured payloads in both inbound and outbound directions upto
1GB. The attachment contents are usually used along with file
processing requirements in integration flow

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

File Based Activities
• Stage Activity
– Provides an abstracted and simple access to the Integration Cloud local file system. The absolute file
paths are abstract for the developer. We can use logical paths just like real file paths
– Provides options to
• List files
• Read files
• Write files
• Zip and Unzip files
• Reading file in segments
(Used to handle large files upto 1 GB)

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
File based Mapper Functions
• encodeReferenceToBase64 : The function
takes a FileReference as input, reads its
contents, performs base64encoding on it and
returns the encoded string. Ex: When sending
file content as inline attachment over to SOAP
endpoints.

• decodeBase64ToReference : The function

takes a base64 encoded string as input. It
decodes the string, writes it to a file and
returns the underlying FileReference. This
function is widely used to read file
attachments from SOAP based endpoints and
then send them over to remote FTP servers

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Use case scenarios where the features can be useful
• A source file of inventory records to be read from FTP and each record to be updated in target system using SOAP
web service (FTP Adapter Structured Read File)
• Accept a pdf file from source FTP and upload to a SOAP web service as MTOM attachment (FTP Adapter Opaque
Read file – SOAP MTOM)
• Read a GL/Invoices file from vendor FTP and load it into Oracle Fusion HCM using UCM web service as inline
attachment (FTP Adapter Opaque Read File – SOAP Inline attachment)
• Read a zip file containing images of Products and upload each image to a REST service as binary payload (FTP
adapter – Download, Unzip and Stage)
• Read a large file (ex: 100 MB) from FTP and process all records to a target database or ERP system (FTP adapter –
Download and Stage activity – Read file in segments)
• Read records over multiple REST calls , aggregate in local file and transfer file to remote FTP server – (Aggregate file
write using Stage Write)
• Read an encrypted file from source FTP. Decrypt the contents and process individual records
• Encrypt a clear text file from source FTP and transfer to target FTP server
• Read an encrypted zip file containing multiple files and process each file to target system

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

ERP Cloud Integration Troubleshooting / Best Practices

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

There are three patterns to extract data from ERP Cloud

• BIP Extract – Designed for extracting data for scenarios

• Get the list of invoices that has been approved since last execution
• Extract payment data since last execution to update downstream or upstream applications to reflect payments.
• The data file can be larger than 10MB and OIC supports handling large files as an attachment (instead of base64 encoding). We
always recommend csv format to reduce the size of the file (that XML generates). The report could be delivered to UCM or
SFTP or it can triggered using ERP integration service (exportBulkData). Please refer blog on handling files from various
interfaces and size constraints
https://blogs.oracle.com/fmw/oracle-integration-cloud-oic-file-based-integration-best-practices
• Short BIP reports – Designed for extracting lookup values or transaction details of a transaction. Example, give chart of account
(COA) values for a specific ledger. The reports are small and can be CSV/XML format

• BI Cloud Connector – This is used mainly for data lake or warehouse applications where you want to periodically extract
incremental data from ERP cloud. This applies more to data integration (ETL) instead of application integrations.

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Bulk Extract Callback or Event flow Upon Job Completion

• Use SOAP Adapter to retrieve

document from UCM to accept
attachments for ERP Cloud operations
ex:
– getDocumentByDocumentId
– getDocumentByFilePrefix
• Allows to pull files of more than 10
MB and for optimized processing

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Basic Checks for not getting events in Oracle Integration..
• Check the pre requisites setup for Event
– Integration account in ERP Cloud
– CSF Key in ERP Cloud
– Integration User should have service user in Oracle Integration
• Check the curl -u <userName>:<passWord> -X GET http://<host>:<port>/soa-
infra/PublicEvent/subscriptions/ “to confirm that you have the events entry there. Make sure to
recnocile the namespace, errors trend etc (subscription catalog)
• Check the curl -u <userName>:<passWord> -X GET http://<host>:<port>/soa-infra/PublicEvent/catalog/
“confirm that you have the events entry there (Event Catalog)
• Check for event xpath filter expressions
• If all the above basic checks looks fine Raise an SR 
• Ref: https://docs.oracle.com/en/cloud/paas/integration-cloud/erp-adapter/event-subscription-error-
troubleshooting.html

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Basic Checks for not getting callbacks in Oracle Integration..
• Check the pre-requisites for callback
– Roles for Integration account in ERP Cloud
– ERP Cloud certs imported into Oracle Integration
– Integration User in Oracle Integration with service user role
• Refer for several trouble shooting tips mentioned ->
https://docs.oracle.com/en/cloud/paas/integration-cloud/erp-
adapter/oracle-erp-cloud-callback-not-received.html

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |