Professional Documents
Culture Documents
12.2.1.4.0
F22173-05
June 2020
Oracle Fusion Middleware Using Oracle Data Integrator on Oracle Cloud Marketplace, 12.2.1.4.0
F22173-05
This software and related documentation are provided under a license agreement containing restrictions on
use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your
license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license,
transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse
engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is
prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If
you find any errors, please report them to us in writing.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on
behalf of the U.S. Government, then the following notice is applicable:
U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software,
any programs embedded, installed or activated on delivered hardware, and modifications of such programs)
and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government
end users are "commercial computer software" or “commercial computer software documentation” pursuant
to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such,
the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works,
and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs
embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle
computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the
license contained in the applicable contract. The terms governing the U.S. Government’s use of Oracle cloud
services are defined by the applicable contract for such services. No other rights are granted to the U.S.
Government.
This software or hardware is developed for general use in a variety of information management applications.
It is not developed or intended for use in any inherently dangerous applications, including applications that
may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you
shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its
safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this
software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of
their respective owners.
Intel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are
used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc,
and the AMD logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered
trademark of The Open Group.
This software or hardware and documentation may provide access to or information about content, products,
and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly
disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise
set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not
be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content,
products, or services, except as set forth in an applicable agreement between you and Oracle.
Contents
Preface
Audience vi
Documentation Accessibility vi
Related Documents vi
Conventions vii
iii
2.2.5.1 Creating a Model 2-7
2.2.5.2 Creating Datastores 2-9
2.2.6 Monitoring and Managing your Execution Results 2-10
2.2.7 Managing Users 2-10
2.3 Working with ODI Web Studio 2-11
2.3.1 Accessing ODI Web Studio 2-12
2.3.2 Create a New Project 2-12
2.3.3 Get Started with Data Flows 2-13
2.3.4 About Data Flow Editor 2-13
2.3.5 Create a New Data Flow 2-14
2.3.5.1 Add Components 2-14
2.3.5.2 Map Data Columns 2-15
2.3.5.3 Set Target Component Options 2-16
2.3.5.4 Validate and Execute a Data Flow 2-16
2.3.6 Introduction to Workflows 2-16
2.3.7 Create a New Workflow 2-17
2.3.8 Execute a Workflow 2-17
2.3.9 Scheduling Data Flows or Workflows 2-17
2.3.10 Managing Jobs 2-18
iv
6 Configuring Oracle Enterprise Manager for Oracle Fusion
Middleware on Oracle Cloud Marketplace
v
Preface
Preface
This book describes how to use Oracle Data Integrator on Oracle Cloud Marketplace.
This preface contains the following topics:
• Audience
• Documentation Accessibility
• Related Documents
• Conventions
Audience
This document helps you to use Oracle Data Integrator on Oracle Cloud Marketplace.
Documentation Accessibility
For information about Oracle's commitment to accessibility, visit the
Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?
ctx=acc&id=docacc.
Related Documents
For more information, see the following documents in Oracle Data Integrator Library:
• Release Notes for Oracle Data Integrator
• Understanding Oracle Data Integrator
• Developing Integration Projects with Oracle Data Integrator
• Installing and Configuring Oracle Data Integrator
• Upgrading Oracle Data Integrator
• Integrating Big Data with Oracle Data Integrator Guide
• Application Adapters Guide for Oracle Data Integrator
• Developing Knowledge Modules with Oracle Data Integrator
vi
Preface
Conventions
The following text conventions are used in this document:
Convention Meaning
boldface Boldface type indicates graphical user interface elements associated
with an action, or terms defined in text or the glossary.
italic Italic type indicates book titles, emphasis, or placeholder variables for
which you supply particular values.
monospace Monospace type indicates commands within a paragraph, URLs, code
in examples, text that appears on the screen, or text that you enter.
vii
1
Getting Started with Oracle Cloud
Marketplace
Oracle Data Integrator on Oracle Cloud Marketplace is a product offering that enables
customers to quickly set up and run Oracle Data Integrator (ODI) on Oracle Cloud. It
provides a fully unified solution for building, deploying, and managing complex data
warehouses or as part of data- centric architectures in a SOA or business intelligence
environment. In addition, it combines all the elements of data integration - data
movement, data synchronization, data quality, data management, and data services -
to ensure that information is timely, accurate, and consistent across complex systems.
This chapter contains the following sections:
• Prerequisites
• Selecting Your Product
• Launching Your Oracle Data Integrator Instance
• Connecting to ODI Compute Instance
• Accessing ODI Studio
• Using Autonomous Databases in ODI
• Installation Locations
• Log Files Location
• Patching
Note:
Refer to Launching Your First Linux Instance documentation before creating
the ODI instance.
1.1 Prerequisites
Make sure you have the following prerequisites before using Oracle Data Integrator on
Oracle Cloud Marketplace:
• Oracle Cloud Account
• Have access to assigned Oracle Cloud Tenant
• Compute node resources within Oracle Cloud Tenant
Go through the following prerequisites carefully before creating the ODI instance:
• Refer to Creating Dynamic Group Polices for compartment, before provisioning
ODI and ADB Instances.
1-1
Chapter 1
Prerequisites
• Read thoroughly the password policies provided in the Stack UI and create
passwords accordingly during provisioning.
• Wait till the completion of ODI instance creation, check for its status and then
proceed working on it.
For more information, refer to the usage instructions in Launching Your Oracle
Data Integrator Instance.
Supported Browsers
Oracle Cloud Infrastructure supports the latest desktop versions of Google Chrome,
Microsoft Edge, Internet Explorer 11, Safari, Firefox, and Firefox ESR. Note that
mobile browsers as well as private browsing mode is not supported for Firefox,
Internet Explorer, or Edge.
2. Enter the path to store this file. By default, this gets saved in your home directory
under a hidden folder called .ssh. Change this default location, if required.
1-2
Chapter 1
Selecting Your Product
ALL {instance.compartment.id =
'ocid1.compartment.oc1..aaaaaaaabgr34tpuanpvq6xfb667nsmy2jz45zj6dexo
jhxdsv4mjayem3cq'}
– ODI tenant level, then all ADW/ATP instances from all the compartments of
the tenancy are listed.
For example - To list ADW/ATP instances from all the compartments of
tenancy, you have to setup the following policy:
1-3
Chapter 1
Selecting Your Product
Note:
This product version is available only from ODI Web
V12.2.1.4.200618 with Bundle Patch 12.2.1.4.200618 release
and later.
Note:
Provisioning your instance on Oracle Cloud Marketplace greatly depends
on your product selection. You cannot use the same VM instance
for multiple product offerings . The procedure listed below are product
specific and may differ based on the product version.
1-4
Chapter 1
Selecting Your Product
1-5
Chapter 1
Launching Your Oracle Data Integrator Instance
1-6
Chapter 1
Launching Your Oracle Data Integrator Instance
Note:
Oracle Data Integrator Repositories that you create during
provisioning are mode specific. You cannot use a repository
created in ODI Web Studio mode with a stack in ODI Studio
mode and you cannot access a repository created in ODI Studio
mode with a stack in ODI Web Studio mode.
Note:
This option is available for both new and existing modes of
network creation.
b. Create or Use a Public ODI Subnet - It creates all the ODI resources
of the newly created instance in a public subnet. This option is
selected by default. If you clear this checkbox, all the ODI resources
are created in a private subnet.
– Existing networking components with be used -
a. Virtual Network CIDR (optional) - A single, contiguous Virtual Cloud
Network (VCN) CIDR block for the VCN. For example: 172.16.0.0/16.
You cannot change this value later.
b. VCN Compartment - Specifies the VCN compartment where the
compute node will be built. It is generally the location that you have
access to build the compute node. From the Virtual cloud network
compartment drop-down list, select the compartment where your VCN
has been setup.
1-7
Chapter 1
Launching Your Oracle Data Integrator Instance
Note:
While creating a subnet, select the option Use DNS
Hostnames in this Subnet. If you do not select this option
you cannot edit the subnet information and as a result of
this, stack creation fails. To rectify this, you have to create a
different subnet again by enabling this option.
Note:
If you are using a private IP address to access the compute
node, you have to setup an IPSec VPN or FastConnect
connection. Refer to OCI documentation for more details.
1-8
Chapter 1
Launching Your Oracle Data Integrator Instance
Note:
ODI Supervisor password should be 6-12 characters in length
and no special characters except $, # and _ are allowed.
– Schema Prefix - Specifies the prefix used for the RCU Schemas, used to
identify them. For existing repository, provide the prefix configured during
repository creation.
Note:
A valid prefix should contain only alpha-numeric characters. It
should not start with a number and should not contain any
special characters. A maximum of 12 characters are allowed.
– Schema Password - Specifies the password used for accessing the RCU
Schemas. For existing repository, provide the password configured during
repository creation.
Note:
Schema password that you provide should :
* contain only alpha-numeric characters,
* be a minimum of 12 characters in length,
* contain at least one alphabet in upper case,
* cannot start with a number
* no special characters except $, # and _ are allowed.
3. Click Next.
4. On the Review page, review the information you provided and then click Create.
5. When you get a message Provisioning Completed/Successful from the OCI
console, the instance is created. However please note the rest of the ODI
configuration happens in the background and takes approximately 20 minutes to
complete. If you attempt to access ODI prior to this, provisioning will be under
progress and you may not view default configurations in ODI Studio.
6. After approximately 20 minutes log into the VNC viewer using your <IP address>:1
and check the status of the configuration. To check the status of the configuration,
open a terminal window and execute the following command: tail -f $MW_HOME/
logs/odiConfigure.log. Upon successful completion, you will get the message
1-9
Chapter 1
Connecting to ODI Compute Instance
ODI Apps started Successfully.. Wait for a few minutes for ODI apps to start up
successfully.
7. Now the newly created ODI Instance is ready for use. Based on the type of ODI
Deployment, you can now launch the ODI Web Studio from a web browser or the
ODI Studio to start working on your data integration projects.
8. To check the logs, navigate to $MW_HOME/logs/odiConfigure.log.
where <private_key> is the full path and name of the file that contains the
private key associated with the instance you want to access.
• Use the following SSH command to access the instance.
where
– <private_key> is the full path and name of the file that contains the
private key associated with the instance you want to access.
– <username> is the default name for the instance. The default user name is
oracle.
– <public-ip-address> is your instance IP address that you retrieved from
the Console.
2. To connect to your ODI compute instance from windows,
• Open putty.exe.
• In the Category pane, select Window, and then select Translation.
• In the Remote character set drop-down list, select UTF-8. The default locale
setting on Linux-based instances is UTF-8, and this configures PuTTY to use
the same locale.
• In the Category pane, select Session and enter the following:
a. Host Name (or IP address):<username>@<public-ip-address> , where
<username> is the default name for the instance. For Oracle Linux and
CentOS images, the default user name is oracle. For the Ubuntu image,
the default name is ubuntu and <public-ip-address> is your instance
public IP address that you retrieved from the console.
1-10
Chapter 1
Accessing the ODI Compute Instance
b. Port: 22
c. Connection type: SSH
• In the Category pane, expand Connection, expand SSH, and then click Auth.
• Click Browse, and then select your private key.
• Click Open to start the session.
If this is your first time connecting to the compute instance, you might see a
message that the server's host key is not cached in the registry. Click Yes to
continue the connection.
4. On your local computer, for the VNC to work, add an Ingress rule as follows:
5. On your local computer, start a VNC viewer and establish a VNC connection to
localhost:1.
6. Enter the VNC password that you had provided during the stack creation.
7. For connecting multiple users, after the vncpasswd utility exits, start the VNC
server by typing vncserver. This will start a VNC server with display number 1
for the oracle user, and the VNC server starts automatically if your instance is
rebooted. For example vncserver@:2 or vncserver@:3.
To start developing your data flows or data mappings/transformations,
• If your product deployment is ODI Studio, then launch studio from the available
options.
• Else, in a web browser which has network connectivity to your compute instance,
type-in the url - http://localhost:9999/odi-web-studio
1. From the Applications menu, navigate to Programming -> ODI Web Studio or ODI
Studio,
or
Double click the short icon for ODI Web Studio or ODI Studio present in your
Desktop,
or
Navigate to the location $MW_HOME/oracle/odi/studio/bin/odi in the VNC.
1-11
Chapter 1
Using Autonomous Databases in ODI
2. Connect to the repository with already populated login credentials. The Login
Name value varies based on the selected repository. For ADB repository, the Login
Name is ODI_ADW_REPO and for MySQL Embedded repository it is ODISA_MYSQL.
3. Post successful configuration, check if the newly created data server is available
for use :
• In ODI Web Studio Administrator - Connections -> Technologies -> Oracle
• In ODI Studio - Topology navigator -> Technologies -> Oracle.
4. In ODI studio, navigate to Topology -> Physical Architecture -> Agent ->
OracleDIAgent1 and click Test, to check if the Standalone Agent is working.
Note:
For more details on services, refer to Managing ODI App Server.
Note:
You need to provide the username and password for the created instance as
prepopulated login credentials may not work.
1-12
Chapter 1
Using Autonomous Databases in ODI
1-13
Chapter 1
Installation Locations
• Connection Details - Click the Connection Details drop down arrow to choose
the required connection URL from the list of available connection URLs
retrieved from tnsnames.ora.
6. Click Test Connection.
Upon successful test connection, the new Dataserver gets created in the ODI
repository.
MW_HOME /u01/oracle/mwh
ODI_HOME MW_HOME+”/odi”
1.9 Patching
The ODI image on the Oracle Cloud Marketplace contains an Enterprise installation of
ODI.
Patching is manual using OPatch.
You can upgrade your existing ODI image to the latest version available on the Oracle
Marketplace directly, by either reusing your existing repository database or by creating
a new repository database. But before this, you should backup your work (export),
create a new instance and then restore (import) it, to proceed with your upgradation
process.
1-14
2
Using ODI Web Studio
This chapter helps you to work with ODI Web Studio on Oracle Cloud Marketplace.
Note:
ODI Web Studio is available only when you selelct Web Studio version from
the ODI product listing on Oracle Cloud Marketplace. For more details, see
Selecting Your Product.
2.1 Introduction
This chapter introduces Oracle Data Integrator Web Studio.
It contains the following sections:
• Introduction to Oracle Data Integrator Web Studio
• Introduction to Oracle Data Integrator Web Studio Administrator
• Introduction to Oracle Data Integrator Repositories
• Introduction to Oracle Data Integrator Run-Time Agent
2-1
Chapter 2
Administering ODI Web Studio
2-2
Chapter 2
Administering ODI Web Studio
For detailed information on the terminologies associated with Web Studio and their
respective functions, see Terminology Information.
$ cd $MW_HOME/odi/studio/bin
$ ./odi -clean
ODI Web Studio Administrator is launched allowing you to manage various aspects of
your integration project.
Connect to the repository with already populated login credentials. The Login Name
value varies based on the selected repository. For ADB repository, the Login Name is
ODI_ADW_REPO and for MySQL Embedded repository it is ODISA_MYSQL.
Post successful configuration, check if the newly created data server is available under
Connections -> Technologies -> Oracle.
To create master and work repositories in Oracle Data Integrator Web Studio
Administrator,
Launch the ODI Web Studio Administrator and click the + icon on the menu and
select the option Create New Master Repository. Enter the required details and
click OK.
2. Using Repository Creation Utility -
2-3
Chapter 2
Administering ODI Web Studio
Use the repository creation utility (rcu) to create the repositories, if you want to
setup J2EE agent/collocated agent.
Follow the below procedure to create repositories using the Repository Creation
Utility:
a. Find the repository creation utility executable available in the $MW_HOME/
oracle_common/bin directory in the Fusion Middleware home and launch rcu.
b. The first screen is just a welcome screen. Click Next.
c. Accept the default values and click Next.
d. Specify the database connection details. Once entered, you can see a dialog
box showing the progress of the three checks carried out on the database,
making sure that the connection is successful.
e. Select the repository components to install. Some are selected by default.
Make sure you select the Master and Work repository option. Click Next.
You can now see a dialog box showing the progress of the checks carried out
on the prerequisites for the components you selected
f. If all the prerequisite checks pass, then enter the password for the schema
users for the repositories. It's easiest to use the same passwords for all
schemas, however, make sure you choose the option that conforms to your
company security policy. Click Next.
g. Enter the password for the Supervisor user, what type the work repository is,
its name and the password for its user. Also specify the encryption algorithm.
Click Next.
h. Map the tablespaces. This is where you specify the name of the schema
owners, and the default and temporary tablespace names for each
component. You can accept the default names, or choose to specify your own.
Click Next.
i. The penultimate screen summarizes your selections, and from here, you
create the repositories. Click Create. Note that it takes several minutes to
create all the repositories.
j. Click Close after you have reviewed the summary of the repositories just
created.
Once you have created a master repository and at least one work repository, you then
create a connection, log in to ODI, and connect to these repositories.
Refer to the tutorial, Oracle Data Integrator 12c - Creating and Connecting to ODI
Master and Work Repositories for detailed steps on how to create your master and
work repositories.
2-4
Chapter 2
Administering ODI Web Studio
Note:
From 12.2.1.4.200618 version of ODI Marketplace release, short url
with TNS_ADMIN is used in ODI Studio/Administrator login field.
5. Enter Supervisor as the user, and the password for it. (Note, it's what you entered
when creating the repositories).
6. Enter the database connection details for the master repository.
7. Select the work repository.
8. Click Test Connection, to test the connection and upon successful test
connection, click OK.
Note:
To configure repository of the ODI Agent/ODI Apps, refer to Switching
Repositories of the ODI App Server.
2-5
Chapter 2
Administering ODI Web Studio
• Casandra
• Derby
• Hypersonic SQL
• IBM DB2 UDB
• IBM DB2/400
• Informix
• Microsoft SQL Server
• MySQL
• Netezza
• PostgreSQL
• Salesforce.com
• Teradata
• Timesten
For detailed information on the terminologies associated with Web Studio and their
respective functions, see Terminology Information.
2-6
Chapter 2
Administering ODI Web Studio
Creating a Schema
To create a new schema for your data server :
1. Right-click the newly-created data server, and select New Schema.
2. Select the schema name from the Schema (Schema) drop-down list in the new
window that appears, then select the appropriate name from the Schema (Work
Schema) drop-down list.
3. Click the Default check box, if this is to be the default schema for this data server.
4. Click Yes in the Confirmation dialog box that displays. Leave all the other fields
unchanged. Click Save.
5. Click OK in the Information dialog box that displays.
Expand the data server hierarchy and confirm that the schema has been added to
the data server.
Once you create a new Data Server and Schema, Logical Schema is automatically
created for you and it gets mapped to your created Schema. For more information, see
Creating a Data Server.
2-7
Chapter 2
Administering ODI Web Studio
You can choose to create the model in selected model folder or you can create a
top-level model.
2. Click the New Model icon.
The window for the new model appears.
3. Enter the name of your model.
Like most objects in ODI, there is also a code to enter. By default, this is the same
as the name, but uppercase with underscores instead of spaces.
4. Specify the technology of the underlying database in this field.
A model must be linked to a specific technology.
5. Logical schema to which the model belongs is automatically created and auto-
populated here. It is created in the following format: <Data Server Name
_Physical Schema Name_ls>. If required, choose the appropriate logical schema.
There is only one context (global) which determines the schema that is be used as
the source of the metadata.
6. Enter a description for your model.
For more information, see Creating a Model.
To perform customized reverse-engineering by using an RKM, perform the following
steps:
1. Click the Reverse Engineer tab of the model.
The Customized option button is selected by default.
2. Fill in the details for the following fields:
• Context: the context used for the reverse-engineering process
• Logical Agent - Choose the required agent for the reverse-engineering
process.
• Object Type: the type of objects to reverse-engineer (tables, view, and so
on). Only object of these types will be taken into account by the reverse-
engineering process.
• Mask: Enter in the Mask field the mask of tables to reverse engineer. The
mask selects the objects to reverse. This mask uses the SQL LIKE syntax. The
percent (%) symbol means zero or more characters, and the underscore (_)
symbol means one character.
• Select the Knowledge Module: Select the KM that you want to use for
performing the reverse-engineering process. This KM is typically called RKM
“<Project_Name>.<Project_Folder>” for example, “Knowledge Module:
HandsOnLabs.HandsOnLoads”
• Optionally, you can specify the characters to remove for the table alias.
These are the characters to delete in order to derive the alias. Note that if
the datastores already exist, the characters specified here will not be removed
from the table alias. Updating a datastore is not applied to the table alias.
3. Click Reverse Engineer in the Model toolbar menu, then Yes to validate the
changes.
4. Click OK. The Session Started dialog appears.
5. Click OK.
2-8
Chapter 2
Administering ODI Web Studio
You can review the reverse-engineering tasks in the Sessions tab of the Operator
Navigator. If the reverse-engineering process completes correctly, reverse-engineered
datastores appear under the model node in the Models panel of Connections tab.
2-9
Chapter 2
Administering ODI Web Studio
2-10
Chapter 2
Working with ODI Web Studio
WARNING:
At least one supervisor user account should be maintained at all times. If all
supervisor user accounts expire, contact Oracle Support.
2-11
Chapter 2
Working with ODI Web Studio
The folder is created with a default folder. You can click the Actions icon ( )
next to the default folder and select Edit to rename it.
2-12
Chapter 2
Working with ODI Web Studio
5. To add more folders to the project, click the Actions icon ( ) next to the project
and click Create Folder.
3. Click the Actions icon ( ) next to folder in which you want to create the data flow,
and click Create Data Flow.
4. This action takes you to the the Data flow editor where you create the data flow.
For more information, see About Data Flow Editor.
• Data Entity Panel: The data entity panel displays the Data Entities that are
available to use in your Data flows. These are organized by Connections,
Schemas and Collections. Expand these tree nodes to find the required Data
Entities, or else use the Filter to use them on the design canvas.
• Transformations Toolbar: The transformations toolbar display the transformation
options that can be used in your data flows. Just like Data Entities, you can drag
and drop the Transformations you want to use on the design canvas.
• Design Canvas: The design canvas is where you build your transformation logic.
After adding the Data Entities and Transformations to the design canvas, you can
connect them in a logical order to complete your data flows.
• Properties Panel: The properties panel displays the properties of the selected
object on the design canvas. The Properties Panel is grouped into four Tabs.
General, Attributes, Column Mapping, and Options. Not all tabs are available
as they vary based on the selected object.
2-13
Chapter 2
Working with ODI Web Studio
4. Click the Actions icon ( ) next to folder in which you want to create the data flow,
and click Create Data Flow.
5. Enter a name for the data flow and click OK.
4. Select an object on the design canvas, and drag the Connector icon ( ) next
to it to connect the components.
5. After saving the data flow, there may be a Transfer icon overlaid on one or more
of the component connections. This indicates that ODI has detected an additional
2-14
Chapter 2
Working with ODI Web Studio
step and it is required to move the data between data servers. You can click on
this Icon to view properties associated with this step.
For example:
Note:
Use the expression editor only if you have complex expressions for a
particular column.
4. Click OK.
2-15
Chapter 2
Working with ODI Web Studio
2. Click the Validate icon ( ) in the toolbar above the design canvas to validate the
data flow.
3. After a successful validation, click the Execute icon ( ) next to the Validate icon
to execute the data flow.
A message appears that displays the execution job ID and name that you can
monitor on the Jobs page. For more information, see Managing Jobs.
2-16
Chapter 2
Working with ODI Web Studio
3. If the "Weekly CDC alerts"data flow execution is successful, execute the "Load
Medicare" data flow.
4. If any of the above data flow fails, then execute the "Audit_error_log" data flow.
3. Click the Actions icon ( ) next to folder in which you want to create the
workflow, and click Create Workflow and provide a name for the workflow.
4. Select the data flow that you want to execute first and select First Step check box
in the Properties Panel.
5. Select either the ok (green arrow) icon, the not ok (red arrow) or the ok/not ok
(black arrow) in the toolbar.
This defines the mode that subsequent links drawn on the canvas will be set to.
• ok (green) defines the success path.
• not ok (red) defines the failure path.
• ok/not ok (black) defines a path that is followed on success or failure.
6. Select the data flow and click the Connector icon ( ) next to it to connect it
with the next data flow.
7. Click Save.
2. Select the object in the canvas and click the Execute Step icon ( ), to execute
only the selected data flow.
You can monitor the execution status on the Jobs page. For more information, see
Managing Jobs.
2-17
Chapter 2
Working with ODI Web Studio
2-18
3
Using ODI Studio
This chapter helps you to configure and use ODI Studio on Oracle Cloud Marketplace.
For detailed information on the terminologies associated with ODI Studio and their
respective functions, see Terminology Information.
It contains the following sections:
• Accessing ODI Studio
• Using Autonomous Databases in ODI
• Working with ODI Instance
4. On your local computer, for the VNC to work, add an Ingress rule as follows:
5. On your local computer, start a VNC viewer and establish a VNC connection to
localhost:1.
6. Enter the VNC password that you had provided during the stack creation.
7. For connecting multiple users, after the vncpasswd utility exits, start the VNC
server by typing vncserver. This will start a VNC server with display number 1
for the oracle user, and the VNC server starts automatically if your instance is
rebooted. For example vncserver@:2 or vncserver@:3.
To launch the ODI instance,
1. From the Applications menu, navigate to Programming -> ODI Studio
or
Double click the short icon for ODI Studio present in your Desktop
or
Navigate to the location $MW_HOME/oracle/odi/studio/bin/odi in the VNC.
3-1
Chapter 3
Accessing ODI Studio
2. Connect to the repository with already populated login credentials. The Login
Name value varies based on the selected repository. For ADB repostiory, the Login
Name is ODI_ADW_REPO and for MySQL Embedded repository it is ODISA_MYSQL.
3. Post successful configuration, check if the newly created data server is available in
the Topology navigator -> Technologies -> Oracle.
3-2
Chapter 3
Working with ODI Instance
Note:
Depending on your OCI network configurations, you may or may not
require access through proxy-hosts. While you are connecting through
proxy, make sure that the proxy address/port or the source dataserver is
allowed through OCI VCN configurations.
Note:
If you are using a BI Cloud Connector Dataserver, you may need to add
the BI Cloud Connector host to the Proxy Exclusion field.
3-3
Chapter 3
Working with ODI Instance
For more information, refer to Overview of Oracle Data Integrator Topology chapter in
Developing Integration Projects with Oracle Data Integrator guide.
3-4
Chapter 3
Working with ODI Instance
For more information, refer to Creating and Using Mappings in Developing Integration
Projects with Oracle Data Integrator guide.
3-5
Chapter 3
Working with ODI Instance
3-6
4
Managing ODI Setup
This chapter helps you to manage the ODI setup that you have provisioned on Oracle
Cloud Marketplace.
It contains the following sections:
• Working with ODI Linux Services
• Changing Repository in ODI Web Studio/ODI Web Studio Administrator
• Switching Repositories of the ODI App Server
• Managing ODI App Server
• Managing ODI Credential
• Configuring Proxy Settings
4-1
Chapter 4
Changing Repository in ODI Web Studio/ODI Web Studio Administrator
Note:
You can switch between repositories only when the repositories in stack
mode and repo mode match.
In ODI App Server, you can switch repository in the following technologies:
• Switching from ADB to ADB
• Switching from MYSQL to ADB or DBCS
Note:
But the reverse (switching back to MYSQL from ADB or DBCS) is not
supported.
4-2
Chapter 4
Switching Repositories of the ODI App Server
dbTech=ADB
rcuCreationMode=false
odiSchemaPassword=<valid password>
odiSchemaUser=<odi schema username>
odiSupervisorPassword=<odi SUPERVISOR password>
walletZipLoc=<path_to_zipped_wallet>
workRepoName=<WORK REPO NAME>
Note:
workRepoName=<WORK REPO NAME> is an optional propery but you may
have to configure this property if your default work respository name is
not WORKREP.
masterReposDriver=oracle.jdbc.OracleDriver
masterReposUser=<odi schema username>
workReposName=<WORK REPO NAME>
python odiMPConfiguration.py
python manageOdiApps.py start
Note:
Stop the server before running any configuration. For more information on
this, refer to Managing ODI App Server.
dbTech=DBCS
dbHost=<IP Address of the DBCS Instance>
dbPort=<port of DBCS Instance>
4-3
Chapter 4
Managing ODI App Server
masterReposDriver=oracle.jdbc.OracleDriver
masterReposUser=<odi schema username>
workReposName=<WORK REPO NAME>
python odiMPConfiguration.py
python manageOdiApps.py start
Note:
Stop the server before running any configuration. For more information on
this, refer to Managing ODI App Server.
APPODIAGENT
You can use ODI App Server to manage all the ODI applications deployed in ODI App
Server.
Navigate to the location $MW_HOME/odi/common/scripts to run the following
commands:
• Use the following command to check the status of the service (as the oracle user):
4-4
Chapter 4
Managing ODI App Server
Note:
You cannot use this command to start or stop the service.
• Use the following command to start all the applications associated with the
service:
• Use the following command to stop all the applications associated with the service:
Note:
When you execute the command python manageOdiApps.py, two log
files odiagent.log and odi_adp_rest_txt.log are created. For details
on the location of the files, refer to Log Files Location.
• Use the following command to get the status of all applications associated with the
service:
• If you have provisoned this stack prior to 12.2.1.4.200618 release version of ODI
Marketplace or if you have provisioned this stack for ODI Studio, follow the below
procedure to manage your ODI Agent lifecycle:
– To stop the ODI Agent:
python stopAgent.py
4-5
Chapter 4
Managing ODI Credential
Note:
The script files stopAgent.py and startAgent.py are not applicable for
Web Studio stack deployment.
Use the following command to set the credential key in the Credential Store:
Use the following command to get the credential key value stored in the Credential
Store:
Note:
Depending on your OCI network configurations, you may or may not require
access through proxy-hosts. While you are connecting through proxy, make
sure that the proxy address/port or the source dataserver is allowed through
OCI VCN configurations.
4-6
Chapter 4
Configuring Proxy Settings
Note:
For backward compatibilty, use the scripts startAgent.py and
stopAgent.py to manage ODI Agent Lifecycle.
-Dhttp.proxyHost=www-proxy-xxx.com -Dhttp.proxyPort=80
-Dhttps.proxyHost=www-proxy-xxx.com -Dhttps.proxyPort=80 -cp
For example, after adding the above lines, your file should be like this:
subprocess.call('nohup java
-Drepo.props=odi-setup.properties
-Xms1024m -Xmx4048 -cp
-Dhttp.proxyHost=www-proxy-xxx.com -Dhttp.proxyPort=80
-Dhttps.proxyHost=www-proxy-xxx.com -Dhttps.proxyPort=80 -cp
$AGENTCLASSPATH oracle.odi.OdiStandaloneAgentStarter'+'
'+oraclediagentPath+"
&", shell=True)
2. Save the file and use the following command to start the agent:
Note:
Ensure you do not add any extra lines or space or tab on the file
startAgent.py. Just add -D option within the line content. It is a python
script and it requires proper line indentation to work.
3. Test the standalone agent from ODI studio to see if the agent has started
successfully. Then execute the packages/mappings using the standalone agent.
4-7
Chapter 4
Configuring Proxy Settings
Note:
If you are using a BI Cloud Connector Dataserver, you may need to add the
BI Cloud Connector host to the Proxy Exclusion field.
3. After the above lines, add the below line before -cp:
4-8
5
Configuring ODI Marketplace Repositories
on DBCS Instance
This chapter helps you to create a new DBCS instance and set up ODI master and
work repositories for the newly created DBCS instance on Oracle Cloud Marketplace.
Perform the following steps to create a DBCS instance and set up ODI master and
work repositories for the created DBCS instance:
Note:
Admin password that you provide should be 9 to 30 characters in length,
contain at least two alphabets in upper case, two alphabets in lower
case, two special characters (which includes _, # or -) and two numeric
characters.
5-1
Chapter 5
Note:
If you wish to use the newly created DBCS repository for your default ODI
Marketplace Agent, refer to Switching Repositories of the ODI App Server.
Note:
For more details on creating and configuring a JEE agent, refer to
Configuring the Domain for a Java EE Agent chapter of Installing and
Configuring Oracle Data Integrator guide.
5-2
6
Configuring Oracle Enterprise Manager for
Oracle Fusion Middleware on Oracle Cloud
Marketplace
This chapter helps you to access Oracle Enterprise Manager Fusion Middleware
Control Console on Oracle Cloud Marketplace, and display WebCenter-related pages
from where you can perform all necessary configuration, monitoring, and management
tasks.
Prerequisites
Ensure to configure the following prerequisites before configuring Oracle Enterprise
Manager for Oracle Fusion Middleware on Oracle Cloud Marketplace:
1. STB and other required schema as created by Repository Creation Utility (RCU).
Note:
Repository Creation Utility (RCU) schemas are created by default in
Oracle Cloud Marketplace and the default STB schema name is in
the format <schema_name>_STB. Check with support team to know the
exact schema name. You can also find the schema name (for example
- ADW11_STB) in odi-setup.properties file along with the following
properties:
• sMasterDBDriver
• sMasterDBUsername
• odiSupervisorUser
• sMasterDBUrl
2. Physical agent in ODI studio with agent name as OracleDIAgent pointing to host
name and port where you plan to configure OracleDIAgent.
Note:
This is a Java EE Agent and when you create it in ODI studio it works
only after completing all the configurations in the configuration wizard
and starting admin and manager servers.
6-1
Chapter 6
oracle.net.authentication_service TCPS
oracle.net.ssl_server_dn_match false
javax.net.ssl.trustStore /u01/oracle/mwh/wallet/cwallet.sso (Path
of the wallet location)
javax.net.ssl.keyStoreType SSO
javax.net.ssl.keyStore /u01/oracle/mwh/wallet/cwallet.sso(Path of
the wallet location)
javax.net.ssl.trustStoreType SSO
Note:
Make sure to provide the path for truststore and keystore parameters
same as the path of your wallet file.
Click Save & Close. Click RCU Configuration and Click Next.
6-2
Chapter 6
Note:
Make sure to change the connection string, connection properties for
each component schema, as they are not carried forward from the
previous screens.
This image is a sample with changes for component LocalSvcTblSchema and you
have to make changes for the following:
• WLS Schema
• ODI Master Schema
• ODI Work Schema
• OPSS Audit Schema
• OPSS Audit Viewer Schema
• OPSS Schema
6-3
Chapter 6
9. After providing all the above details, JDBC Test screen appears. Review all the
details and click Next.
10. From the Credentials screen, enter the Supervisor password for your ODI
instance and click Next.
11. From the Advanced Configuration screen, select the check boxes for the
following servers/services:
6-4
Chapter 6
• Administration Server
• Node Manager
• Topology
• Deployments and Services and click Next.
12. The Administration Server screen displays the port number in which the admin
server listens. It is auto-populated by default. Click Next.
13. From the Node Manager screen, provide the node manager login credentials
in Username, Password and Confirm Password fields and click Next. Node
manager login credentials can be same as Admin server login credentials, which
means you can use the same username and password for logging into both Node
manager and Admin server.
14. From the Managed Servers screen, provide the server name and port details in
Server Name and Listen Port columns respectively.
15. From the Clusters screen, click Add, to create a new cluster. Click Next.
16. The Server Templates screen displays all the available server templates. Click
Add, to add new server templates, if required, else, click Next. Assign the required
server template to the newly created Cluster.
17. The Coherence Cluster screen displays the available cluster name and port
number. Click Next.
18. In the Machines screen, click Add, to add a machine and provide the name, port
and address of the Node Manager. After adding a machine, add Admin server and
ODI_server to the created machine. Click Next.
19. The Virtual Targets screen displays the details of configured virtual targets, if any.
It includes details such as name, target, host names, URI prefix, explicit port and
port offset. Click Next.
20. The Partitions screen displays the details of all the available partitions. Click
Next.
21. The Deployments screen displays a list of all the Deployments and Deployment
Targets. Click Next.
22. The Service Targets screen displays a list of all the Services and Deployment
Targets. Click Next.
23. The Configuration Summary screen displays a summary of all the performed
configurations. Click Create. Configuration Process screen appears displaying
the status of the process and configurations.
24. Upon completion, End of Configuration screen appears, displaying the
Configuration Succeeded message along with the configured Domain Location
and Admin Server URL details. Click Finish.
Other Configurations
1. Locate jps-config.xml file and configure the following properties in your jps-
config.xml file. By default, you can find this file in the following location
- ~/oracle/user_projects/domains/base_domain/config/fmwconfig and this
location is based on the created base domain.
6-5
Chapter 6
value="TCPS"/>
<property name="oracle.net.ssl_server_dn_match"
value="false"/>
<property name="javax.net.ssl.keyStoreType"
value="SSO"/>
<property name="javax.net.ssl.trustStoreType"
value="SSO"/>
<property name="javax.net.ssl.keyStore" value="/u01/
oracle/mwh/wallet_ADWSAT1/cwallet.sso"/>
/u01/oracle/mwh/user_projects/domMains/base_domain/bin>./
startNodeManager.sh
3. In the command prompt, navigate to the domain creation path (configured in the
above steps) and perform the following:
• Start the Admin server using the following command:
/u01/oracle/mwh/user_projects/domains/base_domain/bin> ./
startWebLogic.sh
/u01/oracle/mwh/user_projects/domains/base_domain/bin> ./
startManagedWebLogic.sh ODI_server1 http://localhost:7001
6-6
Chapter 6
For example,
http://localhost:7001/em/
Provide your login credentials and click Sign In, to access the newly configured
Oracle Enterprise Manager Fusion Middleware Control Console on Oracle Cloud
Marketplace.
After logging in, you can check the health of the servers. From the Oracle Enterprise
Manager Fusion Middleware Control Console you can start and stop the servers and
agents. You can also check if, the console is working by logging into the console
http://localhost:managedserverport/odiconsole.
6-7
7
Configuring the Domain for Collocated/
Java EE Agent on Oracle Cloud
Marketplace
This chapter helps you to create and configure a Oracle Data Integrator (ODI) domain
for the collocated/Java EE agent standard installation topology on Oracle Cloud
Marketplace.
Configuring Collocated/Java EE agent on Oracle Cloud Marketplace includes the
following high level steps:
• Navigate to the location (ORACLE_HOME/oracle_common/common/bin) and execute
the script config.sh to launch the WebLogic Server Configuration Wizard.
• Select the ODI collocated / JEE agent template and deploy.
• After succesful deployment, start the collocated / JEE Agent, Weblogic Server and
Managed Server.
For detailed instructions on creating and configuring a WebLogic domain for a Java EE
agent, refer to Configuring the Domain for a Java EE Agent .
7-1
8
Configuring High Availability for ODI on
Oracle Cloud Marketplace
This chapter helps you to configure High Availability (HA) topology for Oracle Data
Integrator on Oracle Cloud Marketplace. The sections in this chapter outline the
concepts and steps that are important for designing high availability deployment.
It contains the following sections:
• Setting Up 2-Node Cluster for High Availability
• Configuring Horizontal Clustering
• Enabling Incoming Ports and Services
Prerequisites
Go through the following prerequisites before setting up 2-Node Cluster for High
Availability. Make sure you have the following before setting up 2-Node Cluster for
High Availability:
1. OCI Virtual Cloud Network (VCN) setup that supports communication with all the
compute instances created in its subnet.
• All the communication channels are through private IPs.
• External communication established outside the subnet are through public IPs.
• Ably configured for Ingress/Egress.
2. ADB or DBaaS instance created in the same subnet and VCN as described in
Step 1.
3. ODI compute instance 1 created in the same subnet and VCN as described in
Step 1.
4. ODI compute instance 2 created in the same subnet and VCN as described in
Step 1.
5. Firewall configurations in all the compute instances that are part of the cluster.
• Enabling Incoming Ports and Services (through firewall commands).
8-1
Chapter 8
Configuring Horizontal Clustering
1. Node-1 and Node-2 have middleware home installed on the same path and have
the same ODI version.
2. Node 1 - Install ODI schemas using RCU
• Here you will be providing the DBaaS or ADW connection credentials.
3. Node 1 - Create ODI Domain ($ODI_DOMAIN_HOME) with Horizontal Clustering
using JEE Agent template.
• RCU association - STB schemas created in step (2)
• Managed Servers - ODI_Server1 on Node1 with port 8001 ; ODI_Server 2 on
Node 2 with port 8001
• Cluster Topology - ODI_Cluster1 with nodes (ODI_Server1, ODI_Server2)
• Node Manager Topology
a. Node Manager on Node-1 (AdminServer; ODI_Server1).
b. Node Manager on Node-2 (ODI_Server2).
4. On Node-1
• Start the Admin Server.
• Start the node manager from $ODI_DOMAIN_HOME/bin.
• Pack the domain using pack.sh (with managed=true).
5. On Node-2
• Unpack the domain in the same domain path of step (3), $ODI_DOMAIN_HOME.
• Start the node manager from $ODI_DOMAIN_HOME/bin.
Note:
Before starting the node manager, in the nodemanager.properties
file, set the property SecureListener=false and make sure the
listner port is matching with the configured port.
8-2
Chapter 8
Configuring Horizontal Clustering
c. webapp(context)- oraclediagent
d. agent work repository (workrep)- jdbc/odiMasterRepository or
jdbc/odiWorkRepository
• Node-1 or Node-2 through WLS Console : Configure Coherence cache
configuration.
– Through WLS Admin Console - Configure Coherence cache properties for
UNICAST clustering in JAVA_OPTIONS for managed server startup.
a. tangosol.coherence.localport configuration parameter defines the
port which a node in the cluster can use for coherence cluster. It is
connected by agent nodes to detect coherence cluster existence and
other coherence communication.
b. All the ODI Agents deployed on a cluster must be connected to the
same Coherence cluster cloud. This enables the agents to share the
knowledge of the tasks performed by each of them as well as allow for
Scheduling Service migration when needed. Following properties are
introduced to configure the Coherence listen addresses.
* oracle.odi.coherence.wkaN : The host name of a Managed
Server.
* oracle.odi.coherence.wkaN.port : Coherence Unicast port
configured on that Managed Server.
* Where N = 1..10
c. Sample Configuration
For Example:
Node 1:
"-Dtangosol.coherence.localport=8095
-Doracle.odi.coherence.wka1=<MS1_HOSTNAME>
-Doracle.odi.coherence.wka1.port=8095
-Doracle.odi.coherence.wka2=<MS2_HOSTNAME>
-Doracle.odi.coherence.wka2.port=8096"
Node 2:
"-Dtangosol.coherence.localport=8096
-Doracle.odi.coherence.wka1=<MS1_HOSTNAME>
-Doracle.odi.coherence.wka1.port=8095
-Doracle.odi.coherence.wka2=<MS2_HOSTNAME>
-Doracle.odi.coherence.wka2.port=8096"
8. Node-1 or Node-2 through WLS console - Start both the managed servers
(ODI_Server1 and ODI_Server2) from the console through node managers.
8-3
Chapter 8
Enabling Incoming Ports and Services
Note:
1. The above steps are guidelines for setting up HA Cluster with 2 nodes.
The same procedure can be extended to have any number of nodes in
the cluster.
2. Load balancer created can either be private or public. It is always
recommended to configure public end-points with SSL. For more
information on this, refer to Overview of Load Balancing.
3. The subnets can be regional or specific to an Availability domain. For
maximum availability, Oracle recommends you to configure regional
subnets. For more information, refer to VCNs and Subnets.
Note:
All the instances participating either directly or indirectly should be following
the below ingress and egress rules.
8-4
Chapter 8
Enabling Incoming Ports and Services
8-5
Chapter 8
Enabling Incoming Ports and Services
8-6
Chapter 8
Enabling Incoming Ports and Services
Firewall Rules
Even after setting the ingress and egress rules, in some cases the instances may not
allow the incoming traffic. This is because of the firewall associated with the instance.
Ensure to enable all the communication ports by configuring the firewall.
The following is an example of firewall commands for the above ingress/egress ports
(that has enabled port communication) :
Firewall command to enable port communication between the nodes (run on both
machines):
8-7
Chapter 8
Enabling Incoming Ports and Services
port=5557/tcp
sudo firewall-cmd --permanent --add-service=odiwlsnodemgr2
sudo firewall-cmd --reload
8-8
9
Troubleshooting ODI on OCI
This chapter describes about various services associated with ODI on OCI and ways
to troubleshoot them when you encounter issues while using them.
Note:
If you are facing issues connecting to ADB dataserver and MySQL repository
after long hours of inactivity, try reconnecting to ODI repository to overcome
this problem.
How Tos
Note:
The following commands are supported in release(s) prior to
12.2.1.4.200618 version of ODI Marketplace. For latest release, refer to
Managing ODI Credential .
$MW_HOME/odi/common/scripts/getPassword.sh
• How to update ODI Repository and schema credentials for ADB technology?
cd $MW_HOME/odi/common/scripts
Create a new file updateCredentials.sh and add below contents
echo "Updating credentials in wallet..."
WCLASSPATH=../../sdk/lib/odi-core.jar:../../sdk/lib/commons-
lang-2.2.jar:../../../oracle_common/modules/oracle.jps/*:../../../
oracle_common/modules/oracle.igf/identitydirectory.jar:../../../
oracle_common/modules/oracle.idm/identitystore.jar:../../../
oracle_common/modules/oracle.osdt/osdt_cert.jar:../../../
oracle_common/modules/oracle.osdt/osdt_core.jar:../../../
oracle_common/modules/oracle.osdt/osdt_xmlsec.jar:../../../
oracle_common/modules/oracle.pki/oraclepki.jar:../../../wlserver/
modules/com.oracle.weblogic.security.jar:../../../wlserver/modules/
com.oracle.weblogic.security.subject.jar
export WCLASSPATH
java -cp $WCLASSPATH oracle.odi.setup.util.ODIWalletSetupUtil
odi ../../common/scripts/jps-config-jse.xml <supervisorPassword>
<schemaPassword>
Save the file and stop the ODI agent.
9-1
Chapter 9
Note:
In the above commands, change the values for <supervisorPassword>
<schemaPassword> fields with your respective ODI supervisor password
and ADB Repository schema password.
9-2
10
Known Issues and Workarounds
This chapter details known issues in this release, and their workarounds.
Mismatch in Server Certificate DN with connection details having tls for ATP/
ADW-D:
When you navigate to Dataserver configured with ATP/ADW-D and select Connection
details with TLS, Test connection fails due to server certificate issue. It happens when
you force JDBC driver to match DN (Server's Distinguished Name) by setting the
property oracle.net.ssl_server_dn_match=true and DN mentioned in JDBC URL
does not match with the DN mentioned in server certificate.
As a workaround, mention the correct DN in JDBC URL or set
oracle.net.ssl_server_dn_match=false in the JDBC tab of ATP/ADW data server.
jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=servername)
(PORT=
2484))(CONNECT_DATA=(SERVICE_NAME=servicename))
(SECURITY=(SSL_SERVER_CERT_DN=\
"CN=server_test,C=US\")))
10-1