PackforSAP 6

WebSphere DataStage Pack for SAP R/3

Version 6.0.0
Integration Guide for WebSphere DataStage Pack for SAP R/3

for WebSphere DataStage, Version 8
SC18-9954-01
WebSphere DataStage Pack for SAP R/3
®

Version 6.0.0
Integration Guide for WebSphere DataStage Pack for SAP R/3

for WebSphere DataStage, Version 8
SC18-9954-01
Note
Before using this information and the product that it supports, read the information in “Notices” on page 103.
© Ascential Software Corporation 2004, 2005.

© Copyright International Business Machines Corporation 2006, 2007. All rights reserved.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
Contents
Chapter 1. Overview . . . . . . . . . 1 Listener subsystem . . . . . . . . . . . . 34
Terminology . . . . . . . . . . . . . . 1 Listener manager . . . . . . . . . . . 34
Listener server . . . . . . . . . . . . 34
Chapter 2. ABAP stage . . . . . . . . 3 Job run mode . . . . . . . . . . . . . 36
Persistent staging area . . . . . . . . . . 37
Functionality . . . . . . . . . . . . . . 3
Configuration files . . . . . . . . . . . 39
Installation prerequisites . . . . . . . . . . 4
IDoc extract for SAP R/3 Enterprise stage . . . . 42
Integrating WebSphere DataStage with SAP R/3
Bookmarking the PSA . . . . . . . . . . 43
enterprise systems . . . . . . . . . . . . 4
Creating jobs for IDoc types . . . . . . . . . 44
Importing the RFC and authorization profile . . 4
Defining the WebSphere DataStage connection to
Creating a job . . . . . . . . . . . . . . 6
SAP . . . . . . . . . . . . . . . . . 44
Defining stage properties . . . . . . . . . 7
Stage General page . . . . . . . . . . . 45
Defining character set maps . . . . . . . . 7
Selecting IDoc types . . . . . . . . . . . 47
Defining output properties. . . . . . . . . 7
Defining IDoc type properties . . . . . . . 48
Creating an ABAP extraction program with RFC data
Specifying SAP R/3 versions . . . . . . . 49
transfer . . . . . . . . . . . . . . . . 8
Defining IDoc extract output links . . . . . . . 50
Creating the RFC destination . . . . . . . . 8
General tab for the Output page . . . . . . 51
Configuring RFC data transfer . . . . . . . 9
Adding columns . . . . . . . . . . . . 51
Configuring MPP ABAP extraction jobs by using
Saving IDoc metadata definitions . . . . . . 52
RFC . . . . . . . . . . . . . . . . 9
Troubleshooting ABAP extraction jobs . . . . 10
Output General page . . . . . . . . . . . 10 Chapter 4. IDoc Load stage . . . . . . 53
Defining SAP connection and logon details . . . 11 Configuration requirements . . . . . . . . . 54
Defining connection properties . . . . . . . 12 IDoc load for SAP R/3 stage . . . . . . . . 54
Selecting the data transfer method . . . . . . . 13 Selecting the WebSphere DataStage connection to
Running programs in the background . . . . 15 SAP . . . . . . . . . . . . . . . . . 55
Output ABAP program page . . . . . . . . 15 Stage General page . . . . . . . . . . . 55
Building SQL queries . . . . . . . . . . . 17 Selecting IDoc types . . . . . . . . . . . 57
Build Open SQL Query Tables page . . . . . 18 IDoc Load Packet Size field . . . . . . . . 58
Build Open SQL Query Select page . . . . . 19 IDoc Components area . . . . . . . . . 58
Build Open SQL Query Where page . . . . . 19 Defining character set mapping . . . . . . . . 58
Build Open SQL Query Having page . . . . . 20 Defining IDoc load input links . . . . . . . . 59
Build Open SQL Query Order By page . . . . 20 Selecting the IDoc component to load . . . . 59
Build Open SQL Query SQL page . . . . . . 20 Connection Properties Connection and Logon
Building extraction objects . . . . . . . . . 21 Details . . . . . . . . . . . . . . . 60
Connecting to the SAP server . . . . . . . 21 Modifying columns . . . . . . . . . . . 60
Searching for tables . . . . . . . . . . . 21 Synchronizing columns . . . . . . . . . 61
Specifying the SQL condition for extraction
objects . . . . . . . . . . . . . . . 24 Chapter 5. BAPI stage . . . . . . . . 63
Using job parameters . . . . . . . . . . 25 Functionality . . . . . . . . . . . . . . 63
Program Generation Options dialog . . . . . . 25 Building a job . . . . . . . . . . . . . 65
Output Columns page . . . . . . . . . . . 25 Defining stage properties . . . . . . . . . . 65
Completing the job . . . . . . . . . . . . 26 Defining character set maps . . . . . . . . 65
SAP R/3 Table Type support . . . . . . . . 26 Defining SAP connection and logon details . . . 66
Data type support . . . . . . . . . . . . 26 Selecting the WebSphere DataStage connection to
Selection criteria data types . . . . . . . . 26 SAP . . . . . . . . . . . . . . . . 66
SAP R/3 Enterprise to WebSphere DataStage Defining connection properties . . . . . . . 67
data type conversion . . . . . . . . . . 27 Defining Input properties . . . . . . . . . . 67
Validating runtime jobs . . . . . . . . . 28 Input General page . . . . . . . . . . . 68
Filtering business objects . . . . . . . . . 68
Chapter 3. IDoc Extract stage . . . . . 31 BAPI Explorer dialog . . . . . . . . . . 68
Functionality . . . . . . . . . . . . . . 31 Defining BAPI interfaces . . . . . . . . . 68
Working with WebSphere DataStage to process SAP Defining log files . . . . . . . . . . . 69
IDocs . . . . . . . . . . . . . . . . 31 Input Columns page . . . . . . . . . . 70
Configuration requirements . . . . . . . . . 33 Defining output properties . . . . . . . . . 70
Runtime components . . . . . . . . . . . 34 Output General page . . . . . . . . . . 71
© Copyright IBM Corp. 2006, 2007 iii

Output BAPI page . . . . . . . . . . . 71 Installing the RFC function code manually . . . . 86
Output Read Logs page . . . . . . . . . 72 Configuring the SAP/Dispatch/Gateway service 89
Output Columns page . . . . . . . . . . 73 Troubleshooting for RFC installation for SAP R/3 93
Completing the job . . . . . . . . . . . . 73 Packaged extractions . . . . . . . . . . . 94
Runtime component . . . . . . . . . . . 73
Chapter 8. Packaged extractions for
Chapter 6. WebSphere DataStage ABAP . . . . . . . . . . . . . . . 95
Administrator for SAP Utility . . . . . 75
WebSphere DataStage connections to SAP R/3 page 75 Chapter 9. File permissions for IDoc
Defining WebSphere DataStage connections to Extract stage . . . . . . . . . . . . 97
SAP R/3 . . . . . . . . . . . . . . 77
Configuring connections and IDoc types . . . . . 97
Connection and Logon Details page . . . . . 77
Running WebSphere DataStage jobs . . . . . 98
Load balancing . . . . . . . . . . . . 77
Unconfigured IDoc types . . . . . . . . . 99
IDoc Listener Settings page . . . . . . . . 78
Configuring IDoc cleanup and archiving . . . 99
WebSphere DataStage Job Options for IDocs page 79
Recommendations . . . . . . . . . . . 99
Reading IDoc metadata . . . . . . . . . 79
Importing new connections . . . . . . . . 79
Importing IDoc type configurations . . . . . 80 Accessing information about the
Exporting connections . . . . . . . . . . 80 product . . . . . . . . . . . . . . 101
IDoc Types dialog . . . . . . . . . . . 81 Providing comments on the documentation . . . 101
IDoc Log dialog . . . . . . . . . . . . 81
Notices . . . . . . . . . . . . . . 103
Chapter 7. SAP Authorization Trademarks . . . . . . . . . . . . . . 105
Requirements for ABAP . . . . . . . 83
SAP authorization requirements . . . . . . . 83 Index . . . . . . . . . . . . . . . 107
Creating an authorization profile manually . . . 83
iv Integration Guide for WebSphere DataStage Pack for SAP R/3

Chapter 1. Overview
The IBM® WebSphere® DataStage® Pack for SAP R/3 includes the following stages
and utility:
v ABAP Extract. Lets WebSphere DataStage extract data from the R/3 Repository
using the ABAP extraction program generated by the stage.
v IDoc Extract. Lets WebSphere DataStage capture IDocs from R/3 source systems
to be used as source data for WebSphere DataStage job data streams.
v IDoc Load. Generates IDocs to load data into R/3 Enterprise.
v BAPI. Loads data into and extracts data from R/3 Enterprise.
v Administrator for SAP. Manages the configurations of R/3 connection and IDoc
type objects.
Terminology
The terms used in the IBM WebSphere DataStage Pack for SAP R/3 to describe all
the stages are defined below:
Term Description
ABAP Advanced Business Application Programming. The language developed by
SAP for application development purposes. All R/3 applications are
written in ABAP.
BAPI Business Application Programming Interface. A precisely defined interface
providing access to processes and data in business application systems.
BAPIs are defined as API methods of SAP objects. These objects and their
methods are stored in the Business Objects Repository.
BOR Business Object Repository, which is the object-oriented Repository in the
SAP BW system. It contains, among other objects, SAP Business Objects
and their methods.
Business Object
The representation of a business entity, such as an employee or a sales
order, in the SAP Enterprise system.
Control record
A special administrative record within an IDoc, one for each IDoc. The
control record contains a standard set of fields that describe the IDoc as a
whole.
ERP Enterprise Resource Planning business management software.
IDoc Intermediate document. An IDoc is a report, that is, a hierarchal package of
related records, generated by R/3 Enterprise in an SAP proprietary format.
An IDoc, whose transmission is initiated by the source database, exchanges
data between applications.
IDoc type
Named metadata describing the structure of an IDoc that is shared across
databases. It consists of a hierarchy of segment record types.
MATMAS01
An example of an IDoc type.
© Copyright IBM Corp. 2006, 2007 1

PSA Persistent Staging Area.
R/3 Real time/three tiers.
RFC Remote Function Call. The SAP implementation of RPC (Remote Procedure
Call) in ABAP. It calls a function module that runs on a different system
from the calling function. The Remote Function Call can also be called
from within the same system, but usually the caller and callee are
dispersed.
RFM Remote Function Module. A function that belongs to a BOR object type
and has a BAPI method name.
SAP Systems, Applications, and Products in Data Processing. SAP is a product
of SAP AG, Walldorf, Germany.
SCM Supply Chain Management. The solution that tracks financial,
informational, and materials processes and identifies processing exceptions.
Segment
A record within an IDoc that is identified by a segment number.
Segment type
A named record definition for segments within an IDoc that is one level in
the hierarchy of segment types within an IDoc type.
tRFC port
Transactional RFC port.
Variant
A collection of predefined criteria, similar to a group of values used as
parameters.
Variants are attached to various processes used by WebSphere DataStage
for the ABAP program process, for example. The ABAP Program
referenced by the ABAP Program process itself has a variant attached to it.)
2 Integration Guide for WebSphere DataStage Pack for SAP R/3

Chapter 2. ABAP stage
The ABAP Extract stage lets WebSphere DataStage extract data from the R/3
Repository using the ABAP extraction program generated by the stage.
It describes the following for the ABAP Extract stage:

v Functionality
v Installation prerequisites
v Integrating WebSphere DataStage with R/3 Enterprise systems
v Creating a WebSphere DataStage job
v Defining ABAP Extract stage properties
v Extracting data from R/3 Enterprise
v R/3 Enterprise table type support
v R/3 Enterprise data type support
The ABAP Extract stage lets your company maximize its existing investments in
large ERP systems by complementing SAP standard software and consulting
services. It does this by automatically generating the ABAP program to extract data
from the R/3 Repository. The ABAP Extract stage lets users of all levels efficiently
build an extraction object, then generate an extraction program written in the SAP
proprietary ABAP programming language. Or, you can use an SQL query to
generate the ABAP program.
See the Terminology section for a list of the terms used in this section.
Functionality
The ABAP Extract stage has the following functionality and enhancements:
v Lets you choose and define SAP connections using the GUI. You can select an
IBM WebSphere DataStage connection to SAP instead of entering the information
into the stage interface.
v Uses the ABAP Program page for all ABAP-related functionality.
v Lets you view the development status of the ABAP program.
v Lets you view referenced and referring tables.
v Lets you use the SQL Query Builder or an extraction object to generate the
ABAP program.
v Lets you overwrite the ABAP program as you save it to R/3.
v Uses the Data Transfer Method page for the functionality of the data transfer
methods.
v Uses the functionality of the former Access page for RFC-connection failures as
you exit the stage editor.
v Lets you synchronize and validate R/3 columns and WebSphere DataStage
columns.
v Supports NLS (National Language Support). For information, see WebSphere
DataStage Server Job Developer Guide.
v Supports the SAP R/3 remote function call (RFC) data transfer method on
Unicode systems.

The following functionality is not supported:
v Data transformations or mappings. Use the Transformer stage to do this.
v The use of sub-queries in the WHERE and HAVING clauses for SQL queries.
ABAP program is automatically generated to extract data from the SAP R/3
repository. This is done by the ABAP Extract stage and is explained below:
1. Link 1. The GUI client component logs on to the SAP R/3 Enterprise
application server and retrieves metadata information.
2. Link 2. The GUI client component reads and writes job properties (for example,
extraction object, ABAP program, logon information, and so on) between the
GUI client component and the WebSphere DataStage repository.
3. Link 3. The runtime server component reads job properties from the
WebSphere DataStage repository.
4. Link 4. The runtime server component makes a remote function call (RFC) to
run the ABAP program and processes the generated data set.
Installation prerequisites
In addition to the software requirements for the Pack, the following are required
for installing the ABAP Extract stage:
v FTP server on the machine where the temporary extraction files from the R/3
Enterprise are stored
v Database account with read access to the R/3 Enterprise data dictionary tables
Integrating WebSphere DataStage with SAP R/3 enterprise systems

In addition to installing the client and server components you also need to
integrate WebSphere DataStage with the R/3 Enterprise system. Do this by
installing or creating the following two components on the R/3 Enterprise system:
v An RFC-enabled function module on the R/3 Enterprise system from which data
is extracted.
v An appropriate profile that can be assigned to the users. Profiles maintain a
secure environment by providing the correct authorizations to the users
according to SAP practices.
The RFC source file (ZDSRFC.TXT for non-unicode and ZDSRFCU.txt for unicode
SAP Systems) required to install the RFC is on the same CD as the GUI client
component. The text file is in the RFC directory.
The R/3 Administrator must install these two components by using the Transport
Management System (Transaction STMS), which is the recommended method
described in the next section. You can also create them manually by following
step-by-step procedures (see the following instructions and those in Creating an
authorization profile manually).
Importing the RFC and authorization profile

The R/3 Administrator must copy the cofiles and data files shipped with the client
CD to the appropriate directory on the R/3 Enterprise host machine (see the next
section, Copying cofiles and data files).
You can then import them within R/3 Enterprise by doing one of the following:

v Using transaction STMS to access the Transportation Management System
v Using the tp import transport control program at the operating system level
Copying cofiles and data files

The cofiles and data files must be copied from the client installation CD to the
appropriate directory on the R/3 Enterprise machine before importing them. To
copy these files:
1. Log on to the R/3 Enterprise machine as <SID>adm where <SID> is the system
number, for example, C46.
2. For non-Unicode SAP systems, copy data files R900323.P46 (for RFC) and
R900337.P46 (for authorization profile) and cofiles K900323.P46 and
K900337.P46 (for Unicode, copy data files R900018.U47 and R900028.U47 and
cofiles K900018.U47 and K900028.U47) from \trans\cofiles and \trans\data
directories on the client CD to /usr/sap/trans/cofiles and /usr/sap/trans/data
directories respectively on the R/3 Enterprise system by doing one of the
following, depending on your network configuration:
a. Accessible file system. If the file system for your R/3 Enterprise system is
visible (for example, by NFS) on your WebSphere DataStage client system,
you can copy the K9nnnnn.xxx cofiles using a DOS command or Windows®
Explorer. Copy them from trans\cofiles on the client CD to
/usr/sap/trans/cofiles/ on the R/3 Enterprise system.
b. Unmapped network drive. If the R/3 Enterprise system is accessible by the
network, but it is not mapped as a network drive on the WebSphere
DataStage client system, you can use FTP in binary mode to transfer the
files using the directories specified in step a.
c. Inaccessible file system. If the R/3 Enterprise system is not accessible by
the network from the WebSphere DataStage client system, you can mount
the WebSphere DataStage client installation CD directly on your R/3
Enterprise system and copy the files specified in step a. Because of
differences in CD file system formats on UNIX® platforms, the path names
may vary as you mount the CD as follows:
LINUX Platforms. The file names are in lower case. Copy the cofiles to their
destination in upper case.
HP-UX Platform. The file names and directory names are in upper case, and
the file names have `;1’ appended to them. Copy the cofiles to their
destination without this `;1’ appendage.
Importing transport requests

To import the transport request, the R/3 Enterprise Administrator can use one of
the following two methods:
Importing using Transport Management System (using transaction STMS):
To successfully import the transport request, the R/3 Enterprise Administrator

must do the following:
1. Log onto the target system and client as the R/3 Enterprise Administrator.
2. Create a development class named ZETL that is assigned to a transport layer
according to the system landscape for your company. You can use transaction
SE80 to create this class.
3. Enter transaction STMS. The initial Transport Management System window
opens.
4. Choose Overview > Imports. The system reads the import queues.
Chapter 2. ABAP stage 5
5. Select <SID> (or double-click <SID> in the queue column), then choose import
queue>Display.
6. Choose Extras > Other requests > Add to add the transport request to selected
queue.
Type the transport request number in the next window as P46K9 nnnnn and
continue.
Repeat this step for the P46K9 nnnnn transport request.
7. Select the transport request in the import queue, and choose Request > Import.
Type the client number to which to import, and continue.
Repeat this step for each transport request.
8. Verify that the Z_RFC_DS_SERVICE RFC function module and
Z_DS-PROFILE profile are successfully imported and activated.
Importing at the operating system level:
If you are importing at this level, you can do the following (instead of the previous
steps 3-7):
1. Log onto the target system and client as the R/3 Enterprise Administrator.
2. Create a development class named ZETL that is assigned to a transport layer
according to the system landscape for your company. You can use Transaction
SE80 to create this class.
3. Add the transport request to the buffer using the following syntax:
tp addtobuffer <transport request> <SID>
For example, tp addtobuffer P46K9 nnnnn DD1
4. Import the transport request using the following syntax:
tp import <transport request> <SID> client=<NNN>
where <NNN> is a client number like 800.
For example, tp import P46K9nnnnn DD1 client=800
5. Verify that the Z_RFC_DS_SERVICE RFC function module and
Z_DS-PROFILE profile are successfully imported and activated.
Creating a job
To create and build your job:
1. Create a WebSphere DataStage job using the IBM WebSphere DataStage and
QualityStage Designer Client.
2. Create a new ABAP Extract stage.
3. Define the properties for the ABAP Extract stage.
4. Define the R/3 Enterprise connection information.
5. Select the data transfer method.
6. Specify the ABAP program parameters.
7. Define the data to be extracted.
8. Compile the job.
These tasks are described in detail in the following sections.

Defining stage properties
When you edit the ABAP Extract stage, the ABAP Extract Stage dialog opens with
the Stage page on top.
The stage dialog has two pages:

v Stage. Displays the name of the stage you are editing. You can define connection
and logon details to a target and source SAP system. You can also describe the
purpose of the stage in the Description field of the General tab, which appears
by default. This page is similar to that in other R/3 and BW stages except the
connection information appears at the stage level, not at the link level.
The NLS tab defines a character set map to use with the stage, if NLS is
enabled. For details, see Defining character set maps.
v Output. Specifies the method and details to connect to the R/3 Enterprise
system, the options that determine how the ABAP extraction program is run,
options to configure how the extracted data is transferred to the WebSphere
DataStage server system, and lets you view metadata for the R/3 Enterprise
fields. For details, see Defining output properties.
Defining character set maps

You can optionally define a character set map for a ABAP Extract stage using the
NLS tab.
You can change the default character set map defined for the project or the job by
selecting a map name from the list. This tab also has the following components:
v Show all maps. Lists all the maps supplied with WebSphere DataStage. Maps
cannot be used unless they are loaded using the WebSphere DataStage
Administrator Client.
v Loaded maps only. Displays the maps that are loaded and ready to use.
v Use Job Parameter . Lets you specify a character set map as a parameter to the
job containing the stage. If the parameter has not yet been defined, you are
prompted to define it from the Job Properties dialog.
v Allow per-column mapping. Lets you specify character set maps for individual
columns within the table definition. If per-column mapping is selected, an extra
property, NLS Map, appears in the grid in the Columns tab.
For more information about NLS or job parameters, see WebSphere DataStage
documentation.
Defining output properties

To define output properties, click the Output page. It has the name of the output
link in Output name. Click Columns to see a list of columns for the link.
This page also has the following tabs:

v General. Specifies the method and details to connect to R/3 Enterprise (see the
Output General page for details).
v Data Transfer Method (formerly the Runtime page). Specifies how to transfer
extracted data to the WebSphere DataStage server (see Selecting the data transfer
method).

v ABAP Program (formerly the ABAP/4 Options page). Specifies the parameters
for ABAP program operations and lets you see the status of the program. You
can also log on to SAP to review code without leaving the application by
clicking ABAP Workbench (formerly Launch SAP GUI on the Output General
page). For details about this page, see Output ABAP Program page.
v Columns. Lets you view the metadata of the R/3 Enterprise field that
corresponds to the currently selected WebSphere DataStage column. You can also
synchronize and validate columns (see the Output Columns page).
Creating an ABAP extraction program with RFC data transfer

The ABAP Extract stage supports the SAP R/3 remote function call (RFC) data
transfer method.
Transactional RFC (tRFC) data transfer uses asynchronous communication and

supports parallel data processing. Queued RFC (qRFC) uses serialized
communication and supports sequential data processing.
To create an ABAP extraction program with RFC data transfer:

1. Create an RFC destination on SAP R/3.
2. Create an ABAP extraction stage that uses RFC as the data transfer method.
3. On the Output page, use the ABAP Program tab to build an SQL query to be
used in the ABAP extraction program.
4. On the Output page, use the ABAP Program tab to validate the program.
5. On the Output page, use the ABAP Program to load the program to SAP R/3.
6. On the Output page, use the General tab to validate the connection.
7. Use WebSphere DataStage to save, compile, and run the job.
8. Use SAP R/3 transaction SMQS to register the RFC destination.
9. On the SAP R/3 server, verify the data transfer results by calling the SMQS
transaction:
a. Select the RFC destination.
b. Depending on the data transfer type in the ABAP program, click TRFC
Monitor or QRFC Monitor.
c. Verify that the RFC Monitor displays no entries for the RFC destination (the
ABAP extraction program).
Creating the RFC destination

Create and register an SAP R/3 remote function call (RFC) destination to use with
an ABAP extraction program.
To create the RFC destination:

1. Log in to the SAP R/3 server to create the RFC destination.
2. Call transaction SM59 to create the RFC destination:
a. Specify the name of the ABAP Extract program as the RFC destination.
b. Specify connection type T (TCP/IP connection).
c. In the technical settings, click Registered Server Program.
d. In the special options, click Unicode.
3. Call transaction SMQS to register your destination.
a. Select the RFC destination name in the registration table and click
Registration.

b. Specify the following registration parameters:
MAXCONN
The number of available connections. This number cannot exceed
the number nodes running the job in parallel.
MAXTIME
The connection timeout limit.
NO-TRFC
Leave this field blank.
Continue by creating an ABAP extraction stage that uses the RFC data transfer
method.
Configuring RFC data transfer

Configure the ABAP extraction program to use the remote function call (RFC) data
transfer method.
To configure RFC data transfer:

1. Create an ABAP extraction stage.
2. Set the data transfer parameters in the Data Transfer Method tab of the Output
page.
a. In the Data Transfer Method field, click RFC.
b. Configure the remaining fields. See “Selecting the data transfer method” on
page 13.
3. Continue defining the ABAP program by defining the SQL query to extract the
data.
a. On the Output page, use the ABAP Program tab to define the SQL query to
extract the data, as described in “Building SQL queries” on page 17.
b. On the Output page, use the ABAP Program tab to load the program to
SAP R/3, as described in “Output ABAP program page” on page 15.
c. On the the Output page, use the ABAP Program tab to validate the
program, as described in “Output ABAP program page” on page 15.
d. On the Output page, use the General tab to validate the stage, as described
in “Validating runtime jobs” on page 28.
4. Use WebSphere DataStage to save, compile, and run the job.
Configuring MPP ABAP extraction jobs by using RFC

You can configure Linux® and UNIX systems to use massively parallel processing
(MPP).
For best performance, define two MPP system nodes for each processor on the SAP
R/3 system.
To configure MPP on a Linux or UNIX system:

1. Ensure that each server has identical installations of IBM Information Server,
the same operating system, and the same directory structure.
2. Install WebSphere DataStage Pack for SAP R/3 on each MPP node.
3. Create identical project names on each MPP node.
4. Create identical job connection names on each MPP node.
5. Create identical configuration files on each server, and set the parameter
APT_CONFIG_FILE to point to the configuration file location. For example:

APT_CONFIG_FILE=/opt/IBM/InformationServer/Server/Configurations/
parallelsa.apt
6. Configure the /etc/hosts.equiv file. For example, to support the MPP nodes
dsserv1 and dsserv2, the /etc/hosts.equiv file must contain:
dsserv1
dsserv2
See the description of configuring the parallel engine in IBM Information Server
Planning, Installation, and Configuration Guide for details.
7. Configure the remote shell test using the rsh command. For example:
rsh dsserv1 uptime
rsh dsserv2 uptime
8. Ensure that the file /etc/hosts has an entry for each node.
9. Review the description of the parallel engine configuration file in IBM
WebSphere DataStage and Quality Stage Parallel Job Developer Guide.
10. Enable ABAP parallel job execution on a non-conductor node in an MPP
system by creating an empty file named slave in the project directory on the
non-conductor node. For example, if the project is named sapr3 and the
conductor node is dsserv1, create the empty slave file on node dsserv2 as
follows:
/opt/IBM/InformationServer/Server/Projects/sapr3/slave
See IBM Information Server Planning, Installation, and Configuration Guide for
information about configuring MPP on Microsoft® Windows systems.
Troubleshooting ABAP extraction jobs

When an ABAP extraction job fails to extract data, use these steps to begin to
diagnose the cause of the error.
1. Use SAP R/3 transaction SM59 to verify the destination for the transported
data. Verify that the destination is configured correctly and that Unicode is
specified in the special options.
2. Use SAP R/3 transaction SM58 to look for and delete outstanding failed
transactions.
3. Use SAP R/3 transaction SMQS to verify that the timeout connection for
destination CPIC is appropriately set to be longer than the job execution.
4. Use SAP R/3 transaction SM59 to verify that there is connectivity between the
WebSphere DataStage server and the SAP R/3 system:
a. Run the ABAP extraction job.
b. Test the connection.
c. Run the Unicode test.
5. Use SAP R/3 transaction SE38 to manually run the ABAP extraction program:
a. Change the run method to SAP administrator runs the program manually.
b. Regenerate the program and run the program.
c. Manually run the program from SAP R/3 transaction SE38.
Output General page

The General page opens by default.

This page works similarly to the General tab of the BW Load Input page for the
BW Load stage. You can use the WebSphere DataStage server to access the list of
connections to SAP, which is stored in a file.
By default, a newly created stage uses the connection most recently selected by the
current user for other stages of this type. Since the number of R/3 systems
accessed by a given WebSphere DataStage installation is likely to be small, the
default connection is generally correct.
Enter the following information on the General tab:

v DataStage Connection to SAP. The WebSphere DataStage connection to the R/3
Enterprise system that is defined on the WebSphere DataStage server machine
and shared by all WebSphere DataStage users connected to that machine. The
fields in this area are read-only and are obtained from the connection that you
selected.
Name. The name of the selected connection to the R/3 Enterprise system that
generates the data to be extracted.
Select. Click to choose a WebSphere DataStage connection to the R/3 Enterprise
system. The selected connection provides all needed connection and default
logon details that are needed to communicate with the corresponding R/3
Enterprise system. This opens the Select DataStage Connection to SAP dialog.
You can add new entries here and modify or delete existing entries from the list
of connections.
Description. Additional information about the selected connection.
Application Server. The name of the host system running R/3.
System Number. The number assigned to the R/3 Enterprise system used to
connect to R/3.
v SAP Logon Details. User Name, Client Number, and Language default to the
values last entered by the current user for the ABAP Extract stage.
User Name. The user name that is used to connect to SAP.
Password. A password for the specified user name.
Client Number. The number of the client system used to connect to SAP.
Language. The language used to connect to SAP.
v Description. Enter text to describe the purpose of the stage.
v Validate Stage (formerly Job Validation). Click to open the Validation Stage
dialog. The validation process begins automatically when the dialog opens.
Extraction Object, Review Code, and Launch SAP GUI (formerly on the General
page) are now on the ABAP Program page.
Defining SAP connection and logon details

Click Select on the General tab of the Output page to choose a WebSphere
DataStage connection to the R/3 Enterprise system. This opens the Select
DataStage Connection to SAP dialog.
The selected connection provides all needed connection and default logon details
to communicate with the corresponding R/3 Enterprise system. You can add new
entries here and modify or delete existing entries from the list of connections.

Although SAP logon details for the ABAP Extract stage are not stored on the
server, the stage can share SAP connections created by other R/3 stages you create,
such as the BAPI, IDoc Load, and IDoc Extract stages.
New, Properties, and Remove are administrative connection operations. They let
you manage the list of connections that is maintained on the WebSphere DataStage
server machine from this dialog:
v New. Click to open the Connection Properties dialog, which lets you define the
properties for the new connection. It is added to the list of connections. Here
you can specify SAP connection details and default logon details for the new
connection. The Connection and Logon Details page opens by default (see the
following section).
v Properties. Click to open the Connection and Logon Details page of the
Connection Properties dialog, showing the properties of the selected connection.
This is the same dialog that is opened when you click New, but in this context
the connection name is read-only (see Defining connection properties).
v Remove. Click to delete the selected connection after your confirmation.
These administrative operations function similarly to the corresponding buttons on

the DataStage Connections to SAP page of the WebSphere DataStage Administrator
for SAP utility (see Administrator for SAP Utility).
Defining connection properties

The Connection Properties dialog has the Connection and Logon Details page.
Use the dialog to create a new SAP connection, or to view the properties of the
currently selected connection, depending on whether you open it using New or
Properties on the Select WebSphere DataStage Connection to SAP dialog
respectively.
To define the WebSphere DataStage connection to SAP:

1. Specify the SAP connection details:
v Application Server. The name of the R/3 server.
v System Number. The system number of the R/3 instance.
v Router String. Optional. The string used to connect to the remote SAP
server. If you use a SAP router to access your system, you must include a
fully-qualified node name for your RFC server machine in the SAP router
table.
2. Select Use load balancing to use load balancing when connecting to R/3. The
Application Server and the System Number controls are replaced by Message
Server, System ID, and Group controls so that connection details specific to
load balancing can be entered.
3. Specify the CPI-C Connection details. These values are read-only and match the
entries in SAP Connection Details if Use load balancing is selected. Otherwise,
the fields are editable and define a non-load balancing connection that can be
used at runtime for CPI-C processing.
Load balancing
Select Use Load balancing on the Connection and Logon Details page of the
Connection Properties dialog (see Connection and Logon details page) to balance
loads for SQL queries when connecting to the SAP R/3 system. Load balancing at
design time works as follows:

v R/3 lets you make logon connections through a message server. The message
server uses an algorithm that considers server workload and availability to
choose an appropriate application server to handle the logon.
v When connections are configured, you can choose a load balancing connection to
a message server rather than a specific R/3 instance to retrieve and validate
IDoc types and metadata, for example.
Note: Load balancing cannot be used at runtime.
Selecting the data transfer method

The data transfer method determines how data is sent from the SAP R/3 server to
DataStage.
Use the Data Transfer tab on the Output page to transfer data. It is similar to the
Runtime page for the previous version of the ABAP Extract stage. It shows only
controls relevant for the selected transfer method.
This page displays the selected Data Transfer Method, which controls how to
process the data set. Choose one of three data transfer methods:
v CPI-C Logon. Default. Transfers data from the SAP application server directly to
the WebSphere DataStage server, without using FTP or other utilities.
The Local File → option has no effect, that is, no flat files are generated in the
SAP application server. The SAP user name must be the CPI-C user type.
v FTP. Uses the FTP service in the SAP application server to get the data set from
the SAP application. (The buttons for ABAP program loading in the former
version are now on the ABAP Program page.)
Choosing FTP enables the FTP Logon fields, which are required. Enter the path
name for the data file on the remote system to use during run time in Path of
Remote File. This path name is used to access the data file when you may not
have direct access to the data file. The data file contains the data extracted from
R/3. This file is transferred back to the WebSphere DataStage server system.
Alias for Remote Path. Optional. You cannot access the data file using the path
name in the Path of Remote File field, for example, if the system administrator
has restricted access to a directory in the path. In this case, use a relative path
name to access the file. If the WebSphere DataStage server cannot transfer the
data file using the path specified in the Path of Remote File field, it uses the
path in the Alias for Remote Path field to access the file.
Alternate Host Name. Specify an alternate host name for FTP to use as an alias
to access files on the application server’s machine (where the ABAP program
runs). This is necessary if, for security reasons, FTP cannot access files on the
application server’s machine using the application server name.
Run the program as a background process. Specifies whether to run ABAP
programs in the background (the default is cleared). For details, see Running
programs in the background.
Use SAP Logon Details. If selected for a CPI-C data transfer method, User
Name and Password are read-only and display the corresponding values from
the General page.
v Local File. Use this option if the file containing the data set cannot be accessed
using FTP. The resultant data set file can be placed on the WebSphere DataStage
server machine by the Administrator, and WebSphere DataStage can then access
it from there using the Local File option.

Choosing this option enables the Local Data File field and Browse . Type a
name for the data source file, or click Browse to search for a name. R/3
Enterprise needs write access to this file.
v RFC. Use this option to use the SAP R/3 remote function call (RFC) data
transfer method. Transactional RFC (tRFC) data transfer uses asynchronous
communication and supports parallel data processing. Queued RFC (qRFC) uses
serialized communication and supports sequential data processing.
Gateway Host
Specifies the host name of the system where the SAP gateway service is
running.
Gateway Service
Specifies the name of the SAP R/3 server gateway to be used by the
RFC destination. The standard Gateway Service is sapgw plus the system
number; for example, sapgw00. To display the gateway services, use the
SAP R/3 gateway monitor. See your SAP R/3 documentation.
Program ID
Specifies the program ID of the server program registered with SAP R/3.
The recommended program ID is the fully qualified job name.
RFC Destination
Specifies the name of the RFC destination created with SAP R/3
transaction SM59.
Use qRFC
Specifies serialized data processing using queued RFC (qRFC).
– Select the check box to use sequential qTRC processing.
– Clear the check box to use asynchronous transactional RFC (tRFC)
processing.
After changing this option, regenerate the ABAP extraction program and
recompile the job to use the new data transfer method.
qRFC queue name
Specifies the name of the RFC queue. The ABAP stage creates a queue
with this name automatically when you run the ABAP extraction
program. Data sent from SAP R/3 is collected in this queue in the order
in which it is sent. Use SAP R/3 transaction SMQS and click QRFC
Monitor to display the number of entries in the queue and to lock and
unlock the queue to stop and start data extraction. See your SAP R/3
documentation for details.
For example, you can use the qRFC data transfer method to control data
flow through a job that you run manually. Lock the qRFC queue to
allow it to fill with messages, manually start the ABAP extraction
program, and unlock the queue to send the data to the ABAP extraction
program.
Security Options
Specifies whether SAP R/3 Secure Network Communications (SNC)
security options are used. See your SAP R/3 documentation for details.

Running programs in the background
This option on the Data Transfer Method tab of the Output page specifies whether
to run ABAP programs in the background (the default is cleared). For details about
installing an RFC manually for this functionality, see “Installing the RFC function
code manually” on page 86
To run ABAP programs in the background:

1. Select Run the program as a background process for FTP data transfers from
the Output Data Transfer Method page. (This check box does not appear for
CPI-C transfers). This functionality lets you run ABAP programs indefinitely as
background processes without being timed out.
2. The stage runs the generated ABAP program as a background process within
R/3. You must provide a unique variant name if you want to pass job
parameters to the ABAP program. This facilitates the creation of a variant that
can be added to the background process during run time.
This variant is automatically deleted after the associated job completes or
aborts.
3. The runtime implements BAPI calls to:
v Log on to the R/3 system as an external user using the XBP interface
v Create a background process job
v Create a variant if necessary
v Add the ABAP program and corresponding variant to the job
v Schedule the job to run immediately
v Check the status of the background process job
v Delete the job and the associated variant after the job completes (or aborts)
v Log off from R/3
4. To monitor the status of the job from Scheduled to Finished, use the Job
Overview window in the SAP GUI.
The job name is the same as the name of the generated ABAP program. This
helps you easily identify the job.
Your job disappears from the list after the ABAP run time deletes it.
Output ABAP program page

The Output page ABAP Program tab constructs the properties and operations of an
ABAP program.
This includes program generation (formerly done using the Extraction Object
dialog on the Access General page), editing, saving, and validation.
This page has the following components:

v ABAP Program ID. Enter the program ID as you sequentially define required
properties on the tabs on the Output page, from left to right. Otherwise, you see
Program needs ID in ABAP Program Development Status.
v Generation Method includes three ways to create the ABAP program:
– Build SQL query. This option (the default) lets you define the data to be
extracted by constructing an SQL query. (It uses the same SQL Builder
interface as that for other stages.) It does this by causing Build to open the
Build Open SQL Query dialog, described in Building SQL queries section.

– Build extraction object. When you select this option, Build opens the Build
Extraction Object dialog, which disables Options. This method gives you
backward compatibility (the dialog is similar to the Extraction Object dialog
in the former version). This dialog includes OK and Cancel (instead of just
Close as in the previous version). Generate ABAP/4 and Connect no longer
exist. See Building Extraction objects for more information.
– Enter program as text If selected, Build, Options, and Generate Program are
disabled. In this case, you can open ABAP Editor (using Edit Program) to
create the program by hand. If you edit a generated program, Generation
Method changes to Enter program as text.
v Build. Opens a dialog to build an extraction object or SQL query, depending on
the selection generation method.
Click to open the Build Extraction Object dialog if Build extraction object is
selected as the Generation Method. (Options is disabled.)
The Build Extraction Object dialog is similar to the Extraction Object dialog in
the former version, except it uses OK and Cancel (instead of merely Close), and
Generate ABAP/4 and Connect no longer exist.
Opens the Build Fully Generated Query dialog if Build SQL Query is selected
as the Generation Method.
v Options. Lets you further customize the generated ABAP by opening the
Program Generation Options dialog. This option is only available if Build SQL
Query is selected.
v Load Method. Specifies how to load the ABAP program. The options depend on
the data transfer method:
– WebSphere DataStage job loads the program. (Program ready for runtime
load appears as status after program generation.)
– Current user loads the program from this dialog box.
– SAP administrator loads the program manually.
v Delete the program from R/3 after it runs.
v Run Method. Specifies how to run the ABAP program. Options depend on the
data transfer and the load methods.
– WebSphere DataStage job runs the program. Selected by default. If the data
transfer method is CPI-C, only this option appears. The same limitation
applies if you select WebSphere DataStage job runs the program as the load
method.
– SAP administrator runs the program manually. Appears if WebSphere
DataStage job loads the program is not selected.
v ABAP Program Development Status. Displays the following read-only status for
the ABAP program you are developing so you know the next step. For example:
– SQL query needs to be built. Appears when you specify an ID, but the
selected Generation Method is Build SQL query, and you need to build the
query.
– Program not used for ″Local File″ data transfer method. Appears when you
select Local File as the data transfer method (everything on this page is
disabled).
– Program needs to be generated. Appears when the program is not yet
generated, or if the relevant stage properties differ since the last generation of
the program (program generation formerly handled in the Extraction Object
dialog).
– Program needs to be loaded to R/3. Appears if you decline the system offer
to load the program.

– Program needs to be saved to a file. Appears if you decline the system offer
to save the program.
– Program loaded to R/3. Appears after you load the program to R/3.
– Program saved to file <file path>. Appears after you save the program to a
file.
– Program ready for runtime load. Appears after program generation when
Load Method is WebSphere DataStage job loads the program.
– Program needs ID. Appears if you need to provide ABAP Program ID.
v Generate Program. Generates the ABAP program, disabling the button. Build
and Options remain enabled, if Generation Method is Build SQL Query. If you
click Build or Options, and if the system knows about any program edits since
generation-time, you must first click Clear Program to delete the existing edited
program to use any changes to regenerate the program.
After generating the program, you can load the program to R/3 or save it to a
file, depending on the Load Method setting (unless the setting is WebSphere
DataStage job loads the program). If the program needs to be generated, the
development status is Program needs to be generated.
If you save or load the program, the development status is Program saved to
file <file path> or Program loaded to R/3, as appropriate.
The button is disabled after you generate the program, and all the previously
disabled buttons are enabled, including Edit Program and Editor Options.
v Load Program to R/3. Appears if you need to load the program to R/3.
v ABAP Workbench. Opens the SAP GUI, replacing Launch SAP GUI on the
General page in the former version.
v Save Program as File , Clear Program, and Save Program as File are the other
buttons you see, if appropriate.
v Edit Program . Lets you edit the program and save the modifications.
v Editor Options. Opens a dialog that lets you indicate what editor to use to edit
the program.
v Validate Program. Verifies the program syntax. If errors exist, the ABAP editor
for the stage opens and highlights the error, similar to validation done from the
editor.
v ABAP Workbench invokes the SAP GUI, (replacing Launch SAP GUI on the
General page in the previous version).
Note: When you save the ABAP program to R/3, you see a warning if a
program with the same name already exists on the R/3 system (you can
overwrite the program).
You can also click Load Program to R/3, Save Program as File , Clear program,
Edit program , Editor options , or Validate program if appropriate.
Building SQL queries

You can define the data to be extracted by constructing a SQL query.
The SQL Builder interface is the same as for other stages, for example, the Oracle
Call Interface stage.
If you select Build SQL Query as the Generation Method on the Output ABAP
Program page and click Build , the Build Open SQL Query dialog opens with the
Tables page on top.

The Build Open SQL Query dialog contains the Tables (the default), Select,
Where, Having, Order By, and SQL pages.
The generated SQL is customized to match the specific SQL syntax of the source
database and conforms to the SAP proprietary Open SQL syntax used by ABAP
programs.
The first five pages of the Build Open SQL Query dialog correspond to a clause in
the generated SQL. The SQL page shows the complete SQL statement as
determined by the settings in the preceding pages.
Build Open SQL Query Tables page

Use to select a table name from the available tables. This page includes the All
Tables (the default) and Search for Table tabs.
Click > and the selected table appears in the Selected tables list.
Join Type determines the type of join (see Join to -> buttons) between two or more
specified tables as follows. The tree representation of join nesting is described later.
v Inner >. Adds the table using inner join.
v Left >. Creates other types of joins.
v Right >. Creates other types of joins.
Click < to remove the selected table or join.
Click << to remove all tables and joins.
Show Related Tables displays tables related to the selected table through foreign
or primary keys. The related tables are listed in two folders that appear in a tree
structure below the selected table. These folders are labelled Referenced Tables
and Referring Tables. When related tables are added to the join tree (Selected
Tables), default join conditions are created automatically based on the key
relationships.
The join condition and join type of the selected join are shown in controls below
the join tree. Through these controls, you can modify the properties of the join.
Build Open SQL Query Tables All Tables page

The All Tables page includes the Available Tables control, which shows the R/3
tables organized according to the SAP application hierarchy. This hierarchy groups
the tables into folders and sub-folders based on the business category of the data
that is contained in the tables. (The organization of these folders is the same as that
in the SAP R/3 Module Tree dialog in the former version of this stage.)
Each table entry in Available Tables includes both the name and the Short Text for
the table to help you identify the entries.
Build Open SQL Query Tables Search for Table page

This page lets you perform a simple query to search for the appropriate table. The
tables resulting from such a search can be added one by one to the join tree for the
query (shown in Selected Tables).

Build Open SQL Query Select page
Use to select the specific columns to be included in the SQL statement.
The fields in Available columns include the Short Text for each field to help you
identify the ones you want.
Select the appropriate columns from Available columns.
Click > and the selected columns appear in the Selected columns list.
Click >> to add all available columns.
The Agg menu includes these options: SUM, AVE, MIN, MAX, and COUNT. The
menu lets you add new columns that apply one of these functions to the selected
available column.
Columns in the Selected columns list can be reordered using the MOVE UP and
MOVE DOWN buttons.
Columns in the Selected columns list can be removed using the < (remove
selected) and << (remove all) buttons.
Click Find to open a dialog that lets you search for a field in Available columns or
Selected columns, depending on where the focus is when you click Find.
Use the Key column check box to specify which columns should be used as key
columns for the link.
Build Open SQL Query Where page

Use to construct the SQL WHERE clause. Operators include: =, <>, <, <=, >, >=,
BETWEEN, NOT BETWEEN, IN, and NOT IN. For example:
When you click OK, the entire query is validated. The stage validates the syntax of
the Where and Having tabs and verifies the presence of at least one table and at
least one selected column. Errors are reported as warnings, but you can exit the
dialog without fixing the errors.
You can use WebSphere DataStage job parameters on the Where tab of the SQL
Builder by typing the parameter reference into one of the grid boxes. In this case,
the parameter references are in the standard format that is used elsewhere in
WebSphere DataStage (#PARAM#, where PARAM is the name of the job
parameter).
For example, the #LANG_LOW# and #LANG_HIGH# parameters specify the range
of values for the Where clause. You can click the SQL page to see the conversion of
parameter references to the format required by ABAP (see Build Open SQL Query
SQL page. This conversion is done automatically when the SQL is generated.
See Using job parameters for information about using WebSphere DataStage job
parameters with extraction objects.

Build Open SQL Query Having page
Use to construct the SQL HAVING clause.
When you click OK, the entire query is validated. The stage validates the syntax of
the Where and Having tabs and verifies the presence of at least one table and at
least one selected column. Errors are reported as warnings, but you can exit the
dialog without fixing the errors.
You can use WebSphere DataStage job parameters on the Having tab of the SQL
Builder by typing the parameter reference into one of the grid boxes. In this case,
the parameter references are in the standard format that is used elsewhere in
WebSphere DataStage (#PARAM#, where PARAM is the name of the job
parameter). You can click the SQL page to see the conversion of parameter
references to the format required by ABAP (see Build Open SQL Query SQL page.
This conversion is done automatically when the SQL is generated.
See Using job parameters for more information about using WebSphere DataStage
job parameters with extraction objects.
Build Open SQL Query Order By page

Use this page to construct the SQL ORDER BY clause.
The Available columns are the ones selected on the Select page. The columns
listed higher in the Order by columns list have a higher precedence in the
resulting sort order. You can modify the precedence by using the MOVE UP and
MOVE DOWN buttons.
To change Ascending to Descending (or vice versa), click Ascending in the list
control. A combo box appears that allows changes.
Build Open SQL Query SQL page

Use this page to display the SQL statement generated from the input to the
previous pages. The statement conforms to the SAP proprietary Open SQL syntax
used by ABAP programs.
The metadata is obtained directly from the R/3 system from which the data is
extracted.
The Open SQL statement is like the statement that appears in the generated ABAP
program, but without the INTO clause.
The SQL page displays the conversion of parameter references to the format
required by ABAP. This conversion is done automatically when the SQL is
generated.
Click OK to accept the SQL statement or Cancel to discard all the input.

Building extraction objects
You can define the data to be extracted by building an extraction object when you
use Build Extraction Object as your generation method.
To build the object:

1. Connect to the SAP server.
2. Search for tables.
3. Specify the SQL conditions.
4. Generate and review the R/3 Enterprise ABAP code.
5. Examine the extraction object metadata.
6. Upload the SAP ABAP code into SAP.
Each of these tasks is described in more detail in the following sections.
Connecting to the SAP server

To connect to the SAP server:
1. Enter the appropriate values on the General tab in the Output page.
2. Select Build → Extraction Object as the Generation Method on the Output
ABAP Program tab.
The Build Extraction Object dialog opens.
Searching for tables

Dialogs such as Extraction Object Editor, ABAP/4 Code Viewer, and so forth, have
a right-click shortcut menu and the usual buttons.
Use any of the following methods to search for an SAP table to add to the
extraction object. Click the appropriate button, choose the appropriate command
from the Extraction Object Editor shortcut menu, or use the shortcut keys.
v Add tables from Search (Alt+s)
v Add tables from Module Tree (Alt+m)
v Add tables from Logical Database (Alt+l)
Before learning how to add a table, read the next section about navigation.
Navigation
When you navigate the SAP R/3 Module Tree or the SAP R/3 Logical Database
Tree, double-click an entry to expand it. Because R/3 Enterprise is such a large
system, load metadata only when you want to interact with the system.
In the Extraction Object Tree, the root is the highest level and the field is the
lowest level. Since WebSphere DataStage processes input as a rowset, the second
level must only have one table.
The following navigation guidelines pertain to the Build Extraction Object dialog.
Common operations have corresponding buttons and shortcut menu commands.
v You cannot delete the root.
v You can relocate a table by selecting and dragging it to the desired location.

v A table can become a subtree of another table tree.
v You can delete tables by clicking Delete or choosing Delete from the shortcut
menu.
v You can move a column within one table, but you cannot move it outside that
table to another table.
v You cannot move columns to other columns.
v You can delete all or selected columns associated with a table by clicking Delete,
choosing Delete from the shortcut menu, or pressing Alt+e.
Adding a table
1. Click Add Tables → From Search or choose Add Tables from the Extraction
Object Editor shortcut menu. The R/3 Table Searching & Adding dialog box
opens.
2. To find a table, do one of the following:
v Type a full or partial table name using the * or % wildcard in the Table
Name field. For example, you can type t00, t00*, or t00%. You can use * or %
interchangeably in one table name.
v Type a description in the Table Description field. You can type a full or
partial name using a wildcard (* or %). This name is case-sensitive.
v Create a combination of table name and table description values. Use the
AND or OR option buttons to specify the Boolean relationships among the
selection criteria.
Click Search. The Search Results for Table grid is filled in. The dialog box
name is replaced by ″Searching result: n entries found.″
3. Do one of the following:
v Highlight a table, then click Table Definition, or choose Table Definition
from the Table List Operations shortcut menu. The Table definition dialog
opens.
v Highlight one or more table columns, then click Add to EO Tree or choose
Add to Extraction Object from the Table Definition Operations shortcut
menu. The columns are added to the Build Extraction Object dialog. Close
the Table definition dialog.
v Click Print, or choose Print from the Table List Operations shortcut menu to
send the search results to the printer.
v Click Save As, or choose Save As from the Table List Operations shortcut
menu to save the search results to a flat file.
4. To view the table contents, click Table Content, or choose Table Content from
the Table List Operations shortcut menu. The Table Contents dialog box
opens.
The table content list on the left under Select Field for Selection Criteria
displays the associated column details. You can search for a string in the field
name or a description using the Find What field, the Direction list, and Find.
Using Find What is especially helpful for searching large tables.
You must specify selection criteria in order to view the table contents.
a. To add a single value, select the column for which you are specifying the
condition. Select a comparison operator from the Operator list under Single
Value Operation.
Type the comparison value in the Value field.
Click Add Single Value. The selection criteria is added to the SQL
Selection Criteria box.

b. You can add a range of values in the same manner. Select the column for
which you are specifying the condition. Select INCLUDE or EXCLUDE
from the Operator list under Range Operation. Type the first value in the
range in the first Range of Values field and the last value in the range in
the second field. Click Add Range. The criteria is added to the SQL
Selection Criteria box.
Selecting INCLUDE or EXCLUDE produces output similar to the following:
v INCLUDE valueA, valueB will get condition
as field >= min(valueA, valueB)
AND
field <= max(valueA valueB),
v EXCLUDE valueA, valueB will get condition
as field > max(valueA, valueB)
OR
field < min(valueA, valueB),
You can create a complex condition by using a combination of single and
range values. Use the AND or OR option buttons to specify the Boolean
relationships among the selection criteria.
Note: If you make an error in specifying the condition, you can remove
the condition from the SQL Selection Criteria box by clicking Clear
Condition. This removes the entire selection criteria definition from the
box. You can also manually edit the SQL condition.
c. If you do not specify any fields, this stage by default loads every field back
into the Fields for Selection list. If you use RFC to connect to the SAP
application server and the table is large (that is, the record length is large),
then you can load nothing back. In this case, make sure that the record you
want to view is less than 512 bytes long. After you specify the selection
criteria, click View Contents. A Table Content dialog showing the table
contents opens. The rows are displayed in 500-row lots.
Select one of the following using the buttons or commands on the Table
Content Operations shortcut menu:
d. Click Save As to save the table contents to a flat file using the Save As
dialog box.
e. Click Print to send the table contents to the printer.
Close this dialog and the Table Content dialog to go to the R/3 Table
Searching & Adding dialog box.
5. After you verify that you have the correct table, select the table from the grid in
your R/3 Table Searching & Adding dialog box, and click Add to Extraction.
Select one of the following using the buttons or the shortcut menu on the R/3
Table Searching & Adding dialog box:
v Select Save As to save the table columns to a flat file using a standard Save
As dialog box.
v Select Print to send the table columns to the printer.
6. Close the R/3 Table Searching & Adding dialog box. The Build Extraction
Object dialog now shows the added table.
Navigating the SAP R/3 Module Tree

1. On the Build Extraction Object dialog, click Add Tables, then From Module
Tree, or choose Navigate Module Tree from the Extraction Object Editor
shortcut menu. The SAP R/3 Module Tree dialog opens.

2. Double-click any entry to expand it to the next level. Click Expand or Collapse
to expand or collapse the entire tree.
3. The lowest level of any node in a module tree is a table. Right-click a table to
see its table definition or table content, or add it to the Build Extraction Object
dialog.
You can select multiple fields or multiple tables at the same time, but you
cannot select both fields and tables at the same time.
Navigating the SAP R/3 Logical Database Tree

1. On the Build Extraction Object dialog, click Add Tables, then From Logical
Database, or choose Navigate Logical Database from the Extraction Object
Editor shortcut menu. The SAP R/3 Logical Database Tree dialog opens.
2. Double-click any entry to expand it to the next level. Click Expand or Collapse
to expand or collapse the entire tree.
3. The lowest level in the logical database tree is a table. Right-click a table to see
its table definition or table content, or add it to the Build Extraction Object
dialog.
You can select multiple fields or multiple tables at the same time, but you
cannot select both fields and tables at the same time.
Specifying the SQL condition for extraction objects

If you added tables to the extraction object, you can return to the Build Extraction
Object dialog to specify the SQL condition. To do this:
1. Click SQL Condition, or choose Specify SQL Condition from the Extraction
Object Editor shortcut menu. The SQL Condition Builder dialog box opens.
2. Select a column from the Table list. You can search for a string in the field
name as well as a description using the Find What field, the Direction list, and
Find. Using Find What is especially helpful for searching large tables.
3. Specify a single value for a field, or use the list to specify job parameters. Click
Add Single Value to add the selection criteria to the ABAP SQL Condition
box.
4. Specify a range of values for a field, or use the drop-down list to specify job
parameters. Click Add Range to add the selection criteria to the ABAP SQL
Condition box.
5. Click OK to add the condition to the Build Extraction Object dialog. You can
see the condition statement at the end of the table description in the tree.
6. If you create a join condition, specify the SQL condition for the child table. The
parent table is displayed in the Join Table list. Double-click the parent table to
list its fields in the Join Field list.
7. Select the column name from the Table list. Select the column on which you are
doing the join from the Join Field list. Click Add Relationship to add the
condition to the ABAP SQL Condition box.
You can specify additional columns for the join condition. Select the columns in
the same manner, and click either AND or OR to specify the Boolean condition.
8. Click OK to add the join condition to the Build Extraction Object dialog. The
condition is added to the table description in the tree.
Note: If you make an error in specifying the condition, you can remove the
condition from the ABAP SQL Condition box by clicking Clear Condition.

This removes the entire selection criteria definition from the box. You can also
manually edit the SQL condition. You can then review your SQL statement by
clicking View SQL.
9. Highlight any entry, and click Property on the Build Extraction Object dialog to
look at its properties and values in the Properties window.
Using job parameters

To use WebSphere DataStage job parameters from within the ABAP code that is
generated from the specified extraction object, you need to perform certain tasks
when designing your job.
1. Define the necessary job parameters in the WebSphere DataStage Designer
Client by choosing Edit > Job Properties. Enter job parameters on the
Parameters page.
In this example, LANG_LOW and LANG_HIGH are defined as job parameters
for this extraction job.
2. You can use job parameters in the SQL Condition Builder dialog by using the
syntax: DS_JOB_PARAM@job_parameter. In the following example, LANG_LOW
and LANG_HIGH specify a range of values in the SQL condition. Likewise,
you can use them in a single value or a join operation.
Click Add Range to add the range condition to the ABAP SQL Condition box.
3. Generate the ABAP code. Save and compile the job when your job design is
complete. When you run this job, WebSphere DataStage prompts for values for
each of the job parameters that are defined for this job. At run time, these
values are passed to the ABAP program that performs the data extraction from
R/3 Enterprise.
Program Generation Options dialog

Click Options on the ABAP Program tab of the Output page to open the Program
Generation Options dialog. It lets you control how the ABAP program is
generated by letting you enter lines of ABAP code that are automatically inserted
into the generated program.
This lets you regenerate the program with all your custom insertions after you
modify the SQL query.
This dialog has the following components:

v Additional Program Header Comments
v Additional Commands to Execute just before the Program Starts
v Additional Commands to Execute just before the Program Exits
v Set as Default Options. If selected, the text in the three edit controls appears as
the defaults for these controls when the current user creates new ABAP Extract
stages.
Output Columns page

This page contains the column definitions for the data being output on the chosen
link. For details about column definitions, see your WebSphere DataStage
documentation.

The list of columns is automatically synchronized with the SQL query (or the
extraction object) whenever you change the set of extracted tables and fields that is
specified in the SQL query (or extraction object). But since you can modify the
columns from the WebSphere DataStage Designer Client, Synchronize Columns is
useful.
In case columns are accidentally deleted or modified, Synchronize Columns lets

you align the column list with the extraction object.
Completing the job

Complete the definition of the other stages in your job according to normal
WebSphere DataStage procedures. Compile and run the job.
SAP R/3 Table Type support

This Pack supports the following tables types and views:
v Transparent Table (SAP Table Type: TRANSP)
v Cluster Table (SAP Table Type: CLUSTER)
Note: WebSphere DataStage models the LRAW fields in table clusters as

WebSphere DataStage VARCHAR fields. However, WebSphere DataStage cannot
interpret the data in an LRAW field in a cluster.
v Pool Table (SAP Table Type: POOL)
v View Table (SAP Table Type: VIEW), which supports these views:
– Database View (View Type: D)
– Projection View (View Type: P)
Data type support

The tables in the following sections document the support for R/3 Enterprise data
types.
Selection criteria data types

When you define selection criteria in the SQL Condition Builder, you must type
the values as specified in the following table:
R/3 Enterprise
data type Description Examples
ACCP Posting period YYYYMM Type the value in single quotation
marks. Type a year and month in the
format YYYYMM, for example, `199906’.
CHAR Character strings Type the value in single quotation
marks, for example, `xyz’.
CLNT Client Type the value in single quotation
marks, for example, `800’ (three digit).
CUKY Currency key, referenced Type the value in single quotation
by CURR fields marks, for example, ÙSD’ for US
dollars, or ÈUROS’ for Euros.
CURR Currency field, stored as Type a numeric value, for example,
DEC 500.00.

R/3 Enterprise
data type Description Examples
DATS Date field (YYYYMMDD) Type the value in single quotation
stored as char (8) marks. Type year, month, and day in the
format YYYYMMDD, for example,
`19990623’.
DEC Counter or amount field Type a numeric value without quotation
with comma and sign marks, for example, 8.0 or -8.0.
FLTP Floating-point number, Type a numeric value without quotation
accurate to 8 bytes marks, for example, 8.0 or -8.0.
INT1 1-byte integer, decimal Type a numeric value without quotation
number <= 254 marks, for example, 1 or 2 through 254.
INT2 2-byte integer, only used Type a numeric value without quotation
for length field before marks, for example, 1 or 2 through
VARC or RAW 32655.
INT4 4-byte integer, decimal Type a numeric value without quotation
number with sign marks, for example, 1 or 2 through 232-1.
LANG Language key Type a 1-digit language identifier. Type
the value in single quotation marks, for
example, È’ for English or `F’ for
French.
LCHR Long character string, Do not use this data type for specifying
requires preceding INT2 a selection condition.
field
LRAW Long byte string, requires Do not use this data type for specifying
preceding INT2 field a selection condition.
NUMC Character field with only Type a numeric value without quotation
digits marks, for example, 8.0 or -8.0.
QUAN Quantity field, points to a Type a numeric value without quotation
unit field with format marks, for example, 8.0 or -8.0.
UNIT
RAW Uninterpreted sequence of Do not use this data type for specifying
bytes a selection condition.
TIMS Time field (hhmmss), Type the value in single quotation
stored as char (6) marks. Type hour, minutes, and seconds
in the format hhmmss, for example,
`091024’.
UNIT Unit key for QUAN fields Type a value in single quotation marks,
for example, `pk’.
SAP R/3 Enterprise to WebSphere DataStage data type

conversion
WebSphere DataStage automatically converts R/3 Enterprise data types to
WebSphere DataStage data types. The following table shows how the data types
are converted:
R/3 Enterprise data type
WebSphere DataStage data type
CHAR(n)
CHAR(n)

CLNT(n)
CHAR(n)
CUKY(n)
CHAR(n)
CURR(n, m)
DECIMAL(n, m)
DATS CHAR(8)
DEC(n)
DECIMAL(n+1, 0)
FLTP(n)
FLOAT(n)
INT1(n)
CHAR(n)
INT2(n)
CHAR(n)
INT4(n)
CHAR(n)
LANG(n)
CHAR(n)
LCHR(n)
VARCHAR(n)
LRAW(n)
VARCHAR(n)
Note: WebSphere DataStage models the LRAW fields in table clusters as

WebSphere DataStage VARCHAR fields. However, WebSphere DataStage
cannot interpret the data in an LRAW field in a cluster.
NUMC(n)
CHAR(n)
QUAN(n, m)
DECIMAL(n, m)
RAW(n)
VARCHAR(n)
TIMS CHAR(6)
VARC(n)
VARCHAR(n)
UNIT(n)
CHAR(n)
Validating runtime jobs

You validate stage properties when you run a job, by using Validate Stage on the
General tab of the Output page. Validate the stage before you save the stage
design. For more information, see the Output General page section.
Validation is based on the data transfer method selected on the Data Transfer
Method tab of the Output page. (The system automatically supplies this data
transfer method information for the Runtime Validations dialog.)

The Runtime Validations window provides a consolidated view of the stage
options and uses color (along with system error and status messages) to indicate
the success or failure of the validation steps:
v Green - success
v Orange - warning
v Red - failure
FTP transfers
For FTP transfers, the following operations are considered:
v Data Transfer Method. The FTP Logon is validated using the FTP logon details
specified in the FTP logon section on the Data Transfer Method tab of the
Output page. The value specified in Application Server on the General tab of
the Output page is used as the FTP server. Path of Remote File is validated to
check if it has a value. If no value is specified in Absolute Remote Path and
Path of Remote File, an error occurs.
v SAP Connection/Logon. SAP connection and logon information is validated
based on the method selected to load and run the ABAP program. For Design
time or Manual Load and Manual Run, SAP logon information is not required
and is not checked. However, for a Runtime Load or when WebSphere
DataStage runs the ABAP program, the SAP connection and logon are validated
using the values specified on the General tab of the Output page.
v Check installed RFC. As for the CPI-C transfer type, the validation routine
checks whether the correct version of RFC (Z_RFC_DS_SERVICE) is installed
and activated on the R/3 Enterprise system.
v ABAP Options. Again, based on the method selected to load and run the ABAP
program, this section requires different validation sequences. For Design time
Load or Manual Load and Manual Run, the validation routine only checks
whether the ABAP program exists in the WebSphere DataStage repository. In the
case of Runtime Load or when WebSphere DataStage runs the ABAP program,
the validation routine also checks the syntax of the ABAP program.
CPI-C transfers
You can also perform validations for CPI-C data transfers. For these transfers, the
following operations are considered:
v SAP Connection/Logon. SAP connection and logon are validated using the
details specified on the General tab of the Output page. However, if you specify
CPI-C User Name and Password in the CPI-C logon details section, these values
are used instead. After a successful logon, the system also determines whether
the user type is CPI-C and indicates this with a message.
v Check installed RFC. This component is shipped with the Pack and must be
installed on the R/3 Enterprise system. The validation routine checks whether
the correct version of RFC (Z_RFC_DS_SERVICE) is installed and activated on
the R/3 Enterprise system.
v ABAP Options. Based on the method selected to load the ABAP program, this
section requires one of the following validation sequences:
– Design time Load and Manual Load. The validation routine only checks
whether the ABAP program exists in the DataStage Repository.
– Runtime Load. The validation routine also checks whether the ABAP
program is syntactically correct.

Local File transfers
Local File transfers require no validations.
RFC transfers
When you validate RFC data transfers, the following operations are considered:
v SAP Connection/Logon. Validates the SAP connection and logon using the
details specified on the General tab of the Output page. After a successful logon,
the system also determines whether the data transfer type is RFC and indicates
this with a message.
v Check installed RFC. This component is shipped with the Pack and must be
installed on the R/3 Enterprise system. The validation routine checks whether
the correct version of RFC (Z_RFC_DS_SERVICE) is installed and activated on
the R/3 Enterprise system.
v ABAP Options. Based on the method selected to load the ABAP program, this
section requires one of the following validation sequences:
– Design Time Load and Manual Load. The validation routine checks whether
the ABAP program exists in the DataStage Repository.
– Runtime Load. The validation routine also checks whether the ABAP
program is syntactically correct.

Chapter 3. IDoc Extract stage
The R/3 Enterprise suite of applications supports ERP (Enterprise Resource
Planning), integrating the supply-chain processes of a company. An IDoc
(intermediate document) is a report, that is, a hierarchal package of related records,
generated by R/3 Enterprise in an SAP proprietary format. An IDoc, whose
transmission is initiated by the source database, exchanges data between
applications. It provides a standard format for exchanging data with R/3
Enterprise. Individual IDocs contain the data that make up a business transaction
(for example, a sales order) or master data (for example, material master) and
include segment records and a control record.
The IBM WebSphere DataStage IDoc Extract stage, which lets WebSphere
DataStage capture IDocs from R/3 source systems to be used as source data for
WebSphere DataStage job data streams. It lets you browse R/3 Enterprise IDoc
metadata, select IDoc types to process, and extract the data from IDocs.
The IDoc Extract stage includes a set of tools and a custom GUI to passively
retrieve and process IDocs generated by R/3 Enterprise. It complements the
WebSphere DataStage ABAP Extract , which is described in part 1 of this guide.
You can use only standard SAP interfaces to access IDoc metadata and content. No
ABAP code is uploaded to R/3 source systems for IDocs.
NLS (National Language Support) is supported for the IDoc Extract stage.
Functionality
The IDoc Extract stage has the following functionality:
v Retrieval of IDocs generated by R/3 Enterprise as source data for IBM
WebSphere DataStage job data streams.
v Simultaneous connections to multiple R/3 Enterprise instances from WebSphere
DataStage.
v Ability to define a unique directory for each IDoc type, R/3 source combination.
v IDoc metadata browser capability.
v Automatic and manual job processing modes.
v Automatic re-connections to R/3 Enterprise.
v A separate client utility, the WebSphere DataStage Administrator for SAP. It
manages and configures R/3 connection and IDoc type properties.
v A persistent staging area (PSA) on the WebSphere DataStage server for storage
of IDocs retrieved from R/3 source systems.
v Performance features for a high volume of data.
v A mechanism for coordinating the processing of IDocs from the PSA.
Working with WebSphere DataStage to process SAP IDocs

The following steps describe how to process SAP IDocs using WebSphere
DataStage:
1. Use WebSphere DataStage Administrator for SAP to define a connection to the
SAP database.

2. Use the SAP GUI to define a tRFC port and a logical system to represent
WebSphere DataStage on the SAP system.
3. Create WebSphere DataStage jobs to process IDocs of particular IDoc types.
4. Configure the SAP system to send IDocs of these types to WebSphere
DataStage.
The source data is delivered to WebSphere DataStage rather than being actively
requested from the source as in the traditional data access stages. Consequently, a
background listener receives the IDocs as an R/3 source system delivers them to
WebSphere DataStage. The listener stores each received IDoc in a persistent staging
area (PSA), which conceptually serves as the data source for the IDoc Extract stage.
This stage is a source stage, which means that it has no input links. It reads the
IDoc from the PSA, parses its content, and forwards this content as relational rows
down the output links of the stage for processing by downstream stages.
The custom GUI, which interfaces directly with an R/3 system, helps you design
jobs that use the IDoc Extract stage. You can select from a list of available IDoc
types and choose the IDoc segments you want. IDoc segment metadata is
automatically retrieved and used to populate a column grid for each output link
associated with a segment.
The following diagram illustrates the architecture of the system. The runtime
components include the listener subsystem and the IDoc Extract stage. These two
components provide the IDoc retrieval and processing functions for the system, as
described in the following sections.

IDoc custom GUI
DataStage
IDoc metadata repository
IDoc
R/3 source R/3 source R/3 source plug-in
system 1 system 2 system n
listener listener listener

server 1 server 2 server n Persistent
staging area
IDocs
Listener Output links to

manager other stages
Configuration requirements
WebSphere DataStage Systems. You need to install the following components: two
on the WebSphere DataStage server system and two on the client.
v WebSphere DataStage server system:
– Listener subsystem, which installs and registers the listener manager as a
service executable. See Listener subsystem for details.
– IDoc Extract stage.
v WebSphere DataStage client system:
– IDoc Extract client GUI.
– WebSphere DataStage Administrator for SAP utility.
As with the ABAP Extract stage, the installation is facilitated by a prior installation
of the SAP GUI.
R/3 Source Systems. Some configuration on all R/3 source systems is required to
identify WebSphere DataStage as a target system. Otherwise, the listeners cannot
connect to RFC ports in order to receive IDocs.
For our system to acknowledge receipt of IDocs from R/3, it must send a message
of type STATUS back to the sending R/3 system. The logical system corresponding
to our RFC server can participate in various distribution model views. In these
Chapter 3. IDoc Extract stage 33

cases, these distribution model views must be configured to let us send messages
of type STATUS that use an IDoc type of SYSTAT01 back to the sending R/3
system. Additionally, the partner profile should be configured to collect status
messages to avoid potential locking issues.
Install the IDoc Extract stage from the CD-ROM.
Note: You must use a user name and non-blank password for WebSphere
DataStage logon credentials to use the IDoc stages. This means you must clear
Omit on the Attach to Project dialog in WebSphere DataStage, otherwise,
unexpected results occur.
Runtime components
The IDoc Extract includes the listener subsystem and the IBM WebSphere
DataStage stage components. The following sections describe the listener
subsystem.
For information about the stage and the administrative functions, see IDoc extract
for SAP R/3 Enterprise stage and Administrator for SAP Utility.
Listener subsystem
The listener subsystem includes the following components:
v Listener manager
v One or more RFC listener servers
This architecture is similar to that of the IBM WebSphere DataStage Pack for SAP
BW. It lets multiple R/3 sources deliver IDocs to WebSphere DataStage. It runs as a
daemon on UNIX, as a service on Microsoft Windows NT®.
Listener manager
The listener manager detects changes in RFC servers by doing the following:
1. At startup, the manager reads a configuration file that contains parameters
describing R/3 connections, which are defined by the IBM WebSphere
DataStage job designer.
2. For each configured R/3 connection, the listener manager starts an RFC listener
server in a separate background process. The manager ensures that all its
associated listener servers remain active and connected to R/3.
3. If it finds any irregularities, it tries to restart the server in question.
4. Additionally, if connection parameters change while a server is connected, the
manager stops the server and uses the updated connection parameters to
restart it.
5. The listener subsystem sends any error messages or messages reported by R/3
to a log file, one for each listener. The log file contains error messages
encountered by the listener subsystem as well any messages reported by R/3.
You can view the contents of the log file manually, or use the WebSphere
DataStage Administrator for SAP utility.
Listener server
Listener servers run in the background, listening on R/3 transactional RFC (tRFC)
ports, waiting for IDocs to be delivered by R/3. An R/3 administrator must

configure tRFC ports, identifying IBM WebSphere DataStage as a target system.
When a server receives IDocs, it saves them as flat files to a PSA on disk and sends
receipt confirmation to R/3.
The stage runtime component parses the IDoc content and forwards the content as
relational rows down the output links for the stage for processing by downstream
stages.
Starting and stopping RFC Listener Manager on Microsoft

Windows NT
The WebSphere DataStage RFC Listener Manager is a Microsoft Windows NT
service that starts up automatically by default when the operating system is
started.
If you need to stop or start the RFC Listener Manager on Windows NT, we
recommend that you use the Windows NT Service Manager dialog as follows:
1. Choose Start > Settings > Control Panel to display the Windows NT Control
Panel.
2. Double-click Services to display the Services dialog.
3. Select the WebSphere DataStage (IDoc Manager) service.
4. Click the start/stop button to start or stop the RFC Listener Manager.
5. Click Close to exit the Services dialog.
If you shut down the RFC Listener Manager, the RFC Listener Manager shuts
down all individual RFC Listener Servers.
For details about using the Windows NT Services dialog, see the Windows NT
documentation and online help.
Starting and stopping the RFC Listener Manager on UNIX

The WebSphere DataStage RFC Listener Manager is a UNIX daemon that starts up
automatically by default when the operating system is started.
If you need to stop or start the RFC Listener Manager on UNIX, we recommend
that you run the dsidocd.rc script for your platform with the stop or start command
options. The location of the script for all UNIX platforms is given below:
Platform
Script
HP-UX 11i (Itanium® 64 bit) v2
$DSSAPHOME/DSSAPbin/dsidocd.rc
Linux $DSSAPHOME/DSSAPbin/dsidocd.rc
Example:
To start the WebSphere DataStage RFC Listener Manager:

$DSSAPHOME/DSSAPbin/dsidocd.rc start
To stop the WebSphere DataStage RFC Listener Manager:

$DSSAPHOME/DSSAPbin/dsidocd.rc stop

If you shut down the RFC Listener Manager, the RFC Listener Manager shuts
down all individual RFC Servers.
The following sections describe the execution modes available on a per-IDoc basis.
Job run mode

You can use automatic or manual modes to run a job on a per-IDoc basis.
v Automatic. The server starts IBM WebSphere DataStage jobs when a threshold of
IDocs of a specific type arrive (see the following section, Batch Processing
Mode).
v Manual. WebSphere DataStage receives IDocs and sends them to a PSA without
processing. The WebSphere DataStage operator schedules and runs the
WebSphere DataStage jobs that process the specified IDoc.
WebSphere DataStage job execution mode is a property of the WebSphere

DataStage configuration for an IDoc type. This implies that all jobs associated with
a particular IDoc type and PSA run in automatic or manual mode.
Batch processing mode

The server can launch an IBM WebSphere DataStage job after a listener receives a
specified number of IDocs of a specified type. For example, you can configure the
listener subsystem to launch WebSphere DataStage jobs that process IDoc type
MATMAS01 after 100 instances of this IDoc are delivered.
An instance of an IDoc type is sometimes called a message. Even if IDoc messages

of a particular type are received from multiple tRFC ports by multiple listener
servers, only the server that received the IDoc completing the batch launches a job.
It resets the batch count so subsequent arrivals of the IDoc do not trigger a job.
As in job execution mode, batch processing is a property of an IDoc type, not a

WebSphere DataStage job. The batch count is used only to determine when a
WebSphere DataStage job is launched. New IDocs can arrive after the job starts.
Therefore, the job can process more IDocs than the batch count for its type
specifies. The default batch count is one. You should adjust this accordingly based
on the expected volume of an IDoc and the desired frequency of WebSphere
DataStage job runs.
Note: A WebSphere DataStage limitation prevents multiple instances of the same

job from running simultaneously. Therefore, the listener has to queue job requests
or combine batches if the threshold for launching a job is reached before the
current job instance is complete. You may need to adjust the batch count to an
appropriate value to prevent this situation from occurring.
Setting batch size for manual IDoc extraction

You can configure the batch size for IDoc extraction jobs that are to be run in
manual mode.
To set the batch size:

1. Create an IDoc extraction job.
2. Set manual job processing in the Stage properties IDoc Type tab by clearing the
field Run jobs that extract IDocs of this type after receiving n IDocs.
3. Right-click the IDoc Extract Stage in the Designer Client canvas and click Grid
Style.
4. In the Stage tab, click the Properties tab.

5. In the MANUAL_BATCH_SIZE field, specify the number of IDocs to be
processed when the job is run manually.
6. Click OK.
Use WebSphere DataStage Director or WebSphere DataStage Designer to run the

IDoc extraction job manually.
Related tasks
“Extracting IDocs by IDoc number”
You can configure the IDoc numbers that an IDoc extraction job processes.
Extracting IDocs by IDoc number

The IDOC_EXTRACT_LIST property specifies the IDoc numbers that an IDoc

extraction job processes. When you do not configure this property, the IDoc
extraction job processes all IDocs in the persistent staging area (PSA) that are not
processed already.
To configure the IDoc numbers:

1. Create or open an IDoc extraction job.
2. Right-click the IDoc Extract Stage in the Designer Client canvas and click Grid
Style.
3. In the Stage tab, click the Properties tab.
4. In the IDOC_EXTRACT_LIST field, specify the IDoc numbers to be processed.
Specify the IDoc numbers as follows:
v A single IDoc number
v A comma-separated list of IDoc numbers, such as 11000,11003,11025
v A range of IDoc numbers, such as 11000-11200
5. Click OK.
6. Run the job.
Related tasks
“Setting batch size for manual IDoc extraction” on page 36
You can configure the batch size for IDoc extraction jobs that are to be run in
manual mode.
“Selecting IDoc types” on page 47
Persistent staging area

Because IDocs are not immediately processed by IBM WebSphere DataStage jobs
when they are received from R/3, they must be stored locally with respect to the
WebSphere DataStage server. When you start a job manually using the WebSphere
DataStage Director Client or start one from a listener server, the IDocs are read as
text files from local storage. Local storage refers to the file system of the computer,
which conceptually serves as the data source for the IDoc Extract stage. This local
storage on the file system is referred to as the Persistent Staging Area (PSA). The
PSA comprises a collection of individual directories that you configure when
designing WebSphere DataStage jobs or configuring IDoc types with the
WebSphere DataStage Administrator for SAP utility.
The PSA is a configurable property of an IDoc type and the R/3 connection from
which it arrives.

v Type. You can define a separate directory for each IDoc type that the WebSphere
DataStage external system can receive. That is, you can specify a directory of
your choice for each IDoc type that is represented by some job.
v Connection. Since WebSphere DataStage jobs must also specify an R/3
connection, IDocs can be segregated by connection as well as type. For example,
a single WebSphere DataStage instance can run production level jobs against an
R/3 production environment while simultaneously serving as a development or
QA environment for the respective R/3 instances. In another scenario, you can
configure WebSphere DataStage to listen to more than one production R/3
instance for the same or different IDoc types.
Example. The following diagram represents four RFC listener servers associated
with a single WebSphere DataStage server instance. Each RFC listener server listens
on one of four R/3 instances: development, QA, and two production instances.
Assume six WebSphere DataStage jobs are on this server.
R/3 Dev R/3 QA R/3 Prod P1 R/3 Prod P2
R/3 instances
Listener servers
A B A A B C PSA/IDoc type
DataStage jobs
J1 J2 J3 J4 J5 J6
The relationship between WebSphere DataStage jobs and the IDoc types they
process is as follows:
v J1 processes IDoc type A from the R/3 development instance.
v J2 processes IDoc type B from the R/3 development and QA instance.
v J3 processes IDoc type A from the R/3 QA instance.
v J4 processes IDoc type A from the R/3 production instance P1.
v J5 processes IDoc type B from the R/3 production instance P1.
v J6 processes IDoc type C from the R/3 production instance P2.
Security. In addition to offering flexibility in determining how IDocs from various

R/3 instances are processed, this method lets you have full control over the file
permissions for the directories where the IDocs are stored. Thus, varying levels of
security can be applied based on IDoc type. However, the owner of the listener
server process must have write permissions on this directory, and the owner of the
process that runs the WebSphere DataStage job must have read and write
permissions.

PSA maintenance
Because of potential limitations in available disk space, the file system may not
have the capacity to store every IDoc that IBM WebSphere DataStage receives. To
minimize the possibility of file systems becoming full due to an excess of IDocs in
the PSA, the IDoc Extract includes the WebSphere DataStage Administrator for
SAP utility for cleaning up or archiving IDocs that have been processed by
WebSphere DataStage jobs and are no longer needed. During installation, a
separate executable is scheduled to be run periodically (weekly every Saturday by
default) by the operating system. Use the utility to modify the default scheduling
of the executable or to run the cleanup/archive manually. A bookmark file
determines when processed IDocs are deleted from persistent storage.
You can achieve an additional level of control by using manual removal or

archival. Even though this task is performed automatically, you can request a
manual removal of processed IDocs of a particular type at any time (see IDoc
Cleanup and Archiving page for more information).
When the process runs, it scans the PSA, identifying all IDocs that have been
successfully processed by all jobs having an interest in the particular IDoc. When
these IDocs are identified, they are deleted from the file system or archived to
another location. If they are archived, it is the responsibility of the user to maintain
the archive.
Inactive jobs
Inactive jobs are those that have not recently run. If a job stops being run, IDocs
that are to be processed by the job accumulate. They are not cleaned up because
the cleanup process detects that the inactive job has not yet processed the IDocs
and may never process them. If the job never runs again, the IDocs will
accumulate indefinitely.
To resolve this issue, a job attains an inactive status with respect to the listener
subsystem when it has not been run for 40 days. Forty days is the default value,
which you or the WebSphere DataStage Administrator Client can modify using the
WebSphere DataStage Administrator for SAP. When a job becomes inactive, its
unprocessed IDocs are not saved or archived by the cleanup process. For more
information, see Administrator for SAP Utility.
Configuration files
Because R/3 connection parameters and IDoc type properties are outside the scope
of individual IBM WebSphere DataStage jobs, they are not stored in job definition
files in the WebSphere DataStage repository. External configuration files store these
parameters and properties. We recommend that you use the IDoc Extract GUI to
modify settings for these configuration files. This section describes the various
configuration files that are used by the server to manage R/3 connection
parameters and IDoc type properties:
v DSSAPConnections.config
v IDocTypes.config
v <IDocType>.config
Note: On UNIX platforms, the dsenv file in the WebSphere DataStage server
directory must contain a umask setting so that users have read and write
permissions on these configuration files. (Windows NT platforms handle this
automatically.)

DSSAPConnections.config file
The DSSAPConnections.config file stores SAP R/3 connection parameters and is
located in the DSSAPConnections subdirectory of the IBM WebSphere DataStage
server directory. It contains all the information specific to the individual physical
SAP R/3 connections that have been configured. SAP R/3 connections are typically
configured during job design (see Defining the WebSphere DataStage connection to
SAP). However, you can also configure them using the WebSphere DataStage
Administrator for SAP. These configuration files should never be directly edited
(see Administrator for SAP Utility).
The example below shows an SAP R/3 connections configuration file:

DSSAPCONNECTIONS=<BEGIN>
<BEGIN>
DSPASSWORD=py~sZiv.
SAPROUTERSTRING=
DEFAULTLANGUAGE=EN
DATAFILESPATH=
SAPSYSNUM=02
DEFAULTUSERNAME=p45user
DESCRIPTION=SALES logical system on ultra - P45
instance
SAPGROUP=
DSUSERNAME=dsuser1
SAPMESSERVER=
DEFAULTCLIENT=800
ALLOWSAPTORUNJOBS=TRUE
SAPAPPSERVER=R3sys1
LISTENFORIDocs=TRUE
NAME=P45VERSION3
SAPSYSID=
REFSERVERPROGID=idoctest3
USELOADBALANCING=FALSE
DEFAULTPASSWORD=JSKM~
<END>
<END>
Fields of the DSSAPConnections.config file
Each connection is delimited by <BEGIN>/<END> pairs. The fields within a

connection block in the DSSAPConnections.config file are defined as follows:
v NAME. A tag identifying the physical connection.
v DSPASSWORD. An encrypted password used by a listener server to launch
WebSphere DataStage jobs.
v DEFAULTLANGUAGE. One of the SAP mnemonics representing the native
language of an RFC client connection. RFC client connections are used by the
IDoc custom GUI and by the listener when sending status information back to
R/3.
v DEFAULTUSERNAME. The user name for connecting to R/3 as an RFC client.
v DSUSERNAME. The user name used by a listener server to launch WebSphere
DataStage jobs.
v DEFAULTCLIENT. The R/3 client number used to connect to R/3 as an R/3
client.
v SAPROUTERSTRING. The router string used to connect to R/3 as an RFC
client.
v SAPSYSNUM. The R/3 system number used to connect to R/3 as an RFC
client.

v REFSERVERPROGID. The program ID of a tRFC port used to connect to R/3
as an RFC server. The listener servers use this when connecting to R/3 to listen
for IDocs.
v DEFAULTPASSWORD. An encrypted password used to connect to R/3 as an
RFC client.
v SAPAPPSERVER. The host name of the system running the R/3 instance.
v USELOADBALANCING. The load balancing specification. If true, client
connections are made through a message server rather than an application
server.
v SAPMESSERVER. The message server host that handles the connection request
when using load balancing.
v SAPSYSID. The R/3 system name that handles the connection request when
using load balancing.
v SAPGROUP. The group name of the application servers when using load
balancing.
IDocTypes.config file
When the listener manager starts, it starts a listener server for each connection
described in this file. The previous example contains a single connection,
PA45VERSION3.
When an IDoc type is first associated with a connection during job design, the
client GUI creates a directory in the IBM WebSphere DataStage server directory
called DSSAPConnections/<ConnectionName>/IDocTypes/<IDocTypeName>.
v ConnectionName specifies one of the connections defined by the NAME=
parameter in the DSSAPConnections.config file.
v IDocTypeName is a directory that specifies the name of an IDoc type.
For each configured connection, a corresponding subdirectory

(DSSAPConnections/<ConnectionName>) is created.
Each of these subdirectories contains an IDocTypes directory that contains a

configuration file named IDocTypes.config and further subdirectories, one for each
IDoc type configured. The type is received from the respective connection.
The IDocTypes.config file specifies the directory locations for writing each type of
IDoc that has been configured for that connection. It has the same format as the
DSSAPConnections.config file, but it contains different fields. For example:
DSIDOCTYPES=<BEGIN>
<BEGIN>
USE_DEFAULT_PATH=FALSE
IDOC_FILES_PATH=/u1/IDocs/MATMAS01
NAME=MATMAS01
<END>
<BEGIN>
USE_DEFAULT_PATH=TRUE
IDOC_FILES_PATH=
NAME=CREMAS01
<END>
<END>
The fields within an IDoc type block are defined as follows:

v NAME. The name of the IDoc type.

v USE_DEFAULT_PATH. A value of TRUE indicates that WebSphere DataStage
defines the default directory to serve as the PSA for this IDoc type. If TRUE,
IDocs received by the listener are stored in the corresponding IDocTypeName
directory in the IDocTypes directory.
v IDOC_FILES_PATH. If USE_DEFAULT_PATH is FALSE, specifies the
user-defined directory that serves as the PSA for this type.
<IDocType>.config file
The <IDocType>.config exists in every IDocTypeName directory that is configured as
a destination for a particular IDoc type, either by default or that you explicitly
specify.
IDocType is the type name of the IDoc. This configuration file contains all the
information that is specific to that IDoc directory location. For example:
IDOC_COUNT=100
DSUSERNAME=johnharvey
DSPASSWORD=py~sZiv.
AUTORUN_ENABLED=TRUE
ARCHIVE_IDOC_FILES=FALSE
USE_DEFAULT_LOGON=FALSE
The fields in the IDocType.config file are defined as follows:

v IDOC_COUNT. The value indicating the number of IDocs to be collected in the
PSA. They are collected before a job is launched by a listener server to process
the IDoc.
v DSUSERNAME. The user name used by the listener server to connect to IBM
WebSphere DataStage to launch a job. If USE_DEFAULT_LOGON is TRUE,
DSUSERNAME is ignored.
v DSPASSWORD. The encrypted WebSphere DataStage password used with
DSUSERNAME. If USE_DEFAULT_LOGON is TRUE, DSPASSWORD is ignored.
v AUTORUN_ENABLED. If set to TRUE, WebSphere DataStage jobs associated
with the IDoc type are launched automatically by the listener server when
IDOC_COUNT is reached.
If AUTORUN_ENABLED is set to FALSE, WebSphere DataStage jobs are not
started by the listener server, and IDOC_COUNT is ignored.
v ARCHIVE_IDOC_FILES. If set to TRUE, the cleanup process moves the IDocs
to a default location rather than deleting them from the PSA.
v USE_DEFAULT_LOGON. If this field is set to TRUE, the listener server uses the
DSUSERNAME/DSPASSWORD combination in the DSSAPConnections.config file
that is configured for the connection. If it is set to FALSE, it uses the
DSUSERNAME and DSPASSWORD combination in this <IDocType>.config file.
For more information about the management of these configuration files, see the
Administrator for SAP Utility section.
IDoc extract for SAP R/3 Enterprise stage

IDocs that are received by a listener server from R/3 are saved to disk. IBM
WebSphere DataStage jobs do not directly read IDocs from an SAP R/3 system.
Instead, they are read from the directory specified in the configuration file for the
particular IDoc type (see Configuration files for more information). You can use the
IDoc Extract stage to start jobs automatically using a request from a listener server,
or you can use the WebSphere DataStage Director Client or dsjob to start them.
The IDoc Extract stage is a passive stage that has output links but no input links.

Each output link from the IDoc Extract stage is mapped to a single IDoc data
segment. You should create links for each IDoc segment whose data is of interest
when you design a job.
If you process only segments E1MARAM and E1MAKTM, for example, the data
contained in other segments is ignored. Segments are records within an IDoc that
can be child or parent segments. They include administrative fields such as the
IDoc number, the segment number, and the number of the parent segment. Child
segments have a many-to-one relationship to parent segments.
The selection of E1MARAM, which has child segments, does not imply that all its
child segments are included. Only the segment fields of E1MARAM are processed.
The segment fields for E1MARAM correlate with the column definitions for the
output link on which E1MARAM is associated. The segment fields for E1MAKTM
become the column definitions for a second output link. Use the custom GUI to
define the column metadata definitions. When a segment is chosen for a link, the
GUI displays the available segments fields that you can select for output. The GUI
then automatically translates the R/3 metadata for the selected segment fields to
equivalent WebSphere DataStage metadata definitions and populates the column
grid.
Examples
Note: WebSphere DataStage uses the segment definition name, for example,
E2MAKTM001, rather than the segment type name. The distinction between the
two is beyond the scope of this document. For more information, consult your SAP
documentation.
The R/3 metadata is translated to WebSphere DataStage metadata.
When the job is run, the stage reads unprocessed IDocs of the specified type from
the appropriate location in the PSA. Unprocessed IDocs are identified by a
bookmark file that is maintained for each IDoc type. (See the next section and PSA
maintenance for details about bookmarks.)
When subsequent jobs are run, only those IDocs that arrived following the current
bookmark are processed. This prevents a job from repeatedly processing the same
set of IDocs.
If a job fails, you can use the reset functionality for the job to restore the bookmark
to its original state.
Test Mode. To prevent the bookmark for the job from being updated, you can
process IDocs in Test Mode (set on the General tab of the Stage page). For more
information, see About the Stage General page.
Bookmarking the PSA

A bookmark file for each IDoc type identifies IDocs that have not yet been
processed by an IBM WebSphere DataStage job. A single bookmark file can
independently account for any number of jobs designed to process the particular
IDoc type.

The bookmark is updated when a job begins. If a job aborts, a job reset causes the
bookmark to be reset to its state before the aborted run. Therefore, the same batch
of IDocs can be reprocessed without data loss.
This scheme lets more than one job process IDocs of the same type from the same
location in the PSA. Since the bookmark coordinates the access for each job to
IDocs of the same type in a given PSA, it is not possible for one job to exhaust the
IDocs before a second job has the opportunity to process them.
Note: On UNIX platforms, the WebSphere DataStage server must be started so that
users who want to run a job have write permission on the bookmark file. This also
applies to the saprfc.ini file, which the job maintains in the WebSphere DataStage
project directory.
Creating jobs for IDoc types

The following steps let you create a WebSphere DataStage job for an IDoc type:
1. Open a new job.
2. Add an IDoc Extract stage.
3. Create additional stages to process IDoc segment records.
4. Create links between the IDoc stage and the stages that process the segment
records.
5. Define the properties of the IDoc Extract stage.
6. Save and compile the job.
For information about using stages in WebSphere DataStage jobs, see WebSphere
Defining the WebSphere DataStage connection to SAP

A summary of the interaction of the stage and the links follows. The subsequent
sections give further details.
v The stage reads multiple IDocs of a particular type arriving through a particular
connection.
v The connection to an R/3 Enterprise system and the IDoc type properties are
defined as stage properties.
For information about selecting an IDoc type and configuring the IDoc
processing properties see Selecting IDoc types and Importing IDoc type
configurations respectively.
v Each link carries segment records from a particular segment type or control
records.
v The segment type or the control record is defined as a link property. Select
segments on the output links to retrieve IDoc metadata, and translate the
segment fields into WebSphere DataStage column definitions (see Defining IDoc
Extract Output links).
When you start the stage GUI from the WebSphere DataStage Designer Client to
edit an IDoc Extract stage, the General tab of the Stage page opens.
This dialog has the Stage and Output pages:

v Stage. This page displays the name of the stage you are editing. This page
contains the General and the IDoc Type tabs. The selected SAP connection

parameters are displayed on the General tab on the Stage page. Generally the
connection is already defined by default. See the following sections for details.
Use the IDoc Type tab of the Stage page to select an IDoc type (see Selecting
IDoc types).
v Output. You assign the control record or a segment to each output link here. You
can also modify the corresponding columns that are generated. (See Defining
IDoc Extract Output links).
Stage General page

The parameters that are needed by WebSphere DataStage to connect to an SAP
database are defined on the General tab on the Stage page.
To connect to an R/3 Enterprise system:

1. DataStage Connection to SAP. The WebSphere DataStage connection to the
R/3 Enterprise system that is defined on the WebSphere DataStage server
machine and shared by all WebSphere DataStage users connected to that
machine. The fields in this area are read-only and are obtained from the
connection that you selected.
Name The name of the selected connection to the SAP R/3 system that
generates the IDocs to be extracted by this stage.
Select...
Click the button to choose a WebSphere DataStage connection to the
SAP R/3 system. The selected connection provides all needed
connection and default logon details that are needed to communicate
with the corresponding SAP R/3 system.
Description
Additional information about the selected connection.
Application Server
The name of the host system running SAP R/3. If the selected
connection uses load balancing, Message Server is used instead.
System Number
The number assigned to the SAP R/3 system used to connect to SAP
R/3. If the selected connection uses load balancing, System ID is used
instead.
2. SAP Logon Details. The fields in this area are read-only unless you clear the
Use connection defaults box.
a. User Name. The user name that is used to connect to SAP.
b. Password. A password for the specified user name.
c. Client Number. The number of the client system used to connect to SAP.
d. Language. The language used to connect to SAP.
e. Use connection defaults. Clear this box to remove the default SAP Logon
Details settings so you can use different logon information. If selected, the
displayed logon details are obtained from the selected connection and are
disabled.
3. Enter an optional description to describe the purpose of the stage in the
Description field.
4. Test Mode. The check box that specifies whether the job runtime component
updates the bookmark file for this stage. If you select this check box, the job

runtime does not update the bookmark file for this stage. Therefore, each time
the job is run, this stage reads all the IDocs that it read when the job was
previously run.
Use Test Mode as you develop a job and need to test it repeatedly with the
same set of input data without causing the bookmark to be updated.
Selecting Test Mode also prevents the job from being run automatically as new
IDocs of this type arrive. The listener server does not launch a job in test mode
even if its IDoc type is designated for automatic execution.
The Administrator can archive processed IDocs rather than permanently
remove them by moving them to a subdirectory of the PSA for the IDoc.
Currently, the only way to reprocess IDocs is to reset an aborted job or set the
job to Test Mode. You cannot reprocess archived IDocs.
For a newly created stage, the connection for the stage defaults to the last one you
selected for another stage of this type. If only one WebSphere DataStage connection
to an R/3 Enterprise system is defined on the WebSphere DataStage server
machine, a new stage defaults to that connection regardless of whether you
previously created an IDoc Extract stage.
Selecting the WebSphere DataStage connection to SAP

When you click Select on the General tab of the Stage page, the Select WebSphere
DataStage Connection to SAP dialog opens. This dialog lets you choose a
WebSphere DataStage connection to the R/3 Enterprise system. The selected
connection provides all needed connection and default logon details that are
needed to communicate with the corresponding R/3 Enterprise system.
The IDoc Listener Settings options function in the same way as the corresponding
options in the IDoc Listener Settings of the WebSphere DataStage Administrator for
SAP utility. In addition, the Number of servers for this connection field selects the
number of listening processes for the IDoc connection in the WebSphere DataStage
IDOC Manager. This property improves the performance during IDoc extraction.
Use SAP R/3 transaction SMSQS to configure multiple IDoc connections.
Add, Properties, Import into, Export and Remove are administrative connection
operations. They let you manage the list of connections that is maintained on the
WebSphere DataStage server machine from the Select DataStage Connection to SAP
dialog. They function in the same way as the corresponding buttons on the
DataStage Connections to SAP page of the WebSphere DataStage Administrator for
SAP utility.
Stage Options page

Use the IDoc Extract Stage Options page to specify design-time connection details.
Name The name of the selected connection to the SAP R/3 system that generates
the IDocs to be extracted by this stage.
Select...
Click the button to choose a WebSphere DataStage connection to the SAP
R/3 system. The selected connection provides all needed connection and
default logon details that are needed to communicate with the
corresponding SAP R/3 system.
Description
Additional information about the selected connection.

Application Server
The name of the host system running SAP R/3. If the selected connection
uses load balancing, Message Server is used instead.
System Number
The number assigned to the SAP R/3 system used to connect to SAP R/3.
If the selected connection uses load balancing, System ID is used instead.
Use this connection for design time
Specifies a separate connection to SAP R/3 that is to be used only at
design time for extracting metadata from the SAP R/3 system. This
connection is not used at runtime.
Separate connections for design time and runtime are required to support
SAP Exchange Infrastructure (SAP XI). The DataStage connection to SAP
defined on the Stage General page is used to connect to SAP XI at job
runtime.
IDoc Data Directory Path
The path that is used to generate and store IDoc data or to read IDoc data
files when the SAP R/3 system is not available. This path is set in the IDoc
Type properties in the field Directory containing temporary IDoc files for
this type and connection.
Enable offline IDoc processing
Select to specify that IDocs are to be read from and stored to the path in
IDoc Data Directory Path when the SAP R/3 system is not available at job
runtime.
Selecting IDoc types

After you provide a WebSphere DataStage connection to SAP on the General tab of
the Stage page, you select an IDoc type on the IDoc Type tab of the Stage page.
Configure an IDoc type as follows. The subsequent sections describe these steps in
detail.
1. Click Select to open the Select IDoc Type dialog for selecting an IDoc type.
2. Select a type, and click OK.
3. A prompt can appear about an unconfigured IDoc type. Click Yes to configure
the IDoc type for use with WebSphere DataStage.
4. Define the IDoc type configuration properties on the IDoc Type Properties
dialog.
5. The selected IDoc type and its component segment types are now visible on the
IDoc Type tab of the Stage page. The Select IDoc Type dialog displays all the
released IDoc types defined on the R/3 Enterprise system. It has the following
components:
v Connection. The connection name for the R/3 system whose IDoc types are
being shown is indicated at the top of the dialog.
v Description. A description of the R/3 system.
v IDoc Types. A list of all the released IDoc types defined on the R/3 system.
v Find. Click to open a Find IDoc Type dialog. It lets you search for IDoc
Types that contain user-specified substrings in their name or description.
v Properties. Click to view and change the WebSphere DataStage configuration
for the selected IDoc type.

v OK. When you select an IDoc type and click OK, the system checks whether
anyone on the WebSphere DataStage system configured the IDoc type for the
current connection. If not, you see the following message:
This IDoc type has not been configured for use with WebSphere
DataStage. Would you like to set these options now?
Click Yes to set the options to default values. The IDoc Type Properties
dialog opens (see the following section).
Related tasks
“Extracting IDocs by IDoc number” on page 37
Defining IDoc type properties

When you confirm the default values after selecting an IDoc type, the IDoc Type
Properties dialog opens.
The IDoc Type Properties dialog contains various parameters set to default values.
The default settings are usually appropriate.
1. Name is the name of the selected IDoc type (read-only) with its description in
the Description field.
2. Directory containing temporary IDoc files for this IDoc type and connection
is the path name for storing IDoc files for this IDoc type and connection.
Initially, the directory is set to a default location, and the edit box is read-only.
To enter your own directory, clear the Use default directory box.
If Use default directory is selected, the default path name is displayed, and the
control is read-only.
3. Click Browse to browse for an alternate directory to store IDoc files for this
IDoc type and connection. If Use default directory is cleared, this button is
enabled.
4. Click Use default directory to browse for an alternate directory for storing
IDoc files for this IDoc type and connection. If cleared, Directory containing
temporary IDoc files for this IDoc type and connection and Browse are
enabled, and you can specify an alternate directory. Otherwise, WebSphere
DataStage uses a default location.
If the directory does not exist, and you save changes to the IDoc type
configuration, the system tries to create the directory. If it cannot, you must
choose another directory.
If the directory has already been configured for this IDoc type using a different
WebSphere DataStage connection to SAP, you are warned that stages that read
IDocs of this type process IDocs arriving from all connections that share this
directory for the type.
After you acknowledge this message, the values displayed in the dialog for the
other configuration details are refreshed to match those already defined for the
type through the other connections. Any changes you make to the
configuration details for an IDoc type whose PSA is shared between two or
more connections affect all connections processing this type from this PSA.
5. Click Archive processed IDoc files to archive processed IDocs rather than
permanently remove them. The archive location is at the same level as the PSA
for the IDoc.
It is the responsibility of the Administrator to oversee the permanent storage of
archived IDocs, for example, moving them to an alternate disk location or
removable storage media. You cannot restore processed IDocs.

6. If Run jobs that extract IDocs of this type after receiving n IDocs is selected,
jobs are automatically run that read IDocs of this type each time another n
IDocs of this type are received by the IDoc listener server. (The default is one.)
If the number of IDocs of this type is expected to be small and to arrive
frequently, increase the number of IDocs that must arrive before jobs are
automatically run.
Alternately, you can disable automatic jobs invocation for this IDoc type by
clearing the check box. In this case, use the WebSphere DataStage Director
Client to schedule jobs.
Note: See “Setting batch size for manual IDoc extraction” on page 36 for
additional information.
7. The DataStage Logon Details for Running the Jobs area specifies the
WebSphere DataStage logon user name and password, and whether to use the
defaults for the connection.
8. Specify a specific job to run automatically in the Assign job section:
v Select Run specific job to specify the project name and job name that is to
run automatically when the stage receives the number of IDocs in the Run
jobs that extract IDocs of this type after receiving field. In the Project
Name field, choose the name of the project that is to run automatically. The
Project Name field includes up to two project names:
– The most recent, saved project associated with this IDoc type
– The current project
v Clear Run specific job to automatically run all jobs that extract this IDoc
type when the stage receives the specified number of IDocs.
9. After you select an IDoc type and optionally define IDoc Type configuration
properties, the Select IDoc Type dialog closes, and you return to the IDoc Type
tab of the Stage page, with the properties of the selected IDoc type now visible.
The IDoc Components area shows the control record and all the segments
defined for the IDoc type with their descriptions. (The control record, one for
each IDoc, is an administrative record that contains a standard set of fields
describing the IDoc as a whole.) This area contains the following information:
v Name. Shows the hierarchical relationship among the segments using a tree
structure with their descriptions.
(A segment type can appear only once within an IDoc type. The names for
the segments are the segment definition names, not the segment type names.
You can infer the segment type name from the segment definition name.)
v Assigned Output Link. After particular segments in the IDoc Type are
assigned to the output links of the stage, the IDoc Components control
shows the names of the links in the Assigned Output Link column of the
control. This gives you an overview of which segments are being extracted
by the stage.
The General tab of the Output page shows the segment type or control record for
each output link as described in subsequent sections.
Specifying SAP R/3 versions

When an R/3 system is upgraded to a later version, you can add new fields to the
segment definitions used by the IDoc types that are currently defined on the
system, thus changing segment metadata for the IDoc types.

To prevent third party applications that receive IDocs from the R/3 system from
having to change to accommodate the new segment definitions, you can specify
that IDocs of a particular IDoc type that are sent through a particular port use the
segment definitions from an earlier version of the R/3 system.
If this is already done in your SAP application, set R/3 Version on the IDoc Type
Properties dialog to the same value that is set for the IDoc type in the R/3 system
itself (See Connection and Logon Details page).
R/3 Version lists the version that is the same or earlier than that of the R/3 system
that is specified in the connection. It defaults to the current version of the R/3
system.
Version 46E and Later. By default, the IDoc Extract product only knows about R/3
46D or earlier. If you have an R/3 system that is later than 46D, the program reads
the version from the R/3 system and automatically adds it to the list.
Missing Versions. But suppose the R/3 system is Version 46F. The combo box then
includes the 46F, but may be missing 46E. If you want to specify 46E, type the
value directly into the edit control of the combo box.
When you click OK, the system verifies that the specified R/3 Version is unknown,
and prompts:
R/3 version "46E" is not among the versions currently
known to DataStage. Would you like "46E" to be added to
the list of known versions?
If you click Cancel, the IDoc Type Properties dialog stays open.
If you enter an incorrectly formatted version and click OK, the following error
appears:
″xx″ is not a valid R/3 version. A correctly formatted version must consist of two digits
followed by an upper case letter.
If you enter a version number that is later than that of the R/3 system specified in
the connection, you see the following error:
R/3 Version ″xxx″ is greater than ″xxx″, the version of the connection. The R/3 version for
the IDoc type must not be greater than the version of the connection.
If you change the port version of the connection for the stage or the R/3 version of
the IDoc type for the stage, the stage automatically refreshes itself using
appropriate IDoc metadata. If the either of these versions is changed before you
open the stage editor, the act of opening the stage editor produces the following
warning:
Either the port version has changed for connection ″xxx″, or the R/3 version in the
configuration for IDoc type ″xxx″ has changed. The IDoc metadata for the stage will now
be fixed.
Defining IDoc extract output links

After you select an IDoc type, you assign the control record or a segment to each
output link. Begin the selection process from the General tab on the Output page.

After you choose the output link from the Output name box, the following steps
summarize how output links function.
The subsequent sections describe these steps in detail.

1. Click Select to open the Select IDoc Component to Extract dialog. The IDoc
Component to Extract, Description, and Fields in Component information is
displayed after you select a control record or a segment for the link.
2. The selected segment type and its fields now appear on the General tab of the
Output page.
The administrative fields common to all segment types appear at the beginning
of the list.
3. Columns are automatically generated for each link when the segment type is
selected.
4. You can delete unnecessary columns (administrative fields that are rarely used
are not visible).
5. Columns for missing fields can be added by clicking Add Columns on the
Columns tab of the Output page.
General tab for the Output page

This tab is displayed by default on the IDoc Extract stage. From this tab, the
following output link functions are available:
v Select the IDoc component to extract
v Add columns
v Save IDoc metadata definitions
Selecting the IDoc component to extract

Click Select on the Output General page to open the Select IDoc Component to
Extract dialog, which contains an IDoc Components control identical to the one on
the IDoc Type tab of the Stage page.
1. Select the control record or a segment, and click OK.
2. The dialog closes, and you return to the General tab of the Output page with
the IDoc Component to Extract, Description, and Fields in Component
information.
If you selected a segment, the first seven fields in the Fields in Component list
are administrative fields common to all segments. These fields are followed by
the data fields specific to this segment type.
Adding columns
When you choose a segment or the control record on the General tab on the
Output page, a corresponding list of columns is automatically generated.
You can view and modify these columns using the Columns tab on the Output
page.
To view and modify the columns:

1. Columns corresponding to segment administration fields are given names
prefixed by ADM_. The default column list does not include columns for the
less frequently used administrative fields (namely, SEGNAM, MANDT,
HLEVEL, and SDATA). You can also delete any unneeded columns.

The Corresponding Extract Field for Column ″ADM_DOCNUM″ is the
single-row grid near the bottom of the tab showing the extract field that
corresponds to the selected column, in this case, ADM_DOCNUM (the value
changes depending on the selected column). It includes Name, Description,
Internal Type, and Length information.
2. Click Add Columns to add columns for any fields that are not currently
represented in the columns list. The Add Columns dialog opens. If you
double-click a field or select one or more fields, and click Add, the fields
disappear from the Add Columns dialog. Corresponding columns appear in the
Columns tab. The new columns are inserted into positions in the column list
that match the sequence of the corresponding fields in the segment. The
columns are shown as selected and are automatically scrolled into view. After
the columns are added, the Add Columns dialog remains open so you can
optionally add columns for remaining fields.
Click Close to exit the dialog and return to the Columns tab of the Output
page. The dialog also closes automatically if there are no columns left to be
added.
3. Click Validate Columns to check whether each column has a matching extract
field and that the properties for the column are consistent with those of the
field. If the properties of the column are inconsistent, the program offers to
correct the properties of the column.
4. Click Save to open the Save IDoc Segment Definition dialog to save metadata
definitions for a job to the WebSphere DataStage repository.
Saving IDoc metadata definitions

You can save metadata definitions for a job to the WebSphere DataStage repository.
Besides being useful to other jobs that may process the target data of the IDoc
Extract, the stored metadata lets the WebSphere DataStage perform where-used
analysis on IDoc metadata. It also lets WebSphere MetaStage® analyze the metadata
and perform various queries against it.
The Save IDoc Segment Definition dialog contains the following fields:
v Data source type. The type is always IDoc.
v IDoc type. The name of the data source.
v Segment name. The name of the table or file.
v Short description. The brief description of the IDoc segment.
v Long description. The detailed description of the IDoc segment.

Chapter 4. IDoc Load stage
The R/3 Enterprise suite of applications supports ERP (Enterprise Resource
Planning), integrating the supply-chain processes of a company. An IDoc
(intermediate document) is a report, that is, a hierarchical package of related
records, generated by R/3 Enterprise in an SAP proprietary format. An IDoc,
whose transmission is initiated by the source database, exchanges data between
applications. It provides a standard format for exchanging data with R/3
Enterprise. Individual IDocs contain the data that make up a business transaction
(for example, a sales order) or master data (for example, material master) and
include segment records.
The IBM WebSphere DataStage Load stage is a passive stage that has input links
but no output links. Use this stage to generate IDocs from source stages to load
data into R/3 Enterprise.
This section describes the Load stage:

v Functionality
v Configuration requirements
v Defining the IDoc Load stage
The user interface for the IDoc Load stage is almost identical to that of the IDoc
Extract stage.
NLS (National Language Support) is supported for the Load stage.
The IDoc Load stage has the following functionality:

v Ability to browse and select IDoc types from a metadata browser in the GUI.
v Selection of IDoc segment types to which non-SAP data is loaded. The data for
each segment is read in on a separate link in parallel processes.
v Ability to allow the WebSphere DataStage job designer to map relevant segment
data from multiple sources. Each IDoc Load stage in a WebSphere DataStage job
can send IDocs of one chosen type to R/3 Enterprise.
v A mapping mechanism that allows relational row data to be converted to SAP’s
proprietary IDoc data format. IDoc structural metadata determines the link
processing order. Join keys, which are sort keys, are identified by the job
designer. The processing order and join keys are used by the stage to assemble
IDocs.
v Support for message types, the scheme for electronically transmitted data used
for one specific business transaction.
v Support for primary key (P-key), and foreign key (F-key) handling, which differs
from that in the IDoc Extract stage. Foreign keys for segments at root level of the
IDoc hierarchy separate data into separate IDocs, allowing one stage to send
several IDocs in one transmission to SAP R/3. Foreign keys in non-root level
segments relate child segment records to their parent segment record.
v NLS (National Language Support). For information, see WebSphere DataStage
Administrator Client Guide.
Automatically running a job is unsupported for the Parallel Canvas.

Configuration requirements
WebSphere DataStage Systems. You need to install the following components: two
on the DataStage server system and two on the client.
v WebSphere DataStage server system:
– IDoc Load for SAP R/3 stage.
v WebSphere DataStage client system:
– IDocLoad for SAP R/3 client GUI.
– Administrator for SAP utility.
As with the ABAP Extract for SAP R/3, the installation is facilitated by a prior
installation of the SAP GUI.
R/3 Source Systems. Some configuration on all R/3 source systems is required to
identify WebSphere DataStage as a target system.
After installing the stage GUI, start the stage editor from the WebSphere DataStage
Designer Client by doing one of the following:
v Double-click the stage in the Diagram window
v Select the stage and choose Properties from the shortcut menu.
v Select the stage and choose Edit > Properties from the WebSphere DataStage
Designer Client window.
Note: You must use a user name and non-blank password for WebSphere
DataStage logon credentials to use the IDoc stages. This means you must clear
Omit on the Attach to Project dialog in WebSphere DataStage, otherwise,
unexpected results occur.
IDoc load for SAP R/3 stage

The user interface for the IDoc Load for SAP R/3 stage is nearly identical to that of
the IDoc Extract for SAP R/3 stage, with input links instead of output links.
Each input link loads records for a particular segment type within the IDoc type
that is selected for the stage. A column list for each link is generated automatically,
based on the fields of the selected segment type. In addition, special columns are
generated to represent primary and foreign key values for each link. These key
values allow the runtime to determine which child segment records flowing into
one link belong to each parent segment record that flows into a different link.
For a root-level segment type, the foreign key value identifies the specific
generated IDoc into which the segment records will be incorporated.
Since the column lists for an IDoc Load for SAP R/3 stage are generated
automatically, Transformer stages map values from source data columns to the
columns generated from the fields of the segment types. In each Transformer stage,
you must also map source data values to values that can be used as primary and
foreign keys for the segments. Values for the key columns not actually loaded as
data into the IDocs, but are only used to correlate records flowing into separate
links.

You can design jobs in any way that provides effective key values. The foreign key
values provided for a link representing a parent segment type must exactly match
primary key values provided for a link that represents the parent segment type.
For root-level segment types, the foreign key identifies the IDoc for the segments.

The GUI for the stage closely resembles that of the IDoc Extract for SAP R/3 stage.
When you start the stage GUI from the WebSphere DataStage Designer Client to
edit an IDoc Load for SAP R/3 stage, the General tab of the Stage page opens by
default.
This dialog box has the Stage and Input pages:

v Stage. This page displays the name of the stage you are editing and contains the
General and IDoc Type pages. The selected SAP connection parameters are
displayed on the General tab on the Stage page. Generally the connection is
already defined by default. See the following sections for details.
Use the IDoc Type tab of the Stage page to select a type of the IDocs to load
(see Selecting IDoc types).
The NLS page defines a character set map to use with the stage. This page
opens only if you have installed NLS for WebSphere DataStage (see Defining
character set mapping).
v Input. This page specifies the IDoc components you assign to each input link
(see Defining IDoc load input links).
The main phases in defining an IDoc Load for SAP R/3 stage from the Stage
dialog box are as follows. See the following sections for details.
1. Select a connection to SAP.
2. Select an IDoc type.
3. Optionally define a character set map.
4. Define the data on the input links.
Click OK to close this dialog box. Changes are saved when you save the job
design.
Stage General page

The parameters that are needed by WebSphere DataStage to connect to an SAP
database are defined on the General tab on the Stage page. You select a logical
WebSphere DataStage connection to SAP from the list of available connections.
This list is maintained on the server. Each connection contains all the information
needed to connect to a particular R/3 system. Logical connections are shared with
the IDoc Extract for SAP R/3 component.
Use the following components on this tab to connect to an SAP R/3 system:
v WebSphere DataStage Connection to SAP. The WebSphere DataStage
connection to the SAP R/3 system that is defined on the server machine and
shared by all users connected to that machine. The fields in this area are
read-only and are obtained from the connection that you selected. For more
information, see Selecting the WebSphere DataStage Connection to SAP.
Chapter 4. IDoc Load stage 55

– Name. The name of the selected connection to the SAP R/3 system that
generates the IDocs to be loaded by this stage. The logical connection
includes RFC client logon details.
– Select. Click to choose a WebSphere DataStage connection to an existing SAP
R/3 system or create a new one. The Select WebSphere DataStage
Connection to SAP dialog opens. The selected connection provides all needed
connection and default logon details that are needed to communicate with the
corresponding SAP R/3 system. The default SAP Logon Details are used if
Use connection defaults is selected.
– Description. Additional information about the selected connection.
– Application Server. The name of the host system running R/3. If the selected
connection uses load balancing, Message Server is used instead.
– System Number. The number assigned to the SAP R/3 system used to
connect to R/3. If the selected connection uses load balancing, System ID is
used instead.
v SAP Logon Details. The fields in this area are read-only unless you clear the
Use connection defaults box.
– User Name. The user name that is used to connect to SAP.
– Password. A password for the specified user name.
– Client Number. The number of the client system used to connect to SAP.
– Language. The language used to connect to SAP.
– Use connection defaults. Clear to remove the default SAP Logon Details
settings so you can use different logon information for only this stage in this
job. If selected, the displayed logon details are obtained from the selected
connection and are disabled.
v Enter an optional description to describe the purpose of the stage in the
Description field.
v Validate All. Select to check the stage and link properties and the column lists
for consistency and completeness.
For a newly created stage, the connection for the stage defaults to the last one you
selected for another stage of this type. If only one WebSphere DataStage connection
to an SAP R/3 system is defined on the server machine, a new stage defaults to
that connection regardless of whether you previously created an IDoc Load for
SAP R/3 stage.
Select WebSphere DataStage connection to SAP dialog

When you click Select on the General tab of the Stage page, the Select WebSphere
DataStage Connection to SAP dialog opens.
This dialog lets you choose a WebSphere DataStage connection to the SAP R/3
system. The selected connection provides all needed connection and default logon
details that are needed to communicate with the corresponding SAP R/3 system.
New, Properties, and Remove are administrative connection operations. They let
you manage the list of connections that is maintained on the WebSphere DataStage
server machine from the Select WebSphere DataStage Connection to SAP dialog:
v New. Click to open the Connection Properties dialog, which lets you define the
properties for the new connection. Here you can specify SAP connection details
and default logon details for the new connection. The Connection and Logon
Details page opens by default.

v Properties. Click to open the Connection Properties dialog, showing the
properties of the selected connection. This is the same dialog that is opened
when you click New, but in this context the connection name is read-only.
These administrative operations function similarly to the corresponding buttons on

the WebSphere DataStage Connections to SAP page of the WebSphere DataStage
Administrator for SAP utility.
Connection and Logon Details page

Use Properties to open the Connection and Logon Details page of the Connection
Properties dialog, showing the properties of the selected connection. This is the
same dialog that is opened when you click New, but in this context the connection
name is read-only.
IDoc Load Settings page

Click IDoc Load Settings on the Connection Properties dialog to complete the
configuration for loading IDocs.
The IDoc Load Settings page contains the following components:

v Partner Number to Use when Sending IDocs. This is the logical name which
represents IBM WebSphere DataStage as the source of IDocs. Specify the
case-sensitive name that corresponds to the Partner number you created to
represent WebSphere DataStage in the Partner profile in SAP.
v Destination Partner Number to Use when Sending IDocs. This is the logical
name which represents the SAP system as the target where WebSphere
DataStage sends IDocs. Specify the case-sensitive name that corresponds to the
Partner number in the SAP Partner profile.
v Location for Temporary Segment Index Files. For example, D:\DataStage\
DSSAPConnections\connection name where connection name is the name of the
WebSphere DataStage connection to SAP on the Select WebSphere DataStage
Connection to SAP dialog.
v Use default. Selected by default. If cleared, Browse is enabled.
v Browse lets you find another location.
Selecting IDoc types

After you provide a WebSphere DataStage connection to SAP on the General tab of
the Stage page, specify the type of the IDocs to be loaded on the IDoc Type tab of
the Stage page.
To configure an IDoc type:

1. Click Select beside the IDoc Type field on the IDoc Type tab of the Stage page
to open the Select IDoc Type dialog for selecting an IDoc type.
All available, released IDoc types, both basic and extended types, that are
defined on the R/3 system are visible.
Connection displays the connection name with its description for the R/3
system whose IDoc types are being shown.
IDoc Types lists all released IDoc types defined on the R/3 system.

2. Click Find on the Select IDoc Type dialog to open the Find IDoc Type dialog.
It lets you search for IDoc Types that contain user-specified substrings in their
name or description.
3. Select a type, and click OK to set as the IDoc type for the stage. The segment
hierarchy (that is, the selected IDoc type and its component segment types) is
now visible on the IDoc Type tab of the Stage page.
IDoc Load Packet Size field

Define the number of IDocs per SAP Load transaction in the IDoc Load Packet
Size field. The possible values are:
0 - All IDocs will be sent in one packet as a part of a single SAP IDoc Load
transaction. If the transaction fails, no IDocs will be loaded into SAP system.
1- The default IDoc packet size is 1, this maintains the current IDoc Load stage
behavior. Each IDoc is loaded as a part of separate SAP IDoc Load transaction.
`n’- `n’ IDocs are part of a single SAP IDoc Load transaction, where `n’ is defined
as a number greater than 1. If `n’ was set to 10, and there are 50 IDocs to be
loaded, then 5 transactions are used, each sending 10 IDocs in the packet. If there
is a problem in the fourth transaction, 30 IDocs will have been committed in three
separate transactions, no IDocs from the forth transaction will be loaded and the
fifth transaction will not occur.
IDoc Components area

The IDoc Components area on the IDoc Type tab of the Stage page shows the
segments defined for the IDoc type with their descriptions. This area contains the
following information:
v Name. Shows the hierarchical relationship among the segments using a tree
structure, with their descriptions.
(A segment type can appear only once within an IDoc type. The names for the
segments are the segment definition names, not the segment type names. You
can infer the segment type name from the segment definition name.)
v Assigned Input Link. After particular segments in the IDoc Type are assigned to
the input links of the stage, the IDoc Components control shows the names of
the links in the Assigned Input Link column of the control. This gives you an
overview of which segments are being loaded by the stage.
Click Select to open the Select Message Type for IDoc Type dialog to choose a
message type from the available types. Message Type defaults to the first available
message type for the selected IDoc type.
Defining character set mapping

You can define a character set map for a stage. Do this from the NLS tab that
appears on the Stage page. The NLS page appears only if you have installed NLS.
Specify information using the following components:

v Map name to use with stage. The default character set map is defined for the
project or the job. You can change the map by selecting a map name from the
list.

v Use Job Parameter. Specifies parameter values for the job. Use the format
#Param#, where Param is the name of the job parameter. The string #Param# is
replaced by the job parameter when the job is run.
v Show all maps. Lists all the maps that are shipped with WebSphere DataStage.
v Loaded maps only. Lists only the maps that are currently loaded.
documentation.
Defining IDoc load input links

After you select an IDoc type, you assign a segment type to each input link.
Begin the selection process from the General tab on the Input page:
The Input page has an Input name field, the General and Columns pages, and the
Columns button. The Input link pages are almost identical to the Output link
pages of the IDoc Extract for SAP R/3 stage. They display the fields to load.
v Input name. The name of the input link. Choose the link you want to edit from
the Input name list box.
v Click the Columns button to display a brief list of the columns designated on
the input link. As you enter detailed metadata in the Columns page, you can
leave this list displayed.
After you choose the input link from the Input name box, the following steps
summarize how input links function. The subsequent sections describe these steps
in detail.
1. Click Select beside IDoc Component to Load on the General tab of the Input
page to open the Select IDoc Component to Load dialog. The IDoc Component
to Load, Description, and Fields in Component information is displayed after
you select a segment for the link.
2. The selected segment type and its fields to load now appear on the General tab
of the Input page.
3. Columns are automatically generated for each link when the segment type is
selected.
4. You can delete unnecessary columns.
5. Columns for missing fields can be added by clicking Add Columns on the
Columns tab of the Input page.
Selecting the IDoc component to load

When you click Select on the General tab of the Input page, the Select IDoc
Component to Load dialog appears. This dialog contains an IDoc Components
control identical to the one on the IDoc Type tab of the Stage page.
You select the segment definition for each link from the Select IDoc Component to
Load dialog.
The most recent released version of the IDoc and its segments are used.
1. Select a segment type and click OK.
2. The dialog closes, and you return to the General tab of the Input page with the
IDoc Component to Load, Description, and Fields in Component information.

Connection Properties Connection and Logon Details
This page lets you view SAP connection and default SAP logon details:
v Select Use load balancing to set the connection to use load balancing since an
RFC server cannot itself make a load balancing connection. If the connection is
not configured to use load balancing, the connection details shown on the RFC
Server Options page are identical to those shown on this page of this dialog, and
the controls are read-only. When the connection uses load-balancing, the controls
change to Message Server, System ID, and Group.
v However, when the connection uses load-balancing, the controls on the RFC
Server Options page are modifiable and required.
Modifying columns
When you choose a segment from the Select IDoc Component to Load dialog, a
corresponding list of columns for the link including key columns is automatically
generated.
P-keys (primary keys) are named after the segment type. F-keys (foreign keys)
relate records for input links to parent records. As a result, metadata is
synchronized throughout the links when changes are made to it. F-keys and P-keys
are automatically updated, giving users flexibility in using data from different
columns in the source data.
You can view and modify these generated columns using the Columns tab on the
Input page.
The Columns tab of the Input page has the following components:
v The Corresponding Load Field for Column ″xx″ is the single-row grid near the
bottom of the tab showing the load field that corresponds to the selected
column. (The value changes depending on the selected column.) It includes
Name, Description, Internal Type, and Length information. In this window, for
example, no values are displayed since a key column was selected with no
corresponding segment field.
v Click Add Columns to add columns for any fields that are not currently
represented in the columns list. The Add Columns dialog appears where you
can select from the available load fields for columns.
You can also delete any unneeded columns.
v Click Validate Columns to check the column list for consistency with the
segment fields.
v Key Columns. Click to open the Key Columns for Link dialog. This lets you
change the number of columns that represent the primary and foreign key for
the link (see Synchronizing columns).
v Save. Click to open the Save IDoc Segment Definition dialog to save the
metadata definitions when you finish defining the columns for the link.
Columns and segments

The first column in the list on the Columns tab of the Input page,
(IDOC_MATMAS03_P_KEY in the previous example) represents the primary key
for the IDocs.
Any generated value can be used as the value for the column, but for each value
of the key, there must be exactly one record that is received through the link.
Select a corresponding segment type for the rest of the input links the same way as
previously described. As each segment type is being selected using the Select IDoc
Component to Load dialog, segment types that have already been assigned a link
are indicated in the Assigned Input Link column.
Parent and child segment types

Generally, you should assign links to parent segment types before assigning links
to the child segment types.
Use Validate All on the General tab of the Stage page to detect when a link is
assigned to a segment, but the links are not assigned to their ancestors. If you try
to assign a segment type whose parent segment type has not been assigned to
some other link, you are warned, but you can continue.
Root Segments. When a segment type is assigned to a link, the automatically

generated column list includes primary and foreign key columns. When the
selected segment type is a root segment type (that is, one that has no parent
segment type in the IDoc), the foreign key column represents the primary key for
the IDoc as a whole.
Non-Root Segments. If a non-root segment type is selected, the resulting generated

foreign key column represents the value of the primary key of the parent segment
type.
Synchronizing columns
It is sometimes more convenient to use more than one column for a particular key.
Use Key Columns to open the Key Columns for Link dialog. This lets you change
the number of columns that are used to represent the primary and foreign key for
the link. If you change the primary key to two, for example, and click OK, the
column list is refreshed, generating as many columns for each key as requested.
There are three key columns.
Changing the number of columns that are used for a primary key also causes the
column lists for links that include a corresponding foreign key to be updated.
Likewise, changing the number of columns used for a foreign key causes the
corresponding primary key columns to be updated in the link representing the
parent segment type.

Chapter 5. BAPI stage
SAP created the Business Framework, which is an open, component-based product
architecture. It allows the technical integration and exchange of business data
among R/3 SAP components and between SAP and non-SAP components.
Business Objects and their BAPIs are important components of the Business
Framework, which are used by the stage to move data across SAP and non-SAP
components in a standard way.
A Business Object is the representation of a business entity, such as an employee or

a sales order, in the SAP R/3 system.
BAPIs are standard SAP interfaces that are used by the stage to integrate with SAP
R/3 systems. They are technically implemented using function modules that are
RFC-enabled (Remote Function Call) inside SAP systems.
The availability of these BAPIs as object-oriented interfaces lets other components

directly access the application layer of an SAP system without implementation
details. BAPIs allow integration with SAP at the business level.
You can use the BAPI stage to interface with the my-SAP Business Suite, which is a
family of solutions and integrated application platforms, such as CRM, SCM, and
so forth that are supplied by SAP. Use the BAPI stage within IBM WebSphere
DataStage as a passive stage, for example, to load data into and extract data from
SAP R/3 Enterprise as the target system. Do this by using the library of Business
Application Programming Interfaces (BAPIs), which is provided and maintained by
SAP. The databases always remain in a consistent state after executing the BAPIs
(methods).
The BAPI stage lets you use these BAPIs to design load or extract jobs. You can do
this because the stage captures the metadata for each BAPI and dynamically builds
the complex RFM call to execute these BAPIs.
You can use the BAPI stage GUI to select a Business Object and its BAPI from the
SAP R/3 system and display the corresponding interface which you can use to
load or extract data. The runtime component for the stage loads or extracts data
into or from the SAP application server.
Functionality
The BAPI stage has the following functionality:
v Lets you choose and define SAP connections using the GUI.
v Lets you explore the SAP BOR, dynamically choose any BAPI, and store its
metadata in the job.
v Lets you view the BAPI interface using a function module (RFC) with its import,
export, and table parameters, and optionally decide which parameters to use
when executing the BAPI.
v Works with the my_SAP Business Suite products.
v Loads and extracts data into and from SAP R/3 using BAPIs with appropriate
log information.

v Supports NLS (National Language Support). For information, see IBM WebSphere
The following functionality is not supported:

v The creation of custom BAPIs using the stage.
v Data transformations or mappings. Use the Transformer stage to do this.
v Testing of the BAPI during design time.
The functionality of the BAPI stage is represented in the following figure:
SAP R/3 Enterprise
Application layer
BAPI BAPI BAPI BAPI
1 4
BAPI PACK GUI BAPI PACK Runtime
2 3 5
DataStage Log files

Repository
1. The GUI client logs on to the SAP R/3 application server and retrieves
metadata information from BOR.
2. The GUI client stores BAPI metadata in the WebSphere DataStage repository.
3. The runtime server reads the metadata from the WebSphere DataStage
repository and dynamically builds a BAPI call.
4. The runtime server makes a BAPI call and processes the generated data set.
5. The returned values are written to the appropriate logs.
In summary, the stage can dynamically call any BAPI and process the returned
data appropriately, based on whether it is a load or an extract BAPI.

Building a job
The BAPI stage can be a data source and a data destination. You can add links into
the stage and out from the stage. Multiple links are allowed in both directions that
let this stage call multiple BAPIs in the same job to load and extract data at the
same time.
If the job uses multiple input or output links, each link must define a unique BAPI,
that is, a link cannot contain a BAPI that is already used in another link in the
same job.
To build a job:
1. Create a WebSphere DataStage job using the WebSphere DataStage Designer
Client.
2. Create the BAPI stage, adding the stages and links you need to load and extract
data. Double-click the BAPI stage icon to open the stage editor dialog (GUI).
3. Define the stage properties.
4. Define the SAP R/3 connection details and the SAP logon information.
5. Specify the properties for input links for loading data to SAP R/3.
6. Specify the properties for output links for extracting data from SAP R/3.
7. Compile the job.
Each task is described in more detail in the following sections.
Defining stage properties

When you open the BAPI stage, the BAPI_PACK_for_R3 Stage dialog opens. The
Stage page for the BAPI stage appears by default.
This dialog has the following pages, depending on your links:

v Stage. Displays the name of the stage you are editing. You can define
WebSphere DataStage connection and logon details to a target or source SAP
system. You can also describe the purpose of the stage in the Description field
of the General tab. This page is similar to that in other R/3 and BW stages
except the connection information is stored at the stage level, not at the link
level. This enables bidirectional links.
The NLS tab defines a character set map to use with the stage, if NLS is
enabled. For details, see Defining character set maps.
v Input. Has an Input name field, the General, BAPI, Logs, and Columns pages,
and the Columns button.
v Output. Has an Output name field, the General, BAPI, Read Logs, and Columns
pages, and the Columns button.
Defining character set maps

You can optionally define a character set map for a BAPI stage using the NLS tab.
You can change the default character set map defined for the project or the job by
selecting a map name from the list.
This tab also has the following components:
Chapter 5. BAPI stage 65

v Show all maps. Lists all the maps supplied with WebSphere DataStage. Maps
cannot be used unless they are loaded using the WebSphere DataStage
Administrator Client.
v Loaded maps only. Displays the maps that are loaded and ready to use.
v Use Job Parameter . Lets you specify a character set map as a parameter to the
job containing the stage.
v Allow per-column mapping. Lets you specify character set maps for individual
columns within the table definition. If per-column mapping is selected, an extra
property, NLS Map, appears in the grid in the Columns tab.
documentation.
Defining SAP connection and logon details

The BAPI stage can use and share SAP connections created by other R/3 stages
you create, such as IDoc Load, IDoc Extract, and ABAP. For security reasons, the
logon details for the BAPI stage are not stored at the server level.
Enter the following information on the General tab:

v WebSphere DataStage Connection to SAP. The WebSphere DataStage
connection to the SAP R/3 system that is defined on the WebSphere DataStage
server machine and shared by all users connected to that machine. The fields in
this area are read-only and get default values from the last connection that you
used.
Name. The name of the selected connection to the SAP R/3 system that
generates the data to be extracted.
Select. Click to choose a WebSphere DataStage connection or define a new
connection to the SAP R/3 system. The selected connection provides all needed
connection details to communicate with the corresponding SAP R/3 system. This
opens the Select WebSphere DataStage Connection to SAP dialog.
Description. Additional information about the selected connection.
Application Server. The name of the host system running R/3.
System Number. The number assigned to the SAP R/3 system used to connect
to R/3.
v SAP Logon Details. Provides all the necessary information to logon to SAP,
which gets stored in the job.
User Name. The user name that is used to logon to SAP.
Password. A password for the specified user name.
Client Number. The number of the client system used to logon to SAP.
Language. The language used to logon to SAP.
v Description. Enter text to describe the purpose of the stage.

Click Select on the General tab of the Stage page to open the Select WebSphere
DataStage Connection to SAP dialog. It lets you view previously-defined
connections to SAP systems and choose a IBM WebSphere DataStage connection to
the SAP R/3 system. The selected connection provides all required connection
details to communicate with the corresponding SAP R/3 system. You can create
and define new connections and delete existing connections.

Since the information is stored on the WebSphere DataStage server, it lets you
share connections across several WebSphere DataStage clients that you created, for
example:
New, Properties, and Remove are administrative connection operations, similar to

those for other R/3 stages with these pages. They let you manage the list of
connections that is maintained on the WebSphere DataStage server machine from
the Select DataStage Connection to SAP dialog. They function in the same way as
the corresponding buttons on the DataStage Connections to SAP page of the
WebSphere DataStage Administrator for SAP utility.
For more information about creating or defining new connections and deleting
existing connections, see the following section and Administrator for SAP Utility.
Defining connection properties

Click Properties on the Select DataStage Connection to SAP dialog to open the
Connection Properties dialog. The Connection Properties dialog has the Connection
and Logon Details and RFC Server Options pages. Use the dialog to create a new
connection or to view the properties of the currently selected connection,
depending on whether you open it using New or Properties respectively.
Information such as Connection Name and Description is automatically added if
upgrading job from previous version of BW.
The DataStage Connections to SAP page of the WebSphere DataStage

Administrator for SAP utility lets you specify a non-load balancing connection
even if load balancing is specified in the connection configuration.
This page has the following components:

v Router String. Optional. The string used to connect to the remote SAP server. If
you use a SAP router to access your system, you must include a fully-qualified
node name for your RFC server machine in the SAP router table.
v Use load balancing. Select to use load balancing when connecting to R/3. Client
connections are made through a message server rather than an application
server.
The Application Server and the System Number controls are replaced by Message
Server, System ID, and Group controls so that connection details specific to load
balancing can be entered.
Defining Input properties

Begin the selection process from the General tab on the Input page. When you
click the Input tab, the General tab appears by default, displaying the fields used
for loading data.
The Input page has an Input name field, the General, BAPI, Logs, and Columns
pages, and the Columns button.
v Input name. The name of the input link. Choose the link you want to edit from
the Input name list.

v Click the Columns button to briefly list the columns designated on the input
link. As you enter detailed metadata in the Columns page, you can leave this list
displayed.
Input General page

This resizable tab has the following read-only components (except Description):
v Business Object. The name of the Business Object in the SAP R/3 system used
to load data.
v Method. The name of the BAPI in the SAP R/3 system you are using.
v Short Text. The text that describes the selected BAPI.
v BAPI Explorer. Click to open the Specify Filter Criteria for BAPIs dialog. It lets
you explore the SAP BOR.
v Description. Enter text to describe the purpose of the link.
After you select a BAPI on the BAPI Explorer dialog, the General tab
automatically displays the selected Business Object, BAPI, and short description of
the selected BAPI.
Filtering business objects

When you click BAPI Explorer on the General tab of the Input or Output page,
the Specify Filter Criteria for BAPIs dialog opens first. The dialog lets you select
an ID to filter the business objects that are based on SAP components.
This interim dialog opens only for release 4.6 and later of SAP. Otherwise, the
BAPI Explorer window opens.
This dialog has the following components:

v Application Component ID. You can select a specific component from the list or
use <All Components> to select all components from the list.
After you select a component ID, this dialog contains only those Business objects
that belong to the selected application component. The list of available application
components to choose from is dynamically built, depending on the accessed
system.
v BAPI’s to Display. You can show only released BAPIs (the default) or all BAPIs
in the BAPI Explorer window. You must ensure unreleased BAPIs are complete
and ready to use, or a warning appears. (Click Show All BAPIs to see those
BAPIs that are unreleased.)
v OK. Click to open the BAPI Explorer dialog.
v Cancel. Click to return to the Input or Output General page.
BAPI Explorer dialog

Click OK on the Specify Filter Criteria for BAPIs dialog to open the BAPI
Explorer dialog. This lets you browse the SAP BOR to select a BAPI.
Defining BAPI interfaces

The BAPI tab of the Input page displays the interface for the selected BAPI. Use
this tab to design the runtime behavior of the selected BAPI.

You can view the parameters or attributes for the selected BAPI on the Import (the
default), Tables, and Export tabs. Fields that are required for import and table
parameters appear on the Columns page.
For information about parameters, see Runtime component.
Input BAPI Import page

When a BAPI is first selected, only required parameters are active on the BAPI
Import tab of the Input page. For input links, fields that are required for import
parameters appear on the Columns page.
You can tell whether a parameter is active by the following indications:

v Green icons beside the parameter names indicate that the parameters are active,
that is, used to dynamically build BAPI calls at run time. (Red icons indicate
that parameters are inactive, unused when calling BAPIs.)
v Green icons display I or E to indicate whether table parameters are activated for
Import or Export.
To activate or deactivate parameters, do one of the following:

v Double-click a parameter name
v Right-click for a shortcut menu
When you move the cursor over a parameter, its shape changes, indicating that
you can activate or deactivate parameters.
About the Input BAPI Tables page

Click the Tables tab on the Input BAPI page to view the table parameters of a
BAPI.
Fields that are required for tables parameters appear on the Columns page.
Input BAPI Export page

Click the Export tab on the Input BAPI page to view the export parameters of a
BAPI. By default, export parameters are optional and initially inactive.
Fields that are required for export parameters appear in a grid on the Input Logs
page, not on the Input Columns page, because they contain return values from the
BAPI call.
The reference structure being used for each parameter appears with a short text
description of the parameter. This helps you decide whether to activate a particular
parameter.
Defining log files

Click the Logs tab of the Input BAPI page to view information about the return
values for exported fields from the BAPI call and the directory where these values
get stored.
The Logs tab has the following options:

v Location for Return Parameters and other Log Files. By default, you see a path
name whose directory corresponds to the selected BAPI. This directory for
temporary log files is created under the DataStage\DSSAPConnections
directory.
For example, if StandardMaterial.SaveData is the selected BAPI for the stage.

Assuming the connection name is C46, the default directory for return values is:
<DSHOME>\DSSAPConnections\C46\BAPIs\Loads\Standard
Material.SaveData
where <DSHOME> is the WebSphere DataStage home directory.

v Browse. Click to open the Browse directories dialog to find existing directories to
store the values. (Clear Use default directory.)
v Use default directory. Clear to use Browse to search for another directory to
store the values or to enter the path name in Location for Return Parameters and
other Log Files.
v Grid. Displays the fields that correspond to the parameters selected in the
Export tab on the Input BAPI page.
Input Columns page

This page stores metadata for all the input fields to this link.
All fields in the grid except the first (BAPISeqNo) are derived from reference
structures for the import and tables parameters that get activated on the Input
BAPI page.
BAPISeqNo is automatically generated by the GUI to let the runtime component

distinguish between parent and child rows in an input data set that represents a
header-detail relationship. For example, if the data set contains a header record
with multiple child records, this field contains the same value for each of those
rows.
As for other stages, you can use the following components for the column
definitions:
v Save. Click to open the Save Table Definition dialog to save column definitions
to the WebSphere DataStage repository.
v Load. Click to open the Table definitions dialog to load the column definitions
from the WebSphere DataStage repository into other stages in your job design.
Defining output properties

When you click the Output tab, the General tab appears by default.
This resizable window looks exactly like the Input General page, except that the
selected BAPI extracts data from SAP.
The Output page has an Output name field, the General (the default), BAPI, Read
Logs, and Columns pages, and the Columns button.
v Output name. The name of the output link. Choose the link you want to edit
from the Output name list.

v Columns. Click to briefly list of the columns designated on the output link. As
you enter detailed metadata in the Columns page, you can leave this list
displayed.
Output General page

This window has the following read-only components (except Description):
v Business Object. The name of the selected Business Object.
v Method. The name of the BAPI you use for extracting data. After you select a
BAPI, the page automatically displays the related information for the other
fields.
v Short Text. The text that describes the selected BAPI.
v BAPI Explorer. Click to open the Specify Filter Criteria for BAPIs dialog to
explore the SAP BOR (see Filtering business objects).
v Description. Enter text to describe the purpose of the link.
After you select a BAPI on the BAPI Explorer dialog (see BAPI Explorer dialog),
the General tab of the Output page automatically displays the selected Business
Object, BAPI, and short description of the selected BAPI.
Output BAPI page

Click the BAPI tab of the Output page to open the Import tab by default. You can
view parameters or attributes of the BAPI on the Import, Tables, or Export page,
depending on their type.
This page displays the attributes for the selected BAPI. As for the Input BAPI page,
you can design and determine the runtime behavior of the selected BAPI.
Output BAPI Import page

When you first select a BAPI, only required parameters are active.
The color of the icons beside parameter names indicate whether they are active:
v Green. Indicate active parameters that are used when dynamically building the
BAPI call at runtime.
v Red. Indicate inactive parameters that are unused when calling the BAPI.
Double-click a parameter name or right-click to open a shortcut menu to activate

or deactivate parameters. When you move the cursor over a parameter, its shape
changes, indicating whether you can activate or deactivate parameters.
Output BAPI Tables page

You can activate or deactivate table parameters for Import or Export. When values
need to be passed to SAP using table parameters, you can activate them for Import
(or Input). When values need to be extracted from SAP, you can activate table
parameters for Export.
Use the shortcut menu to activate table parameters:

v Activate Selected Parameter. By default, if you activate a table parameter by
double-clicking, it is activated for Export.
v Activate Selected Parameter for Input. Indicates that you can activate a table
parameter for Input only and use the shortcut menu.
A green icon displays I or E to indicate whether the table parameter is activated
for Import or Export.
Output BAPI Export page

Export parameters, by default, are optional (initially inactive).
The reference structure is being used for each parameter with a short text
description of the parameter to help you decide whether to activate a particular
parameter.
For output links, the following parameters appear on the Columns page:
v Parameters activated on the Export tab
v Tables parameters that are activated for extracting values
Fields for import parameters and the table parameters activated for Input Import
appear on the Output Read Logs page. This is because these fields are not
extracted when the job runs but need to be read (loaded) to call a BAPI.
Format of BAPI input file for import parameters

The input file for import parameters passes input parameters to SAP R/3 for BAPI
extraction jobs.
The BAPI input file for import parameters passes data to the BAPI as
comma-separated values (CSV). The file has the same encoding as the SAP R/3
platform (therefore, on Unicode platforms, the file encoding is Unicode). You
specify the full path and file name of the input file in the Input File for Import
Parameters field of the Output Read Logs tab.
The input parameters can be structures or tables. The input-file format matches the
columns that are displayed by the BAPI Output Read Logs page.
v Each column displayed by the Read Logs page is represented in the input file as
a comma-delimited field.
v Each line in the file represents a call to the BAPI. Each call takes the input data
from one line. (To pass multiple lines to BAPI tables, use the Load design.)
v Use SAP R/3 to display the data in a table or structure.
Sample
The following example input file makes five BAPI calls. Each call takes the input
data from one line. (Each line in this example is truncated to fit the page.)
4500000153,,,,,,abc,,,,,,12,,,,,org,,,,345,f,,,,,,,,,,,,,,,,,,,,,,,,,IS,22,1,,,
4500000153,12345,d,text0,,,,,,,,,,,,,,,1234567890.123,pou,uis,opu,opi,45678,,,,
4500000153,,,,,,abc,,,,,,12,,,,,org,,,,345,f,,,,,,,,,,,,,,,,,,,,,,,,,IS,22,1,,,
4500000153,22347,d,text1,,,,,,,,,,,,,,,2234567890.123,pou,uis,opu,opi,45678,,,,
4500000153,,,,,,abc,,,,,,12,,,,,org,,,,345,f,,,,,,,,,,,,,,,,,,,,,,,,,IS,22,1,,,
Output Read Logs page

The Output Read Logs page contains information about the imported fields for the
BAPI call and the directory from which these values get loaded.
The grid displays those fields that correspond to the parameters selected on the
Import tab on the Output BAPI page and the fields that correspond to parameters
activated on the Tables tab for Input on the Output BAPI page.

The locations for import parameters and other log files are specified on this
window. By default, a directory name that corresponds to the selected BAPI that
gets created in the DataStage\DSSAPConnections directory appears.
For example, if the selected BAPI for the stage is PurchaseOrder.GetDetails and the
connection name is C46, the default directory for import values is:
DSHOME\DSSAPConnections\C46\BAPIs\Extracts\PurchaseOrde
r.GetDetails
where <DSHOME> is the WebSphere DataStage home directory.
You can specify another directory to store these import values by clearing Use
defaults and browsing for existing directories or by entering a value in the edit
control.
Click Browse beside Location for Log Files to open the Browse directories dialog
to select a directory.
You can also click Browse beside Input File for Import Parameters to open a
dialog to select a file.
Output Columns page

This page stores metadata for all output fields for this link. All fields in the grid
except the first (BAPISeqNo) are derived from reference structures for the export
and tables parameters that get activated on the Output BAPI page.
The BAPISeqNo field is generated automatically so the runtime component can

distinguish between parent and child rows in an output data set. They represent a
header-detail relationship. For example, if the data set contains a header record
with multiple child records, the field for each of those rows contains the same
value.
Completing the job

Complete the definition of the other stages in your job according to normal
WebSphere DataStage procedures. Compile and run the job.
Runtime component
The BAPI stage runtime component is a modified sequential file stage that
supports multiple links in both directions. It dynamically builds a call to the
selected BAPI and runs that call.
Input links. The return values from the BAPI call are stored in the appropriate
directory that you specify on the Input Logs page.
The name of the log files corresponds to the parameter names that are activated for
the BAPI, for example:
v Export parameters. If an export parameter named PURCHASEORDER is
activated for the BAPI, a log file named PurchaseOrder.txt is created in the
Export directory in the directory specified on the Input Logs page.
v Tables parameters. You can use table parameters to import or export values.
Two log files are created for each parameter, which contain the following:

– The parameter name with the .TXT suffix
– The parameter name with the __RETURN.TXT suffix
For example, for the table parameter named DESCRIPTION, log files named
DESCRIPTION.TXT and DESCRIPTION_RETURN.TXT are created.
Output links. The job expects import values to be available in the Import directory
that gets created under the pathname specified on the Output Read Logs page. The
name of the log files corresponds to the parameter names that are activated for the
BAPI, for example:
v Import parameter. If an import parameter named PURCHASEORDER is
activated for the BAPI, a log file named PurchaseOrder.txt is expected to be in
the Import directory in the path name specified on the Output Read Logs page.
v Tables parameters. You can use table parameters to import or export values.
Two log files are created for each active parameter, which contain the following:
– The parameter name with the .TXT suffix
– The parameter name with the __Return suffix
For example, if the table parameter named DESCRIPTION is active,

DESCRIPTION.TXT and DESCRIPTION_RETURN.TXT log files are created.

Chapter 6. WebSphere DataStage Administrator for SAP Utility
Many of the features provided by the stage editor for managing and configuring
both R/3 connection properties and IDoc type properties are not associated with a
specific IBM WebSphere DataStage job but are more global in nature. It is
convenient to have this functionality available during job design using the stage
editor because it is quite probable that a new job might necessitate the
configuration of a new connection and a new IDoc to be received by that
connection.
However, subsequent changes to these configurations have the potential to impact

many jobs. For example, the WebSphere DataStage Administrator Client may
determine that the batch count for an IDoc is set too low. This can cause
WebSphere DataStage jobs that process the IDoc to become backed up since they
cannot complete before the next batch of IDocs arrive. The Administrator can
increase the batch count for this IDoc type, and all jobs processing the IDoc type
are affected by this change.
The Pack includes a stand alone utility called WebSphere DataStage Administrator
for SAP that lets you manage the configurations of the various R/3 connection and
IDoc type objects that are beyond the scope of a WebSphere DataStage job.
You can invoke this utility from the WebSphere DataStage program group on a
WebSphere DataStage client machine to do the following:
v Define the connection to the SAP database.
v Monitor the activity of the IDoc listener.
v Migrate connection and IDoc type configuration settings from one WebSphere
DataStage installation to another.
v Modify connection and IDoc type configuration settings.
v Schedule cleanup of temporary IDoc files.
WebSphere DataStage Administrator for SAP is a utility that contains the

WebSphere DataStage Connections to BW, WebSphere DataStage Connections to
SAP R/3, and IDoc Cleanup and Archiving pages, depending on which stages you
install.
WebSphere DataStage connections to SAP R/3 page

This page displays all the connections to SAP that are currently defined on the
WebSphere DataStage server machine. Add, Properties, Import into, Export, and
Remove let you manage the list of connections that is maintained on the
WebSphere DataStage server machine. They function the same as the
corresponding buttons on the Select DataStage Connection to R/3 dialog.
The following steps let you define a WebSphere DataStage connection to SAP. The
subsequent sections describe these steps in detail.
1. Run the WebSphere DataStage Administrator for SAP on a WebSphere
DataStage client machine.
2. Click Add > New to define the properties for the new connection.
3. Enter a connection name, a description, and SAP connection and logon details.

4. Optionally, use load balancing.
5. Enter an IDoc listener program ID (the same ID must be defined for the tRFC
port in the SAP database).
6. Define the properties for running jobs automatically as IDocs arrive on the
DataStage Job Options for IDocs page.
7. Click Add to test the connection and logon properties and add the connection.
An IDoc listener server starts automatically after the new connection is added.
8. Configure the SAP system to access WebSphere DataStage by doing the
following:
a. Log on to the SAP R/3 system.
b. Create a tRFC port, and assign it the IDoc listener program ID that was set
for the connection.
c. Create a logical system to represent WebSphere DataStage.
d. Attach the tRFC port to the logical system.
9. Click IDoc Types to see the IDoc types for the selected connection. It opens the
IDoc Types dialog where you can set the properties of IDoc types.
The WebSphere DataStage Connections to R/3 page has the following components:
v Add. Click to open a popup menu with the New and Import items. Do the
following:
1. Click New to open the Connection Properties dialog, which lets you define
the properties for the new connection. Here you can specify Connection
Name, Description, and SAP connection and default logon details for the
new connection (See Defining WebSphere DataStage connections to SAP
R/3).
2. Click Import to open a standard Open File dialog that lets you select the
export file to be imported. This lets you import a new connection (see
Importing new connections).
v Properties. Click to open the Connection Properties dialog, showing the
properties of the selected connection. This is the same dialog that is opened
when you click New on the Select DataStage Connection to SAP dialog, but in
this context the connection Name is read-only, and the Add button is replaced
by an OK button.
v Import into. Click to import IDoc type configurations into the selected
connection that already exists. A standard Open File dialog appears so you can
select a file to be imported (see Importing IDoc type configurations).
v Export. Click to save the configuration information for the selected connection
and all its associated IDoc types into a file (see Exporting connections).
v IDoc Types. Click to see the IDoc types for the selected connection. It opens the
IDoc Types dialog (see IDoc Types dialog). This button does not appear on the
Select WebSphere DataStage Connection to SAP dialog.
v IDoc Log. Click to open the IDoc Log dialog. This dialog displays log messages
reported by the IDoc listener for the connection (see the section IDoc Log
dialog). This button does not appear on the Select WebSphere DataStage
Connection to SAP dialog.
If Listen for IDocs received through this connection is cleared on the IDoc
Listener Settings page, the Import Into, IDoc Types, and IDoc Log buttons on this
page are disabled when you select that connection. Also, if this same option is
cleared, Import and Export operations apply only to the properties for the

connection. They do not involve IDoc Type configurations. (This is useful for
connections that are only used for ABAP or BAPI stages, rather than IDoc stages.)
Defining WebSphere DataStage connections to SAP R/3

Click Properties on the DataStage Connections to R/3 page of the WebSphere
DataStage Administrator for SAP utility to open the Connection Properties dialog.
This dialog contains the Connection and Logon Details, IDoc Listener Settings,
and the WebSphere DataStage Job Options for IDocs pages.
The WebSphere DataStage Connections to R/3 page lets the RFC Server make a
non-load balancing connection even if load balancing is specified in the connection
configuration and provides an option that prevents the Listener from sending
status updates back to the R/3 system when IDocs are received.
Connection and Logon Details page

Use this page to specify a name, description, and SAP connection and default SAP
logon details for the new connection.
The Connection and Logon Details page opens by default. Group appears for this
window when Use load balancing is set.
To define the WebSphere DataStage connection to SAP:

1. Specify the SAP connection details:
v Router String. Optional. The string used to connect to the remote SAP
server. If you use a SAP router to access your system, you must include a
fully-qualified node name for your RFC server machine in the SAP router
table.
2. Select Use load balancing to use load balancing when connecting to R/3. The
Application Server and the System Number controls are replaced by Message
Server, System ID, and Group controls so that connection details specific to
load balancing can be entered (see Load Balancing).
3. Specify the default SAP logon details:
v User Name. The name of the user for connecting to SAP.
v Password. The password for User Name.
v Client Number. The SAP client number.
v Language. The language used for connecting to SAP.
Load balancing
Select Use Load balancing on the Connection and Logon Details page of the
Connection Properties dialog to balance loads when connecting to the R/3 system.
Load balancing works as follows:
1. R/3 lets you make logon connections through a message server. The message
server uses an algorithm that considers server workload and availability to
choose an appropriate application server to handle the logon.
When connections are configured, you can choose a load balancing connection
to a message server rather than a specific R/3 instance to retrieve and validate
IDoc types and metadata.
Chapter 6. WebSphere DataStage Administrator for SAP Utility 77

2. If load balancing is selected for this connection, the listener server uses load
balancing when returning status updates to R/3.
3. If load balancing is selected, the stage runtime component uses load balancing
when connecting to R/3 to validate IDoc type metadata at job execution time.
The listener server does NOT use this connection to listen for IDocs arriving on an
RFC port since this is an RFC server connection. Load balancing is only a client
connection feature.
IDoc Listener Settings page

Click IDoc Listener Settings to open the page.
This page lets you make an additional load balancing test connection as follows:
1. Select Listen for IDocs received through this connection so that a listener
server runs continuously on the WebSphere DataStage server machine. This
check box is selected by default. If selected, this option indicates that the
connection is enabled for use with IDoc Extract or IDoc Load.
If Listen for IDocs received through this connection is cleared, the other
controls on the IDoc Listener Settings page and DataStage Job Options for
IDocs page are disabled (including the labels for the controls). Also if cleared,
the Import Into, IDoc Types, and IDoc Log buttons on the DataStage
Connections for R/3 page of the WebSphere DataStage Administrator for SAP
utility are disabled when you select that connection.
2. Specify IDoc Listener Program ID. The listener registers this program ID with
the R/3 system. Use the same program ID for the tRFC port on the SAP R/3
system to be invoked when the R/3 system sends an IDoc to WebSphere
DataStage.
3. Clear Return status update upon successful receipt of IDoc to prevent the
listener from sending status messages to the R/3 system when IDocs are
received. (It is set by default.) Do this to reduce the load incurred by the R/3
system when it sends IDocs to WebSphere DataStage.
4. If you select Use load balancing on the Connection and Logon Details page,
and the load balancing test connection succeeds, an additional test connection
is made using the IDoc Listener SAP Connection Details that you specify.
If this test connection fails, the following warning appears:
Currently unable to connect to SAP using the IDoc Listener
connection information specified for this connection. Do
you want to save your changes anyway?
Separate connection details must be provided for the listener, since the Listener
cannot make a load-balancing connection. If load balancing is indicated on the
Connection and Logon Details page, you can modify the Application Server,
System Number, and Router String controls shown on the IDoc Listener Settings
page.
The Application Server, System Number, and Router String information is used
by the listener when it connects to the R/3 system. If you clear Use load balancing
on the Connection and Logon Details page, the listener uses the connection details
specified on that page. In that case, the Application Server, System Number, and
Router String controls on the IDoc Listener Settings page displays values copied
from the Connection and Logon Details page, but these values are read-only
(disabled).

WebSphere DataStage Job Options for IDocs page
This page contains information used to run WebSphere DataStage jobs
automatically when a specified number of IDocs of a given type are received from
the R/3 system specified for the connection. (This feature also depends on
properties that are defined elsewhere for each IDoc Type.) The properties default to
the current WebSphere DataStage logon details for the user.
Click WebSphere DataStage Job Options for IDocs to open the page:
To specify job options and add the new connection:

1. Select Run appropriate DataStage jobs automatically after receiving IDocs
from this SAP system to run jobs automatically after the specified number of
IDocs of a given type are received from the R/3 system specified for the
connection.
Default WebSphere DataStage Logon Details for Running the Jobs displays
the default user name and password for the WebSphere DataStage user.
2. Click Add to add the new connection. (If you open the Select DataStage
Connection to SAP dialog using Add, you see Add here. If you open the
Connection Properties dialog using Properties, you see OK here.)
3. Specify Port Version. This property specifies the version of the port used to
send IDocs to WebSphere DataStage. It lets the R/3 system appear to a third
party application such as WebSphere DataStage as if the R/3 system has an
earlier version than it actually has. It lets you tell WebSphere DataStage to
expect IDocs to have an earlier version of control record and administrative
field metadata. Thus, the third party applications do not have to change when
the R/3 system is upgraded (see the next section).
Reading IDoc metadata

The Port Version list on the WebSphere DataStage Job Options for IDocs page of
the Connection Properties dialog specifies the version of the port that sends IDocs
to WebSphere DataStage. This property lets the R/3 system appear to a third party
application such as WebSphere DataStage as if the R/3 system has an earlier
version than it actually has. It lets you tell WebSphere DataStage to expect IDocs to
have an earlier version of control record and administrative field metadata. Thus,
third party applications do not have to change when the R/3 system is upgraded.
This property is related to the R/3 Version list on the IDoc Type Properties page.
Select one of the following:

v 4.x. More recent (4.0A or later) metadata is used for control records and
administrative fields. 4.x is the default version. If you select 4.x, but the specified
R/3 system is version 3.2 or earlier, the following error occurs when you try to
save your changes to the connection. (No error occurs if the connection is
unsuccessful.)
v 3.0/3.1. Stages that read IDocs for this connection use metadata for IDoc control
records and segment administrative fields that corresponds to the metadata used
for R/3 Version 3.1 or earlier systems. This metadata is used even if the R/3
system specified in the connection actually has a later version.
Importing new connections

To import new WebSphere DataStage connections to SAP:

1. Click Add > Import from the WebSphere DataStage Connections to R/3 page
to import new connections. The list control shows the IDoc types whose
configurations are to be imported. This includes only those IDoc types that are
not already configured for the connection.
2. A standard Open File dialog opens that lets you select the export file to be
imported. Normally, the export file is created by exporting a connection from
another WebSphere DataStage installation.
3. The Add Connection dialog opens, with property values that default to those of
the connection being imported. If the name of the connection is the same as
that of an existing connection, you are asked if you want to overwrite the
existing connection.
4. The Import IDoc Type Configurations dialog opens, which shows the name and
description of the newly imported connection, and lists the IDoc types whose
configurations are imported with the connection (see the following section).
Importing IDoc type configurations

To import IDoc type configurations into existing connections:
1. Click Import into from the WebSphere DataStage Connections to SAP page to
open the standard Open File dialog. This lets you select a file to import.
2. The Import IDoc Type Configurations dialog opens. This dialog displays the set
of IDoc types that have been configured for the connection whose IDoc types
you have chosen to import. These types are imported into an existing
connection whose name and description are also displayed in the dialog.
This is the same dialog used when importing an entire connection using the
Import menu pick. (The list control for the Import menu pick showing the IDoc
types whose configurations are to be imported includes only those IDoc types
that are not already configured for the connection.)
The capability to export SAP connection definitions with their associated IDoc
types lets you easily migrate your configurations from one WebSphere
DataStage server system to another.
3. The following information is displayed:
v Import into Connection. The existing connection receiving imported IDoc
types.
v Description. A description of the connection.
v Import configurations for these IDoc Types. A list of the IDoc types whose
configurations are imported.
4. Click Configuration to view and modify the configuration for the selected IDoc
type. The list control has check boxes for each item, so you can import
configurations for a subset of the IDoc types shown in the list.
5. When you click OK, the configurations are imported. As each checked IDoc
type configuration is imported, the IDoc type disappears from the list control.
6. If any errors occur while importing a configuration, the import process stops,
the IDoc type with the problem is highlighted, and the problem is reported in a
message box. You can resolve the problem by changing the properties or skip
the import for this particular IDoc type by clearing its check mark and clicking
OK again so any remaining configurations are imported.
Exporting connections
To export connections by saving configuration information into a file:

1. Click Export from the WebSphere DataStage Connections to SAP page to
open the Export Connection dialog. This dialog lets you save the following
configuration information for the selected connection and all its associated IDoc
types into a file:
v Connection. The connection whose configuration you want to export.
v Description. A description of the connection.
2. The Export configurations for these IDoc Types control lists all IDocs types
that are configured for the connection. You can export only a subset of these
configurations by clearing the check boxes next to the IDoc types whose
configurations you do not want.
3. Click Exported Properties to modify the connection properties to be exported.
4. Click Exported Configuration to modify the IDoc type configuration details to
be exported for a selected IDoc type.
5. After you click OK, a Save As dialog opens so you can specify the name and
location of the export file. The file is given a .cxp extension.
IDoc Types dialog

After you export connections, saving configuration information into a file, continue
on the WebSphere DataStage Connections to SAP page.
To set the properties of the IDoc types for the selected connection:
1. Open the IDoc Types dialog by clicking on IDoc Types on the WebSphere
DataStage Connections to SAP page.
This dialog is similar to the Select IDoc Type dialog, except OK is replaced by
Close, and there is no Cancel.
The Connection field specifies the connection whose IDoc types are displayed
in the list with descriptive text in the Description field.
2. Click Find to open a Find IDoc Type dialog to search for IDoc types that
contain user-specified substrings in their name or description as in the Select
IDoc Type dialog.
3. Click Properties to examine and change the WebSphere DataStage
configuration for the selected IDoc type using the IDoc Type Properties dialog.
(You can also do this by double-clicking an IDoc type in the list.)
IDoc Log dialog

To view log messages about the IDoc listener associated with a connection:
1. Open the IDoc Log dialog by clicking IDoc Log on the DataStage Connections
to SAP page.
The Connection field specifies the connection whose IDoc log messages are
displayed. Descriptive text is included in the Description field.
2. The IDoc Log Messages field lists log messages about the activities of the IDoc
listener that is associated with the connection. When the dialog first opens, this
list is automatically scrolled to the end so that the most recent messages are
visible.
3. Click Refresh to reload the log messages, including any that were generated
since you first opened the dialog.
4. Click Clear Log to delete the messages currently in the log (after you provide
confirmation) and refresh the display.

Chapter 7. SAP Authorization Requirements for ABAP
This section describes the SAP authorization requirements to run Version 3.0.1r1 of
the ABAP Extract for SAP R/3 stage. It documents how to manually create an
Z_DS_PROFILE authorization profile and the Z_RFC_DS_SERVICE RFC function
module in SAP R/3.
You already have these two components installed on your R/3 system if you
imported transport requests. However, if this import is unsuccessful, you can
manually create these components in your R/3 systems (see Creating an
authorization profiles manually and Installing the RFC function code manually.
The section also describes the configuration of the SAP Dispatch/Gateway service.
It further includes information about packaged extraction jobs that include
common extraction scenarios which you can customize for your environment.
SAP authorization requirements

When third party products need to communicate with R/3 systems, system
administrators and project managers must enforce security for user access to the
business information.
The IBM WebSphere DataStage ABAP Extract for R/3 stage addresses this concern
and provides a comprehensive and flexible way to communicate with R/3 systems
without compromising the security aspects that already exist.
Depending on the level of security desired and the type of system (DEV, QAS, or
PRD) from which to extract data, you can assign the standard SAP S_A.DEVELOP
profile or Z_DS_PROFILE that is shipped as a transport request on the Pack client
CD to ABAP Extract stage users. The Z_DS_PROFILE, which you can install by
importing the transport request or create manually, contains the minimum
authorizations for designing and running WebSphere DataStage jobs.
These limited authorizations ensure that ABAP Extract stage users can run only
those transactions to which they have permissions and are denied access to other
transactions. For more details about each authorization, see Creating an
authorization profile manually.
Creating an authorization profile manually

The following sections describe how to create a Z_DS_PROFILE authorization
profile manually.
This profile contains these SAP authorization requirements:

v RFC authorization for WebSphere DataStage
v Authorization for WebSphere DataStage
v CPI-C authorization for WebSphere DataStage
v Table display authorization for WebSphere DataStage
v Background processing authorization for WebSphere DataStage

The SAP Administrator needs to create a profile in the R/3 system. This profile
contains the authorization objects referenced in the following text with their
respective authorizations. The Administrator must then assign this profile to
WebSphere DataStage extract users.
The SAP Administrator should do the following:

1. Create a Z_DS_PROFILE profile (SAP transaction SU02) containing the
following authorization objects for RFC authorization for WebSphere DataStage.
Define the authorizations, which are described in the following sections, before
the profile is created.
a. Authorization Objects. The standard SAP objects.
b. Authorizations. The authorizations that must be created with the values
defined in the following sections (SAP Transaction SU03). In some cases,
you can use a standard SAP authorization as suggested. You can use the
naming conventions for your organization instead of those used in these
sections, that is, Z:DS_XXX.
Authorization
Object Text Authorization
S_RFC Authorization check for RFC Z:DS_RFC
access
S_DATASET Authorization for file access Z:DS_DATASET
S_CPIC Calls from ABAP programs Z:DS_CPIC
S_TABU_DIS Table maintenance (using Z:DS_TABU
standard tools, such as
SM30)
S_BTCH_JOB Background Processing: Z:DS_BJOB
Operations on Background
Jobs
S_XMI_PROD Authorization for external Z:DS_XMI
management interfaces (XMI)
2. Assign the profile to ABAP Extract users.
RFC authorization for WebSphere DataStage

Define the following authorization:
v Authorization. Z:DS_RFC - no standard authorization available
v Text. RFC authorization for WebSphere DataStage
v Class. AAAB - cross-application authorization objects
v Object. S_RFC - authorization check for RFC access
Field Description Value

ACTVT Activity 16
RFC_NAME Name of RFC to be protected CADR, CMON, RFC1, SDTX,
Y*, Z*, SYST, SXBP, SPFL, SXMI
RFC_TYPE Type of RFC object to be FUGR
protected
Authorization for WebSphere DataStage


v Authorization. Z:DS_DATASET (you can use the standard authorization
S_DATASET_AL)
v Text file access. Authorization for WebSphere DataStage
v Class. BC_A - Basis: Administration
v Object. S_DATASET - Authorization for file access

ACTVT Activity 34
PROGRAM ABAP program name Y*, Z*
FILENAME Physical file name *
CPI-C authorization for WebSphere DataStage

v Authorization. Z:DS_CPIC (you can use the standard authorization S_SAPCPIC)
v Text. CPI-C authorization for WebSphere DataStage
v Object. S_CPIC - CPIC call from ABAP programs

ACTVT Activity 37
PROGRAM ABAP program name Y*, Z*
CPICDEST Symbolic destination ’’
ABAPFORM Name of an ABAP FORM routine CPIC
Table display authorization for WebSphere DataSTage

v Authorization. Z:DS_TABU (you can use the standard authorization
S_TABU_SHOW)
v Text. Table display authorization for WebSphere DataStage
v Object. S_TABU_DIS - table maintenance (using standard tools, such as SM30)

ACTVT Activity 03
DISBERCLS Authorization group *
To upload or delete ABAP code, SAP users (CPIC and Dialog) must be
registered within SAP as developers. To do this, log on to the Online Support
System (OSS), and select the appropriate SAP installation number. Go to
Register > Register Developer, and type the SAP user name. OSS provides a 20
character access key. Record this access key, and create a dummy ABAP
program. When SAP prompts you for an access key, type this OSS key, and the
system will confirm it.
Chapter 7. SAP Authorization Requirements for ABAP 85

Background processing: Operations on background jobs
v Authorization. Z:DS_BJOB Background Administrator for WebSphere DataStage
v Text. Background Processing: Operations on Background Jobs
v Class. Basis: Administration
v Object. S_BTCH_JOB

JOBACTION Job operations RELE
JOBGROUP Summary of jobs for a group *
Authorization for external management interfaces

v Authorization. Z:DS_XMI for connection to external system
v Text. Authorization for external management interfaces
v Class. Basis: Administration
v Object.

INTERFACE Interface ID XBP
EXTPROD XMI logging: Program name DATASTAGE
external management tool

EXTCOMPANY XMI logging: Company name external DataStage2
management tool
For security purposes, the Program name and Company name are required and
are case sensitive.
Installing the RFC function code manually

To install an RFC manually, you need to consider installation prerequisites before
you begin the installation. The windows in the following sections may differ
depending on your SAP R/3 version.
Installation Prerequisites. To install the Z_RFC_DS_SERVICE RFC function code

manually, follow these guidelines:
v You must be the SAP Administrator or an SAP user with S_A.DEVELOP
authorization to create the RFC function.
v You can create a function group or find an existing function group. If you use an
existing function group and you are not the SAP Administrator, you must be the
owner of the function group.
Installation Steps. To install the RFC code:

1. Log on to SAP R/3.
2. Start transaction SE37. The Function Builder: Initial window dialog opens. Do
the following:
a. Type Z_RFC_DS_SERVICE in the Function module box.

b. Click Create to open the Create Function Module dialog. The Create
Function Module dialog opens.
3. To create a function module:
a. Type the name of the function group in which you want the RFC function
code to be created in the Function group field.
b. Optional. Type Datastage General Service Utility in the Short text field.
c. Click Save to create the module.
4. The Attributes page of the Function Builder: Change Z_RFC_DS_SERVICE
dialog opens.
Specify the following values on the Attributes page.
Use the defaults for the remaining entries:
v Application: Z
v Short text: WebSphere DataStage General Service Utility
v Processing type: Remote-enabled module
v Update module: Start immed.
If you need help determining which function group to use, see step 2 in
Troubleshooting for RFC installation for SAP R/3.
5. Click the Import page to enter the import parameters. New service types exist
in Z_RFC_DS_SERVICE so that job parameters can be passed to ABAP
programs that run in the background. For details about running these
programs in the background, see Running programs in the background.
The Import page opens.
Type the following values shown on this sample Import page for the
Parameter name, Type, Reference type, Default value, Optional, and Pass
value fields:
Default Pass
Parameter name Type Reference type value Optional value
I_SER VE_TYPE LIKE SY-TFILL Yes
I_TABLE_NAME LIKE DD02T-TABNAME Yes Yes
I_REPORT_NAME LIKE D010SINF-PROG Yes Yes
I_DELIMITER LIKE SONV-FLAG SPACE Yes Yes
I_NO_DATA LIKE SONV-FLAG SPACE Yes Yes
I_ROWSKIPS LIKE SOID-ACCNT 0 Yes Yes
I_ROWCOUNT LIKE SOID-ACCNT 0 Yes Yes
I_CURR_VARIANT LIKE RSVAR-VARIANT Yes Yes
I_VARI_DESC LIKE VARID Yes Yes
6. Click the Export page to enter the export parameters.

Type the following values shown on the sample Export page for the
Parameter name, Type spec., Reference type, and Pass val. fields:
Parameter name Type spec. Reference type Pass value

O_TABLE_SIZE LIKE SY-TFILL Yes
O_ERROR_INCLUDE LIKE SY-REPID Yes
O_ERROR_LINE LIKE SY-INDEX Yes
O_ERROR_MESSAGE LIKE RSLINLMSG-MESSAGE Yes

Parameter name Type spec. Reference type Pass value
O_ERROR_OFFSET LIKE SY-TABIX Yes
O_ERROR_SUBRC LIKE SY-SUBRC Yes
O_SAP_VER LIKE SVERS-VERSION Yes
O_DSS_VER LIKE SVERS-VERSION Yes
O_VARIANT LIKE RSVAR-VARIANT Yes
7. Click the Tables page to enter the table parameters.

Type the values shown on this sample Tables page for the Parameter name,
Type spec., Reference type, and Optional fields according to the following
table:
Parameter name Type spec. Reference type Optional

T_SQLCON LIKE RFC_DB_OPT Yes
T_PARAM LIKE RSPARAMS Yes
T_CODE LIKE PROGTAB Yes
T_FIELDS LIKE RFC_DB_FLD Yes
T_DATA LIKE TAB512 Yes
T_VARI_TEXT LIKE VARIT Yes
8. Click the Exceptions page to enter the exceptions parameters.

The Exceptions page opens.
Type the following values shown on this Exceptions page for the Exception
field:
Exception
TABLE_NOT_FOUND
WRONG_TYPE
TABLE_WITHOUT_DATA
OPTION_NOT_VALID
FIELD_NOT_VALID
DATA_BUFFER_EXCEEDED
REPORT_NOT_FOUND
9. Save the RFC interface definition.

10. Click Source code. The source code appears in the ABAP Editor window:
To replace this default code with the source code provided on the CD, that is,
the ZDSRFC.TXT file in the RFC directory:
a. Choose Utilities > more utilities.
b. Choose Upload/Download > Upload from the ABAP Editor menu.
c. Specify the path name for the ZDSRFC.TXT file in the RFC directory on
the CD. For example, type E:\Datastage\RFC\ZDSRFC.TXT in the File name
field.
11. After the upload is complete, save the RFC function code, and activate it.

Configuring the SAP/Dispatch/Gateway service
To use the WebSphere DataStage Pack for SAP R/3 correctly, you must add entries
for the SAP Dispatch/Gateway service to the services file for your DataStage client
and server systems. If SAP R/3 is already configured on the WebSphere DataStage
client and server systems, these entries may already be added.
The location of the services file depends on your platform:
Windows NT: \winnt\system32\drivers\etc\services
Unix: /etc/services
Add the following entries to the services file:
# SAP Port
sapdp00 3200/tcp
sapdp01 3201/tcp
sapdp02 3202/tcp
sapdp03 3203/tcp
sapdp04 3204/tcp
sapdp05 3205/tcp
sapdp06 3206/tcp
sapdp07 3207/tcp
sapdp08 3208/tcp
sapdp09 3209/tcp
sapdp10 3210/tcp
sapdp11 3211/tcp
sapdp12 3212/tcp
sapdp13 3213/tcp
sapdp14 3214/tcp
sapdp15 3215/tcp
sapdp16 3216/tcp
sapdp17 3217/tcp
sapdp18 3218/tcp
sapdp19 3219/tcp
sapdp20 3220/tcp
sapdp21 3221/tcp
sapdp22 3222/tcp
sapdp23 3223/tcp
sapdp24 3224/tcp
sapdp25 3225/tcp
sapdp26 3226/tcp
sapdp27 3227/tcp
sapdp28 3228/tcp
sapdp29 3229/tcp
sapdp30 3230/tcp
sapdp31 3231/tcp
sapdp32 3232/tcp
sapdp33 3233/tcp
sapdp34 3234/tcp

sapdp35 3235/tcp
sapdp36 3236/tcp
sapdp37 3237/tcp
sapdp38 3238/tcp
sapdp39 3239/tcp
sapdp40 3240/tcp
sapdp41 3241/tcp
sapdp42 3242/tcp
sapdp43 3243/tcp
sapdp44 3244/tcp
sapdp45 3245/tcp
sapdp46 3246/tcp
sapdp47 3247/tcp
sapdp48 3248/tcp
sapdp49 3249/tcp
sapdp50 3250/tcp
sapdp51 3251/tcp
sapdp52 3252/tcp
sapdp53 3253/tcp
sapdp54 3254/tcp
sapdp55 3255/tcp
sapdp56 3256/tcp
sapdp57 3257/tcp
sapdp58 3258/tcp
sapdp59 3259/tcp
sapdp60 3260/tcp
sapdp61 3261/tcp
sapdp62 3262/tcp
sapdp63 3263/tcp
sapdp64 3264/tcp
sapdp65 3265/tcp
sapdp66 3266/tcp
sapdp67 3267/tcp
sapdp68 3268/tcp
sapdp69 3269/tcp
sapdp70 3270/tcp
sapdp71 3271/tcp
sapdp72 3272/tcp
sapdp73 3273/tcp
sapdp74 3274/tcp
sapdp75 3275/tcp
sapdp76 3276/tcp
sapdp77 3277/tcp
sapdp78 3278/tcp
sapdp79 3279/tcp
sapdp80 3280/tcp
sapdp81 3281/tcp
sapdp82 3282/tcp
sapdp83 3283/tcp
sapdp84 3284/tcp
sapdp85 3285/tcp
sapdp86 3286/tcp
sapdp87 3287/tcp
sapdp88 3288/tcp
sapdp89 3289/tcp
sapdp90 3290/tcp

sapdp91 3291/tcp
sapdp92 3292/tcp
sapdp93 3293/tcp
sapdp94 3294/tcp
sapdp95 3295/tcp
sapdp96 3296/tcp
sapdp97 3297/tcp
sapdp98 3298/tcp
sapdp99 3299/tcp
sapgw00 3300/tcp
sapgw01 3301/tcp
sapgw02 3302/tcp
sapgw03 3303/tcp
sapgw04 3304/tcp
sapgw05 3305/tcp
sapgw06 3306/tcp
sapgw07 3307/tcp
sapgw08 3308/tcp
sapgw09 3309/tcp
sapgw10 3310/tcp
sapgw11 3311/tcp
sapgw12 3312/tcp
sapgw13 3313/tcp
sapgw14 3314/tcp
sapgw15 3315/tcp
sapgw16 3316/tcp
sapgw17 3317/tcp
sapgw18 3318/tcp
sapgw19 3319/tcp
sapgw20 3320/tcp
sapgw21 3321/tcp
sapgw22 3322/tcp
sapgw23 3323/tcp
sapgw24 3324/tcp
sapgw25 3325/tcp
sapgw26 3326/tcp
sapgw27 3327/tcp
sapgw28 3328/tcp
sapgw29 3329/tcp
sapgw30 3330/tcp
sapgw31 3331/tcp
sapgw32 3332/tcp
sapgw33 3333/tcp
sapgw34 3334/tcp
sapgw35 3335/tcp
sapgw36 3336/tcp
sapgw37 3337/tcp
sapgw38 3338/tcp
sapgw39 3339/tcp
sapgw40 3340/tcp
sapgw41 3341/tcp
sapgw42 3342/tcp
sapgw43 3343/tcp
sapgw44 3344/tcp
sapgw45 3345/tcp
sapgw46 3346/tcp

sapgw47 3347/tcp
sapgw48 3348/tcp
sapgw49 3349/tcp
sapgw50 3350/tcp
sapgw51 3351/tcp
sapgw52 3352/tcp
sapgw53 3353/tcp
sapgw54 3354/tcp
sapgw55 3355/tcp
sapgw56 3356/tcp
sapgw57 3357/tcp
sapgw58 3358/tcp
sapgw59 3359/tcp
sapgw60 3360/tcp
sapgw61 3361/tcp
sapgw62 3362/tcp
sapgw63 3363/tcp
sapgw64 3364/tcp
sapgw65 3365/tcp
sapgw66 3366/tcp
sapgw67 3367/tcp
sapgw68 3368/tcp
sapgw69 3369/tcp
sapgw70 3370/tcp
sapgw71 3371/tcp
sapgw72 3372/tcp
sapgw73 3373/tcp
sapgw74 3374/tcp
sapgw75 3375/tcp
sapgw76 3376/tcp
sapgw77 3377/tcp
sapgw78 3378/tcp
sapgw79 3379/tcp
sapgw80 3380/tcp
sapgw81 3381/tcp
sapgw82 3382/tcp
sapgw83 3383/tcp
sapgw84 3384/tcp
sapgw85 3385/tcp
sapgw86 3386/tcp
sapgw87 3387/tcp
sapgw88 3388/tcp
sapgw89 3389/tcp
sapgw90 3390/tcp
sapgw91 3391/tcp
sapgw92 3392/tcp
sapgw93 3393/tcp
sapgw94 3394/tcp
sapgw95 3395/tcp
sapgw96 3396/tcp
sapgw97 3397/tcp
sapgw98 3398/tcp
sapgw99 3399/tcp

Troubleshooting for RFC installation for SAP R/3
This troubleshooting information applies to all versions of SAP R/3. If the RFC
utility does not work, do the following:
1. Verify that the settings are correct on the Attributes page for transaction SE37.
2. If you do not know which function group to use, start transaction SE84 to see a
list.
a. Click the Programming > Function Builder, then double-click Function
groups. The R/3 Repository Information System: Function groups dialog
opens, where you specify the selection criteria.
b. Type z* in the Func. group name field, and run it to display a list of
available function groups. The Function groups dialog opens.
c. If you are not sure which group to use, start a new transaction SE80. The
Object Navigator dialog opens.
d. Several functions are displayed under the function modules branch. Display
the different function groups (as listed in SE84 in step 2) until you find the
one you want. The Import page appears.
3. Verify that the parameters are correct for the Import page according to the
following guidelines:
a. The LIKE entry for Type is case-sensitive.
b. The Optional value must be clear for the I_SERV_TYPE parameter. The
other parameters must have Optional selected.
c. The RFC function cannot pass a value by reference. Pass value must be
selected.
4. Verify that the parameters for the Export, Tables, and Exceptions tabs are
correct.
5. Click Source code. The function module header appears at the top of the code.
The IMPORTING and EXPORTING values must be passed by VALUE, not
REFERENCE. Instead of uploading the code into the Editor, do the following:
a. Copy the code in the ZDSRFC.TXT file, from TABLES through the last
ENDIF, to the clipboard.
b. Paste the code to the Editor in the function module.
c. Check that the syntax for this function procedure displays No syntax error
found.
6. Activate Z_RFC_DS_SERVICE. Do this by selecting Function module >
Activate (or Ctrl+F3) to activate the code so that it can function properly.
7. As part of the installation process, test that the function works in your
environment by testing the version number or the table content:
SAP and RFC code version number. Do the following:
a. Start Transaction SE37, and type Z_RFC_DS_SERVICE.
b. Press F8. The Test Function Module: Initial window opens.
c. Click Execute. The Test Function Module: Result window appears. The
SAP version number (for example, 46B) and the RFC utility version number
are displayed.
Table content. Do the following:
d. Start Transaction SE37, and type Z_RFC_DS_SERVICE in the Function module
field.
e. Press F8. The Test Function Module: Initial window opens.
f. Type 5 in the I_SERVE_TYPE field.

g. Type T000 for I_TABLE_NAME.
h. Click Execute. The Test Function Module: Result window opens. If
T_FIELDS and T_DATA contain records, the RFC utility works properly.
Packaged extractions
The ABAP Extract for SAP R/3 stage includes packaged extraction jobs that help
you understand common extraction scenarios.
These packaged extractions are provided as an IBM WebSphere DataStage export

file (Extracts.dsx). This file is located on the client CD in the templates directory.
Use WebSphere DataStage import function to import any or all of the jobs
contained in the file.
You can customize these jobs for your ETL initiatives by providing parameters
specific to your environment and by using the extraction object and corresponding
ABAP that already exist in the job. The ABAP Extract stage also includes jobs that
extract long text fields from R/3. The following list describes these jobs and the
SAP tables from which they extract data:
v LongTextCPIC. This job uses CPIC data transfer method to extract long text
from VarChar fields. It decompresses them using the READ_TEXT function
module.
Note: The Extraction Object and the corresponding ABAP program cannot be
regenerated since it overwrites the code to call the function module.
v LongTextFTP. This job uses the FTP data transfer method to extract long text
module.
regenerated as this results in overwriting the code to call the function module. If
Path of Remote File in the Output >Runtime tab is changed, you must
manually search for the old path in the ABAP program and replace it with the
new one.
v MARAextract. Extracts a range of materials from MARA and their
corresponding descriptions from the MAKT table.
v MaterialExtract. Extracts data from the primary material tables, namely, MARA,
MARM, MBEW, MVKE, MARC, MARD, MCHB, MKOL, MLGN, and MLGT.
v SalesExtract. Extracts data from the VBAK and VBAP tables.
v SalesCustomerMasterExtract. Extracts data from the KNA1, KNB1, and KNVV
tables.

Chapter 8. Packaged extractions for ABAP
The IBM WebSphere DataStage ABAP Extract for SAP R/3 stage includes packaged
extraction jobs that help you understand common extraction scenarios.
These packaged extractions are provided as a WebSphere DataStage export file

(Extracts.dsx). This file is located on the pack client CD in the templates directory.
Use the WebSphere DataStage import function to import any or all of the jobs
contained in the file.
You can customize these jobs for your ETL initiatives by providing parameters
specific to your environment and by using the extraction object and corresponding
ABAP that already exist in the job. The ABAP Extract stage also includes jobs that
extract long text fields from R/3. The following list describes these jobs and the
SAP tables from which they extract data:
v LongTextCPIC. This job uses CPIC data transfer method to extract long text
module.
regenerated since it overwrites the code to call the function module.
v LongTextFTP. This job uses the FTP data transfer method to extract long text
module.
regenerated as this results in overwriting the code to call the function module. If
Path of Remote File in the Output >Runtime tab is changed, you must
manually search for the old path in the ABAP program and replace it with the
new one.
v MARAextract. Extracts a range of materials from MARA and their
corresponding descriptions from the MAKT table.
v MaterialExtract. Extracts data from the primary material tables, namely, MARA,
MARM, MBEW, MVKE, MARC, MARD, MCHB, MKOL, MLGN, and MLGT.
v SalesExtract. Extracts data from the VBAK and VBAP tables.
v SalesCustomerMasterExtract. Extracts data from the KNA1, KNB1, and KNVV
tables.

Chapter 9. File permissions for IDoc Extract stage
This section documents permissions on files that are created by the IDoc Extract
Pack. For more information on umask, see the UNIX documentation for your
system.
Configuring connections and IDoc types

When a file is created on UNIX systems, it is given a set of default permissions.
The permissions that are assigned are determined by the file creation mask of the
process that creates the file. The umask changes the file creation mask by removing
permissions from the existing mask. It consists of three groups of three bits each:
v The first group relates to permissions for the user who owns the file
v The second group relates to permissions for the group that owns the file
v The third group relates to everyone else
When a file is created, ownership is set to the user id of the process that created
the file. Group ownership is set to the primary group of the user who created the
file. Its permissions are then determined by its file creation mode minus the value
of its umask. Child processes inherit the umask value and user id of a process.
For example, if the default file creation mode of a file is -rw-rw-rw- (666), and the
umask for the process is 002, the file created has permissions of -rw-rw-r-- (664).
This means the user and the user’s group have read and write permissions but
everyone else only has read permissions.
Files created by the IDoc Extract Pack are subject to two sets of permissions,
depending on how they are created:
v The first set belongs to those files that are created by child processes of the
dsidocmgr executable, such as the dsIDocsvr executable.
v The second set belongs to those that are created by IBM WebSphere DataStage
client connections, such as the IDoc stage editor and the WebSphere DataStage
Administrator for SAP.
These files inherit the user ID and group ID of the user logged into the
WebSphere DataStage client and the umask setting of the WebSphere DataStage
server process owner. The umask is not that of the user logged in to the client.
When the IDoc Extract stage is installed, it creates a directory named

DSSAPConnections in the directory where the server is installed. This directory is
owned by root, and the permissions are set so that everyone has permissions to
read and write, regardless of the umask for the process.
A file called DSSAPConnections.config is created in the DSSAPConnections directory

when a user logs in using the client and has permissions subject to the umask with
which the server is started.
Several other files with the .config suffix reside in this directory. (The client creates
these for internal purposes.)
v The IDoc.SegmentAdminFields.3.config and IDoc.SegmentAdminFields.2.config files
are created the first time a user accesses metadata for an IDoc from a particular
ALE port version.

v The IDocCleanup.config file is created the first time a user accesses the IDoc
Cleanup and Archiving page in the DataStage Administrator for SAP.
v The SAPVersions.config file is created at the time the first connection to SAP is
created.
The user who first logs into WebSphere DataStage to perform these activities owns
these files, which are subject to the umask setting that the server was started with.
When a connection to SAP is created, a directory with the same name given to the
connection is created containing a subdirectory called IDocTypes. These directories
are owned by the user who creates the connection, with permissions subject to the
umask setting that the server was started with.
Note: Users who want to create a connection to SAP either using the stage editor
or the WebSphere DataStage Administrator for SAP must have write permissions
to the DSSAPConnections.config file and the DSSAPConnections directory.
When the first IDoc type is configured, a file called IDocTypes.config is created in
the IDocTypes directory, so the user who configures this first IDoc type must have
write permissions to the IDocTypes directory. Remember that this directory was
created by the user who defined the connection and may be different than the user
defining the first IDoc type.
When a user configures subsequent IDoc types, that user must have write
permissions to the IDocTypes.config file and permissions to create the directory
location (PSA) to be configured for that type. This directory can be configured
anywhere on the file system, but is created by default in the IDocTypes directory.
In the configured PSA, another file with the .config suffix is created that contains
parameters for this IDoc type. The user who logs into WebSphere DataStage to
configure this IDoc type owns these files, which are subject to the umask setting
that the WebSphere DataStage server was started with.
Any user who later modifies the configuration for an IDoc type must have write
permissions to:
v The IDocTypes.config file
v The directory the IDoc type is to be written to
v The IDoc type config file in that directory
If the modification is to specify a new directory location, the user must have
permission to write to the new directory location.
Running WebSphere DataStage jobs

An IBM WebSphere DataStage job uses the user ID of the user logged in to
WebSphere DataStage to run it, no matter how the job is started. If the RFC Server
runs the job, it runs with the WebSphere DataStage user ID configured for that
connection and type, not the user that owns the RFC Server process (the RFC
Server runs as root).
The first user who runs a job that is not in test mode, which reads IDocs of a
specified type from a specified directory location, must have permission to create a
bookmark file. The bookmark file is created in the directory from which the IDocs
for that type are read. This file is named for the IDoc type with the .bmk suffix and

is owned by the user who creates the file. It has permissions subject to the umask
setting with which the WebSphere DataStage server was started.
In addition, a file named saprfc.ini exists in all project directories after the first
WebSphere DataStage job is run. This is the case with any WebSphere DataStage
Pack for SAP R/3, such as the ABAP Extract and the Load Pack for SAP BW. The
user who runs the job owns the saprfc.ini file, with permissions subject to the
umask setting with which the WebSphere DataStage server is started. Each
subsequent job run removes and recreates this file, so all the users who want to
run jobs in this project must be able to write to this file. For example, if user A
runs a Load Pack for SAP BW job, and user B subsequently runs an IDoc Extract
job, the job fails unless user B has permission to remove and recreate this file.
Unconfigured IDoc types

When the RFC Server receives an IDoc that it does not recognize, it creates a
directory in the default location for that IDoc type. Since the RFC Server runs as
root, it can override the umask setting. It does this by creating this directory with
read, write, and execute permissions for user, group, and other so that another user
can later configure that location. All IDoc files that the RFC Server receives and
writes to the PSA are owned by root and subject to the umask settings with which
the RFC Server Manager is started.
Configuring IDoc cleanup and archiving

The cleanup scheduling mechanism uses the crontab command. Consequently, user
permissions to modify the cleanup schedule are subject to that user’s permissions
to run the crontab command.
Since you cannot use the crontab command to manipulate the crontab files of other
users, the information in the IDOC Cleanup and Archiving page of the IBM
WebSphere DataStage Administrator for SAP only applies to the user logging in to
the tool.
For example, if user A schedules a cleanup for 12:00 every Saturday, and user B
wants to change the cleanup schedule, user B can add this entry, but it will not
remove the entry for user A. Likewise, when user B logs into the tool, the cleanup
settings do not reflect the settings user A established. WebSphere DataStage has the
same limitation when you log in to the WebSphere DataStage Director Client.
Recommendations
v The umask setting should apply a consistent set of permissions to all users who
log in to IBM WebSphere DataStage for the purposes of administering the IDoc
Extract functionality. In addition, the umask setting should be consistent for all
users running WebSphere DataStage jobs for all SAP R/3 products.
For example, if only one user will ever administer WebSphere DataStage
connections to SAP or run WebSphere DataStage jobs using SAP R/3 stages, the
umask should be 022. If only users from the same group will perform these
functions, the umask should be 002. If you want users to administer WebSphere
DataStage connections to SAP or run WebSphere DataStage jobs using SAP R/3
stages regardless of user id or group, the umask must be explicitly set to 000.
v For consistent permissions, the dsidocmgr executable should be invoked with
the same umask as the WebSphere DataStage server. The dsidocd.rc script used to
start the dsidocmgr references the dsenv file containing the environment settings
for the WebSphere DataStage server. Therefore, the dsidocmgr executable should
use an explicit umask contained in this file.
Chapter 9. File permissions for IDoc Extract stage 99

v Only one user should change the cleanup settings on the system. This lets the
WebSphere DataStage Administrator for SAP reflect the state of the system
accurately.
Accessing information about the product
IBM has several methods for you to learn about products and services.
You can find the latest information on the Web:

www.ibm.com/software/data/integration/info_server/
To access product documentation, go to publib.boulder.ibm.com/infocenter/

iisinfsv/v8r0/index.jsp.
You can order IBM publications online or through your local IBM representative.
v To order publications online, go to the IBM Publications Center at
www.ibm.com/shop/publications/order.
v To order publications by telephone in the United States, call 1-800-879-2755.
To find your local IBM representative, go to the IBM Directory of Worldwide

Contacts at www.ibm.com/planetwide.
Providing comments on the documentation

Please send any comments that you have about this information or other
documentation.
Your feedback helps IBM to provide quality information. You can use any of the
following methods to provide comments:
v Send your comments using the online readers’ comment form at
www.ibm.com/software/awdtools/rcf/.
v Send your comments by e-mail to comments@us.ibm.com. Include the name of
the product, the version number of the product, and the name and part number
of the information (if applicable). If you are commenting on specific text, please
include the location of the text (for example, a title, a table number, or a page
number).

Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in
other countries. Consult your local IBM representative for information on the
products and services currently available in your area. Any reference to an IBM
product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product,
program, or service that does not infringe any IBM intellectual property right may
be used instead. However, it is the user’s responsibility to evaluate and verify the
operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does not grant you
any license to these patents. You can send license inquiries, in writing, to:
IBM Director of Licensing

IBM Corporation
North Castle Drive
Armonk, NY 10504-1785 U.S.A.
For license inquiries regarding double-byte (DBCS) information, contact the IBM
Intellectual Property Department in your country or send inquiries, in writing, to:
IBM World Trade Asia Corporation

Licensing 2-31 Roppongi 3-chome, Minato-ku
Tokyo 106-0032, Japan
The following paragraph does not apply to the United Kingdom or any other
country where such provisions are inconsistent with local law:
INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS
PUBLICATION ″AS IS″ WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or
implied warranties in certain transactions, therefore, this statement may not apply
to you.
This information could include technical inaccuracies or typographical errors.

Changes are periodically made to the information herein; these changes will be
incorporated in new editions of the publication. IBM may make improvements
and/or changes in the product(s) and/or the program(s) described in this
publication at any time without notice.
Any references in this information to non-IBM Web sites are provided for
convenience only and do not in any manner serve as an endorsement of those Web
sites. The materials at those Web sites are not part of the materials for this IBM
product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it
believes appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purpose
of enabling: (i) the exchange of information between independently created
programs and other programs (including this one) and (ii) the mutual use of the
information which has been exchanged, should contact:
IBM Corporation
J46A/G4
555 Bailey Avenue
San Jose, CA 95141-1003 U.S.A.
Such information may be available, subject to appropriate terms and conditions,

including in some cases, payment of a fee.
The licensed program described in this document and all licensed material
available for it are provided by IBM under terms of the IBM Customer Agreement,
IBM International Program License Agreement or any equivalent agreement
between us.
Any performance data contained herein was determined in a controlled

environment. Therefore, the results obtained in other operating environments may
vary significantly. Some measurements may have been made on development-level
systems and there is no guarantee that these measurements will be the same on
generally available systems. Furthermore, some measurements may have been
estimated through extrapolation. Actual results may vary. Users of this document
should verify the applicable data for their specific environment.
Information concerning non-IBM products was obtained from the suppliers of

those products, their published announcements or other publicly available sources.
IBM has not tested those products and cannot confirm the accuracy of
performance, compatibility or any other claims related to non-IBM products.
Questions on the capabilities of non-IBM products should be addressed to the
suppliers of those products.
All statements regarding IBM’s future direction or intent are subject to change or
withdrawal without notice, and represent goals and objectives only.
This information is for planning purposes only. The information herein is subject to
change before the products described become available.
This information contains examples of data and reports used in daily business
operations. To illustrate them as completely as possible, the examples include the
names of individuals, companies, brands, and products. All of these names are
fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which

illustrate programming techniques on various operating platforms. You may copy,
modify, and distribute these sample programs in any form without payment to
IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating
platform for which the sample programs are written. These examples have not
been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or
imply reliability, serviceability, or function of these programs.
Each copy or any portion of these sample programs or any derivative work, must
include a copyright notice as follows:
© (your company name) (year). Portions of this code are derived from IBM Corp.
Sample Programs. © Copyright IBM Corp. _enter the year or years_. All rights
reserved.
If you are viewing this information softcopy, the photographs and color
illustrations may not appear.
Trademarks
IBM trademarks and certain non-IBM trademarks are marked at their first
occurrence in this document.
See www.ibm.com/legal/copytrade.shtml for information about IBM trademarks.
The following terms are trademarks or registered trademarks of other companies:
Adobe®, the Adobe logo, PostScript®, the PostScript logo are either registered
trademarks or trademarks of Adobe Systems Incorporated in the United States,
and/or other countries.
Cell Broadband Engine™ is a trademark of Sony Computer Entertainment, Inc. in

the United States, other countries, or both and is used under license therefrom.
Intel®, Intel logo, Intel Inside® logo, Intel Centrino®, Intel Centrino logo, Celeron®,
Intel Xeon®, Intel SpeedStep®, Itanium and Pentium® are trademarks of Intel
Corporation in the United States, other countries, or both.
Java™ and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in
the United States, other countries, or both.
Microsoft, Windows, Windows NT and the Windows logo are trademarks of

Microsoft Corporation in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other
countries.
Linux is a registered trademark of Linus Torvalds in the United States, other

countries, or both.
ITIL® is a registered trademark and a registered community trademark of the

Office of Government Commerce, and is registered in the U.S. Patent and
Trademark Office.
IT Infrastructure Library® is a registered trademark of the Central Computer and

Telecommunications Agency which is now part of the Office of Government
Commerce.
Other company, product or service names may be trademarks or service marks of

others.
Notices 105
Index
Special characters batch size
IDoc Extract stage 36
D
.bmk file suffix 98 bookmark files 39 data transfer methods
bookmark files for IDoc Extract ABAP Extract 13
PSA 43 DataStage Administrator for SAP 81
A BOR browsing DataStage Connections to SAP page
DataStage Administrator for SAP 75
ABAP definition 2 BAPI plug-in 68
BOR definition 2 DataStage Job Options for IDocs page
ABAP Editor 16
Build Extraction Object DataStage Administrator for SAP 79
ABAP Extract 15
program generation, ABAP defining
ABAP Extract plug-in
extract 16 ABAP Extract Stage properties 7
functionality 3
Build Extraction Object dialog 16 IDoc Extract stage 45
overview 3
Build SQL query properties for IDoc types for IDoc
ABAP Extract programs
program generation, ABAP Extract 48
RFC data transfer 8
Extract 15 defining ABAP Extract Stage
running in background 15
Build SQL Query dialog 16 properties 7
ABAP Extract Stage dialog 7
building defining BAPIs
ABAP Program Development Status,
extraction objects or SQL queries, BAPI plug-in 68
ABAP Extract 16
ABAP Extract 16 defining character set mapping
ABAP Program page 15
SQL queries, ABAP Extract 16 IDoc Load plug-in 58
ABAP RFC extraction job
Business Object definition 2 defining character set maps
troubleshooting 10
ABAP Extract plug-in 7
accessibility 101
BAPI plug-in 65
activating BAPI parameters
defining IDoc Load input data 59
BAPI plug-in 69 C defining IDoc Load stage 55
Add Columns dialog 52 character set maps defining IDoc type configuration
ADM_ prefixes ABAP Extract plug-in 7 properties
IDoc extraction 51 cleanup scheduling IDoc Extract 47
administrative fields crontab command 99 defining input properties
IDoc extraction 51 cofiles and data files BAPI plug-in
Administrator for SAP utility 75 copying, for ABAP Extract 5 BAPI Input General page 67
architecture columns defining log files 69
BAPI plug-in 64 adding for IDoc extraction 52 defining output properties
IDoc Extract plug-in 32 columns and corresponding segment ABAP Extract 7
archiving IDocs 39 types defining SAP connections
archiving processed IDocs IDoc Load plug-in 61 BAPI plug-in 66
IDoc Extract 48 configure IDoc types defining stage properties
assigned input links 58 IDoc Load plug-in 57 BAPI plug-in 65
authorization profile, ABAP Extract 4 configuring global properties 75 design time connection, IDoc Extract
automatic job mode, IDoc extraction 49 connecting to SAP R/3 systems from stage 46
DataStage destination, RFC 8
IDoc Extract plug-in 45 documentation
B Connection Properties dialog accessible 101
background processing, ABAP accessing from ABAP Extract GUI 12 dsenv file
Extract 15 BAPI plug-in 67 IDoc Extract plug-in 39
BAPI definition 2 connections dsidocd.rc script
BAPI Explorer dialog managing for IDoc Extract 46 permissions 99
BAPI plug-in 68 selecting for ABAP Extract 11 RFD Listener Manager 35
BAPI input file for import connections to SAP
parameters 72 ABAP Extract 11
BAPI Input page
BAPI plug-in 68
Control record definition 2
control records
E
IDoc extractions 49 E2MAKTM
BAPI plug-in 69, 70
CPI-C data transfer method, ABAP example of translation to DataStage
BAPI plug-in runtime component 73
Extract 13 columns 43
BAPI Stage dialog 65
creating programs manually, ABAP Enter program as text
BAPI Stage General page 66
Extract 16 program generation, ABAP
BAPISeqNo field
creating tables 18 Extract 16
BAPI plug-in 70
crontab command 99 ERP definition 2
batch count
errors
WebSphere DataStage limit for
IDoc extraction 50
IDocs 36

example
IDoc Load assigned input links 61
IDoc Type page
IDoc Extract plug-in 47
M
metadata translation to DataStage IDoc Type Properties dialog managing connections for IDoc
columns 43 IDoc Extract plug-in 47, 48 Extract 46
relationships between DataStage jobs running specific job 49 manual job batch size, IDoc Extract
and IDoc types 38 IDoc Types dialog 81 stage 36
RFC listener servers 38 DataStage Administrator for SAP 76 manual job execution
Export Connection dialog IDoc types for IDoc Extract IDoc extraction 36
DataStage Administrator for SAP 81 finding substrings 47 manual job mode, IDoc extraction 49
Export log files IDOC_EXTRACT_LIST 37 MATMAS01 definition 2
BAPI plug-in 73 Import IDoc Type Configurations dialog methods
Export page DataStage Administrator for SAP 80 generating ABAP Extract
Input BAPI page 69 Import page programs 15
BAPI plug-in 69 modifying columns
Importing at the operating system level, IDoc Load plug-in 60
F ABAP Extract 6
importing into existing connections
F-keys 60
filtering Business Objects
DataStage Administrator for SAP 80 N
importing the RFC and authorization NLS page
BAPI plug-in 68
profile, ABAP Extract 4 ABAP Extract plug-in 7
Find IDoc Type dialog 81
Input BAPI Export page BAPI plug-in 65
BAPI plug-in 69 IDoc Load plug-in 58
format error
Input BAPI Import page 69
IDoc extraction 50
Input BAPI Logs page
FTP data transfer method, ABAP
Extract 13
BAPI plug-in 69
input file for import parameters,
O
functionality opening, ABAP Extract 16
BAPI 72
ABAP Extract plug-in 3 Output General page
installation prerequisites
BAPI plug-in 63 ABAP Extract plug-in 10
ABAP Extract plug-in 4
IDoc Extract plug-in 31 IDoc extraction
iRFC function module, ABAP Extract 4
IDoc Load plug-in 53 IDoc Extract Output General
page 50
Output page
G J ABAP Extract plug-in 7
job mode overview
generation methods, ABAP Extract
IDoc Extract stage 49 ABAP Extract plug-in 3
programs 15
I K
IDoc control records 49
Key Columns for Link dialog P
IDoc Load plug-in 61 Pack definition 2
IDoc definition 2
IDoc Extract plug-in parent and child segment types
overview 31 IDoc Load plug-in 61
IDoc Extract stage L partner number
connection, design time 46 legal notices 103 IDoc Load plug-in 57
IDoc extraction listener log files permissions
by IDoc number 37 IDoc Extract plug-in 34 IDoc Extract bookmark files 44
IDoc number, extracting by 37 listener servers permissions on bookmark file 44
IDoc extraction batch size, manual IDoc Extract plug-in 34 Port Version list
job 36 listener subsystem DataStage Administrator for SAP 79
IDoc extraction components IDoc Extract plug-in 34 port versions
selecting 51 load balancing IDoc extraction 50
IDoc extraction job BAPI plug-in 67 preventing updates in bookmark files 43
automatically run all jobs 49 DataStage Administrator for SAP 77 processing IDocs using DataStage
automatically run one job 49 load methods, ABAP Extract 16 summary 31
running manually 49 loading program generation
IDoc Listener Settings page ABAP programs, ABAP Extract 16 ABAP Extract 15
DataStage Administrator for SAP 78 local storage for IDocs 37 Program Generation Options dialog 16
IDoc Load Columns page 60 log files directory programs
IDoc Load Input General page 59 BAPI plug-in 70 running in background, ABAP
IDoc Load plug-in 58, 60 logs Extract 15
overview 53 IDoc listener 81 PSA
IDoc Log dialog bookmark files for IDoc Extract 43
DataStage Administrator for SAP 81 PSA definition 2
IDoc segment definitions PSA property 37
saving for IDoc extraction 52
IDoc type definition 2
Q selecting segment definitions
IDoc Load plug-in 59
viewing Tables parameters
BAPI plug-in 69
qRFC data transfer, ABAP program 9 specifying
ABAP program properties, ABAP
R
Extract 15
Stage General page
W
WebSphere DataStage Administrator for
R/3 source systems configuration IDoc Extract plug-in 45
SAP utility 75
IDoc Extract plug-in 33 IDoc Load plug-in 55
restore bookmark files to original Stage IDoc Type page
state 43 IDoc Extract plug-in 47
return parameters 70 starting and stopping RFC Listener Z
RFC data transfer Manager on UNIX 35 Z_DS-PROFILE profile, ABAP Extract 6
ABAP Extract job 8 status, program development Z_RFC_DS_SERVICE RFC function
RFC data transfer method, ABAP ABAP Extract 16 module, ABAP Extract 6
program 9 summary ZDSRFC.TXT file, ABAP Extract 4
RFC definition 2 output links for IDoc extraction 51 ZETL development class, ABAP
RFC destination, creating 8 processing IDocs using DataStage 31 Extract 6
RFC, importing for ABAP Extract 4 summary of IDoc Extract stage and
root segments links 44
IDoc Load plug-in 61 synchronizing columns
run methods, ABAP Extract 16 IDoc Load plug-in 61
running background processes, ABAP
Extract 15
T
tables
S creating 18
SAP XI support, design time Tables log files
connection 46 BAPI plug-in 73
saprfc.ini file 44, 99 temporary IDoc files directory for IDoc
IDoc Extract plug-in 44 extraction
permissions 99 path name for temporary IDoc files
Save IDoc Segment Definition dialog 52 for IDoc extraction 48
IDoc Load plug-in 60 temporary segment index files
scheduling jobs IDoc Load plug-in 57
IDoc Extract 49 Terminology 1
screen readers 101 Test Mode
Segment definition 2 IDoc Extract Stage General page 45
segment definition names processing IDocs 43
IDoc extraction 49 testing load balancing 78
segment type definition 2 tp import, ABAP Extract 5
Select Application Component ID dialog trademarks 105
BAPI plug-in 68 Transaction SE80, ABAP Extract 6
Select DataStage Connection to SAP Transaction STMS, ABAP Extract 4
dialog transferring data
ABAP Extract plug-in 11 methods for ABAP Extract 13
BAPI plug-in 66 transport control program, ABAP
IDoc Extract plug-in 46 Extract 5
Select IDoc Component to Extract Transport Management System, ABAP
dialog 51 Extract 4
Select IDoc Component to Load dialog tRFC data transfer, ABAP program 9
IDoc Load plug-in 59 tRFC port definition 2
Select IDoc Type dialog tRFC ports 34
IDoc Load plug-in 57, 58
selecting V
ABAP Extract data transfer validating
methods 13 ABAP Extract stage 11
selecting a BAPI validating columns 60
BAPI plug-in 68 Validation Stage dialog 11
selecting DataStage connection to SAP Variant definition 2
IDoc Load plug-in 55 versions of R/3
selecting IDoc types DataStage Administrator for SAP 79
IDoc Load plug-in viewing Export parameters
IDoc Load Stage IDoc Type BAPI plug-in 69
page 57 viewing Import parameters
selecting IDoc Types BAPI plug-in 69
Index 109

Printed in USA
SC18-9954-01

PackforSAP 6

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PackforSAP 6

Uploaded by

Copyright:

Available Formats

WebSphere DataStage Pack for SAP R/3

Integration Guide for WebSphere DataStage Pack for SAP R/3

Integration Guide for WebSphere DataStage Pack for SAP R/3

© Ascential Software Corporation 2004, 2005.

© Copyright IBM Corp. 2006, 2007 iii

iv Integration Guide for WebSphere DataStage Pack for SAP R/3

© Copyright IBM Corp. 2006, 2007 1

2 Integration Guide for WebSphere DataStage Pack for SAP R/3

It describes the following for the ABAP Extract stage:

© Copyright IBM Corp. 2006, 2007 3

Integrating WebSphere DataStage with SAP R/3 enterprise systems

Importing the RFC and authorization profile

4 Integration Guide for WebSphere DataStage Pack for SAP R/3

Copying cofiles and data files

Importing transport requests

Importing using Transport Management System (using transaction STMS):

To successfully import the transport request, the R/3 Enterprise Administrator

Importing at the operating system level:

These tasks are described in detail in the following sections.

6 Integration Guide for WebSphere DataStage Pack for SAP R/3

The stage dialog has two pages:

Defining character set maps

Defining output properties

This page also has the following tabs:

Chapter 2. ABAP stage 7

Creating an ABAP extraction program with RFC data transfer

Transactional RFC (tRFC) data transfer uses asynchronous communication and

To create an ABAP extraction program with RFC data transfer:

Creating the RFC destination

To create the RFC destination:

8 Integration Guide for WebSphere DataStage Pack for SAP R/3

Configuring RFC data transfer

To configure RFC data transfer:

Configuring MPP ABAP extraction jobs by using RFC

To configure MPP on a Linux or UNIX system:

Chapter 2. ABAP stage 9

Troubleshooting ABAP extraction jobs

Output General page

10 Integration Guide for WebSphere DataStage Pack for SAP R/3

Enter the following information on the General tab:

Defining SAP connection and logon details

Chapter 2. ABAP stage 11

These administrative operations function similarly to the corresponding buttons on

Defining connection properties

To define the WebSphere DataStage connection to SAP:

12 Integration Guide for WebSphere DataStage Pack for SAP R/3

Note: Load balancing cannot be used at runtime.

Selecting the data transfer method

Chapter 2. ABAP stage 13

14 Integration Guide for WebSphere DataStage Pack for SAP R/3

To run ABAP programs in the background:

Output ABAP program page

This page has the following components:

Chapter 2. ABAP stage 15

16 Integration Guide for WebSphere DataStage Pack for SAP R/3

Building SQL queries

Chapter 2. ABAP stage 17

Build Open SQL Query Tables page

Click < to remove the selected table or join.

Click << to remove all tables and joins.

Build Open SQL Query Tables All Tables page

Build Open SQL Query Tables Search for Table page

18 Integration Guide for WebSphere DataStage Pack for SAP R/3

Select the appropriate columns from Available columns.