How To Use BusinessObjects Data Services in SAP BI staging process

Attachments:69
Added by Bhargav Gandhi, last edited by Bhargav Gandhi on Jun 09, 2009 (view change)
Data Services tools can be used into the SAP BI staging process, especially for Non-SAP data.
Following is the procedure for using Data Services features into the SAP BI staging process 1)Load the non-SAP data in the Data Services engine.
2)Perform Data Quality operations on the data in Data Services.
3)Load the data into SAP BI
Connecting SAP BI with Data Services
In order to allow the exchange of data and meta data between SAP BI and Data Services, we have to first establish a connection between both the systems.
On the SAP BI side this is achieved by creating a External Source System and in Data Services we have to create a new Datastore.
Screenshot 1:

1) Log on to the SAP BI system.

2) Start the Data Warehousing Workbench using TC RSA1 and select Modeling from the navigation area. Pick the entry Source Systems.
3) Position the cursor on the External System folder, and choose Create from the context menu as shown above.
Screenshot 2:

4) Enter Logical System Name and Source System Name as shown above and hit Continue.
Screenshot 3:

5) Data Services will start an RFC Server program and indicates to SAP BI that it is ready to receive RFC calls. To identify itself as the RFC Server representing this SAP BI
Source System
a keyword is exchanged, in the screen shot above it is "DEMO_SRC". This is the Registered Server Program, the Data Services RFC Server will register itself with at SAP.
Therefore,
provide the same Program ID that you want to use for the call of the RFC Server on Data Services side. All other settings for the Source System can remain on the default

settings.
To complete the definition of the Source System, save it.
6) Now, start the RFC Server on the Data Services side so that we can test the connection and load data from Data services to SAP BI system.

Open MS-DOS command prompt. (Start > Run > Type CMD & hit OK)

Go to the bin directory/folder for BusinessObjects Data Services. E.g . - C:\Program Files\Business Objects\BusinessObjects Data Services\bin>

all the program "rfcsvr -aPROGRAM_ID -gSAP_ROUTER_STRING -xSAP_GATEWAY". The PROGRAM_ID is the keyword we used above, "DEMO_SRC" in our
example.
The Router-String and SAP Gateway is something the BW admins should know. As a guideline, you can open the SAP Logon Pad and try the settings there.
Screenshot 4:

The server here is 172.17.10.162 and the System Number is 00. The router string could hence be "/H/172.17.10.162/S/3300". The /H/ stands for the hostname,
the /S/ for the port number. The port number can be derived from the system number, system 00 means port 3300. Same thing with the SAP Gateway. In a standard

standalone installation the value for this is sapgw00.

Type rfcsvr -aDEMO_SRC -g/H/172.17.10.162/S/3300 -xsapgw00 and hit Enter. It should not return anything
Screenshot 5:

7) To check the connection, click on Connection Test as shown below -

Screenshot 6:

8) The result would be as shown below in case of a success.

Screenshot 7:

9) The newly entered 3rd party system would appear in the list of External Systems as shown below.
Screenshot 8:

PoC - Loading unstructured data from Flat files into SAP BI using BusinessObjects Data Services
Let us take a sample tab delimited flat file as shown below. (Note - Sample data is placed in an excel file only for understanding purpose. In actual PoC we will be using a
text file)
Screenshot 9:

Define the SAP BI system as Datastore in Data Services

Before we start designing a job for our PoC, let us create a datastore for SAP BI system.
1) Open the BusinessObjects Data Services Designer.
2) In the Local Object Library, select the Datastore tab.
3) Right click and select New to create new Datastore as shown below
Screenshot 10:

4) Type in datastore name as shown below.

5) Select SAP BW Target as the Datastore type. This is because we want to load data into SAP BI and hence SAP BI is the target.
6) Enter location/IP address of the SAP BI system server
7) Enter your user name and password for the SAP BI
8) Under Advanced section, enter Client and System Number. ( You can find system number, application server details from SAP Logon Pad > Change Item)
9) In our example (refer screenshot 4), enter '/H/' as the router string.
10) Hit OK.

Screenshot 11:

11) The new datastore BW_PoC_Target_Datastore is now available under Local Object Library as shown below.

Screenshot 12:

Defining new Flat File format for our sample data

1) Open the BusinessObjects Data Services Designer.
2) In the Local Object Library, select the Formats tab
3) Right click on Flat Files format and select New.

Screenshot 13:

4) You will see File Format Editor window

Screenshot 14:

5) Select/Enter the data as given below

Type: Delimited

Name: PoC_File_Format

Root directory: Path where the sample flat file is stored (In our case it is - C:\Documents and Settings\164600\My Documents\BOBJ Data Services\BOBJ Data
Services\Reference Material\PoC)
File name: In our case it is PoC1.txt. You would see a prompt to overwrite the current schema.

Screenshot 15:

Hit Yes. Now, you would see your sample data in the editor window. Note that Designer automatically assigned appropriate data types based on the input data.
Screenshot 16:

Under Delimiters, select Column - Tab

Click Save & Close

PoC_File_Format is now available under the Flat Files tree in the Local Object Library as shown below.
Screenshot 17:

Designing job for the PoC in Data Services Designer

1) Click on Project menu and select New > Project.

Screenshot 18:

2) Let us call our new project as PoC

3) Create new job by selecting New Batch Job from context menu(right click menu). Call it as PoC_Job1
Screenshot 19:

4) Create a new dataflow and name it as PoC_DataFlow1.

Screenshot 20:

5) Open the PoC_DataFlow1 workspace.

6) Drag PoC_File_Format to the workspace as a Source.

Screenshot 21:

7) Let us have some validation done on the source data. Let us assume we want only those records where country is 'IN' (India) or 'US'.
8) Drag the Validation from the Transform tab of the Local Object Library on the Data Flow workspace.

Screenshot 22:

9) Connect source with the validation.

Screenshot 23:

10) Double click on the Validation box

11) You will see Validation Transform Editor as shown below.

Click on Country column in the Schema In section.

Enable Validation

Select 'In' radio button and enter validation criteria as 'IN', 'US'.

This indicates that records with country as either 'IN' or 'US' will pass the validation and rest would fail the validation. Passed and failed records are stored
separately.
Screenshot 24:

12) Drag the target for failed records from any of the available datastores e.g. Database, Flat Files, Excel Workbooks, BW Target, etc. In our case, I have created
a target datastore for Oracle XE schema on my local machine. So I have taken a template table as a target for the failed records.
Screenshot 25:

13) Name it as Invalid_Customer.

Screenshot 26:

14) Connect the Validation box with target Invalid_Customer for failing records.
Screenshot 27:

Now, we need SAP BI target for the records which pass the validation. We first have to create DataSource/InfoSource on the SAP BI side.
In order to create InfoSource, we need InfoObjects corresponding to our sample flat file data.
1) Log on to the SAP BI system.
2) Start the Data Warehousing Workbench using transaction Code RSA1 and select Modeling from the navigation area. Pick the entry InfoObjects.
3) Create the InfoObjects as per the cloumns in Flat File as shown in the following 3 screenshots.
Screenshot 27:

Screenshot 28:

Screenshot 29:

4) Save and Activate the InfoObjects as you create them.

5) Below screenshot shows all the created InfoObjects.
Screenshot 30:

Now, let us create a DataSource on the SAP BI side as a target for our sample data.
Following 4 screenshots shows that System is not allowing us to create a DataSource in a source system of type External System.
It also provides alternate way to achieve the same. It seems we cannot create BI 7.0 dataSource of type external system.
Screenshot 32:

Screenshot 33:

Screenshot 34:

Screenshot 35:

As per the method described in the earlier screenshot, we will create InfoSource 3.x. Follow the steps as given below.
Screenshot 36:

Screenshot 37:

Include all the InfoObjects created earlier in this InfoSource.

Screenshot 38:

Save and Activate InfoSource

Click Yes, for activating all the independent transfer programs.

Screenshot 39:

You would see logs on the screen as shown below. Go back.

Screenshot 40:

Now, create Transfer Rules for the newly created InfoSource.

Screenshot 41:

Screenshot 41:

Click Yes for assigning the DataSource to InfoSource.

Screenshot 42:

Save and Activate the InfoSource.

Screenshot 43:

Go back and you would see the subtree similar to following.

Screenshot 44:

To import the SAP BI structures, open the Datastore tab strip in the Object Library. Search for the Datastore that you have created for the SAP BI system as
target (in our caseBW_PoC_Target_Datastore). Position the cursor on the Datastore name and choose Open from the context menu. Depending on the structures
you want to use, expand the Master InfoSources or Transaction InfoSources tree.
Screenshot 45:

Find your InfoSource, and open its subtree. Position the cursor on the DataSource name, and use the option Import from the context menu.
Screenshot 46:

Afterwards, the DataSource will be available in the Object Library for your SAP BI system.

Screenshot 47:

Open the workflow, you have created in the previous section. Open the Datastore tab strip in the Object Library. Search for the Datastore that you have created
for the SAP BI system as target (in our case BW_PoC_Target_Datastore). Drag the structure to the canvas you want to load the data in the SAP BI system to.
Connect the existing Validation Transform with the new target.
Screenshot 48:

Double click this new target to open the Target Table Editor as shown below. Go to Options tab and select Column Comparison = Compare by position.
This is done so that even if column names do not match (InfoObject names cannot be more that 9 characters), we can still go ahead and carry out the data movement.
Validate your DataFlow.

Screenshot 49:

Since the loading process has to be initiated by the SAP BI system, we have to create a batch file for the execution of the Job in Data Services.
This batch file can subsequently be called by an InfoPackage in the SAP BI system.
Open the Data Services Management Console by choosing the corresponding menu entry from the Tools menu.
Screenshot 50:

Log in with your user credentials. In the Management Console navigate to the Administrator, and open your repository (or All Repositories) of the Batch folder.
Screenshot 51:

Screenshot 52:

Switch to the Batch Job Configuration tab strip and find your Job you want to schedule. Choose the option Export Execution Command.
Screenshot 53:

Provide a File Name for the batch file. Leave the other settings on the default values and press the Export button.

Note Since the maximum length of the file name entry field in the InfoPackage is limited to 44 characters, your file name entered must not exceed 40 characters
(44 characters minus 4 characters for the extension .bat)
Screenshot 54:

Screen would refresh confirming the successful export.

Screenshot 55:

In our case following two files gets created after the export under C:\temp folder.

Screenshot 56:

Before you switch to the SAP BI system, check that the RFC server is still running.
Switch to the SAP BI system you want to load the data to. Open the Data Warehousing Workbench and find your InfoSource / DataSource. Create an InfoPackage for the
DataSource.

Screenshot 57:

Screenshot 58:

Switch to the 3rd Party Selection tab strip. Press the Refresh Sel. Fields button to display the input fields.
Screenshot 59:

Enter the File Name that you have provided in the creation of the batch file.
Screenshot 60:

Switch to the Processing tab strip. Select "Only PSA" for updating the data to. Hit Save.
Screenshot 61:

Switch to Schedule tab strip. Ensure "Start Data Load Immediately" is selected. Hit Start. It would start executing the job designed in the Data Services using the batch file.

Screenshot 62:

A message "Data was requested" would appear on completion of the job.

Click on Monitor (press F6) to check the status of the load.

Screenshot 63:

Following screen shows you the status of the load.

To check whether your data has been transferred as expected, you can check the PSA for the load. From the monitor, you can choose the icon PSA Maintenance (Ctrl+F6) in
the monitor to jump to the PSA display.

Screenshot 64:

Click Continue.

Screenshot 65:

Following screen shows the data loaded into the PSA. In our example we transferred the records which passed all our Data Quality measures, i.e.
all records which Country equal to either 'IN' or 'US'. In total there are 3 records.

Screenshot 66:

Records not satisfying the validation criteria i.e. records with countries other than 'IN' or 'US' are stored the template table as desired.
Screenshot below shows the failing records stored in the Oracle XE schema.
Screenshot 67: