Professional Documents
Culture Documents
Common Issues in Datastage PDF
Common Issues in Datastage PDF
DataStage Developer - user with full access to all areas of a DataStage project
DataStage Operator - has privileges to run and manage deployed DataStage jobs
-none- - no permission to log on to DataStage
There are two ways to analyze a hashed file. Both should be invoked from the datastage
command shell. These are:
FILE.STAT command
ANALYZE.FILE command
There is a few possible methods of sending sms messages from Datastage. However, there
is no easy way to do this directly from Datastage and all methods described below will
require some effort.
The easiest way of doing that from the Datastage standpoint is to configure an SMTP
(email) server as a mobile phone gateway. In that case, a Notification Activity can be
used to send message with a job log and any desired details. DSSendMail Before-job or
After-job subroutine can be also used to send sms messages.
If configured properly, the recipients email address will have the following format:
600123456@oursmsgateway.com
If there is no possibility of configuring a mail server to send text messages, you can to work
it around by using an external application run directly from the operational system.
There is a whole bunch of unix scripts and applications to send sms messages.
In that solution, you will need to create a batch script which will take care of sending
messages and invoke it from Datastage using ExecDOS or ExecSh subroutines passing the
required parameters (like phone number and message body).
Please keep in mind that all these solutions may require a contact to the local cellphone
provider first and, depending on the country, it may not be free of charge and in some
cases the provider may not support the capability at all.
2.
2.1. Error in Link collector - Stage does not support in-process active-to-active
inputs or outputs
To get rid of the error just go to the Job Properties -> Performance and select Enable row
buffer.
Then select Inter process which will let the link collector run correctly.
Buffer size set to 128Kb should be fine, however it's a good idea to increase the timeout.
source odbc stage which fetches one record from the database and maps it to one
column - for example: select sysdate from dual
A transformer which passes that record through. If required, add pl/sql procedure
parameters as columns on the right-hand side of tranformer's mapping
Put Stored Procedure (STP) stage as a destination. Fill in connection parameters,
type in the procedure name and select Transform as procedure type. In the input
tab select 'execute procedure for each row' (it will be run once).
@INROWNUM and @OUTROWNUM are internal datastage variables which do the following:
These variables can be used to generate sequences, primary keys, id's, numbering rows and
also for debugging and error tracing.
They play similiar role as sequences in Oracle.
2.16. Datastage trim function cuts out more characters than expected
By deafult datastage trim function will work this way:
Trim(" a b c d ") will return "a b c d" while in many other programming/scripting languages
"a b c d" result would be expected.
That is beacuse by default an R parameter is assumed which is R - Removes leading and
trailing occurrences of character, and reduces multiple occurrences to a single occurrence.
To get the "a b c d" as a result use the trim function in the following way: Trim(" a b c d
"," ","B")
2.17. Database update actions in ORACLE stage
The destination table can be updated using various Update actions in Oracle stage. Be
aware of the fact that it's crucial to select the key columns properly as it will determine
which column will appear in the WHERE part of the SQL statement. Available actions:
Clear the table then insert rows - deletes the contents of the table (DELETE
statement) and adds new rows (INSERT).
Truncate the table then insert rows - deletes the contents of the table
(TRUNCATE statement) and adds new rows (INSERT).
Insert rows without clearing - only adds new rows (INSERT statement).
Delete existing rows only - deletes matched rows (issues only the DELETE
statement).
Replace existing rows completely - deletes the existing rows (DELETE
statement), then adds new rows (INSERT).
Update existing rows only - updates existing rows (UPDATE statement).
Update existing rows or insert new rows - updates existing data rows (UPDATE)
or adds new rows (INSERT). An UPDATE is issued first and if succeeds the INSERT is
ommited.
Insert new rows or update existing rows - adds new rows (INSERT) or updates
existing rows (UPDATE). An INSERT is issued first and if succeeds the UPDATE is
ommited.
User-defined SQL - the data is written using a user-defined SQL statement.
User-defined SQL file - the data is written using a user-defined SQL statement
from a file.
ICONV and OCONV functions are quite often used to handle data in Datastage.
ICONV converts a string to an internal storage format and OCONV converts an
expression to an output format.
Syntax:
Iconv (string, conversion code)
Oconv(expression, conversion )
Some useful iconv and oconv examples:
Iconv("10/14/06", "D2/") = 14167
Oconv(14167, "D-E") = "14-10-2006"
Oconv(14167, "D DMY[,A,]") = "14 OCTOBER 2006"
Oconv(12003005, "MD2$,") = "$120,030.05"
That expression formats a number and rounds it to 2 decimal places:
Oconv(L01.TURNOVER_VALUE*100,"MD2")
Iconv and oconv can be combined in one expression to reformat date format easily:
Oconv(Iconv("10/14/06", "D2/"),"D-E") = "14-10-2006"
Error message:
Error message:
Error calling subroutine:
Now hang on just a minute, the guy who architected DataStage is now an IBM
Distinguished Engineer! The design is not so much stupid as pragmatically
challenged. DataStage 8 fixes this problem with a new Quick Find and Advanced
Find that looks for any type of object including jobs, columns and
properties. Your solution here is to use the good old google desktop search
(if you only want to view the job).
Environment variables...if I need to choose more than one, I dont want to have to open it up and
hunt the variable down each time. Let me choose multiple ones at once. Also, let me sort them
on variable name rather than description. Sometimes the description is meaningless or
misleading.
I agree. Doesnt the environment property window suck? Two things alleviate
this. 1) Use the Copy Job Utility from Ken Bland and Associates. See my post
DataStage tip 1 of 10: hacking job parameters. 2) Hack the DSParams file in the
DataStage Project home directory and manually add a folder structure to your
new environment variables. This makes them a LOT easier to find. For this
technique read my post DataStage tip: using job parameters without losing your
mind.
Why can we NOT open up multiple panels/panes? How much easier would it be if
we could to allow us to compare variable lists, priority information and so
on?
When you are comparing two jobs it is much better to open two Designers. Like
most Windows applications many of the property windows are modal and you can
only open one of them at a time. Having two Designers lets you open the same
property box in two jobs and compare the side by side or flip between
them.DataStage 8 introduces a compare job function inside the Designer.
Why cant I drag off the side of the screen when highlighting a long stream
of stages within a job?
You wouldnt believe how easy drag and drop becomes if you use the Diagram Zoom Out function. You can zoom out several times on a big job to see every
stage in that job with space to spare around the edges. This makes drag and
drop easy.
MODIFY statement. Why cant Specification be chosen from a pick list of variables that allows
for multiple variable selection?
Im on a personal campaign to rid the world of unnecessary Modify stages. Use
a Transformer first. If performance becomes a problem later on then replace
it with a Modify stage. For more details read my post Is the DataStage parallel
transformer evil?
Copying jobs. Once a job has been copied and given its name, it cannot be copied again without
an error. Why not allow this to take place but prefix the name with "Copy1Of..", "Copy2Of"
and so forth?
I dont like copying jobs this way as you have to then rename them. If I am
copying multiple times I open the job and keep using the Save As.. function
as it lets me enter new job names as I go.
Please add your own feedback or solutions to any of these problems. If you
have a DataStage problem that you cannot get fixed in the Ascential ITToolbox
forum or the dsxchange forum you can email it to me at websphereblog at gmail
dot com.