Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword or section
Like this
1Activity

Table Of Contents

0 of .
Results for:
No results containing your search query
P. 1
DataStage Best Practises1_1

DataStage Best Practises1_1

Ratings: (0)|Views: 49 |Likes:
Published by sam2sung2

More info:

Published by: sam2sung2 on Aug 12, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as DOC, PDF, TXT or read online from Scribd
See more
See less

03/27/2012

pdf

text

original

 
DataStage Best Practices
65063162.docPage 1 of 41
 
CONTENTS
1. INTRODUCTION.....................................................................................................................................5
1.1 O
BJECTIVE
..........................................................................................................................................51.2 D
OCUMENT
U
SAGE
................................................................................................................................5
2. DATASTAGE OVERVIEW......................................................................................................................53. DATASTAGE DEVELOPMENT WORKFLOW.......................................................................................6
3.1 B
UILDING
 
AND
T
ESTING
J
OBS
...................................................................................................................63.1.1 Dummy_Dev Project.............................................................................................................63.2 O
THER
D
ATA
S
TAGE
P
ROJECTS
................................................................................................................6
4. DATASTAGE JOB DESIGN CONSIDERATIONS..................................................................................7
4.1 J
OB
T
YPES
..........................................................................................................................................74.1.1 Import Jobs...........................................................................................................................74.1.2 Transform Jobs.....................................................................................................................84.1.3 Unload Jobs..........................................................................................................................8
5. USE OF STAGES....................................................................................................................................8
5.1 C
OMBINING
D
ATA
..................................................................................................................................85.1.1 Join, Lookup and Merge Stages...........................................................................................85.1.2 Aggregate Stage...................................................................................................................95.1.3 The Funnel Stage.................................................................................................................95.2 S
ORTING
.............................................................................................................................................95.3 D
ATA
M
ANIPULATION
............................................................................................................................105.3.1 Transformer Usage Guidelines...........................................................................................105.3.2 Modify Stage.......................................................................................................................125.4 T
RANSITIONING
D
ATA
...........................................................................................................................135.4.1 External Data......................................................................................................................135.4.2 Parallel Dataset..................................................................................................................135.5 U
NIT
T
EST
........................................................................................................................................135.5.1 Copy Stage.........................................................................................................................135.5.2 Peek Stage.........................................................................................................................135.5.3 Row Generator...................................................................................................................135.5.4 Column Generator..............................................................................................................135.5.5 Manual XLS Generation......................................................................................................14
6. GUI STANDARDS.................................................................................................................................147. DATASTAGE NAMING STANDARDS..................................................................................................148. RUNTIME COLUMN PROPAGATION (RCP).......................................................................................169. STANDARDISED REJECT HANDLING...............................................................................................16
9.1 R
EJECT
C
OMPONENTS
..........................................................................................................................169.2 C
USTOMISED
R
EJECT
M
ESSAGES
............................................................................................................189.3 R
EJECT
L
IMIT
.....................................................................................................................................199.4 B
EFORE
R
OUTINE
................................................................................................................................199.5 N
OTIFICATIONS
...................................................................................................................................199.5.1 In-line Notification of Rejects..............................................................................................199.5.2 Cross Functional Notification of Rejects.............................................................................20
65063162.docPage 2 of 41
 
10. ENVIRONMENT..................................................................................................................................20
10.1 D
EFAULT
E
NVIRONMENT
V
ARIABLES
S
TANDARDS
.......................................................................................2010.2 J
OB
P
ARAMETER
F
ILE
S
TANDARDS
........................................................................................................2010.3 D
IRECTORY
P
ATH
P
ARAMETERS
............................................................................................................2010.4 D
EFAULT
D
IRECTORY
P
ATH
P
ARAMETERS
..............................................................................................2010.5 D
IRECTORY
& D
ATASET
 
NAMING
 
STANDARDS
............................................................................................2110.5.1 Functional Area Input Files...............................................................................................2110.5.2 Functional Area Output Tables.........................................................................................2110.5.3 Functional Area Staging Tables........................................................................................2110.5.4 Internal Module Tables.....................................................................................................2110.5.5 Datasets Produced from Import Processing ....................................................................21
11. METADATA MANAGEMENT..............................................................................................................21
11.1 S
OURCE
 
AND
T
ARGET
M
ETADATA
..........................................................................................................2211.2 I
NTERNAL
M
ETADATA
..........................................................................................................................22
12. STANDARD COMMON COMPONENTS............................................................................................22
12.1 J
OB
T
EMPLATES
................................................................................................................................2212.1.1 Import Jobs.......................................................................................................................2312.1.2 Transform Jobs.................................................................................................................2312.1.3 Unload Jobs......................................................................................................................2412.2 C
ONTAINERS
....................................................................................................................................24
13. DEBUGGING A JOB...........................................................................................................................2514. COMMON ISSUES AND TIPS............................................................................................................25
14.1 1-
WAY
/
N
-
WAY
.................................................................................................................................2514.2 D
UPLICATE
K
EYS
..............................................................................................................................2614.3 R
ESOURCE
 
USAGE
V
S
P
ERFORMANCE
....................................................................................................2714.4 G
ENERAL
T
IPS
.................................................................................................................................28
15. REPOSITORY STRUCTURE..............................................................................................................29
15.1 J
OB
C
ATEGORIES
..............................................................................................................................2915.2 T
ABLE
D
EFINITION
C
ATEGORIES
............................................................................................................2915.3 R
OUTINES
........................................................................................................................................2915.4 S
HARED
C
ONTAINERS
.........................................................................................................................29
16. COMMON COMPONENTS USED IN DUMMY...................................................................................30
16.1
JBT
 _ 
SC
 _ 
JOIN
....................................................................................................................................3016.2
JBT
 _ 
SC
 _ 
SRT
 _ 
CD
 _ 
LKP
.........................................................................................................................3016.3
JBT
 _ 
ENV
 _ 
VAR
...................................................................................................................................3116.4
JBT
 _ 
ANNOTATION
...............................................................................................................................3216.5 J
OB
L
OG
S
NAPSHOT
..........................................................................................................................3216.6 R
ECONCILIATION
R
EPORT
....................................................................................................................3416.7 S
CRIPT
 
TEMPLATE
..............................................................................................................................3616.8 S
PLIT
F
ILE
......................................................................................................................................3616.9 M
AKE
F
ILE
......................................................................................................................................3616.10
JBT
 _ 
IMPORT
...................................................................................................................................3716.11
JST
 _ 
IMPORT
...................................................................................................................................3916.12
JBT
 _ 
UNLOAD
...................................................................................................................................3916.13
JST
 _ 
UNLOAD
...................................................................................................................................41
65063162.docPage 3 of 41

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->