Datastage

Teradata Utility Error Handling - Fastload
===========================================
In the first article in this series on Teradata Error Handling, we introduced th
e larger architecture and load utility environment. in this article the focus wi
ll be on the Fastload Utility and some of the techniques applicable to handling
errors, exceptions and failures that can occur within a High Availability system
.
Fastload
The Teradata Fastload utility loads data into empty tables. It has the ability t
o be restarted after a database reset or after a client process or platform fail
ure. A Fastload job step should provide the capability to either restart at the
point of failure or to completely rerun the step.
Restarts
Fastload will automatically restart a job after a database failure after the cau
se of the failure has been fixed and the database has reset. The database config
uration must be exactly the same as when the Fastload job was initiated and the
Fastload error tables must not have been de-leted. When these conditions are not
true the Fastload job must be re-executed from the beginning.
A checkpoint interval can be established as part of the BEGIN LOADING command. T
his allows a restart to continue reading an input file at the point after the la
st checkpoint taken before a failure or inter-ruption occurs. The checkpoint int
erval is set as number of records read for each checkpoint.
When a database reset occurs the Fastload utility will continue to try to re-est
ablish a database connection. When a connection is established the utility will
continue the interrupted job automatically. If a check-point has been set it wil
l continue at the point in the input file where the last checkpoint was taken. S
pecifying a checkpoint interval will de-crease the time needed for a restart but
there are tradeoffs. Using checkpoints can increase the execution time of a Fas
tload job. The following factors should be considered when specifying a checkpoi
nt in-terval:
* Each checkpoint temporarily halts the multiple session data transfer featu
re of Teradata Fastload, thereby decreasing the speed of the Teradata Fastload j
ob.
* For each checkpoint, Teradata Fastload waits for each session to complete
sending its current request, which interrupts the data transfer operation.
* The record size and the size of the Teradata Database influence how often
you should specify checkpoints. On a smaller Teradata Database, specify checkpoi
nts:
o Every 50,000 records if each record is more then 4 KB
o Every 100,000 records if each record is less than 4 KB
o On a larger Teradata Database, specify higher (less frequent) checkp
oint values.
The procedure used and the Teradata Fastload response to restarting a paused Ter
adata Fastload job depends on the phase that the Teradata Fastload job was in wh
en it was paused.
A Fastload job that was paused during loading can be can be restarted, either fr
om the beginning or from the most recent checkpoint if the BEGIN LOADING command
specified the checkpoint option. A job that was paused during end loading phase
will restart from wherever it was interrupted. This is because the Teradata Dat
abase uses internal checkpointing during this phase.
A Fastload job that terminates because of a Fastload script error may be restart
ed after the error is corrected. Commands that have exe-cuted successfully must
not be changed in the script. This will cause the restart to fail. If commands t
hat have successfully executed have to be changed the entire Fastload job must b
e re-executed.
Restart After Database Reset
The following is an example of an automatic Fastload restart after a database re
set has been initiated to fix a database error condition.
The Fastload script for this example follows:
?
code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
LOGON test/testuser,test;
SET RECORD FORMATTED;
DEFINE FILE=C:\DataIntegration\Data\tran1.dat;
SHOW;
DROP TABLE TEST.TRAN_ERR;
DROP TABLE TEST.TRAN;

DROP TABLE TEST.TRANFLERR1;
DROP TABLE TEST.TRANFLERR2;
CREATE SET TABLE TEST.TRAN ,NO FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT
(
STORE INTEGER,
TRAN_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC NOT NULL,
TRAN_DATE DATE FORMAT 'MM/DD/YYYY',
TRAN_TIME TIME(6),
TRAN_TYPE CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,
TRAN_TERMINAL CHAR(3) CHARACTER SET LATIN NOT CASESPECIFIC,
CLERK VARCHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,
CUST_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,
NAME VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC,
TOTAL_SALE DECIMAL(10,2),
SALES_TAX DECIMAL(10,2),
DISCOUNT DECIMAL(10,2),
DISC_PCT DECIMAL(5,3),
DISC_TAKEN_FLAG CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,
CASH_TENDERED DECIMAL(10,2),
CHANGE_GIVEN DECIMAL(10,2),
CHARGE_TENDERED DECIMAL(10,2),
CHECK_NUMBER VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFIC,
CHECK_TENDERED DECIMAL(10,2),
BANKCARD_TENDERED DECIMAL(10,2),
PURCHASE_ORDER_NUM VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFIC)
UNIQUE PRIMARY INDEX ( TRAN_NO );
CREATE ERROR TABLE TEST.TRAN_ERR FOR TEST.TRAN;
BEGIN LOADING TEST.TRAN
ERRORFILES TEST.TRANFLERR1, TEST.TRANFLERR2
CHECKPOINT 50000
INDICATORS;
INSERT TEST.TRAN.*;
END LOADING;
LOGOFF;
This script drops the target table and the error tables if they exist. It create
s the target table and executes the load for the target table. A checkpoint will
be taken every 50000 input records.
The following is the output from a Fastload job that was restarted automatically
after a database reset occurred during the execution of the job.
?
code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
===================================================================
=
=
=
FASTLOAD UTILITY
VERSION 12.00.00.004
=
=
PLATFORM WIN32
=
=
=
===================================================================
===================================================================
=
=
=
Copyright 1984-2007, Teradata Corporation.
=
=
ALL RIGHTS RESERVED.
=
=
=
===================================================================
**** 16:14:34 Processing starting at: Sun Aug 09 16:14:34 2009
===================================================================
=
=
=
Logon/Connection
=
=
=
===================================================================
0001 LOGON test/testuser,
****
****
****
****
****
****
16:14:37
16:14:37
16:14:37
16:14:37
16:14:38
16:14:38
Teradata Database Release: 12.00.01.24

Teradata Database Version: 12.00.01.24
Current CLI or RDBMS allows maximum row size: 64K
Character set for this job: ASCII
Number of FastLoad sessions connected = 2
FDL4808 LOGON successful
0002 SET RECORD FORMATTED;

**** 16:14:38 Now set to read 'FORMATTED' records
**** 16:14:38 Command completed successfully
0003 DEFINE FILE=C:\DataIntegration\Data\tran1.dat;
**** 16:14:38 FDL4803 DEFINE statement processed
0004 SHOW;
FILE = C:\DataIntegration\Data\tran1.dat
TOTAL RECORD LENGTH = 0
0005 DROP TABLE TEST.TRAN_ERR;
0006 DROP TABLE TEST.TRAN;
0007 DROP TABLE TEST.TRANFLERR1;
**** 16:14:38 RDBMS error 3807: Object 'TEST.TRANFLERR1' does not exist.
0008 DROP TABLE TEST.TRANFLERR2;
**** 16:14:38 RDBMS error 3807: Object 'TEST.TRANFLERR2' does not exist.
0009 CREATE SET TABLE TEST.TRAN ,NO FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT
(
STORE INTEGER,
TRAN_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC NOT NULL,
TRAN_DATE DATE FORMAT 'MM/DD/YYYY',
TRAN_TIME TIME(6),
TRAN_TYPE CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,
TRAN_TERMINAL CHAR(3) CHARACTER SET LATIN NOT CASESPECIFIC,
CLERK VARCHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,
CUST_NO CHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC,
NAME VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC,
TOTAL_SALE DECIMAL(10,2),
SALES_TAX DECIMAL(10,2),
DISCOUNT DECIMAL(10,2),
DISC_PCT DECIMAL(5,3),
DISC_TAKEN_FLAG CHAR(1) CHARACTER SET LATIN NOT CASESPECIFIC,
CASH_TENDERED DECIMAL(10,2),
CHANGE_GIVEN DECIMAL(10,2),
CHARGE_TENDERED DECIMAL(10,2),
CHECK_NUMBER VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFIC,
CHECK_TENDERED DECIMAL(10,2),
BANKCARD_TENDERED DECIMAL(10,2),
PURCHASE_ORDER_NUM VARCHAR(20) CHARACTER SET LATIN NOT CASESPECIFI
UNIQUE PRIMARY INDEX ( TRAN_NO );
0010 CREATE ERROR TABLE TEST.TRAN_ERR FOR TEST.TRAN;
0011 BEGIN LOADING TEST.TRAN
CHECKPOINT 50000
INDICATORS;
**** 16:14:38 Indicator mode is set to ON
**** 16:14:38 Number of AMPs available: 2
**** 16:14:38 BEGIN LOADING COMPLETE
===================================================================
=
=
=
Insert Phase
=
=
=
===================================================================
0012 INSERT TEST.TRAN.*;
****
****
****
****
****
****
****
****
16:14:39
16:14:39
16:14:39
16:14:40
16:14:41
16:14:42
16:14:42
16:15:08
****
****
****
****
****
****
****
****
****
****
****
****
****
16:15:08
16:15:08
16:15:08
16:15:08
16:15:08
16:15:10
16:15:10
16:15:10
16:15:10
16:15:11
16:15:12
16:15:13
16:15:14
Number of recs/msg: 316

Starting to send to RDBMS with record 1
Sending row 50000
Sending row 100000
Sending row 150000
Sending row 200000
Sending row 250000
RDBMS error 2825: NO RECORD OF THE LAST REQUEST WAS FOUND
AFTER DBC RESTART
FastLoad will attempt to retry this request 5 time(s)
The last checkpoint was taken at row: 250000
FastLoad will now restart at row:
250001
Sending row 300000
Sending row 350000
Sending row 400000
Sending row 450000
****
****
****
****
****
16:15:15
16:15:16
16:15:17
16:15:18
16:15:18
Sending row 500000

Sending row 550000
Sending row 600000
Sending row 649472
Finished sending rows to the RDBMS
===================================================================
=
=
=
End Loading Phase
=
=
=
===================================================================
0013 END LOADING;
**** 16:15:42 END LOADING COMPLETE
Total
Total
Total
Total
Total
Records Read
Error Table 1
Error Table 2
Inserts Applied
Duplicate Rows
Start:
End :
=
=
=
=
=
649472
0 ---- Table has been dropped
649472
0
Sun Aug 09 16:15:19 2009

Sun Aug 09 16:15:42 2009
0014 LOGOFF;
===================================================================
=
=
=
Logoff/Disconnect
=
=
=
===================================================================
**** 16:15:43 Logging off all sessions
**** 16:15:44 Total processor time used = '2.64063 Seconds'
.
Start : Sun Aug 09 16:14:34 2009
.
End : Sun Aug 09 16:15:44 2009
.
Highest return code encountered = '0'.
**** 16:15:44 FDL4818 FastLoad Terminated
Restart after client failure
A Fastload job can be restarted after a client process failure. In this case the
error tables and target tables must be in the state they were in when the failu
re occurred, therefore, a restart script must not in-clude the actions to drop e
rror tables, drop target table and create target table. The following is an exam
ple of the restart script.
?
code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
LOGON test/testuser,test;
SET RECORD FORMATTED;
DEFINE FILE=C:\DataIntegration\Data\tran1.dat;
SHOW;
BEGIN LOADING TEST.TRAN
CHECKPOINT 50000
INDICATORS;
INSERT TEST.TRAN.*;
END LOADING;
LOGOFF;
In this example a client failure occurs while a Fastload job is execut-ing.
===================================================================
= =
= Insert Phase =
= =
===================================================================
****
****
****
****
****
****
****
****
****
****
16:33:43
16:33:43
16:33:44
16:33:45
16:33:46
16:33:47
16:33:47
16:33:48
16:33:49
16:33:50

Starting to send to RDBMS with record 1
Sending row 50000
Sending row 100000
Sending row 150000
Sending row 200000
Sending row 250000
Sending row 300000
Sending row 350000
Sending row 400000
When the client failure occurs the Teradata database will log off the Fastload s
essions, however, on a network attached client a gateway session will exist unti
l a timeout has expired and the gateway session is cancelled. The default timeou
t period is 20 minutes. During this time the Fastload job can not be restarted a
nd will return the following error.
?
code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
C:\DataIntegration\Scripts>fastload <tranrestart.fl ============================
=======================================
= =
= FASTLOAD UTILITY VERSION 12.00.00.004 =
= PLATFORM WIN32 =
= =
===================================================================
===================================================================
=
=
=
=
=
=
=
=
===================================================================
===================================================================
=
=
=
Logon/Connection
=
=
=
===================================================================
**** 16:34:29 Teradata Database Release: 12.00.01.24
**** 16:34:29 Teradata Database Version: 12.00.01.24
**** 16:34:29 Current CLI or RDBMS allows maximum row size: 64K
**** 16:34:29 Character set for this job: ASCII
**** 16:34:29 Number of FastLoad sessions connected = 2
**** 16:34:29 FDL4808 LOGON successful
0004 SHOW;
CHECKPOINT 50000
INDICATORS;
**** 16:34:29 Indicator mode is set to ON
**** 16:34:29 RDBMS error 2738: TRAN already has Fastload running
===================================================================
=
=
=
Logoff/Disconnect
=
=
=
===================================================================
.
Start : Sun Aug 09 16:34:26 2009
.
End : Sun Aug 09 16:34:29 2009
.
**** 16:34:29 FDL4818 Fastload Terminated
The Fastload job can be restarted before the Gateway session timeout if the Gate
way sessions for the failed Fastload job are killed. This can be done using the
gtwglobal utility in the following way.
?
code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
C:\Documents and Settings\john>gtwglobal
_______
|
|
|
___
__
____
|
| /
|/ \
____|
____|
| --|
/
| /
|
| \___ |
\____| \____|
____
____|
/
|
\____|
|
__|__
|
|
|__
____
____|
/
|
\____|
Enter gateway command or enter h for Help:

select host 1
Host 1 has been selected.
Enter gateway command or enter h for Help:
1>kill user testuser
User TESTUSER has 4 sessions killed
1059
1060
1061
1062
Once the Fastload sessions have been killed the Fastload job can be restarted. T
he following is an example of the output from the restarted Fastload job after a
client failure.
?
code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
C:\DataIntegration\Scripts>fastload <tranrestart.fl
===================================================================
=
=
=
FASTLOAD UTILITY
VERSION 12.00.00.004
=
=
PLATFORM WIN32
=
=
=
===================================================================
===================================================================
=
=
=
=
=
=
=
=
===================================================================
===================================================================
=
=
=
Logon/Connection
=
=
=
===================================================================
****
****
****
****
****
****
16:53:57
16:53:57
16:53:57
16:53:57
16:53:57
16:53:57


0004 SHOW;
CHECKPOINT 50000
INDICATORS;
****
****
****
****
16:53:57
16:53:57
16:53:57
16:53:57
Indicator mode is set to ON

FastLoad is restarting
Number of AMPs available: 2
BEGIN LOADING COMPLETE
===================================================================
=
=
=
Insert Phase
=
=
=
===================================================================
****
****
****
****
****
****
****
****
****
16:53:57
16:53:57
16:53:57
16:53:58
16:53:59
16:54:00
16:54:01
16:54:01
16:54:01

The last checkpoint was taken at row: 400000
FastLoad will now restart at row:
400001
Sending row 450000
Sending row 500000
Sending row 550000
Sending row 600000
Sending row 649472
Finished sending rows to the RDBMS
===================================================================
=
=
=
End Loading Phase
=
=
=
===================================================================
0007 END LOADING;
**** 16:54:21 END LOADING COMPLETE
Total
Total
Total
Total
Total
Start:
End :
Records Read
Error Table 1
Error Table 2
Inserts Applied
Duplicate Rows
=
=
=
=
=
Sun Aug 09 16:54:02 2009

Sun Aug 09 16:54:21 2009
649472
649472
0
0008 LOGOFF;
===================================================================
=
=
=
Logoff/Disconnect
=
=
=
===================================================================
.
Start : Sun Aug 09 16:53:54 2009
.
End : Sun Aug 09 16:54:22 2009
.
**** 16:54:22 FDL4818 FastLoad Terminated
Rerun
In some cases a Fastload job may need to be re-executed when the error tables or
target tables are not available or the database configuration has changed. In t
he previous example the original Fastload job can be rerun because it drops the
error and target tables and recreates the target table before loading the data.
In the next article in this series we will look at the Multiload Utility.

Datastage

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Datastage

Uploaded by

Copyright:

Available Formats

Teradata Utility Error Handling - Fastload

DROP TABLE TEST.TRAN;

Teradata Database Release: 12.00.01.24

0002 SET RECORD FORMATTED;

Number of recs/msg: 316

Sending row 500000

Sun Aug 09 16:15:19 2009

Number of recs/msg: 316

Enter gateway command or enter h for Help:

Teradata Database Release: 12.00.01.24

0002 SET RECORD FORMATTED;

**** 16:53:57 Now set to read 'FORMATTED' records

Indicator mode is set to ON

Number of recs/msg: 316

Sun Aug 09 16:54:02 2009

You might also like