This action might not be possible to undo. Are you sure you want to continue?
Teradata Client Tools
Client Tools 2
Teradata Database provides several application utilities for data manipulation Each utility exploits the capabilities of Teradata parallel architecture Teradata application utilities are supported on several hardware platforms including a wide range of channel-attached mainframes and LAN-attached clients Regardless of host platform, all access between the host and the Teradata Database relies on Call Level Interface (CLI)
Client Tools 3
Application Utilities «.
The Teradata application utilities run on the host computer, whether it is mainframe or a workstation BTEQ is a transparent interface to CLI, to transmit textual SQL to Teradata server, and deliver response to the user FastLoad, MultiLoad and TPump performs high volume data transfer from the host to Teradata Fast Export performs high volume SELECTs to export data from Teradata server to the host CLI is the vehicle that makes parallel access possible
Client Tools 4
Application Utilities «.
Operating System Support Environment Multi Load CLI Host Fast Export TPum p
Client Tools Request 5 and Response Flow Application Request CLI MTDP MOSI Response Server .
Client Tools 6 Data Flow Host BTEQ FastLoad MultiLoad Gateway TPump Fast Export PE PE AMP Data Disk AMP AMP Data AMP Disk Disk Disk .
Client Tools 7 Application Development INMODs More Flexibility Application Utilities Easy Ease of Use Difficult Less Selection of the right vehicle can be crucial and depends on how difficult it is to implement and maintain Use of application utilities Offer less complexity Takes full advantage of parallel processing .
Client Tools 8 Using An INMOD Routine The application utilities allow input data to be read or pre-processed by a user-written INMOD routine. An INMOD routine can perform various functions: Generate records to be passed to the utility Validate a data record before passing it to the utility Read data directly from one or more database systems Select specific records for input to the Teradata Database Perform data conversions not supported by utilities Add or change data fields in the records .
TPump.Client Tools 9 Using An INMOD Routine IMS ORACLE User± User±written INMOD FastLoad. « Server . MultiLoad.
10 Client Tools BTEQ FastLoad Fast Export MultiLoad TPump .
Client Tools 11 BTEQ Batch mode utility for submitting SQL requests to the Teradata Database Runs on every supported platform ± laptop to mainframe Flexible and easy to use report writer Exports data to a client system from the Teradata Database Reads input data and imports it to the Teradata Database Limited ability to branch forward to a LABEL .
logoff Request Flow CLI T T Network attached Client MTDP MOSI Server P AMP Data AMP Data AMP Data AMP Data .logon select .Client Tools 12 T script .
logon select .logoff Response Flow CLI BTEQ BTEQ Network attached Client MTDP MOSI PE Generated answer set Server AMP AMP AMP AMP Data Data Data Data .Client Tools 13 BTEQ script .
time-consuming steps .Client Tools 14 Conditional Logic T T has the ability to branch forward in a script based on a test of either an error code or an activity count This is not a true loop function yet it can be used to avoid unnecessary.
GOTO TableOK CR AT TA L Customer_Info ( Acc_no INT G R. Cust_Name VARCHAR(20).LA L Continue .IF RRORCOD = 0 TH N . Cust_Name.Acc_no).GOTO Continue . .Acc_no = .xample T D L T Customer_Info ALL. alance D CIMAL(9.LA L TableOK INS RT INTO Customer_Info ( S L CT A.Client Tools 15 Conditional Logic .2)). Acc_Cust .Acc_no. Customer C WH R alance > 100000 AND A.IF ACTIVITYCOUNT > 0 TH N . UIT . . . alance FROM Accounts A.
Client Tools 16 Command Set BTEQ BTEQ commands can be classified as follows: Session control commands File control commands Sequence control commands Format control commands .
Commands: LOGON SESSIONS LOGOFF QUIT.Client Tools 17 Session Control Commands BTEQ Controls session characteristics.EXIT SHOW CONTROLS SESSION TRANSACTION .
Commands: OS RUN IMPORT REPEAT QUIET EXPORT INDICDATA/RECORDMODE .Client Tools 18 File Control Commands BTEQ Specify the format of incoming/outgoing information and identify the source and destination of input/output streams.
Commands: IF THEN ELSE ENDIF GOTO .Client Tools Sequence 19 Control Commands BTEQ Controls the sequence in which other BTEQ commands and SQL statements will be executed within the scripts and macros.
Commands: FOOTING HEADING RETLIMIT FOLDLINE SUPPRESS QUIET .Client Tools 40 Format Control Commands BTEQ To specify the way in which BTEQ presents information on the screen/printer.
EXPORT feature provides the ability to separate the report or output data from the accounting information. The . this mixed output typically renders the data unsuitable for some purposes. If all of this information is captured in a single output file.Client Tools 41 EXPORT BTEQ BTEQ by default delivers a response to all SQL queries that includes a helpful message along with helpful diagnostic information about the time taken to perform the query. .
export data file=x.dat sel * from table.passwd .Client Tools 42 EXPORT BTEQ BTEQ Script . .quit.logon mac/user. Logon complete 1200 rows returned Time 15 seconds BTEQ Default Output 12348009 23498763 8568733 933435490 x.dat .export reset .
Client Tools 43 EXPORT Types BTEQ .EXPORT INDICDATA .EXPORT DATA .EXPORT .EXPORT DIF Output converted to DIF used to transport data to various PC programs.EXPORT REPORT . Example: dBase.EXPORT RESET Reverses the effect of a previous . Lotus 1-2-3 .
column A 1 4 7 column B 2 5 8 column C 3 6 9 Transfers data one column at a time with numeric data converted to character.EXPORT REPORT .EXPORT REPORT Sends results to a host file in field mode.Client Tools 44 Data Modes BTEQ Field mode is set by . . Data set contains column headings and formatted data.
Nulls are represented as zeros or spaces.Client Tools 45 Data Modes «. field1 field2 field3 field4 Transfers data one row at a time in a host format. . BTEQ Record mode is set by .EXPORT DATA Sends results to a host file in record mode.EXPORT DATA. .
Allows host programs to deal with NULLs. Nulls are represented as zeros or spaces. . . sending an indicator variable for nulls.EXPORT INDICDATA.Client Tools 46 Data Modes «. BTEQ Record mode is set by . Ind field1 field2 field3 field4 Var Transfers data one row at a time in a host format.EXPORT INDICDATA Sends query results that contain indicator variables to a host file.
.[SET] INDICDATA [ON] INDICATORS ON INDICATORS Export NULL Columns F 3 F 4 F 5 F 6 F 7 - BTEQ BTEQ FastLoad MultiLoad.. . F1 2 .Client Tools 47 Indicator Variables BTEQ Indicator variables allow utilities to process records that contain NULL indicators. Fast TPump 0100100 00000000 F F 1 2 0 «.EXPORT INDICDATA .
DELETEs and macro transactions BTEQ supports IMPORT in DATA.IMPORT loads data from the host to the Teradata database with a USING clause .Client Tools Data 48 Handling With IMPORT BTEQ Useful to IMPORT data from the host to Teradata as a series of INSERTs. UPDATEs. INDICDATA and REPORT modes only .
Client Tools 50 Application Utility Checklist BTEQ BTE FastLoa Fast Export DDL Functions QAll d All DML Functions Yes Multiple DML Yes Multiple tables Multiple sessions Yes SQ Protocol used Y Conditional Exp. Les Yes Arithmetic Calc Yes Data conversion Error files Error limits User routines No No No MultiLoad TPum p .
Client Tools 51 FastLoad Fast batch mode utility for loading new tables onto the Teradata Database Can reload previously emptied tables Full restart capability Error limits and error tables. accessible using SQL Restartable INMOD routine capability Ability to load data in several stages .
Concepts: Load into an empty table with no secondary indexes. Checkpoints can be taken for restarts. Creates two error tables.Client Tools 52 Purpose/Concepts FastLoad Purpose: at Load large amounts of data into an empty table high speed. .
Client Tools 53 Restrictions FastLoad If an AMP goes down. FastLoad cannot be restarted until it is back online The Teradata Database will accommodate up to 15 FL/ML/FE runs at one time Tables defined with referential integrity or secondary indexes cannot be loaded with FastLoad Duplicate rows cannot be loaded into multi-set table with FastLoad .
Client Tools 54 FastLoad Phases ± Phase 1 FastLoad Phase 1 FastLoad uses one SQL session to define AMP steps The PE sends a block to each AMP AMPs hash each record and redistribute them to the AMP responsible for the hash value Records are written to the target table in unsorted blocks .
Client Tools 55 Phase 1 PE FastLoad Host SQL & Data BYNET AMP AMP Disk Disk .
Client Tools tools 56 Phase 1 PE FastLoad Host Data BYNET AMP AMP Disk Disk .
Client Tools 57
FastLoad Phases ± Phase 2
Phase 2 When loading completes, each AMP sorts the target table, puts the rows into blocks, and writes the blocks to the disk Fall back rows are then generated if required
Client Tools 58
Client Tools 59
fastload < floadjob > floadjob.out
.logon Tdpid/User,Passwd .begin loading Customer errorfiles custerr1, custerr2; define CustNum ( Integer ), Name ( Char(20) ) file = CustData.dat; insert into Customer values (:CustNum, :Name); .end Loading; .logoff;
Name of empty table Starts Phase 1
Defines input record SQL Insert statement Start phase 2; if omitted utility will pause
Client Tools 60
Paused FastLoad Job
SQL & Data
PE FastLoad datafile Data 1 BYNET AMP AMP Disk Disk .Client Tools 61 Paused FastLoad Job «.
PE FastLoad datafile Data n BYNET AMP AMP Disk Disk .Client Tools 62 Paused FastLoad Job «.
Client Tools 63 Paused FastLoad Job «. PE FastLoad BYNET AMP AMP Disk Disk .
Column_Nam e Error_Code Error Field nameParcel Data Content The Error code in DBC.ErrorMsgs The column that cause the error The data record sent by the host Error Table 2 Captures rows that contain duplicate values for UPIs .Client Tools 64 Error Tables FastLoad Error Table 1 Contains one row for each row which failed to be loaded due to constraint violations or translation errors.
Client Tools 65 INMOD FastLoad Data INMOD FastLoa d Teradata Server To acquire data from a non-standard data source. This is done via return codes. FastLoad can use an INMOD or exit routine. and the INMOD will pass data records to FastLoad. Communication needs to be established between the INMOD and FastLoad. . make unusual conversion of data.
INMOD should open files to read data. FastLoad expects a record.Client Tools INMOD/FastLoad 66 Communication FastLoad INMOD to FastLoad return codes 0 Non 0 Indicates that INMOD is returning a record INMOD indicates end-of-file condition FastLoad to INMOD return codes 0 1 Calling for the first time. indicates Next Record Transfer .
FastLoad is not expecting a record and will not send a zero return code. Indicates a Teradata failure. Indicates a checkpoint has been written and INMOD should remember it. 2 4 5 . FastLoad instructs INMOD to clean up. INMOD should 3 position itself to the last checkpoint. INMOD should position itself to the last checkpoint. No record expected.Client Tools 67 Additional Return Codes FastLoad FastLoad to INMOD FastLoad has been restarted. No record expected.
Client Tools Other 68 FastLoad Commands init-string (optional ) max [min] max rejected records hours minutes integerdate/ansidate FastLoad AXSMOD SESSIONS ERRLIMIT TENACITY SLEEP DATEFORM .
Les Arithmetic Calc Yes Data conversion Yes Error files Error limits User routines No No No MultiLoad TPum p .Client Tools 69 Application Utility Checklist FastLoa Fast d Limited Export INS No No Yes FastLoa d No No Limited Yes Yes Yes FastLoad BTE DDL Functions QAll All DML Functions Yes Multiple DML Yes Multiple tables Multiple sessions Yes SQ Protocol used Y Conditional Exp.
Fast Export and TPump Provides a fully nested .Client Tools 70 Support Environment Launch vehicle for utilities such as MultiLoad.RUN file facility Interprets utility commands and provides error reporting Supports a wide range of DDL and DML commands Allows for conditional processing of commands Supports system and user defined variables Provides recovery management from a Teradata or host failure .
BEGIN EXPORT TPump is invoked with: .BEGIN IMPORT MLOAD or .Client Tools 71 Invoking Utilities Support Environme nt MultiLoad is invoked with: .BEGIN LOAD .BEGIN DELETE MLOAD Fast Export is invoked with: .
.logoff Support Environment Commands . .begin export. .Client Tools 72 Invoking Utilities «.passwd. Support Environme nt A sample Fast Export script . .logtable cust_table. select * from Transaction.logon mac/user. .end export.export outfile Dataout.
IF .IF/.LOGTABLE Acquires or creates the restart log table .LOGOFF .DISPLAY Writes messages to a specific destination .RUN .SET Assigns a data type and value to a .LOGON .ELSE .ENDIF condition Connects multiple session to Teradata Terminates the utility operation Specifies an external script file Identifies statements to be executed if certain conditions are true or otherwise Resumes the program after a .Client Tools 73 Commands Support Environme nt .
SET var [TO] expression.Client Tools 74 System Variables Support Environme nt . System X(5) VM/SP &SYSUSER User Id BRM . Variable Example Description Permits a variable to be set or reset to an expression or a preexisting variable Format &SYSDATE System Date YY/MM/DD 98/04/01 &SYSTIME System Time HH:MM:SS 11:05:00 &SYSDAY Day Of Week X(3) MON &SYSOS Host Op.
Write a message and terminate job if not µFRI¶ Invoke utility Support Environme nt .run File Logon. .LOGOFF. .Client Tools 75 Example .DISPLAY µThis job runs Fridays only¶ TO file ErrFile. ****** ****** ****** .IF µ&SYSDAY¶ NE µFRI¶ THEN .logtable cust_logtable. .ENDIF.begin import mload ****** Create or acquire restart log table Accept LOGON string from Logon File Check for Day of week. .
Client Tools 76 Fast Export Exports large volumes of formatted data from Teradata to a host file or user-written application Export from multiple tables Uses support environment Fully automated restart .
BEGIN EXPORT SESSIONS max min TENACITY hours SLEEP minutes NOTIFY high | medium | low | . To Specify: Number of SESSIONS required Number of hours Fast Export will try to establish a connection Number of minutes between each logon attempts Parameter for specifying the notify user exit option.Client Tools 77 BEGIN Export Fast Export . .
END EXPORT Delimits a series of commands that define a single EXPORT action Must be the last directive in a Fast Export script Causes the utility to send the SELECT to the Teradata Database .Client Tools 78 END Export Fast Export .
begin export sessions 12. Define Restart Log Specify sessions. . .end export. .logoff.logtable RestartLog. .Client Tools 79 Sample Fast Export Script Fast Export .run file logon.export outfile dataout. . Destination file. select * from Customer. Send request Terminate sessions .
Les Arithmetic Calc Yes Data conversion Yes Error files Error limits User routines No No No FastLoa Fast MultiLoad TPum d p No Limited Export INS SEL No Yes No Yes Yes Yes FastLoa EXPORT d No Yes No Yes Limited Yes Yes Yes Yes No No Yes .Client Tools 80 Application Utility Checklist Fast Export BTE DDL Functions QAll All DML Functions Yes Multiple DML Yes Multiple tables Multiple sessions Yes SQ Protocol used Y Conditional Exp.
UPDATEs.Client Tools 81 MultiLoad Supports up to five populated tables Multiple operations with one pass of input files Conditional logic for applying changes Supports Inserts. DELETEs and UPSERTs Full restart capability Error reporting via error tables Support for INMODs .
Client Tools 82 MultiLoad Table A update Insert Table B Table C MultiLoad Delete Host Server Table D Table E .
exponentiation or aggregates are not processed Cannot process tables with Secondary Indexes or Referential Integrity defined Import tasks require use of Primary Index .Client Tools 83 MultiLoad Limitations MultiLoad No data retrieval capability Arithmetic functions.
Two distinct tasks are: IMPORT task: These are the tasks which intermix a number of different SQL/DML statements and apply them to up to five different tables depending on the APPLY conditions DELETE task: These are tasks which execute a single DELETE statement on a single table. .Client Tools 84 MultiLoad Tasks MultiLoad MultiLoad allows INSERT. UPDATE. DELETE and UPSERT operations against up to five target tables per task.
Client Tools 85 IMPORT TASK Input Data inserts update s deletes upserts MultiLoad Hos t MultiLoad Apply Conditions Serve r T 1 T 2 T 3 T 4 T 5 .
Client Tools 86 DELETE TASK Data value deletes only MultiLoad Hos t MultiLoad Serve r T 1 delete Rows .
2). . .begin mload tables Emploee. . do insert for missing update rows. .layout Employee_Trans.dml label payroll.Client tools 88 Sample MultiLoad Script MultiLoad . begin loading . :Salary ).field Deptno * smallint. . .field Empno * smallint. define logtable .logtable logtable001.filler Transcode 1 char(3). Emp_History. Salary ) values (:Empno.field Salary * decimal(8. insert into Employee (Empno. update Employee set Salary = :Salary where Empno = :Empno. define .logon TDPID/User/Passwd.
dml label Terminate. .logoff .import infile Filename. delete from Employee where Empno = :Empno. File name layout Employee_Trans Insert apply Payroll where Transcode = µPAY¶ statement apply Terminate where Transcode = µDEL¶. MultiLoad . . Deptno) values (:Empno. insert into Emp_History (Empno.Client Tools 89 Sample MultiLoad Script «. :Deptno ).end mload End loading .
. . .logtable defines the name of the restart log. . .dml label defines a set of DML instructions which will be applied if conditions are met.layout defines the layout of the incoming record(s).field defines the name of an input field.Client Tools 90 Script Components MultiLoad . and its datatype.begin mload tables defines the tables which will participate in the MultiLoad. its position in the record. .logon defines username which will own the sessions.
end mload defines end of MultiLoad script.import infile references filename of the input file.logoff terminate the sessions. . .Client tools 91 Script Components «. apply references label to be applied and conditions under which to do so. layout references previously defined layout. . do insert if not present. MultiLoad do insert for missing update rows indicates that UPSERT processing follows: do update if present. .
Client Tools 92 IMPORT Phases MultiLoad IMPORT consists of five separate phases of processing. Preliminary Phase DML Phase AMPs Acquisition Phase AMPs and Application Phase tables End Phase Basic clean up Basic setup Get DML steps down on Send the input data to sort it Apply the input data to the appropriate target .
Client Tools 93 MultiLoad Preliminary Phase MultiLoad and SQL #AMPS + 2 One per target table Two per target One per Prevent DDL Validate all statements Start all sessions Create work tables Create error tables table Create restart log IMPORT run Apply locks to target tables .
Client Tools 94 DML Transaction Phase Send DML steps in work tables MultiLoad Send prototype DML to the Server Add a USING modifier to the request Host data to be filled in from input file .
Sort by hash value and sequence to be applied Set up transition to the application phase. .Client Tools 95 Acquisition Phase MultiLoad Get the data from host and apply it to appropriate AMP worktables.Sort the reblocked records in the work tables . .Set table headers for application phase .Make blocks and send to AMPs .Duplicate rows for each successful APPLY .Upgrade locks on target tables to write . .Deblock and resend record to µcorrect¶ AMP Reblock and store in worktable of target table.
Restartable based on last checkpoint .Client Tools 96 Application Phase MultiLoad Execute MLOAD for each target table as a single multi-statement request. .Executed as a single transaction without rollback .AMPs independently apply changes to target tables .No Transient Journal needed .End of host interaction until end of phase .
.Client Tools 97 Cleanup Phase MultiLoad Execute END MLOAD processing as a series of transactions performed by the host utility.Target tables are made available to other users .Work tables are dropped .Table headers are restored across all AMPs .Final error code is reported .All locks are released .Logtable is dropped Mload Session logoff.Logoff request sent to each AMP with a session . .Dictionary cache of Target tables is spoiled .Statistics are reported .
Client Tools DELETE/IMPORT 98 Differences MultiLoad DELETE tas s o erate er si ilarl to IMPORT tas s it so e differences. Deletin ased on a Pri ar Inde is not er itted DML DELETE state ent sent to eac atc ta arcel a MP it o c uisition record to a l ase ecause no aria le in ut lication ase reads eac tar et loc and deletes ualif in ro s ll ot er as ects si ilar to IMPORT tas .
Client Tools 99 Restarting MultiLoad MultiLoad DBC restart MLOAD reinitiated automatically after DBC recovery Continue from checkpoint without user interaction Host restart Resubmit the script as-is MLOAD determines its stopping point and restarts .
Client Tools 100 Application Utility Checklist MultiLoad BTE DDL Functions QAll All DML Functions Yes Multiple DML Yes Multiple tables Multiple sessions Yes SQ Protocol used Y Conditional Exp. Les Arithmetic Calc Yes Data conversion Yes Error files Error limits User routines No No No FastLoa Fast MultiLoad TPum d p No Limited Export All SEL Ins/upd/d INS el Yes Yes No Yes Yes No Yes Yes Yes FastLoa EXPORT MultiLoad d No Yes Yes Yes No Yes Limited Yes Yes Yes Yes No No Yes Yes Yes Yes Yes .
or a combination. and DELETE operations.Have secondary indexes and Referential Integrity constraints . to more than 60 tables at a time from the same source Alternative to MultiLoad for low-volume batch maintenance of large databases Allows target tables to: .Be populated or empty . UPDATE.Client Tools 101 TPump Allows near real-time updates from transactional systems into the wearhouse Performs INSERT.Be MULISET or SET .
Client Tools 102 TPump Allows conditional processing Supports automatic restarts No session limit Uses row-hash locks. allowing concurrent updates on the same table User can specify how many updates occur minute by minute No limit to the number of concurrent instances .
Client Tools 103 Limitations TPum p Exponential operators are not allowed Aggregate operators are not allowed Arithmetic functions are not supported There is a limit of four IMPORT commands within a single TPump ³load´ task .
.logon TDPID/User.Client Tools 104 TPump Sample Script TPum p .field AByteInt * byteint. .layout lay1a.Passwd.filler ATest * byteint. .begin load errlimit 50 checkpoint 1 sessions 11 tenacity 2 error table tlnterr BTET pack 10. . . .field AInteger * integer.logtable logtbl002. .
. .dml label labelb ignore missing rows ignore extra rows do insert for missing update rows. update TPumptbl set AInteger = AInteger + 1 where AByteInt = :AByteInt. :AInteger ). TPum p . :Ainteger).dml label labela ignore duplicate rows ignore missing rows ignore extra rows insert into TPumptbl values (:AByteInt. insert into TPumptbl values (:AByteInt.Client Tools 105 TPump Sample Script «.
import infile Tpumptbl. .dat layout lay1a from 1 for 400 apply labelb.logoff. . .Client Tools 106 TPump Sample Script «.end mload. TPum p .import infile TPumptbl. .dat layout lay1a from 1 for 400 apply labela where ATest = 1.
Client Tools 107 TPump / MultiLoad TPum p MultiLoad performance improves as the volume of changes increases. TPump uses row hash locking to allow for concurrent read and write access to target tables. MultiLoad locks tables for write access (Phase 2) until it completes . TPump does better on relatively low volumes of changes TPump uses macros to modify tables rather than actual DML commands MultiLoad uses the DML statements.
Client Tools 108 Application Utility Checklist TPum p BTE DDL Functions QAll All DML Functions Yes Multiple DML Yes Multiple tables Multiple sessions Yes SQ Protocol used Y Conditional Exp. Les Arithmetic Calc Yes Data conversion Yes Error files Error limits User routines No No No FastLoa Fast MultiLoad TPum d p All No Limited Export All SEL Ins/upd/d I/U/D INS el Yes Yes No Yes Yes Yes No Yes Yes Yes Yes Yes FastLoa EXPORT MultiLoad TPump d No Yes Yes Yes No Yes No Yes Yes Yes Limited Yes Yes Yes Yes No No Yes Yes Yes Yes Yes Yes Yes .
What happens if the host fails? Multiple sessions .Do you have control over error handling? Does the utility support INMODs? .What happens with a Teradata restart? .How do you choose the optimum number? Error handling .Is there a restart log? .Client Tools 109 Choosing The Best Method Utility support .Are errors captured in an error file? .Does the utility support multiple sessions? .Does it run on your host? Restart capability .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.