Teradata Utilities: FastExport

Reprinted for KV Satish Kumar, IBM kvskumar@in.ibm.com Reprinted with permission as a subscription benefit of Books24x7, http://www.books24x7.com/

................................................................3 Task Commands...............7 Formats........................................i Table of Contents Chapter 6: FastExport..........................2 Maximum of 15 Loads ..................3 Support Environment Commands................................................................................................................................3 FastExport Support and Task Commands.......................................................................................................................................................................................................................5 SQL Commands ..............................................................8 A FastExport Script using Binary Mode.......................................5 A FastExport in its Simplest Form ............................................................................................................................................................................2 FastExport Supported Operating Systems ........................8 .1 Why it is Called "FAST" Export............................................................................................................5 Sample FastExport Script............................................................................................................................................................7 Modes.........................................................1 FastExport Fundamentals...................................................................................................1 How FastExport Works..............................................6 FastExport Modes and Formats ................................................................................................................4 FastExport Supported SQL Commands.............................................................................................................................................................................................................................................................................................................

when FastExport starts sending the rows back a block at a time. if FastExport terminates abnormally.Chapter 6: FastExport "An invasion of armies can be resisted. From there. Coffing Publishing (c) 2005. we make the products you buy better".com. The other advantage is that if BTEQ terminates abnormally. but it does take the SQL SELECT statement and processes the request with lighting fast parallel processing! Reprinted for ibmkvskumar@in. the ever-growing requirement for tools to export massive amounts of data also increases. FastExport may be required to redistribute the selected data two times across the AMP processors in order to build the blocks in the correct sequence. Copying Prohibited .ibm. which provides a job restart capability from a checkpoint if an error occurs during the process of executing an export job. an OUTMOD routine can be written to modify the result set after it is sent back to the client on either the mainframe or LAN. FastExport has the ability to use OUTMOD routines. Part of this speed is achieved because FastExport takes full advantage of Teradata's parallelism. FastExport is extremely attractive for exporting data because it takes full advantage of multiple sessions. How FastExport Works When FastExport is invoked. which leverages Teradata parallelism. the sole purpose of FastExport is to move data out of Teradata. validate. BTEQ continues to send rows. FastExport can also export from multiple tables during a single operation." . we have already discovered how BTEQ can be utilized to export data from Teradata in a variety of formats. which provide the user the capability to write. BTEQ starts sending rows immediately for storage into a file. it must build blocks to send back to the client. a lot of rows fit into a 64K block and both the rows and the blocks must be sequenced. You must rerun the BTEQ script from the beginning. This is the reason why FastExport (FEXP) is brilliant by design. and preprocess the exported data. While all of this redistribution is occurring. However.Victor Hugo Why it is Called "FAST" Export FastExport is known for its lightning speed when it comes to exporting vast amounts of data from Teradata and transferring the data into flat files on either a mainframe or network-attached computer. but not an idea whose time has come. it quickly overtakes and passes BTEQ's row at time processing. Fastexport will take up 4 sessions that will be running on the local computer. FastExport is getting behind in the processing. if there is a requirement to manipulate the data before storing it on the computer's hard drive. It does this by harnessing the parallelism that Teradata provides. In comparison. FastExport is designed off the same premise. However. then FastExport is the best choice to accomplish this task. In this book. Remember. In addition. all of your rows (which are in SPOOL) are discarded. IBM Coffing Data Warehousing. Keep in mind that FastExport is designed as a one-way utility — that is. the utility logs onto the Teradata database and retrieves the rows that are specified in the SELECT statement and puts them into SPOOL. "We don't make the products you buy. it does not make the SQL SELECT statement faster. If the output data is sorted. select. Just like the BASF commercial states. FastExport utilizes the Support Environment. A good rule of thumb is that if you have more than half a million rows of data to export to either a flat file format or with NULL indicators. all the selected rows are in worktables and it can continue sending them where it left off in a very smart and very fast manner! Also. As the demand increases to store data. By default. In addition.

FastLoad. IBM Coffing Data Warehousing. The FastExport utility will terminate after a certain number of errors have been encountered. TPump. #2: FastExport only supports the SELECT statement. like BTEQ it can output multiple files in a single run. FastExports. validate and preprocess the exported data FastExport Supported Operating Systems The FastExport utility is supported on either the mainframe or on LAN.Teradata Utilities: BTEQ. and conversions. The information below illustrates which operating systems are supported for each environment: The LAN environment supports the following Operating Systems: • UNIX MP-RAS • Windows 2000 • Windows 95/98/XP • Windows NT/2000 • UNIX HP-UX • AIX • Solaris SPARC • Solaris Intel Reprinted for ibmkvskumar@in. FastExport is recommended over BTEQ Export. which is limited to 15. Copying Prohibited . FastExport is flexible and supports the above conditions. Coffing Publishing (c) 2005. arithmetic calculations. MultiLoad. #7: FastExport supports user-written routines INMODs and OUTMODs. The reason they call it FastExport is because it takes data off of Teradata (Exports Data). #3: Choose FastExport over BTEQ when Exporting Data of more than half a million+ rows. #6: FastExport does NOT support error files or error limits. FastExport allows you write INMOD and OUTMOD routines so you can select.com. FastExport does not record particular error types in a table. FastExport does not import data into Teradata. You can have multiple SELECT statements with FastExport and each SELECT can join information up to 64 tables. You SELECT the data you want exported and FastExport will take care of the rest. and MultiLoads that can run at the same time. Second Edition 2 FastExport Fundamentals #1: FastExport EXPORTS data from Teradata. and FastExport. Additionally.ibm. When a large amount of data is being exported. #5: FastExport supports conditional logic. FastExport will work with less data. #4: FastExport supports multiple SELECT statements and multiple tables in a single run. Of course. calculations. and data conversions. but the speed may not be much faster than BTEQ. conditional expressions. The only drawback is the total number of FastLoads. BTEQ Export does not have this restriction. The only DML statement that FastExport understands is SELECT.

this value is set at 5. Second Edition 3 The Mainframe (Channel Attached) environment supports the following Operating Systems: • MVS • VM Maximum of 15 Loads The Teradata RDBMS will only support a maximum of 15 simultaneous FastLoad. However. For example. Teradata does an excellent job of protecting system resources by queuing up additional FastLoad. if two many load jobs are running. and FastExport jobs that are attempting to connect. If more then 15 simultaneous jobs were supported.Teradata Utilities: BTEQ.ibm. MultiLoad. This value can be set from 0 to 15. The FastExport commands can be broken down into support and task activities. These commands provide flexibility and control during the export process. a saturation point could be reached on the availability of resources. The reason for this limitation is that FastLoad. FastExport is clearly the better choice when exporting data. and FastExport. BTEQ does not have this load limitation. then there can be only a total of fifteen of them running at any one time". Copying Prohibited . or FastExport utility jobs. TPump. The table below highlights the key FastExport commands and their definitions. that job will not start. This maximum value is determined and configured in the DBS Control record. When Teradata is initially installed. IBM Coffing Data Warehousing. A tip for remembering how the load limit applies is this. Support Environment Commands (see Support Environment chapter for details) Reprinted for ibmkvskumar@in. FastLoad. Coffing Publishing (c) 2005. MultiLoad. and FastExport all use large blocks to transfer data. MultiLoad. In this case. BTEQ is an alternate choice for exporting data. if the maximum number of utilities on the Teradata system is reached and another job attempts to run.com. MultiLoad. "If the name of the load utility contains either the word "Fast" or the word "Load". This limitation should be viewed as a safety control feature. FastExport Support and Task Commands FastExport accepts both FastExport commands and a subset of SQL statements.

Specifies the data layout for a file. Disconnects all FastExport active sessions and terminates FastExport. MultiLoad. Specifies a field in the input record that will not be sent to Teradata for processing. Provides two things which are: • The client destination and file format specifications for the export data retrieved from Teradata • A generated MultiLoad script file that can be used later to reload the export data back into Teradata FIELD FILLER IMPORT LAYOUT Constitutes a field in the input record section that provides data values for the SELECT statement. Used in conjunction with the IF or ELSE statements. FastLoad. Introduces a conditional expression. Figure 3-2 Reprinted for ibmkvskumar@in. If true then execution of subsequent commands will happen. Defines the file that provides the USING data values for the SELECT. Ends the export task and initiates processing by Teradata.com. This will Invoke the specified external file as the current source of utility and Teradata SQL commands. FastExport utilizes this to specify a restart log table. and FastExport.Teradata Utilities: BTEQ. Specifies the style of the DATE data types for FastExport. LOGON command or string used to connect sessions established through the FastExport utility. ELSE commands and statements will execute when a proceeding IF condition is false. Copying Prohibited . Used in conjunction with the IF statement. It is part of the input record to provide data values for the SELECT statement. The purpose is for FastExport checkpoint information. Suspends the FastExport utility temporarily and executes any valid local operating system command before returning.ibm. Assigns a data type and value to a variable. Second Edition 4 ACCEPT DATEFORM DISPLAY ELSE ENDIF IF LOGOFF LOGON LOGTABLE ROUTE MESSAGES RUN FILE SET SYSTEM Allows the value of utility variables to be accepted directly from a file or from environmental variables. This is used to describe the import file that can optionally provide data values for the SELECT. IBM Coffing Data Warehousing. Figure 3-1 Task Commands BEGIN EXPORT END EXPORT EXPORT Begins the export task and sets the specifications for the number of sessions with Teradata. Writes messages to the specific location. It contains a sequence of FIELD and FILLER commands. Delimits the commands that were subject to previous IF or ELSE conditions. Coffing Publishing (c) 2005. TPump. Used to point to a file that FastExport is to use as standard input. Will route FastExport messages to an alternate destination.

view. Copying Prohibited . FastLoad. COMMENT Store or retrieve a comment string for a particular object. REVOKE Revoke privileges to an object. RENAME Change the name of a table. challenging. GIVE Transfer ownership of a database or user to another user. and FastExport. Second Edition 5 FastExport Supported SQL Commands FastExport accepts the following Teradata SQL statements. With this being stated. UPDATE Change a column value of an existing row or rows in a table. CREATE MACRO Creates a new macro.com. REPLACE VIEW Change a view. REPLACE MACRO Change a macro. CREATE DATABASE Creates a new database. MODIFY DATABASE Change the options for a database. CHECKPOINT Add a checkpoint entry in the journal table. I always remember my driving instructor coaching me during a practice session in a new car around a road course racetrack. DELETE Delete rows from a table. and rewarding all at the same time. Figure 3-3 A FastExport in its Simplest Form The hobby of racecar driving can be extremely frustrating. macros. CREATE VIEW Creates a new view. SQL Commands ALTER TABLE Change a column or table options of a table. or macro. If FastExport is broken into steps. GRANT Grant access privileges to an object. Each has been placed in alphabetic order for your convenience. FastExport can be broken into the following steps: • Logging onto Teradata • Retrieves the rows you specify in your SELECT statement Reprinted for ibmkvskumar@in.ibm. views. then several things that appear to be complicated are really very simple. DELETE DATABASE Removes all tables." This same philosophy can be applied when working with FastExport.Teradata Utilities: BTEQ. IBM Coffing Data Warehousing. Coffing Publishing (c) 2005. He said to me. COLLECT STATISTICS Collect statistics for one or more columns or indexes in a table. TPump. DROP DATABASE Drops a database. "Before you can learn to run. SET SESSION Override the collation specification during the current COLLATION session. and stored procedures from a database. you need to learn how to walk. CREATE TABLE Creates a new table. MultiLoad. DATABASE Specify a default database for the session.

/* Get Data From the Student Table */ SELECT Student_ID (CHAR(11)). FastLoad. .Student_Table.END EXPORT.BEGIN EXPORT SESSIONS 12. . In addition.LOGTABLE sql01. Figure 3-4 Sample FastExport Script Now that the first steps have been taken to understand FastExport. Copying Prohibited . In the script below. Another note is that the column names are listed vertically.Teradata Utilities: BTEQ. Coding this way makes reading or debugging the script easier to accomplish. . This will simplify the importing process into a different database.LOGON CDW/sql01. . FastExport and SQL commands are written in upper case in order to highlight them. . Teradata comment lines have been placed inside the script [/*. NOTE: The selected columns for the export are being converted to character types.ONLY) The SELECT defines the columns used to create the exported file.ibm. The recommendation is to place the comma separator in front of the following column. Begin the Export and set the number of sessions on Teradata Defines the output file name. Reprinted for ibmkvskumar@in. specifies the output mode and format (LAN . First_name (CHAR(14)). */].com.LOGOFF. .whynot.EXPORT OUTFILE Student. End the Export and logoff /* Finish the Export Job and Write to File */ Teradata. In addition. the next step is to journey forward and review another example that shows builds upon what we have learned. TPump. . Logon to Teradata . and FastExport. Grade_Pt (CHAR(9)) FROM SQL_CLASS.txt MODE RECORD FORMAT TEXT. Coffing Publishing (c) 2005. Second Edition 6 • Exports the data to the specified file or OUTMOD routine • Logs off of Teradata /* Created by CoffingDW */ /* Setup the Fast Export Parameters */ Creates the logtable -Required . MultiLoad.SWA_Log. Class_code (CHAR(2)). Last_name (CHAR(20)). IBM Coffing Data Warehousing.

/* Join Data From the Employee and Department Table */ SELECT EMP.Dept_name (CHAR(20)) FROM SQL_CLASS. END THE JOB AND LOGOFF TERADATA. THE SELECT PULLS DATA FROM TWO TABLES.ONLY) MODE RECORD FORMAT TEXT. BEGIN EXPORT STATEMENT. The difference between the two modes is INDICATOR mode will set the indicator bits to 1 for column values containing NULLS.Dept_No (CHAR(6)) . only use RECORD mode.LOGTABLE SQL01.EMP. /* Setup the Fast Export Parameters */ . IT IS GOOD TO QUALILY WHEN DOING A TWO-TABLE JOIN.Department_Table AS DEPT ON EMP. .Teradata Utilities: BTEQ. Copying Prohibited .Employee_table AS EMP INNER JOIN SQL_CLASS.txt MODE RECORD FORMAT TEXT.ibm. IBM Coffing Data Warehousing.whynot. SESSIONS 12. .Dept_No = DEPT.CDW_Log. In the UNIX or LAN environment.Employee_No (CHAR(11)) . TPump. but you can use RECORD mode if desired.com. Reprinted for ibmkvskumar@in. Second Edition 7 /* ---------------------------------------------------*/ /*@(#) FASTEXPORT SCRIPT */ /*@(#) Version 1.END EXPORT. Figure 3-5 FastExport Modes and Formats Modes FastExport has two modes: RECORD or INDICATOR. DEFINES THE OUTPUT FILE NAME. SPECIFIES THE OUTPUT MODE AND FORMAT(LAN .1 /*@(#) Created by CoffingDW */ /* ---------------------------------------------------*/ ALWAYS GOOD TO IDENTIFY THE */ SCRIPT AND AUTHOR IN COMMENTS CREATE LOGTABLE AND LOGON.First_Name (CHAR(14)) .DEPT.Dept_No . FastLoad.Last_Name (CHAR(20)) . and FastExport.EXPORT OUTFILE Join_Export. /* Finish the Export Job and Write to File */ .EMP. IN ADDITION.LOGOFF. In the mainframe world. . . Coffing Publishing (c) 2005. MultiLoad. INDICATOR mode is the default.LOGON CDW/SQL01.DEPT.BEGIN EXPORT SESSIONS 12.

MultiLoad. Coffing Data Warehousing. and FastExport. . . NAME THE OUTPUT FILE AND SET THE FORMAT TO BINARY. It is called FASTLOAD because the data is exported in a format ready for FASTLOAD. /* Get Data From the Student Table */ SELECT Student_ID */ */ */ */ */ CREATE LOGTABLE AND LOGON TO TERADATA.SWA_LOG. Coffing Publishing (c) 2005. A FastExport Script using Binary Mode COMMENTS /*-------------------------------------------------/*@(#)FASTEXPORT SCRIPT .whynot.LOGON CDW/sql01. Formats FastExport has many possible formats in the UNIX or LAN environment. INDICATOR mode will set bit flags that identify the columns that have a null value.LOGTABLE SQL01. followed by an end-of-record marker. Second Edition 8 Both modes return data in a client internal format with variable-length records. FASTLOAD format has a two-byte integer. Each individual record has a value for all of the columns specified by the SELECT statement. All variable-length columns are preceded by a two-byte control value indicating the length of the column data. followed by the data.Teradata Utilities: BTEQ. IBM .BEGIN EXPORT SESSIONS 12.1 /*@(#)Created by CoffingDW /*-------------------------------------------------/* Setup the Fast Export Parameters */ . . BEGIN EXPORT STATEMENT. Copying Prohibited (CHAR(11)). Reprinted for ibmkvskumar@in. The FORMAT statement specifies the format for each record being exported which are: • FASTLOAD • BINARY • TEXT • UNFORMAT The default FORMAT is FASTLOAD in a UNIX or LAN environment. BINARY format is a two-byte integer.txt FORMAT BINARY.SWA /*@(#)Version 1. followed by data. Remember. FastLoad.EXPORT OUTFILE Cdw_import.com. GET THE DATA FROM THE STUDENT TABLE.ibm. NULL columns have a value that is appropriate for the column data type. TEXT format is an arbitrary number of bytes followed by an end-of-record marker. UNFORMAT format is exactly as received from CLIv2 without any client modifications. TPump.

LOGOFF.Student_Table.ibm. Class_code (CHAR(2)). MultiLoad. Grade_Pt (CHAR(8)) FROM SQL_CLASS. Copying Prohibited . . and FastExport. Second Edition 9 Last_name (CHAR(20)).Teradata Utilities: BTEQ.com. /* Finish the Export Job and Write to File */ END THE JOB . IBM Coffing Data Warehousing. FastLoad. Figure 3-6 Reprinted for ibmkvskumar@in. First_name (CHAR(14)). Coffing Publishing (c) 2005. TPump.END EXPORT.

Sign up to vote on this title
UsefulNot useful