You are on page 1of 10

Implementing Teradata Utilities in Informatica

By

Chandru Samuthiraraj (chandru.samuthiraraj@wipro.com) Oct 2009

..............................10 Confidential Page 2 .............................Implementing Teradata Utilities in Informatica Table of Contents 1............ )oad!r )o% (i#!...........................................................................................................................a#id tim!stamp....................................................................................................* +....................! 3u!ry on th! 4rror ta2#! shows that 115th row ........................ 1ata #oad and "sa%! o.......................................................rom th! sourc! %ot r!j!ct!d 2!caus! o..........................................................................................................................................0 h! a2o..................................................... !radata "ti#iti!s... In....... $appin% that us!s &ump'(ast)oad...............or d!cidin% )oad!r yp!......... S!ssion #!..... Introduction....!# &rop!rti!s....................................... Sh!## Script ................................0 -..........10 /...................... Conc#usion..........................................................*.../ .....................3 3.................................3 2...........................

.  &ara##!# !. th! num2!r o. his docum!nt !8p#ains how to imp#!m!nt !radata uti#iti!s in In.o#um!6 2r!adth6 num2!r o. an !8istin% syst!m #in!ar#y incr!as!s th! p!r. us!rs.0.!rs.-.  &!r.. a data2as! syst!m wor7#oad (data .ormatica 0.!rnanc!6 mast!r data mana%!m!nt6 and cross9!nt!rpris! data int!%ration.!rs c!rtain uti#iti!s that assists in data war!housin% mana%!m!nt and maint!nanc! a#on% with th! !radata <1B$S.  "nconditiona# para##!#ism6 with #oad distri2ution shar!d amon% s!.ort .ormanc!. proc!ssors o. Som! o.ormanc! thus do!s not d!t!riorat! with an incr!as!d num2!r o. h! !radata 1B$S is #in!ar#y and pr!dicta2#y sca#a2#! in a## dim!nsions o.ormatica 0.-.  Comp#!8 ad hoc 3u!ri!s with up to -+ joins. th!m ar!  (ast)oad  (ast48port  &ump  $u#ti)oad FastLoad: Confidential Page 3 .0 supports .!ra# s!r.arious !nt!rpris!9wid! data int!%ration and data 3ua#ity so#utions inc#udin% data war!housin%6 data mi%ration6 data conso#idation6 data synchroni:ation6 data %o. Introduction In som! o. !radata acts as a sin%#! data stor! that can acc!pt #ar%! num2!rs o. !radata is a massi.or cr!atin% 1006000 r!cords.  Sca#a2i#ity6 so that incr!asin% o. Teradata Utilities !radata o.!#y para##!# proc!ssin% syst!m runnin% a shar!d nothin% archit!ctur!. Si%ni.!atur!s inc#ud!.or cr!atin% 100 r!cords is sam! as that . us!rs6 comp#!8ity o. concurr!nt r!3u!sts .Implementing Teradata Utilities in Informatica 1.ici!ncy6 such that th! !.icant . In. 2. th! 4 ) oo#s6 Imp#!m!ntin% !radata uti#iti!s #i7! &ump6 (ast #oad !tc is a cha##!n%!.rom mu#tip#! c#i!nt app#ications. 3u!ri!s)..

!m!nt6 and #oadin% ar! automatic and p!r.!rsion6 mo.  1!#imit!d or . data .i!w to which you ha.!r #ar%! amounts o. Confidential Page 4 . th! !radata <!#ationa# 1ata2as! $ana%!m!nt Syst!m (<1B$S) to a c#i!nt92as!d app#ication. h! d!stination .i!ws o.i#! on your chann!#9attach!d or n!twor79attach!d c#i!nt syst!m  >n Output $odi.  h! tar%!t ta2#! must 2! !mpty.rom any ta2#! or .! data .  > .! th! S4)4C acc!ss pri.rom data sourc!s into !radata ta2#!s without #oc7in% th! a.!n uti#ity that us!s mu#tip#! s!ssions to 3uic7#y trans.ormanc! #!.o#um!s.o#um!s o.  =o acc!ss (r!ad or writ!) is a##ow!d on th! tar%!t ta2#! whi#! #oadin%.i8!d .ication (O" $O1) routin! you writ! to s!#!ct6 .or th! !8port!d data can 2!. FastExport: (ast48port is a command9dri.. Constraints.rom data sourc!s on chann!# and n!twor79attach!d c#i!nts into !mpty ta2#!s in th! !radata 1ata2as!. data co##!ct!d .or hu%! data ..!ct!d ta2#!.rom ta2#!s and .  Initia# ta2#! #oad.i#!%!. !8port can 2! don! . Features.! #ar%! .  On#y a sin%#! ta2#! can 2! #oad!d in on! jo2.(ast)oad typica##y o.  B!st .orm!d in para##!#.Implementing Teradata Utilities in Informatica his para##!# #oad uti#ity is d!si%n!d to mo.!#s than a standard app#ication pro%ram writt!n to #oad data to an !mpty data2as! 2!caus! data a##ocation6 con.ormat input.!rs hi%h!r p!r.a#idat!6 and pr!proc!ss th! !8port!d data TPump: &ump is a hi%h#y para##!# uti#ity d!si%n!d to continuous#y mo.

or hi%h9sp!!d 2atch cr!ation and maint!nanc! o. Som! In this mappin% Sourc! is a .i.or up9to9th!9mom!nt d!cision9ma7in%.r!sh6 accurat! data . It Supports up to .!r!ntia# int!%rity6 constraints @ B! popu#at!d or !mpty.or hi%h9.  >##ows tar%!t ta2#!s to. apping t!at uses TPump"FastLoad ar%!t is a !radata ta2#!.orms 2#oc7 #!.or sma## data .!rt Strin% to dat!.i#! and 48pr!ssions ar! us!d to Con.!# op!rations a%ainst popu#at!d ta2#!s 3. ultiLoad: $u#ti)oad is a command9dri.i!ws o.!n para##!# #oad uti#ity . @ Aa. #ar%! data2as!s.  B!st . ?ou can us! &ump to ins!rt6 updat!6 ups!rt6 and d!#!t! data in th! !radata 1ata2as!.! s!condary ind!8!s6 r!.id!s n!ar9r!a#9tim! data into your data war!hous!6 a##owin% you to maintain . It is sp!cia##y d!si%n!d .! popu#at!d ta2#!s and &!r.o#um! 2atch maint!nanc! on mu#tip#! ta2#!s and . th! !radata 1ata2as!.Implementing Teradata Utilities in Informatica &ump pro. Features:  (ast6 sca#a2#! continuous data #oads.o#um!s.#at . Confidential Page 5 .

o#um!.!d in this .!r th! .or sma## data . h! B!#ow Sh!## Script can 2! us!d .rom som! !n. hat is6 wh!n!. th! count !8c!!ds *00006 th! Script wi## s!t th! )oad!r Conn!ction as (ast)oad or !#s! th! Conn!ction wi## 2! &ump. Bas!d on th! (!atur!s o.rom which th! mappin% is %oin% to run. $!ell $cript for deciding Loader T%pe (or #oadin% a ta2#! in !radata w! can us! &ump or (ast)oad "ti#ity o. h! Script can 2! ca##!d . h!r! ar! .i#!.i. S4SSIO=C=>$4  h! s!ssion .or &ump. !radata.rom a Command s!ssion or you can s!t th! &r!9 s!ssion9command in your s!ssion. LoaderConnection_S cript. th!s! "ti#iti!s6 B! can say &ump is suita2#! .Implementing Teradata Utilities in Informatica #.or (ast)oad.ironm!nt6 th! Script wi## ta7! th! Count o. )<(C=>$4  h! Input . &><>$C(I)4  )oad!r Conn!ction typ! wi## 2! sa. &"$&CCO==4C IO=  h! nam! o.i#! %ot %!n!rat!d . ()O>1CCO==4C IO=  h! nam! o. Confidential Page 6 . rows and i.or d!.#at . th! <!#ationa# Conn!ction .inin% th! )oad!r Conn!ction in <untim!.#at .txt In th! a2o. th! <!#ationa# Conn!ction .or hu%! data .! Script th! )oad #imit is d!c#ar!d as *0000.! Input &aram!t!rs ar! us!d in th! Script.i#! nam!.o#um! and (ast)oad is suita2#! .

CSC>SS4 .i#! and s!ttin% th! #oad!r conn!ction accordin% to th! row count.c'IS&C=> IE4'w.Implementing Teradata Utilities in Informatica Presession_Cm d_Scri pt. D&$<oot1ir' %t(i#!s's.CSECCIS&C=> IE4C)<(CI=C.aria2#! usin% th! crC 1CconnCtyp!. his s!ssion is us!d .or th! sCmCIS&C=> IE4CSC>SS4 C)<(CI=C s!ssion wi## 2! stor!d dynamica##y 2y th! crC 1CconnCtyp!.i#!. $ession le'el Properties h! .i#! wi## 2! ca##!d . th! )oad!r Con!!ction d!cidin% script in which th! cod! is writt!n o. his param!t!r .! to 2! %i.7sh is th! dir!ctory and . &.i#! path o.c'IS&C=> IE4'#r.7sh script .i#! path o.txt In th! a2o. sCmCIS&C=> IE4CSC>SS4 C)<(CI=C is th! s!ssion nam!.! to 2! don! in th! s!ssion #!. th! &aram!t!r .!# &rop!rti!s s!ction.dat is th! dir!ctory and .!# prop!rti!s ha.!n in th! &rop!rti!s &an!#.i#! path o.rom th! sCmCIS&C=> IE4CSC>SS4 C)<(CI=C s!ssion to %!t th! #oad!r conn!ction usin% th! #oad!r conn!ction .aria2#! D)oad!rConn!ctionCsCmCIS&C=> IE4CSC>SS4 C)<(CI=C.parm param!t!r . $tep1: h! path and th! nam! o.or th! sCmCIS&C=> IE4CSC>SS4 C)<(CI=C s!ssion in th! w.o##owin% st!ps ha.or !8!cutin% th! mappin% and th! s!ssion #!.i#!.! to 2! s!t in this s!ssion as m!ntion!d 2!#ow in th! S!ssion #!.7sh )oad!r Con!!ction d!cidin% script. th!s! conn!ctions wi## 2! s!t a%ainst th! D)oad!rConn!ctionCsCmCIS&C=> IE4CSC>SS4 C)<(CI=C .i#! ha.! script6 D&$<oot1ir'Script(i#!s'data#oad'crC 1CconnCtyp!.!#.CSECCIS&C=> IE4C)<(CI=C. &"$&CS4<EIC4CIS&C4 )C01 and ()O>1CS4<EIC4CIS&C4 )C01 ar! th! &ump and (#oad conn!ctions. Confidential Page 7 .parm is th! dir!ctory and . th! .#at . ca#cu#atin% th! num2!r o.i#! in which th! #oad!r conn!ction . On! o. r!cords in th! . D&$<oot1ir'BB&aram's.#at . th! &aram!t!r .

i#! nam! and th! dir!ctory path ha. his )oad!r Conn!ction .i#! Bi## 2! %!t %!n!rat!d on th! sam! path o. h! param!t!r .#at . $tep(: In th! ar%!t &rop!rti!s th! Conn!ction typ! has to 2! F)oad!r5 and th! .a#u! wi## 2! .i#!. *ata load and Usage of Loader Log File >. th! $appin% &an!#.!ry !radata #oad6 On! )oad!r #o% .a#u! shou#d 2! th! )oad!rConn!ctionCGS!ssion nam!H.Implementing Teradata Utilities in Informatica $tep2: h! Sourc! . (or !. Confidential Page 8 .!n in th! Sourc! attri2ut! prop!rti!s o.!tch!d . th! Sourc! .a#u! dynamica##y usin% th! Sh!## Script ).t!r s!ttin% a## th! a2o.rom th! param!t!r .i#! stor!s th! conn!ction .! prop!rti!s th! s!ssion can 2! run and th! data wi## %!t #oad!d into th! !radata tar%!t ta2#!.i#!.! to 2! %i.

rows r!ad .rom th! sourc! and 0 rows #oad!d into th! !rror ta2#!.!ry #oad. 4rror ta2#! nam! is 4<<O<C . Example Loader Log File: B!#ow is th! !8amp#! #oad!r #o% .or th! r!j!ction wi## 2! stor!d in a co#umn o. rows #oad!d into th! tar%!t and th! num2!r o.Implementing Teradata Utilities in Informatica )oad!r #o% . th! !rror ta2#!. rows #oad!d into th! 4rror ta2#!. !radata automatica##y %!n!rat!s an 4rror ta2#! .4 CSC>SS4 0 Example Error Table: Confidential Page 9 .or !.rom th! sourc!6 num2!r o.i#! can us!d to .!tch!d .i#! which shows that 20000 rows ar! .y th! num2!r o. h! r!cords which %ot r!j!ct!d on th! #oad wi## 2! #oad!d in to th! !rror ta2#! and th! r!ason .!ri.

rom th! sourc! %ot r!j!ct!d 2!caus! o.ocus!s on imp#!m!ntin% th! !radata #oad uti#iti!s #i7! &ump and (ast)oad usin% In.Implementing Teradata Utilities in Informatica h! a2o.ormatica and th! usa%! o. Confidential Page 10 .! 3u!ry on th! 4rror ta2#! shows that 115th row . Conclusion A!nc! this docum!nt c#!ar#y !8p#ains th! !radata uti#iti!s and main#y .i#! and th! 4rror ta2#!s. +. )oad!r #o% . In.a#id tim!stamp.