This action might not be possible to undo. Are you sure you want to continue?
Page 1 of 8
Toolbox for IT
Kn o w l e d g e S h a r i n g C o m m u n i t i e s
Share and compare information with 1.3 million professionals
Select a Community
Ask Question / Find
Business Intelligence Community
Tooling Around in the IBM InfoSphere
by Vincent McBurney (Deloitte Manager)
Blog Main / Archive / Invite Peers / Connect to this blog
/ Next Entry
DataStage 8 Tutorial: Using Range Lookups
Vincent McBurney (Deloitte Manager) posted 6/13/2007 | Comments (13)
Looking at the new range lookup functionality in DataStage 8. DataStage 8 comes with some range lookup functionality within the lookup stage, a feature that came in at number four in my My top ten features in DataStage Hawk. A field on an input link can be compared to two fields on a lookup link or vice versa using a between clause returning one or more rows from a lookup link. This post has an example of a range lookup using pictures and I have also filled in a wiki page explaining the steps of the different types of range lookups in HOWTO:Do a range lookup in DataStage 8 where you can add your own examples or fix the instructions. Part of my Wiki Wednesday series of using wiki entries to describe an aspect of data integration. DataStage has always performed joins very efficiently when there are exact key fields that match using the lookup, join or merge stage. Range lookups are more challenging as it's a less efficient way to join whether you are doing it in an ETL job or on a database. You can do a range lookup in DataStage 7 using a lookup stage and a filter stage, you can do it using a sparse lookup and you can do it by loading both tables into a database staging area and joining them in SQL. This tutorial shows how to do it in a single Lookup stage providing a much simpler design.
Since timekeeping in the company has always been rubbish they don’t currently track it. They do have employee records with a history of roles. salaries and locations over the years.toolbox. state and local holidays: http://it. I have created three lookup files . This can be done using a simple lookup design: The employee history is processed into a single file showing each change in the location and charge rate of each person.Australia had an unofficial public holiday after we won a boat race. Some are a one off . some are state based. Let's say someone with way too much time on their hands has decided they want to know how much money public holidays have cost the company over the years.DataStage 8 Tutorial: Using Range Lookups Page 2 of 8 I will use a data integration challenge to demonstrate a range lookup.com/blogs/infosphere/datastage-8-tutorial-using-range-lookups-16911 10/11/2008 . The history of public holidays over the years are manually loaded into a text lookup file with a different file for national. In these pictures I will show how to perform the lookups for just the national holidays .one for each type of public holiday. some are specific to a city and some to a region.the same process can be used to create rows for the other types as well. How would you use DataStage to tally the cost of public holidays? Answer The tricky part is that some public holidays are national.
we manually set the Start_Date and End_Date fields and choose the operators ">=" and "<=" to create a range check . Choosing Range for the date field lets us compare the field to two fields on the history stream. This returns multiple holidays from the lookup for the duration of that history .toolbox. Set the Type to Range and then doubleclick right next to it in the empty Expression field to bring up the range Express form shown here: The form will load the Holiday Date field on each row. In DataStage 8 we have a new "Key Type" field with values of Equality.every public holiday between the start and end dates are returned.DataStage 8 Tutorial: Using Range Lookups Page 3 of 8 In DataStage 7 we can only join "country" and "state".com/blogs/infosphere/datastage-8-tutorial-using-range-lookups-16911 10/11/2008 . we cannot join date because it needs to be a range lookup. Caseless and "Range". We then set up the rest of the lookup stage properties as per a normal lookup: http://it.
the steps described in the wiki entry for range lookups.toolbox. state and city holidays will bring in all the public holidays in scope. each using different Equality joins but the same Range join: The range lookup can also be used to compare a single value in the primary stream to two values on the lookup. Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way. Previous Entry / Next Entry http://it. Repeating this type of range lookup against a lookup file with annual. These rows can then be saved to a table or put through an aggregation stage and turned into statistics.DataStage 8 Tutorial: Using Range Lookups Page 4 of 8 Every single input row of employee history can return zero or more output holiday rows so we need to set the lookup properties to drop rows that have not holiday match and accept multiple rows where there is more than one match.com/blogs/infosphere/datastage-8-tutorial-using-range-lookups-16911 10/11/2008 .
6/18/2007 # Raja writes: Hi vincent. for small sources it gets loaded into RAM. Parallel jobs let you use almost anything as a source for lookup data . But I have a question that if we want to do range lookup from lookup files. will be looking at the new slowly changing dimension stage in http://it. lookup filesets.DataStage 8 Tutorial: Using Range Lookups Page 5 of 8 Comments (13) Raja writes: RSS for Comments 6/14/2007 # Good effort 6/14/2007 # Priyadarshi writes: I think "reference link range lookup" has a problem. database stages.sequential files. regards 6/15/2007 # ttteety writes: Good post Vincent. you have any post on scd implementation 6/18/2007 # Vincent McBurney writes: You read my mind! Yes. DataStage converts all of these sources into an internal data format on startup. but overall its a great feature added in Hawk. Can we do that in Range Lookup stage? Thanks 6/15/2007 # Vincent McBurney writes: Yes you can do range lookups against lookup files. Any of these lookup types can be the source of a range lookup. for larger sources it overflows into lookup files on the nodes.com/blogs/infosphere/datastage-8-tutorial-using-range-lookups-16911 10/11/2008 . Lookup filesets are handy in that they are already loaded onto the nodes and need little extra work on startup.toolbox. A nice effort by Vincent McBurney for sorting out the newly added features and describing them as well. datasets etc.
Thanks for all the effort. I took at "What's new in DataStage / QualityStage 8.toolbox. thanks for the good info. I can confirm what Priyadarshi mentioned above . i need some information for DATA STAGE. Can anybody let me know what are the major differences between Dtastage 7. and i need a routine to write and call that routine so how can i write the routine if possible give a small example so that i can get an idea. 8/11/2008 # priya writes: Hi.0 http://it.DataStage 8 Tutorial: Using Range Lookups Page 6 of 8 the next couple weeks and hope to have a post about it. 6/19/2007 # Raghu writes: Hi vincent.5 and 8. Thanks in advance 6/1/2008 # rama writes: Hi guys i am new for this field. 12/26/2007 # praveen writes: Hi all I am new to Datastage.in the current release the reference link range lookups don't work. please help me how to start this technology. Raghu 7/6/2007 # Lakshmi writes: Hi Vincent. Its well explained. regards Lakshmi 9/15/2007 # Andy Sorrell writes: Great Post (as usual) Vincent.0" class from IBM a couple of weeks ago and the instructor confirmed that it has a problem but they are working on it and it will be corrected "soon". Keep posting Thanks.com/blogs/infosphere/datastage-8-tutorial-using-range-lookups-16911 10/11/2008 .
or contain profanity or advertisements may be removed by the author or a community moderator. To uphold community standards.toolbox. c d e f g Connect to this blog to be notified of new entries. job hunters and partners User and Group security for the Information Server and DataStage 8 The 20 Hardest Oracle Interview Questions ever http://it. comments that are inflammatory. offensive. If not logged-in your post will not appear until approved by a community moderator. Tutorial and Certification Online Resources Related Groups New DWH dbase engine to replace Oracle 9i non-equality based lookup Related Wiki Articles Do a range lookup in DataStage 8 Data Warehouse Concepts More from this author Why Teradata NCR split is good news for staff.DataStage 8 Tutorial: Using Range Lookups Page 7 of 8 Thanks in Advance Priya 8/19/2008 # Vincent McBurney writes: @Priya. You are not logged in. Preview Submit Related Blogs DataStage 8 Tutorial: Using Range Lookups 40 DataStage Learning.com/blogs/infosphere/datastage-8-tutorial-using-range-lookups-16911 10/11/2008 . Please preview your message before posting. Have a look at my blog post "What's new in DataStage 8" and look for the DataStage wiki page on this site that has the changes introduced in each version of DataStage. Sign in to post unmoderated comments or join the community to create your free profile today! Name: (Will display on the site) E-mail: (Not displayed. No Spam) Lines break automatically.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.