Professional Documents
Culture Documents
Welcome To The World of "Cache"
Welcome To The World of "Cache"
1)
2)
Index Cache: Server stores key values or condition values used to index values at a faster rate. Data Cache: Server stores output values.
For Index Caches: a) Aggregators store group by values from Group-By ports. b) Rankers store Group-By values c) Joiners store index values for the master (Join condition columns) d) Lookups Stores lookup condition information
For Data Caches: a) Aggregators store aggregate data based on Group-By ports (variable ports, output ports, non group by ports) b) Rankers store ranking based on Group-By port (output rows other than ranked column) c) Joiners store master table (Output columns not in Join condition). d) Look ups Stores stores lookup data that is not stored in the index cache.
Memory Cache :
The server creates a memory cache based on size specified in the session properties which can be done manually based on certain calculations .
By default, the PowerCenter Server allocates 1 GB to the index cache and 2GB to the data cache for each transformation instance. If the PowerCenter Server cannot allocate the configured amount of cache memory, it cannot initialize the session and the session fails. If the PowerCenter Server requires more memory than the configured cache size, it pages to the Disc. Since paging to disk can slow session performance, try to configure the index and data cache sizes to store data in memory.
Three Instances when the Cache File exists even after Session completion: a) The session performs incremental aggregation. b) You configure the Lookup transformation to use a persistent cache. c) The session does not complete successfully.
PMLKUP8_4_2.idx,
PMLKUP transformation type as Lookup, 8 the session ID 4 the transformation ID, 2 the partition index.
Description
Name Prefix
Prefix
Suffix
Overflow Index
Cache Calculations
Aggregator: Index size: (Sum of column sizes in group-by ports + 17) X number of groups. Data Size: (Sum of column sizes of output ports + 7) X number of groups. Rank: Index size: (Sum of column sizes in group-by ports + 17) X number of groups. Data Size: (Sum of column sizes of output ports + 10) X number of groups + 20.
Joiner: Index Size: (Sum of master column sizes in join condition + 16) X number rows in master table. Data Size: (Sum of master column sizes NOT in join condition but on output ports + 8)X number of rows in master table LookUp: Index Size: # rows in lookup table [( S column size) + 16] * 2 Data Size: # rows in lookup table [( S column size) + 8]
Datatype
Binary Date/Time Decimal, high precision off (all precision) Decimal, high precision on (precision <=18) Decimal, high precision on (precision >18, <=28) Decimal, high precision on (precision >28) Decimal, high precision on (negative scale) Double Real Integer String Small integer
Aggregator, Rank
precision + 2 18 10 18 22 10 10 10 10 6
ASCII mode: precision + 3
Joiner, Lookup
precision + 8 Round to nearest multiple of 8 24 16 24 32 16 16 16 16 16
ASCII mode: precision + 9
16
Dynamic cache :
U can not insert or update the cache.
The informatica server inserts rows into cache when the condition is false.This indicates that the the row is not in the cache or target table. U can pass these rows to the target table
You can use a relational look up only
Example: The Lookup transformation, LKP_PROMOS, looks up values based on the ITEM_ID. It uses the following lookup condition: ITEM_ID = IN_ITEM_ID1 ITEM_ID column size Column in lookup condition integer = 16
The lookup condition uses one column, ITEM_ID, and the table contains 60,000 rows. Use the following calculation to determine the minimum index cache requirements: 200 * (16 + 16) = 6,400 Use the following calculation to determine the maximum index cache requirements: 60,000 * (16 + 16) * 2 = 3,840,000 Therefore, this Lookup transformation requires an index cache size between 6,400 and 3,840,000 bytes.
Aggregator Cache
When the PowerCenter Server runs a session with an Aggregator transformation, it stores data in memory until it completes the aggregation. If you use incremental aggregation, the PowerCenter Server saves the cache files in the cache file directory. Note: The PowerCenter Server uses memory to process an Aggregator transformation with sorted ports. It does not use cache memory. You do not need to configure cache memory for Aggregator transformations that use sorted ports.
Joiner Cache
While using joiner cache informatica server first reads the data from master source and built index & data cache in the master rows. After building the cache,the PowerCenter Server then performs the join based on the detail source data and the cache data. Server creates the Index cache as it reads the master source into the data cache. The server uses the Index cache to test the join condition. When it finds a match, it retrieves rows values from the data cache The PowerCenter Server caches all master rows with a unique key in the index cache, and all master rows in the data cache. For instance, Index cache. The PowerCenter Server caches 100 master rows with unique keys. Data cache. The PowerCenter Server caches the master rows in the data cache that correspond to the 100 rows in the index cache. The number of rows it stores in the data cache depends on the data. For example, if every master row contains a unique key, the PowerCenter Server stores 100 rows in the data cache. However, if the master data contains multiple rows with the same key, the PowerCenter Server stores more than 100 rows in the data cache.
ITEM_NAME string 32
PRODUCT CATEGORY decimal 30 Total column size = 62 The master source has 90,000 rows. Use the following calculation to determine the minimum data cache requirements: 90,000 * (62 + 8) = 6,300,000 This Joiner transformation requires a data cache size of 6,300,000 bytes.
Rank Caches
When the PowerCenter Server runs a session with a Rank transformation, it compares an input row with rows in the data cache. If the input row out-ranks a stored row, the PowerCenter Server replaces the stored row with the input row. For example, you configure a Rank transformation to find the top three sales. The PowerCenter Server reads the following input data: SALES 10,000 12,210 5,000 2,455 6,324 The PowerCenter Server caches the first three rows (10,000, 12,210, and 5,000). When the PowerCenter Server reads the next row (2,455) it compares it to the cache values. Since the row is lower in rank than the cached rows, it discards the row with 2,455. The next row (6,324), however, is higher in rank than one of the cached rows. Therefore, the PowerCenter Server replaces the cached row with the higher-ranked input row. If the Rank transformation is configured to rank across multiple groups, the PowerCenter Server ranks incrementally for each group it finds.