You are on page 1of 14

Data Warehousing > Database

Partitioned Primary Indexes

Jerry Klindt
(updated by Paul Sinclair)
October 20, 2004

there were few viable opportunities for a Database Administrator (DBA) to structure the data warehouse in a manner that allowed such queries to avoid full-table scans. Another example is an application can result in partial-table scans instead that compares customer behavior in one of full-table scans with dramatic improve- geographic region to another region. The feature is flexible. A PPI allows a table to be partitioned on columns of interest while retaining the traditional use of the primary index (PI) for data distribution and efficient access when the PI values are specified in the query.Partitioned Primary Indexes Table of Contents Executive Summary 2 Introduction 2 Definitions and Basics 3 How Much Can PPI Improve Performance? 3 How PPI Solves the Business Problem – Example One 4 Can the First Example Be Improved Further? 8 A Second Example 9 Executive Summary Partitioned primary indexes. Starting with Teradata Database V2R5. Introduction Some common business queries generally require a full-table scan of a large table even though it’s predictable that a fairly small percentage of the rows will qualify. and delete operations. update. yet easy to use. and to improve A Final Example 10 Specifics of Defining a PPI Table 11 High-Level Partitioning Guidelines 13 High-Level Trade-off Considerations 13 Summary 14 the performance of high-volume insert. provide an opportunity to greatly improve performance of certain queries. ments in resource consumption and Prior to Teradata® Database V2R5. the DBA has a flexible and powerful EB-1889 > 1204 > PAGE 2 OF 14 tool to structure tables to allow automatic elapsed time (elapsed time decreases of 99% or more are possible). introduced in Teradata Database V2R5. using a table with several years of sales A carefully-chosen partitioning expression detail. Batch insert and update times may also be improved when the partitioning column is chosen to match the arrival pattern of the data (elapsed time decreases of 90% or more are possible). and is largely transparent to end users. . or to the same month of the previous year. That tool is the partitioned primary index (PPI). optimization of frequently used queries of this class. One example of such a query is a trend analysis application that compares current month sales to the previous month.

and by partition number first then PI hash for PPI tables. and specified in equality join terms.535. is simple and straight- PARTITION BY clause following the is not spooled in preparation for a merge forward. and extending to the data the table. As is true for all physical database design decisions. partitioning refers the first row belonging to the partition to the physical ordering of rows within (on each AMP). the PI value is hashed to distribute a row to a particular AMP in an identical fashion for PPI and non-PPI tables. queries that specify one or a few PI values and queries that perform joins on the PI columns. The Optimizer may choose a direct product join when all the partitioning columns are specified in equality join terms. The result join.Partitioned Primary Indexes The process for physically defining the The partitioning expression is specified The term direct merge join is used to partitioning expression. This means that best-case PPI queries can take less than 1/100 of one percent of the time they would take with a non-PPI table. EB-1889 > 1204 > PAGE 3 OF 14 The term direct product join is used to describe a join in which the table of interest is not spooled in preparation for a product join. Definitions and Basics Accessing a particular partition of a table means accessing a subset of the table beginning with the data block containing In the context of PPI. The best performance improvement occurs when . specifically. PRIMARY INDEX definition. How Much Can PPI Improve Performance? The performance gain depends on the number of partitions and the specific query being measured. based on query conditions and the partitioning expression. In the best case. and causes those partitions to be skipped. The ordering is automatically block containing the last row belonging provided by the database management to the partition. The columns referenced in the partitioning expression are called the partitioning columns. We will briefly discuss these trade-off considerations in subsequent sections. It’s beyond the scope of this paper to discuss the trade-off considerations at length. inclusive. More the partition). A PPI table physically is may be necessary to read one data block substantially the same as a non-PPI table to determine that there are no rows for except for the ordering of rows. Generally. The number of data software. The term partition elimination refers to an automatic optimization in which the Optimizer determines. the elapsed time reduction factor for a specific query against a single table can approach the reciprocal of the number of partitions in the table. Partitions that are skipped for a particular query are called eliminated partitions. that some partitions cannot contain qualifying rows. and is determined by a user- blocks will be zero if there are no rows specified expression called the partitioning belonging to that partition (although it expression. A partition number must be The objective of this paper is to provide between 1 and 65. the result indicates the partition number. the greatest benefit of a PPI table is obtained from partition elimination. The ordering of rows within a table is transparent to application developers. via the CREATE on the CREATE TABLE statement in a describe a join in which the table of interest TABLE statement. The Optimizer may choose a direct of the expression must be an integer value merge join when all columns of the PI are or a value that can be cast to integer. This paper gives some examples. there are trade-off considerations associated with each possible choice. realistic examples and actual performance the maximum number of partitions that comparisons using PPI and non-PPI can be defined for a table is 65.535. rows are ordered by PI hash for non-PPI tables. Within each AMP. therefore. but there are trade-off considerations involving queries with partitioning column conditions. solutions.

discuss the optimization opportunities a Each row contains. and may vary. Actual Performance Test Results there are many partitions with reasonably even distribution of rows among the partitions. but your results prior to Teradata Database V2R5. How PPI Solves the Business Problem – the First Example We start the discussion of when a PPI is months plus the current month-to-date. but a small percentage of transactions may be reported a few days after they occur. Current transactions are added to the table nightly using Teradata MultiLoad. Once per month. and the PPI column is the performance we stipulate a table and some processing for a PPI counterpart table. among other things. the transac- Our hypothetical company has a large sales table containing the details of each transaction for the previous 24 full EB-1889 > 1204 > PAGE 4 OF 14 tions are added on the date they occur. and partition elimination excludes all except one partition. For the first example. and the quantity sold. Most transac- Figure 1 shows the results of actual most appropriate by showing the differ- performance tests. The Baseline column ences between a PPI and non-PPI table is the performance for a non-PPI table. These tests are requirements.742 rows per second per node more than ten times faster MultiLoad insert a number of rows equal to 1% of the table size into one partition (of 200) with one NUSI defined on the table 841 rows per second per node 5666 rows per second per node more than six times faster Figure 1. an identifier for the sales agent. the transactions from the oldest month are deleted. . tion date. PPI provides. The number of transactions per month is roughly the same for all months. discuss the options available considered to be realistic. the product code for the item.Partitioned Primary Indexes Test Description Baseline PPI Improvement Select rows that have a specified value of the partitioning column (200 partitions with roughly the same number of rows each) 59 seconds one second 98% reduction in elapsed time Select a month of activity from one partition containing six months of data (11 years of data contained in 40 partitions of unequal size) 58 seconds two seconds 96% reduction in elapsed time Delete rows that have a specified value of the partitioning column (200 partitions of equal size) 239 seconds one second more than 99% reduction in elapsed time Update one column in each row that has a specified value of the partitioning column (200 partitions of equal size) 237 seconds three seconds 98% reduction in elapsed time MultiLoad insert a number of rows equal to 1% of the table size into one partition (of 200) 1394 rows per second per node (larger numbers are better 14. for a few examples. The rows are short.

The sales table is frequently EB-1889 > 1204 > PAGE 5 OF 14 DATE ‘2002-10-01’ AND DATE ‘2004-10-31’ EACH INTERVAL ‘1’ MONTH). had a need to speed up ad hoc With PPI. By converting the sales table into a table partitioned by transaction month. and select appropriate date ranges ending dates and the granularity of the and product code ranges. or CREATE TABLE PPI_SalesTable ( days of a month when a few of the transactions would be from the prior month. quantity_sold INTEGER. The need to partitioning. The for this example scenario. was rejected as being too complicated and non-PPI definition of this table. . The DBA then considered splitting the There are four major categories of queries against this table: > A modest number of short-running queries specify the PI values. know the appropriate table name (from the 25 different tables) would also apply calendar quarter or less. ance. to applications submitting short-running queries that specify the primary index. nightly load jobs. By adding a DBA considered creating a value-ordered PARTITION BY clause to the definition product_code CHAR(8). but it added too much complexity for the end users. and the agent identification. > Many ad hoc queries have the follow- table into 25 separate tables. usually over an interval of a No other tables have the same PI PARTITION BY RANGE_N (sales_date BETWEEN have to understand the structure and > Some queries analyze agent perform- > Some queries examine sales trends over agent_id CHAR(8). usually for This solution would also complicate most or all product code values. there’s an excellent solution queries and agent analysis queries. ing general pattern: another month. secondary index or join index on the of the replacement PPI table. sales_date. that neither index was selective enough to is as follows: CREATE TABLE SalesTable ( be an improvement over a full-table scan.Partitioned Primary Indexes and the data blocks are large. After running month (assuming the current date is in quantity_sold INTEGER. The PI is a joined to relatively small tables containing The solution would also complicate the composite of product code. The each sales agent. showing only a few of the most important columns. the DBA would create a view with – Compare one month of activity to sales to the same days of the sales_date DATE. sales history. Users would The RANGE_N function was used in this code more complicated UNION state- scenario to specify the beginning and ments. and had set up easy to create 25 partitions. one for each agent_id CHAR(8). The DBA concluded that – Compare current-month-to-date product_code CHAR(8). the previous 24 full months. Then. tests for those scenarios. the DBA had October 2004). transaction information about each product code and archive strategy. transaction date column. many of the queries would run faster (in this scenario) with no significant negative trade-off considerations. In the end. other_columns CHAR(50)) found that the Optimizer had determined PRIMARY INDEX (product_code. error-prone. each contain- other_columns CHAR(50)) a UNION of all the tables for use by the PRIMARY INDEX (product_code. targeted queries. this solution date. especially in the first few definition. agent_id). it would be sales_date DATE. and analyzing EXPLAINs. Let’s examine each element of the stated workload as it applies to the newly-partitioned table in more detail. agent_id) applications that analyze 24 months of this solution could indeed speed up the previous month or to the same days of the same month of the previous year for a few product code values. sales_ date. ing transactions for a calendar month. change the table names in their queries. prior to Teradata Database V2R5. The DBA.

the inserted rows would be still roughly a 50% gain in reading twelve so the data block would not have to be concentrated in data blocks for the proper of 25 partitions for the step that reads the read or rewritten. Given the stated assumptions.. as with the analysis is for twelve full months. AMP would contain rows for the oldest "hits per block" count (a key measure of month plus the second oldest month. Additional partitions for future months could be added if desired. to drop the partition and delete in Ad hoc Queries delete rows. The 92% figure applies to the step WITH DELETE. EB-1889 > 1204 > PAGE 6 OF 14 . The number of less evenly among all the data blocks. If the operation assuming there are no second- index values would run approximately analysis is for 24 months plus the current ary indexes or join indexes that require as fast as on the non-PPI table. agent_id) DROP RANGE BETWEEN partition elimination. physically stored contiguously (on each The nightly Teradata MultiLoad insert The same considerations apply to the AMP) instead of being scattered more or job would run faster than it did for the agent analysis queries. partitions (such as NO RANGE) to move index. and the option to make a copy of be significantly changed.e. for example. sales_date. the transient journal as they’re deleted.Partitioned Primary Indexes Faster Monthly Deletes the deleted rows is not specified. Instead of the inserted rows partitions read is determined by the time the non-PPI table. need to touch any of the rows for the Decision support queries that analyze 24 months of sales data would take roughly other month partitions. as in non-PPI table. with a proportional reduction in elapsed that reads the sales table. the resource updates. Most all the data blocks of the table. there is no need to record the individual rows in the ALTER TABLE SalesTable MODIFY PRIMARY INDEX (product_code. the entire table). only two of the 25 partitions would be read instead of the full-table scan required on the non-PPI table. so there would be fewer distributing more or less evenly among period specified in the query. there are no retained or added partitioning column is part of the primary usage is the same as for the non-PPI table. The Faster Teradata MultiLoad other steps should take roughly the same rows for the month being deleted are Inserts amount of time as for the non-PPI table. There is also no read and rewritten. the DBA could submit an the rows for October 2002. the PI access performance would not the rows. that. A delete of all the rows in a partition is optimized in much the same way that a delete of all rows in a table has historically been optimized. In both cases. For Significant Performance Gains Instead of using Teradata MultiLoad to example. and No Degradation to Queries Teradata MultiLoad efficiency) and reduce that would be the only data block read. Since the month (i. not to the sum of all the steps used to accomplish the query. This would increase the average sales table. There would be a small gain from statement is a nearly instantaneous Short-running queries that specify primary reading 24 instead of 25 partitions. This means that the number of disk DATE ‘2002-10-01’ AND DATE ‘2002-10-31’ reads would be reduced by roughly 92% ADD RANGE BETWEEN DATE ‘2004-11-01’ AND DATE ‘2004-11-30’ time. and at the same time create a new partition that would contain data for the upcoming month. Due to oldest partition and delete its rows. Only one data block per month. Dropping the Virtually No Change to Short- the same time and resources as for the non- oldest partition(s) with an ALTER TABLE Running Queries PPI table. compare a recent month basis (see the next example) to drop the you would submit: of sales data to a prior month. there is of the deletes would be full-block deletes non-PPI table. and rewritten. and create a Large gains would be seen in ad hoc queries ALTER TABLE statement on a monthly partition for November 2004. Even if the data blocks with rows to be deleted. Requiring a Full-Table Scan the number of data blocks that must be updated.

For this example. time to archive data by only archiving the recently changed partitions. Joins could even be Required Joins would take roughly the same amount faster depending on the specific query The partitioned sales table would require of time. Example of PPI Improvement Opportunities Virtually No Degradation for strategy is less efficient with the partition- Additional Disk Space Joins ing of the sales table. restored. elimination. Joins No direct merge joins No direct merge joins Little change No direct merge joins due to choice of primary index. the percentage of . since there are no conditions and the possibility of partition somewhat more disk space than the non- other tables with the same primary index. and Figure 2 summarizes the improvement copied.Partitioned Primary Indexes Activity Non-PPI Table PPI Table Improvement Comments Nightly inserts Inserted rows scattered throughout table Inserted rows concentrated in one partition Faster performance No changes to load script needed. partitioned counterpart due to the two- there are no direct merge joins to the sales table. Neither byte partition number recorded in each More Efficient Archiving and Restoring increase would be less than 3%. EB-1889 > 1204 > PAGE 7 OF 14 row. The join strategy would typically be either a duplication of a small table followed by a product join to the sales table. or a redistribution of a spool file followed by a merge join. 2% more data blocks for 100-byte rows. Archive/Restore (in Teradata Database V2R6) Entire table Entire table or selected partitions Faster archives for selected partitions Saves having to re-archive data already archived Figure 2. This can significantly reduce the opportunities for the example. Joins to the product table and agent table would most likely use the same join strategy as when the sales table was not partitioned. partitions can also be selectively archived. In this example. Monthly delete of one month of data MultiLoad job reads most data blocks. In Teradata Database V2R6. Restores of selected partitions can be used to quickly reload critical partitions. updates most data blocks ALTER TABLE statement deletes partition Much faster performance Easier maintenance Primary index access One data block read One data block read No change needed No SQL changes Comparison of current month to prior month All data blocks read Two partitions read Step is 12 times faster (two partitions of 25 read) No SQL changes needed Trend analysis over entire table All data blocks read All data blocks read Little change Rows are two bytes longer for PPI.

the same number of rows would be deleted. Modest Improvement for Some Ad hoc Queries Ad hoc queries that analyze two full months of data would not be impacted. the rows would be inserted into 32 and 36 days of data). a small partition of data) could be selectively archived or restored. was to partition by month since many of the queries use a month as their basic unit of time. which is substantially the performance of the inserts. outlined above. Additionally in Teradata Database V2R6. significantly impact the joins since there are no direct merge joins against this table in this scenario. For example. Virtually No Impact to the Monthly Deletes The monthly process deleting the oldest month of data would virtually be the same. thereby improving the 760 partitions. a day’s transactions (that is. between 28 and 31 smaller partitions would be deleted instead of one larger partition. This is because in this example the partitioning column is part of the primary index. well under one percent involve eight out of 760 partitions (eight of the total.Partitioned Primary Indexes Can the First Example Be Improved Further? The first PPI solution. the query on the three to five smaller partitions of the 760 fifth day of the current month would daily partitions. would be empty. They would now access about 60 partitions out of the 760. In other situations. for this example. Faster Nightly Inserts analyze four days for each of two months. Instead of of the 25 large partitions. However. same as two out of 25 monthly partitions. Instead of being con- the current month might analyze about 30 centrated in one or two partitions out days for each of two months. Let's compare partitioning by month to partitioning by day using the following PARTITION BY clause: PARTITION BY RANGE_N (sales_date BETWEEN DATE ‘2002-10-01’ AND DATE ‘2004-10-31’ EACH INTERVAL ‘1’ DAY). Most of the inserts would be days of data). instead of two out of 25. Some small number of partitions. Nightly inserts would benefit from the while a query submitted on the last day of finer partitioning. However. Depending on the month. as in the last two out of 25 monthly partitions (between example. either case. . when queries vary by the time of month. having a larger number of smaller partitions would produce modest gains and no degradation to performance. there could be a significant impact. No Impact to Short-Running Analysis queries that examine 24 months Queries of data would run in about the same time Having 760 partitions instead of 25 would as they are examining most of the table in not impact short-running PI access queries. The greatest gains would be for queries that analyze only a few days of transactions. roughly the same percentage of the table. and for the nightly loads. Another option to consider is partitioning to a finer level. a query submitted on the fifth day of the current month might EB-1889 > 1204 > PAGE 8 OF 14 The number of partitions would not In summary. and the run time for the job would be roughly the same. The table would now have about 760 partitions (two years with 365 days each plus the current month of about 30 days). the ones corresponding to future dates in the current month. there would be some gain by having the larger number of partitions. which is a smaller percent- directed to the one partition that contains age of the table. The query at the end of the day's activity. This would increase the the month would examine about 60 out of hits per block.

We cannot have 100. A non-PPI definition of which would always be empty because of this table. and the retention period is rarely more than six weeks. or deletes. the ALTER TABLE statement could not be used. If 1000 partitions improve performance. The primary index is the phone number and the call-start timestamp.000 partitions using the first five digits specifying a range of a few days getting the of the phone number: greatest gain. Deletion of rows would not get the same performance gain since the deletes are not strictly by call date and. EB-1889 > 1204 > PAGE 9 OF 14 faster as only one partition would be read out of maybe 500 or more non-empty partitions. facilitate direct merge joins. some of certain criteria. it is not the only choice. and the process would not reap the same performance benefit that deleting entire partitions provides. specifying a phone number to run much other_columns CHAR(30)) PRIMARY INDEX (phone_number. but would allow queries call_duration INTEGER. Some partitions might be empty due to the way phone numbers are assigned. for customers meeting there would be 1000 partitions. columns. maybe 50. If the other_columns CHAR(30)) a particular period of time. The rows are retained for a variable length of time based on the call date and the monthly bill preparation date. but we could use the first five digits and assign two consecutive numbers to each partition. similarly to the solution first three digits identify a particular area. Some queries numbers contain too many digits to give call_start TIMESTAMP. This implies the primary index was chosen to provide good data distribution across the One possibility for partitioning this table to benefit geographic area analysis since would be to cast call_start as a date and (in some parts of the world. in the first example. which would be scattered across all partitions. among other things. This is not the same for every customer. call_start) PARTITION BY RANGE_N ( CAST(phone_number / 100000. There is a row for each outgoing call with the originating phone number.276 billion rows. If the table contains about 3. perhaps for as first (high-order) three digits are used. If 10. If it’s not important to be able to map a geographic area to one or more partitions. at least) the partition by date. CREATE TABLE PPI_CallDetail ( AMPs. call_start). showing only a few critical the way phone numbers are assigned. In this case. Phone phone_number DECIMAL(10) NOT NULL. the timestamp for the start of the call. The analysis queries that are based on the For example. but a call_duration INTEGER. Let's consider a telephone company's table with detailed information about phone calls. It would not help with date- call_start TIMESTAMP.000 partitions (the first four digits of the phone number) would probably be even better. this table definition creates date of the call would benefit with queries 50. another option would be to maximize the number of partitions by using the partitioning expression (phone_number mod 65535) + 1. Other queries analyze all calls for subset of the digits could be used. number. on average each .Partitioned Primary Indexes A Second Example While transaction date is frequently a good choice for the partitioning column.000 partitions. follows: This partitioning expression would not CREATE TABLE CallDetail ( improve the performance of bulk inserts phone_number DECIMAL(10) NOT NULL. 10.00000 AS INTEGER) BETWEEN 0 AND 99999 EACH 2).000 partitions were good. and the call duration. long as a month. the deleted rows would not be clustered in a partition. analyze all calls from a particular phone each number its own partition. A second advantage would be PRIMARY INDEX (phone_number. therefore. This would help with inserting new activity in the same manner as in the previous example.000 would be better yet. based queries. It is also obvious that the primary Another choice would be to use the phone index was not chosen for data access or to number as the partitioning column.

that examine all invoices for some period On a positive note. but would have to be defined as non-unique if the table was partitioned. Here is a table definition to is a moderately heavy volume of queries row using a secondary index would take use this partitioning: that get information about one specified roughly two to three times as long as using CREATE TABLE PPI_CallDetail ( invoice. query initiation and termination overhead. would also be larger. and update operations. A query to return activity for one phone number is a best-case scenario for single-table response time improvement due to PPI. There are ad hoc analysis queries the primary index for the non-PPI table. of these proposed The following are some of the considera- rows. There row. if any. those other tables. EB-1889 > 1204 > PAGE 10 OF 14 further increasing the required disk space. and the oldest month unique secondary index to access the but could be less than 1/10000 of the of data is deleted once per month. but some width was fairly small. we examine a more ambigu- number column. New rows are added nightly using PI access queries would now use the be somewhat less than a factor of 65. by two bytes per row.Partitioned Primary Indexes partition would contain about 50. tables have invoice number as their Doubling or tripling the response time is INTEGER. and may be ranges. considerations apply. For a unique secondary index on the invoice this example. Slower Long-Running Queries The DBA is considering whether it would Direct merge joins (without partition be advantageous to partition the invoice elimination) would at best require more table on invoice date using one-month memory and CPU time. The extended 500 rows per partition. but do not have an invoice likely to go unnoticed to the users who date column. the PI access is a very of time.000 The best choice. Other fast. measurably slower compared to a similar . The base table tee that invoice numbers are unique. The decrease in amount of testing of different scenarios response time of a one-partition scan for will often be required. each partitioning expressions depends on the tions that will apply: AMP would on average contain about mix of anticipated queries.535. sume additional disk space. TIMESTAMP. Teradata MultiLoad. For a system with 100 AMPs. and con- solution is not as evident. accessing the non-PPI time. Including the each invoice issued in the past four years. usually less than a year. phone_number call_start call_duration other_columns DECIMAL(10) NOT NULL. CHAR(30)) PRIMARY INDEX (phone_number. a number of rows logical data model can serve as the starting that might fit in one data block if the row point for making the decision. could be reduced to 1/65535 of the time An invoice table contains data about using the non-PPI table. primary index. As a rule of thumb. Disregarding the overhead cost of initiating the query and returning the answer set. table scan that would result with the non-PPI table. all activity for a particular phone number would be dramatic compared to the full- Additional Disk Space A Final Example There is a business requirement to guaran- The previous examples illustrate scenarios Therefore. usually a sub-second. the DBA would have to define where a PPI table is the correct choice. the elapsed time Required The primary index is currently defined as unique. The unique primary index is invoice Slower Short-Running Queries the total query time improvement would number. call_start) PARTITION BY phone_number MOD 65535 + 1. and the correct delete. This secondary index ous situation in which more trade-off would increase processing times on insert. There are frequent joins with issue those queries. operation.

The amount of perform- of queries. followed by an optional PARTITION BY CREATE VOLATILE TABLE. For sion. This excludes global temporary tables. this may be an opportunity to move to a near-real-time load strategy using. The expression must not require example. If the difference between a PPI and but are not required to be. in the definition of a join index or hash index. if the nightly insert volume is character or graphic comparisons. Alternatively. Two functions. The benefit would be greatest when fewer months are examined. The result of required to determine the overall differ- non-PPI table performance is substantial the partitioning expression must be a ence in performance. Teradata TPump. volatile tables. CREATE partitioning_expression clause. and determine how much each One or more columns can make up the ance degradation will depend on the query query type contributes to the overall partitioning expression although it’s conditions. This restriction does not mean that a PPI table cannot have Specifics of Defining a PPI Table secondary indexes or cannot be referenced The PRIMARY INDEX clause of the TION BY clause is not available on a CREATE TABLE statement may be CREATE GLOBAL TEMPORARY TABLE. CREATE JOIN INDEX. there can be up to Would it be worthwhile to convert the characteristics of the table. unique secondary index can be defined on even a small degradation in those queries the same columns as the primary index. for this case. if cannot be defined as unique although a the response time of PI queries is critical. the additional index on invoice number would partially offset the benefit. for example. However. But the cast to INTEGER. and secondary indexes. measurement and analysis is The same considerations as in the first required to come to a rational decision example apply to the monthly deletes. in either direction. or tioning expression is a general expression CREATE HASH INDEX statement. in Teradata Database V2R6. invoice table to use a PPI? The DBA will RANGE_N and CASE_N. This will anticipated that. Impact on Table Maintenance Nightly inserts would benefit in the same way as in the first example for the same reasons. hash indexes. ment of representative queries will be table. The parti- INDEX. the choice will be scalar value that is INTEGER or can be evident for the overall workload. Similarly. are provided As rows are inserted into the table. Actual measure- load performance with and without a PPI columns can be part of the primary index. how many partitions can be workload involving this table. even a small can be referenced in some circumstances. Faster Ad hoc Queries Ad hoc queries examining several months of invoices would benefit in the same way as in the first example. and the specific join plan provide an estimate of the overall work- column will be specified. Since Teradata MultiLoad does not support unique secondary indexes. In short. It merely means that the PARTI- allowing wide flexibility in tailoring the partitioning expression to the unique In the general case. 65. the index would need to be dropped prior to the MultiLoad job and then recreated after the job.Partitioned Primary Indexes non-PPI table. for most tables. improvement in load time might be If the partitioning columns are not all part considered sufficiently important to offset of the primary index. the need to measure the amount of improve- to simplify the creation of partitioning partitioning expression is evaluated to ment and degradation in the various types expressions. Only base tables can be PPI tables. join indexes. the primary index larger degradations in queries. one eliminated. determine the proper partition placement EB-1889 > 1204 > PAGE 11 OF 14 . might be considered unacceptable even if overall workload performance is improved. Most deterministic DBA should also consider the relative functions can be used within the expres- importance of the various activities. benefits may occur with archives and restores of selected partitions. Similarly. starting to overwhelm the time set aside although character or graphic columns for inserting new activity.535 partitions numbered from one. The partitioning chosen by the Optimizer.

wasn’t partitioned. a simple partitioning expression (for instance. Teradata Database V2R6 also makes a Non-Unique Secondary Index (NUSI) access a single-AMP operation if the well. An example was The ability to ALTER a PPI table provides that compare the partitioning columns to be equal to constant expressions provide The ALTER TABLE statement has been .Partitioned Primary Indexes for that row. than using the NUPI but with the same Sample uses of partitioning expressions thereby. However.1. Secondary indexes cases. there is an equality constraint between the partitioning column of one table and a column of another table. partition elimination. As mentioned earlier. embedded in the row as part of the row expression is a single column or a only rows in the base table referenced by identifier making PPI rows two bytes RANGE_N function on a single column rowids pointing to non-eliminated wider than they would be if the table can provide partition elimination. but it does illustrate the capability. the partitioning rowkey merge join simplifies and improves the number of partitions. Teradata the partitioning columns in the context of Database V2R6 further extends dynamic the partitioning expression. making those rows wider as partition elimination. range constraints on the partitioning column where 1 shown in the section How PPI Solves the Business Problem – the First Example to drop the partition containing the oldest transactions and create expansion partitions for future dates. A data block can contain rows from multiple consecutive partitions. Also. This a set of partitions can be directly read in potentially provides a faster access path a “sliding window” of merge joins and. A join index or hash index that references a table using a row identifier uses the wider format whether or not the table has a partitioned primary index starting with Teradata Database V2R5. A two-byte internal represen- the partitioning column is compared to index. Note mized when there are a small number of that a NUSI on the same columns as the non-eliminated partitions. In Teradata Database V2R5. NUPI is only allowed for a PPI table. In some partitions are read. EB-1889 > 1204 > PAGE 12 OF 14 extended to support PPI. If also joined by teristics. dynamic V2R6 provides for archives and restores of partition elimination can occur when selected partitions. In this case. a of occurrences of a NUSI value is less than the examples were simple. This is useful when looking up a row in one table and matching those rows to corresponding The Optimizer does partition elimination partitions (using a product join) instead of for a query by analyzing the constraints on a product join to the entire table. RANGE_N on a single date column) may provide the best opportunities for partition elimination in queries. Instead of looking up all the rows tation of the partition number is constant expressions and the partitioning in the base table for particular index value. avoid spooling the partitioned single-AMP and rowhash locking charac- were shown in the discussions of the table prior to the join. Constraints partition elimination to merge joins. expression is a general expression.1 Except for the embedded internal Joins on the primary index columns of a NUSI is on the same columns as the partitioned table that are equated to the Non-Unique Primary Index (NUPI) with columns of another table are also opti- an equality condition on the NUSI. which the performance of the merge join. There are no new control structures to implement the partitioning expression. This can occur when the number examples that were presented earlier. While equality on the partitioning columns. Teradata Database makes it possible to define complex partitioning schemes tailored to the processing needs of individual tables. This is a simple example. Another enhancement in Teradata Data- a simple and convenient mechanism for base V2R6 provides partition elimination the DBA to perform periodic maintenance on the referencing rowids of a secondary on a range-based PPI table. the constant expressions may referencing PPI tables use the wider row contain USING variables and still provide identifier. partition number. PPI rows have the same format as non-PPI rows.

and is automatic. Partition on a column that is frequently used as a restrictive query condition.Partitioned Primary Indexes High-Level Partitioning Guidelines Here are some general guidelines. 2. Use RANGE_N or CASE_N in prefer- In the above situations. critical data (for example. For example. index in preference to a column that dramatically faster (nearly instantaneous Choose PI columns that provide good is not. This disadvantage can EB-1889 > 1204 > PAGE 13 OF 14 . The same considerations regarding existing SQL. 3. with limited discussion. unless the primary index is in some cases) when the table is parti- distribution and avoid large clumps of seldom. While there are few restrictions on the most recent partitions) can be restored partitioning expression is more likely to give the maximum amount of A more detailed description of partition- quickly and made available to users partition elimination than a more ing guidelines may be found in the without waiting for the entire table to complex expression. the time the selection of the primary index a column that is part of the primary to delete old rows no longer needed can be apply to PPI tables as non-PPI tables. structure. If the table is partitioned by transaction date. A simple must be reached. the improvement High-Level Trade-off Considerations may be even greater when the partitioning structure makes one or more secondary indexes or join indexes redundant. The Optimizer can a small subset of the table instead of the Offsetting these gains are some potential determine the maximum number entire table. but less costly archives. SQL authors than when the number is assumed to need not be aware of the partitioning be 65. duplicate PI values.535. to help determine whether and how to partition a table: 1. If other factors are equal. For elimination to queries. Similarly. 5. a Teradata Orange Book: Partitioned Primary be restored. 7. about one-twelfth of the table instead of is not part of the PI. This can provide a large perform- accurate when the actual number of ance boost for a wide range of queries. Unless the PI is rarely used for access or direct merge joins. restores. in particular. day partitions is known and fairly small after day.535 partitions otherwise. partitioning expression. and a reason- selected partitions. and which are most commonly used to access individual Finally with Teradata Database V2R6. and will have to with two years of sales history would read may be slower when a partitioning column assume 65. RANGE_N function on a date column Index Usage. Sometimes those two you can perform archives and restores of expression is only useful if the Opti- considerations conflict. This allows for more mizer can effectively apply partition able compromise between the two frequent. keep the number of partitions fairly small when the partitioning expression uses columns that are not part of the PI. A second potential advantage is faster batch loads. in the 4. a partitioning rows in the table. nightly loads of transactions for the current day can be dramatically improved. allow- The greatest potential gain derived from ing those indexes to be dropped. ence to direct use of a column in partitioning a table is the ability to read most situations. can often be an effective partitioning expression for queries with range constraints on the partitioning column. if ever. used for access or joins. partition on Join costing. tioned by transaction date. Large tables are good candidates for partitioning. a query that disadvantages of partitioning. The first of partitions when RANGE_N or examines two months of sales from a table disadvantage is that PI access of the table CASE_N is used. can be more all of it. and no changes are required to 6. For example.

com be offset by choosing partitioning columns that are part of the PI. process or technology described in the document may be the subject of other intellectual property rights reserved by NCR and are not licensed hereunder. NCR continually enhances products as new technologies and components become available. the primary index value is specified in the query. is the exclusive property of NCR Corporation. The disadvantage can be offset partitioning column. design decisions. The extended logical data model can serve slower unless both tables can be identically the transaction date is specified as the as the starting point for making physical partitioned. which includes the information contained herein.A. OH U. but some amount of when the query conditions allow some A partitioned primary index is flexible and partitions to be eliminated from the join.com for more information. Consult your Teradata representative or visit Teradata. specifying the Summary PPI tables can dramatically improve values of the partitioning columns and performance of certain types of queries. Teradata Orange Book: Partitioned Primary Users accessing a PPI table will see no Index Usage.Partitioned Primary Indexes Teradata. All Rights Reserved. print. As with other physical design decisions. The trade-off considerations associated with a PPI should be understood and considered when making the physical design decisions. in some situations.S.A. duplication or disclosure by the United States government is subject to the restrictions set forth in DFARS 252. Use. and distribute this document subject to the following conditions.S. informational purposes only and is provided on an “AS-IS” basis. part of a large table. This document may be used for non-commercial. difference. easy to use. reserves the right to change specifications without prior notice. you must uses of primary indexes to distribute data weigh the trade-off considerations and test evenly and provide very fast access when assumptions to get the best results. Any copy of this document or portion thereof must include this copyright notice and all other restrictive legends appearing in this document.227-19. by especially those that access only a small defining a secondary index. Any person is hereby authorized to view. A second disadvantage is that direct merge load and data maintenance times can joins involving a partitioned table may be also be improved when. NCR. Note that any product. testing of different scenarios will often be required. All features. copy. A more detailed description of trade-off considerations may be found in the No changes to existing SQL are necessary. No license rights will be implied.227-7013 (c) (1) (ii) and FAR 52. High-volume data Whether and how to partition the primary index of a table is a physical design choice. No part of this publication may be reprinted or otherwise reproduced without permission from Teradata. the total workload and relative importance of the workload components must be examined to determine whether the benefits will outweigh the disadvantages for each design decision. Teradata and NCR are registered trademarks of NCR Corporation. EB-1889 > 1204 > PAGE 14 OF 14 Produced in U. therefore. the PI columns. for example. and operations described herein may not be marketed in all parts of the world. functions. except perhaps for different average response times. or. PPI tables retain the traditional As in all physical design choices. This document. . © 2004 NCR Corporation Dayton.