Introduction I/O Parallelism Interquery Parallelism Intraquery Parallelism Intraoperation Parallelism Interoperation Parallelism Design of Parallel Systems

Parallel machines are becoming quite common and affordable

Prices of microprocessors, memory and disks have dropped sharply Recent desktop computers feature multiple processors and this trend is projected to accelerate

Introduction

Databases are growing increasingly large

Large-scale parallel database systems increasingly used for:

large volumes of transaction data are collected and stored for later analysis. multimedia objects like images are increasingly stored in databases storing large volumes of data processing time-consuming decision-support queries providing high throughput for transaction processing

Parallelism in Databases
Data can be partitioned across multiple disks for parallel I/O. Individual relational operations (e.g., sort, join, aggregation) can be executed in parallel
data can be partitioned and each processor can work independently on its own partition.

Queries are expressed in high level language (SQL, translated to relational algebra)
makes parallelization easier.

Different queries can be run in parallel with each other. Concurrency control takes care of conflicts. Thus, databases naturally lend themselves to parallelism.

I/O Parallelism
Partitioning can improve performance and help manage data. In particular, partitioning can help manage large tables and indexes by dividing them into smaller, more manageable pieces. Partitions, like a large-scale index, provide faster and easier access to data. load data and create indexes at the partition level. Yet partitions are transparent to the end user, who can select, insert, and delete data using the same DML commands whether the table is partitioned or not.

I/O Parallelism
There are several ways to partition a table: Round-robin by random assignment. Hash using a hashing function to determine row assignment (semantics-based partitioning). Range according to values in the data row falling within a range of specified values (semantics-based partitioning).

 Data partitions A data partition is an independent database object with a unique partition ID. It is a subset of a table, and shares the column definitions and referential and integrity constraints of the base table. To maximize I/O parallelism, Sybase recommends that you bind each partition to a different segment, and bind each segment to a different storage device. Partition IDs A partition ID is a pseudo-random number similar to object ID. Partition IDs and object IDs are allocated from the same number space. An index or data partition is identified with a unique combination of index ID and partition ID.

Range partitioning:

 Rows in a range-partitioned table or index are distributed among partitions according to values in the partitioning key columns. The partitioning column values of each row are compared with a set of upper and lower bounds to determine the partition to which the row belongs. Every partition has an inclusive upper bound, which is specified by the values <= clause when the partition is created. Every partition except the first has a noninclusive lower bound, which is specified implicitly by the values <= clause on the next-lower partition. Range partitioning is particularly useful for high-performance applications in both OLTP and decision-support environments. Select ranges carefully so that rows are assigned equally to all partitionsknowledge of the data distribution of the partition key columns is crucial to balancing the load evenly among the partitions.

 CREATE TABLE lineitem ( l_orderkey DECIMAL(10,0) NOT NULL, l_quantity DECIMAL(12,2), l_shipdate DATE, l_year_month INT) PARTITION BY RANGE(l_shipdate) (STARTING ('1/1/1992') ENDING ('12/31/1992') ); CREATE TABLE foo(a INT) PARTITION BY RANGE(a) (STARTING FROM (1) ENDING AT (100), STARTING FROM (201) ENDING AT (300))

Hash partitioning
Partitioning is based on a function of one or more columns (the hash partitioning keys) in each record. The hash partitioner examines one or more fields of each input record (the hash key fields). Records with the same values for all hash key fields are assigned to the same processing node. This method is useful for ensuring that related records are in the same partition, which might be a prerequisite for a processing operation. For example, for a remove duplicates operation, you can hash partition records so that records with the same partitioning key values are on the same node. You can then sort the records on each node using the hash key fields as sorting key fields, then remove duplicates, again using the same keys. Although the data is distributed across partitions, the hash partitioner ensures that records with identical keys are in the same partition, allowing duplicates to be found

Hash partitioning

Round-robin partitioning
In round-robin partitioning, Adaptive Server does not use partitioning criteria. Round-robin-partitioned tables have no partition key. Adaptive Server assigns rows in a round-robin manner to each partition so that each partition contains a more or less equal number of rows and load balancing is achieved. Because there is no partition key, rows are distributed randomly across all partitions. The first record goes to the first processing node, the second to the second processing node, and so on . This method is useful for resizing partitions of an input data set that are not equal in size. The round robin method always creates approximately equal-sized partitions

Round-robin partitioning

 The distribution of tuples to disks may be skewed that is, some disks have many tuples, while others may have fewer tuples. Types of skew:
Attribute-value skew.
Some values appear in the partitioning attributes of many tuples; all the tuples with the same value for the partitioning attribute end up in the same partition. Can occur with range-partitioning and hash-partitioning.

Handling of Skew

Partition skew.
With range-partitioning, badly chosen partition vector may assign too many tuples to some partitions and too few to others. Less likely with hash-partitioning if a good hash-function is chosen.

Handling Skew in Range-Partitioning

To create a balanced partitioning vector (assuming partitioning attribute forms a key of the relation):
Sort the relation on the partitioning attribute. Construct the partition vector by scanning the relation in sorted order as follows.
After every 1/nth of the relation has been read, the value of the partitioning attribute of the next tuple is added to the partition vector.

n denotes the number of partitions to be constructed. Duplicate entries or imbalances can result if duplicates are present in partitioning attributes.

Alternative technique based on histograms used in practice

Handling Skew using Histograms

straightforward fashion

Balanced partitioning vector can be constructed from histogram in a relatively

Assume uniform distribution within each range of the histogram

Histogram can be constructed by scanning relation, or sampling (blocks containing)

tuples of the relation

Interquery Parallelism
Queries/transactions execute in parallel with one another. Increases transaction throughput; used primarily to scale up a transaction processing system to support a larger number of transactions per second. Easiest form of parallelism to support, particularly in a sharedmemory parallel database, because even sequential database systems support concurrent processing. More complicated to implement on shared-disk or sharednothing architectures
Locking and logging must be coordinated by passing messages between processors. Data in a local buffer may have been updated at another processor. Cache-coherency has to be maintained reads and writes of data in buffer must find latest version of data.

Cache Coherency Protocol

Example of a cache coherency protocol for shared disk systems:
Before reading/writing to a page, the page must be locked in shared/exclusive mode. On locking a page, the page must be read from disk Before unlocking a page, the page must be written to disk if it was modified.

More complex protocols with fewer disk reads/writes exist.

Intraquery Parallelism
Execution of a single query in parallel on multiple processors/disks; important for speeding up longrunning queries. Two complementary forms of intraquery parallelism :
Intraoperation Parallelism parallelize the execution of each individual operation in the query. Interoperation Parallelism execute the different operations in a query expression in parallel.

the first form scales better with increasing parallelism because the number of tuples processed by each operation is typically more than the number of operations in a query