Clear

Chapter 16: Query Optimization
Database System Concepts, 7th Ed.

©Silberschatz, Korth and Sudarshan
See www.db-book.com for conditions on re-use
Outline
 Introduction
 Transformation of Relational Expressions
 Cost-based optimization
 Equivalence Expressions
 Statistical Information for Cost Estimation
Database System Concepts - 7th Edition 16.2 ©Silberschatz, Korth and Sudarshan
Introduction
 Alternative ways of evaluating a given query

• Equivalent expressions
• Different algorithms for each operation
Introduction (Cont.)
 An evaluation plan defines exactly what algorithm is used for each

operation, and how the execution of the operations is coordinated.
 Find out how to view query execution plans on your favorite database
Equivalence of Expressions
 The first step in selecting a query-processing strategy is
• To find a relational algebra expression that is equivalent to the given

query and is efficient to execute.
 Two relational algebra expressions are said to be equivalent if the two
expressions generate the same set of tuples on every legal database
instance.
 An equivalence rule says that expressions of two forms are equivalent
• Can replace expression of first form by second, or vice versa.
 We'll use the following relations as examples:

 instructor(ID, name, dept name, salary)
 teaches(ID, course id, sec id, semester, year)
 course(course id, title, dept name, credits)
cost-based query optimization
 Steps in cost-based query optimization

1. Generate logically equivalent expressions using equivalence rules
2. Annotate resultant expressions to get alternative query plans
3. Choose the cheapest plan based on estimated cost
 Estimation of plan cost based on:

• Statistical information about relations. Examples:
 number of tuples, number of distinct values for an attribute
• Statistics estimation for intermediate results

 to compute cost of complex expressions
• Cost formulae for algorithms, computed using statistics
Semi Join
Semi Join - Example
Equivalence Rules ( 16 Rules ..)
1. Conjunctive selection operations can be deconstructed into a sequence

of individual selections.
σ1  2 (E) ≡ σ1 (σ2 (E))
2. Selection operations are commutative.

σ1(σ2(E)) ≡ σ2 (σ1(E))
3. Only the last in a sequence of projection operations is needed, the

others can be omitted.
 L1( L2(…( Ln(E))…)) ≡  L1(E)
where L1 ⊆ L2 … ⊆ Ln
4. Selections can be combined with Cartesian products and theta joins.
a. σ (E1 x E2) ≡ E1 ⨝  E2
b. σ 1 (E1 ⨝2 E2) ≡ E1 ⨝ 1∧2 E2
Equivalence Rules (Cont.)
5. Theta-join operations (and natural joins) are commutative.
E1 ⨝ E2 ≡ E2 ⨝ E1
6. (a) Natural join operations are associative:

(E1 ⨝ E2) ⨝ E3 ≡ E1 ⨝ (E2 ⨝ E3)
(b) Theta joins are associative in the following manner:
(E1 ⨝ 1 E2) ⨝ 2  3 E3 ≡ E1 ⨝1  3 (E2 ⨝ 2 E3)
where 2 involves attributes from only E2 and E3.
Pictorial Depiction of Equivalence Rules
7. The selection operation distributes over the theta join operation

under the following two conditions:
(a) When all the attributes in 0 involve only the attributes of
one of the expressions (E1) being joined.
0 E1 ⨝ E2) ≡ (0(E1)) ⨝ E2
(b) When 1 involves only the attributes of E1 and 2 involves

only the attributes of E2.
1  2 E1 ⨝ E2) ≡ (1(E1)) ⨝ (2(E2))
8. The projection operation distributes over the theta join operation as follows:
(a) if  involves only attributes from L1  L2:
 L1  L2(E1 ⨝ E2) ≡  L1(E1) ⨝  L2(E2)
(b) In general, consider a join E1 ⨝ E2.
• Let L1 and L2 be sets of attributes from E1 and E2, respectively.
• Let L3 be attributes of E1 that are involved in join condition , but are
not in L1  L2, and
• let L4 be attributes of E2 that are involved in join condition , but are
not in L1  L2.
 L1  L2(E1 ⨝ E2) ≡  L1  L2( L1  L3(E1) ⨝  L2  L4(E2))
Similar equivalences hold for outer join operations: ⟕, ⟖, and ⟗
9. The set operations union and intersection are commutative

E1  E2 ≡ E2  E1
E1  E2 ≡ E2  E1
(set difference is not commutative).
10. Set union and intersection are associative.
(E1  E2 )  E3 ≡ E1  (E2  E3)
(E1  E2 )  E3 ≡ E1  (E2  E3)
11. The selection operation distributes over ,  and –.
a.  (E1  E2) ≡  (E1)  (E2)
b.  (E1  E2) ≡  (E1)  (E2)
c.  (E1 – E2) ≡  (E1) – (E2)
d.  (E1  E2) ≡ (E1)  E2
e.  (E1 – E2) ≡ (E1) – E2
preceding equivalence does not hold for 
12. The projection operation distributes over union
L(E1  E2) ≡ (L(E1))  (L(E2))
13. Selection distributes over aggregation as below
(G𝛾A(E)) ≡ G𝛾A((E))
provided  only involves attributes in G
14. a. Full outerjoin is commutative:
E1 ⟗ E2 ≡ E2 ⟗ E1
b. Left and right outerjoin are not commutative, but:
E1 ⟕ E2 ≡ E2 ⟖ E1
15. Selection distributes over left and right outerjoins as below, provided 1
only involves attributes of E1
a. 1 (E1 ⟕ E2) ≡ (1 (E1)) ⟕ E2
b. 1 (E1 ⟖ E2) ≡ E2 ⟕ (1 (E1))
16. Outerjoins can be replaced by inner joins under some conditions
a. 1 (E1 ⟕ E2) ≡ 1 (E1 ⨝ E2)
b. 1 (E1 ⟖ E1) ≡ 1 (E1 ⨝ E2)
provided 1 is null rejecting on E2
Transformation Example: Pushing Selections
 Query: Find the names of all instructors in the Music department, along
with the titles of the courses that they teach
• name, title(dept_name= ‘Music’
(instructor ⨝ (teaches ⨝ course_id, title (course))))
 Transformation using rule 7a.
• name, title((dept_name= ‘Music’(instructor)) ⨝
(teaches ⨝ course_id, title (course)))
 Performing the selection as early as possible reduces the size of the
relation to be joined.
Multiple Transformations (Cont.)
Join Ordering Example
 For all relations r1, r2, and r3,
(r1 ⨝ r2) ⨝ r3 = r1 ⨝ (r2 ⨝ r3 )
(Join Associativity) ⨝
 If r2 ⨝ r3 is quite large and r1 ⨝ r2 is small, we choose
(r1 ⨝ r2) ⨝ r3
so that we compute and store a smaller temporary relation.
Join Ordering Example (Cont.)
 Consider the expression

name, title(dept_name= “Music” (instructor) ⨝ teaches)
⨝ course_id, title (course))))
 Could compute teaches ⨝ course_id, title (course) first, and join result with
dept_name= “Music” (instructor)
but the result of the first join is likely to be a large relation.
 Only a small fraction of the university’s instructors are likely to be from
the Music department
• it is better to compute
dept_name= “Music” (instructor) ⨝ teaches
first.
Cost Estimation
 Cost of each operator computer
 Need statistics of input relations
 E.g., number of tuples, sizes of tuples
 Inputs can be results of sub-expressions
• Need to estimate statistics of expression results
• To do so, we require additional statistics
 E.g., number of distinct values for an attribute
Statistical Information for Cost Estimation
 nr: number of tuples in a relation r.
 br: number of blocks containing tuples of r.
 lr: size of a tuple of r.
 fr: blocking factor of r — i.e., the number of tuples of r that fit into one block.
 V(A, r): number of distinct values that appear in r for attribute A; same as
the size of A(r).
 If tuples of r are stored together physically in a file, then:
nr ùú
é
br = ê
f r úú
ê
ê
Join Operation: Running Example
Running example:
student ⨝ takes
Catalog information for join examples:
 nstudent = 5,000.
 fstudent = 50, which implies that bstudent =5000/50 = 100.
 ntakes = 10000.
 ftakes = 25, which implies that

btakes = 10000/25 = 400.
 V(ID, takes) = 2500, which implies that on average, each student who has
taken a course has taken 4 courses.
• Attribute ID in takes is a foreign key referencing student.
• V(ID, student) = 5000 (primary key!)
Estimation of the Size of Joins
 If R  S in S is a foreign key in S referencing R, then the number of tuples

in r ⋈ s is exactly the same as the number of tuples in s.
 In the example query student ⋈ takes, ID in takes is a foreign key

referencing student
• hence, the result has exactly ntakes tuples, which is 10000
Estimation of the Size of Joins (Cont.)
 If R  S = {A} is not a key for R or S.

If we assume that every tuple t in R produces tuples in R S, the number
of tuples in R ⨝ S is estimated to be:
nr * ns
V ( A, s )
If the reverse is true, the estimate obtained will be:
nr * ns
V ( A, r )
The lower of these two estimates is probably the more accurate one.
 Can improve on above if histograms are available
• Use formula similar to above, for each cell of histograms on the two
relations
Estimation of the Size of Joins (Cont.)
 Compute the size estimates for student ⨝ take without using information
about foreign keys:
• V(ID, takes) = 2500, and

V(ID, student) = 5000
• The two estimates are 5000 * 10000/2500 = 20,000 and 5000 *

10000/5000 = 10000
• We choose the lower estimate, which in this case, is the same as our
earlier computation using foreign keys.
chapter 17: Transactions
Database System Concepts, 7th Ed.

©Silberschatz, Korth and Sudarshan
See www.db-book.com for conditions on re-use
Outline
 Transaction Concept
 Transaction State
 Concurrent Executions
 Serializability
 Recoverability
 Transaction Definition in SQL
Transaction Concept
 A transaction is a unit of program execution that accesses and possibly

updates various data items.
 E.g., transaction to transfer $50 from account A to account B:
1. read(A)
2. A := A – 50
3. write(A)
4. read(B)
5. B := B + 50
6. write(B)
 Two main issues to deal with:
• Failures of various kinds, such as hardware failures and system
crashes
• Concurrent execution of multiple transactions
Example of Fund Transfer
 Transaction to transfer $50 from account A to account B:

1. read(A)
2. A := A – 50
3. write(A)
4. read(B)
5. B := B + 50
6. write(B)
 Atomicity requirement
• If the transaction fails after step 3 and before step 6, money will be “lost”
leading to an inconsistent database state
 Failure could be due to software or hardware
• The system should ensure that updates of a partially executed transaction
are not reflected in the database
 Durability requirement — once the user has been notified that the transaction
has completed (i.e., the transfer of the $50 has taken place), the updates to the
database by the transaction must persist even if there are software or hardware
failures.
Example of Fund Transfer (Cont.)
 Consistency requirement in above example:

• The sum of A and B is unchanged by the execution of the transaction
 In general, consistency requirements include
• Explicitly specified integrity constraints such as primary keys and foreign
keys
• Implicit integrity constraints
 e.g., sum of balances of all accounts, minus sum of loan amounts must
equal value of cash-in-hand
• A transaction must see a consistent database.
• During transaction execution the database may be temporarily
inconsistent.
• When the transaction completes successfully the database must be
consistent
 Erroneous transaction logic can lead to inconsistency
Example of Fund Transfer (Cont.)
 Isolation requirement — if between steps 3 and 6, another transaction T2

is allowed to access the partially updated database, it will see an
inconsistent database (the sum A + B will be less than it should be).
T1 T2
1. read(A)
2. A := A – 50
3. write(A)
read(A), read(B), print(A+B)
4. read(B)
5. B := B + 50
6. write(B)
 Isolation can be ensured trivially by running transactions serially
• That is, one after the other.
 However, executing multiple transactions concurrently has significant
benefits, as we will see later.
ACID Properties
A transaction is a unit of program execution that accesses and possibly
updates various data items. To preserve the integrity of data the database
system must ensure:
 Atomicity. Either all operations of the transaction are properly reflected in
the database or none are.
 Consistency. Execution of a transaction in isolation preserves the
consistency of the database.
 Isolation. Although multiple transactions may execute concurrently, each
transaction must be unaware of other concurrently executing transactions.
Intermediate transaction results must be hidden from other concurrently
executed transactions.
• That is, for every pair of transactions Ti and Tj, it appears to Ti that
either Tj, finished execution before Ti started, or Tj started execution
after Ti finished.
 Durability. After a transaction completes successfully, the changes it has
made to the database persist, even if there are system failures.
Transaction State
 Active – the initial state; the transaction stays in this state while it is
executing
 Partially committed – after the final statement has been executed.
 Failed -- after the discovery that normal execution can no longer proceed.
 Aborted – after the transaction has been rolled back and the database
restored to its state prior to the start of the transaction. Two options after it
has been aborted:
• Restart the transaction
 Can be done only if no internal logical error
• Kill the transaction
 Committed – after successful completion.
Transaction State (Cont.)
Schedules
 Schedule – a sequences of instructions that specify the

chronological order in which instructions of concurrent transactions
are executed
• A schedule for a set of transactions must consist of all
instructions of those transactions
• Must preserve the order in which the instructions appear in each
individual transaction.
 A transaction that successfully completes its execution will have a
commit instructions as the last statement
• By default, transaction assumed to execute commit instruction
as its last step
 A transaction that fails to successfully complete its execution will
have an abort instruction as the last statement.
Schedule 1 – serial
 Each serial schedule consists of a sequence of instructions from
various transactions, where the instructions belonging to one single
transaction appear together in that schedule.
 Let T1 transfer $50 from A to B, and T2 transfer 10% of the balance from A
to B.
 A serial schedule in which T1 is followed by T2 :
Precedence graph
Schedule 2
 A serial schedule where T2 is followed by T1
Precedence graph
Concurrent transactions
 When the database system executes several transactions

concurrently, the corresponding schedule no longer needs to be
serial.
 If two transactions are running concurrently, the operating
system may execute one transaction for a little while, then
perform a context switch, execute the second transaction for
some time, and then switch back to the first transaction for
some time, and so on.
 With multiple transactions, the CPU time is shared among all
the transactions.
Concurrent Executions
 Multiple transactions are allowed to run concurrently in the system.
Advantages are:
• Increased processor and disk utilization, leading to better
transaction throughput
 E.g., one transaction can be using the CPU while another is
reading from or writing to the disk
• Reduced average response time for transactions: short
transactions need not wait behind long ones.
 Concurrency control schemes – mechanisms to achieve isolation
• That is, to control the interaction among the concurrent
transactions in order to prevent them from destroying the
consistency of the database.
• It is the job of the database system to ensure that any schedule
that is executed will leave the database in a consistent state.
Schedule 3
 Let T1 and T2 be the transactions defined previously. The following schedule is

not a serial schedule, but it is equivalent to Schedule 1
Concurrent schedule equivalent to schedule 1.

The concept of conflict
equivalence leads to the concept
of conflict serializability.
We say that a schedule S is

conflict serializable if it is conflict
equivalent to a serial schedule.
Thus, schedule 3 is conflict
serializable, since it is conflict
equivalent to the serial schedule 1.
 In Schedules 1, 2 and 3, the sum A + B is preserved.
Schedule 4
 The following concurrent schedule does not preserve the value of (A and B ).
It contains the edge T1 →T2

because T1 executes read(A)
before T2 executes write(A).
It also contains the edge T2 →
T1 because T2 executes read(B)
before T1 executes write(B).
Concurrent schedule resulting in an inconsistent state.
Serializability
 Basic Assumption – Each transaction preserves database consistency.

 Thus, serial execution of a set of transactions preserves database
consistency.
 A (possibly concurrent) schedule is serializable if it is equivalent to a serial
schedule. Different forms of schedule equivalence give rise to the notions of:
Conflict serializability
conflict serializability
 Let us consider a schedule S in which there are two consecutive

instructions, I and J, of transactions Ti and Tj, respectively (i ≠ j).
 If I and J refer to different data items, then we can swap I and J without
affecting the results of any instruction in the schedule. However, if I and
J refer to the same data item Q, then the order of the two steps may
matter.
 Since we are dealing with only read and write instructions, there are four
cases that we need to consider:
1. I = read(Q), J = read(Q). The order of I and J does not matter, since the
same value of Q is read by Ti and Tj, regardless of the order.
2. I = read(Q), J = write(Q). The order of I and J matters.
3. I = write(Q), J = read(Q). The order of I and J matters
4. I = write(Q), J = write(Q). Since both instructions are write operations, the
order of these instructions does not affect either Ti or Tj.
 I and J conflict if they are operations by different transactions on the
same data item, and at least one of these instructions is a write
operation.
conflict serializability…
The concept of conflict

equivalence leads to the concept
of conflict serializability.
We say that a schedule S is

conflict serializable if it is conflict
equivalent to a serial schedule.
Thus, schedule 3 is conflict
serializable, since it is conflict
equivalent to the serial schedule
1.
The write(A) instruction of T1 conflicts with the read(A) instruction of

T2.
However, the write(A) instruction of T2 does not conflict with the
read(B) instruction
Precedence graph
 Consider a schedule S.
 Precedence graph consists of a pair G = (V, E), where V is a set of
vertices and E is a set of edges. The set of vertices consists of all the
transactions participating in the schedule S.
 The set of edges consists of all edges Ti → Tj for which one of three
conditions holds:
1. Ti executes write(Q) before Tj executes read(Q).
2. Ti executes read(Q) before Tj executes write(Q).
3. Ti executes write(Q) before Tj executes write(Q).
 If an edge Ti →Tj exists in the precedence graph, then, in any serial
schedule S′ equivalent to S, Ti must appear before Tj.
Test for Conflict Serializability
 A schedule is conflict serializable if and only if its

precedence graph is acyclic.
 Cycle-detection algorithms exist which take order n2
time, where n is the number of vertices in the graph.
• (Better algorithms take order n + e where e is
the number of edges.)
 If precedence graph is acyclic, the serializability
order can be obtained by a topological sorting of the
graph.
 A topological sorting is the process of finding a linear
order consistent with the partial order of the
precedence graph
 The graph of Figure a has the two
acceptable linear orderings shown in Figure
b and Figure c.
Transaction Definition in SQL
 In SQL, a transaction begins implicitly.

 A transaction in SQL ends by:
• Commit work commits current transaction and begins a new one.
• Rollback work causes current transaction to abort.
 In almost all database systems, by default, every SQL statement also
commits implicitly if it executes successfully
• Implicit commit can be turned off by a database directive
 E.g., in JDBC -- connection.setAutoCommit(false);
 Isolation level can be set at database level
 Isolation level can be changed at start of transaction
 E.g. In SQL set transaction isolation level serializable
 E.g. in JDBC -- connection.setTransactionIsolation(
Connection.TRANSACTION_SERIALIZABLE)
Implementation of Isolation Levels
 Locking
• Lock on whole database vs lock on items
• How long to hold lock?
• Shared vs exclusive locks
 Timestamps
• Transaction timestamp assigned e.g. when a transaction begins
• Data items store two timestamps
 Read timestamp
 Write timestamp
• Timestamps are used to detect out of order accesses
 Multiple versions of each data item
• Allow transactions to read from a “snapshot” of the database
Transactions as SQL Statements
 E.g., Transaction 1:
select ID, name from instructor where salary > 90000
 E.g., Transaction 2:
insert into instructor values ('11111', 'James', 'Marketing', 100000)
 Suppose
• T1 starts, finds tuples salary > 90000 using index and locks them
• And then T2 executes.
• Do T1 and T2 conflict? Does tuple level locking detect the conflict?
• Instance of the phantom phenomenon
 Also consider T3 below, with Wu’s salary = 90000
update instructor
set salary = salary * 0.9
where name = 'Wu’
 Key idea: Detect “predicate” conflicts, and use some form of “predicate
locking”
End of Chapter 17

Clear

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Clear

Uploaded by

Copyright:

Available Formats

Chapter 16: Query Optimization

Database System Concepts, 7th Ed.

 Transformation of Relational Expressions

 Statistical Information for Cost Estimation

 Alternative ways of evaluating a given query

 An evaluation plan defines exactly what algorithm is used for each

 The first step in selecting a query-processing strategy is

• To find a relational algebra expression that is equivalent to the given

 We'll use the following relations as examples:

 Steps in cost-based query optimization

 Estimation of plan cost based on:

• Statistics estimation for intermediate results

• Cost formulae for algorithms, computed using statistics

1. Conjunctive selection operations can be deconstructed into a sequence

2. Selection operations are commutative.

3. Only the last in a sequence of projection operations is needed, the

4. Selections can be combined with Cartesian products and theta joins.

b. σ 1 (E1 ⨝2 E2) ≡ E1 ⨝ 1∧2 E2

5. Theta-join operations (and natural joins) are commutative.

6. (a) Natural join operations are associative:

(b) Theta joins are associative in the following manner:

(E1 ⨝ 1 E2) ⨝ 2  3 E3 ≡ E1 ⨝1  3 (E2 ⨝ 2 E3)

where 2 involves attributes from only E2 and E3.

7. The selection operation distributes over the theta join operation

0 E1 ⨝ E2) ≡ (0(E1)) ⨝ E2

(b) When 1 involves only the attributes of E1 and 2 involves

1  2 E1 ⨝ E2) ≡ (1(E1)) ⨝ (2(E2))

Similar equivalences hold for outer join operations: ⟕, ⟖, and ⟗

9. The set operations union and intersection are commutative

 For all relations r1, r2, and r3,

(r1 ⨝ r2) ⨝ r3 = r1 ⨝ (r2 ⨝ r3 )

 If r2 ⨝ r3 is quite large and r1 ⨝ r2 is small, we choose

so that we compute and store a smaller temporary relation.

 Consider the expression

 Cost of each operator computer

 Need statistics of input relations

 E.g., number of tuples, sizes of tuples

 Inputs can be results of sub-expressions

• Need to estimate statistics of expression results

• To do so, we require additional statistics

 E.g., number of distinct values for an attribute

 nr: number of tuples in a relation r.

 br: number of blocks containing tuples of r.

 lr: size of a tuple of r.

 If tuples of r are stored together physically in a file, then:

Catalog information for join examples:

 fstudent = 50, which implies that bstudent =5000/50 = 100.

 ftakes = 25, which implies that

• Attribute ID in takes is a foreign key referencing student.

• V(ID, student) = 5000 (primary key!)

 If R  S in S is a foreign key in S referencing R, then the number of tuples

 In the example query student ⋈ takes, ID in takes is a foreign key

 If R  S = {A} is not a key for R or S.

• V(ID, takes) = 2500, and

• The two estimates are 5000 * 10000/2500 = 20,000 and 5000 *

Database System Concepts, 7th Ed.

 A transaction is a unit of program execution that accesses and possibly

 Transaction to transfer $50 from account A to account B:

 Consistency requirement in above example:

 Isolation requirement — if between steps 3 and 6, another transaction T2

 Schedule – a sequences of instructions that specify the

 When the database system executes several transactions

 Let T1 and T2 be the transactions defined previously. The following schedule is