Database 2

Lecture 2
Storage hierarchy

During application execution a piece of

information may move between different levels
of storage.

Some data may exist only temporarily during

execution (transient data).

Some data may be stored for later use

(persistent data).

Secondary Storage
Data is stored onto secondary storage as bits. In the case of magnetic disks, an area of the
disk can be magnetized to represent either 0 or 1.

The several read/write heads read

and write data onto an imaginary
cylinder.

The spindle rotates the disks to

the location (on the cylinder)
where data is to be read or
written.

On each disk are multiple tracks (a

track can hold 10-150 Kbytes). The
tracks are divided into sectors.

This formatting divides the track into

equal sized disk blocks (a disk block
can be 512-8192 bytes) with interblock
gaps separating them

The combination of cylinder number, track number, and block number are what define the
hardware address of piece of data.
Buffering
Buffers are reserved in main memory to speed up data transfer between disk and main
memory. Several buffers can be reserved in main memory to allow the CPU to process data
in one buffer while the other buffer is being read/written (this is possible because the I/O
controller handles data block transfer independent of the CPU).

When a program requests a certain block of data it calls the buffer manager, the buffer
manager tries to accommodate the new block of data within the available buffer space, if
the available space is not sufficient then it removes certain blocks based on some
replacement strategy (e.g., FIFO, LRU, Clock policy) and replaces them with the requested
block.

The buffer manager stores some information about each block/page to aid it in its job:

1. Pin-count: the number of times that a block has been requested, if a block is being
pinned many times the block manager keeps it in memory.
2. Dirty bit: initially set to 0 but is set to one when a block is updated by a program, this
means it will have to be written back on to the disk.

File Organization
A database is a set of records organized into a set of files.

*a record having several fields

A file is a sequence of records. If all the records in a file are the same size, the file is said
to be made up of fixed-length records. If different records in the file have different sizes,
the file is said to be made up of variable-length records. Records can be variable-length for
several reasons e.g., having a variable-length field, having an optional field, having a
repeating field, etc.

The number of records that can fit in one block (assuming fixed-length records) is called
the blocking factor. The record size might not divide the block size evenly, in this case one
of two things are done:

1. Fit as many records as possible and leave the rest empty (unspanned)

2. Store the record across two blocks and connect then using a pointer (spanned)
File Operations
File operations are mostly concerned with the retrieval and updating of records. This
usually starts by selecting one more records using some condition (e.g., employees with
salaries above 10000) and then retrieving or updating one or several fields within that
selection.

Some common file operations:

1. Open: allocates appropriate buffers to load the file into memory and sets file pointer
to the start of the file
2. Reset: resets the file pointer to the start of the file
3. Find: searches for the first record that satisfies some search condition, loads the
block it is in into memory, and sets the file pointer on the record
4. Read: copies record from buffer into a program variable
5. FindNext: searches for and loads the next record that satisfies the search condition
6. Delete: deletes the current record
7. Modify: modifies some field values for the current record
8. Insert: inserts a new record into the file
9. Close: releases buffers allocated for file
10. FindAll: locates all the records that satisfy a search condition
11. FindOrdered: retrieves the records in a file in some specified order
12. Reorganize: reorganizes records in a file according to some field

File Organization
The organization of records in a file affects how easily we can preform operations on them
(e.g., searching, inserting, deleting).

The simplest way is a heap, where records are stored in the order they are inserted, and
records are inserted at the end of the file. This makes insertion very efficient 𝑂(1).
However, searching for a record would require a linear search 𝑂(𝑛).

Alternatively the records in a file can be ordered based on some field value (e.g.,
alphabetical order based on name, increasing order based on ID), this field value is called
the ordering key. This makes reading records in the order of the ordering key very efficient,
and makes searching (using the ordering key) more efficient as binary search can be used
𝑂(log 2 𝑛). However, insertion and deletion become more expensive as the order of the file
must be restored after each operation.

Hashing Techniques
Hashing is a type of file organization where a field from the records is chosen as the hash
field/hash key and put through a hash function to yield the address of the disk block where
that record is stored.
We implement hashing using a hash table which has two fields: one for the record and the
other for the hash value, two records might produce the same hash value, this is called a
collision. Collisions can be resolved using several techniques:

1. Open Addressing: when a collision occurs, subsequent positions are checked until
an empty position is found.
2. Chaining: when a collision occurs, the new record is placed in an overflow area and
connected using a pointer.
3. Multiple Hashing: when a collision occurs, a second hash function is used and the
record is placed at (𝑓𝑖𝑟𝑠𝑡 ℎ𝑎𝑠ℎ(𝑘𝑒𝑦 ) + 𝑖(𝑠𝑒𝑐𝑜𝑛𝑑 ℎ𝑎𝑠ℎ(𝑘𝑒𝑦 )) where 𝑖 is the number of
attempts

Hashing is implemented both internally on records to map them to addresses and

externally on files to map them to disk buckets/blocks.

RAID Technology
Redundant Arrays of Independent Disks (RAID) used to increase disk speed and access
times through a technique called data striping. Data striping splits data to several disks to
make them preform as one large disk. Striping can be done at the block or bit level.

When retrieving the data for a certain file all 4 disks participate in the operation making the
access time faster by a factor of 4.

RAID can also be used to improve reliability through mirroring, where data is written
redundantly to multiple disks. Or through storing reconstruction information (parity bits or
Hamming codes) on a separate disk.

Using RAID can also boost performance as it increases the number of read/write requests
that can be processed at once. If data is mirrored RAID allows for parallel access to the
same data, and if data is only split it can still access different parts of a file simultaneously.

There are 7 levels of RAID organization:

- Level 0: data striping across 2 or more disks with no redundancies

- Level 1: redundancy through mirroring. (used for critical applications as its easiest
to rebuild in case of failure)
- Level 2: redundancy through Hamming codes
- Level 3: using a parity disk (used for large volume storage, provides high transfer
rates)
- Level 4: block-level data striping
- Level 5: block-level data striping and distribution of parity information across disks
(used for large volume storage)
- Level 6: P+Q redundancy using Reed-Solomon codes
Selected Book Exercises
16.34. Consider a disk with the following characteristics (these are not parameters of any
particular disk unit): block size B = 512 bytes; interblock gap size G = 128 bytes; number of
blocks per track = 20; number of tracks per surface = 400. A disk pack consists of 15
double-sided disks.

a. What is the total capacity of a track, and what is its useful capacity (excluding interblock
gaps)?

b. How many cylinders are there?

c. What are the total capacity and the useful capacity of a cylinder?
d. What are the total capacity and the useful capacity of a disk pack?

Answers:

a) 𝑢𝑠𝑒𝑓𝑢𝑙 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦 𝑜𝑓 𝑡𝑟𝑎𝑐𝑘 = 𝑛𝑜. 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 𝑝𝑒𝑟 𝑡𝑟𝑎𝑐𝑘 × 𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒 𝑖𝑛 𝑏𝑦𝑡𝑒𝑠
= (20)(512)
= 10420 𝑏𝑦𝑡𝑒𝑠
b) 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟𝑠 𝑜𝑛 𝑡𝑟𝑎𝑐𝑘 = 𝑛𝑜. 𝑜𝑓 𝑡𝑟𝑎𝑐𝑘𝑠 𝑝𝑒𝑟 𝑑𝑖𝑠𝑘
= 400 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟𝑠
c) 1. 𝑡𝑜𝑡𝑎𝑙 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦 = 𝑛𝑜. 𝑜𝑓 𝑡𝑟𝑎𝑐𝑘𝑠 𝑝𝑒𝑟 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟 × 𝑡𝑜𝑡𝑎𝑙 𝑡𝑟𝑎𝑐𝑘 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦
𝑛𝑜. 𝑜𝑓 𝑡𝑟𝑎𝑐𝑘𝑠
=( ) (𝑡𝑜𝑡𝑎𝑙 𝑡𝑟𝑎𝑐𝑘 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦)
𝑛𝑜. 𝑜𝑓 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟𝑠
(400)(15)
=( ) ((20)(128 + 512))
400
= 192000 𝑏𝑦𝑡𝑒𝑠

2. 𝑢𝑠𝑒𝑓𝑢𝑙 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦 = 𝑛𝑜. 𝑜𝑓 𝑡𝑟𝑎𝑐𝑘𝑠 𝑝𝑒𝑟 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟 × 𝑢𝑠𝑒𝑓𝑢𝑙 𝑡𝑟𝑎𝑐𝑘 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦
𝑛𝑜. 𝑜𝑓 𝑡𝑟𝑎𝑐𝑘𝑠
=( ) (𝑢𝑠𝑒𝑓𝑢𝑙 𝑡𝑟𝑎𝑐𝑘 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦 )
𝑛𝑜. 𝑜𝑓 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟𝑠
= (15)((20)(512))
= 153600 𝑏𝑦𝑡𝑒𝑠

d) 1. 𝑡𝑜𝑡𝑎𝑙 𝑑𝑖𝑠𝑘 𝑝𝑎𝑐𝑘 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦 = 𝑡𝑜𝑡𝑎𝑙 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟 𝑐𝑎𝑝𝑐𝑖𝑡𝑦 × 𝑛𝑜. 𝑜𝑓 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟𝑠

= (192000)(400)
= 76800000 𝑏𝑦𝑡𝑒𝑠 𝑜𝑟 76800 𝐾𝑏𝑦𝑡𝑒𝑠
2. 𝑢𝑠𝑒𝑓𝑢𝑙 𝑑𝑖𝑠𝑘 𝑝𝑎𝑐𝑘 𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦 = 𝑢𝑠𝑒𝑓𝑢𝑙 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟 𝑐𝑎𝑝𝑐𝑖𝑡𝑦 × 𝑛𝑜. 𝑜𝑓 𝑐𝑦𝑙𝑖𝑛𝑑𝑒𝑟𝑠
= (153000)(400)
= 61200000 𝑏𝑦𝑡𝑒𝑠 𝑜𝑟 61200 𝐾𝑏𝑦𝑡𝑒𝑠

16.35. A file has r = 20,000 STUDENT records of fixed length. Each record has the following
fields: Name (30 bytes), Ssn (9 bytes), Address (40 bytes), PHONE (10 bytes), Birth_date (8
bytes), Sex (1 byte), Major_dept_code (4 bytes), Minor_dept_code (4 bytes), Class_code (4
bytes, integer), and Degree_program (3 bytes). An additional byte is used as a deletion
marker. The file is stored on the disk whose parameters are given in Exercise 16.27.

a. Calculate the record size R in bytes.

b. Calculate the blocking factor bfr and the number of file blocks b, assuming an unspanned
organization.
Answers:

a) 𝑟𝑒𝑐𝑜𝑟𝑑 𝑠𝑖𝑧𝑒 = ∑ 𝑠𝑖𝑧𝑒 𝑜𝑓 𝑓𝑖𝑒𝑙𝑑𝑖

= 30 + 9 + 40 + 10 + 8 + 1 + 4 + 4 + 4 + 3 + 1
= 114 𝑏𝑦𝑡𝑒𝑠
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
b) 1. 𝑏𝑙𝑜𝑐𝑘𝑖𝑛𝑔 𝑓𝑎𝑐𝑡𝑜𝑟 = 𝑟𝑒𝑐𝑜𝑟𝑑 𝑠𝑖𝑧𝑒
512
=⌊ ⌋
114
= 4 𝑟𝑒𝑐𝑜𝑟𝑑𝑠 𝑝𝑒𝑟 𝑏𝑙𝑜𝑐𝑘
𝑛𝑜. 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠
2. 𝑛𝑜. 𝑜𝑓 𝑓𝑖𝑙𝑒 𝑏𝑙𝑜𝑐𝑘𝑠 =
𝑏𝑙𝑜𝑐𝑘𝑖𝑛𝑔 𝑓𝑎𝑐𝑡𝑜𝑟
20000
=
4
= 5000 𝑏𝑙𝑜𝑐𝑘𝑠

16.48. Write program code to access individual fields of records under each of the following
circumstances. For each case, state the assumptions you make concerning pointers,
separator characters, and so on. Determine the type of information needed in the file
header in order for your code to be general in each case.

a. Fixed-length records with unspanned blocking

b. Fixed-length records with spanned blocking

Answers:

a) Assumptions: blocking factor is a whole number, file header contains block size and
record size, records represented as arrays (can be accessed using an index)

File definition with metadata/header:

class File :
def init(self, record_size, block_size):
self.record_size = record_size
self.block_size = block_size
self.blocking_factor = block_size // record_size
self.data = []
Method for field access:
def access_field(self, record_num, field_offset, field_length):
# Calculate which block the record is in
block = record_num // self.blocking_factor
# Calculate where the record is in the block as an offset from the start
of the block
record_offset = (record_num % self.blocking_factor) * self.record_size
# Calculate the start of the field as an offset from the start of the file
field_start = block_num * self.block_size + record_start + field_offset
# Calculate the end of the field
field_end = field_start + field_length
# Return the field's content
return self.data[field_start:field_end]

b) Assumptions: file header contains block size, record size, and where the first record is
in the block
The field access method itself would be similar to the unspanned version. However we’ll
have specify how to get data for records that span.
File definition with metadata/header:
class File:
def __init__(self, record_size, block_size, first_record_offset):
self.record_size = record_size
self.block_size = block_size
self.blocking_factor = self.block_size // self.record_size
self.first_record_offset = first_record_offset
self.data = []
Method for record access
def access_record(self, record_num):
# Calculate where the record is
record_offset = self.first_record_offset + (record_num *
self.record_size)
# Calculate which block the record starts in
start_block = record_offset // block size
# Calculate where the record is in the start block
record_offset_block = record_offset - (block * block_size)
# Calculate which block the record ends in
end_block = (record_offset + self.record_size) // block_size
# Check if the record spans
spans = False if start_block == end block else True

if spans:
# Get the data from both blocks
start_block_data = access_block(self, start_block)
end_block_data = access_block(self, end_block)
# Calculate where the record ends in the second block
record_end = self.record_size - record_offset_block
# Return the record
return (start_block_data[record_offset_block:] +
end_block_data[:record_end]
else:
# Get the data from the one block
start_block_data = access_block(self, start_block)
# Return the record
return start_block_data[record_offset_block: record_offset_block +
self.record_size]
 16.38. A PARTS file with Part# as the hash key includes records with the following
Part# values: 2369, 3760, 4692, 4871, 5659, 1821, 1074, 7115, 1620, 2428, 3943, 4750, 6975,
4981, and 9208. The file uses eight buckets, numbered 0 to 7. Each bucket is one disk
block and holds two records. Load these records into the file in the given order, using
the hash function h(K) = K mod 8. Calculate the average number of block accesses for a
random retrieval on Part#.

Answer:
ℎ𝑎𝑠ℎ(𝑘𝑒𝑦) = 𝑘𝑒𝑦 𝑚𝑜𝑑 8

Hash table: key hash(key)

2369 1
3760 0 Buckets 4 and 7 are overflowed and
4692 4 will need an additional block access,
4871 7 the rest of the records will only need
5659 3 one block access
1821, 5 13 + 2 + 2
1074 2 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 =
15
7115 3 = 1.133
1620 4
2428 4
3943 7
4750 6
6975 7
4981 5
9208 0

Lecture 3/4
Indexing
An index is an ordered file used to speed up data access and eliminate unnecessary I/O
operations.

The index is defined on a field within the file called the index field, for each index field the
index stores a pointer to the disk block that the record is in. The index for a file is much
smaller than the file itself so it is usually kept in memory.

index
Disk blocks
When a record needs to be retrieved a binary search is done in the index, the block
containing the record is loaded into memory, and the block is searched sequentially for the
record.

Sparse and Dense Indexes

A dense index has an index entry for every record in the data file. A dense index can be
made on an unsorted file.

A sparse index has index entries for only some of the records in the data file. A sparse
index must be made on a ordered file.

Primary Indexes
A primary index is a sparse index made on a data file that is ordered on a primary key. It
has an index entry for every block in the data file. The first record in the block is called the
block anchor.

The issue with primary indexing is inserting or deleting records, since it would move all the
block anchors.
Clustering Indexes
A clustering index is sparse index made on a data file that is ordered on a non-key field
(not unique), this is called the clustering field. It has an index entry for each distinct value in
the clustering field.

Clustering indexes still have the problem of insertion and deletion. The problem can be
alleviated by giving each distinct value in the clustering field it’s own block or series of
blocks.

Secondary Index
A secondary index is a dense index that provides a secondary means to access a datafile
which already has a primary index. The index field for a secondary index can be any key
field in the data file.

A secondary index can also be made on a non-key field in the data file, this can be done in 2
ways:

1. Include all the records with the same index field value.
2. Only include distinct values in the index, but for each index entry keep a list/block of
pointers to the blocks with the same index values
Option 1:

Option 2:
Multilevel Indexes
A multilevel index is an index created on single-level index. The original index file is called
the first-level index and the index to the index is called the second level index and so on.
We can create keep adding index levels indexes as long as the index is larger than one disk
block.

A multilevel index makes searching for records much faster, because we can eliminate
more of the search space faster than with just one index.
Dynamic Multilevel Indexes Using Binary Trees
A dynamic multilevel index is an index that can adapt to insertions and deletions to keep
access to records as efficient as possible.

A binary tree is a hierarchal data structure where each node has at most 2 children. Its
structure includes the root node, internal nodes, and leaf nodes. Binary trees also have
balancing conditions, meaning they can adjust to insertions and deletions to remain
efficient.

When using binary trees for multilevel indexing the data/block pointers can be at any node
(except the root), this makes random and range queries require many I/O operations.

Dynamic Multilevel Indexes Using B+ Trees

B+ trees have the same structure as binary trees, but the internal nodes are used only for
navigation (using keys), with data pointers only at the leaf nodes.

B+ trees are more efficient for random and range queries.

Selected Book Exercises
17.18. Consider a disk with block size B = 512 bytes. A block pointer is P = 6 bytes long, and a
record pointer is PR = 7 bytes long. A file has r = 30,000 EMPLOYEE records of fixed length.
Each record has the following fields: Name (30 bytes),Ssn (9 bytes), Department_code (9
bytes), Address (40 bytes), Phone (10 bytes), Birth_date (8 bytes), Sex (1 byte), Job_code (4
bytes), and Salary (4 bytes, real number). An additional byte is used as a deletion marker.

a. Calculate the record size R in bytes.

b. Calculate the blocking factor bfr and the number of file blocks b, assuming an unspanned
organization.

c. Suppose that the file is ordered by the key field Ssn and we want to construct a primary
index on Ssn. Calculate (i) the index blocking factor bfri (which is also the index fan-out fo);
(ii) the number of first-level index entries and the number of first-level index blocks; (iii)
the number of levels needed if we make it into a multilevel index; (iv) the total number of
blocks required by the multilevel index; and (v) the number of block accesses needed to
search for and retrieve a record from the file—given its Ssn value—using the primary index.

d. Suppose that the file is not ordered by the key field Ssn and we want to construct a
secondary index on Ssn. Repeat the previous exercise (part c) for the secondary index and
compare with the primary index.

e. Suppose that the file is not ordered by the nonkey field Department_code and we want to
construct a secondary index on Department_code, using option 3 of Section 17.1.3, with an
extra level of indirection that stores record pointers. Assume there are 1,000 distinct values
of Department_code and that the EMPLOYEE records are evenly distributed among these
values. Calculate (i) the index blocking factor bfri (which is also the index fan-out fo); (ii) the
number of blocks needed by the level of indirection that stores record pointers; (iii) the
number of first-level index entries and the number of first-level index blocks; (iv) the
number of levels needed if we make it into a multilevel index; (v) the total number of blocks
required by the multilevel index and the blocks used in the extra level of indirection; and (vi)
the approximate number of block accesses needed to search for and retrieve all records in
the file that have a specific Department_code value, using the index.

f. Suppose that the file is ordered by the nonkey field Department_code and we want to
construct a clustering index on Department_code that uses block anchors (every new value
of Department_code starts at the beginning of a new block). Assume there are 1,000 distinct
values of Department_code and that the EMPLOYEE records are evenly distributed among
these values. Calculate (i) the index blocking factor bfri (which is also the index fan-out fo);
(ii) the number of first-level index entries and the number of first-level index blocks; (iii)
the number of levels needed if we make it into a multilevel index; (iv) the total number of
blocks required by the multilevel index; and (v) the number of block accesses needed to
search for and retrieve all records in the file that have a specific Department_code value,
using the clustering index (assume that multiple blocks in a cluster are contiguous).
g. Suppose that the file is not ordered by the key field Ssn and we want to construct a B+-
tree access structure (index) on Ssn. Calculate (i) the orders p and pleaf of the B+-tree; (ii)
the number of leaf-level blocks needed if blocks are approximately 69% full (rounded up for
convenience); (iii) the number of levels needed if internal nodes are also 69% full (rounded
up for convenience); (iv) the total number of blocks required by the B+-tree; and (v) the
number of block accesses needed to search for and retrieve a record from the file—given
its Ssn value—using the B+-tree.

h. Repeat part g, but for a B-tree rather than for a B+-tree. Compare your results for the B-
tree and for the B+-tree.

Answers:

a) 𝑟𝑒𝑐𝑜𝑟𝑑 𝑠𝑖𝑧𝑒 = ∑ 𝑠𝑖𝑧𝑒 𝑜𝑓 𝑓𝑖𝑒𝑙𝑑𝑖

= 30 + 9 + 9 + 40 + 10 + 8 + 1 + 4 + 4 + 1
= 116 𝑏𝑦𝑡𝑒𝑠
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
b) 𝑏𝑓𝑟 = ⌊ ⌋
𝑟𝑒𝑐𝑜𝑟𝑑 𝑠𝑖𝑧𝑒
512
= ⌊ ⌋
116
=4
# 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠
# 𝑜𝑓 𝑓𝑖𝑙𝑒 𝑏𝑙𝑜𝑐𝑘𝑠 = 𝑏𝑓𝑟
30,000
=
4
= 7,500 𝑓𝑖𝑙𝑒 𝑏𝑙𝑜𝑐𝑘𝑠
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
c) (i) 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟 = ⌊ ⌋
𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑦 𝑠𝑖𝑧𝑒
512
⌋ =⌊
9+6
= 34
(ii) # 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑖𝑒𝑠 = # 𝑜𝑓 𝑓𝑖𝑙𝑒 𝑏𝑙𝑜𝑐𝑘𝑠
= 7,500 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑖𝑒𝑠
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑖𝑟𝑒𝑠
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑏𝑙𝑜𝑐𝑘𝑠 = ⌈ 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟
⌉
7,500
=⌈ ⌉
34
= 221 𝑏𝑙𝑜𝑐𝑘𝑠
(iii) 𝑙𝑒𝑣𝑒𝑙 1: 221 𝑏𝑙𝑜𝑐𝑘𝑠
221
𝑙𝑒𝑣𝑒𝑙 2: ⌈ ⌉ = 7 𝑏𝑙𝑜𝑐𝑘𝑠
34
7
𝑙𝑒𝑣𝑒𝑙 3: ⌈34⌉ = 1 𝑏𝑙𝑜𝑐𝑘
3 𝑙𝑒𝑣𝑒𝑙𝑠

(iv) # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 = ∑ # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 𝑎𝑡 𝑙𝑒𝑣𝑒𝑙𝑖

= 221 + 7 + 1
= 229 𝑏𝑙𝑜𝑐𝑘𝑠
(v) 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 = 3 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 𝑓𝑜𝑟 𝑒𝑎𝑐ℎ 𝑙𝑒𝑣𝑒𝑙 + 1 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠 𝑓𝑜𝑟 𝑑𝑎𝑡𝑎
= 4 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠
 𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
d) (i) 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟 = ⌊ ⌋
𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑦 𝑠𝑖𝑧𝑒
512
= ⌊ ⌋
9+7
= 32
(ii) # 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑖𝑒𝑠 = # 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠
= 30,000 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑖𝑒𝑠
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑖𝑟𝑒𝑠
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑏𝑙𝑜𝑐𝑘𝑠 = 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟
30,000
=⌈ ⌉
32
= 938 𝑏𝑙𝑜𝑐𝑘𝑠
(iii) 𝑙𝑒𝑣𝑒𝑙 1: 938 𝑏𝑙𝑜𝑐𝑘𝑠
938
𝑙𝑒𝑣𝑒𝑙 2: ⌈ ⌉ = 30 𝑏𝑙𝑜𝑐𝑘𝑠
32
30
𝑙𝑒𝑣𝑒𝑙 3: ⌈32⌉ = 1 𝑏𝑙𝑜𝑐𝑘
3 𝑙𝑒𝑣𝑒𝑙𝑠

(iv) # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 = ∑ # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 𝑎𝑡 𝑙𝑒𝑣𝑒𝑙𝑖

= 938 + 30 + 1
= 969 𝑏𝑙𝑜𝑐𝑘𝑠
(v) 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 = 3 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 𝑓𝑜𝑟 𝑒𝑎𝑐ℎ 𝑙𝑒𝑣𝑒𝑙 + 1 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠 𝑓𝑜𝑟 𝑑𝑎𝑡𝑎
= 4 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
e) (i) 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟 = ⌊𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑦 𝑠𝑖𝑧𝑒 ⌋
512
= ⌊ ⌋
9+6
= 34
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
(ii) # 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠 𝑝𝑒𝑟 𝑏𝑙𝑜𝑐𝑘 = ⌊ ⌋
𝑠𝑖𝑧𝑒 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑 𝑝𝑜𝑖𝑛𝑡𝑒𝑟
512
=⌊ ⌋
7
= 73 𝑟𝑒𝑐𝑜𝑟𝑑 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠
# 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠
# 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠 𝑝𝑒𝑟 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑣𝑎𝑙𝑢𝑒 = # 𝑜𝑓 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑣𝑎𝑙𝑢𝑒 𝑖𝑛 𝑖𝑛𝑑𝑒𝑥 𝑓𝑖𝑒𝑙𝑑
30,000
=
1,000
= 30 𝑟𝑒𝑐𝑜𝑟𝑑 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠
𝑒𝑎𝑐ℎ 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑣𝑎𝑙𝑢𝑒 𝑖𝑛 𝑖𝑛𝑑𝑒𝑥 𝑓𝑖𝑒𝑙𝑑 𝑔𝑒𝑡𝑠 1 𝑏𝑙𝑜𝑐𝑘 ∴ 1,000 𝑏𝑙𝑜𝑐𝑘𝑠
(iii) # 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑖𝑒𝑠 = # 𝑜𝑓 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑣𝑎𝑙𝑢𝑒𝑠 𝑖𝑛 𝑖𝑛𝑑𝑒𝑥 𝑓𝑖𝑒𝑙𝑑
= 1,000
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑖𝑟𝑒𝑠
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑏𝑙𝑜𝑐𝑘𝑠 = ⌈ 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟
⌉
1,000
=⌈ ⌉
34
= 30
(iv) 𝑙𝑒𝑣𝑒𝑙 1: 30
30
𝑙𝑒𝑣𝑒𝑙 2: ⌈34⌉ = 1
2 𝑙𝑒𝑣𝑒𝑙𝑠
(v) # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 = ∑ # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 𝑎𝑡 𝑙𝑒𝑣𝑒𝑙𝑖 + 𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛 𝑏𝑙𝑜𝑐𝑘𝑠
= (30 + 1) + (1,000)
= 1,031 𝑏𝑙𝑜𝑐𝑘𝑠
 (vi) 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 = 1 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠 𝑓𝑜𝑟 𝑓𝑖𝑟𝑠𝑡 𝑖𝑛𝑑𝑒𝑥 𝑙𝑒𝑣𝑒𝑙 +
1 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠 𝑓𝑜𝑟 𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛 + 30 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 𝑟𝑒𝑐𝑜𝑟𝑑 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠
= 32 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠

𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
f) (i) 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟 = ⌊𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑦 𝑠𝑖𝑧𝑒 ⌋
512
=⌊
⌋
9+6
= 34
(ii) # 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑟𝑖𝑒𝑠 = # 𝑜𝑓 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑣𝑎𝑙𝑢𝑒𝑠 𝑖𝑛 𝑖𝑛𝑑𝑒𝑥 𝑓𝑖𝑒𝑙𝑑
= 1,000
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑒𝑛𝑡𝑖𝑟𝑒𝑠
# 𝑜𝑓 𝑖𝑛𝑑𝑒𝑥 𝑏𝑙𝑜𝑐𝑘𝑠 = ⌈ 𝑖𝑛𝑑𝑒𝑥 𝑏𝑓𝑟
⌉
1,000
=⌈ ⌉
34
= 30
(iii) 𝑙𝑒𝑣𝑒𝑙 1: 30
30
𝑙𝑒𝑣𝑒𝑙 2: ⌈34⌉ = 1
2 𝑙𝑒𝑣𝑒𝑙𝑠
(iv) # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 = ∑ # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 𝑎𝑡 𝑙𝑒𝑣𝑒𝑙𝑖
= 30 + 1
= 31 𝑏𝑙𝑜𝑐𝑘𝑠
# 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠 𝑤𝑖𝑡ℎ 𝑠𝑎𝑚𝑒 𝑣𝑎𝑙𝑢𝑒
(v) # 𝑜𝑓 𝑏𝑙𝑜𝑐𝑘𝑠 𝑝𝑒𝑟 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑣𝑎𝑙𝑢𝑒 = ⌈ ⌉
𝑏𝑓𝑟
30
⌉ =⌈
4
=8
𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠 = 1 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠 𝑓𝑜𝑟 𝑓𝑖𝑟𝑠𝑡 𝑖𝑛𝑑𝑒𝑥 𝑙𝑒𝑣𝑒𝑙 + 8 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠
= 9 𝑏𝑙𝑜𝑐𝑘 𝑎𝑐𝑐𝑒𝑠𝑠𝑒𝑠
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
g) (i) 𝑜𝑟𝑑𝑒𝑟 𝑜𝑓 𝑝 = ⌊𝑠𝑖𝑧𝑒 𝑜𝑓 𝑒𝑛𝑡𝑟𝑦 ⌋
512
= ⌊ ⌋
9+6
= 34
𝑏𝑙𝑜𝑐𝑘 𝑠𝑖𝑧𝑒
𝑜𝑟𝑑𝑒𝑟 𝑜𝑓 𝑝𝑙𝑒𝑎𝑓 = ⌊𝑠𝑖𝑧𝑒 𝑜𝑓 𝑒𝑛𝑡𝑟𝑦 ⌋
512
=⌊
⌋
9+7
= 32
(ii) # 𝑜𝑓 𝑙𝑒𝑎𝑓 𝑙𝑒𝑣𝑒𝑙 𝑏𝑙𝑜𝑐𝑘 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠 = (𝑝𝑙𝑒𝑎𝑓 )(𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦)
= (32)(0.69)
= 22 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠
# 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠
# 𝑜𝑓 𝑙𝑒𝑎𝑓 𝑙𝑒𝑣𝑒𝑙 𝑏𝑙𝑜𝑐𝑘𝑠 = ⌈# 𝑜𝑓 𝑙𝑒𝑎𝑓 𝑙𝑒𝑣𝑒𝑙 𝑏𝑙𝑜𝑐𝑘 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠 ⌉
30,000
= ⌈ ⌉
22
= 1364 𝑙𝑒𝑎𝑓 𝑙𝑒𝑣𝑒𝑙 𝑏𝑙𝑜𝑐𝑘𝑠
(iii) # 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 𝑛𝑜𝑑𝑒 𝑝𝑜𝑖𝑛𝑡𝑒𝑟𝑠 = (𝑝)(𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦)
= (34)(0.69) = 23
ℎ𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝐵+ 𝑡𝑟𝑒𝑒 = ⌈log 𝑝 ((# 𝑜𝑓 𝑙𝑒𝑎𝑓 𝑙𝑒𝑣𝑒𝑙 𝑏𝑙𝑜𝑐𝑘𝑠)(𝑝 − 1) + 1) − 1⌉
= ⌈log 23 ((1364)(23 − 1) + 1) − 1⌉
= 3 𝑙𝑒𝑣𝑒𝑙𝑠
Lecture 5
Relational Algebra
The basic set of operations for the formal relational model is the relational algebra. These
operations enable a user to specify basic retrieval requests as relational algebra
expressions. The result of a retrieval query is a new relation.

COMPANY Database

SELECT Operation
The SELECT operation is used to choose a subset of the tuples/rows from a relation that
satisfies a selection condition.

The SELECT operation is denoted by 𝜎<𝑠𝑒𝑙𝑒𝑐𝑡 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛> (𝑅)

Ex.

Select employees who work in department no. 4:

𝜎𝐷𝑛𝑜=4 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸)

Select employees whose salary is greater than 20,000:

𝜎𝑆𝑎𝑙𝑎𝑟𝑦>20000 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸)

Select all employees who either work in department 4 and make over $25,000 per year, or
work in department 5 and make over $30,000:
𝜎𝐷𝑛𝑜=4 𝐴𝑁𝐷 𝑆𝑎𝑙𝑎𝑟𝑦>25000) 𝑂𝑅 (𝐷𝑛𝑜=5 𝐴𝑁𝐷 𝑆𝑎𝑙𝑎𝑟𝑦>30000) (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸)
PROJECT Operation
The PROJECT operation is used to choose a subset of attributes/columns from a relation. It
also removes any duplicate tuples.

The PROJECT operation is denoted by 𝜋<𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒 𝑙𝑖𝑠𝑡> (𝑅)

Ex.

List each employee’s first name, last name, and salary:

𝜋𝑛𝑎𝑚𝑒,𝐹𝑛𝑎𝑚𝑒,𝑆𝑎𝑙𝑎𝑟𝑦 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝑆)

RENAME Operation
The RENAME operation is used to hold intermediate results in new relations.

The RENAME operation is denoted by either ← or 𝜌<𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒 𝑙𝑖𝑠𝑡> (𝑅)

Ex.

Select the first name, last name, and salary of employees who work in department 5:
𝑇𝐸𝑀𝑃 ← 𝜎𝐷𝑛𝑜=5 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 )
𝑅𝐸𝑆𝑈𝐿𝑇 ← 𝜋𝐹𝑛𝑎𝑚𝑒,𝐿𝑛𝑎𝑚𝑒,𝑆𝑎𝑙𝑎𝑟𝑦 (𝑇𝐸𝑀𝑃)

Rename the attributes in RESULT to First_name, Last_name, and Salary:

𝜌(𝐹𝑖𝑟𝑠𝑡_𝑛𝑎𝑚𝑒,𝐿𝑎𝑠𝑡_𝑛𝑎𝑚𝑒,𝑆𝑎𝑙𝑎𝑟𝑦) (𝑅𝐸𝑆𝑈𝐿𝑇)

UNION, INTERSECTION, and MINUS/DIFFERENCE Operations

These operations are standard operations on sets, they are binary (applied on 2 relations).
For all these operations the relations they are applied on must be type compatible (must
have same number of attributes)

For 2 type compatible relations R and S:

• UNION operation would produce a relation that includes all tuples in both relations
(eliminates duplicates). This is denoted by 𝑅 ∪ 𝑆
• INTERSECTION operation would produce a relation that includes all tuples common in
both relations. This is denoted by 𝑅 ∩ 𝑆
• MINUS/DIFFERENCE operation would produce a relation that includes all tuples that are
in one relation but not the other. This is denoted by 𝑅 − 𝑆
**𝑅 − 𝑆 ≠ 𝑆 − 𝑅

CARTESIAN PRODUCT/CROSS PRODUCT Operation

The CARTESIAN PRODUCT operation is used to produce a relation with all the possible
combinations of tuples between 2 relations.

The CARTESIAN PRODUCT operation is denoted by 𝑅1 × 𝑅2

Ex. Retrieve a list of the names of employees and their dependents:

𝐸𝑀𝑃𝑁𝐴𝑀𝐸𝑆 ← 𝜋𝑛𝑎𝑚𝑒,𝐹𝑛𝑎𝑚𝑒,𝑆𝑠𝑛 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝑆)
𝐸𝑀𝑃𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇𝑆 ← 𝐸𝑀𝑃𝑁𝐴𝑀𝐸𝑆 × 𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇
𝑇𝐸𝑀𝑃 ← 𝜎𝑆𝑠𝑛=𝐸𝑠𝑠𝑛 (𝐸𝑀𝑃𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇𝑆)
𝑅𝐸𝑆𝑈𝐿𝑇 ← 𝜋(𝐹𝑛𝑎𝑚𝑒,𝐿𝑛𝑎𝑚𝑒,𝐷𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡_𝑛𝑎𝑚𝑒) (𝑇𝐸𝑀𝑃)
Lecture 6
JOIN Operation
The JOIN operation is used to produce a relation that combines related tuples from two
relations based on some condition.

The JOIN operation is denoted by 𝑅 ⋈<𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛> 𝑆

Ex. Retrieve a list of the names of employees and their dependents:

𝐸𝑀𝑃𝑁𝐴𝑀𝐸𝑆 ← 𝜋𝑛𝑎𝑚𝑒,𝐹𝑛𝑎𝑚𝑒,𝑆𝑠𝑛 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝑆)
𝑇𝐸𝑀𝑃 ← 𝐸𝑀𝑃𝑁𝐴𝑀𝐸𝑆 ⋈𝑆𝑠𝑛=𝐸𝑠𝑠𝑛 𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇
𝑅𝐸𝑆𝑈𝐿𝑇 ← 𝜋(𝐹𝑛𝑎𝑚𝑒,𝐿𝑛𝑎𝑚𝑒,𝐷𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡_𝑛𝑎𝑚𝑒) (𝑇𝐸𝑀𝑃)

There are 3 types of JOIN operations:

1. THETAJOIN: <condition> is in the form 𝐴 𝜃 𝐵 where 𝐴 and 𝐵 are attributes in their

corresponding relations and 𝜃 is one of the following operators {=, <, ≤, >, ≥, ≠}
2. EQUIJOIN: <condition> is in the form 𝐴 = 𝐵
3. NATURALJOIN: uses no condition. Joins on attributes with the same name in both
relation. Denoted by 𝑅 ∗ − 𝑆

Aggregate Functions and Grouping

Aggregate functions are used to preform mathematical operations on values within tuples.
Some common functions are SUM, AVERAGE, MAXIMUM, MINIMUM, and COUNT.
The results from a aggregate function have to be grouped based on some attribute or
attributes.
Aggregate functions are denoted by <𝑔𝑟𝑜𝑢𝑝𝑖𝑛𝑔 𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒𝑠> ℑ<𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑙𝑖𝑠𝑡>

Ex. Retrieve the average salaries for each department:

𝑇𝐸𝑀𝑃 ← 𝐷𝑛𝑜 ℑ𝐶𝑂𝑈𝑁𝑇 𝑆𝑠𝑛,𝐴𝑉𝐸𝑅𝐴𝐺𝐸 𝑆𝑎𝑙𝑎𝑟𝑦 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 )
𝑅𝐸𝑆𝑈𝐿𝑇(𝐷𝑛𝑜, 𝑁𝑜_𝑜𝑓_𝑒𝑚𝑝𝑙𝑜𝑦𝑦𝑒𝑠, 𝐴𝑣𝑒𝑟𝑎𝑔𝑒_𝑠𝑎𝑙𝑎𝑟𝑦) ← 𝑇𝐸𝑀𝑃

OUTER JOIN Operation

OUTER JOIN operations are used similarly to the JOIN operation, with the difference that
they can include values that have no match.

There are 3 types of OUTER JOIN operations:

1. LEFT OUTER JOIN: keeps every tuple in the left/first relation. Denoted by 𝑅 ⟕ 𝑆
2. RIGHT OUTER JOIN: Keeps every tuple in the right/second relation. Denoted by 𝑅 ⟖ 𝑆
3. FULL OUTER JOIN: keeps every tuple in both relations. Denoted by 𝑅 ⟗ 𝑆
Selected Book Exercises
8.15. Show the result of each of the sample queries in Section 8.5 as it would apply to the
database state in Figure 5.6.

Query 1. Retrieve the name and address of all employees who work for the ‘Research’
department.

Query 2. For every project located in ‘Stafford’, list the project number, the controlling
department number, and the department manager’s last name, address, and birth date.
Query 3. Find the names of employees who work on projects controlled by department
number 5.

Query 4. Make a list of project numbers for projects that involve an employee whose last
name is ‘Smith’, either as a worker or as a manager of the department that controls the
project.

Query 5. List the names of all employees with two or more dependents.

Query 6. Retrieve the names of employees who have no dependents.

Answers:

a)
Fname Lname Address
John Smith 731 Fondren, Houston, TX
Franklin Wong 638 Voss, Houston, TX
Ramesh Narayan 975 Fire Oak, Humble, TX
Joyce English 5631 Rice, Houston, TX

b)
Pnumber Dnum Lname Address Bdate
10 4 Wallace 291 Berry, 1941-06-20
Bellaire, TX
30 4 Wallace 291 Berry, 1941-06-20
Bellaire, TX

c)
Fname Lname

d)
Pnumber
1
2

e)
Fname Lname
Franklin Wong
John Smith

f)
Fname Lname
Alica Zelaya
Jeniffer Wallace
Ramesh Narayan
Joyce English
Ahmed Jabar
James Borg
8.16. Specify the following queries on the COMPANY relational database schema shown in
Figure 5.5 using the relational operators discussed in this chapter. Also show the result of
each query as it would apply to the database state in Figure 5.6.

a. Retrieve the names of all employees in department 5 who work more than 10 hours per
week on the ProductX project.

b. List the names of all employees who have a dependent with the same first name as
themselves.

c. Find the names of all employees who are directly supervised by ‘Franklin Wong’.

d. For each project, list the project name and the total hours per week (by all employees)
spent on that project.

e. Retrieve the names of all employees who work on every project.

f. Retrieve the names of all employees who do not work on any project.

g. For each department, retrieve the department name and the average salary of all
employees working in that department.

h. Retrieve the average salary of all female employees.

Answers

a) 𝑃𝑅𝑂𝐷𝑋𝑊𝑂𝑅𝐾𝐸𝑅𝑆 ← 𝜎𝑃𝑛𝑎𝑚𝑒=”𝑃𝑟𝑜𝑑𝑢𝑐𝑡𝑋” (𝑊𝑂𝑅𝐾𝑆𝑂𝑁⋈𝑃𝑛𝑜=𝑃𝑛𝑢𝑚𝑏𝑒𝑟 𝑃𝑅𝑂𝐽𝐸𝐶𝑇 )

𝑇𝐸𝑁𝐻𝑂𝑈𝑅𝑆 ← 𝜎𝐻𝑜𝑢𝑟𝑠>10 𝑃𝑅𝑂𝐷𝑋𝑊𝑂𝑅𝐾𝐸𝑅𝑆
𝑇𝐸𝑁𝐻𝑂𝑈𝑅𝑆𝐸𝑀𝑃𝑆 ← 𝑇𝐸𝑁𝐻𝑂𝑈𝑅𝑆 ⋈𝐸𝑠𝑠𝑛=𝑆𝑠𝑛 𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸
𝑅𝐸𝑆𝑈𝐿𝑇 = 𝜋𝐹𝑛𝑎𝑚𝑒,𝐿𝑛𝑎𝑚𝑒 (𝜎𝐷𝑛𝑜=5 𝑇𝐸𝑁𝐻𝑂𝑈𝑅𝑆𝐸𝑀𝑃𝑆)

b) 𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇 ← 𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ⋈𝑆𝑠𝑛=𝐸𝑠𝑠𝑛 𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇

𝑆𝐴𝑀𝐸𝑁𝐴𝑀𝐸 ← 𝜎𝐹𝑛𝑎𝑚𝑒=𝐷𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡_𝑛𝑎𝑚𝑒 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝐷𝐸𝑃𝐸𝑁𝐷𝐸𝑁𝑇)
𝑅𝐸𝑆𝑈𝐿𝑇 ← 𝜋𝐹𝑛𝑎𝑚𝑒,𝐿𝑁𝑎𝑚𝑒 (𝑆𝐴𝑀𝐸𝑁𝐴𝑀𝐸)

c) 𝐹𝑅𝐴𝑁𝐾𝐿𝐼𝑁𝑊𝑂𝑁𝐺 ← 𝜎𝐹𝑛𝑎𝑚𝑒=”𝐹𝑟𝑎𝑛𝑘𝑙𝑖𝑛”,𝐿𝑛𝑎𝑚𝑒=”𝑊𝑜𝑛𝑔” (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝑆)

𝑊𝑂𝑁𝐺𝑆𝑆𝑁 ← 𝜋𝑆𝑠𝑛 (𝐹𝑅𝐴𝑁𝐾𝐿𝐼𝑁𝑊𝑂𝑁𝐺 )
𝑊𝑂𝑁𝐺𝐸𝑀𝑃𝑆 ← 𝜎𝑆𝑢𝑝𝑒𝑟_𝑠𝑠𝑛=𝑊𝑂𝑁𝐺𝑆𝑆𝑁 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝑆)
𝑅𝐸𝑆𝑈𝐿𝑇 ← 𝜋𝐹𝑛𝑎𝑚𝑒.𝐿𝑛𝑎𝑚𝑒 (𝑊𝑂𝑁𝐺𝐸𝑀𝑃𝑆)

d)
Lecture 8
Query Processing
A query expressed in a high-level query language such as SQL must first be scanned,
parsed, and validated, before it can be optimized and executed.

The scanner identifies the query tokens—such as SQL keywords, attribute names, and
relation names—that appear in the text of the query.

The parser checks that the query syntax follows the syntax rules of the query language.

The query is also validated to check that all the attribute, and relation names are valid in
the database being queried.

A intermediate representation of the query is then created, called a query tree. The DBMS
devises an execution plan for the query (how to retrieve the results of the query from the
database). Choosing a suitable execution plan for a query is called query optimization.

Translating SQL Queries into Relational Algebra

An SQL query is first translated into an equivalent extended relational algebra expression
(in the form of query tree) that is then optimized.
SQL queries are decomposed into query blocks. A query block contains a single SELECT-
FROM-WHERE expression, as well as GROUP BY and HAVING clauses if these are part of
the block.
SELECT Lname, Fname
Ex.
FROM EMPLOYEE
WHERE Salary > c
SELECT Lname, Fname
FROM EMPLOYEE
WHERE Salary > ( SELECT MAX (Salary) Outer block
FROM EMPLOYEE
WHERE Dno=5 );
SELECT MAX (Salary)
FROM EMPLOYEE
WHERE Dno=5

Ex. cont. Inner block

SELECT MAX (Salary)

ℑ𝑀𝐴𝑋 𝑆𝑎𝑙𝑎𝑟𝑦 (𝜎𝐷𝑛𝑜=5 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ))
FROM EMPLOYEE
WHERE Dno=5

SELECT Lname, Fname

FROM EMPLOYEE 𝜋𝐿𝑛𝑎𝑚𝑒,𝐹𝑛𝑎𝑚𝑒 (𝜎𝑆𝑎𝑙𝑎𝑟𝑦>𝑐 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ))
WHERE Salary > c

The query optimizer chooses and execution plan for each query block.
Algorithms for SELECT Operation
S1. Linear search (brute force approach): retrieve every record in the file, and test
whether its attribute values satisfy the selection condition.

S2. Binary search: used if the selection condition contains an equality comparison on
which the file is ordered.

Ex.
𝜎𝑆𝑠𝑛="12345678" (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 )

*If EMPLOYEE table is ordered on Ssn

S3.
a) Using primary index: used if the selection condition involves an equality
comparison on a key attribute with a primary index.

Ex.
𝜎𝑆𝑠𝑛="12345678" (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 )

*If EMPLOYEE table has a primary index on Ssn

b) Using a hash key: used if the selection condition involves an equality comparison
on a key attribute with a hash key.

Both a and b retrieve a single record only.

S4. Using a primary index to retrieve multiple records: used if the comparison condition
is >, >=, <, or <= on a key field with a primary index.

Ex.
𝜎𝐷𝑛𝑜>5 (𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇)

*If DEPARTMENT has primary index on Dno

S5. Using a clustering index to retrieve multiple records: used if the selection condition
involves an equality comparison on a nonkey attribute with a clustering index.

Ex.
𝜎𝐷𝑛𝑜=5 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 )

*If EMPLOYEE has clustering index on Dno

S6. Using a secondary (B+-tree) index on an equality comparison: used if the indexing
field is a key (has unique values) or to retrieve multiple records if the indexing field
is not a key. Also used for comparisons involving >, >=, <, or <= and a large range of
values.

S7. Conjunctive selection: used if an attribute involved in any single simple condition in
the conjunctive select condition has an access path that permits the use of one of
the methods S2 to S6.

Use that condition to retrieve the records and then check whether each retrieved
record satisfies the remaining simple conditions in the conjunctive select
condition.
 S8. Conjunctive selection using a composite index: used if two or more attributes are
involved in equality conditions in the conjunctive select condition and a composite
index (or hash structure) exists on the combined fields.

Ex.
𝜎𝐸𝑠𝑠𝑛="12345678" 𝐴𝑁𝐷 𝑃𝑛𝑜=10 (𝑊𝑂𝑅𝐾𝑆_𝑂𝑁)

*If WORKS_ON has an index in the composite key (Essn, Pno)

S9. Conjunctive selection by intersection of record pointers: used if secondary indexes

(or other access paths) are available on more than one of the fields involved in
simple conditions in the conjunctive select condition, and if the indexes include
record pointers.

Each index can be used to retrieve the set of record pointers that satisfy the
individual condition. The intersection of these sets of record pointers gives the
record pointers that satisfy the conjunctive select condition.

*Assume each index is on a nonkey attribute

When the optimizer is choosing between multiple simple conditions in a conjunctive select
condition, it typically considers the selectivity of each condition. The selectivity (𝑠𝑙) is
defined as the ratio of the number of records that satisfy the condition to the total number
of records in the file.
𝑛𝑜. 𝑟𝑒𝑐𝑜𝑟𝑑𝑠 𝑡ℎ𝑎𝑡 𝑠𝑎𝑡𝑖𝑠𝑓𝑦 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛
𝑠𝑙 = 𝑠𝑙 ∈ [0,1]
𝑡𝑜𝑡𝑎𝑙 𝑛𝑜. 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠
The Query optimizer receives input from the system catalogue to estimate selectivity.

Algorithms for JOIN Operation

J1. Nested-loop join (brute force approach): For each record 𝑡 in 𝑅 (outer loop), retrieve
every record 𝑠 from 𝑆 (inner loop) and test whether the two records satisfy the join
condition 𝑡[𝐴] = 𝑠[𝐵].

J2. Index-based nested-loop join: used if an index (or hash key) exists for one of the
two join attributes.

retrieve each record 𝑡 in 𝑅 (loop over file 𝑅), and then use the access structure
(such as an index or a hash key) to retrieve directly all matching records 𝑠 from 𝑆
that satisfy 𝑠[𝐵] = 𝑡[𝐴].

J3. Sort-merge join: used if the records in the tables being queried are sorted by the
value of the join attributes. This allows for the most efficient implementation of the
join operation.

The performance of a join operation depends on the available buffer space, join selection
factor, and the type of join (inner or outer)
𝑛𝑜. 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠 𝑖𝑛 𝑓𝑖𝑙𝑒 1 𝑡ℎ𝑎𝑡 𝑤𝑖𝑙𝑙 𝑏𝑒 𝑗𝑜𝑖𝑛𝑒𝑑
𝑗𝑜𝑖𝑛 𝑠𝑒𝑙𝑒𝑐𝑡𝑖𝑜𝑛 𝑓𝑎𝑐𝑡𝑜𝑟 =
𝑛𝑜. 𝑜𝑓 𝑟𝑒𝑐𝑜𝑟𝑑𝑠 𝑖𝑛 𝑓𝑖𝑙𝑒 2 𝑡ℎ𝑎𝑡 𝑤𝑖𝑙𝑙 𝑏𝑒 𝑗𝑜𝑖𝑛𝑒𝑑
Algorithms for PROJECT Operation
Projecting a relation 𝑅 on a list of attributes. Duplicates can be eliminated by using the
DISTINCT keyword.

If < 𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒 𝑙𝑖𝑠𝑡 > has a key of relation 𝑅, extract all tuples from 𝑅 with only the values
for the attributes in < 𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒 𝑙𝑖𝑠𝑡 >.

If < 𝑎𝑡𝑡𝑟𝑖𝑏𝑢𝑡𝑒 𝑙𝑖𝑠𝑡 > does NOT include a key of relation 𝑅, duplicated tuples must be
removed from the results.

Duplicates are removed using either sorting or hashing. Using sorting the duplicates
appear consecutively, and can be removed easily. Using hashing each record is hashed and
inserted into a bucket of the hash file in memory, it is checked against those records
already in the bucket; if it is a duplicate, it is not inserted in the bucket.

Algorithms for SET Operations

Set operations—UNION, INTERSECTION, SET DIFFERENCE, and CARTESIAN PRODUCT—are
expensive to implement, since UNION, INTERSECTION, MINUS or SET DIFFERENCE must
always return distinct results.

CARTESIAN PRODUCT operation 𝑅 × 𝑆 is also expensive because its result includes a

record for each combination of records from 𝑅 and 𝑆. If 𝑅 has 𝑛 records and 𝑗 attributes
and 𝑆 has 𝑚 records and 𝑘 attributes, the result relation will have 𝑛 ∗ 𝑚 records and 𝑗 + 𝑘
attributes.

For UNION, INTERSECTION and SET DIFFERENCE operations, the two relations are sorted
on the same attributes and are scanned and merged concurrently. The result is then
extracted from the merged table according to the logic of the operation (e.g. UNION keeps
only unique tuples, INTERSECTION keeps only tuples that are in both relations, etc.)

Algorithms for Aggregate Operations

The aggregate operators (MIN, MAX, COUNT, AVERAGE, SUM), when applied to an entire
table, can be computed by a table scan or by using an appropriate index.
For MIN and MAX operations if a B+-tree exists on the attribute being aggregated the
minimum or maximum value is found by following the pointers in the tree.

AVERAGE, COUNT and SUM operations can also be computed using indexes. If a dense index
exists on the attribute being aggregated the computation is simply applied to the values in
the index. If the index on the attribute is nondense the actual number of records associated
with each index value must be used for a correct computation.
Selected Book Exercises
18.13. Consider SQL queries Q1 and Q8 in Chapter 6 and Q27 in Chapter 7.

a. Draw at least two query trees that can represent each of these queries. Under what
circumstances would you use each of your query trees?

Q1
SELECT Fname, Lname, Address
FROM EMPLOYEE, DEPARTMENT
WHERE Dname = "Research" AND Dnumber = Dno;

Q8
SELECT E.Fname, E.Lname, S.Fname, S.Lname,
FROM EMPLOYEE AS E, EMPLOYEE AS S
WHERE E.Super_ssn = S.Snn;

Q27
SELECT Pnumber, Pname, COUNT (*)
FROM PROJECT, WORKS_ON, EMPLOYEE
WHERE Pnumber = Pno AND Ssn = Essn AND Dno = 5
GROUP BY Pnumber, Pname;

Answers
Lecture 9
Query Optimization
Query optimization is an activity conducted by the query
optimizer in a DBMS to select the best available
strategy for executing the query.

After the high-level language (SQL) queries are

converted to query trees, heuristic functions are used
to transform them into a more efficient form. Once the
queries have been optimized they can be converted into
execution plans and executed.

The goal of query optimization is to select an execution

plan with the minimum cost.
𝑇𝑜𝑡𝑎𝑙 𝑐𝑜𝑠𝑡 = 𝐼/𝑂 𝑐𝑜𝑠𝑡 + 𝐶𝑃𝑈 𝑐𝑜𝑠𝑡

Cost is generally measured as total elapsed time for

answering a query. Typically disk access is the
predominant cost

Query Trees and Heuristics

A query tree is a tree data structure that corresponds to an extended relational algebra
expression. Inputs are represented as leaf nodes, and operations are represented as
internal nodes. Execution in the query tree starts at the leaf nodes and ends at the root
node.

Ex.
SELECT Pnumber, Dnum, Lname, Address, Bdate
FROM PROJECT, DEPARTMENT, EMPLOYEE
WHERE Dnum = Dnumber AND Mgr_ssn = Ssn AND
Plocation = "Stafford"
Many different relational algebra expressions—and hence many different query trees—can
be semantically equivalent; that is, they can represent the same query and produce the
same results.

The query parser will typically generate a standard initial query tree to correspond to an
SQL query, without doing any optimization. This query tree is never executed but is only
used by the optimizer to produce the final query tree.

Ex.

SELECT E.Lname
FROM EMPLOYEE E, WORKS_ON W, PROJECT P
WHERE P.Pname="Aquarius" AND P.Pnumber=W.Pno AND E.Essn=W.Ssn
AND E.Bdate > "1957-12-31";

First the SELECT operations are moved down the tree (b), then the more restrictive
SELECT operations are applied first (c), then the CARTESIAN PRODUCT and SELECT
operations can be replaced with JOIN (d), then PROJECT operations are moved down the
tree to make joins more efficient (e).
General Transformation Rules
1. Cascade of 𝜎. A conjunctive selection condition can be broken up into a cascade (that is,
a sequence) of individual σ operations:
𝜎𝑐1AND 𝑐2 AND …AND 𝑐𝑛 (𝑅) ≡ 𝜎𝑐1 (𝜎𝑐2 (… (𝜎𝑐𝑛 (𝑅)) … ))

2. Commutativity of 𝜎. The 𝜎 operation is commutative:

𝜎𝑐1 (𝜎𝑐2 (𝑅)) ≡ 𝜎𝑐2 (𝜎𝑐1 (𝑅))

3. Cascade of 𝜋. In a cascade (sequence) of 𝜋 operations, all but the last one can be
ignored:
𝜋𝐿𝑖𝑠𝑡1 (𝜋𝐿𝑖𝑠𝑡2 (… (𝜋𝐿𝑖𝑠𝑡𝑛 (𝑅)) … )) ≡ 𝜋𝐿𝑖𝑠𝑡1 (𝑅)

4. Commuting 𝜎 with 𝜋. If the selection condition 𝑐 involves only those attributes

𝐴1 , … , 𝐴𝑛 in the projection list, the two operations can be commuted:

𝜋𝐴1 ,𝐴2,… ,𝐴𝑛 (𝜎𝑐 (𝑅)) ≡ 𝜎𝑐 (𝜋𝐴1,𝐴2,… ,𝐴𝑛 (𝑅))

Summary of Heuristics
The main heuristic is to apply first the operations that reduce the size of intermediate
results. This means performing SELECT, and PROJECT operations as early as possible.
Another heuristic is to perform the most restrictive (produce the fewest tuples) SELECT
and JOIN operations first.

Execution Plans
An execution plan for a relational algebra expression represented as a query tree includes
information about the access methods available for each relation as well as the algorithms
to be used in computing the relational operators represented in the tree.

The approach taken for executing the query may specify a materialized or a pipelined
evaluation. In general a pipelined evaluation is preferred whenever feasible.

With materialized evaluation, the result of an operation is stored as a temporary relation

(that is, the result is physically materialized).

With pipelined evaluation, as the resulting tuples of an operation are produced, they are
forwarded directly to the next operation in the query sequence.

Nested Subquery Evaluation

The process of removing the nested query and converting the outer and inner query into
one block is called unnesting.

Queries involving a nested subquery connected by IN or ANY connector in SQL can always
be converted into a single block query.

Wherever possible, SQL optimizer tries to convert queries with nested subqueries into a
join operation.
Ex.

DEPARTMENT (Dnumber, Dname, Mgr_ssn, Mgr_start_date, Zipcode)

SELECT Fname, Lname, Salary

FROM EMPLOYEE E
WHERE EXISTS ( SELECT *
FROM DEPARTMENT D
WHERE D.Dnumber = E.Dno AND D.Zipcode=30332);

The nested subquery takes the E.Dno, the department where the employee works, as a
parameter and returns a true or false value as a function depending on whether the
department is located in zip code 30332.

SELECT Fname, Lname, Salary

FROM EMPLOYEE E, DEPARTMENT D
WHERE WHERE D.Dnumber = E.Dno AND D.Zipcode=30332
Here inner join is used, since D.Dnumber is unique and the join is an equi-join; this
guarantees that a tuple from relation EMPLOYEE will match with at most one tuple from
relation DEPARTMENT.

Subquery Merging Transformation

In Instances where a subquery appears in the FROM clause of a query, the subquery is
called an inline view. In such cases, the transformation of the query can be referred to as a
view-merging or subquery merging transformation.

Ex.
EMP (Ssn, Fn, Ln, Dno)
DEPT (Dno, Dname, Dmgrname, Bldg_id)
BLDG (Bldg_id, No_storeys, Addr, Phone)

SELECT E.Ln, V.Addr, V.Phone

FROM EMP E, ( SELECT D.Dno, D.Dname, B.Addr, B.Phone
FROM DEPT D, BLDG B
WHERE D.Bldg_id = B.Bldg_id ) V
WHERE V.Dno = E.Dno AND E.Fn = "John";

The query joins the EMP table with a view called V that provides the address
and phone of the building where the employee works. In turn, the view joins the
two tables DEPT and BLDG.

The view-merging operation merges the tables in the view with the tables from the
outer query block and produces the following query:

SELECT E.Ln, B.Addr, B.Phone

FROM EMP E, DEPT D, BLDG B
WHERE D.Bldg_id = B.Bldg_id AND D.Dno = E.Dno AND E.Fn = "John";

Three tables appear in the FROM clause, thus affording eight possible join orders and
indexes on Dno in DEPT, and Bldg_id in BLDG can be used for index-based nested loop
joins that were previously excluded.
Views containing SELECT-PROJECT-JOIN operations are considered simple views and can
always be subjected to this type of view-merging.

However, view-merging may be invalid under certain conditions where the view is more
complex and involves DISTINCT, OUTER JOIN, AGGREGATION, GROUP BY set operations, and
so forth.

When it comes to GROUP BY operations, delaying the GROUP BY operation after performing
joins may reduce the data subjected to grouping in case the joins have low join selectivity.
Alternately, performing early GROUP BY may reduce the amount of data subjected to
subsequent joins.

The optimizer considers execution plans with and without merging and compares their cost.

Ex.
SALES (Custid, Productid, Date, Qty_sold)
CUST (Custid, Custname, Country, Cemail)
PRODUCT (Productid, Pname, Qty_onhand)

The query: List customers from France who have bought more than 50 units
of a product “Ring_234” may be set up as follows:

CREATE VIEW CP_BOUGHT_VIEW AS

SELECT SUM (S.Qty_sold) as Bought, S.Custid, S.Productid
FROM SALES S
GROUP BY S.Custid, S.Productid;

SELECT C.Custid, C.Custname, C.Cemail

FROM CUST C, PRODUCT P, CP_BOUGHT_VIEW V1
WHERE P.Productid = V1.Productid AND C.Custid = V1.Custid AND V1.Bought >50
AND Pname = "Ring_234" AND C.Country = "France";

Before merging, the view V1 does grouping on the entire SALES table and materializes
the result, and it is expensive to do so.

The view V1 may be evaluated first and its results temporarily materialized, then the
query may be evaluated using the materialized view as one of the tables in the join
using a merging transformation.

SELECT C.Custid, C.Custname, C.Cemail

FROM CUST C, PRODUCT P, SALES S
WHERE P.Productid = S.Productid AND C.Custid = S.Custid AND
Pname = "Ring_234" AND C.Country = "France"
GROUP BY, P.Productid, P.rowid, C.rowid, C.Custid, C.Custname, C.Cemail
HAVING SUM (S.Qty_sold) > 50;

In the transformed query, the grouping is applied to the join of the three tables; in this
operation, a single product tuple is involved from the PRODUCT table, thus filtering the
data from SALES considerably.
Materialized Views
A view is defined in the database as a query, and a materialized view stores the results of
that query. Using materialized views to avoid some of the computation involved in a query is
a query optimization technique.

The main idea behind materialization is that it is much cheaper to read it when needed and
query against it than to recompute it from scratch. The savings can be significant when the
view involves costly operations like join, aggregation, and so forth.

The materialized view must be “maintained” so that it accurately reflects the query it’s
defined by when changes are made. Incremental view maintenance updates the view in
increments, accounting only for the changes since the last time the view was updated.

Cost-Based Query Optimization

In order for the query optimizer to compare execution strategies it needs accurate cost
estimates, it also has to limit the number of strategies it will consider so it doesn’t spend
too much time making cost estimates.

For compiled queries a more elaborate optimization considering many strategies can be
done, and the resulting execution strategy code is stored to be executed later. However, for
interpreted queries a full scale optimization would result in a slow response time, so a
partial less time consuming optimization works better.

Cost-based query optimization, as opposed to heuristic-based optimization, searches

through the equivalent expressions for a query block, calculating their predicted cost. This
way it can keep the cheaper alternatives and prune (not consider) the costlier ones

Cost-based query optimization considers the following components when calculating the
cost of a query:

1. Access cost to secondary storage. This is the cost of transferring (reading and writing)
data blocks between secondary disk storage and main memory buffers. *I/O Cost

2. Disk storage cost. This is the cost of storing on disk any intermediate files that are
generated by an execution strategy for the query.

3. Computation cost. This is the cost of performing in-memory operations on the records
within the data buffers during query execution e.g. searching, sorting, etc. *CPU Cost

4. Memory usage cost. This is the cost pertaining to the number of main memory buffers
needed during query execution.
5. Communication cost. This is the cost of shipping the query and its results from the
database site to the site or terminal where the query originated.
To estimate the costs of various execution strategies, we must keep track of any
information that is needed for the cost functions. This information may be stored in the
DBMS catalogue.

1. File size 4. Distinct values for an attribute

2. Organization (record size, blocking factor, etc.) 5. Attribute selectivity
3. Index information (no. of levels, no. of blocks, etc.)
Cost Functions
Notation:

𝐶𝑆𝑖 : Cost for method 𝑆𝑖 in block accesses

𝑟𝑋 : Number of records (tuples) in a relation 𝑋
𝑏𝑋 : Number of blocks occupied by relation 𝑋 (also referred to as 𝑏)
𝑏𝑓𝑟𝑋 : Blocking factor (i.e., number of records per block) in relation 𝑋
𝑠𝑙𝐴 : Selectivity of an attribute 𝐴 for a given condition
𝑠𝐴 : Selection cardinality of the attribute being selected (= 𝑠𝑙𝐴 ∗ 𝑟)
𝑥𝐴 : Number of levels of the index for attribute 𝐴
𝑏𝐼1 𝐴: Number of first-level blocks of the index on attribute 𝐴
𝑁𝐷𝑉 (𝐴, 𝑋): Number of distinct values of attribute 𝐴 in relation 𝑋

Cost functions for different SELECT algorithms:

S1. Linear search:

𝐶𝑆1 = 𝑏

S2. Binary search:

𝐶𝑆2 = log 2 𝑏 + ⌈𝑠/𝑏𝑓𝑟⌉

S3.

a) Using primary index for single record:

𝐶𝑆3𝑎 = 𝑥 + 1

b) Using a hash key for a single record:

𝐶𝑆3𝑏 = 1

S4. Using an primary index to retrieve multiple records:

𝑏
𝐶𝑆4 = 𝑥 +
2
S5. Using a clustering index to retrieve multiple records:
𝐶𝑆5 = 𝑥 + ⌈𝑠/𝑏𝑓𝑟⌉

S6. Using a secondary (B+-tree) index:

𝐶𝑆6 = 𝑥 + 1 + 𝑠 worst case
𝑏𝐼1 𝑟
𝐶𝑆6 = 𝑥 + + for range queries
2 2
Selected Book Exercises
19.15. Develop cost functions for the PROJECT, UNION, INTERSECTION, SET DIFFERENCE,
and CARTESIAN PRODUCT algorithms discussed in Section 19.4.

Answer

PROJECT:

Assuming the operation is performed on a relation 𝑅 for a key attribute, a read and
write operation is performed on each block giving:
𝐶𝜋 = 2𝑏𝑅

Assuming the operation is performed on a relation 𝑅 for a non-key attribute, the

relation will have to be sorted and duplicates will have to be sorted. So we would add
the cost of sorting as well as the extra block scan to remove duplicates giving:
𝐶𝜋 = 3𝑏𝑅 + 𝑘𝑏𝑅 ∗ log 2 𝑏𝑅

Where 𝑘 is the cost of sorting per block

UNION, INTERSECTION, and SET DIFFERENCE (Set Operations)

Assuming the operations are performed on 2 relations 𝑅 and 𝑆, all the set operations
are performed using a sort and scan algorithm giving:
𝐶𝑆𝐸𝑇 = 𝑘[(𝑏𝑅 + log 2 𝑏𝑅 ) + (𝑏𝑠 + log 2 𝑏𝑠 )] + 𝑏𝑅 + 𝑏𝑆 + 𝑏𝑅𝐸𝑆𝑈𝐿𝑇

CARTESIAN PRODUCT

Assuming the operation is performed on 2 relations 𝑅 and 𝑆, the cartesian product is

simply a join with 𝑗𝑠 = 1 giving:
|𝑅 | ∗ |𝑆 |
𝐶× = 𝑏𝑅 + (𝑏𝑅 ∗ 𝑏𝑆 ) + ( )
𝑏𝑓𝑟𝑅𝑆

19.21. Compare the cost of two different query plans for the following query:
𝜎𝑆𝑎𝑙𝑎𝑟𝑦< 40000 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ⋈𝐷𝑛𝑜=𝐷𝑛𝑢𝑚𝑏𝑒𝑟 𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇)

Use the database statistics shown in Figure 19.6.

Answer

Original query:
𝑄1 = 𝜎𝑆𝑎𝑙𝑎𝑟𝑦< 40000 (𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ⋈𝐷𝑛𝑜=𝐷𝑛𝑢𝑚𝑏𝑒𝑟 𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇)

Alternative query:
𝑄2 = 𝜎𝑆𝑎𝑙𝑎𝑟𝑦< 40000 (𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇 ⋈𝑀𝑔𝑟_𝑠𝑠𝑛=𝑆𝑠𝑛 𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸)

Cost of 𝑄1 :

First we calculate the cost of the JOIN operation:

1
𝑗𝑠 =
|𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇|
1
𝑗𝑠 =
50
𝑗𝑠 ∗ |𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇| ∗ |𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 |
𝐶𝐽𝑂𝐼𝑁 = 𝑏𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 + (𝑏𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ∗ 𝑏𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇 ) + ( )
𝑏𝑓𝑟𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸
1
50 ∗ 50 ∗ 1000
𝐶𝐽𝑂𝐼𝑁 = 2000 + (2000 ∗ 5) + ( ) = 2,502,000
4

Next we calculate the cost of the SELECT operation:

𝑏𝐼1 𝑟𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸
𝐶𝑆𝐸𝐿𝐸𝐶𝑇 = 𝑥 + +
2 2
50 10000
𝐶𝑆𝐸𝐿𝐸𝐶𝑇 = 1 + + = 5026
2 2
Total cost:
𝐶𝑄1 = 2,507,026
Cost of 𝑄2 :

First we calculate the cost of the JOIN operation:

1
𝑗𝑠 =
|𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸𝑆|
1
𝑗𝑠 =
10000
𝑗𝑠 ∗ |𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇| ∗ |𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 |
𝐶𝐽𝑂𝐼𝑁 = 𝑏𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 + (𝑏𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸 ∗ 𝑏𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇 ) + ( )
𝑏𝑓𝑟𝐷𝐸𝑃𝐴𝑅𝑇𝑀𝐸𝑁𝑇𝐸𝑀𝑃𝐿𝑂𝑌𝐸𝐸
1
10000 ∗ 50 ∗ 1000
𝐶𝐽𝑂𝐼𝑁 = 2000 + (2000 ∗ 5) + ( ) = 12,012
4

Next we calculate the cost of the SELECT operation:

𝐶𝑆𝐸𝐿𝐸𝐶𝑇 = 𝑏
𝐶𝑆𝐸𝐿𝐸𝐶𝑇 = 10000
Total cost:
𝐶𝑄2 = 22,012

𝑄2 is much more efficient

Lecture 10
Transactions
A transaction is an atomic unit of work that should either be completed in its entirety or not
done at all. A transaction can include one or more database access operations.

A transaction goes through several states based on the following operations:

1. BEGIN_TRANSACTION 4. COMMIT TRANSACTION

2. READ or WRITE 5. ROLLBACK/ABORT
3. END TRANSACTION

If the database operations in a transaction do not update the database but only retrieve
data, the transaction is called a read-only transaction; otherwise it is known as a read-

write transaction

Transactions follow ACID properties:

• Atomicity: transactions must either be preformed entirely or not at all

• Consistency: the execution of a transaction must take the database from one consistent
state to another

• Isolation: transactions should appear to execute in isolation even if executing in parallel.

i.e. transactions should not affect each other during execution

• Durability: once a transaction is committed the changes it has made to the database
must never be lost
Transaction Processing
Transaction processing systems are large multiuser database systems supporting
thousands of concurrent transactions.

Concurrency can be achieved through interleaved processing, where the execution of

several processes is interleaved in a single CPU. *assumed method

Alternatively, concurrency can be achieved by parallel processing, where processes are

executed concurrently on several CPUs.

Interleaved Processing Parallel Processing

System Log
To be able to recover from failures that affect transactions, the system maintains a log to
keep track of all transaction operations that affect the values of database.

The log is a sequential, append-only file that is kept on disk, so it is not affected by any type
of failure except for disk or catastrophic failure.

The types of entries (log records) in a log file are:

1. [start_transaction, T]. Indicates that transaction T has started execution.

2. [write_item, T, X, old_value, new_value]. Indicates that transaction T has changed the

value of database item X from old_value to new_value.

3. [read_item, T, X]. Indicates that transaction T has read the value of database item X.

4. [commit, T]. Indicates that transaction T has completed successfully, and affirms that its
effect can be committed (recorded permanently) to the database.

5. [abort, T]. Indicates that transaction T has been aborted.

Concurrency Control
Concurrency control is the mechanism responsible for maintaining the consistency of the
database in case of concurrent transactions.

Without Concurrency Control, problems may occur with concurrent transactions:

• The Lost Update Problem. This problem occurs when two transactions that access the
same database items have their operations interleaved in a way that makes the value of
some database items incorrect.

• The Temporary Update (or Dirty Read) Problem. This problem occurs when one
transaction updates a database item and then the transaction fails for some reason.
Meanwhile, the updated item is accessed (read) by another transaction before it is
changed back (or rolled back) to its original value.

• The Incorrect Summary Problem. If one transaction is calculating an aggregate

summary function on a number of database items while other transactions are updating
some of these items, the aggregate function may calculate some values before they are
updated and others after they are updated.
Recovery
Recovery is the process of restoring the database to the most recent consistent state that
existed shortly before the time of failure.

If failure occurs midway through the execution of a transaction, the DBMS must recover the
database to the state before the transaction was executed because the DBMS must not
permit some operations of a transaction T to be applied to the database while other
operations of T are not (Atomicity).

There are several reasons for failure:

• Computer failure (system crash): a hardware or software error occurs during

transaction execution.

• Transaction or system error: Some operation in the transaction may cause it to fail,
such as integer overflow or division by zero, erroneous parameter values, or a logical
programming error. *user interruptions fall under this category

• Local errors or exceptions: Certain conditions necessitate cancellation of the

transaction. For example, data for the transaction may not be found.

• Concurrency control enforcement: The concurrency control method may decide to abort
the transaction, to be restarted later, because it violates serializability or because
several transactions are in a state of deadlock.

• Disk failure: Some disk blocks may lose their data because of a read or write
malfunction or because of a disk read/write head crash.

• Physical problems and catastrophes: This refers to an endless list of problems that
includes power or air conditioning failure, fire, theft, sabotage, overwriting disks or
tapes by mistake, and mounting of a wrong tape by the operator.