Chapter 17

Indexing Structures for Files and Physical Database Design

We assume that a file already exists with some primary organization

unordered, ordered or hash. The index provides alternate ways to
access the records without affecting the existing placement of records
on the disk.
Each indexing approach have a particular data structure to speed up
the search. A variety of indexing techniques are studied here.
Types of Single-Level Ordered Indexes
- Concept of indexes is similar to an index of terms in a book
- Index access structure is usually a single field of a file called indexing
field
- The index stores each value of the field along with all disk blocks
that contain records with this field
- The values in the index are ordered so that a binary search can be
done
- Both the index and data files are ordered, but index file is smaller

Several types of ordered indexes:

- Primary index specified on a key field
- Clustering index, ordering field is not a key field; the data file is
called clustered file
- A file can have at most one physical ordering field; it can have one
primary index, or one clustering index but not both

1
- Secondary index can be specified on any non-ordering field of a file;
a data file can have several secondary indexes in addition to the
primary access method

Primary Indexes
- Ordered file with 2 fields, PK field and data ptr. PK is the primary key
of the data file. Ptr is the pointer to a disk block; PK is the value for
the first record in the block
- Each block in the data file has one entry in the index file
- The two fields <K(i), P(i)>; P(i) is the pointer for the block in data file
- In general the two fields are: <K(i), X>:
o X may be the physical address of a block (or page)

2
 o X may be the record address made up of a block address and a
record id (for offset) with in the block
o X may be a logical address of the block or of the record within
the file and is a relative number that would be mapped to
physical address
Fig. 17-1

- First record in each block of the data file is called an block anchor or
anchor record
- Indexes can be dense or parse
- A dense index has an entry for every search key value; a parse index
has index entries for only for some of the search values
- To retrieve a record given the value of its PK field, we do a binary
search on the index file to find appropriate entry I, and then retrieve
the data field block whose address is P(i).

3
4
Example 1.
Ordered file with records r = 300000
Disk block size = B = 4096 bytes
File records are fixed size and unspanned
Record length = R = 100 bytes
Blocking factor bfr = [B/R] (lower ceiling) = 4096/100 = 40 records per
block
The number of blocks needed for the file = [r/bfr] (upper ceiling)
= 300000/40 = 7500 blocks
A binary search on the data file = log2(7500) (upper ceiling) = 13
Let ordering key field is 9 bytes, block pointer is 6 bytes; total 15 bytes
in each entry of index file
bfr for index file is 4095/15 (lower ceiling) = 273
total number of index entries = total no of blocks
The number of index blocks = [7500/273] (upper ceiling) = 28
To perform binary search on index file: log2(28) (upper ceiling) = 5
To search for a record, we need one additional access to read the data
block, thus we need 5+1 = 6 block accesses using a binary search
Whereas, we need 13 block accesses without an index file
Problems with Primary Index
- Insertion
o Inserting in correct position (make space, change index
entries)
o Move records to make space for new records
5
 o Move will change anchor records of some blocks
o Use linked list or overflow records
- Deletion
o Use delete markers

Clustering Indexes
- Ordered on a non-key field (no distinct values), clustering field
- The data field is ordered on a non-key field called a clustered file
- Seed up retrieval of all records that have the same value for the
clustering field
- Includes one index entry for each distinct value of the field, the
index entry points to the first data block that contains records with
the field value
- Another example of non-dense (or parse) index
- Insertion and deletion problems (reserve one or more blocks for
each value of the clustering field);

Example 2:
r = 300000 records
B = 4096 bytes
It is ordered by zip codes; there are 1000 zip codes in the file
Average 300 records per zip code (assume even distribution)
The index 1000 index entries, 5 bytes zip code, 6 bytes block no, 11
bytes total in each entry;
bfr = 4096/11 (lower ceiling) = 372 index entries per block
The number of index blocks = 1000/372 (upper ceiling) = 3
6
Binary search on index file would require log2(3) = 2 block accesses
The index is loaded in main memory 1000*11 = 11000 bytes.

7
8
Secondary Indexes
- The data field records could be unordered, ordered, or hashed
- A secondary index provides a secondary means of accessing a file for
which some primary access already exists
- The secondary index may be on a field which is a candidate key and
has a unique value in every record, or a non-key with duplicate
values
- The index is an ordered file with two fields:
o The first field is of the same data type as some non-ordering
field of the data file that is an indexing field
o The second field is either a block or record pointer
o There can be many secondary indexes (and hence, indexing
fields) for the same file; each represents an additional means
of accessing that file based on some specific field
- Includes one entry for each record in the data file; hence it is a
dense index (records of the data file are not physically ordered by
secondary key)
- A secondary index needs more storage space and longer search time
than primary index, because of its longer number of entries
- Search time is improved as there is no need to do linear search on
records in a data file (records are directly accessed)

9
10
Example 3:
r = 300000 records
size R = 100 bytes
block size B = 4096 bytes
no of records per block bfr = 4096/100 = 40
no of blocks for the data file = b = 300000/40 = 7500
suppose we want to search for a record with a specific value for the
secondary key ….a non-ordering key with 9 bytes value
without a secondary index; to do a linear search on the file would
require: b/2 searches (7500/2 = 3750) block accesses
suppose that we construct a secondary index on the nonordering key
field of the file; 9+6 = 15 byte entry value;
the blocking factor for the index: 4096/15 = 273 entries per block
in a dense secondary index; total number of index entries is equal to
the number of records; 300000
the number of blocks needed for the secondary index = 300000/273
(upper ceiling) = 1099 blocks
A binary search on this secondary index needs log2b = log2(1099) = 11
block accesses
To search for a record using the index, we need 1 additional block
access to the data file
11+1 = 12 block accesses (compared to 3750 block accesses)

11
We can also create a secondary index on a non-key, non-ordering field.
Numerous records in the data file can have the same value for the
indexing field. There are several options:
1. Include duplicate index entries with the same K(i) value, one for
each record. Dense index.
2. Have variable length records for the index entries, with a
repeating field for the pointer. Keep a list of pointers
<P(i, 1), P(i, 2), …,P(i, k) in the index entry K(i).
3. Create an extra level of indirection to handle the multiple
pointers. Fig. 17.5. If one block of indirection is not enough, a
cluster can be used.

12
13
Multilevel Indexes
We covered ordered index file techniques. A binary search is needed to
locate pointers for disk blocks or records. A binary search requires
log2bi block accesses for an index with bi blocks. Each step of the
binary search reduces the file by 2.
The idea of multilevel index is to reduce the part of the index that we
continue to search by bfri . the blocking factor for the index, which is
larger than 2. The search space is reduced much faster.

The value bfri is called fan-out (fo).

Multilevel index requires logfobi block accesses.

With a 4096 byte block size, 9 byte key (SSN) and 4 byte pointer (13
bytes total index entry)

bfri = 4096/13 (lower ceiling) = 315.

In multilevel index, the index file is the first level (base) of multilevel
index, as an ordered file with distinct values for each K(i).
We can create a primary index for the first level is called the second
level multilevel index. The second level has one entry for each block, as
it is a primary index (we can use block anchors)
The process repeats for 3rd level and so on until all entries fit in one disk
block for the top level.
-problems: insertion and deletion; leave some space in each of its
blocks called dynamic multilevel index

14
Example 4
r = 300000 fixed length records
B = 4096 bytes
Record size R = 100 bytes
bfr = 4096/100 (lower ceiling) = 40 records per block
number of blocks = 300000/40 (upper ceiling) = 7500 blocks
Number of bytes in each index entry = 9 + 6 = 15

bfri = 4096/15 (lower ceiling) = 273 = fo of the multilevel index

number of blocks needed for the index = 300000/273 (upper ceiling) =
b1 = 1099 blocks (number of first level blocks)
(nth level n-1 level ……. 1st level) in multilevel structure
The number of 2nd level blocks = b2 = b1/fo (upper ceiling) =
1099/273 = 5 blocks
The number of 3rd level blocks = b3 = b2/fo (upper ceiling) =
5/273 = 1 block
Hence, the third level is the top level (for multilevel index), t = 3
To access a record by searching a multilevel index, we must access one
block at each level plus 1 block from the data file = 3+1 = 4 blocks (in a
single level it was 12 blocks)

15
16
Dynamic Multilevel Indexes Using B-Tree and B+ Trees
Tree Structure:
- A tree is formed of nodes
- Each node except the root, has one parent node and 0 or more child
nodes
- The root node has no parent
- A node with no child nodes is a leaf node
- A nonleaf node is an internal node
- The level of node is always one higher than its parent
- The level of root node is 0
- A subtree of a node consists of that node and all its descendant
nodes
- If the leaf nodes are at different levels, it is an unbalanced tree
- B-tree nodes are kept 50-100% full
- Pointers to the data blocks are stored at internal and leaf nodes for
B-trees
- Pointers to the data blocks are stored at leaf nodes for B+ trees

Search Trees
A search tree is a special type of tree used to guide the search for a
record given the value of one of the record’s fields.
Fig. 17.8
A search tree of order p is a tree such that each node contains p-1
search values and p pointers.

<P1, K1, P2, K2, ….Pq-1, Kq-1, Pq> Each Pi is a pointer to a

child node or null

17
We can use search tree as a mechanism to search for records on disk.
Two constraints must hold all the time:
1. Within each node K1 < K2 < K3 <…..Kq-1
2. For all values of X in the subtree pointed at by Pi
We have; Ki-1 < X < Ki

18
19
- Search field is same as the index field
- Algorithms needed to insert and delete
- May result in unbalanced tree
- Make sure nodes are evenly distributed
- Make search speed uniform
- Minimize number of levels; also make sure it does not require
restructuring many times

B-Trees
B-Tree solves the above problems:
- Tree is always balanced
- Space wasted by deletion, if any, will not be excessive
- Insertion and deletion algorithms are complex

A B-Tree of order p, when used as an access structure on a key field to

search for records in a data file can be defined as follows:
1. Each internal node in the B-Tree is of the form:

<P1, <K1, Pr1>, P2, <K2, Pr2>, ….Pq-1, <Kq-1,Prq-1>, Pq>

Where, q <= p, each Pi is a tree pointer – a pointer to the record, whose
search key field value is equal to Ki (or the data field value containing
that record)
2. Within each node K1 < K2 < …<Kq-1
3. For all search key field values X in the subtree pointed at by Pi, we
have Ki-1 < X< Ki for 1 < I <q; X < Ki for i = 1; Ki-1 < X for i=q
4. Each node has at most p tree pointers

20
5. Each node, except the root and leaf nodes, has at least [p/2]
(upper ceiling) pointers; the root node has at least two tree
pointers unless it is the only node in the tree
6. A node with q tree pointers, q <=p, has q-1 search key field values
and hence has q-1 data pointers
7. All the leaf nodes are at the same level. Leaf nodes have same
structure as the internal nodes, except they have null pointers.

21
- Fig. 17-10(b) illustrate a B-Tree with p = 3. All search key values are
unique, as it is a key field
- If we use a B-Tree on a nonkey field, we must change the file pointer
Pri to point to a block or a cluster of blocks that contain the
pointers to the records.
- The B-Tree starts with a single root node (which is also a leaf node)
at level 0
- Once the root node is full with p-1 search key values, we attempt to
insert another entry in the tree; the root node splits into two nodes
at level 1. Only the middle value is kept at the root node, the rest of
the values are split evenly on other nodes
- When a nonroot node is full, and a new entry is inserted into it, that
node is split into two nodes at the same level, and the middle entry
is moved into the parent node, along with two pointers to the newly
split nodes. ……
- Read deletion from p/621
- SKIP Example 5
B-Tree (order p)
- No node has more than p children
- Every node except the root and terminal nodes has at least [p/2]
(upper ceiling) children
- The root has at least two children, unless the tree has only one node
- All terminal nodes appear on the same level, i.e., same distance
from the root
- A non-terminal node with k children contains k-1 records; a terminal
node contains at least ([p/2] – 1) records (upper ceiling) and at most
p-1 records
- The largest number of records allowed in a node is p-1

22
 B-Tree Insertion:
- New records are always inserted into terminal nodes
- Every null pointer represents an insertion pointer, where a new
record might go
- To determine the insertion point, searching for a new record as if it
were already in the tree
- Problems with inserting is that nodes can overflow because there is
upper bound p-1 records
- If the node into which we have inserted a record now exceeds the
max size, then redistribute or split on overflow
- The node is split into three parts
- Splitting on overfull node with p records, the middle record is
passed upward and inserted into its parent
B-Tree Exmples: (perform it on the B-Tree, p=3)
(1)Insert 13
(2)Insert 10
(3)Insert 16

23
24
B-Tree Deletion
- Start the delete operation at the lowest level of the tree
- Replace it by a copy of its successor, the record with the next
highest key, the successor will be at the lowest level (predecessor
will also work as well)
- We may have an underflow after deletion (node may be smaller
than the minimum size)
- Use redistribution or concatenation to solve underflow
B-Tree Examples: p=5
Delete 10
Delete 13
Delete 18

25
26
B+ Trees
Knuth proposed a variation on B trees.
- Records on a B+ tree are held only on the terminal nodes
- The terminal nodes are linked together to facilitate sequential
processing of the records and are termed the sequential set
- No need for terminal nodes to have tree pointers
- Terminal nodes have different structure than non-terminal nodes

Each internal node is of the form:

1. <P1, K1, P2, K2, ….Pq-1, Kq-1, Pq>, Where, q ≤ p, each Pi is a

tree pointer
2. Within each node K1 < K2 < …<Kq-1
3. For all search filed values X in the subtree pointed at by Pi, we
have Ki-1 < X ≤ Ki for 1 <i<q, X ≤ Ki for i=1; and Ki-1 < X for i=q
4. Each internal node has at most p tree pointers

5. Each internal node, except the root, has at least ᴦp/2˥ tree
pointers; the root node has at least two tree pointers if it is an

ᴦ ˥
internal node ( p/2 to p)
6. An internal node with q pointers, q≤p, has q-1 search fields values
(Notice that there is no Kq as this pointer will lead to another subtree, it
is simply a pointer, no need for a key)
The structure of the lead nodes of B+ tree of order p is as follows:
1. <<K1,Pr1>, <K2, Pr2>, …….,<Kq-1, Prq-1>, Pnext> where q ≤ p, each Pri
is a data pointer, and Pnext points to the next leaf node

27
 2. Within each leaf node K1 < K2 < …<Kq-1, q≤p
3. Each Pri is a data pointer that points to the record whose search
field value is Ki or to a file block containing the record (or to a
block of record pointers that point to records whose search field
value is Ki, if the search field is not a key)

4. Each leaf node has at least ᴦp/2˥ values

5. All leaf nodes are at the same level

- By starting at the left most leaf node, it is possible to traverse leaf

nodes as a linked list using Pnext pointers
- Provides ordered access to the data records
- A Pprevious can also be included
- As the structures for internal and leaf nodes are different, their
order can be different; order p for internal nodes, and order pleaf for
leaf nodes.
Example 6:
Search key field is V = 9 bytes
Block size is B = 512 bytes
Record pointer is Pr = 7 bytes
Block pointer/tree pointer is P = 6 bytes
An internal node can have up to p tree pointers and p-1 key fields,
these must fit in a single block
Thus,
p * P + (p-1) * V ≤ B
p*6+(p-1)9 ≤ 512

28
15p ≤ 521
P ≤ 34 for intermediate nodes
The leaf nodes have the same number of values and pointers, except
they are data pointers and next pointer.

The order of pleaf can be calculated as follows:

Pleaf * (Pr+V) + P ≤ 512
Pleaf * (7+9) + 6 ≤ 512
Pleaf * 16 ≤ 512 - 6
Pleaf * 16 ≤ 506
Pleaf ≤ 31
Example 7:
Construct a B+ tree on example 6
- Assume each node is 69% full
- p = 34, pleaf = 31
- On the average, each internal node have 0.69 * 34 ~ 23 pointers and
22 key values
- On the average leaf node has 0.69 * 31 ~ 21 data record pointers
Root 1 node 22 key entries (22+1=23) ptrs
Level1 23 nodes 23*22 (506) (506+23=529) ptrs
Level2 23*23(529) 528*22(11638) (11638+529=12167)
Leaf 529*23(12167) 12167*21 (255507) data record ptrs

29
30
B+ Tree Example

31
B+ Tree Insertion
1. If an index node has to split, the algorithm is same as B tree.
2. If the node splits when we insert record into a terminal node, we
put a copy of the key of the central record in TOOBIG into the
index. Thus the central record will also be one of the two halves
after splitting,

B+ Tree Deletion
1. When a record is deleted from B+ tree, no distribution or
concatenation is needed
2. No changes are made to the index, even if the key of the record to
be deleted appears in the record, it can be left as a separator
Use the following B+ tree for deletion.

32
33
Indexes on Multiple Keys
So far, we have considered single attributes as search attributes.
However, in real world, multiple attributes are used to search records.
EMPLOYEE
ssn, dno, age, street, city, zip, salary, skill-code
Query:
List the employees in department number 4, where age is 59.
Both attributes department number and age are non-key attributes,
that is, a search value for either of these will point to multiple records.
Ordered index on multiple attributes:
- Create an index on a search key field that is a combination of <dno,
age>. The search key is a pair of values, <4, 59> in this example.
- In general, <A1, A2, …, An> attributes result in values <v1, v2, …,Vn>
- <3, n> precedes <4,m> in this ordering. The ascending order for dno
keys will be <4,18>, <4,19>,….etc..The composite attribute indexing
can be used to access data
Partitioned Hashing
For a key consisting of n components, the hash function is designed to
produce a result with n separate hash addresses. For example, <Dno,
Age> search key; suppose Dno=4 has a hash function 010 and Age=59
has a hash function 10101. Then, the search value goes to 10010101.
Just to search with employees with Age=59, it will be go through all 8
buckets of 001 – 111 combinations resulting in 00010101, 00110101,
….searches. This approach is only good for equality search, not range
searches.

34
Grid Files
Organize records as a grid file. <Dno, Age> has a 2 dimensional grid. n
dimensional grid can be formed with n attributes, which is hard to
construct and maintain. The scales are made in a way to achieve
uniform distribution. Dn0=4 and Age=59 falls into grid (1,5). Each cell or
cluster of cells can point to one bucket pool. This is suitable for range
queries.

35
SKIP 17.5
General Issues Concerning Indexing
- When physical index changes, then index entry needs to change;
thus one can use a logical address to cope with this problem; it
causes another level of indirect mapping of addresses and more
overhead and maintenance
- Index creation: many RDBMS have commands for creating index

CREATE | [UNIQUE] INDEX <index-name>

ON <table-name> (<column-name> [<order>]…)[CKUSTER];
CREATE INDEX DnoIndex
ON EMPLOYEE (Dno)
CLUSTER;
- Index creation process: index is not part of the data file, but can be
created and discarded dynamically (called access structure)
- Whenever we expect to access a file frequently based on some
search condition involving a particular attribute, we can request the
DBMS to create an index
- Usually, a secondary index is created to avoid reordering of records
on the disk
- Secondary index can be created with any primary record
organization
- Insertion of a large number of entries into index is called bulk
loading the index
- Indexing of strings cause problem as the strings vary in size; prefix
compression is used to reduce the size of strings to short fields
- Tuning indexes: The initial choice of indexes may have to be revisted
for the following reasons:

36
 o Certain queries may take too long to run for the lack of an
index
o Certain indexes may not be utilized at all
o Certain indexes may undergo too much updating due to
frequent changes
- Some indexes may be dropped, some new ones created; trace
facility shows the usage of indexes
- Rebuilding the index: to improve performance and restructure the
tree
- It is common to use an index to enforce a key constraint on an
attribute; while inserting a record, it can be checked to see if
another record exists with the same key attribute (key integrity
constraint)
- If an index is created on a nonkey field, duplicates occur; data
records for the duplicate may contain in the same block or span
across many blocks; some systems use row-id with the record, so
that records with duplicates have their own unique identifiers
- Inverted file: a file that has a secondary index on every one of its
fields is called a fully inverted file. The data file itself is an unordered
file.
- Using indexing hints on queries: provision for allowing hints in
queries that are suggested alternatives or indicators to the query
process and optimization process for expediting query execution
SELECT /*+ INDEX (EMPLOYEE emp_dno_index)*/ Emp_ssn,
Salary, Dno
FROM EMPLOYEE
WHERE Dno < 10;
- Column-based storage of relations

37
 o Vertically partitioning the table column by column, thus a two
column table can be constructed, only the needed columns
can be accessed (index value, data value)
o Using materialized views to support queries on multiple
columns

Physical database design in Relational Databases

The goal of the physical design is to provide appropriate structuring of
data to provide optimal performance.
Factors that influence physical database design:
(a) Analyze the database queries and transactions: intended use of
the database by defining high level form of queries and
transactions that will run; for each retrieval query the following
information would be needed:
i. The files (relations accessed by the query)
ii. Attributes on which selection condition is specified
iii. Selection condition is equality, inequality or a range
iv. Join and multiple tables and attributes
v. The attributes whose value is retrieved by queries
For each update operation or transaction, the following
information will be needed:
i. The files that will be updated
ii. Type of operation (insert, update or delete)
iii. Attributes on selection
iv. Attributes that will be changed

(b) Analyze the expected frequency of invocation of queries and

transactions (80% processing and 20% querying rule)
38
 (c) Analyzing the time constraints on the queries and transactions
(min of 4 seconds and max of 20 seconds)
(d) Analyzing the expected frequency of update operations;
slow down the operation
(e)Analyzing the uniqueness constraints on attributes; checking
uniqueness constraints during inserts will slow the process

Physical database design decisions:

- Most relational databases represent each base relation as a physical
database file
- The access path options include individual or composite attributes
for primary file organization (keys)
- At most one of the indexes on each file may be a primary or
clustering index. Any number of secondary indexes can be created
- The performance largely depends upon which indexes or hashing
schemes exist to expedite the processing of selections and joins
- The physical design decisions for indexing fall into the following
categories:
o Whether to index an attribute (used in a query)
o What attribute or attributes to index on (one or more)
o Whether to set up a clustered index (primary index, key;
clustered index, non-key; which one depends on ordering of
the table on that attribute or attributes)
o Whether to use a hash or tree index (B+ trees support equality
and range queries; hash do not support range queires; most
commonly used are tree index (B+ tree)
o Whether to use dynamic hashing (files that grow and shrink
often; not commonly used).

39