Professional Documents
Culture Documents
Database Storage: Intro To Database Systems Andy Pavlo
Database Storage: Intro To Database Systems Andy Pavlo
04 Part II
ADMINISTRIVIA
U P C O M I N G D ATA B A S E E V E N T S
SalesForce Talk
→ Friday Sep 13th @ 12:00pm
→ CIC 4th Floor
Impira Talk
→ Monday Sep 16th @ 4:30pm
→ GHC 8102
Vertica Talk
→ Monday Sep 23rd @ 4:30pm
→ GHC 8102
CMU 15-445/645 (Fall 2019)
4
D I S K- O R I E N T E D A R C H I T E C T U R E
S L O T T E D PA G E S
Slot Array
The most common layout scheme is
called slotted pages. Header
S L O T T E D PA G E S
Slot Array
The most common layout scheme is
called slotted pages. Header
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
Instead of storing tuples in pages, the
DBMS only stores log records. INSERT id=1,val=a
New Entries
INSERT id=2,val=b
…
attributes that were modified.
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the INSERT id=1,val=a
Reads
DELETE id=4
INSERT id=3,val=c
UPDATE val=X (id=3)
UPDATE val=Y (id=4)
…
CMU 15-445/645 (Fall 2019)
7
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the INSERT id=1,val=a
…
CMU 15-445/645 (Fall 2019)
7
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the id=1,val=a
id=2,val=b
tuple to find what it needs. id=3,val=X
id=4,val=Y
Build indexes to allow it to jump to
locations in the log.
LO G -S T R U C T U R E D F I L E O R G A N I Z AT I O N
Page
To read a record, the DBMS scans the
log backwards and "recreates" the id=1,val=a
id=2,val=b
tuple to find what it needs. id=3,val=X
id=4,val=Y
Build indexes to allow it to jump to
locations in the log.
T O D AY ' S A G E N D A
Data Representation
System Catalogs
Storage Models
TUPLE STORAGE
D ATA R E P R E S E N TAT I O N
INTEGER/BIGINT/SMALLINT/TINYINT
→ C/C++ Representation
FLOAT/REAL vs. NUMERIC/DECIMAL
→ IEEE-754 Standard / Fixed-point Decimals
VARCHAR/VARBINARY/TEXT/BLOB
→ Header with length, followed by data bytes.
TIME/DATE/TIMESTAMP
→ 32/64-bit integer of (micro)seconds since Unix epoch
VA R I A B L E P R E C I S I O N N U M B E R S
VA R I A B L E P R E C I S I O N N U M B E R S
VA R I A B L E P R E C I S I O N N U M B E R S
POSTGRES: NUMERIC
# of Digits
typedef unsigned char NumericDigit;
Weight of 1st Digit typedef struct {
int ndigits;
Scale Factor int weight;
int scale;
Positive/Negative/NaN int sign;
NumericDigit *digits;
Digit Storage } numeric;
POSTGRES: NUMERIC
# of Digits
typedef unsigned char NumericDigit;
Weight of 1st Digit typedef struct {
int ndigits;
Scale Factor int weight;
int scale;
Positive/Negative/NaN int sign;
NumericDigit *digits;
Digit Storage } numeric;
L A R G E VA L U E S
Tuple
Most DBMSs don't allow a tuple to Header a b c d e
exceed the size of a single page.
E X T E R N A L VA L U E S T O R A G E
Tuple
Some systems allow you to store a Header a b c d e
really large value in an external file.
Treated as a BLOB type.
→ Oracle: BFILE data type
→ Microsoft: FILESTREAM data type External File
The DBMS cannot manipulate the
contents of an external file. Data
→ No durability protections.
→ No transaction protections.
E X T E R N A L VA L U E S T O R A G E
Tuple
Some systems allow you to store a Header a b c d e
really large value in an external file.
Treated as a BLOB type.
→ Oracle: BFILE data type
→ Microsoft: FILESTREAM data type External File
The DBMS cannot manipulate the
contents of an external file. Data
→ No durability protections.
→ No transaction protections.
S Y S T E M C ATA L O G S
S Y S T E M C ATA L O G S
A C C E S S I N G TA B L E S C H E M A
\d; Postgres
.tables; SQLite
A C C E S S I N G TA B L E S C H E M A
\d student; Postgres
O B S E R VAT I O N
WIKIPEDIA EXAMPLE
CREATE TABLE useracct ( CREATE TABLE pages (
userID INT PRIMARY KEY, pageID INT PRIMARY KEY,
userName VARCHAR UNIQUE, title VARCHAR UNIQUE,
⋮ latest INT
); ⮱REFERENCES revisions (revID),
);
O LT P
SELECT P.*, R.*
On-line Transaction Processing: FROM pages AS P
→ Simple queries that read/update a small INNER JOIN revisions AS R
amount of data that is related to a single ON P.latest = R.revID
entity in the database. WHERE P.pageID = ?
OLAP
SELECT COUNT(U.lastLogin),
On-line Analytical Processing: EXTRACT(month FROM
→ Complex queries that read large portions U.lastLogin) AS month
FROM useracct AS U
of the database spanning multiple entities. WHERE U.hostname LIKE '%.gov'
GROUP BY
EXTRACT(month FROM U.lastLogin)
You execute these workloads on the
data you have collected from your
OLTP application(s).
HTAP
OLTP
Simple
Writes Reads
Workload Focus [SOURCE]
CMU 15-445/645 (Fall 2019)
28
D ATA S T O R A G E M O D E L S
Header - - - - -
Header - - - - -
Header userID
- userName
- userPass
- hostname
- lastLogin
-
Useless Data
CMU 15-445/645 (Fall 2019)
33
Advantages
→ Fast inserts, updates, and deletes.
→ Good for queries that need the entire tuple.
Disadvantages
→ Not good for scanning large portions of the table and/or
a subset of the attributes.
T U P L E I D E N T I F I C AT I O N
Advantages
→ Reduces the amount wasted I/O because the DBMS only
reads the data that it needs.
→ Better query processing and data compression (more on
this later).
Disadvantages
→ Slow for point queries, inserts, updates, and deletes
because of tuple splitting/stitching.
CONCLUSION
D ATA B A S E S T O R A G E