Transaction Processing

All material (including figures) from:

Concurrency Control and Recovery in Database Systems
Phil Bernstein, Vassos Hadzilacos and Nathan Goodman
(http://research.microsoft.com/en-
us/people/philbe/ccontrol.aspx)
 Transactions
• Interaction with the DBMS through SQL

update Airlines set price = price - price*0.1, status = “cheap”

where price < 5000

A transaction is a unit of interaction

 ACID Properties
• Atomicity
• Consistency
• Isolation
• Durability

Database system must ensure ACID properties

 Atomicity and Consistency
• Single transaction
– Execution of a transaction: “all-or-nothing”
Either a transaction completes in its entirety
Or it “does not even start”
– As if the transaction never existed
– No partial effect must be visible

2 outcomes: A transaction COMMITs or ABORTs

 Consistency and Isolation
• Multiple transactions
– Concurrent execution can cause an inconsistent database state
– Each transaction executed as if isolated from the others
 Durability
• If a transaction commits the effects are permanent

• But, durability has a bigger scope

– Catastrophic failures (floods, fires, earthquakes)
 What we will study…
• Concurrency Control
– Ensuring atomicity, consistency and isolation when multiple transactions
are executed concurrently
• Recovery
– Ensuring durability and consistency in case of software/hardware
failures
 Terminology
• Data item
– A tuple, table, block

• Read (x)
• Write (x, 5)

• Start (T)
• Commit (T)
• Abort (T)
• Active Transaction
– A transaction which has neither committed nor aborted
 High level model
Transaction 1 Transaction 2
Transaction n

Transaction Manager

Scheduler

Recovery Manager
Disk

Cache Manager
 Recoverability (1/2)
• Transaction T Aborts T T’

– T wrote some data items Read (x)

Write (x, k)
– T’ read items that T wrote Read (y)
• DBMS has to… Read (x)
Write (y, k’)
– Undo the effects of T Commit
– Undo effects of T’ Abort

– But, T’ has already committed

 Recoverability (2/2)
• Let T1,…,Tn be a set of transactions
• Ti reads a value written by Tk, k < i
• An execution of transactions is recoverable if
Ti commits after all Tk commit

T1 T2 T1 T2
Write (x,2) Write (x,2)
Read (x) Read (x)
Write (y,2) Write (y,2)
Commit Commit
Commit
 Cascading Aborts (1/2)
• Because T was aborted, T1,…, Tk also have to be aborted

T T’ T’’
Read (x)
Write (x, k)
Read (y)
Read (x)
Write (y, k’)
Abort
Read (y)
 Cascading Aborts (2/2)
• Recoverable executions do not prevent cascading aborts
• How can we prevent them then ?

T1 T2 T1 T2
Write (x,2) Write (x,2)
Read (x)
Write (y,2)
Abort Abort
Abort Read (x)
Write (y,2)
Commit
 What we learnt so far…
Reading a value, committing a transaction

Recoverable with Recoverable without

Not recoverable
cascading aborts cascading aborts

T1 T2 T1 T2 T1 T2
Write (x,2) Write (x,2) Write (x,2)
Read (x) Read (x)
Write (y,2) Write (y,2)
Commit Abort Commit
Abort Read (x)
Write (y,2)
Commit
 Before image/after image
• “Undo”-ing the effects of a transaction
– Restore the before image of the data item

T1 T2
Write (x,1)
T1 T2
Write (y,3) Equivalent to Write (x,1)
Write (y,1) Final value
Write (y,3)
Commit of y: 3
Commit
Read (x)
Abort
 Strict Schedule
Initial value of x: 1

T1 T2 T1 T2 T1 T2
Write (x,2) Write (x,2) Write (x,2)
Write (x,3) Write (x,3)
Abort Abort Abort
Abort Write (x,3)

Should x be restored to 1 or 3? T1 restores x to 3?

T2 restores x to 2?

Do not read or write a value which has been written by

an active transaction until that transaction has committed
or aborted
The Lost Update Problem

T1 T2
Read (x)
Read (x)
Write (x, 200,000)
Commit
Write (x, 200)
Commit

Assume x is your account balance

 Serializable Schedules
• Serial schedule
– Simply execute transactions one after the other
• A serializable schedule is one which equivalent to some serial
schedule
SERIALIZABILITY THEORY
 Serializable Schedules
T1: op11, op12, op13
T2: op21, op22, op23, op24
• Serial schedule
– Simply execute transactions one after the other
op11, op12, op13 op21, op22, op23, op24
op21, op22, op23, op24 op11, op12, op13
• Serializable schedule
– Interleave operations
– Ensure end result is equivalent to some serial
schedule
 Notation
r1[x] = Transaction 1, Read (x)
w1[x] = Transaction 1, Write (x)
c1 = Transaction 1, Commit
a1= Transaction 1, Abort

r1[x], r1[y], w2[x], r2[y], c1, c2

 Histories (1/3)
• Operations of transaction T can be represented
by a partial order.

r1[x]
w1[z] c1
r1[y]

• No assumptions are made regarding values,

computations.
 Histories (2/3)
• Conflicting operations
– Of two ops operating on the same data item, if one of them is a write,
then the ops conflict
– An order has to be specified for conflicting operations
 Histories (3/3)
• Complete History
T1 = r1[x] → w1[x] → c1
T2 = r2 [x] → w2 [y] → w2 [x] → c2
T3 = r3[y] → w3[x] → w3[y] → w3[z] → c3

A complete history over T = {T1,T2 ,T3 }

r2 [x] → w2 [y] → w2 [x] → c2
↑ ↑
H1 = r3[y] → w3[x] → w3[y] → w3[z] → c3
↑
r1[x] → w1[x] → c1
Figures from Bernstein et al.
 Serializable Histories
• The goal: Ensure that the interleaving operations guarantee a
serializable history.
• The method
– When is a history serial?
– When are two histories equivalent?
 Equivalence of Histories (1/2)
H H’ if
1. they are defined over the same set of transactions and they have the
same operations
2. they order conflicting operations the same way
Equivalence of Histories (2/2) 2.2 SERIALIZABLE HISTORIES 31

H, = ~*bl + 7,[Yl - w,[xl - y -

w,[xl Cl
\ / r2[z3- Ll\
w&l ’ c2

/ w[xl ,
7,[xl - rIbI,
H, = AYI ’ c’
T
/ 72[zl - WL[YIy
w2bl ’ c2

FlGURE 2-l
Example Histories
H2 and H3 are equivalent, but H4 is not equivalent to either.

The idea underlying this definition

Figures from Bernstein et al.
is that the outcome of a concurrent execu-
 Serial History
• A complete history is serial if for every pair of transactions Ti
and Tk,
– all operations of Ti occur before Tk OR
– all operations of Tk occur before Ti
• A history is serializable if its committed projection is equivalent to
a serial history.
 Serialization Graph
Let H be a history
SG(H ) = (V, E)
where V = {T1,...,Tn }
and E = {(Ti ,Tj ) | ∃opi ∈ Ti
which precedes and conflicts op j ∈ Tj }

r2 [x] → w2 [y] → w2 [x] → c2 T1 T3 T2

↑ ↑
H1 = r3[y] → w3[x] → w3[y] → w3[z] → c3
↑
r1[x] → w1[x] → c1
 Serializability Theorem
A history H is serializable if its serialization graph SG(H) is acyclic

Reading assignment
How do recoverability, strict schedules, cascading aborts fit into
the big picture?
LOCKING
 High level model
Transaction 1 Transaction 2
Transaction n

Transaction Manager

Scheduler

Recovery Manager
Disk

Cache Manager
 Transaction Management
Transaction Manager
• Receives Transactions
Transaction 1
• Sends operations to
Transaction 2
Transaction 3 scheduler
. Read1(x)
. Write2(y,k)
. Read2(x)
Transaction n Commit1

Scheduler
• Execute op
Disk
• Reject op
• Delay op
 Schedulers
• Conservative schedulers
– Delay operations until more is known
– Declaration of readset and writeset
• Aggressive schedulers
– Schedule immediately
• Efficiency
– Conflicting vs. non-conflicting set of transactions
 Locking
• Each data item x has a lock associated with it
• If T wants to access x
– Scheduler first acquires a lock on x
– Only one transaction can hold a lock on x
• T releases the lock after processing

Locking is used by the scheduler to ensure serializability

 Notation
• Read lock and write lock
rl[x], wl[x]
• Obtaining read and write locks
rli[x], wli[x]
• Lock table
– Entries of the form [x, r, Ti]
• Conflicting locks
– pli[x], qlk[y], x = y and p, q conflict
• Unlock
rui[x], wui[x]
 Basic 2-Phase Locking (2PL)
RULE 1 RULE 2
Receive pi[x]
pli[x] cannot be released
until pi[x] is completed

is qlk[x] set such NO

that p and q Acquire pli[x]
conflict?
RULE 3 (2 Phase Rule)
YES pi[x] scheduled
Once a lock is released no
other locks may be
pi[x] delayed obtained.
 The 2-phase rule
Once a lock is released no other locks may be
obtained.
T T
T1: r1[x] w1[y] c1 1 2

T2: w2[x] w2[y] c2

H = rl1[x] r1[x] ru1[x] wl2[x] w2[x] wl2[y] w2[y]

wu2[x] wu2[y] c2 wl1[y] w1[y] wu1[y] c1
 Correctness of 2PL
2PL always produces serializable histories

STEP 1: Characterize properties of the scheduler

STEP 2: Prove that any history with these properties is
serializable
(That is, SG(H) is acyclic)
 Proof outline (1/2)
Let H be a history produced by a 2PL scheduler.
1. if oi[x] is in H, then so are oli[x] and oui[x] and oli[x] < oi[x] <
oui[x]
2. if pi[x] and qj[x] conflict, then pui[x] < qlj[x] or quj[x] < pli[x]
3. [2 phase rule] for all pi[x] and qi[y], pli[x] < qui[y]
 Proof outline (2/2)
Let SG(H) be the serialization graph of H.
1. if Ti à Tj is in SG(H), pi[x] conflicts with qj[x], and pi[x] executed
before qj[x], therefore, pui[x] < qlj[x]
2. if Ti à Tj à Tk is in SG(H), Ti released some lock before Tk set
some lock
3. [Contradiction] if there is a cyle, Ti à…àTi, Ti released some
lock before Ti set some lock
 Deadlocks (1/2)
T1: r1[x] w1[y] c1
T2: w2[y] w2[x] c2

Scheduler
rl1[x] wl2[y] r1[x] w2[y] <cannot proceed>
 Deadlocks (2/2)
Strategies to deal with deadlocks
• Timeouts
– Leads to inefficiency
• Detecting deadlocks
– Maintain a wait-for graph, cycle indicates deadlock
– Once a deadlock is detected, break the cycle by aborting a
transaction
• New problem: Starvation
 Conservative 2PL
• Avoids deadlocks altogether
– T declares its readset and writeset
– Scheduler tries to acquire all required locks
– If not all locks can be acquired, T waits in a queue
• T never “starts” until all locks are acquired
– Therefore, it can never be involved in a deadlock

Reading HW
Strict 2PL (2PL which ensures only strict schedules)
NON-LOCK-BASED SCHEDULERS
 Timestamp Ordering (1/3)
• Each transaction is associated with a timestamp
– Ti indicates Transaction T with timestamp i.
• Each operation in the transaction has the same timestamp
 Timestamp Ordering (2/3)
TO Rule
If pi[x] and qk[x] are conflicting operations, then pi[x] is processed
before qk[x] iff i < k

Theorem: If H is a history representing an execution produced by a

TO scheduler, then H is serializable.
 Timestamp Ordering (3/3)
For each data item x, maintain: max-r(x), max-w(x)

Receive pi[x]

is i < max-q, NO
pi[x] scheduled
q conflicts with p

YES update max-q

pi[x] rejected
 Reading HW: Validation
• Aggressively schedule all operations
• Do not commit until the transaction is “validated”
SOURCE: Database System: The complete book. Garcia-Molina, Ullman and Widom

RECOVERY
 Logging
• Log the operations in the transaction(s)
• Believe the log
– Does the log say transaction T has committed?
– Or does it say aborted?
– Or has only a partial trace (implicit abort)?
• In case of failures, reconstruct the DB from its log
 Terminology
Data item: an element which can be read or written
– tuple, relation, B+-tree index, etc

Input x: fetch x from the disk to buffer

Read x,t: read x into variable local variable t
Write x,t: write value of t into x
Output x: write x to disk
 Example
update Airlines set price = price - price*0.1, status =
“cheap” where price < 5000

Read P, x
x -= x* 0.1
Write x,P
System fails here
Read S, y
y = “CHEAP”
Write y, S
Output P
System fails here
Output S
System fails here
 Logs
• Sequence of log records
• Need to keep track of
– Start of transaction
– Update operations (Write operations)
– End of transaction (COMMIT or ABORT)
• “Believe” the log, use the log to reconstruct a consistent DB state
 Types of logs
• Undo logs
– Ensure that uncommitted transactions are rolled back (or undone)
• Redo logs
– Ensure that committed transactions are redone
• Undo/Redo logs
– Both of the above
All 3 logging styles ensure atomicity and durability
 Undo Logging (1/3)
• <START T>: Start of transaction T
• <COMMIT T>
• <ABORT T>
• <T, A, x>: Transaction T modified A whose before-image is x.
 Undo Logging (2/3)
<START T>
Read P, x U1: <T, X, v> should be
x -= x* 0.1 flushed before Output X
Write x,P U2: <COMMIT T> should be
<T, P, x’>
Read S, y flushed after all OUTPUTs
y = “CHEAP”
Write y, S
<T, S, y’>
FLUSH LOG
Output P
Output S
<COMMIT T>
FLUSH LOG
 Undo Logging (3/3)
• Recovery with Undo log
1. If T has a <COMMIT T> entry, do nothing
2. If T has a <START T> entry, but no <COMMIT T>
• T is incomplete and needs to be undone
• Restore old values from <T,X,v> records
• There may be multiple transactions
– Start scanning from the end of the log
 Redo Logging (1/3)
• All incomplete transactions can be ignored
• Redo all completed transactions
• <T, A, x>: Transaction T modified A whose after-image is x.
 Redo Logging (2/3)
<START T>
Read P, x R1: <T, X, v> and
x -= x* 0.1 <COMMIT T> should be
flushed before Output X
Write x,P
<T, P, x>
Read S, y
y = “CHEAP”
Write y, S
<T, S, y>
<COMMIT T>
FLUSH LOG Write-ahead
Logging
Output P
Output S
 Redo Logging (3/3)
• Recovery with Redo Logging
– If T has a <COMMIT T> entry, redo T
– If T is incomplete, do nothing (add <ABORT T>)
• For multiple transactions
– Scan from the beginning of the log
 Drawbacks
• Reading HW: What are the drawbacks of UNDO and REDO
logging?
 Undo/Redo Logging (1/2)
<START T>
UR1: <T, X, b, a> should be
Read P, x flushed before Output X
x -= x* 0.1
Write x,P U1: <T, X, b> should be
<T, P, b, a>
Read S, y flushed before Output X
y = “CHEAP”
U2: <COMMIT T> should be
Write y, S
<T, S, b’, a’> flushed after all OUTPUTs
FLUSH LOG
Output P
<COMMIT T> R1: <T, X, a> and
Output S <COMMIT T> should be
flushed before Output X
 Undo/Redo Logging (2/2)
• Recovery with Undo/Redo Logging
– Redo all committed transactions (earliest-first)
– Undo all uncommitted transactions (latest-first)

What happens if there is a crash when you are writing a log? What
happens if there is a crash during recovery?