Distributed Systems

(4th edition, version 01)

Chapter 05: Consistency and Replication

Consistency and replication Introduction

Why Replicate?

Reliability:
• Creating copies of system parts (code and data) enhances reliability.
• If one copy fails, switch to another while fixing the issue.

Performance:
• Distribute requests across replicas to balance load.
• Improve response times by considering proximity of replicas to users.

The Challenge of Replication:

Maintaining Consistency:
• Each time a change is made in one copy, it must be replicated across all copies.
• Essential to keep all replicas identical and up-to-date.

Reasons for replication

Consistency and replication Data-centric consistency models

Data-centric consistency models

What is a Consistency Model?
• A set of rules or a contract between a distributed data store and its
processes.
• Specifies expected outcomes of read and write operations under
concurrent access.
Importance of Consistency Models:
Ensures predictable and reliable data interaction in distributed systems.

Understanding a Data Store in this Context:

• A data store refers to a distributed network of storage resources.
• These storages collectively maintain and manage data in the distributed
system.
Consistency and replication Data-centric consistency models

Some notations
Read and write operations
• Wi (x )a: Process Pi writes value a to x

• Ri (x )b: Process Pi reads value b from x

• All data items initially have value NIL

Possible behavior
We omit the index when possible and draw according to time (x-
axis):

Consistent ordering of operations

Consistency and replication Data-centric consistency models

Sequential consistency
Definition
• Operations by all processes were performed in a specific sequential
order.
• Operations for each process adhere to the order dictated by the program
itself.

A sequentially consistent data

store

A data store that is not sequentially

consistent
Consistent ordering of operations
Consistency and replication Data-centric consistency models

Example
Three concurrent processes (initial values: 0)
Process P1 Process P2 Process P3
x ← 1; y ← 1; z ← 1;
p r i n t ( y, z ) ; print(x,z); print(x,y);

Example execution sequences

Consistent ordering of operations

Consistency and replication Data-centric consistency models

Grouping operations
Entry Consistency: a rule ensuring changes are complete before
others can access the data.
• Access to shared data is controlled by locks.
• Ensures operations are performed in order or sequence.

Write Operations:
• No lock access is permitted until all preceding write operations are
complete across all processes.

Read Operations:
• Ensures no data is accessed until all associated lock operations
have been executed.

Consistent ordering of operations

Consistency and replication Data-centric consistency models

Grouping operations
A valid event sequence for entry consistency

Observation
Entry consistency implies that we need to lock and unlock data.

Consistent ordering of operations

Consistency and replication Client-centric consistency models

Client-centric consistency:
• Ensures clients see updates in an orderly manner.
• Ideal for systems where data updates are not frequent or complex
to merge.
Eventual Consistency:
• Clients may see outdated data temporarily.
• Eventually or at the end, all clients will see the latest updates.
Example:
Social Media Feed:
• You post a photo on a social platform from your phone.
• Your friend might not see the photo immediately if they refresh
their feed from a computer.
• After a short delay, they refresh again and see the latest photo
you posted.
Monotonic reads
Consistency and replication Client-centric consistency models

Monotonic reads vs Monotonic Writes

Monotonic Reads:
Once a value is read, any future read will show the same value or
more a recent value.
Example: Reading shared online document from different locations,
you will always see the latest edits you made, or edits made by others,
but never an older version of the document.

Monotonic Writes:
Writes by a process are completed in the order they're made.
Example: If you update your contact list on your phone, that same
order of updates applies when you access it on your laptop.

Monotonic reads
Consistency and replication Replica management

Strategies for Replica Placement in Distributed Systems

Objective of Replica Placement:
To determine the best available options to place copies of data in a distributed
system.

Strategies:
• Minimize Client Distance: Choose locations closest to clients.
Computationally intensive.
• Autonomous System Approach: Place in largest systems' best-
connected hosts. High computational cost.
• Geometric Space Distribution: Identify nearest regions in a d-
dimensional space for placement based on latency and distance.
Computationally efficient.

Finding the best server location

Consistency and replication Consistency protocols

Consistency Protocols: Primary-based protocols

Primary-backup protocol

Example primary-backup protocol

Traditionally applied in distributed databases and file systems that require
a high degree of fault tolerance. Replicas are often placed on the same
SequentialLAN.
consistency: Primary-based protocols
Consistency and replication Consistency protocols

Consistency Protocols: Replicated-write protocols

Quorum-based protocols
Assume N replicas. Ensure that each operation is carried out in such a
way that a majority vote is established: distinguish read quorum NR and
write quorum NW . Ensure:

1. NR + NW > N (prevent read-write conflicts)

2. NW > N/2 (prevent write-write conflicts)

Correct Write-write conflict Correct (ROWA)

Sequential consistency: Replicated-write protocols

Consistency and replication
Example: Caching and replication in the Web

Example: replication in the Web

Client-side caches
• In the browser
• At a client’s site, notably through a Web proxy

Caches at ISPs
Internet Service Providers also place caches to (1) reduce cross-ISP traffic
and (2) improve client-side performance. May get nasty when a request
needs to pass many ISPs.
Consistency and replication Example: Caching and replication in the Web

Cooperative caching
Consistency and replication Example: Caching and replication in the Web

Web-cache consistency
How to guarantee freshness?
To prevent that stale information is returned to a client:
• Option 1: let the cache contact the original server to see if content is
still up to date.
• Option 2: Assign an expiration time Texpire that depends on how long
ago the document was last modified when it is cached.
Consistency and replication Example: Caching and replication in the Web

Summary

• Why we replicate data and challenges.

• Data-centric consistency models.

• Sequential consistency with examples.

• Grouping operations with example.

• Client-centric consistency.

• Monotonic reads vs Monotonic writes.

• Strategies for Replica Placement in Distributed Systems.

• Consistency Protocols (Extra).

• Replication in the Web and Web-cache consistency.