Professional Documents
Culture Documents
Availability Architectures
Jeremy Cole
jeremy@provenscaling.com
Eric Bergen
eric@provenscaling.com
Who are we?
• Proven Scaling is a consulting company founded
in 2006 by Eric and Jeremy specializing in MySQL
• Partition by “group”
• Partition by “user”
Partitioning Difficulties
• Inter-partition interactions are a lot more difficult
• Example: Partitioning by user, where do we store a
message sent from one user to another? How
about a friend list?
Master
Slave
Master with Many Slaves
Master
Master
Relay
Slave
Slave
Master with Relay and
Many Slaves
Master
Relay
Slave
Master
Master Master
Dual Masters with
Slaves
Slave
Master Master
Relay
Slave
Master Master
Master
High Availability Options
Dual Master
• Two machines with independent storage
configured as master and slave of each other
• Optionally: Any number of slaves for reads only
• Manual (scripted) or automatic (heartbeat-based)
failover is possible
Dual Master Pros
• Very simple configuration
• Simple to understand = simple to maintain
• Very similar to basic master-slave configuration
that many are familiar with
• Allows easy failover in either direction without
reconfiguration or rebuilding
• Allows for easy and reliable failover for non-
emergency situations: upgrades, schema changes,
etc.
• Allows for quick failover in emergency
• Can work between distant sites fairly easily
Dual Master Cons
• Does not help scale writes (no, not at all)
• Limited to two sites; replication does not allow
multiple masters, so three or more is not possible
• Replication is asynchronous, and may get behind --
there is always a chance of data loss (albeit small)
SAN
• Shared storage of a single set of disks by two
MySQL servers, with a single copy of the data on a
FibreChannel or IP/iSCSI SAN
• Automatic (heartbeat) failover by fencing and
mounting the SAN on the other machine
SAN Pros
• Single copy of the data means lower storage cost
for extremely large databases
• No worries about replication getting behind
• SAN systems can achieve very high performance
for same or lower cost as two very large RAIDs
SAN Cons
• Single copy of the data means corruption is
possible, and could be very damaging
• For medium or small databases, cost can be
prohibitive
• FibreChannel requires additional infrastructure
often not present in typical MySQL systems; iSCSI
can be very helpful in this regard
• Single copy of the data -- no schema change tricks
are possible
DRBD
• Block device-level replication between two
machines with their own independent storage
(mirrors of the same data)
• Automatic (heartbeat-based) failover by fencing
and mounting local copy of filesystem is typical
DRBD Pros
• Simple hardware and infrastructure using locally-
attached RAID
• No expensive hardware or network
DRBD Cons
• Complex configuration and maintenance
• May cause performance problems, especially if
poorly configured
• Failure of or problems with mirror can cause
problems in production
• From the software perspective, there is still a
single copy of the data, which may get corrupted
• Single copy of the data -- no schema change tricks
are possible
Putting It All Together
Partitioning + HA
• No partitioning solutions really address HA .. They
treat the “shards” or “partitions” as single MySQL
servers
• In reality you would implement an HA solution for
each partition
• There are many possibilities
HiveDB + Dual Master
• We recommend HiveDB plus Dual Master for most
installations
• While not technically perfect, and with a chance of
data loss, administrative tasks are very simple
• Additionally, LVM for volume management gives
ability to take snapshot backups easily
Any questions?
Discussion!