You are on page 1of 36

Continuent Tungsten

:
Proxies on Steroids for HA
and Performance!
Technology Preview

Robert Hodges, CTO of Continuent
April 15, 2008

© Continuent
4/14/08
What We Will Cover
/ Defining the Scale-Out Problem
/ A Hard Look at MySQL Replication for Scale-Out
/ Can We Do Better?
/ Replication Reloaded: Tungsten Scale-Out Stack
/ How to Get Software and Get Started Today

2 © Continuent
A Word from Our Sponsor: Continuent

/ Company
• Leading provider of database availability solutions for open source
databases
/ Products
• uni/cluster – multi-master database clustering that replicates data across
multiple databases and load balances reads
• Works with off-the-shelf database servers
• Uses “database virtualization” to provide a seamless client interface

/ Value
• Low-cost open source business critical solutions
• Highly available data
• Raise performance and hardware utilization through load balancing
• Remove chance of data loss

3 © Continuent
Defining the Scale-Out Problem

4 4 © Continuent
© Continuent Confidential
Requirement: Availability and Performance

/ Data Availability
• Bring up alternate database after local failure
• Bring up alternate database after site loss
• Keep both sites on-line for immediate failover
• Maintain/upgrade/migrate without disruption
• Restart/restore following data loss or outage due to human error
• Ensure commits to multiple databases and/or sites
• Always accept transactions no matter what
• Stay available in grid environments

/ Performance
• Scale reads transparently
• Scale writes transparently
• Spread transaction load across multiple active sites
• Rapid capacity scaling up/down in grid environments

/ Nirvana: increased data availability and
performance scalability by adding commodity
hardware

5 © Continuent
The Dream - Flexible Scale-Out

App
Scalability

App
Db
App
Db
Db

Availability

6 © Continuent
A Hard Look at MySQL Replication

7 7 © Continuent
© Continuent Confidential
MySQL Replication for Scale-Out

/ MySQL database replication has set the high bar for
horizontal scale-out on large websites for many years
/ Replication is “baked-in” to MySQL
• Initial set-up takes only a few minutes
• Handy SQL commands to configure masters and slaves
/ Flexibly copies all or some databases
/ Offers a choice of statement or row replication
• Row replication only in MySQL 5.0 and previous releases
/ Works across WANs, hence can be used to link
multiple sites
/ In short, it’s a key reason why MySQL is so popular

8 © Continuent
…But Is It Really That Easy?

/ Replication is hard -- no single approach solves all
problems
/ Master is the weak point in the MySQL design
• Replacing a crashed master when there are multiple slaves
• Sync between binlog and database
• Loss of committed transactions
/ Latency due to single threaded update on slaves
/ Administration is harder than it looks
• Parameter settings: innodb_flush_log_at_trx_commit=2
• Provisioning, resynchronizing, monitoring
• Automating failover
/ Read scaling requires application changes
/ Not a write scaling solution

9 © Continuent
A Hard Problem -- Master Election after a Crash

Master Slave

Where was I??

New
Slave Slave
Master
Where was I??

10 © Continuent
Customer Feedback from Our Clustering Work

/ Use cases for data base clustering driven by
HA/scaling/both
/ Analysis of US/Europe sales shows following
patterns:
• Almost 100% of customers are looking for HA solution
• Scalability was a requirement for ~30%

/ What Customers Need
• Transparent, easy-to-deploy HA
• Protection from data loss/corruption
• Higher database performance with fewer application
changes

11 © Continuent
MySQL vs. Customer Requirements

/ MySQL replication is excellent for implementing read
scaling on custom applications
• For the do-it-yourself set, it’s the best thing out there
/ BUT--It does not suit everyone:
/ Users who need easy-to-deploy availability
/ Users who cannot afford to lose data
/ Users who cannot afford intrusive application
changes
/ Users who are not MySQL experts

12 © Continuent
Can We Do Better?
New Technologies for Scale-Out

13 13 © Continuent
© Continuent Confidential
Continuent Database Clustering History

/ Continuent implemented m/cluster for MySQL -- the
first successful commercial cluster based on group
communications
/ Our current uni/cluster product implements clusters
using middleware and off-the-shelf databases
/ We have supported MySQL and PostgreSQL for years
/ Strong experience in commercial/open source
solutions
• Sequoia -- Multi-master state-machine clustering
• Myosotis -- Fast native-to-JDBC proxies
• Hedera -- Generic group communications adapters
• Bristlecone -- Testing tools for scale-out architectures

14 © Continuent
What Have We Learned?

/ Replication works for building clusters - It’s the only
approach that really scales
/ Database virtualization using middleware is essential
for encapsulating location of copies
/ Group communications are hard to configure but
highly effective for controlling clusters
/ Biggest challenges are simple management and
simple application semantics
/ WAN support is the killer application of database
clustering

/ There is no one-size-fits-all solution

15 © Continuent
What Are the Up-And-Coming Technologies?

/ CAP-aware databases (Consistent, Available,
Partition-Tolerant)
• Now appearing in non-SQL databases near you: Amazon
Dynamo and SimpleDB
/ Pluggable replication -- Different replication
mechanisms for different problems
• GORDA research project has pioneered approach
/ Partitioning -- Dividing data and automatically routing
queries
• MySQL community is driving solutions
/ Open source databases -- This stuff is going to be big
some day!

16 © Continuent
Replication Reloaded:
The Tungsten Scale-Out Stack

17 17 © Continuent
© Continuent Confidential
Our Goal

Highly reliable, transparent, easy to
deploy solutions for data availability
and performance scalability that work
with unaltered, stock RDBMS and
economical, off-the-shelf hardware

Tungsten: An architecture for
scale-out that meets the goal

18 © Continuent
Tungsten Overview
/ Data Services integrate multiple copies of data within
and across sites into a single highly available,
performant database
/ Database virtualization with controlled data latency
and consistency
/ Database-neutral replication designed for high
availability
/ Cluster-aware management framework backed by
excellent graphical tools
/ “Stack” organization of technology to allow
substitution and extensions

19 © Continuent
An Open, Database-Neutral Scale-Out Stack
Application Stacks
PHP, Perl,
Perl, Java, Ruby, etc.

Tungsten Connector
Native Wire Protocol Proxying;
Proxying; Connection semantics

Sequoia Tungsten Replicator
Synchrononous Asynchronous
Multi-Master Replication Master/Slave Replication

Tungsten Manager
Cluster-aware service management for LAN/WAN

Bristlecone Hedera
Scale-Out Perf Testing Group Comm Adapters

DBMS Servers Group Communications
MySQL,
MySQL, PostreSQL,
PostreSQL, Oracle JGroups, Appia,
Appia, Spread

20 © Continuent
Why Are Stacks So Important?

/ No single solution can solve all data availability and
scaling use cases
/ Stack approach allows you to reuse components on
different scale-out problems
/ The stack is an extensible framework

Caching Tungsten Connector (Myosotis
(Myosotis))

Tungsten
Sequoia
Replicator
Partioning

Tungsten Manager
Certification-
Based Bristlecone Hedera
Replication

21 © Continuent
Virtualized SQL Connection Semantics

Application Stacks
PHP, Perl,
Perl, Java, Ruby, etc.

Updates
All Updates Own Updates Eventually
Visible Visible Visible

Node Node Node Node

Tungsten Data Service

22 © Continuent
High-Availability Replication
Master-master replication Master-slave replication
/ No master failover headaches / Wider range of supported SQL
/ Automatic load balancing / Faster performance
/ Works over WAN

Node Node

Node Node Node Node

Sequoia Tungsten Replicator

23 © Continuent
Managing within and across Sites
Site 2
Administrative
Client
Host Host Host
JMX
Event Bridge
Group Communication

Service Service Service Service
Manager Manager Manager Manager

Connector Connector Replicator Replicator
Node Node
Connector Connector
Host Host
Host Host

Site 1
24 © Continuent
GUI Tools Demo

25 © Continuent
Now -- Multi-Master Data Services
/ Ensure constant availability and avoid master failure
scenarios
/ Focus: Scalable and easily manageable multi-master
replication
• High quality management tools
• Automatic load balancing of read requests
• 3+ master nodes
• Seamless connections from native clients and Java
• Support for latest MySQL versions and table types
• Solaris and Linux platforms

26 © Continuent
Now -- uni/cluster 2008
LAMP Stack Clients Clients
Client
(PHP, Perl,Applications
C, Ruby, etc.)
Java Clients
uni/cluster Driver

uni/cluster connector
uni/cluster JDBC Driver

Controller Controller Controller Controller

JDBC Driver JDBC Driver JDBC Driver JDBC Driver

DB DB DB DB
Server Server Server Server

27 © Continuent
Phase 1 - Master/Slave Data Services
/ Constant availability with high transparency and
update performance
/ Focus: Seamless upgrade from existing standalone
database
• One-step software installation
• Rapid data service configuration
• Lightweight, multi-platform management tools
• No application code changes
/ High availability replication
• Automatic and manual promotion of slave to master
• Bounded latency between slave and master
• Transparent proxying with read distribution to idle slaves

28 © Continuent
Phase 1 - Master/Slave Data Service
Client Client

Connectors
Tungsten Mamagement Tools

Failover, Connection Semantics

Master DB Slave DB Slave DB
Server Server Server

Replication Replication Replication
Node Mgr Node Mgr Node Mgr

Service Management

29 © Continuent
Phase 2 - Cross-Site Data Services
/ Ensure availability and scaling across sites
/ Focus: Cross-site clustering with efficient failover
• Replication of data to secondary site databases
• Minimal, understandable application changes
• Accept transactions across active systems on multiple sites
/ WAN clustering features
• Tools and admin model for cross-site management
• Handle WAN quality of service (latency, transient failures)
• SQL partitioning
/ Emphasis on WAN solutions that work without corner
cases or data inconsistencies

30 © Continuent
Phase 2 - Cross-Site Data Services
DNS Load Balancing

Read-Only Read-Only
Read/Write Read/Write
Applications, Applications,
Applications Applications
Reports Reports

Connectors
Data Service Connectors

Slave DB Master DB Slave DB
Server Server Server

Site 1 Site 2
31 © Continuent
How to Get Software and Get
Started Today

32 32 © Continuent
© Continuent Confidential
Technology Roadmap
/ Uni/cluster 2007.1 is available today!
/ Uni/cluster 2008 available in late Q2
• GUI administrative tools
• Tungsten management framework
• Multiple controllers
/ Tungsten scale-out stack projects will be published
soon
/ Tungsten master/slave services will be available early
Q4

33 © Continuent
Getting Software
/ For access to commercial and open source products,
visit us at the following web addresses

www.continuent.com
www.continuent.org

/ Visit us at the Continuent booth in the exhibit hall
/ Watch for more information on Tungsten very soon!

34 © Continuent
For More on Scale-Out Performance…

/ Come hear about scale-out testing using Bristlecone

Portable Scale-Out
Benchmarks for MySQL
Wednesday 10:50-11:50am
Ballroom D

35 © Continuent
Questions?

Thank you!

36 36 © Continuent
© Continuent Confidential