Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Download
Standard view
Full view
of .
Look up keyword
Like this
4Activity
0 of .
Results for:
No results containing your search query
P. 1
Redundant Storage Cluster: For When It's Just Too Big

Redundant Storage Cluster: For When It's Just Too Big

Ratings: (0)|Views: 131|Likes:
Published by Best Tech Videos
Our system has a number of large items to store, retrieve quickly, and eventually age out. When that number reached over a million a day (with two million just around the corner), we knew we weren’t “in Kansas anymore”. Ordinary tables just weren’t cutting it. Something had to be done.

Although there are horizontal scaling solutions out there, such as HiveDB and HScale, we need the speed of horizontal scaling plus with the security of redundant storage. Our storage cluster meets these needs.

The seeds for our cluster were planted at last year’s UC. Listening to presentations about horizontal scaling, I imagined a system like that to solve our problem. Everything was there except for data safety. Then I started investigating the Google File System and Bigtables and it all fell into place: directory-based storage plus redundancy and automatic rebalancing!

The goal of my presentation is to completely describe the design concepts and details of our cluster and to share performance measurements and details of how it fits in our database. Hopefully, even if our storage cluster doesn’t exactly meet someone’s requirements, it will inspire them to design something similar. Being written in Lua under MySQL-Proxy, the presentation will also illustrate Lua programming details under Proxy. Also, the basics of federated tables and shell scripting (pertaining to MySQL) will be covered briefly.
Our system has a number of large items to store, retrieve quickly, and eventually age out. When that number reached over a million a day (with two million just around the corner), we knew we weren’t “in Kansas anymore”. Ordinary tables just weren’t cutting it. Something had to be done.

Although there are horizontal scaling solutions out there, such as HiveDB and HScale, we need the speed of horizontal scaling plus with the security of redundant storage. Our storage cluster meets these needs.

The seeds for our cluster were planted at last year’s UC. Listening to presentations about horizontal scaling, I imagined a system like that to solve our problem. Everything was there except for data safety. Then I started investigating the Google File System and Bigtables and it all fell into place: directory-based storage plus redundancy and automatic rebalancing!

The goal of my presentation is to completely describe the design concepts and details of our cluster and to share performance measurements and details of how it fits in our database. Hopefully, even if our storage cluster doesn’t exactly meet someone’s requirements, it will inspire them to design something similar. Being written in Lua under MySQL-Proxy, the presentation will also illustrate Lua programming details under Proxy. Also, the basics of federated tables and shell scripting (pertaining to MySQL) will be covered briefly.

More info:

Published by: Best Tech Videos on May 02, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

05/11/2014

pdf

text

original

 
Redundant Storage Cluster
For When It's Just Too Big 
Bob Burgess
radian
6
TechnologiesMySQL User Conference 2009
 
Scope
Fundamentals of MySQL Proxy
Fundamentals of LuaSQL
Description of the
Redundant Storage Cluster 
Architecture
Sample code
 
Scope
 –
NOT
Complete Course on MySQL Proxy
Lua Programming Course
Amazing Lua Tricks
Complete code listing of cluster(Posted online)

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->