Designing and Implementing Scalable Applications with Memcached and MySQL

Farhan "Frank" Mashraqi
Director of Business Operations and Technical Strategy - Fotolog Inc

Monty Taylor
Senior Consultant - Sun Microsystems

Jimmy Guerrero
Sr Product Marketing Manager - Sun Microsystems, Database Group

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

1

• • • • • •

Sun – MySQL Overview Technical Introduction Architectures Use Cases Solutions Next Steps plus Q & A

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

2

Sun Acquires MySQL
MySQL Helps Sun...
• • • • • Grow in a $15B database market Reach new customers Position its offerings for Web companies Deliver innovation to community & customers Complete its software platform offerings

Sun Helps MySQL…
• • • • Expand market reach and revenue opportunities Enhance enterprise appeal with global services Enhance performance and integration Resources for accelerated growth

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

3

About MySQL

• • • • •

12 years old 400+ employees 750 partners 60K downloads/day Customers across every major operating system, hardware vendor, geography, industry, and application type

Leadership * Innovation * Open * Ubiquitous
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 4

Established & Emerging Companies
Web 2.0

Enterprise 2.0
craigslist

SaaS

Telecom

OEM & ISV
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 5

Technical Introduction

Monty Taylor
Senior Consultant - Sun Microsystems mtaylor@mysql.com

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

6

“Cache is King”

• Browser Cache • Web Server Cache
ms

• Memcached Cache • MySQL Database Cache • Disk Storage

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

7

What is Memcached?
“A high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load” *
* http://www.socialtext.net/memcached/index.cgi?faq

• Created by Danga Interactive to speed up LiveJournal’s 20 million+ dynamic page views per day for 1 million+ users • Significantly dropped database load, delivering faster page loads, better resource utilization, and faster access to databases • Perfect for dynamic sites that generate high database load • Used by Facebook, YouTube, Wikipedia, others!
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 8

What is Memcached?
• Runs wherever RAM is available
– Application, Web, Database or dedicated memcached servers – Low CPU utilization – Multiple operating systems and architectures supported

• Open Source - Community Driven • Memcached Server is licensed under BSD • Current version is 1.2.x series
– http://www.danga.com/memcached/download.bml

• Various Client APIs and libraries available
– Perl, Python, Ruby, Java, C#, C, Lua, MySQL, more…. – http://www.danga.com/memcached/apis.bml
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 9

Why Use Memcached With MySQL?
• Enables massive scale-out of dynamic web-sites • Faster page loads • Allows for more efficient use of existing database resources • Can easily utilize idle computing resources • Dozens to hundreds of nodes can be supported in a memcached cluster • No interconnect or proprietary networking required • Extensible and customizable

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

10

How Does Memcached Work? Hash Function
• A hash is a procedure for turning data into a small integer that serves as an index into an array • Speeds up table lookup or data comparison tasks

Memcached
• Two-stage hash, like a giant hash table looking up key = value pairs • Client hashes the key against a list of servers • When the server is identified, the client sends its request • Server performs a hash key lookup for the actual data
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 11

Memcached Server
• Slab allocator
– Memory permanently allocated – Classes created by chunk size – Cannot reassign slab pages

• Libevent based • Simple protocol (no xml) • Server has internal hash table • Servers know nothing about each other

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

12

Memcached Server
• Limits
– Key size (250 bytes) – Data size (under 1 MB) – 32bit/64bit (maximum size of process) – Maxbytes (limits item cache)

• LRU
– Least recently accessed items are cycled out – One LRU exists per “slab class” – LRU “evictions” need not be common

• Threads
– New in version 1.2 – Big instances (16 GB+) – Large multiget requests
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 13

Memcached Commands
• set/get/replace • append/prepend • increment/decrement • cas (compare and swap atomic) • stats (detail)

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

14

Memcached Clients
• Client hashes key to server list (distribution) • Serializes the object • Compresses data • “Consistent hashing” client • Other things to know about memcached…
– Dumping and loading data – Redundancy – Failover – Authentication

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

15

Basic Memcached Example
X
mc

Y
mc

Z

Client X
mc

1) set key “foo” with value “seattle” 2) hashes the key against server list

hash server list select server connect set key value

get key connect get value

3) Server B is selected 4) connects to Server B and sets key

Client Z
5) get key “foo” 6) connects to Server B
ms ms ms

7) requests “foo” and gets value “seattle”

A

B
key = value foo = seattle

C

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

16

Memcached vs MySQL Query Cache

• • • • •

Distributed Unlimited Size Generic Data Storage No Direct Invalidation Maximum 1 MB

• Centralized (Single Server) • Limited by maximum addressable space of mysqld • Database data only • Invalidated by any write on tables used in SELECT • Configurable per item

ms

ms

ms

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

17

Memcached vs MySQL Cluster

• • • • •

Distributed Unlimited Size Generic Data Storage No Direct Invalidation Maximum 1 MB

• Distributed • Limited Size • Datatype storage limits • No invalidation (DELETE instead) • HA and Redundant • SQL and NDB API support

Application Nodes
(NDB API and/or MySQL Server)

ms

ms

ms
Management Node Data Nodes

NDB Storage Engine

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

18

Architectures

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

19

Memcached & MySQL
read write

Web Servers

Application Servers

Memcached Clients

mc

mc

mc

Memcached Servers

ms

ms

ms

cache update MySQL Server

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

20

On-Demand Read Scalability for MySQL
• • • • • • • MySQL Replication Designed for on-demand read scalability Cost-effective alternative to Scale-Up Asynchronous replication Statement or row-level replication support No special networking components Fully-supported via MySQL Enterprise subscription Enterprise Monitor discovers and monitors replication topologies

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

21

MySQL Replication – Scale-Out
Web/App Servers MySQL Replication

writes

MySQL Replication

reads reads

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

22

Memcached & MySQL Replication
read from cache
Application Servers & Memcached Clients

write

read from slave

mc

mc

mc

Memcached Servers

ms

ms

ms

MySQL Master

cache update

MySQL Replication
MySQL Slaves

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

23

Sharding - Application Partitioning
Web/App Servers Sharding Architecture

Cust_id 1-999

Cust_id 1000-1999

Cust_id 2000-2999

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

24

Memcached, Sharding & MySQL Replication
read from cache write read from slave

mc

mc

mc

shard 1 (cust 1-999)
read from cache
ms ms ms

shard 2 (cust 1000-1999)
read from cache
ms ms ms

cache update

write

cache update

write

MySQL Replication

MySQL Replication

read from slave
Copyright 2008 MySQL AB

read from slave
The World’s Most Popular Open Source Database 25

Use Cases

Farhan "Frank" Mashraqi
Director of Business Operations and Technical Strategy Fotolog Inc fmashraqi@fotolog.com

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

26

Fotolog Introduction
• World’s largest photo blogging community
– > 18 million members – > 155 million page views

• Ranked top 57 site
– 13th before Alexa changed ranking algorithm

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

27

Memcached @ Fotolog
• • • • 51 instances on 21 servers ~254G available to memcached servers ~175G in use Cache everything that is slow to query, fetch or calculate

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

28

Memcached Use Cases
• • • • • • • Non-deterministic cache: Classic memcached model “State” cache Deterministic cache Proactive cache Replacement/add-on for file system cache Partial page cache Application-based memcached replication

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

29

Non-Deterministic Cache: Classic Memcached Model
• Check memcached, check database, then populate memcached • Previously using classic implementation:
– App servers running memcached on “spare” memory
• 90% CPU usage • Memory garbage collected nearly once a minute • Experienced blocking on memcached on app servers

• 45 memcached instances dedicated to nondeterministic cache
– Each instance (on average):
• ~440 gets per second • ~40 sets per second • ~11 gets/set

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

30

Non-Deterministic Cache: Classic Memcached Model
• Ideal for:
– – – – – – Caching objects that are to be queried several times Replacement for MySQL’s query cache Caching relationships and other lists Slow data that’s used across many pages Don’t cache if its more taxing to cache than you’ll save Tag clouds and auto-suggest lists

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

31

State Caching
• • • • “Who’s online?” ~9G dedicated Multiple instances Ideal for caching:
– Sessions – Current state of users
• Especially for IM applications

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

32

Deterministic Cache
• Memcached has what database has…read scalability • Multiple dedicated cache pools • Cache has to be populated when the site “starts”
– – – – – – – – – – Maintained by application from that point onwards ~ 90,000 gets / second across cache cluster ~ 300 sets / second get/set ratio of ~ 300 Heavily accessed data/objects/lists User credentials User profiles User preferences Active media belonging to users Outsourcing logins to memcached
The World’s Most Popular Open Source Database 33

• Ideal for caching:

Copyright 2008 MySQL AB

Proactive Caching
• “It’s magically there” • Fun with binlogs
– Easier to implement for UPDATEs – A little bit of extra logic required for INSERTs – DELETEs can be honored easily

• Ideal for
– Pre-Populating Cache: Keeping memcached updated minimizes calls to database if object not present – “Warm up” cache in cases of cross data-center replication

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

34

Proactive Caching
• Fun with UDFs • Memcached UDFs for MySQL
– – – – – Major contributing author: Patrick Galbraith Allows you to manage memcached when used with triggers. Implemented using libmemcached Compatible with all versions of MySQL Examples:
• • • • • • • memc_servers_set() memc_set() memc_get() memc_delete() memc_increment() memc_decrement() memc_replace()

• Complete list available at
– http://tangent.org/586/Memcached_Functions_for_MySQL.html
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 35

Proactive Caching
• Fun with blackhole
– thanks to Brian Aker

• Create a blackhole table
– CREATE TABLE blackhole_table … ENGINE=blackhole;

• Replicate this table • Add triggers at right places. • Example: Make multiple INSERTs
– INSERT <real_insert>; – INSERT INTO blackhole_table VALUES (memc_set(id));

• Benefits
– Actual data is not sent to slave disks – Not CPU intensive for slaves

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

36

Add-on/Replacement for NFS File System Cache
• NFS has a significant overhead
– File system cache for NFS systems not enough

• Ideal for caching:
– XML files – Small size media stored on NFS mounts

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

37

Partial Page Caching
• Certain partial page elements are…
– Very costly to generate – Shared among many pages – Viewed several times

• Examples:
– Top users within a region – Popular photo lists – Featured users

• Once generated these can be shared

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

38

Application-Based Memcached Replication
• Ideal for deterministic cache cluster • Redundancy
– Application writes to multiple memcached pools

• High availability
– Take node out of rotation

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

39

Memcached Wishlist
• • • • “Built-in” replication Integration with binary logs A “hybrid” storage engine? Production support
– Now fulfilled thanks to Sun/MySQL

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

40

Solutions

Jimmy Guerrero
Sr Product Marketing Manager Sun Microsystems - Database Group jimmy@mysql.com

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

41

Memcached for MySQL
• Support is built into your MySQL Enterprise subscription
http://www.mysql.com/products/enterprise/memcached.html

• MySQL Enterprise
– – – – 24x7 Production Support Enterprise Monitor MySQL Enterprise Server Additional Add-ons Available

• MySQL Professional Services
– MySQL Scale, High-Availability and Replication Jumpstart

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

42

MySQL Enterprise
Enterprise database and monitoring software, plus production support delivered in an annual subscription
MySQL Enterprise

MySQL Enterprise Server

MySQL Enterprise Monitor

24x7 Production Support

MySQL Enterprise Unlimited

Deploy and monitor an unlimited number of servers for the price of a single CPU of Oracle

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

43

Why MySQL Enterprise?
• Deliver on SLAs • Increase availability • Optimize performance • Effectively monitor scale-out environments • Faster time to launch • Develop & deliver on proven open source components • Keep costs predictable
Copyright 2008 MySQL AB The World’s Most Popular Open Source Database 44

MySQL Enterprise Monitor
• • • • • • • Consolidated view of entire MySQL environment Auto discovery of MySQL Servers & replication topologies Over 100 customizable rules for monitoring and alerting Identifies problems before they occur Reduces risk of downtime Improve delivery of SLAs Scale-out with less DBAs

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

45

Why Memcached with MySQL?
• Enables massive scale-out of dynamic web-sites • Faster page loads • Allows for more efficient use of existing database resources • Can easily utilize idle computing resources • Dozens to hundreds of nodes can be supported in a memcached cluster • No interconnect or proprietary networking required • Extensible and customizable

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

46

Next Steps
Memcached for MySQL http://www.mysql.com/products/enterprise/memcached.html

Whitepapers - http://www.mysql.com/why-mysql/white-papers/
• • • “Designing and Implementing Scalable Applications with Memcached and MySQL” “How MySQL Powers Web 2.0” “Enabling Enterprise 2.0 with MySQL”

Documentation http://dev.mysql.com/doc/refman/5.1/en/ha-memcached.html

Discussion Forum - http://forums.mysql.com/list.php?150 Miscellaneous Resources http://www.mysql.com/products/enterprise/memcached.html

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

47

Questions?
Farhan "Frank" Mashraqi
Director of Business Operations and Technical Strategy - Fotolog Inc

Monty Taylor
Senior Consultant - Sun Microsystems

Jimmy Guerrero
Sr Product Marketing Manager - Sun Microsystems, Database Group

Copyright 2008 MySQL AB

The World’s Most Popular Open Source Database

48

Sign up to vote on this title
UsefulNot useful