This action might not be possible to undo. Are you sure you want to continue?
Jeff Stoner Performance Engineer @ OpSource Inc firstname.lastname@example.org I am not a DBA
This presentation is licensed under Creative Commons AttributionShare Alike 3.0 (http://creativecommons.org/licenses/bysa/3.0/us/)
MySQL 3.23 in 2000
over NFS (shudder)
Brief MySQL 4 Heavy MySQL 5.0 since 2005
Standalone and clustered using Red Hat Cluster Sizes from <10 meg to >100 gig
I don't consider myself a DBA
Who is MySQL
Swedish company, MySQL AB, 1995 Sun Microsystems Inc. purchased MySQL AB in 2008 Famous names
Michael ”Monty” Widenius Brian ”Krow” Aker Jeremy Zawodny
What is MySQL
RDBMS with pluggable storage engines ANSI/ISO SQL compliance (see section 1.8 of the online manual for specifics) Spatial Extenstions (see section 20 of the online manual for specifics) Supports INFORMATION_SCHEMA metadata tables Dual license – GPL and commercial
What's this ”Drizzle” thing
A fork of the MySQL code Lead by Brian Aker Removed tons of features like stored procedures, triggers, grants, views, etc. Targeted at web sites looking for raw performance than features Still under heavy development
”Separation of duties”
SQL is processed in the upper layer
Parsing Optimizer Query cache Privileges Indexing Storage Fragmentation and reclaimation
Storage engines handle physical data
Connection Pool Query Cache
Why storage engines?
Allow for specialization of data storage and retrieval Allows the developer to use the appropriate feature set for the application Allows the admin to design and manage the system based on the engine
Always present, must have it Nontransactional No foreign key support Tablelevel locking Fulltext indexes Support inserts concurrent with reads if the inserts happen at the end of the table (no holes) The OS provides data cache, MySQL provides index cache
Transactional, MVCC, ACID Supports foreign keys (only across InnoDB tables) Traditional transaction ”redo log” design Rowlevel locking All tables in single data file(s) or filepertable, your choice Clustered indexes Developed by Oracle (InnoBase), not MySQL
Inmemory tables Uses Hash indexes by default but BTree is available Table definitions persist across restarts but data doesn't
Link to another table on another database server Currently only support MySQL databases
Can only use insert and select – no update, delete or replace Uses zlib to compress each row No indexes
Plain text, CSV files No indexes Easy to import/export data Not efficient
Present several tables as one without using a join or view Table definitions and indexes have to be identical Only works for MyISAM tables You define how inserts happen – first table, last table or not at all
NDB – MySQL's Cluster engine Falcon – MySQL's transactional engine Maria – next generation MyISAM (crashsafe) Blackhole – /dev/null BDB (BerkeleyDB) – dropped in 5.1 Example – template for writing your own Sphinx PBXT – transactional engine from PrimeBase MyBS (BLOB Streaming) – from PrimeBase
MySQL/Sun provides binaries for lots of platforms
Windows, Linux, Red Hat, Suse, Ubuntu, FreeBSD, MacOS X, Solaris, HPUX, AIX, IBM i5/OS, QNX, Netware, SCO x86, x86_64, PowerPC, IA64, POWER, PARISC 1.1 and 2.0, HPPA
And CPU architectures
And source code
Stick to the precompiled binaries unless you have a specific reason to compile yourself Percona provides binaries for patched versions of MySQL
These guys are damn smart Check out www.percona.com/perconalab.html for details
MySQL rpms require perlDBI rpm (Red Hat only?)
You'll have a user and possibly a group for running MySQL Main configuration file is my.cnf (Linux) or my.ini (Windows)
Controls MySQL and many support programs
Uses the traditional Windowsstyle INI format
RPM install will start MySQL with a default configuration that may will not be suitable The configuration sets up InnoDB using a single, monolith data file, two 5 meg log files Anonymous user is created and 'root' has no password Why?
MySQL needs to bootstrap the mysql database InnoDB is enabled by default and needs to start with something
mysmall.cnf, mymedium.cnf, mylarge.cnf, my huge.cnf, etc.
Use these as templates, not production (!!) Online manual has extensive documentation on the variables
[client] #password = your_password port = 3306 socket = /var/lib/mysql/mysql.sock # The MySQL server [mysqld] port = 3306 socket = /var/lib/mysql/mysql.sock skiplocking key_buffer = 256M max_allowed_packet = 1M table_cache = 256 [mysqldump] quick max_allowed_packet = 16M [mysql] noautorehash # Remove the next comment character if you are not familiar with SQL #safeupdates
There should be 2 processes
Shell script that sits in an infinite loop The MySQL daemon
You can run multiple MySQL database processes on the same server
Section 4.3.4 of the online manual
Where things live
User binaries in /usr/bin System binaries in /usr/sbin Data in /var/lib/mysql Configuration in /etc PID, socket and logs are in /var/lib/mysql (!!)
mysql – you'll spend 90%+ of your time here mysqladmin mysqldump backups mysqlimport – imports data myisamchk – recovery/repair of MyISAM tables mysqlbinlog – reading binary logs mysqldumpslow – analyzing slow queries
Client program like sqlplus and isql Connects to MySQL server through local or network socket Does not parse SQL, only concerned with displaying results Can be scripted fairly easily
Limited admin functions like changing passwords, creating/dropping databases, shutting down MySQL, etc. All these functions can be done using the mysql client so, why bother?
Used to produce dumps of databases, tables, data, structure, etc. Wealth of options at your fingertips Restoring a dump made using mysqldump is done using the mysql client – simply pipe the file into it! Backing up databaes that use multiple storage engines is not as simple as it might seem (!!)
Used to import delimited data (comma, tab, space, whatever) Can also be done using the mysql client (LOAD DATA INFILE)
Used to check and repair MyISAM index and data files Data files are very resistent to corruption Index files simply get rebuilt Can also be done using the mysql client (CHECK TABLE, REPAIR TABLE) Only used when MySQL is shut down (!!)
Reads the binary logs and produces plain text output that can be piped directly to mysql client That mysql client sure is a versatile tool, eh? Allows you to do pointintime recovery of databases But only if you enabled binary logging (!!)
Analyzes the slow log (you did enable it, didn't you?) Great for feedback into development Helps identify where indexes may be needed, bad SQL, etc.
Let's take a look
Asynchronous transfer of datachanging statements between 2 servers Statement based You can chain servers Easily allows you to scale reads You can't scale writes
Multimaster loops are possible but very tricky and hard to manage
Section 15 of the online manual
Need to set the serverid variable in my.cnf
All servers involved in replication need to have unique serverid values
Must enable binary logging on the master Create a user in the master database with just the 'REPLICATION SLAVE' priviledge
Initial data for slave
Option 1: use mysqldump on master, load into slave (use masterdata option) Option 2: stop master, copy the data files to slave Option 3: start with an empty master and load data into it In cases 1 or 2, start the slave with the 'skip slave' parameter and manually start replication
Always verify your slave data and database configuration works before opening the flood gates
Let the fun begin
Issue a 'CHANGE MASTER TO ...' statement on slave Issue the 'START SLAVE' command on slave Voila! Check it out with 'SHOW SLAVE STATUS\G'
How it works
2 threads on the slave
Slave IO thread reads the binary log on the master and writes it to the relay binary log Slave SQL thread reads the relay binary log and executes the statements
DROP TABLE FOO; INSERT INTO FOO VALUES (UUID()); Lag Slaves of slaves of slaves become hard to manage Binary log management A slave can only replicate from a single master
That's where the Blackhole engine comes in
All statments are executed serially in a single thread on the slave
Do not get the benefit of parallel execution the master enjoys
If all tables are using the same engine – no problem If tables use mixed engines, how do you synchronize the engines for a consistent backup? singletransaction only for transactional engines Flushing buffers and locking tables functions/procedures not dumped by default
Tools and info
O'Reilly's High Performance MySQL, 2nd ed. Maatkit http://www.maatkit.org
Parallel dump and restore, table checksums, heartbeat, replication tools and more
PlanetMySQL http://www.planetmysql.com Percona http://www.mysqlperformanceblog.com and http://www.percona.com MySQL Forge http://forge.mysql.com
Questions, comments Open SQL Camp (http://www.opensqlcamp.org)
Charlottesville, VA November 1416 Free!!!
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.