M y SQL Magazine

Summer 2007 Issue 1

How Secure Is Your Server?

What will you find here? News, articles on both administration and devlopment, feedback from the users. Simply anything relating to MySQL!!

Welcome to MySQL Magazine. This is the magazine for the community of MySQL database administrators and developers who use MySQL on a daily basis to provide some of the best infrastructure available on the Internet.

MySQL in the News

The biggest news of the past few months for MySQL users has to be the MySQL Conference that was April 23rd through the 26th. If you didnt go-you missed out! There were many speakers including our own Baron Schwartz. Topics ranged from building your basic knowledge, improving the performance of your servers and quite a few presentations on how large Web 2.0 companies such as Digg and Flickr handle their traffic loads. Many of the speakers have their slides available for viewing. You can find them and other informaion about the conference at: http://www.mysqlconf.com. In addition there are several people in the MySQL community who have graciously supplied audio and video from the conference. An RSS feed of mp3s that will work with itunes is at http://download.tailrank.com/mysqlconf/mysqlconf.rss. Kevin Burton created these mp3s from ogg vorbis recording that Baron Schwartz made (available at http://www.xaprb.com/presentations/mysqlconf2007/). In addition, Sheeri Kritzer, has made available her recording (both audio and video) of presentations at the conference. They are available at http://technocation.org/index.php?option=com_content&task=view&id=54 As an aside, Sheeri does a great MySQL podcast called OurSQL (available from the itunes store). MySQL AB has also recently announced the addition of two new certifications. In March MySQL AB announced the new Certified MySQL Cluster DBA. Next, in April, MySQL AB announced the new Certified MySQL Associate. The CMCDBA (rolls off the tongue doesnt it?) shows mastery of all facets of the new MySQL cluster server. The CMA is a more entry-level certification showing basic knowledge of the MySQL server. Next year the conference will be April 15-18. Go ahead and make plans to attend. You wont regret it!

Book Worm

One of the books on my bookshelf is Pro MySQL by Michael Kruckenberg and Jay Pipes. This is an excellent book for someone seeking information about MySQL that goes beyond the basics. At almost 700 pages (not including index) this book is divided into two parts. The first part is devoted to the architecture of the MySQL server and MySQL-based devlopment. The second half of the book devoted to database administration. Some of the topics include benchmarking/profiling (worth the cost of the book alone), storage engines, security, backup, replication and clustering. Each is covered in-depth from true experts.

This is a well written book. If you consider yourself a professional MySQL DBA the information is invauluble. This book makes a great reference for topics that you might not deal with on a day to day basis. My only quible would be that it needs to be updated. It was printed in 2005 and covers up to version five of our favorite database. In particular the informaiton on clustering is dated (because with version 5.1 the tables are no longer required to be in memory). Even so the book is still well worth the money.

Contents
3 6 9 12 14 MySQLs Future? Securing MySQL Server Optimizing MySQL with Rails - Part One Convenient MySQL Privilege Scripting and Versioning Operating System Hardening

Summer 2007

1 1 4 23

Regular Features

News Book Worm Coding Corner - Protecting a MySQL Database from SQL Injection Attacks - Part 1 log-bin

MySQLs Future?
by Keith Murphy So where is MySQL going? Before we examine this question it is a good idea to look at the current feature set of MySQL (as of version 5.1.x). ACID compliance transactions triggers functions cursors views subqueries transactions row-based replication multiple database engines partitioning shared-nothing clustering it would be nice to see some polish applied to this often used area of MySQL. News from the recent MySQL user conference has the 6.1 release having online backup. In addition there should be subquery optimization in version 6.1. These are the major features slated for the next two releases of MySQL. In addition, there will continue to be improvements over many areas of MySQL.

Prior to the release of version 5.1 there was discussion of the implementation of an onlinebackup tool that would be superior to current implementations. This was originally slated for the 5.1 release. This just has not happened. As a DBA who performs back-ups on a routine basis

Another item of interest is implementation of foreign keys for all the major database engines in version 6.1. Currently foreign keys are only supported on InnoDB. Foreign keys are a great help for the developers out there in keeping the data in the database syncronized.

What is on the horizon for our favorite database? Certainly there has been a good deal of buzz of late about the new Falcon database engine. There has been an alpha release (6.0) of the server that includes the the new engine. Please be forwarned that this database engine is in no way production quality. It certainly has great promise. The Falcon db engine was created by Jim Starkey who is one of the pioneers of RDBMS. Some people think that Falcon will become the default engine for MySQL - especially after Oracle bought Innodb. The expected release date for a production version 6.0 is Q2 2008.

We are fortunate that MySQL AB and the community together have been able to implement almost every feature that is of significance. Even though this is the case, there are areas that will continue to move forward. Clustering is a feature that will continue to gain momentum as it becomes better implemented. I find this area of MySQL technology fascinating and I look forward to the combination of growing stability and new feature advancements of the young clustering environment. Keith Murphy is the editor of MySQL Magazine. He is also the chief Dolphin at Broadwick Corporation. About the author

by Peter Brawley http://www.artfulsoftware.com

Coding Corner

Protecting a MySQL Database from SQL Injection Attacks - Part I

If your application's connection supports multiple SQL commands in a single query call, all your customer data just went away.

Web sites that send SQL commands to databases are particularly vulnerable to SQL injection, because they often rely on dynamic SQL, and because it can be easy to mount millions of such attacks until one succeeds. Here is a simple example. A PHP page asks the user for a name and a password, then sends this to the database:
SELECT * FROM mysql.user WHERE user = '$usr' AND password = '$pwd';

SQL injection is a method for attacking databases. The attacker "injects" elements into your program's SQL in order to bypass authorization or damage the database.

The simplest way to prevent this sort of injection is to search the SQL string for semi-colons and double dashes, and remove them before passing the statement to the database. That's easy in an adequate application language, for example in PHP: If $qry has offending characters, sending $protectedqry to the database raises a MySQL error. That provides one level of protection.

LEVEL 1 DEFENSE: NEGATIVE INPUT FILTERS

$protectedqry = str_replace( "--", "", str_replace( ";", "", $qry ));

Is it as innocuous as it looks? Suppose a user enters something like this as a user name: When your application plugs that entry into your SQL, the command becomes:
SELECT * FROM mysql.user WHERE user = '' OR 1>0; -AND password = '' ' OR 1>0; --

Your intruder just retrieved all rows and columns of the mysql.user table. Not exactly what you had in mind.

Or a malevolent user might supply this username: whereupon your application sends this command to the database:
SELECT * FROM mysql.user WHERE user = '' OR 1>0; TRUNCATE customers; -- ' AND password = '' OR 1>0; TRUNCATE customers; --

Now you are fully protected against attacks that use double dashes and semi-colons. Have you covered all possible attacks? Not a chance, human ingenuity having no practical limit. For example, a favorite trick we haven't touched on is introduction of malevolent WHERE clauses. LEVEL 2 DEFENSE: POSITIVE INPUT FILTERS The attacker has to succeed just once. If your database is to be safe, you must succeed every time. You are on better logical ground enforcing a simple positive validation pattern than looking for a limitless number of dangerous or invalid patterns. Positive input filters improve your chances of success enormously.

Better still, search the string for double dashes and semi-colons, and if either is found then refuse to send the query to the database. If you want to be really thoroughgoing, you could blacklist the IP address that launched the attack.

For example, you could decide to accept only alphanumeric characters in user names and passwords. It is easy to enforce that rule in PHP:
if( ereg( '[^A-Za-z0-9]+', $usr.$pwd )) { echo "<script> alert('Alphabetic and numeric characters only, please.'); </script>";

You can formulate more stringent tests based on specific input requirements. LEVEL 3 DEFENSE: OUTPUT FILTERS

$qry = mysql_real_escape_string( $qry, $connection_resource );

Finally, application languages provide generic tools for cleaning up submissions to your database. Again in PHP the function to use is mysql_real_escape_string():

LEVEL 4 DEFENSE: ENCAPSULATION

Enterprise RDBMS policies usually require that all such protective logic be encapsulated in stored procedures. In Part II we will study how to do this in MySQL. To stop SQL injection attacks in their tracks, apply simple positive and negative input filters, and escape possibly problematic characters in what you send to the database. Peter Brawley is president of Artful Software Development and co-author of Getting It Done With MySQL 5. About the author Summary

Securing MySQL Server

by Keith Murphy When looking to secure a MySQL server there are several important areas to consider. We will take time to examine each of these areas in some detail. This will give you strong foundational knowledge when looking at securing a MySQL server.

A very common discussion point for MySQL security is the user accounts of the database. It is certainly foundational to understanding the security of your server. MySQL user security is based on an ACL (Access Control List) model. This is used for everything a user does including query execution, connection capability and server administration capabilities. All of this information is stored in the user table of the mysql database. Permissions are generally managed with the GRANT and REVOKE ocmmands. User management can be one of the more arcane areas of being a DBA. Even so, it is simply to important to neglect. Each user should have the minimal set of permissions to perform the tasks needed. Not every user needs to ability to drop tables/databases!! Dont be lazy in this area as it will bite you eventually. By default the root user does not have a password when you set up a MySQL server. Change this immediately by executing the msyqladmin -u root -p new-password comand.

User accounts

mysql> DELETE FROM user WHERE User = ''; mysql> DELETE FROM db WHERE Host = '%'; Connection methods

Executing the following two commands from the MySQL monitor will achieve the desired results:

MySQL ships with two default users and a default 'test' table. The default users are for connecting to the DBMS without specifying a password, so removing these users is obviously a very good security measure. There are also entries so that tables called or starting with 'test' can be world-writable. You will want to disable these accounts and remove the test database.

It is quite common to connect to a MySQL server using the command line mysql client. If you connect to a server from your desktop using this client all your information is being sent across the network without very strong encryption. This is something you want to avoid doing. It is a best practice to connect the server that MySQL is running on using a tool such as ssh that provides an encrypted communication channel and then use the client tools on the server to work with MySQL.

If you are planning a network for a typical website that is publicly accessible there is one security concern to keep in mind. It is always best to have your web server and your database server running on separate machines. If they are not running on the same machine you will want to have the MySQL server (which does not need to be accessed by the public) on its own private network. Take a look at the following diagram: As you can see we have a small network that is publicly accessable through 72.9.253.1. Each web server will be accessible (through the load balancer). The MySQL server, however, is not accessible. The 192.168.1.x network is not available over the Internet. You would have to be on the internal network to reach the server. This makes it much more difficult for an intruder to reach and attempt to compromise the MySQL server. Of course the MySQL server can communicate freely with the web servers over the internal network.

Network design considerations

When operating overTCP/IP MySQL, by default, uses port 3306. You should have this port blocked on your network firewall(s) to disable access from outside.

Remove any unneccessary users from the operating system. Like the extra packages, unneeded users provide an avenue of access for an attacker. Use strong passwords for your system users. Your dogs name isnot a good password. It should be a mix of letters (capitalized and uncapitalized) and other symbols. Minimally it should be eight characters long, but should probably be more in the ten to twelve character range. In addition, you should consider the physical security of a system. Any system can be compromised if left unattended. Servers should be in dedicated areas behind locked doors minimally. Data center space with locked cabinets is always a good idea if possible.

Whatever operating system you choose to run MySQL on there are a several concerns in preparing the operating system for use. You should always install a minimal set of packages necessary to run your operating system. No need for the everything but the kitchen sink mentality. The reason why is that every additional packages/applications installed gives an attacker another avenue of possible access. As a side benefit, you will get better performance out of your MySQL server.

Operating system considerations

Do not run the MySQL daemon as the Unix root user. It is a very dangerous practice as any user with FILE privileges will be able to create files as root (for example, ~root/.bashrc). To prevent this, mysqld will refuse to run as root unless it is specified directly via --user=root option. The mysqld daemon can be run as any user instead. You can also create a new user mysql to make things more secure. If you run mysqld as another Unix user, you don't need to change the root user name in the user table, because MySQL user names have nothing to do with Unix user names. You can edit the mysql.server script to start mysqld as another Unix user. Normally this is done with the su command. If you put a password for the Unix root user in the mysql.server script, make sure this script is readable only by root. Check that the Unix user that mysqld runs as is the only user with read/write privileges in the database directories. Mysqld file options

User account running the MySQL daemon

The skip-networking options disables TCP/IP network connections. This is a good security feature if you are running MySQL on the same server as the web server (in a typical Internet serving connection). If the MySQL server is located on another server (as it must use TCP/IP to communicate with the other server).

It is important to understand the settings in your mysqld options file. This file is typically named my.conf. There is one settings that it is critical that you examine - the skip-networking options.

Resources

http://dev.mysql.com/doc/refman/5.0/en/security.html About the author Keith Murphy is the editor of MySQL Magazine. He is also the chief Dolphin at Broadwick Corporation.

Optimizing MySQL with Rails - Part One

By Dan Buettner drbuettner@gmail.com Rails philosophy of convention over configuration and plethora of built in helpers makes it almost scary-easy to prototype a database-driven web application, and the transparency with which your applicaton interacts with the database makes it easy to forget about the often-complex beast known as your RDBMS lurking behind the scenes. As with the applications Ruby code, it is easy to create a basic database, and easy to change the database structure as your application evolves - but eventually most web applications need to grow up and be deployed somewhere, and when they do, the database needs to be up to the challenge. MySQL is one of the most widely-used databases for Rails development and deployment. I will present several different methods you can use to improve performance - some fairly general, some MySQL-specific, and some Rails-specific. Index creation and caching are probably the two biggest items in terms of potential payoff, but all the methods can be employed in various combinations for maximum benefit.

Object-oriented languages can present special challenges to relational databases because of the sheer number of queries executed in some cases. This isnt a fault, really, but a result of clean, readable, simple code. With proper testing, and a bit of tweaking, your database can be made to keep up with your application. When you create tables with migrations, the only default index is on the primary key - and while that index is often useful in joins, it is unusual for that to be the only index you need on an important table. It may be that you know right away which fields youll be querying on and can create useful indices right away, or it could be that you only gather a good understanding of the nature of the data late in the project, but at some point prior to launch youll want to spend some time on indices. Creating effective indices

Multi-column indices can be a great way to hone performance in large datasets, but they do require some care when creating them to be most useful. A multi-column index is used from left to right, with no gaps allowed that is to say, if you index several columns like so:
ALTER TABLE t1 CREATE_INDEX multi_col_idx (c1, c2, c3)

It is easy to go overboard creating indices: MySQL can use only one index per instance of a table per query, and extra indices take up disk space and slow down write operations. That said, creating an index is usually a highly effective step in increasing performance, and generally requires very little effort to experiment with.

As an example you might have a table with directory-type information about a companys employees, and users can search by specifying a last name and, optionally, a first name.

This index will be usable to speed up queries that specify c1, c1 and c2, or c1 and c2 and c3 in the WHERE clause; however, it will not be usable to speed up queries against only c2, or only c3, or c2 and c3. A query against c1 and c3 would be assisted with c1 information, but not able to also use the index on the c3 portion.

Something like this code:

In this case, a multi-column index across last_name and then first_name would be more beneficial than separate indices on each column. Specifying last_name as the first column in the index makes sense as (in this somewhat simplistic example) we are requiring that last_name always be present. Also be aware of MySQLs limitation with LIKE string searches and indices, that the search string must match from the beginning of the column. This can use an index:
SELECT * FROM employees WHERE last_name LIKE Smith% ALTER TABLE employees CREATE_INDEX both_names_idx (last_name, first_name)

if params[:first_name].blank? Employee.find :all, :conditions => [last_name = ?, params[:last_name]] else Employee.find :all, :conditions => [last_name = ? AND first_name = ?, params[:last_name], params[:first_name]] end

while this cannot, due to the wildcard in front of Smith: Join tables, which link two other tables in has_and_belongs_to_many relationships, are excellent candidates for indexing, and a case where indexing both columns can help significantly. Examine your typical queries to determine which column to put first. Use the Rails log file in a development environment to evaluate the queries executed and the database time for each page. Extract the queries and use MySQLs EXPLAIN feature to see how indices are being used. I often run a tail -f command on log/development.log in my application, examining the output as I load each page in my Rails application and copy-pasting the queries into a MySQL session for evaluation. Example output shows both the complete query and the execution time:
Webpage Load (0.041033) SELECT * FROM content_items WHERE (content_items.`url_text` = 'home') AND ( (content_items.`type` = 'Webpage' ) ) LIMIT 1 SELECT * FROM employees WHERE last_name LIKE %Smith%

Use MySQLs slow query log and the Rails log

Parsing the output of the EXPLAIN command can be daunting for complex queries, but with a little reading and perseverance, and if you start with simple queries, you can get the hang of it pretty quickly. It is especially easy to notice the lack of index usage:

mysql> EXPLAIN SELECT * FROM content_items WHERE (content_items.`url_text` = 'home') AND ( (content_items.`type` = 'Webpage' ) ) LIMIT 1 \G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: content_items type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 261 Extra: Using where 1 row in set (0.00 sec)

Knowing my application and my data, I know that I will always select on a specific value for the url_text column, and nearly always specify a value for type. If we add an index like so, it should help a lot specifying the column Ill always use first, and the column I may not always use second:
ALTER TABLE content_items ADD INDEX multi_col_idx (url_text, type)

We see that possible_keys, which should be a list of any indices on columns used in the query, is NULL, meaning there is no way were going to get an index hit. In fact, the type of ALL indicates a table scan, and rows shows an estimate of the number of rows MySQL thinks it will have to examine. 261 is small in this single table query, but in complex joins the effect on performance can be exponential when multiple tables do not have index hits.

*************************** 1. row *************************** id: 1 select_type: SIMPLE table: content_items type: ref possible_keys: multi_col_idx key: multi_col_idx key_len: 516 ref: const,const rows: 1 Extra: Using where 1 row in set (0.00 sec)

Now my EXPLAIN output looks a lot better:

MySQLs slow query log can be beneficial in identifying queries that are truly slow, taking one second or more to execute. You can specify the number of seconds in the MySQL server configuration file, but the minimum is one second, which sometimes doesnt give you enough granularity to find subtle slowdowns. A query that takes, say, 90 seconds, but is only executed occasionally can cost you a lot less than a query that takes a quarter or even a tenth of a second but is executed on every page load. While it might be nice to speed up that 90 second query, if its only executed for a report that one or two people use once a week, your time is better spent on that other query. Thats where your Rails development log file can be quite helpful. Next time

You can see that the number of rows MySQL thinks it will have to examine has dropped to just one.

In part two of this article we will discuss sample data and development setups, the MySQL and Rails query caches, a couple of common pitfalls when designing database tables and queries, and other items relating to Rails performance on MySQL! About the author

Dan Buettner works for Outsell, Inc., a San Francisco area market research company tracking the information industry, though he lives in Des Moines, Iowa. In a past life he worked at The Des Moines Register, a midsize newspaper, as an IT manager, newspaper production specialist, software developer, and Sybase and MySQL DBA. He has been using MySQL since version 3.x in 1999, and Rails since Summer 2006. This article was finished while attending RailsConf 2007.

Convenient MySQL Privilege Scripting and Versioning

by Baron Schwartz MySQL provides several levels of privilege granularity, and if you're like me, you may find privileges getting unnecessarily complex quickly. Redundant privileges, or users with similar but not identical privileges, can be a headache to manage, and if you need to change or clean them up at the command line, it can be very tedious to do. I also believe privileges are code and should be automatically saved in a version control system, for all the same reasons I version control other kinds of code. Both tasks are harder than they might seem at first glance, but they can be made easier with a little scripting. What's hard about managing privileges? table are listed as SELECT, INSERT, DELETE, but Anne's are listed as INSERT, SELECT, DELETE. This makes it hard to compare the grants.

Your next thought is to pipe the grants into files and use command-line utilities, such as diff, to compare them. This is a great idea, but here's where you start to notice the little things that make this job harder. The grants are not ordered the same. You can pipe them through sort, but you still don't get them in the same order, because not only are the individual GRANT statements not ordered the same in the output, but the actual grants are sorted differently within the lines! For example, Bob's grants on a given

The most straightforward way is to connect to each server and type SHOW GRANTS FOR <username>, then visually scan the lists and compare. This works okay for a few privileges, but not hundreds.

Though it seems straightforward at first, it can actually be quite tedious to manage user privileges, especially from the command line. For example, let's say you have users Bob and Anne, with nearly the same privileges, and your task is to remove Bob's extra privileges so he has the same rights as Anne. To make it more interesting, imagine the users are not on the same server, and each has about a hundred privileges. How do you do this?

This same difficulty makes it hard to place the grants into version control. In my experience, the grants are frequently printed out in a different order every time you check, even when they haven't changed. This leads to spurious changesets in your version control system, and again makes it hard to compare and see what is truly changing. MySQL Show Grants to the rescue I wrote a tool, unimaginatively called MySQL Show Grants, to help me address these very problems. Superficially similar to "mysql -e 'show grants'", it has some important differences that make it easy to script solutions to the above problems: 1. Grants are sorted within each line, so they always appear in the same order. 2. The lines are sorted.

Imagine that you overcome this somehow, and now you see only five differences between the two. Your remaining task is to type REVOKE statements for Bob's account. This also gets old after you do it a few times. Imagine how hard this process becomes when you have many users on many servers!

These properties are enough to solve the problem of spurious changesets in version control. All you need to do is run a nightly cron job to dump the grants (by default it dumps grants for all users), then check it into your source control system. As for cleaning up user privileges, I also added features to help you diff users conveniently, adding or revoking with ease:

These features make it easy to revoke Bob's excess privileges and make them the same as Anne's. For example,

2. You can convert GRANT statements to REVOKE statements.

1. You can split each GRANT command onto its own line. GRANT SELECT, INSERT becomes GRANT SELECT; GRANT INSERT. This makes it much easier to see which exact priveleges are different.

These commands separate the GRANT statements and convert them into REVOKE statements, then put the results into files. You can easily run diff on these two files now and see what has changed; the files contain valid REVOKE commands, so you can even execute the difference by piping it into the mysql command-line client.

mysql-show-grants --separate --revoke --only bob -host=server1 \ | grep REVOKE > bob-privileges mysql-show-grants --separate --revoke --only anne -host=server2 \ | grep REVOKE | sed -e 's/anne/bob/g' > anne-privileges

Though I have solved my own needs with this tool, I also want it to be useful to others. If you need additional features, feel free to request them via the SourceForge bug tracker, forums, or mailing list. Baron Schwartz works for the Rimm-Kaufman Group, a search marketing agency in Charlottesville, Virginia. He writes frequently about MySQL on his blog (http://www.xaprb.com). He is the author of the innotop MySQL and InnoDB monitor, and MySQL Toolkit, a collection of command-line tools. About the author

MySQL Show Grants is licensed under the GPL, hosted on SourceForge, and can be downloaded from http://sourceforge.net/projects/mysqltoolkit The system requirements are very moderate: Perl and DBD, which might already be on your system.

Operating System Hardening

by Keith Murphy Some might question why there is an article about the operating system in a magazine about MySQL. When considering the overall security of a database it is important to take into account the underlying operating system of the database. You can have a very secure MySQL server compromised through the underlying operating system.

MySQL runs on a variety of platforms. There is not enough space to cover every platform. First I will discuss several principles that are going to be applicable to any operating system. Then I will cover in detail the setup of the operating system. Linux is one of the most popular platforms for MySQL and CentOS is one of the most common Linux distribution.

There are two security principles that apply to server hardening - the principle of separation and the principle of defense in depth. The principle of separation is the idea that a server should perform a minimal set of task. In another words, the server should NOT operating email, web and database services. Why is this? Because less software loaded and running on the server provides fewer avenues for attack. The principle of defense in depth means that only one method of protection should not be used to protect your server/network. For example, many people mistakenly think that a firewall will protect your network completely. In addition to a firewall you should have good password, intrusion detection systems, etc.

Do not underestimate the importance of these two principles. When performing the following sample installation of CentOS I will demonstrate these two principles. The operating system will not have any unneeded services running and there will be multiple forms of protection. Consider this when planning your own deployments. Hardening a CentOS 4.X base system

We will be hardening a basic install of CentOS 4.X. In addition to the basic operating system install, we are installing several additional programs to provide layers of security: tcp wrappers libsafe chkroot tripwire

In preparation for the install we will be using the following partitioning scheme: Partition /boot / swap /usr /var /tmp /home 100MB 6 gb 2x memory 10 gb 10 gb 1 gb remaining space Size Notes

We will make sure all unnecessary services are deactivated, and all file systems permissions are properly set. Physical security of the system should be taken into account as well.

Contains all startup files and the kernel top level partion (root partition) contains paging file for memory management contains programs, libraries and documentation system log files temporary files user files

 Protection against SUID programs Easy backup & upgrade management Ability for better control of mounted file system Limit each file systems ability to grow

Why are we partitioning like this?

Install or upgrade CentOS in graphical mode: <Enter> Install or upgrade CentOS in text mode: linux text <Enter> Use function keys for more information Hit Enter to begin the installation in graphical mode. Before the installation begins, you will be prompted to test the CD media. If you have not already done this, select OK. This should be done at least once for all of the installation CDs. When the media test has completed, re-insert CD #1 and select Continue. If you have already done this, select Skip.

The boot screen will display three options:

Insert the CentOS server cd into the drive.

You will probably want to make some modifiations to this layout. If you are setting up a MySQL server you might want to set up specific partitions for your database logs and tablespace.

The following will walk through our Operation System installation. An explanation of each step is provided on the left side of the screen. Welcome Language Keyboard Mouse Configuration Disc Partitioning Setup Boot Loader Configuration Network Configuration Screen Select Next Select English (default) Select Next Action

Select US English (default) Select Next

Select Manually partition with Disk Druid Select Next Layout the partitioning as described earlier

Select Generic - 3 Button Mouse (PS/2) Select Next

Leave default selection of GRUB boot loader. Select Use a boot loader password, and enter a password. Select Next Configure with your network settings

Screen Firewall Additional Language Support Time Zone Selection

Action Select Enable firewall allow only ssh through the firewall Select Next

Default language for the system: English (USA) Additional languages to install: English (USA) only Select System clock uses UTC Select your timezone (i.e., America/Denver) On the UTC Offset tab Select offset (i.e, UTC-07 US Mountain) Select Use daylight saving time Select Next Enter a root password

Set Root Password Package Defaults Package Group Selection Preparing to install Installing Packages

Select Customize set of packages to be installed. Select only the following package: Minimal

Select Next will begin the installation, this is the last opportunity to cancel the install. Insert disks as prompted When the CD is ejected, remove the media and select Exit to reboot the system. A complete log of the installation can be found in the /root/install.log

Installation Complete Post Installation

Once the installation is complete save the anaconda-ks.cfg file somewhere so it wont be overwritten or deleted. This file is located in roots home directory (/root) and is a complete kickstart configuration file that can be used to build a duplicate system. Also generate a listing of all RPMs installed on the systems using the following command. rpm -qa | sort > /root/rpm-list This system will be using yum on a internal server to get updates. All updates on this server have been reviewed and tested before being made available to machines on the network. With this in mind, we will be using yum to update the machine, which requires it to be on our network. We have provided a secure subnet on which to do this, however, weve also taken the extra precaution during installation by installing a firewall that is currently only allowing ssh traffic, and the required DNS replies and DHCP requests for network connectivity. This update is initially performed by during installation from the update server. Patching the System

Securing services

Now we will remove some services. There are several ways to stop services, I will talk about two of them. running the chkconfig command running the ntsysv command chkconfig --list

First, the chkconfig command. To see what services are currently running on your system: This will show you a complete list of all processes and what run level they are starting at. To make it a little easier to read you might want to run chkconfig -list | grep :on and this will only show you the processes that are starting up. network 0:off 1:off 2:on 3:on 4:on 5:on 6:off gpm 0:off 1:off 2:on 3:on 4:on 5:on 6:off haldaemon 0:off 1:off 2:off 3:on 4:on 5:on 6:off xfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off acpid 0:off 1:off 2:off 3:on 4:on 5:on 6:off rpcsvcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off xinetd 0:off 1:off 2:off 3:on 4:on 5:on 6:off readahead_early0:off 1:off 2:off 3:off 4:off 5:on 6:off netfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off messagebus 0:off 1:off 2:off 3:on 4:on 5:on 6:off mdmonitor 0:off 1:off 2:on 3:on 4:on 5:on 6:off rhnsd 0:off 1:off 2:off 3:on 4:on 5:on 6:off crond 0:off 1:off 2:on 3:on 4:on 5:on 6:off atd 0:off 1:off 2:off 3:on 4:on 5:on 6:off apmd 0:off 1:off 2:on 3:on 4:on 5:on 6:off smartd 0:off 1:off 2:on 3:on 4:on 5:on 6:off autofs 0:off 1:off 2:off 3:on 4:on 5:on 6:off isdn 0:off 1:off 2:on 3:on 4:on 5:on 6:off readahead 0:off 1:off 2:off 3:off 4:off 5:on 6:off rpcidmapd 0:off 1:off 2:off 3:on 4:on 5:on 6:off anacron 0:off 1:off 2:on 3:on 4:on 5:on 6:off irqbalance 0:off 1:off 2:off 3:on 4:on 5:on 6:off pcmcia 0:off 1:off 2:on 3:on 4:on 5:on 6:off nfslock 0:off 1:off 2:off 3:on 4:on 5:on 6:off sendmail 0:off 1:off 2:on 3:on 4:on 5:on 6:off sshd 0:off 1:off 2:on 3:on 4:on 5:on 6:off rpcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off syslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off cups 0:off 1:off 2:on 3:on 4:on 5:on 6:off kudzu 0:off 1:off 2:off 3:on 4:on 5:on 6:off portmap 0:off 1:off 2:off 3:on 4:on 5:on 6:off iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off cpuspeed 0:off 1:on 2:on 3:on 4:on 5:on 6:off rawdevices 0:off 1:off 2:off 3:on 4:on 5:on 6:off Now, with my default system I have this output from the chkconfig --list command:

Now we will stop some (unneeded) services:

[root@rivendale /]# for svc in gpm kudzu netfs atd apmd isdn \ > pcmcia autofs portmap nfslock mdmonitor mdmpd cups ; do > /sbin/chkconfig -level 2345 $svc off > /sbin/service $svc stop > done

This is fairly straightforward, each of the services listed gets removed from the startup list, and also stopped when you run the script. network 0:off 1:off 2:on 3:on 4:on 5:on 6:off haldaemon 0:off 1:off 2:off 3:on 4:on 5:on 6:off acpid 0:off 1:off 2:off 3:on 4:on 5:on 6:off rpcsvcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off xinetd 0:off 1:off 2:off 3:on 4:on 5:on 6:off readahead_early 0:off 1:off 2:off 3:off 4:off 5:on 6:off messagebus 0:off 1:off 2:off 3:on 4:on 5:on 6:off rhnsd 0:off 1:off 2:off 3:on 4:on 5:on 6:off crond 0:off 1:off 2:on 3:on 4:on 5:on 6:off smartd 0:off 1:off 2:on 3:on 4:on 5:on 6:off readahead 0:off 1:off 2:off 3:off 4:off 5:on 6:off rpcidmapd 0:off 1:off 2:off 3:on 4:on 5:on 6:off anacron 0:off 1:off 2:on 3:on 4:on 5:on 6:off irqbalance 0:off 1:off 2:off 3:on 4:on 5:on 6:off sendmail 0:off 1:off 2:on 3:on 4:on 5:on 6:off sshd 0:off 1:off 2:on 3:on 4:on 5:on 6:off rpcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off syslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off cpuspeed 0:off 1:on 2:on 3:on 4:on 5:on 6:off rawdevices 0:off 1:off 2:off 3:on 4:on 5:on 6:off Network security Here is the output of the chkconfig -list command:

In this particular setup, I went from 34 services running to 21 services running. This is much better.

# Kernel sysctl configuration file for Red Hat Linux # # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and # sysctl.conf(5) for more details. # Controls IP packet forwarding # Disable IP Forwarding net.ipv4.ip_forward = 0 # Disables IP Source Routing net.ipv4.conf.all.accept_source_route = 0

We will be modifying the /etc/sysctl.conf file. The final version will look like this:

# Enables SYN flood protection net.ipv4.tcp_max_syn_backlog = 4096 # Enables TCP SYNC flood protection net.ipv4.tcp_syncookies = 1 # Disables the ability to send ICMP Redirect net.ipv4.conf.all.send_redirects = 0 # Disables ICMP Redirect acceptance net.ipv4.conf.all.accept_redirect = 0 # Another parameter that disables ICMP Redirect acceptance net.ipv4.conf.default.accept_redirects = 0 # Controls source route verification # Enable IP Spoofing protection net.ipv4.conf.default.rp_filter = 1 # Controls the System Request debugging functionality of the kernel kernel.sysrq = 0 # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 After changing the file, set the file permisions: # chown root:root /etc/sysctl.conf # chmod 0600 /etc/sysctl.conf

Now, you need to restart the network service: It is always a good idea to have a log-in banner when someone accesses the system. Legally, it is a good idea, plus it also helps to disguise what particular services your system is running. For ssh, add the following line to the /etc/hosts.allow file sshd : ALL : banners /etc/issue # /etc/rc.d/init.d/network restart

create the content of the sshd file:

# mkdir /etc/banners # touch /etc/banners/sshd

then do the following:

220-Hello, %c 220-All activity on rivendale.paragon-cs.com is logged. 220-Thank you

The %c variable shows client information such as the username/hostname.

[root@rivendale /]# cat /etc/securetty tty1 tty2 tty3 tty4 tty5 tty6

You really dont need to be logging in across the network with the root account. To fix this, edit the /etc/securetty file to disable network root login. Here is what it should look like:

Secure Console and Root Account

The root account can log in from the local console and the ssh daemon. To disable ssh root login you need to add PermitRootLogin no to the /etc/ssh/sshd_config file, and then restart sshd. Linux provides a "Single User Mode" to perform system maintenance. By default, Linux does not require the root password to log into single user mode. To require the password, add "~~:S:wait:/sbin/sulogin" to /etc/inittab. Then run /sbin/init q to activate the change. If you type ctrl-alt-del simultaneously, a Linux system will typically reboot. This is easy to fix, and should be done. There are two ways of doing this: ca::ctrlaltdel:/sbin/shutdown -t3 -r now or Edit the /etc/inittab again and comment (#) out the following line: Restricting usage of ctrl-alt-del Restricting root login from the local console

add a -a option to the line. The -a flag tells shutdown to look for the /etc/shutdown.allow file. You can restrict any users who are allowed to shutdown the system using ctrl-alt-del. In /etc/inittab: ca::ctrlaltdel:/sbin/shutdown -a -t3 -r now In /etc/shutdown.allow bmurphy root Disabling console program access

If you want to disallow any user at the console to run poweroff, halt, and reboot run the following commands: [root@rivendale /]# rm -f /etc/security/console.apps/poweroff [root@rivendale /]# rm -f /etc/security/console.apps/halt [root@rivendale /]# rm -f /etc/security/console.apps/reboot

Removing unnecessary system accounts

Some user accounts and groups are not needed by the server. Its best to keep the /etc/password file as small as possible. This will make it easier to see unauthorized additions easier. Some common users you dont need: And the groups: gopher, adm, pcap, rpc, rpcuser, nfsnobody, games, news, uucp apache, xfs, named, ntp

[root@rivendale /]# for user in gopher pcap rpc rpcuser nfsnobody \ > games news uucp apache xfs named ntp adm ; \ > do /usr/sbin/userdel $user ; done [root@rivendale /]# for group in news uucp adm games dip; \ > do /usr/sbin/groupdel $group ; done [root@rivendale /]# pwck [root@rivendale /]# grpck For protection against Trojan binaries being installed in the /usr partition, /usr can be mounted as readonly. In the /etc/fstab file: /usr ext3 ro 1 2 If you need to allow write access (to install a program): [root@rivendale /]# mount -o remount,rw /usr [root@rivendale /]# mount -o remount,ro /usr Additional Software Installation Then after installation, change it back to read only: Mount /usr file system as read-only

news, uucp, adm, games, dip

As we mentioned earlier there are several packages we want to install to perform additional hardening of the system: tcp wrappers, libsafe, chkroot, and tripwire. We will configure TCP Wrapper to control the authentication of the ssh daemon. Two files will be used (/etc/host.allow and /etc/hosts.deny). The hosts.deny file will by default deny everyone, and the hosts.allow file will only allow those to whom we wish to grant access. [root@rivendale /]# rpm -qa | grep wrapper tcp_wrappers-7.6-38 [root@rivendale /]# Make sure you have TCP Wrapper TCP Wrappers

[root@rivendale /]# vi /etc/hosts.deny [root@rivendale /]# vi /etc/hosts.allow [root@rivendale /]# cat /etc/hosts.allow ssh: 192.168.1.101: allow [root@rivendale /]# cat /etc/hosts.deny ALL:ALL: Libsafe

Create the hosts.allow and hosts.deny files (if you dont already have them)

Libsafe is a interesting program that helps protect your system from what is called from stack attacks typically called buffer overflows. A buffer overflow occurs when a memory stack is filled to capacity when a piece of code is executed in the stack. This causes the potential side efect of a root shell. Buffer overflows occur from either poor programming practices or weaknesses in the programming language itself. Libsafe is a way for the operating system to deny buffer overflow attacks. Libsafe monitors system calls and if an attack is detected then the entire process group will be sent a SIGKILL signal. The attempt is also logged in /var/log/secure. Libsafe is available from http://www.research.avayalabs.com/project/libsafe. The chkrootkit script is a tool that checks for various trojans on your system. The homepage is http://www.chkrootkit.org. Chkrootkit is available as a tarball..but since it is just a shell script it is not difficult to install. Just download it and unzip it somewhere (/usr/local maybe?). Have it executed on a regular basis using the crontab utility. Chkrootkit

Tripwire is a very good program that builds a database of the size of your system files and then you periodically run tripwire to compare your current system files against the database. It is important to create a baseline of your system files as soon as you do the installation and preferably before you are plugged into the public Internet. The "official" homepage of tripwire is http://www.tripwiresecurity.com. This page is about the enterprise version of tripwire however. Google tripwire if you need additional information. What have we learned from this exercise? We have reduced the number of services running on a server with a minimal install of packages. In addition we have added several very useful programs to help keep tabs on your systems security. You have applied both the principles of seperation and defense in depth. Do not fall into the trap of thinking that just because you hardened your server to begin with that you are safe. An important aspect of security is staying vigilant and monitoring your system(s). About the author Keith Murphy is the editor of MySQL Magazine. He is also the chief Dolphin at Broadwick Corporation. Conclusion

Tripwire

Previously I had worked as a DBA on an Oracle system running on HP-UX. Contrast this with running MySQL on commodity PC hardware and as a DBA you can see the possibilities of MySQL even at the fairly immature growth stage that was version three. Sure, many of the features of Oracle were not in place. Even so it was solid, very quick and absolutely free.

I have been involved in Unix in general and Linux specifically for quite a while. The first time I booted up a Sun workstation was in 1992 or 1993. Back then Sun used the SunOS which was BSD-Unix based. I moved into using Linux in 1997 with some version of Slackware. When I started an ISP in 1998 I began using RedHat and MySQL.

So what is this all about? I decided to start MySQL Magazine because I see a need for the magazine. The worlds number one open-source database has over 10 million installations. While there are stacks of books available, and numerous resources available online there are no magazines available about MySQL. Sure there is the occassional article about MySQL in the various Linux magazines or Sys Admin Magazine but I want something that is only about my passion - MySQL.

log-bin

Thanks,

I am looking forward to this project. It is my way of giving back to the FOSS community. Also, I want to hear back from the community about your thoughts too!

Over the years I have watched MySQL grow and mature. More and more big companies have started utilizing it. Now with the birth of Web 2.0, there is an absolute need for full featured databases that can scale with the growth of the website and deliver billions of queries a month. Livejournal, YouTube, Google, Kaneva and Flikr are a few of the large sites powered on the backend by MySQL. Why is this? The maturity of MySQL has allowed these companies to scale very quickly. The costeffectiveness is obviously a factor but it really boils down to the fact that MySQL enables these companies to provide a better experience to the end user. And with the maturing of the clustering capabilities in version 5.x there is a whole new level we can take MySQL. While there are commericial RDMS that provide for clustering the costs are astronomical. I would suggest that if you are a MySQL DBA, or you are aspiring to be one, you invest in the hardware needed to run Xen well - any 64-bit processor with hardware virtualization support and a few gigs of RAM. Then load it up with CentOS 5 and create some virtual servers running version 5.1.x of MySQL. Work with clustering. Experiment, play, above all have fun!

Keith Murphy (bmurphy@paragon-cs.com)