KarthikAppigatla 2018 MySQL8CookbookOver150RecipesforHigh-Perf

2018. Packt Publishing. All rights reserved.
May not be reproduced in any form without permission from the publisher, except fair uses permitted under U.S. or applicable copyright law.
Copyright
EBSCO Publishing : eBook Academic Collection (EBSCOhost) - printed on 11/9/2022 10:11 AM via
AN: 1699220 ; Karthik Appigatla.; MySQL 8 Cookbook : Over 150 Recipes for High-performance Database Querying and Administration
Account: s3824264
MySQL 8 Cookbook
0WFSSFDJQFTGPSIJHIQFSGPSNBODFEBUBCBTFRVFSZJOH
BOEBENJOJTUSBUJPO
Karthik Appigatla
BIRMINGHAM - MUMBAI
EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

MySQL 8 Cookbook
Copyright a 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, without the prior written permission of the publisher, except in the case of brief quotations
embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented.
However, the information contained in this book is sold without warranty, either express or implied. Neither the
author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to
have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products
mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy
of this information.
Commissioning Editor: Amey Varangaokar

Acquisition Editor: Viraj Madhav
Content Development Editors: Aishwarya Pandere
Technical Editor: Prasad Ramesh
Copy Editor: Vikrant Phadkay
Project Coordinator: Nidhi Joshi
Proofreader: Safis Editing
Indexer: Pratik Shirodkar
Graphics: Tania Dutta
Production Coordinator: Shraddha Falebhai
First published: January 2018

Production reference: 1230118
Published by Packt Publishing Ltd.

Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78839-580-9
XXXQBDLUQVCDPN

To my mother, A.S Gayathri, and to the memory of my father, A.S.R.V.S.N Murty, for their
sacrifices and for exemplifying the power of determination.
` Karthik Appigatla

NBQUJP
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as
well as industry leading tools to help you plan your personal development and advance
your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos
from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at XXX1BDLU1VCDPN and as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at TFSWJDF!QBDLUQVCDPN for more details.
At XXX1BDLU1VCDPN, you can also read a collection of free technical articles, sign up for a
range of free newsletters, and receive exclusive discounts and offers on Packt books and
eBooks.

Contributors
About the author

Karthik Appigatla is a highly reputed database architect and is very famous for
performance tuning. He has been consulted by many companies all over the world for
designing, performance tuning, building database infrastructure, and training. In his
decade of experience, he has worked for companies such as Yahoo, Pythian, and Percona.
Currently, he is working for LinkedIn, where he has innovated a new way of analyzing
queries. He gave a talk about this at SRECon, Dublin in 2017.
I would like to acknowledge the encouragement from my wife, Lalitha, and my brother,
Kashyap. This book would not have completed without the cooperation of my little
daughter, Samhita.

About the reviewers
Marco Ippolito is an Italian software engineer working as Director of Software
Development for Imagining IT. Marco completed his postgraduate in Software Engineering
in Oxford and has worked for large corporations such as Intel, HP, Google, Dell, and Oracle
(in the acquired MySQL team), as well as for start-ups such as !QMBUGPSNTI and for large
production users of MySQL such as !CPPLJOHDPN (Twitter). He can be reached
at NBSDPJQQPMJUP!JNBHJOJOHJUDPN and has experience working in teams speaking
Italian, English, Spanish, Brazilian Portuguese, German, and French, remotely or onsite.
Kedar Mohaniraj Vaijanapurkar is a MySQL database consultant with over a decade of

experience, ranging from programming to database administration. He aims to spread
happiness with the MySQL database systems he works on. Apart from working with
MySQL and related open source technologies, he also explores cloud, automation, and
NoSQL. He is a 5-year Pythian laureate living with his awesome family in the cultured city
of Vadodara, India. You can reach him at LFEBS!OJUUZXJUUZDPN.
Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit BVUIPSTQBDLUQVCDPN and
apply today. We have worked with thousands of developers and tech professionals, just
like you, to help them share their insight with the global tech community. You can make a
general application, apply for a specific hot topic that we are recruiting an author for, or
submit your own idea.

Table of Contents
Preface 1
Chapter 1: MySQL 8 - Installing and Upgrading 6
Introduction 6
Installing MySQL using YUM/APT 7
How to do it... 8
Using YUM repositories 8
Using APT repositories 12
Installing MySQL 8.0 using RPM or DEB files 14
How to do it... 14
Using the RPM bundle 15
Using the APT bundle 16
Installing MySQL on Linux using Generic Binaries 17
How to do it... 17
There's more... 21
Starting or Stopping MySQL 8 21
How to do it... 21
Starting the MySQL 8.0 server 21
Stopping the MySQL 8.0 server 23
Checking the status of the MySQL 8.0 server 24
Uninstalling MySQL 8 25
How to do it... 25
On YUM-based systems 25
On APT-based systems 26
Uninstalling Binaries 27
Managing the MySQL Server with systemd 28
How to do it... 28
Downgrading from MySQL 8.0 29
How to do it... 29
In-place Downgrades 30
Using YUM repositories 30
Using APT Repositories 31
Using the RPM or APT bundle 31
Using Generic Binaries 32
Logical Downgrades 32
Using YUM Repositories 33
Using APT Repositories 34
Using RPM or APT bundles 34

Table of Contents

Upgrading to MySQL 8.0 35
Getting ready 35
How to do it... 36
In-place upgrades 36
YUM-based systems 37
APT-based systems 38
Using RPM or APT bundles 38
Logical Upgrades 40
Installing MySQL utilities 40
How to do it... 41
On YUM-based systems 41
On APT-based systems 41
Chapter 2: Using MySQL 42
Introduction 43
Connecting to MySQL using the command-line client 43
Getting ready 43
How to do it... 44
See also 45
Creating databases 45
How to do it... 46
See also 48
Creating tables 49
How to do it... 49
Cloning table structure 53
See also 53
Inserting, updating, and deleting rows 53
How to do it... 53
Inserting 54
Updating 55
Deleting 55
REPLACE, INSERT, ON DUPLICATE KEY UPDATE 55
Truncating tables 56
Loading sample data 57
How to do it... 57
Selecting data 59
How to do it... 60
Selecting columns 60
Count 61
Filter based on condition 61
Operators 62
[ ii ]

Table of Contents
Simple pattern matching 63

Regular expressions 64
Limiting results 66
Using the table alias 66
Sorting results 67
How to do it... 67
Grouping results (aggregate functions) 68
How to do it... 68
COUNT 68
SUM 69
AVERAGE 70
DISTINCT 70
Filtering using HAVING 70
See also 71
Creating users 71
How to do it... 71
See also 73
Granting and revoking access to users 73
How to do it... 73
Granting privileges 73
Checking grants 75
Revoking grants 76
Modifying the mysql.user table 76
Setting password expiry for users 78
Locking users 79
Creating roles for users 80
Selecting data into a file and table 81
How to do it... 82
Saving as a file 82
Saving as a table 83
Loading data into a table 84
How to do it... 84
Joining tables 85
How to do it... 87
Identifying Duplicates using SELF JOIN 88
Using SUB queries 89
Finding mismatched rows between tables 91
Stored procedures 92
How to do it... 93
There's more... 96
See also 96
Functions 96
How to do it... 97
[ iii ]

Table of Contents
Inbuilt functions 98
See also 99
Triggers 99
How to do it... 100
See also 102
Views 102
How to do it... 102
Events 104
How to do it... 104
Access control 105
See also 105
Getting information about databases and tables 106
How to do it... 107
TABLES 107
COLUMNS 108
FILES 109
INNODB_SYS_TABLESPACES 109
INNODB_TABLESTATS 110
PROCESSLIST 110
See also 111
Chapter 3: Using MySQL (Advanced) 112
Introduction 112
Using JSON 112
How to do it... 113
Insert JSON 113
Retrieve JSON 113
JSON functions 114
Pretty view 114
Searching 115
Modifying 116
Removing 117
Other functions 118
See also 118
Common table expressions (CTE) 118
How to do it... 119
Non-recursive CTE 119
Recursive CTE 121
Generated columns 124
How to do it... 124
Window functions 126
How to do it... 126
Row number 127
[ iv ]

Table of Contents
Partition results 127

Named windows 128
First, last, and nth values 128
Chapter 4: Configuring MySQL 130
Introduction 130
Using config file 131
How to do it... 131
Using global and session variables 132
How to do it... 132
Using parameters with startup script 133
How to do it... 133
Configuring the parameters 133
How to do it... 133
data directory 133
innodb_buffer_pool_size 134
innodb_buffer_pool_instances 135
innodb_log_file_size 136
Changing the data directory 136
How to do it... 136
Chapter 5: Transactions 138
Introduction 138
Performing transactions 139
How to do it... 140
Autocommit 141
Using savepoints 141
How to do it... 142
Isolation levels 143
How to do it... 143
Read uncommitted 143
Read committed 144
Repeatable read 145
Serializable 148
Locking 149
How to do it... 150
Locking queue 150
Chapter 6: Binary Logging 154
Introduction 154
Using binary logging 155
How to do it... 155
Enabling binary logs 155
[v]

Table of Contents
Disabling binary logs for a session 157

Move to the next log 157
Expire binary logs 158
Binary log format 160
How to do it... 160
See also 161
Extracting statements from a binary log 161
Getting ready 161
How to do it... 163
Observations 163
Extracting based on time and position 166
Extracting based on the database 167
Extracting a row event display 168
Rewriting a database name 169
Disabling a binary log for recovery 170
Displaying events in a binary log file 171
Ignoring databases to write to a binary log 172
How to do it... 172
Example 1 173
Example 2 173
Relocating binary logs 174
How to do it... 174
Chapter 7: Backups 176
Introduction 176
Taking backups using mysqldump 177
How to do it... 177
Full backup of all databases 177
Point-in-time recovery 178
Dumping master binary coordinates 178
Specific databases and tables 179
Ignore tables 179
Specific rows 179
Backup from a remote server 180
Backup to rebuild another server with a different schema 180
Only schema and no data 180
Only data and no schema 180
Backup for merging data with other server 181
REPLACE with new data 181
IGNORE data 181
Taking backups using mysqlpump 181
How to do it... 181
Parallel processing 182
Exclude/include database objects using regex 182
[ vi ]

Table of Contents
Backup users 183

Compressed backups 183
Faster reload 184
Taking backups using mydumper 184
How to do it... 185
Installation 185
Full backup 186
Consistent backup 187
Backup of a single table 188
Backup of specific databases using regex 188
Taking backup of a big table using mydumper 189
Non-blocking backup 189
Compressed backups 190
Backing up only data 190
Taking backups using flat files 190
How to do it... 191
Taking backups using XtraBackup 191
How to do it... 192
Installation 192
On CentOS/Red Hat/Fedora 192
On Debian/Ubuntu 193
Locking instances for backup 194
How to do it... 194
Binary log backup 194
How to do it... 194
Chapter 8: Restoring Data 196
Introduction 196
Recovering from mysqldump and mysqlpump 196
How to do it... 196
There's more... 197
Recovering from mydumper using myloader 197
How to do it... 197
Recovering full database 198
Recover a single database 198
Recovering a single table 198
Recovering from flat file backup 199
How to do it... 199
Performing point-in-time recovery 200
How to do it... 200
mysqldump or mysqlpump 200
mydumper 202
Chapter 9: Replication 203
[ vii ]

Table of Contents
Introduction 203
Setting up replication 204
How to do it... 207
Setting up master-master replication 210
How to do it... 211
Setting up multi-source replication 212
How to do it... 212
Setting up replication filters 218
How to do it... 218
Replicate a database only 218
Replicate specific tables 218
Ignore a database 219
Ignore specific tables 219
See also 219
Switching slave from master-slave to chain replication 220
How to do it... 220
Switching the slave from chain replication to master-slave 225
How to do it... 225
Setting up delayed replication 228
How to do it... 229
Setting up GTID replication 230
How to do it... 231
Setting up semi-synchronous replication 234
How to do it... 235
Chapter 10: Table Maintenance 239
Introduction 239
Installing Percona Toolkit 240
How to do it... 240
On Debian/Ubuntu 240
On CentOS/Red Hat/Fedora 241
Altering tables 242
How to do it... 244
Moving tables across databases 245
How to do it... 245
Altering tables using an online schema change tool 246
How it works... 246
How to do it... 247
Archiving tables 250
How to do it... 251
Purging data 252
[ viii ]

Table of Contents
Archiving data 252

Copying data 252
See also 253
Cloning tables 253
How to do it... 253
Partitioning tables 254
How to do it... 255
RANGE partitioning 256
Removing partitioning 259
RANGE COLUMNS partitioning 259
LIST and LIST COLUMNS partitioning 260
HASH and LINEAR HASH partitioning 261
KEY and LINEAR KEY partitioning 263
Subpartitioning 263
Partition pruning and selection 264
How to do it... 264
Partition pruning 265
Partition selection 266
Partition management 267
How to do it... 268
ADD partitions 268
Reorganizing partitions 269
DROP partitions 269
TRUNCATE partitions 269
Managing HASH and KEY partitions 270
Other operations 270
Partition information 271
How to do it... 271
Using SHOW CREATE TABLE 271
Using SHOW TABLE STATUS 271
Using EXPLAIN 272
Querying the INFORMATION_SCHEMA.PARTITIONS table 273
Efficiently managing time to live and soft delete rows 275
How it works... 275
How to do it... 276
Chapter 11: Managing Tablespace 282
Introduction 282
Changing the number or size of InnoDB redo log files 285
How to do it... 286
Resizing the InnoDB system tablespace 287
How to do it... 288
Increasing the InnoDB system tablespace 288
[ ix ]

Table of Contents
Shrinking the InnoDB system tablespace 290

Creating file-per-table tablespaces outside the data directory 291
How to do it... 292
Copying file-per-table tablespaces to another instance 292
How to do it... 293
Copy full table 294
Copying individual partitions of a table 297
See also 299
Managing UNDO tablespace 299
How to do it... 300
Managing general tablespace 303
How to do it... 304
Create a general tablespace 304
Adding tables to a general tablespace 304
Moving non-partitioned tables between tablespaces 305
Managing partitioned tables in a general tablespace 305
Dropping general tablespace 306
Compressing InnoDB tables 307
How to do it... 308
Enabling Compression for file_per_table Tables 308
Disabling Compression for file_per_table Tables 308
Enabling Compression for General Tablespace 309
Chapter 12: Managing Logs 311
Introduction 311
Managing the error log 311
How to do it... 312
Configuring the error log 312
Rotating the error log 314
Using the system log for logging 314
Error logging in JSON format 317
Managing the general query log and slow query log 319
How to do it... 319
General query log 319
Slow query log 320
Selecting query log output destinations 323
Managing the binary logs 325
How to do it... 325
Chapter 13: Performance Tuning 328
Introduction 329
The explain plan 329
How to do it... 329
[x]

Table of Contents
Using EXPLAIN 329

Using EXPLAIN JSON 330
Using EXPLAIN for connection 333
Benchmarking queries and the server 334
How to do it... 334
Adding indexes 336
Primary key (clustered index) and secondary indexes 336
How to do it... 337
Add index 338
UNIQUE index 339
Prefix index 339
Drop index 339
Index on generated columns 339
Invisible index 342
How to do it... 342
Descending index 344
How to do it... 345
Analyzing slow queries using pt-query-digest 346
How to do it... 346
Slow query log 347
General query log 350
Process list 350
Binary log 351
TCP dump 351
See also 351
Optimizing datatypes 351
How to do it... 352
Removing duplicate and redundant indexes 353
How to do it... 355
pt-duplicate-key-checker 355
mysqlindexcheck 356
Checking index usage 361
How to do it... 361
Controlling the query optimizer 361
How to do it... 363
optimizer_search_depth 363
How to know that the query is spending time in evaluating plans? 364
optimizer_switch 365
Optimizer hints 367
Adjusting the optimizer cost model 372
Using index hints 373
How to do it... 374
[ xi ]

Table of Contents
Indexing for JSON using generated columns 376

How to do it... 376
Using resource groups 379
How to do it... 379
Alter and drop resource group 381
Using performance_schema 382
How to do it... 382
Enable/disable performance_schema 382
Enable/disable consumers and instruments 382
performance_schema tables 384
Using the sys schema 387
How to do it... 387
Statement by type (INSERT and SELECT) from each host 388
Statement by type from each user 389
Redundant indexes 390
Unused indexes 390
Statements executed from each host 391
Table statistics 391
Table statistics with buffer 392
Statement analysis 392
Chapter 14: Security 395
Introduction 395
Securing installation 395
How to do it... 396
The FILE privilege 397
Restricting networks and users 398
How to do it... 399
Password-less authentication using mysql_config_editor 399
How to do it... 401
Resetting the root password 403
How to do it... 403
Using init-file 403
Using --skip-grant-tables 405
Setting up encrypted connections using X509 406
How to do it... 407
Setting up SSL replication 409
How to do it... 409
Index 412
[ xii ]

Preface
MySQL is one of the most popular and widely used relational databases in the world today.
With the recently released MySQL 8, it promises to be better and more efficient than ever,
giving you high-performance querying results and easy configuration as an administrator.
Who this book is for

This book is for a wide range of readers. MySQL database administrators and developers
who have worked on earlier versions of MySQL will learn about the features of MySQL 8
and how they can leverage them. For readers who worked on other RDBMSes such as
Oracle, MSSQL, PostgreSQL, and Db2 this book will be a quick start guide to MySQL 8. For
beginners, this book serves as a handbook; they can refer to the recipes and find quick
solutions to their problems.
Most importantly, this book makes you production-ready. After reading this book, you will
be confident in handling busy database servers with large datasets.
In my 10 years of experience with MySQL, I have witnessed small mistakes leading to major
outages. In this book, many scenarios where a mistake can be made are covered and put
under a warning label.
The topics are introduced in such a way that a beginner need not go back and forth to
understand the concepts. A reference link to the MySQL documentation or any other source
is provided for each topic, and the reader can refer to the link for more details.
Since this book is written to suit beginners as well, there may be a few recipes on topics you
already know; feel free to skip them.
What this book covers

Practice makes a man perfect. But to practice, you need some knowledge and training. This
book helps you with that. Most day-to-day and practical scenarios are covered in this book.
$IBQUFS, MySQL 8 - Installing and Upgrading, describes how to install MySQL 8 on

different flavors of Linux, upgrade to MySQL 8 from previous stable releases, and also
downgrade from MySQL 8.

Preface
$IBQUFS, Using MySQL, takes you through the basic uses of MySQL, such as creating
databases and tables; inserting, updating, deleting, and selecting data in various ways;
saving to different destinations; sorting and grouping results; joining tables; managing
users; other database elements such as triggers, stored procedures, functions, and events;
and getting metadata information.
$IBQUFS, Using MySQL (Advanced), covers the latest additions to MySQL 8, such as the
JSON datatype, common table expressions, and window functions.
$IBQUFS, Configuring MySQL, shows you how to configure MySQL and basic
configuration parameters.
$IBQUFS, Transactions, explains the four isolation levels of RDBMS and how to use MySQL
for transactions.
$IBQUFS, Binary Logging, demonstrates how to enable binary logging, various formats of
binary logs, and how to retrieve data from binary logs.
$IBQUFS, Backups, covers various types of backups, the pros and cons of each method, and
which one to choose based on your requirements.
$IBQUFS, Restoring Data, covers how to recover data from varies backups.
$IBQUFS, Replication, explains how to set up various replication topologies. The recipes on
switching a slave from master-slave to chain replication and switching a slave from chain
replication to master-slave is something that will interest the readers.
$IBQUFS, Table Maintenance, covers cloning tables. Managing big tables is something that
this chapter will make you a maestro of. Installation and usage of third-party tools is also
covered in this chapter.
$IBQUFS, Managing Tablespace, deals with recipes that will teach the readers how to
resize, create, copy, and manage tablespaces.
$IBQUFS, Managing Logs, takes readers through error, general query, slow query, and
binary logs.
$IBQUFS, Performance Tuning, explains query and schema tuning in detail. There are
ample recipes in the chapter that will cover this.
$IBQUFS, Security, focuses on the aspects of security. Recipes on securing installation,

restricting networks and users, setting and resetting of passwords, and much more in
covered are detail.
[2]

Preface
To get the most out of this book

Basic knowledge of any Linux system makes it easy for you to understand this book.
Conventions used
There are a number of text conventions used throughout this book.
$PEF*O5FYU: Indicates code words in text, database table names, folder names, filenames,
file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an
example: "MySQL has a dependency on the MJCBJP library."
When we wish to draw your attention to a particular part of a command line statement, the
relevant lines or items are set in bold:
TIFMM TVEPZVNSFQPMJTUBMM]HSFQNZTRM
mysql80-community/x86_64 MySQL 8.0 Community Server
enabled: 16
NZTRMDPNNVOJUZTPVSDF.Z42-$PNNVOJUZ4FSWFS
EJTBCMFE
Any command-line input or output is written as follows:

NZTRM "-5&35"#-&UBCMF@OBNF3&.07&1"35*5*0/*/(
Bold: Indicates a new term, an important word, or words that you see onscreen. For
example, words in menus or dialog boxes appear in the text like this. Here is an example:
"Select the Development Releases tab for getting MySQL 8.0 and the choose the OS and
version."
Warnings or important notes appear like this.
Tips and tricks appear like this.
[3]

Preface
Sections
In this book, you will find several headings that appear frequently (Getting ready, How to do
it..., How it works..., There's more..., and See also).
To give clear instructions on how to complete a recipe, use these sections as follows:
Getting ready
This section tells you what to expect in the recipe and describes how to set up any software
or any preliminary settings required for the recipe.
How to do it`
This section contains the steps required to follow the recipe.
How it works`
This section usually consists of a detailed explanation of what happened in the previous
section.
There's more`
This section consists of additional information about the recipe in order to make you more
knowledgeable about the recipe.
See also
This section provides helpful links to other useful information for the recipe.
Get in touch
Feedback from our readers is always welcome.
General feedback: Email GFFECBDL!QBDLUQVCDPN and mention the book title in the
subject of your message. If you have questions about any aspect of this book, please email
us at RVFTUJPOT!QBDLUQVCDPN.
[4]

Preface
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you have found a mistake in this book, we would be grateful if you would
report this to us. Please visit XXXQBDLUQVCDPNTVCNJUFSSBUB, selecting your book,
clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the internet, we
would be grateful if you would provide us with the location address or website name.
Please contact us at DPQZSJHIU!QBDLUQVCDPN with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in
and you are interested in either writing or contributing to a book, please visit
BVUIPSTQBDLUQVCDPN.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on
the site that you purchased it from? Potential readers can then see and use your unbiased
opinion to make purchase decisions, we at Packt can understand what you think about our
products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit QBDLUQVCDPN.
[5]

MySQL 8 - Installing and
1
Upgrading
In this chapter, we will cover the following recipes:
Installing MySQL using YUM/APT

Installing MySQL 8.0 using RPM or DEB files
Installing MySQL on Linux using Generic Binaries
Starting or Stopping MySQL 8
Uninstalling MySQL 8
Managing MySQL Server with systemd
Downgrading from MySQL 8.0
Upgrading to MySQL 8.0
Installing MySQL utilities
Introduction
In this chapter, you will learn about the installing, upgrading, and downgrading steps of
MySQL 8. There are five different ways to install or upgrade; the three most widely-used
installation methods are covered in this chapter:
Software repositories (YUM or APT)

RPM or DEB files
Generic Binaries

MySQL 8 - Installing and Upgrading Chapter 1
Docker (not covered)

Source code compilation (not covered)
If you have already installed MySQL and want to upgrade, go through the upgrade steps in
the Upgrade to MySQL 8 section. If your installation is corrupt, go through the uninstallation
steps also in the Upgrade to MySQL 8 section.
Before installation, make a note of OS and CPU architecture. The convention followed is as
follows:
MySQL Linux RPM package distribution identifiers
Distribution value Intended use

el6, el7 Red Hat Enterprise Linux, Oracle Linux, CentOS 6 or 7
fc23, fc24, fc25 Fedora 23, 24, or 25
sles12 SUSE Linux Enterprise Server 12
MySQL Linux RPM package CPU identifiers
CPU value Intended processor type or family

i386, i586, i686 Pentium processor or better, 32-bit
x86_64 64-bit x86 processor
ia64 Itanium (IA-64) processor
MySQL Debian and Ubuntu 7 and 8 installation packages CPU identifiers

i386 Pentium processor or better, 32-bit
amd64 64-bit x86 processor
MySQL Debian 6 Installation package CPU identifiers

i686 Pentium processor or better, 32-bit
x86_64 64-bit x86 processor
[7]

Installing MySQL using YUM/APT

The most common and easiest way of installation is through software repositories where
you add official Oracle MySQL repositories to your list and install MySQL through package
management software.
There are mainly two types of repository software:
YUM (Centos, Red Hat, Fedora and Oracle Linux)

APT (Debian, Ubuntu)
How to do it...
Let's look at steps for installing MySQL 8 in the following ways:
Using YUM repositories

1. Find the Red Hat or CentOS version:
TIFMM DBUFUDSFEIBUSFMFBTF
$FOU04-JOVYSFMFBTF
$PSF
2. Add the MySQL Yum repository to your system's repository list. This is a one-
time operation that can be performed by installing an RPM provided by MySQL.
You can download the MySQL YUM Repository from IUUQEFWNZTRMDPN
EPXOMPBETSFQPZVN and choose the file depending on your OS.
Install the downloaded release package with the following command, replacing
the name with the platform- and version-specific package name of the
downloaded RPM package:
TIFMM TVEPZVNMPDBMJOTUBMMZNZTRMDPNNVOJUZSFMFBTF
FMOPBSDISQN
-PBEFEQMVHJOTGBTUFTUNJSSPS
&YBNJOJOHNZTRMDPNNVOJUZSFMFBTFFMOPBSDISQNNZTRM
DPNNVOJUZSFMFBTFFMOPBSDI
.BSLJOHNZTRMDPNNVOJUZSFMFBTFFMOPBSDISQNUPCFJOTUBMMFE
3FTPMWJOH%FQFOEFODJFT
3VOOJOHUSBOTBDUJPODIFDL
1BDLBHFNZTRMDPNNVOJUZSFMFBTFOPBSDIFMXJMMCF
JOTUBMMFE
'JOJTIFE%FQFOEFODZ3FTPMVUJPO
[8]

_
7FSJGZJOHNZTRMDPNNVOJUZSFMFBTFFMOPBSDI
*OTUBMMFE
NZTRMDPNNVOJUZSFMFBTFOPBSDIFM
$PNQMFUF
3. Or you can copy the link location and install directly using RPM (you can skip
the next step after installing):
TIFMM TVEPSQN6WI
IUUQTEFWNZTRMDPNHFUNZTRMDPNNVOJUZSFMFBTFFMOPBSDI
SQN
3FUSJFWJOH
IUUQTEFWNZTRMDPNHFUNZTRMDPNNVOJUZSFMFBTFFMOPBSDIS
QN
1SFQBSJOH
<>
6QEBUJOHJOTUBMMJOH
NZTRMDPNNVOJUZSFMFBTFFM
<>
4. Verify the installation:
TIFMM ZVNSFQPMJTUFOBCMFE]HSFQ NZTRMDPNNVOJUZ

NZTRMDPOOFDUPSTDPNNVOJUZY@.Z42-$POOFDUPST$PNNVOJUZ

NZTRMUPPMTDPNNVOJUZY@.Z42-5PPMT$PNNVOJUZ

NZTRMDPNNVOJUZY@.Z42-$PNNVOJUZ4FSWFS

5. Set the release series. At the time of writing this book, MySQL 8 is not a general
availability (GA) release. So MySQL 5.7 will be selected as the default release
series. To install MySQL 8, you have to set the release series to 8:
NZTRMDMVTUFSDPNNVOJUZY@.Z42-$MVTUFS$PNNVOJUZ
EJTBCMFE
NZTRMDMVTUFSDPNNVOJUZTPVSDF.Z42-$MVTUFS$PNNVOJUZ
EJTBCMFE
NZTRMDMVTUFSDPNNVOJUZY@.Z42-$MVTUFS$PNNVOJUZ
EJTBCMFE
NZTRMDMVTUFSDPNNVOJUZTPVSDF.Z42-$MVTUFS$PNNVOJUZ
EJTBCMFE
mysql-connectors-community/x86_64 MySQL Connectors Community
enabled: 42
[9]

NZTRMDPOOFDUPSTDPNNVOJUZTPVSDF.Z42-$POOFDUPST$PNNVOJUZ
EJTBCMFE
mysql-tools-community/x86_64 MySQL Tools Community
enabled: 53
NZTRMUPPMTDPNNVOJUZTPVSDF.Z42-5PPMT$PNNVOJUZ4PV
EJTBCMFE
NZTRMUPPMTQSFWJFXY@.Z42-5PPMT1SFWJFX
EJTBCMFE
NZTRMUPPMTQSFWJFXTPVSDF.Z42-5PPMT1SFWJFX4PVSD
EJTBCMFE
EJTBCMFE
EJTBCMFE
EJTBCMFE
EJTBCMFE
enabled: 227
EJTBCMFE
disabled
mysql80-community-source MySQL 8.0 Community Server
disabled
6. Disable NZTRMDPNNVOJUZ and enable NZTRMDPNNVOJUZ:
TIFMM TVEPZVNJOTUBMMZVNVUJMTOPBSDIZ
TIFMM TVEPZVNDPOGJHNBOBHFSEJTBCMFNZTRMDPNNVOJUZ
TIFMM TVEPZVNDPOGJHNBOBHFSFOBCMFNZTRMDPNNVOJUZ
7. Verify that NZTRMDPNNVOJUZ is enabled:
enabled: 16
EJTBCMFE
8. Install MySQL 8:
TIFMM TVEPZVNJOTUBMMZNZTRMDPNNVOJUZTFSWFS
NZTRMDPOOFDUPSTDPNNVOJUZ]L#
NZTRMUPPMTDPNNVOJUZ]L#
[ 10 ]

NZTRMDPNNVOJUZ]L#
-PBEJOHNJSSPSTQFFETGSPNDBDIFEIPTUGJMF
CBTFNJSSPSXFCTUFSDPN
FQFMNJSSPSTDBUQEYFEV
FYUSBTNJSSPSTPJUVDJFEV
VQEBUFTSFQPTMBYRVBESBOFUDPN
3FTPMWJOH%FQFOEFODJFT
_
5SBOTBDUJPOUFTUTVDDFFEFE
3VOOJOHUSBOTBDUJPO
*OTUBMMJOHNZTRMDPNNVOJUZDPNNPOSDFMY@
*OTUBMMJOHNZTRMDPNNVOJUZMJCTSDFMY@
*OTUBMMJOHNZTRMDPNNVOJUZDMJFOUSDFMY@
*OTUBMMJOHNZTRMDPNNVOJUZTFSWFSSDFMY@
7FSJGZJOHNZTRMDPNNVOJUZMJCTSDFMY@
7FSJGZJOHNZTRMDPNNVOJUZDPNNPOSDFMY@
7FSJGZJOHNZTRMDPNNVOJUZDMJFOUSDFMY@
7FSJGZJOHNZTRMDPNNVOJUZTFSWFSSDFMY@
*OTUBMMFE
NZTRMDPNNVOJUZTFSWFSY@SDFM
%FQFOEFODZ*OTUBMMFE
NZTRMDPNNVOJUZDMJFOUY@SDFM
NZTRMDPNNVOJUZDPNNPOY@SDFM
NZTRMDPNNVOJUZMJCTY@SDFM
$PNQMFUF
9. You can check the installed packages using the following:
TIFMM SQNRB]HSFQJ NZTRM

QFSM%#%.Z42-FMY@
NZTRMDPNNVOJUZMJCTSDFMY@
NZTRMDPNNVOJUZDPNNPOSDFMY@
NZTRMDPNNVOJUZDMJFOUSDFMY@
NZTRMDPNNVOJUZTFSWFSSDFMY@
[ 11 ]

Using APT repositories

1. Add the MySQL APT repository to your system's repository list. This is a one-
time operation that can be performed by installing a EFC file provided by
MySQL
You can download the MySQL APT repository from IUUQEFWNZTRMDPN
EPXOMPBETSFQPBQU.
Or you can copy the link location and use XHFU to download directly on to the
server. You might need to install XHFU (TVEPBQUHFUJOTUBMMXHFU):
TIFMM XHFU
IUUQTSFQPNZTRMDPNNZTRMBQUDPOGJH@@BMMEFC
2. Install the downloaded release package with the following command, replacing
the name with platform- and version-specific package name of the downloaded
APT package:
TIFMM TVEPEQLHJNZTRMBQUDPOGJH@@BMMEFC

3FBEJOHEBUBCBTFGJMFTBOEEJSFDUPSJFTDVSSFOUMZ
JOTUBMMFE
1SFQBSJOHUPVOQBDLNZTRMBQUDPOGJH@@BMMEFC
6OQBDLJOHNZTRMBQUDPOGJH
PWFS

4FUUJOHVQNZTRMBQUDPOGJH

8BSOJOHBQULFZTIPVMEOPUCFVTFEJOTDSJQUT
DBMMFEGSPN
QPTUJOTUNBJOUBJOFSTDSJQUPGUIFQBDLBHFNZTRMBQUDPOGJH
0,
3. During the installation of the package, you will be asked to choose the versions of
the MySQL server and other components. Press Enter for selecting and the Up
and Down keys for navigating.
Select MySQL Server and Cluster (Currently selected: mysql-5.7).
Select mysql-8.0 preview (At the time of writing, MySQL 8.0 is not GA). You
might get a warning such as MySQL 8.0-RC Note that MySQL 8.0 is currently
an RC. It should only be installed to preview upcoming features of MySQL,
and is not recommended for use in production environments. (RC is short
for release candidate).
If you want to change the release version, execute the following:
TIFMM TVEPEQLHSFDPOGJHVSFNZTRMBQUDPOGJH
[ 12 ]

4. Update package information from the MySQL APT repository with the following
command (this step is mandatory):
TIFMM TVEPBQUHFUVQEBUF
5. Install MySQL. During installation, you'll need to provide a password for the root
user for your MySQL installation. Remember the password; if you forget it, you'll
have to reset the root password (refer to the Resetting root password section). This
installs the package for the MySQL server, as well as the packages for the client
and for the database common files:
TIFMM TVEPBQUHFUJOTUBMMZNZTRMDPNNVOJUZTFSWFS
_
1SPDFTTJOHUSJHHFSTGPSVSFBEBIFBE

4FUUJOHVQNZTRMDPNNPO
SDVCVOUV
VQEBUFBMUFSOBUJWFTVTJOHFUDNZTRMNZDOGGBMMCBDLUPQSPWJEF
FUDNZTRMNZDOG
NZDOGJOBVUPNPEF
4FUUJOHVQNZTRMDPNNVOJUZDMJFOUDPSF
SDVCVOUV
4FUUJOHVQNZTRMDPNNVOJUZTFSWFSDPSF
SDVCVOUV
_
6. Verify packages. JJ indicates that the package is installed:
TIFMM EQLHM]HSFQJNZTRM
JJNZTRMBQUDPOGJHBMM"VUP
DPOGJHVSBUJPOGPS.Z42-"153FQP
JJNZTRMDMJFOUSDVCVOUVBNE.Z42-
$MJFOUNFUBQBDLBHFEFQFOEJOHPOMBUFTUWFSTJPO
JJNZTRMDPNNPOSDVCVOUVBNE.Z42-
$PNNPO
JJNZTRMDPNNVOJUZDMJFOUSDVCVOUVBNE.Z42-
$MJFOU
JJNZTRMDPNNVOJUZDMJFOUDPSFSDVCVOUVBNE.Z42-
$MJFOU$PSF#JOBSJFT
JJNZTRMDPNNVOJUZTFSWFSSDVCVOUVBNE.Z42-
4FSWFS
JJNZTRMDPNNVOJUZTFSWFSDPSFSDVCVOUVBNE.Z42-
4FSWFS$PSF#JOBJSFT
[ 13 ]

Installing MySQL 8.0 using RPM or DEB files

Installing MySQL using repositories requires access to public internet. As a security
measure, most of the production machines are not connected to the internet. In that case,
you can download the RPM or DEB files on the system administration and copy them to the
production machine.
There are mainly two types of installation files:
RPM (CentOS, Red Hat, Fedora, and Oracle Linux)

DEB (Debian, Ubuntu)
There are multiple packages that you need to install. Here is a list and short description of
each one:
NZTRMDPNNVOJUZTFSWFS: Database server and related tools.

NZTRMDPNNVOJUZDMJFOU: MySQL client applications and tools.
NZTRMDPNNVOJUZDPNNPO: Common files for server and client libraries.
NZTRMDPNNVOJUZEFWFM: Development header files and libraries for MySQL
database client applications, such as the Perl MySQL module.
NZTRMDPNNVOJUZMJCT: The shared libraries (MJCNZTRMDMJFOUTP) that
certain languages and applications need to dynamically load and use MySQL.
NZTRMDPNNVOJUZMJCTDPNQBU: The shared libraries for older releases. Install
this package if you have applications installed that are dynamically linked
against older versions of MySQL but you want to upgrade to the current version
without breaking the library dependencies.
How to do it...
Let's look at how to do it using the following types of bundles:
[ 14 ]

Using the RPM bundle

1. Download the MySQL RPM tar bundle from the MySQL Downloads page,
IUUQEFWNZTRMDPNEPXOMPBETNZTRM, choosing your OS and CPU
architecture. At the time of writing, MySQL 8.0 is not GA. If it is still in the
development series, select the Development Releases tab for getting MySQL 8.0
and the choose the OS and version:
TIFMM XHFU
IUUQTEFWNZTRMDPNHFU%PXOMPBET.Z42-NZTRMSDF
MY@SQNCVOEMFUBS
_
4BWJOHUPbNZTRMSDFMY@SQNCVOEMFUBSc
_
2. Untar the package:
TIFMM UBSYGWNZTRMSDFMY@SQNCVOEMFUBS
3. Install MySQL:
TIFMM TVEPSQNJNZTRMDPNNVOJUZ\TFSWFSDMJFOUDPNNPOMJCT^
4. RPM cannot solve the dependency issues and the installation process might run
issues. If you are facing such issues, use the ZVN command listed here (you
should have access to dependent packages):
TIFMM TVEPZVNJOTUBMMNZTRMDPNNVOJUZ
\TFSWFSDMJFOUDPNNPOMJCT^Z
TIFMM SQNRB]HSFQJNZTRMDPNNVOJUZ
NZTRMDPNNVOJUZMJCTDPNQBUSDFMY@
[ 15 ]

Using the APT bundle

1. Download the MySQL APT TAR from the MySQL Downloads page, IUUQ
EFWNZTRMDPNEPXOMPBETNZTRM:
TIFMM XHFU
IUUQTEFWNZTRMDPNHFU%PXOMPBET.Z42-NZTRMTFSWFS@S
DVCVOUV@BNEEFCCVOEMFUBS
_
4BWJOHUPbNZTRMTFSWFS@SDVCVOUV@BNEEFCCVOEMFUBSc
_
2. Untar the packages:
TIFMM UBSYWGNZTRMTFSWFS@SDVCVOUV@BNEEFC
CVOEMFUBS
3. Install the dependencies. You may need to install the MJCBJP package if it is not
already installed:
TIFMM TVEPBQUHFUJOTUBMMZMJCBJP
4. Upgrade MJCTUED to the latest:
TIFMM TVEPBEEBQUSFQPTJUPSZQQBVCVOUVUPPMDIBJOSUFTU
TIFMM TVEPBQUHFUVQHSBEFZMJCTUED
5. Upgrade MJCNFDBC to the latest. If VOJWFSTF is not included, then add the
following line to the end of the file (for example, [FTUZ):
TIFMM TVEPWJFUDBQUTPVSDFTMJTU
EFCIUUQVTBSDIJWFVCVOUVDPNVCVOUV[FTUZNBJOVOJWFSTF
TIFMM TVEPBQUHFUJOTUBMMMJCNFDBC
6. Preconfigure the MySQL server package with the following command. It asks
you to set the root password:
TIFMM TVEPEQLHQSFDPOGJHVSFNZTRMDPNNVOJUZTFSWFS@EFC
[ 16 ]

7. Install the database common files package, the client package, the client
metapackage, the server package, and the server metapackage (in that order); you
can do that with a single command:
TIFMM TVEPEQLHJNZTRM\DPNNPODPNNVOJUZDMJFOUDPSFDPNNVOJUZ
DMJFOUDMJFOUDPNNVOJUZTFSWFSDPSFDPNNVOJUZTFSWFSTFSWFS^@EFC
8. Install the shared libraries:
TIFMM TVEPEQLHJMJCNZTRMDMJFOU@
ENSVCVOUV@BNEEFC
JJNZTRMDMJFOUSDVCVOUVBNE.Z42-
$MJFOUNFUBQBDLBHFEFQFOEJOHPOMBUFTUWFSTJPO
JJNZTRMDPNNPOSDVCVOUVBNE.Z42-
$PNNPO
JJNZTRMDPNNVOJUZDMJFOUSDVCVOUVBNE.Z42-
$MJFOU
JJNZTRMDPNNVOJUZDMJFOUDPSFSDVCVOUVBNE.Z42-
$MJFOU$PSF#JOBSJFT
JJNZTRMDPNNVOJUZTFSWFSSDVCVOUVBNE.Z42-
4FSWFS
JJNZTRMDPNNVOJUZTFSWFSDPSFSDVCVOUVBNE.Z42-
4FSWFS$PSF#JOBJSFT
JJNZTRMTFSWFSSDVCVOUVBNE.Z42-
4FSWFSNFUBQBDLBHFEFQFOEJOHPOMBUFTUWFSTJPO
Installing MySQL on Linux using Generic

Binaries
Installing using the software packages requires some dependencies to be installed first and
can conflict with other packages. In that case, you can install MySQL using the generic
binaries available on the downloads page. Binaries are precompiled using advanced
compilers and are built with the best possible options for optimal performance.
[ 17 ]

How to do it...
MySQL has a dependency on the MJCBJP library. The EBUBEJSFDUPSZ initialization, and
subsequent server startup steps, will fail if this library is not installed locally.
On YUM-based systems:
TIFMM TVEPZVNJOTUBMMZMJCBJP
On APT-based systems:
TIFMM TVEPBQUHFUJOTUBMMZMJCBJP
Download the TAR binary from the MySQL Downloads page, at IUUQTEFWNZTRMDPN
EPXOMPBETNZTRM, then choose Linux - Generic as the OS and select the version. You can
download directly onto your server directly using the XHFU command:
TIFMM DEPQU
TIFMM XHFU
IUUQTEFWNZTRMDPNHFU%PXOMPBET.Z42-NZTRMSDMJOVYHMJCD
Y@UBSH[
Install MySQL using the following steps:
1. Add the NZTRM group and the NZTRM user. All the files and directories should be
under the NZTRM user:
TIFMM TVEPHSPVQBEENZTRM
TIFMM TVEPVTFSBEESHNZTRMTCJOGBMTFNZTRM
2. This is the installation location (you can change it to another location):
TIFMM DEVTSMPDBM
3. Untar the binary file. Keep the untarred binary file at the same location and
symlink it to the installation location. In this way, you can keep multiple versions
and it is very easy to upgrade. For example, you can download another version
and untar it to a different location; while upgrading, all you need to do is to
change the symlink:
TIFMM TVEPUBS[YWGPQUNZTRMSDMJOVYHMJCD
Y@UBSH[
NZTRMSDMJOVYHMJCDY@CJONZJTBN@GUEVNQ
NZTRMSDMJOVYHMJCDY@CJONZJTBNDIL
[ 18 ]

NZTRMSDMJOVYHMJCDY@CJONZJTBNMPH
NZTRMSDMJOVYHMJCDY@CJONZJTBNQBDL
NZTRMSDMJOVYHMJCDY@CJONZTRM
_
4. Make the symlink:
TIFMM TVEPMOTNZTRMSDMJOVYHMJCDY@NZTRM
5. Create the necessary directories and change the ownership to NZTRM:
TIFMM DENZTRM
TIFMM TVEPNLEJSNZTRMGJMFT
TIFMM TVEPDINPENZTRMGJMFT
TIFMM TVEPDIPXO3NZTRM
TIFMM TVEPDIHSQ3NZTRM
6. Initialize NZTRM, which generates a temporary password:
TIFMM TVEPCJONZTRMEJOJUJBMJ[FVTFSNZTRM
_
5;</PUF>"UFNQPSBSZQBTTXPSEJT
HFOFSBUFEGPSSPPU!MPDBMIPTUAw=ee.rf(6Ua
_
7. Set up the RSA for SSL. Refer to $IBQUFS, Setting up Encrypted Connections
using X509 Section, for more details on SSL. Note that a temporary password is
generated for SPPU!MPDBMIPTUF+2EK$R7.R:
TIFMM TVEPCJONZTRM@TTM@STB@TFUVQ
(FOFSBUJOHBCJU34"QSJWBUFLFZ

XSJUJOHOFXQSJWBUFLFZUP DBLFZQFN


XSJUJOHOFXQSJWBUFLFZUP TFSWFSLFZQFN


XSJUJOHOFXQSJWBUFLFZUP DMJFOULFZQFN

[ 19 ]

8. Change the ownership of binaries to SPPU and EBUB files to NZTRM:
TIFMM TVEPDIPXO3SPPU
TIFMM TVEPDIPXO3NZTRMEBUBNZTRMGJMFT
9. Copy the startup script to JOJUE:
TIFMM TVEPDQTVQQPSUGJMFTNZTRMTFSWFSFUDJOJUENZTRM
10. Export the binary of NZTRM to the 1"5) environment variable:
TIFMM FYQPSU1"5)1"5)VTSMPDBMNZTRMCJO
11. Refer to Starting or Stopping MySQL 8 section to start MySQL.
After installation, you will get the following directories inside VTSMPDBMNZTRM:
Directory Contents of directory

CJO NZTRME server, client, and utility programs
EBUB Log files, databases
EPDT MySQL manual in info format
NBO Unix manual pages
JODMVEF Include (header) files
MJC Libraries
Miscellaneous support files, including error messages, sample configuration
TIBSF
files, SQL for database installation
[ 20 ]

There's more...
There are other installation methods, such as:
1. Compiling from the source code. You can compile and build MySQL from the
source code provided by Oracle where you have the flexibility to customize build
parameters, compiler optimizations, and the installation location. It is highly
recommended to use precompiled binaries provided by Oracle, unless you want
specific compiler options or you are debugging MySQL.
This method is not covered as it is used very rarely and it requires several
development tools, which is beyond the scope of this book. For installation
through source code, you can refer to the reference manual, at IUUQTEFW
NZTRMDPNEPDSFGNBOFOTPVSDFJOTUBMMBUJPOIUNM.
2. Using Docker. The MySQL server can also be installed and managed using
Docker image. Refer to IUUQTIVCEPDLFSDPNSNZTRMNZTRMTFSWFS for
installation, configuration, and also how to use MySQL under Docker.
Starting or Stopping MySQL 8

After the installation is completed, you can start/stop MySQL using the following
commands, which vary from different platforms and installation methods. NZTRME is the
NZTRM server process. All the startup methods invoke the NZTRME script.
How to do it...
Let's look at it in detail. Along with the starting and stopping, we will also learn something
about checking the status of the server. Let's see how.
Starting the MySQL 8.0 server

You can start the server with the following commands:
1. Using TFSWJDF:
TIFMM TVEPTFSWJDFNZTRMTUBSU
[ 21 ]

2. Using JOJUE:
TIFMM TVEPFUDJOJUENZTRMTUBSU
3. If you do not find the startup script (when you are doing binary installation), you
can copy from the location where you untarred.
TIFMM TVEPDQVTSMPDBMNZTRMTVQQPSUGJMFTNZTRMTFSWFS
FUDJOJUENZTRM
4. If your installation includes TZTUFNE support:
TIFMM TVEPTZTUFNDUMTUBSUNZTRME
5. If the TZTUFNE support is not there, MySQL can be started using NZTRME@TBGF.
NZTRME@TBGF is the launcher script for NZTRME that safeguards the NZTRME
process. If NZTRME is killed, NZTRME@TBGF attempts to start the process again:
TIFMM TVEPNZTRME@TBGFVTFSNZTRM
After start,
1. The server is initialized.

2. The SSL certificate and key files are generated in the EBUBEJSFDUPSZ.
3. The WBMJEBUF@QBTTXPSE plugin is installed and enabled.
4. A superuser account, SPPU ! MPDBMIPTU, is created. A password for the
superuser is set and stored in the error log file (not for binary installation). To
reveal it, use the following command:
TIFMM TVEPHSFQUFNQPSBSZQBTTXPSEWBSMPHNZTRMEMPH
HFOFSBUFEGPSSPPU!MPDBMIPTUCLWPUT(IK%
You can connect to MySQL using that temporary password.
TIFMM NZTRMVSPPUQCLWPUT(IK%
NZTRM<8BSOJOH>6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBO
CFJOTFDVSF
8FMDPNFUPUIF.Z42-NPOJUPS$PNNBOETFOEXJUIPS=H
:PVS.Z42-DPOOFDUJPOJEJT
4FSWFSWFSTJPOSDMPH
$PQZSJHIU
D0SBDMFBOEPSJUTBGGJMJBUFT"MMSJHIUT
SFTFSWFE
[ 22 ]

0SBDMFJTBSFHJTUFSFEUSBEFNBSLPG0SBDMF$PSQPSBUJPOBOEPSJUT
BGGJMJBUFT0UIFSOBNFTNBZCFUSBEFNBSLTPGUIFJSSFTQFDUJWF
PXOFST
5ZQF IFMQ PS =I GPSIFMQ5ZQF =D UPDMFBSUIFDVSSFOUJOQVU

TUBUFNFOU
NZTRM
5. Change the root password as soon as possible by logging in with the generated
temporary password and setting a custom password for the superuser account:
:PVXJMMCFQSPNQUFEGPSBQBTTXPSEFOUFSUIFPOFZPVHPUGSPN
UIFQSFWJPVTTUFQ
NZTRM "-5&364&3 SPPU ! MPDBMIPTU *%&/5*'*&%#: /FX1BTT

2VFSZ0,SPXTBGGFDUFE
TFD
QBTTXPSETIPVMEDPOUBJOBUMFBTUPOF6QQFSDBTFMFUUFSPOF
MPXFSDBTFMFUUFSPOFEJHJUBOEPOFTQFDJBMDIBSBDUFSBOEUIBU
UIFUPUBMQBTTXPSEMFOHUIJTBUMFBTUDIBSBDUFST
Stopping the MySQL 8.0 server

Stopping MySQL and checking the status are similar to starting it, except for the change of
one word:
1. Using TFSWJDF:
TIFMM TVEPTFSWJDFNZTRMETUPQ
3FEJSFDUJOHUPCJOTZTUFNDUMTUPQNZTRMETFSWJDF
2. Using JOJUE:
TIFMM TVEPFUDJOJUENZTRMTUPQ
<PL>4UPQQJOHNZTRM
WJBTZTUFNDUMNZTRMTFSWJDF
3. If your installation includes the TZTUFNE support (refer to the Managing MySQL
Server with systemd section):
TIFMM TVEPTZTUFNDUMTUPQNZTRME
4. Using NZTRMBENJO:
TIFMM NZTRMBENJOVSPPUQTIVUEPXO
[ 23 ]

Checking the status of the MySQL 8.0 server

1. Using TFSWJDF:
TIFMM TVEPTZTUFNDUMTUBUVTNZTRME
dNZTRMETFSWJDF.Z42-4FSWFS
-PBEFEMPBEFE
VTSMJCTZTUFNETZTUFNNZTRMETFSWJDFFOBCMFE
WFOEPSQSFTFUEJTBCMFE
%SPQ*OFUDTZTUFNETZTUFNNZTRMETFSWJDFE
efPWFSSJEFDPOG
Active: active (running) since Sat 2017-12-02 07:33:53 UTC; 14s
ago
%PDTNBONZTRME

IUUQEFWNZTRMDPNEPDSFGNBOFOVTJOHTZTUFNEIUNM
1SPDFTT&YFD4UBSUVTSTCJONZTRMEEBFNPOJ[FQJE
GJMFWBSSVONZTRMENZTRMEQJE.:42-%@0154
DPEFFYJUFE
TUBUVT46$$&44
1SPDFTT&YFD4UBSU1SFVTSCJONZTRME@QSF@TZTUFNE

DPEFFYJUFETUBUVT46$$&44
.BJO1*%
NZTRME
$(SPVQTZTUFNTMJDFNZTRMETFSWJDF
efVTSTCJONZTRMEEBFNPOJ[FQJE
GJMFWBSSVONZTRMENZTRMEQJEHFOFSBM@MPH
%FDDFOUPTTZTUFNE<>4UBSUJOH.Z42-4FSWFS
%FDDFOUPTTZTUFNE<>4UBSUFE.Z42-4FSWFS
2. Using JOJUE:
TIFMM TVEPFUDJOJUENZTRMTUBUVT
dNZTRMTFSWJDF-4#TUBSUBOETUPQ.Z42-
-PBEFEMPBEFE
FUDJOJUENZTRMCBEWFOEPSQSFTFUFOBCMFE
"DUJWFJOBDUJWF
EFBE
%PDTNBOTZTUFNETZTWHFOFSBUPS

%FDVCVOUVTZTUFNE<>4UBSUJOH-4#TUBSUBOETUPQ
.Z42-
%FDVCVOUVNZTRM<>4UBSUJOH.Z42-
%FDVCVOUVNZTRM<>
%FDVCVOUVTZTUFNE<>4UBSUFE-4#TUBSUBOETUPQ
.Z42-
%FDVCVOUVNZTRM<>5;
NZTRME@TBGF"NZTRMEQSPDFTTBMSFBEZFYJTUT
%FDVCVOUVTZTUFNE<>4UPQQJOH-4#TUBSUBOETUPQ
.Z42-
%FDVCVOUVNZTRM<>4IVUUJOHEPXO.Z42-
%FDVCVOUVNZTRM<>
[ 24 ]

%FDVCVOUVTZTUFNE<>4UPQQFE-4#TUBSUBOETUPQ
.Z42-
%FDVCVOUVTZTUFNE<>4UPQQFE-4#TUBSUBOETUPQ
.Z42-
3. If your installation includes the TZTUFNE support (refer to the Managing MySQL
Server with systemd section):
TIFMM TVEPTZTUFNDUMTUBUVTNZTRME
Uninstalling MySQL 8
If you have messed up with installation or you do not want MySQL 8 version, you can
uninstall using the following steps. Before uninstalling, make sure to make backup files
(refer to $IBQUFS, Backups), if required, and stop MySQL.
How to do it...
Uninstalling will be dealt in a different way on different systems. Let's look at how.
On YUM-based systems
1. Check whether there are any existing packages:
TIFMM SQNRB]HSFQJNZTRMDPNNVOJUZ
NZTRMDPNNVOJUZMJCTDPNQBUSDFMY@
2. Remove the packages. You may be notified that there are other packages
dependent on MySQL. If you plan on installing MySQL again, you can ignore the
warning by passing the OPEFQT option:
TIFMM SQNFQBDLBHFOBNF
For example:
TIFMM TVEPSQNFNZTRMDPNNVOJUZTFSWFS
[ 25 ]

3. To remove all packages:
TIFMM TVEPSQNRB]HSFQJNZTRMDPNNVOJUZ]YBSHTTVEPSQNF
OPEFQT
XBSOJOHFUDNZDOGTBWFEBTFUDNZDOGSQNTBWF
On APT-based systems
1. Check whether there are any existing packages:
2. Remove the packages using the following:
TIFMM TVEPBQUHFUSFNPWFNZTRMDPNNVOJUZTFSWFSNZTRMDMJFOU
NZTRMDPNNPONZTRMDPNNVOJUZDMJFOUNZTRMDPNNVOJUZDMJFOUDPSF
NZTRMDPNNVOJUZTFSWFSNZTRMDPNNVOJUZTFSWFSDPSFZ
3FBEJOHQBDLBHFMJTUT%POF
#VJMEJOHEFQFOEFODZUSFF
3FBEJOHTUBUFJOGPSNBUJPO%POF
5IFGPMMPXJOHQBDLBHFTXJMMCF3&.07&%
NZTRMDMJFOUNZTRMDPNNPONZTRMDPNNVOJUZDMJFOUNZTRMDPNNVOJUZ
DMJFOUDPSFNZTRMDPNNVOJUZTFSWFSNZTRMDPNNVOJUZTFSWFSDPSF
NZTRMTFSWFS
VQHSBEFEOFXMZJOTUBMMFEUPSFNPWFBOEOPUVQHSBEFE
"GUFSUIJTPQFSBUJPO.#EJTLTQBDFXJMMCFGSFFE

3FBEJOHEBUBCBTFGJMFTBOEEJSFDUPSJFTDVSSFOUMZ
JOTUBMMFE
3FNPWJOHNZTRMTFSWFS
SDVCVOUV
3FNPWJOHNZTRMDPNNVOJUZTFSWFS
SDVCVOUV
VQEBUFBMUFSOBUJWFTVTJOHFUDNZTRMNZDOGGBMMCBDLUPQSPWJEF
FUDNZTRMNZDOG
NZDOGJOBVUPNPEF
3FNPWJOHNZTRMDMJFOU
SDVCVOUV
3FNPWJOHNZTRMDPNNVOJUZDMJFOU
SDVCVOUV
3FNPWJOHNZTRMDPNNPO
SDVCVOUV
3FNPWJOHNZTRMDPNNVOJUZDMJFOUDPSF
SDVCVOUV
3FNPWJOHNZTRMDPNNVOJUZTFSWFSDPSF
SDVCVOUV
1SPDFTTJOHUSJHHFSTGPSNBOEC

Or remove them using:

TIFMM TVEPBQUHFUSFNPWFQVSHFNZTRM=Z
TIFMM TVEPBQUHFUBVUPSFNPWFZ
[ 26 ]

3. Verify that the packages are uninstalled:
JJNZTRMBQUDPOGJHBMM"VUP
DPOGJHVSBUJPOGPS.Z42-"153FQP
SDNZTRMDPNNPOSDVCVOUVBNE
.Z42-$PNNPO
SDNZTRMDPNNVOJUZDMJFOUSDVCVOUVBNE
.Z42-$MJFOU
SDNZTRMDPNNVOJUZTFSWFSSDVCVOUVBNE
.Z42-4FSWFS
SD indicates that the packages have been removed (S), and only config files (D) have been
kept.
Uninstalling Binaries
It is very simple to uninstall a binary installation. All you need to do is to remove the
symlink:
1. Change the directory to the installation path:
TIFMM DEVTSMPDBM
2. Check where NZTRM is pointing to, which will show the path it is referencing to:
TIFMM TVEPMTMINZTRM
3. Remove NZTRM:
TIFMM TVEPSNNZTRM
4. Remove the binaries (optional):
TIFMM TVEPSNGPQUNZTRMSDMJOVYHMJCDY@UBSH[
[ 27 ]

Managing the MySQL Server with systemd

If you install MySQL using an RPM or Debian package server, startup and shutdown is
managed by TZTUFNE. On platforms for which the TZTUFNE support for MySQL is installed,
NZTRME@TBGF, NZTRME@NVMUJ, and NZTRME@NVMUJTFSWFS are not installed. MySQL
server startup and shutdown is managed by TZTUFNE using the TZTUFNDUM command. You
need to configure TZTUFNE as follows.
RPM-based systems use the NZTRMETFSWJDF files, and APT-based

systems use the NZTRMTFSWFS files.
How to do it...
1. Create a localized TZTUFNE configuration file:
TIFMM TVEPNLEJSQWFUDTZTUFNETZTUFNNZTRMETFSWJDFE
2. Create/open the DPOG file:
TIFMM TVEPWJFUDTZTUFNETZTUFNNZTRMETFSWJDFEPWFSSJEFDPOG
3. Enter the following:
<4FSWJDF>
-JNJU/0'*-&NBY@PQFO@GJMFT
FY
1*%'JMFQBUIUPQJEGJMF
FYWBSMJCNZTRMNZTRMQJE
/JDFOJDF@MFWFM
FY
&OWJSPONFOU-%@13&-0"%QBUIUPNBMMPDMJCSBSZ
&OWJSPONFOU5;UJNF@[POF@TFUUJOH
4. Reload TZTUFNE:
TIFMM TVEPTZTUFNDUMEBFNPOSFMPBE
5. For temporary changes, you can reload without editing the DPOG file:
TIFMM TVEPTZTUFNDUMTFUFOWJSPONFOU.:42-%@0154HFOFSBM@MPH
PSVOTFUVTJOH
TIFMM TVEPTZTUFNDUMVOTFUFOWJSPONFOU.:42-%@0154
[ 28 ]

6. After modifying the TZTUFNE environment, restart the server to make the
changes effective.
Enable NZTRMTFSWJDFTIFMM TVEPTZTUFNDUM, and enable NZTRMTFSWJDF:
TIFMM TVEPTZTUFNDUMVONBTLNZTRMTFSWJDF
7. Restart NZTRM:
On RPM platforms:
TIFMM TVEPTZTUFNDUMSFTUBSUNZTRME
On Debian platforms:
TIFMM TVEPTZTUFNDUMSFTUBSUNZTRM
Downgrading from MySQL 8.0

If your application is not performing as expected, you can always downgrade to a previous
GA release (MySQL 5.7). Before downgrading, it is recommended to take a logical backup
(refer to $IBQUFS, Backups). Note that you can downgrade by only one previous release.
Suppose that you want to downgrade from MySQL 8.0 to MySQL 5.6, you have to
downgrade to MySQL 5.7, and then from MySQL 5.7 to MySQL 5.6.
You can do it in two ways:
In-place downgrade (downgrades within MySQL 8)

Logical downgrade
How to do it...
In the following subsections, you will be learning how to handle the
installation/uninstallation/upgrade/downgrade using various repositories, bundles, and so
on.
[ 29 ]

In-place Downgrades
For downgrades between the GA status releases within MySQL 8.0 (note that you cannot
downgrade to MySQL 5.7 using this method):
1. Shut down the old MySQL version

2. Replace the MySQL 8.0 binaries or older binaries
3. Restart MySQL on the existing EBUBEJSFDUPSZ
4. Run the NZTRM@VQHSBEF utility
6TJOH:6.SFQPTJUPSJFT
1. Prepare MySQL for a slow shutdown, which ensures that the undo logs are
empty and data files are fully prepared in case of file format differences between
releases:
NZTRM 4&5(-0#"-JOOPEC@GBTU@TIVUEPXO
2. Shut down the NZTRM server as described in the Stopping MySQL 8.0
Server section:
3. Remove the *OOP%# redo log files (the JC@MPHGJMF files) from the EBUB
EJSFDUPSZ to avoid downgrade issues related to redo log file format changes
that may have occurred between releases:
TIFMM TVEPSNSGWBSMJCNZTRMJC@MPHGJMF
4. Downgrade MySQL. To downgrade the server, you need to uninstall MySQL 8.0,
as described in the Uninstalling MySQL 8 section. The configuration files are
automatically stored as backup.
List the available versions:
TIFMM TVEPZVNMJTUNZTRMDPNNVOJUZTFSWFS
[ 30 ]

Downgrades are tricky; it is better to remove the existing packages before

downgrading:
OPEFQT
Install the older version:

TIFMM TVEPZVNJOTUBMMZNZTRMDPNNVOJUZTFSWFSWFSTJPO
6TJOH"153FQPTJUPSJFT
1. Reconfigure MySQL and choose the older version:
2. Run BQUHFUVQEBUF:
3. Remove the current version:
TIFMM TVEPBQUHFUBVUPSFNPWF
4. Install the older version (autoselected since you have reconfigured):
TIFMM TVEPBQUHFUJOTUBMMZNZTRMTFSWFS
6TJOHUIF31.PS"15CVOEMF
Uninstall the existing packages (refer to the Uninstalling MySQL 8 section) and install the
new packages, which can be downloaded from the MySQL Downloads (refer to the
Installing MySQL 8.0 using RPMs or DEB files section).
[ 31 ]

6TJOH(FOFSJD#JOBSJFT
If you have installed MySQL through binaries, you have to remove the symlink to the old
version (refer to the Uninstalling MySQL 8 section) and do a fresh installation (refer to
the Installing MySQL on Linux Using Generic Binaries section):
1. Start the server as described in the Starting or Stopping MySQL 8 section. Please
note that the start procedure is the same for all the versions.
2. Run the NZTRM@VQHSBEF utility:
TIFMM TVEPNZTRM@VQHSBEFVSPPUQ
3. Restart the MySQL server to ensure that any changes made to the system tables
take effect:
Logical Downgrades
Here is an outline of the steps:
1. Export existing data from the MySQL 8.0 version using logical backup (refer to
$IBQUFS, Backups for logical backup methods)
2. Install MySQL 5.7
3. Load the dump file into the MySQL 5.7 version (refer to $IBQUFS, Restoring
Data for restoring methods)
Here are the detailed steps:
1. You need to take logical backup of the database. (refer to $IBQUFS, Backups for a
quicker backup called NZEVNQFS):
TIFMM NZTRMEVNQVSPPUQBEEESPQUBCMFSPVUJOFTFWFOUT
BMMEBUBCBTFTGPSDF NZTRMTRM
2. Shut down the MySQL server as described in the Starting or Stopping MySQL 8
section.
3. Move the EBUBEJSFDUPSZ. Instead of restoring the SQL backup (in step 1), we
can move back the EBUBEJSFDUPSZ if you want to keep MySQL 8:
TIFMM TVEPNWWBSMJCNZTRMWBSMJCNZTRM
[ 32 ]

4. Downgrade MySQL. To downgrade the server, we need to uninstall MySQL 8.

The configuration files are automatically backed up.
6TJOH:6.3FQPTJUPSJFT
After the uninstallation, install the older version:
1. Switch the repositories:
TIFMM ZVNSFQPMJTUFOBCMFE]HSFQNZTRMDPNNVOJUZ

NZTRMDPOOFDUPSTDPNNVOJUZY@.Z42-$POOFDUPST$PNNVOJUZ

NZTRMUPPMTDPNNVOJUZY@.Z42-5PPMT$PNNVOJUZ


3. Downgrades are tricky; it is better to remove the existing packages before

downgrading:
OPEFQT
4. List the available versions:
TIFMM TVEPZVNMJTUNZTRMDPNNVOJUZTFSWFS
-PBEJOHNJSSPSTQFFETGSPNDBDIFEIPTUGJMF
CBTFNJSSPSSBDLTQBDFDPN
FQFMNJSSPSTEFWFMPPQFSDPN
FYUSBTDFOUPTTVXFEV
VQEBUFTNJSSPSTTZSJOHBOFUXPSLTOFU
"WBJMBCMF1BDLBHFT
NZTRMDPNNVOJUZTFSWFSY@FM
NZTRMDPNNVOJUZ
5. Install MySQL 5.7:
TIFMM TVEPZVNJOTUBMMZNZTRMDPNNVOJUZTFSWFS
[ 33 ]

6TJOH"153FQPTJUPSJFT
1. Reconfigure BQU to switch to MySQL 5.7:
4. Install MySQL 5.7:
TIFMM TVEPBQUHFUJOTUBMMZNZTRMTFSWFS
6TJOH31.PS"15CVOEMFT
new packages, which can be downloaded from MySQL Downloads (refer to the Installing
MySQL 8 using RPM or DEB files section).
version (refer to the Uninstalling MySQL 8 section) and do a fresh installation (refer to the
Installing MySQL on Linux using Generic Binaries section).
Once you have downgraded MySQL, you have to restore the backup and run
the NZTRM@VQHSBEF utility:
1. Start MySQL (refer to the Starting or Stopping MySQL 8 section). You need to reset
the password again.
2. Restore the backup (this may take a long time, depending up on the size of
backup). Refer to $IBQUFS, Restoring Data, for a quick restoration method called
NZMPBEFS:
TIFMM NZTRMVSPPUQNZTRMTRM
[ 34 ]

3. Run NZTRM@VQHSBEF:
TIFMM NZTRM@VQHSBEFVSPPUQ
4. Restart the MySQL server to ensure that any changes made to the system tables
take effect. Refer to the Starting or Stopping MySQL 8 section:
TIFMM TVEPFUDJOJUENZTRMSFTUBSU
Upgrading to MySQL 8.0

MySQL 8 uses a global EBUBEJDUJPOBSZ containing information about database objects in
transactional tables. In previous versions, the dictionary data was stored in metadata files
and non-transactional system tables. You need to upgrade your EBUBEJSFDUPSZ from the
file-based structure to the data-dictionary structure.
Just like a downgrade, you can upgrade using two methods:
In-place upgrade
Logical upgrade
You also should check a few prerequisites before the upgrade.
Getting ready
1. Check for obsolete datatypes or triggers that have a missing or empty definer or
an invalid creation context:
TIFMM TVEPNZTRMDIFDLVSPPUQBMMEBUBCBTFTDIFDLVQHSBEF
2. There must be no partitioned tables that use a storage engine that does not have
native partitioning support. To identify these tables, execute this query:
TIFMM 4&-&$55"#-&@4$)&."5"#-&@/".&'30.
*/'03."5*0/@4$)&."5"#-&48)&3&&/(*/&/05*/
JOOPEC
OECDMVTUFS "/%$3&"5&@015*0/4-*,& QBSUJUJPOFE
[ 35 ]

If there are any of these tables, change them to *OOP%#:

NZTRM "-5&35"#-&UBCMF@OBNF&/(*/&*//0%#
Or remove the partitioning:

NZTRM "-5&35"#-&UBCMF@OBNF3&.07&1"35*5*0/*/(
3. There must be no tables in the MySQL 5.7 NZTRM system database that have the
same name as a table used by the MySQL 8.0 EBUBEJDUJPOBSZ. To identify
tables with those names, execute this query:
NZTRM 4&-&$55"#-&@4$)&."5"#-&@/".&'30.
*/'03."5*0/@4$)&."5"#-&48)&3&-08&3
5"#-&@4$)&." NZTRM BOE
-08&3
5"#-&@/".&*/
DBUBMPHT DIBSBDUFS@TFUT DPMMBUJPOT
DPMVNO@UZQF@FMFNFOUT DPMVNOT FWFOUT
GPSFJHO@LFZ@DPMVNO@VTBHF GPSFJHO@LFZT JOEFY@DPMVNO@VTBHF
JOEFY@QBSUJUJPOT JOEFY@TUBUT JOEFYFT
QBSBNFUFS@UZQF@FMFNFOUT QBSBNFUFST SPVUJOFT TDIFNBUB
TU@TQBUJBM@SFGFSFODF@TZTUFNT UBCMF@QBSUJUJPO@WBMVFT
UBCMF@QBSUJUJPOT UBCMF@TUBUT UBCMFT UBCMFTQBDF@GJMFT
UBCMFTQBDFT USJHHFST WFSTJPO WJFX@SPVUJOF@VTBHF
WJFX@UBCMF@VTBHF
4. There must be no tables that have foreign key constraint names longer than 64
characters. To identify tables with constraint names that are too long, execute this
query:
NZTRM 4&-&$5$0/453"*/5@4$)&."5"#-&@/".&$0/453"*/5@/".&'30.
*/'03."5*0/@4$)&."3&'&3&/5*"-@$0/453"*/548)&3&
-&/(5)
$0/453"*/5@/".&
5. Tables not supported by MySQL 8.0, such as OEC, should be moved to *OOP%#:
NZTRM "-5&35"#-&UBCMFOBNF&/(*/&*OOP%#
How to do it...
Just like the previous recipe, the following subsections will take you through the details
with various systems, bundles, and so on.
[ 36 ]

In-place upgrades
1. Shut down the old MySQL version.

2. Replace the old MySQL binaries or packages with the new ones (detailed steps
for different types of installation methods are covered).
3. Restart MySQL on the existing EBUBEJSFDUPSZ.
4. Run the NZTRM@VQHSBEF utility.
5. In the MySQL 5.7 server, if there are encrypted *OOP%# tablespaces, rotate the
LFZSJOH master key by executing this statement:
NZTRM "-5&3*/45"/$&305"5&*//0%#."45&3,&:
1. Configure your MySQL 5.7 server to perform a slow shutdown. With a slow
shutdown, *OOP%# performs a full purge and change buffer merge before
shutting down, which ensures that the undo logs are empty and the data files are
fully prepared in case of file format differences between releases.
This step is the most important because, without it, you will end up with the
following error:
<&3303>*OOP%#6QHSBEFBGUFSBDSBTIJTOPUTVQQPSUFE
This redo log was created with MySQL 5.7.18. Please follow the instructions at
IUUQEFWNZTRMDPNEPDSFGNBOFOVQHSBEJOHIUNM:
NZTRM 4&5(-0#"-JOOPEC@GBTU@TIVUEPXO
2. Shut down the MySQL server as described in the Starting or Stopping MySQL 8
section.
Upgrade the MySQL binaries or packages.
:6.CBTFETZTUFNT
1. Switch the repositories:
[ 37 ]

enabled: 16
EJTBCMFE
3. Run the yum update:
TIFMM TVEPZVNVQEBUFNZTRMTFSWFS
"15CBTFETZTUFNT
1. Reconfigure the BQU to switch to MySQl 8.0:
4. Install MySQL 8:
TIFMM TVEPBQUHFUJOTUBMMNZTRMTFSWFS
TIFMM TVEPBQUHFUJOTUBMMMJCNZTRMDMJFOU
6TJOH31.PS"15CVOEMFT
new packages, which can be downloaded from MySQL Downloads (refer to the Installing
MySQL 8.0 using RPM or DEB files section).
[ 38 ]

version (refer to the Uninstalling MySQL 8 section) and do a fresh installation (refer to the
Installing MySQL on Linux using generic binaries section).
Start the MySQL 8.0 server (refer to the Starting or Stopping MySQL 8 to start
MySQL section). If there are encrypted *OOP%# tablespaces, use the FBSMZQMVHJOMPBE
option to load the LFZSJOH plugin.
The server automatically detects whether EBUBEJDUJPOBSZ tables are present. If not, the
server creates them in the EBUBEJSFDUPSZ , populates them with metadata, and then
proceeds with its normal startup sequence. During this process, the server upgrades
metadata for all database objects, including databases, tablespaces, system and user tables,
views, and stored programs (stored procedures and functions, triggers, event scheduler
events). The server also removes files that previously were used for metadata storage. For
example, after upgrading, you will notice that your tables no longer have GSN files.
The server creates a directory named CBDLVQ@NFUBEBUB@ and moves the files used by
MySQL 5.7 into it. The server renames the FWFOU and QSPD tables to FWFOU@CBDLVQ@
and QSPD@CBDLVQ@. If this upgrade fails, the server reverts all changes to the EBUB
EJSFDUPSZ. In this case, you should remove all redo log files, start your MySQL 5.7 server
on the same EBUBEJSFDUPSZ, and fix the cause of any errors. Then, perform another slow
shutdown of the 5.7 server and start the MySQL 8.0 server to try again.
Run the NZTRM@VQHSBEF utility:

NZTRM@VQHSBEF examines all tables in all databases for incompatibilities with the current
version of MySQL. It makes any remaining changes required in the NZTRM system database
between MySQL 5.7 and MySQL 8.0, so that you can take advantage of new privileges or
capabilities. NZTRM@VQHSBEF also brings the performance schema, */'03."5*0/@4$)&.",
and TZTTDIFNB objects up to date for MySQL 8.0.
Restart the MySQL server (refer to the Starting or Stopping MySQL 8 to start MySQL section).
[ 39 ]

Logical Upgrades
1. Export existing data from the old MySQL version using NZTRMEVNQ
2. Install the new MySQL version
3. Load the dump file into the new MySQL version
1. You need to take a logical backup of the database (refer to $IBQUFS, Backups for
a quicker backup called NZEVNQFS):
TIFMM NZTRMEVNQVSPPUQBEEESPQUBCMFSPVUJOFTFWFOUT
BMMEBUBCBTFTJHOPSFUBCMFNZTRMJOOPEC@UBCMF@TUBUTJHOPSF
UBCMFNZTRMJOOPEC@JOEFY@TUBUTGPSDF EBUBGPSVQHSBEFTRM
2. Shut down the MySQL server (refer to the Starting or Stopping MySQL 8 section).
3. Install the new MySQL version (refer to the methods mentioned in the In-place
upgrades section).
4. Start the MySQL server (refer to the Starting or Stopping MySQL 8 section).
5. Reset the temporary SPPU password:
TIFMM NZTRMVSPPUQ
&OUFSQBTTXPSE
FOUFSUFNQPSBSZSPPUQBTTXPSEGSPNFSSPSMPH
NZTRM "-5&364&364&3
*%&/5*'*&%#: ZPVSOFXQBTTXPSE
6. Restore the backup (this may take a long time depending up on the size of the
backup). Refer to $IBQUFS, Restoring Data for a quick restoration method called
NZMPBEFS:
TIFMM NZTRMVSPPUQGPSDFEBUBGPSVQHSBEFTRM
7. Run the NZTRM@VQHSBEF utility:
8. Restart the MySQL server (refer to the Starting or Stopping MySQL 8 section).
[ 40 ]

Installing MySQL utilities

MySQL utilities gives you very handy tools to smoothly carry out day-to-day operations
without much manual effort.
How to do it...
It can be installed on YUM-based and APT-based systems in the following manner. Let's
take a look.
On YUM-based systems
Download the files from the MySQL downloads page, IUUQTEFWNZTRMDPNEPXOMPBET
VUJMJUJFT, by selecting Red Hat Enterprise Linux/Oracle Linux, or directly from this link,
using XHFU:
TIFMM XHFU
IUUQTDEONZTRMDPN%PXOMPBET.Z42-(6*5PPMTNZTRMVUJMJUJFTFM
OPBSDISQN
TIFMM TVEPZVNMPDBMJOTUBMMZNZTRMVUJMJUJFTFMOPBSDISQN
On APT-based systems
Download the files from the MySQL Downloads page, IUUQTEFWNZTRMDPN
EPXOMPBETVUJMJUJFT, by selecting Ubuntu Linux, or directly from this link, using XHFU:
TIFMM XHFU
IUUQTDEONZTRMDPN%PXOMPBET.Z42-(6*5PPMTNZTRMVUJMJUJFT@VCVO
UV@BMMEFC
TIFMM TVEPEQLHJNZTRMVUJMJUJFT@VCVOUV@BMMEFC
TIFMM TVEPBQUHFUJOTUBMMG
[ 41 ]

Using MySQL
2
Connecting to MySQL using the command-line client

Creating databases
Creating tables
Inserting, updating, and deleting rows
Loading sample data
Selecting data
Sorting results
Grouping results (aggregate functions)
Creating users
Granting and revoking access to users
Selecting data into a file and table
Loading data into a table
Joining tables
Stored procedures
Functions
Triggers
Views
Events
Getting information about databases and tables

Using MySQL Chapter 2
Introduction
We are going to learn a lot of things in the following recipes. Let's take a look at each one in
detail.
Connecting to MySQL using the command-

line client
So far, you have learned how to install MySQL 8.0 on various platforms. Along with the
installation, you will get the command-line client utility called NZTRM, which we use to
connect to any MySQL server.
Getting ready
First you need to know to which server you need to connect. If you have the MySQL server
installed on one host and you are trying to connect to the server from a different host
(usually called client), you should specify the hostname or IP address of the server and
the NZTRMDMJFOU package should be installed on the client. In the previous chapter, you
installed both MySQL server and client packages. If you are already on the server (through
SSH), you can specify MPDBMIPTU, , or .
Second, since you are connected to the server, the next thing you need to specify is to which
port you want to connect on the server. By default, MySQL runs on port . So, you
should specify .
Now you know where to connect. The next obvious thing is the username and password to
log in to the server. You have not created any users yet, so use the root user to connect.
While installing, you would have supplied a password, use that to connect. In case you
changed it, use the new password.
[ 43 ]

How to do it...
Connecting to the MySQL client can be done with any of the following commands:
TIFMM NZTRMIMPDBMIPTU1VVTFSOBNF QQBTTXPSE
TIFMM NZTRMIPTUMPDBMIPTUQPSUVTFSSPPUQBTTXPSEQBTTXPSE
TIFMM NZTRMIPTUMPDBMIPTUQPSUVTFSSPPUQBTTXPSEQBTTXPSE
It is highly recommended not to give the password in the command line, instead you can
leave the field blank; you will be prompted for a password:
TIFMM NZTRMIPTUMPDBMIPTUQPSUVTFSSPPUQBTTXPSE
&OUFS1BTTXPSE
1. The 1 argument (in uppercase) is passed for specifying the port.

2. The Q argument (in lowercase) is passed for specifying the password.
3. There is no space after the Q argument.
4. For the password, there is no space after .
By default, the host is taken as MPDBMIPTU, the port is taken as , and the user is taken
as the current shell user.
1. To know the current user:
TIFMM XIPBNJ
2. To disconnect, press Ctrl + D or type FYJU:
NZTRM ?%#ZF
TIFMM
Or use:
NZTRM FYJU
#ZF
TIFMM
[ 44 ]

3. After connecting to the NZTRM prompt, you can execute statements followed by
the delimiter. The default delimiter is a semicolon ():
NZTRM 4&-&$5

]]

]]

SPXJOTFU
TFD
Ś. To cancel a command, press Ctrl + C or type =D:
NZTRM 4&-&$5?$
NZTRM 4&-&$5=D
Connecting to MySQL using the root user is not recommended. You can
create users and restrict the user by granting appropriate privileges, which
will be discussed in the Creating Users and Granting and revoking access to
users sections. Till then, you can use the root user to connect to MySQL.
See also
After connecting, you might have noticed a warning:
8BSOJOH6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBOCFJOTFDVSF
To learn about the secure ways of connecting, refer to $IBQUFS, Security.
Once you are connected to the command-line prompt, you can execute the
SQL statements, which can be terminated by , =H, or =(.
or =Hgoutput is displayed horizontally, =(goutput is displayed
vertically.
Creating databases
Well, you have installed MySQL 8.0 and connected to it. Now it is time to store some data in
it, that's what the database is meant for, after all. In any relational database management
system (RDBMS), data is stored in rows, which is the basic building block of the database.
Rows contain columns in which we can store several set of values.
For example, if you want to store information about your customers in a database.
[ 45 ]

Here is the dataset:

DVTUPNFSJEGJSTU@OBNF.JLFMBTU@OBNF$ISJTUFOTFODPVOUSZ64"
DVTUPNFSJEGJSTU@OBNF"OEZMBTU@OBNF)PMMBOETDPVOUSZ"VTUSBMJB
DVTUPNFSJEGJSTU@OBNF3BWJMBTU@OBNF7FEBOUBNDPVOUSZ*OEJB
DVTUPNFSJEGJSTU@OBNF3BKJWMBTU@OBNF1FSFSBDPVOUSZ4SJ-BOLB
You should save them as rows:

.JLF $ISJTUFOTFO 64" ,
"OEZ
)PMMBOET "VTUSBMJB ,
3BWJ 7FEBOUBN *OEJB ,
3BKJW
1FSFSB 4SJ-BOLB . For this dataset, there are four rows described by three
columns
JEGJSTU@OBNFMBTU@OBNFBOEDPVOUSZ, which are stored in a table. The
number of columns that a table can hold should be defined at the time of the creation of the
table, which is the major limitation of RDBMS. However, we can alter the definition of the
table any time, but the full table should be rebuilt while doing so. In some cases, the table
will be unavailable while doing an alter. Altering a table will be discussed in detail in
$IBQUFS, Table Maintenance.
A database is a collection of many tables, and a database server can hold many of these
databases. The flow is as follows:
Database Server g> Databases g> Tables (defined by columns) g> Rows
Databases and tables are referred to as database objects. Any operation, such as creating,
modifying, or deleting database objects, is called Data Definition Language (DDL).
The organization of data as a blueprint of how the database is constructed (divided into
database and tables) is called a schema.
How to do it...
Connect to the MySQL server:
TIFMM NZTRMVSPPUQ
&OUFS1BTTXPSE
NZTRM $3&"5&%"5"#"4&DPNQBOZ
NZTRM $3&"5&%"5"#"4&ANZDPOUBDUTA
The back tick character (A) is used to quote identifiers, such as database and table names.
You need to use it when the database name contains special characters, such as dot ().
You can switch between databases:

NZTRM 64&DPNQBOZ
NZTRM 64&ANZDPOUBDUTA
[ 46 ]

Instead of switching, you can directly connect to the database you want by specifying it in
the command line:
TIFMM NZTRMVSPPUQDPNQBOZ
To find which database you are connected to, use the following:
NZTRM 4&-&$5%"5"#"4&

]%"5"#"4&
]

]DPNQBOZ]

SPXJOTFU
TFD
To find all the databases you have access to, use:

NZTRM 4)08%"5"#"4&4

]%BUBCBTF]

]DPNQBOZ]
]NZDPOUBDUT]
]JOGPSNBUJPO@TDIFNB]
]NZTRM]
]QFSGPSNBODF@TDIFNB]
]TZT]

SPXTJOTFU
TFD
The database is created as a directory inside EBUBEJSFDUPSZ. The default

EBUBEJSFDUPSZ is WBSMJCNZTRM for repository-based installation and
VTSMPDBMNZTRMEBUB for installation through binaries. To know your current EBUB
EJSFDUPSZ, you can execute:
NZTRM 4)087"3*"#-&4-*,& EBUBEJS

]7BSJBCMF@OBNF]7BMVF]

]EBUBEJS]VTSMPDBMNZTRMEBUB]

SPXJOTFU
TFD
[ 47 ]

Check the files inside EBUBEJSFDUPSZ :

TIFMM TVEPMTMIUSVTSMPDBMNZTRMEBUB
UPUBM.
SXSNZTRMNZTRM+VOBVUPDOG
SXSNZTRMNZTRM+VOQFSGPSNBODF@TDIF@4%*
ESXYSYNZTRMNZTRM,+VOQFSGPSNBODF@TDIFNB
ESXYSYNZTRMNZTRM,+VONZTRM
SXSNZTRMNZTRM+VOTZT@4%*
ESXYSYNZTRMNZTRM,+VOTZT
SXNZTRMSPPU,+VODBLFZQFN
SXSSNZTRMSPPU,+VODBQFN
SXNZTRMSPPU,+VOTFSWFSLFZQFN
SXSSNZTRMSPPU,+VOTFSWFSDFSUQFN
SXNZTRMSPPU,+VODMJFOULFZQFN
SXSSNZTRMSPPU,+VODMJFOUDFSUQFN
SXNZTRMSPPU,+VOQSJWBUF@LFZQFN
SXSSNZTRMSPPU+VOQVCMJD@LFZQFN
SXSNZTRMNZTRM,+VOJC@CVGGFS@QPPM
SXSNZTRMNZTRM+VOTFSWFSQJE
SXSNZTRMNZTRM+VODPNQBOZ@4%*
ESXYSYNZTRMNZTRM,+VODPNQBOZ
SXSNZTRMNZTRM,+VOTFSWFSFSS
SXSNZTRMNZTRM+VOFNQMPZFFT@4%*
ESXYSYNZTRMNZTRM,+VOFNQMPZFFT
SXSNZTRMNZTRM.+VOJCEBUB
SXSNZTRMNZTRM.+VOJC@MPHGJMF
SXSNZTRMNZTRM.+VOJC@MPHGJMF
SXSNZTRMNZTRM.+VOJCUNQ
See also
You might be wondering about other files and directories, such
as JOGPSNBUJPO@TDIFNB and QFSGPSNBODF@TDIFNB, which you have not
created. JOGPSNBUJPO@TDIFNB will be discussed in the Getting information about databases
and tables section and QFSGPSNBODF@TDIFNB will be discussed in $IBQUFS, Performance
Tuning, in the Using performance_schema section.
[ 48 ]

Creating tables
While defining columns in a table, you should mention the name of the column, datatype
(integer, floating point, string, and so on), and default value (if any). MySQL supports
various datatypes. Refer to the MySQL documentation for more details (IUUQTEFW
NZTRMDPNEPDSFGNBOFOEBUBUZQFTIUNM). Here is an overview of all datatypes.
The +40/ datatype is a new extension, which will be discussed in $IBQUFS, Using MySQL
(Advanced):
1. Numeric: 5*/:*/5, 4."--*/5, .&%*6.*/5, */5, #*(*/5, and #*5.

2. Floating numbers: %&$*."-, '-0"5, and %06#-&.
3. Strings: $)"3, 7"3$)"3, #*/"3:, 7"3#*/"3:, #-0#, 5&95, &/6., and 4&5.
4. Spatial datatypes are also supported. Refer to IUUQTEFWNZTRMDPNEPD
SFGNBOFOTQBUJBMFYUFOTJPOTIUNM for more details.
5. The +40/ datatype - discussed in detail in the next chapter.
You can create many tables inside a database.
How to do it...
The table contains the column definition:
NZTRM $3&"5&5"#-&*'/05&9*454ADPNQBOZAADVTUPNFSTA
AJEAJOUVOTJHOFE"650@*/$3&.&/513*."3:,&:
AGJSTU@OBNFAWBSDIBS

AMBTU@OBNFAWBSDIBS

ADPVOUSZAWBSDIBS

&/(*/&*OOP%#
The options are explained as follows:
Dot notation: Tables can be referenced using database name dot table name
(EBUBCBTFUBCMF). If you are connected to the database, you can simply use
DVTUPNFST instead of DPNQBOZDVTUPNFST.
*'/05&9*454: If a table with the same name exists and you specify this clause,
MySQL simply throws a warning that the table already exists. Otherwise, MySQL
will throw an error.
JE: It is declared as an integer since it contains only integers. Along with that,
there are two key words: "650@*/$3&.&/5 and 13*."3:,&:.
[ 49 ]

"650@*/$3&.&/5: A linearly incremental sequence is automatically generated, so

you do not need to worry about assigning JE to each row.
13*."3:,&:: Each row is identified by a 6/*26& column that is /05/6--.
Only one of these columns should be defined in a table. If a table contains
an "650@*/$3&.&/5 column, it is taken as 13*."3:,&:.
GJSTU@OBNF, MBTU@OBNF, and DPVOUSZ: They contain strings, so they are
defined as WBSDIBS.
Engine: Along with the column definition, you should mention the storage
engine. Some types of storage engines include *OOP%#, .Z*4"., '&%&3"5&%,
#-"$,)0-&, $47, and .&.03:. Out of all the engines, *OOP%# is the only
transactional engine and it is the default engine. To learn more about
transactions, refer to $IBQUFS, Transactions.
To list all the storage engines, execute the following:

NZTRM 4)08&/(*/&4=(
SPX
&OHJOF.3(@.:*4".
4VQQPSU:&4
$PNNFOU$PMMFDUJPOPGJEFOUJDBM.Z*4".UBCMFT
5SBOTBDUJPOT/0
9"/0
4BWFQPJOUT/0
SPX
&OHJOF'&%&3"5&%
4VQQPSU/0
$PNNFOU'FEFSBUFE.Z42-TUPSBHFFOHJOF
5SBOTBDUJPOT/6--
9"/6--
4BWFQPJOUT/6--
SPX
&OHJOF*OOP%#
4VQQPSU%&'"6-5
$PNNFOU4VQQPSUTUSBOTBDUJPOTSPXMFWFMMPDLJOHBOEGPSFJHOLFZT
5SBOTBDUJPOT:&4
9":&4
4BWFQPJOUT:&4
SPX
&OHJOF#-"$,)0-&
4VQQPSU:&4
$PNNFOUEFWOVMMTUPSBHFFOHJOF
BOZUIJOHZPVXSJUFUPJU
EJTBQQFBST
5SBOTBDUJPOT/0
9"/0
[ 50 ]

4BWFQPJOUT/0
SPX
&OHJOF$47
4VQQPSU:&4
$PNNFOU$47TUPSBHFFOHJOF
5SBOTBDUJPOT/0
9"/0
4BWFQPJOUT/0
SPX
&OHJOF.&.03:
4VQQPSU:&4
$PNNFOU)BTICBTFETUPSFEJONFNPSZVTFGVMGPSUFNQPSBSZUBCMFT
5SBOTBDUJPOT/0
9"/0
4BWFQPJOUT/0
SPX
&OHJOF1&3'03."/$&@4$)&."
4VQQPSU:&4
$PNNFOU1FSGPSNBODF4DIFNB
5SBOTBDUJPOT/0
9"/0
4BWFQPJOUT/0
SPX
&OHJOF"3$)*7&
4VQQPSU:&4
$PNNFOU"SDIJWFTUPSBHFFOHJOF
5SBOTBDUJPOT/0
9"/0
4BWFQPJOUT/0
SPX
&OHJOF.Z*4".
4VQQPSU:&4
$PNNFOU.Z*4".TUPSBHFFOHJOF
5SBOTBDUJPOT/0
9"/0
4BWFQPJOUT/0
SPXTJOTFU
TFD
You can create many tables in a database.
Create one more table to track the payments:

NZTRM $3&"5&5"#-&ADPNQBOZAAQBZNFOUTA
ADVTUPNFS@OBNFAWBSDIBS
13*."3:,&:
AQBZNFOUAGMPBU

[ 51 ]

To list all the tables, use:

NZTRM 4)085"#-&4

]5BCMFT@JO@DPNQBOZ]

]DVTUPNFST]
]QBZNFOUT]

SPXTJOTFU
TFD
To see the structure of the table, execute the following:

NZTRM 4)08$3&"5&5"#-&DVTUPNFST=(
SPX
5BCMFDVTUPNFST
$SFBUF5BCMF$3&"5&5"#-&ADVTUPNFSTA
AJEAJOU
VOTJHOFE/05/6--"650@*/$3&.&/5
AGJSTU@OBNFAWBSDIBS
%&'"6-5/6--
AMBTU@OBNFAWBSDIBS
%&'"6-5/6--
ADPVOUSZAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AJEA
&/(*/&*OOP%#"650@*/$3&.&/5%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
Or use this:
NZTRM %&4$DVTUPNFST

]'JFME]5ZQF]/VMM],FZ]%FGBVMU]&YUSB]

]JE]JOU
VOTJHOFE]/0]13*]/6--]BVUP@JODSFNFOU]
]GJSTU@OBNF]WBSDIBS
]:&4]]/6--]]
]MBTU@OBNF]WBSDIBS
]:&4]]/6--]]
]DPVOUSZ]WBSDIBS
]:&4]]/6--]]

SPXTJOTFU
TFD
MySQL creates JCE files inside the EBUBEJSFDUPSZ :

TIFMM TVEPMTMIUSVTSMPDBMNZTRMEBUBDPNQBOZ
UPUBM,
SXSNZTRMNZTRM,+VODVTUPNFSTJCE
SXSNZTRMNZTRM,+VOQBZNFOUTJCE
[ 52 ]

Cloning table structure

You can clone the structure of one table into a new table:
NZTRM $3&"5&5"#-&OFX@DVTUPNFST-*,&DVTUPNFST
2VFSZ0,SPXTBGGFDUFE
TFD
You can verify the structure of the new table:

NZTRM 4)08$3&"5&5"#-&OFX@DVTUPNFST=(
SPX
5BCMFOFX@DVTUPNFST
$SFBUF5BCMF$3&"5&5"#-&AOFX@DVTUPNFSTA
AJEAJOU
VOTJHOFE/05/6--"650@*/$3&.&/5
AGJSTU@OBNFAWBSDIBS
%&'"6-5/6--
AMBTU@OBNFAWBSDIBS
%&'"6-5/6--
ADPVOUSZAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AJEA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
See also
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFODSFBUFUBCMFIUNM for many other
options in $SFBUF5BCMF. Partitioning a table and compressing a table will be discussed in
$IBQUFS, Table Maintenance and $IBQUFS, Managing Tablespace, respectively.
Inserting, updating, and deleting rows

The */4&35, 61%"5&, %&-&5&, and 4&-&$5 operations are called Data Manipulation
Language (DML) statements. */4&35, 61%"5&, and %&-&5& are also called write
operations, or simply write(s). 4&-&$5 is a read operation and is simply called read(s).
How to do it...
Let's look at each of them in detail. I am sure you will enjoy learning this. I would suggest
that you try a few things on your own as well, later. By the end of this recipe, we will also
have gotten to grips with truncating tables.
[ 53 ]

Inserting
The */4&35 statement is used to create new records in a table:
NZTRM */4&35*(/03&*/50ADPNQBOZAADVTUPNFSTA
GJSTU@OBNF
MBTU@OBNFDPVOUSZ
7"-6&4

.JLF $ISJTUFOTFO 64"

"OEZ )PMMBOET "VTUSBMJB

3BWJ 7FEBOUBN *OEJB

3BKJW 1FSFSB 4SJ-BOLB
Or you can explicitly mention the JE column, if you want to insert the specific JE:
NZTRM */4&35*(/03&*/50ADPNQBOZAADVTUPNFSTA
JEGJSTU@OBNF
MBTU@OBNFDPVOUSZ
7"-6&4

.JLF $ISJTUFOTFO 64"

"OEZ )PMMBOET "VTUSBMJB

3BWJ 7FEBOUBN *OEJB

3BKJW 1FSFSB 4SJ-BOLB
2VFSZ0,SPXTBGGFDUFEXBSOJOHT
TFD
3FDPSET%VQMJDBUFT8BSOJOHT
*(/03&: If the row already exists and the *(/03& clause is given, the new data is ignored
and the */4&35 statement still succeeds in producing a warning and a number of
duplicates. Otherwise, if the *(/03& clause is not given, the */4&35 statement produces an
error. The uniqueness of a row is identified by the primary key:
NZTRM 4)088"3/*/(4

]-FWFM]$PEF].FTTBHF]

]8BSOJOH]]%VQMJDBUFFOUSZ GPSLFZ 13*."3: ]

SPXTJOTFU
TFD
[ 54 ]

Updating
The 61%"5& statement is used to modify the existing records in a table:
NZTRM 61%"5&DVTUPNFST4&5GJSTU@OBNF 3BKJW DPVOUSZ 6, 8)&3&JE
2VFSZ0,SPXBGGFDUFE
TFD
3PXTNBUDIFE$IBOHFE8BSOJOHT
8)&3&: This is the clause used for filtering. Whatever condition(s) are issued after the 8)&3&
clause are evaluated and the filtered rows are updated.
The 8)&3& clause is mandatory. Failing to give it will 61%"5& the whole
table.
It is recommended to do data modification in a transaction, so that you can
easily rollback the changes if you find anything wrong. You can refer
to $IBQUFS, Transactions to learn more about transactions.
Deleting
Deleting a record can be done as follows:
NZTRM %&-&5&'30.DVTUPNFST8)&3&JE"/%GJSTU@OBNF 3BKJW
2VFSZ0,SPXBGGFDUFE
TFD
The 8)&3& clause is mandatory. Failing to give it will %&-&5& all the rows
of the table.
It is recommended to do data modification in a transaction, so that you can
easily rollback the changes if you find anything wrong.
REPLACE, INSERT, ON DUPLICATE KEY UPDATE

There are many cases where you need to handle the duplicates. The uniqueness of a row is
identified by the primary key. If a row already exists, 3&1-"$& simply deletes the row and
inserts the new row. If a row is not there, 3&1-"$& behaves as */4&35.
0/%61-*$"5&,&:61%"5& is used when you want to take action if the row already
exists. If you specify the 0/%61-*$"5&,&:61%"5& option and the */4&35 statement
causes a duplicate value in the 13*."3:,&:, MySQL performs an update to the old row
based on the new values.
[ 55 ]

Suppose you want to update the previous amount whenever you get payment from the
same customer and concurrently insert a new record if the customer is paying for the first
time. To do this, you will define an amount column and update it whenever a new payment
comes in:
NZTRM 3&1-"$&*/50DVTUPNFST7"-6&4
.JLF $ISJTUFOTFO "NFSJDB
2VFSZ0,SPXTBGGFDUFE
TFD
You can see that two rows are affected, one duplicate row is deleted and a new row is
inserted:
NZTRM */4&35*/50QBZNFOUT7"-6&4
.JLF$ISJTUFOTFO 0/%61-*$"5&
,&:61%"5&QBZNFOUQBZNFOU 7"-6&4
QBZNFOU
2VFSZ0,SPXBGGFDUFE
TFD
3BWJ7FEBOUBN 0/%61-*$"5&,&:
61%"5&QBZNFOUQBZNFOU 7"-6&4
QBZNFOU
2VFSZ0,SPXBGGFDUFE
TFD
When .JLF$ISJTUFOTFO pays $300 next time, this will update the row and add this
payment to the previous payment:
.JLF$ISJTUFOTFO 0/%61-*$"5&
,&:61%"5&QBZNFOUQBZNFOU 7"-6&4
QBZNFOU
2VFSZ0,SPXTBGGFDUFE
TFD
7"-6&4 (payment): refers to the value given in the */4&35 statement. Payment refers to the
column of the table.
Truncating tables
Deleting the whole table takes lot of time, as MySQL performs operations row by row. The
quickest way to delete all of the rows of a table (preserving the table structure) is to use
the 536/$"5&5"#-& statement.
Truncate is a DDL operation in MySQL, meaning once the data is truncated, it cannot be
rolled back:
NZTRM 536/$"5&5"#-&DVTUPNFST
2VFSZ0,SPXTBGGFDUFE
TFD
[ 56 ]

Loading sample data

You have created the schema (databases and tables) and some data (through */4&35,
61%"5&, and %&-&5&). To explain the further chapters, more data is needed. MySQL has
provided a sample FNQMPZFF database and a lot of data to play around with. In this
chapter, we will discuss how to get that data and store it in our database.
How to do it...
1. Download the zipped file:
TIFMM XHFU
IUUQTDPEFMPBEHJUIVCDPNEBUBDIBSNFSUFTU@EC[JQNBTUFS 0
NBTUFS[JQ
2. Unzip the file:
TIFMM VO[JQNBTUFS[JQ
3. Load the data:
TIFMM DEUFTU@ECNBTUFS
TIFMM NZTRMVSPPUQFNQMPZFFTTRM
CFJOTFDVSF
*/'0
$3&"5*/(%"5"#"4&4536$563&
*/'0
TUPSBHFFOHJOF*OOP%#
*/'0
-0"%*/(EFQBSUNFOUT
*/'0
-0"%*/(FNQMPZFFT
*/'0
-0"%*/(EFQU@FNQ
*/'0
-0"%*/(EFQU@NBOBHFS
*/'0
-0"%*/(UJUMFT
*/'0
-0"%*/(TBMBSJFT
EBUB@MPBE@UJNF@EJGG
/6--
[ 57 ]

4. Verify the data:
TIFMM NZTRMVSPPUQFNQMPZFFT"
CFJOTFDVSF
4FSWFSWFSTJPOSDMPH.Z42-$PNNVOJUZ4FSWFS
(1-
$PQZSJHIU
SFTFSWFE
PXOFST

TUBUFNFOU
NZTRM 4)085"#-&4

]5BCMFT@JO@FNQMPZFFT]

]DVSSFOU@EFQU@FNQ]
]EFQBSUNFOUT]
]EFQU@FNQ]
]EFQU@FNQ@MBUFTU@EBUF]
]EFQU@NBOBHFS]
]FNQMPZFFT]
]TBMBSJFT]
]UJUMFT]

SPXTJOTFU
TFD
NZTRM %&4$FNQMPZFFT=(
SPX
'JFMEFNQ@OP
5ZQFJOU

/VMM/0
,FZ13*
%FGBVMU/6--
&YUSB
SPX
'JFMECJSUI@EBUF
5ZQFEBUF
/VMM/0
,FZ
%FGBVMU/6--
&YUSB
SPX
[ 58 ]

'JFMEGJSTU@OBNF
5ZQFWBSDIBS

/VMM/0
,FZ
%FGBVMU/6--
&YUSB
SPX
'JFMEMBTU@OBNF
5ZQFWBSDIBS

/VMM/0
,FZ
%FGBVMU/6--
&YUSB
SPX
'JFMEHFOEFS
5ZQFFOVN
. '
/VMM/0
,FZ
%FGBVMU/6--
&YUSB
SPX
'JFMEIJSF@EBUF
5ZQFEBUF
/VMM/0
,FZ
%FGBVMU/6--
&YUSB
SPXTJOTFU
TFD
Selecting data
You have inserted and updated data in the tables. Now it is time to learn how to retrieve
information from the database. In this section, we will discuss how to retrieve data from the
sample FNQMPZFF database that we have created.
There are many things that you can do with 4&-&$5. The most common use cases will be
discussed in this section. For more details on syntax and other use cases, refer to IUUQT
EFWNZTRMDPNEPDSFGNBOFOTFMFDUIUNM.
[ 59 ]

How to do it...
Select all data from the EFQBSUNFOUT table of the FNQMPZFF database. You can use an
asterisk () to select all columns from a table. It is not recommended to use it, you should
always select only the data you need:
NZTRM 4&-&$5'30.EFQBSUNFOUT

]EFQU@OP]EFQU@OBNF]

]E]$VTUPNFS4FSWJDF]
]E]%FWFMPQNFOU]
]E]'JOBODF]
]E])VNBO3FTPVSDFT]
]E].BSLFUJOH]
]E]1SPEVDUJPO]
]E]2VBMJUZ.BOBHFNFOU]
]E]3FTFBSDI]
]E]4BMFT]

SPXTJOTFU
TFD
Selecting columns
Suppose you need FNQ@OP and EFQU@OP from EFQU@NBOBHFS:
NZTRM 4&-&$5FNQ@OPEFQU@OP'30.EFQU@NBOBHFS

]FNQ@OP]EFQU@OP]

]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
[ 60 ]

]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]
]]E]

SPXTJOTFU
TFD
Count
Find the count of employees from the FNQMPZFFT table:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT

]$06/5
]

]]

SPXJOTFU
TFD
Filter based on condition

Find FNQ@OP of employees with GJSTU@OBNF as (FPSHJ and MBTU@OBNF as 'BDFMMP:
NZTRM 4&-&$5FNQ@OP'30.FNQMPZFFT8)&3&GJSTU@OBNF (FPSHJ "/%
MBTU@OBNF 'BDFMMP

]FNQ@OP]

]]
]]

SPXTJOTFU
TFD
All the filtering conditions are given through the 8)&3& clause. Except integers and floating
points, everything else should be put inside quotes.
[ 61 ]

Operators
MySQL supports many operators for filtering results. Refer to IUUQTEFWNZTRMDPN
EPDSFGNBOFODPNQBSJTPOPQFSBUPSTIUNM for a list of all the operators. We will
discuss a few operators here. -*,& and 3-*,& are explained in detail in the next examples:
Equality: Refer to the preceding example where you have filtered using .
*/: Check whether a value is within a set of values.
For example, find the count of all employees whose last name is either
$ISJTU, -BNCB, or #BCB:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&MBTU@OBNF*/
$ISJTU
-BNCB #BCB

]$06/5
]

]]

SPXJOTFU
TFD
#&58&&/"/%: Check whether a value is within a range of values.

For example, find the number of employees who were hired in December 1986:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&IJSF@EBUF#&58&&/
"/%

]$06/5
]

]]

SPXJOTFU
TFD
/05: You can simply negate the results by preceding with the /05 operator.
For example, find the number of employees who were /05 hired in December
1986:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&IJSF@EBUF/05#&58&&/
"/%

]$06/5
]

]]

SPXJOTFU
TFD
[ 62 ]

Simple pattern matching

You can use the -*,& operator. Use underscore (@) for matching exactly one character. Use
for matching any number of characters.
Find the count of all employees whose first name starts with $ISJTU:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&GJSTU@OBNF-*,&
DISJTU

]$06/5
]

]]

SPXJOTFU
TFD
Find the count of all employees whose first name starts with $ISJTU and ends
with FE:
NZTRM 4&-&$5$06/5
DISJTUFE

]$06/5
]

]]

SPXJOTFU
TFD
Find the count of all employees whose first name contains TSJ:
NZTRM 4&-&$5$06/5
TSJ

]$06/5
]

]]

SPXJOTFU
TFD
[ 63 ]

Find the count of all employees whose first name ends with FS:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&GJSTU@OBNF-*,& FS

]$06/5
]

]]

SPXJOTFU
TFD
Find the count of all employees whose first name starts with any two characters
followed by LB and then followed by any number of characters:
NZTRM 4&-&$5$06/5
@@LB

]$06/5
]

]]

SPXJOTFU
TFD
Regular expressions
You can use regular expressions in the 8)&3& clause by using the 3-*,& or 3&(&91
operators. There are many ways to use 3&(&91, refer to IUUQTEFWNZTRMDPNEPD
SFGNBOFOSFHFYQIUNM for more examples:
Expression Description
Zero or more repetitions
One or more repetitions
Optional character
Any character
= Period
? Starts with
Ends with
<BCD> Only a, b, or c
<?BCD> Neither a, b, nor c
[ 64 ]

<B[> Characters a to z
<> Numbers 0 to 9
? Starts and ends
=E Any digit
=% Any non-digit character
=T Any whitespace
=4 Any non-whitespace character
=X Any alphanumeric character
=8 Any non-alphanumeric character
\N^ m repetitions
\NO^ m to n repetitions
Find the count of all employees whose first name starts with $ISJTU:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&GJSTU@OBNF3-*,&
?DISJTU

]$06/5
]

]]

SPXJOTFU
TFD
Find the count of all employees whose last name ends with CB:
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&MBTU@OBNF3&(&91 CB

]$06/5
]

]]

SPXJOTFU
TFD
[ 65 ]

Find the count of all employees whose last name does not contain vowels (a, e, i,
o, or u):
NZTRM 4&-&$5$06/5
'30.FNQMPZFFT8)&3&MBTU@OBNF/053&(&91
<BFJPV>

]$06/5
]

]]

SPXJOTFU
TFD
Limiting results
Select the names of any 10 employees whose IJSF@EBUF is before 1986. You can get this by
using the -*.*5 clause at the end of the statement:
NZTRM 4&-&$5GJSTU@OBNFMBTU@OBNF'30.FNQMPZFFT8)&3&IJSF@EBUF
-*.*5

]GJSTU@OBNF]MBTU@OBNF]

]#F[BMFM]4JNNFM]
]4VNBOU]1FBD]
]&CFSIBSEU]5FSLLJ]
]0UNBS])FSCTU]
]'MPSJBO]4ZSPUJVL]
]5TF])FSCFS]
]6EJ]+BOTDI]
]3FVWFO](BSJHMJBOP]
]&SF[]3JU[NBOO]
]1SFNBM]#BFL]

SPXTJOTFU
TFD
Using the table alias

By default, whatever column you have given in the 4&-&$5 clause will appear in the
results. In the previous examples, you have found out the count, but it is displayed as
$06/5
. You can change it by using the "4 alias:
NZTRM 4&-&$5$06/5
"4DPVOU'30.FNQMPZFFT8)&3&IJSF@EBUF#&58&&/
"/%

[ 66 ]

]DPVOU]

]]

SPXJOTFU
TFD
Sorting results
You can order the result based on the column or aliased column. You can be specify %&4$
for descending order or "4$ for ascending. By default, ordering will be ascending. You can
combine the -*.*5 clause with 03%&3#: to limit the results.
How to do it...
Find the employee IDs of the first five top-paid employees.
NZTRM 4&-&$5FNQ@OPTBMBSZ'30.TBMBSJFT03%&3#:TBMBSZ%&4$-*.*5

]FNQ@OP]TBMBSZ]

]]]
]]]
]]]
]]]
]]]

SPXTJOTFU
TFD
Instead of specifying the column name, you can also mention the position of the column in
the 4&-&$5 statement. For example, you are selecting the salary at the second position in
the 4&-&$5 statement. So, you can specify 03%&3#::
NZTRM 4&-&$5FNQ@OPTBMBSZ'30.TBMBSJFT03%&3#:%&4$-*.*5

]FNQ@OP]TBMBSZ]

]]]
]]]
]]]
]]]
]]]

SPXTJOTFU
TFD
[ 67 ]

Grouping results (aggregate functions)

You can group the results using the (3061#: clause on a column and then use "((3&("5&
functions, such as $06/5, ."9, .*/, and "7&3"(&. You can also use the function on a
column in a group by clause. See the 46. example where you will use the :&"3
function.
How to do it...
Each of the previously-mentioned aggregate functions will be introduced to you here in
detail.
COUNT
1. Find the count of male and female employees:
NZTRM 4&-&$5HFOEFS$06/5
"4DPVOU'30.FNQMPZFFT(3061#:
HFOEFS

]HFOEFS]DPVOU]

].]]
]']]

SPXTJOTFU
TFD
2. You want to find the 10 most common first names of the employees. You can use
(3061#:GJSTU@OBNF to group all the first names, then $06/5
GJSTU@OBNF
to find the count inside the group, and finally the 03%&3#: count to sort the
results. -*.*5 these results to the top 10:
NZTRM 4&-&$5GJSTU@OBNF$06/5
GJSTU@OBNF"4DPVOU'30.FNQMPZFFT
(3061#:GJSTU@OBNF03%&3#:DPVOU%&4$-*.*5

]GJSTU@OBNF]DPVOU]

]4IBIBC]]
]5FUTVTIJ]]
]&MHJO]]
]"OZVBO]]
])VJDBO]]
].BLF]]
]1BOBZPUJT]]
[ 68 ]

]4SFFLSJTIOB]]
])BUFN]]
](JSJ]]

SPXTJOTFU
TFD
SUM
Find the sum of the salaries given to employees in each year and sort the results by salary.
The :&"3
function returns the :&"3 of the given date:
NZTRM 4&-&$5 :&"3

]]:&"3
]

]]]

SPXJOTFU
TFD
NZTRM 4&-&$5:&"3
GSPN@EBUF46.
TBMBSZ"4TVN'30.TBMBSJFT(3061#:
:&"3
GSPN@EBUF03%&3#:TVN%&4$

]:&"3
GSPN@EBUF]TVN]

]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]

SPXTJOTFU
TFD
[ 69 ]

AVERAGE
Find the 10 employees with the highest average salaries:
NZTRM 4&-&$5FNQ@OP"7(
TBMBSZ"4BWH'30.TBMBSJFT(3061#:FNQ@OP
03%&3#:BWH%&4$-*.*5

]FNQ@OP]BWH]

]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]

SPXTJOTFU
TFD
DISTINCT
You can use the %*45*/$5 clause to filter the distinct entries in a table:
NZTRM 4&-&$5%*45*/$5UJUMF'30.UJUMFT

]UJUMF]

]4FOJPS&OHJOFFS]
]4UBGG]
]&OHJOFFS]
]4FOJPS4UBGG]
]"TTJTUBOU&OHJOFFS]
]5FDIOJRVF-FBEFS]
].BOBHFS]

SPXTJOTFU
TFD
Filtering using HAVING

You can filter results of the (3061#: clause by adding the )"7*/( clause.
[ 70 ]

For example, find the employees with an average salary of more than 140,000:
TBMBSZ"4BWH'30.TBMBSJFT(3061#:FNQ@OP
)"7*/(BWH 03%&3#:BWH%&4$

]FNQ@OP]BWH]

]]]
]]]

SPXTJOTFU
TFD
See also
There are many other aggregate functions, refer to IUUQTEFWNZTRMDPNEPDSFGNBO
FOHSPVQCZGVODUJPOTIUNM for more information.
Creating users
So far, you have used only the root user to connect to MySQL and execute statements. The
root user should never be used while accessing MySQL, except for administrative tasks
from MPDBMIPTU. You should create users, restrict the access, restrict the resource usage,
and so on. For creating new users, you should have the $3&"5&64&3 privilege that will be
discussed in the next section. During the initial set up, you can use the root user to create
other users.
How to do it...
Connect to mysql using the root user and execute $3&"5&64&3 command to create new
users.
NZTRM $3&"5&64&3*'/05&9*454 DPNQBOZ@SFBE@POMZ ! MPDBMIPTU
*%&/5*'*&%8*5)NZTRM@OBUJWF@QBTTXPSE
#: DPNQBOZ@QBTT
8*5)."9@26&3*&4@1&3@)063
."9@61%"5&4@1&3@)063
You might get the following error if the password is not strong.
&3303
)::PVSQBTTXPSEEPFTOPUTBUJTGZUIFDVSSFOUQPMJDZ
SFRVJSFNFOUT
[ 71 ]

The preceding statement will create users with:
6TFSOBNF: DPNQBOZ@SFBE@POMZ.
BDDFTTPOMZGSPN: MPDBMIPTU.
You can restrict the access to the IP range. For example: . By giving
, the user can access from any host.
QBTTXPSE: DPNQBOZ@QBTT.
VTJOHNZTRM@OBUJWF@QBTTXPSE (default) authentication.
You can also specify any pluggable authentication, such
as TIB@QBTTXPSE, -%"1, or Kerberos.
The NBYJNVNOVNCFSPGRVFSJFT the user can execute in an hour is 500.
The NBYJNVNOVNCFSPGVQEBUFT the user can execute in an hour is 100.
When a client connects to the MySQL server, it undergoes two stages:
1. Access controlgconnection verification

2. Access controlgrequest verification
During the connection verification, the server identifies the connection by the username and
the hostname from which it is connected. The server invokes the authentication plugin for
the user and verifies the password. It also checks whether the user is locked or not.
During the request verification stage, the server checks whether the user has sufficient
privileges for each operation.
In the preceding statement, you have to give the password in clear text, which can be
recorded in the command history file, )0.&NZTRM@IJTUPSZ. To avoid that, you can
compute the hash on your local server and directly specify the hashed string. The syntax for
it is the same, except NZTRM@OBUJWF@QBTTXPSE#: DPNQBOZ@QBTT changes to
NZTRM@OBUJWF@QBTTXPSE"4 IBTIFE@TUSJOH :
NZTRM 4&-&$51"44803%
DPNQBOZ@QBTT

]1"44803%
DPNQBOZ@QBTT ]

]&#%&#'%$"&#%#'''&$]

SPXJOTFUXBSOJOH
TFD
NZTRM $3&"5&64&3*'/05&9*454 DPNQBOZ@SFBE@POMZ ! MPDBMIPTU

*%&/5*'*&%8*5)NZTRM@OBUJWF@QBTTXPSE
[ 72 ]

"4 &#%&#'%$"&#%#'''&$
8*5)."9@26&3*&4@1&3@)063
."9@61%"5&4@1&3@)063
You can directly create users by granting privileges. Refer to the next
section on how to grant privileges. However, MySQL will deprecate this
feature in the next release.
See also
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFODSFBUFVTFSIUNM for more options
on creating users. More secure options, such as SSL, using other authentication methods
will be discussed in $IBQUFS, Security.
Granting and revoking access to users

You can restrict the user to access specific databases or tables and also only specific
operations, such as 4&-&$5, */4&35, and 61%"5&. For granting privileges to other users,
you should have the (3"/5 privilege.
How to do it...
During the initial setup, you can use the root user to grant privileges. You can also create an
administrative account to manage the users.
Granting privileges
Grant the 3&"%0/-:
4&-&$5 privileges to the DPNQBOZ@SFBE@POMZ user:
NZTRM (3"/54&-&$50/DPNQBOZ50

DPNQBOZ@SFBE@POMZ ! MPDBMIPTU
2VFSZ0,SPXTBGGFDUFE
TFD
The asterisk () represents all tables inside the database.
[ 73 ]

Grant the */4&35 privilege to the new DPNQBOZ@JOTFSU@POMZ user:
NZTRM (3"/5*/4&350/DPNQBOZ50

DPNQBOZ@JOTFSU@POMZ ! MPDBMIPTU *%&/5*'*&%#: YYYY
2VFSZ0,SPXTBGGFDUFEXBSOJOH
TFD
NZTRM 4)088"3/*/(4=(
SPX
-FWFM8BSOJOH
$PEF
.FTTBHF6TJOH(3"/5GPSDSFBUJOHOFXVTFSJTEFQSFDBUFEBOEXJMM
CFSFNPWFEJOGVUVSFSFMFBTF$SFBUFOFXVTFSXJUI$3&"5&64&3
TUBUFNFOU
SPXJOTFU
TFD
Grant the 83*5& privileges to the new DPNQBOZ@XSJUF user:
NZTRM (3"/5*/4&35%&-&5&61%"5&0/DPNQBOZ50

DPNQBOZ@XSJUF ! *%&/5*'*&%8*5)NZTRM@OBUJWF@QBTTXPSE"4
&#%&#'%$"&#%#'''&$
TFD
Restrict to a specific table. Restrict the FNQMPZFFT@SFBE@POMZ user to 4&-&$5

only from the FNQMPZFFT table:
NZTRM (3"/54&-&$50/FNQMPZFFTFNQMPZFFT50
FNQMPZFFT@SFBE@POMZ ! *%&/5*'*&%8*5)NZTRM@OBUJWF@QBTTXPSE"4
&#%&#'%$"&#%#'''&$
TFD
You can further restrict to specific columns. Restrict the FNQMPZFFT@SP user to
the GJSTU@OBNF and MBTU@OBNF columns of the FNQMPZFFT table:
NZTRM (3"/54&-&$5
GJSTU@OBNFMBTU@OBNF0/FNQMPZFFTFNQMPZFFT
50 FNQMPZFFT@SP ! *%&/5*'*&%8*5)NZTRM@OBUJWF@QBTTXPSE"4
&#%&#'%$"&#%#'''&$
TFD
Extending grants. You can extend the grants by executing the new grant. Extend
the privilege to the FNQMPZFFT@DPM@SP user to access the salary of the TBMBSJFT
table:
NZTRM (3"/54&-&$5
TBMBSZ0/FNQMPZFFTTBMBSJFT50
FNQMPZFFT@SP !
2VFSZ0,SPXTBGGFDUFE
TFD
[ 74 ]

Creating the 461&3 user. You need an administrative account to manage the
server. "-- signifies all privileges expect the (3"/5 privilege:
NZTRM $3&"5&64&3 ECBENJO ! *%&/5*'*&%8*5)

NZTRM@OBUJWF@QBTTXPSE#: %#!ENO
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/5"--0/50 ECBENJO !

2VFSZ0,SPXTBGGFDUFE
TFD
Granting the (3"/5 privilege. The user should have the (3"/5015*0/ privilege
to grant privileges to other users. You can extend the (3"/5 privilege to
the ECBENJO super user:
NZTRM (3"/5(3"/5015*0/0/50 ECBENJO !

2VFSZ0,SPXTBGGFDUFE
TFD
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOHSBOUIUNM for more privilege types.
Checking grants
You can check all the user's grants. Check grants for the FNQMPZFF@DPM@SP user:
NZTRM 4)08(3"/54'03 FNQMPZFFT@SP ! =(
SPX
(SBOUTGPSFNQMPZFFT@SP!(3"/564"(&0/50AFNQMPZFFT@SPA!AA
SPX
(SBOUTGPSFNQMPZFFT@SP!(3"/54&-&$5
AGJSTU@OBNFAAMBTU@OBNFA0/
AFNQMPZFFTAAFNQMPZFFTA50AFNQMPZFFT@SPA!AA
SPX
(SBOUTGPSFNQMPZFFT@SP!(3"/54&-&$5
ATBMBSZA0/
AFNQMPZFFTAATBMBSJFTA50AFNQMPZFFT@SPA!AA
Check grants for the ECBENJO user. You can see all the grants that are available to
the ECBENJO user:
NZTRM 4)08(3"/54'03 ECBENJO ! =(
SPX
(SBOUTGPSECBENJO!(3"/54&-&$5*/4&3561%"5&%&-&5&$3&"5&%301
3&-0"%4)65%08/130$&44'*-&3&'&3&/$&4*/%&9"-5&34)08%"5"#"4&4
461&3$3&"5&5&.103"3:5"#-&4-0$,5"#-&4&9&$65&3&1-*$"5*0/4-"7&
3&1-*$"5*0/$-*&/5$3&"5&7*&84)087*&8$3&"5&3065*/&"-5&33065*/&
$3&"5&64&3&7&/553*((&3$3&"5&5"#-&41"$&$3&"5&30-&%30130-&0/
50AECBENJOA!AA8*5)(3"/5015*0/
SPX
[ 75 ]

(SBOUTGPSECBENJO!(3"/5
#*/-0(@"%.*/$0//&$5*0/@"%.*/&/$3:15*0/@,&:@"%.*/(3061@3&1-*$"5*0/@"%.*/
3&1-*$"5*0/@4-"7&@"%.*/30-&@"%.*/4&5@64&3@*%4:45&.@7"3*"#-&4@"%.*/0/
50AECBENJOA!AA
SPXTJOTFU
TFD
Revoking grants
Revoking grants has the same syntax as creating grants. You grant a privilege 50 the user
and revoke a privilege '30. the user.
Revoke the %&-&5& access from the DPNQBOZ@XSJUF ! user:
NZTRM 3&70,&%&-&5&0/DPNQBOZ'30. DPNQBOZ@XSJUF !

2VFSZ0,SPXTBGGFDUFE
TFD
Revoke the access to the salary column from the FNQMPZFF@SP user:
NZTRM 3&70,&4&-&$5
TBMBSZ0/FNQMPZFFTTBMBSJFT'30.
FNQMPZFFT@SP !
2VFSZ0,SPXTBGGFDUFE
TFD
Modifying the mysql.user table

All the user information, along with privileges, is stored in the NZTRMVTFS table. If you
have the privilege to access the NZTRMVTFS table, you can directly modify the NZTRMVTFS
table to create users and grant privileges.
If you modify the grant tables indirectly, using account-management statements such as
(3"/5, 3&70,&, 4&51"44803%, or 3&/".&64&3, the server notices these changes and
loads the grant tables into memory again immediately.
If you modify the grant tables directly, using statements such as */4&35, 61%"5&, or
%&-&5&, your changes have no effect on privilege checking until you either restart the
server or tell it to reload the tables. If you change the grant tables directly but forget to
reload them, your changes have no effect until you restart the server.
The reloading of the (3"/5 tables can be done by issuing a '-64)13*7*-&(&4 statement.
[ 76 ]

Query the NZTRMVTFS table to find out all the entries for the ECBENJO user:
NZTRM 4&-&$5'30.NZTRMVTFS8)&3&VTFS ECBENJO =(
SPX
)PTU
6TFSECBENJO
4FMFDU@QSJW:
*OTFSU@QSJW:
6QEBUF@QSJW:
%FMFUF@QSJW:
$SFBUF@QSJW:
%SPQ@QSJW:
3FMPBE@QSJW:
4IVUEPXO@QSJW:
1SPDFTT@QSJW:
'JMF@QSJW:
(SBOU@QSJW:
3FGFSFODFT@QSJW:
*OEFY@QSJW:
"MUFS@QSJW:
4IPX@EC@QSJW:
4VQFS@QSJW:
$SFBUF@UNQ@UBCMF@QSJW:
-PDL@UBCMFT@QSJW:
&YFDVUF@QSJW:
3FQM@TMBWF@QSJW:
3FQM@DMJFOU@QSJW:
$SFBUF@WJFX@QSJW:
4IPX@WJFX@QSJW:
$SFBUF@SPVUJOF@QSJW:
"MUFS@SPVUJOF@QSJW:
$SFBUF@VTFS@QSJW:
&WFOU@QSJW:
5SJHHFS@QSJW:
$SFBUF@UBCMFTQBDF@QSJW:
TTM@UZQF
TTM@DJQIFS
Y@JTTVFS
Y@TVCKFDU
NBY@RVFTUJPOT
NBY@VQEBUFT
NBY@DPOOFDUJPOT
NBY@VTFS@DPOOFDUJPOT
QMVHJONZTRM@OBUJWF@QBTTXPSE
BVUIFOUJDBUJPO@TUSJOH"#"%%$#&%#&###'&$&%
QBTTXPSE@FYQJSFE/
QBTTXPSE@MBTU@DIBOHFE
[ 77 ]

QBTTXPSE@MJGFUJNF/6--
BDDPVOU@MPDLFE/
$SFBUF@SPMF@QSJW:
%SPQ@SPMF@QSJW:
SPXJOTFU
TFD
You can see that the ECBENJO user can access the database from any host (%). You can
restrict them to MPDBMIPTU just by updating the NZTRMVTFS table and reloading the grant
tables:
NZTRM 61%"5&NZTRMVTFS4&5IPTU MPDBMIPTU 8)&3&VTFS ECBENJO
2VFSZ0,SPXBGGFDUFE
TFD
NZTRM '-64)13*7*-&(&4
2VFSZ0,SPXTBGGFDUFE
TFD
Setting password expiry for users

You can expire the passwords of users for a specific interval; after this, they need to change
their password.
When an application developer asks for database access, you can create the account with a
default password and then set it to expire. You can share the password with the developers,
then they have to change the password to continue using MySQL.
All the accounts are created with a password expiry equal to

the EFGBVMU@QBTTXPSE@MJGFUJNF variable, which is disabled by default:
Create a user with an expired password. When the developer logs in for the first
time and tries to execute any statement, &3303
): is thrown. The
password must be reset using the "-5&364&3 statement before executing this
statement:
NZTRM $3&"5&64&3 EFWFMPQFS ! *%&/5*'*&%8*5)

NZTRM@OBUJWF@QBTTXPSE"4
&#%&#'%$"&#%#'''&$ 1"44803%&91*3&
2VFSZ0,SPXTBGGFDUFE
TFD
TIFMM NZTRMVEFWFMPQFSQDPNQBOZ@QBTT
CFJOTFDVSF
4FSWFSWFSTJPOSDMPH
[ 78 ]

$PQZSJHIU
SFTFSWFE
PXOFST

TUBUFNFOU
NZTRM 4)08%"5"#"4&4
&3303
)::PVNVTUSFTFUZPVSQBTTXPSEVTJOH"-5&364&3
TUBUFNFOUCFGPSFFYFDVUJOHUIJTTUBUFNFOU
The developer has to change their password using the following command:
NZTRM "-5&364&3 EFWFMPQFS ! *%&/5*'*&%8*5)NZTRM@OBUJWF@QBTTXPSE#:
OFX@DPNQBOZ@QBTT
2VFSZ0,SPXTBGGFDUFE
TFD
Manually expire the existing user:
NZTRM "-5&364&3 EFWFMPQFS ! 1"44803%&91*3&

2VFSZ0,SPXTBGGFDUFE
TFD
Require the password to be changed every 180 days:
NZTRM "-5&364&3 EFWFMPQFS ! 1"44803%&91*3&*/5&37"-%":

2VFSZ0,SPXTBGGFDUFE
TFD
Locking users
If you find any issues with the account, you can lock it. MySQL supports locking while
using $3&"5&64&3 or "-5&364&3.
Lock the account by adding the "$$06/5-0$, clause to the "-5&364&3 statement:
NZTRM "-5&364&3 EFWFMPQFS ! "$$06/5-0$,
2VFSZ0,SPXTBGGFDUFE
TFD
[ 79 ]

The developer will get an error saying that the account is locked:
TIFMM NZTRMVEFWFMPQFSQOFX@DPNQBOZ@QBTT
NZTRM<8BSOJOH>6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBOCF
JOTFDVSF
&3303
):"DDFTTEFOJFEGPSVTFS EFWFMPQFS ! MPDBMIPTU "DDPVOU
JTMPDLFE
You can unlock the account after confirming:

NZTRM "-5&364&3 EFWFMPQFS ! "$$06/56/-0$,
2VFSZ0,SPXTBGGFDUFE
TFD
Creating roles for users

A MySQL role is a named collection of privileges. Like user accounts, roles can have
privileges granted to and revoked from them. A user account can be granted roles, which
grants to the account the role privileges. Earlier, you created separate users for reads,
writes, and administration. For write privilege, you have granted */4&35, %&-&5&, and
61%"5& to the user. Instead, you can grant those privileges to a role and then assign the user
to that role. By this way, you can avoid granting privileges individually to possibly many
user accounts.
Creating roles:
NZTRM $3&"5&30-& BQQ@SFBE@POMZ BQQ@XSJUFT BQQ@EFWFMPQFS

2VFSZ0,SPXTBGGFDUFE
TFD
Assigning privileges to the roles using the (3"/5 statement:
NZTRM (3"/54&-&$50/FNQMPZFFT50 BQQ@SFBE@POMZ

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/5*/4&3561%"5&%&-&5&0/FNQMPZFFT50 BQQ@XSJUFT

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/5"--0/FNQMPZFFT50 BQQ@EFWFMPQFS

2VFSZ0,SPXTBGGFDUFE
TFD
[ 80 ]

Creating users. If you do not specify any host, will be taken:
NZTRM $3&"5&VTFSFNQ@SFBE@POMZ*%&/5*'*&%#: FNQ@QBTT

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $3&"5&VTFSFNQ@XSJUFT*%&/5*'*&%#: FNQ@QBTT

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $3&"5&VTFSFNQ@EFWFMPQFS*%&/5*'*&%#: FNQ@QBTT

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $3&"5&VTFSFNQ@SFBE@XSJUF*%&/5*'*&%#: FNQ@QBTT

2VFSZ0,SPXTBGGFDUFE
TFD
Assigning the roles to the users using the (3"/5 statement. You can assign
multiple roles to a user.
For example, you can assign both read and write access to the FNQ@SFBE@XSJUF
user:
NZTRM (3"/5 BQQ@SFBE@POMZ 50 FNQ@SFBE@POMZ !

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/5 BQQ@XSJUFT 50 FNQ@XSJUFT !

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/5 BQQ@EFWFMPQFS 50 FNQ@EFWFMPQFS !

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/5 BQQ@SFBE@POMZ BQQ@XSJUFT 50 FNQ@SFBE@XSJUF !

2VFSZ0,SPXTBGGFDUFE
TFD
As a security measure, avoid using and restrict the access to the IP where the application
is deployed.
Selecting data into a file and table

You can save the output into a file using the 4&-&$5*/50065'*-& statement.
You can specify the column and line delimiters, and later you can import the data into other
data platforms.
[ 81 ]

How to do it...
You can save the output destination as a file or a table.
Saving as a file
To save the output into a file, you need the '*-& privilege. '*-& is a global
privilege, which means you cannot restrict it for a particular database. However,
you can restrict what the user selects:
NZTRM (3"/54&-&$50/FNQMPZFFT50 VTFS@SP@GJMF ! *%&/5*'*&%

8*5)NZTRM@OBUJWF@QBTTXPSE"4
&#%&#'%$"&#%#'''&$
TFD
NZTRM (3"/5'*-&0/50 VTFS@SP@GJMF ! *%&/5*'*&%8*5)

NZTRM@OBUJWF@QBTTXPSE"4
&#%&#'%$"&#%#'''&$
TFD
On Ubuntu, by default, MySQL will not allow you to write to file. You should set
TFDVSF@GJMF@QSJW in the config file and restart MySQL. You will learn more on
config in $IBQUFS, Configuring MySQL. On CentOS, Red
Hat, TFDVSF@GJMF@QSJW is set to WBSMJCNZTRMGJMFT, which means all the
files will be saved in that directory.
For now, enable like this. Open the config file and add TFDVSF@GJMF@QSJW
WBSMJCNZTRM:
TIFMM TVEPWJFUDNZTRMNZTRMDPOGENZTRMEDOG
Restart the MySQL server:
The following statement will save the output into a CSV format:
NZTRM 4&-&$5GJSTU@OBNFMBTU@OBNF*/50065'*-& SFTVMUDTW
'*&-%45&3.*/"5&%#: 015*0/"--:&/$-04&%#:
-*/&45&3.*/"5&%#: =O
'30.FNQMPZFFT8)&3&IJSF@EBUF -*.*5
2VFSZ0,SPXTBGGFDUFE
TFD
[ 82 ]

You can check the output of the file, which will be created in the path specified by
\TFDVSF@GJMF@QSJW^\EBUBCBTF@OBNF^, it is WBSMJCNZTRMFNQMPZFFT in this
case. The statement will fail if the file already exists, so you need to give a unique name
every time you execute or move the file to a different location:
TIFMM TVEPDBUWBSMJCNZTRMFNQMPZFFTSFTVMUDTW
#F[BMFM4JNNFM
4VNBOU1FBD
&CFSIBSEU5FSLLJ
0UNBS)FSCTU
'MPSJBO4ZSPUJVL
5TF)FSCFS
6EJ+BOTDI
3FVWFO(BSJHMJBOP
&SF[3JU[NBOO
1SFNBM#BFL
Saving as a table
You can save the results of a 4&-&$5 statement into a table. Even if the table does not exist,
you can use $3&"5& and 4&-&$5 to create the table and load the data. If the table already
exists, you can use */4&35 and 4&-&$5 to load the data.
You can save the titles into a new UJUMFT@POMZ table:

NZTRM $3&"5&5"#-&UJUMFT@POMZ"44&-&$5%*45*/$5UJUMF'30.UJUMFT
2VFSZ0,SPXTBGGFDUFE
TFD
If the table already exists, you can use the */4&35*/504&-&$5 statement:
NZTRM */4&35*/50UJUMFT@POMZ4&-&$5%*45*/$5UJUMF'30.UJUMFT
2VFSZ0,SPXTBGGFDUFE
TFD
To avoid duplicates, you can use */4&35*(/03&. However, in this case, there is no
13*."3:,&: on the UJUMFT@POMZ table. So the *(/03& clause does not make any
difference.
[ 83 ]

Loading data into a table

The way you can dump a table data into a file, you can do vice-versa, that is, load the data
from the file into a table. This is widely used for loading bulk data and is a super fast way to
load data into tables. You can specify the column delimiters to load the data into respective
columns. You should have the '*-& privilege and the */4&35 privilege on the table.
How to do it...
Earlier, you have saved GJSTU@OBNF and MBTU@OBNF to a file. You can use the same file to
load the data into another table. Before loading, you should create the table. If the table
already exists, you can directly load. The columns of the table should match the fields of the
file.
Create a table to hold the data:

NZTRM $3&"5&5"#-&FNQMPZFF@OBNFT
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
&/(*/&*OOP%#
2VFSZ0,SPXTBGGFDUFE
TFD
Make sure that the file is present:

TIFMM TVEPMTMIUSWBSMJCNZTRMFNQMPZFFTSFTVMUDTW
SXSXSXNZTRMNZTRM+VO
WBSMJCNZTRMFNQMPZFFTSFTVMUDTW
Load the data using the -0"%%"5"*/'*-& statement:

NZTRM -0"%%"5"*/'*-& SFTVMUDTW */505"#-&FNQMPZFF@OBNFT
'*&-%45&3.*/"5&%#:
015*0/"--:&/$-04&%#:
-*/&45&3.*/"5&%#: =O
2VFSZ0,SPXTBGGFDUFE
TFD
3FDPSET%FMFUFE4LJQQFE8BSOJOHT
[ 84 ]

The file can be given as a full path name to specify its exact location. If given as a relative
path name, the name is interpreted relative to the directory in which the client program was
started.
If the file contains any headers you want to ignore, specify *(/03&O-*/&4:
NZTRM -0"%%"5"*/'*-& SFTVMUDTW */505"#-&FNQMPZFF@OBNFT

'*&-%45&3.*/"5&%#:
015*0/"--:&/$-04&%#:
-*/&45&3.*/"5&%#: =O
*(/03&-*/&4
You can specify 3&1-"$& or *(/03& to deal with duplicates:
NZTRM -0"%%"5"*/'*-& SFTVMUDTW REPLACE INTO TABLE

FNQMPZFF@OBNFT'*&-%45&3.*/"5&%#: 015*0/"--:&/$-04&%#:
-*/&45&3.*/"5&%#: =O
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM -0"%%"5"*/'*-& SFTVMUDTW IGNORE INTO TABLE

FNQMPZFF@OBNFT'*&-%45&3.*/"5&%#: 015*0/"--:&/$-04&%#:
-*/&45&3.*/"5&%#: =O
2VFSZ0,SPXTBGGFDUFE
TFD
MySQL assumes that the file you want to load is available on the server. If you
are connected to the server from a remote client machine, you can specify -0$"-
to load the file located on the client. The local file will be copied from the client to
the server. The file is saved in the standard temporary location of the server. In
Linux machines, it is UNQ:
NZTRM -0"%%"5"-0$"-*/'*-& SFTVMUDTW *(/03&*/505"#-&

FNQMPZFF@OBNFT'*&-%45&3.*/"5&%#: 015*0/"--:&/$-04&%#:
-*/&45&3.*/"5&%#: =O
Joining tables
So far you have looked at inserting and retrieving data from a single table. In this section,
we will discuss how to combine two or more tables to retrieve the results.
[ 85 ]

A perfect example is that you want to find the employee name and department number of a
employee with FNQ@OP:
The department number and name are stored in the EFQBSUNFOUT table
The employee number and other details, such as GJSTU@OBNF and MBTU@OBNF,
are stored in the FNQMPZFFT table
The mapping of employee and department is stored in the EFQU@NBOBHFS table
If you do not want to use +0*/, you can do this:
1. Find the employee name with FNQ@OP as from the FNQMPZFF table:
NZTRM 4&-&$5FNQFNQ@OPFNQGJSTU@OBNFFNQMBTU@OBNF
'30.FNQMPZFFT"4FNQ
8)&3&FNQFNQ@OP

]FNQ@OP]GJSTU@OBNF]MBTU@OBNF]

]].BSHBSFUB].BSLPWJUDI]

SPXJOTFU
TFD
2. Find the department number from the EFQBSUNFOUT table:
NZTRM 4&-&$5EFQU@OP'30.EFQU@NBOBHFS"4EFQU@NHS8)&3&
EFQU@NHSFNQ@OP

]EFQU@OP]

]E]

SPXJOTFU
TFD
3. Find the department name from the EFQBSUNFOUT table:
NZTRM 4&-&$5EFQU@OBNF'30.EFQBSUNFOUTEFQU8)&3&
EFQUEFQU@OP E

]EFQU@OBNF]

].BSLFUJOH]

SPXJOTFU
TFD
[ 86 ]

How to do it...
To avoid look ups on three different tables using three statements, you can use +0*/ to club
them. The important thing to note here is to join two tables, you should have one, or more,
common column to join. You can join employees and EFQU@NBOBHFS based on FNQ@OP,
they both have the FNQ@OP column. Though the names don't need to match, you should
figure out the column on which you can join. Similarly, EFQU@NHS and EFQBSUNFOUT have
EFQU@OP as a common column.
Like a column alias, you can give table an alias and refer columns of that table using an
alias. For example, you can give employees an alias using '30.FNQMPZFFT"4FNQ and
refer columns of the FNQMPZFFT table using dot notation, such as FNQFNQ@OP:
NZTRM 4&-&$5
FNQFNQ@OP
FNQGJSTU@OBNF
FNQMBTU@OBNF
EFQUEFQU@OBNF
'30.
FNQMPZFFT"4FNQ
+0*/EFQU@NBOBHFS"4EFQU@NHS
0/FNQFNQ@OPEFQU@NHSFNQ@OP"/%FNQFNQ@OP
+0*/EFQBSUNFOUT"4EFQU
0/EFQU@NHSEFQU@OPEFQUEFQU@OP

]FNQ@OP]GJSTU@OBNF]MBTU@OBNF]EFQU@OBNF]

]].BSHBSFUB].BSLPWJUDI].BSLFUJOH]

SPXJOTFU
TFD
Let's look at another examplegyou want to find out the average salary for each
department. For this you can use the "7( function and group by EFQU@OP. To find out the
department name, you can join the results with the EFQBSUNFOUT table on EFQU@OP:
NZTRM 4&-&$5
EFQU@OBNF
"7(
TBMBSZ"4BWH@TBMBSZ
'30.
TBMBSJFT
+0*/EFQU@FNQ
0/TBMBSJFTFNQ@OPEFQU@FNQFNQ@OP
+0*/EFQBSUNFOUT
0/EFQU@FNQEFQU@OPEFQBSUNFOUTEFQU@OP
(3061#:
[ 87 ]

EFQU@FNQEFQU@OP
03%&3#:
BWH@TBMBSZ
%&4$

]EFQU@OBNF]BWH@TBMBSZ]

]4BMFT]]
].BSLFUJOH]]
]'JOBODF]]
]3FTFBSDI]]
]1SPEVDUJPO]]
]%FWFMPQNFOU]]
]$VTUPNFS4FSWJDF]]
]2VBMJUZ.BOBHFNFOU]]
])VNBO3FTPVSDFT]]

SPXTJOTFU
TFD
Identifying Duplicates using SELF JOIN

You want to find the duplicate rows in a table for specific columns. For example, you want
to find out which employees have the same GJSTU@OBNF, same MBTU@OBNF, same HFOEFS,
and same IJSF@EBUF. In that case, you can join the FNQMPZFFT table with itself while
specifying the columns where you want to find duplicates in the +0*/ clause. You need to
use different aliases for each table.
You need to add an index on the columns you want to join. The indexes will be discussed in
$IBQUFS, Performance Tuning. For now, you can execute this command to add an index:
NZTRM "-5&35"#-&FNQMPZFFT"%%*/%&9OBNF
GJSTU@OBNFMBTU@OBNF
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5
FNQ
'30.
FNQMPZFFTFNQ
+0*/FNQMPZFFTFNQ
0/FNQGJSTU@OBNFFNQGJSTU@OBNF
"/%FNQMBTU@OBNFFNQMBTU@OBNF
"/%FNQHFOEFSFNQHFOEFS
"/%FNQIJSF@EBUFFNQIJSF@EBUF
"/%FNQFNQ@OPFNQFNQ@OP
03%&3#:
[ 88 ]

GJSTU@OBNFMBTU@OBNF

]FNQ@OP]CJSUI@EBUF]GJSTU@OBNF]MBTU@OBNF]HFOEFS]IJSF@EBUF]

]]],FVOH])FVTDI].]]
]]],FVOH])FVTDI].]]
]]].BSJU],PMWJL]']]
]]].BSJU],PMWJL]']]
]]].BSTIB]'BSSPX].]]
]]].BSTIB]'BSSPX].]]
]]]/BGUBMJ].BXBUBSJ].]]
]]]/BGUBMJ].BXBUBSJ].]]
]]]5BJTPPL])VUUFS]']]
]]]5BJTPPL])VUUFS]']]

SPXTJOTFU
TFD
You have to mention FNQFNQ@OPFNQFNQ@OP because the employees will have

different FNQ@OP. Otherwise, the same employee will appear.
Using SUB queries

A subquery is a 4&-&$5 statement within another statement. Suppose you want to find the
name of the employees who started as a 4FOJPS&OHJOFFS on .
You can get the FNQ@OP from the UJUMFT table, and name from the FNQMPZFFT table. You
can also use +0*/ to find out the results.
To get the FNQ@OP from titles:

NZTRM 4&-&$5FNQ@OP'30.UJUMFT8)&3&UJUMF4FOJPS&OHJOFFS"/%
GSPN@EBUF

]FNQ@OP]

]]
]]
]]
]]
]]

SPXTJOTFU
TFD
[ 89 ]

To find the name:

NZTRM 4&-&$5GJSTU@OBNFMBTU@OBNF'30.FNQMPZFFT8)&3&FNQ@OP*/

PVUQVUGSPNQSFDFEJOHRVFSZ
NZTRM 4&-&$5GJSTU@OBNFMBTU@OBNF'30.FNQMPZFFT8)&3&FNQ@OP*/


](FPSHJ]'BDFMMP]
].JOHIPOH],BMMPVGJ]
]/FDIBNB]#FOOFU]
]/BHVJ]3FTUJWP]
]4IV[P],JSLFSVE]

SPXTJOTFU
TFD
Other clauses such as &9*454 and &26"- are also supported in MySQL. Refer to the
reference manual, IUUQTEFWNZTRMDPNEPDSFGNBOFOTVCRVFSJFTIUNM, for
more details:
NZTRM 4&-&$5
GJSTU@OBNF
MBTU@OBNF
'30.
FNQMPZFFT
8)&3&
FNQ@OP
*/
4&-&$5FNQ@OP'30.UJUMFT8)&3&UJUMF4FOJPS&OHJOFFS"/%
GSPN@EBUF


](FPSHJ]'BDFMMP]
].JOHIPOH],BMMPVGJ]
]/BHVJ]3FTUJWP]
]/FDIBNB]#FOOFU]
]4IV[P],JSLFSVE]

SPXTJOTFU
TFD
[ 90 ]

Find the employee making the maximum salary:

NZTRM 4&-&$5FNQ@OP'30.TBMBSJFT8)&3&TBMBSZ
4&-&$5."9
TBMBSZ'30.
TBMBSJFT

]FNQ@OP]

]]

SPXJOTFU
TFD
4&-&$5."9
TBMBSZ'30.TBMBSJFT is the subquery that gives the maximum salary, to
find the employee number corresponding to that salary, you can use that subquery in the
8)&3& clause.
Finding mismatched rows between tables

Suppose you want to find rows in a table that are not in other tables. You can achieve this in
two ways. Using the /05*/ clause or using 065&3+0*/.
To find the matched rows, you can use normal +0*/, if you want to find mismatched rows,
you can use 065&3+0*/. Normal +0*/ means A intersection B. 065&3+0*/ gives
matching records of both A and B and also gives unmatched records of A with /6--. If you
want the output of "#, you can use the 8)&3&+0*/$0-6./*/# *4/6-- clause.
To understand the usage of 065&3+0*/, create two FNQMPZFF tables and insert some
values:
NZTRM $3&"5&5"#-&FNQMPZFFT@MJTU"44&-&$5'30.FNQMPZFFT8)&3&
GJSTU@OBNF-*,& BB
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $3&"5&5"#-&FNQMPZFFT@MJTU"44&-&$5'30.FNQMPZFFT8)&3&FNQ@OP

#&58&&/"/%"/%HFOEFS '
2VFSZ0,SPXTBGGFDUFE
TFD
You already know how to find the employees who exist in both lists:
NZTRM 4&-&$5'30.FNQMPZFFT@MJTU8)&3&FNQ@OP*/
4&-&$5FNQ@OP'30.
FNQMPZFFT@MJTU
[ 91 ]

Or you can use +0*/:

NZTRM 4&-&$5M'30.FNQMPZFFT@MJTUM+0*/FNQMPZFFT@MJTUM0/
MFNQ@OPMFNQ@OP
To find out the employees who exist in FNQMPZFFT@MJTU but not in FNQMPZFFT@MJTU:
NZTRM 4&-&$5'30.FNQMPZFFT@MJTU8)&3&FNQ@OP/05*/
4&-&$5FNQ@OP
'30.FNQMPZFFT@MJTU
Or you can use 065&3+0*/:

NZTRM 4&-&$5M'30.FNQMPZFFT@MJTUM-&'5065&3+0*/FNQMPZFFT@MJTU
M0/MFNQ@OPMFNQ@OP8)&3&MFNQ@OP*4/6--
The outer join creates /6-- columns of the second table in the join list for each unmatched
row. If you use 3*()5+0*/, the first table will get /6-- values for the unmatched rows.
You can also use 065&3+0*/ to find matched rows. Instead of 8)&3&MFNQ@OP*4
/6--, give 8)&3&FNQ@OP*4/05/6--:
NZTRM 4&-&$5M'30.FNQMPZFFT@MJTUM-&'5065&3+0*/FNQMPZFFT@MJTU

M0/MFNQ@OPMFNQ@OP8)&3&MFNQ@OP*4/05/6--
Stored procedures
Suppose you need to execute a series of statements in MySQL, instead of sending all SQL
statements every time, you can encapsulate all the statements in a single program and call it
whenever required. A stored procedure is a set of SQL statements for which no return value
is needed.
Apart from SQL statements, you can make use of variables to store results and do
programmatical stuff inside a stored procedure. For example, you can write *', $"4&
clauses, logical operations, and 8)*-& loops.
Stored functions and procedures are also referred to as stored routines.

For creating a stored procedure, you should have the $3&"5&3065*/& privilege.
Stored functions will have a return value.
Stored procedures do not have a return value.
[ 92 ]

All the code is written inside the #&(*/BOE&/% block.

Stored functions can be called directly in a 4&-&$5 statement.
Stored procedures can be called using the $"-- statement.
Since the statements inside stored routines should end with a delimiter (), you
have to change the delimiter for MySQL so that MySQL won't interpret the SQL
statements inside a stored routine with normal statements. After the creation of
the procedure, you can change the delimiter back to the default value.
How to do it...
For example, you want to add a new employee. You should update three tables, namely
FNQMPZFFT, TBMBSJFT, and UJUMFT. Instead of executing three statements, you can
develop a stored procedure and call it to create a new FNQMPZFF.
You have to pass the employee's GJSTU@OBNF, MBTU@OBNF, HFOEFS, and CJSUI@EBUF, as
well as the department the employee joins. You can pass those using input variables and
you should get the employee number as output. The stored procedure does not return a
value, but it can update a variable and you can use it.
Here is a simple example of a stored procedure to create a new employee and update
the TBMBSZ and EFQBSUNFOU tables:
%301UIFFYJTUJOHQSPDFEVSFJGBOZXJUIUIFTBNFOBNFCFGPSFDSFBUJOH
%301130$&%63&*'&9*454DSFBUF@FNQMPZFF
$IBOHFUIFEFMJNJUFSUP
%&-*.*5&3
*/TQFDJGJFTUIFWBSJBCMFTUBLFOBTBSHVNFOUT*/065TQFDJGJFTUIF
PVUQVUWBSJBCMF
$3&"5&130$&%63&DSFBUF@FNQMPZFF
065OFX@FNQ@OP*/5*/GJSTU@OBNF
WBSDIBS
*/MBTU@OBNFWBSDIBS
*/HFOEFSFOVN
. ' */
CJSUI@EBUFEBUF*/FNQ@EFQU@OBNFWBSDIBS
*/UJUMFWBSDIBS

#&(*/
%FDMBSFWBSJBCMFTGPSFNQ@EFQU@OPBOETBMBSZ
%&$-"3&FNQ@EFQU@OPDIBS

%&$-"3&TBMBSZJOU%&'"6-5
4FMFDUUIFNBYJNVNFNQMPZFFOVNCFSJOUPUIFWBSJBCMFOFX@FNQ@OP
4&-&$5NBY
FNQ@OP*/50OFX@FNQ@OP'30.FNQMPZFFT
*ODSFNFOUUIFOFX@FNQ@OP
4&5OFX@FNQ@OPOFX@FNQ@OP
*/4&35UIFEBUBJOUPFNQMPZFFTUBCMF
[ 93 ]

5IFGVODUJPO$63%"5&
HJWFTUIFDVSSFOUEBUF
*/4&35*/50FNQMPZFFT7"-6&4
OFX@FNQ@OPCJSUI@EBUFGJSTU@OBNF
MBTU@OBNFHFOEFS$63%"5&

'JOEPVUUIFEFQU@OPGPSEFQU@OBNF
4&-&$5FNQ@EFQU@OBNF
4&-&$5EFQU@OP*/50FNQ@EFQU@OP'30.EFQBSUNFOUT8)&3&
EFQU@OBNFFNQ@EFQU@OBNF
4&-&$5FNQ@EFQU@OP
*OTFSUJOUPEFQU@FNQ
*/4&35*/50EFQU@FNQ7"-6&4
OFX@FNQ@OPFNQ@EFQU@OP$63%"5&

*OTFSUJOUPUJUMFT
*/4&35*/50UJUMFT7"-6&4
OFX@FNQ@OPUJUMF$63%"5&

'JOETBMBSZCBTFEPOUJUMF
*'UJUMF 4UBGG
5)&/4&5TBMBSZ
&-4&*'UJUMF 4FOJPS4UBGG
5)&/4&5TBMBSZ
&/%*'
*OTFSUJOUPTBMBSJFT
*/4&35*/50TBMBSJFT7"-6&4
OFX@FNQ@OPTBMBSZ$63%"5&

&/%

$IBOHFUIFEFMJNJUFSCBDLUP
%&-*.*5&3
To create a stored procedure, you can:
Paste it in the command-line client

Save it in the file and import it in MySQL using NZTRMVVTFS Q
FNQMPZFFTTUPSFE@QSPDFEVSFTRM
Source the NZTRM 4063$&TUPSFE@QSPDFEVSFTRM file
To use the stored procedure, grant the execute privilege to the FNQ@SFBE@POMZ user:
NZTRM (3"/5&9&$65&0/FNQMPZFFT50 FNQ@SFBE@POMZ !
2VFSZ0,SPXTBGGFDUFE
TFD
Invoke the stored procedure using the $"--TUPSFE@QSPDFEVSF

065WBSJBCMF*/
WBMVFT statement and name of the routine.
[ 94 ]

Connect to MySQL using the FNQ@SFBE@POMZ account:

TIFMM NZTRMVFNQ@SFBE@POMZQFNQ@QBTTFNQMPZFFT"
Pass the variable where you want to store the !OFX@FNQ@OP output and also pass the
required input values:
NZTRM $"--DSFBUF@FNQMPZFF
!OFX@FNQ@OP +PIO 4NJUI .
3FTFBSDI 4UBGG
2VFSZ0,SPXBGGFDUFE
TFD
Select the value of FNQ@OP, which is stored in the !OFX@FNQ@OP variable:

NZTRM 4&-&$5!OFX@FNQ@OP

]!OFX@FNQ@OP]

]]

SPXJOTFU
TFD
Check that the rows are created in the FNQMPZFFT, TBMBSJFT, and UJUMFT tables:
NZTRM 4&-&$5'30.FNQMPZFFT8)&3&FNQ@OP


]]]+PIO]4NJUI].]]

SPXJOTFU
TFD
NZTRM 4&-&$5'30.TBMBSJFT8)&3&FNQ@OP

]FNQ@OP]TBMBSZ]GSPN@EBUF]UP@EBUF]

]]]]]

SPXJOTFU
TFD
NZTRM 4&-&$5'30.UJUMFT8)&3&FNQ@OP

]FNQ@OP]UJUMF]GSPN@EBUF]UP@EBUF]

]]4UBGG]]]

SPXJOTFU
TFD
[ 95 ]

You can see that, even though FNQ@SFBE@POMZ has no write access on the tables, it is able to
write by calling the stored procedure. If the 42-4&$63*5: of the stored procedure is
created as */70,&3, FNQ@SFBE@POMZ cannot modify the data. Note that if you are
connecting using MPDBMIPTU, create the privileges for the MPDBMIPTU user.
To list all the stored procedures in a database, execute 4)08130$&%63&45"564=(. To

check the definition of an existing stored routine, you can execute 4)08$3&"5&
130$&%63&QSPDFEVSF@OBNF =(.
There's more...
Stored procedures are also used to enhance the security. The user needs
the &9&$65& privilege on the stored procedure to execute it.
By the definition of a stored routine:
The %&'*/&3 clause specifies the creator of the stored routine. If nothing is
specified, the current user is taken.
The 42-4&$63*5: clause specifies the execution context of the stored routine. It
can be either %&'*/&3 or */70,&3.
%&'*/&3: A user even with only the &9&$65& permission for routine can call and get the
output of the stored routine, regardless of whether that user has permission on the
underlying tables or not. It is enough if %&'*/&3 has privileges.
*/70,&3: The security context is switched to the user who invokes the stored routine. In
this case, the invoker should have access to the underlying tables.
See also
Refer to the documentation for more examples and syntax, at IUUQTEFWNZTRMDPNEPD
SFGNBOFODSFBUFQSPDFEVSFIUNM.
Functions
Just like stored procedures, you can create stored functions. The main difference is
functions should have a return value and they can be called in 4&-&$5. Usually, stored
functions are created to simplify complex calculations.
[ 96 ]

How to do it...
Here is an example of how to write a function and how to call it. Suppose a banker wants to
give a credit card based on income level, instead of exposing the actual salary, you can
expose this function to find out the income level:
TIFMM WJGVODUJPOTRM
%301'6/$5*0/*'&9*454HFU@TBM@MFWFM
%&-*.*5&3
$3&"5&'6/$5*0/HFU@TBM@MFWFM
FNQJOU3&563/47"3$)"3

%&5&3.*/*45*$
#&(*/
%&$-"3&TBM@MFWFMWBSDIBS

%&$-"3&BWH@TBM'-0"5
4&-&$5"7(
TBMBSZ*/50BWH@TBM'30.TBMBSJFT8)&3&FNQ@OPFNQ
*'BWH@TBM5)&/
4&5TBM@MFWFM #30/;&
&-4&*'
BWH@TBM "/%BWH@TBM5)&/
4&5TBM@MFWFM 4*-7&3
&-4&*'
BWH@TBM "/%BWH@TBM5)&/
4&5TBM@MFWFM (0-%
&-4&*'
BWH@TBM 5)&/
4&5TBM@MFWFM 1-"5*/6.
&-4&
4&5TBM@MFWFM /05'06/%
&/%*'
3&563/
TBM@MFWFM
&/%

%&-*.*5&3
To create the function:

NZTRM 4063$&GVODUJPOTRM
2VFSZ0,SPXTBGGFDUFE
TFD
2VFSZ0,SPXTBGGFDUFE
TFD
:PVIBWFUPQBTTUIFFNQMPZFFOVNCFSBOEUIFGVODUJPOSFUVSOTUIFJODPNF
MFWFM
NZTRM 4&-&$5HFU@TBM@MFWFM

]HFU@TBM@MFWFM
]

]4*-7&3]

[ 97 ]

SPXJOTFU
TFD

]HFU@TBM@MFWFM
]

](0-%]

SPXJOTFU
TFD

]HFU@TBM@MFWFM
]

]/05'06/%]

SPXJOTFU
TFD
To list all the stored functions in a database, execute 4)08'6/$5*0/45"564=(. To check

the definition of the existing stored function, you can execute 4)08$3&"5&'6/$5*0/
GVODUJPO@OBNF =(.
It is very important to give the %&5&3.*/*45*$ keyword in the function

creation. A routine is considered %&5&3.*/*45*$ if it always produces
the same result for the same input parameters, and /05%&5&3.*/*45*$
otherwise. If neither %&5&3.*/*45*$ nor /05%&5&3.*/*45*$ is given
in the routine definition, the default is /05%&5&3.*/*45*$. To declare
that a function is deterministic, you must specify %&5&3.*/*45*$
explicitly.
Declaring a /0/%&5&3.*/*45*$ routine as %&5&3.*/*45*$ might lead
to unexpected results, by causing the optimizer to make incorrect
execution plan choices. Declaring a %&5&3.*/*45*$ routine as /0/
%&5&3.*/*45*$ might diminish performance, by causing available
optimizations not to be used.
Inbuilt functions
MySQL provides numerous inbuilt functions. You have already used the $63%"5&

function to get the current date.
[ 98 ]

You can use the functions in the 8)&3& clause:

NZTRM 4&-&$5'30.FNQMPZFFT8)&3&IJSF@EBUF$63%"5&

For example, the following function gives the date from exactly one week ago:
NZTRM 4&-&$5%"5&@"%%
$63%"5&
*/5&37"-%":"4 %BZT"HP
Add two strings:
NZTRM 4&-&$5$0/$"5
GJSTU@OBNF MBTU@OBNF'30.FNQMPZFFT
-*.*5

]$0/$"5
GJSTU@OBNF MBTU@OBNF]

]"BNFS"OHFS]

SPXJOTFU
TFD
See also
Refer to the MySQL reference manual for a complete list of functions, at IUUQTEFW
NZTRMDPNEPDSFGNBOFOGVODPQTVNNBSZSFGIUNM.
Triggers
A trigger is used to activate something before or after the trigger event. For example, you
can have a trigger activate before each row that is inserted into a table or after each row that
is updated.
Triggers are highly useful while altering a table without downtime (Refer to $IBQUFS,
Table Maintenance, in the Alter table using online schema change tool section) and also for
auditing purposes. Suppose you want to find out the previous value of a row, you can write
a trigger that saves those rows in another table before updating. The other table serves as an
audit table that has the previous records.
The trigger action time can be #&'03& or "'5&3, which indicates whether the trigger
activates before or after each row to be modified.
[ 99 ]

Trigger events can be */4&35, %&-&5&, or 61%"5&:
*/4&35: Whenever a new row gets inserted through */4&35, 3&1-"$&, or -0"%
%"5", the trigger gets activated
61%"5&: Through the 61%"5& statement
%&-&5&: Through the %&-&5& or 3&1-"$& statements
From MySQL 5.7, a table can have multiple triggers at the same time. For example, a table
can have two #&'03&*/4&35 triggers. You have to specify which trigger should go first
using '0--084 or 13&$&%&4.
How to do it...
For example, you want to round off the salary before inserting it into the TBMBSJFT table.
/&8 refers to the new value that is being inserted:
TIFMM WJCFGPSF@JOTFSU@USJHHFSTRM
%30153*((&3*'&9*454TBMBSZ@SPVOE
%&-*.*5&3
$3&"5&53*((&3TBMBSZ@SPVOE#&'03&*/4&350/TBMBSJFT
'03&"$)308
#&(*/
4&5/&8TBMBSZ306/%
/&8TBMBSZ
&/%

%&-*.*5&3
Create the trigger by sourcing the file:

NZTRM 4063$&CFGPSF@JOTFSU@USJHHFSTRM
2VFSZ0,SPXTBGGFDUFE
TFD
2VFSZ0,SPXTBGGFDUFE
TFD
Test the trigger by inserting a floating number into the salary:

NZTRM */4&35*/50TBMBSJFT7"-6&4
$63%"5&

2VFSZ0,SPXBGGFDUFE
TFD
[ 100 ]

You can see that the salary is rounded off:

NZTRM 4&-&$5'30.TBMBSJFT8)&3&FNQ@OP"/%GSPN@EBUF$63%"5&

]FNQ@OP]TBMBSZ]GSPN@EBUF]UP@EBUF]

]]]]]

SPXJOTFU
TFD
Similarly, you can create a #&'03&61%"5& trigger to round off the salary. Another
example: you want to log which user has inserted into the TBMBSJFT table. Create an BVEJU
table:
NZTRM $3&"5&5"#-&TBMBSZ@BVEJU
FNQ@OPJOUVTFSWBSDIBS

EBUF@NPEJGJFEEBUF
Note that the following trigger precedes the TBMBSZ@SPVOE trigger, which is specified by
13&$&%&4TBMBSZ@SPVOE:
TIFMM WJCFGPSF@JOTFSU@USJHHFSTRM
%&-*.*5&3
$3&"5&53*((&3TBMBSZ@BVEJU
#&'03&*/4&35
0/TBMBSJFT'03&"$)30813&$&%&4TBMBSZ@SPVOE
#&(*/
*/4&35*/50TBMBSZ@BVEJU7"-6&4
/&8FNQ@OP64&3
$63%"5&

&/%
%&-*.*5&3
Insert it into TBMBSJFT:

NZTRM */4&35*/50TBMBSJFT7"-6&4
$63%"5&

2VFSZ0,SPXBGGFDUFE
TFD
Find out who inserted the salary by querying the TBMBSZ@BVEJU table:
NZTRM 4&-&$5'30.TBMBSZ@BVEJU8)&3&FNQ@OP

]FNQ@OP]VTFS]EBUF@NPEJGJFE]

]]SPPU!MPDBMIPTU]]

SPXJOTFU
TFD
[ 101 ]

In case the TBMBSZ@BVEJU table is dropped or is not available, all the

inserts on the TBMBSJFT table will be blocked. If you do not want to do
auditing, you should drop the trigger first and then the table.
Triggers can make overhead on the write speed based on the complexity
of it.
To check all the triggers, execute 4)0853*((&34=(.
To check the definition of an existing trigger, execute 4)08$3&"5&
53*((&3USJHHFS@OBNF .
See also
Refer to the MySQL reference manual, at IUUQTEFWNZTRMDPNEPDSFGNBOFO
USJHHFSTZOUBYIUNM, for more details.
Views
View is a virtual table based on the result-set of an SQL statement. It will also have rows
and columns just like a real table, but few restrictions, which will be discussed later. Views
hide the SQL complexity and, more importantly, provide additional security.
How to do it...
Suppose you want to give access only to the FNQ@OP and TBMBSZ columns of the TBMBSJFT
table, and GSPN@EBUF is after . For this, you can create a view with the SQL
that gives the required result.
NZTRM $3&"5&"-(03*5).6/%&'*/&%
%&'*/&3ASPPUA!AMPDBMIPTUA
42-4&$63*5:%&'*/&37*&8TBMBSZ@WJFX
"4
4&-&$5FNQ@OPTBMBSZ'30.TBMBSJFT8)&3&GSPN@EBUF
Now the TBMBSZ@WJFX view is created and you can query it just like any other table:
TBMBSZBTBWH'30.TBMBSZ@WJFX(3061#:FNQ@OP
03%&3#:BWH%&4$-*.*5
[ 102 ]

You can see that the view has access to particular rows (that is, GSPN@EBUF
) and not all of the rows. You can use the view to restrict user access to
particular rows.
To list all views, execute:

NZTRM 4)08'6--5"#-&48)&3&5"#-&@5:1&-*,& 7*&8
To check the definition of the view, execute:

NZTRM 4)08$3&"5&7*&8TBMBSZ@WJFX=(
You might have noticed the DVSSFOU@EFQU@FNQ and EFQU@FNQ@MBUFTU@EBUF views,

which are there as part of the FNQMPZFF database. You can explore the definition and find
out their purpose.
Simple views that do not have sub-queries, +0*/4, (3061#: clauses, union, and so on, can
be updated. TBMBSZ@WJFX is a simple view that can be updated or inserted if the
underlying tables have a default value:
NZTRM 61%"5&TBMBSZ@WJFX4&5TBMBSZ8)&3&FNQ@OP
2VFSZ0,SPXBGGFDUFE
TFD
NZTRM */4&35*/50TBMBSZ@WJFX7"-6&4

&3303
):'JFMEPGWJFX FNQMPZFFTTBMBSZ@WJFX VOEFSMZJOHUBCMF
EPFTO UIBWFBEFGBVMUWBMVF
If the table has a default value, you could have inserted a row even if it does not match the
filter criteria in the view. To avoid that, and to insert rows that satisfy the view condition,
you have to provide 8*5)$)&$,015*0/ in the definition.
The 7*&8 algorithms:
.&3(&: MySQL combines the input query and the view definition into a single
query and then executes the combined query. The .&3(& algorithm is allowed
only on simple views.
5&.15"#-&: MySQL stores the results in the temporary table and then it executes
the input query against this temporary table.
6/%&'*/&% (default): MySQL automatically chooses the .&3(& or 5&.15"#-&
algorithm. MySQL prefers the .&3(& algorithm to the 5&.15"#-& algorithm
because the .&3(& algorithm is much more efficient.
[ 103 ]

Events
Just like a cron on a Linux server, MySQL has &7&/54 to handle the scheduled tasks.
MySQL uses a special thread called the event schedule thread to execute all scheduled
events. By default, the event scheduler thread is not enabled (version < 8.0.3), to enable it,
execute:
NZTRM 4&5(-0#"-FWFOU@TDIFEVMFS0/
How to do it...
Suppose you no longer need to keep salary audit records that are more than a month old,
you can schedule an event that runs daily and deletes the records from the TBMBSZ@BVEJU
table that are a month old.
NZTRM %301&7&/5*'&9*454QVSHF@TBMBSZ@BVEJU
%&-*.*5&3
$3&"5&&7&/5*'/05&9*454QVSHF@TBMBSZ@BVEJU
0/4$)&%6-&
&7&3:8&&,
45"354$633&/5@%"5&
%0#&(*/
%&-&5&'30.TBMBSZ@BVEJU8)&3&EBUF@NPEJGJFE%"5&@"%%
$63%"5&

*/5&37"-EBZ
&/%
%&-*.*5&3
Once the event is created, it will automatically do the job of purging the salary audit
records.
To check the events, execute:
NZTRM 4)08&7&/54=(
SPX
%CFNQMPZFFT
/BNFQVSHF@TBMBSZ@BVEJU
%FGJOFSSPPU!MPDBMIPTU
5JNF[POF4:45&.
5ZQF3&$633*/(
&YFDVUFBU/6--
*OUFSWBMWBMVF
*OUFSWBMGJFME.*/65&
4UBSUT
&OET/6--
[ 104 ]

4UBUVT&/"#-&%
0SJHJOBUPS
DIBSBDUFS@TFU@DMJFOUVUG
DPMMBUJPO@DPOOFDUJPOVUG@HFOFSBM@DJ
%BUBCBTF$PMMBUJPOVUGNC@@BJ@DJ
SPXJOTFU
TFD
To check the definition on the event, execute:
NZTRM 4)08$3&"5&&7&/5QVSHF@TBMBSZ@BVEJU=(
To disable/enable the event, execute the following:
NZTRM "-5&3&7&/5QVSHF@TBMBSZ@BVEJU%*4"#-&
NZTRM "-5&3&7&/5QVSHF@TBMBSZ@BVEJU&/"#-&
Access control
All stored programs (procedures, functions, triggers, and events) and views have a
%&'*/&3. If the %&'*/&3 is not specified, the user who creates the object will be chosen as
%&'*/&3.
Stored routines (procedures and functions) and views have an 42-4&$63*5:

characteristic with a value of %&'*/&3 or */70,&3 to specify whether the object executes in
the definer or invoker context. Triggers and events have no 42-4&$63*5: characteristic
and always execute in the definer context. The server invokes these objects automatically as
necessary, so there is no invoking user.
See also
There are lots of ways to schedule events, refer to IUUQTEFWNZTRMDPNEPDSFGNBO
FOFWFOUTDIFEVMFSIUNM for more details.
[ 105 ]

Getting information about databases and

tables
You might have already noticed an JOGPSNBUJPO@TDIFNB database in the list of databases.
JOGPSNBUJPO@TDIFNB is a collection of views that consist of metadata about all the
database objects. You can connect to JOGPSNBUJPO@TDIFNB and explore all the tables. The
most widely-used tables are explained in this chapter. You either query the
JOGPSNBUJPO@TDIFNB tables or use the 4)08 command, which essentially does the same.
*/'03."5*0/@4$)&." queries are implemented as views over the EBUB

EJDUJPOBSZ tables. There are two types of metadata in the */'03."5*0/@4$)&." tables:
Static table metadata: 5"#-&@4$)&.", 5"#-&@/".&, 5"#-&@5:1&, and &/(*/&.

These statistics will be read directly from the EBUBEJDUJPOBSZ.
Dynamic table metadata: "650@*/$3&.&/5, "7(@308@-&/(5), and %"5"@'3&&.
Dynamic metadata frequently changes (for example, the "650@*/$3&.&/5 value
will advance after each */4&35). In many cases, the dynamic metadata will also
incur some cost to accurately calculate on demand, and accuracy may not be
beneficial for the typical query. Consider the case of the %"5"@'3&& statistic that
shows the number of free bytes in a tablega cached value is usually sufficient.
In MySQL 8.0, the dynamic table metadata will default to being cached. This is configurable
via the JOGPSNBUJPO@TDIFNB@TUBUT setting (default cached), and can be changed to 4&5
!!(-0#"-JOGPSNBUJPO@TDIFNB@TUBUT -"5&45 in order to always retrieve the
dynamic information directly from the storage engine (at the cost of slightly higher query
execution).
As an alternative, the user can also execute "/"-:;&5"#-& on the table, to update the
cached dynamic statistics.
Most of the tables have the 5"#-&@4$)&." column, which refers to the database name, and
the 5"#-&@/".& column, which refers to the table name.
Refer to IUUQTNZTRMTFSWFSUFBNDPNNZTRMJNQSPWFNFOUTUPJOGPSNBUJPO@
TDIFNB for more details.
[ 106 ]

How to do it...
Check the list of all the tables:
NZTRM 64&*/'03."5*0/@4$)&."
NZTRM 4)085"#-&4
TABLES
The 5"#-&4 table contains all the information about the table, such as which database
belongs to 5"#-&@4$)&.", the number of rows (5"#-&@3084), &/(*/&, %"5"@-&/(5),
*/%&9@-&/(5), and %"5"@'3&&:
mysql> DESC INFORMATION_SCHEMA.TABLES;

]'JFME]5ZQF
]/VMM],FZ]%FGBVMU]&YUSB]

]5"#-&@$"5"-0(]WBSDIBS

]/0]]/6--]]
]5"#-&@4$)&."]WBSDIBS

]/0]]/6--]]
]5"#-&@/".&]WBSDIBS

]/0]]/6--]]
]5"#-&@5:1&]FOVN
#"4&5"#-& 7*&8 4:45&.7*&8
]/0]]/6--]]
]&/(*/&]WBSDIBS

]:&4]]/6--]]
]7&34*0/]JOU

]:&4]]/6--]]
]308@'03."5]
FOVN
'JYFE %ZOBNJD $PNQSFTTFE 3FEVOEBOU $PNQBDU 1BHFE ]:&4]
]/6--]]
]5"#-&@3084]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]"7(@308@-&/(5)]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]%"5"@-&/(5)]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]."9@%"5"@-&/(5)]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]*/%&9@-&/(5)]CJHJOU
VOTJHOFE
[ 107 ]

]:&4]]/6--]]
]%"5"@'3&&]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]"650@*/$3&.&/5]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]$3&"5&@5*.&]UJNFTUBNQ
]/0]]$633&/5@5*.&45".1]POVQEBUF$633&/5@5*.&45".1]
]61%"5&@5*.&]UJNFTUBNQ
]:&4]]/6--]]
]$)&$,@5*.&]UJNFTUBNQ
]:&4]]/6--]]
]5"#-&@$0--"5*0/]WBSDIBS

]:&4]]/6--]]
]$)&$,46.]CJHJOU
VOTJHOFE
]:&4]]/6--]]
]$3&"5&@015*0/4]WBSDIBS

]:&4]]/6--]]
]5"#-&@$0..&/5]WBSDIBS

]:&4]]/6--]]

SPXTJOTFU
TFD
For example, you want to know %"5"@-&/(5), */%&9@-&/(5), and %"5&@'3&& inside the
FNQMPZFFT database:
NZTRM 4&-&$546.
%"5"@-&/(5)"4%"5"@4*;&@.#
46.
*/%&9@-&/(5)"4*/%&9@4*;&@.#46.
%"5"@'3&&"4
%"5"@'3&&@.#'30.*/'03."5*0/@4$)&."5"#-&48)&3&5"#-&@4$)&." FNQMPZFFT

]%"5"@4*;&@.#]*/%&9@4*;&@.#]%"5"@'3&&@.#]

]]]]

SPXJOTFU
TFD
COLUMNS
This table lists all the columns and its definition for each table:
NZTRM 4&-&$5'30.$0-6./48)&3&5"#-&@/".& FNQMPZFFT =(
[ 108 ]

FILES
You have already seen that MySQL stores the *OOP%# data in the JCE files inside a
directory (with the same name as the database name) in the EBUBEJSFDUPSZ . To get more
information about the files, you can query the '*-&4 table:
NZTRM 4&-&$5'30.'*-&48)&3&'*-&@/".&-*,&
FNQMPZFFTFNQMPZFFTJCE =(
___
&95&/5@4*;&
"650&95&/%@4*;&
%"5"@'3&&
___
You should be keen at %"5"@'3&&, which represents the unallocated segments plus the data
that is free inside the segments due to fragmentation. When you rebuild the table, you can
free up bytes shown in %"5"@'3&&.
INNODB_SYS_TABLESPACES
The size of the file is also available in the *//0%#@5"#-&41"$&4 table:
NZTRM 4&-&$5'30.*//0%#@5"#-&41"$&48)&3&/".& FNQMPZFFTFNQMPZFFT =(
SPX
41"$&
/".&FNQMPZFFTFNQMPZFFT
'-"(
308@'03."5%ZOBNJD
1"(&@4*;&
;*1@1"(&@4*;&
41"$&@5:1&4JOHMF
'4@#-0$,@4*;&
'*-&@4*;&32505856
"--0$"5&%@4*;&
SPXJOTFU
TFD
You can verify the same in the filesystem:

TIFMM TVEPMTMUSWBSMJCNZTRMFNQMPZFFTFNQMPZFFTJCE
SXSNZTRMNZTRM32505856+VO
WBSMJCNZTRMFNQMPZFFTFNQMPZFFTJCE
[ 109 ]

INNODB_TABLESTATS
The size of the index and approximate number of rows is available in
the *//0%#@5"#-&45"54 table:
NZTRM 4&-&$5'30.*//0%#@5"#-&45"548)&3&/".& FNQMPZFFTFNQMPZFFT =(
SPX
5"#-&@*%
/".&FNQMPZFFTFNQMPZFFT
45"54@*/*5*"-*;&%*OJUJBMJ[FE
/6.@3084
$-645@*/%&9@4*;&
05)&3@*/%&9@4*;&
.0%*'*&%@$06/5&3
"650*/$
3&'@$06/5
SPXJOTFU
TFD
PROCESSLIST
One of the most used views is the process list. It lists all the queries running on the server:
NZTRM 4&-&$5'30.130$&44-*45=(
SPX
*%
64&3FWFOU@TDIFEVMFS
)045MPDBMIPTU
%#/6--
$0.."/%%BFNPO
5*.&
45"5&8BJUJOHGPSOFYUBDUJWBUJPO
*/'0/6--
SPX
*%
64&3SPPU
)045MPDBMIPTU
%#JOGPSNBUJPO@TDIFNB
$0.."/%2VFSZ
5*.&
45"5&FYFDVUJOH
*/'04&-&$5'30.130$&44-*45
SPXTJOTFU
TFD
[ 110 ]

Or you can execute 4)08130$&44-*45 to get the same output.
Other tables: 3065*/&4 contains the definition of functions and stored routines. 53*((&34
contains the definition of triggers. 7*&84 contains the definition of views.
See also
To learn about the improvements in */'03."5*0/@4$)&.", refer to IUUQ
NZTRMTFSWFSUFBNDPNNZTRMJNQSPWFNFOUTUPJOGPSNBUJPO@TDIFNB.
[ 111 ]

Using MySQL (Advanced)
3
Using JSON
Common table expressions (CTE)
Generated columns
Window functions
Introduction
In this chapter, you will learn about the newly introduced features of MySQL.
Using JSON
As you have seen in the previous chapter, to store data in MySQL, you have to define the
database and table structure (schema), which is a major limitation. To cope with that, from
MySQL 5.7, MySQL supports the JavaScript Object Notation (JSON) datatype. Earlier
there was no separate datatype and it was stored as a string. The new JSON datatype
provides automatic validation of JSON documents and optimized storage format.
JSON documents are stored in binary format, which enables the following:
Quick-read access to document elements

No need for the value to be parsed from a text representation when the server
reads the JSON again
Looking up subobjects or nested values directly by key or array index without
reading all values before or after them in the document

Using MySQL (Advanced) Chapter 3
How to do it...
Suppose you want to store more details about your employees; you can save them using
JSON:
$3&"5&5"#-&FNQ@EFUBJMT
FNQ@OPJOUQSJNBSZLFZ
EFUBJMTKTPO

Insert JSON
*/4&35*/50FNQ@EFUBJMT
FNQ@OPEFUBJMT
7"-6&4

\MPDBUJPO*/QIPOF FNBJMBCD!FYBNQMFDPN
BEESFTT\MJOFBCDMJOFYZ[TUSFFUDJUZ#BOHBMPSF
QJO^^

Retrieve JSON
You can retrieve the fields of the JSON column using the and operators:
NZTRM 4&-&$5FNQ@OPEFUBJMT BEESFTTQJO QJO'30.FNQ@EFUBJMT

]FNQ@OP]QJO]

]]]

SPXJOTFU
TFD
To retrieve data without quotes, use the operator:

NZTRM 4&-&$5FNQ@OPEFUBJMT BEESFTTQJO QJO'30.FNQ@EFUBJMT

]FNQ@OP]QJO]

]]]

SPXJOTFU
TFD
[ 113 ]

JSON functions
MySQL provides many functions to deal with JSON data. Let's look into the most used
ones.
1SFUUZWJFX
To display JSON values in pretty format, use the +40/@13&55:
function:
NZTRM 4&-&$5FNQ@OP+40/@13&55:
EFUBJMT'30.FNQ@EFUBJMT=(
SPX
FNQ@OP
+40/@13&55:
EFUBJMT\
FNBJMBCD!FYBNQMFDPN
QIPOF
BEESFTT\
QJO
DJUZ#BOHBMPSF
MJOFBCD
MJOFYZ[TUSFFU
^
MPDBUJPO*/
^
SPXJOTFU
TFD
Without pretty:
NZTRM 4&-&$5FNQ@OPEFUBJMT'30.FNQ@EFUBJMT=(
SPX
FNQ@OP
EFUBJMT\FNBJMBCD!FYBNQMFDPNQIPOF BEESFTT
\QJODJUZ#BOHBMPSFMJOFBCDMJOFYZ[
TUSFFU^MPDBUJPO*/^
SPXJOTFU
TFD
[ 114 ]

Searching
You can reference a JSON column using the DPM QBUI operator in the 8)&3& clause:
NZTRM 4&-&$5FNQ@OP'30.FNQ@EFUBJMT8)&3&
EFUBJMT BEESFTTQJO

]FNQ@OP]

]]

SPXJOTFU
TFD
You can also use the +40/@$0/5"*/4 function to search for data. It returns if the data is
found and otherwise:
NZTRM 4&-&$5+40/@$0/5"*/4
EFUBJMT BEESFTTQJO '30.
FNQ@EFUBJMT

]+40/@$0/5"*/4
EFUBJMT BEESFTTQJO ]

]]

SPXJOTFU
TFD
How to search for a key? Suppose you want to check whether BEESFTTMJOF exists or
not:
NZTRM 4&-&$5+40/@$0/5"*/4@1"5)
EFUBJMT POF BEESFTTMJOF'30.
FNQ@EFUBJMT

]+40/@$0/5"*/4@1"5)
EFUBJMT POF BEESFTTMJOF
]

]
]

SPXJOTFU
TFD
[ 115 ]

Here, POF indicates at least one key should exists. Suppose you want to check whether
BEESFTTMJOF or BEESFTTMJOF exists:
NZTRM 4&-&$5+40/@$0/5"*/4@1"5)
EFUBJMT POF BEESFTTMJOF
BEESFTTMJOF'30.FNQ@EFUBJMT

]+40/@$0/5"*/4@1"5)
EFUBJMT POF BEESFTTMJOFBEESFTTMJOF
]

]
]

If you want to check whether both BEESFTTMJOF and BEESFTTMJOF exist, you can use
BOE instead of POF:
NZTRM 4&-&$5+40/@$0/5"*/4@1"5)
EFUBJMT BMM BEESFTTMJOF
BEESFTTMJOF'30.FNQ@EFUBJMT

]+40/@$0/5"*/4@1"5)
EFUBJMT BMM BEESFTTMJOFBEESFTTMJOF
]

]
]

SPXJOTFU
TFD
Modifying
You can modify the data using three different functions: +40/@4&5
, +40/@*/4&35
,
+40/@3&1-"$&
. Before MySQL 8, we needed a full update of the entire column, which is
not the optimal way:
+40/@4&5: Replaces existing values and adds non-existing values.

Suppose you want to replace the pin code of the employee and also add details of
a nickname:
NZTRM 61%"5&
FNQ@EFUBJMT
[ 116 ]

4&5
EFUBJMT+40/@4&5
EFUBJMTBEESFTTQJO
OJDLOBNFLBJ
8)&3&
FNQ@OP
2VFSZ0,SPXBGGFDUFE
TFD
+40/@*/4&35
: Inserts values without replacing existing values
Suppose you want to add a new column without updating the existing values;
you can use +40/@*/4&35
:
NZTRM 61%"5&FNQ@EFUBJMT4&5EFUBJMT+40/@*/4&35
EFUBJMT
BEESFTTQJOBEESFTTMJOF"8JOH8)&3&
FNQ@OP
2VFSZ0,SPXBGGFDUFE
TFD
In this case, QJO won't be updated; only a new BEESFTTMJOF field will be
added.
+40/@3&1-"$&
: Replaces only existing values
Suppose you want to just replace the fields without adding new fields:
NZTRM 61%"5&FNQ@EFUBJMT4&5EFUBJMT+40/@3&1-"$&
EFUBJMT
BEESFTTQJOBEESFTTMJOF-BOENBSL8)&3&
FNQ@OP
2VFSZ0,SPXBGGFDUFE
TFD
In this case, MJOF won't be added. Only QJO will be updated.
Removing
+40/@3&.07& removes data from a JSON document.
Suppose you no longer need MJOF in the address:

NZTRM 61%"5&FNQ@EFUBJMT4&5EFUBJMT+40/@3&.07&
EFUBJMT
BEESFTTMJOF8)&3&FNQ@OP
2VFSZ0,SPXBGGFDUFE
TFD
[ 117 ]

Other functions
Some of the other functions are as follows:
+40/@,&:4
: Gets all the keys in a JSON document:
NZTRM 4&-&$5+40/@,&:4
EFUBJMT'30.FNQ@EFUBJMT8)&3&FNQ@OP
SPX
+40/@,&:4
EFUBJMT<FNBJMQIPOFBEESFTTOJDLOBNF
MPDBUBUJPO>
+40/@-&/(5)
: Gives the number of elements in a JSON document:
NZTRM 4&-&$5+40/@-&/(5)
EFUBJMT'30.FNQ@EFUBJMT8)&3&FNQ@OP

SPX
+40/@-&/(5)
EFUBJMT
See also
You can check out the complete list of functions at IUUQTEFWNZTRMDPNEPDSFGNBO
FOKTPOGVODUJPOSFGFSFODFIUNM.
Common table expressions (CTE)

MySQL 8 supports common table expressions, both non-recursive and recursive.
Common table expressions enable the use of named temporary result sets, implemented by
permitting a 8*5) clause preceding 4&-&$5 statements and certain other statements.
Why do you need CTEs?

It is not possible to refer to a derived table twice in the same query. So the
derived tables are evaluated twice or as many times as referred, which
indicates a serious performance problem. Using CTE, the subquery is
evaluated only once.
[ 118 ]

How to do it...
Recursive and non-recursive CTE will be looked into in the following sections.
Non-recursive CTE
A Common table expression (CTE) is just like a derived table, but its declaration is put
before the query block instead of in '30. clause.
Derived Table
4&-&$5'30.
TVCRVFSZ"4EFSJWFEU
CTE
4&-&$58*5)EFSJWFE"4
TVCRVFSZ4&-&$5'30.EFSJWFEU
A CTE may precede 4&-&$5/61%"5&/%&-&5&, including subqueries 8*5) derived "4

(subquery), for example:
%&-&5&'30.U8)&3&UB*/
4&-&$5C'30.EFSJWFE
Suppose you want to find out the percentage increase in the salary of each year with respect
to the previous year. Without CTE, you need to write two subqueries, and they are
essentially the same. MySQL is not smart enough to detect that and the subqueries are
executed twice:
NZTRM 4&-&$5
RZFBS
RZFBS"4OFYU@ZFBS
RTVN
RTVN"4OFYU@TVN

RTVNRTVNRTVN"4QDU
'30.

4&-&$5ZFBS
GSPN@EBUFBTZFBSTVN
TBMBSZBTTVN'30.TBMBSJFT(3061
#:ZFBS"4R
4&-&$5ZFBS
GSPN@EBUFBTZFBSTVN
TBMBSZBT
TVN'30.TBMBSJFT(3061#:ZFBS"4R
8)&3&RZFBSRZFBS

]ZFBS]OFYU@ZFBS]TVN]OFYU@TVN]QDU]

]]]]]]
]]]]]]
]]]]]]
]]]]]]
[ 119 ]

]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]

SPXTJOTFU
TFD
With non-recursive CTE, the derived query is executed only once and reused:
NZTRM
8*5)$5&"4

4&-&$5ZFBS
GSPN@EBUF"4ZFBS46.
TBMBSZ"4TVN'30.TBMBSJFT(3061
#:ZFBS
4&-&$5
RZFBSRZFBSBTOFYU@ZFBSRTVNRTVNBTOFYU@TVN
RTVN
RTVNRTVNBTQDU'30.
$5&"4R
$5&"4R
8)&3&
RZFBSRZFBS

]ZFBS]OFYU@ZFBS]TVN]OFYU@TVN]QDU]

]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]
[ 120 ]

]]]]]]
]]]]]]
]]]]]]
]]]]]]
]]]]]]

SPXTJOTFU
TFD
You may notice that with CTE, the results are the same and query time improves by 50%;
the readability is good and can be referenced multiple times.
Derived queries cannot refer to other derived queries:

4&-&$5
'30.
4&-&$5'30."4E
4&-&$5'30.E"4E
&3303
45BCMFbECEcEPFTOcUFYJTU
CTEs can refer to other CTEs:

8*5)E"4
4&-&$5'30.E"4
4&-&$5'30.E
4&-&$5
'30.EE
Recursive CTE
A recursive CTE is a CTE with a subquery that refers to its own name. The 8*5) clause
must begin with 8*5)3&$634*7&. The recursive CTE subquery has two parts, seed query
and recursive query, separated by 6/*0/<"--> or 6/*0/%*45*/$5.
Seed 4&-&$5 is executed once to create the initial data subset; recursive 4&-&$5 is
repeatedly executed to return subsets of data until the complete result set is obtained.
Recursion stops when an iteration does not generate any new rows. This is useful to dig
into hierarchies (parent/child or part/subpart):
8*5)3&$634*7&DUF"4

4&-&$5'30.UBCMF@OBNFTFFE4&-&$5
6/*0/"--
4&-&$5'30.DUFUBCMF@OBNFSFDVSTJWF4&-&$5
4&-&$5'30.DUF
[ 121 ]

Suppose you want to print all numbers from to :

NZTRM 8*5)3&$634*7&DUF
O"4

4&-&$5TFFERVFSZ
6/*0/"--
4&-&$5O '30.DUF8)&3&OSFDVSTJWFRVFSZ

4&-&$5'30.DUF

]O]

]]
]]
]]
]]
]]

SPXTJOTFU
TFD
At each iteration, 4&-&$5 produces a row with a new value one more than the value of O
from the previous row set. The first iteration operates on the initial row set (1) and produces
1+1=2; the second iteration operates on the first iteration's row set (2) and produces 2+1=3;
and so on and so forth. This continues until the recursion ends, when O is no longer less
than .
Suppose you want to do hierarchical data traversal to produce an organizational chart with
the management chain for each employee (that is, the path from CEO to employee). Use a
recursive CTE!
Create a test table with NBOBHFS@JE:

NZTRM $3&"5&5"#-&FNQMPZFFT@NHS
JE*/513*."3:,&:/05/6--
OBNF7"3$)"3
/05/6--
NBOBHFS@JE*/5/6--
*/%&9
NBOBHFS@JE
'03&*(/,&:
NBOBHFS@JE3&'&3&/$&4FNQMPZFFT@NHS
JE

[ 122 ]

Insert sample data:

NZTRM */4&35*/50FNQMPZFFT@NHS7"-6&4

:BTNJOB/6--:BTNJOBJTUIF$&0
NBOBHFS@JEJT/6--

+PIO+PIOIBT*%BOESFQPSUTUP
:BTNJOB

5BSFL

1FESP

4BSBI

1JFSSF

"EJM
Execute the recursive CTE:

NZTRM 8*5)3&$634*7&FNQMPZFF@QBUIT
JEOBNFQBUI"4

4&-&$5JEOBNF$"45
JE"4$)"3

'30.FNQMPZFFT@NHS
8)&3&NBOBHFS@JE*4/6--
6/*0/"--
4&-&$5FJEFOBNF$0/$"5
FQQBUI FJE
'30.FNQMPZFF@QBUIT"4FQ+0*/FNQMPZFFT@NHS"4F
0/FQJEFNBOBHFS@JE

4&-&$5'30.FNQMPZFF@QBUIT03%&3#:QBUI
It produces the following results:

]JE]OBNF]QBUI]

]]:BTNJOB]]
]]+PIO]]
]]1FESP]]
]]4BSBI]]
]]1JFSSF]]
]]5BSFL]]
]]"EJM]]

SPXTJOTFU
TFD
8*5)3&$634*7&FNQMPZFF@QBUIT
JEOBNFQBUI"4 is the name of the CTE and
the columns are (JE, OBNF, QBUI).
[ 123 ]

4&-&$5JEOBNF$"45
JE"4$)"3
'30.FNQMPZFFT@NHS8)&3&
NBOBHFS@JE*4/6-- is the seed query where the CEO is selected (CEO does not have a
manager).
4&-&$5FJEFOBNF$0/$"5
FQQBUI FJE'30.FNQMPZFF@QBUIT"4
FQ+0*/FNQMPZFFT@NHS"4F0/FQJEFNBOBHFS@JE is the recursive query.
Each row produced by the recursive query finds all employees who report directly to an
employee produced by a previous row. For every such employee, the row includes the
employee ID, name, and employee management chain. The chain is the manager's chain
with the employee ID added at the end.
Generated columns
The generated columns are also known as virtual or computed columns. The values of a
generated column are computed from an expression included in the column definition.
There are two types of generated columns:
Virtual: The column will be calculated on the fly when a record is read from a
table
Stored: The column will be calculated when a new record is written in the table
and will be stored in the table as a regular column
Virtual generated columns are more useful than stored generated columns because a virtual
column does not take any storage space. You can simulate the behavior of stored generated
columns using triggers.
How to do it...
Suppose your application is using GVMM@OBNF as DPODBU
GJSTU@OBNF
MBTU@OBNF while retrieving the data from the FNQMPZFFT table; instead of using the
expression, you can use a virtual column, which calculates GVMM@OBNF on the fly. You can
add another column followed by the expression:
NZTRM $3&"5&5"#-&AFNQMPZFFTA
AFNQ@OPAJOU
/05/6--
ACJSUI@EBUFAEBUF/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
[ 124 ]

AIJSF@EBUFAEBUF/05/6--
`full_name` VARCHAR(30) AS (CONCAT(first_name,' ',last_name)),
13*."3:,&:
AFNQ@OPA
,&:AOBNFA
AGJSTU@OBNFAAMBTU@OBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
Note that you should modify the insert statement according to the virtual column. You have
an option to use the full insert like this:
NZTRM */4&35*/50FNQMPZFFT
FNQ@OPCJSUI@EBUFGJSTU@OBNFMBTU@OBNF
HFOEFSIJSF@EBUF7"-6&4
"#$ 9:; '

If you want to include GVMM@OBNF in the */4&35 statement, you can specify it as %&'"6-5
only. All other values throw an &3303
): error. The value specified for the
generated column, GVMM@OBNF, in the FNQMPZFFT table is not allowed:
NZTRM */4&35*/50FNQMPZFFT
FNQ@OPCJSUI@EBUFGJSTU@OBNFMBTU@OBNF
HFOEFSIJSF@EBUFGVMM@OBNF7"-6&4
"#$ 9:;
' %&'"6-5
You can directly select GVMM@OBNF from the FNQMPZFFT table:

NZTRM 4&-&$5'30.FNQMPZFFT8)&3&FNQ@OP

full_name]

]]]"#$]9:;]']]ABC
XYZ]

SPXJOTFU
TFD
If you have already created the table and wish to add new generated column, execute the
ALTER TABLE statement which will be covered in detail in Chapter 10 - Table Maintenance
Example:
NZTRM "-5&35"#-&FNQMPZFFT"%%IJSF@EBUF@ZFBS:&"3"4
:&"3
IJSF@EBUF
7*356"-
[ 125 ]

Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFODSFBUFUBCMFHFOFSBUFEDPMVNOT
IUNM to know more about generated columns. You will understand the other uses of virtual
columns in $IBQUFS, Performance Tuning, in the Adding indexes and Index for JSON using
generated columns sections.
Window functions
By using window functions, for each row from a query, you can perform a calculation using
rows related to that row. This is accomplished by using the 07&3 and 8*/%08 clauses.
Here is the list of calculations that you can do:
$6.&@%*45
: Cumulative distribution value
%&/4&@3"/,
: Rank of the current row within its partition, without gaps
'*345@7"-6&
: The value of the argument from the first row of the window
frame
-"(
: The argument value from the row lagging the current row within
partition
-"45@7"-6&
: Value of argument from the first row of window frame
-&"%
: Value of argument from row leading current row within partition
/5)@7"-6&
: Argument value from n-th row of window frame
/5*-&
: Bucket number of the current row within its partition
1&3$&/5@3"/,
: Percentage rank value
3"/,
: Rank of the current row within its partition, with gaps
308@/6.#&3
: Number of the current row within its partition
How to do it...
Window functions can be used in various ways. Let's get to grips with each one of them in
the following sections. For these examples to work, you need to add hire_date_year virtual
column
:&"3
IJSF@EBUF
7*356"-
[ 126 ]

Row number
You can get the row number for each row to rank the results:
NZTRM 4&-&$5$0/$"5
GJSTU@OBNFMBTU@OBNF"4GVMM@OBNFTBMBSZ
308@/6.#&3
07&3
03%&3#:TBMBSZ%&4$"4 3BOL '30.FNQMPZFFT+0*/
TBMBSJFT0/TBMBSJFTFNQ@OPFNQMPZFFTFNQ@OP-*.*5

]GVMM@OBNF]TBMBSZ]3BOL]

]5PLVZBTV1FTDI]]]
]5PLVZBTV1FTDI]]]
])POFTUZ.VLBJEPOP]]]
]9JBIVB8IJUDPNC]]]
]4BOKBJ-VEFST]]]
]5TVUPNV"MBNFMEJO]]]
]8JMMBSE#BDB]]]

SPXTJOTFU
TFD
Partition results
You can partition the results in the 07&3 clause. Suppose you want to find out the salary
rank for each year; it can be done as follows:
NZTRM 4&-&$5IJSF@EBUF@ZFBSTBMBSZ308@/6.#&3
07&3
1"35*5*0/#:
IJSF@EBUF@ZFBS03%&3#:TBMBSZ%&4$"4 3BOL '30.FNQMPZFFT+0*/TBMBSJFT
0/TBMBSJFTFNQ@OPFNQMPZFFTFNQ@OP03%&3#:TBMBSZ%&4$-*.*5

]IJSF@EBUF@ZFBS]TBMBSZ]3BOL]

]]]]
]]]]
| 1986 | 156286 | 1 |
]]]]
| 1987 | 155513 | 1 |
]]]]
]]]]
]]]]
]]]]
]]]]

SPXTJOTFU
TFD
[ 127 ]

You can notice that the rank changed for the years and but the ranking for
continued.
Named windows
You can name a window and use it as many times as you need rather than redefining it
every time:
NZTRM 4&-&$5IJSF@EBUF@ZFBSTBMBSZ3"/,
07&3X"4 3BOL '30.
FNQMPZFFTKPJOTBMBSJFT0/TBMBSJFTFNQ@OPFNQMPZFFTFNQ@OP WINDOW w AS
(PARTITION BY hire_date_year ORDER BY salary DESC)03%&3#:TBMBSZ%&4$
-*.*5

]IJSF@EBUF@ZFBS]TBMBSZ]3BOL]

]]]]
]]]]
]]]]
]]]]
]]]]
]]]]
]]]]
]]]]
]]]]
]]]]

SPXTJOTFU
TFD
First, last, and nth values

You can select the first, last, and nth values in the window results. If the row does not exist,
a /6-- value is returned.
Suppose you want to find the first, last, and third values from the window:
NZTRM 4&-&$5IJSF@EBUF@ZFBSTBMBSZ3"/,
07&3X"4 3BOL
'*345@7"-6&
TBMBSZ07&3X"4 GJSTU
/5)@7"-6&
TBMBSZ07&3X"4 UIJSE
-"45@7"-6&
TBMBSZ07&3X"4 MBTU
'30.FNQMPZFFTKPJOTBMBSJFT0/TBMBSJFTFNQ@OPFNQMPZFFTFNQ@OP
8*/%08X"4
1"35*5*0/#:IJSF@EBUF@ZFBS03%&3#:TBMBSZ%&4$
03%&3#:TBMBSZ%&4$-*.*5

]IJSF@EBUF@ZFBS]TBMBSZ]3BOL]GJSTU]UIJSE]MBTU]
[ 128 ]


]]]]]/6--]]
]]]]]/6--]]
]]]]]/6--]]
]]]]]]]
]]]]]/6--]]
]]]]]]]
]]]]]]]
]]]]]]]
]]]]]]]
]]]]]]]

SPXTJOTFU
TFD
To know more about the other use cases of window functions, refer to IUUQT
NZTRMTFSWFSUFBNDPNNZTRMJOUSPEVDJOHXJOEPXGVODUJPOT and IUUQTEFW
NZTRMDPNEPDSFGNBOFOXJOEPXGVODUJPOEFTDSJQUJPOTIUNMGVODUJPO@SPX
OVNCFS.
[ 129 ]

Configuring MySQL
4
Using config file

Using global and session variables
Using parameters with startup script
Configuring the parameters
Changing the data directory
Introduction
MySQL has two types of parameters:
Static, which takes effect after restarting MySQL server

Dynamic, which can be changed on the fly without restarting MySQL server
Variables can be set through the following:
Config file: MySQL has a configuration file in which we can specify the location
of data, the memory that MySQL can use, and various other parameters.
Startup script: You can directly pass the parameters to the NZTRME process. It
remains in effect only for that invocation of the server.
Using SET command (only dynamic variables): This will last until the server
restarts. You also need to set the variable in the config file to make the change
persistent across restarts. Another way to make changes persistent is by
preceding the variable name by the 1&34*45 keyword or !!QFSTJTU.

Configuring MySQL Chapter 4
Using config file

The default config file is FUDNZDOG (on Red Hat and CentOS systems) and
FUDNZTRMNZDOG (Debian systems). Open the file in your favorite editor and modify
the parameters as needed. The main parameters are discussed in this chapter.
How to do it...
The config file has sections specified by TFDUJPO@OBNF. All the parameters related to a
section can be put under them, for example:
[mysqld] <---section name
QBSBNFUFS@OBNF WBMVF QBSBNFUFSWBMVFT
[client]
QBSBNFUFS@OBNF WBMVF
[mysqldump]
[mysqld_safe]
[server]
<NZTRM>: Section is read by the NZTRM command-line client

<DMJFOU>: Section is read by all connecting clients (including NZTRMDMJ)
<NZTRME>: Section is read by the NZTRM server
<NZTRMEVNQ>: The section is read by the backup utility called NZTRMEVNQ
<NZTRME@TBGF>: Read by the NZTRME@TBGF process (MySQL Server Startup
Script)
Apart from that the NZTRME@TBGF process reads all options from the <NZTRME> and
<TFSWFS> sections in option files.
For example, NZTRME@TBGF process reads the QJEGJMF option from NZTRME section.
TIFMM TVEPWJFUDNZDOG
<NZTRME>
QJEGJMFWBSMJCNZTRMNZTRMEQJE
In systems that use TZTUFNE, NZTRME@TBGF will not be installed. To configure the startup
script, you need to set the values
in FUDTZTUFNETZTUFNNZTRMETFSWJDFEPWFSSJEFDPOG.
[ 131 ]

For example:
<4FSWJDF>
-JNJU/0'*-&NBY@PQFO@GJMFT
1*%'JMFQBUIUPQJEGJMF
-JNJU$PSFDPSF@GJMF@MJNJU
&OWJSPONFOU-%@13&-0"%QBUIUPNBMMPDMJCSBSZ
&OWJSPONFOU5;UJNF@[POF@TFUUJOH
Using global and session variables

As you have seen in the previous chapters, you can set the parameters by connecting to
MySQL and executing the 4&5 command.
There are two types of variables based on the scope of the variable:
Global: Applies to all the new connections

Session: Applies only to the current connection (session)
How to do it...
For example, if you want to log all queries that are slower than one second, you can execute:
NZTRM 4&5(-0#"-MPOH@RVFSZ@UJNF
To make the changes persistent across restarts use:

NZTRM 4&51&34*45MPOH@RVFSZ@UJNF
2VFSZ0,SPXTBGGFDUFE
TFD
Or:
NZTRM 4&5!!QFSTJTUMPOH@RVFSZ@UJNF
2VFSZ0,SPXTBGGFDUFE
TFD
The persisted global system variable settings are stored in mysqld-auto.cnf which is located
in data directory.
Suppose you want to log queries only for this session and not for all the connections. You
can use the following command:
NZTRM 4&54&44*0/MPOH@RVFSZ@UJNF
[ 132 ]

Using parameters with startup script

Suppose you wish to start MySQL using a startup script and not through TZTUFNE,
especially for testing or for some temporary change. You can pass the variables to the script
rather than changing it in the config file.
How to do it...
TIFMM VTSMPDBMNZTRMCJONZTRMECBTFEJSVTSMPDBMNZTRM
EBUBEJSVTSMPDBMNZTRMEBUBQMVHJOEJSVTSMPDBMNZTRMMJCQMVHJO
VTFSNZTRMMPHFSSPSVTSMPDBMNZTRMEBUBDFOUPTFSSQJE
GJMFVTSMPDBMNZTRMEBUBDFOUPTQJEJOJUGJMFUNQNZTRMJOJU
You can see that the JOJUGJMF parameter is passed to the server. The server executes
the SQL statements in that file before starting.
Configuring the parameters

After installation, the basic things you need to configure are covered in this section. The rest
all can be left as default or tuned later according to the load.
How to do it...
Let's get into the details.
data directory
Data managed by the MySQL server is stored under a directory known as the EBUB
EJSFDUPSZ. Each sub-directory of the EBUBEJSFDUPSZ is a database directory and
corresponds to a database managed by the server. By default, the
EBUBEJSFDUPSZ has three sub directories:
NZTRM: MySQL system database

QFSGPSNBODF@TDIFNB: Provides information used to inspect the internal
execution of the server at runtime
TZT: Provides a set of objects to help interpret performance schema information
more easily
[ 133 ]

Apart from these, the EBUBEJSFDUPSZ contains the log files, *OOP%# tablespace and
*OOP%# log files, SSL and RSA key files, QJE of NZTRME, and NZTRMEBVUPDOG, which
stores persisted global system variable settings.
To set the EBUBEJSFDUPSZ change/add the value of EBUBEJS to the config file. The default
is WBSMJCNZTRM:
<NZTRME>
EBUBEJSEBUBNZTRM
You can set it to wherever you want to store the data, but you should change the ownership
of the EBUBEJSFDUPSZ to NZTRM.
Make sure that the disk volume bearing the EBUBEJSFDUPSZ has
sufficient space to hold all your data.
innodb_buffer_pool_size
This is the most important tuning parameter that decides how much memory the *OOP%#
storage engine can use to cache data and indexes in memory. Setting it too low can degrade
the performance of the MySQL server, and setting it too high can increase the memory
consumption of MySQL process. The best thing about MySQL 8 is
that JOOPEC@CVGGFS@QPPM@TJ[F is dynamic, meaning you can vary
JOOPEC@CVGGFS@QPPM@TJ[F without restarting the server.
Here is a simple guide on how to tune it:
1. Find out the size of your dataset. Do not set the value of
JOOPEC@CVGGFS@QPPM@TJ[F higher than that of your dataset. Suppose you have
a 12 GB RAM machine and your dataset is 3 GB; then you can set
JOOPEC@CVGGFS@QPPM@TJ[F to 3 GB. If you expect growth in your data, you can
increase it as and when needed without restarting MySQL.
[ 134 ]

2. Usually, the size of the dataset is much bigger than the available RAM. Out of the
total RAM, you can set some for the operating system, some for other processes,
some for per-thread buffers inside MySQL, and some for the MySQL server apart
from *OOP%#. The rest can be assigned to the *OOP%# buffer pool size.
This is a very generic table and gives you a good value to start with, assuming
that it is a dedicated MySQL server, all the tables are *OOP%#, and per-thread
buffers are left as default. If the system is running out of memory, you can
decrease the buffer pool dynamically.
RAM Buffer Pool Size (Range)
4 GB 1 GB-2 GB
8 GB 4 GB-6 GB
12 GB 6 GB-10 GB
16 GB 10 GB-12 GB
32 GB 24 GB-28 GB
64 GB 45 GB-56 GB
128 GB 108 GB-116 GB
256 GB 220 GB-245 GB
innodb_buffer_pool_instances
You can divided the *OOP%# buffer pool into separate regions to improve concurrency, by
reducing contention as different threads read and write to cached pages. For example, if the
buffer pool size is 64 GB and JOOPEC@CVGGFS@QPPM@JOTUBODFT are 32, the buffer is split
into 32 regions with 2 GB each.
If the buffer pool size is more than 16 GB, you can set the instances so that each region gets
at least 1 GB of space.
[ 135 ]

innodb_log_file_size
This is the size of the redo log space used to replay committed transactions in case of a
database crash. The default is 48 MB, which may not be sufficient for production workloads.
To start with, you can set 1 GB or 2 GB. This change requires a restart. Stop the MySQL
server and make sure that it shuts down without errors. Make the changes in NZDOG and
start the server. In earlier versions, you need to stop the server, remove the log files, and
then start the server. In MySQL 8, it is automatic. Modifying the redo log files is explained
in $IBQUFS, Managing Tablespace, in the Changing the number or size of InnoDB redo log files
section.
Changing the data directory

Your data can grow over time, and when it outgrows the filesystem, you need to add a disk
or move the EBUBEJSFDUPSZ to a bigger volume.
How to do it...
1. Check the current EBUBEJSFDUPSZ. By default, the EBUBEJSFDUPSZ is
WBSMJCNZTRM:
NZTRM TIPXWBSJBCMFTMJLF EBUBEJS


]EBUBEJS]WBSMJCNZTRM]

SPXJOTFU
TFD
2. Stop NZTRM and make sure it has stopped successfully:
TIFMM TVEPTZTUFNDUMTUPQNZTRM
3. Check the status:
TIFMM TVEPTZTUFNDUMTUBUVTNZTRM
It should show 4UPQQFE.Z42-$PNNVOJUZ4FSWFS.
[ 136 ]

4. Create the directory at the new location and change the ownership to NZTRM:
TIFMM TVEPNLEJSQWEBUB
TIFMM TVEPDIPXO3NZTRMNZTRMEBUB
5. Move the files to the new EBUBEJSFDUPSZ:
TIFMM TVEPSTZODBWWBSMJCNZTRMEBUB
6. In Ubuntu, if you've enabled AppArmor, you need to configure the Access

Control:
TIFMM WJFUDBQQBSNPSEUVOBCMFTBMJBT
BMJBTWBSMJCNZTRM EBUBNZTRM
TIFMM TVEPTZTUFNDUMSFTUBSUBQQBSNPS
7. Start MySQL server and verify that the EBUB directory has changed:
TIFMM TVEPTZTUFNDUMTUBSUNZTRM
NZTRM TIPXWBSJBCMFTMJLF EBUBEJS


]EBUBEJS]EBUBNZTRM]

SPXJOTFU
TFD
8. Verify that the data is intact and remove the old EBUBEJSFDUPSZ:
TIFMM TVEPSNSGWBSMJCNZTRM
If MySQL fails to starts with the errorg.Z42-EBUBEJSOPUGPVOEBU

WBSMJCNZTRMQMFBTFDSFBUFPOF:
Execute, TVEPNLEJSWBSMJCNZTRMNZTRMQ
If it says .Z42-TZTUFNEBUBCBTFOPUGPVOE,
run the ;NZTRM@JOTUBMM@EC tool, which creates the required directories.
[ 137 ]

Transactions
5
Performing transactions
Using savepoints
Isolation levels
Locking
Introduction
In the following recipes, we will discuss the transactions and various isolation levels in
MySQL. Transaction means a set of SQL statements that should succeed or fail together.
Transactions should also satisfy Atomicity, Consistency, Isolation, and Durability(ACID)
properties. Take a very basic example of a money transfer from account " to account #.
Assume that " has $600, # has $400, and # wishes to transfer $100 from " to itself.
The bank would deduct $100 from " and add to # using the following SQL code (for
illustration):
NZTRM 4&-&$5CBMBODF*/50!BCBM'30.BDDPVOU8)&3&BDDPVOU@OVNCFS "
Programmatically, check whether !BCBM is greater than or equal to 100:

NZTRM 61%"5&BDDPVOU4&5CBMBODF!BCBM8)&3&BDDPVOU@OVNCFS "
NZTRM 4&-&$5CBMBODF*/50!CCBM'30.BDDPVOU8)&3&BDDPVOU@OVNCFS #

Transactions Chapter 5
Programmatically, check whether !CCBM is /05/6--:

NZTRM 61%"5&BDDPVOU4&5CBMBODF!CCBM 8)&3&BDDPVOU@OVNCFS #
These four SQL lines should be part of a single transaction and satisfy the following ACID
properties:
Atomicity: Either all the SQLs should be successful or all should fail. There
should not be any partial updates. If this property is not obeyed and if the
database crashes after running two SQLs, then " would lose 100.
Consistency: A transaction must change affected data only in allowed ways. In
this example, if BDDPVOU@OVNCFS with # does not exist, the whole transaction
should be rolled back.
Isolation: Transactions that occur at the same time (concurrent transactions)
should not lead the database to an inconsistent state. Each of the transactions
should be executed as if it were the only transaction in the system. No transaction
should affect the existence of any other transaction. Suppose " transfers all of this
600 exactly at the same time while transferring to #; both transactions should act
independently, ensuring the balance before transferring the amount.
Durability: The data should be persistent on disk and should not be lost despite
any database or system failure.
*OOP%#, the default storage engine in MySQL, supports transactions whereas MyISAM
does not support them.
Performing transactions
Create dummy tables and sample data to understand this recipe:
NZTRM $3&"5&%"5"#"4&CBOL
NZTRM 64&CBOL
NZTRM $3&"5&5"#-&BDDPVOU
BDDPVOU@OVNCFSWBSDIBS
13*."3:,&:CBMBODF
JOU
NZTRM */4&35*/50BDDPVOU7"-6&4
"
#
[ 139 ]

How to do it...
To start a transaction (set of SQLs), execute the 45"3553"/4"$5*0/ or #&(*/ statement:
NZTRM 45"3553"/4"$5*0/
PS
NZTRM #&(*/
Then execute all the statements that you wish to be inside a transaction, such as transferring
100 from " to #:
1SPHSBNNBUJDBMMZDIFDLJG!BCBMJTHSFBUFSUIBOPSFRVBMUP
NZTRM 4&-&$5CBMBODF*/50!CCBM'30.BDDPVOU8)&3&BDDPVOU@OVNCFS #
1SPHSBNNBUJDBMMZDIFDLJG!CCBM*4/05/6--
NZTRM 61%"5&BDDPVOU4&5CBMBODF!CCBM 8)&3&BDDPVOU@OVNCFS #
After making sure that all the SQLs are executed successfully, execute the $0..*5
statement, which will finish the transaction and commit the data:
NZTRM $0..*5
If you encounter any error in between and wish to abort the transaction, you can issue a
30--#"$, statement instead of $0..*5.
For example, instead of sending to #, if " wants to transfer to an account that does not exist,
you should abort the transaction and refund the amount to ":
NZTRM #&(*/
NZTRM 4&-&$5CBMBODF*/50!CCBM'30.BDDPVOU8)&3&BDDPVOU@OVNCFS $
TFD
NZTRM 4)088"3/*/(4

]-FWFM]$PEF].FTTBHF]

]8BSOJOH]]/PEBUB[FSPSPXTGFUDIFETFMFDUFEPSQSPDFTTFE]

SPXJOTFU
TFD
[ 140 ]

NZTRM 4&-&$5!CCBM

]!CCBM]

]/6--]

SPXJOTFU
TFD
NZTRM 30--#"$,
2VFSZ0,SPXTBGGFDUFE
TFD
Autocommit
By default, autocommit is 0/, which means that all individual statements are committed as
soon as they are executed unless they are in a #&(*/$0..*5 block. If autocommit is
0'', you need to explicitly issue a $0..*5 statement to commit a transaction. To disable it,
execute:
NZTRM 4&5BVUPDPNNJU
DDL statements, such as $3&"5& or %301 for databases and $3&"5&, %301, or "-5&3 for
tables or stored routines cannot be rolled back.
There are certain statements such as DDLs, -0"%%"5"*/'*-&, "/"-:;&

5"#-&, replication-related statements and so on that cause implicit
$0..*5. For more details on these statements, refer IUUQTEFWNZTRM
DPNEPDSFGNBOFOJNQMJDJUDPNNJUIUNM.
Using savepoints
Using savepoints, you can roll back to certain points in the transaction without terminating
the transaction. You can use 4"7&10*/5JEFOUJGJFS to set a name for the transaction and
use the 30--#"$,50JEFOUJGJFS statement to roll back a transaction to the named
savepoint without terminating the transaction.
[ 141 ]

How to do it...
Suppose " wants to transfer to multiple accounts; even if a transfer to one account fails, the
others should not be rolled back:
NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD

2VFSZ0,SPXBGGFDUFE
TFD

2VFSZ0,SPXBGGFDUFE
TFD
NZTRM 61%"5&BDDPVOU4&5CBMBODFCBMBODF 8)&3&BDDPVOU@OVNCFS #

2VFSZ0,SPXBGGFDUFE
TFD
mysql> SAVEPOINT transfer_to_b;

2VFSZ0,SPXTBGGFDUFE
TFD

2VFSZ0,SPXBGGFDUFE
TFD
NZTRM 61%"5&BDDPVOU4&5CBMBODFCBMBODF 8)&3&BDDPVOU@OVNCFS $

2VFSZ0,SPXTBGGFDUFE
TFD
### Since there are no rows updated, meaning there is no account with 'C',
you can rollback the transaction to SAVEPOINT where transfer to B is
successful. Then 'A' will get back 100 which was deducted to transfer to C.
If you wish not to use the save point, you should do these in two
transactions.
mysql> ROLLBACK TO transfer_to_b;

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $0..*5
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5CBMBODF'30.BDDPVOU8)&3&BDDPVOU@OVNCFS "

]CBMBODF]

]]

[ 142 ]

SPXJOTFU
TFD
NZTRM 4&-&$5CBMBODF'30.BDDPVOU8)&3&BDDPVOU@OVNCFS #

]CBMBODF]

]]

SPXJOTFU
TFD
Isolation levels
When two or more transactions occur at the same time, the isolation level defines the degree
at which a transaction is isolated from the resource or data modifications made by other
transactions. There are four types of isolation levels; to change the isolation level, you need
to set the UY@JTPMBUJPO variable which is dynamic and has session level scope.
How to do it...
To change this level, execute 4&5!!USBOTBDUJPO@JTPMBUJPO 3&"%$0..*55&% .
Read uncommitted
The current transaction can read data written by another uncommitted transaction, which is
also called dirty read.
For example, " wants to add some amount to his account and transfer it to #. Assume both
the transactions happen at the same time; the flow will be like this.
[ 143 ]

" initially has $400 and wants to transfer $500 to # after adding $500 to his account.
# Transaction 1 (adding amount) # Transaction 2 (transferring amount)

#&(*/ #&(*/
61%"5&BDDPVOU
4&5CBMBODFCBMBODF --
8)&3&BDDPVOU@OVNCFS "
4&-&$5CBMBODF*/50!BCBM
'30.BDDPVOU
-- 8)&3&BDDPVOU@OVNCFS "
# A sees 900 here
30--#"$,
# Assume due to some reason the --
transaction got rolled back
# A transfers 900 to B since
A has 900 in previous SELECT
-- 61%"5&BDDPVOU
4&5CBMBODFCBMBODF
# B receives the amount
61%"5&BDDPVOU
-- 4&5CBMBODFCBMBODF
8)&3&BDDPVOU@OVNCFS #
5SBOTBDUJPODPNQMFUFTTVDDFTTGVMMZ
-- $0..*5
You can notice that Transaction 2 has read the data that is not committed or rolled back from
Transaction 1, causing account " to go into negative balance after this transaction, which is
clearly not desired.
Read committed
The current transaction can read only the data committed by another transaction, which is
also called non-repeatable read.
[ 144 ]

Take the same example again where " has $400 and # has $600.
# Transaction 1 (adding amount) # Transaction 2 (transferring amount)

#&(*/ #&(*/
61%"5&BDDPVOU4&5
CBMBODFCBMBODF --
'30.BDDPVOU
# A sees 400 here because transaction 1
has not committed the data yet
$0..*5 --
'30.BDDPVOU
# A sees 900 here because transaction 1
has committed the data.
You can notice that, in the same transaction, different results are fetched for the same
4&-&$5 statement.
Repeatable read
A transaction will see the same data that is read by the first statement even though another
transaction has committed the data. All consistent reads within the same transaction read
the snapshot established by the first read. An exception is a transaction that can read the
data changed within the same transaction.
When a transaction starts and executes its first read, a read view will be created and stay
open until the end of the transaction. In order to provide the same result set until the end of
the transaction, *OOP%# uses row versioning and 6/%0 information. Suppose Transaction 1
has selected a few rows and another transaction has deleted those rows and committed the
data. If Transaction 1 is open, it should be able to see the rows it has selected at the
beginning. The deleted rows are preserved in the 6/%0 log space to fulfill Transaction 1.
Once Transaction 1 finishes, the rows are marked to be deleted from the 6/%0 logs. This is
called Multi-Version Concurrency Control (MVCC).
[ 145 ]

Take the same example again where " has 400 and # has 600.
# Transaction 1 (adding the amount) # Transaction 2 (transferring the amount)

#&(*/ #&(*/
'30.BDDPVOU
"TFFTIFSF
61%"5&BDDPVOU
4&5CBMBODFCBMBODF --
'30.BDDPVOU
# A sees still 400 even though
transaction 1 is committed
$0..*5 --
-- $0..*5
'30.BDDPVOU
# A sees 900 here because this is a
freshtransaction
This applies only to 4&-&$5 statements and not necessarily to DML

statements. If you insert or modify some rows and then commit that
transaction, a %&-&5& or 61%"5& statement issued from another
concurrent 3&1&"5"#-&3&"% transaction could affect those just-
committed rows, even though the session cannot query them. If a
transaction does update or delete rows committed by a different
transaction, those changes become visible to the current transaction.
[ 146 ]

For example:
# Transaction 1 # Transaction 2
#&(*/ #&(*/
4&-&$5'30.BDDPVOU
# 2 rows are returned --
*/4&35*/50BDDPVOU7"-6&4
$
-- # New account is created
-- $0..*5
4&-&$5'30.BDDPVOU8)&3&
BDDPVOU@OVNCFS $
# no rows are returned because of --
MVCC
%&-&5&'30.BDDPVOU8)&3&
BDDPVOU@OVNCFS $
# Surprisingly account C gets --
deleted
4&-&$5'30.BDDPVOU
-- # 3 rows are returned because
transaction 1 is not yet committed
$0..*5 --
4&-&$5'30.BDDPVOU
-- # 2 rows are returned because
transaction 1 is committed
Here's another example:
#&(*/ #&(*/
4&-&$5'30.BDDPVOU
# 2 rows are returned --
*/4&35*/50BDDPVOU
-- 7"-6&4
%
-- $0..*5
4&-&$5'30.BDDPVOU
# 3 rows are returned because of MVCC --
[ 147 ]

61%"5&BDDPVOU4&5CBMBODF8)&3&
BDDPVOU@OVNCFS % --
Surprisingly account D gets updated
4&-&$5'30.BDDPVOU
# Surprisingly 4 rows are returned --
Serializable
This provides the highest level of isolation by locking all the rows that are being selected.
This level is like 3&1&"5"#-&3&"%, but *OOP%# implicitly converts all plain 4&-&$5
statements to 4&-&$5-0$,*/4)"3&.0%& if autocommit is disabled. If autocommit is
enabled, 4&-&$5 is its own transaction.
For example:
#&(*/ #&(*/
4&-&$5'30.BDDPVOU8)&3&
BDDPVOU@OVNCFS " --
61%"5&BDDPVOU4&5CBMBODF8)&3&
BDDPVOU@OVNCFS "
-- # This will wait until the lock held by
transaction 1
on row A is released
$0..*5 --
-- # UPDATE will be successful now
Another example:
#&(*/ #&(*/
4&-&$5'30.BDDPVOU8)&3&BDDPVOU@OVNCFS "
# Selects values of A --
*/4&35*/50BDDPVOU
-- 7"-6&4
%
# Inserts D
[ 148 ]

4&-&$5'30.BDDPVOU8)&3&BDDPVOU@OVNCFS %
5IJTXJMMXBJUVOUJMUIFUSBOTBDUJPO --
DPNQMFUFT
-- $0..*5
# Now the preceding select statement returns
values of D --
So, serializable waits for the locks and always reads the latest committed data.
Locking
There are two types of locking:
Internal locking: MySQL performs internal locking within the server itself to
manage contention for table contents by multiple sessions
External locking: MySQL gives the option to client sessions to acquire a table
lock explicitly for preventing other sessions from accessing the table
Internal locking: There are mainly two types of locks:
Row-level locks: The locking is granular to the level of rows. Only the rows that
are accessed are locked. This allows simultaneous write access by multiple
sessions, making them suitable for multi-user, highly concurrent, and OLTP
applications. Only *OOP%# supports row-level locks.
Table-level locks: MySQL uses table-level locking for .Z*4"., .&.03:, and
.&3(& tables, permitting only one session to update those tables at a time. This
locking level makes these storage engines more suitable for read-only, read-
mostly, or single-user applications.
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOJOUFSOBMMPDLJOHIUNM and

IUUQTEFWNZTRMDPNEPDSFGNBOFOJOOPECMPDLJOHIUNM to know more about
the *OOP%# locks.
External Locking: You can use -0$,5"#-& and 6/-0$,5"#-&4 statements to control the
locks.
[ 149 ]

Table locking for 3&"% and 83*5& are explained as follows:
3&"%: When a table is locked for 3&"%, multiple sessions can read data from the
table without acquiring a lock. Also, multiple sessions can acquire a lock on the
same table, that is why a 3&"% lock is also called a shared lock. When a 3&"% lock
is held, no session can write data into the table (including the session that held
the lock). If any write attempt is make, it will be in waiting state until the 3&"%
lock is released.
83*5&: When a table is locked for 83*5&, no other session can read and write
data from the table except the session that held the lock. No other session can
even acquire any lock until the existing lock is released. That is why this is called
FYDMVTJWFMPDL. If any read/write attempt is make, it will be in waiting state
until the 83*5& lock is released.
All the locks are released when the ;6/-0$,5"#-&4 statement is executed or when the
session terminates.
How to do it...
The syntax is as follows:
NZTRM -0$,5"#-&4UBCMF@OBNF<3&"%]83*5&>
To unlock the tables, use:

NZTRM 6/-0$,5"#-&4
To lock all tables across all databases, execute the following statement. It is used when
taking a consistent snapshot of the database. It freezes all writes to the database:
NZTRM '-64)5"#-&48*5)3&"%-0$,
Locking queue
No two locks can be held together on a table except shared locks (a table can have multiple
shared locks). If a table already has a shared lock and an exclusive lock comes, it will be
kept in a queue until the shared lock is released. When an exclusive lock is in a queue, all
subsequent shared locks are also blocked and kept in a queue.
[ 150 ]

*OOP%# acquires a metadata lock when reading/writing from a table. If a second transaction
requests 83*5&-0$,, it will be kept in a queue until the first transaction completes. If a
third transaction wants to read the data, it has to wait until the second transaction
completes.
Transaction 1:
NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5'30.FNQMPZFFT-*.*5


]]](FPSHJ]'BDFMMP].]]
]]]#F[BMFM]4JNNFM]']]
]]]1BSUP]#BNGPSE].]]
]]]$IJSTUJBO],PCMJDL].]]
]]],ZPJDIJ].BMJOJBL].]]
]]]"OOFLF]1SFVTJH]']]
]]]5[WFUBO];JFMJOTLJ]']]
]]]4BOJZB],BMMPVGJ].]]
]]]4VNBOU]1FBD]']]
]]]%VBOHLBFX]1JWFUFBV]']]

SPXTJOTFU
TFD
Note that $0..*5 is not executed. The transaction is kept open.
Transaction 2:
NZTRM -0$,5"#-&FNQMPZFFT83*5&
This statement has to wait until transaction 1 completes.
Transaction 3:
[ 151 ]

Even transaction 3 will not give any result because an exclusive lock is in the queue (it is
waiting for transaction 2 to complete). Moreover, it is blocking all operations on the table.
You can check this by checking 4)08130$&44-*45 from another session:

mysql> SHOW PROCESSLIST;

]*E]6TFS])PTU]EC]$PNNBOE]5JNF]4UBUF
]*OGP]

]]SPPU]MPDBMIPTU]FNQMPZFFT]4MFFQ]]
]/6--]
| 21 | root | localhost | employees | Query | 34 | Waiting for table
metadata lock | LOCK TABLE employees WRITE |
metadata lock | SELECT * FROM employees LIMIT 10 |
]]SPPU]MPDBMIPTU]FNQMPZFFT]2VFSZ]]TUBSUJOH
]4)08130$&44-*45]

SPXTJOTFU
TFD
You can notice that both transaction 2 and transaction 3 are waiting for transaction 1.
To know more about a metadata lock, refer to IUUQTEFWNZTRMDPNEPDSFGNBO

FONFUBEBUBMPDLJOHIUNM. The same behavior can be observed while using '-64)
5"#-&48*5)3&"%-0$,.
Transaction 1:
NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD


]]](FPSHJ]'BDFMMP].]]
]]]#F[BMFM]4JNNFM]']]
]]]1BSUP]#BNGPSE].]]
]]]$IJSTUJBO],PCMJDL].]]
]]],ZPJDIJ].BMJOJBL].]]
]]]"OOFLF]1SFVTJH]']]
]]]5[WFUBO];JFMJOTLJ]']]
]]]4BOJZB],BMMPVGJ].]]
[ 152 ]

]]]4VNBOU]1FBD]']]
]]]%VBOHLBFX]1JWFUFBV]']]

SPXTJOTFU
TFD
Note that $0..*5 is not executed. The transaction is kept open.
Transaction 2:
NZTRM '-64)5"#-&48*5)3&"%-0$,
Transaction 3:
Even transaction 3 will not give any result because '-64)5"#-&4 is waiting for all the
operations on the table to complete before getting a lock. Moreover, it is blocking all the
operations on the table.
You can check this by checking 4)08130$&44-*45 from another session.

NZTRM 4)08130$&44-*45

]*E]6TFS])PTU]EC]$PNNBOE]5JNF]4UBUF
]*OGP]

]]SPPU]MPDBMIPTU]FNQMPZFFT]2VFSZ]]$SFBUJOHTPSUJOEFY
]4&-&$5'30.FNQMPZFFT03%&3#:GJSTU@OBNF%&4$]
flush | FLUSH TABLES WITH READ LOCK |
flush | SELECT * FROM employees LIMIT 10 |
]]SPPU]MPDBMIPTU]FNQMPZFFT]2VFSZ]]TUBSUJOH
]4)08130$&44-*45]

SPXTJOTFU
TFD
For consistent backup, all the backup methods use '-64)5"#-&48*5)3&"%-0$,,

which can be very dangerous if there is a long-running transaction on table.
[ 153 ]

Binary Logging
6
Using binary logging

Binary log format
Extracting statements from a binary log
Ignoring databases to write to a binary log
Relocating binary logs
Introduction
The binary log contains a record of all changes to the database, both data and structure. The
binary log is not used for statements such as 4&-&$5 or 4)08 that do not modify
data. Running a server with binary logging enabled has a slight performance effect. The
binary log is crash-safe. Only complete events or transactions are logged or read back.
Why should you use binary log?
Replication: You stream the changes made to a server to another server using
binary logs. The slave acts as a mirror copy and can be used to distribute the load.
The server that accepts the writes is referred to as a master and the mirror copy
server is referred to as a slave.
Point-in-time recovery: Suppose you take a backup at 00:00 on Sunday and your
database crashed at 8:00 on Sunday. Using backup, you can recover till 00:00
Sunday. Using binary logs you can reply to them, to recover till 08:00.

Binary Logging Chapter 6
Using binary logging

To enable CJOMPH, you have to set MPH@CJO and TFSWFS@JE and restart the server. You can
mention the path and base name in the MPH@CJO itself. For example, MPH@CJO is set to
EBUBNZTRMCJOMPHTTFSWFS, the binary logs are stored in
the EBUBNZTRMCJOMPHT folder with the name TFSWFS, TFSWFS,
and so on. The server creates a new file in the series each time it starts or flushes the logs or
the current log's size reaches NBY@CJOMPH@TJ[F. It maintains the TFSWFSJOEFY file,
which contains the location of each binary log.
How to do it...
Let's see how to play with the logs. I am sure you are going to love learning about them.
Enabling binary logs

1. Enable binary logging and set the TFSWFS@JE. Open the MySQL DPOGJH file in
your favorite editor and append the following lines. Choose TFSWFS@JE such that
it will be unique to each MySQL server in your infrastructure.
You can also simply put the MPH@CJO variable in NZDOG without any value. In
that case, the binary log is created in the EBUBEJSFDUPSZ directory and
uses IPTUOBNF as its name.
<NZTRME>
MPH@CJOEBUBNZTRMCJOMPHTTFSWFS
TFSWFS@JE
2. Restart the MySQL server:
3. Verify that binary logs are created:
NZTRM 4)087"3*"#-&4-*,& MPH@CJO

]7BSJBCMF@OBNF]7BMVF
]

[ 155 ]

]MPH@CJO]0/
]
]MPH@CJO@CBTFOBNF]EBUBNZTRMCJOMPHTTFSWFS
]
]MPH@CJO@JOEFY]
EBUBNZTRMCJOMPHTTFSWFSJOEFY]
]MPH@CJO@USVTU@GVODUJPO@DSFBUPST]0''
]
]MPH@CJO@VTF@W@SPX@FWFOUT]0''
]

SPXTJOTFU
TFD
NZTRM 4)08."45&3-0(4

]-PH@OBNF]'JMF@TJ[F]

]TFSWFS]]

SPXJOTFU
TFD
TIFMM TVEPMTMIUSEBUBNZTRMCJOMPHT
UPUBM,
SXSNZTRMNZTRM"VHTFSWFSJOEFY
SXSNZTRMNZTRM"VHTFSWFS
4. Execute 4)08#*/"3:-0(4 or 4)08."45&3-0(4 to display all the binary

logs of the server.
5. Execute the 4)08."45&345"564 command to get the current binary log
position:
NZTRM 4)08."45&345"564

]'JMF]1PTJUJPO]#JOMPH@%P@%#]#JOMPH@*HOPSF@%#]
&YFDVUFE@(UJE@4FU]

]TFSWFS]]]]
]

SPXJOTFU
TFD
[ 156 ]

As soon as TFSWFS reaches NBY@CJOMPH@TJ[F (1 GB default), a new

file, TFSWFS, will be created and added to TFSWFSJOEFY. You can configure to
set NBY@CJOMPH@TJ[F dynamically with 4&5!!HMPCBMNBY@CJOMPH@TJ[F.
Disabling binary logs for a session

There may be cases where you do not want the statements to be replicated to other servers.
For that, you can use the following command to disable binary logging for that session:
NZTRM 4&542-@-0(@#*/
All the SQL statements logged after executing the previous statement are not logged to the
binary log. This applies only for that session.
To enable, you can execute the following:

NZTRM 4&542-@-0(@#*/
Move to the next log

You can use the '-64)-0(4 command to close the current binary log and open a new one:
NZTRM 4)08#*/"3:-0(4


]TFSWFS]]

SPXJOTFU
TFD
NZTRM '-64)-0(4
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)08#*/"3:-0(4


]TFSWFS]]
]TFSWFS]]

SPXTJOTFU
TFD
[ 157 ]

Expire binary logs

Binary logs consume a lot of space based on the number of writes. Leaving them as-is can
fill up the disk in no time. It is essential to clean them up:
1. Set the expiry of logs using CJOMPH@FYQJSF@MPHT@TFDPOET and

FYQJSF@MPHT@EBZT.
If you want to set an expiry period in days, set FYQJSF@MPHT@EBZT. For example,
if you want to delete all the binary logs that are older than two days, 4&5
!!HMPCBMFYQJSF@MPHT@EBZT. Setting the value to disables automatic
expiry.
If you want to have more granularity, you can use
the CJOMPH@FYQJSF@MPHT@TFDPOET variable, which sets the binary log
expiration period in seconds.
The effects of this variable and FYQJSF@MPHT@EBZT are cumulative. For example,
if FYQJSF@MPHT@EBZT is and CJOMPH@FYQJSF@MPHT@TFDPOET is , then
the binary log is purged every 1.5 days. This produces the same result as setting
CJOMPH@FYQJSF@MPHT@TFDPOET to and leaving FYQJSF@MPHT@EBZT set
to 0. In MySQL 8.0, both CJOMPH@FYQJSF@MPHT@TFDPOET and
FYQJSF@MPHT@EBZT must be set to 0 to disable automatic purging of the binary
log.
2. To purge the logs manually, execute 163(&#*/"3:-0(450 GJMF@OBNF .
For example, there are files such as TFSWFS, TFSWFS,
TFSWFS, and TFSWFS, and if you execute 163(&#*/"3:
-0(450 TFSWFS , all the files up to TFSWFS will be
deleted and TFSWFS will not be touched:
NZTRM 4)08#*/"3:-0(4


]TFSWFS]]
]TFSWFS]]
]TFSWFS]]
]TFSWFS]]

SPXTJOTFU
TFD
NZTRM 163(&#*/"3:-0(450 TFSWFS

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)08#*/"3:-0(4

[ 158 ]


]TFSWFS]]

SPXJOTFU
TFD
You can also execute the command 163(&#*/"3:-0(4#&'03&

rather than specifying a log file. You can also use the word ."45&3
instead of #*/"3:.
NZTRM 163(&."45&3-0(450 TFSWFS also works.
3. To delete all the binary logs and start from the beginning again, execute 3&4&5
."45&3:
NZTRM 4)08#*/"3:-0(4

]
]TFSWFS]]

SPXJOTFU
TFD
NZTRM 3&4&5."45&3
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)08#*/"3:-0(4


]TFSWFS]]

SPXJOTFU
TFD
Purging or expiring logs is a very unsafe method if you are using

replication. The safe way to purge binary logs is to use
the NZTRMCJOMPHQVSHF script, which will be covered in $IBQUFS,
Managing Logs.
[ 159 ]

Binary log format

Binary logs can be written in three formats:
1. 45"5&.&/5: Actual SQL statements are logged.

2. 308: Changes made to each row are logged.
For example, an update statement updates 10 rows, the updated information of
all 10 rows is written to the log. Whereas in statement-based replication, only the
update statement is written. The default format is 308.
3. .*9&%: MySQL switches from 45"5&.&/5 to 308 as and when needed.
There are statements that can cause different results when executed on different servers. For
example, the output of the 66*%
function differs from server to server. Those statements
are called non-deterministic and are unsafe for statement-based replication. In those
situations, a MySQL server switches to row-based format when you set the .*9&% format.
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOCJOBSZMPHNJYFEIUNM to learn

more about the unsafe statements and when switching happens.
How to do it...
You can set the format using the dynamic variable, CJOMPH@GPSNBU, which has both global
and session scope. Setting it at global level makes all clients use the specified format:
NZTRM 4&5(-0#"-CJOMPH@GPSNBU 45"5&.&/5
Or:
NZTRM 4&5(-0#"-CJOMPH@GPSNBU 308
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOSFQMJDBUJPOTCSSCSIUNMto learn

about the advantages and disadvantages of various formats.
1. MySQL 8.0 uses version 2 binary log row events, which cannot be read by
MySQL Server releases prior to MySQL 5.6.6. Set MPHCJOVTFWSPXFWFOUT
to to use version 1 so that it can be read by versions prior to MySQL 5.6.6. The
default is .
NZTRM 4&5!!(-0#"-MPH@CJO@VTF@W@SPX@FWFOUT
[ 160 ]

2. When you create a stored function, you must declare either that it is deterministic
or that it does not modify data. Otherwise, it may be unsafe for binary logging.
By default, for a $3&"5&'6/$5*0/ statement to be accepted, at least one of
%&5&3.*/*45*$, /042-, or 3&"%442-%"5" must be specified explicitly.
Otherwise an error occurs:
&3303
):5IJTGVODUJPOIBTOPOFPG%&5&3.*/*45*$/0
42-PS3&"%442-%"5"JOJUTEFDMBSBUJPOBOECJOBSZMPHHJOHJT
FOBCMFE
ZPVNJHIUXBOUUPVTFUIFMFTTTBGF
MPH@CJO@USVTU@GVODUJPO@DSFBUPSTWBSJBCMF
You can write non-deterministic statements inside a routine and still declare as
%&5&3.*/*45*$ (not a good practice), if you want to replicate routines that are not
declared as %&5&3.*/*45*$, you can set the MPH@CJO@USVTU@GVODUJPO@DSFBUPST
variable:
NZTRM 4&5(-0#"-MPH@CJO@USVTU@GVODUJPO@DSFBUPST
See also
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOTUPSFEQSPHSBNTMPHHJOHIUNM to
learn more about how the stored programs are replicated.
Extracting statements from a binary log

You can use the NZTRMCJOMPH utility (shipped along with MySQL) to extract the contents
from binary logs and apply them to other servers.
Getting ready
Execute a few statements using various binary formats. When you set the CJOMPH@GPSNBU
at (-0#"- level, you have to disconnect and reconnect to get the changes. If you want to be
connected, set at 4&44*0/ level.
Change to statement-based replication (SBR):

NZTRM 4&5!!(-0#"-#*/-0(@'03."5 45"5&.&/5
2VFSZ0,SPXTBGGFDUFE
TFD
[ 161 ]

Update a few rows:

NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 61%"5&TBMBSJFT4&5TBMBSZTBMBSZ8)&3&FNQ@OP

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $0..*5
2VFSZ0,SPXTBGGFDUFE
TFD
Change to row-based replication (RBR):

NZTRM 4&5!!(-0#"-#*/-0(@'03."5 308
2VFSZ0,SPXTBGGFDUFE
TFD
Update a few rows:

NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $0..*5
2VFSZ0,SPXTBGGFDUFE
TFD
Change to .*9&% format:

NZTRM 4&5!!(-0#"-#*/-0(@'03."5 .*9&%
2VFSZ0,SPXTBGGFDUFE
TFD
Update a few rows:

NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM */4&35*/50EFQBSUNFOUT7"-6&4
E 66*%

2VFSZ0,SPXBGGFDUFE
TFD
NZTRM $0..*5
2VFSZ0,SPXTBGGFDUFE
TFD
[ 162 ]

How to do it...
To display the contents of TFSWFS, execute the following:
TIFMM TVEPNZTRMCJOMPHEBUBNZTRMCJOMPHTTFSWFS
You will get output similar to the following:

BU
TFSWFSJEFOE@MPH@QPT$3$YCG2VFSZ
UISFBE@JEFYFD@UJNFFSSPS@DPEF
#*/-0(
_
_
In the first line, the number following BU indicates the starting position (file offset) of the
event in the binary log file. The second line contains the timestamp at which the statement
started on the server. The timestamp is followed by TFSWFSJE, FOE@MPH@QPT,
UISFBE@JE, FYFD@UJNF, and FSSPS@DPEF.
TFSWFSJE: Is the TFSWFS@JE value of the server where the event originated
( in this case).
FOE@MPH@QPT: Is the start position of the next event.
UISFBE@JE: Indicates which thread executed the event.
FYFD@UJNF: Is the time spent executing the event, on a master server. On a slave,
it is the difference of the end execution time on the slave minus the beginning
execution time on the master. The difference serves as an indicator of how much
replication lags behind the master.
FSSPS@DPEF: Indicates the result from executing the event. Zero means that no
error occurred.
Observations
1. You executed the 61%"5& statement in statement-based replication and the same
statement is logged in the binary log. Apart from the server, session variables are
also saved in the binary log to replicate the same behavior on the slave:
BU
TFSWFSJEFOE@MPH@QPT$3$YEGD
2VFSZUISFBE@JEFYFD@UJNFFSSPS@DPEF
4&55*.&45".1
[ 163 ]

4&5!!TFTTJPOQTFVEP@UISFBE@JE
4&5!!TFTTJPOGPSFJHO@LFZ@DIFDLT!!TFTTJPOTRM@BVUP@JT@OVMM
!!TFTTJPOVOJRVF@DIFDLT!!TFTTJPOBVUPDPNNJU
4&5!!TFTTJPOTRM@NPEF
4&5!!TFTTJPOBVUP@JODSFNFOU@JODSFNFOU
!!TFTTJPOBVUP@JODSFNFOU@PGGTFU
=$VUG
4&5
!!TFTTJPODIBSBDUFS@TFU@DMJFOU!!TFTTJPODPMMBUJPO@DPOOFDUJPO
!!TFTTJPODPMMBUJPO@TFSWFS
4&5!!TFTTJPOMD@UJNF@OBNFT
4&5!!TFTTJPODPMMBUJPO@EBUBCBTF%&'"6-5
#&(*/

BU
TFSWFSJEFOE@MPH@QPT$3$YDCB
VTFAFNQMPZFFTA
4&55*.&45".1
UPDATE salaries SET salary=salary*2 WHERE emp_no<10002

BU
TFSWFSJEFOE@MPH@QPT$3$YCDGFEB
9JE
$0..*5
2. When row-based replication is used, instead of the statement, the 308 is saved,
which is in binary format and you cannot read. Moreover, you can observe the
length, a single update statement generated so much data. Check the Extract row
event display section, which explains how to view the binary format.
#&(*/

BU
TFSWFSJEFOE@MPH@QPT$3$YFBFD
5BCMF@NBQAFNQMPZFFTAATBMBSJFTANBQQFEUPOVNCFS
BU
TFSWFSJEFOE@MPH@QPT$3$YGFGC
6QEBUF@SPXTUBCMFJEGMBHT45.5@&/%@'
BINLOG '
HveSWRPIAAAAPgAAANICAAAAAKUAAAAAAAEACWVtcGxveWVlcwAIc2FsYXJpZXMABAM
DCgoAAAEBAHTsouA=HveSWR/IAAAAQAIAABIFAAAAAKUAAAAAAAEAAgAE///wEScAAF
SrAwDahA/ahg/wEScAAKrVAQDahA/ahg/wEScAAFjKAwDahg/ZiA/wEScAACzlAQDah
g/ZiA/wEScAAGgIBADZiA/Zig/wEScAADQEAgDZiA/Zig/wEScAAJAQBADZig/ZjA/w
EScAAEgIAgDZig/ZjA/wEScAAEQWBADZjA/Zjg/wEScAACILAgDZjA/Zjg/wEScAABh
[ 164 ]

WBADZjg/YkA/wEScAAAwrAgDZjg/YkA/wEScAAHSJBADYkA/Ykg/wEScAALpEAgDYkA
/Ykg/wEScAAFiYBADYkg/YlA/wEScAACxMAgDYkg/YlA/wEScAAGijBADYlA/Ylg/wE
ScAALRRAgDYlA/Ylg/wEScAAFCxBADYlg/XmA/wEScAAKhYAgDYlg/XmA/wEScAADTi
BADXmA/Xmg/wEScAABpxAgDXmA/Xmg/wEScAAATyBADXmg/XnA/wEScAAAJ5AgDXmg/
XnA/wEScAACTzBADXnA/Xng/wEScAAJJ5AgDXnA/Xng/wEScAANQuBQDXng/WoA/wES
cAAGqXAgDXng/WoA/wEScAAOAxBQDWoA/Wog/wEScAAPCYAgDWoA/Wog/wEScAAKQxB
QDWog/WpA/wEScAANKYAgDWog/WpA/wEScAAIAaBgDWpA8hHk7wEScAAEANAwDWpA8h
Hk7wEScAAIAaBgDSwg8hHk7wEScAAEANAwDSwg8hHk4Fi+/w

BU
TFSWFSJEFOE@MPH@QPT$3$YBEBDED
9JE
$0..*5
3. When .*9&% format is used, the 61%"5& statement is logged as SQL, whereas the
*/4&35 statement is logged in row-based format since the */4&35 has an 66*%

function that is non-deterministic:
#&(*/

BU
TFSWFSJEFOE@MPH@QPT$3$YDEGC
4&55*.&45".1

BU
TFSWFSJEFOE@MPH@QPT$3$YBFGF
5BCMF@NBQAFNQMPZFFTAAEFQBSUNFOUTANBQQFEUPOVNCFS
BU
TFSWFSJEFOE@MPH@QPT$3$YBDEBD
8SJUF@SPXTUBCMFJEGMBHT45.5@&/%@'
BINLOG
'TveSWRPIAAAARQAAALMGAAAAAKYAAAAAAAMACWVtcGxveWVlcwALZGVwYXJ0bWVudH
MAAv4PBP4QoAAAAgP8/wB+D64DTveSWR7IAAAATgAAAAEHAAAAAKYAAAAAAAEAAgAC/
/wEZDAxMSRkMDNhMjQwZS04MWJkLTExZTctODQxMC00MjAxMGE5NDAwMDKsXTyk

BU
TFSWFSJEFOE@MPH@QPT$3$YGBBF
9JE
$0..*5
[ 165 ]

The extracted log can be piped to MySQL to replay the events. It is better to use the force
option while replaying binlogs because, if it is stuck at one point, it won't stop executing.
Later, you can figure out the errors and fix the data manually.
TIFMM TVEPNZTRMCJOMPHEBUBNZTRMCJOMPHTTFSWFS]NZTRMGI
SFNPUF@IPTU VVTFSOBNF Q
Or you can save into a file and execute later:

TFSWFSCJOMPH@FYUSBDU
TIFMM DBUTFSWFSCJOMPH@FYUSBDU]NZTRMISFNPUF@IPTU VVTFSOBNF Q
Extracting based on time and position

You can extract the partial data from the binary log by specifying the position. Suppose you
want to do point-in-time recovery. Assume a %301%"5"#"4& command was executed at
and the latest available backup was ,
which you already restored. Now, you need to restore data from till
. Remember, if you extract the full log, it will also contain the %301%"5"#"4&
command and it will wipe your data again.
You can extract the data by specifying the time window through TUBSUEBUFUJNF and
TUPQEBUBUJNF options.
TIFMM TVEPNZTRMCJOMPHEBUBNZTRMCJOMPHTTFSWFSTUBSU
EBUFUJNFTUPQEBUFUJNF
CJOMPH@FYUSBDU
The disadvantage of using a time window is you will miss the transactions that happened at
the second the disaster occurred. To avoid this, you have to use the file offset of the event in
the binary log file.
A consistent backup saves the binlog file offset until which it has already backed up. Once
the backup is restored, you have to extract the binlogs from the offset provided by the
backup. You will learn more about backups in the next chapter.
Assume that the backup has given an offset of and the %301%"5"#"4& command was
executed at an offset of . You can use TUBSUQPTJUJPO and TUPQQPTJUJPO
options to extract a log between offsets:
QPTJUJPOTUPQQPTJUJPO CJOMPH@FYUSBDU
[ 166 ]

Make sure that the %301%"5"#"4& command does not appear again in the extracted
binlog.
Extracting based on the database

Using the EBUBCBTF option, you can filter events of a specific database. If you give this
multiple times, only the last option will be considered. This works very well for row-based
replication. But for statement-based replication and .*9&%, this gives output only when the
default database is selected.
The following command extracts events from the employees database:

EBUBCBTFFNQMPZFFT CJOMPH@FYUSBDU
As explained in the MySQL 8 reference manual, suppose the binary log was created by
executing these statements using statement-based-logging:
NZTRM
*/4&35*/50UFTUU
J7"-6&4

*/4&35*/50ECU
K7"-6&4

64&UFTU
*/4&35*/50UFTUU
J7"-6&4

*/4&35*/50U
J7"-6&4

*/4&35*/50ECU
K7"-6&4

64&EC
*/4&35*/50UFTUU
J7"-6&4

*/4&35*/50ECU
K7"-6&4

*/4&35*/50U
K7"-6&4

NZTRMCJOMPHEBUBCBTFUFTU does not output the first two */4&35 statements because
there is no default database.
It outputs the three */4&35 statements following 64&UFTU, but not the three */4&35
statements following 64&EC.
NZTRMCJOMPHEBUBCBTFEC does not output the first two */4&35 statements because
there is no default database.
It does not output the three */4&35 statements following 64& test, but does output the three
*/4&35 statements following 64&EC.
[ 167 ]

Extracting a row event display

In row-based replication, by default, binary format is displayed. To view the 308
information, you have to pass the WFSCPTF or W option to NZTRMCJOMPH. The binary
format of row events are shown as comments in the form of pseudo-SQL statements with
lines beginning with . You can see that a single 61%"5& statement is rewritten as
an 61%"5& statement for each row:
shell> mysqlbinlog /data/mysql/binlogs/server1.000001 --start-position=660
--stop-position=1298 --verbose
_
_
BU
BU
#*/-0(
)WF4831*""""1H"""/*$""""",6"""""""&"$87UD(YWF87MDX"*D'T:9+Q;9."#".%$HP"""&
#
")5TPV"
_
_

61%"5&AFNQMPZFFTAATBMBSJFTA
8)&3&
!
!
!
!
4&5
!
!
!
!
_
_
8)&3&
!
!
!
!
4&5
!
[ 168 ]

!
!
!
4&5!!4&44*0/(5*%@/&95 "650."5*$ BEEFECZNZTRMCJOMPH
%&-*.*5&3
&OEPGMPHGJMF
4&5$0.1-&5*0/@5:1&!0-%@$0.1-&5*0/@5:1&
4&5!!4&44*0/14&6%0@4-"7&@.0%&
If you want to just see the pseudo-SQL without the binary row information, specify
CBTFPVUQVUEFDPEFSPXT along with WFSCPTF:
shell> sudo mysqlbinlog /data/mysql/binlogs/server1.000001 --start-

position=660 --stop-position=1298 --verbose --base64-output="decode-rows"
4&5!!4&44*0/14&6%0@4-"7&@.0%&
4&5!0-%@$0.1-&5*0/@5:1&!!$0.1-&5*0/@5:1&$0.1-&5*0/@5:1&
%&-*.*5&3
BU
BU
8)&3&
!
!
!
!
4&5
!
!
!
!
_
Rewriting a database name

Suppose you want to restore the binary log of the FNQMPZFFT database on a production
server as FNQMPZFFT@EFW on a development server. You can use the SFXSJUF
EC GSPN@OBNF UP@OBNF option. This will rewrite all occurrences of GSPN@OBNF to
UP@OBNF.
[ 169 ]

To convert multiple databases, specify the option multiple times:

shell> sudo mysqlbinlog /data/mysql/binlogs/server1.000001 --start-
position=1499 --stop-position=1646 --rewrite-db='employees->employees_dev'
_
BU
TFSWFSJEFOE@MPH@QPT$3$YDEGC2VFSZ
UISFBE@JEFYFD@UJNFFSSPS@DPEF
use èmployees_dev`/*!*/;
_
_

4&5!!4&44*0/(5*%@/&95 "650."5*$ BEEFECZNZTRMCJOMPH
%&-*.*5&3
&OEPGMPHGJMF
_
You can see that the statement VTFAFNQMPZFFT@EFWA is used. So, while restoring,
all the changes will be applied to FNQMPZFFT@EFWEBUBCBTF.
As explained in the MySQL reference manual:
When used together with the EBUBCBTF option, the SFXSJUFEC

option is applied first, then the EBUBCBTF option is applied using the
rewritten database name. The order in which the options are provided
makes no difference in this regard. This means that, for example, if
NZTRMCJOMPH is started with SFXSJUFEC NZEC ZPVSEC
EBUBCBTFZPVSEC, then all updates to any tables in databases NZEC and
ZPVSEC are included in the output.
On the other hand, if it is started with SFXSJUFEC NZEC ZPVSEC
EBUBCBTFNZEC, then NZTRMCJOMPH outputs no statements at all: since
all updates to NZEC are first rewritten as updates to ZPVSEC before
applying the EBUBCBTF option, there remain no updates that match
EBUBCBTFNZEC.
Disabling a binary log for recovery

While restoring binary logs, if you donct want the NZTRMCJOMPH process to create binary
logs, you can use the EJTBCMFMPHCJO option so that binary logs won't be written:
QPTJUJPOTUPQQPTJUJPOEJTBCMFMPHCJO CJOMPH@SFTUPSF
[ 170 ]

You can see that 42-@-0(@#*/ is written to the CJOMPH restore file, which will prevent
creating the binlogs.
4&5!0-%@42-@-0(@#*/!!42-@-0(@#*/42-@-0(@#*/
Displaying events in a binary log file

Apart from using NZTRMCJOMPH, you can also use the 4)08#*/-0(&7&/54 command to
display the events.
The following command will display the events in the TFSWFS binary log. If
-*.*5 is not specified, all the events are displayed:
mysql> SHOW BINLOG EVENTS IN 'server1.000008' LIMIT 10;

]-PH@OBNF]1PT]&WFOU@UZQF]4FSWFS@JE]&OE@MPH@QPT]*OGP
]

]TFSWFS]]'PSNBU@EFTD]]]4FSWFS
WFSSDMPH#JOMPHWFS]
]TFSWFS]]1SFWJPVT@HUJET]]]
]
]TFSWFS]]"OPOZNPVT@(UJE]]]4&5
!!4&44*0/(5*%@/&95 "/0/:.064 ]
]TFSWFS]]2VFSZ]]]ESPQ
EBUBCBTFDPNQBOZYJE]
!!4&44*0/(5*%@/&95 "/0/:.064 ]
]TFSWFS]]2VFSZ]]]#&(*/
]
]TFSWFS]]5BCMF@NBQ]]]
UBCMF@JE
FNQMPZFFTFNQ@EFUBJMT]
]TFSWFS]]8SJUF@SPXT]]]
UBCMF@JEGMBHT45.5@&/%@']
]TFSWFS]]9JE]]]$0..*5
YJE]
!!4&44*0/(5*%@/&95 "/0/:.064 ]

SPXTJOTFU
TFD
[ 171 ]

You can also specify the position and offset:

mysql> SHOW BINLOG EVENTS IN 'server1.000008' FROM 123 LIMIT 2,1;

]-PH@OBNF]1PT]&WFOU@UZQF]4FSWFS@JE]&OE@MPH@QPT]*OGP
]

]TFSWFS]]2VFSZ]]]ESPQ
EBUBCBTFDPNQBOZYJE]

SPXJOTFU
TFD
Ignoring databases to write to a binary log

You can choose which databases should be written to the binary log by specifying the
CJOMPHEPECEC@OBNF option in NZDOG. To specify multiple databases you must use
multiple instances of this option. Because database names can contain commas, the list will
be treated as the name of a single database if you supply a comma-separated list. You need
to restart the MySQL server to effect changes.
How to do it...
Open NZDOG and add the following lines:
<NZTRME>
CJOMPH@EP@ECEC
CJOMPH@EP@ECEC
The behavior on CJOMPHEPEC changes from statement-based logging to row-based

logging just like the EBUBCBTF option in the NZTRMCJOMPH utility.
In statement-based logging, only those statements are written to the binary log where the
default database (that is, the one selected by 64&) is written to the binary log. You should be
very careful while using the CJOMPHEPEC option because it does not work as you might
expect when using statement-based logging. Go through the following examples mentioned
in the reference manual.
[ 172 ]

Example 1
If the server is started with CJOMPHEPECTBMFT and you issue the following
statements, the 61%"5& statement is not logged:
NZTRM 64&QSJDFT
NZTRM 61%"5&TBMFTKBOVBSZ4&5BNPVOUBNPVOU
The main reason for this just check the default database behavior is that it is difficult from the
statement alone to know whether it should be replicated. It is also faster to check only the
default database rather than all databases if there is no need.
Example 2
If the server is started with CJOMPHEPECTBMFT, the following 61%"5& statement is
logged even though prices were not included when setting CJOMPHEPEC:
NZTRM 64&TBMFT
NZTRM 61%"5&QSJDFTEJTDPVOUT4&5QFSDFOUBHFQFSDFOUBHF
Because sales is the default database when the 61%"5& statement is issued, the 61%"5& is
logged.
In row-based logging, it is restricted to database EC@OBNF. Only changes to tables belonging

to EC@OBNF are logged; the default database has no effect on this.
Another important difference in CJOMPHEPEC handling for statement-based logging, as

opposed to the row-based logging, occurs with regard to statements that refer to multiple
databases. Suppose that the server is started with CJOMPHEPECEC, and the following
statements are executed:
NZTRM 64&EC
NZTRM 61%"5&ECUBCMF4&5DPMECUBCMF4&5DPM
If you are using statement-based logging, the updates to both tables are written to the
binary log. However, when using the row-based format, only the changes to UBCMF are
logged; UBCMF is in a different database, so it is not changed by the 61%"5&.
Similarly, you can use the CJOMPHJHOPSFECEC@OBNF option to ignore the database
from writing to the binary log.
Refer to the manual for more information, at: IUUQTEFWNZTRMDPNEPDSFGNBOFO

SFQMJDBUJPOSVMFTIUNM.
[ 173 ]

Relocating binary logs

Since binary logs occupy more space, and sometimes you may wish to change the location
of binary logs, the following procedure helps. Changing the MPH@CJO alone is not sufficient,
you have to move all the binary logs and update the index file with the new location.
The NZTRMCJOMPHNPWF utility eases your work by automating those tasks.
How to do it...
MySQL Utilities should be installed for using the NZTRMCJOMPHNPWF script. Refer to
$IBQUFS, MySQL 8.0 ` Installing and Upgrading, for installation steps.
1. Stop the MySQL server:
2. Start the NZTRMCJOMPHNPWF utility. If you want to change the binary logs from
EBUBNZTRMCJOMPHT to CJOMPHT, the following command should be used. If
your base name is not default, you have to mention your base name through
the CJOMPHCBTFOBNF option:
shell> sudo mysqlbinlogmove --bin-log-base name=server1 --binlog-

dir=/data/mysql/binlogs /binlogs

.PWJOHCJOMPHGJMFT
TFSWFS
TFSWFS
TFSWFS
TFSWFS
TFSWFS

EPOF

3. Edit the NZDOG file and update the new location of MPH@CJO:
<NZTRME>
MPH@CJOCJOMPHT
4. Start the MySQL server:
[ 174 ]

The new location is updated in AppArmor or SELinux.
If there are a lot of binlogs, the downtime of the server will be high. To avoid that you can
use the TFSWFS option to relocate all binary logs except the ones currently in use (with
the higher sequence number). Then stop the server, use the preceding method, and relocate
the last binary log, which will be much faster because only one file is there. Then you can
change the NZDOG and start the server.
For example:
TIFMM TVEPNZTRMCJOMPHNPWFTFSWFSSPPUQBTT!IPTUOFXMPDBUJPO
[ 175 ]

7
Backups
Taking backups using mysqldump

Taking backups using mysqlpump
Taking backups using mydumper
Taking backups using flat files
Taking backups using XtraBackup
Locking instances for backup
Binary log backup
Introduction
After setting up the database, the next important thing is to set up backups. In this chapter,
you will learn how to set up various types of backups. There are two main ways to perform
a backup. One is logical backup, which exports all the databases, table structures, data, and
stored routines into a set of SQL statements that can be executed again to recreate the state
of the database. The other type is physical backup, which contains all the files on the system
that the databases used to store all the database entities:
Logical backup utilities: NZTRMEVNQ, NZTRMQVNQ, and NZEVNQFS (not shipped

with MySQL)
Physical backup utilities: XtraBackup (not shipped with MySQL) and flat file
backup
For point-in-time recovery, the backup should be able to provide the binary log positions
up to which the backups are taken. This is called a consistent backup.

Backups Chapter 7
It is highly recommended to take backups from a slave onto a filer mounted on it.
Taking backups using mysqldump

NZTRMEVNQ is a widely used logical backup tool. It gives a variety of options to include or
exclude databases, select specific data to be backed up, back up only the schema without
data, or just take a backup of stored routines without anything else, and more.
How to do it...
The NZTRMEVNQ utility comes along with the NZTRM binary, so you do need to install it
separately. Most production scenarios are covered in this section.
The syntax is as follows:

TIFMM NZTRMEVNQ<PQUJPOT>
In the options, you can specify the username, password, and hostname to connect to the
database, like this:
VTFSVTFS@OBNF QBTTXPSEQBTTXPSE
PS
VVTFS@OBNF QQBTTXPSE
In this chapter, VTFS and QBTTXPSE are not mentioned in every example, to keep the
reader focused on other important options.
Full backup of all databases

This can be done with the following:
TIFMM NZTRMEVNQBMMEBUBCBTFT EVNQTRM
The BMMEBUBCBTFT option takes a backup of all databases and all tables. The operator
redirects the output to the EVNQTRM file. Prior to MySQL 8, stored procedures and events
were stored in the NZTRMQSPD and NZTRMFWFOU tables. From MySQL 8 onward,
definitions for corresponding objects are stored in EBUBEJDUJPOBSZ tables, but those
tables are not dumped. To include stored routines and events in a dump made using
BMMEBUBCBTFT, use the SPVUJOFT and FWFOUT options.
[ 177 ]

Backups Chapter 7
To include the routines and events:

TIFMM NZTRMEVNQBMMEBUBCBTFTSPVUJOFTFWFOUT EVNQTRM
You can open the EVNQTRM file to see how it is structured. The first few lines are the
session variables at the time of dumping. Next is the $3&"5&%"5"#"4& statement,
followed by the 64&%"5"#"4& command. Next is the %3015"#-&*'&9*454 statement,
followed by $3&"5&5"#-&; and then we have the actual */4&35 statements that insert the
data. Since the data is stored as SQL statements, this is called logical backup.
You will notice that when you restore the dump, the %3015"#-& statement will wipe off
all the tables before creating the tables.
Point-in-time recovery
To get point-in-time recovery, you should specify TJOHMFUSBOTBDUJPO and
NBTUFSEBUB.
The TJOHMFUSBOTBDUJPO option provides consistent backup by changing the

transaction isolation mode to 3&1&"5"#-&3&"% and executing 45"3553"/4"$5*0/
before taking a backup. It is useful only with transactional tables, such as *OOP%#, because
then it dumps the consistent state of the database at the time 45"3553"/4"$5*0/ was
issued without blocking any applications.
The NBTUFSEBUB option prints the binary log coordinates of the server to the EVNQ file.
If NBTUFSEBUB, it prints as a comment. This also uses the '-64)5"#-&48*5)
3&"%-0$, statement to get a snapshot for the binary logs. As explained in $IBQUFS,
Transactions, this can be very dangerous when there is any long-running transaction:
TIFMM NZTRMEVNQBMMEBUBCBTFTSPVUJOFTFWFOUTTJOHMFUSBOTBDUJPO
NBTUFSEBUB EVNQTRM
Dumping master binary coordinates

Backups are always taken on slaves. To get the binary log coordinates of the master when
the backup was taken, you can use the EVNQTMBWF option. If you are taking the binary
log backup from the master, use this option. Otherwise, use the NBTUFSEBUB option:
TIFMM NZTRMEVNQBMMEBUBCBTFTSPVUJOFTFWFOUTTJOHMFUSBOTBDUJPO
EVNQTMBWF EVNQTRM
[ 178 ]

Backups Chapter 7
The output will be like this:

1PTJUJPOUPTUBSUSFQMJDBUJPOPSQPJOUJOUJNFSFDPWFSZGSPN
UIFNBTUFS
PGUIJTTMBWF

$)"/(&."45&350."45&3@-0(@'*-& DFOUPTCJO ."45&3@-0(@104
Specific databases and tables

To back up only a specific database, execute this:
TIFMM NZTRMEVNQEBUBCBTFTFNQMPZFFT FNQMPZFFT@CBDLVQTRM
To back up only a specific table, execute this:

TIFMM NZTRMEVNQEBUBCBTFTFNQMPZFFTUBCMFTFNQMPZFFT
FNQMPZFFT@CBDLVQTRM
Ignore tables
To ignore certain tables, you can use the JHOPSFUBCMFEBUBCBTFUBCMF option. To
specify more than one table to ignore, use the directive multiple times:
TIFMM NZTRMEVNQEBUBCBTFTFNQMPZFFTJHOPSFUBCMFFNQMPZFFTTBMBSZ
FNQMPZFFT@CBDLVQTRM
Specific rows
NZTRMEVNQ helps you filter the data you back up. Suppose you want to take a backup of
employees who joined after 2000:
TIFMM NZTRMEVNQEBUBCBTFTFNQMPZFFTUBCMFTFNQMPZFFTEBUBCBTFT
FNQMPZFFTUBCMFTFNQMPZFFTXIFSFIJSF@EBUF
FNQMPZFFT@BGUFS@TRM
You can use the -*.*5 clause to limit the results:

TIFMM NZTRMEVNQEBUBCBTFTFNQMPZFFTUBCMFTFNQMPZFFTEBUBCBTFT
FNQMPZFFTUBCMFTFNQMPZFFTXIFSFIJSF@EBUF -*.*5
FNQMPZFFT@BGUFS@@MJNJU@TRM
[ 179 ]

Backups Chapter 7
Backup from a remote server

Sometimes, you may not have SSH access to the database server (as in the case of cloud
instances such as Amazon RDS). In those cases, you can use NZTRMEVNQ to take a backup
from a remote server to the local server. For this, you need to mention IPTUOBNF using
the IPTUOBNF option. Make sure that the user has appropriate privileges to connect and
perform a backup:
TIFMM NZTRMEVNQBMMEBUBCBTFTSPVUJOFTFWFOUTUSJHHFSTIPTUOBNF
SFNPUF@IPTUOBNF EVNQTRM
Backup to rebuild another server with a different

schema
There can be a situation where you want to have a different schema on another server. In
that case, you have to dump and restore the schema, alter the schema as you need, and then
dump and restore the data. Altering the schema with data can take a long time, depending
on the amount of data you have. Note that this method will work only when the modified
schema is compatible with inserts. The modified table can have extra columns but it should
have all the columns present in the original table.
Only schema and no data

You can use OPEBUB to dump only the schema:
TIFMM NZTRMEVNQBMMEBUBCBTFTSPVUJOFTFWFOUTUSJHHFSTOPEBUB
TDIFNBTRM
Only data and no schema

You can use the following options to take only a data dump, excluding the schema.
DPNQMFUFJOTFSU will print the column names in the */4&35 statement, which will
help when you have extra columns in the modified table:
TIFMM NZTRMEVNQBMMEBUBCBTFTOPDSFBUFECOPDSFBUFJOGP
DPNQMFUFJOTFSU EBUBTRM
[ 180 ]

Backups Chapter 7
Backup for merging data with other server

You can take backup either way to replace the old data or keep the old data incase of
conflict.
3&1-"$&XJUIOFXEBUB
Suppose you want to restore data from the production database to a development machine
that already has some data. If you want to merge the data from production with
development, you can use the SFQMBDF option, which will use the 3&1-"$&*/50
statement instead of the */4&35 statement. You should also include the TLJQBEE
ESPQUBCMF option, which will not write a %3015"#-& statement to the EVNQ file. If you
have the same number of tables and structure, you can also include the OPDSFBUFJOGP
option, which will skip the $3&"5&5"#-& statement in the EVNQ file:
TIFMM NZTRMEVNQEBUBCBTFTFNQMPZFFTTLJQBEEESPQUBCMFOPDSFBUF
JOGPSFQMBDF UP@EFWFMPQNFOUTRM
If you have some extra tables in production, the preceding dump will fail while restoring
because the table does not exist on development server. In that case, you should not add
the OPDSFBUFJOGP option and use the GPSDF option while restoring. Otherwise, the
restore will fail at $3&"5&5"#-& saying that the table already exists.
Unfortunately, NZTRMEVNQ has not provided the option of $3&"5&5"#-&*'/05
&9*454.
*(/03&EBUB
Instead of 3&1-"$&, you can use the */4&35*(/03& statement when writing to the EVNQ
file. This will keep the existing data on the server and insert new data.
Taking backups using mysqlpump

NZTRMQVNQ is a very similar program to NZTRMEVNQ with some extra features.
How to do it...
There are numerous ways to do it. Let's have a look at each one in detail.
[ 181 ]

Backups Chapter 7
Parallel processing
You can speed up the process of dumping by specifying the number of threads (based on
the number of CPUs). For example, use eight threads to take a full backup:
TIFMM NZTRMQVNQEFGBVMUQBSBMMFMJTN GVMM@CBDLVQTRM
You can even specify the number of threads for each database. In our case, the FNQMPZFFT
database is very big compared to the DPNQBOZ database. So you can spawn four threads to
FNQMPZFFT and two threads to the DPNQBOZ database:
shell> mysqlpump -u root --password --parallel-schemas=4:employees --

default-parallelism=2 > full_backup.sql
%VNQQSPHSFTTUBCMFTSPXT
%VNQDPNQMFUFEJO
Another example to distribute the threads has three threads for EC and EC, two threads
for EC and EC, and four threads for the rest of the databases:
TIFMM NZTRMQVNQQBSBMMFMTDIFNBTECECQBSBMMFMTDIFNBTECEC
EFGBVMUQBSBMMFMJTN GVMM@CBDLVQTRM
You will notice that there is a progress bar that will help you estimate the time.
Exclude/include database objects using regex

Take a backup of all databases ending with QSPE:
TIFMM NZTRMQVNQJODMVEFEBUBCBTFTQSPESFTVMUGJMFEC@QSPETRM
Suppose there are some test tables in some databases and you want to exclude them from
the backup; you can specify using the FYDMVEFUBCMFT option, which will exclude
tables with the name UFTU across all databases:
TIFMM NZTRMQVNQFYDMVEFUBCMFTUFTUSFTVMU
GJMFCBDLVQ@FYDMVEJOH@UFTUTRM
[ 182 ]

Backups Chapter 7
The value of each inclusion and exclusion option is a comma-separated list of names of the
appropriate object type. Wildcard characters are permitted in object names:
matches any sequence of zero or more characters
@ matches any single character
Apart from databases and tables, you can also include or exclude triggers, routines, events,
and users, for example, JODMVEFSPVUJOFT, JODMVEFFWFOUT, and FYDMVEF
USJHHFST.
To know more about the include and exclude options, refer to IUUQTEFWNZTRMDPN
EPDSFGNBOFONZTRMQVNQIUNMNZTRMQVNQGJMUFSJOH.
Backup users
In NZTRMEVNQ, you will not get a backup of the users in $3&"5&64&3 or (3"/5 statements;
instead you have to take a backup of the NZTRMVTFS table. Using NZTRMQVNQ, you can
dump user accounts as account management statements ($3&"5&64&3 and (3"/5) rather
than as inserts into the NZTRM system database:
TIFMM NZTRMQVNQFYDMVEFEBUBCBTFTVTFST VTFST@CBDLVQTRM
You can also exclude some users by specifying the FYDMVEFVTFST option:
TIFMM NZTRMQVNQFYDMVEFEBUBCBTFTFYDMVEFVTFSTSPPUVTFST
VTFST@CBDLVQTRM
Compressed backups
You can compress the backup to minimize disk space and network bandwidth. You can use
DPNQSFTTPVUQVUM[ or DPNQSFTTPVUQVU[MJC.
Note that you should have the appropriate decompress utilities:

TIFMM NZTRMQVNQVSPPUQYYYYDPNQSFTTPVUQVUM[ EVNQM[
[ 183 ]

Backups Chapter 7
To decompress execute this:

TIFMM M[@EFDPNQSFTTEVNQM[EVNQTRM
Using [MJC execute this:

TIFMM NZTRMQVNQVSPPUQYYYYDPNQSFTTPVUQVU[MJC EVNQ[MJC
To decompress execute this:

TIFMM [MJC@EFDPNQSFTTEVNQ[MJCEVNQTRM
Faster reload
You will notice that in the output, the secondary indexes are omitted from the $3&"5&
5"#-& statement. This will speed up the restoration process. The indexes are added at the
end of the */4&35 using the "-5&35"#-& statement.
Indexes will be covered in $IBQUFS, Performance Tuning.
Previously, it was possible to dump all tables in the NZTRM system

database. As of MySQL 8, NZTRMEVNQ and NZTRMQVNQ dump only non-
EBUBEJDUJPOBSZ tables in that database.
Taking backups using mydumper

NZEVNQFS is a logical backup tool that's similar to NZTRMQVNQ.
NZEVNQFS has these advantages over NZTRMEVNQ:
Parallelism (hence, speed) and performance (avoids expensive character set

conversion routines and has efficient code overall).
Consistency. It maintains snapshots across all threads, provides accurate master
and slave log positions, and so on. NZTRMQVNQ does not guarantee consistency.
Easier to manage output (separate files for tables and dumped metadata, and it is
easy to view/parse data). NZTRMQVNQ writes everything to one file, which limits
the option of loading selective database objects.
Inclusion and exclusion of database objects using regex.
The option to kill long-running transactions that block the backup and all
subsequent queries.
[ 184 ]

Backups Chapter 7
NZEVNQFS is an open source backup tool, which you need to install separately. In this
section, installation steps on Debian and Red Hat systems and the usage of NZEVNQFS will
be covered.
How to do it...
Let's start with the installation and then we will learn a lot of things related to backup in
every subsection that is listed within this recipe.
Installation
Install the prerequisites:
On Ubuntu/Debain:
TIFMM TVEPBQUHFUJOTUBMMMJCHMJCEFWMJCNZTRMDMJFOUEFW[MJCHEFW
MJCQDSFEFWDNBLFHJU
On Red Hat/CentOS/Fedora:
TIFMM ZVNJOTUBMMHMJCEFWFMNZTRMEFWFM[MJCEFWFMQDSFEFWFMDNBLFHDD
D HJU
TIFMM DEPQU
TIFMM HJUDMPOFIUUQTHJUIVCDPNNBYCVCFNZEVNQFSHJU
TIFMM DENZEVNQFS
TIFMM DNBLF
TIFMM NBLF
4DBOOJOHEFQFOEFODJFTPGUBSHFUNZEVNQFS
<>#VJMEJOH$PCKFDU$.BLF'JMFTNZEVNQFSEJSNZEVNQFSDP
<>#VJMEJOH$PCKFDU$.BLF'JMFTNZEVNQFSEJSTFSWFS@EFUFDUDP
<>#VJMEJOH$PCKFDU$.BLF'JMFTNZEVNQFSEJSH@VOJY@TJHOBMDP
TIFMM NBLFJOTUBMM
<>#VJMUUBSHFUNZEVNQFS
<>#VJMUUBSHFUNZMPBEFS
-JOLJOH$FYFDVUBCMF$.BLF'JMFT$.BLF3FMJOLEJSNZEVNQFS
-JOLJOH$FYFDVUBCMF$.BLF'JMFT$.BLF3FMJOLEJSNZMPBEFS
*OTUBMMUIFQSPKFDU
*OTUBMMDPOGJHVSBUJPO
*OTUBMMJOHVTSMPDBMCJONZEVNQFS
*OTUBMMJOHVTSMPDBMCJONZMPBEFS
[ 185 ]

Backups Chapter 7
Alternatively, using YUM or APT, you can find the releases here at IUUQTHJUIVCDPN
NBYCVCFNZEVNQFSSFMFBTFT:
:6.
TIFMM TVEPZVNJOTUBMMZ
IUUQTHJUIVCDPNNBYCVCFNZEVNQFSSFMFBTFTEPXOMPBEWNZEVNQFS
FMY@SQN
"15
TIFMM XHFU
IUUQTHJUIVCDPNNBYCVCFNZEVNQFSSFMFBTFTEPXOMPBEWNZEVNQFS@
KFTTJF@BNEEFC
TIFMM TVEPEQLHJNZEVNQFS@KFTTJF@BNEEFC
Full backup
The following command takes a backup of all databases into the CBDLVQT folder:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE PVUQVUEJSCBDLVQT
Several files are created in the CBDLVQT folder. Each database has its $3&"5&%"5"#"4&
statement as EBUBCBTF@OBNF TDIFNBDSFBUFTRM and each table will have its own
schema and data files. Schema files are stored as EBUBCBTF@OBNF UBCMF
TDIFNBTRM and data files are stored as EBUBCBTF@OBNF UBCMF TRM.
The views are stored as EBUBCBTF@OBNF UBCMF TDIFNBWJFXTRM. Stored routines,

triggers, and events are stored as EBUBCBTF@OBNF TDIFNBQPTUTRM (use TVEPNLEJS
hQWCBDLVQT if directory is not created):
TIFMM MTMIUSCBDLVQTDPNQBOZ
SXSSSPPUSPPU"VHCBDLVQTDPNQBOZTDIFNBDSFBUFTRM
SXSSSPPUSPPU"VHCBDLVQTDPNQBOZQBZNFOUTTRM
SXSSSPPUSPPU"VHCBDLVQTDPNQBOZOFX@DVTUPNFSTTRM
SXSSSPPUSPPU"VHCBDLVQTDPNQBOZQBZNFOUT
TDIFNBTRM
SXSSSPPUSPPU"VHCBDLVQTDPNQBOZOFX@DVTUPNFST
TDIFNBTRM
SXSSSPPUSPPU"VHCBDLVQTDPNQBOZDVTUPNFST
TDIFNBTRM
[ 186 ]

Backups Chapter 7
If there are any queries longer than 60 seconds, NZEVNQFS will fail with the following error:

NZEVNQFS$3*5*$"-5IFSFBSFRVFSJFTJO130$&44-*45SVOOJOH
MPOHFSUIBOTBCPSUJOHEVNQ
VTFMPOHRVFSZHVBSEUPDIBOHFUIFHVBSEWBMVFLJMMRVFSJFT
LJMM
MPOHRVFSJFTPSVTFEJGGFSFOUTFSWFSGPSEVNQ
To avoid this, you can pass the LJMMMPOHRVFSJFT option or set MPOHRVFSZ
HVBSE to a higher value.
The LJMMMPOHRVFSJFT option kills all the queries that are greater than 60 seconds or a
value set by MPOHRVFSZHVBSE. Please note that LJMMMPOHRVFSJFT also kills the
replication thread due to a bug (IUUQTCVHTMBVODIQBEOFUNZEVNQFS CVH):
TIFMM TVEPNZEVNQFSLJMMMPOHRVFSJFTPVUQVUEJSCBDLVQT

NZEVNQFS8"3/*/(6TJOHUSY@DPOTJTUFODZ@POMZCJOMPH
DPPSEJOBUFTXJMMOPUCFBDDVSBUFJGZPVBSFXSJUJOHUPOPOUSBOTBDUJPOBM
UBCMFT

NZEVNQFS8"3/*/(,JMMFEBRVFSZUIBUXBTSVOOJOHGPST
Consistent backup
The metadata file in the CBDLVQ directory contains the binary log coordinates for consistent
backup.
On a master, it captures the binary log positions:

TIFMM TVEPDBUCBDLVQTNFUBEBUB
4UBSUFEEVNQBU
4)08."45&345"564
-PHTFSWFS
1PT
(5*%
On a slave, it captures the binary log positions of both master and slave:
TIFMM DBUCBDLVQTNFUBEBUB
4UBSUFEEVNQBU
4)08."45&345"564
-PHTFSWFS
1PT
(5*%
4)084-"7&45"564
)PTU
-PHNBTUFSCJO
[ 187 ]

Backups Chapter 7
1PT
(5*%
'JOJTIFEEVNQBU
Backup of a single table

The following command takes the backup of the FNQMPZFFT table of the FNQMPZFFT
database into the CBDLVQT directory:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE #FNQMPZFFT5FNQMPZFFT
USJHHFSTFWFOUTSPVUJOFTPVUQVUEJSCBDLVQTFNQMPZFF@UBCMF
TIFMM MTMIUSCBDLVQTFNQMPZFF@UBCMF
UPUBM.
SXSSSPPUSPPU"VHFNQMPZFFTTDIFNBDSFBUFTRM
SXSSSPPUSPPU"VHFNQMPZFFTFNQMPZFFTTDIFNBTRM
SXSSSPPUSPPU,"VHFNQMPZFFTTDIFNBQPTUTRM
SXSSSPPUSPPU"VHNFUBEBUB
SXSSSPPUSPPU."VHFNQMPZFFTFNQMPZFFTTRM
The convention of the files are as follows:
FNQMPZFFTTDIFNBDSFBUFTRM contains the $3&"5&%"5"#"4& statement

FNQMPZFFTFNQMPZFFTTDIFNBTRM contains the $3&"5&5"#-& statement
FNQMPZFFTTDIFNBQPTUTRM contains the 3065*/&4, 53*((&34, and &7&/54
FNQMPZFFTFNQMPZFFTTRM contains the actual data in the form of */4&35
statements
Backup of specific databases using regex

You can include/exclude specific databases using the SFHFY option. The following
command will exclude NZTRM and UFTU databases from the backup:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE SFHFY ?

NZTRM]UFTU
PVUQVUEJSCBDLVQTTQFDJGJD@ECT
[ 188 ]

Backups Chapter 7
Taking backup of a big table using mydumper

To speed up the dump and restore of a big table, you can split it into small chunks. The
chunk size can be specified by the number of rows it contains and each chunk will be
written into a separate file:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE #FNQMPZFFT5FNQMPZFFT
USJHHFSTFWFOUTSPVUJOFTSPXTUUSYDPOTJTUFODZPOMZ
PVUQVUEJSCBDLVQTFNQMPZFF@UBCMF@DIVOLT
U: Specifies the number of threads

USYDPOTJTUFODZPOMZ: If you are using only transnational tables, such
as *OOP%#, using this option will minimize the locking
SPXT: Split the table into chunks of this number of rows
For each chunk, a file is created as EBUBCBTF@OBNF UBCMF@OBNF OVNCFS TRM; the
number is padded with five zeros:
TIFMM MTMISCBDLVQTFNQMPZFF@UBCMF@DIVOLT
UPUBM.
SXSSSPPUSPPU"VHFNQMPZFFTTDIFNBDSFBUFTRM
SXSSSPPUSPPU"VHFNQMPZFFTFNQMPZFFTTDIFNBTRM
SXSSSPPUSPPU,"VHFNQMPZFFTTDIFNBQPTUTRM
SXSSSPPUSPPU,"VHFNQMPZFFTFNQMPZFFTTRM
SXSSSPPUSPPU,"VHFNQMPZFFTFNQMPZFFTTRM
Non-blocking backup
To provide consistent backup, NZEVNQFS acquires (-0#"--0$, by executing '-64)
5"#-&48*5)3&"%-0$,.
You have already seen how dangerous it is to use '-64)5"#-&48*5)3&"%-0$, if

there are any long-running transactions (explained in $IBQUFS, Transactions). To avoid
that, you can pass the LJMMMPOHRVFSJFT option to kill blocking queries rather than
aborting NZEVNQFS.
[ 189 ]

Backups Chapter 7
USYDPOTJTUFODZPOMZ: This is equivalent to TJOHMFUSBOTBDUJPO for

NZTRMEVNQ but with a CJOMPH position. Obviously, this position only applies to
transactional tables. One of the advantages of using this option is that the global
read lock is only held for the threads' coordination, so it's released as soon as the
transactions are started.
VTFTBWFQPJOUT reduces metadata locking issues (needs the 461&3
privilege).
Compressed backups
You can specify the DPNQSFTT option to compress the backup:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE #FNQMPZFFT5FNQMPZFFTU
USYDPOTJTUFODZPOMZDPNQSFTTPVUQVUEJSCBDLVQTFNQMPZFFT@DPNQSFTT
TIFMM MTMIUSCBDLVQTFNQMPZFFT@DPNQSFTT
UPUBM.
SXSSSPPUSPPU"VHFNQMPZFFTTDIFNBDSFBUFTRMH[
SXSSSPPUSPPU"VHFNQMPZFFTFNQMPZFFTTDIFNBTRMH[
SXSSSPPUSPPU."VHFNQMPZFFTFNQMPZFFTTRMH[
Backing up only data

You can use the OPTDIFNBT option to skip the schema and take a data-only backup:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE #FNQMPZFFT5FNQMPZFFTU
OPTDIFNBTDPNQSFTTUSYDPOTJTUFODZPOMZPVUQVUEJS
CBDLVQTFNQMPZFFT@EBUB
Taking backups using flat files

This is a physical backup method whereby you take a backup by directly copying the files
inside EBUBEJSFDUPSZ. Since new data is written while you copy the files, the backup will
be inconsistent and cannot be used. To avoid that, you have to shut down MySQL, copy the
files, and then start MySQL. This method is not used for daily backups but is well suited
during maintenance windows for upgrades or downgrades or while doing a host swap.
[ 190 ]

Backups Chapter 7
How to do it...
1. Shut down the MySQL server:
TIFMM TVEPTFSWJDFNZTRMETUPQ
2. Copy the files into the EBUBEJSFDUPSZ (your directory may be different):
TIFMM TVEPSTZODBWEBUBNZTRMCBDLVQT
PSEPSTZODPWFSTTIUPSFNPUFTFSWFS
TIFMM STZODFTTIB[EBUBNZTRM
CBDLVQ@VTFS!SFNPUF@TFSWFSCBDLVQT
TIFMM TVEPTFSWJDFNZTRMETUBSU
Taking backups using XtraBackup

XtraBackup is an open source backup software provided by Percona. It copies flat files
without shutting down the server, but to avoid inconsistencies, it uses a redo log file. It is
widely used by many companies as a standard backup tool. The advantages are that it is
very fast compared to logical backup tools and recovery is also very fast.
This is how Percona XtraBackup works (taken from the Percona XtraBackup
documentation):
1. It copies your *OOP%# data files, which results in data that is internally
inconsistent; but then it performs crash recovery on the files to make them a
consistent, usable database again.
2. This works because *OOP%# maintains a redo log, also called the transaction log.
This contains a record of every change to the *OOP%# data. When *OOP%# starts, it
inspects the data files and the transaction log, and performs two steps. It applies
committed transaction log entries to the data files, and it performs an undo
operation on any transactions that modified data but did not commit.
[ 191 ]

Backups Chapter 7
3. Percona XtraBackup works by remembering the log sequence number (LSN)

when it starts, and then copying away the data files. It takes some time to do this,
so if the files are changing, then they reflect the state of the database at different
points in time. At the same time, Percona XtraBackup runs a background process
that watches the transaction log files, and copies changes from it. Percona
XtraBackup needs to do this continually because the transaction logs are written
in a round-robin fashion, and can be reused after a while. Percona XtraBackup
needs the transaction log records for every change to the data files since it began
execution.
How to do it...
At the time of writing, Percona XtraBackup is not supported for MySQL 8. Eventually,
Percona will release a new version of XtraBackup supporting MySQL 8; hence only the
installation is covered.
Installation
The installation steps are in the following sections.
0O$FOU043FE)BU'FEPSB
1. Install NZTRMDPNNVOJUZMJCTDPNQBU:
TIFMM TVEPZVNJOTUBMMZNZTRMDPNNVOJUZMJCTDPNQBU
2. Install the Percona repository:
TIFMM TVEPZVNJOTUBMM
IUUQXXXQFSDPOBDPNEPXOMPBETQFSDPOBSFMFBTFSFEIBUQFSDP
OBSFMFBTFOPBSDISQN
You should see some output such as the following:

3FUSJFWJOH
IUUQXXXQFSDPOBDPNEPXOMPBETQFSDPOBSFMFBTFSFEIBUQFSDP
OBSFMFBTFOPBSDISQN
1SFQBSJOH
<>
QFSDPOBSFMFBTF
<>
[ 192 ]

Backups Chapter 7
3. Test the repository:
shell> yum list | grep xtrabackup

IPMMBOEYUSBCBDLVQOPBSDIFMFQFM
QFSDPOBYUSBCBDLVQY@FMQFSDPOBSFMFBTFY@
QFSDPOBYUSBCBDLVQEFCVHJOGPY@FMQFSDPOB
SFMFBTFY@
QFSDPOBYUSBCBDLVQEFCVHJOGPY@FMQFSDPOBSFMFBTF
Y@
QFSDPOBYUSBCBDLVQEFCVHJOGPY@FMQFSDPOBSFMFBTF
Y@
QFSDPOBYUSBCBDLVQUFTUY@FMQFSDPOBSFMFBTFY@
QFSDPOBYUSBCBDLVQUFTUY@FMQFSDPOBSFMFBTF
Y@
QFSDPOBYUSBCBDLVQUFTUY@FMQFSDPOBSFMFBTF
Y@
4. Install XtraBackup:
TIFMM TVEPZVNJOTUBMMQFSDPOBYUSBCBDLVQ
0O%FCJBO6CVOUV
1. Fetch the repository packages from Percona:
TIFMM XHFUIUUQTSFQPQFSDPOBDPNBQUQFSDPOB
SFMFBTF@
MTC@SFMFBTFTD@BMMEFC
2. Install the downloaded package with EQLH. To do that, run the following
commands as SPPU or with TVEP:
TIFMM TVEPEQLHJQFSDPOBSFMFBTF@
MTC@SFMFBTF
TD@BMMEFC
Once you install this package, the Percona repositories should be added. You can
check the repository setup in the FUDBQUTPVSDFTMJTUEQFSDPOB
SFMFBTFMJTU file.
3. Remember to update the local cache:
[ 193 ]

Backups Chapter 7
4. After that, you can install the package:
TIFMM TVEPBQUHFUJOTUBMMQFSDPOBYUSBCBDLVQ
Locking instances for backup

As of MySQL 8, you can lock the instance for backup, which will allow the DML during
online backup and block all the operations that could result in an inconsistent snapshot.
How to do it...
Before you begin your backup, lock the instance for backup:
NZTRM -0$,*/45"/$&'03#"$,61
Perform the backup, and after completion, unlock the instance:

NZTRM 6/-0$,*/45"/$&
Binary log backup

You know that binary logs are needed for point-in-time recovery. In this section, you will
understand how to take a backup of binary logs. The process streams the binary logs from
the database server to a remote backup server. You can take the binary log backup from
either the slave or the master. If you are taking the binary log backup from the master and
the actual backup from the slave, you should use EVNQTMBWF to get the corresponding
master log position. If you are using NZEVNQFS or XtraBackup, it gives both the master and
slave binary log positions.
How to do it...
1. Create a replication user on the server. Create a strong password:
NZTRM (3"/53&1-*$"5*0/4-"7&0/50 CJOMPH@VTFS !

*%&/5*'*&%#: CJOMPH@QBTT
TFD
[ 194 ]

Backups Chapter 7
2. Check the binary logs on the server:
NZTRM 4)08#*/"3:-0(4


]TFSWFS]]
]TFSWFS]]
]TFSWFS]]
]TFSWFS]]
]TFSWFS]]

SPXTJOTFU
TFD
You can find the first binary log available on the server; from this, you can start
the backup. In this case, it is TFSWFS.
3. Log in to the backup server and execute the following command. This will copy
the binary logs from the MySQL server to the backup server. You can start using
OPIVQ or EJTPXO:
TIFMM NZTRMCJOMPHVVTFS QQBTT ITFSWFS SFBEGSPN

SFNPUFTFSWFSTUPQOFWFS
UPMBTUMPHSBXTFSWFS
TIFMM EJTPXOB
4. Verify that the binary logs are being backed up:
TIFMM MTMIUSTFSWFS
SXSNZTRMNZTRM,"VHTFSWFS
SXSNZTRMNZTRM,"VHTFSWFS
[ 195 ]

Restoring Data
8
Recovering from mysqldump and mysqlpump

Recovering from mydumper using myloader
Recovering from flat file backup
Performing point-in-time recovery
Introduction
In this chapter, you will learn about various backup restoration methods. Assume that the
backups and binary logs are available on the server.
Recovering from mysqldump and

mysqlpump
The logical backup tools NZTRMEVNQ and NZTRMQVNQ write data to a single file.
How to do it...
Log in to the server where the backups are available:
TIFMM DBUCBDLVQTGVMM@CBDLVQTRM]NZTRMVVTFS Q
PS
TIFMM NZTRMVVTFS QCBDLVQTGVMM@CBDLVQTRM

Restoring Data Chapter 8
To restore on the remote server, you can mention the IIPTUOBNF option:
TIFMM DBUCBDLVQTGVMM@CBDLVQTRM]NZTRMVVTFS QI
SFNPUF@IPTUOBNF
When you are restoring a backup, the backup statements will be logged to the binary log,
which can slow down the restoration process. If you do not want the restoration process to
write to the binary log, you can disable it at the session level using the 4&5
42-@-0(@#*/ option:
TIFMM
FDIP4&542-@-0(@#*/DBUCBDLVQTGVMM@CBDLVQTRM]NZTRMV
VTFS QISFNPUF@IPTUOBNF
Or using:
NZTRM 4&542-@-0(@#*/4063$&GVMM@CBDLVQTRM
There's more...
1. Since backup restoration takes a very long time, it is recommended to start the
restoration process inside a screen session so that even if you lose connectivity to
the server, the restoration will continue.
2. Sometimes, there can be failures during restoration. If you pass the GPSDF
option to MySQL, restoration will continue:
TIFMM
FDIP4&542-@-0(@#*/DBUCBDLVQTGVMM@CBDLVQTRM]
NZTRMVVTFS QISFNPUF@IPTUOBNF G
Recovering from mydumper using myloader

NZMPBEFS is a tool used for multi-threaded restoration of backups taken using NZEVNQFS.
NZMPBEFS comes along with NZEVNQFS and you don't need to install it separately. In this
section, you will learn the various ways to restore a backup.
How to do it...
The common options for NZMPBEFS are the hostname of the MySQL server to connect to
(the default is MPDBMIPTU), username, password, and port.
[ 197 ]

Recovering full database

TIFMM NZMPBEFSEJSFDUPSZCBDLVQTVTFSVTFS QBTTXPSEQBTTXPSE
RVFSJFTQFSUSBOTBDUJPOUISFBETDPNQSFTTQSPUPDPMPWFSXSJUF
UBCMFT
The options are explained as follows:
PWFSXSJUFUBCMFT: This option drops the tables if they already exist

DPNQSFTTQSPUPDPM : This option uses compression on the MySQL
connection
UISFBET : This option specifies the number of threads to use; the default is
RVFSJFTQFSUSBOTBDUJPO : This specifies the number of queries per
transaction; the default is
EJSFDUPSZ: This specifies the directory of the dump to import
Recover a single database

You can specify TPVSDFECEC@OBNF to restore only a single database.
Suppose you want to restore the DPNQBOZ database:

TIFMM NZMPBEFSEJSFDUPSZCBDLVQTRVFSJFTQFSUSBOTBDUJPO
UISFBETDPNQSFTTQSPUPDPMVTFSVTFS QBTTXPSEQBTTXPSE TPVSDF
ECDPNQBOZPWFSXSJUFUBCMFT
Recovering a single table

NZEVNQFS writes the backup of each table to a separate TRM file. You can pick up the TRM
file and restore:
TIFMM NZTRMVVTFS QQBTTXPSE IIPTUOBNF DPNQBOZ"G
DPNQBOZQBZNFOUTTRM
If the table is split into chunks, you can copy all the chunks and information related to the
table to a directory and specify the location.
[ 198 ]

Copy the required files:

TIFMM TVEPDQCBDLVQTFNQMPZFF@UBCMF@DIVOLTFNQMPZFFTFNQMPZFFT=
CBDLVQTFNQMPZFF@UBCMF@DIVOLTFNQMPZFFTFNQMPZFFTTDIFNBTRM=
CBDLVQTFNQMPZFF@UBCMF@DIVOLTFNQMPZFFTTDIFNBDSFBUFTRM=
CBDLVQTFNQMPZFF@UBCMF@DIVOLTNFUBEBUB=
CBDLVQTTJOHMF@UBCMF
Use NZMPBEFS to load; it will automatically detect the chunks and load them:
TIFMM NZMPBEFSEJSFDUPSZCBDLVQTTJOHMF@UBCMFRVFSJFTQFS
USBOTBDUJPOUISFBETDPNQSFTTQSPUPDPMPWFSXSJUFUBCMFT
Recovering from flat file backup

Recovering from flat files requires you to stop the MySQL server, replace all the files,
change the permissions, and start MySQL.
How to do it...
1. Stop the MySQL server:
2. Move the files to the EBUBEJSFDUPSZ:
TIFMM TVEPNWCBDLVQNZTRMWBSMJC
3. Change the ownership to NZTRM:
TIFMM TVEPDIPXO3NZTRMNZTRMWBSMJCNZTRM
4. Start MySQL:
[ 199 ]

To minimize the downtime, if you have enough space on disk, you can
copy to the backup to WBSMJCNZTRM. Then stop MySQL, rename the
directory, and start the server:
TIFMM TVEPNWCBDLVQNZTRMWBSMJCNZTRM
TIFMM TVEPNWWBSMJCNZTRMWBSMJCNZTRM
TIFMM TVEPDIPXO3NZTRMNZTRMWBSMJCNZTRM
Performing point-in-time recovery

Once the full backup is restored, you need to restore binary logs to get point-in-time
recovery. The backups provide the binary log coordinates up to which the backups are
available.
As explained in $IBQUFS, Backups, in the Locking instance for backup section, you should
choose the binary log backup from the right server, based on the EVNQTMBWF or
NBTUFSEBUB option specified in NZTRMEVNQ.
How to do it...
Let's get into the details of doing it. There's a lot to learn here though.
mysqldump or mysqlpump
The binary log information is stored in the SQL file as the $)"/(&."45&350 command
based on the options you passed to NZTRMEVNQ/NZTRMQVNQ.
1. If you have used NBTUFSEBUB, you should use the binary logs of the slave:
shell> head -30 /backups/dump.sql

.Z42-EVNQ%JTUSJCSDGPS-JOVY
Y@

)PTUMPDBMIPTU%BUBCBTF

4FSWFSWFSTJPOSDMPH
4&5!0-%@$)"3"$5&3@4&5@$-*&/5!!$)"3"$5&3@4&5@$-*&/5
4&5!0-%@$)"3"$5&3@4&5@3&46-54!!$)"3"$5&3@4&5@3&46-54
[ 200 ]

4&5!0-%@$0--"5*0/@$0//&$5*0/!!$0--"5*0/@$0//&$5*0/
4&5/".&4VUG
4&5!0-%@5*.&@;0/&!!5*.&@;0/&
4&55*.&@;0/&
4&5
!0-%@*//0%#@45"54@"650@3&$"-$!!*//0%#@45"54@"650@3&$"-$
4&5(-0#"-*//0%#@45"54@"650@3&$"-$0''
4&5!0-%@6/*26&@$)&$,4!!6/*26&@$)&$,46/*26&@$)&$,4

4&5!0-%@'03&*(/@,&:@$)&$,4!!'03&*(/@,&:@$)&$,4
'03&*(/@,&:@$)&$,4
4&5!0-%@42-@.0%&!!42-@.0%&
42-@.0%& /0@"650@7"-6&@0/@;&30
4&5!0-%@42-@/05&4!!42-@/05&442-@/05&4

-- Position to start replication or point-in-time recovery from

CHANGE MASTER TO MASTER_LOG_FILE='server1.000008',
MASTER_LOG_POS=154;
In this case, you should start the restore from the TFSWFS file at
position on the slave.
TIFMM NZTRMCJOMPHTUBSUQPTJUJPOEJTBCMFMPHCJO
CBDLVQTCJOMPHTTFSWFS]NZTRMVVTFS QIIPTU G
2. If you have used EVNQTMBWF, you should use the binary logs of the master:


UIFNBTUFSPGUIJTTMBWF

$)"/(&."45&350."45&3@-0(@'*-& DFOUPTCJO
."45&3@-0(@104
In this case, you should start the restore from the DFOUPT
CJO file at position located from the master.
CBDLVQTCJOMPHTDFOUPTCJO]NZTRMVVTFS QIIPTU G
[ 201 ]

mydumper
The binary log information is available in the metadata:
shell> sudo cat /backups/metadata
4UBSUFEEVNQBU
4)08."45&345"564
-PHTFSWFS
1PT
TQBO (5*%
4)084-"7&45"564
)PTU
-PHDFOUPTCJO
1PT
(5*%
'JOJTIFEEVNQBU
If you have taken the binary log backup from the slave, you should start the restore from
the TFSWFS file at position (4)08."45&345"564):
CBDLVQTCJOMPHTTFSWFS]NZTRMVVTFS QIIPTU G
If you have a binary log backup from the master, you should start the restore from
DFOUPTCJO at position (4)084-"7&45"564):
CBDLVQTCJOMPHTDFOUPTCJO]NZTRMVVTFS QIIPTU G
[ 202 ]

9
Replication
Setting up replication
Setting up master-master replication
Setting up multi-source replication
Setting up replication filters
Switching a slave from master-slave to chain replication
Switching a slave from chain replication to master-slave
Setting up delayed replication
Setting up GTID replication
Setting up semi-synchronous replication
Introduction
As explained in $IBQUFS, Binary Logging, replication enables data from one MySQL
database server (the master) to be copied to one or more MySQL database servers (the
slaves). Replication is asynchronous by default; slaves do not need to be
permanently connected to receive updates from the master. You can configure to replicate
all databases, selected databases, or even selected tables within a database.
In this chapter, you will learn how to set up traditional replication; replicate selected
databases and tables; and set up multi-source replication, chain replication, delayed
replication, and semi-synchronous replication.

Replication Chapter 9
On a high level, replication works like this: all DDL and DML statements executed on a
server (master) are logged into binary logs, which are pulled by the servers connecting to it
(called slaves). The binary logs are simply copied to the slaves and are saved as relay logs.
This process is taken care of by a thread called IO thread. There is one more thread called
SQL thread, that executes the statements in the relay log sequentially.
How replication works is very clearly explained in this blog:

IUUQTXXXQFSDPOBDPNCMPHIPXEPFTNZTRMSFQMJDBUJPOSFBMMZXPSL
The advantages of replication are (taken from the manual, at IUUQTEFWNZTRMDPNEPD

SFGNBOFOSFQMJDBUJPOIUNM):
Scale-out solutions: Spreading the load among multiple slaves to improve

performance. In this environment, all writes and updates must take place on the
master server. Reads, however, may take place on one or more slaves. This model
can improve the performance of writes (since the master is dedicated to updates),
while dramatically increasing read speed across an increasing number of slaves.
Data security: Because data is replicated to the slave and the slave can pause the
replication process, it is possible to run backup services on the slave without
corrupting the corresponding master data.
Analytics: Live data can be created on the master, while the analysis of the
information can take place on the slave without affecting the performance of the
master.
Long-distance data distribution: You can use replication to create a local copy of
data for a remote site to use without permanent access to the master.
Setting up replication
There are many replication topologies. Some of them are the traditional master-slave
replication, chain replication, master-master replication, multi-source replication, and so on.
Traditional replication involves a single master and multiple slaves.
[ 204 ]

Chain replication means one server replicates from another, which in turn replicates from
another. The middle server is referred to as the relay master (master ---> relay master --->
slave).
This is mainly used when you want to set up replication between two data centers. The
primary master and its slaves will be in one data center. The secondary master (relay)
replicates from the primary master in the other data center. All the slaves of the other data
center are replicated from the secondary master.
[ 205 ]

Master-master replication: In this topology, both the masters accept writes and replicate
between each other.
Multi-source replication: In this topology, a slave will replicate from multiple masters
instead of one.
If you want to set up chain replication, you can follow the same steps mentioned here,
replacing the master with the relay master.
[ 206 ]

How to do it...
In this section, setting up of single slave is explained. The same principles can be applied to
set up chain replication. Usually the backups are taken from the slave when setting up
another slave.
Outline:
1. Enable binary logging on the master

2. Create a replication user on the master
3. Set the unique TFSWFS@JE on the slave
4. Take backup from the master
5. Restore the backup on the slave
6. Execute the $)"/(&."45&350 command
7. Start the replication
Steps:
1. On master: Enable binary logging on the master and set 4&37&3@*%. Refer to
$IBQUFS, Binary Logging, to learn how to enable binary logging.
2. On master: Create a replication user. The slave connects to the master using this
account:
NZTRM (3"/53&1-*$"5*0/4-"7&0/50 CJOMPH@VTFS !

*%&/5*'*&%#: CJOMPH@1!TT
TFD
3. On slave: Set the unique 4&37&3@*% option (it should be different from what you
have set on master):
NZTRM 4&5!!(-0#"-4&37&3@*%
4. On slave: Take backup from the master by remotely connecting. You can
use either NZTRMEVNQ or NZEVNQFS. NZTRMQVNQ cannot be used because the
binary log positions won't be consistent.
NZTRMEVNQ:
TIFMM NZTRMEVNQINBTUFS@IPTU VCBDLVQ@VTFSQBTTXPSEQBTT

BMMEBUBCBTFTSPVUJOFTFWFOUTTJOHMFUSBOTBDUJPONBTUFS
EBUB EVNQTRM
[ 207 ]

You have to pass the TMBWFEVNQ option when taking backup from another
slave.
NZEVNQFS:
TIFMM NZEVNQFSINBTUFS@IPTU VCBDLVQ@VTFSQBTTXPSEQBTT

VTFTBWFQPJOUTUSYDPOTJTUFODZPOMZLJMMMPOHRVFSJFT
PVUQVUEJSCBDLVQT
5. On slave: After the backup completes, restore the backup. Refer to $IBQUFS,
Restoring Data, for the restoration methods.
NZTRMEVNQ:
TIFMM NZTRMVVTFS QGEVNQTRM
NZEVNQFS:
TIFMM NZMPBEFSEJSFDUPSZCBDLVQTVTFSVTFS

QBTTXPSEQBTTXPSE RVFSJFTQFSUSBOTBDUJPOUISFBET
PWFSXSJUFUBCMFT
6. On slave: After restoring the backup, you have to execute the following
command:
NZTRM $)"/(&."45&350."45&3@)045 NBTUFS@IPTU

."45&3@64&3 CJOMPH@VTFS ."45&3@1"44803% CJOMPH@1!TT
."45&3@-0(@'*-& MPH@GJMF@OBNF ."45&3@-0(@104QPTJUJPO
NZTRMEVNQ: MPH@GJMF@OBNF and QPTJUJPO are included in the backup

dump file. For example:
TIFMM MFTTEVNQTRM


UIFNBTUFSPGUIJTTMBWF

CHANGE MASTER TO MASTER_LOG_FILE='centos7-bin.000001',
MASTER_LOG_POS=463;
NZEVNQFS: MPH@GJMF@OBNF and QPTJUJPO are stored in the metadata file:
TIFMM DBUNFUBEBUB
4UBSUFEEVNQBU
4)08."45&345"564
-PHTFSWFS
1PT
(5*%
4)084-"7&45"564
[ 208 ]

)PTUYYYYYYYYYYY
-PHDFOUPTCJO
1PT
(5*%
'JOJTIFEEVNQBU
If you are taking backup from a slave or master to set up another slave, you have to use
positions from 4)084-"7&45"564. If you want to set up the chain replication, you can
use the positions from 4)08."45&345"564.
1. On the slave, execute the 45"354-"7& command:
NZTRM 45"354-"7&
2. You can check the status of replication by executing:
NZTRM 4)084-"7&45"564=(
SPX
Slave_IO_State: Waiting for master to send event
.BTUFS@)PTUYYYYYYYYYYY
.BTUFS@6TFSCJOMPH@VTFS
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFSCJO
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@'JMFTFSWFSSFMBZCJO
3FMBZ@-PH@1PT
3FMBZ@.BTUFS@-PH@'JMFTFSWFSCJO
4MBWF@*0@3VOOJOH:FT
4MBWF@42-@3VOOJOH:FT
3FQMJDBUF@%P@%#
3FQMJDBUF@*HOPSF@%#
3FQMJDBUF@%P@5BCMF
3FQMJDBUF@*HOPSF@5BCMF
3FQMJDBUF@8JME@%P@5BCMF
3FQMJDBUF@8JME@*HOPSF@5BCMF
-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@"MMPXFE/P
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
[ 209 ]

.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
Seconds_Behind_Master: 0
.BTUFS@44-@7FSJGZ@4FSWFS@$FSU/P
-BTU@*0@&SSOP
-BTU@*0@&SSPS
-BTU@42-@&SSOP
-BTU@42-@&SSPS
3FQMJDBUF@*HOPSF@4FSWFS@*ET
.BTUFS@4FSWFS@*E
.BTUFS@66*%CFGBGGFB
.BTUFS@*OGP@'JMFWBSMJCNZTRMNBTUFSJOGP
42-@%FMBZ
42-@3FNBJOJOH@%FMBZ/6--
4MBWF@42-@3VOOJOH@4UBUF4MBWFIBTSFBEBMMSFMBZMPH
XBJUJOHGPSNPSFVQEBUFT
.BTUFS@3FUSZ@$PVOU
.BTUFS@#JOE
-BTU@*0@&SSPS@5JNFTUBNQ
-BTU@42-@&SSPS@5JNFTUBNQ
.BTUFS@44-@$SM
.BTUFS@44-@$SMQBUI
3FUSJFWFE@(UJE@4FU
&YFDVUFE@(UJE@4FU
"VUP@1PTJUJPO
3FQMJDBUF@3FXSJUF@%#
$IBOOFM@/BNF
.BTUFS@5-4@7FSTJPO
SPXJOTFU
TFD
You should look for 4FDPOET@#FIJOE@.BTUFS, which shows the replication lag. If it is , it
means the slave is in sync with the master; any non-zero value indicates the number of
seconds of lag, and if it is /6--, it means replication is not happening.
Setting up master-master replication

This recipe will interest many people since many of us try doing this. Let's get into the
details of it.
[ 210 ]

How to do it...
Assume that the masters are NBTUFS and NBTUFS.
Steps:
1. Set up replication between NBTUFS and NBTUFS as described in $IBQUFS,

Replication.
2. Make NBTUFS read-only:
NZTRM 4&5!!(-0#"-3&"%@0/-:0/
3. On NBTUFS, check the current binary log coordinate.
NZTRM 4)08."45&345"564

]'JMF]1PTJUJPO]#JOMPH@%P@%#]#JOMPH@*HOPSF@%#]
&YFDVUFE@(UJE@4FU]

]TFSWFS]]]]
]

SPXJOTFU
TFD
From the preceding output, you can start the replication on NBTUFS from
TFSWFS and position .
4. From the positions taken from the preceding step, execute the $)"/(&."45&3
50 command on NBTUFS:
NZTRM $)"/(&."45&350."45&3@)045 NBTUFS@IPTU

."45&3@64&3 CJOMPH@VTFS ."45&3@1"44803% CJOMPH@1!TT
."45&3@-0(@'*-& MPH@GJMF@OBNF ."45&3@-0(@104QPTJUJPO
5. Start the slave on NBTUFS:
NZTRM 45"354-"7&
6. Finally, you can make NBTUFS read-write, and applications can start writing to
it.
NZTRM 4&5!!(-0#"-3&"%@0/-:0''
[ 211 ]

Setting up multi-source replication

MySQL multi-source replication enables a replication slave to receive transactions from
multiple sources simultaneously. Multi-source replication can be used to back up multiple
servers to a single server, merge table shards, and consolidate data from multiple servers to
a single server. Multi-source replication does not implement any conflict detection or
resolution when applying transactions, and those tasks are left to the application if
required. In a multi-source replication topology, a slave creates a replication channel for
each master that it should receive transactions from.
In this section, you will learn how to set up a slave with multiple masters. This method is
the same as setting up traditional replication over the channels.
How to do it...
Assume that you are setting up TFSWFS as a slave of TFSWFS and TFSWFS. You need to
create traditional replication from TFSWFS to TFSWFS over a channel and from TFSWFS
to TFSWFS over another channel. To ensure that data is consistent on the slave, make sure
that different sets of databases are replicated or the application takes care of the conflicts.
Before you begin, take a backup from server1 and restore on TFSWFS; similarly take a
backup from TFSWFS and restore on TFSWFS, as described in $IBQUFS, Replication.
1. On TFSWFS, modify the replication repositories to 5"#-& from '*-&. You can
change it dynamically by running the following commands:
NZTRM 45014-"7&*GTMBWFJTBMSFBEZSVOOJOH
NZTRM 4&5(-0#"-NBTUFS@JOGP@SFQPTJUPSZ 5"#-&
NZTRM 4&5(-0#"-SFMBZ@MPH@JOGP@SFQPTJUPSZ 5"#-&
Also make the changes in the configuration file:

<NZTRME>
NBTUFSJOGPSFQPTJUPSZ5"#-&
SFMBZMPHJOGPSFQPTJUPSZ5"#-&
[ 212 ]

2. On TFSWFS, execute the $)"/(&."45&350 command to make it a slave of

TFSWFS over a channel named NBTUFS. You can name it anything:
NZTRM $)"/(&."45&350."45&3@)045 TFSWFS

."45&3@64&3 CJOMPH@VTFS ."45&3@1035
."45&3@1"44803% CJOMPH@1!TT ."45&3@-0(@'*-& TFSWFS
."45&3@-0(@104'03$)"//&- NBTUFS
3. On TFSWFS, execute the $)"/(&."45&350 command to make it a slave of

TFSWFS over channel NBTUFS:
NZTRM $)"/(&."45&350."45&3@)045 TFSWFS

."45&3@64&3 CJOMPH@VTFS ."45&3@1035
."45&3@1"44803% CJOMPH@1!TT ."45&3@-0(@'*-& TFSWFS
."45&3@-0(@104'03$)"//&- NBTUFS
4. Execute the 45"354-"7&'03$)"//&- statement for each channel as follows:
NZTRM 45"354-"7&'03$)"//&- NBTUFS

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 45"354-"7&'03$)"//&- NBTUFS

2VFSZ0,SPXTBGGFDUFE
TFD
5. Verify the slave status by executing the 4)084-"7&45"564 statement:
NZTRM 4)084-"7&45"564=(
SPX
4MBWF@*0@4UBUF8BJUJOHGPSNBTUFSUPTFOEFWFOU
.BTUFS@)PTUTFSWFS
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFS
3FBE@.BTUFS@-PH@1PT
NBTUFS!E
3FMBZ@-PH@1PT
3FMBZ@.BTUFS@-PH@'JMFTFSWFS
4MBWF@*0@3VOOJOH:FT
3FQMJDBUF@%P@%#
3FQMJDBUF@*HOPSF@%#
3FQMJDBUF@%P@5BCMF
[ 213 ]

-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
4FDPOET@#FIJOE@.BTUFS
-BTU@*0@&SSOP
-BTU@*0@&SSPS
-BTU@42-@&SSOP
-BTU@42-@&SSPS
.BTUFS@4FSWFS@*E
.BTUFS@66*%DDGDBEFCFBFB
.BTUFS@*OGP@'JMFNZTRMTMBWF@NBTUFS@JOGP
42-@%FMBZ
.BTUFS@3FUSZ@$PVOU
.BTUFS@#JOE
.BTUFS@44-@$SM
.BTUFS@44-@$SMQBUI
3FUSJFWFE@(UJE@4FU
&YFDVUFE@(UJE@4FU
"VUP@1PTJUJPO
$IBOOFM@/BNFNBTUFS
.BTUFS@5-4@7FSTJPO
SPX
.BTUFS@)PTUTFSWFS
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFS
[ 214 ]

3FBE@.BTUFS@-PH@1PT
NBTUFS!E
3FMBZ@-PH@1PT
3FMBZ@.BTUFS@-PH@'JMFTFSWFS
4MBWF@*0@3VOOJOH:FT
3FQMJDBUF@%P@%#
3FQMJDBUF@*HOPSF@%#
3FQMJDBUF@%P@5BCMF
-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
-BTU@*0@&SSOP
-BTU@*0@&SSPS
-BTU@42-@&SSOP
-BTU@42-@&SSPS
.BTUFS@4FSWFS@*E
.BTUFS@66*%CFGBGGFB
42-@%FMBZ
.BTUFS@3FUSZ@$PVOU
.BTUFS@#JOE
.BTUFS@44-@$SM
.BTUFS@44-@$SMQBUI
3FUSJFWFE@(UJE@4FU
[ 215 ]

&YFDVUFE@(UJE@4FU
"VUP@1PTJUJPO
$IBOOFM@/BNFNBTUFS
.BTUFS@5-4@7FSTJPO
SPXTJOTFU
TFD
6. To get the slave status for a particular channel, execute:
NZTRM 4)084-"7&45"564'03$)"//&- NBTUFS =(
7. This is the other way you can use a performance schema to monitor the metrics:
NZTRM 4&-&$5'30.

QFSGPSNBODF@TDIFNBSFQMJDBUJPO@DPOOFDUJPO@TUBUVT=(
SPX
$)"//&-@/".&NBTUFS
(3061@/".&
4063$&@66*%
DDGDBEFCFBFB
5)3&"%@*%
4&37*$&@45"5&0/
$06/5@3&$&*7&%@)&"35#&"54
-"45@)&"35#&"5@5*.&45".1

3&$&*7&%@53"/4"$5*0/@4&5
-"45@&3303@/6.#&3
-"45@&3303@.&44"(&
-"45@&3303@5*.&45".1

-"45@26&6&%@53"/4"$5*0/
-"45@26&6&%@53"/4"$5*0/@03*(*/"-@$0..*5@5*.&45".1

-"45@26&6&%@53"/4"$5*0/@*..&%*"5&@$0..*5@5*.&45".1

-"45@26&6&%@53"/4"$5*0/@45"35@26&6&@5*.&45".1

-"45@26&6&%@53"/4"$5*0/@&/%@26&6&@5*.&45".1

26&6&*/(@53"/4"$5*0/
26&6&*/(@53"/4"$5*0/@03*(*/"-@$0..*5@5*.&45".1

26&6&*/(@53"/4"$5*0/@*..&%*"5&@$0..*5@5*.&45".1

26&6&*/(@53"/4"$5*0/@45"35@26&6&@5*.&45".1

SPX
[ 216 ]

$)"//&-@/".&NBTUFS
(3061@/".&
4063$&@66*%
CFGBGGFB
5)3&"%@*%
4&37*$&@45"5&0/
$06/5@3&$&*7&%@)&"35#&"54
-"45@)&"35#&"5@5*.&45".1

3&$&*7&%@53"/4"$5*0/@4&5
-"45@&3303@/6.#&3
-"45@&3303@.&44"(&
-"45@&3303@5*.&45".1

-"45@26&6&%@53"/4"$5*0/
-"45@26&6&%@53"/4"$5*0/@03*(*/"-@$0..*5@5*.&45".1

-"45@26&6&%@53"/4"$5*0/@*..&%*"5&@$0..*5@5*.&45".1

-"45@26&6&%@53"/4"$5*0/@45"35@26&6&@5*.&45".1

-"45@26&6&%@53"/4"$5*0/@&/%@26&6&@5*.&45".1

26&6&*/(@53"/4"$5*0/
26&6&*/(@53"/4"$5*0/@03*(*/"-@$0..*5@5*.&45".1

26&6&*/(@53"/4"$5*0/@*..&%*"5&@$0..*5@5*.&45".1

26&6&*/(@53"/4"$5*0/@45"35@26&6&@5*.&45".1

SPXTJOTFU
TFD
You can specify all the slave-related commands for a channel by

appending '03$)"//&- DIBOOFM@OBNF :
NZTRM 45014-"7&'03$)"//&- NBTUFS

NZTRM 3&4&54-"7&'03$)"//&- NBTUFS
[ 217 ]

Setting up replication filters

You can control which tables or databases are to be replicated. On the master, you can
control which databases to log changes for by using the CJOMPHEPEC and CJOMPH
JHOPSFEC options to control binary logging, as mentioned in $IBQUFS, Binary Logging.
The better way is to control on the slave side. You can execute or ignore statements received
from the master by using SFQMJDBUF options or dynamically by creating replication
filters.
How to do it...
To create a filter, you need to execute the $)"/(&3&1-*$"5*0/'*-5&3 statement.
Replicate a database only

Assume that you want to replicate EC and EC only. Use the following statement to create
the replication filter.
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@%0@%#
ECEC
Note that you should specify all the databases inside parentheses.
Replicate specific tables

You can specify the tables you want to be replicated using 3&1-*$"5&@%0@5"#-&:
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@%0@5"#-&
ECUBCMF
Suppose that you want to use regex for tables; you can use
the 3&1-*$"5&@8*-%@%0@5"#-& option:
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@8*-%@%0@5"#-&
ECJNQ
You can mention some databases or tables with regex using various *(/03& options.
[ 218 ]

Ignore a database
Just like you can choose to replicate a database, you can ignore a database from replication
using 3&1-*$"5&@*(/03&@%#:
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@*(/03&@%#
ECEC
Ignore specific tables

You can ignore certain tables using the 3&1-*$"5&@*(/03&@5"#-& and
3&1-*$"5&@8*-%@*(/03&@5"#-& options. The 3&1-*$"5&@8*-%@*(/03&@5"#-& option
allows wildcard characters, where as 3&1-*$"5&@*(/03&@5"#-& only accepts full table
names:
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@*(/03&@5"#-&
ECUBCMF
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@8*-%@*(/03&@5"#-&
ECOFX
ECOFX
You can also set filters for a channel by specifying the channel name:
NZTRM $)"/(&3&1-*$"5*0/'*-5&33&1-*$"5&@%0@%#
E'03$)"//&-
NBTUFS
See also
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFODIBOHFSFQMJDBUJPOGJMUFSIUNM
for more details on replication filters. If you are using more than one filter, refer to IUUQT
EFWNZTRMDPNEPDSFGNBOFOSFQMJDBUJPOSVMFTIUNM to know more about how
MySQL evaluates filters.
[ 219 ]

Switching slave from master-slave to chain

replication
If you have a master-slave replication set up, Servers B and C replicating from Server A:
Server A --> (Server B, Server C) and you wish to make Server C a slave of Server B, you
have to stop replication on both Server B and Server C. Then bring them to the same master
log position using the 45"354-"7&6/5*- command. After that, you can get the master
log coordinates from Server B and execute the $)"/(&."45&350 command on Server C.
How to do it...
1. On Server C: Stop slave and note the 3FMBZ@.BTUFS@-PH@'JMF and
&YFD@.BTUFS@-PH@1PT positions in the 4)084-"7&45"564=( command:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)084-"7&45"564=(
SPX
4MBWF@*0@4UBUF
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFS@"CJO
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@'JMFTFSWFS@$SFMBZCJO
3FMBZ@-PH@1PT
Relay_Master_Log_File: server_A-bin.000023
_
Exec_Master_Log_Pos: 2604
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
_
SPXJOTFU
TFD
[ 220 ]

2. On Server B: Stop slave and note the 3FMBZ@.BTUFS@-PH@'JMF and

&YFD@.BTUFS@-PH@1PT positions in the 4)084-"7&45"564=( command:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)084-"7&45"564=(
SPX
4MBWF@*0@4UBUF
.BTUFS@1PSU
$POOFDU@3FUSZ
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@'JMFTFSWFS@#SFMBZCJO
3FMBZ@-PH@1PT
_
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
_
SPXJOTFU
TFD
3. Compare the Server B log positions with Server C and find out which is the latest
sync with Server A. Usually, since you have stopped the slave on Server C first,
Server B will be ahead. In our case, the log positions are:
Server C: (TFSWFS@"CJO, )
Server B: (TFSWFS@"CJO, )
Server B is ahead, so we have to bring Server C to the position of Server B.
4. On Server C: Use the 45"354-"7&6/5*- statement to sync up to the position
of server B:
NZTRM 45"354-"7&6/5*-."45&3@-0(@'*-& DFOUPTCJO

."45&3@-0(@104
TFD
NZTRM 4)088"3/*/(4=(
SPX
-FWFM/PUF
$PEF
.FTTBHF*UJTSFDPNNFOEFEUPVTFTLJQTMBWFTUBSUXIFOEPJOH
[ 221 ]

TUFQCZTUFQSFQMJDBUJPOXJUI45"354-"7&6/5*-PUIFSXJTFZPV
XJMMHFUQSPCMFNTJGZPVHFUBOVOFYQFDUFETMBWF TNZTRMESFTUBSU
SPXJOTFU
TFD
5. On Server C: Wait for server C to catch up by checking &YFD@.BTUFS@-PH@1PT

and 6OUJM@-PH@1PT (both should be the same) in the output of 4)084-"7&
45"564:
NZTRM 4)084-"7&45"564=(
SPX
.BTUFS@1PSU
$POOFDU@3FUSZ
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@1PT
3FMBZ@.BTUFS@-PH@'JMFTFSWFS@"CJO
4MBWF@*0@3VOOJOH:FT
4MBWF@42-@3VOOJOH/P
_
-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO.BTUFS
6OUJM@-PH@'JMFTFSWFS@"CJO
Until_Log_Pos: 8250241
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
Seconds_Behind_Master: NULL
_
SPXJOTFU
TFD
[ 222 ]

6. On Server B: Find out the master status, start the slave, and make sure it is
replicating:
NZTRM 4)08."45&345"564

]'JMF]1PTJUJPO]#JOMPH@%P@%#]#JOMPH@*HOPSF@%#
]&YFDVUFE@(UJE@4FU]

]TFSWFS@#CJO]]]
]]

SPXJOTFU
TFD
NZTRM 45"354-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)084-"7&45"564=(
SPX
4MBWF@*0@4UBUF
.BTUFS@1PSU
$POOFDU@3FUSZ
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@1PT
_
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
_
SPXJOTFU
TFD
7. On Server C: Stop the slave, execute the $)"/(&."45&350 command, and

point to server B. You have to use the positions that you have got from the
preceding step:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
[ 223 ]

NZTRM $)"/(&."45&350."45&3@)045 4FSWFS# ."45&3@64&3

CJOMPH@VTFS ."45&3@1"44803% CJOMPH@1!TT
."45&3@-0(@'*-& TFSWFS@#CJO ."45&3@-0(@104
TFD
8. On Server C: Start replication and verify the slave status:
NZTRM 45"354-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)084-"7&45"564=(
TFD
SPX
.BTUFS@)PTUYYYYYYYYYY
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFS@#CJO
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@1PT
3FMBZ@.BTUFS@-PH@'JMFTFSWFS@#CJO
4MBWF@*0@3VOOJOH:FT
_
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
_
-BTU@*0@&SSOP
-BTU@*0@&SSPS
-BTU@42-@&SSOP
-BTU@42-@&SSPS
_
SPXJOTFU
TFD
[ 224 ]

Switching the slave from chain replication to

master-slave
If you have a chain replication setup (such as server A --> sever B --> server C) and you wish
to make server C a direct slave of server A, you have to stop replication on server B, let
server C catch up with server B, and then find the coordinates of server A corresponding to
the position where server B stopped. Using those coordinates, you can execute a $)"/(&
."45&350 command on server C and make it a slave of server A.
How to do it...
1. On server B: Stop the slave and note down the master status:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)08."45&345"564

]'JMF]1PTJUJPO]#JOMPH@%P@%#]#JOMPH@*HOPSF@%#
]&YFDVUFE@(UJE@4FU]

]TFSWFS@#CJO]]]
]]

SPXJOTFU
TFD
2. On server C: Make sure that the slave delay is caught up.

3FMBZ@.BTUFS@-PH@'JMF and &YFD@.BTUFS@-PH@1PT should be equal to the
output of the master status on server B. Once the delay is caught up, stop the
slave:
NZTRM 4)084-"7&45"564=(
SPX
.BTUFS@)PTU
.BTUFS@6TFSSFQM
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFS@#CJO
[ 225 ]

3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@'JMFVCVOUVSFMBZCJO
3FMBZ@-PH@1PT
Relay_Master_Log_File: server_B-bin.000003
4MBWF@*0@3VOOJOH:FT
3FQMJDBUF@%P@%#
3FQMJDBUF@*HOPSF@%#
3FQMJDBUF@%P@5BCMF
-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
_
SPXJOTFU
TFD
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
3. On server B: Get the coordinates of server A from the 4)084-"7&45"564

output (note down 3FMBZ@.BTUFS@-PH@'JMF and &YFD@.BTUFS@-PH@1PT) and
start the slave:
NZTRM 4)084-"7&45"564=(
SPX
4MBWF@*0@4UBUF
.BTUFS@6TFSSFQM
.BTUFS@1PSU
$POOFDU@3FUSZ
3FBE@.BTUFS@-PH@1PT
[ 226 ]

3FMBZ@-PH@1PT
4MBWF@*0@3VOOJOH/P
4MBWF@42-@3VOOJOH/P
3FQMJDBUF@%P@%#
3FQMJDBUF@*HOPSF@%#
3FQMJDBUF@%P@5BCMF
-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
4FDPOET@#FIJOE@.BTUFS/6--
NZTRM 45"354-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
4. On server C: Stop the slave and execute $)"/(&."45&350$0.."/% to point

to server A. Use the positions noted down from the preceding step (TFSWFS@"
CJO and ). Finally start the slave and verify the slave status:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $)"/(&."45&350."45&3@)045 4FSWFS" ."45&3@64&3

CJOMPH@VTFS ."45&3@1"44803% CJOMPH@1!TT
."45&3@-0(@'*-& TFSWFS@"CJO ."45&3@-0(@104
TFD
NZTRM 45"354-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)084-"7&45"564=(
SPX
4MBWF@*0@4UBUF
[ 227 ]

.BTUFS@1PSU
$POOFDU@3FUSZ
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@1PT
4MBWF@*0@3VOOJOH/P
4MBWF@42-@3VOOJOH/P
_
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
_
SPXJOTFU
TFD
Setting up delayed replication

Sometimes, you need a delayed slave for disaster recovery purpose. Suppose a disastrous
statement (such as a %301%"5"#"4& command) was executed on the master. You have to
use point-in-time recovery from backups to restore the database. It will lead to a huge
downtime depending on the size of the database. To avoid that situation, you can use a
delayed slave, which will be always delayed from the master by a configured amount of
time. If a disaster occurs and that statement is not applied by the delayed slave, you can
stop the slave and start until the disastrous statement, so that the disastrous statement won't
be executed. Then promote it to master.
The procedure is exactly the same as setting up normal replication, except that you specify
."45&3@%&-": in the $)"/(&."45&350 command.
[ 228 ]

How is the delay measured?
In versions earlier than MySQL 8.0, the delay is measured based on

the 4FDPOET@#FIJOE@.BTUFS value. In MySQL 8.0, it is measured based
on PSJHJOBM@DPNNJU@UJNFTUBNQ and JNNFEJBUF@DPNNJU@UJNFTUBNQ, which are
written to the binary log.
PSJHJOBM@DPNNJU@UJNFTUBNQ is the number of microseconds since the epoch when the

transaction was written (committed) to the binary log of the original master.
JNNFEJBUF@DPNNJU@UJNFTUBNQ is the number of microseconds since the epoch when the

transaction was written (committed) to the binary log of the immediate master.
How to do it...
1. Stop the slave:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
2. Execute $)"/(&."45&350."45&3@%&-": and start the slave. Suppose you

want a 1-hour delay, you can set ."45&3@%&-": to seconds:
NZTRM $)"/(&."45&350."45&3@%&-":
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 45"354-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
3. Check for the following in the slave status:

42-@%FMBZ: Number of seconds by which the slave must lag the master.
42-@3FNBJOJOH@%FMBZ: Number of seconds left of the delay. This is NULL
when there is delay is maintained.
4MBWF@42-@3VOOJOH@4UBUF: The state of the SQL thread.
NZTRM 4)084-"7&45"564=(
SPX
.BTUFS@)PTU
.BTUFS@6TFSSFQM
.BTUFS@1PSU
$POOFDU@3FUSZ
[ 229 ]

3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@1PT
4MBWF@*0@3VOOJOH:FT
_
-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
_
-BTU@*0@&SSOP
-BTU@*0@&SSPS
-BTU@42-@&SSOP
-BTU@42-@&SSPS
_
SQL_Delay: 3600
SQL_Remaining_Delay: 3549
Slave_SQL_Running_State: Waiting until MASTER_DELAY seconds
after master executed event
.BTUFS@3FUSZ@$PVOU
.BTUFS@#JOE
_
SPXJOTFU
TFD
Note that 4FDPOET@#FIJOE@.BTUFS will be shown as once the delay is maintained.
Setting up GTID replication

A global transaction identifier (GTID) is a unique identifier created and associated with
each transaction committed on the server of origin (master). This identifier is unique, not
only to the server on which it originated, but also across all servers in a given replication
setup. There is a one-to-one mapping between all transactions and all GTIDs.
[ 230 ]

A GTID is represented as a pair of coordinates, separated by a colon character ():

(5*%TPVSDF@JEUSBOTBDUJPO@JE
The TPVSDF@JE option identifies the originating server. Normally, the server's
TFSWFS@VVJE option is used for this purpose. The USBOTBDUJPO@JE option is a sequence
number determined by the order in which the transaction was committed on this server. For
example, the first transaction to be committed has as its USBOTBDUJPO@JE, and the tenth
transaction to be committed on the same originating server is assigned a USBOTBDUJPO@JE
of .
As you have seen in previous methods, you have to mention the binary log file and position
as the starting point for replication. If you are switching a slave from one master to another,
especially during a failover, you have to get the positions from the new master to sync the
slave, which can be painful. To avoid these, you can use GTID-based replication, where
MySQL automatically detects binary log positions using GTIDs.
How to do it...
If the replication is already set up between the servers, follow these steps:
1. Enable GTIDs in NZDOG:
shell> sudo vi /etc/my.cnf

<NZTRME>
HUJE@NPEF0/
FOGPSDFHUJEDPOTJTUFODZUSVF
TLJQ@TMBWF@TUBSU
2. Set the master as read-only and make sure that all the slaves catch up with the
master. This is very important because there should not be any data inconsistency
between master and slaves:
On master
NZTRM 4&5!!HMPCBMSFBE@POMZ0/
On Slaves (if replication is already setup)

NZTRM 4)084-"7&45"564=(
[ 231 ]

3. Restart all the slaves to put GTID into effect. Since the TLJQ@TMBWF@TUBSU is
given in the configuration file, the slave won't start until you specify the 45"35
4-"7& command. If you start the slave, it will fail with this errorg5IF
SFQMJDBUJPOSFDFJWFSUISFBEDBOOPUTUBSUCFDBVTFUIFNBTUFSIBT
(5*%@.0%&0''BOEUIJTTFSWFSIBT(5*%@.0%&0/:
4. Restart the master. When you restart the master, it begins in read-write mode and
starts accepting writes in GTID mode:
5. Execute the $)"/(&."45&350 command to set up GTID replication:
NZTRM $)"/(&."45&350."45&3@)045NBTUFS@IPTU ."45&3@1035

QPSU ."45&3@64&3 CJOMPH@VTFS ."45&3@1"44803%
CJOMPH@1!TT ."45&3@"650@104*5*0/
You can observe that the binary log file and positions are not given; instead,
."45&3@"650@104*5*0/ is given, which automatically finds the GTIDs executed.
6. Execute 45"354-"7& on all slaves:
NZTRM 45"354-"7&
7. Verify that the slave is replicating:
NZTRM 4)084-"7&45"564=(
SPX
.BTUFS@1PSU
$POOFDU@3FUSZ
.BTUFS@-PH@'JMFTFSWFSCJO
3FBE@.BTUFS@-PH@1PT
3FMBZ@-PH@1PT
3FMBZ@.BTUFS@-PH@'JMFTFSWFSCJO
4MBWF@*0@3VOOJOH:FT
3FQMJDBUF@%P@%#
3FQMJDBUF@*HOPSF@%#
3FQMJDBUF@%P@5BCMF
[ 232 ]

-BTU@&SSOP
-BTU@&SSPS
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
.BTUFS@44-@$"@'JMF
.BTUFS@44-@$"@1BUI
.BTUFS@44-@$FSU
.BTUFS@44-@$JQIFS
.BTUFS@44-@,FZ
-BTU@*0@&SSOP
-BTU@*0@&SSPS
-BTU@42-@&SSOP
-BTU@42-@&SSPS
.BTUFS@4FSWFS@*E
.BTUFS@66*%CFGBGGFB
.BTUFS@*OGP@'JMFWBSMJCNZTRMNBTUFSJOGP
42-@%FMBZ
.BTUFS@3FUSZ@$PVOU
.BTUFS@#JOE
.BTUFS@44-@$SM
.BTUFS@44-@$SMQBUI
3FUSJFWFE@(UJE@4FU
CFGBGGFB
&YFDVUFE@(UJE@4FU
CFGBGGFB
Auto_Position: 1
$IBOOFM@/BNF
.BTUFS@5-4@7FSTJPO
SPXJOTFU
TFD
[ 233 ]

To know more about GTID, refer to IUUQTEFWNZTRMDPNEPDSFGNBO

FOSFQMJDBUJPOHUJETDPODFQUTIUNM.
Setting up semi-synchronous replication

Replication is asynchronous by default. The master is not aware of whether the writes have
reached the slaves or not. If there is a delay between master and slave, and if the master
crashes, you will lose the data that has not reached the slave. To overcome this situation,
you can use semi-synchronous replication.
In semi-synchronous replication, the master waits until at least one slave has received the
writes. By default, the value of SQM@TFNJ@TZOD@NBTUFS@XBJU@QPJOU is "'5&3@4:/$; this
means that the master syncs the transaction to the binary log, which is consumed by the
slave.
After that, the slave sends an acknowledgement to the master, then the master commits the
transaction and returns the result to the client. So, it is enough if the writes have reached the
relay log; the slave need not commit the transaction. You can change this behavior by
changing the variable SQM@TFNJ@TZOD@NBTUFS@XBJU@QPJOU to "'5&3@$0..*5. In this,
the master commits the transaction to the storage engine but does not return the result to
the client. Once the transaction is committed on the slave, the master receives an
acknowledgment of transaction and then returns a result to the client.
If you want the transaction to be acknowledged on more slaves, you can increase the value
of the dynamic variable SQM@TFNJ@TZOD@NBTUFS@XBJU@GPS@TMBWF@DPVOU. You can also
set how many milliseconds the master has to wait for to get the acknowledgement from the
slave through the dynamic variable SQM@TFNJ@TZOD@NBTUFS@UJNFPVU; the default is
seconds.
In fully synchronous replication, the master waits until all the slaves have committed the
transaction. To implement this, you have to use Galera Cluster.
[ 234 ]

How to do it...
On a high level, you need to install and enable semi-synchronous plugins on both the
master and all slaves where you want semi-synchronous replication. You have to restart the
slave IO thread to bring the changes into effect. You can adjust the value of
SQM@TFNJ@TZOD@NBTUFS@UJNFPVU according to your network and application. A value of
second is a good start:
1. On the master, install the SQM@TFNJ@TZOD@NBTUFS plugin:
NZTRM */45"--1-6(*/SQM@TFNJ@TZOD@NBTUFS40/".&
TFNJTZOD@NBTUFSTP
2VFSZ0,SPXTBGGFDUFE
TFD
Verify that the plugin is activated:

NZTRM 4&-&$51-6(*/@/".&1-6(*/@45"564'30.
*/'03."5*0/@4$)&."1-6(*/48)&3&1-6(*/@/".&-*,& TFNJ

]1-6(*/@/".&]1-6(*/@45"564]

]SQM@TFNJ@TZOD@NBTUFS]"$5*7&]

SPXJOTFU
TFD
2. On the master, enable semi-synchronous replication and adjust the timeout (say 1
second):
NZTRM 4&5!!(-0#"-SQM@TFNJ@TZOD@NBTUFS@FOBCMFE
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)087"3*"#-&4-*,& SQM@TFNJ@TZOD@NBTUFS@FOBCMFE


]SQM@TFNJ@TZOD@NBTUFS@FOBCMFE]0/]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"-SQM@TFNJ@TZOD@NBTUFS@UJNFPVU
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)087"3*"#-&4-*,& SQM@TFNJ@TZOD@NBTUFS@UJNFPVU


[ 235 ]

]SQM@TFNJ@TZOD@NBTUFS@UJNFPVU]]

SPXJOTFU
TFD
3. On the slave, install the SQM@TFNJ@TZOD@TMBWF plugin:
NZTRM */45"--1-6(*/SQM@TFNJ@TZOD@TMBWF40/".&
TFNJTZOD@TMBWFTP
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$51-6(*/@/".&1-6(*/@45"564'30.
*/'03."5*0/@4$)&."1-6(*/48)&3&1-6(*/@/".&-*,& TFNJ

]1-6(*/@/".&]1-6(*/@45"564]

]SQM@TFNJ@TZOD@TMBWF]"$5*7&]

SPXJOTFU
TFD
4. On the slave, enable semi-synchronous replication and restart the slave IO thread:
NZTRM 4&5(-0#"-SQM@TFNJ@TZOD@TMBWF@FOBCMFE
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 45014-"7&*0@5)3&"%
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 45"354-"7&*0@5)3&"%
2VFSZ0,SPXTBGGFDUFE
TFD
5. You can monitor the status of the semi-synchronous replication through the
following:
To find the number of clients connected as semi-syncgon master, execute:
NZTRM 4)0845"564-*,& 3QM@TFNJ@TZOD@NBTUFS@DMJFOUT


]3QM@TFNJ@TZOD@NBTUFS@DMJFOUT]]

SPXJOTFU
TFD
[ 236 ]

The master switches between asynchronous and semi-synchronous replication

when the timeout occurs and the slaves catch up. To check what type of
replication the master is using, check the status of
3QM@TFNJ@TZOD@NBTUFS@TUBUVT (on means semi-sync and off means async):
NZTRM 4)0845"564-*,& 3QM@TFNJ@TZOD@NBTUFS@TUBUVT


]3QM@TFNJ@TZOD@NBTUFS@TUBUVT]0/]

SPXJOTFU
TFD
You can verify the semi-synchronous replication using this method:
1. Stop the slave:
NZTRM 45014-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
2. On the master, execute any statement:
NZTRM 64&FNQMPZFFT
%BUBCBTFDIBOHFE
NZTRM %3015"#-&*'&9*454FNQMPZFFT@UFTU
TFD
You will notice that the master has switched to asynchronous replication since it
did not get any acknowledgement from the slave even after 1 second (the value of
SQM@TFNJ@TZOD@NBTUFS@UJNFPVU):



SPXJOTFU
TFD
NZTRM %3015"#-&*'&9*454FNQMPZFFT@UFTU
2VFSZ0,SPXTBGGFDUFE
TFD
[ 237 ]



]3QM@TFNJ@TZOD@NBTUFS@TUBUVT]0'']

SPXJOTFU
TFD
3. Start the slave:
NZTRM 45"354-"7&
2VFSZ0,SPXTBGGFDUFE
TFD
4. On the master, you will notice that the master switched back to semi-
synchronous replication:



SPXJOTFU
TFD
[ 238 ]

Table Maintenance
10
Installing Percona Toolkit

Altering tables
Moving tables across databases
Altering tables using an online schema change tool
Archiving tables
Cloning tables
Partitioning tables
Partition pruning and selection
Partition management
Partition information
Efficiently managing time to live and soft delete rows
Introduction
One of the key aspects in maintaining a database is managing tables. Often, you need to
alter a big table or clone a table. In this chapter, you will learn about managing big tables.
Some open source third-party tools are used as MySQL does not support certain operations.
The installation and usage of third-party tools are also covered in this chapter.

Table Maintenance Chapter 10
Installing Percona Toolkit

Percona Toolkit is a collection of advanced open source command-line tools, developed and
used by Percona to perform a variety of tasks that are too difficult or complex to perform
manually. The installation is covered in this section. In the later sections, you will learn how
to use it.
How to do it...
Let us see how to install Percona Toolkit on various operating systems.
On Debian/Ubuntu
1. Download the repository package:
TIFMM XHFUIUUQTSFQPQFSDPOBDPNBQUQFSDPOB
SFMFBTF@
MTC@SFMFBTFTD@BMMEFC
2. Install the repository package:
TIFMM TVEPEQLHJQFSDPOBSFMFBTF@
MTC@SFMFBTF
TD@BMMEFC
3. Update local package list:
4. Make sure that Percona packages are available:
TIFMM BQUDBDIFTFBSDIQFSDPOB
[ 240 ]

You should see output similar to the following:

QFSDPOBYUSBCBDLVQECH%FCVHTZNCPMTGPS1FSDPOB9USB#BDLVQ
QFSDPOBYUSBCBDLVQUFTU5FTUTVJUFGPS1FSDPOB9USB#BDLVQ
QFSDPOBYUSBECDMVTUFSDMJFOU1FSDPOB9USB%#$MVTUFSEBUBCBTF
DMJFOU
QFSDPOBYUSBECDMVTUFSTFSWFS1FSDPOB9USB%#$MVTUFSEBUBCBTF
TFSWFS
QFSDPOBYUSBECDMVTUFSUFTUTVJUF1FSDPOB9USB%#$MVTUFSEBUBCBTF
SFHSFTTJPOUFTUTVJUF
QFSDPOBYUSBECDMVTUFSUFTUTVJUF1FSDPOB4FSWFSEBUBCBTFUFTU
TVJUF

5. Install the QFSDPOBUPPMLJU package:
TIFMM TVEPBQUHFUJOTUBMMQFSDPOBUPPMLJU
If you do not want to install a repository, you can also install directly:
TIFMM XHFU
IUUQTXXXQFSDPOBDPNEPXOMPBETQFSDPOBUPPMLJUCJOBSZEFCJBOYFOJB
MY@QFSDPOBUPPMLJU@YFOJBM@BNEEFC
TIFMM TVEPEQLHJQFSDPOBUPPMLJU@ZBLLFUZ@BNEEFC
On CentOS/Red Hat/Fedora
1. Install the repository package:
TIFMM TVEPZVNJOTUBMMIUUQXXXQFSDPOBDPNEPXOMPBETQFSDPOB
SFMFBTFSFEIBUQFSDPOBSFMFBTFOPBSDISQN
You should see the following if successful:

*OTUBMMFE
QFSDPOBSFMFBTFOPBSDI
$PNQMFUF
[ 241 ]

2. Make sure that Percona packages are available:
TIFMM TVEPZVNMJTU]HSFQQFSDPOB
You should see output similar to the following:

QFSDPOBSFMFBTFOPBSDI
!QFSDPOBSFMFBTFOPBSDI
1FSDPOB4FSWFSEFCVHJOGPY@SFMFM
QFSDPOBSFMFBTFY@
1FSDPOB4FSWFSEFCVHJOGPY@SFMFM
QFSDPOBSFMFBTFY@
1FSDPOB4FSWFSEFCVHJOGPY@FM
QFSDPOBSFMFBTFY@

3. Install Percona Toolkit:
TIFMM TVEPZVNJOTUBMMQFSDPOBUPPMLJU
If you do not want to install a repository, you can directly install using YUM:
TIFMM TVEPZVNJOTUBMM
IUUQTXXXQFSDPOBDPNEPXOMPBETQFSDPOBUPPMLJUCJOBSZSFEIBUY
@QFSDPOBUPPMLJUFMY@SQN
Altering tables
"-5&35"#-& changes the structure of a table. For example, you can add or delete columns,
create or destroy indexes, change the type of existing columns, or rename columns or the
table itself.
While performing certain alter operations such as changing a column data type, adding a
41"5*"-*/%&9, dropping a primary key, converting a character set, adding/removing
encryption, and so on, DML operations on the table are blocked. If the table is big, it will
take even more time to alter, and the application cannot access the table during that time,
which is not desired. In those situations, a QUPOMJOFTDIFNB change is helpful, where
DML statements are allowed.
There are two algorithms for alter operations:
In-place (default): Does not require copying whole table data

Copy: Copies the data into a temporary disk file and renames it
[ 242 ]

Only certain alter operations can be done in-place. The performance of an online DDL
operation is largely determined by whether the operation is performed in-place, or requires
copying and rebuilding the entire table. Refer to IUUQTEFWNZTRMDPNEPDSFGNBO
FOJOOPECDSFBUFJOEFYPWFSWJFXIUNMJOOPECPOMJOFEEMTVNNBSZHSJE to see what
kinds of operations can be performed in-place, and any requirements for avoiding table-
copy operations.
How copy algorithm works (taken from the reference manualgIUUQTEFWNZTRMDPNEPD

SFGNBOFOBMUFSUBCMFIUNM)
"-5&35"#-& operations that are not performed in-place make a temporary copy of the
original table. MySQL waits for other operations that are modifying the table, then
proceeds. It incorporates the alteration into the copy, deletes the original table, and renames
the new one. While "-5&35"#-& is executing, the original table is readable by other
sessions. Updates and writes to the table that begin after the "-5&35"#-& operation begins
are stalled until the new table is ready, then are automatically redirected to the new table
without any failed updates. The temporary copy of the original table is created in the
database directory of the new table. This can differ from the database directory of the
original table for "-5&35"#-& operations that rename the table to a different database.
To get an idea on whether a DDL operation performs in-place or a table copy, look at the
SPXTBGGFDUFE value displayed after the command finishes:
Changing the default value of a column (superfast, does not affect the table data
at all), the output would be some thing like this:
2VFSZ0,SPXTBGGFDUFE
TFD
Adding an index (takes time, but SPXTBGGFDUFE shows that the table is not
copied), the output would be some thing like this:
2VFSZ0,SPXTBGGFDUFE
TFD
Changing the data type of a column (takes substantial time and does require
rebuilding all the rows of the table), , the output would be some thing like this:
2VFSZ0,1671168 rows affected

NJOTFD
Changing the data type of a column requires rebuilding all the rows of the table with the
exception of changing the 7"3$)"3 size, which may be performed using online "-5&3
5"#-&. See the example mentioned in the Altering tables using an online schema change tool
section, which shows how to modify column properties using QUPOMJOFTDIFNB.
[ 243 ]

How to do it...
If you want to add a new column to the FNQMPZFFT table, you can execute the "%%$0-6./
statement:
NZTRM "-5&35"#-&FNQMPZFFT"%%$0-6./BEESFTTWBSDIBS

2VFSZ0,SPXTBGGFDUFE
TFD
You can see that the number of rows affected is , which means that the table is not copied
and the operation is done in-place.
If you want to increase the length of the WBSDIBS column, you can execute the .0%*':
$0-6./ statement:
NZTRM "-5&35"#-&FNQMPZFFT.0%*':$0-6./BEESFTT7"3$)"3

2VFSZ0,SPXTBGGFDUFE
TFD
If you think that WBSDIBS

is not sufficient to store addresses, and you would like to
change it to UJOZUFYU, you can use the .0%*':$0-6./ statement. However, in this case,
since you are modifying the data type of a column, all the rows of the existing table should
be modified, which requires table copy, and DMLs are blocked:
NZTRM "-5&35"#-&FNQMPZFFT.0%*':$0-6./BEESFTTUJOZUFYU
2VFSZ0,SPXTBGGFDUFE
TFD
You will notice that the rows affected are , which is the size of the table.
There are various other operations that you can do, such as renaming a column, changing
the default value, reordering the column positions, and so on; refer to the manual
at IUUQTEFWNZTRMDPNEPDSFGNBOFOJOOPECDSFBUFJOEFYPWFSWJFXIUNM for
more details.
Adding a virtual generated column is just a metadata change and is almost instantaneous:
NZTRM "-5&35"#-&FNQMPZFFT"%%$0-6./GVMM@OBNF7"3$)"3
"4

$0/$"5
GJSTU@OBNF MBTU@OBNF
2VFSZ0,SPXTBGGFDUFE
TFD
[ 244 ]

However, adding a 4503&%(&/&3"5&% column and modifying the 7*356"-(&/&3"5&%

column is not online:
NZTRM "-5&35"#-&FNQMPZFFT.0%*':$0-6./GVMM@OBNF7"3$)"3
"4

$0/$"5
GJSTU@OBNF MBTU@OBNF7*356"-
2VFSZ0,SPXTBGGFDUFE
TFD
Moving tables across databases

You can rename a table by executing the 3&/".&5"#-& statement.
For the following illustrations to work, create sample tables and databases
NZTRM $3&"5&%"5"#"4&QSPE
NZTRM $3&"5&5"#-&QSPEBVEJU@MPH
JEJOU/05/6--NTHWBSDIBS

NZTRM $3&"5&%"5"#"4&BSDIJWF
How to do it...
For example, if you want to rename the BVEJU@MPH table BVEJU@MPH@BSDIJWF@, you
can execute the following:
NZTRM 64&QSPE
%BUBCBTFDIBOHFE
NZTRM 3&/".&5"#-&BVEJU@MPH50BVEJU@MPH@BSDIJWF@
2VFSZ0,SPXTBGGFDUFE
TFD
If you want to move the table from one database to an other, you can use dot notation to
specify the database name. For example, if you want to move the BVEJU@MPH table from the
database named QSPE to the database named BSDIJWF, execute the following:
NZTRM 64&QSPE
3FBEJOHUBCMFJOGPSNBUJPOGPSDPNQMFUJPOPGUBCMFBOEDPMVNOOBNFT
:PVDBOUVSOPGGUIJTGFBUVSFUPHFUBRVJDLFSTUBSUVQXJUI"
NZTRM 4)085"#-&4

]5BCMFT@JO@QSPE]

]BVEJU@MPH@BSDIJWF@]

[ 245 ]

SPXJOTFU
TFD
mysql> RENAME TABLE audit_log_archive_2018TO archive.audit_log;

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)085"#-&4
&NQUZTFU
TFD
NZTRM 64&BSDIJWF
3FBEJOHUBCMFJOGPSNBUJPOGPSDPNQMFUJPOPGUBCMFBOEDPMVNOOBNFT
:PVDBOUVSOPGGUIJTGFBUVSFUPHFUBRVJDLFSTUBSUVQXJUI"
%BUBCBTFDIBOHFE
NZTRM 4)085"#-&4

]5BCMFT@JO@BSDIJWF]

]BVEJU@MPH]

SPXJOTFU
TFD
Altering tables using an online schema

change tool
In this section, you will learn about Percona's QUPOMJOFTDIFNBDIBOHF (QUPTD) tool,
which is used to perform "-5&35"#-& operations without blocking DMLs.
QUPTD comes along with Percona Toolkit. Installation of Percona Toolkit has already been
described earlier in the chapter.
How it works...
(Taken from IUUQTXXXQFSDPOBDPNEPDQFSDPOBUPPMLJU-"5&45QUPOMJOFTDIFNB
DIBOHFIUNM.)
QUPOMJOFTDIFNBDIBOHF works by creating an empty copy of the table to alter,

modifying it as desired, and then copying rows from the original table into the new table.
When the copy is complete, it moves away the original table and replaces it with the new
one. By default, it also drops the original table.
[ 246 ]

The data copy process is performed in small chunks of data, which are varied to attempt to
make them execute in a specific amount of time. Any modifications to data in the original
tables during the copy will be reflected in the new table, because the tool creates triggers on
the original table to update the corresponding rows in the new table. The use of triggers
means that the tool will not work if any triggers are already defined on the table.
When the tool finishes copying data into the new table, it uses an atomic 3&/".&5"#-&
operation to simultaneously rename the original and new tables. After this is complete, the
tool drops the original table.
Foreign keys complicate the tool's operation and introduce additional risk. The technique of
atomically renaming the original and new tables does not work when foreign keys refer to
the table. The tool must update foreign keys to refer to the new table after the schema
change is complete. The tool supports two methods for accomplishing this. You can read
more about this in the documentation for BMUFSGPSFJHOLFZTNFUIPE.
How to do it...
Modifying the column data type can be done as follows:
shell> pt-online-schema-change D=employees,t=employees,h=localhost -u root
--ask-pass --alter="MODIFY COLUMN address VARCHAR(100)" --alter-foreign-
keys-method=auto --execute
&OUFS.Z42-QBTTXPSE
/PTMBWFTGPVOE4FFSFDVSTJPONFUIPEJGIPTUTFSWFSIBTTMBWFT
/PUDIFDLJOHTMBWFMBHCFDBVTFOPTMBWFTXFSFGPVOEBOEDIFDLTMBWFMBH
XBTOPUTQFDJGJFE
0QFSBUJPOUSJFTXBJU
BOBMZ[F@UBCMF
DPQZ@SPXT
DSFBUF@USJHHFST
ESPQ@USJHHFST
TXBQ@UBCMFT
VQEBUF@GPSFJHO@LFZT
$IJMEUBCMFT
AFNQMPZFFTAAEFQU@FNQA
BQQSPYSPXT
AFNQMPZFFTAAUJUMFTA
BQQSPYSPXT
AFNQMPZFFTAATBMBSJFTA
BQQSPYSPXT
AFNQMPZFFTAAEFQU@NBOBHFSA
BQQSPYSPXT
8JMMBVUPNBUJDBMMZDIPPTFUIFNFUIPEUPVQEBUFGPSFJHOLFZT
"MUFSJOHAFNQMPZFFTAAFNQMPZFFTA
$SFBUJOHOFXUBCMF
$SFBUFEOFXUBCMFFNQMPZFFT@FNQMPZFFT@OFX0,
"MUFSJOHOFXUBCMF
[ 247 ]

"MUFSFEAFNQMPZFFTAA@FNQMPZFFT@OFXA0,
5$SFBUJOHUSJHHFST
5$SFBUFEUSJHHFST0,
5$PQZJOHBQQSPYJNBUFMZSPXT
5$PQJFESPXT0,
5.BYSPXTGPSUIFSFCVJME@DPOTUSBJOUTNFUIPE
%FUFSNJOJOHUIFNFUIPEUPVQEBUFGPSFJHOLFZT
5AFNQMPZFFTAAEFQU@FNQAUPPNBOZSPXTNVTU
VTFESPQ@TXBQ
5%SPQTXBQQJOHUBCMFT
5"OBMZ[JOHOFXUBCMF
5%SPQQFEBOETXBQQFEUBCMFT0,
/PUESPQQJOHPMEUBCMFCFDBVTFOPESPQPMEUBCMFXBTTQFDJGJFE
5%SPQQJOHUSJHHFST
5%SPQQFEUSJHHFST0,
Successfully altered èmployees`.èmployees`.
You will have noticed that the tool has created a new table with a modified structure,
created triggers on the table, copied the rows to a new table, and finally, renamed the new
table.
If you want to alter the TBMBSJFT table, which already has triggers, you need to specify
the QSFTFSWFSUSJHHFST option, otherwise you will end up with the error: 5IFUBCMF
AFNQMPZFFTAATBMBSJFTAIBTUSJHHFSTCVUQSFTFSWFUSJHHFSTXBTOPU
TQFDJGJFE:
TIFMM QUPOMJOFTDIFNBDIBOHF%FNQMPZFFTUTBMBSJFTIMPDBMIPTUVVTFS
BTLQBTTBMUFS.0%*':$0-6./TBMBSZJOUBMUFSGPSFJHOLFZT
NFUIPEBVUPFYFDVUFOPESPQPMEUBCMFQSFTFSWFUSJHHFST
/PTMBWFTGPVOE4FFSFDVSTJPONFUIPEJGIPTUTFSWFSIBTTMBWFT
/PUDIFDLJOHTMBWFMBHCFDBVTFOPTMBWFTXFSFGPVOEBOEDIFDLTMBWFMBH
XBTOPUTQFDJGJFE
0QFSBUJPOUSJFTXBJU
BOBMZ[F@UBCMF
DPQZ@SPXT
DSFBUF@USJHHFST
ESPQ@USJHHFST
TXBQ@UBCMFT
VQEBUF@GPSFJHO@LFZT
/PGPSFJHOLFZTSFGFSFODFAFNQMPZFFTAATBMBSJFTAJHOPSJOHBMUFSGPSFJHO
LFZTNFUIPE
"MUFSJOHAFNQMPZFFTAATBMBSJFTA
$SFBUJOHOFXUBCMF
$SFBUFEOFXUBCMFFNQMPZFFT@TBMBSJFT@OFX0,
"MUFSJOHOFXUBCMF
"MUFSFEAFNQMPZFFTAA@TBMBSJFT@OFXA0,
[ 248 ]

5$SFBUJOHUSJHHFST
5$SFBUFEUSJHHFST0,
5$PQJFESPXT0,
5"EEJOHPSJHJOBMUSJHHFSTUPOFXUBCMF
5"OBMZ[JOHOFXUBCMF
54XBQQJOHUBCMFT
54XBQQFEPSJHJOBMBOEOFXUBCMFT0,
5%SPQQJOHUSJHHFST
5%SPQQFEUSJHHFST0,
4VDDFTTGVMMZBMUFSFEAFNQMPZFFTAATBMBSJFTA
If the server has slaves, this tool can create a slave lag while copying from an existing table
to a new table. To avoid this, you can specify DIFDLTMBWFMBH (enabled by default); it
pauses the data copy until this replica's lag is less than NBYMBH, which is 1 second by
default. You can specify NBYMBH by passing the NBYMBH option.
If you want to make sure that the slaves will not be lagged by more than 10 seconds, pass
NBYMBH:
TIFMM QUPOMJOFTDIFNBDIBOHF%FNQMPZFFTUFNQMPZFFTIMPDBMIPTUVVTFS
BTLQBTTBMUFS.0%*':$0-6./BEESFTT7"3$)"3
BMUFSGPSFJHO
LFZTNFUIPEBVUPFYFDVUFQSFTFSWFUSJHHFSTNBYMBH
&OUFS.Z42-QBTTXPSE
'PVOETMBWFT
TFSWFS YYYYYYYYYYTPDLFU
8JMMDIFDLTMBWFMBHPO
TFSWFS YYYYYYYYYYTPDLFU
0QFSBUJPOUSJFTXBJU
BOBMZ[F@UBCMF
DPQZ@SPXT
DSFBUF@USJHHFST
ESPQ@USJHHFST
TXBQ@UBCMFT
VQEBUF@GPSFJHO@LFZT
$IJMEUBCMFT
AFNQMPZFFTAAEFQU@FNQA
BQQSPYSPXT
AFNQMPZFFTAAUJUMFTA
BQQSPYSPXT
AFNQMPZFFTAATBMBSJFTA
BQQSPYSPXT
AFNQMPZFFTAAEFQU@NBOBHFSA
BQQSPYSPXT
8JMMBVUPNBUJDBMMZDIPPTFUIFNFUIPEUPVQEBUFGPSFJHOLFZT
"MUFSJOHAFNQMPZFFTAAFNQMPZFFTA
$SFBUJOHOFXUBCMF
$SFBUFEOFXUBCMFFNQMPZFFT@FNQMPZFFT@OFX0,
8BJUJOHGPSFWFSGPSOFXUBCMFAFNQMPZFFTAA@FNQMPZFFT@OFXAUPSFQMJDBUFUP
VCVOUV
[ 249 ]

"MUFSJOHOFXUBCMF
"MUFSFEAFNQMPZFFTAA@FNQMPZFFT@OFXA0,
5$SFBUJOHUSJHHFST
5$SFBUFEUSJHHFST0,
5$PQJFESPXT0,
5.BYSPXTGPSUIFSFCVJME@DPOTUSBJOUTNFUIPE
%FUFSNJOJOHUIFNFUIPEUPVQEBUFGPSFJHOLFZT
5AFNQMPZFFTAAEFQU@FNQAUPPNBOZSPXTNVTU
VTFESPQ@TXBQ
54LJQQJOHUSJHHFSTDSFBUJPOTJODFOPTXBQUBCMFTXBT
TQFDJGJFEBMPOHXJUIESPQOFXUBCMF
5%SPQTXBQQJOHUBCMFT
5"OBMZ[JOHOFXUBCMF
5%SPQQFEBOETXBQQFEUBCMFT0,
5%SPQQJOHUSJHHFST
5%SPQQFEUSJHHFST0,
4VDDFTTGVMMZBMUFSFEAFNQMPZFFTAAFNQMPZFFTA
For more details and options, refer to the Percona documentation, atIUUQTXXX
QFSDPOBDPNEPDQFSDPOBUPPMLJU-"5&45QUPOMJOFTDIFNBDIBOHFIUNM
QUPOMJOFTDIFNBDIBOHF works only when there is a primary key or

unique key, otherwise it will fail with the following error:
5IFOFXUBCMFAFNQMPZFFTAA@FNQMPZFFT@OFXAEPFTOPUIBWFB
13*."3:,&:PSBVOJRVFJOEFYXIJDIJTSFRVJSFEGPSUIF
%&-&5&USJHHFS
So, if the table does not have any unique key, you cannot use QUPOMJOFTDIFNBDIBOHF.
Archiving tables
Sometimes, you do not want to keep older data and wish to delete it. If you want to delete
all the rows which were last accessed over a month ago, if the table is small (<10k rows),
you can straight away use the following:
%&-&5&'30.5"#-& 8)&3&MBTU@BDDFTTFE%"5&@"%%
/08
*/5&37"-.0/5)
[ 250 ]

What happens if the table is big? You know *OOP%# creates an 6/%0 log to restore failed
transactions. So all the deleted rows are saved in the 6/%0 log space to be used to restore in
case the %&-&5& statement aborts in between. Unfortunately, if the %&-&5& statement is
aborted in between, *OOP%# copies the rows from the 6/%0 log space to table, which can
make the table inaccessible.
To overcome this behavior, you can -*.*5 the number of rows deleted and $0..*5 the
transaction, running the same thing in a loop until all the unwanted rows are deleted.
This is an example pseudo code:

8)*-&DPVOU
%&-&5&'30.5"#-& 8)&3&MBTU@BDDFTTFE%"5&@"%%
/08
*/5&37"-
.0/5)-*.*5
DPVOU4&-&$5$06/5
'30.5"#-& 8)&3&MBTU@BDDFTTFE%"5&@"%%
/08

*/5&37"-.0/5)
If there is no */%&9 on MBTU@BDDFTTFE, it can lock the table. In that case, you need to find
out the primary key for the deleted rows and delete based on 13*."3:,&:.
This is the pseudo code, assuming JE is the 13*."3:,&::

8)*-&DPVOU
4&-&$5JE'30.5"#-& 8)&3&MBTU@BDDFTTFE%"5&@"%%
/08
*/5&37"-
.0/5)-*.*5
%&-&5&'30.5"#-& 8)&3&JE*/
JETGSPNBCPWFTUBUFNFOU
DPVOU4&-&$5$06/5
'30.5"#-& 8)&3&
MBTU@BDDFTTFE%"5&@"%%
/08
*/5&37"-.0/5)
Instead of writing the code for deleting the rows, you can use Percona's QUBSDIJWFS tool,
which essentially does the same and provides many other options, such as saving the rows
into another table or file, fine control over the load and replication delay, and so on.
How to do it...
There are many options in QUBSDIJWFS, we will start with simple purging.
[ 251 ]

Purging data
If you want to delete all the rows from the FNQMPZFFT table for which IJSF@EBUF is older
than 30 years, you can execute the following:
TIFMM QUBSDIJWFSTPVSDFIMPDBMIPTU%FNQMPZFFTUFNQMPZFFTVVTFS
QQBTT XIFSFIJSF@EBUF%"5&@"%%
/08
*/5&37"-:&"3OPDIFDL
DIBSTFUMJNJUDPNNJUFBDI
You can pass the hostname, database name, and table name through the TPVSDF option.
You can limit the number of rows to delete in a batch using the MJNJU option.
If you specify QSPHSFTT, the output is a header row, plus status output at intervals. Each
row in the status output lists the current date and time, how many seconds QUBSDIJWFS
has been running, and how many rows it has archived.
If you specify TUBUJTUJDT, QUBSDIJWFS outputs timing and other information to help
you identify which part of your archiving process takes the most time.
If you specify DIFDLTMBWFMBH, the tool will pause archiving until the slave lag is less
than NBYMBH.
Archiving data
If you want to save the rows after deletion into a separate table or file, you can specify the
EFTU option.
Suppose you want to move all the rows of the FNQMPZFFT table from the FNQMPZFFT
database to the FNQMPZFFT@BSDIJWF table, you can execute the following:
TIFMM QUBSDIJWFSTPVSDFIMPDBMIPTU%FNQMPZFFTUFNQMPZFFTEFTU
IMPDBMIPTU%FNQMPZFFT@BSDIJWFVVTFS QQBTT XIFSFOP
DIFDLDIBSTFUMJNJUDPNNJUFBDI
If you specify XIFSF, it copies all rows.
Copying data
If you want to copy data from one table to another, you can either use NZTRMEVNQ or
NZTRMQVNQ to back up certain rows and then load them into the destination table. As an
alternative, you can also use QUBSDIJWF. If you specify the OPEFMFUF option, QU
BSDIJWFS will not delete the rows from the source:
[ 252 ]

TIFMM QUBSDIJWFSTPVSDFIMPDBMIPTU%FNQMPZFFTUFNQMPZFFTEFTU
IMPDBMIPTU%FNQMPZFFT@BSDIJWFVVTFS QQBTT XIFSFOP
DIFDLDIBSTFUMJNJUDPNNJUFBDIOPEFMFUF
See also
Refer to IUUQTXXXQFSDPOBDPNEPDQFSDPOBUPPMLJU-"5&45QUBSDIJWFSIUNM for
more details about and options for QUBSDIJWFS.
Cloning tables
If you want to clone a table, there are many options.
How to do it...
1. Use the */4&35*/504&-&$5 statement:
NZTRM $3&"5&5"#-&FNQMPZFFT@DMPOF-*,&FNQMPZFFT
NZTRM */4&35*/50FNQMPZFFT@DMPOF4&-&$5'30.FNQMPZFFT
Note that if there are any generated columns, the above statement would not
work. In that case, you should give full insert statement excluding the generated
columns.
NZTRM */4&35*/50FNQMPZFFT@DMPOF4&-&$5'30.FNQMPZFFT
&3303
):5IFWBMVFTQFDJGJFEGPSHFOFSBUFEDPMVNO
IJSF@EBUF@ZFBS JOUBCMF FNQMPZFFT@DMPOF JTOPUBMMPXFE
NZTRM */4&35*/50FNQMPZFFT@DMPOF
FNQ@OPCJSUI@EBUFGJSTU@OBNF
MBTU@OBNFHFOEFSIJSF@EBUF4&-&$5FNQ@OPCJSUI@EBUFGJSTU@OBNF
MBTU@OBNFHFOEFSIJSF@EBUF'30.FNQMPZFFT
2VFSZ0,SPXTBGGFDUFE
TFD
But the preceding statement is very slow and dangerous on big tables.
Remember, if the statement fails, to restore the table state, *OOP%# saves all
the rows in 6/%0 logs.
[ 253 ]

2. Use NZTRMEVNQ or NZTRMQVNQ and take a backup of a single table and restore it
on destination. This can take very long time if the table is big.
3. Use *OOPCBDLVQFY to take a backup of specific table and restore the data file
onto destination.
4. Use QUBSDIJWFS with the OPEFMFUF option, which will copy the desired
rows or all rows to the destination table.
You can also clone tables by using transportable tablespaces, which are explained in
the Copying file-per-table tablespaces to another instance section of $IBQUFS, Managing
Tablespace.
Partitioning tables
You can distribute portions of individual tables across a filesystem using partitions. The
user-selected rule by which the division of data is accomplished is known as a partitioning
function, which can be modulus, simple matching against a set of ranges or value lists, an
internal hashing function, or a linear hashing function.
Different rows of a table may be assigned to different physical partitions, which is called
horizontal partitioning. MySQL does not have support for vertical partitioning, in which
different columns of a table are assigned to different physical partitions.
There are many ways to partition a table:
3"/(&: This type of partitioning assigns rows to partitions based on column

values falling within a given range.
-*45: Similar to partitioning by 3"/(&, except that the partition is selected based
on columns matching one of a set of discrete values.
)"4): With this type of partitioning, a partition is selected based on the value
returned by a user-defined expression that operates on column values in rows to
be inserted into the table. The function may consist of any expression valid in
MySQL that yields a non-negative integer value.
,&:: This type of partitioning is similar to partitioning by )"4), except that only
one or more columns to be evaluated are supplied, and the MySQL server
provides its own hashing function. These columns can contain other than integer
values, since the hashing function supplied by MySQL guarantees an integer
result regardless of the column data type.
[ 254 ]

Each of the preceding partition types have an extension. 3"/(& has 3"/(&$0-6./4, -*45
has -*45$0-6./4, )"4) has -*/&"3)"4), and ,&: has -*/&"3,&:.
For <-*/&"3>,&:, 3"/(&$0-6./4, and -*45$0-6./4 partitioning, the partitioning

expression consists of a list of one or more columns.
In the case of 3"/(&, -*45, and <-*/&"3>)"4) partitioning, the value of the partitioning
column is passed to the partitioning function, which returns an integer value representing
the number of the partition in which that particular record should be stored. This function
must be non-constant and non-random.
A very common use of database partitioning is to segregate data by date.
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOQBSUJUJPOJOHPWFSWJFXIUNM for

advantages and other details on partitioning.
Note that partitions work only for *OOP%# tables, and foreign keys are not yet supported in
conjunction with partitioning.
How to do it...
You can specify the partitioning while creating the table or by executing the "-5&35"#-&
command. The partition column should be part of all the unique keys in the table.
If you defined partitions based on the DSFBUFE@BU column and JE is the primary key, you
should include the DSFBUF@BU column as part of 13*."3:,&:, that is, (JE, DSFBUFE@BU).
The following example assumes that there are no foreign keys referenced to the table.
If you wish to implement a partitioning scheme based on ranges or intervals of time in

MySQL 8.0, you have two options:
Partition the table by 3"/(&, and for the partitioning expression, employ a
function operating on a %"5&, 5*.&, or %"5&5*.& column and returning an
integer value
Partition the table by 3"/(&$0-6./4, using a %"5& or %"5&5*.& column as the
partitioning column
[ 255 ]

RANGE partitioning
If you want to partition the FNQMPZFFT table based on FNQ@OP and you want to keep
100,000 employees in one partition, you can create it like this:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
FNQ@OP

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
So, all the employees having FNQ@OP less than 100,000 will go to partition Q and all the
employees having the FNQ@OP less than and greater than will go to
partition Q, and so on.
If the employee number is above , since there is no partition defined for them, the
insert will fail with error. To avoid this, you have to regularly check and add partitions or
create a ."97"-6& partition to catch all such exceptions:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
FNQ@OP

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
[ 256 ]

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
PARTITION pmax VALUES LESS THAN MAXVALUE ENGINE = InnoDB

If you want to partition based on IJSF@EBUF, you can use the :&"3
IJSF@EBUF function
as the partition expression:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
:&"3
IJSF@EBUF

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#

Partitioning in MySQL is widely used on EBUF, EBUFUJNF, or UJNFTUBNQ columns. If you

want to store some events on the database, and all the queries are based out of a time range,
you can use partitioning like this.
The partitioning function UP@EBZT

returns the number of days since , which
is a integer:
NZTRM $3&"5&5"#-&AFWFOU@IJTUPSZA
AFWFOU@JEAJOU
/05/6--
AFWFOU@OBNFAWBSDIBS
/05/6--
ADSFBUFE@BUAEBUFUJNF/05/6--
AMBTU@VQEBUFEAUJNFTUBNQ%&'"6-5$633&/5@5*.&45".10/61%"5&
$633&/5@5*.&45".1
AFWFOU@UZQFAWBSDIBS
/05/6--
ANTHAUJOZUFYU/05/6--
13*."3:,&:
AFWFOU@JEAADSFBUFE@BUA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
to_days
DSFBUFE@BU

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
[ 257 ]

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#

If you want to convert an existing table into a partitioned table, and if the partition key is
not part of 13*."3:,&:, you need to drop 13*."3:,&: and add the partition key as part
of 13*."3:,&: and all unique keys. Otherwise, you will get the error &3303

):"13*."3:,&:NVTUJODMVEFBMMDPMVNOTJOUIFUBCMF T
QBSUJUJPOJOHGVODUJPO. You can do that as follows:
NZTRM "-5&35"#-&FNQMPZFFT%30113*."3:,&:"%%13*."3:
,&:
FNQ@OPIJSF@EBUF
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM "-5&35"#-&FNQMPZFFT1"35*5*0/#:3"/(&
:&"3
IJSF@EBUF

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QNBY7"-6&4-&445)"/."97"-6&&/(*/&*OOP%#

2VFSZ0,SPXTBGGFDUFE
TFD
For more details on 3"/(& partitioning, refer to IUUQTEFWNZTRMDPNEPDSFGNBO

FOQBSUJUJPOJOHSBOHFIUNM.
[ 258 ]

Removing partitioning
If you wish to remove partitioning, you can execute the 3&.07&1"35*5*0/*/( statement:
NZTRM "-5&35"#-&FNQMPZFFT3&.07&1"35*5*0/*/(
2VFSZ0,SPXTBGGFDUFE
TFD
RANGE COLUMNS partitioning

3"/(&$0-6./4 partitioning is similar to 3"/(& partitioning, but enables you to define
partitions using ranges based on multiple column values. In addition, you can define the
ranges using columns of types other than integer types. 3"/(&$0-6./4 partitioning
differs significantly from 3"/(& partitioning in the following ways:
3"/(&$0-6./4 does not accept expressions, only names of columns

3"/(&$0-6./4 accepts a list of one or more columns
3"/(&$0-6./4 partitioning columns are not restricted to integer columns;
string, %"5& and %"5&5*.& columns can also be used as partitioning columns
Instead of using the UP@EBZT

or ZFBS
function, you can directly use the column
IJSF@EBUF in 3"/(&$0-6./4:
NZTRM "-5&35"#-&FNQMPZFFT
1"35*5*0/#:RANGE COLUMNS
IJSF@EBUF

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/
."97"-6&

2VFSZ0,SPXTBGGFDUFE
TFD
Or you can divide employees based on their MBTU@OBNF. This will not guarantee uniform
distribution across the partitions:
NZTRM "-5&35"#-&FNQMPZFFT
1"35*5*0/#:3"/(&$0-6./4
MBTU@OBNF

1"35*5*0/Q7"-6&4-&445)"/
C
1"35*5*0/Q7"-6&4-&445)"/
G
1"35*5*0/Q7"-6&4-&445)"/
M
1"35*5*0/Q7"-6&4-&445)"/
R
[ 259 ]

1"35*5*0/Q7"-6&4-&445)"/
V
1"35*5*0/Q7"-6&4-&445)"/
[

2VFSZ0,SPXTBGGFDUFE
TFD
Using 3"/(&$0-6./4, you can put multiple columns in the partition function:
NZTRM $3&"5&5"#-&SBOHF@DPMVNOT@FYBNQMF
B*/5
C*/5
D*/5
E*/5
F*/5
13*."3:,&:
BCD

1"35*5*0/#:3"/(&$0-6./4
BCD
1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/
."97"-6&."97"-6&."97"-6&

If you insert values B, C, D, E, F, it goes to Q. When designing tables
partitioned by 3"/(&$0-6./4, you can always test successive partition definitions by
comparing the desired tuples using the NZTRM client, like this:
NZTRM 4&-&$5

Q

Q

Q

]Q]Q]Q]

]]]]

SPXJOTFU
TFD
In this case, the insert goes to Q.
LIST and LIST COLUMNS partitioning

-*45 partitioning is similar to 3"/(& partitioning, where each partition is defined and
selected based on the membership of a column value in one of a set of value lists, rather
than in one of a set of contiguous ranges of values.
[ 260 ]

You need to define it by 1"35*5*0/#:-*45

FYQS , where FYQS is a column value or
an expression based on a column value and returning an integer value.
Partition definition contains 7"-6&4*/

WBMVF@MJTU , where WBMVF@MJTU is a
comma-separated list of integers rather than 7"-6&4-&445)"/
WBMVF .
If you wish to use data types other than integers, you can use -*45$0-6./4.
Unlike the case with 3"/(& partitioning, there is no DBUDIBMM such as ."97"-6&; all
expected values for the partitioning expression should be covered in the 1"35*5*0/
expression.
Suppose there is a customer table with a zip code and city. If, for example, you want to
divide the customers with certain zip codes in a partition, you can use -*45 partitions:
NZTRM $3&"5&5"#-&DVTUPNFS
DVTUPNFS@JE*/5
[JQDPEF*/5
DJUZWBSDIBS

13*."3:,&:
DVTUPNFS@JE[JQDPEF

1"35*5*0/#:-*45
[JQDPEF
1"35*5*0/QOPSUI7"-6&4*/

1"35*5*0/QFBTU7"-6&4*/

1"35*5*0/QXFTU7"-6&4*/

1"35*5*0/QDFOUSBM7"-6&4*/

If you wish to use the columns directly, rather than integers, you can use -*45$0-6./4:
NZTRM $3&"5&5"#-&DVTUPNFS
DVTUPNFS@JE*/5
[JQDPEF*/5
DJUZWBSDIBS

13*."3:,&:
DVTUPNFS@JEDJUZ

1"35*5*0/#:-*45$0-6./4
DJUZ
1"35*5*0/QOPSUI7"-6&4*/
DJUZ DJUZ DJUZ
1"35*5*0/QFBTU7"-6&4*/
DJUZ DJUZ DJUZ
1"35*5*0/QXFTU7"-6&4*/
DJUZ DJUZ DJUZ
1"35*5*0/QDFOUSBM7"-6&4*/
DJUZ DJUZ DJUZ

[ 261 ]

HASH and LINEAR HASH partitioning

Partitioning by )"4) is used primarily to ensure an even distribution of data among a
predetermined number of partitions. With range or list partitioning, you must specify
explicitly which partition a given column value or set of column values should be stored in;
with hash partitioning, this decision is taken care of for you, and you need only specify a
column value or expression based on a column value to be hashed and the number of
partitions into which the partitioned table is to be divided.
If you want to evenly distribute employees, instead of 3"/(& partitioning over

:&"3
IJSF@EBUF, you can use )"4)PG:&"3
IJSF@EBUF and specify the number of
partitions. When 1"35*5*0/#:)"4) is used, the storage engine determines which
partition to use based on the modulus of the result of the expression.
For example, if the IJSF@EBUF is , :&"3

IJSF@EBUF would be and
.0%
is . So the row goes to third partition:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:)"4)
:&"3
IJSF@EBUF
1"35*5*0/4
The most efficient hashing function is one which operates upon a single table column and
whose value increases or decreases consistently with the column value.
In -*/&"3)"4) partitioning, you can use the same syntax, except for adding a -*/&"3
keyword. Instead of a .0%6-64 operation, MySQL uses a powers-of-two algorithm for
determining the partition. Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFO
QBSUJUJPOJOHMJOFBSIBTIIUNM for more details:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
[ 262 ]

ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:-*/&"3)"4)
:&"3
IJSF@EBUF
1"35*5*0/4
KEY and LINEAR KEY partitioning

Partitioning by key is similar to partitioning by hash, except that, where hash partitioning
employs a user-defined expression, the hashing function for key partitioning is supplied by
the MySQL server. This internal hashing function is based on the same algorithm as
the 1"44803%
function.
,&: takes only a list of zero or more column names. Any columns used as the partitioning
key must comprise part or all of the table's primary key, if the table has one. Where no
column name is specified as the partitioning key, the table's primary key is used, if there is
one:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:,&:

1"35*5*0/4
Subpartitioning
You can further divide each partition into a partitioned table. This is called subpartitioning
or composite partitioning:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
[ 263 ]

AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
:&"3
IJSF@EBUF
46#1"35*5*0/#:)"4)
FNQ@OP
46#1"35*5*0/4
1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/

1"35*5*0/Q7"-6&4-&445)"/."97"-6&

Partition pruning and selection

MySQL does not scan partitions where there are no matching values; this is automatic and
is called partition pruning. The MySQL optimizer evaluates the partitioning expression for
the value given, determines which partition contains that value, and scans only that
partition.
4&-&$5, %&-&5&, and 61%"5& statements support partition pruning. */4&35 statements
currently cannot be pruned.
You can also explicitly specify partitions and subpartitions for rows matching a given
8)&3& condition.
How to do it...
Partition pruning applies only to queries, but explicit selection of partitions is supported for
both queries and a number of DML statements.
[ 264 ]

Partition pruning
Take the example of the FNQMPZFFT table, which is partitioned based on FNQ@OP:
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
:&"3
IJSF@EBUF

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QNBY7"-6&4-&445)"/."97"-6&&/(*/&*OOP%#

Suppose the following 4&-&$5 query is executed:

NZTRM 4&-&$5MBTU@OBNFCJSUI@EBUF'30.FNQMPZFFT8)&3&
IJSF@EBUF "/%GJSTU@OBNF .BSJBOHJPMB
MySQL optimizer detects that partitioned column is being used in the query and
automatically determines the partition to scan.
In this query, it first calculates the :&"3

, which is , and scans the
Q partition rather than the whole table. This dramatically reduces the query time.
Instead of IJSF@EBUF , if a range is given, such

as IJSF@EBUF , then the partitions Q, Q, Q, and QNBY are
scanned.
If the expression IJSF@EBUF is not given in the 8)&3& clause, MySQL has
to scan the whole table.
[ 265 ]

To know which partitions the optimizer scans, you can execute the &91-"*/ plan of the
query, which is explained in the Explain plan section of $IBQUFS, Performance Tuning:
NZTRM &91-"*/4&-&$5MBTU@OBNFCJSUI@EBUF'30.FNQMPZFFT8)&3&
IJSF@EBUF "/%GJSTU@OBNF .BSJBOHJPMB =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
partitions: p2000
UZQFSFG
QPTTJCMF@LFZTOBNF
LFZOBNF
LFZ@MFO
SFGDPOTU
SPXT
GJMUFSFE
&YUSB6TJOHJOEFYDPOEJUJPO
NZTRM &91-"*/4&-&$5MBTU@OBNFCJSUI@EBUF'30.FNQMPZFFT8)&3&
IJSF@EBUF "/%GJSTU@OBNF .BSJBOHJPMB =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
partitions: p2000,p2010,p2020,pmax
UZQFSFG
QPTTJCMF@LFZTOBNF
LFZOBNF
LFZ@MFO
SFGDPOTU
SPXT
GJMUFSFE
&YUSB6TJOHJOEFYDPOEJUJPO
SPXJOTFUXBSOJOH
TFD
Partition selection
Partition pruning is automatic selection based on the 8)&3& clause. You can explicitly
specify the partitions to scan in the query. The queries can be 4&-&$5, %&-&5&, */4&35,
3&1-"$&, 61%"5&, -0"%%"5", and -0"%9.-. The 1"35*5*0/ option is used to select
partitions from a given table, you should specify the keyword 1"35*5*0/ <partition name>
immediately following the name of the table, before all other options, including any table
alias, for example:
[ 266 ]

NZTRM 4&-&$5FNQ@OPIJSF@EBUF'30.FNQMPZFFT1"35*5*0/
Q-*.*5

]FNQ@OP]IJSF@EBUF]

]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]
]]]

SPXTJOTFU
TFD
Similarly, we can delete:

NZTRM %&-&5&'30.FNQMPZFFTPARTITION (p1980, p1990)8)&3&GJSTU@OBNF-*,&
K
2VFSZ0,SPXTBGGFDUFE
TFD
Partition management
The most important thing when it comes to managing partitions is adding sufficient
partitions in advance for time-based 3"/(& partitioning. Failure to do so leads to errors
while inserting or, if the ."97"-6& partition is defined, all the inserts go into the ."97"-6&
partition. For example, take the FWFOU@IJTUPSZ table without the QNBY partition:
NZTRM $3&"5&5"#-&AFWFOU@IJTUPSZA
AFWFOU@JEAJOU
/05/6--
AFWFOU@OBNFAEBUF/05/6--
AMBTU@VQEBUFEAUJNFTUBNQ%&'"6-5$633&/5@5*.&45".10/61%"5&
$633&/5@5*.&45".1
AFWFOU@UZQFAWBSDIBS
/05/6--
13*."3:,&:
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
to_days
DSFBUFE@BU

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
[ 267 ]

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#

The table accepts the */4&354 till October 15, 2017; after that, the */4&354 fail.
Another important thing is to %&-&5& the data after it crosses retention.
How to do it...
To perform these operations, you need to execute the "-5&3 command.
ADD partitions
To add a new partition, execute the "%%1"35*5*0/
1"35*5*0/%&'*/*5*0/
statement:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ"%%1"35*5*0/
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#

This statement locks the whole table for a very short time.
[ 268 ]

Reorganizing partitions
If the ."97"-6& partition is there, you cannot add a partition after ."97"-6&; in that case,
you need to the 3&03("/*;&."97"-6& partition into two partitions:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ3&03("/*;&1"35*5*0/QNBY*/50
1"35*5*0/
Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QNBY7"-6&4-&445)"/."97"-6&&/(*/&*OOP%#
Remember MySQL has to substantially move the data while reorganizing partitions and the
table will be locked during that period.
You can also reorganize multiple partitions into a single partition:

NZTRM "-5&35"#-&FWFOU@IJTUPSZ3&03("/*;&1"35*5*0/
QQQQQQQ
*/50
1"35*5*0/Q@PDU@XFFL7"-6&4-&445)"/

DROP partitions
If the data has crossed the retention, you can %301 the whole partition, which is superquick
compared with conventional %&-&5&'30.5"#-& statement. This is very helpful in
archiving the data efficiently.
If Q has crossed the retention, you can %301 the partition using the "-5&35"#-&
%3011"35*5*0/ statement:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ%3011"35*5*0/Q
2VFSZ0,SPXTBGGFDUFE
TFD
Dropping the partition removes the 1"35*5*0/%&'*/*5*0/ from the table.
TRUNCATE partitions
If you wish to keep 1"35*5*0/%&'*/*5*0/ in the table and remove only the data, you
can execute the 536/$"5&1"35*5*0/ command:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ536/$"5&1"35*5*0/Q
2VFSZ0,SPXTBGGFDUFE
TFD
[ 269 ]

Managing HASH and KEY partitions

The operations performed on )"4) and ,&: partitions are quite different. You can only
reduce or increase the number of partitions.
Suppose the FNQMPZFFT table is partitioned based on )"4):

AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:)"4)
:&"3
IJSF@EBUF
1"35*5*0/4
To reduce the partitions from to , you can execute the $0"-&4$&1"35*5*0/ statement
and specify the number of partitions you want to reduce, that is, 8-6=2:
NZTRM "-5&35"#-&FNQMPZFFT$0"-&4$&1"35*5*0/
2VFSZ0,SPXTBGGFDUFE
TFD
To increase the partitions from to , you can execute the "%%1"35*5*0/ statement and
specify the number of partitions you want to increase, that is, 16-6=10:
NZTRM "-5&35"#-&FNQMPZFFT"%%1"35*5*0/1"35*5*0/4
2VFSZ0,SPXTBGGFDUFE
TFD
Other operations
You can also perform other operations, such as 3&#6*-%, 015*.*;&, "/"-:;&, and 3&1"*3
statements, for a particular partition, for example:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ3&1"*31"35*5*0/QQ
[ 270 ]

Partition information
This section discusses obtaining information about existing partitions, which can be done in
a number of ways.
How to do it...
Using SHOW CREATE TABLE

To know whether a table is partitioned or not, you can execute the 4)08$3&"5&5"#-&=(
statement, which shows the table definition along with partitions, for example:
NZTRM 4)08$3&"5&5"#-&FNQMPZFFT=(
SPX
5BCMFFNQMPZFFT
$SFBUF5BCMF$3&"5&5"#-&AFNQMPZFFTA
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
ABEESFTTAWBSDIBS
%&'"6-5/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
:&"3
IJSF@EBUF

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QNBY7"-6&4-&445)"/."97"-6&&/(*/&*OOP%#
Using SHOW TABLE STATUS

You can execute the 4)085"#-&45"564 command and check $SFBUF@PQUJPOT in the
output:
NZTRM 4)085"#-&45"564-*,& FNQMPZFFT =(
[ 271 ]

SPX
/BNFFNQMPZFFT
&OHJOF*OOP%#
7FSTJPO
3PX@GPSNBU%ZOBNJD
3PXT/6--
"WH@SPX@MFOHUI/6--
%BUB@MFOHUI/6--
.BY@EBUB@MFOHUI/6--
*OEFY@MFOHUI/6--
%BUB@GSFF/6--
"VUP@JODSFNFOU/6--
$SFBUF@UJNF
6QEBUF@UJNF/6--
$IFDL@UJNF/6--
$PMMBUJPOVUGNC@@BJ@DJ
$IFDLTVN/6--
Create_options: partitioned
$PNNFOU
SPXJOTFU
TFD
Using EXPLAIN
The &91-"*/ plan shows all the partitions scanned for a query. If you run the &91-"*/ plan
for 4&-&$5'30.UBCMF , it lists all the partitions, for example:
NZTRM &91-"*/4&-&$5'30.FNQMPZFFT=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
partitions: p1980,p1990,p2000,p2010,p2020,pmax
UZQF"--
QPTTJCMF@LFZT/6--
LFZ/6--
LFZ@MFO/6--
SFG/6--
SPXT
GJMUFSFE
&YUSB/6--
SPXJOTFUXBSOJOH
TFD
[ 272 ]

Querying the INFORMATION_SCHEMA.PARTITIONS

table
Compared to all the preceding methods, */'03."5*0/@4$)&."1"35*5*0/4 gives more
information about the partitions:
NZTRM 4)08$3&"5&5"#-&*/'03."5*0/@4$)&."1"35*5*0/4=(
SPX
5BCMF1"35*5*0/4
$SFBUF5BCMF$3&"5&5&.103"3:5"#-&A1"35*5*0/4A
A5"#-&@$"5"-0(AWBSDIBS
/05/6--%&'"6-5
A5"#-&@4$)&."AWBSDIBS
/05/6--%&'"6-5
A5"#-&@/".&AWBSDIBS
/05/6--%&'"6-5
A1"35*5*0/@/".&AWBSDIBS
%&'"6-5/6--
A46#1"35*5*0/@/".&AWBSDIBS
%&'"6-5/6--
A1"35*5*0/@03%*/"-@104*5*0/ACJHJOU
VOTJHOFE%&'"6-5/6--
A46#1"35*5*0/@03%*/"-@104*5*0/ACJHJOU
VOTJHOFE%&'"6-5/6--
A1"35*5*0/@.&5)0%AWBSDIBS
%&'"6-5/6--
A46#1"35*5*0/@.&5)0%AWBSDIBS
%&'"6-5/6--
A1"35*5*0/@&913&44*0/AMPOHUFYU
A46#1"35*5*0/@&913&44*0/AMPOHUFYU
A1"35*5*0/@%&4$3*15*0/AMPOHUFYU
A5"#-&@3084ACJHJOU
VOTJHOFE/05/6--%&'"6-5
A"7(@308@-&/(5)ACJHJOU
VOTJHOFE/05/6--%&'"6-5
A%"5"@-&/(5)ACJHJOU
VOTJHOFE/05/6--%&'"6-5
A."9@%"5"@-&/(5)ACJHJOU
VOTJHOFE%&'"6-5/6--
A*/%&9@-&/(5)ACJHJOU
VOTJHOFE/05/6--%&'"6-5
A%"5"@'3&&ACJHJOU
VOTJHOFE/05/6--%&'"6-5
A$3&"5&@5*.&AEBUFUJNF%&'"6-5/6--
A61%"5&@5*.&AEBUFUJNF%&'"6-5/6--
A$)&$,@5*.&AEBUFUJNF%&'"6-5/6--
A$)&$,46.ACJHJOU
VOTJHOFE%&'"6-5/6--
A1"35*5*0/@$0..&/5AWBSDIBS
/05/6--%&'"6-5
A/0%&(3061AWBSDIBS
/05/6--%&'"6-5
A5"#-&41"$&@/".&AWBSDIBS
%&'"6-5/6--
&/(*/&*OOP%#%&'"6-5$)"34&5VUG
SPXJOTFU
TFD
To find out more details about a table partition, you can query the
*/'03."5*0/@4$)&."1"35*5*0/4 table by specifying the database name through
5"#-&@4$)&." and table name through 5"#-&@/".&, for example:
NZTRM 4&-&$51"35*5*0/@/".&'30.*/'03."5*0/@4$)&."1"35*5*0/48)&3&
5"#-&@4$)&." FNQMPZFFT "/%5"#-&@/".& FNQMPZFFT

]1"35*5*0/@/".&]
[ 273 ]


]Q]
]Q]
]Q]
]Q]
]Q]
]QNBY]

SPXTJOTFU
TFD
You can get details such as 1"35*5*0/@.&5)0%, 1"35*5*0/@&913&44*0/,

1"35*5*0/@%&4$3*15*0/, and 5"#-&@3084 in that partition:
mysql> SELECT * FROM INFORMATION_SCHEMA.PARTITIONS WHERE

TABLE_SCHEMA='employees' AND TABLE_NAME='employees' AND
PARTITION_NAME='p1990'\G
SPX
5"#-&@$"5"-0(EFG
5"#-&@4$)&."FNQMPZFFT
5"#-&@/".&FNQMPZFFT
1"35*5*0/@/".&Q
46#1"35*5*0/@/".&/6--
1"35*5*0/@03%*/"-@104*5*0/
46#1"35*5*0/@03%*/"-@104*5*0//6--
1"35*5*0/@.&5)0%3"/(&
46#1"35*5*0/@.&5)0%/6--
1"35*5*0/@&913&44*0/:&"3
IJSF@EBUF
46#1"35*5*0/@&913&44*0//6--
1"35*5*0/@%&4$3*15*0/
5"#-&@3084
"7(@308@-&/(5)
%"5"@-&/(5)
."9@%"5"@-&/(5)/6--
*/%&9@-&/(5)
%"5"@'3&&
$3&"5&@5*.&/6--
61%"5&@5*.&/6--
$)&$,@5*.&/6--
$)&$,46./6--
1"35*5*0/@$0..&/5
/0%&(3061EFGBVMU
5"#-&41"$&@/".&/6--
SPXJOTFU
TFD
For more details, refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOQBSUJUJPOTUBCMF

IUNM.
[ 274 ]

Efficiently managing time to live and soft

delete rows
3"/(&$0-6./4 is highly useful in managing time to live and soft delete rows. Suppose
you have a application which specifies the expiry time of row (row to be deleted after it
crosses the expiry time) and the expiry is varying.
Suppose the application can do the following types of inserts:
Insert persistent data

Insert with expiry
If the expiry is constant i.e all the rows inserted will be deleted after certain time, we can go
with RANGE partitioning. But if the expiry is varying i.e some rows will be deleted in a
week, some in a month, some in a year and some have no expiry, it is not possible to create
partitions. In that case, you can use the 3"/(&$0-6./4 partitioning explained below.
How it works...
We introduce a column called TPGU@EFMFUF which will be set by trigger. The
TPGU@EFMFUF column will be part of range column partitioning.
Partitioning will be like (TPGU@EFMFUF, expires). The TPGU@EFMFUF and expires together
controls which partition a row should go. soft_delete column decides the retention of the
row. If the expires is 0, the trigger sets the TPGU@EFMFUF value to 0 which puts the row in a
OP@SFUFOUJPO partition and if the value of expires is out of partition bounds, trigger sets
the TPGU@EFMFUF value to 1 and the row will be put into a MPOH@SFUFOUJPO partition. If
the value of expires is with in the partition bounds, trigger sets the TPGU@EFMFUF value to
. Depending on the value of the expires, the row will be put in respective partition.
To summarize, TPGU@EFMFUF will be :
: If the value of expires is 0

: If the expires is more than 30 days away from the timestamp
: If the expires is less than or equal to 30 days away from the timestamp
[ 275 ]

We create
1 OP@SFUFOUJPO partition (TPGU@EFMFUF)

1 MPOH@SFUFOUJPO partition (TPGU@EFMFUF)
8 daily partitions (TPGU@EFMFUF)
How to do it...
You can create a table like this:
NZTRM $3&"5&5"#-&ADVTUPNFS@EBUBA
AJEAJOU
/05/6--"650@*/$3&.&/5
ANTHAUFYU
AUJNFTUBNQACJHJOU
/05/6--%&'"6-5
AFYQJSFTACJHJOU
/05/6--%&'"6-5
ATPGU@EFMFUFAUJOZJOU
VOTJHOFE/05/6--%&'"6-5
13*."3:,&:
AJEAAFYQJSFTAATPGU@EFMFUFA
&/(*/&*OOP%#%&'"6-5$)"34&5VUG
1"35*5*0/#:3"/(&$0-6./4
TPGU@EFMFUFFYQJSFT

1"35*5*0/OP@SFUFOUJPO7"-6&4-&445)"/
."97"-6&&/(*/&*OOP%#
1"35*5*0/MPOH@SFUFOUJPO7"-6&4-&445)"/
."97"-6&&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#

There will be a buffer weekly partition which will be 42 days away and will be always
empty so that we can split and 7+2 daily partitions with 2 buffer.
NZTRM %30153*((&3*'&9*454DVTUPNFS@EBUB@JOTFSU
%&-*.*5&3
$3&"5&53*((&3DVTUPNFS@EBUB@JOTFSU
#&'03&*/4&35
0/DVTUPNFS@EBUB'03&"$)308
#&(*/
4&5/&8TPGU@EFMFUF
*'
/&8FYQJSFT
*'
306/%
/&8FYQJSFT/&8UJNFTUBNQ

[ 276 ]

&/%

%&-*.*5&3
NZTRM %30153*((&3*'&9*454DVTUPNFS@EBUB@VQEBUF
%&-*.*5&3
$3&"5&53*((&3DVTUPNFS@EBUB@VQEBUF
#&'03&61%"5&
#&(*/
4&5/&8TPGU@EFMFUF
*'
/&8FYQJSFT
*'
306/%

&/%

%&-*.*5&3
Suppose the client inserts a row with timestamp of 1508265000 (2017-10-17

18:30:00) and expiry value of 1508351400 (2017-10-18 18:30:00), the soft_delete will
be 2 which makes it into partition pd20171019
NZTRM */4&35*/50DVTUPNFS@EBUB
JENTHUJNFTUBNQFYQJSFT
7"-6&4
UFTU
2VFSZ0,SPXBGGFDUFE
TFD
NZTRM 4&-&$5'30.DVTUPNFS@EBUB1"35*5*0/

QE

]JE]NTH]UJNFTUBNQ]FYQJSFT]TPGU@EFMFUF]

]]UFTU]]]]

SPXJOTFU
TFD
Suppose the client does not set expiry, expires column will be 0 which makes the
TPGU@EFMFUF to and it will go to OP@SFUFOUJPO partition.
7"-6&4
OPO@FYQJSZ@SPX
2VFSZ0,SPXBGGFDUFE
TFD

OP@SFUFOUJPO


]]OPO@FYQJSZ@SPX]]]]

SPXJOTFU
TFD
[ 277 ]

Suppose the client wants to have expiry (assume 2017-10-19 06:30:00), expiry
column can be updated which moves the row from OP@SFUFOUJPO partition to
the respective partition (this has some performance impact because the row has
to be moved across partitions)
NZTRM 61%"5&DVTUPNFS@EBUB4&5FYQJSFT8)&3&JE
2VFSZ0,SPXBGGFDUFE
TFD

OP@SFUFOUJPO
&NQUZTFU
TFD

QE


]]OPO@FYQJSZ@SPX]]]]

SPXJOTFU
TFD
Suppose the client sets a expiry which is beyond our partitions, it will
automatically go into MPOH@SFUFOUJPO partition.
7"-6&4
MPOH@FYQJSZ

MPOH@SFUFOUJPO


]]MPOH@FYQJSZ]]]]

SPXJOTFU
TFD
Movement of rows across partitions is slow, if you update the TPGU@EFMFUF, the row will
be moved from default partition to other partition which will be slow.
Extending the logic
We can extend the logic and increase the value of TPGU@EFMFUF to accommodate more
types of partitions.
: If the value of expires is 0

[ 278 ]

: If the expires is more than 60 days away from the timestamp
The TPGU@EFMFUF column will be part of partitioning. We create
Single OP@SFUFOUJPO partition if value of TPGU@EFMFUF is

Single MPOH@SFUFOUJPO partition if the value of TPGU@EFMFUF
Weekly partitions if the value of TPGU@EFMFUF
Daily partitions if the value of TPGU@EFMFUF
Example partitioned table structure
There will be a buffer weekly partition which will be 42 days away and will be always
empty so that we can split and 7+2 daily partitions with 2 buffer.
NZTRM %30153*((&3*'&9*454DVTUPNFS@EBUB@JOTFSU
%&-*.*5&3
$3&"5&53*((&3DVTUPNFS@EBUB@JOTFSU
#&'03&*/4&35
#&(*/
4&5/&8TPGU@EFMFUF
*'
/&8FYQJSFT
*'
306/%

*'
306/%

&/%

%&-*.*5&3
NZTRM %30153*((&3*'&9*454DVTUPNFS@EBUB@VQEBUF
%&-*.*5&3
$3&"5&53*((&3DVTUPNFS@EBUB@VQEBUF
#&'03&*/4&35
#&(*/
4&5/&8TPGU@EFMFUF
*'
/&8FYQJSFT
*'
306/%

*'
306/%

&/%

%&-*.*5&3
NZTRM $3&"5&5"#-&ADVTUPNFS@EBUBA
AJEAJOU
/05/6--"650@*/$3&.&/5
[ 279 ]

ANTHAUFYU
AUJNFTUBNQACJHJOU
/05/6--%&'"6-5
AFYQJSFTACJHJOU
/05/6--%&'"6-5
ATPGU@EFMFUFAUJOZJOU
VOTJHOFE/05/6--%&'"6-5
13*."3:,&:
AJEAAFYQJSFTAATPGU@EFMFUFA
&/(*/&*OOP%#"650@*/$3&.&/5%&'"6-5$)"34&5VUG
1"35*5*0/#:3"/(&$0-6./4
ATPGU@EFMFUFAAFYQJSFTA

1"35*5*0/OP@SFUFOUJPO7"-6&4-&445)"/
."97"-6&&/(*/&*OOP%#
1"35*5*0/MPOH@SFUFOUJPO7"-6&4-&445)"/
."97"-6&&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QX7"-6&4-&445)"/
&/(*/&*OOP%#
CVGGFSQBSUJUJPOXIJDIXJMMCFEBZTBXBZBOEXJMMCFBMXBZTFNQUZTP
UIBUXFDBOTQMJU
PARTITION pw20171210 VALUES LESS THAN (2,1512864000000) ENGINE = InnoDB,
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/QE7"-6&4-&445)"/
&/(*/&*OOP%#

Managing partitions
You can create a $30/ in Linux or &7&/5 in mysql to manage the partitions. As the retention
approaches, the partition management tool should reorganize the buffer partition into one
usable partition and a buffer partition and also drop the partitions which crossed the
retention.
[ 280 ]

For example, take the DVTUPNFS@EBUB table mentioned previously.
On 20171203, you have to split the partition pw20171210 into pw20171210 and
pw20171217.
On 20171017, you have to split the pd20171024 into pd20171024 and pd20171025.
Splitting(reorganizing) partitions would be very quick (~milli seconds if there are no

queries locking the table) only if there is no (or very less) data. So we should aim to keep the
partition empty by reorganizing it before the data enters the partition.
[ 281 ]

Managing Tablespace
11
Changing the number or size of InnoDB REDO log files

Resizing the InnoDB system tablespace
Creating file-per-table tablespaces outside the data directory
Copying file-per-table tablespaces to another instance
Managing UNDO tablespace
Managing general tablespace
Compressing InnoDB tables
Introduction
Before you begin this chapter, you should understand the basics of *OOP%#.
As per the MySQL documentation,
System Tablespace (Shared tablespace)

"The InnoDB system tablespace contains the InnoDB data dictionary (metadata for InnoDB-related
objects) and is the storage area for the doublewrite buffer, the change buffer, and undo logs. The
system tablespace also contains table and index data for any user-created tables that are created in the
system tablespace. The system tablespace is considered a shared tablespace since it is shared by
multiple tables.
The system tablespace is represented by one or more data files. By default, one system data file,
named ibdata1, is created in the MySQL data directory. The size and number of system data files is
controlled by the innodb_data_file_path startup option."

Managing Tablespace Chapter 11
File-per-table tablespace
A file-per-table tablespace is a single-table tablespace that is created in its own data file
rather than in the system tablespace. Tables are created in file-per-table tablespaces when
the JOOPEC@GJMF@QFS@UBCMF option is enabled. Otherwise, *OOP%# tables are created in
the system tablespace. Each file-per-table tablespace is represented by a single JCE data
file, which is created in the database directory by default.
File-per-table tablespaces support %:/".*$ and $0.13&44&% row formats, which support
features such as off-page storage for variable length data and table compression.
To know the advantages and disadvantages of file-per-table tablespaces, refer to IUUQT

EFWNZTRMDPNEPDSFGNBOFOJOOPECNVMUJQMFUBCMFTQBDFTIUNM and IUUQT
EFWNZTRMDPNEPDSFGNBOFOJOOPECQBSBNFUFSTIUNMTZTWBS@JOOPEC@GJMF@QFS@
UBCMF.
General tablespace
A general tablespace is a shared *OOP%# tablespace created using $3&"5&5"#-&41"$&

syntax. General tablespaces can be created outside of the MySQL EBUBEJSFDUPSZ , are
capable of holding multiple tables, and support tables of all row formats.
UNDO tablespace
An undo log is a collection of undo log records associated with a single transaction. An
undo log record contains information about how to undo the latest change by a transaction
to a clustered index record. If another transaction needs to see the original data (as part of a
consistent read operation), the unmodified data is retrieved from the undo log records.
Undo logs exist within undo log segments, which are contained within rollback segments.
Rollback segments reside in the system tablespace, temporary tablespace, and undo
tablespaces.
An 6/%0 tablespace comprises one or more files that contain undo logs. The number of
undo tablespaces used by *OOP%# is defined by the JOOPEC@VOEP@UBCMFTQBDFT
configuration option.
These logs are used to roll back transactions and also for multi-version concurrency control.
[ 283 ]

Data dictionary
The EBUBEJDUJPOBSZ is metadata that keeps track of database objects such as tables,
indexes, and table columns. For the MySQL EBUBEJDUJPOBSZ, introduced in MySQL 8.0,
metadata is physically located in *OOP%# file-per-table tablespace files in the
MySQL database directory. For the *OOP%#EBUBEJDUJPOBSZ, metadata is physically
located in the *OOP%# system tablespace.
MySQL data dictionary
MySQL server incorporates a transactional EBUBEJDUJPOBSZ that stores information

about database objects. In previous MySQL releases, dictionary data was stored in metadata
files, nontransactional tables, and storage engine-specific EBUBEJDUJPOBSJFT.
In previous MySQL releases, dictionary data was partially stored in metadata files. Issues
with file-based metadata storage included expensive file scans, susceptibility to filesystem-
related bugs, complex code for handling of replication and crash recovery failure states, and
a lack of extensibility that made it difficult to add metadata for new features and relational
objects.
Benefits of the MySQL EBUBEJDUJPOBSZ include:
Simplicity of a centralized EBUBEJDUJPOBSZ schema that uniformly stores

dictionary data
Removal of file-based metadata storage
Transactional, crash-safe storage of dictionary data
Uniform and centralized caching for dictionary objects
A simpler and improved implementation for some */'03."5*0/@4$)&." tables
Atomic DDL
The metadata files listed as follows are removed from MySQL. Unless otherwise noted, data
previously stored in metadata files is now stored in EBUBEJDUJPOBSZ tables:
GSN files: Table metadata files for table definition.

QBS files: Partition definition files. *OOP%# stopped using the
EFGJOJUJPO partition files in MySQL 5.7 with the introduction of native
partitioning support for *OOP%# tables.
USO files: Trigger namespace files.
USH files: Trigger parameter files.
[ 284 ]

JTM files: The *OOP%# symbolic link files containing the location of file-per-table
tablespace files created outside of the MySQL EBUBEJSFDUPSZ .
ECPQU files: Database configuration files. These files, one per database directory,
contain database default character set attributes.
The limitations of MySQL EBUBEJDUJPOBSZ are as follows:
Manual creation of database directories under the EBUBEJSFDUPSZ (for

example, with NLEJS) is unsupported. Manually created database directories are
not recognized by the MySQL server.
Moving data stored in MyISAM tables by copying and moving MyISAM data
files is unsupported. Tables moved using this method are not discovered by the
server.
Simple backup and restore of individual MyISAM tables using copied data files is
unsupported.
DDL operations take longer due to writing to storage, undo logs, and redo logs
instead of GSN files.
Transactional storage of dictionary data

The EBUBEJDUJPOBSZ schema stores dictionary data in transactional (*OOP%#) tables.
EBUBEJDUJPOBSZ tables are located in the NZTRM database together with OPOEBUB
EJDUJPOBSZ system tables.
EBUBEJDUJPOBSZ tables are created in a single *OOP%# tablespace named NZTRMJCE in

the MySQL EBUBEJSFDUPSZ . The NZTRMJCE tablespace file must reside in the MySQL
EBUBEJSFDUPSZ and its name cannot be modified or used by another tablespace.
Previously, these tables were created in individual tablespace files in the MySQL database
directory.
Changing the number or size of InnoDB redo

log files
The JC@MPHGJMF file and JC@MPHGJMF are the default *OOP%# redo log files created
inside the EBUBEJSFDUPSZ, with 48 MB each. If you wish to change the size of the redo log
files, you can simply change it in the configuration file and restart MySQL. In previous
versions, you had to do a slow shutdown of MySQL server, remove the redo log files,
change the config file, and then start MySQL server.
[ 285 ]

As of MySQL 8, *OOP%# detects that the JOOPEC@MPH@GJMF@TJ[F differs from the redo log
file size. It writes a log checkpoint, closes and removes the old log files, creates new log files
at the requested size, and opens the new log files.
How to do it...
1. Check the sizes of the current files:
TIFMM TVEPMTMIUSWBSMJCNZTRMJC@MPHGJMF
SXSNZTRMNZTRM.0DU
WBSMJCNZTRMJC@MPHGJMF
SXSNZTRMNZTRM.0DU
2. Stop the MySQL server and make sure that it shuts down without errors:
3. Edit the configuration file:
<NZTRME>
JOOPEC@MPH@GJMF@TJ[F.
JOOPEC@MPH@GJMFT@JO@HSPVQ
5. You can verify what MySQL did in the log file:
shell> sudo less /var/log/mysqld.log

2017-10-07T11:09:35.111926Z 1 [Warning] InnoDB: Resizing redo log
from 2*3072 to 4*8192 pages, LSN=249633608
5;<8BSOJOH>*OOP%#4UBSUJOHUPEFMFUF
BOESFXSJUFMPHGJMFT
5;</PUF>*OOP%#4FUUJOHMPHGJMF
JC@MPHGJMFTJ[FUP.#
5;</PUF>*OOP%#1SPHSFTTJO.#

JC@MPHGJMFTJ[FUP.#

[ 286 ]

JC@MPHGJMFTJ[FUP.#

JC@MPHGJMFTJ[FUP.#

5;</PUF>*OOP%#3FOBNJOHMPHGJMF
JC@MPHGJMFUPJC@MPHGJMF
5;<8BSOJOH>*OOP%#/FXMPHGJMFT
DSFBUFE-4/
6. You can also see the new log files created:
TIFMM TVEPMTMIUSWBSMJCNZTRMJC@MPHGJMF
SXSNZTRMNZTRM.0DU
SXSNZTRMNZTRM.0DU
SXSNZTRMNZTRM.0DU
SXSNZTRMNZTRM.0DU
Resizing the InnoDB system tablespace

The JCEBUB file in the EBUBEJSFDUPSZ is the default system tablespace. You can
configure JCEBUB using the JOOPEC@EBUB@GJMF@QBUI and JOOPEC@EBUB@IPNF@EJS
configuration options. The JOOPEC@EBUB@GJMF@QBUI configuration option is used to
configure the *OOP%# system tablespace data files. The value of JOOPEC@EBUB@GJMF@QBUI
should be a list of one or more data file specifications. If you name two or more data files,
separate them by semicolon () characters.
If you want a tablespace containing a fixed-size 50 MB data file named JCEBUB and a 50
MB auto-extending file named JCEBUB in the EBUBEJSFDUPSZ, it can be configured like
this:
<NZTRME>
JOOPEC@EBUB@GJMF@QBUIJCEBUB.JCEBUB.BVUPFYUFOE
If the JCEBUB files become so big, especially when JOOPEC@GJMF@QFS@UBCMF is not

enabled and the disk becomes full, you might want to add another data file on another disk.
[ 287 ]

How to do it...
Resizing the *OOP%# system tablespace is one topic that you would love to know more and
more about. Let's get into its details.
Increasing the InnoDB system tablespace

Assuming that JOOPEC@EBUB@GJMF@QBUI is JCEBUB.BVUPFYUFOE, the size has
reached 76 MB, and your disk is just 100 MB, you can add another disk and configure to
add another tablespace onto the new disk:
1. Stop MySQL server:
2. Check the size of the existing JCEBUB file:
TIFMM TVEPMTMIUSWBSMJCNZTRMJCEBUB
SXSNZTRMNZTRM.0DUWBSMJCNZTRMJCEBUB
3. Mount the new disk. Assuming it is mounted on WBSMJCNZTRM@FYUFOE,

change the ownership to NZTRM; make sure that the file is not already created. If
you are using AppArmour or SELinux, make sure you set the alias or context
correctly:
TIFMM TVEPDIPXONZTRMNZTRMWBSMJCNZTRM@FYUFOE
TIFMM TVEPDINPEWBSMJCNZTRM@FYUFOE
TIFMM TVEPMTMIUSWBSMJCNZTRM@FYUFOE
4. Open the NZDOG and add the following:
shell> sudo vi /etc/my.cnf

<NZTRME>
JOOPEC@EBUB@IPNF@EJS
JOOPEC@EBUB@GJMF@QBUI
JCEBUB76MWBSMJCNZTRM@FYUFOEJCEBUB.BVUPFYUFOE
[ 288 ]

Since the existing size of JCEBUB is 76 MB, you have to choose a maxvalue of at
least 76 MB. The next JCEBUB file will be created on the new disk mounted on
WBSMJCNZTRM@FYUFOE. The JOOPEC@EBUB@IPNF@EJS option should be
specified; otherwise, NZTRME looks at a different path and fails with an error:
5;<&3303>*OOP%#0QFSBUJOHTZTUFN
FSSPSOVNCFSJOBGJMFPQFSBUJPO
5;<&3303>*OOP%#5IFFSSPSNFBOTUIF
TZTUFNDBOOPUGJOEUIFQBUITQFDJGJFE
5;<&3303>*OOP%#*GZPVBSFJOTUBMMJOH
*OOP%#SFNFNCFSUIBUZPVNVTUDSFBUFEJSFDUPSJFTZPVSTFMG*OOP%#
EPFTOPUDSFBUFUIFN
5;<&3303>*OOP%#'JMF
WBSMJCNZTRM@FYUFOEJCEBUB DSFBUF SFUVSOFE04FSSPS
$BOOPUDPOUJOVFPQFSBUJPO
5. Start MySQL server:
6. Verify the new file. Since you have mentioned it as 50 MB, the initial size of the
file would be 50 MB:
TIFMM TVEPMTMIUSWBSMJCNZTRM@FYUFOE
UPUBM.
SXSNZTRMNZTRM.0DUJCEBUB
NZTRM 4)087"3*"#-&4-*,& JOOPEC@EBUB@GJMF@QBUI

]7BSJBCMF@OBNF]7BMVF
]

]JOOPEC@EBUB@GJMF@QBUI]
JCEBUB.WBSMJCNZTRM@FYUFOEJCEBUB.BVUPFYUFOE]

SPXJOTFU
TFD
[ 289 ]

Shrinking the InnoDB system tablespace

If you are not using JOOPEC@GJMF@QFS@UBCMF, then all of the table data is stored in system
tablespace. If you drop a table, the space is not reclaimed. You can shrink the system
tablespace and reclaim the disk space. This requires a major downtime, so it is
recommended to perform the task on a slave by taking it out of rotation and then
promoting it to master.
You can check the free space by querying the */'03."5*0/@4$)&." tables:
NZTRM 4&-&$546.
EBUB@GSFF'30.*/'03."5*0/@4$)&."5"#-&4

]TVN
EBUB@GSFF]

]]

SPXJOTFU
TFD
1. Stop the writes to the database. If it is a master, NZTRM 4&5

!!(-0#"-3&"%@0/-:; if it is a slave, stop the replication and save the binary
log coordinates:
NZTRM 45014-"7&
NZTRM 4)084-"7&45"564=(
2. Take a full backup using NZTRMEVNQ or NZEVNQFS, excluding the TZT database:
TIFMM NZEVNQFSVSPPUQBTTXPSEQBTTXPSE USYDPOTJTUFODZ

POMZLJMMMPOHRVFSJFTMPOHRVFSZHVBSESFHFY ?
TZT
PVUQVUEJSCBDLVQT
3. Stop MySQL server:
4. Remove all the JCE, JC@MPH, and JCEBUB files. If you are using only *OOP%#
tables, you can wipe off the EBUBEJSFDUPSZ and all the locations where system
table spaces are stored (JOOPEC@EBUB@GJMF@QBUI):
TIFMM TVEPSNSGWBSMJCNZTRMJCWBSMJCNZTRMEBUBCBTF

EJSFDUPSJFT
TIFMM TVEPSNSGWBSMJCNZTRM@FYUFOE
[ 290 ]

5. Initialize EBUBEJSFDUPSZ:
TIFMM TVEPNZTRMEJOJUJBMJ[FEBUBEJSWBSMJCNZTRM
TIFMM DIPXO3NZTRMNZTRMWBSMJCNZTRM
TIFMM DIPXO3NZTRMNZTRMWBSMJCNZTRM@FYUFOE
6. Get the temporary password:
TIFMM TVEPHSFQUFNQPSBSZQBTTXPSEJTHFOFSBUFE
WBSMPHNZTRMFSSPSMPH]UBJM
HFOFSBUFEGPSSPPU!MPDBMIPTUM*RFSSBHQB
7. Start MySQL and change the password:
TIFMM NZTRMVSPPUQM*RFSSBHQB
NZTRM "-5&364&3 SPPU ! MPDBMIPTU *%&/5*'*&%#: YYYY

2VFSZ0,SPXTBGGFDUFE
TFD
8. Restore the backup. Use the temporary password to connect to MySQL:
TIFMM PQUNZEVNQFSNZMPBEFSEJSFDUPSZCBDLVQTRVFSJFTQFS
USBOTBDUJPOUISFBETVTFSSPPUQBTTXPSEYYYY
PWFSXSJUFUBCMFT
9. If it is a master, enable writes by

NZTRM 4&5!!(-0#"-3&"%@0/-:. If it is a slave, restore the replication by
executing the $)"/(&."45&350$0.."/% and 45"354-"7&.
Creating file-per-table tablespaces outside

the data directory
In the previous section, you understood how to create a system tablespace in another disk.
In this section, you will learn how to create an individual tablespace in another disk.
[ 291 ]

How to do it...
You can mount a new disk with particular performance or capacity characteristics, such as a
fast SSD or a high-capacity HDD, onto a directory and configure *OOP%# to use that. Within
the destination directory, MySQL creates a subdirectory corresponding to the database
name, and within that, a JCE file for the new table. Remember, you cannot use the %"5"
%*3&$503: clause with the "-5&35"#-& statement:
1. Mount the new disk and change the permissions. If you are using AppArmour or
SELinux, make sure you set the alias or context correctly:
TIFMM TVEPDIPXO3NZTRMNZTRMWBSMJCNZTRM@GBTU@TUPSBHF
TIFMM TVEPDINPEWBSMJCNZTRM@GBTU@TUPSBHF
2. Create a table:
NZTRM $3&"5&5"#-&FWFOU@USBDLFS
FWFOU@JE*/56/4*(/&%"650@*/$3&.&/513*."3:,&:
FWFOU@OBNFWBSDIBS

UTUJNFTUBNQ/05/6--
FWFOU@UZQFWBSDIBS

5"#-&41"$&JOOPEC@GJMF@QFS@UBCMF
%"5"%*3&$503: WBSMJCNZTRM@GBTU@TUPSBHF
3. Check the JCE file created in the new device:
TIFMM TVEPMTMIUSWBSMJCNZTRM@GBTU@TUPSBHFFNQMPZFFT
UPUBM,
SXSNZTRMNZTRM,0DUFWFOU@USBDLFSJCE
Copying file-per-table tablespaces to

another instance
Copying the tablespace file (the JCE file) is the fastest way of moving data around, rather
than exporting and importing through NZTRMEVNQ or NZEVNQFS. The data is available
immediately rather than having to be reinserted and the indexes rebuilt. There are many
reasons why you might copy an *OOP%# file-per-table tablespace to a different instance:
To run reports without putting extra load on a production server

To set up identical data for a table on a new slave server
[ 292 ]

To restore a backed-up version of a table or partition after a problem or mistake

To have busy tables on an SSD device, or large tables on a high-capacity HDD
device
How to do it...
The outline is: you create the table on the destination with the same table definition and
execute the %*4$"3%5"#-&41"$& command on the destination. Execute '-64)5"#-&4
'03&91035 on the source, which ensures that changes to the named table have been
flushed to disk, and so a binary table copy can be made while the instance is running. After
that statement, the table is locked and does not accept any writes; however, reads can
happen. You can copy the JCE file of that table to the destination, execute 6/-0$, tables
on source, and finally execute the *.10355"#-&41"$& command, which accepts the
copied JCE file.
For example, you want to copy the FWFOUT@IJTUPSZ table in a test database from one
server (source) to another server (destination).
Create FWFOU@IJTUPSZ if not created already and insert a few rows for the demo:
NZTRM 64&UFTU
NZTRM $3&"5&5"#-&*'/05&9*454AFWFOU@IJTUPSZA
AFWFOU@JEAJOU
/05/6--
AFWFOU@OBNFAWBSDIBS
%&'"6-5/6--
AMBTU@VQEBUFEAUJNFTUBNQ/6--%&'"6-5$633&/5@5*.&45".10/61%"5&
$633&/5@5*.&45".1
AFWFOU@UZQFAWBSDIBS
/05/6--
13*."3:,&:
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
UP@EBZT
ADSFBUFE@BUA

1"35*5*0/@PDU@XFFL7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
[ 293 ]

NZTRM */4&35*/50FWFOU@IJTUPSZ7"-6&4

UFTU DMJDL UFTU@NFTTBHF







2VFSZ0,SPXTBGGFDUFE
TFD
Copy full table

1. On destination: Create the table with the same definition as on the source:
NZTRM 64&UFTU
NZTRM $3&"5&5"#-&*'/05&9*454AFWFOU@IJTUPSZA
AFWFOU@JEAJOU
/05/6--
AFWFOU@OBNFAWBSDIBS
%&'"6-5/6--
AMBTU@VQEBUFEAUJNFTUBNQ/6--%&'"6-5$633&/5@5*.&45".10/61%"5&
$633&/5@5*.&45".1
AFWFOU@UZQFAWBSDIBS
/05/6--
13*."3:,&:
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
1"35*5*0/#:3"/(&
UP@EBZT
ADSFBUFE@BUA

1"35*5*0/@PDU@XFFL7"-6&4-&445)"/
&/(*/&
*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
2. On destination: Discard the tablespace:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ%*4$"3%5"#-&41"$&
2VFSZ0,SPXTBGGFDUFE
TFD
[ 294 ]

3. On source: Execute '-64)5"#-&4'03&91035:
NZTRM '-64)5"#-&4FWFOU@IJTUPSZ'03&91035
2VFSZ0,SPXTBGGFDUFE
TFD
4. On source: Copy all the table-related files (JCE, DGH) from the EBUB
EJSFDUPSZ directory of the source to the EBUBEJSFDUPSZ of the destination:
TIFMM TVEPTDQJIPNFNZTRMTTIJE@STB
WBSMJCNZTRMUFTUFWFOU@IJTUPSZ1
NZTRM!YYYYYYYYYYYWBSMJCNZTRMUFTU
5. On source: Unlock the table for writes:
NZTRM 6/-0$,5"#-&4
2VFSZ0,SPXTBGGFDUFE
TFD
6. On destination: Make sure that the ownership of the files is set to NZTRM:
TIFMM TVEPMTMIUSWBSMJCNZTRMUFTU
UPUBM.
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
FWFOU@IJTUPSZ1@PDU@XFFLJCE
[ 295 ]

7. On destination: Import the tablespace. You can ignore the warnings as long as
the table definitions are the same. If you have copied the DGH files as well, the
warnings won't occur:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ*.10355"#-&41"$&
TFD
8. On destination: Verify the data:
NZTRM 4&-&$5'30.FWFOU@IJTUPSZ

]FWFOU@JE]FWFOU@OBNF]DSFBUFE@BU]MBTU@VQEBUFE
]FWFOU@UZQF]NTH]

]]UFTU]]
]DMJDL]UFTU@NFTTBHF]
]]UFTU]]
]]UFTU]]
]]UFTU]]
]]UFTU]]
]]UFTU]]
]]UFTU]]
]]UFTU]]

SPXTJOTFU
TFD
If you are doing it on a production system, to minimize downtime, you can copy to the files
locally, which is very fast. Immediately execute 6/-0$,5"#-&4 and then copy the files to
the destination. If you cannot afford downtime, you can use Percona XtraBackup, back up
the single table, and apply the redo logs, which generate the JCE files. You can copy them
to the destination and import.
[ 296 ]

Copying individual partitions of a table

You added a new partition of the FWFOUT@IJTUPSZ table on the source and you wish to
copy only the new partitions to the destination. For the sake of your understanding, create
new partitions on the FWFOUT@IJTUPSZ table and insert a few rows:

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM */4&35*/50FWFOU@IJTUPSZ7"-6&4

UFTU
DMJDL UFTU@NFTTBHF
2VFSZ0,SPXBGGFDUFE
TFD
NZTRM 4&-&$5'30.FWFOU@IJTUPSZ1"35*5*0/

QQ

]FWFOU@JE]FWFOU@OBNF]DSFBUFE@BU]MBTU@VQEBUFE]
FWFOU@UZQF]NTH]

]]UFTU]]]DMJDL
]UFTU@NFTTBHF]
]]UFTU]]]DMJDL
]UFTU@NFTTBHF]

SPXTJOTFU
TFD
Suppose you want to copy the newly created partition to the destination.
1. On destination: Create the partitions:

1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
1"35*5*0/Q7"-6&4-&445)"/
&/(*/&*OOP%#
2VFSZ0,SPXTBGGFDUFE
TFD
2. On destination: Discard only the partitions you want to import:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ%*4$"3%1"35*5*0/Q
Q5"#-&41"$&
2VFSZ0,SPXTBGGFDUFE
TFD
[ 297 ]

3. On source: Execute '-64)5"#-&'03&91035:
NZTRM '-64)5"#-&4FWFOU@IJTUPSZ'03&91035
2VFSZ0,SPXTBGGFDUFE
TFD
4. On source: Copy the JCE files of the partitions to the destination:
TIFMM TVEPTDQJIPNFNZTRMTTIJE@STB=
WBSMJCNZTRMUFTUFWFOU@IJTUPSZ1QJCE=
WBSMJCNZTRMUFTUFWFOU@IJTUPSZ1QJCE=
NZTRM!WBSMJCNZTRMUFTU
FWFOU@IJTUPSZ1QJCE
,#,#TFWFOU@IJTUPSZ1QJCE
,#,#T
5. On destination: Make sure that the JCE files of the required partitions are
copied and have the owner as NZTRM:
TIFMM TVEPMTMIUS
WBSMJCNZTRMUFTUFWFOU@IJTUPSZ1QJCE
SXSNZTRMNZTRM,0DU
TIFMM TVEPMTMIUS
SXSNZTRMNZTRM,0DU
6. On destination: Execute *.10351"35*5*0/5"#-&41"$&:
NZTRM "-5&35"#-&FWFOU@IJTUPSZ*.10351"35*5*0/Q
Q5"#-&41"$&
TFD
You can ignore the warnings as long as the table definitions are the same. If you
have copied the DGH files also, the warning won't occur:
NZTRM 4)088"3/*/(4

]-FWFM]$PEF].FTTBHF
]

]8BSOJOH]]*OOP%#*03FBEFSSPS
/PTVDIGJMFPS
[ 298 ]

EJSFDUPSZ&SSPSPQFOJOH UFTUFWFOU@IJTUPSZ1QDGH

XJMMBUUFNQUUPJNQPSUXJUIPVUTDIFNBWFSJGJDBUJPO]
]8BSOJOH]]*OOP%#*03FBEFSSPS
/PTVDIGJMFPS
EJSFDUPSZ&SSPSPQFOJOH UFTUFWFOU@IJTUPSZ1QDGH
XJMMBUUFNQUUPJNQPSUXJUIPVUTDIFNBWFSJGJDBUJPO]

SPXTJOTFU
TFD
7. On destination: Verify the data:
NZTRM 4&-&$5'30.FWFOU@IJTUPSZ1"35*5*0/

QQ

]FWFOU@JE]FWFOU@OBNF]DSFBUFE@BU]MBTU@VQEBUFE
]FWFOU@UZQF]NTH]

]]UFTU]]
]]UFTU]]

SPXTJOTFU
TFD
See also
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOUBCMFTQBDFDPQZJOHIUNM to find
out more about the limitations of this procedure.
Managing UNDO tablespace

You can manage the size of an 6/%0 tablespace through the dynamic variable
JOOPEC@NBY@VOEP@MPH@TJ[F, which is 1 GB by default, and the number of 6/%0
tablespaces through JOOPEC@VOEP@UBCMFTQBDFT, which is 2 GB by default and dynamic
from MySQL 8.0.2.
[ 299 ]

By default, JOOPEC@VOEP@MPH@USVODBUF is enabled. Tablespaces that exceed the threshold

value defined by JOOPEC@NBY@VOEP@MPH@TJ[F are marked for truncation. Only undo
tablespaces can be truncated. Truncating undo logs that reside in the system tablespace is
not supported. For truncation to occur, there must be at least two undo tablespaces.
How to do it...
Verify the size of the 6/%0 logs:
TIFMM TVEPMTMIUSWBSMJCNZTRMVOEP@
SXSNZTRMNZTRM.0DUWBSMJCNZTRMVOEP@
Suppose you want to reduce the files that are more than 15 MB. Remember, only one undo
tablespace can be truncated. Selection of an undo tablespace for truncation is performed in a
circular fashion to avoid truncating the same undo tablespace every time. After all the
rollback segments in the undo tablespace are freed, the truncate operation runs and the
undo tablespace is truncated to its initial size. The initial size of an undo tablespace file is 10
MB:
1. Make sure that the JOOPEC@VOEP@MPH@USVODBUF is enabled:
NZTRM 4&-&$5!!(-0#"-JOOPEC@VOEP@MPH@USVODBUF

]!!(-0#"-JOOPEC@VOEP@MPH@USVODBUF]

]]

SPXJOTFU
TFD
2. Set JOOPEC@NBY@VOEP@MPH@TJ[F to 15 MB:
NZTRM 4&-&$5!!(-0#"-JOOPEC@NBY@VOEP@MPH@TJ[F

]!!(-0#"-JOOPEC@NBY@VOEP@MPH@TJ[F]

]]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"-JOOPEC@NBY@VOEP@MPH@TJ[F

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5!!(-0#"-JOOPEC@NBY@VOEP@MPH@TJ[F
[ 300 ]


]!!(-0#"-JOOPEC@NBY@VOEP@MPH@TJ[F]

]]

SPXJOTFU
TFD
3. An undo tablespace cannot be truncated until its rollback segments are freed.
Normally, the purge system frees rollback segments once every 128 times that
purge is invoked. To expedite the truncation of undo tablespaces, use
the JOOPEC@QVSHF@STFH@USVODBUF@GSFRVFODZ option to temporarily increase
the frequency with which the purge system frees rollback segments:
NZTRM 4&-&$5!!(-0#"-JOOPEC@QVSHF@STFH@USVODBUF@GSFRVFODZ

]!!(-0#"-JOOPEC@QVSHF@STFH@USVODBUF@GSFRVFODZ]

]]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"-JOOPEC@QVSHF@STFH@USVODBUF@GSFRVFODZ
2VFSZ0,SPXTBGGFDUFE
TFD


]]

SPXJOTFU
TFD
4. Usually on a busy system, at least one purge operation might have initiated and
the truncation would have started. If you are practicing on your machine, you
can initiate the purge by creating a big transaction:
NZTRM #&(*/
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM %&-&5&'30.FNQMPZFFT
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 30--#"$,
2VFSZ0,SPXTBGGFDUFE
TFD
[ 301 ]

5. While the delete is in progress, you can watch the growth of the 6/%0 log files:
You may notice that VOEP@ is truncated to 10 MB while VOEP@ is growing,

accommodating the deleted rows of the %&-&5& statement.
6. After some time, you can notice that VOPE@ is also truncated to 10 MB:
ŝ. Once you've achieved the reduction of the 6/%0 tablespace, set the
JOOPEC@QVSHF@STFH@USVODBUF@GSFRVFODZ to a default of :


]]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"-JOOPEC@QVSHF@STFH@USVODBUF@GSFRVFODZ
2VFSZ0,SPXTBGGFDUFE
TFD
[ 302 ]



]]

SPXJOTFU
TFD
Managing general tablespace

Until MySQL 8, there were two types of tablespaces: system tablespace and individual
tablespace. There are advantages and disadvantages of both types. To overcome the
disadvantages, general tablespaces are introduced in MySQL 8. Similar to system
tablespaces, general tablespaces are shared tablespaces that can store data for multiple
tables. But you have fine control over a general tablespace. Multiple tables in fewer general
tablespaces consume less memory for tablespace metadata than the same number of tables
in separate file-per-table tablespaces.
The limitations are as follows:
Similarly to the system tablespace, truncating or dropping tables stored in a

general tablespace creates free space internally in the general tablespace JCE
data file, which can only be used for new *OOP%# data. Space is not released back
to the operating system as it is for file-per-table tablespaces.
Transportable tablespaces are not supported for tables that belong to a general
tablespace.
In this section, you will learn how to create a general tablespace and add and remove the
tables from it.
Practical usage:
Initially, *OOP%# maintains a GSN file, which contains table structure. MySQL needs to
open and close the GSN file, which degrades performance. With MySQL 8, the GSN files
are removed and all of the metadata is handled using a transactional EBUBEJDUJPOBSZ.
This enables the use of the general tablespace.
Suppose you are using MySQL 5.7 or earlier for SaaS or multi-tenant, where you have a
separate schema for each customer and each customer has hundreds of tables. If your
customers grow, you will notice the performance issues. But with the removal of GSN files
from MySQL 8, the performance is greatly improved. Moreover, you can create a separate
tablespace for each schema (customer).
[ 303 ]

How to do it...
Let's get started with creating it, first.
Create a general tablespace

You can create a general tablespace either in the MySQL EBUBEJSFDUPSZ or outside it.
To create one in a MySQL EBUBEJSFDUPSZ:

NZTRM $3&"5&5"#-&41"$&AUTA"%%%"5"'*-& UTJCE &OHJOF*OOP%#
2VFSZ0,SPXTBGGFDUFE
TFD
To create the tablespace outside it, mount the new disk on WBSMJCNZTRM@HFOFSBM@UT
and change the ownership to NZTRM:
TIFMM TVEPDIPXONZTRMNZTRMWBSMJCNZTRM@HFOFSBM@UT
NZTRM $3&"5&5"#-&41"$&AUTA"%%%"5"'*-&
WBSMJCNZTRM@HFOFSBM@UTUTJCE &OHJOF*OOP%#2VFSZ0,SPXTBGGFDUFE

TFD
Adding tables to a general tablespace

You can add a table to a tablespace while creating it, or you can run the "-5&3 command to
move a table from one tablespace to another:
NZTRM $3&"5&5"#-&FNQMPZFFTUBCMF@HFO@UT
JE*/513*."3:,&:5"#-&41"$&
UT
2VFSZ0,SPXTBGGFDUFE
TFD
Suppose you want to move the FNQMPZFFT table to 5"#-&41"$&UT:

NZTRM 64&FNQMPZFFT
%BUBCBTFDIBOHFE
NZTRM "-5&35"#-&FNQMPZFFT5"#-&41"$&UT
2VFSZ0,SPXTBGGFDUFE
TFD
You can notice the increase in the UTJCE file:

TIFMM TVEPMTMIUSWBSMJCNZTRM@HFOFSBM@UTUTJCE
SXSNZTRMNZTRM.0DU
WBSMJCNZTRM@HFOFSBM@UTUTJCE
[ 304 ]

Moving non-partitioned tables between tablespaces

You can move tables as follows:
1. This is how to move tables from one general tablespace to another.

Suppose you want to move the FNQMPZFFT table from UT to UT:
NZTRM "-5&35"#-&FNQMPZFFT5"#-&41"$&UT
2VFSZ0,SPXTBGGFDUFE
TFD
TIFMM TVEPMTMIUSWBSMJCNZTRMUTJCE
SXSNZTRMNZTRM.0DUWBSMJCNZTRMUTJCE
2. This is how to move tables to file-per-table.

Suppose you want to move the FNQMPZFFT table from UT to file per table:
NZTRM "-5&35"#-&FNQMPZFFT5"#-&41"$&JOOPEC@GJMF@QFS@UBCMF
2VFSZ0,SPXTBGGFDUFE
TFD
TIFMM TVEPMTMIUSWBSMJCNZTRMFNQMPZFFTFNQMPZFFTJCE
SXSNZTRMNZTRM.0DU
WBSMJCNZTRMFNQMPZFFTFNQMPZFFTJCE
3. This is how to move tables to the system tablespace.

Suppose you want to move the FNQMPZFFT table from file per table to the system
tablespace:
NZTRM "-5&35"#-&FNQMPZFFT5"#-&41"$&JOOPEC@TZTUFN
2VFSZ0,SPXTBGGFDUFE
TFD
Managing partitioned tables in a general tablespace

You can create a table with partitions in multiple tablespaces:
NZTRM $3&"5&5"#-&UBCMF@HFO@QBSU@UT
JE*/5WBMVFWBSDIBS
&/(*/&
*OOP%#
1"35*5*0/#:3"/(&
JE
1"35*5*0/Q7"-6&4-&445)"/
5"#-&41"$&UT
1"35*5*0/Q7"-6&4-&445)"/
5"#-&41"$&UT
1"35*5*0/Q7"-6&4-&445)"/
5"#-&41"$&
JOOPEC@GJMF@QFS@UBCMF
1"35*5*0/QNBY7"-6&4-&445)"/
."97"-6&5"#-&41"$&
[ 305 ]

JOOPEC@TZTUFN
2VFSZ0,SPXTBGGFDUFE
TFD
You can add a new partition in another tablespace, or if you do not mention anything, it
will be created in table's default tablespace. An "-5&35"#-&UCM@OBNF5"#-&41"$&
UBCMFTQBDF@OBNF operation on a partitioned table only modifies the table's default
tablespace. It does not move the table partitions. However, after changing the default
tablespace, an operation that rebuilds the table (such as an "-5&35"#-& operation that
uses "-(03*5).$01:) moves the partitions to the default tablespace if another tablespace
is not defined explicitly using the 5"#-&41"$& clause.
If you wish to move the partitions across a tablespace, you need to do 3&03("/*;& on the
partition. For example, you want to move partition Q to UT:
NZTRM "-5&35"#-&UBCMF@HFO@QBSU@UT3&03("/*;&1"35*5*0/Q*/50

1"35*5*0/Q7"-6&4-&445)"/
5"#-&41"$&UT
Dropping general tablespace

You can use the %3015"#-&41"$& command to drop the tablespace. However, all the
tables inside that tablespace should be either dropped or moved:
NZTRM %3015"#-&41"$&UT
&3303
):5BCMFTQBDFAUTAJTOPUFNQUZ
You have to move the partitions Q and Q of table UBCMF@HFO@QBSU@UT in the UT
tablespace to other tablespace before dropping:

1"35*5*0/Q7"-6&4-&445)"/
5"#-&41"$&UT

1"35*5*0/Q7"-6&4-&445)"/
5"#-&41"$&UT
Now you can drop the tablespace:

NZTRM %3015"#-&41"$&UT
2VFSZ0,SPXTBGGFDUFE
TFD
[ 306 ]

Compressing InnoDB tables

You can create tables where the data is stored in compressed form. Compression can help to
improve both raw performance and scalability. Compression means less data is transferred
between disk and memory, and it takes up less space on disk and in memory.
As per the MySQL documentation:
"Because processors and cache memories have increased in speed more than disk storage devices,
many workloads are disk-bound. Data compression enables smaller database size, reduced I/O, and
improved throughput, at the small cost of increased CPU utilization. Compression is especially
valuable for read-intensive applications, on systems with enough RAM to keep frequently used data
in memory. The benefits are amplified for tables with secondary indexes, because index data is
compressed also."
To enable compression, you need to create or alter the table with

the 308@'03."5$0.13&44&%,&:@#-0$,@4*;& option. You can vary the
,&:@#-0$,@4*;& parameter, which uses a smaller page size on disk than the configured
JOOPEC@QBHF@TJ[F value. Compression won't work if the table is in system tablespace.
To create a compressed table in a general tablespace, '*-&@#-0$,@4*;& must be defined

for the general tablespace, which is specified when the tablespace is created. The
'*-&@#-0$,@4*;& value must be a valid compressed page size in relation to the
JOOPEC@QBHF@TJ[F value, and the page size of the compressed table, defined by the
$3&"5&5"#-& or "-5&35"#-&,&:@#-0$,@4*;& clause, must be equal to
'*-&@#-0$,@4*;&.
In the buffer pool, the compressed data is held in small pages, with a page size based on the
,&:@#-0$,@4*;& value. For extracting or updating column values, MySQL also creates an
uncompressed page in the buffer pool with uncompressed data. Within the buffer pool, any
updates to the uncompressed page are also rewritten back to the equivalent compressed
page. You might need to size your buffer pool to accommodate the additional data of both
compressed and uncompressed pages, although uncompressed pages are evicted from the
buffer pool when space is needed and then uncompressed again on the next access.
When to use compression?
In general, compression works best on tables that include a reasonable number of character
string columns and where data is read far more often than it is written. Because there are no
guaranteed ways to predict whether or not compression benefits a particular situation,
always test with a specific workload and dataset running on a representative configuration.
[ 307 ]

How to do it...
You need to choose the parameter ,&:@#-0$,@4*;&. JOOPEC@QBHF@TJ[F is 16,000; ideally,
half of that is 8,000, which is a good start. To tune compression, refer to IUUQTEFW
NZTRMDPNEPDSFGNBOFOJOOPECDPNQSFTTJPOUVOJOHIUNM.
Enabling Compression for file_per_table Tables

1. Make sure that GJMF@QFS@UBCMF is enabled:
NZTRM 4&5(-0#"-JOOPEC@GJMF@QFS@UBCMF
2. Specify the 308@'03."5$0.13&44&%,&:@#-0$,@4*;& in the create

statement:
NZTRM $3&"5&5"#-&DPNQSFTTFE@UBCMF
JE*/513*."3:,&:
308@'03."5$0.13&44&%,&:@#-0$,@4*;&
2VFSZ0,SPXTBGGFDUFE
TFD
If the table already exists, you can execute "-5&3:

NZTRM "-5&35"#-&FWFOU@IJTUPSZ308@'03."5$0.13&44&%,&:@#-0$,@4*;&
2VFSZ0,SPXTBGGFDUFE
TFD
If you try to compress a table that is in the system tablespace, you will get an error:
NZTRM "-5&35"#-&FNQMPZFFT308@'03."5$0.13&44&%,&:@#-0$,@4*;&
&3303
):*OOP%#5BCMFTQBDFAJOOPEC@TZTUFNADBOOPUDPOUBJOB
$0.13&44&%UBCMF
Disabling Compression for file_per_table Tables

To disable compression, execute the "-5&3 table and specify 308@'03."5%:/".*$ or
308@'03."5$0.1"$5, followed by ,&:@#-0$,@4*;&.
For example, if you do not want compression on the FWFOU@IJTUPSZ table:

NZTRM "-5&35"#-&FWFOU@IJTUPSZ308@'03."5%:/".*$,&:@#-0$,@4*;&
2VFSZ0,SPXTBGGFDUFE
TFD
[ 308 ]

Enabling Compression for General Tablespace

First, you need to create a compressed tablespace by mentioning '*-&@#-0$,@4*;&; you
cannot alter the tablespace's '*-&@#-0$,@4*;&.
If you wish to create a compressed table, you need to create the table in the general
tablespace, where the compression is enabled; moreover ,&:@#-0$,@4*;& must be equal to
'*-&@#-0$,@4*;&. If you do not mention ,&:@#-0$,@4*;&, the value will be
automatically taken from '*-&@#-0$,@4*;&.
You can create multiple compressed general tablespaces with different

'*-&@#-0$,@4*;& values and just add the tables to the desired tablespace:
1. Create a general compressed tablespace. You can create one with

'*-&@#-0$,@4*;& of 8k and other with '*-&@#-0$,@4*;& of 4k, and move all
the tables with ,&:@#-0$,@4*;& of 8 to 8k and 4 to 4k:
NZTRM $3&"5&5"#-&41"$&AUT@LA"%%%"5"'*-& UT@LJCE

'*-&@#-0$,@4*;&&OHJOF*OOP%#
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM $3&"5&5"#-&41"$&AUT@LA"%%%"5"'*-& UT@LJCE

'*-&@#-0$,@4*;&&OHJOF*OOP%#
2VFSZ0,SPXTBGGFDUFE
TFD
2. Create compressed tables in those tablespaces by mentioning the

308@'03."5$0.13&44&%:
NZTRM $3&"5&5"#-&DPNQSFTT@UBCMF@@L
JE*/513*."3:,&:
5"#-&41"$&UT@L308@'03."5$0.13&44&%
2VFSZ0,SPXTBGGFDUFE
TFD
If you do not mention 308@'03."5$0.13&44&% you will get an error:

JE*/513*."3:,&:
5"#-&41"$&UT@L
&3303
):*OOP%#5BCMFTQBDFAUT@LAVTFTCMPDLTJ[F
BOEDBOOPUDPOUBJOBUBCMFXJUIQIZTJDBMQBHFTJ[F
Optionally, you can mention the ,&:@#-0$,@4*;&'*-&@#-0$,@4*;&:

NZTRM $3&"5&5"#-&DPNQSFTT@UBCMF@L
JE*/513*."3:,&:
5"#-&41"$&UT@L308@'03."5$0.13&44&%,&:@#-0$,@4*;&
2VFSZ0,SPXTBGGFDUFE
TFD
[ 309 ]

If you mention anything other than '*-&@#-0$,@4*;&, you will get an

error:
JE*/513*."3:,&:
5"#-&41"$&UT@L308@'03."5$0.13&44&%,&:@#-0$,@4*;&
&3303
3. You can move the tables from the GJMF@QFS@UBCMF tablespace to the
compressed general tablespace only if the ,&:@#-0$,@4*;& matches. Otherwise,
you will get an error:
NZTRM $3&"5&5"#-&DPNQSFTT@UBCMFT@L
JE*/513*."3:,&:
5"#-&41"$&JOOPEC@GJMF@QFS@UBCMF308@'03."5$0.13&44&%
,&:@#-0$,@4*;&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM "-5&35"#-&DPNQSFTT@UBCMFT@L5"#-&41"$&UT@L
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM "-5&35"#-&DPNQSFTT@UBCMFT@L5"#-&41"$&UT@L
&3303
[ 310 ]

Managing Logs
12
Managing the error log

Managing the general query log and slow query log
Managing the binary logs
Introduction
You will now learn about managing different type of logs: error log, general query log, slow
query log, binary logs, relay logs, and DDL logs.
Managing the error log

The error log contains a record of NZTRME startup and shutdown times. It also contains
diagnostic messages such as errors, warnings, and notes that occur during server startup
and shutdown, and while the server is running.

Managing Logs Chapter 12
The error log subsystem consists of components that perform log event filtering and
writing, as well as a system variable called MPH@FSSPS@TFSWJDFT that configures which
components to enable to achieve the desired logging result. The default value of
HMPCBMMPH@FSSPS@TFSWJDFT is MPH@GJMUFS@JOUFSOBMMPH@TJOL@JOUFSOBM:
NZTRM 4&-&$5!!HMPCBMMPH@FSSPS@TFSWJDFT

]!!HMPCBMMPH@FSSPS@TFSWJDFT]

]MPH@GJMUFS@JOUFSOBMMPH@TJOL@JOUFSOBM]

That value indicates that log events first pass through the built-in filter component,
MPH@GJMUFS@JOUFSOBM, then through the built-in log writer component,
MPH@TJOL@JOUFSOBM. Component order is significant because the server executes
components in the order listed. Any loadable (not built in) component named in the
MPH@FSSPS@TFSWJDFT value must first be installed with */45"--$0.10/&/5 which will
be described in this section.
To know about all types of error logging, refer to IUUQTEFWNZTRMDPNEPDSFGNBO

FOFSSPSMPHIUNM
How to do it...
Error logs are easy, in a way. Let's see how to configure an error log first.
Configuring the error log

The error logging is controlled by the MPH@FSSPS variable (MPHFSSPS for a startup
script).
If MPHFSSPS is not given, the default destination is the console.

If MPHFSSPS is given without naming a file, the default destination is a file named
IPTU@OBNFFSS in the EBUBEJSFDUPSZ .
If MPHFSSPS is given to name a file, the default destination is that file (with an FSS
suffix added if the name has no suffix), located under the EBUBEJSFDUPSZ unless an
absolute path name is given to specify a different location.
[ 312 ]

The MPH@FSSPS@WFSCPTJUZ system variable controls server verbosity for writing error,
warning, and note messages to the error log. Permitted MPH@FSSPS@WFSCPTJUZ values are
(errors only), (errors and warnings), and (errors, warnings, and notes), with a default
of .
To change the error log location, edit the configuration file and restart MySQL:
TIFMM TVEPNLEJSWBSMPHNZTRM
TIFMM TVEPDIPXO3NZTRMNZTRMWBSMPHNZTRM
<NZTRME>
MPHFSSPSWBSMPHNZTRMNZTRMEMPH
Verify the error log:

NZTRM 4)087"3*"#-&4-*,& MPH@FSSPS


]MPH@FSSPS]WBSMPHNZTRMNZTRMEMPH]

SPXJOTFU
TFD
To adjust the verbosity, you can change the MPH@FSSPS@WFSCPTJUZ variable dynamically.
However, it is recommended to keep the default value of so that error, warning, and note
messages are logged:
NZTRM 4&5!!(-0#"-MPH@FSSPS@WFSCPTJUZ
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5!!(-0#"-MPH@FSSPS@WFSCPTJUZ

]!!(-0#"-MPH@FSSPS@WFSCPTJUZ]

]]

SPXJOTFU
TFD
[ 313 ]

Rotating the error log

Suppose the error log file has become bigger and you want to rotate it; you can simply
move the file and execute the '-64)-0(4 command:
TIFMM TVEPNWWBSMPHNZTRMNZTRMEMPHWBSMPHNZTRMNZTRMEMPH
TIFMM NZTRMBENJOVSPPUQQBTTXPSE GMVTIMPHT

NZTRMBENJO<8BSOJOH>6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBOCF
JOTFDVSF
TIFMM MTMIUSWBSMPHNZTRMNZTRMEMPH
SXSNZTRMNZTRM0DUWBSMPHNZTRMNZTRMEMPH
TIFMM MTMIUSWBSMPHNZTRMNZTRMEMPH
SXSNZTRMNZTRM,0DUWBSMPHNZTRMNZTRMEMPH
You can automate the preceding steps using some scripts and put them into DSPO.
If the location of an error log file is not writable by the server, the log-flushing operation
fails to create a new log file:
TIFMM TVEPNWWBSMPHNZTRMEMPHWBSMPHNZTRMEMPHNZTRMBENJO
GMVTIMPHTVSPPUQQBTTXPSE
NZTRMBENJO<8BSOJOH>6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBOCF
JOTFDVSF
NZTRMBENJOSFGSFTIGBJMFEFSSPS 6OLOPXOFSSPS
Using the system log for logging

To use the system log for logging, you need to load the system log writer called
MPH@TJOL@TZTFWFOUMPH. You can use the built-in filter, MPH@GJMUFS@JOUFSOBM, for
filtering:
1. Load the system log writer:
NZTRM */45"--$0.10/&/5 GJMFDPNQPOFOU@MPH@TJOL@TZTFWFOUMPH

2VFSZ0,SPXTBGGFDUFE
TFD
[ 314 ]

2. Make it persistent across restarts:
NZTRM 4&51&34*45MPH@FSSPS@TFSWJDFT MPH@GJMUFS@JOUFSOBM

MPH@TJOL@TZTFWFOUMPH
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)087"3*"#-&4-*,& MPH@FSSPS@TFSWJDFT


]MPH@FSSPS@TFSWJDFT]MPH@GJMUFS@JOUFSOBMMPH@TJOL@TZTFWFOUMPH]

SPXJOTFU
TFD
3. You can verify that the logs will be directed to the syslog. On CentOS and Red
Hat, you can check in WBSMPHNFTTBHFT; on Ubuntu, you can check in
WBSMPHTZTMPH.
For the sake of a demo, the server was restarted. You can see those logs in the
syslog:
TIFMM TVEPHSFQNZTRMEWBSMPHNFTTBHFT]UBJM
0DUDFOUPTNZTRME<>*OOP%##VGGFSQPPM
TEVNQ
DPNQMFUFEBU
0DUDFOUPTNZTRME<>*OOP%#4IVUEPXODPNQMFUFE
MPHTFRVFODFOVNCFS
0DUDFOUPTNZTRME<>*OOP%#3FNPWFEUFNQPSBSZ
UBCMFTQBDFEBUBGJMFJCUNQ
0DUDFOUPTNZTRME<>4IVUUJOHEPXOQMVHJO
.&.03:
0DUDFOUPTNZTRME<>4IVUUJOHEPXOQMVHJO $47
TIB@QBTTXPSE
NZTRM@OBUJWF@QBTTXPSE
CJOMPH
0DUDFOUPTNZTRME<>VTSTCJONZTRME4IVUEPXO
DPNQMFUF
0DUDFOUPTNZTRME<>VTSTCJONZTRMESFBEZGPS
DPOOFDUJPOT7FSTJPO SDMPH TPDLFU
WBSMJCNZTRMNZTRMTPDL QPSU.Z42-$PNNVOJUZ4FSWFS

(1-
[ 315 ]

If you have multiple NZTRME processes running, you can differentiate using the PID
specified in <>. Otherwise, you can set the MPH@TZTMPH@UBH variable, which appends the
server identifier with a leading hyphen, resulting in an identifier of NZTRMEUBH@WBM.
For example, you can tag an instance with something like JOTUBODF:
NZTRM 4&-&$5!!(-0#"-MPH@TZTMPH@UBH

]!!(-0#"-MPH@TZTMPH@UBH]

]]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"-MPH@TZTMPH@UBH JOTUBODF

2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5!!(-0#"-MPH@TZTMPH@UBH

]!!(-0#"-MPH@TZTMPH@UBH]

]JOTUBODF]

SPXJOTFU
TFD
TIFMM TVEPHSFQNZTRMEWBSMPHNFTTBHFT]UBJM
0DUDFOUPTNZTRMEJOTUBODF<>*OOP%##VGGFSQPPM
T
EVNQDPNQMFUFEBU
0DUDFOUPTNZTRMEJOTUBODF<>*OOP%#4IVUEPXO
DPNQMFUFEMPHTFRVFODFOVNCFS
0DUDFOUPTNZTRMEJOTUBODF<>*OOP%#3FNPWFEUFNQPSBSZ
UBCMFTQBDFEBUBGJMFJCUNQ
0DUDFOUPTNZTRMEJOTUBODF<>4IVUUJOHEPXOQMVHJO
.&.03:
0DUDFOUPTNZTRMEJOTUBODF<>4IVUUJOHEPXOQMVHJO $47
TIB@QBTTXPSE
NZTRM@OBUJWF@QBTTXPSE
CJOMPH
0DUDFOUPTNZTRMEJOTUBODF<>VTSTCJONZTRME4IVUEPXO
DPNQMFUF
0DUDFOUPTNZTRME<>VTSTCJONZTRMESFBEZGPS
DPOOFDUJPOT7FSTJPO SDMPH TPDLFU WBSMJCNZTRMNZTRMTPDL
QPSU.Z42-$PNNVOJUZ4FSWFS
(1-
[ 316 ]

You will notice that the JOTUBODF tag is appended to the log so that you can easily
identify between multiple instances.
If you wish to switch back to the original logging, you can set MPH@FSSPS@TFSWJDFT to
MPH@GJMUFS@JOUFSOBMMPH@TJOL@JOUFSOBM :
NZTRM 4&5!!HMPCBMMPH@FSSPS@TFSWJDFT MPH@GJMUFS@JOUFSOBM

MPH@TJOL@JOUFSOBM
2VFSZ0,SPXTBGGFDUFE
TFD
Error logging in JSON format

To use the JSON format for logging, you need to load the JSON log writer called
MPH@TJOL@KTPO. You can use the built-in filter, MPH@GJMUFS@JOUFSOBM, for filtering:
1. Install the JSON log writer:
NZTRM */45"--$0.10/&/5 GJMFDPNQPOFOU@MPH@TJOL@KTPO

2VFSZ0,SPXTBGGFDUFE
TFD
2. Make it persistent across restarts:
NZTRM 4&51&34*45MPH@FSSPS@TFSWJDFT MPH@GJMUFS@JOUFSOBM

MPH@TJOL@KTPO
2VFSZ0,SPXTBGGFDUFE
TFD
3. The JSON log writer determines its output destination based on the default error
log destination, which is given by the MPH@FSSPS system variable:
NZTRM 4)087"3*"#-&4-*,& MPH@FSSPS


]MPH@FSSPS]WBSMPHNZTRMNZTRMEMPH]

SPXJOTFU
TFD
[ 317 ]

4. The log will be something like NZTRMEMPHKTPO.

After a restart, the JSON log file looks like this:
TIFMM TVEPMFTTWBSMPHNZTRMNZTRMEMPHKTPO
\QSJPFSS@DPEFTVCTZTUFN42-@TUBUF
):TPVSDF@GJMFTRM@QMVHJODDGVODUJPO
SFBQ@QMVHJOTNTH4IVUUJOHEPXOQMVHJO TIB@QBTTXPSE
UJNF5;FSS@TZNCPM
&3@1-6(*/@4)655*/(@%08/@1-6(*/MBCFM/PUF^
SFBQ@QMVHJOTNTH4IVUUJOHEPXOQMVHJO
NZTRM@OBUJWF@QBTTXPSE UJNF5;
FSS@TZNCPM&3@1-6(*/@4)655*/(@%08/@1-6(*/MBCFM/PUF^
SFBQ@QMVHJOTNTH4IVUUJOHEPXOQMVHJO CJOMPH UJNF
5;FSS@TZNCPM
&3@1-6(*/@4)655*/(@%08/@1-6(*/MBCFM/PUF^
):TPVSDF@GJMFNZTRMEDDGVODUJPODMFBO@VQ
NTHVTSTCJONZTRME4IVUEPXODPNQMFUF=VBUJNF
5;FSS@TZNCPM
&3@4)65%08/@$0.1-&5&MBCFM/PUF^
\MPH@UZQFQSJPFSS@DPEFNTH
VTSTCJONZTRMESFBEZGPSDPOOFDUJPOT7FSTJPO SDMPH
TPDLFU WBSMJCNZTRMNZTRMTPDL QPSU.Z42-$PNNVOJUZ
4FSWFS
(1-UJNF5;FSS@TZNCPM
&3@45"356142-@TUBUF):MBCFM/PUF^
If you wish to switch back to original logging, you can set MPH@FSSPS@TFSWJDFT to
MPH@GJMUFS@JOUFSOBMMPH@TJOL@JOUFSOBM :
NZTRM 4&5!!HMPCBMMPH@FSSPS@TFSWJDFT MPH@GJMUFS@JOUFSOBM

MPH@TJOL@JOUFSOBM
2VFSZ0,SPXTBGGFDUFE
TFD
To know more about the error logging configuration, refer to IUUQTEFWNZTRMDPNEPD

SFGNBOFOFSSPSMPHDPNQPOFOUDPOGJHVSBUJPOIUNM.
[ 318 ]

Managing the general query log and slow

query log
There are two ways you can log the queries. One way is through general query log, and
other way is through slow query log. In this section, you will learn about configuring them.
How to do it...
We will get into the details in the following subsections.
General query log

The general query log is a general record of what NZTRME is doing. The server writes
information to this log when clients connect or disconnect, and it logs each SQL statement
received from clients. The general query log can be very useful when you suspect an error
in a client and want to know exactly what the client sent to NZTRME:
1. Specify the file for logging. If you do not specify, it will be created in the
EBUBEJSFDUPSZ with the name IPTUOBNFMPH.
The server creates the file in the EBUBEJSFDUPSZ unless an absolute path name
is given to specify a different directory:
NZTRM 4&5
!!(-0#"-HFOFSBM@MPH@GJMF WBSMPHNZTRMHFOFSBM@RVFSZ@MPH
2VFSZ0,SPXTBGGFDUFE
TFD
2. Enable the general query log:
NZTRM 4&5(-0#"-HFOFSBM@MPH 0/
2VFSZ0,SPXTBGGFDUFE
TFD
[ 319 ]

3. You can see that the queries are logged:
TIFMM TVEPDBUWBSMPHNZTRMHFOFSBM@RVFSZ@MPH
VTSTCJONZTRME7FSTJPOSDMPH
.Z42-$PNNVOJUZ4FSWFS

(1-TUBSUFEXJUI
5DQQPSU6OJYTPDLFUWBSMJCNZTRMNZTRMTPDL
5JNF*E$PNNBOE"SHVNFOU
2017-10-11T04:21:00.118944Z 220 Connect root@localhost on
using Socket
2017-10-11T04:21:00.119212Z 220 Query select
@@version_comment limit 1
2017-10-11T04:21:03.217603Z 220 Query SELECT DATABASE()
5;*OJU%#FNQMPZFFT
5;2VFSZTIPXEBUBCBTFT
5;2VFSZTIPXUBCMFT
5;'JFME-JTUDVSSFOU@EFQU@FNQ
5;'JFME-JTUEFQBSUNFOUT
5;'JFME-JTUEFQU@FNQ
5;'JFME-JTU
EFQU@FNQ@MBUFTU@EBUF
5;'JFME-JTUEFQU@NBOBHFS
5;'JFME-JTUFNQMPZFF@OBNFT
5;'JFME-JTUFNQMPZFFT
5;'JFME-JTUUJUMFT
5;'JFME-JTUUJUMFT@POMZ
_
_
2017-10-11T04:21:09.483117Z 220 Query select count(*) from
employees
5;2VJU
General query log generates a very big log file. Be very cautious when enabling it on a
production server. It drastically affects the server's performance.
Slow query log

"The slow query log consists of SQL statements that took more than long_query_time
seconds to execute and required at least min_examined_row_limit rows to be examined."
To log all the queries, you can set the value of MPOH@RVFSZ@UJNF to . The default value of
MPOH@RVFSZ@UJNF is seconds and NJO@FYBNJOFE@SPX@MJNJU is .
[ 320 ]

By default, queries that do not use indexes for lookups and administrative statements (such
as "-5&35"#-&, "/"-:;&5"#-&, $)&$,5"#-&, $3&"5&*/%&9, %301*/%&9,
015*.*;&5"#-&, and 3&1"*35"#-&) are not logged. This behavior can be changed using
MPH@TMPX@BENJO@TUBUFNFOUT and MPH@RVFSJFT@OPU@VTJOH@JOEFYFT.
To enable slow query log, you can dynamically set TMPX@RVFSZ@MPH and you can set the
filename using TMPX@RVFSZ@MPH@GJMF. To specify the log destination, use MPHPVUQVU:
1. Verify MPOH@RVFSZ@UJNF and adjust it as per your requirement:
NZTRM 4&-&$5!!(-0#"--0/(@26&3:@5*.&

]!!(-0#"--0/(@26&3:@5*.&]

]]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"--0/(@26&3:@5*.&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5!!(-0#"--0/(@26&3:@5*.&

]!!(-0#"--0/(@26&3:@5*.&]

]]

SPXJOTFU
TFD
2. Verify the slow query file. By default, it would be in the EBUBEJSFDUPSZ with
the IPTUOBNFTMPX log:
NZTRM 4&-&$5!!(-0#"-TMPX@RVFSZ@MPH@GJMF

]!!(-0#"-TMPX@RVFSZ@MPH@GJMF]

]WBSMJCNZTRMTFSWFSTMPXMPH]

SPXJOTFU
TFD
NZTRM 4&5
!!(-0#"-TMPX@RVFSZ@MPH@GJMF WBSMPHNZTRMNZTRM@TMPXMPH
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5!!(-0#"-TMPX@RVFSZ@MPH@GJMF

[ 321 ]

]!!(-0#"-TMPX@RVFSZ@MPH@GJMF]

]WBSMPHNZTRMNZTRM@TMPXMPH]

SPXJOTFU
TFD
NZTRM '-64)-0(4
2VFSZ0,SPXTBGGFDUFE
TFD
3. Enable the slow query log:
NZTRM 4&-&$5!!(-0#"-TMPX@RVFSZ@MPH

]!!(-0#"-TMPX@RVFSZ@MPH]

]]

SPXJOTFU
TFD
NZTRM 4&5!!(-0#"-TMPX@RVFSZ@MPH
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5!!(-0#"-TMPX@RVFSZ@MPH

]!!(-0#"-TMPX@RVFSZ@MPH]

]]

SPXJOTFU
TFD
4. Verify that the queries are logged (You have to execute few long running queries
to see them in slow query log):
NZTRM 4&-&$54-&&1

]4-&&1
]

]]

SPXJOTFU
TFD
TIFMM TVEPMFTTWBSMPHNZTRMNZTRM@TMPXMPH
VTSTCJONZTRME7FSTJPOSDMPH
.Z42-$PNNVOJUZ4FSWFS

(1-TUBSUFEXJUI
5DQQPSU6OJYTPDLFUWBSMJCNZTRMNZTRMTPDL
5JNF*E$PNNBOE"SHVNFOU
[ 322 ]

5JNF5;
6TFS!)PTUSPPU<SPPU>!MPDBMIPTU<>*E
2VFSZ@UJNF-PDL@UJNF3PXT@TFOU
3PXT@FYBNJOFE
4&5UJNFTUBNQ
4&-&$54-&&1

Selecting query log output destinations

You can log the queries to either '*-& or 5"#-& in MySQL itself by specifying
the MPH@PVUQVU variable, which can be either '*-& or 5"#-&, or both '*-& and 5"#-&.
If you specify MPH@PVUQVU as '*-&, the general query log and the slow query log will be
written to the files specified by HFOFSBM@MPH@GJMF and TMPX@RVFSZ@MPH@GJMF,
respectively.
If you specify MPH@PVUQVU as 5"#-&, the general query log and the slow query log will be
written to the NZTRMHFOFSBM@MPH and NZTRMTMPX@MPH tables respectively. Log contents
are accessible through SQL statements.
For example:
NZTRM 4&5!!(-0#"-MPH@PVUQVU 5"#-&
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&5!!(-0#"-HFOFSBM@MPH 0/
2VFSZ0,SPXTBGGFDUFE
TFD
Execute a few queries and then query the NZTRMHFOFSBM@MPH table:

NZTRM 4&-&$5'30.NZTRMHFOFSBM@MPH8)&3&DPNNBOE@UZQF 2VFSZ =(
_
_
SPX
FWFOU@UJNF
VTFS@IPTUSPPU<SPPU>!MPDBMIPTU<>
UISFBE@JE
TFSWFS@JE
DPNNBOE@UZQF2VFSZ
BSHVNFOUTIPXEBUBCBTFT
SPX
FWFOU@UJNF
UISFBE@JE
TFSWFS@JE
[ 323 ]

DPNNBOE@UZQF2VFSZ
BSHVNFOUTIPXUBCMFT
SPX
FWFOU@UJNF
UISFBE@JE
TFSWFS@JE
DPNNBOE@UZQF2VFSZ
BSHVNFOUTFMFDUGSPNTBMBSJFTMJNJU
SPX
FWFOU@UJNF
UISFBE@JE
TFSWFS@JE
DPNNBOE@UZQF2VFSZ
BSHVNFOU4&-&$5'30.NZTRMHFOFSBM@MPH8)&3&DPNNBOE@UZQF 2VFSZ
You can use the TMPX@MPH table in a similar way:

NZTRM 4&5!!(-0#"-TMPX@RVFSZ@MPH
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&5!!(-0#"-MPOH@RVFSZ@UJNF
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$54-&&1

]4-&&1
]

]]

SPXJOTFU
TFD
NZTRM 4&-&$5'30.NZTRMTMPX@MPH=(

SPX
TUBSU@UJNF
RVFSZ@UJNF
MPDL@UJNF
SPXT@TFOU
SPXT@FYBNJOFE
ECFNQMPZFFT
MBTU@JOTFSU@JE
JOTFSU@JE
TFSWFS@JE
TRM@UFYU4&-&$54-&&1

UISFBE@JE
SPXJOTFU
TFD
[ 324 ]

If the slow query log table has become huge, you can rotate it by creating a new table and
swapping it:
1. Create a new table, NZTRMHFOFSBM@MPH@OFX:
NZTRM %3015"#-&*'&9*454NZTRMHFOFSBM@MPH@OFX
TFD
NZTRM $3&"5&5"#-&NZTRMHFOFSBM@MPH@OFX-*,&NZTRMHFOFSBM@MPH
2VFSZ0,SPXTBGGFDUFE
TFD
2. Swap the tables using the 3&/".&5"#-& command:
NZTRM 3&/".&5"#-&NZTRMHFOFSBM@MPH50NZTRMHFOFSBM@MPH@
NZTRMHFOFSBM@MPH@OFX50NZTRMHFOFSBM@MPH
2VFSZ0,SPXTBGGFDUFE
TFD
Managing the binary logs

In this section, managing the binary logs in a replication environment is covered. Basic
binary log handling is already covered in $IBQUFS, Binary Logging, using the 163(&
#*/"3:-0(4 command and FYQJSF@MPHT@EBZT variable.
Using those methods is unsafe in a replication environment because if any one of the slaves
has not consumed the binary logs and you have deleted them, the slave will go out of sync
and you'll need to rebuild it.
The safe way to delete the binary logs is by checking which binary logs have been read on
each slave and deleting them. You can use the NZTRMCJOMPHQVSHF utility to achieve this.
How to do it...
Execute the NZTRMCJOMPHQVSHF script on any of the servers and specify the master and
slave hosts. The script connects to all the slaves and finds out the latest binary log applied.
Then it purges the master binary logs until that point. You need a superuser to connect to
all slaves:
1. Connect to any of the servers and execute the NZTRMCJOMPHQVSHF script:
TIFMM NZTRMCJOMPHQVSHFNBTUFSECBENJOQBTT !NBTUFS

TMBWFTECBENJOQBTT !TMBWFECBENJOQBTT !TMBWF
[ 325 ]

NZTRM 4)08#*/"3:-0(4


]NBTUFSCJO]]
_
]NBTUFSCJO]]
]NBTUFSCJO]]
]NBTUFSCJO]]
]NBTUFSCJO]]
]NBTUFSCJO]]
]NBTUFSCJO]]

SPXTJOTFU
TFD
TIFMM mysqlbinlogpurge --master=dbadmin:<pass>@master:3306 --

slaves=dbadmin:<pass>@slave1:3306,dbadmin:<pass>@slave2:3306
-BUFTUCJOMPHGJMFSFQMJDBUFECZBMMTMBWFTNBTUFSCJO
1VSHJOHCJOBSZMPHTQSJPSUP NBTUFSCJO
2. If you wish to discover all the slaves without specifying in the command, you
should set SFQPSU@IPTU and SFQPSU@QPSU on all slaves and restart MySQL
server. On each slave:
<NZTRME>
SFQPSUIPTUTMBWF
SFQPSUQPSU
NZTRM 4)087"3*"#-&4-*,& SFQPSU


]SFQPSU@IPTU]TMBWF]
]SFQPSU@QBTTXPSE]]
]SFQPSU@QPSU]]
]SFQPSU@VTFS]]

SPXTJOTFU
TFD
[ 326 ]

3. Execute NZTRMCJOMPHQVSHF with the EJTDPWFSTMBWFTMPHJO option:
NZTRM 4)08#*/"3:-0(4


]DFOUPTCJO]]
]DFOUPTCJO]]
]DFOUPTCJO]]
]DFOUPTCJO]]

SPXTJOTFU
TFD
TIFMM NZTRMCJOMPHQVSHFNBTUFSECBENJOQBTT !NBTUFSEJTDPWFS

TMBWFTMPHJOECBENJOQBTT
%JTDPWFSJOHTMBWFTGPSNBTUFSBUNBTUFS
%JTDPWFSJOHTMBWFBUTMBWF
'PVOETMBWFTMBWF
%JTDPWFSJOHTMBWFBUTMBWF
'PVOETMBWFTMBWF
-BUFTUCJOMPHGJMFSFQMJDBUFECZBMMTMBWFTNBTUFSCJO
1VSHJOHCJOBSZMPHTQSJPSUP NBTUFSCJO
[ 327 ]

Performance Tuning
13
The explain plan

Benchmarking queries and the server
Adding indexes
Invisible index
Descending index
Analyzing slow queries using pt-query-digest
Optimizing datatypes
Removing duplicate and redundant indexes
Checking index usage
Controlling the query optimizer
Using index hints
Indexing for JSON using generated columns
Using resource groups
Using performance_schema
Using the sys schema

Performance Tuning Chapter 13
Introduction
This chapter will take you through query and schema tuning. The database is meant for the
execution of queries; making it run faster is the end goal of tuning. The database's
performance depends on many factors, mainly queries, schema, configuration settings, and
hardware.
In this chapter, we are going to take the employees database to explain all the examples.
You might have transformed the employees database in many ways in the preceding
chapters. It is recommended to load the sample employees data again before trying the
examples mentioned in this chapter. You can refer to Section Loading sample data in Chapter 2
to know how to load sample data.
The explain plan

How MySQL executes queries is one of the major factors of database performance. You can
verify the MySQL execution plan using the &91-"*/ command. Starting from MySQL 5.7.2,
you can use &91-"*/ to examine queries currently executing in other sessions. &91-"*/
'03."5+40/ gives detailed information.
How to do it...
Using EXPLAIN
The explain plan gives information on how the optimizer is going to execute the query. You
just need to prefix the &91-"*/ keyword to the query:
NZTRM &91-"*/4&-&$5EFQU@OBNF'30.EFQU@FNQ+0*/FNQMPZFFT0/
EFQU@FNQFNQ@OPFNQMPZFFTFNQ@OP+0*/EFQBSUNFOUT0/
EFQBSUNFOUTEFQU@OPEFQU@FNQEFQU@OP8)&3&FNQMPZFFTGJSTU@OBNF "BNFS =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZT13*."3:OBNF
LFZOBNF
[ 329 ]

LFZ@MFO
SFGDPOTU
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFEFQU@FNQ
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZT13*."3:EFQU@OP
LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTFNQMPZFFTFNQ@OP
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFEFQBSUNFOUT
QBSUJUJPOT/6--
UZQFFR@SFG
QPTTJCMF@LFZT13*."3:
LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTEFQU@FNQEFQU@OP
SPXT
GJMUFSFE
&YUSB/6--
SPXTJOTFUXBSOJOH
TFD
6TJOH&91-"*/+40/
Using the explain plan in JSON format gives complete information about the query
execution:
NZTRM &91-"*/'03."5+40/4&-&$5EFQU@OBNF'30.EFQU@FNQ+0*/FNQMPZFFT0/
EFQU@FNQFNQ@OPFNQMPZFFTFNQ@OP+0*/EFQBSUNFOUT0/
EFQBSUNFOUTEFQU@OPEFQU@FNQEFQU@OP8)&3&FNQMPZFFTGJSTU@OBNF "BNFS =(
SPX
&91-"*/\
RVFSZ@CMPDL\
TFMFDU@JE
DPTU@JOGP\
RVFSZ@DPTU
[ 330 ]

^
OFTUFE@MPPQ<
\
UBCMF\
UBCMF@OBNFFNQMPZFFT
BDDFTT@UZQFSFG
QPTTJCMF@LFZT<
13*."3:
OBNF
>
LFZOBNF
VTFE@LFZ@QBSUT<
GJSTU@OBNF
>
LFZ@MFOHUI
SFG<
DPOTU
>
SPXT@FYBNJOFE@QFS@TDBO
SPXT@QSPEVDFE@QFS@KPJO
GJMUFSFE
VTJOH@JOEFYUSVF
DPTU@JOGP\
SFBE@DPTU
FWBM@DPTU
QSFGJY@DPTU
EBUB@SFBE@QFS@KPJO,
^
VTFE@DPMVNOT<
FNQ@OP
GJSTU@OBNF
>
^
^
\
UBCMF\
UBCMF@OBNFEFQU@FNQ
BDDFTT@UZQFSFG
QPTTJCMF@LFZT<
13*."3:
EFQU@OP
>
LFZ13*."3:
VTFE@LFZ@QBSUT<
FNQ@OP
>
LFZ@MFOHUI
SFG<
[ 331 ]

FNQMPZFFTFNQMPZFFTFNQ@OP
>
GJMUFSFE
VTJOH@JOEFYUSVF
DPTU@JOGP\
SFBE@DPTU
FWBM@DPTU
QSFGJY@DPTU
EBUB@SFBE@QFS@KPJO,
^
VTFE@DPMVNOT<
FNQ@OP
EFQU@OP
>
^
^
\
UBCMF\
UBCMF@OBNFEFQBSUNFOUT
BDDFTT@UZQFFR@SFG
QPTTJCMF@LFZT<
13*."3:
>
LFZ13*."3:
VTFE@LFZ@QBSUT<
EFQU@OP
>
LFZ@MFOHUI
SFG<
FNQMPZFFTEFQU@FNQEFQU@OP
>
GJMUFSFE
DPTU@JOGP\
SFBE@DPTU
FWBM@DPTU
QSFGJY@DPTU
EBUB@SFBE@QFS@KPJO,
^
VTFE@DPMVNOT<
EFQU@OP
EFQU@OBNF
>
^
^
[ 332 ]

>
^
^
6TJOH&91-"*/GPSDPOOFDUJPO
You can run the explain plan for an already-running session. You need to specify the
connection ID:
To get connection ID, execute:
NZTRM 4&-&$5$0//&$5*0/@*%

]$0//&$5*0/@*%
]

]]

SPXJOTFU
TFD
NZTRM &91-"*/'03."5+40/'03$0//&$5*0/=(
SPX
&91-"*/\
RVFSZ@CMPDL\
TFMFDU@JE
DPTU@JOGP\
RVFSZ@DPTU
^
OFTUFE@MPPQ<
\
UBCMF\
UBCMF@OBNFFNQMPZFFT
BDDFTT@UZQFJOEFY
QPTTJCMF@LFZT<
13*."3:
>
LFZOBNF
VTFE@LFZ@QBSUT<
GJSTU@OBNF
MBTU@OBNF
>
LFZ@MFOHUI
GJMUFSFE
VTJOH@JOEFYUSVF
DPTU@JOGP\
SFBE@DPTU
FWBM@DPTU
[ 333 ]

QSFGJY@DPTU
EBUB@SFBE@QFS@KPJO
^
_
_
SPXJOTFU
TFD
If the connection is not running any 4&-&$5/61%"5&/*/4&35/%&-&5&/3&1-"$& queries, it

will throw an error:
NZTRM &91-"*/'03$0//&$5*0/
&3303
):&91-"*/'03$0//&$5*0/DPNNBOEJTTVQQPSUFEPOMZGPS
4&-&$561%"5&*/4&35%&-&5&3&1-"$&
Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOFYQMBJOPVUQVUIUNM to learn more

about the explain plan format. The JSON format is very clearly explained at IUUQTXXX
QFSDPOBDPNCMPHDBUFHPSZFYQMBJOFYQMBJOGPSNBUKTPOJTDPPM.
Benchmarking queries and the server

Suppose you want to find out which of the queries is faster. The explain plan gives you an
idea, but sometimes you cannot decide based on it. You can execute them on the server and
find which one is faster if the query time is in the order of tens of seconds. However, if the
query time is in the order of a few milliseconds, you cannot decide based on a single
execution.
You can use the NZTRMTMBQ utility (it comes along with MySQL-client installation), which
emulates client load for a MySQL server and reports the timing of each stage. It works as if
multiple clients are accessing the server. In this section, you will learn about the usage of
NZTRMTMBQ; in later sections, you will learn about the power of the NZTRMTMBQ.
How to do it...
Suppose you want to measure the query time of a query; if you execute that in the MySQL
client, you can know the approximate execution time with a granularity of 100 milliseconds:
NZTRM QBHFSHSFQSPXT
1"(&3TFUUP HSFQSPXT
NZTRM 4&-&$5FFNQ@OPTBMBSZ'30.TBMBSJFTT+0*/FNQMPZFFTF0/
TFNQ@OPFFNQ@OP8)&3&
GJSTU@OBNF "EBN
2384 rows in set (0.00 sec)
[ 334 ]

You can emulate the client load using NZTRMTMBQ and run the preceding SQL concurrently
over multiple iterations:
TIFMM NZTRMTMBQVVTFS QQBTT DSFBUFTDIFNBFNQMPZFFT
RVFSZ4&-&$5FFNQ@OPTBMBSZ'30.TBMBSJFTT+0*/FNQMPZFFTF0/
TFNQ@OPFFNQ@OP8)&3&
GJSTU@OBNF "EBN DJ
NZTRMTMBQ<8BSOJOH>6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBOCF
JOTFDVSF
#FODINBSL
"WFSBHFOVNCFSPGTFDPOETUPSVOBMMRVFSJFTTFDPOET
.JOJNVNOVNCFSPGTFDPOETUPSVOBMMRVFSJFTTFDPOET
.BYJNVNOVNCFSPGTFDPOETUPSVOBMMRVFSJFTTFDPOET
/VNCFSPGDMJFOUTSVOOJOHRVFSJFT
"WFSBHFOVNCFSPGRVFSJFTQFSDMJFOU
The preceding query was executed with 1,000 concurrencies and 100 iterations, and on
average, it took 3.216 seconds.
You can specify multiple SQLs in a file and specify the delimiter. NZTRMTMBQ runs all the
queries in the file:
TIFMM DBURVFSJFTTRM
4&-&$5FFNQ@OPTBMBSZ'30.TBMBSJFTT+0*/FNQMPZFFTF0/
TFNQ@OPFFNQ@OP8)&3&
GJSTU@OBNF "EBN
4&-&$5'30.FNQMPZFFT8)&3&GJSTU@OBNF "EBN 03MBTU@OBNF "EBN
4&-&$5'30.FNQMPZFFT8)&3&GJSTU@OBNF "EBN
TIFMM NZTRMTMBQVVTFS QQBTT DSFBUFTDIFNBFNQMPZFFT

DPODVSSFODZJUFSBUJPOTRVFSZRVFSZTRMRVFSZRVFSJFTTRM
EFMJNJUFS
JOTFDVSF
#FODINBSL
You can even autogenerate the table and SQL statements. In this way, you can compare the
results with earlier server settings:
TIFMM NZTRMTMBQVVTFS QQBTT DPODVSSFODZJUFSBUJPOT
OVNCFSJOUDPMTOVNCFSDIBSDPMTBVUPHFOFSBUFTRM
JOTFDVSF
#FODINBSL
[ 335 ]

You can also use QFSGPSNBODF@TDIFNB for all query-related metrics,

which is explained in the Using performance_schema section.
Adding indexes
Without an index, MySQL must scan the entire table row by row to find the relevant rows.
If the table has an index on the columns that you are filtering for, MySQL can quickly find
the rows in the big data file without scanning the whole file.
MySQL can use an index for filtering of rows in 8)&3&, 03%&3#:, and (3061#: clauses,
and also for joining tables. If there are multiple indexes on a column, MySQL chooses the
index that gives maximum filtering of rows.
You can execute the "-5&35"#-& command to add or drop the index. Both index addition
and dropping are online operations and do not hinder the DMLs on the table, but they take
lot of time on larger tables.
Primary key (clustered index) and secondary

indexes
Before you proceed further, it is important to understand what a primary key (or clustered
index) is, and what a secondary index is.
*OOP%# stores rows in a primary key in order to speed up queries and sorts involving the
primary key columns. This is also called an index-organized table, in Oracle terms. All
other indexes are referred to as secondary keys, which store the value of primary keys (they
do not refer to the row directly).
[ 336 ]

Suppose the table is:

NZTRM $3&"5&5"#-&JOEFY@FYBNQMF
DPMJOU13*."3:,&:
DPMDIBS

,&:ADPMA
ADPMA

The table rows are sorted and stored based on the value of DPM. If you search for any value
of DPM, it can directly point to the physical row; this is why a clustered index is lightning-
fast. The index on DPM also contains the value of DPM, and if you search for DPM, the
value of DPM is returned, which in turn is searched in the clustered index to return the
actual row.
Tips on choosing the primary key:
It should be 6/*26& and /05/6--.

Choose the smallest possible key because all the secondary indexes store the
primary key. So if it is large, the overall index size uses more space.
Choose a monotonically increasing value. The physical rows are ordered based
on the primary key. So if you choose a random key, more rearrangement of rows
is needed, which leads to degraded performance. "650@*/$3&.&/5 is a perfect
fit for primary key.
Always choose a primary key; if you cannot find any, add an "650@*/$3&.&/5
column. If you do not choose any, *OOP%# internally generates a hidden clustered
index with a 6-byte row ID.
How to do it...
You can see the indexes of a table by viewing its definition. You will notice that there is an
index on GJSTU@OBNF and MBTU@OBNF. If you filter the rows by specifying GJSTU@OBNF or
by both (GJSTU@OBNF and MBTU@OBNF), MySQL can use the index to speed up the query.
However, if you specify only MBTU@OBNF, the index cannot be used; this is because the
optimizer can only use any of the leftmost prefixes of the index. Refer to IUUQTEFW
NZTRMDPNEPDSFGNBOFONVMUJQMFDPMVNOJOEFYFTIUNM for more detailed
examples:
NZTRM "-5&35"#-&FNQMPZFFT"%%*/%&9OBNF
GJSTU@OBNFMBTU@OBNF
2VFSZ0,SPXTBGGFDUFE
TFD
[ 337 ]

SPX
5BCMFFNQMPZFFT
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
KEY `name` (`first_name`,`last_name`)
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
Add index
You can add an index by executing the "-5&35"#-&"%%*/%&9 command. For example,
if you want to add an index on MBTU@OBNF, see the following code:
NZTRM "-5&35"#-&FNQMPZFFT"%%*/%&9
MBTU@OBNF
2VFSZ0,SPXTBGGFDUFE
TFD
SPX
5BCMFFNQMPZFFT
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
,&:AOBNFA
KEY `last_name` (`last_name`)
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
You can specify the name of the index; if not, the leftmost prefix will be used as the name. If
there are any duplicates, the name will be appended by @, @, and so on.
For example:
NZTRM "-5&35"#-&FNQMPZFFT"%%*/%&9JOEFY@MBTU@OBNF
MBTU@OBNF
[ 338 ]

UNIQUE index
If you want the index to be unique, you can specify the keyword 6/*26&. For example:
NZTRM "-5&35"#-&FNQMPZFFT"%%6/*26&*/%&9VOJRVF@OBNF
MBTU@OBNF
GJSTU@OBNF
5IFSFBSFGFXEVQMJDBUFFOUSJFTJOFNQMPZFFTEBUBCBTFUIFBCPWF
TUBUFNFOUJTTIPXOGPSJMMVTUSBUJPOQVSQPTFPOMZ
Prefix index
For string columns, indexes that use only the leading part of column values, rather than the
full column, can be created. You need to specify the length of the leading part:
AMBTU@OBNFAWBSDIBS
/05/6--
MBTU@OBNF

2VFSZ0,SPXTBGGFDUFE
TFD
The maximum length of MBTU@OBNF is characters, but the index is created only on the
first 10 characters.
Drop index
You can drop an index using the "-5&35"#-& command:
NZTRM "-5&35"#-&FNQMPZFFT%301*/%&9MBTU@OBNF
2VFSZ0,SPXTBGGFDUFE
TFD
Index on generated columns

An index cannot be used on a column wrapped in a function. Suppose you add an index on
IJSF@EBUF:
IJSF@EBUF
2VFSZ0,SPXTBGGFDUFE
TFD
[ 339 ]

The index on IJSF@EBUF can be used for queries having IJSF@EBUF in the 8)&3& clause:
NZTRM &91-"*/4&-&$5$06/5
'30.FNQMPZFFT8)&3&
IJSF@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFSBOHF
QPTTJCMF@LFZTIJSF@EBUF
LFZIJSF@EBUF
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
Extra: Using where; Using index
SPXJOTFUXBSOJOH
TFD
Instead, if you put IJSF@EBUF inside a function, MySQL has to scan the full table:
NZTRM &91-"*/4&-&$5$06/5
'30.FNQMPZFFT8)&3&YEAR(hire_date) =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY
QPTTJCMF@LFZT/6--
LFZIJSF@EBUF
LFZ@MFO
SFG/6--
rows: 291892
GJMUFSFE
&YUSB6TJOHXIFSF6TJOHJOEFY
SPXJOTFUXBSOJOH
TFD
So try to avoid putting an indexed column inside a function. If you cannot avoid using a
function, create a virtual column and add an index on the virtual column:
:&"3
IJSF@EBUF
7*356"-"%%*/%&9
IJSF@EBUF@ZFBS
2VFSZ0,SPXTBGGFDUFE
TFD
SPX
5BCMFFNQMPZFFT
[ 340 ]

AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
`hire_date_year` year(4) GENERATED ALWAYS AS (year(`hire_date`)) VIRTUAL,
13*."3:,&:
AFNQ@OPA
,&:AOBNFA
,&:AIJSF@EBUFA
AIJSF@EBUFA
KEY `hire_date_year` (`hire_date_year`)
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
Now, instead of using the :&"3

function in the query, you can directly use
IJSF@EBUF@ZFBS in the 8)&3& clause:
NZTRM &91-"*/4&-&$5$06/5
'30.FNQMPZFFT8)&3&hire_date_year>=2000=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFSBOHF
QPTTJCMF@LFZTIJSF@EBUF@ZFBS
key: hire_date_year
LFZ@MFO
SFG/6--
rows: 15
GJMUFSFE
SPXJOTFUXBSOJOH
TFD
Note that even if you use :&"3

IJSF@EBUF, the optimizer recognizes that the expression
:&"3
matches the definition of IJSF@EBUF@ZFBS and that IJSF@EBUF@ZFBS is indexed;
so it considers that index during execution plan construction:
NZTRM &91-"*/4&-&$5$06/5
'30.FNQMPZFFT8)&3&:&"3
IJSF@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFSBOHF
QPTTJCMF@LFZTIJSF@EBUF@ZFBS
key: hire_date_year
[ 341 ]

LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHXIFSF
SPXJOTFUXBSOJOH
TFD
Invisible index
If you want to drop an unused index, then instead of dropping immediately, you can mark
it as invisible, monitor the application behavior, and later drop it. Later, if you need that
index, you can mark it as visible, which is very fast compared to dropping and re-adding
indexes.
To explain the invisible index, you need to add normal index if not already there. Example:
MBTU@OBNF
2VFSZ0,SPXTBGGFDUFE
TFD
How to do it...
If you wish to drop the index on MBTU@OBNF, rather than directly dropping, you can mark it
as invisible using the "-5&35"#-& command:
NZTRM &91-"*/4&-&$5'30.FNQMPZFFT8)&3&MBTU@OBNF "BNPEU =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZTMBTU@OBNF
LFZMBTU@OBNF
LFZ@MFO
SFGDPOTU
SPXT
GJMUFSFE
&YUSB/6--
SPXJOTFUXBSOJOH
TFD
mysql> ALTER TABLE employees ALTER INDEX last_name INVISIBLE;

2VFSZ0,SPXTBGGFDUFE
TFD
[ 342 ]

NZTRM &91-"*/4&-&$5'30.FNQMPZFFT8)&3&MBTU@OBNF "BNPEU =(

SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQF"--
QPTTJCMF@LFZT/6--
LFZ/6--
LFZ@MFO/6--
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHXIFSF
SPXJOTFUXBSOJOH
TFD
SPX
5BCMFFNQMPZFFT
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
,&:AOBNFA
KEY `last_name` (`last_name`) /*!80000 INVISIBLE */
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
You will notice that the query filtering through MBTU@OBNF is using the MBTU@OBNF index;
after marking it as invisible, it is not able to use. You can mark it as visible again:
NZTRM "-5&35"#-&FNQMPZFFT"-5&3*/%&9MBTU@OBNF7*4*#-&
2VFSZ0,SPXTBGGFDUFE
TFD
[ 343 ]

Descending index
Prior to MySQL 8, an index definition could contain the order (ascending or descending),
but it was only parsed and not implemented. The index values were always stored in
ascending order. MySQL 8.0 introduced support for descending indexes. Thus, the specified
order in the index definition is not ignored. A descending index actually stores key values
in descending order. Remember that scanning an ascending index in reverse is not efficient
for a descending query.
Consider a case where, in a multi-column index, you can specify certain columns to be
descending. This can help for queries wherein we have both ascending and descending
03%&3#: clauses.
Suppose you want to sort the FNQMPZFFT table with GJSTU@OBNF ascending and
MBTU@OBNF descending; MySQL cannot use the index on GJSTU@OBNF and MBTU@OBNF.
Without a descending index:
SPX
5BCMFFNQMPZFFT
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
KEY `name` (`first_name`,`last_name`),
,&:AMBTU@OBNFA
AMBTU@OBNFA*/7*4*#-&
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
In the explain plan, you will notice that the index name (GJSTU@OBNF and MBTU@OBNF) is
not used:
NZTRM &91-"*/4&-&$5'30.FNQMPZFFT03%&3#:GJSTU@OBNF"4$MBTU@OBNF
%&4$-*.*5=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQF"--
QPTTJCMF@LFZT/6--
LFZ/6--
[ 344 ]

LFZ@MFO/6--
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHGJMFTPSU
How to do it...
1. Add a descending index:
NZTRM "-5&35"#-&FNQMPZFFT"%%*/%&9OBNF@EFTD
GJSTU@OBNF"4$
MBTU@OBNF%&4$
2VFSZ0,SPXTBGGFDUFE
TFD
2. After adding the descending index, the query is able to use the index:
NZTRM &91-"*/4&-&$5'30.FNQMPZFFT03%&3#:GJSTU@OBNF"4$

MBTU@OBNF%&4$-*.*5=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY
QPTTJCMF@LFZT/6--
key: name_desc
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
&YUSB/6--
3. The same index can be used for the other way of ordering, that is, ordering by
GJSTU@OBNF descending and MBTU@OBNF ascending through the backward index
scan:
NZTRM &91-"*/4&-&$5'30.FNQMPZFFT03%&3#:GJSTU@OBNF%&4$

MBTU@OBNF"4$-*.*5=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY
[ 345 ]

QPTTJCMF@LFZT/6--
LFZOBNF@EFTD
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
Extra: Backward index scan
Analyzing slow queries using pt-query-

digest
QURVFSZEJHFTU, which is part of the Percona Toolkit, is used for analyzing queries. The
queries can be collected through any of the following:
Slow query log

General query log
Process list
Binary log
TCP dump
Installation of the Percona Toolkit is covered in $IBQUFS, Table Maintenance, Installing

Percona Toolkit section. In this section, you will learn how to use QURVFSZEJHFTU. There
are drawbacks for each method. The slow query log does not include all queries unless you
specify MPOH@RVFSZ@UJNF as , which drastically slows down a system. The general query
log does not include query time. You cannot get complete queries from the process list.
Only writes can be analyzed using binary log and using TCP dump causes server
degradation. Usually, this tool is used on the slow query log with a MPOH@RVFSZ@UJNF of 1
second or higher.
How to do it...
Let's get into the details of analyzing slow queries using QURVFSZEJHFTU.
[ 346 ]

Slow query log

Enabling and configuring the slow query log is explained in $IBQUFS, Managing Logs,
Managing the general query log and slow query log. Once the slow query log is enabled and
queries are collected, you can run QURVFSZEJHFTU by passing the slow query log.
Suppose the slow query file is at WBSMJCNZTRMNZTRMTMPXMPH:

TIFMM TVEPQURVFSZEJHFTUWBSMJCNZTRMVCVOUVTMPXMPH RVFSZ@EJHFTU
The digest report contains queries ranked by the number of query executions multiplied by
the query time. Query details such as the query checksum (a unique value for each type of
query), average time, percentage time, and number of executions are shown for all queries
in the summary. You can drill down to the specific query by searching for the query
checksum.
The digest report looks like the following:

TVTFSUJNFNTTZTUFNUJNF.STT.WT[
$VSSFOUEBUF4BU/PW
)PTUOBNFEC
'JMFTWBSMJCNZTRMECTMPXMPH
3BUFMJNJUTBQQMZ
0WFSBMMLUPUBMLVOJRVF214YDPODVSSFODZ@@@@@@
5JNFSBOHFUP
"UUSJCVUFUPUBMNJONBYBWHTUEEFWNFEJBO

&YFDUJNFTVTTNTNTTVT
-PDLUJNFTTNTVTNTVT
3PXTTFOU.LL
3PXTFYBNJOF((LL.
3PXTBGGFDUF.LL
#ZUFTTFOU(.LLL
.FSHFQBTTFTL
5NQUBCMFTL
5NQEJTLUCML
5NQUCMTJ[F(.L.
2VFSZTJ[F.LL
*OOP%#
*0SCZUFT((L.
*0SPQT.LL
*0SXBJUTTNTT
QBHFTEJTUJO.LLL
RVFVFXBJU
SFDMPDLXBJTTVTNT
#PPMFBO
[ 347 ]

'JMFTPSUZFTOP
'JMFTPSUPOZFTOP
'VMMKPJOZFTOP
'VMMTDBOZFTOP
5NQUBCMFZFTOP
5NQUBCMFPOZFTOP
The query profile will look like:

3BOL2VFSZ*%3FTQPOTFUJNF$BMMT3$BMM7.*UFN

Y'"#$#4&-&$5
PSEFST
Y"'#%"#$#4&-&$5
PSEFSTPSEFS@TUBUVT
Y$$"4&-&$5
PSEFSTVTFST
Y$&"%%'&%4&-&$5
TFTTJPOT
Y%$%&"&"%4&-&$5
PSEFSTTFTTJPOT
Y%'#"%61%"5&
4&-&$5
Y$'"%&#"*/4&35
4&-&$5UNQ.PWF
Y#'%"#''61%"5&
4&-&$5PSEFSTUNQ%FM
From the preceding output, you can infer that for query (Y'"#$#), the
cumulative response time for all executions is seconds. This accounts for 42.7% of the
cumulative response time of all queries. The number of executions is 47 and the average
query time is seconds.
You can go to any query by searching for the checksum. The complete query, commands for
the explain plan, and the table status are displayed. For example:
2VFSZ214YDPODVSSFODZ*%Y'"#$#BUCZUF

5IJTJUFNJTJODMVEFEJOUIFSFQPSUCFDBVTFJUNBUDIFTMJNJU
4DPSFT7.
5JNFSBOHFUP
"UUSJCVUFQDUUPUBMNJONBYBWHTUEEFWNFEJBO

$PVOU
&YFDUJNFTTTTTTT
-PDLUJNFTVTNTNTNTNTVT
3PXTTFOULLLLL
[ 348 ]

3PXTFYBNJOF(((((.(
3PXTBGGFDUF
#ZUFTTFOU.LLLLLL
.FSHFQBTTFT
5NQUBCMFT
5NQEJTLUCM
5NQUCMTJ[F
2VFSZTJ[FL
*OOP%#
*0SCZUFT(.....
*0SPQTLLLLLL
*0SXBJUTTTTTT
QBHFTEJTUJOLLLLLL
RVFVFXBJU
SFDMPDLXBJ
#PPMFBO
'VMMTDBOZFTOP
4USJOH
%BUBCBTFTMBTISFOFX@
CFUTZ@EC

)PTUT
*OOP%#USY*%$'$
$'"
NPSF
-BTUFSSOP
SBUFMJNJURVFSZ
6TFSTEC@
ECB

2VFSZ@UJNFEJTUSJCVUJPO
VT
VT
VT
NT
NT
NT
T
T
5BCMFT
4)085"#-&45"564'30.AECA-*,& PSEFST =(
4)08$3&"5&5"#-&AECAAPSEFSTA=(
4)085"#-&45"564'30.AECA-*,& TIJQQJOH@USBDLJOH@IJTUPSZ =(
4)08$3&"5&5"#-&AECAATIJQQJOH@USBDLJOH@IJTUPSZA=(
&91-"*/1"35*5*0/4
4&-&$5USBDLJOH@OVNDBSSJFSPSEFS@JEVTFS*%'30.PSEFSTP8)&3&
USBDLJOH@OVN
BOE/05&9*454
4&-&$5'30.TIJQQJOH@USBDLJOH@IJTUPSZTUI8)&3&
TUIPSEFS@JEPPSEFS@JE"/%TUIJT@GJOBM
"/%PEBUF@GJOBMJ[FE EBUF@BEE
DVSEBUF
JOUFSWBMNPOUI=(
[ 349 ]

General query log

You can use QURVFSZEJHFTU to analyze the general query log by passing the argument
UZQFHFOMPH. Since general logs do not report query times, only the count aggregate is
shown:
TIFMM TVEPQURVFSZEJHFTUUZQFHFOMPHWBSMJCNZTRMECMPH
HFOFSBM@RVFSZ@EJHFTU
The output will be something like this:

NTVTFSUJNFTZTUFNUJNF.STT.WT[
$VSSFOUEBUF4BU/PW
)PTUOBNFEC
'JMFTWBSMJCNZTRMECMPH
0WFSBMMUPUBMVOJRVF214YDPODVSSFODZ@@@@@@@@@@@@@@@
5JNFSBOHFUP
"UUSJCVUFUPUBMNJONBYBWHTUEEFWNFEJBO

&YFDUJNF
2VFSZTJ[FLL
The query profile will look something like this:

1SPGJMF
3BOL2VFSZ*%3FTQPOTFUJNF$BMMT3$BMM7.*UFN

Y#''%4&-&$5
GBDFCPPL@MJLF@EFUBJMT
Y""%&$$#"%.*/26*5
Y%&'##&"%.*/$0//&$5
Process list
Instead of a log file, you can use QURVFSZEJHFTU to read queries from the process list:
TIFMM QURVFSZEJHFTUQSPDFTTMJTUIMPDBMIPTUJUFSBUJPOTSVO
UJNFNVVTFS QQBTT
SVOUJNF specifies how long each iteration should run. In the preceding example, the tool
generates reports every minute for 10 minutes.
[ 350 ]

Binary log
To analyze the binary log using QURVFSZEJHFTU, you should convert it to text format
using the NZTRMCJOMPH utility:
TIFMM TVEPNZTRMCJOMPHWBSMJCNZTRMCJOMPH CJOMPH
TIFMM QURVFSZEJHFTUUZQFCJOMPHCJOMPH CJOMPH@EJHFTU
TCP dump
You can capture TCP traffic using the UDQEVNQ command and send it to QURVFSZEJHFTU
for analysis:
TIFMM TVEPUDQEVNQTYOORUUUUJBOZDQPSU
NZTRMUDQUYU
TIFMM QURVFSZEJHFTUUZQFUDQEVNQNZTRMUDQUYU UDQEVNQ@EJHFTU
There are plenty of options available in QURVFSZEJHFTU, such as filtering queries for a
specific time window, filtering a specific query, and generating reports. Please refer to the
Percona documentation at IUUQTXXXQFSDPOBDPNEPDQFSDPOBUPPMLJU-"5&45QU
RVFSZEJHFTUIUNM for more details.
See also
Refer to IUUQTFOHJOFFSJOHMJOLFEJODPNCMPHRVFSZBOBMZ[FSBUPPM
GPSBOBMZ[JOHNZTRMRVFSJFTXJUIPVUPWFSI to know more about the new way of
analyzing all queries without any overhead.
Optimizing datatypes
You should define tables such that they occupy minimum space on disk while
accommodating all possible values.
[ 351 ]

If the size is smaller:
Less data is written to or read from the disk, which makes queries faster.
The contents on the disk are loaded to the main memory while processing
queries. So, smaller tables occupy less space in the main memory.
Less space is occupied by indexes.
How to do it...
1. If you want to store an employee number, for which the maximum possible value
is 500,000, the optimum datatype is .&%*6.*/56/4*(/&% (which occupies 3
bytes). If you are storing it as */5, which occupies 4 bytes, you are wasting a byte
for each row.
2. If you want to store the first name, for which the length is varying and the
maximum possible value is 20, it is optimal to declare it as WBSDIBS
. If you
are storing it as DIBS
, and just a few names are 20 characters long while the
remaining are less than 10 characters long, you are wasting space of 10 characters.
3. While declaring WBSDIBS columns, you should consider the length. Though
WBSDIBS is optimized on-disk, while loading into memory, it occupies the full
length. For example, if you store GJSTU@OBNF in WBSDIBS
and the actual
length is 10, on the disk it occupies 10 + 1 (an additional byte for storing length);
but in the memory, it occupies the full length of 255 bytes.
4. If the length of the WBSDIBS column is more than 255 chars, it requires 2 bytes to
store the length.
5. Declare the columns as /05/6-- if you are not storing null values. This avoids
the overhead for testing whether each value is null and also saves some storage
space: 1 bit per column.
6. If the length is fixed, use DIBS instead of WBSDIBS, because WBSDIBS takes a byte
or two to store the length of the string.
7. If the values are fixed, use &/6. rather than WBSDIBS. For example, if you want to
store values that can be pending, approved, rejected, deployed, undeployed,
failed, or deleted, you can use &/6.. It takes 1 or 2 bytes, rather than DIBS
,
which occupies 10 bytes.
8. Prefer integers over strings.
9. Try to leverage the prefix index.
10. Try to leverage the *OOP%# compression.
[ 352 ]

Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFOTUPSBHFSFRVJSFNFOUTIUNM to
know more about the storage requirements of each datatype and IUUQTEFWNZTRMDPN
EPDSFGNBOFOJOUFHFSUZQFTIUNM to know about the range of each integer type.
If you want to know the optimized datatype, you can use the function 130$&%63&
"/"-:;&. Though it is not accurate, it gives a fair idea of the fields. Unfortunately, it is
deprecated in MySQL 8:
NZTRM 4&-&$5VTFS@JEGJSTU@OBNF'30.VTFS130$&%63&"/"-:4&
=(
SPX
'JFME@OBNFECVTFSVTFS@JE
.JO@WBMVF!OBUUFTUOFU
.BY@WBMVFUFTU!OBUUFTUOFU
.JO@MFOHUI
.BY@MFOHUI
&NQUJFT@PS@[FSPT
/VMMT
"WH@WBMVF@PS@BWH@MFOHUI
4UE/6--
0QUJNBM@GJFMEUZQF7"3$)"3
/05/6--
SPX
'JFME@OBNFECVTFSGJSTU@OBNF
.JO@WBMVF"MBO
.BY@WBMVF;VOJHB
.JO@MFOHUI
.BY@MFOHUI
&NQUJFT@PS@[FSPT
/VMMT
"WH@WBMVF@PS@BWH@MFOHUI
4UE/6--
0QUJNBM@GJFMEUZQF7"3$)"3
/05/6--
SPXTJOTFU
TFD
Removing duplicate and redundant indexes

You can define several indexes on a column. By mistake, you might have defined the same
index again (same column, same order of columns, or same order of keys), which is called
a duplicate index. If only partial indexes (leftmost columns) are repetitive, they are called
redundant indexes. A duplicate index has no advantages. Redundant indexes might be
useful in some cases (a use case is mentioned in the note at the end of this section), but both
slow down the inserts. So, it is important to identify and remove them.
[ 353 ]

There are three tools that can help with finding out duplicate indexes:
QUEVQMJDBUFLFZDIFDLFS, which is part of the Percona Toolkit. Installing

the Percona Toolkit is covered in $IBQUFS, Table Maintenance, Installing Percona
Toolkit section.
NZTRMJOEFYDIFDL, which is part of MySQL utilities. Installing MySQL utilities is
covered in $IBQUFS, MySQL 8.0 ` Installing and Upgrading.
Using the TZT schema, which will be covered in the next section.
Consider the following FNQMPZFFT table:

SPX
5BCMFFNQMPZFFT
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
,&:AMBTU@OBNFA
,&:AGVMM@OBNFA
,&:AGVMM@OBNF@EFTDA
AGJSTU@OBNFA%&4$AMBTU@OBNFA
,&:AGJSTU@OBNFA
AGJSTU@OBNFA
,&:AGVMM@OBNF@A
,&:AGJSTU@OBNF@FNQ@OPA
AGJSTU@OBNFAAFNQ@OPA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
The index GVMM@OBNF@ is the duplicate of GVMM@OBNF because both indexes are on the
same columns, the same order of columns, and the same order of keys (ascending or
descending).
The index GJSTU@OBNF is a redundant index because the column GJSTU@OBNF is already
covered in the leftmost suffix of the GJSTU@OBNF index.
The index GJSTU@OBNF@FNQ@OP is a redundant index because it contains the primary

key in the rightmost suffix. The *OOP%# secondary indexes already contain the primary key,
so it is redundant to declare the primary key as part of the secondary index. However, it can
be useful in queries that filter by GJSTU@OBNF and sort by FNQ@OP:
4&-&$5'30.FNQMPZFFT8)&3&GJSTU@OBNF "EBN 03%&3#:FNQ@OP
[ 354 ]

The GVMM@OBNF@EFTD option is not a duplicate of GVMM@OBNF because the

ordering of keys is different.
How to do it...
Let's get into the details of removing duplicate and redundant indexes.
pt-duplicate-key-checker
QUEVQMJDBUFLFZDIFDLFS gives the exact "-5&3 statements to drop duplicate keys:
TIFMM QUEVQMJDBUFLFZDIFDLFSVVTFS QQBTT
"TPGUXBSFVQEBUFJTBWBJMBCMF

FNQMPZFFTFNQMPZFFT

GVMM@OBNF@JTBEVQMJDBUFPGGVMM@OBNF
,FZEFGJOJUJPOT
,&:AGVMM@OBNF@A
,&:AGVMM@OBNFA
$PMVNOUZQFT
AGJSTU@OBNFAWBSDIBS
OPUOVMM
AMBTU@OBNFAWBSDIBS
OPUOVMM
5PSFNPWFUIJTEVQMJDBUFJOEFYFYFDVUF
"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGVMM@OBNF@A
GJSTU@OBNFJTBMFGUQSFGJYPGGVMM@OBNF
,FZEFGJOJUJPOT
,&:AGJSTU@OBNFA
AGJSTU@OBNFA
,&:AGVMM@OBNFA
$PMVNOUZQFT
AGJSTU@OBNFAWBSDIBS
OPUOVMM
AMBTU@OBNFAWBSDIBS
OPUOVMM
5PSFNPWFUIJTEVQMJDBUFJOEFYFYFDVUF
"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGJSTU@OBNFA
,FZGJSTU@OBNF@FNQ@OPFOETXJUIBQSFGJYPGUIFDMVTUFSFEJOEFY
,FZEFGJOJUJPOT
13*."3:,&:
AFNQ@OPA
[ 355 ]

$PMVNOUZQFT
AGJSTU@OBNFAWBSDIBS
OPUOVMM
AFNQ@OPAJOU
OPUOVMM
5PTIPSUFOUIJTEVQMJDBUFDMVTUFSFEJOEFYFYFDVUF
"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGJSTU@OBNF@FNQ@OPA"%%
*/%&9AGJSTU@OBNF@FNQ@OPA
AGJSTU@OBNFA
The tool suggests that you shorten the duplicate clustered index by removing the 13*."3:
,&: from the rightmost suffix. Note that it may result in another duplicate index. If you
wish to ignore the duplicate clustered indexes, you can pass the OPDMVTUFSFE option.
To check the duplicate indexes of a particular database, you can pass the EBUBCBTFT
EBUBCBTFOBNF option:
TIFMM QUEVQMJDBUFLFZDIFDLFSVVTFS QQBTT EBUBCBTFFNQMPZFFT
To drop the keys, you can even pipe the output of QUEVQMJDBUFLFZDIFDLFS to NZTRM:
TIFMM QUEVQMJDBUFLFZDIFDLFSVVTFS QQBTT ]NZTRMVVTFS
QQBTT
mysqlindexcheck
Note that NZTRMJOEFYDIFDL ignores descending indexes. For example, GVMM@OBNF@EFTD
(GJSTU@OBNF descending and MBTU@OBNF) is treated as a duplicate index of GVMM@OBNF
(GJSTU@OBNF and MBTU@OBNF):
shell> mysqlindexcheck --server=<user>:<pass>@localhost:3306 employees --
show-drops
8"3/*/(6TJOHBQBTTXPSEPOUIFDPNNBOEMJOFJOUFSGBDFDBOCFJOTFDVSF
4PVSDFPOMPDBMIPTUDPOOFDUFE
5IFGPMMPXJOHJOEFYFTBSFEVQMJDBUFTPSSFEVOEBOUGPSUBCMF
FNQMPZFFTFNQMPZFFT

$3&"5&*/%&9AGVMM@OBNF@EFTDA0/AFNQMPZFFTAAFNQMPZFFTA
AGJSTU@OBNFA
AMBTU@OBNFA64*/(#53&&
NBZCFSFEVOEBOUPSEVQMJDBUFPG
$3&"5&*/%&9AGVMM@OBNFA0/AFNQMPZFFTAAFNQMPZFFTA
AGJSTU@OBNFA

$3&"5&*/%&9AGJSTU@OBNFA0/AFNQMPZFFTAAFNQMPZFFTA
AGJSTU@OBNFA64*/(
#53&&
AGJSTU@OBNFA
[ 356 ]


$3&"5&*/%&9AGVMM@OBNF@A0/AFNQMPZFFTAAFNQMPZFFTA
AGJSTU@OBNFA
AGJSTU@OBNFA

%301TUBUFNFOUT

"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGVMM@OBNF@EFTDA
"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGJSTU@OBNFA
"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGVMM@OBNF@A

5IFGPMMPXJOHJOEFYGPSUBCMFFNQMPZFFTFNQMPZFFTDPOUBJOTUIFDMVTUFSFE
JOEFYBOENJHIUCFSFEVOEBOU

$3&"5&*/%&9AGJSTU@OBNF@FNQ@OPA0/AFNQMPZFFTAAFNQMPZFFTA
AGJSTU@OBNFA
AFNQ@OPA64*/(#53&&

%301"%%TUBUFNFOU

"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9AGJSTU@OBNF@FNQ@OPA"%%
*/%&9AGJSTU@OBNF@FNQ@OPA
GJSTU@OBNF

As mentioned earlier, redundant indexes can be useful in some cases. You

have to consider whether these kinds of cases are needed by your
application.
Create indexes to understand the following examples

NZTRM "-5&35"#-&FNQMPZFFT%30113*."3:,&:"%%13*."3:,&:
FNQ@OP
IJSF@EBUF"%%*/%&9AOBNFA
NZTRM "-5&35"#-&TBMBSJFT"%%*/%&9GSPN@EBUF
GSPN@EBUF"%%*/%&9
GSPN@EBUF@
GSPN@EBUFFNQ@OP
Consider the following FNQMPZFFT and TBMBSJFT tables:

SPX
5BCMFFNQMPZFFT
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
[ 357 ]

AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPAAIJSF@EBUFA
,&:AOBNFA
5"#-&41"$&AJOOPEC@TZTUFNA&/(*/&*OOP%#%&'"6-5
$)"34&5VUGNC
NZTRM 4)08$3&"5&5"#-&TBMBSJFT=(
SPX
$SFBUF5BCMF$3&"5&5"#-&ATBMBSJFTA
AFNQ@OPAJOU
/05/6--
ATBMBSZAJOU
/05/6--
AGSPN@EBUFAEBUF/05/6--
AUP@EBUFAEBUF/05/6--
13*."3:,&:
AFNQ@OPAAGSPN@EBUFA
KEY `from_date` (`from_date`),
KEY `from_date_2` (`from_date`,èmp_no`)
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
It seems like GSPN@EBUF is a redundant index of GSPN@EBUF@, but check the explain plan
of the following query! It is using an intersection of both the indexes. The
GSPN@EBUF index is used for filtering and GSPN@EBUF@ is used for joining with
the FNQMPZFFT table. The optimizer is scanning only one row in each table:
NZTRM &91-"*/4&-&$5FFNQ@OPTBMBSZ'30.TBMBSJFTT+0*/FNQMPZFFTF0/
TFNQ@OPFFNQ@OP8)&3&GSPN@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQFJOEFY@NFSHF
QPTTJCMF@LFZT13*."3:GSPN@EBUF@GSPN@EBUF
key: from_date_2,from_date
LFZ@MFO
SFG/6--
rows: 1
GJMUFSFE
Extra: Using intersect(from_date_2,from_date); Using where
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFSFG
[ 358 ]

LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTTFNQ@OP
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPXTJOTFUXBSOJOH
TFD
Now drop the redundant index GSPN@EBUF and check the explain plan. You can see that the
optimizer is scanning 90 rows in the TBMBSJFT table and one row in the FNQMPZFFT table.
But look at the SFG column; it shows that constants are compared to the index named in the
LFZ column (GSPN@EBUF@) to select rows from the table. Rather than dropping the
indexes, you can test this behavior by passing optimizer hints or index hints, which are
covered in the next section:
TFNQ@OPFFNQ@OP8)&3&GSPN@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZT13*."3:GSPN@EBUF@
LFZGSPN@EBUF@
LFZ@MFO
ref: const
SPXT
GJMUFSFE
&YUSB/6--
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFSFG
LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTTFNQ@OP
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPXTJOTFUXBSOJOH
TFD
[ 359 ]

Now you need to determine which of the queries is faster:
Plan 1: Using JOUFSTFDU

GSPN@EBUFGSPN@EBUF@; scanning one row with
ref as null
Plan 2: Using GSPN@EBUF@; scanning 90 rows with ref as constant
You can use the NZTRMTMBQ utility to find that (do not run this directly on the production
host) and make sure that the concurrency is less than NBY@DPOOFDUJPOT.
The benchmark for plan 1 is as follows:

TIFMM NZTRMTMBQVVTFS QQBTT DSFBUFTDIFNB FNQMPZFFT DJ
TFNQ@OPFFNQ@OP8)&3&GSPN@EBUF
JOTFDVSF
#FODINBSL
"WFSBHFOVNCFSPGTFDPOETUPSVOBMMRVFSJFT0.466 seconds
The benchmark for plan 2 is:

TIFMM NZTRMTMBQVVTFS QQBTT DSFBUFTDIFNB FNQMPZFFT DJ
TFNQ@OPFFNQ@OP8)&3&GSPN@EBUF
JOTFDVSF
#FODINBSL
"WFSBHFOVNCFSPGTFDPOETUPSVOBMMRVFSJFT0.435 seconds
It turns out that the average query time for plan 1 and plan 2 are 0.466 seconds and 0.435
seconds, respectively. Since the results are very close, you can take a call and drop the
redundant index. Use plan 2.
This is just an example that will enable you to learn and apply the concept in your
application scenarios.
[ 360 ]

Checking index usage

In the preceding section, you learned about removing redundant and duplicate indexes.
While designing an application, you might have thought about filtering a query based on a
column and added index. But over a period of time, because of changes in the application,
you might not need that index. In this section, you will learn about identifying those
unused indexes.
There are two ways you can find unused indexes:
Using QUJOEFYVTBHF (covered in this section)

Using TZT schema (covered in the next section)
How to do it...
We can use the QUJOEFYVTBHF tool from the Percona Toolkit to get the index analysis. It
takes queries from the slow query log, runs the explain plan for each and every query, and
identifies the unused indexes. If you have a list of queries, you can save them in slow query
format and pass that to the tool. Note that this is only an approximation because the slow
query log does not include all the queries:
TIFMM TVEPQUJOEFYVTBHFTMPXVVTFS QQBTTXPSE WBSMJCNZTRMEC
TMPXMPH VOVTFE@JOEFYFT
Controlling the query optimizer

The task of the query optimizer is to find an optimal plan for executing a SQL query. There
can be multiple plans to execute a query, especially when joining a table, where the number
of plans to be examined increases exponentially. In this section, you will learn about
adjusting the optimizer to your needs.
Take the example of the FNQMPZFFT table and add the necessary index;
NZTRM $3&"5&5"#-&AFNQMPZFFT@JOEFY@FYBNQMFA
AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
[ 361 ]

,&:AMBTU@OBNFA
,&:AGVMM@OBNFA
,&:AGVMM@OBNF@EFTDA
,&:AGJSTU@OBNFA
AGJSTU@OBNFA
,&:AGVMM@OBNF@A
,&:AMBTU@OBNF@A
AMBTU@OBNFA

&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
TFD
NZTRM 4)088"3/*/(4

]-FWFM]$PEF].FTTBHF
]

]8BSOJOH]]%VQMJDBUFJOEFY GVMM@OBNF@ EFGJOFEPOUIFUBCMF
FNQMPZFFTFNQMPZFFT@JOEFY@FYBNQMF 5IJTJTEFQSFDBUFEBOEXJMMCF
EJTBMMPXFEJOBGVUVSFSFMFBTF]

SPXJOTFU
TFD
NZTRM */4&35*/50FNQMPZFFT@JOEFY@FYBNQMF4&-&$5
FNQ@OPCJSUI@EBUFGJSTU@OBNFMBTU@OBNFHFOEFSIJSF@EBUF'30.FNQMPZFFT
NZTRM 3&/".&5"#-&FNQMPZFFT50FNQMPZFFT@PME
NZTRM 3&/".&5"#-&FNQMPZFFT@JOEFY@FYBNQMF50FNQMPZFFT
Suppose you want to check whether any of GJSTU@OBNF or MBTU@OBNF is "EBN:
The explain plan is as follows:

NZTRM &91-"*/4&-&$5FNQ@OP'30.FNQMPZFFT8)&3&GJSTU@OBNF "EBN 03
MBTU@OBNF "EBN =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY@NFSHF
QPTTJCMF@LFZT
GVMM@OBNFGVMM@OBNF@EFTDGJSTU@OBNFGVMM@OBNF@GJSTU@OBNF@FNQ@OPMBTU@OBNF
@
[ 362 ]

LFZGJSTU@OBNFMBTU@OBNF@
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHTPSU@VOJPO
GJSTU@OBNFMBTU@OBNF@6TJOHXIFSF
SPXJOTFUXBSOJOH
TFD
You will notice that there are many options available for the optimizer to fulfill the query. It
can use any of the indexes listed in QPTTJCMF@LFZT

GVMM@OBNFGVMM@OBNF@EFTDGJSTU@OBNFGVMM@OBNF@GJSTU@OBNF@FNQ@OPMBTU
@OBNF@. The optimizer verifies all the plans and determines which plan involves the least
cost.
Some examples of costs involved in the query are accessing data from the disk, accessing
data from the memory, creating a temp table, sorting the results in the memory, and so on.
MySQL assigns a relative value for each operation and sums the total cost for each plan. It
executes the plan that involves the least cost.
How to do it...
You can control the optimizer either by passing hints to the query or by adjusting the
variable at a global or session level. You can even adjust the cost of the operations. It is
recommended to leave these values as default unless you know what you are doing.
optimizer_search_depth
The Jirgen's point of view, taken from IUUQKPSHFOMPMBOECMPHTQPUJO
JNQSPWFNFOUTGPSNBOZUBCMFKPJOTJOIUNM, states that:
"MySQL uses greedy search algorithm to to find the best order to join tables. When you
join just a few tables, there's no problem calculating the cost of all join order combinations
and then pick the best plan. However, since there are (#tables)! possible combinations, the
cost of calculating them all soon becomes too high: for five tables, e.g., there are 120
combinations which is no problem to compute. For 10 tables there are 3.6 million
combinations and for 15 tables there are 1307 billion. For this reason, MySQL makes a
trade off: use heuristics to only explore promising plans. This is supposed to significantly
reduce the number of plans MySQL needs to calculate, but at the same time you risk not
finding the best one."
[ 363 ]

The MySQL documentation says:
"The optimizer_search_depth variable tells how far into the "future" of each incomplete
plan the optimizer should look to evaluate whether it should be expanded further. Smaller
values of optimizer_search_depth may result in orders of magnitude smaller query
compilation times. For example, queries with 12, 13, or more tables may easily require
hours and even days to compile if optimizer_search_depth is close to the number of tables
in the query. At the same time, if compiled with optimizer_search_depth equal to 3 or 4,
the optimizer may compile in less than a minute for the same query. If you are unsure of
what a reasonable value is for optimizer_search_depth, this variable can be set to 0 to tell
the optimizer to determine the value automatically."
The default value of PQUJNJ[FS@TFBSDI@EFQUI is , which is very greedy, but because of
heuristics, MySQL picks up the plan very quickly. It is not clear from the documentation
why the default value is set to instead of .
If you are joining more than seven tables, you can set PQUJNJ[FS@TFBSDI@EFQUI to or
pass the optimizer hint (you will learn that in the next section). Automatic selection picks
the value of min (number of tables, seven), limiting the search depth to a reasonable value:
NZTRM 4)087"3*"#-&4-*,& PQUJNJ[FS@TFBSDI@EFQUI


]PQUJNJ[FS@TFBSDI@EFQUI]]

SPXJOTFU
TFD
NZTRM 4&5!!4&44*0/PQUJNJ[FS@TFBSDI@EFQUI
2VFSZ0,SPXTBGGFDUFE
TFD
)PXUPLOPXUIBUUIFRVFSZJTTQFOEJOHUJNFJOFWBMVBUJOHQMBOT
If you are joining 10 tables (mostly autogenerated by ORM), run an explain plan. If it takes
more time, it means that the query is spending too much time in evaluating plans. Adjust
the value of PQUJNJ[FS@TFBSDI@EFQUI (probably set to ) and check how much time the
explain plan takes. Also note down the change in plans when you adjust the value of
PQUJNJ[FS@TFBSDI@EFQUI.
[ 364 ]

optimizer_switch
The PQUJNJ[FS@TXJUDI system variable is a set of flags. You can set each of those flags to
0/ or 0'' to enable or disable the corresponding optimizer behavior. You can set it at the
session level or global level dynamically. If you adjust the optimizer switch at the session
level, all the queries in that session are affected, and if it is at the global level, all queries are
affected.
For example, you have noticed that the preceding query, 4&-&$5FNQ@OP'30.
FNQMPZFFT8)&3&GJSTU@OBNF "EBN 03MBTU@OBNF "EBN , is using
TPSU@VOJPO
GJSTU@OBNFMBTU@OBNF@. If you think that optimization is not correct for
that query, you can adjust PQUJNJ[FS@TXJUDI to switch to another optimization:
NZTRM 4)087"3*"#-&4-*,& PQUJNJ[FS@TXJUDI =(
SPX
7BSJBCMF@OBNFPQUJNJ[FS@TXJUDI
7BMVF
JOEFY@NFSHFPOJOEFY@NFSHF@VOJPOPOindex_merge_sort_union=onJOEFY@NFSHF@J
OUFSTFDUJPOPOFOHJOF@DPOEJUJPO@QVTIEPXOPOJOEFY@DPOEJUJPO@QVTIEPXOPONSS
PONSS@DPTU@CBTFEPOCMPDL@OFTUFE@MPPQPOCBUDIFE@LFZ@BDDFTTPGGNBUFSJBMJ
[BUJPOPOTFNJKPJOPOMPPTFTDBOPOGJSTUNBUDIPOEVQMJDBUFXFFEPVUPOTVCRVF
SZ@NBUFSJBMJ[BUJPO@DPTU@CBTFEPOVTF@JOEFY@FYUFOTJPOTPODPOEJUJPO@GBOPVU@G
JMUFSPOEFSJWFE@NFSHFPO
SPXJOTFU
TFD
Initially, JOEFY@NFSHF@VOJPO is on:

MBTU@OBNF "EBN =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY@NFSHF
QPTTJCMF@LFZT
@
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
Extra: Using sort_union(first_name,last_name_2); Using where
SPXJOTFUXBSOJOH
TFD
[ 365 ]

The optimizer is able to use TPSU@VOJPO:

NZTRM 4&5!!4&44*0/PQUJNJ[FS@TXJUDIJOEFY@NFSHF@TPSU@VOJPOPGG
2VFSZ0,SPXTBGGFDUFE
TFD
You can turn off JOEFY@NFSHF@TPSU@VOJPO optimization at the session level so that only
queries in this session are affected:
NZTRM 4)087"3*"#-&4-*,& PQUJNJ[FS@TXJUDI =(
SPX
7BSJBCMF@OBNFPQUJNJ[FS@TXJUDI
7BMVF
JOEFY@NFSHFPOJOEFY@NFSHF@VOJPOPOindex_merge_sort_union=offJOEFY@NFSHF@
JOUFSTFDUJPOPOFOHJOF@DPOEJUJPO@QVTIEPXOPOJOEFY@DPOEJUJPO@QVTIEPXOPONS
SPONSS@DPTU@CBTFEPOCMPDL@OFTUFE@MPPQPOCBUDIFE@LFZ@BDDFTTPGGNBUFSJBM
J[BUJPOPOTFNJKPJOPOMPPTFTDBOPOGJSTUNBUDIPOEVQMJDBUFXFFEPVUPOTVCRV
FSZ@NBUFSJBMJ[BUJPO@DPTU@CBTFEPOVTF@JOEFY@FYUFOTJPOTPODPOEJUJPO@GBOPVU@
GJMUFSPOEFSJWFE@NFSHFPO
SPXJOTFU
TFD
You will notice the plan change after JOEFY@NFSHF@TPSU@VOJPO is turned off; it is no
longer using TPSU@VOJPO optimization:
MBTU@OBNF "EBN =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY
QPTTJCMF@LFZT
@
LFZGVMM@OBNF
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
SPXJOTFUXBSOJOH
TFD
You can further find that, in this case, using TPSU@VOJPO is the best choice. Refer
to IUUQTEFWNZTRMDPNEPDSFGNBOFOTXJUDIBCMFPQUJNJ[BUJPOTIUNM for more
details on all types of optimizer switches.
[ 366 ]

Optimizer hints
Instead of adjusting the optimizer switch or PQUJNJ[FS@TFBSDI@EFQUI variables at
session level, you can hint the optimizer to use, or not to use, certain optimizations. The
scope of the optimizer hint is limited to the statement that gives you finer control over the
queries, whereas the optimizer switch can be at session or global level.
Again, take the example of the preceding query; if you feel that using TPSU@VOJPO is not
optimal, you can turn it off by passing it as a hint in the query itself:
NZTRM &91-"*/4&-&$5/*+ NO_INDEX_MERGE(employees first_name,last_name_2)
*/'30.FNQMPZFFT8)&3&GJSTU@OBNF "EBN 03MBTU@OBNF "EBN =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQF"--
QPTTJCMF@LFZT
@
LFZ/6--
LFZ@MFO/6--
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHXIFSF
SPXJOTFUXBSOJOH
TFD
Remember that in the redundant index section, we dropped the redundant index to find
which plan was better. Instead, you can use the optimizer hint to ignore the intersect of
GSPN@EBUF and GSPN@EBUF@:
NZTRM &91-"*/4&-&$5 /0@*/%&9@.&3(&

TGSPN@EBUFGSPN@EBUF@
FFNQ@OPTBMBSZ'30.TBMBSJFTT+0*/FNQMPZFFTF0/TFNQ@OPFFNQ@OP
8)&3&GSPN@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZT13*."3:GSPN@EBUFGSPN@EBUF@
key: from_date
LFZ@MFO
ref: const
[ 367 ]

SPXT
GJMUFSFE
&YUSB/6--
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFSFG
LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTTFNQ@OP
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPXTJOTFUXBSOJOH
TFD
Another good example of using optimizer hints is setting the +0*/ order:
TFNQ@OPFFNQ@OP8)&3&
GJSTU@OBNF "EBN 03MBTU@OBNF "EBN 03%&3#:
GSPN@EBUF%&4$=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFJOEFY@NFSHF
QPTTJCMF@LFZT
13*."3:GVMM@OBNFGVMM@OBNF@EFTDGJSTU@OBNFGVMM@OBNF@GJSTU@OBNF@FNQ@OPM
BTU@OBNF@
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
GJSTU@OBNFMBTU@OBNF@6TJOHXIFSF6TJOH
UFNQPSBSZ6TJOHGJMFTPSU
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQFSFG
LFZ13*."3:
LFZ@MFO
[ 368 ]

SFGFNQMPZFFTFFNQ@OP
SPXT
GJMUFSFE
&YUSB/6--
SPXTJOTFUXBSOJOH
TFD
In the preceding query, the optimizer is first considering the FNQMPZFFT table, and joining
with the TBMBSJFT table. You can change that by passing the hint, +0*/@03%&3
TF
:
NZTRM &91-"*/4&-&$5/*+ JOIN_ORDER(s, e) */FFNQ@OPTBMBSZ'30.

TBMBSJFTT+0*/FNQMPZFFTF0/TFNQ@OPFFNQ@OP8)&3&
GJSTU@OBNF "EBN
03MBTU@OBNF "EBN 03%&3#:GSPN@EBUF%&4$=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQF"--
LFZ/6--
LFZ@MFO/6--
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHGJMFTPSU
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFFR@SFG
QPTTJCMF@LFZT
13*."3:GVMM@OBNFGVMM@OBNF@EFTDGJSTU@OBNFGVMM@OBNF@GJSTU@OBNF@FNQ@OPM
BTU@OBNF@
LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTTFNQ@OP
SPXT
GJMUFSFE
&YUSB6TJOHXIFSF
SPXTJOTFUXBSOJOH
TFD
[ 369 ]

You will now notice that the TBMBSJFT table is considered first, which avoids creating a
temporary table, but it is going for a full table scan on the TBMBSJFT table.
Another use case of optimizer hints is as follows: rather than setting the session variables
for each statement or session, you can set them only for the statement. Suppose you are
using an 03%&3#: clause that sorts the query results, but you do not have index on the
03%&3#: clause. Optimizer makes use of TPSU@CVGGFS@TJ[F to speed up sorting. By
default, the value of TPSU@CVGGFS@TJ[F is ,. If TPSU@CVGGFS@TJ[F is not sufficient,
the number of merge passes that the sort algorithm has to do increases. You can measure
this through the session variable TPSU@NFSHF@QBTTFT:
NZTRM 4)084&44*0/TUBUVT-*,& TPSU@NFSHF@QBTTFT


]4PSU@NFSHF@QBTTFT]]

SPXJOTFU
TFD
NZTRM QBHFSHSFQSPXTJOTFU4&-&$5'30.FNQMPZFFT03%&3#:IJSF@EBUF

%&4$OPQBHFS
1"(&3TFUUP HSFQSPXTJOTFU
SPXTJOTFU
TFD
1"(&3TFUUPTUEPVU



SPXJOTFU
TFD
You will notice that MySQL did not have enough TPSU@CVGGFS@TJ[F, and it has to do
eight TPSU@NFSHF@QBTTFT. You can set TPSU@CVGGFS@TJ[F to some large value such
as . through an optimizer hint and check TPSU@NFSHF@QBTTFT:



SPXJOTFU
TFD
NZTRM QBHFSHSFQSPXTJOTFU4&-&$5/*+ SET_VAR(sort_buffer_size = 16M)
[ 370 ]

*/'30.FNQMPZFFT03%&3#:IJSF@EBUF%&4$OPQBHFS
1"(&3TFUUP HSFQSPXTJOTFU
SPXTJOTFU
TFD
1"(&3TFUUPTUEPVU



SPXJOTFU
TFD
You will notice that TPSU@NFSHF@QBTTFT is when TPSU@CVGGFS@TJ[F is set to ..
It is highly recommended to optimize your queries by using indexes rather than relying on
TPSU@CVGGFS@TJ[F. You can consider increasing the TPSU@CVGGFS@TJ[F value to speed
up 03%&3#: or (3061#: operations that cannot be improved with query optimization or
improved indexing.
Using 4&5@7"3, you can set PQUJNJ[FS@TXJUDI at the statement level:

NZTRM &91-"*/4&-&$5/*+ SET_VAR(optimizer_switch =
'index_merge_sort_union=off') */FFNQ@OPTBMBSZ'30.TBMBSJFTT+0*/
FNQMPZFFTF0/TFNQ@OPFFNQ@OP8)&3&GSPN@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFJOEFY
LFZOBNF
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQFFR@SFG
LFZ13*."3:
LFZ@MFO
[ 371 ]

SFGFNQMPZFFTFFNQ@OPDPOTU
SPXT
GJMUFSFE
&YUSB/6--
SPXTJOTFUXBSOJOH
TFD
You can also set the maximum execution time for a query, meaning the query is
automatically terminated after the specified time using ."9@&9&$65*0/@5*.&
NJMMJ
TFDPOET:
NZTRM 4&-&$5/*+ MAX_EXECUTION_TIME(100) */'30.FNQMPZFFT03%&3#:

IJSF@EBUF%&4$
&3303
):4PSUBCPSUFE2VFSZFYFDVUJPOXBTJOUFSSVQUFENBYJNVN
TUBUFNFOUFYFDVUJPOUJNFFYDFFEFE
You can hint many other things to the optimizer, refer to IUUQTEFWNZTRMDPNEPD
SFGNBOFOPQUJNJ[FSIJOUTIUNM to get complete list and more examples.
Adjusting the optimizer cost model

To generate execution plans, the optimizer uses a cost model that is based on estimates of
the cost of various operations that occur during query execution. The optimizer has a set of
compiled-in default cost constants available to it, to make decisions regarding execution
plans. You can adjust them by updating or inserting the NZTRMFOHJOF@DPTU table and
executing the '-64)015*.*;&3@$0454 command:
NZTRM 4&-&$5'30.NZTRMFOHJOF@DPTU=(
SPX
FOHJOF@OBNF*OOP%#
EFWJDF@UZQF
DPTU@OBNFJP@CMPDL@SFBE@DPTU
cost_value: 1
MBTU@VQEBUF
DPNNFOU/6--
EFGBVMU@WBMVF
SPX
FOHJOF@OBNF*OOP%#
EFWJDF@UZQF
DPTU@OBNFNFNPSZ@CMPDL@SFBE@DPTU
DPTU@WBMVF
MBTU@VQEBUF
DPNNFOU/6--
EFGBVMU@WBMVF
SPXTJOTFU
TFD
[ 372 ]

Suppose you have a superfast disk; you can decrease the DPTU@WBMVF for
JP@CMPDL@SFBE@DPTU:
NZTRM 61%"5&NZTRMFOHJOF@DPTU4&5DPTU@WBMVF8)&3&
DPTU@OBNF JP@CMPDL@SFBE@DPTU
2VFSZ0,SPXBGGFDUFE
TFD
NZTRM '-64)015*.*;&3@$0454
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5'30.NZTRMFOHJOF@DPTU=(

SPX
FOHJOF@OBNF*OOP%#
EFWJDF@UZQF
DPTU@OBNFJP@CMPDL@SFBE@DPTU
cost_value: 0.5
MBTU@VQEBUF
DPNNFOU/6--
EFGBVMU@WBMVF
SPX
FOHJOF@OBNF*OOP%#
EFWJDF@UZQF
DPTU@OBNFNFNPSZ@CMPDL@SFBE@DPTU
DPTU@WBMVF
MBTU@VQEBUF
DPNNFOU/6--
EFGBVMU@WBMVF
SPXTJOTFU
TFD
To know more about the optimizer cost model, refer to IUUQTEFWNZTRMDPNEPD

SFGNBOFODPTUNPEFMIUNM.
Using index hints

Using index hints, you can hint the optimizer to use or ignore indexes. This is different from
optimizer hints. In optimizer hints, you hint the optimizer to use or ignore certain
optimization methods. Index and optimizer hints can be used separately or together to
achieve the desired plan. Index hints are specified following a table name.
When you are executing a complex query involving multiple table joins, and if the
optimizer is taking too much time in evaluating the plans, you can determine the best plan
and give it a hint to the query. But make sure that the plan you are suggesting is the best
and should work in all cases.
[ 373 ]

How to do it...
Take the same query where you evaluated the use of the redundant index as an example; it
is using JOUFSTFDU
GSPN@EBUFGSPN@EBUF@. By passing the optimizer hint

/0@*/%&9@.&3(&
TGSPN@EBUFGSPN@EBUF@, you avoided the use of intersect.
You can achieve the same behavior by hinting the optimizer to ignore the GSPN@EBUF@
index:
NZTRM &91-"*/4&-&$5FFNQ@OPTBMBSZ'30.TBMBSJFTTIGNORE
INDEX(from_date_2)+0*/FNQMPZFFTF0/TFNQ@OPFFNQ@OP8)&3&
GSPN@EBUF =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFT
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZT13*."3:GSPN@EBUF
LFZGSPN@EBUF
LFZ@MFO
SFGDPOTU
SPXT
GJMUFSFE
&YUSB/6--
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFF
QBSUJUJPOT/6--
UZQFSFG
LFZ13*."3:
LFZ@MFO
SFGFNQMPZFFTTFNQ@OP
SPXT
GJMUFSFE
&YUSB6TJOHJOEFY
SPXTJOTFUXBSOJOH
TFD
Another use case is hinting the optimizer and saving the cost of evaluating multiple plans.
Consider the following FNQMPZFFT table and the query (the same as the one discussed at
the beginning of the Controlling query optimizer section):
SPX
5BCMFFNQMPZFFT
[ 374 ]

AFNQ@OPAJOU
/05/6--
AGJSTU@OBNFAWBSDIBS
/05/6--
AMBTU@OBNFAWBSDIBS
/05/6--
AHFOEFSAFOVN
. ' /05/6--
13*."3:,&:
AFNQ@OPA
,&:AMBTU@OBNFA
,&:AGVMM@OBNFA
,&:AGVMM@OBNF@EFTDA
,&:AGJSTU@OBNFA
AGJSTU@OBNFA
,&:AGVMM@OBNF@A
,&:AMBTU@OBNF@A
AMBTU@OBNFA

&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD

MBTU@OBNF "EBN =(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY@NFSHF
QPTTJCMF@LFZT
@
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
SPXJOTFUXBSOJOH
TFD
You can see that the optimizer has to evaluate the indexes
GVMM@OBNF, GVMM@OBNF@EFTD, GJSTU@OBNF, GVMM@OBNF@, GJSTU@OBNF@FNQ@OP, MBTU@
OBNF@ to arrive at the best plan. You can hint the optimizer by passing 64&
*/%&9
GJSTU@OBNFMBTU@OBNF@, which will eliminate scanning of other indexes:
NZTRM &91-"*/4&-&$5FNQ@OP'30.FNQMPZFFT64&
*/%&9
GJSTU@OBNFMBTU@OBNF@8)&3&GJSTU@OBNF "EBN 03MBTU@OBNF "EBN =(
SPX
JE
[ 375 ]

TFMFDU@UZQF4*.1-&
UBCMFFNQMPZFFT
QBSUJUJPOT/6--
UZQFJOEFY@NFSHF
possible_keys: first_name,last_name_2
key: first_name,last_name_2
LFZ@MFO
SFG/6--
SPXT
GJMUFSFE
SPXJOTFUXBSOJOH
TFD
Since this is a simple query and the table is very small, the performance gain is negligible.
The performance gain can be significant when the query is complex and is executed
millions of times an hour.
Indexing for JSON using generated columns

JSON columns cannot be indexed directly. So if you want to use an index on a JSON
column, you can extract the information using virtual columns and a created index on the
virtual column.
How to do it...
1. Consider the FNQ@EFUBJMT table that you created in $IBQUFS, Using MySQL
(Advanced), Using JSON section:
NZTRM 4)08$3&"5&5"#-&FNQ@EFUBJMT=(
SPX
5BCMFFNQ@EFUBJMT
$SFBUF5BCMF$3&"5&5"#-&AFNQ@EFUBJMTA
AFNQ@OPAJOU
/05/6--
AEFUBJMTAKTPO%&'"6-5/6--
13*."3:,&:
AFNQ@OPA
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
[ 376 ]

2. Insert a few dummy records:
NZTRM */4&35*(/03&*/50FNQ@EFUBJMT
FNQ@OPEFUBJMT7"-6&4

\MPDBUJPO*/QIPOF FNBJM
BCD!FYBNQMFDPNBEESFTT\MJOFBCDMJOFYZ[
TUSFFUDJUZ#BOHBMPSFQJO^^

EFG!FYBNQMFDPNBEESFTT\MJOFBCDMJOFYZ[
TUSFFUDJUZ%FMIJQJO^^

HIJ!FYBNQMFDPNBEESFTT\MJOFBCDMJOFYZ[
TUSFFUDJUZ.VNCBJQJO^^

KLM!FYBNQMFDPNBEESFTT\MJOFBCDMJOFYZ[
TUSFFUDJUZ%FMIJQJO^^

\MPDBUJPO64QIPOF FNBJM
NOP!FYBNQMFDPNBEESFTT\MJOFBCDMJOFYZ[
TUSFFUDJUZ4VOOZWBMFQJO^^
2VFSZ0,SPXTBGGFDUFE
TFD
3. Suppose you want to retrieve the FNQ@OP whose city is #BOHBMPSF:
NZTRM &91-"*/4&-&$5FNQ@OP'30.FNQ@EFUBJMT8)&3&
EFUBJMT BEESFTTDJUZ #BOHBMPSF=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQ@EFUBJMT
QBSUJUJPOT/6--
UZQF"--
possible_keys: NULL
key: NULL
LFZ@MFO/6--
SFG/6--
SPXT
GJMUFSFE
&YUSB6TJOHXIFSF
SPXJOTFUXBSOJOH
TFD
You will notice that the query is not able to use the index and scan all the rows.
[ 377 ]

4. You can retrieve the city as a virtual column and add an index on it:
NZTRM "-5&35"#-&FNQ@EFUBJMT"%%$0-6./DJUZWBSDIBS
"4

EFUBJMT BEESFTTDJUZ "%%*/%&9
DJUZ
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4)08$3&"5&5"#-&FNQ@EFUBJMT=(
SPX
5BCMFFNQ@EFUBJMT
$SFBUF5BCMF$3&"5&5"#-&AFNQ@EFUBJMTA
AFNQ@OPAJOU
/05/6--
AEFUBJMTAKTPO%&'"6-5/6--
`city` varchar(20) GENERATED ALWAYS AS
(json_unquote(json_extract(`details`,_utf8'$.address.city')))
VIRTUAL,
13*."3:,&:
AFNQ@OPA
KEY `city` (`city`)
&/(*/&*OOP%#%&'"6-5$)"34&5VUGNC
SPXJOTFU
TFD
5. If you check the explain plan now, you can notice that the query is able to use the
index on DJUZ and scan only one row:
NZTRM &91-"*/4&-&$5FNQ@OP'30.FNQ@EFUBJMT8)&3&
EFUBJMT BEESFTTDJUZ #BOHBMPSF=(
SPX
JE
TFMFDU@UZQF4*.1-&
UBCMFFNQ@EFUBJMT
QBSUJUJPOT/6--
UZQFSFG
QPTTJCMF@LFZTDJUZ
key: city
LFZ@MFO
ref: const
rows: 1
GJMUFSFE
&YUSB/6--
SPXJOTFUXBSOJOH
TFD
To know more about secondary indexes on generated columns, refer to IUUQTEFW

NZTRMDPNEPDSFGNBOFODSFBUFUBCMFTFDPOEBSZJOEFYFTIUNM.
[ 378 ]

Using resource groups

You can restrict the queries to use only a certain number of system resources using the
resource groups. Currently, only CPU time is a manageable resource represented by virtual
CPU (VCPU), which includes CPU cores, hyperthreads, hardware threads, and more. You
can create a resource group and assign the VCPUs to it. Apart from the CPU, the attribute to
the resource group is thread priority.
You can assign a resource group to a thread, set the default resource group at the session
level, or pass the resource group as an optimizer hint. For example, you want to run some
queries (say, reporting queries) with lowest priority; you can assign them to a resource
group that has minimum resources.
How to do it...
1. Set the $"1@4:4@/*$& capability to NZTRME:
TIFMM QTBVY]HSFQNZTRME]HSFQWHSFQ
NZTRM4M/PW
VTSTCJONZTRMEEBFNPOJ[FQJEGJMFWBSSVONZTRMENZTRMEQJE
TIFMM TVEPTFUDBQDBQ@TZT@OJDF FQVTSTCJONZTRME
TIFMM HFUDBQVTSTCJONZTRME
VTSTCJONZTRMEDBQ@TZT@OJDF FQ
2. Create a resource group using the $3&"5&3&4063$&(3061 statement. You

have to mention the resource group name, number of VCPUS, thread priority,
and type, which can be either 64&3 or 4:45&.. If you do not specify the VCPU,
all the CPUs will be used:
NZTRM $3&"5&3&4063$&(3061SFQPSU@HSPVQ
5:1&64&3
7$16
5)3&"%@13*03*5:
&/"#-&
:PVTIPVMEIBWFBUMFBTU$16TGPSUIFBCPWFSFTPVSDFHSPVQUP
DSFBUF*GZPVIBWFMFTT$16TZPVDBOVTF7$16GPSUFTUJOH
UIFFYBNQMF
The VCPU represents the CPU number as 0-5, including CPUs 0, 1, 2, 3, 4, and 5;
and 0-3, 8-9, and 11 include CPUs 0, 1, 2, 3, 8, 9, and 11.
[ 379 ]

The 5)3&"%@13*03*5: is like a nice value for the CPU; it ranges from -20 to 0 for
system resource groups and 0 to 19 for user groups. -20 is the highest priority and
19 is the lowest priority.
You can also enable or disable a resource group. By default, the resource group is
enabled at creation. A disabled group cannot have threads assigned to it.
3. After creating, you can verify the resource groups created:
NZTRM 4&-&$5'30.*/'03."5*0/@4$)&."3&4063$&@(30614=(

SPX
3&4063$&@(3061@/".&643@EFGBVMU
3&4063$&@(3061@5:1&64&3
3&4063$&@(3061@&/"#-&%
7$16@*%4
5)3&"%@13*03*5:
SPX
3&4063$&@(3061@/".&4:4@EFGBVMU
3&4063$&@(3061@5:1&4:45&.
3&4063$&@(3061@&/"#-&%
7$16@*%4
5)3&"%@13*03*5:
SPX
RESOURCE_GROUP_NAME: report_group
RESOURCE_GROUP_TYPE: USER
RESOURCE_GROUP_ENABLED: 1
VCPU_IDS: 2-3
THREAD_PRIORITY: 15
643@EFGBVMU and 4:4@EFGBVMU are default resource groups which cannot be

dropped or modified.
4. Assign a group to a thread:
NZTRM 4&53&4063$&(3061SFQPSU@HSPVQ'03UISFBE@JE
5. Set the sessions resource group; all the queries in that session will be executed
under SFQPSU@HSPVQ:
NZTRM 4&53&4063$&(3061SFQPSU@HSPVQ
[ 380 ]

6. Use the 3&4063$&@(3061 optimizer hint to execute a single statement

using SFQPSU@HSPVQ:
NZTRM 4&-&$5 3&4063$&@(3061

SFQPSU@HSPVQ'30.FNQMPZFFT
Alter and drop resource group

You can dynamically adjust the number of CPUs or UISFBE@QSJPSJUZ of a resource group.
If the system is heavily loaded, you can decrease the thread priority:
NZTRM "-5&33&4063$&(3061SFQPSU@HSPVQ7$165)3&"%@13*03*5:
2VFSZ0,SPXTBGGFDUFE
TFD
Similarly, you can increase the priority when the system is lightly loaded:
NZTRM "-5&33&4063$&(3061SFQPSU@HSPVQ7$165)3&"%@13*03*5:
2VFSZ0,SPXTBGGFDUFE
TFD
You can disable a resource group:

NZTRM "-5&33&4063$&(3061SFQPSU@HSPVQ%*4"#-&'03$&
2VFSZ0,SPXTBGGFDUFE
TFD
You can also drop a resource group using the %3013&4063$&(3061 statement:
NZTRM %3013&4063$&(3061SFQPSU@HSPVQ'03$&
If '03$& is given, the threads running are moved to the default resource group (system
threads to 4:4@EFGBVMU and user threads to 643@EFGBVMU).
If '03$& is not given, existing threads in the group continue to run until they terminate, but
new threads cannot be assigned to the group.
The resource group is restricted to the local server, and none of the
resource-group-related statements are replicated. To know more about
resource groups, refer to IUUQTEFWNZTRMDPNEPDSFGNBOFO
SFTPVSDFHSPVQTIUNM.
[ 381 ]

Using performance_schema
You can inspect the internal execution of the server at runtime using
QFSGPSNBODF@TDIFNB. This should not be confused with information schema, which is
used to inspect metadata.
There are many event consumers in QFSGPSNBODF@TDIFNB that influence the timings of a
server, such as a function call, a wait for the operating system, a stage of an SQL statement
execution (say, parsing or sorting), a single statement, or a group of statements. All the
collected information is stored in QFSGPSNBODF@TDIFNB and is not replicated.
QFSGPSNBODF@TDIFNB is enabled by default; if you want to disable it, you can set
QFSGPSNBODF@TDIFNB0'' in the NZDOG file. By default, not all the consumers and
instruments are enabled; you can turn them off/on by updating the
QFSGPSNBODF@TDIFNBTFUVQ@JOTUSVNFOUT and
QFSGPSNBODF@TDIFNBTFUVQ@DPOTVNFST tables.
How to do it...
We will see how to use the QFSGPSNBODF@TDIFNB.
Enable/disable performance_schema
To disable it, set QFSGPSNBODF@TDIFNB to :
<NZTRME>
QFSGPSNBODF@TDIFNB
Enable/disable consumers and instruments

You see a list of consumers available in the TFUVQ@DPOTVNFST table, as follows:
NZTRM 4&-&$5'30.QFSGPSNBODF@TDIFNBTFUVQ@DPOTVNFST

]/".&]&/"#-&%]

]FWFOUT@TUBHFT@DVSSFOU]/0]
]FWFOUT@TUBHFT@IJTUPSZ]/0]
]FWFOUT@TUBHFT@IJTUPSZ@MPOH]/0]
]FWFOUT@TUBUFNFOUT@DVSSFOU]:&4]
[ 382 ]

]FWFOUT@TUBUFNFOUT@IJTUPSZ]:&4]
]FWFOUT@TUBUFNFOUT@IJTUPSZ@MPOH]/0]
]FWFOUT@USBOTBDUJPOT@DVSSFOU]:&4]
]FWFOUT@USBOTBDUJPOT@IJTUPSZ]:&4]
]FWFOUT@USBOTBDUJPOT@IJTUPSZ@MPOH]/0]
]FWFOUT@XBJUT@DVSSFOU]/0]
]FWFOUT@XBJUT@IJTUPSZ]/0]
]FWFOUT@XBJUT@IJTUPSZ@MPOH]/0]
]HMPCBM@JOTUSVNFOUBUJPO]:&4]
]UISFBE@JOTUSVNFOUBUJPO]:&4]
]TUBUFNFOUT@EJHFTU]:&4]

SPXTJOTFU
TFD
Suppose you want to enable FWFOUT@XBJUT@DVSSFOU:

NZTRM 61%"5&QFSGPSNBODF@TDIFNBTFUVQ@DPOTVNFST4&5&/"#-&% :&4 8)&3&
/".& FWFOUT@XBJUT@DVSSFOU
Similarly, you can disable or enable instruments from the TFUVQ@JOTUSVNFOUT table. There
are around 1182 instruments (depending on the version):
NZTRM 4&-&$5/".&&/"#-&%5*.&%'30.TFUVQ@JOTUSVNFOUT-*.*5

]/".&]&/"#-&%]5*.&%
]

]XBJUTZODINVUFYQGT-0$,@QGT@TIBSF@MJTU]/0]/0
]
]XBJUTZODINVUFYTRM5$@-0(@.."1-0$,@UD]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@DPNNJU]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@DPNNJU@RVFVF]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@EPOF]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@GMVTI@RVFVF]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@JOEFY]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@MPH]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@CJOMPH@FOE@QPT]/0]/0
]
]XBJUTZODINVUFYTRM.:42-@#*/@-0(-0$,@TZOD]/0]/0
[ 383 ]

]

SPXTJOTFU
TFD
performance_schema tables
There are five main types of table in QFSGPSNBODF@TDIFNB. They are current events tables,
event history tables, event summary tables, object instances tables, and setup
(configuration) tables:
NZTRM 4)085"#-&4-*,& DVSSFOU

]5BCMFT@JO@QFSGPSNBODF@TDIFNB
DVSSFOU]

]FWFOUT@TUBHFT@DVSSFOU]
]FWFOUT@TUBUFNFOUT@DVSSFOU]
]FWFOUT@USBOTBDUJPOT@DVSSFOU]
]FWFOUT@XBJUT@DVSSFOU]

SPXTJOTFU
TFD
NZTRM 4)085"#-&4-*,& IJTUPSZ

IJTUPSZ]

]FWFOUT@TUBHFT@IJTUPSZ]
]FWFOUT@TUBHFT@IJTUPSZ@MPOH]
]FWFOUT@TUBUFNFOUT@IJTUPSZ]
]FWFOUT@TUBUFNFOUT@IJTUPSZ@MPOH]
]FWFOUT@USBOTBDUJPOT@IJTUPSZ]
]FWFOUT@USBOTBDUJPOT@IJTUPSZ@MPOH]
]FWFOUT@XBJUT@IJTUPSZ]
]FWFOUT@XBJUT@IJTUPSZ@MPOH]

SPXTJOTFU
TFD
NZTRM 4)085"#-&4-*,& TVNNBSZ

TVNNBSZ]

]FWFOUT@FSSPST@TVNNBSZ@CZ@BDDPVOU@CZ@FSSPS]
]FWFOUT@FSSPST@TVNNBSZ@CZ@IPTU@CZ@FSSPS]
_
_
]UBCMF@JP@XBJUT@TVNNBSZ@CZ@UBCMF]
[ 384 ]

]UBCMF@MPDL@XBJUT@TVNNBSZ@CZ@UBCMF]

SPXTJOTFU
TFD
NZTRM 4)085"#-&4-*,& TFUVQ

TFUVQ]

]TFUVQ@BDUPST]
]TFUVQ@DPOTVNFST]
]TFUVQ@JOTUSVNFOUT]
]TFUVQ@PCKFDUT]
]TFUVQ@UISFBET]
]TFUVQ@UJNFST]

SPXTJOTFU
TFD
Suppose you want to find out which file is accessed the most:
NZTRM 4&-&$5&7&/5@/".&$06/5@45"3GSPNGJMF@TVNNBSZ@CZ@FWFOU@OBNF03%&3
#:DPVOU@TUBS%&4$-*.*5

]&7&/5@/".&]$06/5@45"3]

]XBJUJPGJMFJOOPECJOOPEC@EBUB@GJMF]]
]XBJUJPGJMFTRMJP@DBDIF]]
]XBJUJPGJMFTRMCJOMPH]]
]XBJUJPGJMFJOOPECJOOPEC@MPH@GJMF]]
]XBJUJPGJMFTRMRVFSZ@MPH]]
]XBJUJPGJMFJOOPECJOOPEC@UFNQ@GJMF]]
]XBJUJPGJMFJOOPECJOOPEC@UBCMFTQBDF@PQFO@GJMF]]
]XBJUJPGJMFTRMDBTFUFTU]]
]XBJUJPGJMFTRMCJOMPH@JOEFY]]
]XBJUJPGJMFNZTZTDOG]]

SPXTJOTFU
TFD
Or you want to find out which file has taken the most time in writing:
NZTRM 4&-&$5&7&/5@/".&46.@5*.&3@83*5&'30.GJMF@TVNNBSZ@CZ@FWFOU@OBNF
03%&3#:46.@5*.&3@83*5&%&4$-*.*5

]&7&/5@/".&]46.@5*.&3@83*5&]

]XBJUJPGJMFJOOPECJOOPEC@EBUB@GJMF]]
]XBJUJPGJMFJOOPECJOOPEC@MPH@GJMF]]
]XBJUJPGJMFTRMJP@DBDIF]]
[ 385 ]

]XBJUJPGJMFTRMRVFSZ@MPH]]
]XBJUJPGJMFTRMCJOMPH]]
]XBJUJPGJMFJOOPECJOOPEC@UFNQ@GJMF]]
]XBJUJPGJMFJOOPECJOOPEC@UBCMFTQBDF@PQFO@GJMF]]
]XBJUJPGJMFTRM4%*]]
]XBJUJPGJMFTRMQJE]]
]XBJUJPGJMFBSDIJWF'3.]]

You can use the FWFOUT@TUBUFNFOUT@TVNNBSZ@CZ@EJHFTU table to get the query report,
just like you did for QURVFSZEJHFTU. Top query by amount of time taken:
NZTRM 4&-&$54$)&."@/".&EJHFTUEJHFTU@UFYUSPVOE
TVN@UJNFS@XBJU
BTBWH@UJNFDPVOU@TUBS'30.
QFSGPSNBODF@TDIFNBFWFOUT@TUBUFNFOUT@TVNNBSZ@CZ@EJHFTU03%&3#:
TVN@UJNFS@XBJU%&4$-*.*5=(
SPX
4$)&."@/".&/6--
EJHFTUGGDEBEBCD
EJHFTU@UFYU4&-&$5ATMFFQA

BWH@UJNF
DPVOU@TUBS
SPXJOTFU
TFD
Top query by number of executions:

NZTRM 4&-&$54$)&."@/".&EJHFTUEJHFTU@UFYUSPVOE
TVN@UJNFS@XBJU
BTBWH@UJNFDPVOU@TUBS'30.
QFSGPSNBODF@TDIFNBFWFOUT@TUBUFNFOUT@TVNNBSZ@CZ@EJHFTU03%&3#:DPVOU@TUBS
%&4$-*.*5=(
SPX
4$)&."@/".&FNQMPZFFT
EJHFTUGFDDGCCCBCBC
EJHFTU@UFYU*/4&35*/50ATBMBSJFTA7"-6&4

BWH@UJNF
DPVOU@TUBS
SPXJOTFU
TFD
Suppose you want to find the statistics of a particular query; rather than depending on
NZTRMTMBQ benchmark, you can check all the statistics using QFSGPSNBODF@TDIFNB:
NZTRM 4&-&$5'30.FWFOUT@TUBUFNFOUT@TVNNBSZ@CZ@EJHFTU8)&3&%*(&45@5&95

-*,& 4&-&$5FNQMPZFF03%&3 -*.*5=(
SPX
4$)&."@/".&FNQMPZFFT
%*(&45ECGGGCGCCGBDBC
%*(&45@5&95&91-"*/4&-&$5 4&5@7"3
ATPSU@CVGGFS@TJ[FAAFAAFNQ@OPAATBMBSZA'30.ATBMBSJFTAATA
[ 386 ]

+0*/AFNQMPZFFTAAFA0/ATAAFNQ@OPAAFAAFNQ@OPA8)&3&
AGJSTU@OBNFA03AMBTU@OBNFA03%&3#:AGSPN@EBUFA%&4$
$06/5@45"3
46.@5*.&3@8"*5
.*/@5*.&3@8"*5
"7(@5*.&3@8"*5
."9@5*.&3@8"*5
46.@-0$,@5*.&
46.@&33034
46.@8"3/*/(4
46.@3084@"''&$5&%
46.@3084@4&/5
46.@3084@&9".*/&%
46.@$3&"5&%@5.1@%*4,@5"#-&4
46.@$3&"5&%@5.1@5"#-&4
46.@4&-&$5@'6--@+0*/
_
'*345@4&&/
-"45@4&&/
26"/5*-&@
26"/5*-&@
26"/5*-&@
QUERY_SAMPLE_TEXT: EXPLAIN SELECT /*+ SET_VAR(sort_buffer_size =
16M) */ e.emp_no, salary FROM salaries s JOIN employees e ON
s.emp_no=e.emp_no WHERE (first_name='Adam' OR last_name='Adam') ORDER BY
from_date DESC
26&3:@4".1-&@4&&/
26&3:@4".1-&@5*.&3@8"*5
Using the sys schema

The TZT schema helps you interpret the data collected from the QFSGPSNBODF@TDIFNB in
an easy and more understandable form. QFSGPSNBODF@TDIFNB should be enabled for TZT
schema to work. To use the TZT schema to its fullest extent, you need to enable all the
consumers and timers on QFSGPSNBODF@TDIFNB, but this impacts the performance of the
server. So, enable consumers for only those that you are looking for.
A view with the Y prefix displays data in picoseconds, which is used by other tools for
further processing; other tables are human readable.
[ 387 ]

How to do it...
Enable a instrument from the TZT schema:
NZTRM $"--TZTQT@TFUVQ@FOBCMF@JOTUSVNFOU
TUBUFNFOU

]TVNNBSZ]

]&OBCMFEJOTUSVNFOUT]

SPXJOTFU
TFD
2VFSZ0,SPXTBGGFDUFE
TFD
If you want to reset to default, do this:

mysql> CALL sys.ps_setup_reset_to_default(TRUE)\G
SPX
TUBUVT3FTFUUJOHTFUVQ@BDUPST
%&-&5&'30.QFSGPSNBODF@TDIFNBTFUVQ@BDUPST8)&3&/05
)045 "/%64&3
"/%A30-&A
SPXJOTFU
TFD
_
SPX
TUBUVT3FTFUUJOHUISFBET
61%"5&QFSGPSNBODF@TDIFNBUISFBET4&5*/4536.&/5&% :&4
SPXJOTFU
TFD
2VFSZ0,SPXTBGGFDUFE
TFD
There are numerous tables in the TZT schema; some of the most used ones are shown in this
section.
Statement by type (INSERT and SELECT) from each

host
NZTRM 4&-&$5TUBUFNFOUUPUBMUPUBM@MBUFODZSPXT@TFOUSPXT@FYBNJOFE
SPXT@BGGFDUFEGVMM@TDBOT'30.TZTIPTU@TVNNBSZ@CZ@TUBUFNFOU@UZQF8)&3&
IPTU MPDBMIPTU 03%&3#:UPUBM%&4$-*.*5

]TUBUFNFOU]UPUBM]UPUBM@MBUFODZ]SPXT@TFOU]SPXT@FYBNJOFE]
SPXT@BGGFDUFE]GVMM@TDBOT]

[ 388 ]

]TFMFDU]]E]]]
]]
]2VJU]]T]]]
]]
]JOTFSU]]N]]]
]]
]1JOH]]NT]]]
]]
]TFU@PQUJPO]]NT]]]
]]

SPXTJOTFU
TFD
Statement by type from each user

NZTRM 4&-&$5TUBUFNFOUUPUBMUPUBM@MBUFODZSPXT@TFOUSPXT@FYBNJOFE
SPXT@BGGFDUFEGVMM@TDBOT'30.TZTVTFS@TVNNBSZ@CZ@TUBUFNFOU@UZQF03%&3#:
UPUBM%&4$-*.*5

]TUBUFNFOU]UPUBM]UPUBM@MBUFODZ]SPXT@TFOU]SPXT@FYBNJOFE]
SPXT@BGGFDUFE]GVMM@TDBOT]

]TFMFDU]]E]]]
]]
]2VJU]]T]]]
]]
]JOTFSU]]N]]]
]]
]1JOH]]NT]]]
]]
]TFU@PQUJPO]]NT]]]
]]

SPXTJOTFU
TFD
[ 389 ]

Redundant indexes
NZTRM 4&-&$5'30.TZTTDIFNB@SFEVOEBOU@JOEFYFT8)&3&
UBCMF@OBNF FNQMPZFFT =(
SPX
UBCMF@TDIFNBFNQMPZFFT
UBCMF@OBNFFNQMPZFFT
SFEVOEBOU@JOEFY@OBNFGJSTU@OBNF
SFEVOEBOU@JOEFY@DPMVNOTGJSTU@OBNF
SFEVOEBOU@JOEFY@OPO@VOJRVF
EPNJOBOU@JOEFY@OBNFGJSTU@OBNF@FNQ@OP
EPNJOBOU@JOEFY@DPMVNOTGJSTU@OBNFFNQ@OP
EPNJOBOU@JOEFY@OPO@VOJRVF
TVCQBSU@FYJTUT
TRM@ESPQ@JOEFY"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9
AGJSTU@OBNFA
_
SPX
UBCMF@OBNFFNQMPZFFT
SFEVOEBOU@JOEFY@OBNFMBTU@OBNF@
SFEVOEBOU@JOEFY@DPMVNOTMBTU@OBNF
SFEVOEBOU@JOEFY@OPO@VOJRVF
EPNJOBOU@JOEFY@OBNFMBTU@OBNF
EPNJOBOU@JOEFY@DPMVNOTMBTU@OBNF
EPNJOBOU@JOEFY@OPO@VOJRVF
TVCQBSU@FYJTUT
TRM@ESPQ@JOEFY"-5&35"#-&AFNQMPZFFTAAFNQMPZFFTA%301*/%&9
AMBTU@OBNF@A
SPXTJOTFU
TFD
Unused indexes
NZTRM 4&-&$5'30.TZTTDIFNB@VOVTFE@JOEFYFT8)&3&
PCKFDU@TDIFNB FNQMPZFFT

]PCKFDU@TDIFNB]PCKFDU@OBNF]JOEFY@OBNF]

]FNQMPZFFT]EFQBSUNFOUT]EFQU@OBNF]
]FNQMPZFFT]EFQU@FNQ]EFQU@OP]
]FNQMPZFFT]EFQU@NBOBHFS]EFQU@OP]
]FNQMPZFFT]FNQMPZFFT]OBNF]
]FNQMPZFFT]FNQMPZFFT]MBTU@OBNF]
]FNQMPZFFT]FNQMPZFFT]GVMM@OBNF]
]FNQMPZFFT]FNQMPZFFT]GVMM@OBNF@EFTD]
]FNQMPZFFT]FNQMPZFFT]GJSTU@OBNF]
[ 390 ]

]FNQMPZFFT]FNQMPZFFT]GVMM@OBNF@]
]FNQMPZFFT]FNQMPZFFT]GJSTU@OBNF@FNQ@OP]
]FNQMPZFFT]FNQMPZFFT]MBTU@OBNF@]
]FNQMPZFFT]FNQMPZFFT@NHS]NBOBHFS@JE]
]FNQMPZFFT]FNQMPZFFT@UFTU]OBNF]
]FNQMPZFFT]FNQ@EFUBJMT]DJUZ]

SPXTJOTFU
TFD
Statements executed from each host

NZTRM 4&-&$5'30.TZTIPTU@TVNNBSZ03%&3#:TUBUFNFOUT%&4$-*.*5=(
SPX
IPTUMPDBMIPTU
TUBUFNFOUT
TUBUFNFOU@MBUFODZE
TUBUFNFOU@BWH@MBUFODZNT
UBCMF@TDBOT
GJMF@JPT
GJMF@JP@MBUFODZN
DVSSFOU@DPOOFDUJPOT
UPUBM@DPOOFDUJPOT
VOJRVF@VTFST
DVSSFOU@NFNPSZCZUFT
UPUBM@NFNPSZ@BMMPDBUFECZUFT
SPXJOTFU
TFD
Table statistics
NZTRM 4&-&$5'30.TZTTDIFNB@UBCMF@TUBUJTUJDT-*.*5=(
SPX
UBCMF@OBNFFNQMPZFFT
UPUBM@MBUFODZI
SPXT@GFUDIFE
GFUDI@MBUFODZI
SPXT@JOTFSUFE
JOTFSU@MBUFODZT
SPXT@VQEBUFE
VQEBUF@MBUFODZQT
SPXT@EFMFUFE
EFMFUF@MBUFODZQT
JP@SFBE@SFRVFTUT/6--
JP@SFBE/6--
JP@SFBE@MBUFODZ/6--
[ 391 ]

JP@XSJUF@SFRVFTUT/6--
JP@XSJUF/6--
JP@XSJUF@MBUFODZ/6--
JP@NJTD@SFRVFTUT/6--
JP@NJTD@MBUFODZ/6--
SPXJOTFU
TFD
Table statistics with buffer

NZTRM 4&-&$5'30.TZTTDIFNB@UBCMF@TUBUJTUJDT@XJUI@CVGGFS-*.*5=(
SPX
UBCMF@OBNFFNQMPZFFT
SPXT@GFUDIFE
GFUDI@MBUFODZI
SPXT@JOTFSUFE
JOTFSU@MBUFODZT
SPXT@VQEBUFE
VQEBUF@MBUFODZQT
SPXT@EFMFUFE
EFMFUF@MBUFODZQT
_
JOOPEC@CVGGFS@BMMPDBUFE.J#
JOOPEC@CVGGFS@EBUB.J#
JOOPEC@CVGGFS@GSFF,J#
JOOPEC@CVGGFS@QBHFT
JOOPEC@CVGGFS@QBHFT@IBTIFE
JOOPEC@CVGGFS@QBHFT@PME
JOOPEC@CVGGFS@SPXT@DBDIFE
SPXJOTFU
TFD
Statement analysis
This output is similar to the output of
QFSGPSNBODF@TDIFNBFWFOUT@TUBUFNFOUT@TVNNBSZ@CZ@EJHFTU and QURVFSZ
EJHFTU.
[ 392 ]

The top query by execution count is as follows:

NZTRM 4&-&$5'30.TZTTUBUFNFOU@BOBMZTJT03%&3#:FYFD@DPVOU%&4$-*.*5
=(
SPX
RVFSZ4&-&$5AFAAFNQ@OPAATBMBSFNQ@OPA8)&3&
AGSPN@EBUFA
ECFNQMPZFFT
GVMM@TDBO
FYFD@DPVOU
FSS@DPVOU
XBSO@DPVOU
UPUBM@MBUFODZI
NBY@MBUFODZNT
BWH@MBUFODZNT
MPDL@MBUFODZN
SPXT@TFOU
SPXT@TFOU@BWH
SPXT@FYBNJOFE
SPXT@FYBNJOFE@BWH
SPXT@BGGFDUFE
SPXT@BGGFDUFE@BWH
UNQ@UBCMFT
UNQ@EJTL@UBCMFT
SPXT@TPSUFE
TPSU@NFSHF@QBTTFT
EJHFTUDDGFECGF
GJSTU@TFFO
MBTU@TFFO
SPXJOTFU
TFD
The statement that consumed maximum UNQ@EJTL@UBCMFT:

NZTRM 4&-&$5'30.TZTTUBUFNFOU@BOBMZTJT03%&3#:UNQ@EJTL@UBCMFT%&4$
-*.*5=(
SPX
RVFSZ4&-&$5ADBUAAOBNFA"4A5"#-4&ADPMAAUZQFA
8)&/5)&/
ECFNQMPZFFT
GVMM@TDBO
FYFD@DPVOU
FSS@DPVOU
XBSO@DPVOU
UPUBM@MBUFODZNT
NBY@MBUFODZNT
BWH@MBUFODZNT
MPDL@MBUFODZNT
SPXT@TFOU
[ 393 ]

SPXT@TFOU@BWH
SPXT@FYBNJOFE
SPXT@FYBNJOFE@BWH
SPXT@BGGFDUFE
SPXT@BGGFDUFE@BWH
UNQ@UBCMFT
UNQ@EJTL@UBCMFT
SPXT@TPSUFE
TPSU@NFSHF@QBTTFT
EJHFTUFDBBFGDGFEE
GJSTU@TFFO
MBTU@TFFO
SPXJOTFU
TFD
To know more about the TZT schema objects, refer to IUUQTEFWNZTRMDPNEPDSFGNBO

FOTZTTDIFNBPCKFDUJOEFYIUNM.
[ 394 ]

14
Security
Securing installation
Restricting networks and users
Password-less authentication using mysql_config_editor
Resetting the root password
Setting up encrypted connections using X509
Setting up SSL replication
Introduction
In this chapter, security aspects of MySQL are covered, which include restricting the
network, strong passwords, using SSL, access control within a database, securing an
installation, and security plugins.
Securing installation
As soon as the installation is done, it is recommended that you secure your installation
using the NZTRM@TFDVSF@JOTUBMMBUJPO utility.

Security Chapter 14
How to do it...
TIFMM NZTRM@TFDVSF@JOTUBMMBUJPO
4FDVSJOHUIF.Z42-TFSWFSEFQMPZNFOU
&OUFSQBTTXPSEGPSVTFSSPPU
5IF WBMJEBUF@QBTTXPSE QMVHJOJTJOTUBMMFEPOUIFTFSWFS
5IFTVCTFRVFOUTUFQTXJMMSVOXJUIUIFFYJTUJOHDPOGJHVSBUJPO
PGUIFQMVHJO
6TJOHFYJTUJOHQBTTXPSEGPSSPPU
&TUJNBUFETUSFOHUIPGUIFQBTTXPSE
$IBOHFUIFQBTTXPSEGPSSPPU
1SFTTZ]:GPS:FTBOZPUIFSLFZGPS/P
TLJQQJOH
#ZEFGBVMUB.Z42-JOTUBMMBUJPOIBTBOBOPOZNPVTVTFS
BMMPXJOHBOZPOFUPMPHJOUP.Z42-XJUIPVUIBWJOHUPIBWF
BVTFSBDDPVOUDSFBUFEGPSUIFN5IJTJTJOUFOEFEPOMZGPS
UFTUJOHBOEUPNBLFUIFJOTUBMMBUJPOHPBCJUTNPPUIFS
:PVTIPVMESFNPWFUIFNCFGPSFNPWJOHJOUPBQSPEVDUJPO
FOWJSPONFOU
3FNPWFBOPOZNPVTVTFST
1SFTTZ]:GPS:FTBOZPUIFSLFZGPS/PZ
4VDDFTT
/PSNBMMZSPPUTIPVMEPOMZCFBMMPXFEUPDPOOFDUGSPN
MPDBMIPTU 5IJTFOTVSFTUIBUTPNFPOFDBOOPUHVFTTBU
UIFSPPUQBTTXPSEGSPNUIFOFUXPSL
%JTBMMPXSPPUMPHJOSFNPUFMZ
4VDDFTT
#ZEFGBVMU.Z42-DPNFTXJUIBEBUBCBTFOBNFE UFTU UIBU

BOZPOFDBOBDDFTT5IJTJTBMTPJOUFOEFEPOMZGPSUFTUJOH
BOETIPVMECFSFNPWFECFGPSFNPWJOHJOUPBQSPEVDUJPO
FOWJSPONFOU
3FNPWFUFTUEBUBCBTFBOEBDDFTTUPJU
1SFTTZ]:GPS:FTBOZPUIFSLFZ
GPS/PZ
%SPQQJOHUFTUEBUBCBTF
4VDDFTT
3FNPWJOHQSJWJMFHFTPOUFTUEBUBCBTF
4VDDFTT
3FMPBEJOHUIFQSJWJMFHFUBCMFTXJMMFOTVSFUIBUBMMDIBOHFT
[ 396 ]

Security Chapter 14
NBEFTPGBSXJMMUBLFFGGFDUJNNFEJBUFMZ
3FMPBEQSJWJMFHFUBCMFTOPX
4VDDFTT
"MMEPOF
By default, the NZTRME process runs under the NZTRM user. You can also run NZTRME under
another user by changing the ownership of all the directories used by NZTRME (such
as EBUBEJS, the CJOMPH directory if any, tablespaces in other disks, and so on) and adding
VTFSVTFS in NZDOG. Refer to IUUQTEFWNZTRMDPNEPDSFGNBOFODIBOHJOH
NZTRMVTFSIUNM to know more about changing the MySQL user.
It is strongly recommended not to run NZTRME as a Unix root user. One reason is that any
user with the '*-& privilege is able to cause the server to create files as root.
The FILE privilege

Be cautious while granting the '*-& privilege to any user because the user can write a file
anywhere in the filesystem with privileges of the NZTRME daemon, which includes the
server's EBUBEJSFDUPSZ. However, they cannot overwrite existing files. Also, users can
read any file accessible to MySQL (or the user that runs NZTRME) into a database table. '*-&
is a global privilege, meaning you cannot restrict it to a particular database:
NZTRM 4)08(3"/54

](SBOUTGPSDPNQBOZ@BENJO!]

](3"/5'*-&0/50ADPNQBOZ@BENJOA!AA]
](3"/54&-&$5*/4&35$3&"5&0/ADPNQBOZA50ADPNQBOZ@BENJOA!AA]

SPXTJOTFU
TFD
NZTRM 64&DPNQBOZ
%BUBCBTFDIBOHFE
NZTRM $3&"5&5"#-&IBDL
JCEGJMFMPOHCMPC
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM -0"%%"5"*/'*-& WBSMJCNZTRMFNQMPZFFTTBMBSJFTJCE */505"#-&

IBDL$)"3"$5&34&5MBUJO'*&-%45&3.*/"5&%#: !!!!!
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM 4&-&$5'30.IBDL
[ 397 ]

Security Chapter 14
Notice that the company user with the '*-& privilege is able to read data from the
FNQMPZFFT table.
You do not need to worry about the preceding hack as the location in which files can be
read and written is limited to WBSMJCNZTRMGJMFT by default, using the
TFDVSF@GJMF@QSJW variable. The problem arises when you set the TFDVSF@GJMF@QSJW
variable to /6--, an empty string, the MySQL EBUBEJSFDUPSZ, or any sensitive directory
that MySQL has access to (for example, the tablespaces outside the MySQL EBUB
EJSFDUPSZ). If you set TFDVSF@GJMF@QSJW to a non-existent directory, it leads to an error.
It is recommended to leave TFDVSF@GJMF@QSJW as default:

NZTRM 4)087"3*"#-&4-*,& TFDVSF@GJMF@QSJW


]TFDVSF@GJMF@QSJW]WBSMJCNZTRMGJMFT]

SPXJOTFU
TFD
Never give anyone access to the NZTRMVTFS table. To know more about
the security guidelines, refer to IUUQTEFWNZTRMDPNEPDSFGNBO
FOTFDVSJUZHVJEFMJOFTIUNM and IUUQTEFWNZTRMDPNEPDSFGNBO
FOTFDVSJUZBHBJOTUBUUBDLIUNM.
Restricting networks and users

Do not open your database to the whole network, meaning the port on which MySQL runs
() should not be accessed from other networks. It should be open only to the
application server. You can set up a firewall using iptables or the IPTUBDDFTT file to
restrict access to port . If you are using MySQL on the cloud, the provider will also
give a firewall.
[ 398 ]

Security Chapter 14
How to do it...
To test this, you can use UFMOFU:
TIFMM UFMOFUNZTRMJQ
JGUFMOFUJTOPUJOTUBMMFEZPVDBOJOTUBMMJUPSVTFOD
OFUDBU
If telnet hangs or the connection is refused, it means that the port is closed. Please note that
if you see an output like this, it means that the port is not blocked:
TIFMM UFMOFU
5SZJOH
Connected to 188.158.186.35.bc.googleusercontent.com.
Escape character is '^]'.
')PTU JTOPUBMMPXFEUPDPOOFDUUPUIJT.Z42-
TFSWFS$POOFDUJPODMPTFECZGPSFJHOIPTU
It means that the port is open but MySQL is restricting the access.
When creating users, avoid giving access from anywhere (the option). Restrict access to an
IP range or subdomain. Also restrict the user to access only the database that is needed. For
example, the SFBE@POMZ user of the FNQMPZFFT database should not be able to access other
databases:
NZTRM $3&"5&VTFS FNQMPZFF@SFBE@POMZ !'10.10.%.%'*%&/5*'*&%#:
4USOH@1!XPSE
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM (3"/54&-&$50/FNQMPZFF50 FNQMPZFF@SFBE@POMZ !'10.10.%.%'

2VFSZ0,SPXTBGGFDUFE
TFD
The FNQMPZFF@SFBE@POMZ user will be able to access only from the subnet and
access only the FNQMPZFF database.
Password-less authentication using

mysql_config_editor
Whenever you enter a password using a command-line client, you might have noticed the
following warning:
TIFMM NZTRMVECBENJOQ USP/H1!XSE
mysql: [Warning] Using a password on the command line interface can be
insecure.
[ 399 ]

Security Chapter 14
(1-
_
NZTRM
If you do not pass the password in the command line and enter when it prompts, you won't
get that warning:
TIFMM NZTRMVECBENJOQ
&OUFSQBTTXPSE
(1-
_
NZTRM
However, when you are developing some scripts over the client utilities, it is difficult to use
with password prompt. One way to avoid this is by storing the password in the NZDOG
file in the IPNF directory. The NZTRM command-line utility, by default, reads the NZDOG
file and does not ask for a password:
TIFMM DBU)0.&NZDOG
<DMJFOU>
VTFSECBENJO
QBTTXPSEUSP/H1!XSE
TIFMM NZTRM
(1-
_
NZTRM
Notice that you are able to connect without giving any password, but this leads to a security
concern; the password is in cleartext. To overcome this, MySQL has introduced
NZTRM@DPOGJH@FEJUPS, which stores the password in encrypted format. The file can be
decrypted by client programs (only used in memory) to connect to the server.
[ 400 ]

Security Chapter 14
How to do it...
Create the NZMPHJODOG file using NZTRM@DPOGJH@FEJUPS:
TIFMM NZTRM@DPOGJH@FEJUPSTFUMPHJOQBUIECBENJO@MPDBMIPTUMPDBMIPTU
VTFSECBENJOQBTTXPSE
&OUFSQBTTXPSE
You can add multiple hostnames and passwords by changing the login path. If the
password is changed, you can run this utility again, which updates the password in the file:
TIFMM NZTRM@DPOGJH@FEJUPSTFUMPHJOQBUIECBENJO@SFNPUF
IPTUVTFSECBENJOQBTTXPSE
&OUFSQBTTXPSE
If you want to log in to using the ECBENJO user, you can simply execute
NZTRMMPHJOQBUIECBENJO@SFNPUF:
TIFMM NZTRMMPHJOQBUIECBENJO@SFNPUF
_
NZTRM 4&-&$5!!TFSWFS@JE

]!!TFSWFS@JE]

]]

SPXJOTFU
TFD
To connect to MPDBMIPTU, you can simply execute NZTRMPSNZTRMMPHJO

QBUIECBENJO@MPDBM:
TIFMM NZTRM
_

]!!TFSWFS@JE]

]]

SPXJOTFU
TFD
TIFMM NZTRMMPHJOQBUIECBENJO@MPDBM
[ 401 ]

Security Chapter 14
_

]!!TFSWFS@JE]

]]

SPXJOTFU
TFD
If the password for ECBENJO is the same across all your servers, you can connect to any of
them by specifying the hostname. You do not need to specify the password:
TIFMM NZTRMI
_

]!!TFSWFS@JE]

]]

SPXJOTFU
TFD
If you want to print all the login paths, do this:

TIFMM NZTRM@DPOGJH@FEJUPSQSJOUBMM
<ECBENJO@MPDBM>
VTFSECBENJO
QBTTXPSE
IPTUMPDBMIPTU
<ECBENJO@SFNPUF>
VTFSECBENJO
QBTTXPSE
IPTU
You can notice that the utility masks the passwords. If you try to read the file, you will only
see gibberish characters:
TIFMM DBUNZMPHJODOG
[]#E6C[8HR#N7,*
I C@!7WMJ+9AR1
[ 402 ]

Security Chapter 14
This utility only helps you to avoid storing cleartext passwords and ease
the process of connecting to MySQL. There are many methods to decrypt
the passwords stored in the NZMPHJODOG file. So do not think that the
password is safe if you use NZTRM@DPOGJH@FEJUPS. Instead of creating
the NZMPHJODOG file every time, you can copy this file to other servers
also (this works only if the username and password are the same).
Resetting the root password

If you forget the root password, you can reset it by two methods, explained as follows.
How to do it...
Using init-file
On Unix-like systems, you stop the server and start it by specifying init-file. You can save
the "-5&364&3 SPPU ! MPDBMIPTU *%&/5*'*&%#: /FXUSPOH1BTT SQL
code in that file. MySQL executes the contents of the file at startup, changing the password
of the root user:
1. Stop the server:
TIFMM QHSFQNZTRME
2. Save the SQL code in WBSMJCNZTRMNZTRMJOJUQBTTXPSE; make it

readable to MySQL only:
TIFMM WJWBSMJCNZTRMNZTRMJOJUQBTTXPSE
"-5&364&3 SPPU ! MPDBMIPTU *%&/5*'*&%#: /FXUSPOH1BTT
TIFMM TVEPDINPEWBSMJCNZTRMNZTRMJOJUQBTTXPSE
TIFMM TVEPDIPXONZTRMNZTRMWBSMJCNZTRMNZTRMJOJUQBTTXPSE
[ 403 ]

Security Chapter 14
3. Start the MySQL server with the JOJUGJMF option and other options as
required:
TIFMM TVEPVNZTRMVTSTCJONZTRMEEBFNPOJ[FQJE
GJMFWBSSVONZTRMENZTRMEQJEVTFSNZTRMJOJU
GJMFWBSMJCNZTRMNZTRMJOJUQBTTXPSE
NZTRMEXJMMMPHFSSPSTUPWBSMPHNZTRMEMPH
NZTRMEJTSVOOJOHBTQJE
4. Verify the error log file:
TIFMM TVEPUBJMWBSMPHNZTRMEMPH
_
2017-11-27T07:32:25.219483Z 0 [Note] Execution of init_file
'/var/lib/mysql/mysql-init-password' started.
5;</PUF>&WFOU4DIFEVMFSTDIFEVMFS
UISFBETUBSUFEXJUIJE
2017-11-27T07:32:25.223528Z 0 [Note] Execution of init_file
'/var/lib/mysql/mysql-init-password' ended.
5;</PUF>VTSTCJONZTRMESFBEZGPS
DPOOFDUJPOT7FSTJPO SDMPH TPDLFU
WBSMJCNZTRMNZTRMTPDL QPSU.Z42-$PNNVOJUZ4FSWFS

(1-
5. Verify that you are able to log in with the new password:
TIFMM NZTRMVSPPUQ /FXUSPOH1BTT

CFJOTFDVSF
(1-
_
NZTRM
6. Now, the most important thing! Remove the WBSMJCNZTRMNZTRMJOJU

QBTTXPSE file:
TIFMM TVEPSNSGWBSMJCNZTRMNZTRMJOJUQBTTXPSE
7. Optionally, you can stop the server and start it normally without the JOJU
GJMF option.
[ 404 ]

Security Chapter 14
Using --skip-grant-tables
In this method, you stop the server and start it by specifying TLJQHSBOUUBCMFT,
which will not load the grant tables. You can connect to the server as root without a
password and set the password. Since the server runs without grants, it is possible for users
from other networks to connect to the server. So as of MySQL 8.0.3, TLJQHSBOU
UBCMFT automatically enables TLJQOFUXPSLJOH, which does not allow remote
connections:
1. Stop the server:
2. Start the server with the TLJQHSBOUUBCMFT option:
TIFMM TVEPVNZTRMVTSTCJONZTRMEEBFNPOJ[FQJE
GJMFWBSSVONZTRMENZTRMEQJEVTFSNZTRMTLJQHSBOUUBCMFT
NZTRMEXJMMMPHFSSPSTUPWBSMPHNZTRMEMPH
NZTRMEJTSVOOJOHBTQJE
3. Connect to MySQL without a password, execute '-64)13*7*-&(&4 to reload

the grants, and alter the user to change the password:
TIFMM NZTRMVSPPU
(1-
_
NZTRM '-64)13*7*-&(&4
2VFSZ0,SPXTBGGFDUFE
TFD
NZTRM "-5&364&3 SPPU ! MPDBMIPTU *%&/5*'*&%#:

/FXUSPOH1BTT
2VFSZ0,SPXTBGGFDUFE
TFD
4. Test the connection to MySQL with the new password:
TIFMM NZTRMVSPPUQ /FXUSPOH1BTT

CFJOTFDVSF
_
NZTRM
[ 405 ]

Security Chapter 14
5. Restart the MySQL server:
NZTRM4M
VTFSNZTRMTLJQHSBOUUBCMFT
TIFMM TVEPLJMM
NZTRM4M
Setting up encrypted connections using

X509
If the connections between the client and MySQL server are not encrypted, anyone who has
access to the network could inspect the data. If the client and server are in different data
centers, it is recommended to use encrypted connections. By default, MySQL 8 uses an
encrypted connection, but if the encrypted connection fails, it falls back to an unencrypted
connection. You can test that by checking the status of the 4TM@DJQIFS variable. If the
connection is established by MPDBMIPTU, cipher won't be used:
NZTRM 4)0845"564-*,& 4TM@DJQIFS


]4TM@DJQIFS]%)&34""&44)"]

SPXJOTFU
TFD
If you are not using SSL, 4TM@DJQIFS will be blank.
You can mandate some users to connect only through an encrypted connection (by
specifying the 3&26*3&44- clause) and leave it as optional for other users.
[ 406 ]

Security Chapter 14
MySQL supports encrypted connections between clients and the server using the TLS
(Transport Layer Security) protocol. TLS is sometimes referred to as SSL (Secure Sockets
Layer) but MySQL does not actually use the SSL protocol for encrypted connections
because its encryption is weak. TLS uses encryption algorithms to ensure that data received
over a public network can be trusted. It has mechanisms to detect data change, loss, or
replay. TLS also incorporates algorithms that provide identity verification using the X509
standard.
In this section, you will learn about setting up SSL connections using X509.
All the SSL (X509) related files (DBQFN, TFSWFSDFSUQFN, TFSWFSLFZQFN, DMJFOU
DFSUQFN, and DMJFOULFZQFN) are created by MySQL during installation and kept
under the EBUBEJSFDUPSZ. The server needs the DBQFN, TFSWFSDFSUQFN,
and TFSWFSLFZQFN files, and the clients use the DMJFOUDFSUQFN and DMJFOU
LFZQFN files to connect to the server.
How to do it...
1. Verify the files in the EBUBEJSFDUPSZ, update NZDOG, restart the server, and
check the SSL-related variables. In MySQL 8, by default, the following values are
set:
TIFMM TVEPMTMIUSWBSMJCNZTRM]HSFQQFN
SXNZTRMNZTRM,/PWDBLFZQFN
-rw-r--r--. 1 mysql mysql 1.1K Nov 19 13:53 ca.pem
-rw-------. 1 mysql mysql 1.7K Nov 19 13:53 server-key.pem
-rw-r--r--. 1 mysql mysql 1.1K Nov 19 13:53 server-cert.pem
-rw-------. 1 mysql mysql 1.7K Nov 19 13:53 client-key.pem
-rw-r--r--. 1 mysql mysql 1.1K Nov 19 13:53 client-cert.pem
SXNZTRMNZTRM,/PWQSJWBUF@LFZQFN
SXSSNZTRMNZTRM/PWQVCMJD@LFZQFN
<NZTRME>
TTMDBWBSMJCNZTRMDBQFN
TTMDFSUWBSMJCNZTRMTFSWFSDFSUQFN
TTMLFZWBSMJCNZTRMTFSWFSLFZQFN
NZTRM 4)087"3*"#-&4-*,& TTM
[ 407 ]

Security Chapter 14


]IBWF@PQFOTTM]:&4]
]IBWF@TTM]:&4]
]TTM@DB]WBSMJCNZTRMDBQFN]
]TTM@DBQBUI]]
]TTM@DFSU]WBSMJCNZTRMTFSWFSDFSUQFN]
]TTM@DJQIFS]]
]TTM@DSM]]
]TTM@DSMQBUI]]
]TTM@LFZ]WBSMJCNZTRMTFSWFSLFZQFN]

SPXTJOTFU
TFD
2. Copy the DMJFOUDFSUQFN and DMJFOULFZQFN files from the server's EBUB
EJSFDUPSZ to the client location:
TIFMM TVEPTDQJ)0.&TTIJE@STBWBSMJCNZTRMDMJFOULFZQFN
WBSMJCNZTRMDMJFOUDFSUQFNVTFS !DMJFOU@JQ
DIBOHFUIFTTIQSJWBUFLFZQBUIBTOFFEFE
3. Connect to the server by passing the TTMDFSU and TTMLFZ options:
TIFMM NZTRM--ssl-cert=client-cert.pem --ssl-key=client-key.pemI

(1-
_
NZTRM
4. Mandate the user to connect only by X509:
NZTRM "-5&364&3AECBENJOA!AA3&26*3&9
2VFSZ0,SPXTBGGFDUFE
TFD
5. Test the connection:
TIFMM NZTRMMPHJOQBUIECBENJO@SFNPUFI--ssl-
cert=client-cert.pem --ssl-key=client-key.pem
(1-
_
NZTRM ?%#ZF
[ 408 ]

Security Chapter 14
6. If you do not specify TTMDFSU or TTMLFZ, you will not be able to log in:
TIFMM NZTRMMPHJOQBUIECBENJO@SFNPUFI
&3303
"DDFTTEFOJFEGPSVTFS
ECBENJO !
VTJOHQBTTXPSE:&4
TIFMM NZTRMMPHJOQBUIECBENJO@SFNPUFITTM
DFSUDMJFOUDFSUQFN
NZTRM<&3303>44-FSSPS6OBCMFUPHFUQSJWBUFLFZGSPN DMJFOU
DFSUQFN
&3303
):44-DPOOFDUJPOFSSPS6OBCMFUPHFUQSJWBUFLFZ
TIFMM NZTRMMPHJOQBUIECBENJO@SFNPUFITTM
LFZDMJFOULFZQFN
NZTRM<&3303>44-FSSPS6OBCMFUPHFUDFSUJGJDBUFGSPN DMJFOU
LFZQFN
&3303
):44-DPOOFDUJPOFSSPS6OBCMFUPHFUDFSUJGJDBUF
By default, all SSL-related files are kept in the60;EBUBEJSFDUPSZ. If you

want to keep them elsewhere, you can set TTM@DB, TTM@DFSU, and
TTM@LFZ in the NZDOG file and restart the server. You can generate a new
set of SSL files through either MySQL or OpenSSL. To know the more
detailed steps, refer to IUUQTEFWNZTRMDPNEPDSFGNBOFO
DSFBUJOHTTMSTBGJMFTIUNM. There are many other authentication
plugins available. You can refer to IUUQTEFWNZTRMDPNEPDSFGNBO
FOBVUIFOUJDBUJPOQMVHJOTIUNM to know more details.
Setting up SSL replication

If you enable SSL replication, the binary log transfer between master and slave will be sent
through an encrypted connection. This is similar to the server/client connection explained in
the preceding section.
How to do it...
1. On the master, as explained in the preceding section, you need to enable SSL.
2. On the master, copy the DMJFOU certificates to the slave:
NZTRM TVEPTDQJ)0.&TTIJE@STBWBSMJCNZTRMDMJFOULFZQFN
WBSMJCNZTRMDMJFOUDFSUQFNVTFS !DMJFOU@JQ
[ 409 ]

Security Chapter 14
3. On the slave, create the NZTRMTTM directory to hold the SSL-related files and set
the permissions correctly:
TIFMM TVEPNLEJSFUDNZTRMTTM
TIFMM TVEPDQDMJFOULFZQFNDMJFOUDFSUQFNFUDNZTRMTTM
TIFMM TVEPDIPXO3NZTRMNZTRMFUDNZTRMTTM
TIFMM TVEPDINPEFUDNZTRMTTMDMJFOULFZQFN
TIFMM TVEPDINPEFUDNZTRMTTMDMJFOUDFSUQFN
4. On the slave, execute the $)"/(&@."45&3 command with the SSL-related

changes on the slave:
NZTRM 45014-"7&
NZTRM $)"/(&."45&350."45&3@44-."45&3@44-@$&35 FUDNZTRM

TTMDMJFOUDFSUQFN ."45&3@44-@,&: FUDNZTRMTTMDMJFOU
LFZQFN
NZTRM 45"354-"7&
5. Verify the slave's status:
NZTRM 4)084-"7&45"564=(
SPX
.BTUFS@)PTU
_
4MBWF@*0@3VOOJOH:FT
_
4LJQ@$PVOUFS
&YFD@.BTUFS@-PH@1PT
3FMBZ@-PH@4QBDF
6OUJM@$POEJUJPO/POF
6OUJM@-PH@'JMF
6OUJM@-PH@1PT
Master_SSL_Allowed: Yes
Master_SSL_CA_File: /etc/mysql-ssl/ca.pem
.BTUFS@44-@$"@1BUI
Master_SSL_Cert: /etc/mysql-ssl/client-cert.pem
.BTUFS@44-@$JQIFS
Master_SSL_Key: /etc/mysql-ssl/client-key.pem
_
.BTUFS@66*%GFCCDEFCDCB
[ 410 ]

Security Chapter 14
42-@%FMBZ
_
SPXJOTFU
TFD
6. Once you have made the SSL-related changes on all the slaves, on the master,
enforce the replication user to use X509:
NZTRM "-5&364&3ASFQMA!AA3&26*3&9
2VFSZ0,SPXTBGGFDUFE
TFD
Note that, this can affect other replication users. As an alternative, you can create
one replication user with SSL and one normal replication user.
7. Verify the slave status on all slaves.
[ 411 ]

Index
obtaining, mysqldump used

A obtaining, mysqlpump used
aggregate functions obtaining, XtraBackup used ,
reference link of big table, mydumper used
alter operations of databases , ,
copy of databases, regex used
in-place of rows
APT bundle of tables
using , parallel processing
APT repositories point-in-time recovery
used, for installing MySQL schema
using server, rebuilding with different schema
Atomicity setting up
Atomicity, Consistency, Isolation, and Durability single table
(ACID) properties tables, ignoring
users, backup
B basic parameters
backups configuring
compressed backups , , data directory
consistent backup innodb_buffer_pool_instances
data innodb_buffer_pool_size
data, backing up innodb_log_file_size
data, merging with server binary logging
database objects, excluding regex used , about
point in time recovery
database objects, including regex used , replication
faster reload using
from remote server binary logs
full backup , backup ,
installation , based on database, extracting
instances, locking based on time, extracting
master binary coordinates, dumping , database name, rewriting
no data databases, selecting
no schema disabling, for recovery
non-blocking backup disabling, for session
obtaining, flat files used , enabling
obtaining, mydumper used , events, displaying

example databases
expiring creating , , ,
flush log information, obtaining
format , datatypes
managing , optimizing , ,
observations DEB files
position, extracting used, for installing MySQL 8.0
relocating delayed replication
row event display, extracting setting up ,
statements, extracting , delete statement
descending indexes ,
C dirty read
chain replication duplicate index
slave, switching from master-slave , about
common table expression (CTE) mysqlindexcheck , ,
about , pt-duplicate-key-checker
non-recursive CTE removing , ,
recursive CTE Durability
composite partitioning dynamic table metadata
config file
using E
Consistency encrypted connections
consistent backup setting up, X509 used , ,
error log
D configuring ,
Data Definition Language (DDL) in JSON format ,
data directory managing ,
changing , reference link
Data Manipulation Language (DML) rotating
data system log, used for logging , , ,
columns, selecting events
count about ,
filter based, on condition access control
ignoring EXPLAIN JSON
loading, into table using
operators explain plan
regular expressions about
replacing reference link
results, limiting EXPLAIN
selecting used, for connection
selecting, into file using
selecting, into table external locking
simple pattern matching
table alias, used
[ 413 ]

GTID replication
F setting up , ,
file-per-table tablespaces
copying, to another instance , I
creating, outside data directory ,
in-place downgrades, MySQL 8.0
partition of table, copying ,
APT bundle, used
table, copying , ,
APT repositories, used
file
generic binaries, used
data, selecting
RPM bundle, used
saving
YUM repositories, used
flat file backup
in-place upgrades, MySQL 8.0
recovering
APT bundle, used
flat files
APT-based systems
used, for obtaining backups ,
generic binaries, used
function and operator reference
RPM bundle, used
reference link
YUM-based systems
function syntax
index hints
reference link
using
functions
index usage
about
checking
inbuilt functions
index-organized table
indexes
G adding ,
general availability (GA) Drop index
general query log on generated columns ,
about , Prefix index
managing primary key ,
query log output destinations, selecting , secondary index ,
general tablespace UNIQUE index
creating init-file
dropping using ,
managing InnoDB locks
non-partitioned tables, moving between reference link
tablespaces InnoDB redo log files
partitioned tables, managing number, changing ,
tables, adding size, changing ,
generated columns InnoDB system tablespace
about increasing ,
stored resizing ,
virtual shrinking ,
generic binaries InnoDB tables
used, for installing MySQL on Linux compressing ,
global transaction identifier (GTID) compression, disabling for file_per_table Tables
global variables
using compression, enabling for file_per_table Tables
[ 414 ]

queue ,
compression, enabling for general tablespace log sequence number (LSN)
, logical backup
insert statement Logical backup utilities
insert logical downgrades, MySQL 8.0
on duplicate key update APT bundle, used
internal locking APT repositories, used
about generic binaries, used
reference link RPM bundle, used
row-level locks YUM repositories, used
table-level locks
invisible index M
IO thread master-master replication
Isolation setting up ,
isolation levels multi-master replication
about setting up , ,
read committed Multi-Version Concurrency Control (MVCC)
read uncommitted mydumper
repeatable read full database, recovering
serializable recovering, using myloader
single database, recovering
J single table, recovering ,
JavaScript Object Notation (JSON) used, for obtaining backups ,
about MySQL 8.0
functions downgrading ,
indexing, generated columns used , , in-place downgrades
in-place upgrades
inserting installing
modifying installing, DEB files used
other functions installing, RPM files used
pretty view logical downgrades
removing logical upgrades
retrieving server status, checking
searching server, starting
using server, stopping
JSON format starting
reference link upgrading , ,
JSON function MySQL 8
reference link stopping
uninstalling
L uninstalling, on APT-based systems
locking uninstalling, on YUM-based systems
about MySQL APT repository
external locking reference link
internal locking MySQL server
[ 415 ]

managing, with systemd INFORMATION_SCHEMA.PARTITIONS table,
MySQL utilities querying
installing SHOW CREATE TABLE, used
installing, on APT-based systems SHOW TABLE STATUS, used
installing, on YUM-based systems partition management
MySQL YUM Repository about
reference link ADD partitions
MySQL DROP partitions
about HASH partitions, managing
connecting, command-line client used , , KEY partitions
dynamic other operations
installing, APT repositories reorganizing partitions
installing, on Linux using generic binaries , TRUNCATE partitions
installing, YUM repositories partition pruning ,
static partition selection ,
uninstalling, on binaries password-less authentication
mysql_config_editor mysql_config_editor, used ,
used, for password-less authentication , Percona Toolkit
mysql_secure_installation installing
FILE privilege installing, on CentOS/Red Hat/Fedora ,
securing , installing, on Debian/Ubuntu ,
mysqldump performance_schema
recovering , consumers, disabling ,
used, for obtaining backups consumers, enabling ,
mysqlpump disabling
recovering , enabling
used, for backups obtaining instruments, disabling ,
instruments, enabling ,
N tables ,
networks using
restricting , Physical backup utilities
non-recursive CTE point-in-time recovery
non-repeatable read mydumper
mysqldump
O mysqlpump
performing
operators
procedures
reference link
reference link
P pt-archiver
reference link
parameters pt-query-digest
used, with startup script binary log
partition information general query log
about process list
EXPLAIN, used slow query log ,
[ 416 ]

TCP dump resource group
used, for analyzing slow queries alter resource group
drop resource group
Q using ,
queries results
benchmarking , average
time, spending in evaluating plans count
query optimizer distinct
controlling , filtering, having clause used
optimizer cost model, adjusting grouping
optimizer hints , , sorting
optimizer_search_depth , sum
optimizer_switch , root password
init-file
R init-file, used
resetting
RANGE partitioning
skip-grant-tables, used ,
reference link
row-based replication (RBR)
redundant indexes
rows
removing , ,
deleting
regular expressions
inserting
reference link
updating
relational database management system (RDBMS)
RPM bundle
release candidate (RC) using
replace RPM files
on duplicate key update used, for installing MySQL 8.0
replication filters
database, ignoring S
database, replicating sample data
setting up loading
tables, ignoring savepoints
tables, replicating using
replication secondary indexes
about reference link
analytics Secure Sockets Layer (SSL)
chain replication SELECT syntax
data security reference link
long-distance data distribution semi-synchronous replication
master-master replication setting up , ,
multi-master replication server
reference link benchmarking ,
scale-out solutions session variables
setting up , , , , using
traditional master-slave replication shared lock
skip-grant-tables
[ 417 ]

using , archiving ,
slave cloning
switching, from chain replication to master-slave COLUMNS
replication , creating , ,
switching, from master-slave replication to chain data, archiving
replication , data, copying
slaves data, loading
slow queries data, purging
analyzing, pt-query-digest used data, selecting
slow query log duplicates, identifying SELF JOIN
about , FILES
managing HASH partitioning
query log output destinations, selecting , information, obtaining
soft delete rows INNODB_SYS_TABLESPACES
source code INNODB_SYS_TABLESTATS
URL, for installing joining ,
SQL thread KEY partitioning
SSL replication LINEAR HASH partitioning
setting up , LINEAR KEY partitioning
statement-based replication (SBR) LIST COLUMNS partitioning
statements LIST partitioning
extracting, from binary logs , mismatched rows, finding
static table metadata moving, across databases
stored procedures , , , partitioning ,
subpartitioning PROCESSLIST
sys schema RANGE COLUMNS partitioning ,
redundant indexes RANGE partitioning ,
statement analysis , REMOVE partitioning
statement by type, from host saving
statement by type, from user SUB queries, used
statements, executed from host subpartitioning
table statistics TABLES
table statistics, with buffer truncating
unused indexes time to live
using , managing
systemd transactions
MySQL server, managing autocommit
performing
T Transport Layer Security (TLS)
table structure trigger syntax
cloning reference link
table triggers ,
altering , , ,
altering, online schema change tool used ,

[ 418 ]

U W
UNDO tablespace window functions
managing , , about
update statement first values
user syntax, creating last values
reference link named windows
users nth values
access, granting partition results
access, revoking row number
creating ,
grants, checking X
grants, revoking X509
locking used, for setting up encrypted connections ,
mysql.user table, modifying ,
password expiry, setting XtraBackup
privileges, granting installing
restricting , installing, on CentOS/Red Hat/Fedora
roles, creating installing, on Debian/Ubuntu
used, for obtaining backups ,
V
variables Y
config file YUM repositories
SET command, using used, for installing MySQL
startup script using
views ,

KarthikAppigatla 2018 MySQL8CookbookOver150RecipesforHigh-Perf

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

KarthikAppigatla 2018 MySQL8CookbookOver150RecipesforHigh-Perf

Uploaded by

Copyright:

Available Formats

2018. Packt Publishing. All rights reserved.

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Commissioning Editor: Amey Varangaokar

First published: January 2018

Published by Packt Publishing Ltd.

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

About the author

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Kedar Mohaniraj Vaijanapurkar is a MySQL database consultant with over a decade of

Packt is searching for authors like you

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Using Generic Binaries 34

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Simple pattern matching 63

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Partition results 127

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Disabling binary logs for a session 157

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Backup users 183

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Archiving data 252

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Shrinking the InnoDB system tablespace 290

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Using EXPLAIN 329

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Indexing for JSON using generated columns 376

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Who this book is for

What this book covers

$IBQUFS, MySQL 8 - Installing and Upgrading, describes how to install MySQL 8 on

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

$IBQUFS, Security, focuses on the aspects of security. Recipes on securing installation,

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

To get the most out of this book

Any command-line input or output is written as follows:

Warnings or important notes appear like this.

Tips and tricks appear like this.

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

For more information about Packt, please visit QBDLUQVCDPN.

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Installing MySQL using YUM/APT

Software repositories (YUM or APT)

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Docker (not covered)

MySQL Linux RPM package distribution identifiers

Distribution value Intended use

CPU value Intended processor type or family

CPU value Intended processor type or family

CPU value Intended processor type or family

EBSCOhost - printed on 11/9/2022 10:11 AM via . All use subject to https://www.ebsco.com/terms-of-use

Installing MySQL using YUM/APT

There are mainly two types of repository software:

YUM (Centos, Red Hat, Fedora and Oracle Linux)

$IBQUFS, MySQL 8 - Installing and Upgrading, describes how to install MySQL 8 on

$IBQUFS, Security, focuses on the aspects of security. Recipes on securing installation,

For more information about Packt, please visit QBDLUQVCDPN.

TIFMM ZVNSFQPMJTUFOBCMFE]HSFQ NZTRMDPNNVOJUZ

6. Disable NZTRMDPNNVOJUZ and enable NZTRMDPNNVOJUZ:

7. Verify that NZTRMDPNNVOJUZ is enabled:

TIFMM SQNRB]HSFQJ NZTRM

NZTRMDPNNVOJUZTFSWFS: Database server and related tools.

4. Upgrade MJCTUED to the latest:

TIFMM TVEPEQLHQSFDPOGJHVSFNZTRMDPNNVOJUZTFSWFS@EFC

9. Copy the startup script to JOJUE:

5ZQF IFMQ PS =I GPSIFMQ5ZQF =D UPDMFBSUIFDVSSFOUJOQVU

NZTRM "-5&364&3 SPPU ! MPDBMIPTU %&/5'*&%#: /FX1BTT

RPM-based systems use the NZTRMETFSWJDF files, and APT-based