Professional Documents
Culture Documents
CJ
Introduction MySQL
• MySQL server is one of the most widely used relational database management systems.
( Structured collection of Data as its stored in separate tables )
• Supports all major OS – Linux flavors (Mac), Windows ( from 1998 ) etc
My Oracle Support (MOS): This is the preferred location to download commercial builds as it includes all releases
since 2011 and some earlier releases.
Oracle Software Delivery Cloud (eDelivery): This only includes the latest patch release of the latest GA version
(currently the 8.0 series). As the repository is only updated once a month, it can lag compared to the release available
in My Oracle Support.
MySQL Downloads: This is for the Community releases only. (https://dev.mysql.com/downloads/)
Compatibility Between MySQL Servers and MySQL Clients/Tools
Client/Tool Client/Tool Version MySQL 5.6 MySQL 5.7 MySQL 8.0
8.0 ✗ ✓ ✓
MySQL Shell
8.0 ✗ ✓ ✓
MySQL Router
1.2 ✓ ✓ ✓
2.0 ✓ ✓ ✗
MySQL for Visual Studio
1.3 ✓ ✓ ✓
MySQL for Excel
13.2.4.0.0 ✓ ✓ ✓
3.12 ✓ ✗ ✗
MySQL Enterprise Backup 4.1 ✗ ✓ ✗
8.0 ✗ ✗ ✓
4.0 ✓ ✓ ✗
8.0 ✓ ✓ ✓
MySQL Enterprise Monitor
1.1 ✓ ✓ ✓
MySQL Notifier
8.0 ✓ ✓ ✓
MySQL Workbench
MySQL Server Versions
Change History :
InnoDB
MyISAM
Memory
CSV
Merge
Archive
Federated
Blackhole
Example
Change History :
•MySQL Server 5.5
•MySQL Server 5.6 (MySQL 5.6.10 (2013-02-05))
•MySQL Server 5.7
Guide:
How to set SELinux contexts for MySQL Server (Doc ID 2296123.1)
MySQL Physical Architecture
MySQL Logical Architecture
MySQL Logical Architecture
My.cnf
You can find the my.cnf file in one of the following locations:
/etc/my.cnf
/etc/mysql/my.cnf
$MYSQL_HOME/my.cnf
[datadir]/my.cnf
datadir = dir_name ( Directory location to the MySQL database data, status and log files. )
pid-file = file_name ( Filename in which the mysqld server should write its process ID information. )
socket = file_name, -S file_name ( On Unix, the name of the Unix socket file to use, for connections made using a named pipe to a local server. )
mysql --version
mysql -V
mysql
In MySQL, physically, a schema is synonymous with a database. You can substitute the keyword SCHEMA instead
of DATABASE in MySQL SQL syntax, for example using CREATE SCHEMA instead of CREATE DATABASE.
Both MySQL DATABASE and SCHEMA are a collection of tables.
MySQL objects such as tables, table columns, views, users, indexes, functions, procedures, etc.
MySQL Objects Database Objects
CREATE DATABASE STORED PROCEDURE
CREATE EVENT STORED FUNCTION
CREATE FUNCTION TRIGGER
CREATE FUNCTION UDF EVENT
CREATE INDEX VIEW
CREATE LOGFILE GROUP TABLE
CREATE PROCEDURE INDEX
CREATE SCHEMA STORED ROUTINES
CREATE SERVER ..
CREATE TABLE etc
CREATE TABLESPACE
CREATE TRIGGER
CREATE USER
CREATE VIEW
Simple User related
Create/Connect /Drop Database :
CREATE DATABASE <db_name>;
CREATE Schema <db_name>;
Use <db_name>;
SELECT USER(),CURRENT_USER(),session_user(),system_user();
SELECT db, host, user FROM mysql.db ;
select DISTINCT User,host,account_locked,password_expired FROM mysql.user;
User Management
CREATE USER:
CREATEUSER'hostuser'@'localhost'IDENTIFIEDBY'user_password’;
CREATEUSER'hostuser'@’%'IDENTIFIEDBY'user_password’;
DROP USER:
DROP USER 'hostuser'@'localhost’;
RESET PASSWORD :
SET PASSWORD FOR 'hostuser'@'localhost’= 'auth_string’;
or
ALTER USER 'hostuser'@'localhost’ IDENTIFIED BY 'auth_string';
REVOKE PRIV :
REVOKE type_of_permission ON database_name.table_name TO'hostuser'@'localhost’;
REVOKE permission1, permission2 ON database_name.table_name TO'hostuser'@'localhost’;
CHECK :
SHOW GRANTS FOR 'hostuser'@'localhost’;
SELECT CONCAT('SHOW GRANTS FOR ''',user,'''@''',host,''';') FROM mysql.user;
SELECT * FROM information_schema.user_privileges;
SELECT * FROM information_schema.user_privileges WHERE GRANTEE="'hostuser'@'localhost’";
Security
SSL AUDITING
Enable SSL: Enable Auditing:
my.cnf variables : my.cnf variables :
[mysqld] / [mysqlclient] plugin-load=audit_log.so
ssl-cert=/apps/mysql/ssl/scpl-dbmysq-001.fhlbsf-i.com.cer audit-log-file=/apps/mysql/mysql_5.7.29/audit/audit.log
ssl-key=/apps/mysql/ssl/scpl-dbmysq-001.fhlbsf-i.com.key audit-log=FORCE_PLUS_PERMANENT
ssl-ca=/apps/mysql/ssl/scpl-dbmysq-001_2020-interim-ca.pem audit_log_format=JSON
Force SSL : audit_log_policy=LOGINS
require_secure_transport=ON audit_log_rotate_on_size=8M
audit_log_connection_policy=ALL
Enable SSL to user: audit_log_statement_policy=ALL
GRANT ALL PRIVILEGES ON database.* to 'hostuser'@'localhost’; REQUIRE SSL; audit_log_exclude_accounts=root
or
ALTER USER 'hostuser'@'localhost’ REQUIRE SSL; Extra Steps :
Disable or Remove SSL Type for user: By default, rule-based audit log filtering logs no auditable events for any
ALTER USER 'hostuser'@'localhost’ REQUIRE NONE; users.
User connect:
mysql –u<username> -h ‘hostname’ -P –p 1216 –sql-mode=REQUIRED SELECT audit_log_filter_set_filter('log_all', '{ "filter": { "log": true } }');
SELECT audit_log_filter_set_user('%', 'log_all');
Tool continuously monitor your databases and alerts you to Graphical user interface tool used for the MySQL database
potential problems before they impact your system management.
Enables you to monitor MySQL instances on your network It facilitates creation of new physical data models and
or on a cloud service. modification of existing MySQL databases with
reverse/forward engineering and change management
functions.
hot backup with MySQL Enterprise Backup Logical backups are performed with the MySQL server
running. The server is not taken offline.
Physical backups is suitable for large, important databases logical backup with mysqldump for smaller data volumes or
that need to be recovered quickly when problems occur. to record the structure of schema objects.
cold backup by copying files while the MySQL server is shut down;
Hot and cold backups are physical backups that copy actual data files, which can be used directly by the mysqld server for
faster restore.
MySQL DB Refresh Steps
1) Export from DB from PROD using mysqldump ( Verify DB Size and Table count ) :
mysqldump --port 1216 --add-drop-database --flush-privileges --flush-logs --single-transaction --set-gtid-purged=OFF --triggers --routines --events
--databases db_name > db_name_"$(date +%d-%m-%Y-%H_%M)".sql
2) scp the file to target server
3) Import dump into the server ( Verify DB Size and Table count ) ;
mysql>
source /apps/mysql/mysqldump/db_name_30-09-2020-18_59_20.sql;
Or
mysql < /apps/mysql/mysqldump/db_name_30-09-2020-18_59_20.sql;
4) Verify Query :
SELECT table_schema AS "Database", ROUND(SUM(data_length + index_length) / 1024 / 1024, 2) AS "Size (MB)",
count(*) as "Total Table Counts",MAX(create_time) create_time,MAX(update_time) update_time
FROM information_schema.TABLES WHERE table_schema = "db_name" GROUP BY table_schema Order by 2 desc;
• The newer method based on global transaction identifiers (GTIDs) is transactional and therefore does not require working with log files
or positions within these files.
• Master-Slave
• Master_Master (Active-Active)
• Master_Master (Active-Passive)
• Group Replication ( Clustered - InnoDB Cluster )
Replication in-Short
1) All database operations are copied to the master’s binary log.
5) The relay log is read by the SQL thread process and it applies all the operations/data to
the slave’s database and its binary log.
Asynchronous replication
• The primary executes transactions, commits them and then
they are later (thus asynchronously) sent to the secondaries to
be either re-executed (in statement-based replication) or
applied (in row-based replication).
• Master keeps writing data
Semi-synchronous replication
• The master does not confirm/commit transactions to the client
until at least one slave has copied the change to its relay log,
and flushed it to disk.
• Master waits for the data – received in any one slave but does
care/bother/wait for commit to happen on Slave.
Replication basics…
Client Master Slave
DATA DATA
INSERT
INTO …
DATA
DATA
Binary INSERT
Log INTO …
DATA
DATA
• If the slave is online, the command is transmitted to the slave in parallel (well, immediately following)
to being written in the local binary log
• Otherwise, when the slave next connects it will receive a list of all pending statements from the
master server’s binary log
• The slave’s replication IO thread stores the command in the local relay log
Replication basics… Slave
DATA
Master
Client
DATA
Thread
SQL
DATA INSERT
INTO …
Replication INSERT I/O
Binary Logs Relay Log
Thread INTO … Thread
• Once the data is received in the slave’s relay log, the slave SQL thread executes the
command locally, bringing the slave up-to-date with the master
• In MySQL 3.23, the IO and SQL threads were just one thread. In later versions this
was changed to 2 threads.
Replication Setup Steps
Common Steps for Master and Slave :
• Add replication variables to my.cnf in both the Master and Slave/Master nodes
• Restart mysql server/service
On Primary / Master:
• Create a replication user and grant REPLICATION SLAVE privilege
• Create a snap-shot / dump of Master Server
• Copy dump file or snap-shop to Slave Server
On Standby / Slave :
• Start P2 mysql_server in read-only
• Update Slave : CHANGE MASTER ( inserting the Master’s replication user credentials ) – channel is created for replication
• On Standby / Slave :
• Start slave mysql in read-only
• Update Slave : CHANGE MASTER ( inserting the Master’s replication user credentials ) – channel is created for replication
• CHANGE MASTER TO MASTER_HOST='<NODE_1>', MASTER_PORT=<port>, MASTER_USER='<user>', MASTER_PASSWORD='<password>',
MASTER_AUTO_POSITION=1 FOR CHANNEL 'Node1';
• Restore dump file or snap-shop into Slave Server ( now both Master and Slave in same sync )
• Start Slave process.
• Restore dump file or snap-shop into Slave Server ( now both Master and Slave in same sync )
• Start Slave process.
Replication Threads
Slave Threads
IO thread continuously receives updates from master and writes to local relay log
When a START SLAVE statement is issued on a replica server, the replica creates an I/O thread, which connects to the source
and asks it to send the updates recorded in its binary logs.
The replication I/O thread reads the updates that the source's Binlog Dump thread sends (see previous item) and copies
them to local files that comprise the replica's relay log.
The state of this thread is shown as Slave_IO_running in the output of SHOW SLAVE STATUS.
SQL thread continuously executes statements in relay log.
The replica creates an SQL thread to read the relay log that is written in relay log by the replication I/O thread and execute
the transactions contained in it.
Master Threads / Binary log dump thread.:
• Dedicated to scanning the binary logs on the master and sending updates to the connected slave – whenever binary
requests for it.
• The source creates a thread to send the binary log contents to a replica when the replica connects. This thread can be
identified in the output of SHOW PROCESSLIST on the source as the Binlog Dump thread.
• The binary log dump thread acquires a lock on the source's binary log for reading each event that is to be sent to the
replica. As soon as the event has been read, the lock is released, even before the event is sent to the replica.
Replicaiton my.cnf variables Master-Slave
Master Slave
##replication ##replication
server-id=1 server-id=2
gtid_mode=ON gtid_mode=ON
enforce-gtid-consistency=ON enforce-gtid-consistency=ON
binlog_format = ROW binlog_format = ROW
master_info_repository='TABLE' master_info_repository='TABLE'
relay_log_info_repository='TABLE’ relay_log_info_repository='TABLE’
read_only=ON
super_read_only=ON
log-bin=/backup/mysql-binlogs/mysql-bin log-bin=/backup/mysql-binlogs/mysql-bin
log-bin-index=/backup/mysql-binlogs/mysql-bin.index log-bin-index=/backup/mysql-binlogs/mysql-bin.index
master-info-file=/backup/mysql-binlogs/master.info master-info-file=/backup/mysql-binlogs/master.info
relay-log=/backup/mysql-relaylogs/relay-bin
relay_log_index=/backup/mysql-relaylogs/lvpl-dbmysq-001-relay-bin.index
max_binlog_size=1G max_binlog_size=1G
expire_logs_days=10 expire_logs_days=10
relay_log_space_limit=2G relay_log_space_limit=2G
Replicaiton my.cnf variables - (Master(active)-Master(passive))
Master-Active Master - Passive
##replication ##replication
server-id=1 server-id=2
gtid_mode=ON gtid_mode=ON
enforce-gtid-consistency=ON enforce-gtid-consistency=ON
binlog_format = ROW binlog_format = ROW
master_info_repository='TABLE' master_info_repository='TABLE'
relay_log_info_repository='TABLE' relay_log_info_repository='TABLE’
auto_increment_offset = 1 auto_increment_offset = 2
auto_increment_increment = 2 auto_increment_increment = 2
read_only=ON
super_read_only=ON
log-bin=/backup/mysql-binlogs/mysql-bin log-bin=/backup/mysql-binlogs/mysql-bin
log-bin-index=/backup/mysql-binlogs/mysql-bin.index log-bin-index=/backup/mysql-binlogs/mysql-bin.index
master-info-file=/backup/mysql-binlogs/master.info master-info-file=/backup/mysql-binlogs/master.info
relay-log=/backup/mysql-relaylogs/relay-bin relay-log=/backup/mysql-relaylogs/relay-bin
relay_log_index=/backup/mysql-relaylogs/scpl-dbmysq-001-relay-bin.index relay_log_index=/backup/mysql-relaylogs/lvpl-dbmysq-001-relay-bin.index
max_binlog_size=1G max_binlog_size=1G
expire_logs_days=10 expire_logs_days=10
relay_log_space_limit=2G relay_log_space_limit=2G
Replication Types : MySQL Replication Topology (Doc ID 1489113.1)