关于mysqldump备份非事务表的注意事项
Preface
We're used to get a logical backup set(whole instance) by simply specifying "--single-transaction" and "--master-data" options.For innodb engine tables,we can get a consistent backup.But the non-innodb(not supporting transaction) tables may be not backed up consistently in MySQL 5.7.
Procedure
Check and open general log(if have not enabled it yet).
(zlm@192.168.1.101 )[(none)]>show variables like 'general_log';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| general_log | OFF |
+---------------+-------+
row in set (0.00 sec) (zlm@192.168.1.101 )[(none)]>set @@global.general_log=on;
Query OK, rows affected (0.01 sec) (zlm@192.168.1.101 )[(none)]>show variables like 'general_log';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| general_log | ON |
+---------------+-------+
row in set (0.00 sec)
Clear the content of general log and tail it.
[root@zlm2 :: /data/mysql/mysql3306/data]
#echo ''> zlm2.log [root@zlm2 :: /data/mysql/mysql3306/data]
#tail -f zlm2.log
Create test tables.
(zlm@192.168.1.101 )[(none)]>create database zlm;
Query OK, row affected (0.00 sec) (zlm@192.168.1.101 )[(none)]>use zlm;
Database changed (zlm@192.168.1.101 )[(none)]>create table test_trx(
-> id int,
-> name varchar()
-> ) engine=innodb;
Query OK, rows affected (0.01 sec) (zlm@192.168.1.101 )[zlm]>insert into test_trx values(,'zlm');
Query OK, row affected (0.01 sec) (zlm@192.168.1.101 )[zlm]>create table test_notrx like test_trx;
Query OK, rows affected (0.01 sec) (zlm@192.168.1.101 )[zlm]>insert into test_notrx values(,'aaron8219');
Query OK, row affected (0.00 sec) (zlm@192.168.1.101 )[zlm]>alter table test_notrx engine=MyISAM;
Query OK, row affected (0.01 sec)
Records: Duplicates: Warnings: (zlm@192.168.1.101 )[zlm]>show create table test_trx\G
*************************** . row ***************************
Table: test_trx
Create Table: CREATE TABLE `test_trx` (
`id` int() DEFAULT NULL,
`name` varchar() DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8
row in set (0.00 sec) (zlm@192.168.1.101 )[zlm]>show create table test_notrx\G
*************************** . row ***************************
Table: test_notrx
Create Table: CREATE TABLE `test_notrx` (
`id` int() DEFAULT NULL,
`name` varchar() DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8
row in set (0.00 sec) (zlm@192.168.1.101 )[zlm]>select * from test_trx;
+------+------+
| id | name |
+------+------+
| | zlm |
+------+------+
row in set (0.00 sec) (zlm@192.168.1.101 )[zlm]>select * from test_notrx;
+------+-----------+
| id | name |
+------+-----------+
| | aaron8219 |
+------+-----------+
row in set (0.00 sec)
Generate a backup with mysqldump specifying common option of "--single-transaction" together with "--master-data=2".
[root@zlm2 :: ~]
#mysqldump --single-transaction --master-data= zlm > test1.sql //General log output below.
--30T09::17.815021+: Connect zlm@zlm2 on using TCP/IP
--30T09::17.815290+: Query /*!40100 SET @@SQL_MODE='' */
--30T09::17.815445+: Query /*!40103 SET TIME_ZONE='+00:00' */
--30T09::17.815862+: Query FLUSH /*!40101 LOCAL */ TABLES
--30T09::17.823118+: Query FLUSH TABLES WITH READ LOCK //Generate a FTWRL.
--30T09::17.823435+: Query SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ
--30T09::17.823684+: Query START TRANSACTION /*!40100 WITH CONSISTENT SNAPSHOT */
--30T09::17.823904+: Query SHOW VARIABLES LIKE 'gtid\_mode'
--30T09::17.826626+: Query SELECT @@GLOBAL.GTID_EXECUTED
--30T09::17.827032+: Query SHOW MASTER STATUS
--30T09::17.827278+: Query UNLOCK TABLES //Release the locks on all tables in whole instance.
--30T09::17.827480+: Query SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
--30T09::17.828912+: Query SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
--30T09::17.830942+: Query SHOW VARIABLES LIKE 'ndbinfo\_version'
--30T09::17.833548+: Init DB zlm
--30T09::17.833785+: Query SAVEPOINT sp
--30T09::17.833963+: Query show tables
--30T09::17.834252+: Query show table status like 'test\_notrx'
--30T09::17.834449+: Query SET SQL_QUOTE_SHOW_CREATE=
--30T09::17.834582+: Query SET SESSION character_set_results = 'binary'
--30T09::17.834781+: Query show create table `test_notrx`
--30T09::17.835621+: Query SET SESSION character_set_results = 'utf8'
--30T09::17.835851+: Query show fields from `test_notrx`
--30T09::17.836362+: Query show fields from `test_notrx`
--30T09::17.836843+: Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_notrx`
--30T09::17.837327+: Query SET SESSION character_set_results = 'binary'
--30T09::17.838289+: Query use `zlm`
--30T09::17.838723+: Query select @@collation_database
--30T09::17.838908+: Query SHOW TRIGGERS LIKE 'test\_notrx'
--30T09::17.839457+: Query SET SESSION character_set_results = 'utf8'
--30T09::17.839678+: Query ROLLBACK TO SAVEPOINT sp
--30T09::17.839821+: Query show table status like 'test\_trx'
--30T09::17.840112+: Query SET SQL_QUOTE_SHOW_CREATE=
--30T09::17.840248+: Query SET SESSION character_set_results = 'binary'
--30T09::17.840364+: Query show create table `test_trx`
--30T09::17.840675+: Query SET SESSION character_set_results = 'utf8'
--30T09::17.840828+: Query show fields from `test_trx`
--30T09::17.841258+: Query show fields from `test_trx`
--30T09::17.841774+: Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_trx`
--30T09::17.842205+: Query SET SESSION character_set_results = 'binary'
--30T09::17.842370+: Query use `zlm`
--30T09::17.842646+: Query select @@collation_database
--30T09::17.842814+: Query SHOW TRIGGERS LIKE 'test\_trx'
--30T09::17.844200+: Query SET SESSION character_set_results = 'utf8'
--30T09::17.844540+: Query ROLLBACK TO SAVEPOINT sp
--30T09::17.844803+: Query RELEASE SAVEPOINT sp
--30T09::17.847871+: Quit //The "unlock tables" operation was executed very soon after a short period of time of getting gtid information.
//After the releasing lock operation,the consistency of transaction table can be guaranteed while the non-transaction table cannot be.
This time merely use "--master-data=2".
[root@zlm2 :: ~]
#mysqldump --master-data= zlm > test2.sql //General log output below.
--30T10::41.766819+: Connect zlm@zlm2 on using TCP/IP
--30T10::41.767265+: Query /*!40100 SET @@SQL_MODE='' */
--30T10::41.767495+: Query /*!40103 SET TIME_ZONE='+00:00' */
--30T10::41.767852+: Query FLUSH /*!40101 LOCAL */ TABLES
--30T10::41.768067+: Query FLUSH TABLES WITH READ LOCK //It still generate a FTWRL.
--30T10::41.768237+: Query SHOW VARIABLES LIKE 'gtid\_mode'
--30T10::41.770245+: Query SELECT @@GLOBAL.GTID_EXECUTED
--30T10::41.770659+: Query SHOW MASTER STATUS //No releasing lock operation here.
--30T10::41.770955+: Query SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
--30T10::41.773632+: Query SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
--30T10::41.775734+: Query SHOW VARIABLES LIKE 'ndbinfo\_version'
--30T10::41.777953+: Init DB zlm
--30T10::41.778223+: Query show tables
--30T10::41.778529+: Query show table status like 'test\_notrx'
--30T10::41.778898+: Query SET SQL_QUOTE_SHOW_CREATE=
--30T10::41.779149+: Query SET SESSION character_set_results = 'binary'
--30T10::41.780362+: Query show create table `test_notrx`
--30T10::41.780722+: Query SET SESSION character_set_results = 'utf8'
--30T10::41.780907+: Query show fields from `test_notrx`
--30T10::41.781665+: Query show fields from `test_notrx`
--30T10::41.782447+: Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_notrx`
--30T10::41.782788+: Query SET SESSION character_set_results = 'binary'
--30T10::41.782945+: Query use `zlm`
--30T10::41.783298+: Query select @@collation_database
--30T10::41.783425+: Query SHOW TRIGGERS LIKE 'test\_notrx'
--30T10::41.784086+: Query SET SESSION character_set_results = 'utf8'
--30T10::41.790618+: Query show table status like 'test\_trx'
--30T10::41.791015+: Query SET SQL_QUOTE_SHOW_CREATE=
--30T10::41.791094+: Query SET SESSION character_set_results = 'binary'
--30T10::41.791192+: Query show create table `test_trx`
--30T10::41.791285+: Query SET SESSION character_set_results = 'utf8'
--30T10::41.791360+: Query show fields from `test_trx`
--30T10::41.792328+: Query show fields from `test_trx`
--30T10::41.792837+: Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_trx`
--30T10::41.793036+: Query SET SESSION character_set_results = 'binary'
--30T10::41.793147+: Query use `zlm`
--30T10::41.793277+: Query select @@collation_database
--30T10::41.793381+: Query SHOW TRIGGERS LIKE 'test\_trx'
--30T10::41.794223+: Query SET SESSION character_set_results = 'utf8'
--30T10::41.799961+: Quit //No RR isolation level and consistent snapshot this time what means the consistency of transaction table cannot be guaranteed,too.
//As for the non-transaction table,it will hold the FTWRL until the backup finishes.That means it's consistent in the course.
//The bad influence is that the locking time will be extremly long.Because option of "--master-data" will enbale "--lock-all-tables" or "-x" option.
This time execute mysqldump without any option.
[root@zlm2 :: ~]
#mysqldump zlm > test3.sql //General log output below.
--30T10::04.371650+: Connect zlm@zlm2 on using TCP/IP
--30T10::04.371651+: Query /*!40100 SET @@SQL_MODE='' */
--30T10::04.371885+: Query /*!40103 SET TIME_ZONE='+00:00' */
--30T10::04.372048+: Query SHOW VARIABLES LIKE 'gtid\_mode'
--30T10::04.374903+: Query SELECT @@GLOBAL.GTID_EXECUTED
--30T10::04.376278+: Query SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
--30T10::04.378718+: Query SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
--30T10::04.380381+: Query SHOW VARIABLES LIKE 'ndbinfo\_version'
--30T10::04.383136+: Init DB zlm
--30T10::04.383512+: Query show tables
--30T10::04.383922+: Query LOCK TABLES `test_notrx` READ /*!32311 LOCAL */,`test_trx` READ /*!32311 LOCAL */ //There're two individual locks here.
--30T10::04.384267+: Query show table status like 'test\_notrx'
--30T10::04.384870+: Query SET SQL_QUOTE_SHOW_CREATE=
--30T10::04.385115+: Query SET SESSION character_set_results = 'binary'
--30T10::04.385241+: Query show create table `test_notrx`
--30T10::04.385479+: Query SET SESSION character_set_results = 'utf8'
--30T10::04.385651+: Query show fields from `test_notrx`
--30T10::04.386440+: Query show fields from `test_notrx`
--30T10::04.387011+: Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_notrx`
--30T10::04.387314+: Query SET SESSION character_set_results = 'binary'
--30T10::04.387446+: Query use `zlm`
--30T10::04.387704+: Query select @@collation_database
--30T10::04.388009+: Query SHOW TRIGGERS LIKE 'test\_notrx'
--30T10::04.388492+: Query SET SESSION character_set_results = 'utf8'
--30T10::04.388576+: Query show table status like 'test\_trx'
--30T10::04.389393+: Query SET SQL_QUOTE_SHOW_CREATE=
--30T10::04.389691+: Query SET SESSION character_set_results = 'binary'
--30T10::04.390024+: Query show create table `test_trx`
--30T10::04.390330+: Query SET SESSION character_set_results = 'utf8'
--30T10::04.392180+: Query show fields from `test_trx`
--30T10::04.393918+: Query show fields from `test_trx`
--30T10::04.394497+: Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_trx`
--30T10::04.394847+: Query SET SESSION character_set_results = 'binary'
--30T10::04.395108+: Query use `zlm`
--30T10::04.395955+: Query select @@collation_database
--30T10::04.396372+: Query SHOW TRIGGERS LIKE 'test\_trx'
--30T10::04.397040+: Query SET SESSION character_set_results = 'utf8'
--30T10::04.397295+: Query UNLOCK TABLES //The lock was released after finished backup.
--30T10::04.401336+: Quit //If you don not specify any option of mysqldump,default option "--lock-tables" or "-l" will be enable.
//The "--lock-tables" option will add a read-only lock by executing "lock table xxx read;" individually on which table you want to backup.
//There's neither RR isolation leve nor consistent snapshot.Therefore,only the non-transaction table was guaranteed a consistent backup.
Summary
- We cannot get a consistent backup of non-transaction table with simply using option of "--singel-transaction" and "--master-data" simultaneously.
- If you merely specify the option "--master-data".It can guarantee a consistent backup of non-transaction table since it will enable a FTWRL.That is,it will enable the option of "--lock-all-tables"(short option is "-x").The global read lock on table will sustain until to the end of backup.
- If you do not specify any option of mysqldump,the default option of "--lock-table"(short option is "-l") will be enabled.That will also guarantee a consisten backup of non-transaction table.
- Notice the above situations when backup your non-transaction tables when using mysqldump otherwise you'll get an inconsistent backup.That's really an awful thing.
关于mysqldump备份非事务表的注意事项的更多相关文章
- MySQL事务表和非事务表
查看 max_binlog_stmt_cache_size 参数解释时,有这么一句话 If nontransactional statements within a transaction requi ...
- mysqldump 备份某张表 Warning: A partial dump from a server that has GTIDs will by default include the GTIDs of all transactions,
[root@NB ok]# mysqldump -uemove -h xx.xx.xx.xx -P9906 DBname t_name -p >2t_tname.sqlWarning: A pa ...
- mysql INNODB_TRX 事务表
demo:/root# mysql -uroot -pkjk7787czcb --socket=/data01/mysql/mysql.sock -e"show processlist&qu ...
- Hive ACID和事务表支持详解
一.ACID介绍 ACID就是常见数据库事务的四大特性:Atomicity(原子性).Consistency(一致性).Isolation(隔离性).Durability(持久性). 在Hive 0. ...
- mysqldump 备份数据说明+ 避免锁表
1.mysqldump命令备份Mysql数据库的参数说明 在用mysqldump备份使用那些参数选项是最完美的组合呢?--skip-opt--create-option ...
- mysqldump备份表中有大字段失败的排错过程
几天前收到某个业务项目,MySQL数据库逻辑备份mysqldump备份失败的邮件,本是在休假,但本着工作认真负责,7*24小时不间断运维的高尚职业情操,开始了DBA的排错之路(一开始数据库的备份都是成 ...
- 使用mysqldump备份表数据
使用mysqldump备份远程表数据到本地 下面的命令是使用mysqldump命令备份远程数据库的一张表的信息,并将信息保存到本地的一个文件的一个示例: mysqldump -h 192.168.1. ...
- mysql 5.5 mysqldump备份原理
开启general_log日志,获取mysqldump执行语句 show VARIABLES like 'general_log%' set GLOBAL general_log=on 执行备份命令 ...
- mysql的日志及利用mysqldump备份及还原
日志文件:6类 一般查询日志:log,general_log,log_output 慢查询日志: 错误日志 二进制日志 中继日志 ...
随机推荐
- framework7 1.3.5 路由跳转后DOM失效问题
再这个版本的7会存在一个问题,那就是loadpage到指定页面后才做其中的DOM比如DIV里面的text或者HTML,虽然控制台会显示改变后的值但是页面上却还是原值,这时候需要改变方法使用reload ...
- Maven项目导出可执行jar
配置文件中添加插件 <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>m ...
- 一步步做程序优化-讲一个用于OpenACC优化的程序（转载）
一步步做程序优化[1]讲一个用于OpenACC优化的程序 分析下A,B,C为三个矩阵,A为m*n维,B为n*k维,C为m*k维,用A和B来计算C,计算方法是:C = alpha*A*B + beta* ...
- git fetch和push的区别
获取fetch的用法 git-fetch用于从另一个reposoitory下载objects和refs. 命令格式为:git fetch … 其中表示远端的仓库路径.git remote add or ...
- redis事务中的WATCH命令和基于CAS的乐观锁
转自:http://blog.sina.com.cn/s/blog_ae8441630101cgy3.html 在Redis的事务中,WATCH命令可用于提供CAS(check-and-set)功能. ...
- c++ 11 线程池的简单封装
#include <condition_variable> #include <queue> #include <thread> #include <vect ...
- python读写dbf数据库
dbf数据库作为一种简单的数据库,曾经广泛使用.现在在金融领域还是有很多的应用之处,工作中遇到此类的问题,在此记录一下. 1. 读取dbf ''' 读取DBF文件 ''' def readDbfFil ...
- poj_1284_Primitive root
We say that integer x, 0 < x < p, is a primitive root modulo odd prime p if and only if the se ...
- Linux Centos 通过虚拟用户访问FTP的配置
Linux Centos 通过虚拟用户访问FTP的配置 实验需求: 让下面4个虚拟用户使用系统用户ftpvu的权限来连接到Linux FTP服务器,并确保都锁定在 自己的虚拟用户目录,不能切换到其他目 ...
- Logrotate实现Catalina.out日志每俩小时切割
一.Logrotate工具介绍 Logrotate是一个日志文件管理工具,它是Linux默认自带的一个日志切割工具.用来把旧文件轮转.压缩.删除,并且创建新的日志文件.我们可以根据日志文件的大小.天数 ...