MySQL备份恢复之mysqldump
Preface
The day before yesterday,there's a motif about the lock procedure when backing up MySQL using mysqldump or Xtrabackup in "Ask Ye" which is like a kind of Q&A originated by Mr. Ye.In my essay today,I'm gonna use the tool mysqlsump to do some tests then observe procedures of backing up MySQL database,here we go.
Introduction
mysqldump is a protogenic logical backup tool of MySQL.As soon as you've installed MySQL server,it can be used in command line directly.
mysqldump logically dumps data in the target database then gernerate a sql file which can be used to restore the the very database later.it's rather convenient to backup a single or multiple databases online(innodb only) but the side-effect is that it can hold lock when backing up.
Procedure
In order to see the intact procedure of backup,we open the general log by setting "general_log=on" in runtime first.
###Open General Log###
(root@localhost mysql3306.sock)[performance_schema]::>show variables like '%gener%';
+------------------+-------------------------------------+
| Variable_name | Value |
+------------------+-------------------------------------+
| general_log | OFF |
| general_log_file | /data/mysql/mysql3306/data/zlm2.log |
+------------------+-------------------------------------+
rows in set (0.00 sec) (root@localhost mysql3306.sock)[performance_schema]::>set general_log=on;
ERROR (HY000): Variable 'general_log' is a GLOBAL variable and should be set with SET GLOBAL
(root@localhost mysql3306.sock)[performance_schema]::>set global general_log=on;
Query OK, rows affected (0.14 sec) ###Create innodb table and myisam table in database zlm.###
(root@localhost mysql3306.sock)[zlm]::>create table test_innodb(id int primary key) engine=innodb;
Query OK, rows affected (0.02 sec) (root@localhost mysql3306.sock)[zlm]::>create table test_myisam(id int primary key) engine=myisam;
Query OK, rows affected (0.01 sec) (root@localhost mysql3306.sock)[zlm]::>show tables;
+---------------+
| Tables_in_zlm |
+---------------+
| test_innodb |
| test_myisam |
+---------------+
rows in set (0.00 sec) (root@localhost mysql3306.sock)[zlm]::> ###Gnerate First Backup(disable triggers)###
[root@zlm2 :: ~]
#mysqldump --triggers=false -B zlm > /data/backup/first_3306-`date +%Y%m%d`.sql
Warning: A partial dump from a server that has GTIDs will by default include the GTIDs of all transactions, even those that changed suppressed parts of the database. If you don't want to restore GTIDs, pass --set-gtid-purged=OFF. To make a complete dump, pass --all-databases --triggers --routines --events. [root@zlm2 :: ~]
#ls -l /data/backup
total
-rw-r--r-- root root Jun : first_3306-.sql ###Check General Log###
[root@zlm2 :: /data/mysql/mysql3306/data]
#cat zlm2.log --15T07::.345850Z Connect root@localhost on using Socket
--15T07::.346193Z Query /*!40100 SET @@SQL_MODE='' */
--15T07::.346252Z Query /*!40103 SET TIME_ZONE='+00:00' */
--15T07::.346332Z Query SHOW VARIABLES LIKE 'gtid\_mode'
--15T07::.348422Z Query SELECT @@GLOBAL.GTID_EXECUTED
--15T07::.350309Z Query SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
--15T07::.351603Z Query SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
--15T07::.352151Z Query SHOW VARIABLES LIKE 'ndbinfo\_version'
--15T07::.354634Z Init DB zlm
--15T07::.354659Z Query SHOW CREATE DATABASE IF NOT EXISTS `zlm`
--15T07::.354660Z Query show tables
--15T07::.354747Z Query LOCK TABLES `test_innodb` READ /*!32311 LOCAL */,`test_myisam` READ /*!32311 LOCAL */
--15T07::.354815Z Query show table status like 'test\_innodb'
--15T07::.355067Z Query SET SQL_QUOTE_SHOW_CREATE=
--15T07::.355110Z Query SET SESSION character_set_results = 'binary'
--15T07::.355144Z Query show create table `test_innodb`
--15T07::.355188Z Query SET SESSION character_set_results = 'utf8'
--15T07::.355227Z Query show fields from `test_innodb`
--15T07::.355412Z Query show fields from `test_innodb`
--15T07::.355631Z Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_innodb`
--15T07::.356449Z Query show table status like 'test\_myisam'
--15T07::.356723Z Query SET SQL_QUOTE_SHOW_CREATE=
--15T07::.356759Z Query SET SESSION character_set_results = 'binary'
--15T07::.356819Z Query show create table `test_myisam`
--15T07::.356863Z Query SET SESSION character_set_results = 'utf8'
--15T07::.356900Z Query show fields from `test_myisam`
--15T07::.357109Z Query show fields from `test_myisam`
--15T07::.357349Z Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_myisam`
--15T07::.357420Z Query UNLOCK TABLES --Release the lock only after all tables have finished backup.
--15T07::.361654Z Quit
I've make the output be simplest by disable triggers' backup.We can see from general log is that "LOCK TABLES test_innodb READ" then the "UNLOCK TABLES" statment.Let's see the difference with parameter "single-transaction".
###Gnerate Sceond Backup(enable transaction consistent backup)###
[root@zlm2 :: ~]
#mysqldump --triggers=false --single-transaction -B zlm > /data/backup/second_3306-`date +%Y%m%d`.sql
Warning: A partial dump from a server that has GTIDs will by default include the GTIDs of all transactions, even those that changed suppressed parts of the database. If you don't want to restore GTIDs, pass --set-gtid-purged=OFF. To make a complete dump, pass --all-databases --triggers --routines --events. [root@zlm2 :: ~]
#ls -l /data/backup
total
-rw-r--r-- root root Jun : first_3306-.sql
-rw-r--r-- root root Jun : second_3306-.sql ###Check General Log###
[root@zlm2 :: /data/mysql/mysql3306/data]
#cat zlm2.log --15T08::.911566Z Connect root@localhost on using Socket
--15T08::.911733Z Query /*!40100 SET @@SQL_MODE='' */
--15T08::.911848Z Query /*!40103 SET TIME_ZONE='+00:00' */
--15T08::.912749Z Query SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ
--15T08::.912839Z Query START TRANSACTION /*!40100 WITH CONSISTENT SNAPSHOT */
--15T08::.912919Z Query SHOW VARIABLES LIKE 'gtid\_mode'
--15T08::.915228Z Query SELECT @@GLOBAL.GTID_EXECUTED
--15T08::.915371Z Query UNLOCK TABLES --Release lock here untill end,there're no more locks.
--15T08::.915568Z Query SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
--15T08::.916737Z Query SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
--15T08::.918498Z Query SHOW VARIABLES LIKE 'ndbinfo\_version'
--15T08::.920665Z Init DB zlm
--15T08::.920742Z Query SHOW CREATE DATABASE IF NOT EXISTS `zlm`
--15T08::.920787Z Query SAVEPOINT sp --Notice,there's a save opoint here.
--15T08::.920837Z Query show tables
--15T08::.921068Z Query show table status like 'test\_innodb'
--15T08::.921242Z Query SET SQL_QUOTE_SHOW_CREATE=
--15T08::.921290Z Query SET SESSION character_set_results = 'binary'
--15T08::.921334Z Query show create table `test_innodb`
--15T08::.921397Z Query SET SESSION character_set_results = 'utf8'
--15T08::.921444Z Query show fields from `test_innodb`
--15T08::.921833Z Query show fields from `test_innodb`
--15T08::.922279Z Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_innodb`
--15T08::.922380Z Query ROLLBACK TO SAVEPOINT sp --Rollback to savepoint sp
--15T08::.922487Z Query show table status like 'test\_myisam'
--15T08::.922694Z Query SET SQL_QUOTE_SHOW_CREATE=
--15T08::.922730Z Query SET SESSION character_set_results = 'binary'
--15T08::.922763Z Query show create table `test_myisam`
--15T08::.922872Z Query SET SESSION character_set_results = 'utf8'
--15T08::.922929Z Query show fields from `test_myisam`
--15T08::.923140Z Query show fields from `test_myisam`
--15T08::.923395Z Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_myisam`
--15T08::.923459Z Query ROLLBACK TO SAVEPOINT sp --Rollback to savepoint sp.
--15T08::.923487Z Query RELEASE SAVEPOINT sp --Release it where backup finish.
--15T08::.928411Z Quit
There is only one piece of "UNLOCK TABLES" which is related with lock can be found this time.At the very beginning of the general log,it shows "START TRANSACTION WITH CONSISTENT SNAPSHOT".As soon as it gets the gtid_executed variable,then it will realse the lock by execute statment "UNLOCK TABLES".Obviously,it's a very short time.
After get the statment of creating database,there's a savepoint created.The rollback operation of the savepoint hapens after it gets the full table backup.Let's see another parameter "master-data" which can make something different,too.
###Gnerate Third Backup(add replication information)###
[root@zlm2 :: ~]
#mysqldump --triggers=false --single-transaction --master-data= -B zlm > /data/backup/third_3306-`date +%Y%m%d`.sql
Warning: A partial dump from a server that has GTIDs will by default include the GTIDs of all transactions, even those that changed suppressed parts of the database. If you don't want to restore GTIDs, pass --set-gtid-purged=OFF. To make a complete dump, pass --all-databases --triggers --routines --events. [root@zlm2 :: ~]
#ls -l /data/backup
total
-rw-r--r-- root root Jun : first_3306-.sql
-rw-r--r-- root root Jun : second_3306-.sql
-rw-r--r-- root root Jun : third_3306-.sql ###Check General Log###
[root@zlm2 :: /data/mysql/mysql3306/data]
#cat zlm2.log --15T08::.848849Z Connect root@localhost on using Socket
--15T08::.849052Z Query /*!40100 SET @@SQL_MODE='' */
--15T08::.849153Z Query /*!40103 SET TIME_ZONE='+00:00' */
--15T08::.849290Z Query FLUSH /*!40101 LOCAL */ TABLES -- Difference .
--15T08::.855139Z Query FLUSH TABLES WITH READ LOCK -- Difference .
--15T08::.855196Z Query SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ
--15T08::.855225Z Query START TRANSACTION /*!40100 WITH CONSISTENT SNAPSHOT */
--15T08::.855272Z Query SHOW VARIABLES LIKE 'gtid\_mode'
--15T08::.857074Z Query SELECT @@GLOBAL.GTID_EXECUTED
--15T08::.857406Z Query SHOW MASTER STATUS
--15T08::.857498Z Query UNLOCK TABLES
--15T08::.857622Z Query SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm'))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
--15T08::.858662Z Query SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN ('zlm')) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
--15T08::.859309Z Query SHOW VARIABLES LIKE 'ndbinfo\_version'
--15T08::.861396Z Init DB zlm
--15T08::.862152Z Query SHOW CREATE DATABASE IF NOT EXISTS `zlm`
--15T08::.862255Z Query SAVEPOINT sp
--15T08::.862322Z Query show tables
--15T08::.862485Z Query show table status like 'test\_innodb'
--15T08::.862665Z Query SET SQL_QUOTE_SHOW_CREATE=
--15T08::.862724Z Query SET SESSION character_set_results = 'binary'
--15T08::.862777Z Query show create table `test_innodb`
--15T08::.862827Z Query SET SESSION character_set_results = 'utf8'
--15T08::.862880Z Query show fields from `test_innodb`
--15T08::.863198Z Query show fields from `test_innodb`
--15T08::.863476Z Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_innodb`
--15T08::.863597Z Query ROLLBACK TO SAVEPOINT sp
--15T08::.863668Z Query show table status like 'test\_myisam'
--15T08::.865590Z Query SET SQL_QUOTE_SHOW_CREATE=
--15T08::.865833Z Query SET SESSION character_set_results = 'binary'
--15T08::.865853Z Query show create table `test_myisam`
--15T08::.865853Z Query SET SESSION character_set_results = 'utf8'
--15T08::.865854Z Query show fields from `test_myisam`
--15T08::.866059Z Query show fields from `test_myisam`
--15T08::.867277Z Query SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_myisam`
--15T08::.867367Z Query ROLLBACK TO SAVEPOINT sp
--15T08::.867398Z Query RELEASE SAVEPOINT sp
--15T08::.869643Z Quit
There're two differences above,one is "FLUSH TABLES" another one is "FLUSH TABLE WITH READ LOCK".In case of other transactoins hold the table lock related with the table to be backed up,the backup operation will be blocked until those transactions release the lock or waiting until time out(according to value of "lock_wait_timeout") occurs.
As a matter of fact,we usually use "master-data" to backup our databases,we shoudn't make it happen in the case of there're massive dml operations continueously executed in rush hour.This will lead to failure of your backup.
Summary
- You've been clear about the whole procedure of backup using mysqldump now.But,there's still one thing make me confused is that why I used "master-data" in mysqldump but not got the change master statement.Is it due to my Group Replication environment?
- Look out,this tool doesnot support backing up in parallel mode.If your database is huge enough,do think twice or use another backup tool in stead of it.
MySQL备份恢复之mysqldump的更多相关文章
- Mysql 备份恢复之 Mysqldump 工具
目前正在学习中,看到mysqldump工具导出的数据都是文本形式的,如果是blob或text大对象类型导出的是什么格式的?这个需要后续研究.下面只先总结下简单的. 一.备份1.备份Mysql一个数据库 ...
- MySQL 备份脚本--mysqldump在slave 上进行备份
MySQL 备份脚本--mysqldump在slave 上进行备份 使用mysqldump在slave上进行备份,建议使用stop slave sql_thread,start slave sql_t ...
- MySQL备份还原——mysqldump工具介绍
mysqldump是一款MySQL逻辑备份的工具,他将数据库里面的对象(表)导出成SQL脚本文件.有点类似于SQL SEVER的"任务-生成脚本"的逻辑备份功能.mysqldump ...
- Mysql备份工具mysqldump和mysqlhotcopy
(1).Mysql备份类型 1)按照备份时对数据库的影响分为 Hot backup(热备):也叫在线备份.指在数据库运行中直接备份,对正在运行的数据库没有任何影响. Cold backup(冷备):也 ...
- MySQL备份命令mysqldump参数说明与示例
1. 语法选项说明 -h, --host=name主机名 -P[ port_num], --port=port_num用于连接MySQL服务器的的TCP/IP端口号 --master-data这个选项 ...
- Mysql备份迁移——Mysqldump（.NET调用Mysqldump.exe方式）——（解决视图嵌视图报错）
利用Mysqldump备份和迁移,我想很多人都用过,具体参数不介绍了,这里主要讲.NET调用Mysqldump进行备份和.NET调用Mysql.exe进行导入数据. 这里使用的是5.1版的Mysqld ...
- MySQL备份恢复-mysqldump原理
+++++++++++++++++++++++++++++++++++++++++++标题:mysqldump对MySQL数据库备份恢复原理时间:2019年2月23日内容:mysqldump工具重点: ...
- mysql备份脚本-mysqldump
背景:全库备份 备份流程: 1.生成DB列表,将DB名字写入文件 2.定义备份函数,结果写入SQL文件 3.压缩文件,减少磁盘占用量 4.设置保留天数,定期删除n天之前的 5.通过for循环读取DB列 ...
- MySQL备份恢复之mydumper
Preface In my previous two blogs,we have known about the tool of backing up MySQL db.I'm gon ...
随机推荐
- SQL使用bcp方式导入，导出数据2
select * from A_Account EXEC sp_configure 'allow_updates' GO EXEC sp_configure 'allow_updates',0; ...
- 【基础笔记】《html&CSS设计与构造网站》一书导读
◉HTML 1.结构网页使用HTML HyperText Markup Language 来描述页面结构超文本标记语言允许对文本建立链接,允许对文本进行标记网页开头都有一个DOCTYPE 文档类型 声 ...
- Java入门到精通——框架篇之Hadoop概述
一.Hadoop来历 Hadoop的思想来源于Google在做搜索引擎的时候出现一个很大的问题就是这么多网页我如何才能以最快的速度来搜索到,由于这个问题Google发明了倒排索引算法,通过加入了Map ...
- 如何开放 Azure 虚拟机 Ping 功能
前言 文章<使用 PsPing & PaPing 进行 TCP 端口连通性测试>中提到,ICMP 协议的数据包无法通过 Azure 的防火墙和负载均衡器,所以不能直接使用 Ping ...
- rel 属性<small>H5保留属性</small>
源文件
- oracle 12c使用dblink克隆pdb
Multitenant : Hot Clone a Remote PDB or Non-CDB in Oracle Database 12c Release 2 (12.2)https://oracl ...
- Opportunity的chance of success的赋值逻辑
该字段的值和另外两个字段Sales Stage和Status都相关. 从下列function module CRM_OPPORT_H_PROB_SET_EC可看出,当status不为代码中的这些硬编码 ...
- POJ 3321 DFS序
Apple Tree Time Limit: 2000MS Memory Limit: 65536K Total Submissions: 30636 Accepted: 9162 Descr ...
- 2016-2017 ACM Central Region of Russia Quarterfinal Programming Contest
2016-2017 ACM Central Region of Russia Quarterfinal Programming Contest A. Fried Fish 题意:有N条鱼,有一个同时可 ...
- pytorch 反向梯度计算问题
计算如下\begin{array}{l}{x_{1}=w_{1} * \text { input }} \\ {x_{2}=w_{2} * x_{1}} \\ {x_{3}=w_{3} * x_{2} ...