一次基于innobackupex备份及binlog的单表恢复操作
【环境介绍】
系统环境:Red Hat Enterprise Linux Server release 7.0 (Maipo) + Server version: 5.7.18-log MySQL Community Server (GPL)
+innobackupex version 2.4.12 Linux (x86_64)+binlog2sql
【背景描述】
当用户进行误操作时,需要使用innobackupex的备份及binlog进行恢复操作。
【实施步骤】
创建测试表:
-> `id` int(11) NOT NULL AUTO_INCREMENT,
-> `name` char(10) DEFAULT NULL,
-> PRIMARY KEY (`id`)
-> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
+----+------+
| id | name |
+----+------+
| 1 | aa |
| 2 | bb |
| 3 | cc |
| 4 | dd |
+----+------+
4 rows in set (0.10 sec)
Query OK, 4 rows affected (0.12 sec)
Records: 4 Duplicates: 0 Warnings: 0
+----+------+
| id | name |
+----+------+
| 1 | aa |
| 2 | bb |
| 3 | cc |
| 4 | dd |
| 5 | aa2 |
| 6 | bb2 |
| 7 | cc2 |
| 8 | dd2 |
+----+------+
8 rows in set (0.00 sec)
Query OK, 0 rows affected (0.29 sec)
+----------------+
| Tables_in_test |
+----------------+
| orders_range |
| rang_test |
| range_datetime |
| test1 |
| test2 |
| test3 |
+----------------+
6 rows in set (0.00 sec)
# WARNING: Cannot generate character set or collation names without the --server option.
# CAUTION: The diagnostic mode is a best-effort parse of the .frm file. As such, it may not identify all of the components of the table correctly. This is especially true for damaged files. It will also not read the default values for the columns and the resulting statement may not be syntactically correct.
# Reading .frm file for /root/backup/full/2018-12-15_12-12-51/test/tb1.frm:
# The .frm file is a TABLE.
# CREATE TABLE Statement:
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` char(30) DEFAULT NULL,
PRIMARY KEY `PRIMARY` (`id`)
) ENGINE=InnoDB;
mysql> CREATE TABLE `test`.`tb1` (
-> `id` int(11) NOT NULL AUTO_INCREMENT,
-> `name` char(30) DEFAULT NULL,
-> PRIMARY KEY `PRIMARY` (`id`)
-> ) ENGINE=InnoDB;
Query OK, 0 rows affected (0.18 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.13 sec)
[root@mysqldb2 test]# chown -R mysql:mysql /mysqldata/mysql01_1/test/tb1.ibd
[root@mysqldb2 test]# ls -trl /mysqldata/mysql01_1/test/tb1.*
-rw-r----- 1 mysql mysql 8586 12月 15 12:20 /mysqldata/mysql01_1/test/tb1.frm
-rw-r----- 1 mysql mysql 98304 12月 15 12:23 /mysqldata/mysql01_1/test/tb1.ibd
[root@mysqldb2 test]#
mysql> alter table tb1 import tablespace;
Query OK, 0 rows affected, 1 warning (0.10 sec)
+---------+------+------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1810 | InnoDB: IO Read error: (2, No such file or directory) Error opening './test/tb1.cfg', will attempt to import without schema verification |
+---------+------+------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
+----+------+
| id | name |
+----+------+
| 1 | aa |
| 2 | bb |
| 3 | cc |
| 4 | dd |
+----+------+
4 rows in set (0.00 sec)
Query OK, 0 rows affected (0.08 sec)
[root@mysqldb2 logs]# mysqlbinlog -v --base64-output=DECODE-ROWS /mysql/mysql57/logs/binlog011.000002 | grep -C 10 -i "DROP"
# at 19330
#181215 12:15:24 server id 21 end_log_pos 19357 Xid = 811
COMMIT/*!*/;
# at 19357
#181215 12:15:57 server id 21 end_log_pos 19418 GTID last_committed=36 sequence_number=37
SET @@SESSION.GTID_NEXT= '4ee31762-fe82-11e8-9aea-000c29c913a2:37'/*!*/;
# at 19418
#181215 12:15:57 server id 21 end_log_pos 19530 Query thread_id=66 exec_time=1 error_code=0
SET TIMESTAMP=1544847357/*!*/;
DROP TABLE `tb1` /* generated by server */
/*!*/;
# at 19530
#181215 12:20:55 server id 21 end_log_pos 19591 GTID last_committed=37 sequence_number=38
SET @@SESSION.GTID_NEXT= '4ee31762-fe82-11e8-9aea-000c29c913a2:38'/*!*/;
# at 19591
#181215 12:20:55 server id 21 end_log_pos 19807 Query thread_id=66 exec_time=0 error_code=0
SET TIMESTAMP=1544847655/*!*/;
CREATE TABLE `test`.`tb1` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` char(30) DEFAULT NULL,
[root@mysqldb2 logs]#
[root@mysqldb2 logs]# less recovery_tb1.sql
[root@mysqldb2 logs]# cat recovery_tb1.sql
INSERT INTO `test`.`tb1`(`id`, `name`) VALUES (5, 'aa2'); #start 19069 end 19330 time 2018-12-15 12:15:24
INSERT INTO `test`.`tb1`(`id`, `name`) VALUES (6, 'bb2'); #start 19069 end 19330 time 2018-12-15 12:15:24
INSERT INTO `test`.`tb1`(`id`, `name`) VALUES (7, 'cc2'); #start 19069 end 19330 time 2018-12-15 12:15:24
INSERT INTO `test`.`tb1`(`id`, `name`) VALUES (8, 'dd2'); #start 19069 end 19330 time 2018-12-15 12:15:24
[root@mysqldb2 logs]#
[root@mysqldb2 logs]# python /root/binlog2sql-master/binlog2sql/binlog2sql.py -uroot -pmysql -dtest -ttb1 -P33061 --start-position=19008 --stop-position=19418 --start-file='binlog011.000002' -B
DELETE FROM `test`.`tb1` WHERE `id`=8 AND `name`='dd2' LIMIT 1; #start 19069 end 19330 time 2018-12-15 12:15:24
DELETE FROM `test`.`tb1` WHERE `id`=7 AND `name`='cc2' LIMIT 1; #start 19069 end 19330 time 2018-12-15 12:15:24
DELETE FROM `test`.`tb1` WHERE `id`=6 AND `name`='bb2' LIMIT 1; #start 19069 end 19330 time 2018-12-15 12:15:24
DELETE FROM `test`.`tb1` WHERE `id`=5 AND `name`='aa2' LIMIT 1; #start 19069 end 19330 time 2018-12-15 12:15:24
[root@mysqldb2 logs]#
+----+------+
| id | name |
+----+------+
| 1 | aa |
| 2 | bb |
| 3 | cc |
| 4 | dd |
+----+------+
4 rows in set (0.00 sec)
Query OK, 1 row affected (0.05 sec)
| id | name |
+----+------+
| 1 | aa |
| 2 | bb |
| 3 | cc |
| 4 | dd |
| 5 | aa2 |
| 6 | bb2 |
| 7 | cc2 |
| 8 | dd2 |
+----+------+
8 rows in set (0.00 sec)
yum -y install epel-release
yum -y install git
[root@mysqldb2 binlog2sql-master]# wget https://bootstrap.pypa.io/get-pip.py
--2018-12-15 12:45:30-- https://bootstrap.pypa.io/get-pip.py
正在解析主机 bootstrap.pypa.io (bootstrap.pypa.io)... 151.101.0.175, 151.101.64.175, 151.101.128.175, ...
正在连接 bootstrap.pypa.io (bootstrap.pypa.io)|151.101.0.175|:443... 已连接。
已发出 HTTP 请求,正在等待回应... 200 OK
长度:1661676 (1.6M) [text/x-python]
正在保存至: “get-pip.py”
binlog2sql example get-pip.py LICENSE README.md requirements.txt tests
[root@mysqldb2 binlog2sql-master]# python get-pip.py
Collecting pip
/tmp/tmpS0wyK9/pip.zip/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
/tmp/tmpS0wyK9/pip.zip/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
/tmp/tmpS0wyK9/pip.zip/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
100% |████████████████████████████████| 1.3MB 105kB/s
Collecting wheel
Downloading https://files.pythonhosted.org/packages/ff/47/1dfa4795e24fd6f93d5d58602dd716c3f101cfd5a77cd9acbe519b44a0a9/wheel-0.32.3-py2.py3-none-any.whl
Installing collected packages: pip, wheel
Successfully installed pip-18.1 wheel-0.32.3
/tmp/tmpS0wyK9/pip.zip/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
[root@mysqldb2 binlog2sql-master]#
Collecting PyMySQL==0.7.11 (from -r requirements.txt (line 1))
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/c6/42/c54c280d8418039bd2f61284f99cb6d9e0eae80383fc72ceb6eac67855fe/PyMySQL-0.7.11-py2.py3-none-any.whl (78kB)
100% |████████████████████████████████| 81kB 106kB/s
Collecting wheel==0.29.0 (from -r requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/8a/e9/8468cd68b582b06ef554be0b96b59f59779627131aad48f8a5bce4b13450/wheel-0.29.0-py2.py3-none-any.whl (66kB)
100% |████████████████████████████████| 71kB 288kB/s
Collecting mysql-replication==0.13 (from -r requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/dd/23/384047702e694139e9fe75a8ba7ad007e8942fd119ebadabc32ce19f70f2/mysql-replication-0.13.tar.gz
Building wheels for collected packages: mysql-replication
Running setup.py bdist_wheel for mysql-replication ... done
Stored in directory: /root/.cache/pip/wheels/91/33/05/32b16ccadd4fc566ff38af96afdeb5d57d49c2f1eff0402164
Successfully built mysql-replication
Installing collected packages: PyMySQL, wheel, mysql-replication
Found existing installation: wheel 0.32.3
Uninstalling wheel-0.32.3:
Successfully uninstalled wheel-0.32.3
Successfully installed PyMySQL-0.7.11 mysql-replication-0.13 wheel-0.29.0
[root@mysqldb2 binlog2sql-master]#
MyFlash是由美团点评公司技术工程部开发维护的一个回滚DML操作的工具。该工具通过解析v4版本的binlog,完成回滚操作。相对已有的回滚工具,其增加了更多的过滤选项,让回滚更加容易。
**该工具已经在美团点评内部使用**
### 二、详细说明
1. [安装](./doc/INSTALL.md)
2. [使用](./doc/how_to_use.md)
3. [测试用例](./doc/TestCase.md)
### 三、限制
1. binlog格式必须为row,且binlog_row_image=full
2. 仅支持5.6与5.7
3. 只能回滚DML(增、删、改)
### 四、FAQ
1. 实现的原理是什么?
- 答:参考文章http://url.cn/5yVTfLY
- 答:支持。请参考 [使用](./doc/how_to_use.md)
- 答:在导入时加入--skip-gtids
mysqlbinlog --skip-gtids <flashbacklog> | mysql -uxxx -pxxx
- 答:参考 [使用](./doc/how_to_use.md) ,搜索maxSplitSize。使用该参数可以对文件进行切片
一次基于innobackupex备份及binlog的单表恢复操作的更多相关文章
- 通过Xtrabackup实现MySQL实例的全库备份与按需单库恢复
在实际的生产环境中,为了管理方便,我们一般是通过 Xtrabackup实现实例的全库备份,即将实例上的所有数据库备份. 但是,考虑到快速恢复 我们常常面临的需求是快速还原单个数据库.针对初学者来说,网 ...
- MSSQL·备份数据库中的单表
阅文时长 | 0.11分钟 字数统计 | 237.6字符 主要内容 | 1.引言&背景 2.声明与参考资料 『MSSQL·备份数据库中的单表』 编写人 | SCscHero 编写时间 | 20 ...
- MongoDB数据库备份与还原、单表的导入导出
-------------------MongoDB备份与恢复------------------- 1.MongoDB数据库备份 1.语法: mongodump -h dbh ...
- java基于xml配置的通用excel单表数据导入组件(五、Action处理类)
package xxxxxx.manage.importexcel; import java.io.File; import java.util.HashMap; import java.util.M ...
- java基于xml配置的通用excel单表数据导入组件(四、DAO主处理类)
package XXXXX.manage.importexcel; import java.beans.IntrospectionException; import java.io.BufferedR ...
- java基于xml配置的通用excel单表数据导入组件(三、负责数据转换处理的类)
package xxxxxxx.manage.importexcel; import java.util.Map; import java.util.logging.Logger; import xx ...
- java基于xml配置的通用excel单表数据导入组件(二、xml配置文件解析加载)
1.BN_ImportExcel.java 对应xml主节点属性 package XXXXX.manage.importexcel; import java.io.Serializable; impo ...
- java基于xml配置的通用excel单表数据导入组件(一、实际应用过程)
主要应用技术:poi + betwixt + reflect 一.实际应用过程 1.创建与目标表结构一样,表名为‘{目标表名}_import’的临时表: 2.创建用于存储导入问题数据的表:t_impo ...
- 基于Azure blob storage T级别HBase表恢复
为减少Hbase集群压力,提高性能,我们将HBase库中的数据移到另外的存储,下面记录当我需要对数据进行计算时,数据恢复的过程 目录: Azure storage explorer 工具 数据复制 元 ...
随机推荐
- C#的扩展方法(this)
先在StringLibrary类中定义一个静态方法,如下: public static class StringLibrary { //第一个参数指定该方法作用于哪个类型,并且该参数以 this 修饰 ...
- java jdk动态代理(proxy)
1. 涉及主要jdk api java.lang.reflect.InvocationHandler: public interface InvocationHandler { /** * Proce ...
- mysql8.0.主从复制搭建
搭建主从数据库 一.准备两台以上对的数据库 数据库1(主服务器):192.168.2.2 数据库2(从服务器):192.168.2.4 1.1 配置主服务器 .在 /et ...
- linux的常用命令介绍
1.ls 列出当前目录下的所有的文件和文件夹的名称. 参数如下:-a 显示隐藏文件 -l 显示方式为列表 -h 以可读性高的方式输出 eg: ls -lh /logs/tran 目录如果不指定(相 ...
- git tag 打标签
注意:在哪个分支上打tag一定要先提交该分支到远程gitlab仓库 标签(tag)操作 1. 查看所有标签 git tag 默认标签是打在最新提交的commit上的 2.本地打新标签 git tag ...
- python学习——读取染色体长度(一、简化问题)
# 读取fasta # 解析每条序列的长度 chr1_len = 10 chr2_len = 20 chr3_len = 30 chr4_len = 40 chr5_len = 50 # 求和 tot ...
- 设置SSH Key
初始设置git config --global user.name "qian" git config --global user.email "1067803326@q ...
- 解决hash冲突的方法
复制粘贴于:https://www.cnblogs.com/wuchaodzxx/p/7396599.html#H1_2 开放地址法(线性探测法.二次探测.伪随机探测) 再哈希法 链地址法 建立公共溢 ...
- JEECG BOOT
JEECGBOOT - 开源搜索 - 开源中国https://www.oschina.net/search?scope=blog&q=JEECGBOOT JEECG 基于代码生成器J2EE智能 ...
- springmvc的前端控制器
<servlet> <servlet-name>xxx</servlet-name> <servlet-class>org.springframewor ...