rds下载备份集在另外一台机器上恢复并应用binlog日志
-----------------------------备份还原并启动数据库----------------------------------
1.创建目录,并把下载的压缩文件拷贝到相应的目录
[root@localhost opt]# mkdir -p /opt/xbackup_file
[root@localhost opt]# cd /soft
[root@localhost soft]# cp hins5246445_data_20180903064437.tar.gz /opt/xbackup_file/
解压缩
[root@localhost xbackup_file]# tar -zxvf hins5246445_data_20180903064437.tar.gz
查看解压后的文件
[root@localhost xbackup_file]# ls
[root@localhost xbackup_file]# ls -al
total 362780
drwxr-xr-x 15 root root 4096 Feb 16 11:33 .
drwxr-xr-x. 4 root root 130 Feb 16 11:14 ..
-rw-rw---- 1 root root 450 Feb 15 01:41 backup-my.cnf
drwxr-xr-x 2 root root 4096 Feb 16 11:32 db_admin
drwxr-xr-x 2 root root 12288 Feb 16 11:32 db_xdmwxxc
-rw-rw---- 1 root root 209715200 Feb 15 01:41 ibdata1
-rw-rw---- 1 root root 99777236 Feb 15 01:40 log000000000003.tokulog29
drwxr-xr-x 2 root root 4096 Feb 16 11:32 mysql
drwxr-xr-x 2 root root 4096 Feb 16 11:32 performance_schema
drwxr-xr-x 2 root root 294 Feb 16 11:32 push
drwxr-xr-x 2 root root 4096 Feb 16 11:32 server_statistics
drwxr-xr-x 2 root root 20 Feb 16 11:32 test
-rw-rw---- 1 root root 16384 Aug 15 2016 tokudb.directory
-rw-rw---- 1 root root 32768 Apr 23 2018 tokudb.environment
-rw-rw---- 1 root root 32768 Apr 23 2018 tokudb.rollback
-rw-rw---- 1 root root 124 Feb 15 01:41 xtrabackup_binlog_info
-rw-rw---- 1 root root 123 Feb 15 01:41 xtrabackup_checkpoints
-rw-rw---- 1 root root 823 Feb 15 01:41 xtrabackup_info
-rw-rw---- 1 root root 61804032 Feb 15 01:41 xtrabackup_logfile
-rw-rw---- 1 root root 81 Feb 15 01:41 xtrabackup_slave_filename_info
-rw-rw---- 1 root root 164 Feb 15 01:41 xtrabackup_slave_info
drwxr-xr-x 2 root root 12288 Feb 16 11:32 yf_zdyu
drwxr-xr-x 2 root root 4096 Feb 16 11:32 yf_leif
drwxr-xr-x 2 root root 4096 Feb 16 11:32 zprogram
2.修改目录权限
将刚才解压的目录权限赋予mysql
[root@localhost opt]# cd /opt
[root@localhost opt]# chown -R mysql:mysql xbackup_file
3.停掉原来的数据库
要是恢复的机器上之前安装了mysql,需要将其停掉,若没有安装mysql的话,需要事先安装mysql,注意安装的版本需要跟rds的要一致
[root@localhost bin]# ./mysqladmin -h localhost -uroot -pmysql shutdown
4.删除之前数据库的data目录下的文件
我的做法是重命名data目录,重新创建
[root@localhost mysql]# cd /db/mysql
[root@localhost mysql]# mv data bakdata
[root@localhost mysql]# mkdir data
[root@localhost mysql]# chown -R mysql:mysql ./data
5.准备配置文件(my.cnf)
拷贝配置文件到数据库安装目录
[root@localhost xbackup_file]# cp backup-my.cnf /opt/mysql-5.6.40/conf/
重命名为my.cnf(若原来的目录有my.cnf,需要重命名该文件)
[mysql@localhost conf]$ mv backup-my.cnf my.cnf
文件拷贝后需要注意修改权限
[root@localhost opt]# chown -R mysql:mysql ./mysql-5.6.40/
修改该文件的内容如下:
[root@localhost conf]# more my.cnf
# This MySQL options file was generated by innobackupex.
# The MySQL server
[mysqld]
innodb_checksum_algorithm=innodb
innodb_data_file_path=ibdata1:200M:autoextend
innodb_log_files_in_group=2
innodb_log_file_size=1572864000
innodb_page_size=16384
innodb_undo_directory=.
innodb_undo_tablespaces=0
port=3306
server-id=1
datadir=/opt/mysql-5.6.40/data
max_connections = 1500
character_set_server=utf8mb4
collation-server=utf8mb4_general_ci
init_connect='SET collation_connection = utf8mb4_general_ci'
init_connect='SET NAMES utf8mb4'
gtid_mode=on
log-bin=mysql-bin
log_slave_updates=1
enforce_gtid_consistency=ON
binlog_format=row
skip-grant-tables
6.删除掉redolog目录下的日志组
要是redolog有日志组的话需要删除掉,这里根据个人的环境配置不同而不同,若redolog也是放在data目录下的话,可以忽略此步骤.
7.进行恢复
首先执行:
/opt/xtrabackup247/bin/innobackupex --defaults-file=/db/mysql/conf/my.cnf --user=root --apply-log /opt/xbackup_file
[root@localhost conf]# /opt/xtrabackup247/bin/innobackupex --defaults-file=/db/mysql/conf/my.cnf --user=root --apply-log /opt/xbackup_file
180903 16:51:56 innobackupex: Starting the apply-log operation
IMPORTANT: Please check that the apply-log run completes successfully.
At the end of a successful apply-log run innobackupex
prints "completed OK!".
/opt/xtrabackup247/bin/innobackupex version 2.4.7 based on MySQL server 5.7.13 Linux (x86_64) (revision id: 05f1fcf)
xtrabackup: cd to /opt/xbackup_file/
xtrabackup: This target seems to be not prepared yet.
InnoDB: Number of pools: 1
xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(133592877048)
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = .
xtrabackup: innodb_data_file_path = ibdata1:200M:autoextend
xtrabackup: innodb_log_group_home_dir = .
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 8388608
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = .
xtrabackup: innodb_data_file_path = ibdata1:200M:autoextend
xtrabackup: innodb_log_group_home_dir = .
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 8388608
xtrabackup: Starting InnoDB instance for recovery.
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
InnoDB: PUNCH HOLE support available
InnoDB: Mutexes and rw_locks use GCC atomic builtins
InnoDB: Uses event mutexes
InnoDB: GCC builtin __sync_synchronize() is used for memory barrier
InnoDB: Compressed tables use zlib 1.2.3
InnoDB: Number of pools: 1
InnoDB: Not using CPU crc32 instructions
InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M
InnoDB: Completed initialization of buffer pool
InnoDB: page_cleaner coordinator priority: -20
InnoDB: Highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 133592877048
InnoDB: Doing recovery: scanned up to log sequence number 133592931244 (0%)
InnoDB: Doing recovery: scanned up to log sequence number 133592931244 (0%)
InnoDB: Database was not shutdown normally!
InnoDB: Starting crash recovery.
InnoDB: Doing recovery: scanned up to log sequence number 133592931244 (0%)
InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: xtrabackup: Last MySQL binlog file position 406368, file name mysql-bin.000078
InnoDB: Creating shared tablespace for temporary tables
InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
InnoDB: File './ibtmp1' size is now 12 MB.
InnoDB: 96 redo rollback segment(s) found. 1 redo rollback segment(s) are active.
InnoDB: 32 non-redo rollback segment(s) are active.
InnoDB: page_cleaner: 1000ms intended loop took 10438ms. The settings might not be optimal. (flushed=0 and evicted=0, during the time.)
InnoDB: 5.7.13 started; log sequence number 133592931244
InnoDB: xtrabackup: Last MySQL binlog file position 406368, file name mysql-bin.000078
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
InnoDB: FTS optimize thread exiting.
InnoDB: Starting shutdown...
InnoDB: Shutdown completed; log sequence number 133592932517
InnoDB: Number of pools: 1
xtrabackup: using the following InnoDB configuration for recovery:
xtrabackup: innodb_data_home_dir = .
xtrabackup: innodb_data_file_path = ibdata1:200M:autoextend
xtrabackup: innodb_log_group_home_dir = .
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 1572864000
InnoDB: PUNCH HOLE support available
InnoDB: Mutexes and rw_locks use GCC atomic builtins
InnoDB: Uses event mutexes
InnoDB: GCC builtin __sync_synchronize() is used for memory barrier
InnoDB: Compressed tables use zlib 1.2.3
InnoDB: Number of pools: 1
InnoDB: Not using CPU crc32 instructions
InnoDB: Initializing buffer pool, total size = 100M, instances = 1, chunk size = 100M
InnoDB: Completed initialization of buffer pool
InnoDB: page_cleaner coordinator priority: -20
InnoDB: Setting log file ./ib_logfile101 size to 1500 MB
InnoDB: Progress in MB:
100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500
InnoDB: Setting log file ./ib_logfile1 size to 1500 MB
InnoDB: Progress in MB:
100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500
InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
InnoDB: New log files created, LSN=133592932517
InnoDB: Highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 133592932876
InnoDB: Doing recovery: scanned up to log sequence number 133592932885 (0%)
InnoDB: Doing recovery: scanned up to log sequence number 133592932885 (0%)
InnoDB: Database was not shutdown normally!
InnoDB: Starting crash recovery.
InnoDB: xtrabackup: Last MySQL binlog file position 406368, file name mysql-bin.000078
InnoDB: Removed temporary tablespace data file: "ibtmp1"
InnoDB: Creating shared tablespace for temporary tables
InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
InnoDB: File './ibtmp1' size is now 12 MB.
InnoDB: 96 redo rollback segment(s) found. 1 redo rollback segment(s) are active.
InnoDB: 32 non-redo rollback segment(s) are active.
InnoDB: page_cleaner: 1000ms intended loop took 243181ms. The settings might not be optimal. (flushed=0 and evicted=0, during the time.)
InnoDB: 5.7.13 started; log sequence number 133592932885
xtrabackup: starting shutdown with innodb_fast_shutdown = 1
InnoDB: FTS optimize thread exiting.
InnoDB: Starting shutdown...
InnoDB: Shutdown completed; log sequence number 133592934827
180903 16:56:18 completed OK!
接着执行:
/opt/xtrabackup247/bin/innobackupex --defaults-file=/db/mysql/conf/my.cnf --user=root --copy-back --rsync /opt/xbackup_file
这里输出很多,省略.
8.修改权限
[root@localhost db]# cd /db
[root@localhost db]# chown -R mysql:mysql ./mysql
9.启动数据库
[root@localhost bin]# cd /db/mysql/bin
[root@localhost bin]#./mysqld_safe --defaults-file=/db/mysql/conf/my.cnf --user=mysql &
启动保持,发现有不支持的参数
2018-09-03 17:57:19 24869 [ERROR] /db/mysql/bin/mysqld: unknown variable 'innodb_log_checksum_algorithm=innodb'
2018-09-03 18:01:11 25745 [ERROR] /db/mysql/bin/mysqld: unknown variable 'innodb_fast_checksum=false'
2018-09-03 18:02:04 26077 [ERROR] /db/mysql/bin/mysqld: unknown variable 'innodb_log_block_size=512'
2018-09-03 18:03:14 26412 [ERROR] /db/mysql/bin/mysqld: unknown variable 'rds_encrypt_data=false'
2018-09-03 18:03:52 26701 [ERROR] /db/mysql/bin/mysqld: unknown variable 'innodb_encrypt_algorithm=aes_128_ecb'
将这些参数去掉
innodb_log_checksum_algorithm=innodb
innodb_fast_checksum=false
innodb_log_block_size=512
rds_encrypt_data=false
innodb_encrypt_algorithm=aes_128_ecb
10.跳过密码登陆启动
步骤9启动后,发现登陆不了数据库,这个时候需要跳过密码启动,修改root账号的密码
在my.cnf配置文件添加如下项目
skip-grant-tables
停掉(kill掉mysql进程)再启动
[root@localhost bin]# ./mysqld_safe --defaults-file=/db/mysql/conf/my.cnf --user=mysql &
11.修改root账号密码
mysql -h localhost -uroot
use mysql;
update user set password=password("mysql") where user="root";
flush privileges;
12.去掉skip-grant-tables后再启动后进行登陆
[root@localhost bin]# ./mysql -h 127.0.0.1 -uroot -pmysql
到这里数据库恢复完成,下面进行日志的应用.
----------------------------日志的应用--------------------------------------
从上面的恢复可以看到,数据库目前恢复到的位置为:
InnoDB: xtrabackup: Last MySQL binlog file position 406368, file name mysql-bin.000078
若是上面没有输出的话,也可以从xtrabackup信息里查看,如下:
[root@localhost xbackup_file]# more xtrabackup_info
binlog_pos = filename 'mysql-bin.000078', position 406368, GTID of the last change '8bfac327-62cb-11e6-9715-6c92bf2c45fc:1-661898,
9f7dd74e-62cb-11e6-9716-6c92bf315fa3:1-1185674168'
我们要应用该位置之后的日志
可以查看mysql-bin.000078日志内容:
[root@localhost bin]# ./mysqlbinlog --base64-output=decode-rows -v /soft/rds_binlog/mysql-bin.000078>/tmp/aa.txt
我这里输出到aa.txt文件,我们这里从位置开始应用日志:
[root@localhost rds_binlog]# /db/mysql/bin/mysqlbinlog /soft/rds_binlog/mysql-bin.000078 --start-position=""|/db/mysql/bin/mysql -h 127.0.0.1 -uroot -pmysql
Warning: Using a password on the command line interface can be insecure.
ERROR 1781 (HY000) at line 15: @@SESSION.GTID_NEXT cannot be set to UUID:NUMBER when @@GLOBAL.GTID_MODE = OFF
解决办法:
1.查看当前数据库的gtid_mode模式
mysql> show global variables like 'gtid_mode';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | OFF |
+---------------+-------+
1 row in set (0.00 sec)
2. 5.6版本需要修改配置文件my.cnf,添加如下项目重启动mysql
gtid_mode=on
log-bin=mysql-bin
log_slave_updates=1
enforce_gtid_consistency=ON
5.7版本的话可以执行如下命令
set @@GLOBAL.GTID_MODE = OFF_PERMISSIVE;
重启动mysql
mysqld_safe --defaults-file=/db/mysql/conf/my.cnf --user=mysql &
再次执行应用日志,发现报错:
[root@localhost rds_binlog]# /db/mysql/bin/mysqlbinlog /soft/rds_binlog/mysql-bin.000078 --start-position="406368"|/db/mysql/bin/mysql -h 127.0.0.1 -uroot -pmysql
Warning: Using a password on the command line interface can be insecure.
ERROR 1666 (HY000) at line 48: Cannot execute statement: impossible to write to binary log since statement is in row format and BINLOG_FORMAT = STATEMENT.
解决办法:
因为rds数据库的BINLOG_FORMAT设置的为ROW模式,所以本地数据库也要设置为ROW模式,如下
binlog_format=row
修改完配置文件后需要重启动
再次执行应用日志
应用mysql-bin.000078
/db/mysql/bin/mysqlbinlog /soft/rds_binlog/mysql-bin.000078 --start-position=""|/db/mysql/bin/mysql -h 127.0.0.1 -uroot -pmysql
应用mysql-bin.000079
/db/mysql/bin/mysqlbinlog /soft/rds_binlog/mysql-bin.000079|/db/mysql/bin/mysql -h 127.0.0.1 -uroot -pmysql
应用mysql-bin.000080
/db/mysql/bin/mysqlbinlog /soft/rds_binlog/mysql-bin.000080|/db/mysql/bin/mysql -h 127.0.0.1 -uroot -pmysql
应用mysql-bin.000081
/db/mysql/bin/mysqlbinlog /soft/rds_binlog/mysql-bin.000081|/db/mysql/bin/mysql -h 127.0.0.1 -uroot -pmysql
附上整个还原环境的配置文件参数(my.cnf)
[root@localhost conf]# more my.cnf
# This MySQL options file was generated by innobackupex.
# The MySQL server
[mysqld]
innodb_checksum_algorithm=innodb
innodb_data_file_path=ibdata1:200M:autoextend
innodb_log_files_in_group=2
innodb_log_file_size=1572864000
innodb_page_size=16384
innodb_undo_directory=.
innodb_undo_tablespaces=0
port=3306
server-id=1
datadir=/opt/mysql-5.6.40/data
max_connections = 1500
character_set_server=utf8mb4
collation-server=utf8mb4_general_ci
init_connect='SET collation_connection = utf8mb4_general_ci'
init_connect='SET NAMES utf8mb4'
gtid_mode=on
log-bin=mysql-bin
log_slave_updates=1
enforce_gtid_consistency=ON
binlog_format=row
skip-grant-tables
-- The End--
rds下载备份集在另外一台机器上恢复并应用binlog日志的更多相关文章
- rds下载备份集
python版本[testuser@localhost tmp]$ python -VPython 2.7.5 需要提前安装RDS[root@localhost ~]# yum -y install ...
- 如何在同一台机器上安装多个MySQL的实例
转自:'http://www.cnblogs.com/shangzekai/p/4375271.html 最近由于工作的需要,需要在同一台机器上搭建两个MySQL的实例,(注:已经存在了一个3306的 ...
- Hexo博客系列(二)-在多台机器上利用Hexo发布博客
[原文链接]:https://www.tecchen.xyz/blog-hexo-env-02.html 我的个人博客:https://www.tecchen.xyz,博文同步发布到博客园. 由于精力 ...
- 如何在同一台机器上安装多个MySQL的实例 转
https://www.cnblogs.com/shangzekai/p/4375271.html 最近由于工作的需要,需要在同一台机器上搭建两个MySQL的实例,(注:已经存在了一个3306的MyS ...
- 如何在同一台机器上安装多个MySQL的实例(转)
最近由于工作的需要,需要在同一台机器上搭建两个MySQL的实例,(注:已经存在了一个3306的MySQL的实例). 先说下,什么是mysql的多实例,简单的来说就是一台机器上安装了多个mysql的服务 ...
- 解决mysql跟php不在同一台机器上,编译安装php服务报错问题:configure: error: Cannot find MySQL header files under /application/mysql.
在编译安装php服务时报错: configure: error: Cannot find MySQL header files under /application/mysql. Note that ...
- window下在同一台机器上安装多个版本jdk,修改环境变量不生效问题处理办法
window下在同一台机器上安装多个版本jdk,修改环境变量不生效问题处理办法 本机已经安装了jdk1.7,而比较早期的项目需要依赖jdk1.6,于是同时在本机安装了jdk1.6和jdk1.7. 安装 ...
- 不要将缓存服务器与Tomcat放在单台机器上,否则出现竞争内存问题
缓存分为本地缓存和远程分布式缓存,本地缓存访问速度更快但缓存数据量有限,同时存在与应用程序争用内存的情况. 1.不要将缓存服务器与Tomcat放在单台机器上,否则出现竞争内存问题 2.不要将缓存服务器 ...
- 通过Mouse Without Borders在多台机器上共享键盘鼠标
博客搬到了fresky.github.io - Dawei XU,请各位看官挪步.最新的一篇是:通过Mouse Without Borders在多台机器上共享键盘鼠标.
随机推荐
- 从写json作业谈起
json的数据格式我经常见到,但是真正的写json的处理时,我又不会了,com.alibaba.fast.json. com.jackson.看了网上的博客,我可以写出简单java对象转换为json字 ...
- 45道sql
一.设有一数据库,包括四个表:学生表(Student).课程表(Course).成绩表(Score)以及教师信息表(Teacher).四个表的结构分别如表1-1的表(一)~表(四)所示,数据如表1-2 ...
- cmder 常用配置(包括默认管理员运行和解决中文乱码)
简介 cmder是一个增强型命令行工具,不仅可以使用windows下的所有命令,更爽的是可以使用linux的命令,shell命令. 下载 官网地址:http://cmder.net/ 下载的时候,会有 ...
- GCD与LCM
求最大公约数(GCD)和求最小公倍数(LCM): 首先是求最大公约数,我们可以利用辗转相除法来求 1 int gcd(int a,int b) 2 { 3 if(b==0) 4 return a; 5 ...
- 剑指offer——python【第43题】左旋转字符串
题目描述 汇编语言中有一种移位指令叫做循环左移(ROL),现在有个简单的任务,就是用字符串模拟这个指令的运算结果.对于一个给定的字符序列S,请你把其循环左移K位后的序列输出.例如,字符序列S=”abc ...
- python全栈开发 * 22 面向对象 知识点汇总 * 180703
22 面向对象 -----属性,类方法,静态方法,反射一.属性1.属性的定义:将方法伪装成属性,虽然在代码层面上没有任何高深之处,但让其看起来更合理. (类似于属性的方法)class Person: ...
- 树状数组 || 线段树 || Luogu P5200 [USACO19JAN]Sleepy Cow Sorting
题面:P5200 [USACO19JAN]Sleepy Cow Sorting 题解: 最小操作次数(记为k)即为将序列倒着找第一个P[i]>P[i+1]的下标,然后将序列分成三部分:前缀部分( ...
- css学习_css浮动
1.文档流介绍 网页布局的核心就是利用css来摆放盒子, 把盒子摆放在合适的位置. css的定位机制有以下3种(网页布局一般需要3种搭配使用): a.普通流(标准流) b.浮动 1.浮动只有左右. 2 ...
- 线程安全 Thread Safety Problem scala concurrency 并发
小结: 1.基于java并发模型 Scala concurrency is built on top of the Java concurrency model. 2. 将每个请求放入一个新的线程 T ...
- Java发送邮件功能
package com.hd.all.test.testjava; import java.util.Properties; import javax.mail.Address; import jav ...