GTID环境中手动修复主从故障一例(Error 1236/Error 1396)
root@localhost:mysql3306.sock [(none)]>alter user zlm@'192.168.56.%' identified by 'zlmzlm';
Query OK, rows affected (0.00 sec)
#mysql -uroot -p -S /var/lib/mysql/mysql.sock
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is
Server version: 5.7.-log MySQL Community Server (GPL) Copyright (c) , , Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. root@localhost:mysql.sock [(none)]>start slave;
Query OK, rows affected (0.00 sec) root@localhost:mysql.sock [(none)]>show slave status\G
*************************** . row ***************************
Slave_IO_State:
Master_Host: 192.168.56.100
Master_User: repl
Master_Port:
Connect_Retry:
Master_Log_File: mysql-bin.
Read_Master_Log_Pos:
Relay_Log_File: relay-bin.
Relay_Log_Pos:
Relay_Master_Log_File: mysql-bin.
Slave_IO_Running: No //The IO_Thread is not running on account of Error 1236
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno:
Last_Error:
Skip_Counter:
Exec_Master_Log_Pos:
Relay_Log_Space:
Until_Condition: None
Until_Log_File:
Until_Log_Pos:
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno:
Last_IO_Error: Got fatal error from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'
Last_SQL_Errno:
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id:
Master_UUID: 2a4b3562-2ab6-11e8-be7a-080027de0e0e
Master_Info_File: /data/mysql/mysql3306/data/master.info
SQL_Delay:
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count:
Master_Bind:
Last_IO_Error_Timestamp: ::
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 27af30ca--11e8-ad7e-080027de0e0e:-,
2a4b3562-2ab6-11e8-be7a-080027de0e0e:- //This is the id of maximum transaction executed on slave.
Auto_Position:
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
row in set (0.00 sec)
root@localhost:mysql.sock [(none)]>reset master;
Query OK, rows affected (0.05 sec) root@localhost:mysql.sock [(none)]>show slave status\G
*************************** . row ***************************
Slave_IO_State:
Master_Host: 192.168.56.100
Master_User: repl
Master_Port:
Connect_Retry:
Master_Log_File: mysql-bin.
Read_Master_Log_Pos:
Relay_Log_File: relay-bin.
Relay_Log_Pos:
Relay_Master_Log_File: mysql-bin.
Slave_IO_Running: No
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno:
Last_Error:
Skip_Counter:
Exec_Master_Log_Pos:
Relay_Log_Space:
Until_Condition: None
Until_Log_File:
Until_Log_Pos:
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno:
Last_IO_Error: Got fatal error from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'
Last_SQL_Errno:
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id:
Master_UUID: 2a4b3562-2ab6-11e8-be7a-080027de0e0e
Master_Info_File: /data/mysql/mysql3306/data/master.info
SQL_Delay:
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count:
Master_Bind:
Last_IO_Error_Timestamp: ::
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: //The value has been cleared after operation of "reset master;".
Auto_Position:
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
row in set (0.00 sec)
zlm@192.168.56.100: [(none)]>show master status;
+------------------+----------+--------------+------------------+-------------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------------------------------------+
| mysql-bin. | | | | 2a4b3562-2ab6-11e8-be7a-080027de0e0e:- |
+------------------+----------+--------------+------------------+-------------------------------------------------+
row in set (0.00 sec) zlm@192.168.56.100: [(none)]>flush logs;
Query OK, rows affected (0.04 sec) zlm@192.168.56.100: [(none)]>show master status;
+------------------+----------+--------------+------------------+-------------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------------------------------------+
| mysql-bin. | | | | 2a4b3562-2ab6-11e8-be7a-080027de0e0e:- |
+------------------+----------+--------------+------------------+-------------------------------------------------+
row in set (0.00 sec) [root@zlm1 :: /data/mysql/mysql3306/logs]
#mysqlbinlog -vv --base64-output=decode-rows mysql-bin. > /root/.log
[root@zlm1 :: /data/mysql/mysql3306/logs]
#cd [root@zlm1 :: ~]
#ls -l|grep .log
-rw-r--r-- root root Jul : .log
[root@zlm1 :: ~]
#less .log
... /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0x6e71f8f2 Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x074cfb08 Previous-GTIDs
# 2a4b3562-2ab6-11e8-be7a-080027de0e0e:-
# at
# :: server id end_log_pos CRC32 0x9125f364 GTID last_committed= sequence_number= rbr_only=no
SET @@SESSION.GTID_NEXT= '2a4b3562-2ab6-11e8-be7a-080027de0e0e:12715627'/*!*/;
# at
# :: server id end_log_pos CRC32 0x7a5353bf Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
SET @@session.pseudo_thread_id=/*!*/;
SET @@session.foreign_key_checks=, @@session.sql_auto_is_null=, @@session.unique_checks=, @@session.autocommit=/*!*/;
SET @@session.sql_mode=/*!*/;
SET @@session.auto_increment_increment=, @@session.auto_increment_offset=/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=,@@session.collation_connection=,@@session.collation_server=/*!*/;
SET @@session.lc_time_names=/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
ALTER USER 'zlm'@'192.168.56.%' IDENTIFIED WITH 'mysql_native_password' AS '*512FB3FAA6F522E351929CAC70AD8EBB0F5901B6'
/*!*/;
# at
# :: server id end_log_pos CRC32 0x0882ea24 Rotate to mysql-bin. pos:
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file //I've known that the operation of altering password is in the uuid:12715627.Therefore,I could set gtid_purged to the previous gtid(uuid:5-12715626).
root@localhost:mysql.sock [(none)]>set @@global.gtid_purged='2a4b3562-2ab6-11e8-be7a-080027de0e0e:5-12715626';
Query OK, rows affected (0.00 sec) root@localhost:mysql.sock [(none)]>start slave io_thread;
Query OK, rows affected (0.01 sec) root@localhost:mysql.sock [(none)]>show slave status\G
*************************** . row ***************************
Slave_IO_State:
Master_Host: 192.168.56.100
Master_User: repl
Master_Port:
Connect_Retry:
Master_Log_File: mysql-bin.
Read_Master_Log_Pos:
Relay_Log_File: relay-bin.
Relay_Log_Pos:
Relay_Master_Log_File: mysql-bin.
Slave_IO_Running: No //It was not running again.
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno:
Last_Error:
Skip_Counter:
Exec_Master_Log_Pos:
Relay_Log_Space:
Until_Condition: None
Until_Log_File:
Until_Log_Pos:
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno:
Last_IO_Error: Got fatal error from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'
Last_SQL_Errno:
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id:
Master_UUID: 2a4b3562-2ab6-11e8-be7a-080027de0e0e
Master_Info_File: /data/mysql/mysql3306/data/master.info
SQL_Delay:
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count:
Master_Bind:
Last_IO_Error_Timestamp: ::
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 2a4b3562-2ab6-11e8-be7a-080027de0e0e:-
Auto_Position:
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
row in set (0.00 sec) root@localhost:mysql.sock [(none)]>reset master;
Query OK, rows affected (0.03 sec) root@localhost:mysql.sock [(none)]>set @@global.gtid_purged='2a4b3562-2ab6-11e8-be7a-080027de0e0e:1-12715626'; //I began with the first gtid this time.
Query OK, rows affected (0.00 sec) root@localhost:mysql.sock [(none)]>stop slave;
Query OK, rows affected (0.00 sec) root@localhost:mysql.sock [(none)]>start slave;
Query OK, rows affected (0.00 sec) root@localhost:mysql.sock [(none)]>show slave status\G
*************************** . row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.56.100
Master_User: repl
Master_Port:
Connect_Retry:
Master_Log_File: mysql-bin.
Read_Master_Log_Pos:
Relay_Log_File: relay-bin.
Relay_Log_Pos:
Relay_Master_Log_File: mysql-bin.
Slave_IO_Running: Yes //IO_Thread could work normally.
Slave_SQL_Running: No //SQL_Thread was down instead.
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: //A new arrival error 1396 occurred.
Last_Error: Error 'Operation ALTER USER failed for 'zlm'@'192.168..%'' on query. Default database: ''. Query: 'ALTER USER 'zlm'@'192.168..%' IDENTIFIED WITH 'mysql_native_password' AS '*512FB3FAA6F522E351929CAC70AD8EBB0F5901B6''
Skip_Counter:
Exec_Master_Log_Pos:
Relay_Log_Space:
Until_Condition: None
Until_Log_File:
Until_Log_Pos:
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno:
Last_IO_Error:
Last_SQL_Errno:
Last_SQL_Error: Error 'Operation ALTER USER failed for 'zlm'@'192.168..%'' on query. Default database: ''. Query: 'ALTER USER 'zlm'@'192.168..%' IDENTIFIED WITH 'mysql_native_password' AS '*512FB3FAA6F522E351929CAC70AD8EBB0F5901B6''
Replicate_Ignore_Server_Ids:
Master_Server_Id:
Master_UUID: 2a4b3562-2ab6-11e8-be7a-080027de0e0e
Master_Info_File: /data/mysql/mysql3306/data/master.info
SQL_Delay:
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count:
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp: ::
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 2a4b3562-2ab6-11e8-be7a-080027de0e0e:
Executed_Gtid_Set: 2a4b3562-2ab6-11e8-be7a-080027de0e0e:-
Auto_Position:
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
row in set (0.00 sec) root@localhost:mysql.sock [(none)]>ALTER USER 'zlm'@'192.168.56.%' IDENTIFIED WITH 'mysql_native_password' AS '*512FB3FAA6F522E351929CAC70AD8EBB0F5901B6'; //I couldn't executed it in mysql cli either.
ERROR (HY000): Operation ALTER USER failed for 'zlm'@'192.168.56.%'
root@localhost:mysql.sock [(none)]>
//Master
root@localhost:mysql3306.sock [(none)]>select user,host from mysql.user;
+---------------+--------------+
| user | host |
+---------------+--------------+
| repl | 192.168..% |
| zlm | 192.168..% |
| bkuser | localhost |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+---------------+--------------+
rows in set (0.00 sec) //Slave
root@localhost:mysql.sock [(none)]>select user,host from mysql.user;
+---------------+--------------+
| user | host |
+---------------+--------------+
| repl | 192.168..% |
| bkuser | localhost |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+---------------+--------------+
rows in set (0.00 sec) root@localhost:mysql.sock [(none)]>create user zlm@'192.168.56.%'; //I created the absent user "zlm" on slave.
Query OK, rows affected (0.00 sec) root@localhost:mysql.sock [(none)]>start slave sql_thread;
Query OK, rows affected (0.01 sec) root@localhost:mysql.sock [(none)]>show slave status\G
*************************** . row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.56.100
Master_User: repl
Master_Port:
Connect_Retry:
Master_Log_File: mysql-bin.
Read_Master_Log_Pos:
Relay_Log_File: relay-bin.
Relay_Log_Pos:
Relay_Master_Log_File: mysql-bin.
Slave_IO_Running: Yes
Slave_SQL_Running: Yes //It was ok.
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno:
Last_Error:
Skip_Counter:
Exec_Master_Log_Pos:
Relay_Log_Space:
Until_Condition: None
Until_Log_File:
Until_Log_Pos:
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master:
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno:
Last_IO_Error:
Last_SQL_Errno:
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id:
Master_UUID: 2a4b3562-2ab6-11e8-be7a-080027de0e0e
Master_Info_File: /data/mysql/mysql3306/data/master.info
SQL_Delay:
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count:
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 2a4b3562-2ab6-11e8-be7a-080027de0e0e:
Executed_Gtid_Set: 27af30ca--11e8-ad7e-080027de0e0e:, //This gtid was generated by the statment of "create user ... ".
2a4b3562-2ab6-11e8-be7a-080027de0e0e:- //The executed gtid was equal with the master.
Auto_Position:
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
row in set (0.00 sec)
- Error 1236 is common phenomenon in our replication of MySQL which always due to lack of specific binlog files.
- Before you execute a statement on master.You'd better start your slave replication at first.Obviously it means our test environment.
- If you were not sure which transaction was stuck.Analyze the binlog file is always a good idea to solve these kind of issues.
- In my case,I know clearly about what I've operated in master.So I can easily execute the same operation on slave.But the product environment is complecated and it's not recommended to do so.
GTID环境中手动修复主从故障一例(Error 1236/Error 1396)的更多相关文章
- GTID环境中手动修复主从故障一例(Error 1146)
Preface In my last test of pt-heartbeat,both of master and slave were out of disk.And the my ...
- SqlServer 禁止架构更改的复制中手动修复使发布和订阅中分别增加的字段同步
原文:SqlServer 禁止架构更改的复制中手动修复使发布和订阅中分别增加的字段同步 由于之前的需要,禁止了复制架构更改,以至在发布中添加一个字段,并不会同步到订阅中,而现在又在订阅中添加了一个同名 ...
- 企业运维 | MySQL关系型数据库在Docker与Kubernetes容器环境中快速搭建部署主从实践
[点击 关注「 WeiyiGeek」公众号 ] 设为「️ 星标」每天带你玩转网络安全运维.应用开发.物联网IOT学习! 希望各位看友[关注.点赞.评论.收藏.投币],助力每一个梦想. 本章目录 目录 ...
- 故障案例 | 主从复制环境中tokudb引擎报错排查过程
欢迎来到 GreatSQL社区分享的MySQL技术文章,如有疑问或想学习的内容,可以在下方评论区留言,看到后会进行解答 GreatSQL社区原创内容未经授权不得随意使用,转载请联系小编并注明来源. 0 ...
- 业务零影响!如何在Online环境中巧用MySQL传统复制技术【转】
业务零影响!如何在Online环境中巧用MySQL传统复制技术 这篇文章我并不会介绍如何部署一个MySQL复制环境或keepalived+双主环境,因为此类安装搭建的文章已经很多,大家也很熟悉.在这篇 ...
- .NET 环境中使用RabbitMQ RabbitMQ与Redis队列对比 RabbitMQ入门与使用篇
.NET 环境中使用RabbitMQ 在企业应用系统领域,会面对不同系统之间的通信.集成与整合,尤其当面临异构系统时,这种分布式的调用与通信变得越发重要.其次,系统中一般会有很多对实时性要求不高的 ...
- 5.7 并行复制配置 基于GTID 搭建中从 基于GTID的备份与恢复,同步中断处理
5.7 并行复制配置 基于GTID 搭建中从 基于GTID的备份与恢复,同步中断处理 这个文章包含三个部分 1:gtid的多线程复制2:同步中断处理3:GTID的备份与恢复 下面文字相关的东西 大部分 ...
- 生产环境中的kubernetes 优先级与抢占
kubernetes 中的抢占功能是调度器比较重要的feature,但是真正使用起来还是比较危险,否则很容易把低优先级的pod给无辜kill.为了提高GPU集群的资源利用率,决定勇于尝试一番该feat ...
- Redis 哨兵模式实现主从故障互切换
200 ? "200px" : this.width)!important;} --> 介绍 Redis Sentinel 是一个分布式系统, 你可以在一个架构中运行多个 S ...
随机推荐
- 332. Reconstruct Itinerary (leetcode)
1. build the graph and then dfs -- graph <String, List<String>>, (the value is sorted a ...
- 【转载】#346 - Polymorphism
Recall that polymorphism is one of the three core principles of object-oriented programming. Polymor ...
- npy数据的保存与读取
保存 利用这种方法,保存文件的后缀名字一定会被置为.npy x = numpy.save("data_x.npy",x) 读取 data = numpy.load("da ...
- git中Bash基本操作命令
).cd : 改变目录. ).cd . . 回退到上一个目录,直接cd进入默认目录 ).pwd : 显示当前所在的目录路径. ).ls(ll): 都是列出当前目录中的所有文件,只不过ll(两个ll)列 ...
- 旧文备份:rtlinux安装手册
前段时间接触了几天RTLinux,折腾了好几天才总算把它安装上,得益于Prof. Chang-Gun Lee的安装建议,觉得该文档可能会对准备尝试安装RTLinux的朋友们有帮助,本人英语很烂,也比较 ...
- caffe在 14.04安装
同事安装遇到的问题,记录一下 需要把cuda里面带的opengl不安装才行,否则冲突.在安装时,首先和之前一样,切换到无图形界面,关掉lightdm,安装cuda时选择--no-opengl-lib, ...
- 瓣呀,一个基于豆瓣api仿网易云音乐的开源项目
整体采用material design 风格,本人是网易云音乐的粉丝,所以界面模仿了网页云音乐,另外,项目中尽量使用了5.0之后的新控件. 项目整体采用mvp+rxjava+retrofit 框架,使 ...
- v-if
vue中通过v-if,v-else-if,v-else的对应的Boolean值来操作元素在dom中是否移除. 这里就以单纯的true,false来模拟一下.注:标签属性去出来的值为string类型. ...
- SAP标准导出功能 - 删除默认选定格式
我们经常会使用SAP系统的标准功能导出ALV显示的数据,一般会选择电子表格. 选择电子表格之后,需要选择电子表格的具体格式. 选择格式之后点击确定,会弹出保存对话框. 如果在使用这个功能的时候,选择了 ...
- OB如何创建租户
一. 先导知识: 资源隔离是保证用户间相互不受影响的重要手段.数据库的资源隔离主要有以下方式: l 服务器隔离 l 数据库隔离:sqlserver.oceanbase.oracle ...