MySQL-Utilities：mysqldbcompare及跳过复制错误

mysqldbcompare也是MySQL-Utilities工具集的一个脚本。mysqldbcompare从两个数据库比较对象和数据的不同。数据库中的对象包括：表、视图、触发器、存储过程、函数和事件。每一个对象类型计数可以使用-vv选项显示。通过一系列步骤检查进行测试，默认情况下，一旦测试失败就终止检测。可以指定--run-all-tests选项来进行所有的测试。
环境：CentOS6.5源码安装多个MySQL实例及复制搭建，之前做复制时Master实例的test库中的数据表没有复制到Slave实例

#使用Master中的test.test1的建表语句创建Slave中的test.test1，修改列c的说明，并插入数据

use test;

create table test1

(id int not null primary key,

a varchar(10) not null,

b varchar(10),

c varchar(10) comment 'cc',

d int

)

ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='test1';

insert into test1 values(1,'a','b','c',1);

此时两表的定义和数据是不一致的。
比较检测的步骤
1、数据库定义的检查检查对比的数据库是否存在

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test1

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

ERROR: The database test1 does not exist.

2、检测数据库的对象检查两者数据库中的对象是否丢失。可以使用--skip-object-compare选项跳过这步。

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

ERROR: The list of objects differs among database test and test.

3、比较对象的定义对对象的定义（CREATE语句）进行比较和显示不同。可以使用--skip-diff选项跳过这步。

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test --skip-object-compare

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# TABLE     test1                                   FAIL    ERROR: The object definitions do not match.

4、检测表的行数检查表是否有相同的行数，但这并不确保表的数据是一致性的。可以使用--skip-row-count选项跳过这步。

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test --skip-object-compare --skip-diff

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# TABLE     test1                                   SKIP    FAIL    ERROR: Row counts are not the same among `test`.`test1` and `test`.`test1`.

5、检测表数据的一致性检查行数同时检查数据是否一致，两表需要有主键或唯一索引。可以使用--skip-checksum-table选项跳过表校验，使用--skip-data-check选项跳过数据检查。

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test --skip-object-compare --skip-diff --skip-row-count

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# TABLE     test1                                   SKIP    SKIP    -

#           - Compare table checksum                                FAIL

#           - Find row differences                                  FAIL

#

# Transformation for --changes-for=server2:

#

DELETE FROM `test`.`test1` WHERE `id` = '';

#

# Transformation for reverse changes (--changes-for=server1):

#

# INSERT INTO `test`.`test1` (`id`, `a`, `b`, `c`, `d`) VALUES('', 'a', 'b', 'c', '');

#

# Database consistency check failed.

#

# ...done

#使用--skip-checksum-table选项跳过表校验

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test --skip-object-compare --skip-diff --skip-row-count --skip-checksum-table

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# TABLE     test1                                   SKIP    SKIP    -

#           - Compare table checksum                                SKIP

#           - Find row differences                                  FAIL

#

# Transformation for --changes-for=server2:

#

DELETE FROM `test`.`test1` WHERE `id` = '';

#

# Transformation for reverse changes (--changes-for=server1):

#

# INSERT INTO `test`.`test1` (`id`, `a`, `b`, `c`, `d`) VALUES('', 'a', 'b', 'c', '');

#

# Database consistency check failed.

#

# ...done

#使用--skip-data-check选项跳过数据检查

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test --skip-object-compare --skip-diff --skip-row-count --skip-checksum-table --skip-data-check

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# TABLE     test1                                   SKIP    SKIP    SKIP   

# Databases are consistent given skip options specified.

#

# ...done

指定--run-all-tests选项来进行所有的检测

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql test:test --run-all-tests

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases test on server1 and test on server2

#

# WARNING: Objects in server1.test but not in server2.test:

#        TABLE: test2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# TABLE     test1                                   FAIL    FAIL    -

#           - Compare table checksum                                FAIL

#           - Find row differences                                  FAIL

#

# Transformation for --changes-for=server2:

#

ALTER TABLE `test`.`test1`

  CHANGE COLUMN c c varchar() NULL COMMENT 'c';

#

# Transformation for reverse changes (--changes-for=server1):

#

# ALTER TABLE `test`.`test1`

#   CHANGE COLUMN c c varchar() NULL COMMENT 'cc';

#

# Row counts are not the same among `test`.`test1` and `test`.`test1`.

#

# Transformation for --changes-for=server2:

#

DELETE FROM `test`.`test1` WHERE `id` = '';

#

# Transformation for reverse changes (--changes-for=server1):

#

# INSERT INTO `test`.`test1` (`id`, `a`, `b`, `c`, `d`) VALUES('', 'a', 'b', 'c', '');

#

# Database consistency check failed.

#

# ...done

检测做复制的sakila库

[root@VMUest ~]# mysqldbcompare --server1=mydba:mysql5635@192.168.85.129: --server2=mydba:mysql5635@192.168.85.129: --changes-for=server2 --show-reverse --difftype=sql sakila:sakila

# WARNING: Using a password on the command line interface can be insecure.

# server1 on 192.168.85.129: ... connected.

# server2 on 192.168.85.129: ... connected.

# Checking databases sakila on server1 and sakila on server2

#

#                                                   Defn    Row     Data

# Type      Object Name                             Diff    Count   Check

# -------------------------------------------------------------------------

# FUNCTION  get_customer_balance                    pass    -       -

# FUNCTION  inventory_held_by_customer              pass    -       -

# FUNCTION  inventory_in_stock                      pass    -       -

# PROCEDURE film_in_stock                           pass    -       -

# PROCEDURE film_not_in_stock                       pass    -       -

# PROCEDURE rewards_report                          pass    -       -

# TABLE     actor                                   pass    pass    -

#           - Compare table checksum                                pass

# TABLE     address                                 pass    pass    -

#           - Compare table checksum                                pass

# TABLE     category                                pass    pass    -

#           - Compare table checksum                                pass

# TABLE     city                                    pass    pass    -

#           - Compare table checksum                                pass

# TABLE     country                                 pass    pass    -

#           - Compare table checksum                                pass

# TABLE     customer                                pass    pass    -

#           - Compare table checksum                                pass

# TABLE     film                                    pass    pass    -

#           - Compare table checksum                                pass

# TABLE     film_actor                              pass    pass    -

#           - Compare table checksum                                pass

# TABLE     film_category                           pass    pass    -

#           - Compare table checksum                                pass

# TABLE     film_text                               pass    pass    -

#           - Compare table checksum                                pass

# TABLE     inventory                               pass    pass    -

#           - Compare table checksum                                pass

# TABLE     language                                pass    pass    -

#           - Compare table checksum                                pass

# TABLE     payment                                 pass    pass    -

#           - Compare table checksum                                pass

# TABLE     rental                                  pass    pass    -

#           - Compare table checksum                                pass

# TABLE     staff                                   pass    pass    -

#           - Compare table checksum                                pass

# TABLE     store                                   pass    pass    -

#           - Compare table checksum                                pass

# TRIGGER   customer_create_date                    pass    -       -

# TRIGGER   del_film                                pass    -       -

# TRIGGER   ins_film                                pass    -       -

# TRIGGER   payment_date                            pass    -       -

# TRIGGER   rental_date                             pass    -       -

# TRIGGER   upd_film                                pass    -       -

# VIEW      actor_info                              pass    -       -

# VIEW      customer_list                           pass    -       -

# VIEW      film_list                               pass    -       -

# VIEW      nicer_but_slower_film_list              pass    -       -

# VIEW      sales_by_film_category                  pass    -       -

# VIEW      sales_by_store                          pass    -       -

# VIEW      staff_list                              pass    -       -      

# Databases are consistent.

#

# ...done

感觉mysqldbcompare的--run-all-tests选项包括了mysqldiff。mysqldiff如果指定数据库对(db1:db2)，将对比数据库下的对象(不会检查对象的定义)；如果指定具体对象(db1.obj1:db2.obj2)，将对比其定义。
跳过复制错误
在前面对比数据的第5步做Find row differences操作时，会将下面语句写到二进制日志(实际是对比的双方都会有这样的操作，可开启log_bin查看)

现在已经知道主从的test库不一致，下面测试如何跳过复制错误。在主库删除表test2

mysql> drop table test2;

Query OK, 0 rows affected

mysql> SHOW BINLOG EVENTS in 'mysql-bin.000002' from 16061;

+------------------+-------+------------+-----------+-------------+----------------------------------------------------------+

| Log_name         | Pos   | Event_type | Server_id | End_log_pos | Info                                                     |

+------------------+-------+------------+-----------+-------------+----------------------------------------------------------+

| mysql-bin.000002 | 16061 | Query      |         6 |       16179 | use `test`; DROP TABLE `test2` /* generated by server */ |

+------------------+-------+------------+-----------+-------------+----------------------------------------------------------+

1 row in set

在从库查看复制状态

mysql> show slave status\G

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: 127.0.0.1

                  Master_User: repl

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000002

          Read_Master_Log_Pos: 16179

               Relay_Log_File: mysql-relay-bin.000005

                Relay_Log_Pos: 16224

        Relay_Master_Log_File: mysql-bin.000002

             Slave_IO_Running: Yes

            Slave_SQL_Running: No

              Replicate_Do_DB:

          Replicate_Ignore_DB:

           Replicate_Do_Table:

       Replicate_Ignore_Table:

      Replicate_Wild_Do_Table:

  Replicate_Wild_Ignore_Table:

                   Last_Errno: 1051

                   Last_Error: Error 'Unknown table 'test.test2'' on query. Default database: 'test'. Query: 'DROP TABLE `test2` /* generated by server */'

                 Skip_Counter: 0

          Exec_Master_Log_Pos: 16061

              Relay_Log_Space: 16678

              Until_Condition: None

               Until_Log_File:

                Until_Log_Pos: 0

           Master_SSL_Allowed: No

           Master_SSL_CA_File:

           Master_SSL_CA_Path:

              Master_SSL_Cert:

            Master_SSL_Cipher:

               Master_SSL_Key:

        Seconds_Behind_Master: NULL

Master_SSL_Verify_Server_Cert: No

                Last_IO_Errno: 0

                Last_IO_Error:

               Last_SQL_Errno: 1051

               Last_SQL_Error: Error 'Unknown table 'test.test2'' on query. Default database: 'test'. Query: 'DROP TABLE `test2` /* generated by server */'

  Replicate_Ignore_Server_Ids:

             Master_Server_Id: 6

                  Master_UUID: 02a05b2c-0557-11e7-bb02-000c29493a20

             Master_Info_File: /usr/local/mysql3307/log/master.info

                    SQL_Delay: 0

          SQL_Remaining_Delay: NULL

      Slave_SQL_Running_State:

           Master_Retry_Count: 86400

                  Master_Bind:

      Last_IO_Error_Timestamp:

     Last_SQL_Error_Timestamp: 170315 14:21:55

               Master_SSL_Crl:

           Master_SSL_Crlpath:

           Retrieved_Gtid_Set:

            Executed_Gtid_Set:

                Auto_Position: 0

1 row in set (0.00 sec)

报错Unknown table，使用下面语句跳过一个事务

mysql> stop slave;

mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;

mysql> start slave;

重新查看复制状态

mysql> show slave status\G

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: 127.0.0.1

                  Master_User: repl

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000002

          Read_Master_Log_Pos: 16179

               Relay_Log_File: mysql-relay-bin.000006

                Relay_Log_Pos: 283

        Relay_Master_Log_File: mysql-bin.000002

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

              Replicate_Do_DB:

          Replicate_Ignore_DB:

           Replicate_Do_Table:

       Replicate_Ignore_Table:

      Replicate_Wild_Do_Table:

  Replicate_Wild_Ignore_Table:

                   Last_Errno: 0

                   Last_Error:

                 Skip_Counter: 0

          Exec_Master_Log_Pos: 16179

              Relay_Log_Space: 16678

              Until_Condition: None

               Until_Log_File:

                Until_Log_Pos: 0

           Master_SSL_Allowed: No

           Master_SSL_CA_File:

           Master_SSL_CA_Path:

              Master_SSL_Cert:

            Master_SSL_Cipher:

               Master_SSL_Key:

        Seconds_Behind_Master: 0

Master_SSL_Verify_Server_Cert: No

                Last_IO_Errno: 0

                Last_IO_Error:

               Last_SQL_Errno: 0

               Last_SQL_Error:

  Replicate_Ignore_Server_Ids:

             Master_Server_Id: 6

                  Master_UUID: 02a05b2c-0557-11e7-bb02-000c29493a20

             Master_Info_File: /usr/local/mysql3307/log/master.info

                    SQL_Delay: 0

          SQL_Remaining_Delay: NULL

      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it

           Master_Retry_Count: 86400

                  Master_Bind:

      Last_IO_Error_Timestamp:

     Last_SQL_Error_Timestamp:

               Master_SSL_Crl:

           Master_SSL_Crlpath:

           Retrieved_Gtid_Set:

            Executed_Gtid_Set:

                Auto_Position: 0

1 row in set (0.00 sec)

对于这种需求明确的通过跳过出错事务就能将主从保持一致。

MySQL-Utilities：mysqldbcompare及跳过复制错误的更多相关文章

mysql主从复制跳过复制错误【转】
跳过复制错误 mysql因为binlog机制问题,有些时候会出现从库重放sql执行失败的情况,特别是旧的STATEMENT模式最容易出现这种情况(因为函数和存储过程等原因),这也是为什么强调使用mix ...
跳过复制错误——sql_slave_skip_counter
昨天不少同学讨论<小心,前方有雷 —— sql_slave_skip_counter>,有说作者在玩文字游戏,扯了那么多sql_slave_skip_counter=1不还是跳过一个事务嘛 ...
跳过复制错误——slave_skip_errors、slave_exec_mode
这一篇写写复制错误处理相关的另两个参数slave_skip_errors.slave_exec_mode,基本环境参考<复制错误处理——sql_slave_skip_counter> 一. ...
MySQL Replication--跳过复制错误
在MySQL中,有两种跳过复制错误的方法:1.对于未使用GTID的复制,可以使用sql_slave_skip_counter来跳过错误2.对于使用GTID的复制,可以使用GTID_NEXT模拟空事务来 ...
MySQL utilities介绍&出现 No module named utilities
目录安装 mysqlreplicate mysqlrplcheck mysqlrplshow mysqlrpladmin mysqlfailover mysqldbcompare 详细介绍 mysq ...
MySQL Utilities管理工具
前提: 1.安装MySQL Utilities工具 2.复制my_print_defaults命令至/usr/bin下或写入环境变量. 卸载方式: python ./setup.py clean -- ...
（5.6）mysql高可用系列——MySQL Utilities 管理工具
关键词:mysql工具集,mysql管理工具,mysql utilities [1]安装mysql utilities cd /download wget https://cdn.mysql.com/ ...
mysql 5.6在gtid复制模式下复制错误，如何跳过？？
mysql 5.6在gtid复制模式下复制错误,如何跳过?? http://www.xuchanggang.cn/archives/918.html
replicate-do-db参数引起的MySQL复制错误及处理办法
replicate-do-db配置在MySQL从库的my.cnf文件中,可以指定只复制哪个库的数据.但是这个参数有个问题就是主库如果在其他的schema环境下操作,其binlog不会被从库应用,从而出 ...

随机推荐

02->交互式图形学--用glut库实现Sierpinski镂垫
Sierpinski图形是一类有趣的图形,本文结合交互式图形学教材,讲解Sierpinski镂垫图形生成,作为入门图形学编程的第一课.为了简便,本文就2D来进行讨论.实验环境见01 算法主要是根据随 ...
java 冒泡排序法、选择排序
1.冒泡排序 /* * 冒泡排序 * 外层控制循环多少趟,内层控制每一趟的循环次数 */ public class Test08 { public static void main(String[] ...
python的update方法
b = {"c":0, "position":{}} b["position"]["IF"] = {} print(b) ...
同步锁与GIL的关系
#_author:来童星#date:2019/12/2# Python的线程在GIL的控制之下,线程之间,对整个python解释器,对python提供的CAPI的访问都是互斥的,# 这可以看作是Pyt ...
学习Caffe（一）使用Caffe
如何使用Caffe Caffe教程(http://robots.princeton.edu/courses/COS598/2015sp/slides/Caffe/caffe_tutorial.pdf) ...
npm run server报错
从git上clone的vue项目npm install后npm run server报错 $ npm run dev > lufei@1.0.0 dev E:\pythonProject\luf ...
SpringBoot_03_SpringBoot对其他技术的整合
1.SpringBoot整合Mybatis 1.2 添加Mybatis的起步依赖  <dependency> <groupId> ...
尚学linux课程---7、linux系统管理命令
尚学linux课程---7.linux系统管理命令一.总结一句话总结: 查网络:netstat -ntpl 查进程:ps 1.需要下载163yum源(从外部源同步仓库)里面的所有rpm文件? re ...
http://edu.manew.com/ ,蛮牛教育（很少免费），主要是unty3D和大数据方向。适合扫盲
http://edu.manew.com/ ,蛮牛教育(很少免费),主要是unty3D和大数据方向.
JS 基本的介绍
JS中的注释 HTML的注释:<!—注释内容--> CSS注释:/* 注释 */ JavaScript的注释:// 或 /* 多行注释 */ 变量 1.变量的概念变量是变化 ...

MySQL-Utilities：mysqldbcompare及跳过复制错误

MySQL-Utilities：mysqldbcompare及跳过复制错误的更多相关文章

随机推荐

热门专题