一. MySQL 5.6引入了GTID的概念,那么GTID是何方神圣?其实也不复杂,就是一个全局事务标示符。使用GTID时,每次事务提交都会在binlog里生成1个唯一的标示符,它由UUID和事务ID组成。首次提交的事务ID为1,第二次为2,第三次为3,以此例推。uuid是服务器的身份ID,在第一次启动MySQL时,会自动生成一个server_uuid, 并且默认写入到数据目录下的auto.cnf文件里。我们一般无需修改,官方也不建议修改。更为详细的可以参考MariaDB官方文档

https://mariadb.com/kb/en/mariadb/mariadb-documentation/replication-cluster-multi-master/replication/parallel-replication/

[root@mysql-server- data]# pwd
/root/sandboxes/msb_5_6_19/data
[root@mysql-server- data]# cat auto.cnf
[auto]
server-uuid=9b0147c3-eed0-11e3--000c29e3621e
[root@mysql-server- data]#

那么基于GTID的Replication有啥好处?好处主要有以下2点:

(1)在传统的复制里面,当发生故障,需要主从切换,需要找到binlog和pos点,然后change master to指向新的master,相对来说比较麻烦,也容易出错。在MySQL 5.6里面,不用再找binlog和pos点,我们只需要知道master的ip,端口,以及账号密码就行,因为复制是自动的,MySQL会通过内部机制GTID自动找点同步。

(2)多线程复制(基于库)。在MySQL 5.6以前的版本,slave的复制是单线程的。一个事件一个事件的读取应用。而master是并发写入的,所以延时是避免不了的。唯一有效的方法是把多个库放在多台slave,这样又有点浪费服务器。在MySQL 5.6里面,我们可以把多个表放在多个库,这样就可以使用多线程复制,当只有1个库,多线程复制是没有用的。

GTID相关特性默认是关闭的(难道官方还觉得不够成熟),如下:

mysql [localhost] {msandbox} ((none)) > show variables like '%gtid%';
+--------------------------+-----------+
| Variable_name | Value |
+--------------------------+-----------+
| enforce_gtid_consistency | OFF |
| gtid_executed | |
| gtid_mode | OFF |
| gtid_next | AUTOMATIC |
| gtid_owned | |
| gtid_purged | |
+--------------------------+-----------+
rows in set (0.01 sec) mysql [localhost] {msandbox} ((none)) >

binlog里面也不会有GTID相关的记录,和普通复制时是一样的,如下:

[root@mysql-server- data]# mysqlbinlog --no-defaults -v --base64-output=DECODE-ROWS mysql_sandbox5619-bin.
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos Start: binlog v , server v 5.6.-log created :: at startup
# Warning: this binlog is either in use or was not closed properly.
ROLLBACK/*!*/;
# at
# :: server id end_log_pos Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
SET @@session.pseudo_thread_id=/*!*/;
SET @@session.foreign_key_checks=, @@session.sql_auto_is_null=, @@session.unique_checks=, @@session.autocommit=/*!*/;
SET @@session.sql_mode=/*!*/;
SET @@session.auto_increment_increment=, @@session.auto_increment_offset=/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=,@@session.collation_connection=,@@session.collation_server=/*!*/;
SET @@session.lc_time_names=/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
drop database yayunz
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@mysql-server- data]#

当开启GTID特性以后,binlog记录格式是这样的(具体开启后面介绍)

注意:(当启用GTID以后,binlog格式变化很大,如果继续采用低版本的mysqlbinlog命令查看,将会得到如下错误)。

[root@mysql-server- data]# mysqlbinlog --no-defaults -v --base64-output=DECODE-ROWS mysql-bin.
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos Start: binlog v , server v 5.6.-log created ::
ERROR: Error in Log_event::read_log_event(): 'Sanity check failed', data_len: , event_type:
ERROR: Could not read entry at offset : Error in log format or read error.
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@mysql-server- data]#

下面是使用MySQL 5.6的mysqlbinlog命令查看开启了GTID日志的记录。

[root@mysql-server- data]# /data/sandbox_mysql/5.6./bin/mysqlbinlog --no-defaults -v --base64-output=DECODE-ROWS mysql-bin. | tail -n
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xcb49c4fe Start: binlog v , server v 5.6.-log created ::
# at
# :: server id end_log_pos CRC32 0x0e40da64 Previous-GTIDs
# 9b0147c3-eed0-11e3--000c29e3621e:
# at
# :: server id end_log_pos CRC32 0x52fc16ed GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '9b0147c3-eed0-11e3-9821-000c29e3621e:2'/*!*/;
# at
# :: server id end_log_pos CRC32 0xf652f593 Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
SET @@session.pseudo_thread_id=/*!*/;
SET @@session.foreign_key_checks=, @@session.sql_auto_is_null=, @@session.unique_checks=, @@session.autocommit=/*!*/;
SET @@session.sql_mode=/*!*/;
SET @@session.auto_increment_increment=, @@session.auto_increment_offset=/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=,@@session.collation_connection=,@@session.collation_server=/*!*/;
SET @@session.lc_time_names=/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
create database dyy
/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
# at
# :: server id end_log_pos CRC32 0x68ff3fc0 Rotate to mysql-bin. pos:
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@mysql-server- data]#

我们从上面输出可以清楚的看见事务id如下(根据上面提到的,这个是第二个事务):

SET @@SESSION.GTID_NEXT= '9b0147c3-eed0-11e3-9821-000c29e3621e:2'/*!*/;

GTID的简单工作流程如下:

(1)在master上产生一个事务并且提交,并且写入binlog

(2)master上的binlog发送到slave,slave接收完毕并且写入relay log,slave读取到这个GTID,并设置gtid_next的值,例如:

SET @@SESSION.GTID_NEXT= '9b0147c3-eed0-11e3-9821-000c29e3621e:2

然后告诉slave接下来的事务必须使用GTID,并写入到它自己的binlog里。
(3)slave检查并确认这个GTID没有被使用,如果没有被使用,那么开始执行这个事务并写入到它自己的binlog里。

(4)由于gtid_next的值不是空的,slave不会尝试去生成一个新的gtid,而是通过主从复制来获取GTID。

二.  基于GTID的Replication的配置如下(最少配置选项,master和slave一样,server-id则需要不一样了,下面的参数只是针对于GTID,对于复制过滤选项和普通复制一样,因为GTID模式的复制也可以转换为普通模式的复制)

binlog_format = row
gtid-mode = ON
enforce-gtid-consistency = ON
log-bin=mysql-bin
log_slave_updates

少了相关选项,启动则报错,这是其中的报错日志,如下:

-- ::  [ERROR] --gtid-mode=ON or UPGRADE_STEP_1 or UPGRADE_STEP_2 requires --log-bin and --log-slave-updates

其中binlog_format = row不是必须的,但是推荐使用ROW格式,具体的参考我前面的文章提到的原因。

上面参数主从都配置以后,下面我们就来完成一个基于GTID的Replication(和普通复制有少量区别)

(1)在master上授权账户;

mysql [localhost] {root} ((none)) > GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.0.10' IDENTIFIED BY '';
Query OK, rows affected (0.00 sec) mysql [localhost] {root} ((none)) > flush privileges;
Query OK, rows affected (0.00 sec) mysql [localhost] {root} ((none)) >

(2)在master上备份要同步的库(这里的备份参数大家自行查阅文档,当然有几个参数或许用不到)

[root@mysql-server- msb_5_6_19]# mysqldump -uroot -pmsandbox -S /tmp/mysql_sandbox5619.sock -q -R --triggers --opt --single-transaction --flush-logs --master-data= dyy > /tmp/dyy.sql
[root@mysql-server- msb_5_6_19]#

(3)将备份的数据在slave上导入:

[root@mysql-server- ~]# mysql -uroot -pmsandbox -S /tmp/mysql_sandbox5619.sock < dyy.sql
[root@mysql-server- ~]#

(4)进行change master to操作

mysql [localhost] {root} (dyy) > CHANGE MASTER TO MASTER_HOST='192.168.0.100',MASTER_PORT=,MASTER_USER='repl',MASTER_PASSWORD='',master_auto_position=;
Query OK, rows affected, warnings (0.03 sec) mysql [localhost] {root} (dyy) > show warnings;
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note | | Sending passwords in plain text without SSL/TLS is extremely insecure. |
| Note | | Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information. |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
rows in set (0.00 sec) mysql [localhost] {root} (dyy) >

可以看见成功,但是有两个警告,原来在MySQL 5.6里面越来越严格了,说不推荐这种使用方法。

mysql [localhost] {root} (dyy) > start slave;
Query OK, rows affected (0.01 sec) mysql [localhost] {root} (dyy) > show slave status\G
*************************** . row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.0.100
Master_User: repl
Master_Port:
Connect_Retry:
Master_Log_File: mysql-bin.
Read_Master_Log_Pos:
Relay_Log_File: mysql_sandbox5619-relay-bin.
Relay_Log_Pos:
Relay_Master_Log_File: mysql-bin.
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno:
Last_Error:
Skip_Counter:
Exec_Master_Log_Pos:
Relay_Log_Space:
Until_Condition: None
Until_Log_File:
Until_Log_Pos:
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master:
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno:
Last_IO_Error:
Last_SQL_Errno:
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id:
Master_UUID: 9b0147c3-eed0-11e3--000c29e3621e
Master_Info_File: /root/sandboxes/msb_5_6_19/data/master.info
SQL_Delay:
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Master_Retry_Count:
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 9b0147c3-eed0-11e3--000c29e3621e:-,
d3c1aada-fad2-11e3-a66f-000c29ce3f18:-
Auto_Position:
row in set (0.00 sec) mysql [localhost] {root} (dyy) >

如果使用GTID模式的复制而change master to还使用普通模式的,则会报错。 

三. GTID的局限性(或者说还不够完善的地方)

(1)目前GTID还不支持MyISAM表(GTID是基于事务的)

(2)对CREATE TABLE ..... SELECT语句不支持。

mysql [localhost] {root} (dyy) > create table dyy select * from t1;
ERROR (HY000): CREATE TABLE ... SELECT is forbidden when @@GLOBAL.ENFORCE_GTID_CONSISTENCY = .
mysql [localhost] {root} (dyy) >

(3)必须开启enforce-gtid-consistency参数,否则启动MySQL报错,如下所示:

-- ::  [ERROR] --gtid-mode=ON or UPGRADE_STEP_1 requires --enforce-gtid-consistency
-- :: [ERROR] Aborting

注:在5.6.9的版本还不支持创建临时表,我测试的是5.6.19的版本,已经可以创建临时表了。

四.  多线程复制(基于库)

多线程复制由参数slave-paralles-workers控制,设定从服务器的SQL线程数;0表示关闭多线程复制功能;默认为0,最大可以设置为1024个线程。在从库的配置文件[mysqld]段落添加配置:

slave_parallel_workers = 

重启MySQL以后使用show processlist看到如下所示:

mysql [localhost] {msandbox} ((none)) > show processlist;
+----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+
| | system user | | NULL | Connect | | Slave has read all relay log; waiting for the slave I/O thread to update it | NULL |
| | system user | | NULL | Connect | | Waiting for master to send event | NULL |
| | system user | | NULL | Connect | | Waiting for an event from Coordinator | NULL |
| | system user | | NULL | Connect | | Waiting for an event from Coordinator | NULL |
| | system user | | NULL | Connect | | Waiting for an event from Coordinator | NULL |
| | system user | | NULL | Connect | | Waiting for an event from Coordinator | NULL |
| | system user | | NULL | Connect | | Waiting for an event from Coordinator | NULL |
| | system user | | NULL | Connect | | Waiting for an event from Coordinator | NULL |
| | msandbox | localhost | NULL | Query | | init | show processlist |
+----+-------------+-----------+------+---------+------+-----------------------------------------------------------------------------+------------------+
rows in set (0.00 sec) mysql [localhost] {msandbox} ((none)) >

此外用于保证relog信息以及master信息不丢失,需要设置相关参数来保证,可以参考我前面的文章。

总结:

我们尽可能的把一个库中的表按照业务逻辑拆分为多个库,这样在master上写操作时,slave就可以根据我们设置的线程数进行多线程复制,减少了传统复制的问题--延时。比如2个库我们就可以开启2个线程,3个库就可以开启3个线程,以此类推。还有目前GTID好像还不太成熟,官方的版本里面也没有默认开启,所以还需要进行大量的测试。

参考资料

http://dev.mysql.com/doc/refman/5.6/en/replication-options-gtids.html

http://dev.mysql.com/doc/refman/5.5/en/replication-howto.html

转自

MySQL 5.6 GTID Replication - yayun - 博客园
http://www.cnblogs.com/gomysql/p/3816031.html

MySQL 5.6 GTID Replication【转】的更多相关文章

  1. Repair MySQL 5.6 GTID replication by injecting empty transactions

    Since SQL_SLAVE_SKIP_COUNTER doesn’t work with GTID we need to find a way to ignore that transaction ...

  2. MySQL 5.6 GTID Replication

    一. MySQL 5.6引入了GTID的概念,那么GTID是何方神圣?其实也不复杂,就是一个全局事务标示符.使用GTID时,每次事务提交都会在binlog里生成1个唯一的标示符,它由UUID和事务ID ...

  3. MYSQL Statement violates GTID consistency: Updates to non-transactional tables can only be done in either autocommitted statements or single-statement transactions, and never in the same statement as

    [2019-04-21 10:17:20] [ERROR] [org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144] Statement viola ...

  4. (5.7)mysql高可用系列——MySQL中的GTID复制(理论篇)【转】

    转自:https://blog.csdn.net/wmq880204/article/details/53160078 一.GTID的概述: 1.全局事物标识:global transaction i ...

  5. MySQL5.7 GTID学习笔记,[MySQL 5.6] GTID实现、运维变化及存在的bug

      GTID(global transaction identifier)是对于一个已提交事务的全局唯一编号,前一部分是server_uuid,后面一部分是执行事务的唯一标志,通常是自增的. 下表整理 ...

  6. [MySQL 5.6] GTID实现、运维变化及存在的bug

    [MySQL 5.6] GTID实现.运维变化及存在的bug http://www.tuicool.com/articles/NjqQju 由于之前没太多深入关注gtid,这里给自己补补课,本文是我看 ...

  7. How to create/restore a slave using GTID replication in MySQL 5.6

    MySQL 5.6 is GA! Now we have new things to play with and in my personal opinion the most interesting ...

  8. 基于GTID Replication主从数据不一致操作

    基本的M-S结构   现在master与slave主机数据一致:   mysql> select * from t1; +------+ | id   | +------+ |    1 | | ...

  9. 【MySql】——MHA+GTID+failover+binlog-server+Atlas

    一.环境准备 1.mysql-db01 #系统版本 [root@mysql-db01 ~]# cat /etc/redhat-release CentOS release 6.7 (Final) #内 ...

随机推荐

  1. hdu 5475 (线段树)

    An easy problem Time Limit: 8000/5000 MS (Java/Others)    Memory Limit: 65536/65536 K (Java/Others)T ...

  2. hbase 多个过滤器组合(列表)

    使用FilterList要保证过滤器的顺序需要使用List<Filter> private static void mutilFilterData() throws IOException ...

  3. MT【130】Heilbronn问题

    (清华THUSSAT,多选题) 平面上 4 个不同点 \(P_1,P_2,P_3,P_4\),在每两个点之间连接线段得到 6 条线段. 记 \[L=\max_{1\leq i<j\leq 4}| ...

  4. 【刷题】HDU 3435 A new Graph Game

    Problem Description An undirected graph is a graph in which the nodes are connected by undirected ar ...

  5. 【转】rt-thread的位图调度算法分析

    序言 期待读者 本文期待读者有C语言编程基础,后文中要分析代码,对其中的一些C语言中的简单语句不会介绍,但是并不要求读者有过多的C基础,比如指针和链表等不会要求太多,后面在分析代码时,会附带地介绍相关 ...

  6. 解题:ZJOI 2015 幻想乡战略游戏

    题面 神**所有点都爆int,我还以为我写出什么大锅了,不开long long见祖宗... 动态点分治利用点分树树高不超过log的性质,我们对每个点维护一个子树和,一个点分树子树和,一个点分树上父亲的 ...

  7. echarts彩虹柱状图 每个bar显示不同颜色, 标题在不同位置 ,工具中有可以直接保存为图片下载,平均线的添加

    可以参考: https://echarts.baidu.com/echarts2/doc/example.html https://echarts.baidu.com/echarts2/doc/doc ...

  8. 星号三角形 I

    N = int(eval(input())) for row in range(1,N+1): if row%2 != 0: a = '*'*row print ('{}'.format(a.cent ...

  9. JAVA 线程池基本总结

    合理利用线程池能够带来三个好处. 第一:降低资源消耗.通过重复利用已创建的线程降低线程创建和销毁造成的消耗. 第二:提高响应速度.当任务到达时,任务可以不需要等到线程创建就能立即执行. 第三:提高线程 ...

  10. Windows 2012 R2 安装net4.6.1

    下载并安装Net4.6.1 根据提示下载如下,并安装 https://support.microsoft.com/zh-cn/help/2919355/windows-rt-8-1--windows- ...