gtid_executed和gtid_purged变量是如何初始化的
一、官方释义
1.1、gtid_executed、gtid_purged
https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html#sysvar_gtid_executed
• gtid_executed
When used with global scope, this variable contains a representation of the set of all transactions executed on the server and GTIDs that have been set by a SET gtid_purged statement. This is the same as the value of the Executed_Gtid_Set column in the output of SHOW MASTER STATUS and SHOW SLAVE STATUS.
• gtid_purged
The set of all transactions that have been purged from the binary log. This is a subset of the set of transactions in gtid_executed.
gtid_executed(global)：MySQL数据库已经执行过的Gtid事务，处于内存中。show master status/show slave status中的Executed_Gtid_Set也取自这里
gtid_purged(global)：由于binlog文件的删除(如purge binary logs或者超过expire_logs_days设置)已经丢失的Gtid事务，它是gtid_executed的子集
1.2、binlog_gtid_simple_recovery
https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html#sysvar_binlog_gtid_simple_recovery
binlog_gtid_simple_recovery=FALSE
• To initialize gtid_executed, binary log files are iterated from the newest file, stopping at the first binary log that has any Previous_gtids_log_event. All GTIDs from Previous_gtids_log_event and Gtid_log_events are read from this binary log file. This GTID set is stored internally and called gtids_in_binlog. The value of gtid_executed is computed as the union of this set and the GTIDs stored in the mysql.gtid_executed table.
This process could take a long time if you had a large number of binary log files without GTID events, for example created when gtid_mode=OFF.
• To initialize gtid_purged, binary log files are iterated from the oldest to the newest, stopping at the first binary log that contains either a Previous_gtids_log_event that is non-empty (that has at least one GTID) or that has at least one Gtid_log_event. From this binary log it reads Previous_gtids_log_event. This GTID set is subtracted from gtids_in_binlog and the result stored in the internal variable gtids_in_binlog_not_purged. The value of gtid_purged is initialized to the value of gtid_executed, minus gtids_in_binlog_not_purged.
binlog_gtid_simple_recovery=TRUE
which is the default in MySQL 5.7.7 and later, the server iterates only the oldest and the newest binary log files and the values of gtid_purged and gtid_executed are computed based only on Previous_gtids_log_event or Gtid_log_event found in these files. This ensures only two binary log files are iterated during server restart or when binary logs are being purged.
数据库服务启动时，gtid_executed和gtid_purged按下面方式初始化
binlog_gtid_simple_recovery=FALSE
• gtid_executed：从mysql-bin.index的末行往首行所对应的binlog查找，直到首个被找到包含Previous_gtids_log_event的binlog。然后读取这个binlog的Previous_gtids_log_event和Gtid_log_events中的所有Gtid集合保存到内部变量gtids_in_binlog。然后使用gtids_in_binlog和mysql.gtid_executed表的并集初始化gtid_executed变量
如果你有大量非GTID的binlog(比如gtid_mode=off的情况下创建)，初始化gtid_executed的过程会消耗较长的时间
• gtid_purged：从mysql-bin.index的首行往末行所对应的binlog查找，直到首个被找到包含非空Previous_gtids_log_event或者Gtid_log_event的binlog。然后读取这个binlog的Previous_gtids_log_event，将gtids_in_binlog - Previous_gtids_log_event得到的集合保存到内部变量gtids_in_binlog_not_purged。最后使用gtid_executed - gtids_in_binlog_not_purged初始化gtid_purged变量
binlog_gtid_simple_recovery=TRUE(MySQL5.7.7及以上默认)
只迭代mysql-bin.index的首行和末行所对应的binlog，gtid_executed和gtid_purged的值就是取这两个binlog中的Previous_gtids_log_event/Gtid_log_event计算，当然gtid_executed变量的值还要结合mysql.gtid_executed
1.3、mysql.gtid_executed
https://dev.mysql.com/doc/refman/5.7/en/replication-gtids-concepts.html#replication-gtids-gtid-executed-table
GTIDs are stored in the mysql.gtid_executed table only when gtid_mode is ON or ON_PERMISSIVE. The point at which GTIDs are stored depends on whether binary logging is enabled or disabled:
• If binary logging is disabled (log_bin is OFF), or if log_slave_updates is disabled, the server stores the GTID belonging to each transaction together with the transaction in the table. In addition, the table is compressed periodically at a user-configurable rate; see mysql.gtid_executed Table Compression, for more information. This situation can only apply on a replication slave where binary logging or slave update logging is disabled. It does not apply on a replication master, because on a master, binary logging must be enabled for replication to take place.
• If binary logging is enabled (log_bin is ON), whenever the binary log is rotated or the server is shutdown, the server writes GTIDs for all transactions that were written into the previous binary log into the mysql.gtid_executed table. This situation applies on a replication master, or a replication slave where binary logging is enabled.
In the event of the server stopping unexpectedly, the set of GTIDs from the current binary log is not saved in the mysql.gtid_executed table. In this case, these GTIDs are added to the table and to the set of GTIDs in the gtid_executed system variable during recovery.
When binary logging is enabled, the mysql.gtid_executed table does not provide a complete record of the GTIDs for all executed transactions. That information is provided by the global value of the gtid_executed system variable.
如果没有开启log_bin或者没有开启log_slave_updates，从库在应用relay-log中的每个事务会执行一个insert mysql.gtid_executed操作。这只针对从库而言~
如果开启log_bin，在binlog发生rotate(flush binary logs/达到max_binlog_size)或者关闭服务时，会把所有写入到binlog中的Gtid信息写入到mysql.gtid_executed表。这适用于主库和从库~
log_bin=on，MySQL 8.0.17 起每个事务提交时会更新mysql.gtid_executed表
例子：从库log_bin=on，log_slave_updates=off，那么在应用relay-log时会实时写入mysql.gtid_executed，而在从库直接写入数据，需要等到发生rotate或者关闭服务才写入~
mysql.gtid_executed压缩：log-bin=off，每gtid_executed_compression_period压缩一次；log-bin=on，日志切换时压缩~
如果发生异常crash，当前binlog中的Gtids信息没能写入到mysql.gtid_executed表，在恢复过程通过读取binlog中的Previous_gtids_log_event/Gtid_log_event信息把这些Gtids添加到mysql.gtid_executed表和gtid_executed系统变量
二、实验
基本环境：官方社区版MySQL 5.7.19
[mysqld]
gtid-mode = on
binlog_gtid_simple_recovery = true
log-bin = /data/mysql/mysql3306/logs/mysql-bin
2.1、the oldest and newest file
前面初始化gtid_executed和gtid_purged时需迭代the newest和the oldest file. 最新和最旧分别对应的是哪个binary log？
2.1.1、binary log与index file一致
# 置空所有Gtid信息
mydba@192.168.85.132,3308 [(none)]> reset master;
Query OK, 0 rows affected (0.04 sec)
# 关闭服务
mydba@192.168.85.132,3308 [(none)]> shutdown; # 将另一实例下的binlog拷贝到当前实例的/data/mysql/mysql3308/logs/目录，并修改文件属主 # binlog列表
[root@ZST1 logs]# ll
total 32
-rw-r-----. 1 mysql mysql 2012 Jan 19 15:33 mysql-bin.000202
-rw-r-----. 1 mysql mysql 1528 Jan 19 15:33 mysql-bin.000203
-rw-r-----. 1 mysql mysql 545 Jan 19 15:33 mysql-bin.000204
-rw-r-----. 1 mysql mysql 873 Jan 19 15:33 mysql-bin.000205
-rw-r-----. 1 mysql mysql 217 Jan 19 15:33 mysql-bin.000206
-rw-r-----. 1 mysql mysql 1056 Jan 19 15:33 mysql-bin.000207
-rw-r-----. 1 mysql mysql 545 Jan 19 15:33 mysql-bin.000208
-rw-r--r--. 1 mysql mysql 308 Jan 19 15:40 mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3308/logs/mysql-bin.000202
/data/mysql/mysql3308/logs/mysql-bin.000203
/data/mysql/mysql3308/logs/mysql-bin.000204
/data/mysql/mysql3308/logs/mysql-bin.000205
/data/mysql/mysql3308/logs/mysql-bin.000206
/data/mysql/mysql3308/logs/mysql-bin.000207
/data/mysql/mysql3308/logs/mysql-bin.000208
[root@ZST1 logs]#
# 启动后登录
mydba@192.168.85.132,3308 [(none)]> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000202 | 2012 |
| mysql-bin.000203 | 1528 |
| mysql-bin.000204 | 545 |
| mysql-bin.000205 | 873 |
| mysql-bin.000206 | 217 |
| mysql-bin.000207 | 1056 |
| mysql-bin.000208 | 545 |
| mysql-bin.000209 | 194 |
+------------------+-----------+
8 rows in set (0.00 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507528 |
+--------------------------------------+----------------+--------------+
1 row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132,3308 [(none)]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507528 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged';
+---------------+-----------------------------------------------+
| Variable_name | Value |
+---------------+-----------------------------------------------+
| gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507510 |
+---------------+-----------------------------------------------+
1 row in set (0.04 sec)
解析mysql-bin.000208、mysql-bin.000202
# 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
...
# :: server id end_log_pos CRC32 0xed94846f Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
# :: server id end_log_pos CRC32 0x8b964e9a GTID last_committed= sequence_number= rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '8ab82362-9c37-11e7-a858-000c29c1025c:507528'/*!*/;
# at
# :: server id end_log_pos CRC32 0x15e5bb70 Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
BEGIN
/*!*/;
# at
# :: server id end_log_pos CRC32 0x16207d9b Table_map: `replcrash`.`py_user` mapped to number
# at
# :: server id end_log_pos CRC32 0x4fedbc9d Write_rows: table id flags: STMT_END_F
### INSERT INTO `replcrash`.`py_user`
### SET
### @= /* INT meta=0 nullable=0 is_null=0 */
### @='d6366f0e-fcb9-11e7-ad55-000c29' /* VARSTRING(96) meta=96 nullable=1 is_null=0 */
### @='2018-01-19 09:41:15' /* DATETIME(0) meta=0 nullable=1 is_null=0 */
### @='' /* VARSTRING(30) meta=30 nullable=1 is_null=0 */
# at
# :: server id end_log_pos CRC32 0x46352a20 Xid =
COMMIT/*!*/;
# at
# :: server id end_log_pos CRC32 0x6e53eee8 Stop
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]# gtids_in_binlog：[-]
mysql.gtid_executed：空
gtid_executed变量：gtids_in_binlog ∪ mysql.gtid_executed = [-] # 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xbb10fc9f Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x1e8fe6f0 Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
... gtids_in_binlog：[-]
oldest Previous_gtids_log_event：[-]
gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [-] - [-] = [-]
gtid_purged变量：gtid_executed - gtids_in_binlog_not_purged = [-] - [-] = [-]
binlog解析得到的数值和库中查询结果一致~
2.1.2、binary log与index file不一致
# 置空所有Gtid信息
mydba@192.168.85.132,3308 [(none)]> reset master;
Query OK, 0 rows affected (0.04 sec)
# 关闭服务
mydba@192.168.85.132,3308 [(none)]> shutdown; # 将另一实例下的binlog拷贝到当前实例的/data/mysql/mysql3308/logs/目录，并修改文件属主
# 并调整mysql-bin.index中的顺序 # binlog列表
[root@ZST1 logs]# ll
total 32
-rw-r-----. 1 mysql mysql 2012 Jan 19 17:11 mysql-bin.000202
-rw-r-----. 1 mysql mysql 1528 Jan 19 17:11 mysql-bin.000203
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000204
-rw-r-----. 1 mysql mysql 873 Jan 19 17:11 mysql-bin.000205
-rw-r-----. 1 mysql mysql 217 Jan 19 17:11 mysql-bin.000206
-rw-r-----. 1 mysql mysql 1056 Jan 19 17:11 mysql-bin.000207
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000208
-rw-r-----. 1 mysql mysql 308 Jan 19 17:12 mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3308/logs/mysql-bin.000203
/data/mysql/mysql3308/logs/mysql-bin.000202
/data/mysql/mysql3308/logs/mysql-bin.000204
/data/mysql/mysql3308/logs/mysql-bin.000205
/data/mysql/mysql3308/logs/mysql-bin.000206
/data/mysql/mysql3308/logs/mysql-bin.000208
/data/mysql/mysql3308/logs/mysql-bin.000207
[root@ZST1 logs]#
# 启动后登录
mydba@192.168.85.132,3308 [(none)]> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000203 | 1528 |
| mysql-bin.000202 | 2012 |
| mysql-bin.000204 | 545 |
| mysql-bin.000205 | 873 |
| mysql-bin.000206 | 217 |
| mysql-bin.000208 | 545 |
| mysql-bin.000207 | 1056 |
| mysql-bin.000209 | 194 |
+------------------+-----------+
8 rows in set (0.00 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507527 |
+--------------------------------------+----------------+--------------+
1 row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132,3308 [(none)]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged';
+---------------+-----------------------------------------------+
| Variable_name | Value |
+---------------+-----------------------------------------------+
| gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507517 |
+---------------+-----------------------------------------------+
1 row in set (0.01 sec)
解析mysql-bin.000207、mysql-bin.000203
# 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
...
COMMIT/*!*/;
# at
# :: server id end_log_pos CRC32 0x91996947 GTID last_committed= sequence_number= rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '8ab82362-9c37-11e7-a858-000c29c1025c:507527'/*!*/;
# at
# :: server id end_log_pos CRC32 0x41cdb52a Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
BEGIN
/*!*/;
# at
# :: server id end_log_pos CRC32 0xfcf11bdb Table_map: `replcrash`.`py_user` mapped to number
# at
# :: server id end_log_pos CRC32 0x0550d2ab Write_rows: table id flags: STMT_END_F
### INSERT INTO `replcrash`.`py_user`
### SET
### @= /* INT meta=0 nullable=0 is_null=0 */
### @='d5986a46-fcb7-11e7-ad55-000c29' /* VARSTRING(96) meta=96 nullable=1 is_null=0 */
### @='2018-01-19 09:26:55' /* DATETIME(0) meta=0 nullable=1 is_null=0 */
### @='' /* VARSTRING(30) meta=30 nullable=1 is_null=0 */
# at
# :: server id end_log_pos CRC32 0x15796ccd Xid =
COMMIT/*!*/;
# at
# :: server id end_log_pos CRC32 0xaf26375b Rotate to mysql-bin. pos:
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]# gtids_in_binlog：[-]
mysql.gtid_executed：空
gtid_executed变量：gtids_in_binlog ∪ mysql.gtid_executed = [-] # 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xd61e82fe Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x962d8c48 Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
... gtids_in_binlog：[-]
oldest Previous_gtids_log_event：[-]
gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [-] - [-] = [-]
gtid_purged变量：gtid_executed - gtids_in_binlog_not_purged = [-] - [-] = [-]
binlog解析得到的数值和库中查询结果一致~
上面的结果说明mysql.index中的首行和末行对应的就是oldest和newest
2.2、gtid_executed是来自gtids_in_binlog和mysql.gtid_executed的并集
前面的例子在服务启动时，读取最新的binlog中的Previous_gtids_log_event/Gtid_log_event，使用这些Gtid信息初始化gtid_executed，并将其写入到mysql.gtid_executed表
此时不再重置Gtid信息，而是继续缩小mysql-bin.index首行和末行的范围
# 关闭服务
mydba@192.168.85.132,3308 [(none)]> shutdown; # 将另一实例下的binlog拷贝到当前实例的/data/mysql/mysql3308/logs/目录，并修改文件属主
# 调整mysql-bin.index文件中列表顺序 # binlog列表
[root@ZST1 logs]# ll
total 32
-rw-r-----. 1 mysql mysql 2012 Jan 19 17:11 mysql-bin.000202
-rw-r-----. 1 mysql mysql 1528 Jan 19 17:11 mysql-bin.000203
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000204
-rw-r-----. 1 mysql mysql 873 Jan 19 17:11 mysql-bin.000205
-rw-r-----. 1 mysql mysql 217 Jan 19 17:11 mysql-bin.000206
-rw-r-----. 1 mysql mysql 1056 Jan 19 17:11 mysql-bin.000207
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000208
-rw-r-----. 1 mysql mysql 308 Jan 19 17:31 mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3308/logs/mysql-bin.000204
/data/mysql/mysql3308/logs/mysql-bin.000203
/data/mysql/mysql3308/logs/mysql-bin.000202
/data/mysql/mysql3308/logs/mysql-bin.000205
/data/mysql/mysql3308/logs/mysql-bin.000207
/data/mysql/mysql3308/logs/mysql-bin.000208
/data/mysql/mysql3308/logs/mysql-bin.000206
[root@ZST1 logs]#
# 启动后登录
mydba@192.168.85.132,3308 [(none)]> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000204 | 545 |
| mysql-bin.000203 | 1528 |
| mysql-bin.000202 | 2012 |
| mysql-bin.000205 | 873 |
| mysql-bin.000207 | 1056 |
| mysql-bin.000208 | 545 |
| mysql-bin.000206 | 217 |
| mysql-bin.000209 | 194 |
+------------------+-----------+
8 rows in set (0.03 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507527 |
+--------------------------------------+----------------+--------------+
1 row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132,3308 [(none)]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged';
+---------------+-------------------------------------------------------------+
| Variable_name | Value |
+---------------+-------------------------------------------------------------+
| gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507521:507525-507527 |
+---------------+-------------------------------------------------------------+
1 row in set (0.00 sec)
解析mysql-bin.000206、mysql-bin.000204
# 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0x17238df5 Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x3c03e609 Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
# :: server id end_log_pos CRC32 0x53b3b407 Stop
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]# gtids_in_binlog：[-]
mysql.gtid_executed：[-]
gtid_executed变量：gtids_in_binlog ∪ mysql.gtid_executed = [-] # 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xee45d7bf Start: binlog v , server v 5.7.-log created ::
# at
# :: server id end_log_pos CRC32 0xf8024ebe Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
... gtids_in_binlog：[-]
oldest Previous_gtids_log_event：[-]
gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [-] - [-] = [-]
gtid_purged变量：gtid_executed - gtids_in_binlog_not_purged = [-] - [-] = [-:-]
上面的结果说明gtid_executed变量取的是gtids_in_binlog和mysql.gtid_executed的并集，并不是单纯来自某一个值~
2.3、binlog在Rotate或者关闭服务时，会把Gtid信息写入到mysql.gtid_executed表
2.3.1、关闭服务
写入一批数据，关闭服务，删除binlog后再启动服务，查看mysql.gtid_executed表
# 置空所有Gtid信息
mydba@192.168.85.132, [replcrash]> reset master;
Query OK, rows affected (0.01 sec)
# 清空测试数据表
mydba@192.168.85.132, [replcrash]> truncate table py_user;
Query OK, rows affected (1.49 sec)
# 写入数据(执行两次)
mydba@192.168.85.132, [replcrash]> insert into py_user(name,server_id) select left(uuid(),),@@server_id;
Query OK, row affected (0.00 sec)
Records: Duplicates: Warnings:
# 查看表中数据
mydba@192.168.85.132, [replcrash]> select * from py_user;
+-----+--------------------------------+---------------------+-----------+
| uid | name | add_time | server_id |
+-----+--------------------------------+---------------------+-----------+
| | 5a57574a--11e8--000c29 | -- :: | |
| | 65e773f6--11e8--000c29 | -- :: | |
+-----+--------------------------------+---------------------+-----------+
rows in set (0.00 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
Empty set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_purged | |
+---------------+-------+
row in set (0.00 sec)
# 关闭服务
mydba@192.168.85.132, [replcrash]> shutdown; # 此时的binlog
[root@ZST1 logs]# pwd
/data/mysql/mysql3306/logs
[root@ZST1 logs]# ll
total
-rw-r----- mysql mysql Jan : mysql-bin.
-rw-r----- mysql mysql Jan : mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3306/logs/mysql-bin.
[root@ZST1 logs]#
mysql-bin. 记录 60863f8d-01af-11e8-bfdf-000c29c1025c:- 的信息
# 删除binlog
[root@ZST1 logs]# cp -r ../logs ../logs_bak
[root@ZST1 logs]# rm -rf * # 启动服务
[root@ZST1 logs]# sh ~/start3306.sh
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 60863f8d-01af-11e8-bfdf-000c29c1025c | | |
+--------------------------------------+----------------+--------------+
row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------+
| gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+---------------+------------------------------------------+
row in set (0.01 sec)
重启前mysql.gtid_executed的Gtid为空，关闭服务后删除所有binlog，重启后mysql.gtid_executed的Gtid:1-3。它的信息不可能来自binlog，只有可能在关闭服务时Gtid已写入到mysql.gtid_executed
2.3.2、flush binary logs
写入一批数据，flush binary logs，查看mysql.gtid_executed表，并解析binlog日志
# 写入数据(执行两次)
mydba@192.168.85.132, [replcrash]> insert into py_user(name,server_id) select left(uuid(),),@@server_id;
Query OK, row affected (0.04 sec)
Records: Duplicates: Warnings:
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 60863f8d-01af-11e8-bfdf-000c29c1025c | | |
+--------------------------------------+----------------+--------------+
row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看mysql.gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------+
| gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+---------------+------------------------------------------+
row in set (0.01 sec)
写入数据，mysql.gtid_executed表并不会时时更新 # 切换日志
mydba@192.168.85.132, [replcrash]> flush binary logs;
Query OK, rows affected (0.01 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 60863f8d-01af-11e8-bfdf-000c29c1025c | | |
+--------------------------------------+----------------+--------------+
row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看mysql.gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------+
| gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+---------------+------------------------------------------+
row in set (0.01 sec)
flush logs后gtid_purged变量的值马上更新到mysql.gtid_executed表
mysql.gtid_executed表并不是实时更新，flush logs后gtid_executed变量的值马上更新到mysql.gtid_executed表
解析binlog日志
# 解析切换后的日志
[root@ZST1 logs]# ll
total
-rw-r----- mysql mysql Jan : mysql-bin.
-rw-r----- mysql mysql Jan : mysql-bin.
-rw-r----- mysql mysql Jan : mysql-bin.index
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xea188782 Start: binlog v , server v 5.7.-log created ::
# Warning: this binlog is either in use or was not closed properly.
# at
# :: server id end_log_pos CRC32 0xdf719f1d Previous-GTIDs
# 60863f8d-01af-11e8-bfdf-000c29c1025c:-
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]#
注意Rotate后的binlog中Previous-GTIDs是4-5 ，而不是1-5。因为Rotate只把binary log中的Gtid写入到新binary log，同时更新mysql.gtid_executed表
2.4、如何通过binlog和mysql.gtid_executed得到gtid_executed、gtid_purged
搞清楚服务启动前mysql.gtid_executed的值。对于5.7.7及以后默认只需迭代最新和最旧binlog中的Previous_gtids_log_event/Gtid_log_event，计算gtids_in_binlog、gtids_in_binlog_not_purged，然后根据公式计算gtid_executed、gtid_purged变量。可以查看前面解析binlog的例子
三、参考文档
Global Transaction ID Options and Variables：https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html
MySQL 5.7 Gtid内部学习(五) mysql.gtid_executed表/gtid_executed变量/gtid_purged变量的更改时机：https://yq.aliyun.com/articles/294004
[MySQL 5.6] GTID实现、运维变化及存在的bug：http://www.cnblogs.com/MYSQLZOUQI/p/3850578.html
binlog rotate引发的MySQL阻塞事件：https://mp.weixin.qq.com/s/XSnFkuYzIlGWMaXIl-oPeQ
gtid_executed和gtid_purged变量是如何初始化的的更多相关文章
- MySQL Replication--全局参数gtid_executed和gtid_purged
参数定义gtid_executed,全局参数,GTID集合包含所有在该服务器上执行过的事务编号和使用set gtid_purged语句设置过的事务编号,使用SHOW MASTER STATUS和SHO ...
- 关于Javascript循环体变量声明与初始化的效率问题
针对循环体变量声明与初始化的效率问题,将执行的简单测试代码如下: function test(n) { console.time('Internally initialized'); for (var ...
- C++变量的默认初始化规则
定义没有初始化式的变量时,系统有时候会帮我们初始化变量.系统如何初始化取决于变量的类型以及变量定义的位置. 内置类型变量是否自动初始化取决于变量定义的位置.函数体外定义的变量初始成0:函数体内定义的变 ...
- GoLang学习之变量定义和初始化
变量命名原则 go语言的变量名有字母数字和下划线组成,首字母不能为数字,但是字母不仅仅只限于英文字母,所有的UTF-8字符都是可以的. 变量声明和初始化方式 使用var关键字 var a int = ...
- Java初始化顺序（静态变量、静态初始化块、实例变量、实例初始化块、构造方法）
1.执行顺序 1.1.一个类中的初始化顺序 类内容(静态变量.静态初始化块) => 实例内容(变量.初始化块.构造器) 1.2.两个具有继承关系类的初始化顺序 父类的(静态变量.静态初始化块)= ...
- 重要：C/C++变量的自动初始化
对于内置变量的自动初始化 代码1: #include<stdio.h> #define CONST 100 int *p1; ]; int b; static int c; main() ...
- Java类的初始化顺序 （静态变量、静态初始化块、变量、初始...
很有意思的一篇文章 1.没有继承 静态变量->静态初始化块->变量->变量初始化块->构造方法 2.有继承的情况 父类静态变量->父类静态初始化块->子类静态变量- ...
- 【C++】const，static和static const类型成员变量声明及其初始化
1)const定义的常量在超出其作用域之后其空间会被释放,而static定义的静态常量在函数执行后不会释放其存储空间 void f1() { ; cout<<x<<endl; ...
- QT，静态变量要记得初始化
//DbUtil.h #ifndef DBUTIL_H #define DBUTIL_H using namespace std; QString md5Encode(QString passwd); ...
随机推荐
- StringBuilder String string.Concat 字符串拼接速度再议
首先看测试代码: public class StringSpeedTest { "; public string StringAdd(int count) { string str = st ...
- day11 filter函数
场景模拟:我想判断某个列表里面的某个元素怎么怎么样 基础方法,如果需要判断多次则重复代码 ret = [] move_peole = ["alex","sb_wupeiq ...
- 【AtCoder010】B - Boxes（差分）
AtCoder Grand Contest 010 B题 题目链接 题意 n个盒子,第i个盒子有ai个石头. 重复这个步骤:选一个盒子i,每次从第i+j个盒子中移走j个石头,j从1到n,第n+k个盒子 ...
- LOJ #2721. 「NOI2018」屠龙勇士（set + exgcd）
题意 LOJ #2721. 「NOI2018」屠龙勇士 题解 首先假设每条龙都可以打死,每次拿到的剑攻击力为 \(ATK\) . 这个需要支持每次插入一个数,查找比一个 \(\le\) 数最大的数(或 ...
- 自学Linux Shell3.1-帮助命令man
点击返回 自学Linux命令行与Shell脚本之路 3.1-帮助命令man 1.man命令概述 默认bash shell提示符是美元符号($),这个符号表明shell在等待用户输入. Linux ma ...
- 【BZOJ4830】[HNOI2017]抛硬币（组合计数，拓展卢卡斯定理）
[BZOJ4830][HNOI2017]抛硬币(组合计数,拓展卢卡斯定理) 题面 BZOJ 洛谷 题解 暴力是啥? 枚举\(A\)的次数和\(B\)的次数,然后直接组合数算就好了:\(\display ...
- 【BZOJ1820】[JSOI2010]快递服务（动态规划）
[BZOJ1820][JSOI2010]快递服务(动态规划) 题面 BZOJ 洛谷 题解 考虑无脑四维\(dp\).\(f[i][a][b][c]\),表示当前处理到第\(i\)个任务,三辆车的位置分 ...
- [系统]安装fedora 19
再也没有什么大道至简了. ==== 步骤如下: 1. 备份. 2. 刻镜像. 选fedora-kde,gnome呵呵. 3. 分区,格式化,安装. 上面3步没什么好说的,按照官网installatio ...
- tesseract-ocr 识别中文扫描图片
原文链接:http://www.cnblogs.com/alex-blog/articles/2714984.html 项目主页地址:http://code.google.com/p/tesser ...
- 使用selenium模拟登陆oschina
Selenium把元素定位接口封装得更简单易用了,支持Xpath.CSS选择器.以及标签名.标签属性和标签文本查找. from selenium.webdriver import PhantomJS ...