MySQL增强半同步几个重要参数搭配的测试

 

Preface

 

    Semi-synchronous replication is supported since MySQL 5.5 and then enhanced gradually in 5.6 & 5.7.It's also called enhanced semi-synchronous replicaiton in MySQL 5.7.The new method of ACK rule in 5.7(after_sync) make the consistency be strongly guaranteed.Maybe that is the reason why it gets the name.Today I'm gonna demonstrate the effect of sevaral variables of semi-sync.

 

Introduce

 

    Semi-synchronized replicatin can be installed as a plugin.There're a lot of variables with prefix of "repl_semi_sync" after you install the "rpl_semi_sync_master" plugin on your MySQL server.Meanwhile,there're a series of global status about it as well.Let's see the details about it.

 

Framework

Hostname	IP/Port	Identity	OS Version	MySQL Version	GTID Mode	Binlog Format	Semi-Sync status
zlm2	192.168.1.101/3306	master	CentOS 7.0	5.7.21	on	row	on
zlm3	192.168.1.102/3306	slave1	CentOS 7.0	5.7.21	on	row	on
zlm4	192.168.1.103/3306	slave2	CentOS 7.0	5.7.21	on	row	on

 

Example

 

Check the variables relevant to semi-sync replication.

 //Master
 (zlm@192.168.1.101 )[(none)]>show variables like 'rpl_semi%';
 +-------------------------------------------+------------+
 | Variable_name                             | Value      |
 +-------------------------------------------+------------+
 | rpl_semi_sync_master_enabled              | OFF        |
 | rpl_semi_sync_master_timeout              |       |
 | rpl_semi_sync_master_trace_level          |          |
 | rpl_semi_sync_master_wait_for_slave_count |           |
 | rpl_semi_sync_master_wait_no_slave        | ON         |
 | rpl_semi_sync_master_wait_point           | AFTER_SYNC |
 +-------------------------------------------+------------+
  rows in set (0.00 sec)

 //rpl_semi_sync_master_enabled --Specify whether to use the feature of semi-sync(master side,default "off").
 //rpl_semi_sync_master_timeout --Specify the threshold of timeout between master and slave(default value is "10000" means 10s).
 //rpl_semi_sync_master_trace_level --Specify the debug trace level on master(default "32",others "1","16","64").
 //rpl_semi_sync_master_wait_for_slave_count --Specify how many slaves the master should wait when timeout happens(default "1").
 //rpl_semi_sync_master_wait_no_slave --Specify whether the master switches to normal replication(default "on").When setting "on",it permits the amount of slave can be less than the value of "rpl_semi_sync_master_wait_for_slave_count" you specified and prevents it becoming normal replication.
 //rpl_semi_sync_master_wait_point --Specify the ACK mode(default "after-sync").It's the most important variable of semi-sync.

 //Slave1
 (zlm@192.168.1.102 )[(none)]>show variables like 'rpl_semi%';
 +---------------------------------+-------+
 | Variable_name                   | Value |
 +---------------------------------+-------+
 | rpl_semi_sync_slave_enabled     | OFF   |
 | rpl_semi_sync_slave_trace_level |     |
 +---------------------------------+-------+
  rows in set (0.00 sec)

 //There're only two variables after install the rpl_semi_sync_slave plugin.
 //rpl_semi_sync_slave_enabled -- Specify whether use the feature of semi-sync(slave side,default "off").
 //rpl_semi_sync_slave_trace_level -- Specify the debug trace level on slave(default "32").

Enable semi-sync replication on both master and slaves.

 //Master
 (zlm@192.168.1.101 )[(none)]>set rpl_semi_sync_master_enabled=on;
 ERROR  (HY000): Variable 'rpl_semi_sync_master_enabled' is a GLOBAL variable and should be set with SET GLOBAL
 (zlm@192.168.1.101 )[(none)]>set global rpl_semi_sync_master_enabled=on;
 Query OK,  rows affected (0.00 sec)

 (zlm@192.168.1.101 )[(none)]>show variables like 'rpl_semi_sync_master_enabled';
 +------------------------------+-------+
 | Variable_name                | Value |
 +------------------------------+-------+
 | rpl_semi_sync_master_enabled | ON    |
 +------------------------------+-------+
  row in set (0.01 sec)

 //Slave1
 (zlm@192.168.1.102 )[(none)]>set @@global.rpl_semi_sync_slave_enabled=on;
 Query OK,  rows affected (0.00 sec)

 (zlm@192.168.1.102 )[(none)]>show variables like 'rpl_semi_sync%';
 +---------------------------------+-------+
 | Variable_name                   | Value |
 +---------------------------------+-------+
 | rpl_semi_sync_slave_enabled     | ON    |
 | rpl_semi_sync_slave_trace_level |     |
 +---------------------------------+-------+
  rows in set (0.00 sec)

 //Slave2
 (zlm@192.168.1.103 )[(none)]>set @@global.rpl_semi_sync_slave_enabled=on;
 Query OK,  rows affected (0.00 sec)

 (zlm@192.168.1.103 )[(none)]>show variables like 'rpl_semi_sync%';
 +---------------------------------+-------+
 | Variable_name                   | Value |
 +---------------------------------+-------+
 | rpl_semi_sync_slave_enabled     | ON    |
 | rpl_semi_sync_slave_trace_level |     |
 +---------------------------------+-------+
  rows in set (0.01 sec)

 //Master
 (zlm@192.168.1.101 )[(none)]>show global status like 'Rpl_semi_sync_master_clients';
 +------------------------------+-------+
 | Variable_name                | Value |
 +------------------------------+-------+
 | Rpl_semi_sync_master_clients |      | //It means there're two slaves using semi-sync replicaiton follow master now.
 +------------------------------+-------+
  row in set (0.00 sec)

Execute sysbench to generate some trasactions.

 [root@zlm2 :: ~/sysbench-1.0/src/lua]
 #sysbench oltp_insert.lua --mysql-host=192.168.1.101 --mysql-port= --mysql-user=zlm --mysql-password=zlmzlm --mysql-db=sysbench --tables= --table-size= --mysql-storage-engine=innodb prepare
 sysbench 1.0. (using bundled LuaJIT 2.1.-beta2)

 Creating table 'sbtest1'...
 Inserting  records into 'sbtest1'
 Creating a secondary index on 'sbtest1'...
 Creating table 'sbtest2'...
 Inserting  records into 'sbtest2'
 Creating a secondary index on 'sbtest2'...
 Creating table 'sbtest3'...
 Inserting  records into 'sbtest3'
 Creating a secondary index on 'sbtest3'...
 Creating table 'sbtest4'...
 Inserting  records into 'sbtest4'
 Creating a secondary index on 'sbtest4'...
 Creating table 'sbtest5'...
 Inserting  records into 'sbtest5'
 Creating a secondary index on 'sbtest5'...

 [root@zlm2 :: ~/sysbench-1.0/src/lua]
 #sysbench oltp_insert.lua --mysql-host=192.168.1.101 --mysql-port= --mysql-user=zlm --mysql-password=zlmzlm --mysql-db=sysbench --threads= --time= --report-interval= --rand-type=uniform run
 sysbench 1.0. (using bundled LuaJIT 2.1.-beta2)

 Running the test with following options:
 Number of threads:
 Report intermediate results every  second(s)
 Initializing random number generator from current time

 Initializing worker threads...

 Threads started!

 [ 30s ] thds:  tps: 931.95 qps: 931.95 (r/w/o: 0.00/931.95/0.00) lat (ms,%): 6.09 err/s: 0.00 reconn/s: 0.00
 [ 60s ] thds:  tps: 971.62 qps: 971.62 (r/w/o: 0.00/971.62/0.00) lat (ms,%): 5.18 err/s: 0.00 reconn/s: 0.00
 [ 90s ] thds:  tps: 949.33 qps: 949.33 (r/w/o: 0.00/949.33/0.00) lat (ms,%): 4.41 err/s: 0.00 reconn/s: 0.00

Check the global status of semi-sync on master.

 (zlm@192.168.1.101 )[(none)]>show global status like 'Rpl_semi_sync%';
 +--------------------------------------------+------------+
 | Variable_name                              | Value      |
 +--------------------------------------------+------------+
 | Rpl_semi_sync_master_clients               |           |
 | Rpl_semi_sync_master_net_avg_wait_time     |           |
 | Rpl_semi_sync_master_net_wait_time         |           |
 | Rpl_semi_sync_master_net_waits             |     |
 | Rpl_semi_sync_master_no_times              |           |
 | Rpl_semi_sync_master_no_tx                 |     |
 | Rpl_semi_sync_master_status                | ON         |
 | Rpl_semi_sync_master_timefunc_failures     |           |
 | Rpl_semi_sync_master_tx_avg_wait_time      |        |
 | Rpl_semi_sync_master_tx_wait_time          |  |
 | Rpl_semi_sync_master_tx_waits              |      |
 | Rpl_semi_sync_master_wait_pos_backtraverse |           |
 | Rpl_semi_sync_master_wait_sessions         |           |
 | Rpl_semi_sync_master_yes_tx                |      |
 +--------------------------------------------+------------+
  rows in set (0.00 sec)

Set the variable "rpl_semi_sync_master_timeout" to 60000.

 (zlm@192.168.1.101 )[(none)]>set @@global.rpl_semi_sync_master_timeout=;
 Query OK,  rows affected (0.00 sec)

 (zlm@192.168.1.101 )[(none)]>show global variables like 'rpl_semi_sync_master_timeout';
 +------------------------------+-------+
 | Variable_name                | Value |
 +------------------------------+-------+
 | rpl_semi_sync_master_timeout |  |
 +------------------------------+-------+
  row in set (0.00 sec)

Stop slave on slave1 and record the execution time.

 (zlm@192.168.1.102 )[(none)]>system date
 Mon Jul  :: CEST
 (zlm@192.168.1.102 )[(none)]>stop slave; //The stopping time was 11:20:40 around.
 Query OK,  rows affected (0.84 sec)

Check the global status of semi-sync and error log on master.

 (zlm@192.168.1.101 )[(none)]>show global status like 'Rpl_semi_sync%';
 +--------------------------------------------+------------+
 | Variable_name                              | Value      |
 +--------------------------------------------+------------+
 | Rpl_semi_sync_master_clients               |           | //The amount of client turned into "1".
 | Rpl_semi_sync_master_net_avg_wait_time     |           |
 | Rpl_semi_sync_master_net_wait_time         |           |
 | Rpl_semi_sync_master_net_waits             |     |
 | Rpl_semi_sync_master_no_times              |           |
 | Rpl_semi_sync_master_no_tx                 |     |
 | Rpl_semi_sync_master_status                | ON         | //Although the amount of client has been less than "slave_count" but it still work in semi-sync mode.
 | Rpl_semi_sync_master_timefunc_failures     |           |
 | Rpl_semi_sync_master_tx_avg_wait_time      |        |
 | Rpl_semi_sync_master_tx_wait_time          |  |
 | Rpl_semi_sync_master_tx_waits              |      |
 | Rpl_semi_sync_master_wait_pos_backtraverse |           |
 | Rpl_semi_sync_master_wait_sessions         |           |
 | Rpl_semi_sync_master_yes_tx                |      |
 +--------------------------------------------+------------+
  rows in set (0.00 sec)

 --16T11::39.628398+:  [ERROR] mysqld: Got an error reading communication packets //It's the time I stopped the slave on slave1.
 --16T11::40.355498+:  [ERROR] Semi-sync master failed on net_flush() before waiting for slave reply
 --16T11::40.355532+:  [Note] Stop semi-sync binlog_dump to slave (server_id: )
 --16T11::41.358003+:  [Note] Aborted connection  to db: 'unconnected' user: 'repl' host: 'zlm3' (Found net error)
 --16T11::43.367327+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 6128ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::59.683904+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 15315ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::05.709621+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 5025ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::40.353597+:  [Warning] Timeout waiting for reply of binlog (file: mysql-bin., pos: ), semi-sync up to file mysql-bin., position .
 --16T11::40.353630+:  [Note] Semi-sync replication switched OFF. //Until the timeout was accumulated to 60s(11:21:40),the semi-sync mode reverts "OFF" what means the master work as asynchronous replication.

 (zlm@192.168.1.101 )[(none)]>show global status like 'Rpl_semi_sync%';
 +--------------------------------------------+------------+
 | Variable_name                              | Value      |
 +--------------------------------------------+------------+
 | Rpl_semi_sync_master_clients               |           |
 | Rpl_semi_sync_master_net_avg_wait_time     |           |
 | Rpl_semi_sync_master_net_wait_time         |           |
 | Rpl_semi_sync_master_net_waits             |     |
 | Rpl_semi_sync_master_no_times              |          |
 | Rpl_semi_sync_master_no_tx                 |     |
 | Rpl_semi_sync_master_status                | OFF        | //Now the status became "OFF".
 | Rpl_semi_sync_master_timefunc_failures     |           |
 | Rpl_semi_sync_master_tx_avg_wait_time      |        |
 | Rpl_semi_sync_master_tx_wait_time          |  |
 | Rpl_semi_sync_master_tx_waits              |      |
 | Rpl_semi_sync_master_wait_pos_backtraverse |           |
 | Rpl_semi_sync_master_wait_sessions         |           |
 | Rpl_semi_sync_master_yes_tx                |      |
 +--------------------------------------------+------------+
  rows in set (0.00 sec)

Change the variable "rpl_semi_sync_master_wait_no_slave" ot off.

 (zlm@192.168.1.101 )[(none)]>set @@global.rpl_semi_sync_master_wait_no_slave=off;
 Query OK,  rows affected (0.00 sec)

 (zlm@192.168.1.101 )[(none)]>show global variables like 'rpl_semi%';
 +-------------------------------------------+------------+
 | Variable_name                             | Value      |
 +-------------------------------------------+------------+
 | rpl_semi_sync_master_enabled              | ON         |
 | rpl_semi_sync_master_timeout              |       |
 | rpl_semi_sync_master_trace_level          |          |
 | rpl_semi_sync_master_wait_for_slave_count |           |
 | rpl_semi_sync_master_wait_no_slave        | OFF        |
 | rpl_semi_sync_master_wait_point           | AFTER_SYNC |
 +-------------------------------------------+------------+
  rows in set (0.00 sec)

Start slave on slave1 to restore the initial status of semi-sync replication.

 //Slave
 (zlm@192.168.1.102 )[(none)]>start slave;
 Query OK,  rows affected (0.01 sec)

 //Error log of master
 --16T11::22.727473+:  [Note] Start binlog_dump to master_thread_id() slave_server(), pos(, )
 --16T11::31.005135+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 8431ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::31.838932+:  [Note] Start semi-sync binlog_dump to slave (server_id: ), pos(, )
 --16T11::50.117239+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 6667ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::59.977585+:  [Note] Semi-sync replication switched ON at (mysql-bin., )

Stop slave on slave1 again and record the execution time.

 (zlm@192.168.1.102 )[(none)]>system date
 Mon Jul  :: CEST
 (zlm@192.168.1.102 )[(none)]>stop slave; //The stopping time was 11:35:58 around.
 Query OK,  rows affected (1.60 sec)

Check the global status of semi-sync and error log on master one more time. 

 (zlm@192.168.1.101 )[(none)]>show global status like 'Rpl_semi_sync%';
 +--------------------------------------------+------------+
 | Variable_name                              | Value      |
 +--------------------------------------------+------------+
 | Rpl_semi_sync_master_clients               |           |
 | Rpl_semi_sync_master_net_avg_wait_time     |           |
 | Rpl_semi_sync_master_net_wait_time         |           |
 | Rpl_semi_sync_master_net_waits             |     |
 | Rpl_semi_sync_master_no_times              |          |
 | Rpl_semi_sync_master_no_tx                 |     |
 | Rpl_semi_sync_master_status                | ON         |
 | Rpl_semi_sync_master_timefunc_failures     |           |
 | Rpl_semi_sync_master_tx_avg_wait_time      |        |
 | Rpl_semi_sync_master_tx_wait_time          |  |
 | Rpl_semi_sync_master_tx_waits              |      |
 | Rpl_semi_sync_master_wait_pos_backtraverse |           |
 | Rpl_semi_sync_master_wait_sessions         |           |
 | Rpl_semi_sync_master_yes_tx                |      |
 +--------------------------------------------+------------+
  rows in set (0.00 sec)

 --16T11::58.920988+:  [ERROR] mysqld: Got an error reading communication packets
 --16T11::00.321218+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 14988ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::11.150976+:  [Note] InnoDB: page_cleaner: 1000ms intended loop took 8718ms. The settings might not be optimal. (flushed= and evicted=, during the time.)
 --16T11::28.973672+:  [Note] Stop semi-sync binlog_dump to slave (server_id: )
 --16T11::29.012288+:  [Note] Semi-sync replication switched OFF. //The master didn't wait the timeout occur and revert to aynchronous replication directly within about merely 30s.

 (zlm@192.168.1.101 )[(none)]>show global status like 'Rpl_semi_sync%';
 +--------------------------------------------+------------+
 | Variable_name                              | Value      |
 +--------------------------------------------+------------+
 | Rpl_semi_sync_master_clients               |           |
 | Rpl_semi_sync_master_net_avg_wait_time     |           |
 | Rpl_semi_sync_master_net_wait_time         |           |
 | Rpl_semi_sync_master_net_waits             |     |
 | Rpl_semi_sync_master_no_times              |          |
 | Rpl_semi_sync_master_no_tx                 |     |
 | Rpl_semi_sync_master_status                | OFF        |
 | Rpl_semi_sync_master_timefunc_failures     |           |
 | Rpl_semi_sync_master_tx_avg_wait_time      |        |
 | Rpl_semi_sync_master_tx_wait_time          |  |
 | Rpl_semi_sync_master_tx_waits              |      |
 | Rpl_semi_sync_master_wait_pos_backtraverse |           |
 | Rpl_semi_sync_master_wait_sessions         |           |
 | Rpl_semi_sync_master_yes_tx                |      |
 +--------------------------------------------+------------+
  rows in set (0.00 sec)

Summay

• The behavior of semi-sync replication depends on the variables that you specified.
• If the value of "rpl_semi_sync_master_wait_no_slave" is "on",the status of semi-sync won't change until it reaches the value of timeout,even though the amount of slaves are less than it in "rpl_semi_sync_master_slave_count".
• If the value of "rpl_semi_sync_master_wait_no_slave" is "off",master won't wait for timeout before it reverts to normal replication when slave decreases to less than the value above.
• In order to observe the result conveniently,I increase the value of "rpl_semi_sync_master_timeout" to 60s.
• In most high consistent scenarios,the value can be tremendous enough to avoid degenerating to asynchronous replication.
• If the degeneration is due to network failure such as instant glitch.The semi-sync status will be ok after the network becomes normal.
• I'm afraid the best practice is to set "rpl_semi_sync_master_wait_no_slave" to "on" and the "rpl_semi_sync_master_timeout" to a comparative value.