KingbaseES V8R3集群运维案例之---流复制异步同步及全同步模式配置

案例说明：

通过案例描述KingbaseES V8R3集群异步、同步及全同步强一致性配置，本案例为一主二备的架构。

适用版本：

KingbaseES V8R3

集群架构：

集群复制配置参数说明：

1）sync_flag

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i sync_
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=0

说明：此参数强制配置流复制同异步模式，0为异步模式，1为同步模式。

2）synchronous_standby_names

[kingbase@node101 bin]$ cat ../data/kingbase.conf |grep -i standby_name
##synchronous_standby_names = ''        # standby servers that provide sync rep
#synchronous_standby_names='1 (node1, node2)'

说明：此参数用于配置在流复制同步模式下，参与同步复制的备库节点；
例如一主二备集群：'1 (node1,node2,node3)',其中node1是主库，则node2为同步备库，node3为潜在同步备库；
'2 (node1,node2,node3)'，其中node1是主库，则node2为同步备库，node3为同步备库；
synchronous_standby_names = '' ，则node2，node3均为异步备库。

3）synchronous_commit

[kingbase@node101 bin]$ cat ../data/kingbase.conf |grep _commit
#synchronous_commit = on                # synchronization level;

说明：用于同步模式下接收到wal日志流的处理方式：
remote_write：备库将wal日志写入到文件系统缓存后，给主库返回commit的ack信息。
on：          备库将wal日志写入到wal日志文件后，给主库返回commit的ack信息。
remote_apply：备库将wal日志写入到wal日志文件并apply后，给主库返回commit的ack信息，
              强制主备事务同步。

如下图所示：

一、流复制异步模式配置

1、sync_flag配置

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i flag
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=0

2、集群启动后流复制状态查看

TEST=# select * from sys_stat_replication;
  PID  | USESYSID | USENAME | APPLICATION_NAME |  CLIENT_ADDR  | CLIENT_HOSTNAME | CLIENT_PORT |         BACKEND_STAR
T         | BACKEND_XMIN |   STATE   | SENT_LOCATION | WRITE_LOCATION | FLUSH_LOCATION | REPLAY_LOCATION | SYNC_PRIOR
ITY | SYNC_STATE
-------+----------+---------+------------------+---------------+-----------------+-------------+---------------------
----------+--------------+-----------+---------------+----------------+----------------+-----------------+-----------
----+------------
 31144 |       10 | SYSTEM  | node2            | 192.168.1.102 |                 |       16989 | 2023-05-07 00:55:00.
110856+08 |              | streaming | 1/500000D0    | 1/500000D0     | 1/500000D0     | 1/500000D0      |
  0 | async
 31275 |       10 | SYSTEM  | node3            | 192.168.1.103 |                 |       30046 | 2023-05-07 00:55:03.
014221+08 |              | streaming | 1/500000D0    | 1/500000D0     | 1/500000D0     | 1/500000D0      |
  0 | async
(2 rows)

# SYNCHRONOUS_STANDBY_NAMES默认配置
[kingbase@node101 data]$ cat kingbase.auto.conf
# Do not edit this file manually!
# It will be overwritten by ALTER SYSTEM command.
SYNCHRONOUS_STANDBY_NAMES = ''

TEST=# show synchronous_standby_names ;
 synchronous_standby_names
---------------------------

(1 row)

如下图所示，两个备库节点流复制均为async模式：

二、流复制同步模式配置

1、sync_flag配置

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i flag
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=1

2、集群启动后流复制状态查看

TEST=# select * from sys_stat_replication;
 PID  | USESYSID | USENAME | APPLICATION_NAME |  CLIENT_ADDR  | CLIENT_HOSTNAME | CLIENT_PORT |         BACKEND_START
         | BACKEND_XMIN |   STATE   | SENT_LOCATION | WRITE_LOCATION | FLUSH_LOCATION | REPLAY_LOCATION | SYNC_PRIORI
TY | SYNC_STATE
------+----------+---------+------------------+---------------+-----------------+-------------+----------------------
---------+--------------+-----------+---------------+----------------+----------------+-----------------+------------
---+------------
 3966 |       10 | SYSTEM  | node2            | 192.168.1.102 |                 |       18965 | 2023-05-07 01:01:12.3
76208+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 2 | sync
 3970 |       10 | SYSTEM  | node3            | 192.168.1.103 |                 |       30106 | 2023-05-07 01:01:14.9
62029+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 3 | potential
(2 rows)

说明：node2同步模式sync，node3同步模式potential（潜在同步，实为异步）

# SYNCHRONOUS_STANDBY_NAMES默认配置
[kingbase@node101 bin]$ cat ../data/kingbase.auto.conf
# Do not edit this file manually!
# It will be overwritten by ALTER SYSTEM command.
SYNCHRONOUS_STANDBY_NAMES = '1 (node1, node2, node3)'

TEST=# show synchronous_standby_names ;
 synchronous_standby_names
---------------------------
 1 (node1, node2, node3)
(1 row)

如下图，默认流复制同步模式配置，其中一个节点为sync，其余为potential：

三、集群流复制全同步模式配置

1、sync_flag配置

[kingbase@node101 bin]$ cat ../etc/HAmodule.conf |grep -i flag
#1->synchronous cluster,0->asynchronous cluster ,default 1.example:SYNC_FLAG=1
SYNC_FLAG=1

2、配置 synchronous_standby_names

TEST=# alter system set synchronous_standby_names='2 (node1, node2, node3)';
ALTER SYSTEM

TEST=# select sys_reload_conf();
 SYS_RELOAD_CONF
-----------------
 t
(1 row)

TEST=# show synchronous_standby_names ;
 synchronous_standby_names
---------------------------
 2 (node1, node2, node3)
(1 row)

[kingbase@node101 bin]$ cat ../data/kingbase.auto.conf
# Do not edit this file manually!
# It will be overwritten by ALTER SYSTEM command.
SYNCHRONOUS_STANDBY_NAMES = '2 (node1, node2, node3)'

3、查看流复制同步模式

TEST=# select * from sys_stat_replication;
 PID  | USESYSID | USENAME | APPLICATION_NAME |  CLIENT_ADDR  | CLIENT_HOSTNAME | CLIENT_PORT |         BACKEND_START
         | BACKEND_XMIN |   STATE   | SENT_LOCATION | WRITE_LOCATION | FLUSH_LOCATION | REPLAY_LOCATION | SYNC_PRIORI
TY | SYNC_STATE
------+----------+---------+------------------+---------------+-----------------+-------------+----------------------
---------+--------------+-----------+---------------+----------------+----------------+-----------------+------------
---+------------
 3966 |       10 | SYSTEM  | node2            | 192.168.1.102 |                 |       18965 | 2023-05-07 01:01:12.3
76208+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 2 | sync
 3970 |       10 | SYSTEM  | node3            | 192.168.1.103 |                 |       30106 | 2023-05-07 01:01:14.9
62029+08 |              | streaming | 1/510000D0    | 1/510000D0     | 1/510000D0     | 1/510000D0      |
 3 | sync
(2 rows)

如下图所示，两个备库节点node2和node3流复制均为sync模式：

4、读写分离配置备库强一致性

[kingbase@node101 log]$ cat ../db/data/kingbase.conf|grep _commit
synchronous_commit = remote_apply                # synchronization level;

说明：配置synchronous_commit = remote_apply，主库事务commit，需要等待所有备库
接收到wal日志流并写入到wal日志文件，执行apply完成事务的操作后，向主库返回ack，
主库完成事务的commit。
因此，此时主库及备库查询到的事务的结果是一致的，但是对事务繁忙的业务， 性能
会有很大的影响。

四、总结

KingbaseES V8R3和KingbaseES V8R6集群在流复制同异步模式配置不同，通过以上案例的描述，可以了解到在KingbaseES V8R3集群下流复制模式的详细配置过程。