pg_receivewal实践

测试从pg_receivewal的日志中恢复从库为主库：

主从配置async模式，配置pg_receivewal接收日志pg_receivewal -D /dbaas/pg/data/pg_receivewal_data -v -h 10.9.10.202

主插入1000万数据，当插入一半时，停止从库

主库插完数据，停止

将pg_receivewal的日志拷贝到从库/dbaas/pg/data/pg_receivewal_data下

修改从库recovery.conf文件，添加内容：

restore_command = 'cp /dbaas/pg/data/pg_receivewal_data/%f "%p"'

启动数据库，数据库成功恢复到10000000行。

另外一种用法：

restore_command = 'cp /dbaas/pg/data/pg_receivewal_data/%f "%p"'

standby_mode = on

primary_conninfo = 'host=10.9.10.203 port=5432 user=postgres connect_timeout=60'

recovery_target_timeline = 'latest'

trigger_file = '/dbaas/pg/data/.tfile'

这种把恢复和备库设置放一起，会先去从归档恢复，当恢复完成后，直接建立流复制关系，这样就相当方便了，当有归档时，不用重做从库。

测试pg_receivewal的同步机制：

-S slotname

--slot=slotname

Require pg_receivewal to use an existing replication slot (see Section 26.2.6). When this option is used, pg_receivewal will report a flush position to the server, indicating when each segment has been synchronized to disk so that the server can remove that segment if it is not otherwise needed.

When the replication client of pg_receivewal is configured on the server as a synchronous standby, then using a replication slot will report the flush position to the server, but only when a WAL file is closed. Therefore, that configuration will cause transactions on the primary to wait for a long time and effectively not work satisfactorily. The option --synchronous (see below) must be specified in addition to make this work correctly.

--synchronous

Flush the WAL data to disk immediately after it has been received. Also send a status packet back to the server immediately after flushing, regardless of --status-interval.

This option should be specified if the replication client of pg_receivewal is configured on the server as a synchronous standby, to ensure that timely feedback is sent to the server.

在这个说明中，使用了--slot=for_pgreceivewal --synchronous后，理论上是有点像sync同步一样，数据库会等待接收端进行回放，回放结束，再进行commit。

1）使slot和synchronous参数：

数据库端：

修改postgresql.conf参数synchronous_standby_name=‘pg_receivewal’

重启数据库

postgres=# select pg_create_physical_replication_slot('for_pgreceivewal');

pg_create_physical_replication_slot

-------------------------------------

(for_pgreceivewal,)

(1 row)

postgres=# select pg_get_replication_slots();

pg_get_replication_slots

----------------------------------------

(for_pgreceivewal,,physical,,f,f,,,,,)

(1 row)

postgres=# select pg_get_replication_slots();

pg_get_replication_slots

--------------------------------------------------------

(for_pgreceivewal,,physical,,f,t,629488,,,0/9DD77B00,)

(1 row)

接收端：

pg_receivewal -D /dbaas/pg/data/pg_receivewal_data --slot=for_pgreceivewal --synchronous -v -h 10.9.10.202

数据库端：

postgres=# select * from pg_stat_replication ;

pid   | usesysid | usename  | application_name | client_addr | client_hostname | client_port |         backend_start         | backend_xmin |   state   |  sent_lsn  | write_lsn  | flush_lsn  | replay_lsn |    write_lag    |    flush_lag    |   replay_lag    | sync_priority | sync_state

--------+----------+----------+------------------+-------------+-----------------+-------------+-------------------------------+--------------+-----------+------------+------------+------------+------------+-----------------+-----------------+-----------------+---------------+------------

912703 |       10 | postgres | pg_receivewal    | 10.9.10.203 |                 |       63103 | 2019-05-15 02:52:30.029814+00 |              | streaming | 0/D9E44528 | 0/D9E44528 | 0/D9E44528 |            | 00:00:00.010972 | 00:00:00.010972 | 00:06:49.964729 |             1 | sync

(1 row)

在数据库端，使用pgbench测试：

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf /dbaas/pg/data]$ pgbench -c 10 -j 5 -r -T 100 pgbench

starting vacuum...end.

transaction type: <builtin: TPC-B (sort of)>

scaling factor: 100

query mode: simple

number of clients: 10

number of threads: 5

duration: 100 s

number of transactions actually processed: 15290

latency average = 65.426 ms

tps = 152.845128 (including connections establishing)

tps = 152.855112 (excluding connections establishing)

statement latencies in milliseconds:

0.002  \set aid random(1, 100000 * :scale)

0.001  \set bid random(1, 1 * :scale)

0.001  \set tid random(1, 10 * :scale)

0.000  \set delta random(-5000, 5000)

0.046  BEGIN;

0.300  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;

0.111  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;

0.417  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;

2.878  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;

0.092  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);

61.539  END;

2）不使用slot和synchronous参数

数据库端，删除slot

postgres=# select pg_drop_replication_slot('for_pgreceivewal');

pg_drop_replication_slot

--------------------------

(1 row)

postgres=# select pg_get_replication_slots();

pg_get_replication_slots

--------------------------

(0 rows)

接收端：

pg_receivewal -D /dbaas/pg/data/pg_receivewal_data  -v -h 10.9.10.202

压测：

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf ~]$ pgbench -c 10 -j 5 -r -T 100 pgbench

starting vacuum...end.

transaction type: <builtin: TPC-B (sort of)>

scaling factor: 100

query mode: simple

number of clients: 10

number of threads: 5

duration: 100 s

number of transactions actually processed: 27015

latency average = 37.021 ms

tps = 270.120226 (including connections establishing)

tps = 270.138206 (excluding connections establishing)

statement latencies in milliseconds:

0.002  \set aid random(1, 100000 * :scale)

0.001  \set bid random(1, 1 * :scale)

0.001  \set tid random(1, 10 * :scale)

0.000  \set delta random(-5000, 5000)

0.042  BEGIN;

0.242  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;

0.110  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;

0.291  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;

1.778  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;

0.091  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);

34.396  END;

同步模式下，停止pg_receivewal，会导致主hang住：

postgres=# create table test(id int, info varchar(8), crt_time timestamp);

^CCancel request sent

WARNING:  canceling wait for synchronous replication due to user request

DETAIL:  The transaction has already committed locally, but might not have been replicated to the standby.

CREATE TABLE

pg_receivewal的使用场景思考：

1）当配置高可用集群时，一个备库很危险，容易出现单点故障，当同步节点死掉后，主节点会hang住（这是没有专业高可用工具的情况下，ecox这种高可用工具，会自动处理为单节点可写）。而部署两个同步节点又会浪费资源（不做读写分离的情况下），这个时候，可以创建一个pg_receivewal节点，使用同步模式，当一个同步节点停止后，它可以充当sync节点，给主节点报告LSN。

为了减少网络开销，还可以将pg_receivewal节点设置为主库本地，即起到了归档作用，又起到了同步节点的功能。唯一注意的是，要定期清理日志空间。

2）第二种情况，数据库集群不使用sync模式，想使用更加高性能的async模式。这时，使用pg_receivewal来做日志归档，至少比aync节点同步的日志要多。这样基于他的恢复，也将丢更少的数据。当数据库更新插入极端的大时，pg_receivewal归档的日志，可能还没有归档命令归档的多。归档是到本地，如果服务器挂了，那么归档也没有了，可以使用pgbackrest或者rync工具将归档同步到其他服务器。

用更加精炼的话说：

1）避免单点故障。这点比较鸡肋，某些高级的高可用工具本身就能解决单点故障。

2）用来做日志归档，归档到块级别，且能归档到远端。比archive_command要更加多（默认归档是完成一个wal文件才归档一个，还未写满的wal是不会归档的）。主从之间保持sync模式，那没必要用pg_receivewal来归档；如果是async模式追求性能，pg_receivewal也不能配成同步来影响性能啊（让步：将pg_receivewal放到本地，这样减少网络传输开销，保持同步，能否接受，答案是同步下性能还是很差，而且差距很大，下面是测试数据）。

postgres=# select * from pg_stat_replication ;
pid   | usesysid | usename  | application_name | client_addr | client_hostname | client_port |         backend_start         | backend_xmin |   state   |  sent_lsn  | write_lsn  | flush_lsn  | replay_lsn |    write_lag    |    flush_lag    |   replay_lag    | sync_priority | sync_state
--------+----------+----------+------------------+-------------+-----------------+-------------+-------------------------------+--------------+-----------+------------+------------+------------+------------+-----------------+-----------------+-----------------+---------------+------------
 745643 |       10 | postgres | pg_receivewal    | ::1         |                 |       58465 | 2019-05-16 03:53:06.628954+00 |              | streaming | 1/CB059FC8 | 1/CB059FC8 | 1/CB059FC8 |            | 00:00:00.191089 | 00:00:00.191089 | 00:00:32.187898 |             1 | sync
 668818 |       10 | postgres | walreceiver      | 10.9.10.203 |                 |       42007 | 2019-05-16 03:37:21.983067+00 |              | streaming | 1/CB059FC8 | 1/CB059FC8 | 1/CB059FC8 | 1/CB059FC8 |                 |                 |                 |             0 | async
(2 rows)

(END)

在本地同步性能:

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf /dbaas/pg]$ pgbench -c 10 -j 5 -r -T 100 pgbench
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 100
query mode: simple
number of clients: 10
number of threads: 5
duration: 100 s
number of transactions actually processed: 17313
latency average = 57.785 ms
tps = 173.054591 (including connections establishing)
tps = 173.066373 (excluding connections establishing)
statement latencies in milliseconds:
         0.002  \set aid random(1, 100000 * :scale)
         0.001  \set bid random(1, 1 * :scale)
         0.001  \set tid random(1, 10 * :scale)
         0.000  \set delta random(-5000, 5000)
         0.048  BEGIN;
         0.268  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         0.111  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         0.457  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         2.694  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         0.092  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
        53.995  END;

异地同步性能更低：

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf /dbaas/pg]$ pgbench -c 10 -j 5 -r -T 100 pgbench
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 100
query mode: simple
number of clients: 10
number of threads: 5
duration: 100 s
number of transactions actually processed: 14289
latency average = 70.130 ms
tps = 142.591560 (including connections establishing)
tps = 142.601721 (excluding connections establishing)
statement latencies in milliseconds:
         0.002  \set aid random(1, 100000 * :scale)
         0.001  \set bid random(1, 1 * :scale)
         0.001  \set tid random(1, 10 * :scale)
         0.000  \set delta random(-5000, 5000)
         0.045  BEGIN;
         0.244  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         0.111  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         0.388  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         2.975  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         0.091  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
        65.978  END;

异地异步，性能高很多：

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf /dbaas/pg]$ pgbench -c 10 -j 5 -r -T 100 pgbench
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 100
query mode: simple
number of clients: 10
number of threads: 5
duration: 100 s
number of transactions actually processed: 26333
latency average = 37.987 ms
tps = 263.246388 (including connections establishing)
tps = 263.264004 (excluding connections establishing)
statement latencies in milliseconds:
         0.002  \set aid random(1, 100000 * :scale)
         0.001  \set bid random(1, 1 * :scale)
         0.001  \set tid random(1, 10 * :scale)
         0.000  \set delta random(-5000, 5000)
         0.042  BEGIN;
         0.228  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         0.109  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         0.310  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         1.764  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         0.090  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
        35.349  END;

本地异步呢，更快：

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf /dbaas/pg]$ pgbench -c 10 -j 5 -r -T 100 pgbench
starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 100
query mode: simple
number of clients: 10
number of threads: 5
duration: 100 s
number of transactions actually processed: 29572
latency average = 33.821 ms
tps = 295.670234 (including connections establishing)
tps = 295.691390 (excluding connections establishing)
statement latencies in milliseconds:
         0.002  \set aid random(1, 100000 * :scale)
         0.001  \set bid random(1, 1 * :scale)
         0.001  \set tid random(1, 10 * :scale)
         0.000  \set delta random(-5000, 5000)
         0.042  BEGIN;
         0.219  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
         0.108  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
         0.253  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
         1.592  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
         0.091  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
        31.412  END;

接下来要思考的场景：

那如果遇到一个落后很久的备库，想不重建的情况下恢复。有归档日志，我们应该怎么做？

1）先从归档恢复，再建立同步？不行，恢复后，日志线就+1了，不能再和主建立同步了。

2）将归档日志，从库缺少的地方开始，拷贝到主库pg_wal下，再进行启动从库。待验证。

3）直接在从库的recovery.conf文件配置如下，先从归档恢复，再恢复流复制关系，这样最靠谱：

restore_command = 'cp /dbaas/pg/data/pg_receivewal_data/%f "%p"'

standby_mode = on

primary_conninfo = 'host=10.9.10.203 port=5432 user=postgres connect_timeout=60'

recovery_target_timeline = 'latest'

trigger_file = '/dbaas/pg/data/.tfile'