postgresql双机热备、高可用方案（采用pacemaker+corosync实现）
http://blog.csdn.net/qguanri/article/details/51151974
需求描述
我们有两台centos7的数据库主机A、B。要对A、B实现双机热备，A作为数据库master主机，对外提供读写服务，B作为slave主机能实时同步A的数据。当A发生故障时，B从slave状态切换到master状态，以保证pg数据库能正常对外提供服务。
经过调研，最终我们给出的解决方案是采用pacemaker和corosync组件来实现postgresql的高可用。
所有部署脚本可以下github中找到：https://github.com/qinguanri/pg-ha
环境说明
- 操作系统centos7：
[root@10 ~]# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
[root@10 ~]# uname -a
Linux 10.16.93.179 3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux- 1
- 2
- 3
- 4
postgresql版本：9.2.15
- 2台主机：
| 角色 | IP |
|---|---|
| node1 | 10.16.93.179 |
| node2 | 10.16.93.182 |
- 分配2个虚IP。我们的数据库对外提供服务时，暴露的是虚IP：
vip-master: 10.16.93.180
vip-slave: 10.16.93.181
配置Linux集群环境
注意：请严格按照下文所给的顺序进行配置
1.安装Pacemaker和Corosync包
需要在所有节点执行：
$ yum install -y pacemaker pcs psmisc policycoreutils-python postgresql-server- 1
2.禁用防火墙
在所有节点执行：
$ systemctl disable firewalld.service
$ systemctl stop firewalld.service- 1
- 2
3.启用pcs
在所有节点执行：
$ systemctl start pcsd.service
$ systemctl enable pcsd.service
ln -s '/usr/lib/systemd/system/pcsd.service' '/etc/systemd/system/multi-user.target.wants/pcsd.service'
$ echo hacluster | passwd hacluster --stdin
Changing password for user hacluster.
Changing password for user hacluster.
passwd: all authentication tokens updated successfully.- 1
- 2
- 3
- 4
- 5
- 6
- 7
4.集群认证
在任何一个节点上执行，这里选择node1：
$ pcs cluster auth -u hacluster -p hacluster 10.16.93.179 10.16.93.182
10.16.93.179: Authorized
10.16.93.182: Authorized- 1
- 2
- 3
5.同步配置
在node1上执行：
$ pcs cluster setup --last_man_standing=1 --name pgcluster 10.16.93.179 10.16.93.182
Shutting down pacemaker/corosync services...
Redirecting to /bin/systemctl stop pacemaker.service
Redirecting to /bin/systemctl stop corosync.service
Killing any remaining services...
Removing all cluster configuration files...
10.16.93.179: Succeeded
10.16.93.182: Succeeded- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
6.启动集群
在node1上执行。（有时需要稍等十几秒，才能看到命令的输出结果）：
$ pcs cluster start --all
172.17.5.90: Starting Cluster...
172.17.5.91: Starting Cluster...
172.17.5.92: Starting Cluster...- 1
- 2
- 3
- 4
7.检验
1）检验corosync
在node1上执行：
$ pcs status corosync
Membership information
----------------------
Nodeid Votes Name
1 1 10.16.93.179 (local)
2 1 10.16.93.182- 1
- 2
- 3
- 4
- 5
- 6
2）检验pacemaker
root@10 data]# pcs status
Cluster name: pgcluster
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Last updated: Mon Apr 4 23:45:48 2016 Last change: Fri Apr 1 09:17:03 2016 by root via crm_attribute on tsung_test
Stack: corosync
Current DC: tsung_test (version 1.1.13-10.el7_2.2-44eb2dd) - partition with quorum
2 nodes and 5 resources configured
Online: [ 10.16.93.179 tsung_test ]
Full list of resources:
Master/Slave Set: pgsql-cluster [pgsql]
Masters: [ tsung_test ]
Slaves: [ 10.16.93.179 ]
Resource Group: master-group
vip-master (ocf::heartbeat:IPaddr2): Started tsung_test
Resource Group: slave-group
vip-slave (ocf::heartbeat:IPaddr2): Started 10.16.93.179
PCSD Status:
10.16.93.179: Online
tsung_test (10.16.93.182): Online
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
注意：当vip-master和vip-slave都是Started状态时，集群节点配置正确。否则，请检查你的配置是否有误
安装和配置PostgreSQL
1.创建数据库目录
需要在所有节点上执行：
$ mkdir -p /data/postgresql/{data,xlog_archive}
$ chown -R postgres:postgres /data/postgresql/
$ chmod 0700 /data/postgresql/data- 1
- 2
- 3
2.初始化db
在node1上执行：
$ su - postgres
$ initdb -D /data/postgresql/data/
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
fixing permissions on existing directory /data/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in /data/postgresql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
postgres -D /data/postgresql/data
or
pg_ctl -D /data/postgresql/data -l logfile start- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
3.修改配置文件
在node1上执行：
$ vim /data/postgresql/data/postgresql.conf
listen_addresses = '*'
wal_level = hot_standby
synchronous_commit = on
archive_mode = on
archive_command = 'cp %p /data/postgresql/xlog_archive/%f'
max_wal_senders=5
wal_keep_segments = 32
hot_standby = on
restart_after_crash = off
replication_timeout = 5000
wal_receiver_status_interval = 2
max_standby_streaming_delay = -1
max_standby_archive_delay = -1
synchronous_commit = on
restart_after_crash = off
hot_standby_feedback = on
$ vim /data/postgresql/data/pg_hba.conf
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
host all all 10.16.0.0/16 md5
host all all 10.18.0.0/16 md5
host replication all 10.16.0.0/16 md5- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
注意：pg_hba.conf文件用于配置postgresql的访问控制，例如：host all all 10.16.0.0/16 md5,表示允许10.16.0.0/16网段的主机访问所有用户、所有数据库，密码使用md5密文的方式。你需要根据你的实际网段进行设置。当出现访问权限错误时，请检查你的pg_hba.conf文件
4.启动
启动数据库，创建replicator用户。将用户postgres的密码修改为postgres（pg初始安装时，postgres用户的登录密码默认为空）
在node1上执行：
$ pg_ctl -D /data/postgresql/data/ start
server starting
[ 2015-10-16 08:51:31.451 UTC 53158 5620ba93.cfa6 1 0]LOG: redirecting log output to logging collector process
[ 2015-10-16 08:51:31.451 UTC 53158 5620ba93.cfa6 2 0]HINT: Future log output will appear in directory "pg_log".
$ psql -U postgres
psql (9.2.13)
Type "help" for help.
postgres=# create role replicator with login replication password '8d5e9531-3817-460d-a851-659d2e51ca99';
CREATE ROLE
postgres=# alter user postgres with password 'postgres';
postgres=# \q- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
5.制作slave
在node2上执行：
$ su - postgres
$ pg_basebackup -h 10.16.93.179 -U postgres -D /data/postgresql/data/ -X stream -P
could not change directory to "/home/wenhang.pan"
20127/20127 kB (100%), 1/1 tablespace
$ vim /data/postgresql/data/recovery.conf
standby_mode = 'on'
primary_conninfo = 'host=10.16.93.179 port=5432 user=replicator application_name=zhaopin-5-92 password=8d5e9531-3817-460d-a851-659d2e51ca99 keepalives_idle=60 keepalives_in
terval=5 keepalives_count=5'
restore_command = 'cp /data/postgresql/xlog_archive/%f %p'
recovery_target_timeline = 'latest'- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
6.启动slave
在node2上执行：
$ pg_ctl -D /data/postgresql/data/ start
pg_ctl: another server might be running; trying to start server anyway
server starting
-bash-4.2$ LOG: database system was interrupted while in recovery at log time 2015-10-16 08:19:07 GMT
HINT: If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.
LOG: entering standby mode
LOG: redo starts at 0/3000020
LOG: consistent recovery state reached at 0/30000E0
LOG: database system is ready to accept read only connections
LOG: streaming replication successfully connected to primary- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
7.查看集群状态
在node1上执行一下命令。这个时候，可看到node2是一个replication。同步状态为sync
[root@10 ~]# psql -U postgres
psql (9.2.15)
Type "help" for help.
postgres=# select * from pg_stat_replication ;
pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start
| state | sent_location | write_location | flush_location | replay_location | sync_priority | sync_state
-------+----------+------------+------------------+--------------+-----------------+-------------+-----------------------
--------+-----------+---------------+----------------+----------------+-----------------+---------------+------------
17349 | 16384 | replicator | 10.16.93.182 | 10.16.93.182 | | 22690 | 2016-04-14 14:07:25.18
1297+08 | streaming | 0/9014508 | 0/9014508 | 0/9014508 | 0/9014508 | 1 | sync
(1 row)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
8.停止PostgreSQL服务
在node1、node2上执行：
$ pg_ctl -D /data/postgresql/data/ -mi stop
waiting for server to shut down.... done
server stopped- 1
- 2
- 3
注意：在配置自动切换前，一定要先把node1、node2上的PostgreSQL服务停掉
配置自动切换
上面的配置完成了数据库的初始化。下面进行自动切换相关的配置。当发生故障时，master/slave身份实现自动切换。
1.配置
在node1执行：
1）配置项较多，我们先将配置步骤写到脚本里
[root@10 ~]# vim cluster_setup.sh
# 将cib配置保存到文件
pcs cluster cib pgsql_cfg
# 在pacemaker级别忽略quorum
pcs -f pgsql_cfg property set no-quorum-policy="ignore"
# 禁用STONITH
pcs -f pgsql_cfg property set stonith-enabled="false"
# 设置资源粘性，防止节点在故障恢复后发生迁移
pcs -f pgsql_cfg resource defaults resource-stickiness="INFINITY"
# 设置多少次失败后迁移
pcs -f pgsql_cfg resource defaults migration-threshold="3"
# 设置master节点虚ip
pcs -f pgsql_cfg resource create vip-master IPaddr2 ip="10.16.93.180" cidr_netmask="24" op start timeout="60s" interval="0s" on-fail="restart" op monitor timeout="60s" interval="10s" on-fail="restart" op stop timeout="60s" interval="0s" on-fail="block"
# 设置slave节点虚ip
pcs -f pgsql_cfg resource create vip-slave IPaddr2 ip="10.16.93.181" cidr_netmask="24" op start timeout="60s" interval="0s" on-fail="restart" op monitor timeout="60s" interval="10s" on-fail="restart" op stop timeout="60s" interval="0s" on-fail="block"
# 设置pgsql集群资源
# pgctl、psql、pgdata和config等配置根据自己的环境修改,node list填写节点的hostname，master_ip填写虚master_ip
pcs -f pgsql_cfg resource create pgsql pgsql pgctl="/usr/bin/pg_ctl" psql="/usr/bin/psql" pgdata="/data/postgresql/data/" config="/data/postgresql/data/postgresql.conf" rep_mode="sync" node_list="tsung_test 10.16.93.179" master_ip="10.16.93.180" repuser="replicator" primary_conninfo_opt="password=8d5e9531-3817-460d-a851-659d2e51ca99 keepalives_idle=60 keepalives_interval=5 keepalives_count=5" restore_command="cp /data/postgresql/xlog_archive/%f %p" restart_on_promote='true' op start timeout="60s" interval="0s" on-fail="restart" op monitor timeout="60s" interval="4s" on-fail="restart" op monitor timeout="60s" interval="3s" on-fail="restart" role="Master" op promote timeout="60s" interval="0s" on-fail="restart" op demote timeout="60s" interval="0s" on-fail="stop" op stop timeout="60s" interval="0s" on-fail="block"
# 设置master/slave模式，clone-max=2，两个节点
pcs -f pgsql_cfg resource master pgsql-cluster pgsql master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
# 配置master ip组
pcs -f pgsql_cfg resource group add master-group vip-master
# 配置slave ip组
pcs -f pgsql_cfg resource group add slave-group vip-slave
# 配置master ip组绑定master节点
pcs -f pgsql_cfg constraint colocation add master-group with master pgsql-cluster INFINITY
# 配置启动master节点
pcs -f pgsql_cfg constraint order promote pgsql-cluster then start master-group symmetrical=false score=INFINITY
# 配置停止master节点
pcs -f pgsql_cfg constraint order demote pgsql-cluster then stop master-group symmetrical=false score=0
# 配置slave ip组绑定slave节点
pcs -f pgsql_cfg constraint colocation add slave-group with slave pgsql-cluster INFINITY
# 配置启动slave节点
pcs -f pgsql_cfg constraint order promote pgsql-cluster then start slave-group symmetrical=false score=INFINITY
# 配置停止slave节点
pcs -f pgsql_cfg constraint order demote pgsql-cluster then stop slave-group symmetrical=false score=0
# 把配置文件push到cib
pcs cluster cib-push pgsql_cfg- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
2）执行操作文件
$ sh cluster_setup.sh- 1
2.查看状态
1）查看cluster状态
在node1上执行：
[root@10 ~]# pcs status
Cluster name: pgcluster
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Last updated: Mon Apr 4 23:15:56 2016 Last change: Fri Apr 1 09:17:03 2016 by root via crm_attribute on tsung_test
Stack: corosync
Current DC: tsung_test (version 1.1.13-10.el7_2.2-44eb2dd) - partition with quorum
2 nodes and 5 resources configured
Online: [ 10.16.93.179 tsung_test ]
Full list of resources:
Master/Slave Set: pgsql-cluster [pgsql]
Masters: [ tsung_test ]
Slaves: [ 10.16.93.179 ]
Resource Group: master-group
vip-master (ocf::heartbeat:IPaddr2): Started tsung_test
Resource Group: slave-group
vip-slave (ocf::heartbeat:IPaddr2): Started 10.16.93.179
PCSD Status:
10.16.93.179: Online
tsung_test (10.16.93.182): Online
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
2）查看PostgreSQL集群状态
在node1上执行：
[root@tsung_test data]# psql -U postgres
psql (9.2.15)
Type "help" for help.
postgres=# select * from pg_stat_replication ;
pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | state | sent_location | write_locatio
n | flush_location | replay_location | sync_priority | sync_state
-------+----------+------------+------------------+--------------+-----------------+-------------+-------------------------------+-----------+---------------+--------------
--+----------------+-----------------+---------------+------------
19415 | 16384 | replicator | 10.16.93.182 | 10.16.93.182 | | 18289 | 2016-04-01 09:17:29.037197-04 | streaming | 0/40F0ED8 | 0/40F0ED8
| 0/40F0ED8 | 0/40F0ED8 | 0 | async
(1 row)- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
验证自动切换
参考文献
- http://clusterlabs.org/wiki/PgSQL_Replicated_Cluster
- http://m.oschina.net/blog/518928
- http://m.oschina.net/blog/519458?mType=Group
postgresql双机热备、高可用方案（采用pacemaker+corosync实现）的更多相关文章
- Mysql+Keepalived双主热备高可用操作记录
我们通常说的双机热备是指两台机器都在运行,但并不是两台机器都同时在提供服务.当提供服务的一台出现故障的时候,另外一台会马上自动接管并且提供服务,而且切换的时间非常短.MySQL双主复制,即互为Mast ...
- 使用Keepalived实现Nginx的自动重启及双主热备高可用
1.概述 之前我们使用Keepalived实现了Nginx服务的双机主备高可用,但是有几个问题没有解决,今天一起探讨一下. 1)在双机主备机制中,Keepalived服务如果宕了,会自动启用备机进行服 ...
- 采用pacemaker+corosync实现postgresql双机热备、高可用方案
环境说明 参照上章已完成postgresql流复制配置,并关闭postgres服务. su - postgres pg_ctl -D /data/postgresql/data/ stop -m fa ...
- mysql+keepalived 双主热备高可用
理论介绍:我们通常说的双机热备是指两台机器都在运行,但并不是两台机器都同时在提供服务.当提供服务的一台出现故障的时候,另外一台会马上自动接管并且提供服务,而且切换的时间非常短.MySQL双主复制,即互 ...
- Redis双机热备方案--转
http://luyx30.blog.51cto.com/1029851/1350832 参考资料: http://patrick-tang.blogspot.com/2012/06/redis-ke ...
- Redis+Keepalived高可用方案详细分析
背景 目前,Redis集群的官方方案还处在开发测试中,未集成到稳定版中.且目前官方开发中的Redis Cluster提供的功能尚不完善(可参考官方网站或http://www.redisdoc.com/ ...
- MySQL高可用方案－PXC环境部署记录
之前梳理了Mysql+Keepalived双主热备高可用操作记录,对于mysql高可用方案,经常用到的的主要有下面三种: 一.基于主从复制的高可用方案:双节点主从 + keepalived 一般来说, ...
- 云数据库Redis版256M双机热备款
云数据库Redis版是兼容Redis协议标准的.提供持久化的缓存式数据库服务,基于高可靠双机热备架构:全新推出的256M小规格款,适用于高QPS.小数据量业务,并支持免费全量迁移,完美服务于个人开发者 ...
- 使用sqlserver搭建高可用双机热备的Quartz集群部署【附源码】
一般拿Timer和Quartz相比较的,简直就是对Quartz的侮辱,两者的功能根本就不在一个层级上,如本篇介绍的Quartz强大的序列化机制,可以序列到 sqlserver,mysql,当然还可以在 ...
随机推荐
- 20145219《网络对抗》MSF基础应用
20145219<网络对抗>MSF基础应用 基础问题回答 用自己的话解释什么是exploit,payload,encode exploit:把实现设置好的东西送到要攻击的主机里. payl ...
- 20145315 《Java程序设计》第五周学习总结
20145315 <Java程序设计>第五周学习总结 教材学习内容总结 第八章 8.1语法与继承架构 8.1.1使用try,catch 所有的错误都会被打包为对象,使用try,catch可 ...
- kernel: swapper: page allocation failure. order:1, mode:0x20
场景:领导电话通知,我们的主站宕机了,到家后从另外一台机器上ssh一直处于等待状态,开始怀疑机器的负载比较高,后查看监控机器,发现网卡.cpu.nginx连接数.....通通都没有数据了,显然不是负载 ...
- [javaScript]身份证号信息解析
之前一直在思考是不是该把工作中一些问题写出来(可能是简单的问题),现在的想法是应该写出来这些简单的问题.虽然工作中可能并没有很多特别难的问题让你去解决,因为公司的招人就是根据你的能力来匹配的嘛. 简单 ...
- 【javascript】数据结构-队列
<!DOCTYPE html> <html> <head> <title>queue</title> <meta charset=&q ...
- awk根据指定的字符串分割字符串
以从字符串"hello-kitty-red-for-you"中获取-for前面的内容为例: echo "hello-kitty-red-for-you" |aw ...
- js的函数作用域
1.js作用域 //在函数内部声明的变量,如果不加var,则自动变成window的成员//预处理:扫描代码,看到var或者函数就生效./*function f(){a = 8;//var a = 8; ...
- 数据库常见的三种join方式
数据库常见的join方式有三种:inner join, left outter join, right outter join(还有一种full join,因不常用,本文不讨论).这三种连接方式都是将 ...
- Symmetric Tree，对称树
问题描述: Given a binary tree, check whether it is a mirror of itself (ie, symmetric around its center). ...
- 这些HTML、CSS知识点，面试和平时开发都需要 No10-No11（知识点：表格操作、代码编写规则）
系列知识点汇总 1.基础篇 这些HTML.CSS知识点,面试和平时开发都需要 No1-No4(知识点:HTML.CSS.盒子模型.内容布局) 这些HTML.CSS知识点,面试和平时开发都需要 No5- ...