Nginx + Keepalived 高可用集群部署
负载均衡技术对于一个网站尤其是大型网站的web服务器集群来说是至关重要的！做好负载均衡架构，可以实现故障转移和高可用环境，避免单点故障，保证网站健康持续运行。在使用 Nginx 做反向代理或者负载均衡的时候，都是以 Nginx 为入口，如果 Nginx 宕机了，那么所有的服务都无法正常提供，影响非常严重。
为了避免负载均衡服务器宕机造成严重影响，就需要建立一个备份机。主服务器和备份机上都运行高可用（High Availability）监控程序，通过传送诸如“I am alive”这样的信息来监控对方的运行状况。当备份机不能在一定的时间内收到这样的信息时，它就接管主服务器的服务IP并继续提供负载均衡服务；当备份管理器又从主管理器收到“I am alive”这样的信息时，它就释放服务IP地址，这样的主服务器就开始再次提供负载均衡服务。
高可用（High Availability）是分布式系统架构设计中必须考虑的因素之一，它通常是指，通过设计减少系统不能提供服务的时间。如果一个系统能够一直提供服务，那么这个可用性则是百分之百，但是我们不能保证一个系统能永远不出问题，所以我们只能通过设计来尽可能的去减少由于系统的故障所造成的影响。
由于业务扩展，网站的访问量不断加大，负载越来越高。现需要在web前端放置nginx负载均衡,同时结合keepalived对前端nginx实现HA高可用。
前文分享了《Linux下Nginx基础应用》，《Linux下实现高可用软件-Keepalived基础知识梳理》；今天简单分享Nginx + Keepalived 高可用集群部署。
主从集群架构图
环境说明
| hostname | ip | 说明 |
| Client-01 | 172.16.70.171 | 客户端测试机 |
| KeepMaster | 172.16.70.181 | keepalived 主服务器 (nginx 主负载均衡器) |
| KeepBackup | 172.16.70.182 | keepalived 备服务器 (nginx 备负载均衡器) |
| VIP | 172.16.70.183 | vrrp HA 虚拟地址，可有多个IP |
| Web1 | 172.16.70.191 | 后端web主服务器1 (nginx 站点) |
| Web2 | 172.16.70.192 | 后端web备服务器2 (nginx 站点) |
准备环境
# 本次所有部署服务器都配置
# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
# uname -r
3.10.0-1160.83.1.el7.x86_64 # systemctl stop firewalld
# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/sysconfig/selinux
# setenforce 0
# ntpdate 0.centos.pool.ntp.org
# yum install net-tools vim wget curl -y
搭建后端web服务器
- 主备一样操作
# 这里以 web-01 为例
[root@web-01 ~]# wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~]# rpm -ivh nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~]# nginx -v
nginx version: nginx/1.20.2 [root@web-01 ~]# #echo "`hostname` `ifconfig ens33 |sed -n 's#.*inet \(.*\)netmask.*#\1#p'`" > /usr/share/nginx/html/index.html
[root@web-01 ~]# cat /usr/share/nginx/html/index.html
web-01 172.16.70.191 [root@web-01 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful [root@web-01 ~]# systemctl start nginx
[root@web-01 ~]# systemctl enable nginx
[root@web-01 ~]# netstat -ntupl | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 10687/nginx: master
[root@web-01 ~]# ps -ef | grep nginx
root 10687 1 0 16:36 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx 10688 10687 0 16:36 ? 00:00:00 nginx: worker process
nginx 10689 10687 0 16:36 ? 00:00:00 nginx: worker process
nginx 10690 10687 0 16:36 ? 00:00:00 nginx: worker process
nginx 10691 10687 0 16:36 ? 00:00:00 nginx: worker process
root 10761 10586 0 16:45 pts/1 00:00:00 grep --color=auto nginx # Client-01 测试访问
[root@Client-01 ~]# curl 172.16.70.191
web-01 172.16.70.191
浏览器测试访问 http://ip/
Keep服务器上部署nginx负载均衡器
- 主备一样操作
# 这里以 KeepMaster 为例
# 安装部署nginx
[root@KeepMaster ~]# wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~]# rpm -vih nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~]# nginx -v
nginx version: nginx/1.18.0 # 新建
[root@KeepMaster ~]# cat /etc/nginx/conf.d/web.conf
upstream web {
server 172.16.70.191:80 weight=1 max_fails=3 fail_timeout=20s;
server 172.16.70.192:80 weight=2 max_fails=3 fail_timeout=20s;
}
# weight(权重)和访问比率成正比，默认值为1
# max_fails 为允许失败的次数，默认值为1
# fail_timeout 当max_fails次失败后，暂停将请求分发到该后端服务器的时间 server {
listen 80;
server_name www.zhangwencheng.org; location / {
proxy_pass http://web;
proxy_set_header HOST $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
} [root@KeepMaster ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful [root@KeepMaster ~]# systemctl start nginx
[root@KeepMaster ~]# systemctl enable nginx [root@KeepMaster ~]# ps -ef | grep nginx
root 1677 1 0 17:28 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx 1678 1677 0 17:28 ? 00:00:00 nginx: worker process
root 1708 1444 0 17:33 pts/0 00:00:00 grep --color=auto nginx
[root@KeepMaster ~]# netstat -tnpl | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1677/nginx: master
测试机Client-01验证负载均衡
- Keep主备服务器上的nginx负载均衡
# 在测试机上添加host解析, KeepMaster/KeepBackup主机IP
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.181 www.zhangwencheng.org
172.16.70.182 www.zhangwencheng.org # 测试时候轮流关闭Keep节点，关闭后还是能够访问并看到轮循效果即表示nginx负载均衡器集群搭建成功。
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
Keep服务器上部署keepalived
- 主备一样操作
[root@KeepMaster ~]# yum install -y openssl openssl-devel libnl libnl-devel gcc
[root@KeepMaster ~]# mkdir /data/apps/keepalived -p
[root@KeepMaster ~]# wget --no-check-certificate http://www.keepalived.org/software/keepalived-2.2.4.tar.gz
[root@KeepMaster ~]# tar -xf keepalived-2.2.4.tar.gz
[root@KeepMaster ~]# cd keepalived-2.2.4
[root@KeepMaster keepalived-2.2.4]# ls
aclocal.m4 autogen.sh build-aux ChangeLog configure.ac COPYING INSTALL keepalived.spec.in m4 Makefile.in snap tools
AUTHOR bin_install build_setup configure CONTRIBUTORS doc keepalived lib Makefile.am README.md TODO [root@KeepMaster keepalived-2.2.4]# ./configure --prefix=/data/apps/keepalived
....
....
# 最后编译正常输出如下
Keepalived configuration
------------------------
Keepalived version : 2.2.4
Compiler : gcc gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Preprocessor flags : -D_GNU_SOURCE
Compiler flags : -g -g -O2 -Wextra -Wunused -Wstrict-prototypes -Wabi -Wbad-function-cast -Wcast-align -Wcast-qual -Wdisabled-optimization -Wdouble-promotion \
-Wfloat-equal -Wframe-larger-than=5120 -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical-op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs \
-Wmissing-prototypes -Wnested-externs -Wold-style-definition -Woverlength-strings -Wpointer-arith -Wredundant-decls -Wshadow -Wstack-protector -Wstrict-overflow=4 -Wsuggest-attribute=format \
-Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wsync-nand -Wtrampolines -Wundef -Wuninitialized -Wunknown-pragmas -Wunsafe-loop-optimizations -Wunsuffixed-float-constants -Wvariadic-macros \
-Wwrite-strings -fPIE -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -O2
Linker flags : -pie -Wl,-z,relro -Wl,-z,now
Extra Lib : -lm -lcrypto -lssl -lnl
Use IPVS Framework : Yes
IPVS use libnl : Yes
IPVS syncd attributes : No
IPVS 64 bit stats : No
HTTP_GET regex support : No
fwmark socket support : Yes
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
Use VRRP authentication : Yes
With track_process : Yes
With linkbeat : Yes
Use BFD Framework : No
SNMP vrrp support : No
SNMP checker support : No
SNMP RFCv2 support : No
SNMP RFCv3 support : No
DBUS support : No
Use JSON output : No
libnl version : 1
Use IPv4 devconf : No
Use iptables : No
Use nftables : No
init type : systemd
systemd notify : No
Strict config checks : No
Build documentation : No
Default runtime options : -D [root@KeepMaster keepalived-2.2.4]# make -j 4 && make install
[root@KeepMaster keepalived-2.2.4]# ls
aclocal.m4 bin build_setup config.status CONTRIBUTORS INSTALL keepalived.spec.in Makefile README TODO
AUTHOR bin_install ChangeLog configure COPYING keepalived lib Makefile.am README.md tools
autogen.sh build-aux config.log configure.ac doc keepalived.spec m4 Makefile.in snap [root@KeepMaster keepalived-2.2.4]# cp keepalived/keepalived /usr/local/sbin/ -a
[root@KeepMaster keepalived-2.2.4]# keepalived -v
Keepalived v2.2.4 (08/21,2021) Copyright(C) 2001-2021 Alexandre Cassen, <acassen@gmail.com> Built with kernel headers for Linux 3.10.0
Running on Linux 3.10.0-1160.83.1.el7.x86_64 #1 SMP Wed Jan 25 16:41:43 UTC 2023
Distro: CentOS Linux 7 (Core) configure options: --prefix=/data/apps/keepalived Config options: LVS VRRP VRRP_AUTH VRRP_VMAC OLD_CHKSUM_COMPAT INIT=systemd System options: VSYSLOG LIBNL1 RTA_ENCAP RTA_EXPIRES RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTA_VIA IFA_FLAGS \
NET_LINUX_IF_H_COLLISION LIBIPTC_LINUX_NET_IF_H_COLLISION LIBIPVS_NETLINK IFLA_LINK_NETNSID GLOB_BRACE GLOB_ALTDIRFUNC INET6_ADDR_GEN_MODE SO_MARK [root@KeepMaster keepalived-2.2.4]# cd /data/apps/keepalived/
[root@KeepMaster keepalived]# ls
bin etc sbin share [root@KeepMaster keepalived]# mv etc/keepalived/keepalived.conf etc/keepalived/keepalived.conf_bak
[root@KeepMaster keepalived]# vim etc/keepalived/keepalived.conf
! Configuration File for keepalived global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
} vrrp_script chk_nginx {
script "/data/apps/keepalived/chk_nginx.sh"
interval 2
weight -5
} vrrp_instance VI_1 {
state MASTER # 备服务器这为 BACKUP
interface ens33
virtual_router_id 51
priority 110 # 备服务器这小于110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.183
}
track_script {
chk_nginx
}
}
优先级不会不断的提高或者降低，最终优先级的范围是在[1,254]，不会出现优先级小于等于0或者优先级大于等于255的情况。
在MASTER节点的vrrp_instance中配置nopreempt，当它异常恢复后，即使它prio更高也不会抢占，这样可以避免正常情况下做无谓的切换。
nginx检测脚本
编写脚本来判断本机nginx是否正常，如果发现NginX不正常，自重启nginx。等待2秒再次校验，仍然失败则不再尝试，关闭keepalived，让其他主机此时会接管VIP。
此脚本必须在keepalived服务运行的前提下才有效！如果在keepalived服务先关闭的情况下，那么nginx服务关闭后就不能实现自启动了。
[root@KeepMaster keepalived]# cat chk_nginx.sh
#!/bin/bash
chk=$(ps -C nginx --no-heading|wc -l)
if [ "${chk}" = "0" ]; then
systemctl start nginx
sleep 2
chk=$(ps -C nginx --no-heading|wc -l)
if [ "${chk}" = "0" ]; then
systemctl stop keepalived
fi
fi [root@KeepMaster keepalived]# chmod +x chk_nginx.sh
测试机Client-01验证VIP
- Keep服务器上的VIP
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.183 www.zhangwencheng.org [root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
故障转移测试
#手动关闭Master机器上的nginx服务，最多2秒钟后就会自启动
[root@KeepMaster ~]# systemctl stop nginx
[root@KeepMaster ~]# ps -ef | egrep 'nginx|keepalived'
root 57266 1 0 16:21 ? 00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root 57267 57266 0 16:21 ? 00:00:01 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root 60019 1 0 16:42 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx 60020 60019 0 16:42 ? 00:00:00 nginx: worker process
root 60027 1444 0 16:42 pts/0 00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.183/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever #手动关闭Master机器上的keepalived服务，已经发现没VIP了
[root@KeepMaster ~]# systemctl stop keepalived
[root@KeepMaster ~]# ps -ef | egrep 'nginx|keepalived'
root 60019 1 0 16:42 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx 60020 60019 0 16:42 ? 00:00:00 nginx: worker process
root 60348 1444 0 16:45 pts/0 00:00:00 grep -E --color=auto nginx|keepalived
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever [root@KeepMaster keepalived]# tail /var/log/messages
Mar 7 17:01:01 Keepalived-01 systemd: Started Session 27 of user root.
Mar 7 17:01:28 Keepalived-01 Keepalived[60703]: Stopping
Mar 7 17:01:28 Keepalived-01 systemd: Stopping LVS and VRRP High Availability Monitor...
Mar 7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) sent 0 priority
Mar 7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) removing VIPs.
Mar 7 17:01:28 Keepalived-01 NetworkManager[570]: <info> [1678179688.0176] policy: set-hostname: current hostname was changed outside NetworkManager: 'KeepMaster'
Mar 7 17:01:29 Keepalived-01 Keepalived_vrrp[60704]: Stopped - used (self/children) 0.005506/0.797936 user time, 0.172766/0.818969 system time
Mar 7 17:01:29 Keepalived-01 Keepalived[60703]: CPU usage (self/children) user: 0.000000/0.803442 system: 0.001394/0.994146
Mar 7 17:01:29 Keepalived-01 Keepalived[60703]: Stopped Keepalived v2.2.4 (08/21,2021)
Mar 7 17:01:29 Keepalived-01 systemd: Stopped LVS and VRRP High Availability Monitor
- KeepBackup查看，已经接管VIP
[root@KeepBackup keepalived]# ps -ef | egrep 'nginx|keepalived'
root 65036 1 0 17:01 ? 00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root 65037 65036 0 17:01 ? 00:00:00 /data/apps/keepalived/sbin/keepalived -f /data/apps/keepalived/etc/keepalived/keepalived.conf -D
root 65067 1 0 17:01 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx 65068 65067 0 17:01 ? 00:00:00 nginx: worker process
root 65122 1514 0 17:01 pts/0 00:00:00 grep -E --color=auto nginx|keepalived [root@KeepBackup keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
inet 172.16.70.182/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.183/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
[root@KeepBackup keepalived]# tail /var/log/messages
Mar 7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:28 Keepalived-02 NetworkManager[573]: <info> [1678179688.8137] policy: set-hostname: current hostname was changed outside NetworkManager: 'KeepBackup'
Mar 7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: (VI_1) Sending/queueing gratuitous ARPs on ens33 for 172.16.70.183
Mar 7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
Mar 7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33 for 172.16.70.183
双机高可用一般是通过虚拟IP（飘移IP）方法来实现的，基于Linux/Unix的IP别名技术，目前分为两种：
- 双机主从模式：即前端使用两台服务器，一台主服务器和一台热备服务器，正常情况下，主服务器绑定一个公网虚拟IP，提供负载均衡服务，热备服务器处于空闲状态；当主服务器发生故障时，热备服务器接管主服务器的公网虚拟IP，提供负载均衡服务；但是热备服务器在主机器不出现故障的时候，永远处于浪费状态，对于服务器不多的网站，该方案不经济实惠。
- 双机主主模式：即前端使用两台负载均衡服务器，互为主备，且都处于活动状态，同时各自绑定一个公网虚拟IP，提供负载均衡服务；当其中一台发生故障时，另一台接管发生故障服务器的公网虚拟IP（这时由非故障机器一台负担所有的请求）。这种方案，经济实惠，非常适合于当前架构环境。
主主模式集群架构图
当了解主备模式后，双主模式就容易配置多了。只需要在每台keepalived配置文件，加上一个vrrp_instance命名vrrp_instance VI_2即可，更改几个参数，设置另一个VIP：172.16.70.184
- KeepMaster：state BACKUP ，priority 100， virtual_router_id 52
- KeepBackup：state MASTER ，priority 110， virtual_router_id 52
KeepMaster上的keepalived.conf
[root@KeepMaster keepalived]# cat etc/keepalived/keepalived.conf
! Configuration File for keepalived global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
} vrrp_script chk_nginx {
script "/data/apps/keepalived/chk_nginx.sh"
interval 2
weight -5
} vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.183
}
track_script {
chk_nginx
}
} vrrp_instance VI_2 {
state BACKUP
interface ens33
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.184
}
track_script {
chk_nginx
}
} [root@KeepMaster keepalived]# systemctl restart keepalived
KeepBackup上的keepalived.conf
[root@KeepBackup keepalived]# cat etc/keepalived/keepalived.conf
! Configuration File for keepalived global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
} vrrp_script chk_nginx {
script "/data/apps/keepalived/chk_nginx.sh"
interval 2
weight -5
} vrrp_instance VI_1 {
state BAKCUP
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.183
}
track_script {
chk_nginx
}
} vrrp_instance VI_2 {
state MASTER
interface ens33
virtual_router_id 52
priority 110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.70.184
}
track_script {
chk_nginx
}
} [root@KeepBackup keepalived]# systemctl restart keepalived
测试机Client-01验证VIP2
- Keep服务器上的VIP2
# 注释VIP1解析，此时仅测试VIP2；测试成功后再取消VIP1注释
[root@Client-01 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
#172.16.70.183 www.zhangwencheng.org
172.16.70.184 www.zhangwencheng.org [root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
[root@Client-01 ~]# curl www.zhangwencheng.org
web-02 172.16.70.192
[root@Client-01 ~]# curl www.zhangwencheng.org
web-01 172.16.70.191
- 查看VIP情况
[root@KeepMaster ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
inet 172.16.70.181/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.183/32 scope global ens33 # VIP1
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@KeepBackup ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
inet 172.16.70.182/24 brd 172.16.70.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet 172.16.70.184/32 scope global ens33 # VIP2
valid_lft forever preferred_lft forever
inet6 fe80::7726:d409:2cf4:babd/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::833:43b:7d2:6e4c/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::c2be:590b:1ae6:42e3/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
Nginx + Keepalived 高可用集群部署的更多相关文章
- Nginx+Keepalived高可用集群应用实践
Nginx+Keepalived高可用集群应用实践 1.Keepalived高可用软件 1.1 Keepalived服务的三个重要功能 1.1.1管理LVS负载均衡软件 早期的LVS软件,需要通过命令 ...
- Nginx入门篇（七）之Nginx+keepalived高可用集群
一.keepalived介绍 keepalived软件最开始是转为负载均衡软件LVS而设计,用来管理和监控LVS集群系统中各个服务节点的状态,后来又加入了可实现高可用的VRRP功能.所以Keepali ...
- Keepalived高可用集群部署
KeepAlived 目录 KeepAlived KeepAlived安装 KeepAlived部署 准备工作 主备模式 节点配置 验证 正常状态 故障 故障恢复 1+N(一主多备)模式 节点配置 验 ...
- nginx Keepalived高可用集群
一.Keepalived高可用 1.简介 Keepalived软件起初是专为LvS负载均衡软件设计的,用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能.因此, ...
- Keepalived高可用集群应用
Keepalived高可用集群应用 1.keepalived服务说明 1.1.keepalived介绍 Keepalived是一个用C语言编写的路由软件.该项目的主要目标是为Linux系统和基于Lin ...
- 分布式FastDfs+nginx缓存高可用集群构建
介绍: FastDFS:开源的高性能分布式文件系统:主要功能包括:文件存储,文件同步和文件访问,以及高容量和负载平衡 FastDFS:角色:跟踪服务器(Tracker Server).存储服务器(St ...
- Haproxy+keepalived高可用集群实战
1.1 Haproxy+keepalived高可用集群实战 随着互联网火热的发展,开源负载均衡器的大量的应用,企业主流软件负载均衡如LVS.Haproxy.Nginx等,各方面性能不亚于硬件负载均衡 ...
- Nginx 笔记（四）nginx 原理与优化参数配置 与 nginx 搭建高可用集群
个人博客网:https://wushaopei.github.io/ (你想要这里多有) 一.nginx 原理与优化参数配置 master-workers 的机制的好处 首先,对于每个 ...
- hbase高可用集群部署（cdh）
一.概要 本文记录hbase高可用集群部署过程,在部署hbase之前需要事先部署好hadoop集群,因为hbase的数据需要存放在hdfs上,hadoop集群的部署后续会有一篇文章记录,本文假设had ...
- （十）RabbitMQ消息队列-高可用集群部署实战
原文:(十)RabbitMQ消息队列-高可用集群部署实战 前几章讲到RabbitMQ单主机模式的搭建和使用,我们在实际生产环境中出于对性能还有可用性的考虑会采用集群的模式来部署RabbitMQ. Ra ...
随机推荐
- Python 封装cmd 执行命令
1.利用shell中执行成功返回0 失败非零 封装成函数 # coding: utf-8 from subprocess import Popen, PIPE, STDOUT import sys ...
- 使用骨架创建maven的web工程 maven工程servlet实例之间指定web资源包 实例之导入项目依赖的jar包 maven工程servlet实例之jar包冲突解决
使用骨架创建maven的web工程 操作一样把勾进行勾选找到webapp选项 下一步创建就行 创建好会有显示 目录结构其余的都需要自己手动补齐 有小蓝点的才是一个web项目, 如果上面都正确的话没有 ...
- iOS block相关面试题
一.前言本文重点来研究一下 objc 的 block,并具体来分析一下以下一些面试题目:block 的内部实现,结构体是什么样?block 是类吗?有哪些类型?一个 int 变量被 __block 修 ...
- 手动导入jar
mvn install:install-file -Dfile=D:\java_tools\maven\maven-repository\cn\afterturn\easypoi-base-cy\4. ...
- leetcode-560-和为 K 的子数组
给你一个整数数组 nums 和一个整数 k ,请你统计并返回 该数组中和为 k 的连续子数组的个数 . 示例 1: 输入:nums = [1,1,1], k = 2 输出:2 示例 2: 输入:num ...
- 项目实训DAY 11-12 学习
在神经网络可视化工具中,选择了三种,NNSVG,PlotNeuralNet,GraphCore 前两者应该比较好实现,例子都跑通了,对于定制的代码读起来也不难.PNN的示例图如下 最后一个虽然有实例图 ...
- chatGpt启示
在应用软件领域实现自动化编程,我现在确实不怀疑了,只是还要等几年的问题.所以我的建议是:如果还想撸代码,可以往基础软件方向走,或者直接去大公司做 AI 工具:不想搞软件了,就去多了解业务吧.早做准备早 ...
- 内存模型和名称空间复习题(c++ prime plus )
第一题 注意到setgolf()被重载,可以这样使用其第一个版本: golf ann; setgolf(ann, "Ann Birdfree", 24); 上述函数调用提供了存储在 ...
- node 版本管理
32位版本的node,运行较大的项目,会内存溢出.所以建议安装64位的版本,且运行速度比32位快.node14以下的版本支持node-sass,版本node16以上的不再支持node-sass,而sa ...
- Jmeter一、开源软件的崛起
一.jmeter自身特点: 1.开源,轻量级,更适合自动化和持续集成. 2.学习难度大. 3.资料少.多英文. 二.性能测试工具选型的原则 1.成本: a.工具成本 b.学习成本 2.通信协议: a. ...