Keepalived+LVS(dr)高可用负载均衡集群的实现
一 环境介绍
1.操作系统
CentOS Linux release 7.2.1511 (Core)
2.服务
keepalived+lvs双主高可用负载均衡集群及LAMP应用
keepalived-1.2.13-7.el7.x86_64
ipvsadm-1.27-7.el7.x86_64
httpd-2.4.6-45.el7.centos.x86_64
mariadb-5.5.52-1.el7.x86_64
php-5.4.16-42.el7.x86_64
二 原理及拓扑图
1.vrrp协议
vrrp(Virtual Redundant Routing Protocol)协议:
在现实的网络环境中,两台需要通信的主机大多数情况下并没有直接的物理连接。对于这样的情况,它们之间路由怎样选择?主机如何选定到达目的主机的下一跳路由,这个问题通常的解决方法有两种:
在主机上使用动态路由协议(RIP、OSPF等)
在主机上配置静态路由
很明显,在主机上配置动态路由是非常不切实际的,因为管理、维护成本以及是否支持等诸多问题。配置静态路由就变得十分流行,但路由器(或者说默认网关default gateway)却经常成为单点故障。VRRP的目的就是为了解决静态路由单点故障问题,VRRP通过一竞选(election)协议来动态的将路由任务交给LAN中虚拟路由器中的某台VRRP路由器。
2.keepalived简介
Keepalived 是一个基于VRRP协议来实现的LVS服务高可用方案,可以利用其来避免单点故障。一个LVS服务会有2台服务器运行Keepalived,一台为主服务器(MASTER),一台为备份服务器(BACKUP),但是对外表现为一个虚拟IP,主服务器会发送特定的消息给备份服务器,当备份服务器收不到这个消息的时候,即主服务器宕机的时候,备份服务器就会接管虚拟IP,继续提供服务,从而保证了高可用性。Keepalived是VRRP的完美实现。
3.lvs-dr
Direct Routing,直接路由,通过为请求报文重新封装一个MAC首部进行转发,源MAC是DIP所在的接口的MAC,目标MAC是某挑选出的RS的RIP所在接口的MAC地址;源IP/PORT,以及目标IP/PORT均保持不变。
Director和各RS都得配置使用VIP;
(1) 确保前端路由器将目标IP为VIP的请求报文发往Director;
(a) 在前端网关做静态绑定;
(b) 在RS上使用arptables;
(c) 在RS上修改内核参数以限制arp通告及应答级别;
arp_announce
arp_ignore
(2) RS的RIP可以使用私网地址,也可以是公网地址;RIP与DIP在同一IP网络;RIP的网关不能指向DIP,以确保响应报文不会经由Director;
(3) RS跟Director要在同一个物理网络;
(4) 请求报文要经由Director,但响应不能经由Director,而是由RS直接发往Client;
(5) 不支持端口映射。
4.IP分配
VIP1:172.18.67.66
VIP2:172.18.67.88
DIP1:172.18.67.13
DIP2:172.18.67.14
RIP1:172.18.67.11
RIP2:172.18.67.12
CIP:172.18.67.3
5.拓扑图
三 keepalived配置
1.安装keepalived
[root@inode2 ~]# yum install -y keepalived
[root@inode3 ~]# yum install -y keepalived
2.高可用的ipvs双主集群配置
第一个节点:
[root@inode2 ~]# cd /etc/keepalived/
[root@inode2 keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost #接受通知的邮件地址
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node2 #路由节点
vrrp_mcast_group4 224.0.67.67 #多播地址,范围224.0.0.0~239.255.255.255
}
vrrp_instance myr1 {
state MASTER
interface eno16777736 #网卡接口
virtual_router_id 167 #虚拟路由ID号,0~255
priority 100 #优先级,MASTER比BACKUP优先级高
advert_int 1
authentication {
auth_type PASS
auth_pass f1bf7fda
}
virtual_ipaddress {
172.18.67.66/16 dev eno16777736 label eno16777736:0
}
track_interface {
eno16777736
}
notify_master "/etc/keepalived/notify.sh master" #调用通知脚本
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance myr2 {
state BACKUP
interface eno16777736
virtual_router_id 168
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass f2bf7ade
}
virtual_ipaddress {
172.18.67.88/16 dev eno16777736 label eno16777736:1
}
track_interface {
eno16777736
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 172.18.67.66 80 { #VIP
delay_loop 2
lb_algo wrr #lvs负载均衡调度算法
lb_kind DR #负载均衡类型
protocol TCP #传输协议
sorry_server 127.0.0.1 80 #localhost
real_server 172.18.67.11 80 { #后端RIP
weight 1 #调度权重
HTTP_GET { #http请求方式
url {
path /
status_code 200 #状态码
}
connect_timeout 2 #连接超时
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.18.67.12 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 2
nb_get_retry 3
delay_before_retry 3
}
}
}
第二个节点:
[root@inode3 ~]# cd /etc/keepalived/
[root@inode3 keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node3
vrrp_mcast_group4 224.0.67.67
}
vrrp_instance myr1 {
state BACKUP
interface eno16777736
virtual_router_id 167
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass f1bf7fda
}
virtual_ipaddress {
172.18.67.66/16 dev eno16777736 label eno16777736:0
}
track_interface {
eno16777736
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance myr2 {
state MASTER
interface eno16777736
virtual_router_id 168
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass f2bf7ade
}
virtual_ipaddress {
172.18.67.88/16 dev eno16777736 label eno16777736:1
}
track_interface {
eno16777736
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 172.18.67.88 80 {
delay_loop 2
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.18.67.11 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 2
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.18.67.12 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 2
nb_get_retry 3
delay_before_retry 3
}
}
}
3.邮件通知脚本
当双主高可用集群主备切换时可通过邮件通知管理员,此时在配置文件中可自动调用实现编辑好的脚本
[root@inode2 ~]# vim notify.sh
#!/bin/bash
#
contact='root@localhost'
notify() {
mailsubject="$(hostname) to be $1, vip floating"
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
节点二同样配置。
四 后端RS服务器的配置
1.配置LAMP环境
[root@inode4 ~]# yum install httpd mariadb-server php -y
[root@inode5 ~]# yum install httpd mariadb-server php -y
2.简单编辑测试网页
[root@inode4 ~]# echo "RS1:172.18.67.11" > /var/www/html/index.html
[root@inode5 ~]# echo "RS2:172.18.67.12" > /var/www/html/index.html
3.修改RS内核参数
dr模型中,各主机上均需要配置VIP,解决地址冲突的方式有三种:
(1) 在前端网关做静态绑定;
(2) 在各RS使用arptables;
(3) 在各RS修改内核参数,来限制arp响应和通告的级别;
限制响应级别:arp_ignore
0:默认值,表示可使用本地任意接口上配置的任意地址进行响应;
1: 仅在请求的目标IP配置在本地主机的接收到请求报文接口上时,才给予响应;
限制通告级别:arp_announce
0:默认值,把本机上的所有接口的所有信息向每个接口上的网络进行通告;
1:尽量避免向非直接连接网络进行通告;
2:必须避免向非本网络通告。
可通过编辑脚本实现:
[root@inode4 ~]# vim dr.sh
#!/bin/bash
#
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce
;;
stop)
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/eth0/arp_announce
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
;;
esac
同理第二个RS需同样配置
4.添加路由
节点一:
[root@inode4 ~]# ifconfig lo:0 172.18.67.66 netmask 255.255.255.255 broadcast 172.18.67.66 up
[root@inode4 ~]# ifconfig lo:1 172.18.67.88 netmask 255.255.255.255 broadcast 172.18.67.88 up
[root@inode4 ~]# route add -host 172.18.67.66 dev lo:0
[root@inode4 ~]# route add -host 172.18.67.88 dev lo:1
节点二:
[root@inode5 ~]# ifconfig lo:0 172.18.67.88 netmask 255.255.255.255 broadcast 172.18.67.88 up
[root@inode5 ~]# ifconfig lo:1 172.18.67.66 netmask 255.255.255.255 broadcast 172.18.67.66 up
[root@inode5 ~]# route add -host 172.18.67.88 dev lo:0
[root@inode5 ~]# route add -host 172.18.67.66 dev lo:1
五 测试
1.启动服务
RS:
[root@inode4 ~]# systemctl start httpd
[root@inode5 ~]# systemctl start httpd
节点一:
[root@inode2 ~]# systemctl start keepalived.service
[root@inode2 ~]# systemctl status -l keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since Sun 2017-05-14 01:19:27 CST; 17s ago
Process: 2120 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 2121 (keepalived)
CGroup: /system.slice/keepalived.service
├─2121 /usr/sbin/keepalived -D
├─2122 /usr/sbin/keepalived -D
└─2123 /usr/sbin/keepalived -D
May 14 01:19:29 inode2 Keepalived_vrrp[2123]: Opening script file /etc/keepalived/notify.sh
May 14 01:19:29 inode2 Keepalived_healthcheckers[2122]: Netlink reflector reports IP 172.18.67.66 added
May 14 01:19:31 inode2 Keepalived_vrrp[2123]: VRRP_Instance(myr2) Transition to MASTER STATE
May 14 01:19:32 inode2 Keepalived_vrrp[2123]: VRRP_Instance(myr2) Entering MASTER STATE
May 14 01:19:32 inode2 Keepalived_vrrp[2123]: VRRP_Instance(myr2) setting protocol VIPs.
May 14 01:19:32 inode2 Keepalived_vrrp[2123]: VRRP_Instance(myr2) Sending gratuitous ARPs on eno16777736 for 172.18.67.88
May 14 01:19:32 inode2 Keepalived_vrrp[2123]: Opening script file /etc/keepalived/notify.sh
May 14 01:19:32 inode2 Keepalived_healthcheckers[2122]: Netlink reflector reports IP 172.18.67.88 added
May 14 01:19:34 inode2 Keepalived_vrrp[2123]: VRRP_Instance(myr1) Sending gratuitous ARPs on eno16777736 for 172.18.67.66
May 14 01:19:37 inode2 Keepalived_vrrp[2123]: VRRP_Instance(myr2) Sending gratuitous ARPs on eno16777736 for 172.18.67.88
[root@inode2 ~]# ip a l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:8b:08:6f brd ff:ff:ff:ff:ff:ff
inet 172.18.67.13/16 brd 172.18.255.255 scope global eno16777736
valid_lft forever preferred_lft forever
inet 172.18.67.66/16 scope global secondary eno16777736:0
valid_lft forever preferred_lft forever
inet 172.18.67.88/16 scope global secondary eno16777736:1
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe8b:86f/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
节点二:
[root@inode3 ~]# systemctl start keepalived.service
[root@inode3 ~]# systemctl status -l keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since Sun 2017-05-14 01:20:25 CST; 6s ago
Process: 2110 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 2111 (keepalived)
CGroup: /system.slice/keepalived.service
├─2111 /usr/sbin/keepalived -D
├─2112 /usr/sbin/keepalived -D
└─2113 /usr/sbin/keepalived -D
May 14 01:20:25 inode3 Keepalived_vrrp[2113]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
May 14 01:20:26 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) Transition to MASTER STATE
May 14 01:20:26 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) Received lower prio advert, forcing new election
May 14 01:20:26 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) Received lower prio advert, forcing new election
May 14 01:20:26 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) Received lower prio advert, forcing new election
May 14 01:20:27 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) Entering MASTER STATE
May 14 01:20:27 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) setting protocol VIPs.
May 14 01:20:27 inode3 Keepalived_vrrp[2113]: VRRP_Instance(myr2) Sending gratuitous ARPs on eno16777736 for 172.18.67.88
May 14 01:20:27 inode3 Keepalived_vrrp[2113]: Opening script file /etc/keepalived/notify.sh
May 14 01:20:27 inode3 Keepalived_healthcheckers[2112]: Netlink reflector reports IP 172.18.67.88 added
[root@inode3 ~]# ip a l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:78:24:c3 brd ff:ff:ff:ff:ff:ff
inet 172.18.67.14/16 brd 172.18.255.255 scope global eno16777736
valid_lft forever preferred_lft forever
inet 172.18.67.88/16 scope global secondary eno16777736:1
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe78:24c3/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
2.访问测试
[root@inode1 ~]# curl http://172.18.67.66
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.66
RS1:172.18.67.11
[root@inode1 ~]# curl http://172.18.67.66
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.66
RS1:172.18.67.11
[root@inode1 ~]# curl http://172.18.67.88
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.88
RS1:172.18.67.11
[root@inode1 ~]# curl http://172.18.67.88
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.88
RS1:172.18.67.11
高可用负载集群成功搭建完成。
3.模拟一台web服务器宕机
将RS1的httpd服务停掉,并再此访问:
[root@inode4:~]# systemctl stop httpd
[root@inode1 ~]# curl http://172.18.67.66
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.66
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.88
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.88
RS2:172.18.67.12
发现照样可以访问服务器,实际环境中两台RS的内容应该是一模一样的,在这里我为了以示区别将两台内容编辑成不同。
4.模拟一台高可用负载集群宕机
将VS1的keepalived服务关闭并测试:
[root@inode2 ~]# systemctl stop keepalived.service
[root@inode1 ~]# curl http://172.18.67.88
RS1:172.18.67.11
[root@inode1 ~]# curl http://172.18.67.88
RS2:172.18.67.12
[root@inode1 ~]# curl http://172.18.67.88
RS1:172.18.67.11
[root@inode1 ~]# curl http://172.18.67.88
RS2:172.18.67.12
我们发现即使高可用负载均衡集群中的某一个主机宕机了,我们任然可以通过其中的一个IP访问web服务器,体现出了高可用的实用性,并且在访问中lvs调度器将客户端请求按设置的权重分别向后端的服务器实现调度。
Keepalived+LVS(dr)高可用负载均衡集群的实现的更多相关文章
- LVS+Keepalived搭建MyCAT高可用负载均衡集群
LVS+Keepalived 介绍 LVS LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统.本项目在1998年5月由章文嵩博士成立,是中国 ...
- Haproxy+Keepalived搭建Weblogic高可用负载均衡集群
配置环境说明: KVM虚拟机配置 用途 数量 IP地址 机器名 虚拟IP地址 硬件 内存3G 系统盘20G cpu 4核 Haproxy keepalived 2台 192.168.1.10 192 ...
- Keepalived+Nginx实现高可用负载均衡集群
一 环境介绍 1.操作系统CentOS Linux release 7.2.1511 (Core) 2.服务keepalived+nginx双主高可用负载均衡集群及LAMP应用keepalived-1 ...
- Heartbeat+LVS构建高可用负载均衡集群
1.heartbeat简介: Heartbeat 项目是 Linux-HA 工程的一个组成部分,它实现了一个高可用集群系统.心跳服务和集群通信是高可用集群的两个关键组件,在 Heartbeat 项目里 ...
- LVS+Keepalived+Nginx+Tomcat高可用负载均衡集群配置(DR模式,一个VIP,多个端口)
一.概述 LVS作用:实现负载均衡 Keepalived作用:监控集群系统中各个服务节点的状态,HA cluster. 配置LVS有两种方式: 1. 通过ipvsadm命令行方式配置 2. 通过Red ...
- docker下用keepalived+Haproxy实现高可用负载均衡集群
启动keepalived后宿主机无法ping通用keepalived,报错: [root@localhost ~]# ping 172.18.0.15 PING () bytes of data. F ...
- Linux 笔记 - 第十八章 Linux 集群之(三)Keepalived+LVS 高可用负载均衡集群
一.前言 前两节分别介绍了 Linux 的高可用集群和负载均衡集群,也可以将这两者相结合,即 Keepalived+LVS 组成的高可用负载均衡集群,Keepalived 加入到 LVS 中的原因有以 ...
- LVS+Keepalived 实现高可用负载均衡集群
LVS+Keepalived 实现高可用负载均衡集群 随着网站业务量的增长,网站的服务器压力越来越大?需要负载均衡方案!商业的硬件如 F5 ,Array又太贵,你们又是创业型互联公司如何有效 ...
- 测试LVS+Keepalived高可用负载均衡集群
测试LVS+Keepalived高可用负载均衡集群 1. 启动LVS高可用集群服务 此时查看Keepalived服务的系统日志信息如下: [root@localhost ~]# tail -f /va ...
随机推荐
- C语言小知识点汇集
1. 三元表达式“?:”冒号两边的两个操作数类型必须一致. 2. 枚举变量的sizeof值同int型一样为4. 3. 为了消除不必要的计算 &&两侧的表达式,当左侧即第1个表达式不成立 ...
- Linux文件的默认权限:umask
1. 文件的默认权限 Linux下当我们新建一个文件和目录时,该文件和目录的默认权限是什么? 通过umask命令来查看: $ umask0002 $ umask -Su=rwx,g=rwx,o=rx ...
- react login page demo
1. login form import React from "react"; import {Row, Col} from "antd"; import { ...
- Flask的消息message机制flash
Flask的消息机制flash message是一个基于session实现的用于保存数据的集合,其特点是:使用一次就删除. 原理就是 操作成功 session['操作'] = 'msg' # 设置 s ...
- 项目工程结构说明(Internal)
注意:想要彻底把Internal关键字搞清楚,就耐着性子把她读完.当然了这篇文章只是对其他文章的总结.也算是引用吧.主要还是为了把知识点搞清楚 进入主题之前先来了解一下,项目.解决方案.程序集.命名空 ...
- 《Drools7.0.0.Final规则引擎教程》第3章 3.2 KIE概念&FACT对象
3.2.1 什么是KIE KIE(Knowledge Is Everything),知识就是一切的简称.JBoss一系列项目的总称,在<Drools使用概述>章节已经介绍了KIE包含的大部 ...
- WindowManager实现悬浮可拖动效果
现在360手机卫士有个流量统计的效果,开启流量统计后,在桌面上会出现一个显示流量的窗体,在任何界面都可以自由拖动. 模仿这个功能,做了一个统计手机信号强度的Demo, 界面效果如下: 从上面的截图可以 ...
- golang包快速生成base64验证码
base64Captcha快速生成base64编码图片验证码字符串 支持多种样式,算术,数字,字母,混合模式,语音模式. Base64是网络上最常见的用于传输8Bit字节代码的编码方式之一.Base6 ...
- sublime上配置markdown
等等等等 简书一个不错的教程:Sublime Text3的Markdown配置 补充说明:第一步可以直接找 Tools-->install package control. ^.^ ...
- LINUX 命令—netstat [简单实用]
1.--当我们在检查程序是否启动或者网络状况的时候 会查看本机活跃的端口,就需要这个命令: |--"netstat – Print network connections, routing ...