Keepalived + LVS-NAT 实现高可用四层 TCP/UDP 负载均衡器
目录
文章目录
前文列表
在 LVS1/2 安装 Keepalived & LVS
[root@control01 ~]# yum install -y keepalived ipvsadm
[root@control01 ~]# keepalived --version
Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
[root@control01 ~]# ipvsadm --version
ipvsadm v1.27 2008/5/15 (compiled with popt and IPVS v1.2.1)
Keepalived + LVS-NAT 实现 TCP 负载均衡
官方文档：LVS NAT + Keepalived HOWTO
IP 规划
- Client：192.168.1.100/24
- LVS1：
- 外部网卡 192.168.1.110/24
- 内部网卡 10.0.0.103/24
- LVS2：
- 外部网卡 192.168.1.111/24
- 内部网卡 10.0.0.104/24
- VS 外部 VIP：192.168.1.112
- VS 内部 DIP：10.0.0.105
- RS1：
- IP 10.0.0.101/24
- 网关 10.0.0.105/24
- RS2：
- IP 10.0.0.102/24
- 网关 10.0.0.105/24
网络架构参考
LVS1 配置
启动 ipvsadm：
[root@localhost ~]# touch /etc/sysconfig/ipvsadm
[root@localhost ~]# systemctl start ipvsadm.service
启动 keepalived：
[root@localhost ~]# cat /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp -P Only run with VRRP subsystem.
# --check -C Only run with Health-checker subsystem.
# --dont-release-vrrp -V Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs -I Dont remove IPVS topology on daemon stop.
# --dump-conf -d Dump the configuration data.
# --log-detail -D Detailed log messages.
# --log-facility -S 0-7 Set local syslog facility (default=LOG_DAEMON)
#
KEEPALIVED_OPTIONS="-D -d"
[root@localhost ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
lvs_id LVS_01
}
vrrp_sync_group VG1 {
group {
VI_1
VI_GATEWAY
}
}
vrrp_instance VI_1 {
state MASTER
interface eno16777736
lvs_sync_daemon_inteface eno16777736
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.112
}
}
vrrp_instance VI_GATEWAY {
state MASTER
interface eno33554960
lvs_sync_daemon_inteface eno33554960
virtual_router_id 52
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass example
}
virtual_ipaddress {
10.0.0.105
}
}
virtual_server 192.168.1.112 80 {
delay_loop 6
lb_algo rr
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 10.0.0.101 80 {
weight 1
}
real_server 10.0.0.102 80 {
weight 1
}
}
[root@localhost ~]# systemctl start keepalived
Keepalived 的启动日志：
[root@localhost ~]# journalctl -f -u keepalived
-- Logs begin at Sun 2019-01-06 07:05:29 EST. --
Jan 06 09:57:02 localhost.localdomain systemd[1]: Starting LVS and VRRP High Availability Monitor...
Jan 06 09:57:02 localhost.localdomain Keepalived[18040]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Jan 06 09:57:02 localhost.localdomain Keepalived[18040]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 06 09:57:02 localhost.localdomain Keepalived[18041]: Starting Healthcheck child process, pid=18042
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Initializing ipvs
Jan 06 09:57:02 localhost.localdomain Keepalived[18041]: Starting VRRP child process, pid=18043
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Unknown keyword 'lvs_id'
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Unknown keyword 'nat_mask'
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Registering Kernel netlink reflector
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Registering Kernel netlink command channel
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Registering gratuitous ARP shared channel
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Unknown keyword 'lvs_id'
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Unknown keyword 'lvs_sync_daemon_inteface'
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Unknown keyword 'lvs_sync_daemon_inteface'
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_1) removing protocol VIPs.
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_GATEWAY) removing protocol VIPs.
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ------< Global definitions >------
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Router ID = localhost
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Default interface = eth0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: LVS flush = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP IPv4 mcast group = 224.0.0.18
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP IPv6 mcast group = ff02::12
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP refresh timer = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP refresh repeat = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP lower priority delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP lower priority repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Send advert after receive lower priority advert = true
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Send advert after receive higher priority advert = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP interval = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous NA interval = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP default protocol version = 2
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Iptables input chain = INPUT
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Using ipsets = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ipset IPv4 address set = keepalived
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ipset IPv6 address set = keepalived6
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ipset IPv6 address,iface set = keepalived_if6
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP check unicast_src = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP skip check advert addresses = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP strict mode = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP process priority = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP don't swap = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Checker process priority = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Checker don't swap = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: SNMP keepalived disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: SNMP checker disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: SNMP RFCv2 disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: SNMP RFCv3 disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: SNMP traps disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: SNMP socket = default (unix:/var/agentx/master)
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Network namespace = (default)
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Script security disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Default script uid:gid 0:0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ------< VRRP Topology >------
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP Instance = VI_1
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Using VRRPv2
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Want State = MASTER
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Running on device = eno16777736
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Skip checking advert IP addresses = no
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Enforcing strict VRRP compliance = no
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Using src_ip = 192.168.1.110
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP refresh timer = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP refresh repeat = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP lower priority delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP lower priority repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Send advert after receive lower priority advert = true
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Send advert after receive higher priority advert = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Virtual Router ID = 51
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Priority = 150
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Advert interval = 1 sec
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Accept enabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Promote_secondaries disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Authentication type = SIMPLE_PASSWORD
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Password = 1111
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Virtual IP = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: ------< Global definitions >------
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Router ID = localhost
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: 192.168.1.112/32 dev eno16777736 scope global
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Default interface = eth0
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: LVS flush = false
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP IPv4 mcast group = 224.0.0.18
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP IPv6 mcast group = ff02::12
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP refresh timer = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP refresh repeat = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP lower priority delay = 4294
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP lower priority repeat = -1
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Send advert after receive lower priority advert = true
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Send advert after receive higher priority advert = false
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous ARP interval = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Gratuitous NA interval = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP Instance = VI_GATEWAY
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP default protocol version = 2
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Using VRRPv2
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Iptables input chain = INPUT
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Want State = MASTER
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Using ipsets = true
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Running on device = eno33554960
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: ipset IPv4 address set = keepalived
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Skip checking advert IP addresses = no
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: ipset IPv6 address set = keepalived6
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Enforcing strict VRRP compliance = no
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: ipset IPv6 address,iface set = keepalived_if6
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Using src_ip = 10.0.0.103
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP check unicast_src = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP skip check advert addresses = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP strict mode = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP refresh timer = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP process priority = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP refresh repeat = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VRRP don't swap = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP lower priority delay = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Checker process priority = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Gratuitous ARP lower priority repeat = 5
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Checker don't swap = false
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Send advert after receive lower priority advert = true
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: SNMP keepalived disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Send advert after receive higher priority advert = false
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: SNMP checker disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Virtual Router ID = 52
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: SNMP RFCv2 disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Priority = 150
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: SNMP RFCv3 disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Advert interval = 1 sec
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: SNMP traps disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Accept enabled
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: SNMP socket = default (unix:/var/agentx/master)
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Promote_secondaries disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Network namespace = (default)
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Authentication type = SIMPLE_PASSWORD
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Script security disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Password = example
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Default script uid:gid 0:0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Virtual IP = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: ------< SSL definitions >------
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: 10.0.0.105/32 dev eno33554960 scope global
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Using autogen SSL context
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ------< VRRP Sync groups >------
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: ------< LVS Topology >------
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP Sync Group = VG1, BACKUP
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: System is compiled with LVS v1.2.1
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: monitor = VI_1
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: VIP = 192.168.1.112, VPORT = 80
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: monitor = VI_GATEWAY
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Address family = inet
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ------< NIC >------
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: delay_loop = 6, lb_algo = rr
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Name = eno16777736
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: Hashed = disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: index = 2
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: flag-1 = disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: IPv4 address = 192.168.1.110
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: flag-2 = disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: IPv6 address = fe80::20c:29ff:fe27:d53b
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: flag-3 = disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: MAC = 00:0c:29:27:d5:3b
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: One packet scheduling = disabled
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: is UP
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: protocol = TCP
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: is RUNNING
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: alpha is OFF, omega is OFF
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: MTU = 1500
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: quorum = 1, hysteresis = 0
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: HW Type = ETHERNET
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: lb_kind = NAT
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: ------< NIC >------
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: RIP = 10.0.0.101, RPORT = 80, WEIGHT = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Name = eno33554960
Jan 06 09:57:02 localhost.localdomain Keepalived_healthcheckers[18042]: RIP = 10.0.0.102, RPORT = 80, WEIGHT = 1
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: index = 3
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: IPv4 address = 10.0.0.103
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: IPv6 address = fe80::20c:29ff:fe27:d545
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: MAC = 00:0c:29:27:d5:45
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: is UP
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: is RUNNING
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: MTU = 1500
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: HW Type = ETHERNET
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: Using LinkWatch kernel netlink reflector...
Jan 06 09:57:02 localhost.localdomain systemd[1]: Started LVS and VRRP High Availability Monitor.
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Jan 06 09:57:02 localhost.localdomain Keepalived_vrrp[18043]: VRRP sockpool: [ifindex(3), proto(112), unicast(0), fd(12,13)]
Jan 06 09:57:03 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_GATEWAY) Transition to MASTER STATE
Jan 06 09:57:03 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_GATEWAY) Entering MASTER STATE
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_GATEWAY) setting protocol VIPs.
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_GATEWAY) Sending/queueing gratuitous ARPs on eno33554960 for 10.0.0.105
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Group(VG1) Syncing instances to MASTER state
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eno16777736 for 192.168.1.112
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:04 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_GATEWAY) Sending/queueing gratuitous ARPs on eno33554960 for 10.0.0.105
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno33554960 for 10.0.0.105
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eno16777736 for 192.168.1.112
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
Jan 06 09:57:09 localhost.localdomain Keepalived_vrrp[18043]: Sending gratuitous ARP on eno16777736 for 192.168.1.112
查看 VIP 和 DIP：
[root@localhost ~]# ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:27:d5:3b brd ff:ff:ff:ff:ff:ff
inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic eno16777736
valid_lft 6646sec preferred_lft 6646sec
inet 192.168.1.112/32 scope global eno16777736
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe27:d53b/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
3: eno33554960: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:27:d5:45 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.103/24 brd 10.0.0.255 scope global eno33554960
valid_lft forever preferred_lft forever
inet 10.0.0.105/32 scope global eno33554960
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe27:d545/64 scope link
valid_lft forever preferred_lft forever
查看 ipvs 规则：
[root@localhost ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.112:80 rr
-> 10.0.0.101:80 Masq 1 0 0
-> 10.0.0.102:80 Masq 1 0 0
NOTE：LVS1 上并不会真的开启 80 端口，只是 80 只是一个 VPORT。
[root@localhost ~]# netstat -lpntu | grep 80
开启路由转发功能：
[root@localhost ~]# cat /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward = 1
[root@localhost ~]# sysctl -p
net.ipv4.ip_forward = 1
清空防火墙规则：
[root@localhost ~]# iptables -F -t filter
[root@localhost ~]# iptables -F -t raw
[root@localhost ~]# iptables -F -t mangle
[root@localhost ~]# iptables -F -t nat
LVS2 配置
配置 LVS2 和配置 LVS1 的步骤基本一致，但 Keepalived 的配置文件有些许变动。
[root@localhost ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
lvs_id LVS_01
}
vrrp_sync_group VG1 {
group {
VI_1
VI_GATEWAY
}
}
vrrp_instance VI_1 {
state BACKUP
interface eno16777736
lvs_sync_daemon_inteface eno16777736
virtual_router_id 51
priority 140
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.112
}
}
vrrp_instance VI_GATEWAY {
state BACKUP
interface eno33554960
lvs_sync_daemon_inteface eno33554960
virtual_router_id 52
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass example
}
virtual_ipaddress {
10.0.0.105
}
}
virtual_server 192.168.1.112 80 {
delay_loop 6
lb_algo rr
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 10.0.0.101 80 {
weight 1
}
real_server 10.0.0.102 80 {
weight 1
}
}
配置 RS1
网卡的 Gateway 指向 DIP：
[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-eno33554960
HWADDR=00:0C:29:15:40:15
TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno33554960
UUID=be63d7a3-f7eb-4204-9c1d-cecb2e857d0b
ONBOOT=yes
IPADDR=10.0.0.101
GATEWAY=10.0.0.105
NETMASK=255.255.255.0
DNS1=114.114.114.114
清空防火墙规则：
[root@localhost ~]# iptables -F -t filter
[root@localhost ~]# iptables -F -t raw
[root@localhost ~]# iptables -F -t mangle
[root@localhost ~]# iptables -F -t nat
安装 TCP 80 的 httpd 服务：
[root@localhost ~]# yum install -y httpd
[root@localhost ~]# cat /var/www/html/index.html
<html>
<body>
<h1>RS1</h1>
</body>
</html>
[root@localhost ~]# systemctl start httpd
[root@localhost ~]# netstat -lpntu | grep 80
tcp6 0 0 :::80 :::* LISTEN 18227/httpd
配置 RS2
配置 RS2 与 RS1 的步骤基本一致，只是 httpd 的 index.html 有些许改变：
[root@localhost ~]# cat /var/www/html/index.html
<html>
<body>
<h1>RS2</h1>
</body>
</html>
验证
在客户端 curl VIP:VPort 会轮询的访问 RS1 和 RS2。
[root@localhost ~]# curl 192.168.1.112
<html>
<body>
<h1>RS2</h1>
</body>
</html>
[root@localhost ~]# curl 192.168.1.112
<html>
<body>
<h1>RS1</h1>
</body>
</html>
在 MASTER 查看连接表：
[root@localhost ~]# ipvsadm -Lnc
IPVS connection entries
pro expire state source virtual destination
TCP 01:27 TIME_WAIT 192.168.1.100:52034 192.168.1.112:80 10.0.0.101:80
在 BACKUP 查看连接表：
[root@localhost ~]# ipvsadm -Lnc
IPVS connection entries
pro expire state source virtual destination
Failover：关闭 MASTER 电源之后还可以继续通过客户端访问 curl VIP:VPort，查看发现 VIP 漂移到了 BACKUP，再次查看 BACKUP 的 ipvs 连接表：
[root@localhost ~]# ipvsadm -Lnc
IPVS connection entries
pro expire state source virtual destination
TCP 01:57 TIME_WAIT 192.168.1.100:52115 192.168.1.112:80 10.0.0.101:80
TS
开始为了方便操作 RS1/RS2 所以也为这两天机器添加了 192.168.1.0/24 的网卡，但这样做的话 VIP 无法生效，后来卸载掉即可，暂不清楚什么原因。
Keepalived + LVS-NAT 实现 UDP 负载均衡
配置 RS1/2
启用 nc udp 服务器，接收外部传输过来的文件：
[root@localhost ~]# yum install -y nc
[root@localhost ~]# nc -ul 9999 > file.txt
[root@localhost ~]# netstat -lpntu | grep 9999
udp 0 0 0.0.0.0:9999 0.0.0.0:* 2618/nc
udp6 0 0 :::9999 :::* 2618/nc
配置 LVS1/2
配置 Keepalived，添加 UDP 协议负载均衡 virtual_server：
[root@localhost ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
lvs_id LVS_01
}
vrrp_sync_group VG1 {
group {
VI_1
VI_GATEWAY
}
}
vrrp_instance VI_1 {
state MASTER
interface eno16777736
lvs_sync_daemon_inteface eno16777736
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.112
}
}
vrrp_instance VI_GATEWAY {
state MASTER
interface eno33554960
lvs_sync_daemon_inteface eno33554960
virtual_router_id 52
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass example
}
virtual_ipaddress {
10.0.0.105
}
}
virtual_server 192.168.1.112 80 {
delay_loop 6
lb_algo rr
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 10.0.0.101 80 {
weight 1
}
real_server 10.0.0.102 80 {
weight 1
}
}
virtual_server 192.168.1.112 9999 {
delay_loop 6
lb_algo rr
lb_kind NAT
nat_mask 255.255.255.0
protocol UDP
real_server 10.0.0.101 9999 {
weight 1
}
real_server 10.0.0.102 9999 {
weight 1
}
}
[root@localhost ~]# systemctl restart keepalived
[root@localhost ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.112:80 rr
-> 10.0.0.101:80 Masq 1 0 0
-> 10.0.0.102:80 Masq 1 0 0
UDP 192.168.1.112:9999 rr
-> 10.0.0.101:9999 Masq 1 0 0
-> 10.0.0.102:9999 Masq 1 0 0
NOTE：LVS1/2 的配置大同小异，只是 vrrp_instance 的角色和权重有所区别而已。
验证
在客户端上准备两个文件：
╭─mickeyfan@localhost ~/test
╰─$ cat 1.txt 1 ↵
11111111111
╭─mickeyfan@localhost ~/test
╰─$ cat 2.txt
22222222222
执行文件传输：
╭─mickeyfan@localhost ~/test
╰─$ nc -u -w 1 192.168.1.112 9999 < 1.txt 1 ↵
╭─mickeyfan@localhost ~/test
╰─$ nc -u -w 1 192.168.1.112 9999 < 2.txt
可以看见这两个文件分别被 RS1 和 RS2 的 nc udp 服务端接收。
- RS1
[root@localhost ~]# nc -ul 9999 > file.txt
Ncat: Connection refused.
[root@localhost ~]# cat file.txt
22222222222
- RS2
[root@localhost ~]# nc -ul 9999 > file.txt
Ncat: Connection refused.
[root@localhost ~]# cat file.txt
11111111111
查看 ipvs 转发表：
[root@localhost ~]# ipvsadm -Lnc
IPVS connection entries
pro expire state source virtual destination
UDP 04:20 UDP 192.168.1.100:65136 192.168.1.112:9999 10.0.0.101:9999
UDP 04:19 UDP 192.168.1.100:51930 192.168.1.112:9999 10.0.0.102:9999
总结
使用 Keepalived + LVS-NAT 模式需要注意以下几点：
- LVS 服务器应该具有两张网卡，并且通过 Keepalived 分别配置 VIP 和 DIP。VIP 是给外部客户端访问的，DIP 是给内部后端服务器访问的。
- LVS 服务器应该开启路由转发功能。
- RS 服务器的网关应该指向 DIP。
- ipvs 的规则应该是 VIP 转发到后端真实服务器，而不是 DIP 转发到后端真实服务器。
Keepalived + LVS-NAT 实现高可用四层 TCP/UDP 负载均衡器的更多相关文章
- Keepalived+LVS DR模式高可用架构实践
Keepalived最初是为LVS设计,专门监控各服务器节点的状态(LVS不带健康检查功能,所以使用keepalived进行健康检查),后来加入了VRRP(虚拟路由热备协议(Virtual Route ...
- 014.Docker Harbor+Keepalived+LVS+共享存储高可用架构
一 多Harbor高可用介绍 共享后端存储是一种比较标准的方案,将多个Harbor实例共享同一个后端存储,任何一个实例持久化到存储的镜像,都可被其他实例中读取.通过前置LB组件,如Keepalived ...
- LVS 四层 TCP/UDP 负载均衡器
目录 文章目录 目录 LVS LVS 应用结构 LVS 提供的三种模式 LVS-NAT LVS-TUN LVS_DR LVS 负载均衡算法 静态负载均衡 动态负载均衡 LVS-ipvsadm 指令集 ...
- Lvs+keepAlived实现负载均衡高可用集群（DR实现）
第1章 LVS 简介 1.1 LVS介绍 LVS是Linux Virtual Server的简写,意为Linux虚拟服务器,是虚拟的服务器集群系统,可在UNIX/LINUX平台下实现负载均衡集群功能. ...
- 基于Keepalived实现LVS双主高可用集群
Reference: https://mp.weixin.qq.com/s?src=3×tamp=1512896424&ver=1&signature=L1C7us ...
- lvs+keep搭建高可用web服务
title: lvs+keep搭建高可用web服务 date: 2015-11-26 22:11:55 tags: --- 第一部分 概念 负载均衡 生产环境下必不可少的基础手段当前大部分互联网都使用 ...
- 高并发场景 LVS 安装及高可用实现
1.1 负载均衡介绍 1.1.1 负载均衡的妙用 负载均衡(Load Balance)集群提供了一种廉价.有效.透明的方法,来扩展网络设备和服务器的负载.带宽.增加吞吐量.加强网络数据处理能力.提高网 ...
- [svc]高并发场景 LVS DR +KeepAlive高可用实现及ka的persistence_timeout参数
LVS-DR+keepalived模式是一种非常经典的常用生产组合 高可用场景及LVS架构 一般都用一(负载)拖多(Server Array)方式 使用LVS架设的服务器集群系统有三个部分组成: (1 ...
- Nginx(haproxy)+keepalived+Tomcat双主高可用负载均衡
周末的时候一个正在学Linux的朋友问我,高可用怎么玩?我和他微信了将近三个小时,把Nginx和haproxy双主高可用教给他了,今天突然想把这个给写进博客里,供给那些正在学习Linux系统的朋友们, ...
随机推荐
- iscsi原理
iscsi原理 一,[名词解释] SCSI:小型计算机系统接口,SCSI作为i输入/输出接口, FC:光纤通道 DAS:直连式存储,指将存储设备通过SCSI接口或光纤通道直接连到一台计算机上. NAS ...
- Linux系统组成和获取命令帮助1
在GNU上边发布的都是源码,不可以直接拿来使用 源代码都是文本格式的,需要找个编译器编译成不同机器上使用的二进制,这样机器才可以运行的起来 英特儿的CPU有着x86,x64架构之分,x64又叫amd6 ...
- IIS7设置限制IP地址访问
1.拒绝访问设置,选择“一组计算机”,下面重点说明如何填写“网络标识”和“子网掩码”. IP地址按照IPV4的标准来分,分为A类地址.B类地址.C类地址,一般我们是屏蔽C类或者B类地址. A类地址:如 ...
- 可执行程序加一个dl
add_executable(forwarder app/main.cxx) TARGET_LINK_LIBRARIES(forwarder dl)
- 磁盘IO及性能指标
一.磁盘 I/O 的概念 I/O 的概念,从字义来理解就是输入输出.操作系统从上层到底层,各个层次之间均存在 I/O.比如,CPU 有 I/O,内存有 I/O, VMM 有 I/O, 底层磁盘上也有 ...
- 判断当前环境是ios还是安卓
/** * @name 判断iOS */ export const isiOS = ()=>{ let u = navigator.userAgent; let iOs = !!u.match( ...
- async异步改同步后怎么监听错误
当我们使用readFile()这种api,它第一个参数是报的错误,当使用async.await把它改写成同步,我们可以使用try { }catch{ }解决.
- MFC:OnCreate PreCreateWindow PreSubclassWindow
OnCreate PreCreateWindow PreSubclassWindow PreCreateWindow和PreSubclassWindow是虚函数,而OnCreate是一个消息响应函数. ...
- CSS3做出条纹大背景
㈠实现不等宽背景条纹 实现如上图所示的效果,代码如下: <!DOCTYPE html> <html lang="en"> <head> &l ...
- head first 设计模式笔记5-单例模式
目录: 1.单例模式(Singleton Pattern) 2.概念 3.饿汉式:不是延迟加载,加载类的时候直接初始化 4.懒汉式:延迟加载,首次需要使用的时候在实例化,需要考虑线程安全 5.静态内部 ...