LVS入门篇（四）之LVS实战

• 一、LVS的NAT模式实战

1、环境说明：

HOST	OS	role	remask
192.168.56.12	Centos 7.4	LVS调度器（1.2.7）	VIP：192.168.0.104
192.168.56.11	Centos 7.4	RS1	LNMP
192.168.56.13	Centos 7.4	RS2	LNMP

2、LVS调度器环境配置

（1）增加一块外网网卡eth1

（2）拷贝eth0配置文件为eth1，并修改网卡配置

[root@lb01 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth1
[root@lb01 ~]# vim !$
vim /etc/sysconfig/network-scripts/ifcfg-eth1
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
NAME="eth1"
DEVICE="eth1"
ONBOOT="yes"
IPADDR=192.168.0.104
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
DNS1=8.8.8.8
DNS2=114.114.114.114
[root@lb01 ~]# systemctl restart network
[root@lb01 ~]# ifconfig eth1
eth1: flags=<UP,BROADCAST,RUNNING,MULTICAST>  mtu
        inet 192.168.0.104  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::20c:29ff:fe95:  prefixlen   scopeid 0x20<link>
        ether :0c::::  txqueuelen   (Ethernet)
        RX packets   bytes  (265.1 KiB)
        RX errors   dropped   overruns   frame
        TX packets   bytes  (5.4 KiB)
        TX errors   dropped  overruns   carrier   collisions 

（3）YUM安装ipvsadm，并对ipvsadm规则进行配置

1、开启调度器上的路由转发功能
[root@lb01 ~]# echo  > /proc/sys/net/ipv4/ip_forward　

2、关闭ICMP重定向　
[root@lb01 ~]# echo  > /proc/sys/net/ipv4/conf/all/send_redirects 　　
[root@lb01 ~]# echo  > /proc/sys/net/ipv4/conf/default/send_redirects
[root@lb01 ~]# echo  > /proc/sys/net/ipv4/conf/eth0/send_redirects
[root@lb01 ~]# echo  > /proc/sys/net/ipv4/conf/eth1/send_redirects 

3、配置防火墙的NAT规则
[root@lb01 ~]# iptables -t nat -F　　
[root@lb01 ~]# iptables -t nat -X
[root@lb01 ~]# iptables -t nat -A POSTROUTING -s 192.168.56.0/ -j MASQUERADE

4、配置ipvsadm并查看调度器状态信息
[root@lb01 ~]# ipvsadm -C
[root@lb01 ~]# ipvsadm -A -t 192.168.0.104: -s rr
[root@lb01 ~]# ipvsadm -a -t 192.168.0.104: -r 192.168.56.11: -m -w
[root@lb01 ~]# ipvsadm -a -t 192.168.0.104: -r 192.168.56.13: -m -w
[root@lb01 ~]# ipvsadm -L -n
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.104: rr
  -> 192.168.56.11:             Masq
  -> 192.168.56.13:             Masq                             

参数说明：

-A：表示增加一个虚拟服务

-a：表示增加一个real server

-t：表示TCP服务

-r：指定real server的ip地址

-s：指定调度算法

-m：选择NAT方式调度

-w：指定权重

（3）修改real server：RS1和RS2的网关

[root@rs1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
GATEWAY=192.168.56.12
[root@rs2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
GATEWAY=192.168.56.12

（4）测试访问

[root@lb01 ~]# curl 192.168.0.104
<h1>welcome to RS1 192.168.56.11</h1>
[root@lb01 ~]# curl 192.168.0.104
<h1> welcome to use RS2 192.168.56.13</h1>
[root@lb01 ~]# watch -n  ipvsadm -L -n　　#动态查看访问记录
Every .0s: ipvsadm -L -n                                                                                                                                            Tue Jul  :: 

IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.104: rr
  -> 192.168.56.11:             Masq
  -> 192.168.56.13:             Masq                   

• 二、LVS的DR模式实战

1、环境准备和说明

HOST	OS	ROLE	REMASK
192.168.56.12	Centos 7.4	LVS调度器	VIP：192.168.56.200
192.168.56.11	Centos 7.4	RS1	LNMP
192.168.56.13	Centos 7.4	RS2	LNMP

[root@lb01 ~]# ipvsadm -ln　　#查看NAT模式下的ipvsadm规则
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.104: rr
  -> 192.168.56.11:             Masq
  -> 192.168.56.13:             Masq
[root@lb01 ~]# ipvsadm -C　　#清空NAT模式下的规则
[root@lb01 ~]# ipvsadm -ln　　#确认规则是否清空
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
[root@lb01 ~]# iptables -t nat -F　　#情况防火墙的nat表规则
[root@rs1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0 　　#修改rs1网卡网关
GATEWAY=192.168.56.2
[root@rs2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0 　　#修改rs2网卡网关
GATEWAY=192.168.56.2
[root@rs1 ~]# systemctl restart network　　#重启网络服务
[root@rs2 ~]# systemctl restart network

　[root@lb01 ~]# ifdown eth1　　#DR模式调度器用不到eth1网卡，down掉
  Device 'eth1' successfully disconnected.

2、在LVS调度节点上配置vip

[root@lb01 ~]# ifconfig eth0: 192.168.56.200/ up
[root@lb01 ~]# ifconfig eth0:
eth0:: flags=<UP,BROADCAST,RUNNING,MULTICAST>  mtu
        inet 192.168.56.200  netmask 255.255.255.0  broadcast 192.168.56.255
        ether :0c::::  txqueuelen   (Ethernet)

3、在lvs调度节点上配置LVS服务并增加2台节点服务器

[root@lb01 ~]# ipvsadm -C　　#清空LVS配置
[root@lb01 ~]# ipvsadm --set   60　　#设置连接超时时间
[root@lb01 ~]# ipvsadm -A -t 192.168.56.200: -s rr -p 20　　#新增lvs服务
[root@lb01 ~]# ipvsadm -a -t 192.168.56.200: -r 192.168.56.11 -g -w
[root@lb01 ~]# ipvsadm -a -t 192.168.56.200: -r 192.168.56.13 -g -w
[root@lb01 ~]# ipvsadm -L -n
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.56.200: rr persistent
  -> 192.168.56.11:             Route
  -> 192.168.56.13:             Route                    

4、在RS节点上的lo（网卡的回环接口）接口上配置vip（192.168.56.200）

[root@rs1 ~]# ifconfig lo: 192.168.56.200/ up
[root@localhost ~]# ifconfig lo:
lo:: flags=<UP,LOOPBACK,RUNNING>  mtu
        inet 192.168.56.200  netmask 0.0.0.0
        loop  txqueuelen   (Local Loopback)
[root@rs2 ~]# ifconfig lo: 192.168.56.200/ up
[root@localhost ~]# ifconfig lo:
lo:: flags=<UP,LOOPBACK,RUNNING>  mtu
        inet 192.168.56.200  netmask 0.0.0.0
        loop  txqueuelen   (Local Loopback)

5、在RS节点上抑制ARP响应 （RS1和RS2上分别执行）

[root@rs1 ~]# echo  > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo  > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs1 ~]# echo  > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo  > /proc/sys/net/ipv4/conf/all/arp_announce

[root@rs2 ~]# echo  > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo  > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# echo  > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo  > /proc/sys/net/ipv4/conf/all/arp_announce

6、测试LVS调度

访问：http://192.168.56.200 ，如图：

[root@lb01 ~]# watch -n  ipvsadm -L -n 　　#查看访问状态
Every .0s: ipvsadm -L -n                                                                                                                                            Tue Jul  :: 

IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.56.200: rr persistent
  -> 192.168.56.11:             Route
  -> 192.168.56.13:             Route                  

7、删除节点和服务

[root@lb01 ~]# ipvsadm -d -t 192.168.56.200: -r 192.168.56.13   #删除RS2节点
[root@lb01 ~]# ipvsadm -Ln　　#查看是否已删除RS2
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.56.200: rr persistent
  -> 192.168.56.11:             Route
[root@lb01 ~]# ipvsadm -D -t 192.168.56.200:  　　#直接删除LVS服务

 8、LVS脚本配置（设置为开机启动，确保长期生效）

（1）Server端脚本

[root@lb01 ~]# vim /scripts/ipvs_server.sh
#!/bin/sh
. /etc/init.d/functions
VIP=192.168.56.200
PORT=
RIP=(
   192.168.56.11
   192.168.56.13
)
start(){
ifconfig eth0: $VIP/ up
route add -host $VIP dev eth0
ipvsadm -C
ipvsadm --set
ipvsadm -A -t $VIP:$PORT -s rr -p
for ((i=;i<${#RIP[*]};i++))
do
ipvsadm -a -t $VIP:$PORT -r ${RIP[$i]} -g -w
done
echo "ipvs is started"
}
stop(){
ipvsadm -C
ifconfig eth0:  down
route del -host $VIP dev eth0
echo "ipvs is stopped"
}
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "UASGE: $0  {start|stop|restart}"
esac
[root@lb01 ~]# cp /scripts/ipvs_server.sh /usr/local/sbin/ipvs
[root@lb01 ~]# cd /usr/local/sbin/
[root@lb01 ~]# chmod +x ipvs

（2）Client端脚本

[root@rs1 ~]# vi /scripts/ipvs_client.sh
#!/bin/sh
. /etc/init.d/functions
VIP=(
    192.168.56.200
)

case "$1" in
    start)
        echo "start LVS of RealServer IP"
        for ((i=;i<`echo ${#VIP[*]}`;i++))
        do
            interface="lo:`echo ${VIP[$i]}|awk -F . '{print $4}'`"
            /sbin/ifconfig $interface ${VIP[$i]} broadcast ${VIP[$i]} netmask 255.255.255.255 up
            route add -host ${VIP[$i]} dev $interface
        done
        echo "" >/proc/sys/net/ipv4/conf/lo/arp_ignore
        echo "" >/proc/sys/net/ipv4/conf/lo/arp_announce
        echo "" >/proc/sys/net/ipv4/conf/all/arp_ignore
        echo "" >/proc/sys/net/ipv4/conf/all/arp_announce
        ;;
    stop)
        for ((i=;i<`echo ${#VIP[*]}`;i++))
        do
            interface="lo:`echo ${VIP[$i]}|awk -F . '{print $4}'`"
            /sbin/ifconfig $interface ${VIP[$i]} broadcast ${VIP[$i]} netmask 255.255.255.255 down
            route del -host ${VIP[$i]} dev $interface
        done
        echo "stOP LVS of RealServer IP"
        echo "" >/proc/sys/net/ipv4/conf/lo/arp_ignore
        echo "" >/proc/sys/net/ipv4/conf/lo/arp_announce
        echo "" >/proc/sys/net/ipv4/conf/all/arp_ignore
        echo "" >/proc/sys/net/ipv4/conf/all/arp_announce
        ;;
    *)
        echo "Usage: $0 {start | stop}"
        exit
esac