lvs之DR模式的实操演练
理论
我是内部服务，代替我访问外部网络，这是正向代理；代替外部网络访问我，这是反向代理
槽位
sh根据源地址。调度到某个节点，dh,根据目标地址，调度到某个节点，
实战演练
默认策略以及修改策略
查看
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 wlc
-> 172.168.1.15:80 Masq 1 0 0
-> 172.168.1.16:80 Masq 1 0 0
TCP 10.0.0.14:443 wlc
-> 172.168.1.16:443 Masq 1 0 0
-> 172.168.1.25:443 Masq 1 0 0
[root@mcw04 ~]#
清除后添加，默认是wlc
[root@mcw04 ~]# ipvsadm -C
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@mcw04 ~]# ipvsadm -A -t 10.0.0.14:80
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 wlc
[root@mcw04 ~]#
wrr理论
wrr实践演练
清空以前的，然后添加wrr的lvs集群
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 wlc
[root@mcw04 ~]# ipvsadm -C
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@mcw04 ~]# ipvsadm -A -t 10.0.0.14:80 -s wrr
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 wrr
[root@mcw04 ~]#
添加rs 1 2 3 ，权重分别是1 2 3 。并且是nat模式，我们也能查看到权重信息
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 wrr
[root@mcw04 ~]# ipvsadm -a -t 10.0.0.14:80 -r 172.168.1.15 -m -w 1
[root@mcw04 ~]# ipvsadm -a -t 10.0.0.14:80 -r 172.168.1.16 -m -w 2
[root@mcw04 ~]# ipvsadm -a -t 10.0.0.14:80 -r 172.168.1.25 -m -w 3
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 wrr
-> 172.168.1.15:80 Masq 1 0 0
-> 172.168.1.16:80 Masq 2 0 0
-> 172.168.1.25:80 Masq 3 0 0
[root@mcw04 ~]#
我们在客户端mcw03上访问，可以看到，的确是 1 2 3 的比例，被调度到
[root@mcw03 ~]#
[root@mcw03 ~]# for i in {1..6}
> do
> curl 10.0.0.14
> done
hello rs3
machangwei06 rs2
machangwei05 rs1
hello rs3
hello rs3
machangwei06 rs2
[root@mcw03 ~]#
如果是访问12次呢，我们可以看到，的确还是1:2:3的概率
[root@mcw03 ~]# for i in {1..12}; do curl 10.0.0.14; done
hello rs3
machangwei06 rs2
machangwei05 rs1
hello rs3
hello rs3
machangwei06 rs2
hello rs3
machangwei06 rs2
machangwei05 rs1
hello rs3
hello rs3
machangwei06 rs2
[root@mcw03 ~]#
SH实践
测试效果，某个主机访问第一次调度到哪里，那么后面一直会调度到哪里
我们创建集群，添加rs。指定调度算法sh
[root@mcw04 ~]# ipvsadm -C
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@mcw04 ~]# ipvsadm -A -t 10.0.0.14:80 -s sh
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 sh
[root@mcw04 ~]# ipvsadm -a -t 10.0.0.14:80 -r 172.168.1.15 -m
[root@mcw04 ~]# ipvsadm -a -t 10.0.0.14:80 -r 172.168.1.16 -m
[root@mcw04 ~]# ipvsadm -a -t 10.0.0.14:80 -r 172.168.1.25 -m
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.14:80 sh
-> 172.168.1.15:80 Masq 1 0 0
-> 172.168.1.16:80 Masq 1 0 0
-> 172.168.1.25:80 Masq 1 0 0
[root@mcw04 ~]#
客户端请求，可以看到，客户端一直被调度到rs2上，符合预期
[root@mcw03 ~]# for i in {1..12}; do curl 10.0.0.14; done
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
[root@mcw03 ~]# for i in {1..12}; do curl 10.0.0.14; done
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
machangwei06 rs2
[root@mcw03 ~]#
再换一台机器访问，还是总是调度到后面同一台机器处理
[root@mcw04 ~]# for i in {1..12}; do curl 10.0.0.14; done
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
[root@mcw04 ~]# for i in {1..12}; do curl 10.0.0.14; done
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
[root@mcw04 ~]#
[root@mcw04 ~]# for i in {1..12}; do curl 10.0.0.14; done
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
[root@mcw04 ~]# for i in {1..12}; do curl 10.0.0.14; done
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
hello rs3
[root@mcw04 ~]#
不过我在把rs2防火墙开起来后
[root@mcw06 ~]# systemctl start firewalld.service
[root@mcw06 ~]# curl localhost
machangwei06 rs2
[root@mcw06 ~]#
这时它还是去请求rs2,此时有问题，但是它还是调度这个绑定的节点，而不是其它节点
[root@mcw03 ~]# for i in {1..12}; do curl 10.0.0.14; done
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
^C
[root@mcw03 ~]# for i in {1..12}; do curl 10.0.0.14; done
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
curl: (7) Failed connect to 10.0.0.14:80; No route to host
[root@mcw03 ~]#
DR模式前置知识理论
DR模式理论
直接路由的意思，就是后端rs真实主机能直接和客户端进行通信。lvs主机和rs在同一个交换机上面，可以基于mac地址进行通信。rs要和客户端通信，需要有外网ip。客户端需要访问公司服务的时候，只会访问一个ip地址。因此我们想要请求随机跳到任何一个主机上面，那么这些所有的后端主机上面必须都有一个相同的对外的ip，也就是vip。
数据请求是这样的：数据从客户端经过路由器，转交到lvs主机，lvs主机把请求分发给后端的真实主机，后端的真实主机的响应报文会通过vip接口直接返回给客户端，这就是一个完整的数据流程。
公网网段：cip，vip，；私网网段，lvs和rs都是通过交换机，在内网网段。lvs主机在整个当中属于一个分发的效果。而且是基于交换机分发，基于mac地址进行分发。
数据来的时候需要经过lvs主机，走的时候不需要经过lvs主机，直接通过rs主机返回给客户端。这种情况下lvs主机的压力会少一些。
下面最重要的一个就是配置我们的网关，让vip能找到客户端ip。
因为下面的主机都用了同一个vip，那么地址冲突的问题需要解决了。这里我们通过内核方式的修改，达到所有的内网主机上面，都可以使用相同的vip效果。
DR模式简单实践
理论
路由器，一个外网一个内网
因为是直接路由，所以我的rs想要和外网直接通信的话，需要配置一个vip，vip和外网直接通信，还需要设置我们的网络网关。
因为lvs机器上面，作为流量的入口，所以vip最好是配置在真实网卡上面，而rs上的vip，仅供响应数据报文的出口，所以没必要放在eth0网卡上面，可以放在lo上面。而lvs主机，必须得绑定到真实网卡上面，不然没办法接收数据了，
效果，注意这里访问，客户端是访问路由器对外的ip 192.168.8.14,不是访问我们的vip。
实践
mcw03 客户端 10.0.0.13(ens33)
mcw15 路由器 10.0.0.25(ens33 进网网关) 172.168.1.25(ens34 出网网关)
mcw04 lvs 172.168.1.14(ens33) 172.168.1.100(ens34) 关闭外网网卡
mcw05 rs1 172.168.1.15(ens34) 172.168.1.100(ens34) 关闭外网网卡
mcw06 rs2 172.168.1.16(ens34) 172.168.1.100(ens34) 关闭外网网卡
因为路由器有ens34网卡IP。所以三个内网机器，都可可以用内网IP从路由器上去登录
mcw04 5 6上增加出网网关，指向路由器内网网卡IP。此时，内网机器，可以ping通路由器外网网卡的IP。虽然mcw05没有外网IP，但是因为添加了路由器的内网网卡IP作为默认路由，当ping路由器的外网网卡IP的时候，因为出流量会走到mcw05的默认网关也就是路由器的内网网卡IP上面去，然后访问到
给三个内网机器添加VIP
mcw04 LVS机器加到ens34上面
ifconfig ens34:0 172.168.1.100/32 up
rs添加VIP，添加到lo上面
重新来，添加网卡
[root@mcw05 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:43:64:95 txqueuelen 1000 (Ethernet)
RX packets 500038 bytes 33458865 (31.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15512 bytes 1443348 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.168.1.15 netmask 255.255.255.0 broadcast 172.168.1.255
inet6 fe80::c7c4:97e9:a77b:a70b prefixlen 64 scopeid 0x20<link>
inet6 fe80::428e:4a2b:802a:fccc prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:43:64:9f txqueuelen 1000 (Ethernet)
RX packets 1843 bytes 122461 (119.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 281 bytes 33083 (32.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@mcw05 ~]# ifconfig lo 172.168.1.100/32 up
[root@mcw05 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:43:64:95 txqueuelen 1000 (Ethernet)
RX packets 500038 bytes 33458865 (31.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15512 bytes 1443348 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.168.1.15 netmask 255.255.255.0 broadcast 172.168.1.255
inet6 fe80::c7c4:97e9:a77b:a70b prefixlen 64 scopeid 0x20<link>
inet6 fe80::428e:4a2b:802a:fccc prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:43:64:9f txqueuelen 1000 (Ethernet)
RX packets 1871 bytes 124817 (121.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 296 bytes 35749 (34.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 172.168.1.100 netmask 0.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 175 bytes 14640 (14.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 175 bytes 14640 (14.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@mcw05 ~]#
宕掉网卡
[root@mcw05 ~]# ifconfig lo 172.168.1.100/32 down
[root@mcw05 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:43:64:95 txqueuelen 1000 (Ethernet)
RX packets 500038 bytes 33458865 (31.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15512 bytes 1443348 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.168.1.15 netmask 255.255.255.0 broadcast 172.168.1.255
inet6 fe80::c7c4:97e9:a77b:a70b prefixlen 64 scopeid 0x20<link>
inet6 fe80::428e:4a2b:802a:fccc prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:43:64:9f txqueuelen 1000 (Ethernet)
RX packets 1901 bytes 127333 (124.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 312 bytes 38973 (38.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@mcw05 ~]# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noqueue state DOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 172.168.1.100/0 scope global lo
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:43:64:95 brd ff:ff:ff:ff:ff:ff
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:43:64:9f brd ff:ff:ff:ff:ff:ff
inet 172.168.1.15/24 brd 172.168.1.255 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::428e:4a2b:802a:fccc/64 scope link
valid_lft forever preferred_lft forever
inet6 fe80::c7c4:97e9:a77b:a70b/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
[root@mcw05 ~]#
在添加VIP之前，我们应该先修改rs内核参数。查看参数
[root@mcw05 ~]# cat /proc/sys/net/ipv4/conf/all/arp_ignore
0
[root@mcw05 ~]# cat /proc/sys/net/ipv4/conf/all/arp_announce
0
[root@mcw05 ~]# cat /proc/sys/net/ipv4/conf/lo/arp_ignore
0
[root@mcw05 ~]# cat /proc/sys/net/ipv4/conf/lo/arp_announce
0
[root@mcw05 ~]#
修改参数
[root@mcw05 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@mcw05 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@mcw05 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@mcw05 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@mcw05 ~]#
然后添加VIP
root@mcw05 ~]# ifconfig lo 172.168.1.100/32 up
[root@mcw05 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:43:64:95 txqueuelen 1000 (Ethernet)
RX packets 500136 bytes 33465129 (31.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15512 bytes 1443348 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.168.1.15 netmask 255.255.255.0 broadcast 172.168.1.255
inet6 fe80::c7c4:97e9:a77b:a70b prefixlen 64 scopeid 0x20<link>
inet6 fe80::428e:4a2b:802a:fccc prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:43:64:9f txqueuelen 1000 (Ethernet)
RX packets 2511 bytes 179461 (175.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 638 bytes 76453 (74.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 172.168.1.100 netmask 0.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 175 bytes 14640 (14.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 175 bytes 14640 (14.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@mcw05 ~]#
此时客户端上访问VIP，试试，没有通，这是因为没有添加网关
添加网关，是路由器外网网卡，还是不通。
路由器开启ipv4转发
然后就可以ping通VIP了
宕掉rs1 mcw05的VIP，此时只有lvs服务器有这个VIP，然后从外网客户端继续ping,可以ping通VIP。因为ping VIP的时候，客户端默认网关是到路由器外网网卡，通过ipvs4内核转发到路由器内网网卡，内网网卡能通lvs内网服务器，到达VIP，返回数据包的时候，因为lvs服务器的默认网关是路由器的内网网卡，路由器又通过内核转发到外网网卡，然后到达外部网络客户端。
rs2也配置一下VIP
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig lo 172.168.1.100/32 up
ssl报错了
后端服务开启，因为rs2用了https了，所以启停特别慢
添加lvs集群以及添加后端服务器。使用直接路由的模式
[root@mcw04 ~]# ipvsadm -C
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@mcw04 ~]# ipvsadm -A -t 172.168.1.100:80
[root@mcw04 ~]# ipvsadm -a -t 172.168.1.100:80 -r 172.168.1.15
[root@mcw04 ~]# ipvsadm -a -t 172.168.1.100:80 -r 172.168.1.16
[root@mcw04 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.168.1.100:80 wlc
-> 172.168.1.15:80 Route 1 0 0
-> 172.168.1.16:80 Route 1 0 0
[root@mcw04 ~]#
在客户端访问，可以成功访问到。
回头测试，把内核配置取消掉，是什么结果,测试了，貌似没有影响，四个内核配置都设置为0.结果还能被调度访问到服务
DR模型原理详解
DR模型应用技巧
上面的案例原理解析抓包分析
客户端请求访问到rs1
[root@mcw03 ~]# curl 172.168.1.100
machangwei05 rs1
[root@mcw03 ~]#
路由器外网网卡ens33
[root@mcw15 ~]#
[root@mcw15 ~]# tcpdump -nn -i ens33 port 80 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
00:06:12.468817 00:0c:29:ae:54:49 > 00:0c:29:aa:cb:e8, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:12.469374 00:0c:29:aa:cb:e8 > 00:0c:29:ae:54:49, ethertype IPv4 (0x0800), length 74: 172.168.1.100.80 > 10.0.0.13.21076: Flags [S.], seq 3273677818, ack 1733961332, win 28960, options [mss 1460,sackOK,TS val 279961132 ecr 279975180,nop,wscale 9], length 0
00:06:12.469696 00:0c:29:ae:54:49 > 00:0c:29:aa:cb:e8, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:12.469727 00:0c:29:ae:54:49 > 00:0c:29:aa:cb:e8, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:12.470284 00:0c:29:aa:cb:e8 > 00:0c:29:ae:54:49, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [.], ack 78, win 57, options [nop,nop,TS val 279961132 ecr 279975181], length 0
00:06:12.470891 00:0c:29:aa:cb:e8 > 00:0c:29:ae:54:49, ethertype IPv4 (0x0800), length 325: 172.168.1.100.80 > 10.0.0.13.21076: Flags [P.], seq 1:260, ack 78, win 57, options [nop,nop,TS val 279961133 ecr 279975181], length 259: HTTP: HTTP/1.1 200 OK
00:06:12.471041 00:0c:29:ae:54:49 > 00:0c:29:aa:cb:e8, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:12.471122 00:0c:29:ae:54:49 > 00:0c:29:aa:cb:e8, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:12.471587 00:0c:29:aa:cb:e8 > 00:0c:29:ae:54:49, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [F.], seq 260, ack 79, win 57, options [nop,nop,TS val 279961134 ecr 279975183], length 0
00:06:12.471712 00:0c:29:ae:54:49 > 00:0c:29:aa:cb:e8, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
路由器内网网卡ens34
[root@mcw15 ~]# tcpdump -nn -i ens34 port 80 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes
00:06:12.468838 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:12.469244 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:12.469366 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 74: 172.168.1.100.80 > 10.0.0.13.21076: Flags [S.], seq 3273677818, ack 1733961332, win 28960, options [mss 1460,sackOK,TS val 279961132 ecr 279975180,nop,wscale 9], length 0
00:06:12.469702 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:12.469729 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:12.469837 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:12.469839 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:12.470275 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [.], ack 78, win 57, options [nop,nop,TS val 279961132 ecr 279975181], length 0
00:06:12.470883 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 325: 172.168.1.100.80 > 10.0.0.13.21076: Flags [P.], seq 1:260, ack 78, win 57, options [nop,nop,TS val 279961133 ecr 279975181], length 259: HTTP: HTTP/1.1 200 OK
00:06:12.471046 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:12.471126 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:12.471518 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:12.471521 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:12.471582 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [F.], seq 260, ack 79, win 57, options [nop,nop,TS val 279961134 ecr 279975183], length 0
00:06:12.471716 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
00:06:12.471884 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
lvs网卡ens34
[root@mcw04 ~]# tcpdump -nn -i ens34 port 80 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes
00:06:13.879854 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:13.879890 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:13.880193 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 74: 172.168.1.100.80 > 10.0.0.13.21076: Flags [S.], seq 3273677818, ack 1733961332, win 28960, options [mss 1460,sackOK,TS val 279961132 ecr 279975180,nop,wscale 9], length 0
00:06:13.880541 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:13.880554 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:13.880587 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:13.880591 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:13.880856 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [.], ack 78, win 57, options [nop,nop,TS val 279961132 ecr 279975181], length 0
00:06:13.881687 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 325: 172.168.1.100.80 > 10.0.0.13.21076: Flags [P.], seq 1:260, ack 78, win 57, options [nop,nop,TS val 279961133 ecr 279975181], length 259: HTTP: HTTP/1.1 200 OK
00:06:13.881976 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.881989 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.882037 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.882042 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.882187 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [F.], seq 260, ack 79, win 57, options [nop,nop,TS val 279961134 ecr 279975183], length 0
00:06:13.882573 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
00:06:13.882582 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
rs1网卡ens34
[root@mcw05 ~]# tcpdump -nnn -i ens34 port 80 -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes
00:06:13.875934 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:13.876087 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 74: 10.0.0.13.21076 > 172.168.1.100.80: Flags [S], seq 1733961331, win 29200, options [mss 1460,sackOK,TS val 279975180 ecr 0,nop,wscale 9], length 0
00:06:13.876112 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 74: 172.168.1.100.80 > 10.0.0.13.21076: Flags [S.], seq 3273677818, ack 1733961332, win 28960, options [mss 1460,sackOK,TS val 279961132 ecr 279975180,nop,wscale 9], length 0
00:06:13.876684 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:13.876689 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:13.876797 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 0
00:06:13.876806 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 143: 10.0.0.13.21076 > 172.168.1.100.80: Flags [P.], seq 1:78, ack 1, win 58, options [nop,nop,TS val 279975181 ecr 279961132], length 77: HTTP: GET / HTTP/1.1
00:06:13.876835 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [.], ack 78, win 57, options [nop,nop,TS val 279961132 ecr 279975181], length 0
00:06:13.877662 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 325: 172.168.1.100.80 > 10.0.0.13.21076: Flags [P.], seq 1:260, ack 78, win 57, options [nop,nop,TS val 279961133 ecr 279975181], length 259: HTTP: HTTP/1.1 200 OK
00:06:13.877983 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.878043 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.878135 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.878213 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [F.], seq 78, ack 260, win 60, options [nop,nop,TS val 279975183 ecr 279961133], length 0
00:06:13.878241 00:0c:29:43:64:9f > 00:0c:29:aa:cb:f2, ethertype IPv4 (0x0800), length 66: 172.168.1.100.80 > 10.0.0.13.21076: Flags [F.], seq 260, ack 79, win 57, options [nop,nop,TS val 279961134 ecr 279975183], length 0
00:06:13.878647 00:0c:29:aa:cb:f2 > 00:0c:29:73:fb:66, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
00:06:13.878750 00:0c:29:73:fb:66 > 00:0c:29:43:64:9f, ethertype IPv4 (0x0800), length 66: 10.0.0.13.21076 > 172.168.1.100.80: Flags [.], ack 261, win 60, options [nop,nop,TS val 279975183 ecr 279961134], length 0
如下，命令行添加的VIP，当重启网络之后，就会消失，会失效，
lvs之DR模式的实操演练的更多相关文章
- 第十章 Fisco Bcos 权限控制下的数据上链实操演练
一.目的 前面已经完成fisco bcos 相关底层搭建.sdk使用.控制台.webase中间件平台等系列实战开发, 本次进行最后一个部分,体系化管理区块链底层,建立有序的底层控管制度,实现权限化管理 ...
- ABP入门系列（1）——学习Abp框架之实操演练
作为.Net工地搬砖长工一名,一直致力于挖坑(Bug)填坑(Debug),但技术却不见长进.也曾热情于新技术的学习,憧憬过成为技术大拿.从前端到后端,从bootstrap到javascript,从py ...
- LVS：DR模式（Direct Routing）部署实验
本文介绍怎样在kvm的虚拟环境下,部署实验LVS的DR模式.包含网络结构图,怎样配置.以及使用tcpdump分析ip包. 网络结构图 kvm ...
- ASP.NET Core托管和部署Linux实操演练手册
一.课程介绍 ASP.NET Core 是一种全新的跨平台开源 .NET 框架,能够在 IIS.Nginx.Apache.Docker 上进行托管或在自己的进程中进行自托管. 作为一个.NET Web ...
- LVS的DR模式
DR模式: 请求由LVS接受,由真实提供服务的服务器(RealServer, RS)直接返回给用户,返回的时候不经过LVS. DR模式下需要LVS和绑定同一个VIP(RS通过将VIP绑定在loopba ...
- LVS的DR模式负载均衡
参考项目:http://www.cnblogs.com/along21/p/7833261.html#auto_id_3 LVS的DR模式实现负载均衡 1.环境 lvs-server :192.168 ...
- Lvs Keepalive DR模式高可用配置
Lvs Keepalive DR模式配置 一.环境 #DIP# eth0:192.168.233.145#VIP# eth0:0 192.168.233.250/32 #RIP1:192.168.23 ...
- lvs中dr模式配置脚本
1 dr模式介绍 1.1 lvs的安装 安装具体解释:http://blog.csdn.net/CleverCode/article/details/50586957. 1.2 lvs模式 lvs有三 ...
- lvs部署-DR模式
DR模式 角色 IP地址 备注 LVS负载均衡器 192.168.119.132 VIP:192.168.119.150 ipvsadm http_Real server 192.168.119 ...
- CentOS6.4 配置LVS（DR模式）
DR模式中LVS主机与实际服务器都有一块网卡连在同一物理网段上. IP分配 VIP:10.10.3.170 RIP1:10.10.3.140 RIP2:10.10.3.141 1.安装所需的依赖包 y ...
随机推荐
- 内存分析利器之UMDH
近两周投入分析产品的内存泄漏问题. 测试团队反馈产品在安卓平台运行时,随用户操作,应用占用的内存出现上涨的趋势,停止操作并等待一段时间之后,应用占用的内存没有下降,怀疑存在内存泄漏问题. 结合复现的情 ...
- Windows 杀毒简单有效的方式
Windows 电脑杀毒通常会选择杀毒软件,这样太笨重,且容易占内存和存在流氓软件侵入. 推荐使用 Windows 自带的恶意软件删除工具 按住 Win + R 键,弹出运行窗口,输入 mrt. 系统 ...
- 运动App如何实现端侧后台保活，让运动记录更完整？
你在锻炼健身时,有没有遇到这样的情况?辛辛苦苦锻炼了几小时,却发现App停止了运行,本次运动并没有被记录到App上,从而失去了一个查看完整运动数据的机会? 运动类App是通过手机或者穿戴设备的传感器, ...
- 报表格式太难？一文教你Excel融合分析
Excel是我们最经常使用的处理报表工具了,尤其是金融行业更是需要天天和报表打交道.但是Excel有一个问题,如果面对的数据庞杂,则处理起来就会相当麻烦,当需要从多个数据源获取数据来制作一个报表时,这 ...
- C++执行Linux命令
一.执行简单命令 比如需要创建文件.文件夹.删除文件 #include <iostream> #include <stdio.h> #include <stdlib.h& ...
- MogDB/openGauss存储过程的修改
MogDB/openGauss 存储过程的修改 SQL 中没有提供显式的存储过程修改命令,通常需要通过 REPLACE 关键字来指定使用当前的存储过程替代之前的同名存储过程. 将前文定义的存储过程替换 ...
- 解密方舟的高性能内存回收技术——HPP GC
原文:https://mp.weixin.qq.com/s/o8uuP1XViIyviveL4m-8ZQ,点击链接查看更多技术内容. 众所周知,内存是操作系统的一项重要资源,直接影响系统性能.而在应用 ...
- 10.1K star ！牛逼了！开源技术速查表，推荐人手一份！
1.前言 在当今信息爆炸的时代,知识的获取.整理和应用显得尤为重要.随着个人职业发展和学习需求的不断提升,搭建一个个人知识库已成为提升竞争力的关键一环.个人知识库不仅是一个信息的存储库,更是一个思维的 ...
- 演示webuploader和cropperjs图片裁剪上传
最近有个项目要在浏览器端裁剪并上传图片.由于缺乏人力,只能我上阵杀敌.通过参考各种文章,最后决定用cropperjs进行图片裁剪,用webuploader上传文件.本文涉及到的知识至少有Java基础. ...
- TextIn.com API使用心得
我们参加了本次大学生创新创业服务外包大赛,在项目中大量使用到了合合信息所提供的api进行相关功能实现,所以在这里写一篇博客分享一下我们在项目的实际推进中关于TextIn.com API使用心得 我们的 ...