[转]Examining Open vSwitch Traffic Patterns
In this post, I want to provide some additional insight on how the use of Open vSwitch (OVS) affects—or doesn’t affect, in some cases—how a Linux host directs traffic through physical interfaces, OVS internal interfaces, and OVS bridges. This is something that I had a hard time understanding as I started exploring more advanced OVS configurations, and hopefully the information I share here will be helpful to others.
To help structure this discussion, I’m going to walk through a few different OVS configurations and scenarios. In these scenarios, I’ll use the following assumptions:
· The physical host has four interfaces (eth0, eth1, eth2, and eth3)
· The host is running Linux with KVM, libvirt, and OVS installed
Scenario 1: Simple OVS Configuration
In this first scenario let’s look at a relatively simple OVS configuration, and examine how Linux host and guest domain traffic moves into or out of the network.
Let’s assume that our OVS configuration looks something like this (this is the output from ovs-vsctl show):
bc6b9e64-11d6-415f-a82b-5d8a61ed3fbd
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "eth0"
Interface "eth0"
Bridge "br1"
Port "br1"
Interface "br1"
type: internal
Port "eth1"
Interface "eth1"
ovs_version: "1.7.1"
This is a pretty simple configuration; there are two bridges, each with a single physical interface. Let’s further assume, for the purposes of this scenario, that eth2 has an IP address and is working properly to communicate with other hosts on the network. The eth3 interface is shutdown.
So, in this scenario, how does traffic move into or out of the host?
1. Traffic from a guest domain: Traffic from a guest domain will travel through the OVS bridge to which it is attached (you’d see an additional “vnet0″ port and interface appear on that bridge when you start the guest domain). So, a guest domain attached to br0 would communicate via eth0, and a guest domain attached to br1 would communicate via eth1. No real surprises here.
2. Traffic from the Linux host: Traffic from the Linux host itself will notcommunicate over any of the configured OVS bridges, but will instead use its native TCP/IP stack and any configured interfaces. Thus, since eth2 is configured and operational, all traffic to/from the Linux host itself will travel through eth2.
The interesting point (to me, at least) about #2 above is that this includes traffic from the OVS process itself. In other words, if the OVS process(es) need to communicate across the network, they won’t use the bridges—they’ll use whatever interfaces the Linux host uses to communicate. This is one thing that threw me off: because OVS is itself a Linux process, when OVS needs to communicate across the network it will use the Linux network stack to do so. In this scenario, then, OVS would not communicate over any configured bridge, but instead using eth2. (This makes perfect sense now, but I recall that it didn’t earlier. Maybe it’s just me.)
Scenario 2: Adding Bonding
In this second scenario, our OVS configuration changes only slightly:
bc6b9e64-11d6-415f-a82b-5d8a61ed3fbd
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "bond0"
Interface "eth0"
Interface "eth1"
ovs_version: "1.7.1"
In this case, we’re now leveraging a bond that contains two physical interfaces (eth0 and eth1). (By the way, I have a write-up on configuring OVS and bonds, if you need/want more information.) The eth2 interface still has an IP address assigned and is up and communicating properly. The physical eth3 interface is shutdown.
How does this affect the way in which traffic is handled? It doesn’t, really.Traffic from guest domains will still travel across br0 (since this is the only configured OVS bridge), and traffic from the Linux host—including traffic from OVS itself—will still use whatever interfaces are determined by the host’s TCP/IP stack. In this case, that would be eth2.
Scenario 3: The Isolated Bridge
Let’s look at another OVS configuration, the so-called “isolated bridge”. This is a configuration that is commonly found in implementations using NVP, OpenStack, and others, and it’s a configuration that I recently discussed in my post on GRE tunnels and OVS.
Here’s the configuration:
bc6b9e64-11d6-415f-a82b-5d8a61ed3fbd
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "bond0"
Interface "eth0"
Interface "eth1"
Bridge "br-int"
Port "br-int"
Interface "br-int"
type: internal
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.1.100"}
ovs_version: "1.7.1"
As with previous configurations, we’ll assume that eth2 is up and operational, and eth3 is shutdown. So how does traffic get directed in this configuration?
1. Traffic from guest domains attached to br0: This is as before—traffic will go out one of the physical interfaces in the bond, according to the bonding configuration (active-standby, LACP, etc.). Nothing unusual here.
2. Traffic from the Linux host: As before, traffic from processes on the Linux host will travel out according to the host’s TCP/IP stack. There are no changes from previous configurations.
3. Traffic from guest domains attached to br-int: Now, this is where it gets interesting. Guest domains attached to br-int (named “br-int” because in this configuration the isolated bridge is often called the “integration bridge”) don’t have any physical interfaces they can use; they can only use the GRE tunnel. Here’s the “gotcha”, so to speak: the GRE tunnel is created and maintained by the OVS process, and therefore it uses the host’s TCP/IP stack to communicate across the network. Thus, traffic from guest domains attached to br-int would hit the GRE tunnel, which would travel through eth2.
I’ll give you a second to let that sink in.
Ready now? Good! The key to understanding #3 is, in my opinion, understanding that the tunnel (a GRE tunnel in this case, but the same would apply to a VXLAN or STT tunnel) is created and maintained by the OVS process.Thus, because it is created and maintained by a process on the Linux host (OVS itself), the traffic for the tunnel is directed according to the host’s TCP/IP stack and IP routing table(s). In this configuration, the tunnels don’t travel through any of the configured OVS bridges.
Scenario 4: Leveraging an OVS Internal Interface
Let’s keep ramping up the complexity. For this scenario, we’ll use an OVS configuration that is the same as in the previous scenario:
bc6b9e64-11d6-415f-a82b-5d8a61ed3fbd
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "bond0"
Interface "eth0"
Interface "eth1"
Bridge "br-int"
Port "br-int"
Interface "br-int"
type: internal
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.1.100"}
ovs_version: "1.7.1"
The difference, this time, is that we’ll assume that eth2 and eth3 are both shutdown. Instead, we’ve assigned an IP address to the br0 interface on bridge br0. OVS internal interfaces, like br0, can appear as “physical” interfaces to the Linux host, and therefore can be assigned IP addresses and used for communication. This is the approach I used in describing how to run host management across OVS.
Here’s how this configuration affects traffic flow:
1. Traffic from guest domains attached to br0: No change here. Traffic from guest domains attached to br0 will continue to travel across the physical interfaces in the bond (eth0 and eth1, in this case).
2. Traffic from the Linux host: This time, the only interface that the Linux host has is the br0 internal interface. The br0 internal interface is attached to br0, so all traffic from the Linux host will travel across the physical interfaces attached to the bond (again, eth0 and eth1).
3. Traffic from guest domains attached to br-int: Because Linux host traffic is directed through br0 by virtue of using the br0 internal interface, this means that tunnel traffic is also directed through br0, as dictated by the Linux host’s TCP/IP stack and IP routing table(s).
As you can see, assigning an IP address to an OVS internal interface has a real impact on the way in which the Linux host directs traffic through OVS. This has both positive and negative impacts:
· One positive impact is that it allows for Linux host traffic (such as management or tunnel traffic) to take advantage of OVS bonds, thus gaining some level of NIC redundancy.
· A negative impact is that OVS is now “in band,” so upgrades to OVS will be disruptive to all traffic moving through OVS—which could potentially include host management traffic.
Let’s take a look at one final scenario.
Scenario 5: Using Multiple Bridges and Internal Interfaces
In this configuration, we’ll use an OVS configuration that is very similar to the configuration I showed in my post on GRE tunnels with OVS:
bc6b9e64-11d6-415f-a82b-5d8a61ed3fbd
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "mgmt0"
Interface "mgmt0"
type: internal
Port "bond0"
Interface "eth0"
Interface "eth1"
Bridge "br1"
Port "br1"
Interface "br1"
type: internal
Port "tep0"
Interface "tep0"
type: internal
Port "bond1"
Interface "eth2"
Interface "eth3"
Bridge "br-int"
Port "br-int"
Interface "br-int"
type: internal
Port "gre0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.1.100"}
ovs_version: "1.7.1"
In this configuration, we have three bridges. br0 uses a bond that contains eth0 and eth1; br1 uses a bond that contains eth2 and eth3; and br-int is an isolated bridge with no physical interfaces. We have two “custom” internal interfaces, mgmt0 (on br0) and tep0 (on br1), to which IP addresses have been assigned and which are successfully communicating across the network. We’ll assume that mgmt0 and tep0 are on different subnets, and that tep0 is assigned to the 192.168.1.0/24 subnet.
How does traffic flow in this scenario?
1. Traffic from guest domains attached to br0: The behavior here is as it has been in previous configurations—guest domains attached to br0 will communicate across the physical interfaces in the bond.
2. Traffic from the Linux host: As it has been in previous scenarios, traffic from the Linux host is driven by the host’s TCP/IP stack and IP routing table(s). Because mgmt0 and tep0 are on different subnets, traffic from the Linux host will go out either br0 (for traffic moving through mgmt0) or br1 (for traffic moving through tep0), and thus will utilize the corresponding physical interfaces in the bonds on those bridges.
3. Traffic from guest domains attached to br-int: Because the GRE tunnel is on the 192.168.1.0/24 subnet, traffic for the GRE tunnel—which is created and maintained by the OVS process on the Linux host itself—will travel through tep0, which is attached to br1. Thus, the physical interfaces eth2 and eth3 would be leveraged for the GRE tunnel traffic.
Summary
The key takeaway from this post, in my mind, is understanding where traffic originates, and separating the idea of OVS as a switching mechanism (to handle guest domain traffic) as well as a Linux host process itself (to create and maintain tunnels between hosts).
Hopefully this information is helpful. I am, of course, completely open to your comments, questions, and corrections, so feel free to speak up in the comments below. Courteous comments are always welcome!
[转]Examining Open vSwitch Traffic Patterns的更多相关文章
- Examining Open vSwitch Traffic Patterns
In this post, I want to provide some additional insight on how the use of Open vSwitch (OVS) affects ...
- (转载)研究openvswitch的流量模式
最近又开始弄openvswitch,网上也有不少的资料,但是发觉都集中在openvswitch安装及简单使用或者一些原码分析,从心里来讲,感觉这些和心里得到的差距有点大啊,其实更希望能类似资料在ope ...
- Enhancing the Scalability of Memcached
原文地址: https://software.intel.com/en-us/articles/enhancing-the-scalability-of-memcached-0 1 Introduct ...
- 【译】在ASP.Net和IIS中删除不必要的HTTP响应头
引入 每次当浏览器向Web服务器发起一个请求的时,都会伴随着一些HTTP头的发送.而这些HTTP头是用于给Web服务器提供一些额外信息以便于处理请求.比如说吧.如果浏览器支持压缩功能,则浏览器会发送A ...
- 在ASP.Net和IIS中删除不必要的HTTP响应头
引入 每次当浏览器向Web服务器发起一个请求的时,都会伴随着一些HTTP头的发送.而这些HTTP头是用于给Web服务器提供一些额外信息以便于处理请求.比如说吧.如果浏览器支持压缩功能,则浏览器会发送A ...
- Optimal Logging
by Anthony Vallone How long does it take to find the root cause of a failure in your system? Five mi ...
- HDU 2722 Here We Go(relians) Again (spfa)
Here We Go(relians) Again Time Limit : 2000/1000ms (Java/Other) Memory Limit : 32768/32768K (Java/ ...
- Here We Go(relians) Again
Here We Go(relians) Again Time Limit: 2000/1000 MS (Java/Others) Memory Limit: 32768/32768 K (Java/O ...
- Revisiting Network Support for RDMA
重新审视RDMA的网络支持 本文为SIGCOMM 2018会议论文. 笔者翻译了该论文.由于时间仓促,且笔者英文能力有限,错误之处在所难免:欢迎读者批评指正. 本文及翻译版本仅用于学习使用.如果有任何 ...
随机推荐
- pythone函数基础(12)连接Redis,写数据,读数据,修改数据
需要导入Resdis模块 import redisip = '127.0.0.1'password='123456'r = redis.Redis(host=ip,password=password, ...
- Windows网络发现无法启动
解决方法: 运行services.msc命令,打开服务界面.分别将Function Discovery Resource Publication.SSDP Discovery.UPnP Device ...
- ServletConfig、ServletContext 的应用
一.ServletConfig对象及其应用(用的不多) 1. Context和ContextPath:一个web工程,若名为JavaWeb,访问的路径为:http://localhost:8080/J ...
- 201621123002《JAVA程序设计》第十四周学习总结
1. 本周学习总结 1.1 以你喜欢的方式(思维导图或其他)归纳总结与数据库相关内容. 2. 使用数据库技术改造你的系统 2.1 简述如何使用数据库技术改造你的系统.要建立什么表?截图你的表设计. 用 ...
- Centos启动流程及grub legacy
Linux系统的组成部分:内核+根文件系统 内核的功能:进程管理.内存管理.网络管理.文件系统.驱动程序.安全功能 系统在运行时要么就是在运行内核代码,要么就是在运行应用程序代码.如果一个程序大多数时 ...
- Oracle启动和停止
概述 只有具备sysdba和sysoper系统特权的用户才能启动和关闭数据库. 在启动数据库之前应该启动监听程序,否则就不能利用命令方式来管理数据库,包括启动和关闭数据库. 虽然数据库正常运行,但如果 ...
- Acceptance Test - Business Readable Acceptance Test using - Specflow
Agenda Benefits Living document Frameworks specflow supports How to integrate it in development cycl ...
- Jmeter如何把响应数据的结果保存到本地的一个文件
当做性能压测时,可能会需要把响应数据的一些字段统计出来.这里简单介绍一下. 1.首先把接口调通,确定需要统计的字段,这里以统计ccmpSeq字段来做例子. 2.添加正则表达式提取器,用来提取响应结果中 ...
- Web3D
https://baike.baidu.com/item/WEB%203D/11066359?fr=aladdin https://zhidao.baidu.com/question/17325151 ...
- 20175316 盛茂淞 MyCP(课下作业,必做)
题目要求 编写MyCP.java 实现类似Linux下cp XXX1 XXX2的功能,要求MyCP支持两个参数: java MyCP -tx XXX1.txt XXX2.bin 用来把文本文件(内容为 ...