OpenStack/devstack with Neutron on Ubuntu 14 (2)

在前面的文章中，已经完成了devstack的安装。下面，我会介绍如何使用neutron

• 首先创建两个neutron net, vmnet1 和vmnet2

stack@ubuntu:~/devstack$ source openrc admin admin
stack@ubuntu:~/devstack$ neutron net-list
+--------------------------------------+---------+-------------------------------------------------------+
| id                                   | name    | subnets                                               |
+--------------------------------------+---------+-------------------------------------------------------+
| 5365b56f--40d4-b065-a9d50126b55b | private | fc3a839e-412c-4f0d-b1ed-87be1a7b0bf8 192.168.106.0/ |
| a101e11f-7bbe-4f4d-a52e-5bb312d5f8b9 | public  | 570e8b64--41f7--12d9bc54f195 172.24.4.0/    |
+--------------------------------------+---------+-------------------------------------------------------+
stack@ubuntu:~/devstack$ neutron net-create vmnet1
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | 54028daa--4fc7-9f97-d400dc801e97 |
| name                      | vmnet1                               |
| provider:network_type     | vlan                                 |
| provider:physical_network | physnet1                             |
| provider:segmentation_id  |                                   |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tenant_id                 | dcf6685a3e304e4c92af08bca762aa33     |
+---------------------------+--------------------------------------+
stack@ubuntu:~/devstack$ neutron net-create vmnet2
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | 7b0394ec-34c0-48d3-a17d-82ef9bbc2cd9 |
| name                      | vmnet2                               |
| provider:network_type     | vlan                                 |
| provider:physical_network | physnet1                             |
| provider:segmentation_id  |                                   |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tenant_id                 | dcf6685a3e304e4c92af08bca762aa33     |
+---------------------------+--------------------------------------+
stack@ubuntu:~/devstack$ neutron net-list
+--------------------------------------+---------+-------------------------------------------------------+
| id                                   | name    | subnets                                               |
+--------------------------------------+---------+-------------------------------------------------------+
| 5365b56f--40d4-b065-a9d50126b55b | private | fc3a839e-412c-4f0d-b1ed-87be1a7b0bf8 192.168.106.0/ |
| a101e11f-7bbe-4f4d-a52e-5bb312d5f8b9 | public  | 570e8b64--41f7--12d9bc54f195 172.24.4.0/    |
| 54028daa--4fc7-9f97-d400dc801e97 | vmnet1  |                                                       |
| 7b0394ec-34c0-48d3-a17d-82ef9bbc2cd9 | vmnet2  |                                                       |
+--------------------------------------+---------+-------------------------------------------------------+

• 现在两个网络就创建好了，但现在其实这是两个独立的网络，是不互通的，接下来创建router： router-vmnet1-vmnet2

stack@ubuntu:~/devstack$ neutron router-create router-vmnet1-vmnet2
Created a new router:
+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| admin_state_up        | True                                 |
| distributed           | False                                |
| external_gateway_info |                                      |
| ha                    | False                                |
| id                    | 4bef86df-3b13-4b23-80f0-3a2bd8d0c746 |
| name                  | router-vmnet1-vmnet2                 |
| routes                |                                      |
| status                | ACTIVE                               |
| tenant_id             | dcf6685a3e304e4c92af08bca762aa33     |
+-----------------------+--------------------------------------+

• 在vmnet1和vmnet2中，分别创建subnet

stack@ubuntu:~/devstack$ neutron subnet-create 54028daa--4fc7-9f97-d400dc801e97 30.30.30.0/
Created a new subnet:
+-------------------+------------------------------------------------+
| Field             | Value                                          |
+-------------------+------------------------------------------------+
| allocation_pools  | {"start": "30.30.30.2", "end": "30.30.30.254"} |
| cidr              | 30.30.30.0/                                  |
| dns_nameservers   |                                                |
| enable_dhcp       | True                                           |
| gateway_ip        | 30.30.30.1                                     |
| host_routes       |                                                |
| id                | c0be451f-91a1-4f9f-a29b-13adea79798e           |
| ip_version        |                                               |
| ipv6_address_mode |                                                |
| ipv6_ra_mode      |                                                |
| name              |                                                |
| network_id        | 54028daa--4fc7-9f97-d400dc801e97           |
| tenant_id         | dcf6685a3e304e4c92af08bca762aa33               |
+-------------------+------------------------------------------------+
stack@ubuntu:~/devstack$ neutron subnet-create 7b0394ec-34c0-48d3-a17d-82ef9bbc2cd9 40.40.40.0/
Created a new subnet:
+-------------------+------------------------------------------------+
| Field             | Value                                          |
+-------------------+------------------------------------------------+
| allocation_pools  | {"start": "40.40.40.2", "end": "40.40.40.254"} |
| cidr              | 40.40.40.0/                                  |
| dns_nameservers   |                                                |
| enable_dhcp       | True                                           |
| gateway_ip        | 40.40.40.1                                     |
| host_routes       |                                                |
| id                | 0b9c9ad8--4c96-afda-ecb7e53dc519           |
| ip_version        |                                               |
| ipv6_address_mode |                                                |
| ipv6_ra_mode      |                                                |
| name              |                                                |
| network_id        | 7b0394ec-34c0-48d3-a17d-82ef9bbc2cd9           |
| tenant_id         | dcf6685a3e304e4c92af08bca762aa33               |
+-------------------+------------------------------------------------+

• 接下来，给router加入interface，分别连接到vmnet1的subnet和vmnet2的subnet

stack@ubuntu:~/devstack$ neutron router-interface-add 4bef86df-3b13-4b23-80f0-3a2bd8d0c746 subnet=c0be451f-91a1-4f9f-a29b-13adea79798e
Added interface 8fc6f138-150c-47de-8c88-fa924ce2302d to router 4bef86df-3b13-4b23-80f0-3a2bd8d0c746.
stack@ubuntu:~/devstack$ neutron router-interface-add 4bef86df-3b13-4b23-80f0-3a2bd8d0c746 subnet=0b9c9ad8--4c96-afda-ecb7e53dc519
Added interface 4e830e98-e24b-45dd-bc6e-338e47b31b1b to router 4bef86df-3b13-4b23-80f0-3a2bd8d0c746.

• 然后，我们看，Linux 网络配置上的变化，首先是network namespace， 你会发现名为qrouter-4bef86df-3b13-4b23-80f0-3a2bd8d0c746 的Linux Namespace创建成功

stack@ubuntu:~/devstack$ sudo ip netns exec qrouter-4bef86df-3b13-4b23-80f0-3a2bd8d0c746  ip addr
: lo: <LOOPBACK,UP,LOWER_UP> mtu  qdisc noqueue state UNKNOWN group default
    link/loopback ::::: brd :::::
    inet 127.0.0.1/ scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::/ scope host
       valid_lft forever preferred_lft forever
: qr-4e830e98-e2: <BROADCAST,UP,LOWER_UP> mtu  qdisc noqueue state UNKNOWN group default
    link/ether fa::3e:8c:4d:c7 brd ff:ff:ff:ff:ff:ff
    inet 40.40.40.1/ brd 40.40.40.255 scope global qr-4e830e98-e2
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe8c:4dc7/ scope link
       valid_lft forever preferred_lft forever
: qr-8fc6f138-: <BROADCAST,UP,LOWER_UP> mtu  qdisc noqueue state UNKNOWN group default
    link/ether fa::3e::b5: brd ff:ff:ff:ff:ff:ff
    inet 30.30.30.1/ brd 30.30.30.255 scope global qr-8fc6f138-
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe67:b514/ scope link
       valid_lft forever preferred_lft forever

• 上图的qr-4e830e98-e2和qr-8fc6f138-15，可以理解为vmnet1和vmnet2的gateway， 同时它们也都连接在ovs的br-int bridge上，如下：

stack@ubuntu:~/devstack$ sudo ovs-vsctl show
.......
    Bridge br-int
        fail_mode: secure
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "qr-4e830e98-e2"
            tag:
            Interface "qr-4e830e98-e2"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "int-br-eth1"
            Interface "int-br-eth1"
                type: patch
                options: {peer="phy-br-eth1"}
        Port "qr-8fc6f138-15"
            tag:
            Interface "qr-8fc6f138-15"
                type: internal
        Port "qr-a6466103-17"
            tag:
            Interface "qr-a6466103-17"
                type: internal

从上面的输出可以看出"qr-4e830e98-e2" is tagged with 3, "qr-8fc6f138-15"is tagged with 2

• 接下来，我要在vmnet1和vmnet2上分别创建一个virtual machine instance，看看openstack neutron中发生了什么！

#注意，这个一定要先增加这个，不然，后面的vm启动无法得到IP， 参见我的异常处理部分
stack@ubuntu:~/devstack$ nova secgroup-add-rule default icmp - - 0.0.0.0/
stack@ubuntu:~/devstack$ nova secgroup-add-rule default tcp   0.0.0.0/0
#创建两个vm
stack@ubuntu:~/devstack$ nova boot --image 54bdfba1-27f7-4fae-99f1-cfbbca78c9bd --flavor m1.tiny --nic net-id=54028daa-9158-4fc7-9f97-d400dc801e97 vm1
stack@ubuntu:~/devstack$ nova boot --image 54bdfba1-27f7-4fae-99f1-cfbbca78c9bd --flavor m1.tiny --nic net-id=7b0394ec-34c0-48d3-a17d-82ef9bbc2cd9 vm2
#查看vm是否ready
stack@ubuntu:~/devstack$ nova list
+--------------------------------------+------+--------+------------+-------------+-------------------+
| ID                                   | Name | Status | Task State | Power State | Networks          |
+--------------------------------------+------+--------+------------+-------------+-------------------+
| b22e706b-a1aa-4f73-9bd5-5f70bbf7e299 | vm1  | ACTIVE | -          | Running     | vmnet1=30.30.30.2 |
| 1f4fad7d-8eb9-4011-b196-337dea3d7d66 | vm2  | ACTIVE | -          | Running     | vmnet2=40.40.40.2 |
+--------------------------------------+------+--------+------------+-------------+-------------------+

现在这两个ip 30.30.30.2 和40.40.40.2是无法直接ping同的，因为是被Linux Namespace隔离的

stack@ubuntu:~/devstack$ ping 30.30.30.2
PING 30.30.30.2 (30.30.30.2) () bytes of data.
^C
--- 30.30.30.2 ping statistics ---
 packets transmitted,  received, % packet loss, time 3000ms
 
stack@ubuntu:~/devstack$ ping 40.40.40.2
PING 40.40.40.2 (40.40.40.2) () bytes of data.
^C
--- 40.40.40.2 ping statistics ---
 packets transmitted,  received, % packet loss, time 3000ms

• 你可以这样在主机上ssh到vm上，这里我们以ssh到vm1为例：首先确定vm1所在的network为vmnet1， vmnet1对应的namespace为qdhcp-54028daa-9158-4fc7-9f97-d400dc801e97，然后

ip netns exec qqdhcp-54028daa-9158-4fc7-9f97-d400dc801e97 ssh cirros@30.30.30.2

密码为   cubswin:)

这样一个基于neutron network的devstack环境就搭建好了，大家有兴趣的话，可以一起探讨关于neutron跟多的东西，推荐大家几篇oracle上的文章，讲的深入浅出的，很不错

https://blogs.oracle.com/ronen/entry/diving_into_openstack_network_architecture

https://blogs.oracle.com/ronen/entry/diving_into_openstack_network_architecture1

https://blogs.oracle.com/ronen/entry/diving_into_openstack_network_architecture2

https://blogs.oracle.com/ronen/entry/diving_into_openstack_network_architecture3

https://blogs.oracle.com/ronen/entry/running_openstack_icehouse_with_zfs