OpenStack 实现技术分解 (5) 应用开发 — 使用 OpenStackClients 进行二次开发

文件夹

• 文件夹
• 前文列表
• 參考阅读
• 前言
• OpenStackClients
  • 使用 OpenStackClients 获取 project_client object 的 demo
  • 调用 project_client object 实例方法实现对 project 操作的 demo
• 最后

前文列表

OpenStack 实现技术分解 (1) 开发环境 — Devstack 部署案例具体解释

OpenStack 实现技术分解 (2) 虚拟机初始化工具 — Cloud-Init & metadata & userdata

OpenStack 实现技术分解 (3) 开发工具 — VIM & dotfiles

OpenStack 实现技术分解 (4) 通用技术 — TaskFlow

參考阅读

Openstack API 类型 & REST 风格

OpenStackClients

Python bindings to the OpenStack Identity API

Python Bindings for the OpenStack Images API

Python bindings to the OpenStack Nova API

Cinder Python API

Python bindings to the OpenStack Networking API

前言

OpenStack 为用户提供了三种操作方式, Web界面/CLI/RESTAPI, 实际上前两者是对 RESTAPI 做了两种不同形式的包装, 使用户能够通过网页或者指令行的方式来调用 RESTAPI 接口.

本篇博文主要记录了 使用 OpenStackClients (OSC 命令行客户端) 项目所提供了Python Bindings API 来进行二次开发的技巧, 以及实现一个启动虚拟机并部署 Workpass+MySQL 自己主动化脚本的 Demo. 源代码详见 GitHub: openstackclient-api-demo

在介绍 OpenStackClients 之前, 我们能够尝试直接使用 curl 指令来查看一个 tenant 所含有的虚拟机列表.

• Step 1: 获取账户 admin 的身份验证 Temporary token

curl -k -X 'POST' -v http://200.21.18.2:5000/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"fanguiju"}}}' -H 'Content-type: application/json' | python -mjson.tool

Response:

{
    "access": {
        "metadata": {
            "is_admin": 0,
            "roles": []
        },
        "serviceCatalog": [],
        "token": {
            "audit_ids": [
                "AOMhHXq_Qx2Nz41RVoUy7g"
            ],
            "expires": "2017-03-19T05:41:20Z",
            "id": "16ae22b6c36f4ebc97938f51b7d0631b",
            "issued_at": "2017-03-19T04:41:20.039145"
        },
        "user": {
            "id": "135b2cb86962401c82044fd4ca9daae4",
            "name": "admin",
            "roles": [],
            "roles_links": [],
            "username": "admin"
        }
    }
}

获取到 Temporary token: 16ae22b6c36f4ebc97938f51b7d0631b, 表示我们的账户信息通过了验证流程.

• Step 2: 使用 Temporary token 来获取 tenants list

curl -X 'GET' -H  "X-Auth-Token:16ae22b6c36f4ebc97938f51b7d0631b" -v http://200.21.18.2:5000/v2.0/tenants | python -mjson.tool

Response:

{
    "tenants": [
        {
            "description": "",
            "enabled": true,
            "id": "6c4e4d58cb9d4451b36e774b348e8813",
            "name": "admin"
        },
        {
            "description": "",
            "enabled": true,
            "id": "ad9a69f3da8f4aa280389fcdf855aeb5",
            "name": "demo"
        }
    ],
    "tenants_links": []
}

能够看出 admin 账户含有 admin tenant 和 demo tenant.

• Step 3: 获取 admin tenant 的 token 信息

curl -k -X 'POST' -v http://200.21.18.2:5000/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"fanguiju"},"tenantId":"6c4e4d58cb9d4451b36e774b348e8813"}}' -H 'Content-type: application/json' | python -mjson.tool

Response:

{
    "access": {
        "metadata": {
            "is_admin": 0,
            "roles": [
                "14a6da35e3ef4e47a540c6608aa00ca7"
            ]
        },
        "serviceCatalog": [
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813",
                        "id": "705f599f3bae42ceb4a70616d9663ad8",
                        "internalURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813",
                        "publicURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "nova",
                "type": "compute"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813",
                        "id": "39ceecd18b754c9495834d0155fe91bf",
                        "internalURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813",
                        "publicURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "egis",
                "type": "recovery"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813",
                        "id": "218769a91d0943ff8db44887645ec0ff",
                        "internalURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813",
                        "publicURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "cinderv2",
                "type": "volumev2"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:9292",
                        "id": "7f2f8036b0194ea0bd5231710b2cddf4",
                        "internalURL": "http://200.21.18.2:9292",
                        "publicURL": "http://200.21.18.2:9292",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "glance",
                "type": "image"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813",
                        "id": "054567bc62ce4b4fbdbdcd7c3a23748e",
                        "internalURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813",
                        "publicURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "nova_legacy",
                "type": "compute_legacy"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813",
                        "id": "2eefe27748774693b635bf48f486f225",
                        "internalURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813",
                        "publicURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "cinder",
                "type": "volume"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:8773/",
                        "id": "4d8f727748924cdf9d23591bad2bbd19",
                        "internalURL": "http://200.21.18.2:8773/",
                        "publicURL": "http://200.21.18.2:8773/",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "ec2",
                "type": "ec2"
            },
            {
                "endpoints": [
                    {
                        "adminURL": "http://200.21.18.2:35357/v2.0",
                        "id": "16e2a0df7fa64c8cbcdb5936e23b19cc",
                        "internalURL": "http://200.21.18.2:5000/v2.0",
                        "publicURL": "http://200.21.18.2:5000/v2.0",
                        "region": "RegionOne"
                    }
                ],
                "endpoints_links": [],
                "name": "keystone",
                "type": "identity"
            }
        ],
        "token": {
            "audit_ids": [
                "4zrwvCd7TySk7jJKuO4G1Q"
            ],
            "expires": "2017-03-19T05:48:41Z",
            "id": "74e396f8202b481a9cbd95b319a4314b",
            "issued_at": "2017-03-19T04:48:42.002243",
            "tenant": {
                "description": "",
                "enabled": true,
                "id": "6c4e4d58cb9d4451b36e774b348e8813",
                "name": "admin"
            }
        },
        "user": {
            "id": "135b2cb86962401c82044fd4ca9daae4",
            "name": "admin",
            "roles": [
                {
                    "name": "admin"
                }
            ],
            "roles_links": [],
            "username": "admin"
        }
    }
}

须要注意的是, 这一步骤所获取的 Tenant token 是差别于 Temporary token 的, Temporary token 作为暂时 token 是为了实现多租户的场景所提供的鉴权条件(外部鉴权). 而 Tenant token 才是联系不同 OpenStack Project 间的认证通行证(内部鉴权). 从这一步骤能够看出想要获取 Tenant token 就须要同一时候向 Keystone 提供账户信息和 tenant_id, 此时用户不仅得到了 Tenant token 还获取了对应的 endpoints list. 而且用户能够通过 endpints list 进一步的去訪问注冊在 Keystone 中的其它 OpenStack 组件.

• Step 4: 使用 Tenant token 和 tenant_id 获取 admin tenant 的虚拟机列表

curl -v -H "X-Auth-Token:74e396f8202b481a9cbd95b319a4314b" http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers

Response:

{
    "servers": [
        {
            "id": "138ecea2-1656-46bd-aefd-39449e11c356",
            "links": [
                {
                    "href": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers/138ecea2-1656-46bd-aefd-39449e11c356",
                    "rel": "self"
                },
                {
                    "href": "http://200.21.18.2:8774/6c4e4d58cb9d4451b36e774b348e8813/servers/138ecea2-1656-46bd-aefd-39449e11c356",
                    "rel": "bookmark"
                }
            ],
            "name": "aju_test_dvs"
        },
        {
            "id": "42da5d12-a470-4193-8410-0209c04f333a",
            "links": [
                {
                    "href": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers/42da5d12-a470-4193-8410-0209c04f333a",
                    "rel": "self"
                },
                {
                    "href": "http://200.21.18.2:8774/6c4e4d58cb9d4451b36e774b348e8813/servers/42da5d12-a470-4193-8410-0209c04f333a",
                    "rel": "bookmark"
                }
            ],
            "name": "TestVMwareInterface"
        }
    ]
}

终于, 我们从 Response 中得到了 admin tenant 所具有的两台虚拟机的信息.

完整的 RESTAPI 请求流程图片例如以下:

显然, 使用 curl 请求 RESTAPI 的方式过于繁复, 不能满足用户对 OpenStack 多方位的应用需求(e.g. 实现 OpenStack 的自己主动化操作脚本). 对此, OpenStack 为用户提供了更高级别的 RESTAPI 调用封装 — OpenStackClients

OpenStackClients

(摘自 OpenStackClients 官方文档)Each OpenStack project has a related client project that includes Python API bindings and a CLI.

每个 OpenStack 项目都具有一个包括了 Python API bindings 和 CLI 相关的 client 项目. 如图:

有图可见, OpenStackClients 项目主要实现了将 OpenStack 计算(Compute)、身份识别(Keystone)、镜像(Glance)、网络(Neutron)、对象存储(Swift)和卷存储(Cinder) 等核心组件所提供出来的 REST API 整合封装为具有统一指令结构的 CLI. 简而言之, 就是 OpenStackClients 项目使得用户能够通过 CLI 的形式调用以上组件提供的 REST API, 从而实现操作. 而且我们也能够从代码的层面直接导入 OpenStackClients, 更加便于开发人员对 OpenStack 功能模块的调用.

使用 OpenStackClients 获取 project_client object 的 demo

vim openstack_clients.py

#!/usr/bin/env python
#encoding=utf8

from openstackclient.identity.client import identity_client_v2
from keystoneclient import session as identity_session
import glanceclient
import novaclient.client as novaclient
import cinderclient.client as cinderclient

# 定义 project_client version
NOVA_CLI_VER = 2
GLANCE_CLI_VER = 2
CINDER_CLI_VER = 2

class OpenstackClients(object):
    """Clients generator of openstack."""

    def __init__(self, auth_url, username, password, tenant_name):

        ### Identity authentication via keystone v2
        # An authentication plugin to authenticate the session with.
        # 通过身份验证信息获取 keystone 的 auth object
        # Keystoneclient v2 的具体使用介绍请浏览 https://docs.openstack.org/developer/python-keystoneclient/using-api-v2.html
        auth = identity_client_v2.v2_auth.Password(
            auth_url=auth_url,          # http://200.21.18.3:35357/v2.0/
            username=username,          # admin
            password=password,          # fanguiju
            tenant_name=tenant_name)    # admin
        try:
            # 通过 auth object 获取 Keystone 的 session object
            self.session = identity_session.Session(auth=auth)
        except Exception as err:
            raise

        # Return a token as provided by the auth plugin.
        # 通过 session object 获取 Tenant token
        self.token = self.session.get_token()

    def get_glance_client(self, interface='public'):
        """Get the glance-client object."""

        # Get an endpoint as provided by the auth plugin.
        # 默认获取 glance project 的 public endpoint
        glance_endpoint = self.session.get_endpoint(service_type="image",
                                                    interface=interface)
        # Client for the OpenStack Images API.
        # 通过 glance endpoint 和 token 获取 glance_client object
        # 然后就能够使用 glance_client 调用事实上例方法来实现对 glance project 的操作了
        # glanceclient v2 所提供的实例方法列表请浏览 https://docs.openstack.org/developer/python-glanceclient/ref/v2/images.html
        glance_client = glanceclient.Client(GLANCE_CLI_VER,
                                            endpoint=glance_endpoint,
                                            token=self.token)
        return glance_client

    def get_nova_client(self):
        """Get the nova-client object."""

        # Initialize client object based on given version. Don't need endpoint.
        # 也能够 不指定 endpoint 的类型, 仅使用 session object 来获取 nove_client
        # novaclient v2 的实例方法列表请浏览 https://docs.openstack.org/developer/python-novaclient/api.html#usage
        nova_client = novaclient.Client(NOVA_CLI_VER, session=self.session)
        return nova_client

    def get_cinder_client(self, interface='public'):
        """Get the cinder-client object."""

        cinder_endpoint = self.session.get_endpoint(service_type='volume',
                                                    interface=interface)
        # cinder_client v2 的实例方法列表请查看 https://docs.openstack.org/developer/python-cinderclient/
        cinder_client = cinderclient.Client(CINDER_CLI_VER, session=self.session)
        return cinder_client

调用 project_client object 实例方法实现对 project 操作的 demo

vim auto_dep.py

#!/usr/bin/env python
#encoding=utf8

import os
from os import path
import time

import openstack_clients as os_cli

# FIXME(Fan Guiju): Using oslo_config and logging
AUTH_URL = 'http://200.21.18.3:35357/v2.0/'
USERNAME = 'admin'
PASSWORD = 'fanguiju'
PROJECT_NAME = 'admin'

DISK_FORMAT = 'qcow2'
IMAGE_NAME = 'ubuntu_server_1404_x64'
IMAGE_PATH = path.join(path.curdir, 'images',
                       '.'.join([IMAGE_NAME, DISK_FORMAT]))

MIN_DISK_SIZE_GB = 20

KEYPAIR_NAME = 'jmilkfan-keypair'
KEYPAIT_PUB_PATH = '/home/stack/.ssh/id_rsa.pub'

DB_NAME = 'blog'
DB_USER = 'wordpress'
DB_PASS = 'fanguiju'
DB_BACKUP_SIZE = 5
DB_VOL_NAME = 'mysql-volume'
DB_INSTANCE_NAME = 'AUTO-DEP-DB'
MOUNT_POINT = '/dev/vdb'

BLOG_INSTANCE_NAME = 'AUTO-DEP-BLOG'

TIMEOUT = 60

class AutoDep(object):
    def __init__(self, auth_url, username, password, tenant_name):
        # 实例化上述的 openstack_client.OpenstackClients 的对象
        openstack_clients = os_cli.OpenstackClients(
            auth_url,
            username,
            password,
            tenant_name)

        # 通过 openstack_clients 的实例方法获取 project_client 对象
        self._glance = openstack_clients.get_glance_client()
        self._nova = openstack_clients.get_nova_client()
        self._cinder = openstack_clients.get_cinder_client()

    def _wait_for_done(self, objs, target_obj_name):
        """Wait for action done."""
        count = 0
        while count <= TIMEOUT:
            for obj in objs.list():
                if obj.name == target_obj_name:
                    return
            time.sleep(3)
            count += 3
        raise

    def upload_image_to_glance(self):
        images = self._glance.images.list()
        for image in images:
            if image.name == IMAGE_NAME:
                return image
        # 调用 glanceclient.images.create method 创建一个 image object.
        new_image = self._glance.images.create(name=IMAGE_NAME,
                                               disk_format=DISK_FORMAT,
                                               container_format='bare',
                                               min_disk=MIN_DISK_SIZE_GB,
                                               visibility='public')
        # Open image file with read+binary.
        # 调用 glanceclient.images.upload method 上传一个 image
        self._glance.images.upload(new_image.id, open(IMAGE_PATH, 'rb'))
        self._wait_for_done(objs=self._glance.images,
                            target_obj_name=IMAGE_NAME)
        image = self._glance.images.get(new_image.id)
        return image

    def create_volume(self):
        # 调用 cinderclient.volumes.list method 获取 volumes 的列表
        volumes = self._cinder.volumes.list()
        for volume in volumes:
            if volume.name == DB_VOL_NAME:
                return volume
        # cinderclient.v2.volumes:VolumeManager
        # 调用 minderclient.volumes.create method 创建一个 volume
        new_volume = self._cinder.volumes.create(
            size=DB_BACKUP_SIZE,
            name=DB_VOL_NAME,
            volume_type='lvmdriver-1',
            availability_zone='nova',
            description='backup volume of mysql server.')
        if new_volume:
            return new_volume
        else:
            raise

    def get_flavor_id(self):
        # 调用 novaclient.flavors.list method 获取全部 flavors 的列表
        flavors = self._nova.flavors.list()
        for flavor in flavors:
            if flavor.disk == MIN_DISK_SIZE_GB:
                return flavor.id

    def _get_ssh_pub_key(self):
        if not path.exists(KEYPAIT_PUB_PATH):
            raise
        return open(KEYPAIT_PUB_PATH, 'rb').read()

    def import_keypair_to_nova(self):
        # 调用 novaclient.keypairs.list method 获取 keypairs 的列表
        keypairs = self._nova.keypairs.list()
        for keypair in keypairs:
            if keypair.name == KEYPAIR_NAME:
                return None
        keypair_pub = self._get_ssh_pub_key()
        # 调用 nova client.keypairs.create method 创建 keypair
        self._nova.keypairs.create(KEYPAIR_NAME, public_key=keypair_pub)

    def nova_boot(self, image, volume):
        flavor_id = self.get_flavor_id()
        self.import_keypair_to_nova()
        db_instance = False

        # 调用 novaclient.servers.list method 获取 servers 的列表
        servers = self._nova.servers.list()
        server_names = []
        for server in servers:
            server_names.append(server.name)
            if server.name == DB_INSTANCE_NAME:
                db_instance = server

        if not db_instance:
            # Create the mysql server
            db_script_path = path.join(path.curdir, 'scripts/db_server.txt')
            db_script = open(db_script_path, 'r').read()
            db_script = db_script.format(DB_NAME, DB_USER, DB_PASS)
            # 通过 nova client.servers.create method 创建一个 server
            # 这里由于希望创建 server 并对其进行预设置, 所以使用了 userdata 參数
            # userdata 參数会接收一个 script 文件, 并在 server 第一次启动的时候运行
            db_instance = self._nova.servers.create(
                # FIXME(Fan Guiju): Using the params `block_device_mapping` to attach the volume.
                DB_INSTANCE_NAME,
                image.id,
                flavor_id,
                key_name=KEYPAIR_NAME,
                userdata=db_script)
            # 通过 novaclient.server.get method 和 server_id 来获取单个 server 的具体信息
            if not self._nova.server.get(db_instance.id):
                self._wait_for_done(objs=self._nova.servers,
                                    target_obj_name=DB_INSTANCE_NAME)
        # Attach the mysql-vol to mysql server, device type is `vd`.
        # 通过 cinderclient.volumes.attach method 挂在一个 volume 到 server 上
        # mountpoint 參数运行了挂载到 server 的设备路径, e.g. /dev/vdb
        self._cinder.volumes.attach(volume=volume,
                                    instance_uuid=db_instance.id,
                                    mountpoint=MOUNT_POINT)
        time.sleep(5)

        if BLOG_INSTANCE_NAME not in server_names:
            # Create the wordpress blog server
            # Nova-Network
            db_instance_ip = self._nova.servers.\
                get(db_instance.id).networks['private'][0]
            blog_script_path = path.join(path.curdir, 'scripts/blog_server.txt')
            blog_script = open(blog_script_path, 'r').read()
            blog_script = blog_script.format(DB_NAME,
                                             DB_USER,
                                             DB_PASS,
                                             db_instance_ip)
            self._nova.servers.create(BLOG_INSTANCE_NAME,
                                      image.id,
                                      flavor_id,
                                      key_name=KEYPAIR_NAME,
                                      userdata=blog_script)
            self._wait_for_done(objs=self._nova.servers,
                                target_obj_name=BLOG_INSTANCE_NAME)

        servers = self._nova.servers.list(search_opts={'all_tenants': True})
        return servers

def main():
    """FIXME(Fan Guiju): Operation manual."""
    os.environ['LANG'] = 'en_US.UTF8'

    deploy = AutoDep(auth_url=AUTH_URL,
                     username=USERNAME,
                     password=PASSWORD,
                     tenant_name=PROJECT_NAME)
    image = deploy.upload_image_to_glance()
    volume = deploy.create_volume()
    deploy.nova_boot(image, volume)

if __name__ == '__main__':
    main()

最后

上面给出了一个自己主动化运行 OpenStack Project 功能模块的脚本, 但实际上, 我们能够使用 OpenStackClients 进行更加复杂的工作, 比如: 自己定义一个新的 OpenStack Project, 并使之与 OpenStack 的原生 Project 进行互动, 这才是真正意义上的二次开发.