前文列表

横向扩展裸金属管理服务节点

当前云基础设备服务清单：

[root@controller ~]# openstack compute service list

+----+------------------+------------+----------+---------+-------+----------------------------+

| ID | Binary           | Host       | Zone     | Status  | State | Updated At                 |

+----+------------------+------------+----------+---------+-------+----------------------------+

|  1 | nova-scheduler   | controller | internal | enabled | up    | 2019-05-08T09:51:29.000000 |

|  2 | nova-consoleauth | controller | internal | enabled | up    | 2019-05-08T09:51:22.000000 |

|  3 | nova-conductor   | controller | internal | enabled | up    | 2019-05-08T09:51:27.000000 |

|  6 | nova-compute     | controller | nova     | enabled | up    | 2019-05-08T09:51:24.000000 |

|  7 | nova-compute     | compute    | nova     | enabled | up    | 2019-05-08T09:51:24.000000 |

+----+------------------+------------+----------+---------+-------+----------------------------+

[root@controller ~]# openstack volume service list

+------------------+-----------------+------+---------+-------+----------------------------+

| Binary           | Host            | Zone | Status  | State | Updated At                 |

+------------------+-----------------+------+---------+-------+----------------------------+

| cinder-scheduler | controller      | nova | enabled | up    | 2019-05-08T09:51:40.000000 |

| cinder-volume    | controller@lvm  | nova | enabled | up    | 2019-05-08T09:51:43.000000 |

| cinder-volume    | controller@ceph | nova | enabled | up    | 2019-05-08T09:51:39.000000 |

| cinder-backup    | controller      | nova | enabled | up    | 2019-05-08T09:51:42.000000 |

+------------------+-----------------+------+---------+-------+----------------------------+

[root@controller ~]# openstack network agent list

+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+

| ID                                   | Agent Type         | Host       | Availability Zone | Alive | State | Binary                    |

+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+

| 41925586-9119-4709-bc23-4668433bd413 | Metadata agent     | controller | None              | :-)   | UP    | neutron-metadata-agent    |

| 43281ac1-7699-4a81-a5b6-d4818f8cf8f9 | Open vSwitch agent | controller | None              | :-)   | UP    | neutron-openvswitch-agent |

| b815e569-c85d-4a37-84ea-7bdc5fe5653c | DHCP agent         | controller | nova              | :-)   | UP    | neutron-dhcp-agent        |

| d1ef7214-d26c-42c8-ba0b-2a1580a44446 | L3 agent           | controller | nova              | :-)   | UP    | neutron-l3-agent          |

| f55311fc-635c-4985-ae6b-162f3fa8f886 | Open vSwitch agent | compute    | None              | :-)   | UP    | neutron-openvswitch-agent |

+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+

[root@controller ~]# openstack catalog list

+-----------+-----------+------------------------------------------------------------------------+

| Name      | Type      | Endpoints                                                              |

+-----------+-----------+------------------------------------------------------------------------+

| nova      | compute   | RegionOne                                                              |

|           |           |   internal: http://controller:8774/v2.1                                |

|           |           | RegionOne                                                              |

|           |           |   admin: http://controller:8774/v2.1                                   |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:8774/v2.1                                  |

|           |           |                                                                        |

| cinderv2  | volumev2  | RegionOne                                                              |

|           |           |   admin: http://controller:8776/v2/a2b55e37121042a1862275a9bc9b0223    |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:8776/v2/a2b55e37121042a1862275a9bc9b0223   |

|           |           | RegionOne                                                              |

|           |           |   internal: http://controller:8776/v2/a2b55e37121042a1862275a9bc9b0223 |

|           |           |                                                                        |

| neutron   | network   | RegionOne                                                              |

|           |           |   internal: http://controller:9696                                     |

|           |           | RegionOne                                                              |

|           |           |   admin: http://controller:9696                                        |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:9696                                       |

|           |           |                                                                        |

| glance    | image     | RegionOne                                                              |

|           |           |   admin: http://controller:9292                                        |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:9292                                       |

|           |           | RegionOne                                                              |

|           |           |   internal: http://controller:9292                                     |

|           |           |                                                                        |

| keystone  | identity  | RegionOne                                                              |

|           |           |   admin: http://controller:5000/v3/                                    |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:5000/v3/                                   |

|           |           | RegionOne                                                              |

|           |           |   internal: http://controller:5000/v3/                                 |

|           |           |                                                                        |

| placement | placement | RegionOne                                                              |

|           |           |   internal: http://controller:8778                                     |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:8778                                       |

|           |           | RegionOne                                                              |

|           |           |   admin: http://controller:8778                                        |

|           |           |                                                                        |

| cinderv3  | volumev3  | RegionOne                                                              |

|           |           |   internal: http://controller:8776/v3/a2b55e37121042a1862275a9bc9b0223 |

|           |           | RegionOne                                                              |

|           |           |   admin: http://controller:8776/v3/a2b55e37121042a1862275a9bc9b0223    |

|           |           | RegionOne                                                              |

|           |           |   public: http://controller:8776/v3/a2b55e37121042a1862275a9bc9b0223   |

|           |           |                                                                        |

+-----------+-----------+------------------------------------------------------------------------+

BareMetal Node：

ens160: 172.18.22.233/24
[可选] ens192: OvS Provider Bridge NIC of Provisioning Network

配置基础设施

DNS 解析

[root@localhost ~]# cat /etc/hosts

...

172.18.22.231 controller

172.18.22.232 compute

172.18.22.233 baremetal

NTP 时间同步

[root@baremetal ~]# cat /etc/chrony.conf | grep -v ^# | grep -v ^$

server controller iburst

driftfile /var/lib/chrony/drift

makestep 1.0 3

rtcsync

logdir /var/log/chrony

[root@baremetal ~]# systemctl enable chronyd.service

[root@baremetal ~]# systemctl start chronyd.service

[root@baremetal ~]# chronyc sources

210 Number of sources = 1

MS Name/IP address         Stratum Poll Reach LastRx Last sample

===============================================================================

^? controller                    3   6     3     0  -9548us[-9548us] +/-   37ms

YUM 源

yum install centos-release-openstack-rocky -y

yum upgrade -y

yum install python-openstackclient -y

yum install openstack-selinux -y

安装 Ironic（BareMetal）

NOTE：请注意每个步骤的操作节点

添加 Ironic 用户及其鉴权信息

openstack service create --name ironic --description "Ironic baremetal provisioning service" baremetal

openstack user create --domain default --password-prompt ironic

openstack role add --project service --user ironic admin

openstack endpoint create --region RegionOne baremetal admin http://baremetal:6385

openstack endpoint create --region RegionOne baremetal public http://baremetal:6385

openstack endpoint create --region RegionOne baremetal internal http://baremetal:6385

openstack catalog list

安装软件包

yum install openstack-ironic-api openstack-ironic-conductor python-ironicclient -y

配置 ironic-api & ironic-conductor

[DEFAULT]

my_ip=172.18.22.233

transport_url = rabbit://openstack:fanguiju@controller

auth_strategy = keystone

state_path = /var/lib/ironic

debug = True

[api]

port = 6385

[conductor]

automated_clean = false

clean_callback_timeout = 1800

rescue_callback_timeout = 1800

soft_power_off_timeout = 600

power_state_change_timeout = 30

power_failure_recovery_interval = 300

[database]

connection=mysql+pymysql://ironic:fanguiju@controller/ironic?charset=utf8

[dhcp]

dhcp_provider = neutron

[neutron]

auth_type = password

auth_url = http://controller:5000

username=ironic

password=fanguiju

project_name=service

project_domain_id=default

user_domain_id=default

region_name = RegionOne

valid_interfaces=public

[glance]

url = http://controller:9292

auth_url = http://controller:5000

auth_type = password

project_domain_name = default

user_domain_name = default

region_name = RegionOne

project_name = service

username = glance

password = fanguiju

[cinder]

region_name = RegionOne

project_domain_id = default

user_domain_id = default

project_name = service

password = fanguiju

username = ironic

auth_url = http://controller:5000

auth_type = password

[service_catalog]

region_name = RegionOne

project_domain_id = default

user_domain_id = default

project_name = service

password = fanguiju

username = ironic

auth_url = http://controller:5000

auth_type = password

[keystone_authtoken]

auth_type=password

www_authenticate_uri = http://controller:5000

auth_url = http://controller:5000

username=ironic

password=fanguiju

project_name=service

project_domain_name=default

user_domain_name=default

NOTE：本文 ironic-api 和 ironic-conductor 同节点

创建数据库（Controller）

CREATE DATABASE ironic CHARACTER SET utf8;

GRANT ALL PRIVILEGES ON ironic.* TO 'ironic'@'localhost' IDENTIFIED BY 'fanguiju';

GRANT ALL PRIVILEGES ON ironic.* TO 'ironic'@'%' IDENTIFIED BY 'fanguiju';

初始化数据库

ironic-dbsync --config-file /etc/ironic/ironic.conf create_schema

启动服务

systemctl enable openstack-ironic-api openstack-ironic-conductor

systemctl start openstack-ironic-api openstack-ironic-conductor

systemctl status openstack-ironic-api openstack-ironic-conductor

验证

[root@controller ~]# openstack baremetal driver list

+---------------------+----------------+

| Supported driver(s) | Active host(s) |

+---------------------+----------------+

| ipmi                | baremetal      |

+---------------------+----------------+

安装 Nova Compute（BareMetal）

NOTE：该节点的 nova-compute service 作为 BareMetal 的管理与调度层，所以不需要支持嵌套虚拟化。e.g.

[root@baremetal ~]# egrep -c '(vmx|svm)' /proc/cpuinfo

0

软件包

yum install openstack-nova-compute -y

配置

# /etc/nova/nova.conf

[DEFAULT]

my_ip = 172.18.22.233

transport_url = rabbit://openstack:fanguiju@controller

debug = True

use_neutron = true

compute_driver=ironic.IronicDriver

firewall_driver=nova.virt.firewall.NoopFirewallDriver

reserved_host_cpus=0

reserved_host_memory_mb=0

reserved_host_disk_mb=0

update_resources_interval=10

cpu_allocation_ratio=1.0

ram_allocation_ratio=1.0

disk_allocation_ratio=1.0

bandwidth_poll_interval=-1

[ironic]

api_retry_interval = 5

api_max_retries = 300

auth_type=password

auth_url=http://controller:5000/v3

project_name=service

username=ironic

password=fanguiju

project_domain_name=default

user_domain_name=default

[glance]

api_servers = http://controller:9292

[cinder]

os_region_name = RegionOne

[neutron]

url = http://controller:9696

auth_url = http://controller:5000

auth_type = password

project_domain_name = default

user_domain_name = default

region_name = RegionOne

project_name = service

username = neutron

password = fanguiju

[placement]

region_name = RegionOne

project_domain_name = Default

project_name = service

auth_type = password

user_domain_name = Default

auth_url = http://controller:5000/v3

username = placement

password = fanguiju

启动服务

systemctl enable openstack-nova-compute.service

systemctl start openstack-nova-compute.service

systemctl status openstack-nova-compute.service

将 BareMetal Node 的 nova-compute 注册到 Cell

[root@controller ~]# nova-manage cell_v2 discover_hosts --by-service

[root@controller ~]# nova-manage cell_v2 list_cells

+-------+--------------------------------------+------------------------------------+-------------------------------------------------+----------+

|  Name |                 UUID                 |           Transport URL            |               Database Connection               | Disabled |

+-------+--------------------------------------+------------------------------------+-------------------------------------------------+----------+

| cell0 | 00000000-0000-0000-0000-000000000000 |               none:/               | mysql+pymysql://nova:****@controller/nova_cell0 |  False   |

| cell1 | 51e0592e-b622-4365-814c-98ce96bcce7b | rabbit://openstack:****@controller |    mysql+pymysql://nova:****@controller/nova    |  False   |

+-------+--------------------------------------+------------------------------------+-------------------------------------------------+----------+

[root@controller ~]# nova-manage cell_v2 list_hosts

+-----------+--------------------------------------+------------+

| Cell Name |              Cell UUID               |  Hostname  |

+-----------+--------------------------------------+------------+

|   cell1   | 51e0592e-b622-4365-814c-98ce96bcce7b | baremetal  |

|   cell1   | 51e0592e-b622-4365-814c-98ce96bcce7b |  compute   |

|   cell1   | 51e0592e-b622-4365-814c-98ce96bcce7b | controller |

+-----------+--------------------------------------+------------+

验证

[root@controller ~]# openstack compute service list

+----+------------------+------------+----------+---------+-------+----------------------------+

| ID | Binary           | Host       | Zone     | Status  | State | Updated At                 |

+----+------------------+------------+----------+---------+-------+----------------------------+

|  1 | nova-scheduler   | controller | internal | enabled | up    | 2019-05-08T11:28:57.000000 |

|  2 | nova-consoleauth | controller | internal | enabled | up    | 2019-05-08T11:28:55.000000 |

|  3 | nova-conductor   | controller | internal | enabled | up    | 2019-05-08T11:28:58.000000 |

|  6 | nova-compute     | controller | nova     | enabled | up    | 2019-05-08T11:28:54.000000 |

|  7 | nova-compute     | compute    | nova     | enabled | up    | 2019-05-08T11:28:56.000000 |

|  8 | nova-compute     | baremetal  | nova     | enabled | up    | 2019-05-08T11:28:58.000000 |

+----+------------------+------------+----------+---------+-------+----------------------------+

因为是裸机环境，所以关闭 Nova Scheduler 跟踪虚拟机信息

[filter_scheduler]

track_instance_changes=False

重启 Nova Scheduler

systemctl restart openstack-nova-scheduler

配置 Neutron 提供 Provisioning Network

抽象网络模型

Provisioning Network 是注册并部署裸金属实例的网络，所以要求连通 IPMI、DHCP、PXE 以及裸金属服务器。其中 DHCP 由 Neutron DHCP Agent 提供，PXE 由 Ironic Conductor 提供。有 Flat、VLAN、SDN 等多种方案，本文选择使用 Flat 类型。
Cleaning Network 是初始化裸金属节点的网络，主要完成抹盘、初始化硬件配置信息等工作，所以要求连通 IPMI 以及裸金属服务器。
Tenant Network 是常规的 Neutron 租户网络，在裸金属服务器部署完成之后会将裸金属节点的部署端口（PXE 网卡）从 Provisioning Network 切换到 Tenant Network。在此之上可以通过 L3 Router 完成裸机与虚拟机的跨网络通信。

Flat 网络模型

Flat 网络模型下，所有的物理服务器（裸金属节点、OpenSack 节点）都处于同一个 Flat（扁平）网络中，无需交换机或者为透明交换机。物理机网络由运维人员完成预配置，Neutron 在 Flat 网络模型中只负责提供 DHCP 服务。

IPMI OOB Network、OpenStack MGMT Network 二层互通：Ironic Conductor 节点可以通过 ipmitool 管理 BM Node。
Provisioning Network 复用 External Network 与 BM Nodes 二层互通：BM Node 可以从 Provisioning Network DHCP 获取到 IP 地址和 TFTP 服务器的访问接口。
OpenStack MGMT Network 与 External Network 二层互通：BM Node 可以经 OpenStack MGMT Network 访问到 TFTP 服务器；运行在 BM Node 上的 IPA 与 Ironic Conductor 通信。

VLAN 网络模型

VLAN 网络模型下，Neutron 可以通过 Networking Generic Switch 来实现对物理交换机的接管。在此基础上无论是 Provisioning Network 还是 Tenant Network 都通过 VLAN 网络类型（Physical Network）接入到物理交换机，由 Neutron 完成对物理交换机端口配置的切换控制。比如：部署时，BM Node 接入端口为 Provisioning Network VLAN；部署完成后，BM Node 接入端口为 Tenant Network VLAN。

Provisioning Network 与 Tenant Network 复用一个 VLAN Physical Network，并具有不同的 VLAN ID。
Provisioning Network、Tenant Network、BM Nodes 全部接入一个二层交换网络，通过对 BM Node 上联端口 VLAN ID 的动态配置实现 BM Node 接入网络的切换。
- 部署时，BM Node VLAN 接入 Provisioning Network：BM Node 从 DHCP 获取 IP 地址。
- 部署完成后，BM Node VLAN 接入 Tenant Network：BM Node 与同租户网络中的其他节点互通。
OpenStack MGMT Network 要求与 IPMI OOB Network、Provisioning Network、物理交换机管理 IP 打通，可以部署为同一个网络。
- 与 IPMI OOB Network 打通：Ironic Conductor 通过 ipmitool 接管 IPMI。
- 与 Provisioning Network 打通：BM Node 访问 TFTP 服务器；运行在 BM Node 上的 IPA 与 Ironic Conductor 通信。
- 与物理交换机管理 IP 打通：Neutron 接管交换机。

配置 Open vSwitch Agent（Controller）

NOTE：因为在本文环境中已经存在一个基础的 OpenStack 环境，包括已经配置好了的 Flat Physical Network。这里本质的需求是打造一个可以提供 Provisioning Network 的基础网络配置，所以根据个人的实际情况来选择是否执行下列配置。

安装 OvS Agent

yum install openstack-neutron-openvswitch ipset -y

配置 OvS Agent

# /etc/neutron/plugins/ml2/openvswitch_agent.ini

[ovs]

datapath_type = system

bridge_mappings = provider:br-provider

[agent]

l2_population = True

[securitygroup]

firewall_driver = openvswitch

启动 vswitchd 守护进程

systemctl enable openvswitch

systemctl start openvswitch

systemctl status openvswitch

手动创建 OvS Provider Bridge

ovs-vsctl add-br br-provider

ovs-vsctl add-port br-provider ens192

启动 OvS Agent

systemctl enable neutron-openvswitch-agent.service

systemctl start neutron-openvswitch-agent.service

systemctl status neutron-openvswitch-agent.service

验证

[root@baremetal ~]# ovs-vsctl show

52fd6a40-ed6b-460c-8af9-8b13239a9ad5

    Manager "ptcp:6640:127.0.0.1"

        is_connected: true

    Bridge br-provider

        Controller "tcp:127.0.0.1:6633"

            is_connected: true

        fail_mode: secure

        Port br-provider

            Interface br-provider

                type: internal

        Port "ens192"

            Interface "ens192"

        Port phy-br-provider

            Interface phy-br-provider

                type: patch

                options: {peer=int-br-provider}

    Bridge br-int

        Controller "tcp:127.0.0.1:6633"

            is_connected: true

        fail_mode: secure

        Port int-br-provider

            Interface int-br-provider

                type: patch

                options: {peer=phy-br-provider}

        Port br-int

            Interface br-int

                type: internal

    ovs_version: "2.10.1"

[root@controller ~]# openstack network agent list

+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+

| ID                                   | Agent Type         | Host       | Availability Zone | Alive | State | Binary                    |

+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+

| 02ac17a4-9a27-4dd6-b11f-a6eada895432 | Open vSwitch agent | baremetal  | None              | :-)   | UP    | neutron-openvswitch-agent |

...

+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+

在 Neutron ML2 配置 Flat Physical Network，用于创建 Provisioning Network

# /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2_type_flat]

flat_networks = provider,external

[ml2_type_vlan]

network_vlan_ranges = provider1:1:1000

[ml2_type_vxlan]

vni_ranges = 1:1000

重启 Neutron Server

systemctl restart neutron-server

配置 Networking-baremetal ML2 mechanism driver（Controller）[可选]

以往使用 Flat 网络接口时创建的裸机 Port 状态会一直处于 DOWN，但裸机操作系统的部署依然能够成功且正常工作。而 Networking-baremetal 项目正是希望解决裸机 Port 状态不正常的问题，该项目提供了网络服务和裸机服务深度集成的功能，不仅能完成裸机 Port 状态变更，还能提供 Routed Networks 功能。

PS：Routed Networks & Multi-Segments

Networking-baremetal ML2 mechanism driver 是一个 Neutron ML2 Mechanism Driver，主要用于 Fake Neutron Port Attach 动作，使其状态保存健康。但实际上 Networking-baremetal ML2 mechanism driver 是可选的。因为 Ironic Driver in Nova 的 Port Binding 允许失败。类比：

VM：将 Neutron Port 绑定到计算节点的 Tap

BM：将 Neutron Port Fake Attach

安装 Networking-baremetal ML2 mechanism driver

yum install python2-networking-baremetal -y

配置使用 Networking-baremetal ML2 mechanism driver

# /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]

type_drivers = local,flat,vlan,vxlan

tenant_network_types = vxlan

extension_drivers = port_security

mechanism_drivers = openvswitch,l2population,baremetal

重启服务

systemctl restart neutron-server

配置 Ironic Neutron Agent（Controller）[可选]

Ironic Neutron Agent 和 Networking-baremetal ML2 mechanism driver 配合使用。

安装 Ironic Neutron Agent

yum install -y python2-ironic-neutron-agent

配置 Ironic Neutron Agent

# /etc/neutron/plugins/ml2/ironic_neutron_agent.ini

[DEFAULT]

debug = true

[agent]

log_agent_heartbeats = true

[ironic]

project_domain_name = default

project_name = service

user_domain_name = default

password = fanguiju

username = ironic

auth_url = http://controller:5000/v3

auth_type = password

region_name = RegionOne

启动 Ironic Neutron Agent

systemctl enable ironic-neutron-agent

systemctl start ironic-neutron-agent

systemctl status ironic-neutron-agent

创建 Provisioning Network

如前文所述，我们创建一个 Flat 类型的 Provisioning Network，该网络的本质是一个运营商网络，通过运营商的物理交换机连通到裸金属服务器，使得裸金属服务器可以借助该网络的 DHCP 服务获取到 IP 地址和 PXE 服务器的信息。所以 Subnet 一定要 Enable DHCP。

openstack network create --project admin provisioning-net-1 --share --provider-network-type flat --provider-physical-network provider

openstack subnet create provisioning-subnet-1 --network provisioning-net-1 \

  --subnet-range 172.18.22.0/24 --ip-version 4 --gateway 172.18.22.1 \

  --allocation-pool start=172.18.22.237,end=172.18.22.240 --dhcp

配置 Neutron 提供 Cleaning Network（BareMetal）

本文环境将 Provisioning Network 和 Cleaning Network 合并。

获取 Provisioning Network UUID

[root@controller ~]# openstack network list

+--------------------------------------+--------------------+--------------------------------------+

| ID                                   | Name               | Subnets                              |

+--------------------------------------+--------------------+--------------------------------------+

| 3e8d84ab-9d6e-4194-b8c0-4a14807cf8ed | ext_net            | 8792cf1d-51e8-49b7-80ae-656226c440e6 |

| b90fce07-0f32-4ba5-a1fd-a8e5e00f9c65 | provisioning-net-1 | 67327a38-4dd1-41bb-99cc-2be0bd2de00a |

| be8ca1f5-f243-4640-b7e1-4107fe16dd70 | vxlan-net-1000     | 85c68fdd-85f7-4f19-9538-ff82b5c8c5f0 |

+--------------------------------------+--------------------+--------------------------------------+

配置 ironic-conductor

# /etc/ironic/ironic.conf

[neutron]

cleaning_network = b90fce07-0f32-4ba5-a1fd-a8e5e00f9c65

重启服务

systemctl restart openstack-ironic-conductor

配置 Ironic 使用 Neutron Networking（BareMetal）

修改配置让 Ironic 默认使用 Flat Network Interface 以及填入 Provisioning/Cleaning Network

# /etc/ironic/ironic.conf

[DEFAULT]

...

enabled_network_interfaces=noop,flat,neutron

default_network_interface=flat

[neutron]

...

cleaning_network = b90fce07-0f32-4ba5-a1fd-a8e5e00f9c65

cleaning_network_security_groups = b9ce73bb-58c1-44f6-91cf-f66d5f55f57f

provisioning_network = b90fce07-0f32-4ba5-a1fd-a8e5e00f9c65

provisioning_network_security_groups = b9ce73bb-58c1-44f6-91cf-f66d5f55f57f

NOTE: The “provisioning” and “cleaning” networks may be the same network or distinct networks. To ensure that communication between the Bare Metal service and the deploy ramdisk works, it is important to ensure that security groups are disabled for these networks, or that the default security groups allow:

DHCP
TFTP
egress port used for the Bare Metal service (6385 by default)
ingress port used for ironic-python-agent (9999 by default)
if using iSCSI deploy, the ingress port used for iSCSI (3260 by default)
if using Direct deploy, the egress port used for the Object Storage service (typically 80 or 443)
if using iPXE, the egress port used for the HTTP server running on the ironic-conductor nodes (typically 80).
重启服务

systemctl restart openstack-ironic-api

systemctl restart openstack-ironic-conductor

构建 Images

安装 Disk Image Builder

$ virtualenv dib

$ source dib/bin/activate

(dib) $ pip install diskimage-builder

构建 Deploy Images

官方文档：Building or downloading a deploy ramdisk image

官方文档：Installing Ironic Python Agent!

NOTE：这里我们没有定制需求，直接下载。

wget https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe.vmlinuz

wget https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe_image-oem.cpio.gz

上传至 Glance

glance image-create --name deploy-vmlinuz --visibility public --disk-format aki --container-format aki < coreos_production_pxe.vmlinuz

glance image-create --name deploy-initrd --visibility public --disk-format ari --container-format ari < coreos_production_pxe_image-oem.cpio.gz

验证

[root@baremetal deploy_images]# openstack image list

+--------------------------------------+----------------+--------+

| ID                                   | Name           | Status |

+--------------------------------------+----------------+--------+

| d18923bd-86fc-4f77-b5e8-976d3b1c367c | cirros_raw     | active |

| 6000a17f-0ab7-418a-990c-2009a59c3392 | deploy-initrd  | active |

| e650d33b-8fad-42f7-948c-5c12526bcd07 | deploy-vmlinuz | active |

+--------------------------------------+----------------+--------+

构建 User Images

Partition images

# 支持 Cloud Init

# 设置登录账户

$ DIB_CLOUD_INIT_DATASOURCES="ConfigDrive, OpenStack"

   DIB_DEV_USER_USERNAME=root \

   DIB_DEV_USER_PWDLESS_SUDO=yes \

   DIB_DEV_USER_PASSWORD=fanguiju \

   disk-image-create \

         centos7 \

         dhcp-all-interfaces \

         baremetal \

         grub2 \

         -o my-image

$ ls

my-image.d  my-image.initrd  my-image.qcow2  my-image.vmlinuz

The partition image command creates my-image.qcow2, my-image.vmlinuz and my-image.initrd files. The grub2 element in the partition image creation command is only needed if local boot will be used to deploy my-image.qcow2, otherwise the images my-image.vmlinuz and my-image.initrd will be used for PXE booting after deploying the bare metal with my-image.qcow2.

上传至 Glance

glance image-create --name my-image.vmlinuz --visibility public --disk-format aki --container-format aki < my-image.vmlinuz

glance image-create --name my-image.initrd --visibility public --disk-format ari --container-format ari < my-image.initrd

export MY_VMLINUZ_UUID=$(openstack image list | awk '/my-image.vmlinuz/ { print $2 }')

export MY_INITRD_UUID=$(openstack image list | awk '/my-image.initrd/ { print $2 }')

glance image-create --name my-image --visibility public --disk-format qcow2 --container-format bare --property kernel_id=$MY_VMLINUZ_UUID --property ramdisk_id=$MY_INITRD_UUID < my-image.qcow2

验证

(dib) [root@baremetal user_images]# openstack image list

+--------------------------------------+------------------+--------+

| ID                                   | Name             | Status |

+--------------------------------------+------------------+--------+

| d18923bd-86fc-4f77-b5e8-976d3b1c367c | cirros_raw       | active |

| 6000a17f-0ab7-418a-990c-2009a59c3392 | deploy-initrd    | active |

| e650d33b-8fad-42f7-948c-5c12526bcd07 | deploy-vmlinuz   | active |

| 5e756d4d-b4e9-43a9-9e49-d530c72a7674 | my-image         | active |

| 24c9d142-3589-420a-b59c-f70e04575dbe | my-image.initrd  | active |

| 3bf6aaa0-58b6-4037-803a-43ee6d8937c4 | my-image.vmlinuz | active |

+--------------------------------------+------------------+--------+

配置 Bare Metal Provisioning 驱动

根据裸机集群的具体厂商和硬件设备来配置 Bare Metal Provisioning Drivers。Ironic 支持的驱动类型非常之多，具体可浏览官方文档。

官方文档：Set up the drivers for the Bare Metal service

常见组合：

pxe + ipmi：IPMI 控制硬件设备、使用 PXE 实施部署
pxe + drac：DRAC 控制硬件设备、使用 PXE 实施部署
pxe + ilo：iLO 控制硬件设备、使用 PXE 实施部署
pxe + iboot：iBoot 控制硬件设备、使用 PXE 实施部署
pxe + ssh：SSH 控制硬件设备、使用 PXE 实施部署
配置

# /etc/ironic/ironic.conf

[DEFAULT]

...

enabled_hardware_types = ipmi,redfish

# boot

enabled_boot_interfaces = pxe

# console

enabled_console_interfaces = ipmitool-socat,no-console

# deploy

enabled_deploy_interfaces = direct,iscsi

# inspect

enabled_inspect_interfaces = inspector

# management

enabled_management_interfaces = ipmitool,redfish

# power

enabled_power_interfaces = ipmitool,redfish

# raid

enabled_raid_interfaces = agent

# vendor

enabled_vendor_interfaces = ipmitool, no-vendor

# storage

enabled_storage_interfaces = cinder, noop

# network

enabled_network_interfaces = flat,neutron

重启

systemctl restart openstack-ironic-conductor

验证

[root@controller ~]# openstack baremetal driver list

+---------------------+----------------+

| Supported driver(s) | Active host(s) |

+---------------------+----------------+

| ipmi                | baremetal      |

| redfish             | baremetal      |

+---------------------+----------------+

配置 PXE 服务器

编辑

# /etc/ironic/ironic.conf

[ipmi]

retry_timeout=60

[pxe]

ipxe_enabled = False

pxe_append_params = nofb nomodeset vga=normal console=ttyS0 systemd.journald.forward_to_console=yes

tftp_root=/tftpboot

tftp_server=172.18.22.233

重启服务

systemctl restart openstack-ironic-conductor

配置 IPMI Tool

在 Ironic Conductor 节点安装 IPMI Tool。

安装

yum install ipmitool -y

验证

# ipmitool -I lanplus -H <ip-address> -U <username> -P <password> chassis power status

[root@baremetal ~]# ipmitool -I lanplus -H 172.18.22.106 -U admin -P admin chassis power status

Chassis Power is on

配置 TFTP 服务器

在 Ironic Conductor 节点配置 TFTP。

安装

sudo mkdir -p /tftpboot

sudo chown -R ironic /tftpboot

sudo yum install tftp-server syslinux-tftpboot xinetd -y

配置

# /etc/xinetd.d/tftp

service tftp

{

  protocol        = udp

  port            = 69

  socket_type     = dgram

  wait            = yes

  user            = root

  server          = /usr/sbin/in.tftpd

  server_args     = -v -v -v -v -v --map-file /tftpboot/map-file /tftpboot

  disable         = no

  flags           = IPv4

}

准备 pxelinux.0、chain.c32 文件

sudo cp /usr/share/syslinux/pxelinux.0 /tftpboot

# If whole disk images need to be deployed via PXE-netboot, copy the chain.c32 image to /tftpboot to support it

sudo cp /usr/share/syslinux/chain.c32 /tftpboot/

If the version of syslinux is greater than 4 we also need to make sure that we copy the library modules into the /tftpboot directory. Create a map file in the tftp boot directory

echo 're ^(/tftpboot/) /tftpboot/\2' > /tftpboot/map-file

echo 're ^/tftpboot/ /tftpboot/' >> /tftpboot/map-file

echo 're ^(^/) /tftpboot/\1' >> /tftpboot/map-file

echo 're ^([^/]) /tftpboot/\1' >> /tftpboot/map-file

启动

sudo systemctl enable xinetd

sudo systemctl restart xinetd

sudo systemctl status xinetd

验证

# 服务端

[root@baremetal ~]# echo 'test tftp' > /tftpboot/aaa

# 客户端

[root@controller ~]# tftp baremetal -c get aaa

[root@controller ~]# cat aaa

test tftp

配置支持 PXE UEFI

NOTE：Make sure that bare metal node is configured to boot in UEFI boot mode and boot device is set to network/pxe.

Install Grub2 and shim packages

sudo yum install grub2-efi shim -y

Copy grub and shim boot loader images to /tftpboot directory

sudo cp /boot/efi/EFI/centos/shim.efi /tftpboot/bootx64.efi

sudo cp /boot/efi/EFI/centos/grubx64.efi /tftpboot/grubx64.efi

Create master grub.cfg

$ GRUB_DIR=/tftpboot/EFI/centos

$ sudo mkdir -p $GRUB_DIR

$ cat $GRUB_DIR/grub.cfg

set default=master

set timeout=5

set hidden_timeout_quiet=false

menuentry "master"  {

configfile /tftpboot/$net_default_mac.conf

}

$ sudo chmod 644 $GRUB_DIR/grub.cfg

Update the bare metal node with boot_mode capability in node’s properties field

openstack baremetal node set <node-uuid> --property capabilities='boot_mode:uefi'

配置支持 iSCSI-base Driver

因为使用 iSCSI Deploy 方式的话，Ironic Conductor 节点会作为 iSCSI Client 并执行镜像的注入，所以需要安装 qemu-img 和 iscsiadm 指令行工具。

安装 QEMU 镜像操作工具

yum install qemu-img

安装 iSCSI 客户端

yum -y install iscsi-initiator-utils

参考文章

https://www.cnblogs.com/zhangyufei/p/8473306.html

手动集成 Ironic 裸金属管理服务（Rocky）的更多相关文章

Ironic 裸金属管理服务
目录文章目录目录 Ironic 软件架构设计资源模型设计全生命周期的状态机设计 Inspection 裸金属上架自检阶段 Provision 裸金属部署阶段 Clean 裸金属回收阶段快速体 ...
Ironic 裸金属管理服务的网络模型
目录文章目录目录 Bare-Metal networking in Neutron 核心网络类型网络拓扑抽象网络拓扑图 Neutron Implementation Neutron 了解裸金属 ...
Ironic 裸金属管理服务的底层技术支撑
目录文章目录目录底层技术支撑 DHCP NBP TFTP IPMI PXE & iPXE Cloud Init Linux 操作系统启动引导过程底层技术支撑 PXE:预启动执行环境,支 ...
注册 Ironic 裸金属节点并部署裸金属实例
目录文章目录目录前文列表注册(Enrollment)裸机创建裸金属实例的 Flavor 部署裸金属实例日志分析问题:Failed to create neutron ports for ...
Ironic 裸金属实例的部署流程
目录文章目录目录逻辑架构部署架构前提条件部署流程 iSCSI Deploy UML PXE Deploy Driver Direct Deploy UML IPA Deploy Drive ...
使用disk-image-builder（DIB）制作Ironic 裸金属镜像
export DIB_DEV_USER_USERNAME=centos export DIB_DEV_USER_PASSWORD= export DIB_DEV_USER_PWDLESS_SUDO=Y ...
OpenStack-Ironic裸金属简介
一,Ironic简述简而言之,OpenStack Ironic就是一个进行裸机部署安装的项目. 所谓裸机,就是指没有配置操作系统的计算机.从裸机到应用还需要进行以下操作: (1)硬盘RAID ...
OpenStack Newton：集虚拟化，裸金属和容器部署的统一云平台(转载)
2016-10-08木屐大数据在线国庆长假第六天,OpenStack第十四版本Newton(牛顿?)发布,官方介绍中强调这是一个集虚拟化.裸金属和容器技术的一体化平台,可通过一套API来管理裸金属. ...
ironic组件硬件自检服务——ironic-inspector
介绍 ironic-inspector是一个用于硬件自检的辅助型服务,它可以对被ironic组件管理的裸金属节点进行硬件自检,通过在裸金属节点上运行内存系统,发现裸金属节点的硬件信息,例如CPU数量和 ...

随机推荐

《Linux就该这么学》day1-day2
ps:原谅我的书法出自鲁迅的<野草> <Linux就该这么学>书本介绍: 本书是由全国多名红帽架构师(RHCA)基于最新Linux系统共同编写的高质量Linux技术自学教程,极 ...
日常系统维护之修复linux的grub引导
恩,不知道怎么表达为win10的更新很烦人,其实我感觉windows用的不是那么顺手,但是制作视频的时候还是要用到,图片的话linux也可以胜任,毕竟GUN IMAGE还是挺不错的,但到了视频制作方面 ...
PTA ”写出这个数“
读入一个正整数 n,计算其各位数字之和,用汉语拼音写出和的每一位数字. 输入格式: 每个测试输入包含 1 个测试用例,即给出自然数 n 的值.这里保证 n 小于 10的100次方. 输出格式: 在 ...
Webpack编译提示内存溢出解决方案
在项目开发中,随着业务需求的复杂项目随之增大,再加上同一个文件被引用次数过于频繁在开发编译或者上线打包时经常会出现如下错误: 这个报错的意思就是Node内存不足所导致的,我们都知道 Node 是基于V ...
【NOIP2017】小凯的疑惑
原题: 小凯手中有两种面值的金币,两种面值均为正整数且彼此互素.每种金币小凯都有无数个.在不找零的情况下,仅凭这两种金币,有些物品他是无法准确支付的.现在小凯想知道在无法准确支付的物品中,最贵的价 ...
Hdu 1851 Nim+巴什博弈
写出SG表发现SG函数规律为Mi%(Li+1) #include<bits/stdc++.h> using namespace std; int main() { int t; cin ...
Python中关于txt的简单读写模式与操作
Python中关于txt的简单读写操作常用的集中读写模式: 1.r 打开只读文件,该文件必须存在. 2.r+ 打开可读写的文件,该文件必须存在. 3.w 打开只写文件,若文件存在则文件长度清为0,即 ...
java学习笔记（二）分布式框架Dubbo+zookeeper搭建
参考文章http://www.cnblogs.com/sxjun/p/6963844.html 注意的几个问题: 1.配置dubbo.properties 将以下地址改为你Zookeeper注册的地址 ...
论文参考文献中的M R J意义
1 期刊作者.题名［J］.刊名,出版年,卷(期):起止页码 2 专著作者.书名［M］.版本(第一版不著录).出版地:出版者,出版年.起止页码 3 论文集作者.题名［C］.//编者.论文集名.出版地:出 ...
Linux下mount存储盘遇到的错误
一.注意点 1.超过1T的盘,创建的分区要查看是否初始化为了GPT格式. 2.如果新添加的盘是从存储上挂载的,涉及到多路径的问题,挂载的是多路径的盘符,比如:/dev/mapper/mpatha(对应 ...

手动集成 Ironic 裸金属管理服务（Rocky）

目录

文章目录