服务器配置要求

  1. x86服务器内存不能低于32GB
  2. 服务器CPU需要支持虚拟化和设备虚拟化 VT-x VT-d,SR-IOV 功能，并且在BIOS中能启用了SR-IOV
  3. 网卡配置最起码为千兆配置
  4. 支持 SR-IOV 的网卡必须插在总线带宽 X8 以上的扩展槽中

基本定义

SR-IOV 技术标准允许在虚拟机之间高效共享 PCIe（Peripheral Component Interconnect Express，快速外设组件互连）设备，并且它是在硬件中实现的，可以获得能够与本机性能媲美的 I/O 性能。SR-IOV 规范定义了新的标准，根据该标准创建的新设备可允许将虚拟机直接连接到 I/O 设备。

SRIOV的功能类型

物理功能 (Physical Function, PF)

用于支持 SR-IOV 功能的 PCI 功能，如 SR-IOV 规范中定义。PF 包含 SR-IOV 功能结构，用于管理 SR-IOV 功能。PF 是全功能的 PCIe 功能，可以像其他任何 PCIe 设备一样进行发现、管理和处理。PF 拥有完全配置资源，可以用于配置或控制 PCIe 设备。

虚拟功能 (Virtual Function, VF)

与物理功能关联的一种功能。VF是一种轻量级 PCIe功能，可以与物理功能以及与同一物理功能关联的其他 VF共享一个或多个物理资源。VF仅允许拥有用于其自身行为的配置资源。

功能简介

每个 SR-IOV 设备都可有一个物理功能 (Physical Function, PF)，并且每个 PF 最多可有 64,000 个与其关联的虚拟功能 (Virtual Function, VF)。PF 可以通过寄存器创建 VF，这些寄存器设计有专用于此目的的属性。
如果在 PF 中启用了 SR-IOV，就可以通过 PF 的总线、设备和功能编号（路由 ID）访问各个 VF 的 PCI 配置空间。每个 VF 都具有一个PCI内存空间，用于映射其寄存器集。VF设备驱动程序对寄存器集进行操作以启用其功能，并且显示为实际存在的PCI设备。创建VF后，可以直接将其指定给IO来宾域或各个应用程序。此功能使得虚拟功能可以共享物理设备，并在没有 CPU 和虚拟机管理程序软件开销的情况下执行 I/O。

GPU SRIOV

System Requirements

Operating System Requirements

Ubuntu 16.04. has been fully validated as host, other Linux operating system like Centos 7 are also OK.

Hardware Requirements

The device supports VT-d, and the physical graphics card supports SRIOV technology.

Modify the BIOS

we need reset bios to defaults,and enable VT-d and SRIOV function

Kernel preparation

In order to use the graphics cards and all its cores, we’ll have to recompile the Linux Kernel, modifying some values and applying some paches.

Download kernel source

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
vi /etc/apt/sources.list

# deb cdrom:[Ubuntu-Server 16.04.6 LTS _Xenial Xerus_ - Release amd64 (20190226)]/ xenial main restricted

#deb cdrom:[Ubuntu-Server 16.04.6 LTS _Xenial Xerus_ - Release amd64 (20190226)]/ xenial main restricted

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://us.archive.ubuntu.com/ubuntu/ xenial main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://us.archive.ubuntu.com/ubuntu/ xenial universe
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial universe
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates universe
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu 
## team, and may not be under a free licence. Please satisfy yourself as to 
## your rights to use the sof 大专栏  基于SR-IOV的IO虚拟化技术tware. Also, please note that software in 
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://us.archive.ubuntu.com/ubuntu/ xenial multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial multiverse
deb http://us.archive.ubuntu.com/ubuntu/ xenial-updates multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://us.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ xenial-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu xenial partner
deb-src http://archive.canonical.com/ubuntu xenial partner

deb http://security.ubuntu.com/ubuntu xenial-security main restricted
deb-src http://security.ubuntu.com/ubuntu xenial-security main restricted
deb http://security.ubuntu.com/ubuntu xenial-security universe
deb-src http://security.ubuntu.com/ubuntu xenial-security universe
deb http://security.ubuntu.com/ubuntu xenial-security multiverse
deb-src http://security.ubuntu.com/ubuntu xenial-security multiverse

apt update && apt upgrade -y

apt install -y dpkg-dev
apt source linux-image-$(uname -r)

Download gim modules

git clone https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization

Patch the kernel

The patch is located in the SRC_DIR/MxGPU-Virtualization/patch directory.

1
2
3
4
5
6
 
cd linux-4.4.0 && make oldconfig && make menuconfig

patch < ../MxGPU-Virtualization/patch/0001-Added-legacy-endpoint-type-to-sriov-for-ubuntu-4.4.0-75-generic.diff
File to patch: ./drivers/pci/iov.c
patch < ../MxGPU-Virtualization/patch/0002-add-pci-io-access-cap-for-ubuntu-4.4.0-75-generic.diff
File to patch: ./drivers/vfio/pci/vfio_pci_config.c

Kernel compilation

make -j deb-pkg LOCALVERSION=-vm-firepro

Kernel installation

1
2
 
cd ..
dpkg -i *.deb

GRUB and BLACKLIST preparation

GRUB

1
2
3
4
5
6
7
8
 
vim /etc/default/grub
---
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on"
---
Check if IOMMU is supported after rebooting the system.
Command is as follows:
dmesg | grep -e DMAR -e IOMMU
You should be able to see something like the following:DMAR: IOMMU enabled

Blacklist amdgpu and amdkfd

adding the following line to the end of file

1
2
3
4
5
 
vi /etc/modprobe.d/blacklist.conf
---
blacklist amdgpu
blacklist amdkfd
---

gim module compilation

Compile gim module

cd MxGPU-Virtualization && make && make install

Use gim module

1
2
3
4
5
6
7
8
9
10
11
 
modprobe gim
modprobe -r gim
vi /etc/gim_config
----
...
vf_num=16 # This is going to activate the 16 cores of the GPU
...
----
modprobe gim
dmesg # Finally, we need to check that the gim module has successfully been loaded, without errors
lspci -vvv # We can now see that the FirePro S7150 has all the 16 cores activated

基于SR-IOV的IO虚拟化技术的更多相关文章

  1. 基于python自动化测试平台与虚拟化技术结合的思考

    背景: 自动化测试行业内,个人觉得主力语言是python.java.这里讨论下基于python自动化框架设计与case开发,用过python的都知道它的好处,但是根据实际项目需要有了很多迎面而来的困难 ...

  2. KVM基于X86硬件辅助的虚拟化技术实现机制【转】

    内存虚拟化 Shadow Paging 作者 Shawn 在其中文博客中很详尽地介绍了 KVM 在只支持一级分页的 x86 平台上用 “Shadow Paging”进行 MMU 虚拟化的实现,由于目前 ...

  3. 虚拟化--IO虚拟化基本原理

    本文话题: IO虚拟化概述 设备发现 访问截获 设备模拟 设备共享基于软件的IO虚拟化 基于前端后端的IO虚拟化基于硬件的IO虚拟化 概述 从处理器的角度看,外设是通过一组I/O资源(端口I/O或者是 ...

  4. 虚拟化技术原理（CPU、内存、IO）

    本文来自:http://www.ywnds.com/?p=5856 虚拟化 云计算现在已经非常成熟了,而虚拟化是构建云计算基础架构不可或缺的关键技术之一. 云计算的云端系统, 其实质上就是一个大型的分 ...

  5. 基于KVM、Xen、OpenVZ等虚拟化技术的WEB在线管理工具

    1.Proxmox proxmox是一个开源的虚拟化管理平台,支持集群管理和HA.在存储方面,proxmox除了支持常用的lvm,nfs,iscsi,还支持集群存储glusterfs和ceph,这也是 ...

  6. 基于SDN的IP RAN网络虚拟化技术

    http://www.zte.com.cn/cndata/magazine/zte_technologies/2014/2014_4/magazine/201404/t20140421_422858. ...

  7. 四种主要网络IO虚拟化模型

    本文主要为大家简要介绍VMware.Redhat.Citrix.Microsoft主要虚拟化厂商使用的4种主要的虚拟化IO模型 (emulation.para-virtualization.pass- ...

  8. 虚拟化技术比较 PV HVM

    很多人看到同样配置的VPS价格相差很大,甚是不理解,其实VPS使用的虚拟技术种类有很多,如OpenVZ.Xen.KVM.Xen和HVM与PV.在XEN中pv是半虚拟化,hvm是全虚拟化,pv只能用于L ...

  9. 关于Linux虚拟化技术KVM的科普 科普二(KVM虚拟机代码揭秘)

    代码分析文章<KVM虚拟机代码揭秘--QEMU代码结构分析>.<KVM虚拟机代码揭秘--中断虚拟化>.<KVM虚拟机代码揭秘--设备IO虚拟化>.<KVM虚拟 ...

随机推荐

  1. python-day8爬虫基础之数据存储

    数据存储,在爬虫中也是十分的重要,因为我们要把我们想要的数据保存到本地,其中最简单直接的就是保存为文件文本,比如:TXT.JSON.CSV等等,除此之外,我们还可以将其保存到数据库中,常见的数据库类型 ...

  2. QeePHP

    百度百科: https://baike.baidu.com/item/qeephp/8328612?fr=aladdin 官方地址: http://www.qeephp.cn/app/index.ph ...

  3. 剑指offer【10】- 变态跳台阶

    题目:一只青蛙一次可以跳上1级台阶,也可以跳上2级……它也可以跳上n级.求该青蛙跳上一个n级的台阶总共有多少种跳法. 关于本题,前提是n个台阶会有一次n阶的跳法.分析如下: f(1) = 1 f(2) ...

  4. Python笔记_第四篇_高阶编程_GUI编程之Tkinter_6.附录

    1. 事件附录: 2. 事件属性附录:

  5. tensorflow用法记录

    使用 embedding 变量 import tensorflow as tf import numpy as np sess = tf.InteractiveSession() M = list(' ...

  6. ruoyi HttpUtils

    package com.ruoyi.common.utils.http; import java.io.BufferedReader; import java.io.IOException; impo ...

  7. android studio 3.2 bundle.gradle 与2.2区别

    参考:https://blog.csdn.net/MakerCloud/article/details/82898305

  8. TPO1-1groundwater

    Thus a proportion of the total volume of any sediment, loose or cemented, consists of empty space. M ...

  9. E - Apple Tree(树状数组+DFS序）

    There is an apple tree outside of kaka's house. Every autumn, a lot of apples will grow in the tree. ...

  10. win10//ubuntu安装tensorflow-gpu与kears，并用minist测试

    WIn10 安装cuda 先安装VS,然后根据自己的版本安装CUDA. 安装完后,打开cmd命令行输入nvcc -V,检测是否安装成功 安装cuDDN 安装对应版本,解压后覆盖到CUDA的地址,默认为 ...