Which Queue Pair type to use?
Unconnected: one QP send/receive with any QP
The following mechanisms are being used in RDMA:
* CRC: The CRC field which validates that packets weren't corrupted along the path.
* PSN: The Pa
2024-10-16 13:25:48 原文Which Queue Pair type to use?
Contents [hide]
When writing a new RDMA application (just like when writing a new application over sockets), one should decide which QP type he should work with.
In this post, I will describe in detail the characteristics of each transport type.
In RDMA, there are several QP types. They can be represented by : XY
X can be:
Reliable: There is a guarantee that messages are delivered at most once, in order and without corruption.
Unreliable: There isn't any guarantee that the messages will be delivered or about the order of the packets.
In RDMA, every packet has a CRC and corrupted packets are being dropped (for any transport type). The Reliability of a QP transport type refers to the whole message reliability.
Y can be:
Connected: one QP send/receive with exactly one QP
Unconnected: one QP send/receive with any QP
The following mechanisms are being used in RDMA:
* CRC: The CRC field which validates that packets weren't corrupted along the path.
* PSN: The Packet Serial Number makes sure that packets are being received by the order. This helps detect missing packets and packet duplications.
* Acknowledgement: (only in RC QP) Only after a message is being written successfully on the responder side, an ack packet is being sent back to the requestor. If an ack isn't being sent by the requestor, it resend the message again according to the QP's attributes. If there won't be any ack (or nack) from a QP, it will report that there is an error (retry exceeded).
If there is any kind of error on the responder side (protection, resources, etc.) an ack will be sent to the requestor and it will report that there is an error.
Reliable Connected (RC) QP
One RC QP is being connected (i.e. send and receive messages) to exactly one RC QP in a reliable way. It is guaranteed that messages are delivered from a requester to a responder at most once, in order and without corruption. The maximum supported message size is up to 2GB (this value may be lower, depends on the supported RDMA device attributes). RC QP supports Send operations (w/o immediate), RDMA Write operations (w/o immediate), RDMA Read operations and Atomic operations (it depends on the RDMA device support level in atomic operations).
If a message size is bigger than the path MTU, it is being fragmented in the side that sends the data and being reassembled in the receiver side.
Requester considers a message operation complete once there is an ack from the responder side that the message was read/written to its memory.
Responder considers a message operation complete once the message was read/written to its (local) memory.
Unreliable Connected (UC) QP
One UC QP is being connected (i.e. send and receive messages) to exactly one UC QP in an unreliable way. There isn't any guaranteed that the messages will be received by the other side: corrupted or out of sequence packets are silently dropped. If a packet is being dropped, the whole message that it belongs to will be dropped. In this case, the responder won't stop, but continues to receive incoming packets. There isn't any guarantee about the packet ordering. The maximum supported message size is up to 2GB (this value may be lower, depends on the support RDMA device attributes). RC QP supports Send operations (w/o immediate) and RDMA Write operations (w/o immediate).
If a message size is bigger than the path MTU, it is being fragmented in the side that sends the data and being reassembled in the receiver side.
Requester considers a message operation complete once all of the message was sent to the fabric.
Responder considers a message operation complete once it received a complete message in correct sequence and it written the data to its (local) memory.
Unreliable Datagram (UD) QP
One QP can send and receive message to any other UD QP in either unicast (one to one) or multicast (one to many) way in an unreliable way. There isn't any guaranteed that the messages will be received by the other side: corrupted or out of sequence packets are silently dropped. There isn't any guarantee about the packet ordering. The maximum supported message size is the maximum path MTU. UD QP supports only Send operations.
Requester considers a message operation complete once the (one packet) message was sent to the fabric.
Responder considers a message operation complete once it received a complete message and it written the data to its (local) memory.
Choosing the right QP type
Choosing the right QP type is critical to the correction and scalability of an application.
RC QP should be chosen if:
- Reliability by the fabric is needed
- Fabric size isn't big or the cluster size is big, but not all nodes send traffic to the same node (one victim)
Several uses for a RC QP can be: FTP over RDMA or file system over RDMA.
UC QP should be chosen if:
- Reliability by the fabric isn't needed (i.e. reliability isn't important at all or it is being taken care of by the application)
- Fabric size isn't big or the cluster size is big, but not all nodes send traffic to the same node (one victim)
- Big messages (more than the path MTU) are being sent
One use for an UC QP can be: video over RDMA.
UD QP should be chosen if:
- Reliability by the fabric isn't needed (i.e. reliability isn't important at all or it is being taken care of by the application)
- Fabric size is big and all nodes and every node send messages to any other node in the fabric. UD is one of the best solutions for scalability problems.
- Multicast messages are needed
One use for an UD QP can be: voice over RDMA.
Summary
The following table describes the characteristics of each QP Transport Service Type:
Metric | UD | UC | RC |
---|---|---|---|
Opcode: SEND (w/o immediate) | Supported | Supported | Supported |
Opcode: RDMA Write (w/o immediate) | Not supported | Supported | Supported |
Opcode: RDMA Read | Not supported | Not supported | Supported |
Opcode: Atomic operations | Not supported | Not supported | Supported |
Reliability | No | No | Yes |
Connection type | Datagram (One to any/many) | Connected (one to one) | Connected (one to one) |
Maximum message size | Maximum path MTU | 2 GB | 2 GB |
Multicast | supported | Not supported | Not supported |
Share:
Written by: Dotan Barak on June 1, 2013.on January 11, 2019.
Related
Which Queue Pair type to use?的更多相关文章
- Queue Pair in RDMA (zz)
Queue Pair in RDMA 首页分类标签留言关于订阅2018-03-21 | 分类 Network | 标签 RDMA 一个CA(Channel Adapter)可以包含多个QP,QP相当 ...
- C++的队列和pair
C++队列的成员函数: back()返回最后一个元素 empty()如果队列空则返回真 front()返回第一个元素 pop()删除第一个元素 push()在末尾加入一个元素 size()返回队列中元 ...
- AMQP 0-9-1 Model Explained Why does the queue memory grow and shrink when publishing/consuming? AMQP和AMQP Protocol的是整体和部分的关系 RabbitMQ speaks multiple protocols.
AMQP 0-9-1 Model Explained — RabbitMQ http://next.rabbitmq.com/tutorials/amqp-concepts.html AMQP 0-9 ...
- jQuery.queue源码分析
作者:禅楼望月(http://www.cnblogs.com/yaoyinglong ) 队列是一种特殊的线性表,它的特殊之处在于他只允许在头部进行删除,在尾部进行插入.常用来表示先进先出的操作(FI ...
- jQuery源代码学习之七—队列模块queue
一.jQuery种的队列模块 jQuery的队列模块主要是为动画模块EFFECTS提供支持,(不过到现在为了支持动画队列的inprogress的出入队还是搞不太清楚),单独抽取出一个命名空间是为了使程 ...
- jquery源码学习之queue方法
队列模块的代码结构 静态方法jQuery下有queue,dequeue,_queueHooks这三种方法:静态方法不建议直接在外部调用: 实例方法.queue,.dequeue,.clearQueue ...
- QUEUE——队列(procedure)
#include <stdio.h> #include <stdlib.h> #include "queue.h" int main() { int i; ...
- Objective-C priority queue
http://stackoverflow.com/questions/17684170/objective-c-priority-queue PriorityQueue.h // // Priorit ...
- 找最大重复次数的数和重复次数(C++ Pair)
Problem A: 第一集 你好,世界冠军 Time Limit: 10 Sec Memory Limit: 128 MBSubmit: 265 Solved: 50[Submit][Statu ...
随机推荐
- 微信小程序-滚动Tab选项卡
前言:今天呢 给大家详细讲解一下滚动Tab选项卡:左右可滑动切换的效果,希望对大家做项目时候有用! 以前也遇到过这个,但是没有做记录.转载来源于:https://www.jianshu.com/p/9 ...
- Linux基本指令
常用目录文件作用 - / 根目录 - /bin 命令保存目录(普通用户就可以读取的命令) - /boot 启动目录,启动相关文件 - /dev 设备文件保存目录 - /etc ...
- js中各种弹窗
** * JS提示跳转 * @param $tip 弹窗口提示信息(为空没有提示) * @param $type 设置类型 close = 关闭 ,back=返回 ,refresh=提示重载,j ...
- hdfs webhdfs 写文件(create file)
# _*_ coding=utf-8 _*_ import sys import os hosts = {} cmd1 = ''' curl -i -X PUT "http://%s:500 ...
- json jmespath的使用
在接口测试的过程中,我们经常会接触到json格式respones,我们会经常去做取JSON中某个值的操作,可以用jmespath分析.转换和选择性的提取JSON数据进行操作,功能非常强大,易读. 一. ...
- HBASE分布式集群搭建(ubuntu 16.04)
1.hbase是依赖Hadoop运行的,因此先确保自己已搭建好Hadoop集群环境 没安装的可以参考这里:https://www.cnblogs.com/chaofan-/p/9740408.html ...
- vue中less文件全局引用
1.先安装sass-resources-loader npm install sass-resources-loader 2.然后在build->utils.js修改less配置 在less ...
- python测试开发django-1.开始hello world!
前言 当你想走上测试开发之路,用python开发出一个web页面的时候,需要找一个支持python语言的web框架.django框架有丰富的文档和学习资料,也是非常成熟的web开发框架,想学pytho ...
- 2018-2019-2 《网络对抗技术》Exp0 Kali安装
2018-2019-2 <网络对抗技术>Exp0 Kali安装 ---20165110 一.实验要求 1.下载 2.安装Kali 3.连接网络 4.共享文件 5.更新软件源 二.实验步骤 ...
- 算法练习LeetCode初级算法之其他
位1的个数 解法一: class Solution { // you need to treat n as an unsigned value public int hammingWeight(int ...