[转帖]高性能异步io机制：io_uring
文章目录
io_uring 是 linux 内核 5.10 引入的异步 io 接口。相比起用户态的DPDK、SPDK，io_uring 作为内核的一部分，通过 mmap 的方式实现用户和内核共享内存，并基于 memory barrier 在这块内存上实现了两个无锁环形队列：
submission queue ring(sq) 和
completion queue ring(cq)。 sq 用于用户程序向内核提交 IO 任务，内核执行完成的任务会放入cq，用户程序从 cq 获取结果。在提交任务和返回任务结果时，用户程序和内核共用环形队列中的数据，不再需要额外的数据拷贝。此外，io_uring 还提供了两种轮询 Polling 模式，可以避免提交任务时的系统调用，以及io完成后的中断通知。
1、性能测试
1.1、FIO
iops 是指单位时间内系统能处理的I/O请求数量，用于存储设备性能测试。这里我们使用硬盘性能辅助测试工具 FIO，来直观感受异步 io: io_uring 的性能优势。
# 安装 fio
sudo apt install fio
# 运行方式
fio job_file
需要通过编写一个配置文件来预定义 FIO 将要以什么样的模式来执行任务。
FIO 的基本参数：
rw readwrite：定义 IO 类型。随机读 randread、随机写 randwrite、顺序读 read、顺序写 write、顺序读写 rw readwrite ，随机混合读写 randrwbs, blocksize：IO 的块大小。默认4ksize: IO 传输的数据大小ioengine：IO 引擎。同步模式psync、异步模式io_uringiodepth：I/O 引擎若使用异步模式，保持队列深度direct: 是否使用非缓冲 io ，默认 false 缓冲 io
编写的 posix.fio 配置文件如下
[global]
thread=1
group_reporting=1
direct=1
verify=0
time_based=1
runtime=10
bs=16K
size=16384
iodepth=64
rw=randwrite
filename=Cat
ioengine=io_uring
[test]
stonewall
description="variable bs"
实验结果：iops：psync 8k, io_uring 19.0k，由此可以看出异步 io 的性能优势。
1.2、rust_echo_benc
服务器性能测试方法
- 连接数
- 每个请求连接的大小
- 持续时间
epoll 与 io_uring 事件的区别
- epoll 设置完后，不更改。
- io_uring 设置一次，触发一次。
接下来，进行同步 epoll 与异步 io_uring 服务器的测试对比，代码见 liburing 测试代码
# 安装 rust_echo_benc
git clone https://github.com/haraldh/rust_echo_bench.git
cargo run --release
测试
cargo run --release -- --address "127.0.0.1:9999" --number 1000 --duration 60 --length 512
实验结果：在网络 io 方面，io_uring并不明显。在磁盘 io 方面，io_uring 具有一定的优势。
2、io_uring
io_uring 提供了三个系统调用接口 io_uring_setup、io_uring_enter、io_uring_register
2.1、io_uring_setup
在 kernel 中创建：
- 提交队列 SQ：里面每一项是 sqe(submission queue event)，描述1个任务
- 完成队列 CQ：里面每一项是 cqe(completion queue event)，描述1个任务返回结果
- 提交队列项 SQEs 数组（Submission Queue Entries）
SQ 和 CQ 采用 Ringbuffer 的结构，有 head 和 tail 两个成员，head = tail 时队列为空。每个节点保存的是 SQEs 数组的偏移量，实际的请求保存在 SQEs 数组中，这样就可以批量提交一组 SQEs 上不连续的请求。SQ 和 CQ 本身没有提供锁等同步机制，向 SQ中放入 sqe，从 CQ 中取出 cqe，都需要通过 memory barrier 来实现。
函数返回1个 fd 用于 io_uring 管理。用户将 fd 以 mmap 的方式映射到内存，实现了用户态和内核态的共享内存。
/*
- 参数1 entries：期望的 sq 长度。默认cq长度是sq的两倍
- 参数2 params: 配置io_uring，内核返回的 sq/cq 配置信息也通过它带回来
*/
int io_uring_setup(unsigned entries, struct io_uring_params *params)
struct io_uring_params {
__u32 sq_entries;
__u32 cq_entries;
__u32 flags;
__u32 sq_thread_cpu;
__u32 sq_thread_idle;
__u32 resv[5];
struct io_sqring_offsets sq_off;
struct io_cqring_offsets cq_off;
};
2.2、io_uring_enter
调用时，执行两个操作
- 提交 IO 请求：把 sqe 的索引尾插到 SQ 中，调用
io_uring_enter提交到内核 - 等待 IO 完成：内核将完成的 IO 放到 CQ 中，用户轮询 CQ 来等待结果
/*
- 参数1 fd：io_uring_setup返回的fd
- 参数2 to_submit: 一次提交多少个 sqe 到内核
- 参数3 min_complete: 要求内核至少等待min_complete个任务完成再返回
- 参数4 flags：接口控制行为，IORING_ENTER_GETEVENTS
*/
int io_uring_enter(unsigned int fd, u32 to_submit, u32 min_complete, u32 flags);
2.3、io_uring_register
注册用于异步 I/O 的文件或用户缓冲区
对于文件， 保持内核长时间持有该文件的索引。每次通过 sqe 向内核传递一个 fd，内核都需要通过 fd 找到对应的文件索引，完成该sqe 处理后，则将该索引释放。对于高 iops 的场景，这个开销会拖慢请求的速度。通过预先注册一组已经打开的文件。
对于缓冲区，保持内存的长期映射。内核在读写前进行page map，读写完成后，执行unmap。类似的，通过预注册，来避免多次的 map 和 unmap。
/*
- 参数1 fd：io_uring_setup返回的fd
- 参数2 opcode: 注册类型。
文件类型: IORING_REGISTER_FILES；
用户缓冲类型 buffer: IORING_REGISTER_BUFFERS
- 参数3 arg:
文件类型: 指向一个fd数组；
用户缓冲类型：指向一个struct iovec的数组。
- 参数4 nr_args：arg数组的长度
*/
int io_uring_register(unsigned int fd, unsigned int opcode,
void *arg, unsigned int nr_args);
2.4、使用方法：cat 程序为例
接下来，基于 io_uring 的系统调用接口进行封装，实现自定义的 uring_cat 程序
// gcc -o uring_cat uring_cat.c
// ./uring_cat filename
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <sys/syscall.h>
#include <sys/mman.h>
#include <sys/uio.h>
#include <linux/fs.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <linux/io_uring.h>
#define URING_QUEUE_DEPTH 1024
#define BLOCK_SZ 1024
// sqring
struct app_io_sq_ring {
<span class="token keyword">unsigned</span> <span class="token operator">*</span>head<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>tail<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>ring_mask<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>ring_entries<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>flags<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>array<span class="token punctuation">;</span>
};
// cqring
struct app_io_cq_ring {
<span class="token keyword">unsigned</span> <span class="token operator">*</span>head<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>tail<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>ring_mask<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> <span class="token operator">*</span>ring_entries<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_cqe</span> <span class="token operator">*</span>cqes<span class="token punctuation">;</span>
};
// 提交器: cq, sq, sqe
struct submitter {
<span class="token keyword">int</span> ring_fd<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">app_io_sq_ring</span> sq_ring<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">app_io_cq_ring</span> cq_ring<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_sqe</span> <span class="token operator">*</span>sqes<span class="token punctuation">;</span>
};
-------------------
struct file_info {
off_t file_sz;
struct iovec iovecs[];
};
-------------------
// 利用系统调用执行 io_uring_setup 流程
// 1、int 0x80 中断信号
// 2、mv arg1, eax
// 3、mv arg2, ebx
// 4、call sys_call_table: sys_call_table[__NR_io_uring_setup]
int io_uring_setup(unsigned entries, struct io_uring_params *p)
{
return (int) syscall(__NR_io_uring_setup, entries, p);
}
int io_uring_enter(int ring_fd, unsigned int to_submit,
unsigned int min_complete, unsigned int flags)
{
return (int) syscall(__NR_io_uring_enter, ring_fd, to_submit, min_complete,
flags, NULL, 0);
}
int app_setup_uring(struct submitter *s) {
<span class="token keyword">struct</span> <span class="token class-name">io_uring_params</span> p<span class="token punctuation">;</span>
<span class="token function">memset</span><span class="token punctuation">(</span><span class="token operator">&</span>p<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span>p<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// 创建sq, cq, sqes</span>
s<span class="token operator">-></span>ring_fd <span class="token operator">=</span> <span class="token function">io_uring_setup</span><span class="token punctuation">(</span>URING_QUEUE_DEPTH<span class="token punctuation">,</span> <span class="token operator">&</span>p<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>s<span class="token operator">-></span>ring_fd <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token comment">// 获取初始的sq,cq的大小，sq_off, cq_off起始偏移地址</span>
<span class="token keyword">int</span> sring_sz <span class="token operator">=</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>array <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_entries <span class="token operator">*</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">unsigned</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">int</span> cring_sz <span class="token operator">=</span> p<span class="token punctuation">.</span>cq_off<span class="token punctuation">.</span>cqes <span class="token operator">+</span> p<span class="token punctuation">.</span>cq_entries <span class="token operator">*</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">io_uring_cqe</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// io_uring特性：IORING_FEAT_SINGLE_MMAP：内核通过一次mmap完成sq, cq的映射</span>
<span class="token comment">// 即sq，cq共用1块内存，则两者大小必须设置相同</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>p<span class="token punctuation">.</span>features <span class="token operator">&</span> IORING_FEAT_SINGLE_MMAP<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>cring_sz <span class="token operator">></span> sring_sz<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
sring_sz <span class="token operator">=</span> cring_sz<span class="token punctuation">;</span>
<span class="token punctuation">}</span>
cring_sz <span class="token operator">=</span> sring_sz<span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token comment">// 1、将 sq 的映射到用户空间，sq_ptr 指向sq首地址</span>
<span class="token keyword">void</span> <span class="token operator">*</span>sq_ptr <span class="token operator">=</span> <span class="token function">mmap</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> sring_sz<span class="token punctuation">,</span> PROT_READ<span class="token operator">|</span>PROT_WRITE<span class="token punctuation">,</span> MAP_SHARED<span class="token operator">|</span>MAP_POPULATE<span class="token punctuation">,</span>
s<span class="token operator">-></span>ring_fd<span class="token punctuation">,</span> IORING_OFF_SQ_RING<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>sq_ptr <span class="token operator">==</span> MAP_FAILED<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token comment">// 2、将 cq 的映射到用户空间，cq_ptr 指向cq首地址</span>
<span class="token keyword">void</span> <span class="token operator">*</span>cq_ptr<span class="token punctuation">;</span>
<span class="token comment">// 若共用一块内存，则两个指针指向相同</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>p<span class="token punctuation">.</span>features <span class="token operator">&</span> IORING_FEAT_SINGLE_MMAP<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
cq_ptr <span class="token operator">=</span> sq_ptr<span class="token punctuation">;</span>
<span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{<!-- --></span>
<span class="token comment">// 若使用两块内存，则重新对cq进行mmap，</span>
cq_ptr <span class="token operator">=</span> <span class="token function">mmap</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> sring_sz<span class="token punctuation">,</span> PROT_READ<span class="token operator">|</span>PROT_WRITE<span class="token punctuation">,</span> MAP_SHARED<span class="token operator">|</span>MAP_POPULATE<span class="token punctuation">,</span>
s<span class="token operator">-></span>ring_fd<span class="token punctuation">,</span> IORING_OFF_CQ_RING<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>cq_ptr <span class="token operator">==</span> MAP_FAILED<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">struct</span> <span class="token class-name">app_io_sq_ring</span> <span class="token operator">*</span>sring <span class="token operator">=</span> <span class="token operator">&</span>s<span class="token operator">-></span>sq_ring<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">app_io_cq_ring</span> <span class="token operator">*</span>cring <span class="token operator">=</span> <span class="token operator">&</span>s<span class="token operator">-></span>cq_ring<span class="token punctuation">;</span>
sring<span class="token operator">-></span>head <span class="token operator">=</span> sq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>head<span class="token punctuation">;</span>
sring<span class="token operator">-></span>tail <span class="token operator">=</span> sq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>tail<span class="token punctuation">;</span>
sring<span class="token operator">-></span>ring_mask <span class="token operator">=</span> sq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>ring_mask<span class="token punctuation">;</span>
sring<span class="token operator">-></span>ring_entries <span class="token operator">=</span> sq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>ring_entries<span class="token punctuation">;</span>
sring<span class="token operator">-></span>flags <span class="token operator">=</span> sq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>flags<span class="token punctuation">;</span>
sring<span class="token operator">-></span>array <span class="token operator">=</span> sq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>sq_off<span class="token punctuation">.</span>array<span class="token punctuation">;</span>
<span class="token comment">// 3、将 seqs 映射到用户空间</span>
s<span class="token operator">-></span>sqes <span class="token operator">=</span> <span class="token function">mmap</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">,</span> p<span class="token punctuation">.</span>sq_entries <span class="token operator">*</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">io_uring_sqe</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
PROT_READ <span class="token operator">|</span> PROT_WRITE<span class="token punctuation">,</span> MAP_SHARED <span class="token operator">|</span> MAP_POPULATE<span class="token punctuation">,</span> s<span class="token operator">-></span>ring_fd<span class="token punctuation">,</span> IORING_OFF_SQES<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>s<span class="token operator">-></span>sqes <span class="token operator">==</span> MAP_FAILED<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">return</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
cring<span class="token operator">-></span>head <span class="token operator">=</span> cq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>cq_off<span class="token punctuation">.</span>head<span class="token punctuation">;</span>
cring<span class="token operator">-></span>tail <span class="token operator">=</span> cq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>cq_off<span class="token punctuation">.</span>tail<span class="token punctuation">;</span>
cring<span class="token operator">-></span>ring_mask <span class="token operator">=</span> cq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>cq_off<span class="token punctuation">.</span>ring_mask<span class="token punctuation">;</span>
cring<span class="token operator">-></span>ring_entries <span class="token operator">=</span> cq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>cq_off<span class="token punctuation">.</span>ring_entries<span class="token punctuation">;</span>
cring<span class="token operator">-></span>cqes <span class="token operator">=</span> cq_ptr <span class="token operator">+</span> p<span class="token punctuation">.</span>cq_off<span class="token punctuation">.</span>cqes<span class="token punctuation">;</span>
<span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span>
}
off_t get_file_size(int fd) {
struct stat st;
if(fstat(fd, &st) < 0) {
perror("fstat");
return -1;
}
if (S_ISBLK(st.st_mode)) {
unsigned long long bytes;
if (ioctl(fd, BLKGETSIZE64, &bytes) != 0) {
perror("ioctl");
return -1;
}
return bytes;
} else if (S_ISREG(st.st_mode))
return st.st_size;
return -1;
}
void output_to_console(char buf, int len) {
while (len--) {
fputc(buf++, stdout);
}
}
void read_from_cq(struct submitter *s) {
<span class="token keyword">struct</span> <span class="token class-name">file_info</span> <span class="token operator">*</span>fi<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">app_io_cq_ring</span> <span class="token operator">*</span>cring <span class="token operator">=</span> <span class="token operator">&</span>s<span class="token operator">-></span>cq_ring<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_cqe</span> <span class="token operator">*</span>cqe<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> head <span class="token operator">=</span> <span class="token operator">*</span>cring<span class="token operator">-></span>head<span class="token punctuation">;</span>
<span class="token keyword">while</span> <span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token comment">//read_barrier();</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>head <span class="token operator">==</span> <span class="token operator">*</span>cring<span class="token operator">-></span>tail<span class="token punctuation">)</span> <span class="token keyword">break</span><span class="token punctuation">;</span>
cqe <span class="token operator">=</span> <span class="token operator">&</span>cring<span class="token operator">-></span>cqes<span class="token punctuation">[</span>head <span class="token operator">&</span> <span class="token operator">*</span>s<span class="token operator">-></span>cq_ring<span class="token punctuation">.</span>ring_mask<span class="token punctuation">]</span><span class="token punctuation">;</span>
fi <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">file_info</span><span class="token operator">*</span><span class="token punctuation">)</span>cqe<span class="token operator">-></span>user_data<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>cqe<span class="token operator">-></span>res <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token function">fprintf</span><span class="token punctuation">(</span><span class="token constant">stderr</span><span class="token punctuation">,</span> <span class="token string">"Error: %d\n"</span><span class="token punctuation">,</span> cqe<span class="token operator">-></span>res<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">int</span> blocks <span class="token operator">=</span> fi<span class="token operator">-></span>file_sz <span class="token operator">/</span> BLOCK_SZ<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>fi<span class="token operator">-></span>file_sz <span class="token operator">%</span> BLOCK_SZ<span class="token punctuation">)</span> blocks <span class="token operator">++</span><span class="token punctuation">;</span>
<span class="token keyword">int</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
<span class="token keyword">while</span> <span class="token punctuation">(</span><span class="token operator">++</span>i <span class="token operator"><</span> blocks<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token function">output_to_console</span><span class="token punctuation">(</span>fi<span class="token operator">-></span>iovecs<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>iov_base<span class="token punctuation">,</span> fi<span class="token operator">-></span>iovecs<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">.</span>iov_len<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token function">printf</span><span class="token punctuation">(</span><span class="token string">"------------------------i : %d, blocks: %d\n"</span><span class="token punctuation">,</span> i<span class="token punctuation">,</span> blocks<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
head <span class="token operator">++</span><span class="token punctuation">;</span>
<span class="token function">printf</span><span class="token punctuation">(</span><span class="token string">"head: %d, tail: %d, blocks: %d\n"</span><span class="token punctuation">,</span>
head<span class="token punctuation">,</span> <span class="token operator">*</span>cring<span class="token operator">-></span>tail<span class="token punctuation">,</span> blocks<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token operator">*</span>cring<span class="token operator">-></span>head <span class="token operator">=</span> head<span class="token punctuation">;</span>
<span class="token function">printf</span><span class="token punctuation">(</span><span class="token string">"exit read_from_cq\n"</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">//write_barrier();</span>
}
int submit_to_sq(char file_path, struct submitter s) {
<span class="token keyword">int</span> filefd <span class="token operator">=</span> <span class="token function">open</span><span class="token punctuation">(</span>file_path<span class="token punctuation">,</span> O_RDONLY<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>filefd <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">struct</span> <span class="token class-name">app_io_sq_ring</span> <span class="token operator">*</span>sring <span class="token operator">=</span> <span class="token operator">&</span>s<span class="token operator">-></span>sq_ring<span class="token punctuation">;</span>
<span class="token class-name">off_t</span> filesz <span class="token operator">=</span> <span class="token function">get_file_size</span><span class="token punctuation">(</span>filefd<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>filesz <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token class-name">off_t</span> bytes_remaining <span class="token operator">=</span> filesz<span class="token punctuation">;</span>
<span class="token keyword">int</span> blocks <span class="token operator">=</span> filesz <span class="token operator">/</span> BLOCK_SZ<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>filesz <span class="token operator">%</span> BLOCK_SZ<span class="token punctuation">)</span> blocks <span class="token operator">++</span><span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">file_info</span> <span class="token operator">*</span>fi <span class="token operator">=</span> <span class="token function">malloc</span><span class="token punctuation">(</span><span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">file_info</span><span class="token punctuation">)</span> <span class="token operator">+</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">iovec</span><span class="token punctuation">)</span> <span class="token operator">*</span> blocks<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>fi<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">2</span><span class="token punctuation">;</span>
fi<span class="token operator">-></span>file_sz <span class="token operator">=</span> filesz<span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> current_block<span class="token punctuation">;</span>
<span class="token keyword">while</span> <span class="token punctuation">(</span>bytes_remaining<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token class-name">off_t</span> bytes_to_read <span class="token operator">=</span> bytes_remaining<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>bytes_to_read <span class="token operator">></span> BLOCK_SZ<span class="token punctuation">)</span> bytes_to_read <span class="token operator">=</span> BLOCK_SZ<span class="token punctuation">;</span>
fi<span class="token operator">-></span>iovecs<span class="token punctuation">[</span>current_block<span class="token punctuation">]</span><span class="token punctuation">.</span>iov_len <span class="token operator">=</span> bytes_to_read<span class="token punctuation">;</span>
<span class="token keyword">void</span> <span class="token operator">*</span>buf<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">posix_memalign</span><span class="token punctuation">(</span><span class="token operator">&</span>buf<span class="token punctuation">,</span> BLOCK_SZ<span class="token punctuation">,</span> BLOCK_SZ<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">return</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
fi<span class="token operator">-></span>iovecs<span class="token punctuation">[</span>current_block<span class="token punctuation">]</span><span class="token punctuation">.</span>iov_base <span class="token operator">=</span> buf<span class="token punctuation">;</span>
current_block <span class="token operator">++</span><span class="token punctuation">;</span>
bytes_remaining <span class="token operator">-=</span> bytes_to_read<span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">unsigned</span> next_tail <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">,</span> tail <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">,</span> index <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
next_tail <span class="token operator">=</span> tail <span class="token operator">=</span> <span class="token operator">*</span>sring<span class="token operator">-></span>tail<span class="token punctuation">;</span>
next_tail <span class="token operator">++</span><span class="token punctuation">;</span>
index <span class="token operator">=</span> tail <span class="token operator">&</span> <span class="token operator">*</span>s<span class="token operator">-></span>sq_ring<span class="token punctuation">.</span>ring_mask<span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_sqe</span> <span class="token operator">*</span>sqe <span class="token operator">=</span> <span class="token operator">&</span>s<span class="token operator">-></span>sqes<span class="token punctuation">[</span>index<span class="token punctuation">]</span><span class="token punctuation">;</span>
sqe<span class="token operator">-></span>fd <span class="token operator">=</span> filefd<span class="token punctuation">;</span>
sqe<span class="token operator">-></span>flags <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
sqe<span class="token operator">-></span>opcode <span class="token operator">=</span> IORING_OP_READV<span class="token punctuation">;</span>
sqe<span class="token operator">-></span>addr <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token keyword">unsigned</span> <span class="token keyword">long</span><span class="token punctuation">)</span>fi<span class="token operator">-></span>iovecs<span class="token punctuation">;</span>
sqe<span class="token operator">-></span>len <span class="token operator">=</span> blocks<span class="token punctuation">;</span>
sqe<span class="token operator">-></span>off <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
sqe<span class="token operator">-></span>user_data <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token keyword">unsigned</span> <span class="token keyword">long</span> <span class="token keyword">long</span><span class="token punctuation">)</span>fi<span class="token punctuation">;</span>
sring<span class="token operator">-></span>array<span class="token punctuation">[</span>index<span class="token punctuation">]</span> <span class="token operator">=</span> index<span class="token punctuation">;</span>
tail <span class="token operator">=</span> next_tail<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">*</span>sring<span class="token operator">-></span>tail <span class="token operator">!=</span> tail<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token operator">*</span>sring<span class="token operator">-></span>tail <span class="token operator">=</span> tail<span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">int</span> ret <span class="token operator">=</span> <span class="token function">io_uring_enter</span><span class="token punctuation">(</span>s<span class="token operator">-></span>ring_fd<span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> IORING_ENTER_GETEVENTS<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>ret <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">return</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span>
}
int main(int argc, char *argv[]) {
<span class="token keyword">struct</span> <span class="token class-name">submitter</span> <span class="token operator">*</span>s <span class="token operator">=</span> <span class="token function">malloc</span><span class="token punctuation">(</span><span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">submitter</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>s<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token function">perror</span><span class="token punctuation">(</span><span class="token string">"malloc"</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token function">memset</span><span class="token punctuation">(</span>s<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">submitter</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// 1、setup</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">app_setup_uring</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token keyword">int</span> i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token keyword">for</span> <span class="token punctuation">(</span>i <span class="token operator">=</span> <span class="token number">1</span><span class="token punctuation">;</span>i <span class="token operator"><</span> argc<span class="token punctuation">;</span>i <span class="token operator">++</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token comment">// 2、submit</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">submit_to_sq</span><span class="token punctuation">(</span>argv<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">,</span> s<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token comment">//fprintf(stderr, "Error reading file\n");</span>
<span class="token keyword">return</span> <span class="token number">1</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token function">read_from_cq</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token keyword">return</span> <span class="token number">0</span><span class="token punctuation">;</span>
}
3、liburing
由于 io_uring 使用起来比较麻烦，作者封装了 io_uring 接口，创作了 liburing 库。
# 安装 liburing
git clone https://github.com/axboe/liburing.git
./configure
make && make install
- 1
- 2
- 3
- 4
3.1、liburing api
// 初始化io_uring，内部调用io_uring_setup
int io_uring_queue_init_params(unsigned entries, struct io_uring *ring,
struct io_uring_params *p); // 提交 sq 到内核，内核完成后移动到 cq，内部调用 io_uring_enter
// 1、提交io请求：将sqe的偏移信息加入到sq，提交sq到内核，不阻塞等待其完成
// 2、等待io完成：内核在io完成后，自动将sqe的偏移信息加入到cq
int io_uring_submit(struct io_uring *ring); // 等待io完成，获取cqe
// 阻塞等待
unsigned io_uring_peek_batch_cqe(struct io_uring ring,
struct io_uring_cqe cqes, unsigned count);
// 不阻塞等待
int io_uring_wait_cqes(struct io_uring ring, struct io_uring_cqe cqe_ptr,
unsigned wait_nr, struct __kernel_timespec ts,
sigset_t sigmask); // 轮询 cq 队列，将 cq 队首后移动 nr 个
static inline void io_uring_cq_advance(struct io_uring *ring, unsigned nr) // 和libaio封装的io_prep_writev一样
static inline void io_uring_prep_writev(struct io_uring_sqe sqe, int fd,const struct iovec iovecs, unsigned nr_vecs, off_t offset) // 和libaio封装的io_prep_readv一样
static inline void io_uring_prep_readv(struct io_uring_sqe sqe, int fd, const struct iovec iovecs, unsigned nr_vecs, off_t offset) // 销毁 io
void io_uring_queue_exit(struct io_uring *ring);
3.2、测试代码
利用 liburing 编写的简单测试 iouring_server
// gcc -o iouring_server iouring_server.c -luring
#include <liburing.h>
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#define ENTRIES_LENGTH 4096
#define MAX_CONNECTIONS 1024
#define BUFFER_LENGTH 1024
char buf_table[MAX_CONNECTIONS][BUFFER_LENGTH] = {0};
// 传递的事件
enum {
READ,
WRITE,
ACCEPT,
};
// 连接信息
struct conninfo {
int connfd; // fd
int type; // 事件类型
};
void set_read_event(struct io_uring ring, int fd, void buf, size_t len, int flags) {
<span class="token keyword">struct</span> <span class="token class-name">io_uring_sqe</span> <span class="token operator">*</span>sqe <span class="token operator">=</span> <span class="token function">io_uring_get_sqe</span><span class="token punctuation">(</span>ring<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// io_uring 读事件</span>
<span class="token function">io_uring_prep_recv</span><span class="token punctuation">(</span>sqe<span class="token punctuation">,</span> fd<span class="token punctuation">,</span> buf<span class="token punctuation">,</span> len<span class="token punctuation">,</span> flags<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">conninfo</span> ci <span class="token operator">=</span> <span class="token punctuation">{<!-- --></span>
<span class="token punctuation">.</span>connfd <span class="token operator">=</span> fd<span class="token punctuation">,</span>
<span class="token punctuation">.</span>type <span class="token operator">=</span> READ
<span class="token punctuation">}</span><span class="token punctuation">;</span>
<span class="token function">memcpy</span><span class="token punctuation">(</span><span class="token operator">&</span>sqe<span class="token operator">-></span>user_data<span class="token punctuation">,</span> <span class="token operator">&</span>ci<span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">conninfo</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
}
void set_write_event(struct io_uring ring, int fd, const void buf, size_t len, int flags) {
<span class="token keyword">struct</span> <span class="token class-name">io_uring_sqe</span> <span class="token operator">*</span>sqe <span class="token operator">=</span> <span class="token function">io_uring_get_sqe</span><span class="token punctuation">(</span>ring<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// io_uring 写事件</span>
<span class="token function">io_uring_prep_send</span><span class="token punctuation">(</span>sqe<span class="token punctuation">,</span> fd<span class="token punctuation">,</span> buf<span class="token punctuation">,</span> len<span class="token punctuation">,</span> flags<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">conninfo</span> ci <span class="token operator">=</span> <span class="token punctuation">{<!-- --></span>
<span class="token punctuation">.</span>connfd <span class="token operator">=</span> fd<span class="token punctuation">,</span>
<span class="token punctuation">.</span>type <span class="token operator">=</span> WRITE
<span class="token punctuation">}</span><span class="token punctuation">;</span>
<span class="token function">memcpy</span><span class="token punctuation">(</span><span class="token operator">&</span>sqe<span class="token operator">-></span>user_data<span class="token punctuation">,</span> <span class="token operator">&</span>ci<span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">conninfo</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
}
void set_accept_event(struct io_uring ring, int fd,
struct sockaddr cliaddr, socklen_t *clilen, unsigned flags) {
<span class="token comment">// 获取 sq 队列的空 sqe</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_sqe</span> <span class="token operator">*</span>sqe <span class="token operator">=</span> <span class="token function">io_uring_get_sqe</span><span class="token punctuation">(</span>ring<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// io_uring的accept事件：将fd放入到sqe里</span>
<span class="token function">io_uring_prep_accept</span><span class="token punctuation">(</span>sqe<span class="token punctuation">,</span> fd<span class="token punctuation">,</span> cliaddr<span class="token punctuation">,</span> clilen<span class="token punctuation">,</span> flags<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// 用于回调函数</span>
<span class="token keyword">struct</span> <span class="token class-name">conninfo</span> ci <span class="token operator">=</span> <span class="token punctuation">{<!-- --></span>
<span class="token punctuation">.</span>connfd <span class="token operator">=</span> fd<span class="token punctuation">,</span>
<span class="token punctuation">.</span>type <span class="token operator">=</span> ACCEPT
<span class="token punctuation">}</span><span class="token punctuation">;</span>
<span class="token function">memcpy</span><span class="token punctuation">(</span><span class="token operator">&</span>sqe<span class="token operator">-></span>user_data<span class="token punctuation">,</span> <span class="token operator">&</span>ci<span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">conninfo</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
}
int main() {
<span class="token keyword">int</span> listenfd <span class="token operator">=</span> <span class="token function">socket</span><span class="token punctuation">(</span>AF_INET<span class="token punctuation">,</span> SOCK_STREAM<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>listenfd <span class="token operator">==</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">sockaddr_in</span> servaddr<span class="token punctuation">,</span> clientaddr<span class="token punctuation">;</span>
servaddr<span class="token punctuation">.</span>sin_family <span class="token operator">=</span> AF_INET<span class="token punctuation">;</span>
servaddr<span class="token punctuation">.</span>sin_addr<span class="token punctuation">.</span>s_addr <span class="token operator">=</span> <span class="token function">htonl</span><span class="token punctuation">(</span>INADDR_ANY<span class="token punctuation">)</span><span class="token punctuation">;</span>
servaddr<span class="token punctuation">.</span>sin_port <span class="token operator">=</span> <span class="token function">htons</span><span class="token punctuation">(</span><span class="token number">9999</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">-</span><span class="token number">1</span> <span class="token operator">==</span> <span class="token function">bind</span><span class="token punctuation">(</span>listenfd<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">sockaddr</span><span class="token operator">*</span><span class="token punctuation">)</span><span class="token operator">&</span>servaddr<span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span>servaddr<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">return</span> <span class="token operator">-</span><span class="token number">2</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token function">listen</span><span class="token punctuation">(</span>listenfd<span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_params</span> params<span class="token punctuation">;</span>
<span class="token function">memset</span><span class="token punctuation">(</span><span class="token operator">&</span>params<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span>params<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// 初始化队列，内部调用io_uring_setup</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring</span> ring<span class="token punctuation">;</span>
<span class="token function">io_uring_queue_init_params</span><span class="token punctuation">(</span>ENTRIES_LENGTH<span class="token punctuation">,</span> <span class="token operator">&</span>ring<span class="token punctuation">,</span> <span class="token operator">&</span>params<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token class-name">socklen_t</span> clilen <span class="token operator">=</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span>clientaddr<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token function">set_accept_event</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> listenfd<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">sockaddr</span><span class="token operator">*</span><span class="token punctuation">)</span><span class="token operator">&</span>clientaddr<span class="token punctuation">,</span> <span class="token operator">&</span>clilen<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">while</span> <span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token comment">// 封装 io_uring_enter</span>
<span class="token comment">// 1、提交io请求：将sqe的偏移信息加入到sq，提交sq到内核，不阻塞等待其完成</span>
<span class="token comment">// 2、等待io完成：内核在io完成后，自动将sqe的偏移信息加入到cq</span>
<span class="token function">io_uring_submit</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// 从获取 cqe 的两种方式</span>
<span class="token comment">// 1、阻塞等待io完成，获取 cqe</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_cqe</span> <span class="token operator">*</span>cqe<span class="token punctuation">;</span>
<span class="token keyword">int</span> ret <span class="token operator">=</span> <span class="token function">io_uring_wait_cqe</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> <span class="token operator">&</span>cqe<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// 2、不阻塞等待io完成，没有cqe返回错误，获取 cqe</span>
<span class="token keyword">struct</span> <span class="token class-name">io_uring_cqe</span> <span class="token operator">*</span>cqes<span class="token punctuation">[</span><span class="token number">10</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
<span class="token keyword">int</span> cqecount <span class="token operator">=</span> <span class="token function">io_uring_peek_batch_cqe</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> cqes<span class="token punctuation">,</span> <span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">int</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
<span class="token keyword">unsigned</span> count <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
<span class="token keyword">for</span> <span class="token punctuation">(</span>i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator"><</span> cqecount<span class="token punctuation">;</span> <span class="token operator">++</span>i<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
cqe <span class="token operator">=</span> cqes<span class="token punctuation">[</span>i<span class="token punctuation">]</span><span class="token punctuation">;</span>
count <span class="token operator">++</span><span class="token punctuation">;</span>
<span class="token keyword">struct</span> <span class="token class-name">conninfo</span> ci<span class="token punctuation">;</span>
<span class="token function">memcpy</span><span class="token punctuation">(</span><span class="token operator">&</span>ci<span class="token punctuation">,</span> <span class="token operator">&</span>cqe<span class="token operator">-></span>user_data<span class="token punctuation">,</span> <span class="token keyword">sizeof</span><span class="token punctuation">(</span>ci<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>ci<span class="token punctuation">.</span>type <span class="token operator">==</span> ACCEPT<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">int</span> connfd <span class="token operator">=</span> cqe<span class="token operator">-></span>res<span class="token punctuation">;</span>
<span class="token keyword">char</span> <span class="token operator">*</span>buffer <span class="token operator">=</span> buf_table<span class="token punctuation">[</span>connfd<span class="token punctuation">]</span><span class="token punctuation">;</span>
<span class="token function">set_read_event</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> connfd<span class="token punctuation">,</span> buffer<span class="token punctuation">,</span> <span class="token number">1024</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token comment">// io_uring 设置一次，触发一次</span>
<span class="token function">set_accept_event</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> listenfd<span class="token punctuation">,</span> <span class="token punctuation">(</span><span class="token keyword">struct</span> <span class="token class-name">sockaddr</span><span class="token operator">*</span><span class="token punctuation">)</span><span class="token operator">&</span>clientaddr<span class="token punctuation">,</span> <span class="token operator">&</span>clilen<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>ci<span class="token punctuation">.</span>type <span class="token operator">==</span> READ<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">int</span> bytes_read <span class="token operator">=</span> cqe<span class="token operator">-></span>res<span class="token punctuation">;</span>
<span class="token keyword">if</span> <span class="token punctuation">(</span>bytes_read <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token function">close</span><span class="token punctuation">(</span>ci<span class="token punctuation">.</span>connfd<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>bytes_read <span class="token operator"><</span> <span class="token number">0</span><span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">char</span> <span class="token operator">*</span>buffer <span class="token operator">=</span> buf_table<span class="token punctuation">[</span>ci<span class="token punctuation">.</span>connfd<span class="token punctuation">]</span><span class="token punctuation">;</span>
<span class="token function">set_write_event</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> ci<span class="token punctuation">.</span>connfd<span class="token punctuation">,</span> buffer<span class="token punctuation">,</span> bytes_read<span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token keyword">if</span> <span class="token punctuation">(</span>ci<span class="token punctuation">.</span>type <span class="token operator">==</span> WRITE<span class="token punctuation">)</span> <span class="token punctuation">{<!-- --></span>
<span class="token keyword">char</span> <span class="token operator">*</span>buffer <span class="token operator">=</span> buf_table<span class="token punctuation">[</span>ci<span class="token punctuation">.</span>connfd<span class="token punctuation">]</span><span class="token punctuation">;</span>
<span class="token function">set_read_event</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> ci<span class="token punctuation">.</span>connfd<span class="token punctuation">,</span> buffer<span class="token punctuation">,</span> <span class="token number">1024</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
<span class="token comment">// cq队列一次轮询完成后，因为cqe的取出，需要调整队首的位置，以便下次使用</span>
<span class="token function">io_uring_cq_advance</span><span class="token punctuation">(</span><span class="token operator">&</span>ring<span class="token punctuation">,</span> count<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>
}
4、参考
[转帖]高性能异步io机制：io_uring的更多相关文章
- 【译】深入理解python3.4中Asyncio库与Node.js的异步IO机制
转载自http://xidui.github.io/2015/10/29/%E6%B7%B1%E5%85%A5%E7%90%86%E8%A7%A3python3-4-Asyncio%E5%BA%93% ...
- {python之IO多路复用} IO模型介绍 阻塞IO(blocking IO) 非阻塞IO(non-blocking IO) 多路复用IO(IO multiplexing) 异步IO(Asynchronous I/O) IO模型比较分析 selectors模块
python之IO多路复用 阅读目录 一 IO模型介绍 二 阻塞IO(blocking IO) 三 非阻塞IO(non-blocking IO) 四 多路复用IO(IO multiplexing) 五 ...
- Tornado异步IO
Tornado提供了强大的异步IO机制,提高了服务器的响应能力. @tornado.web.asynchronous tornado默认在处理函数返回时关闭链接,@tornado.web.asynch ...
- node.js零基础详细教程(4)：node.js事件机制、node异步IO操作
第四章 建议学习时间3小时 课程共10章 学习方式:详细阅读,并手动实现相关代码 学习目标:此教程将教会大家 安装Node.搭建服务器.express.mysql.mongodb.编写后台业务逻辑. ...
- python 全栈开发，Day44(IO模型介绍,阻塞IO,非阻塞IO,多路复用IO,异步IO,IO模型比较分析,selectors模块,垃圾回收机制)
昨日内容回顾 协程实际上是一个线程,执行了多个任务,遇到IO就切换 切换,可以使用yield,greenlet 遇到IO gevent: 检测到IO,能够使用greenlet实现自动切换,规避了IO阻 ...
- (IO模型介绍,阻塞IO,非阻塞IO,多路复用IO,异步IO,IO模型比较分析,selectors模块,垃圾回收机制)
参考博客: https://www.cnblogs.com/xiao987334176/p/9056511.html 内容回顾 协程实际上是一个线程,执行了多个任务,遇到IO就切换 切换,可以使用yi ...
- python全栈开发，Day44（IO模型介绍，阻塞IO，非阻塞IO，多路复用IO，异步IO，IO模型比较分析，selectors模块，垃圾回收机制）
昨日内容回顾 协程实际上是一个线程,执行了多个任务,遇到IO就切换 切换,可以使用yield,greenlet 遇到IO gevent: 检测到IO,能够使用greenlet实现自动切换,规避了IO阻 ...
- 服务器端高性能的IO模型 转自酷勤网
服务器端编程经常需要构造高性能的IO模型,常见的IO模型有四种: (1)同步阻塞IO(BlockingIO):即传统的IO模型. (2)同步非阻塞IO(Non-blockingIO):默认创建的soc ...
- 异步IO简介
最近想学习一下libevent,就先翻译一下libevent的官方文档吧. 英文原文链接:http://www.wangafu.net/~nickm/libevent-book/01_intro.ht ...
- Python高级编程和异步IO并发编程
第1章 课程简介介绍如何配置系统的开发环境以及如何加入github私人仓库获取最新源码. 1-1 导学 试看 1-2 开发环境配置 1-3 资源获取方式第2章 python中一切皆对象本章节首先对比静 ...
随机推荐
- Llama2-Chinese项目：7-外延能力LangChain集成
本文介绍了Llama2模型集成LangChain框架的具体实现,这样可更方便地基于Llama2开发文档检索.问答机器人和智能体应用等. 1.调用Llama2类 针对LangChain[1]框架 ...
- MySQL篇：bug2_ Navicate无法添加或更新子行-外键约束失败
问题产生原因 Mysql中如果表和表之间建立的外键约束,则无法删除表及修改表结构. 解决办法 解决方法是在Mysql中取消外键约束: SET FOREIGN_KEY_CHECKS=0; 再添加值, 然 ...
- 云图说｜将源端MongoDB业务搬迁至华为云DDS的几种方式
摘要:华为云文档数据库服务DDS能帮您在业务需要时,快速便捷的搬迁源端MongoDB业务上云. 如果您因业务调整或需要使用华为云文档数据库DDS特性功能时,可以通过数据迁移功能将原有MongoDB数据 ...
- 将模型转为NNIE框架支持的wk模型第一步：tensorflow->caffe
摘要:本系列文章旨在分享tensorflow->onnx->Caffe->wk模型转换流程,主要针对的是HI3516CV500, Hi3519AV100 支持NNIE推理框架的海思芯 ...
- 浏览器层面优化前端性能(2):Reader引擎线程与模块分析优化点
Reader 引擎线程与模块分析 首先是网页内容,加载完输入到HTML解释器,解释后构成DOM树,这期间如果遇到JavaScript代码就交给JavaScript引擎去处理,如果网页中包含CSS,就交 ...
- 火山引擎 DataTester：构建增长闭环，3-5 人即可搭建企业增长团队
更多技术交流.求职机会,欢迎关注字节跳动数据平台微信公众号,回复[1]进入官方交流群 增长是一个老生常谈的话题,对许多企业而言这个词都很熟悉,但在实践过程中仍然有很多的疑问,比如指标体系应该怎么搭建? ...
- 熔断、限流、降级 —— SpringCloud Alibaba Sentinel
Sentinel 简介 Sentinel 是阿里中间件团队开源的,面向分布式服务架构的高可用流量防护组件,主要以流量为切入点,从限流.流量整形.熔断降级.系统负载保护.热点防护等多个维度来帮助开发者保 ...
- C++11实用特性1
1 原始字面量 有时候在输出一个路径字符串时,编译器会将其中的部分内容识别成转义字符进行输出,可以用R "xxx(原始字符串)xxx"其中()两边的字符串可以省略.原始字面量R可以 ...
- 【JAVA基础】日志管理
LOGGER.debug("Request uri: {}, headers: {}", signedRequest.getURI(), signedRequest.getAllH ...
- PySpark 报错 java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver
解决方案: mv mysql-connector-java-8.0.20.jar $SPARK_HOME/jars/ 驱动文件mysql-connector-java-8.0.20.jar是从mave ...