Ceph 之 Background on http frontends
Background on http frontends
-civetweb frontend
--thread per connection, requires a lot of threads
---qos or priority queuing would block frontend threads
-beast frontend
--boost::beast for http parsing
--boost::asio for async networking/io
--async for accepting connections and reading headers
---good model for qos - can queue requests without blocking threads
--synchronous call to process_request()
---thread per request, still need lots of threads
goal: scale requests independently of threads
-why boost::asio
--doesn't impose a threading model. io_service object is a reactor, call run() from any thread
--mature library, basis for C++ std::net library in Networking TS [1]
--the Extensible Asynchronous Model [2] provides several options for async primitives (callbacks, futures, coroutines)
--boost::asio::spawn() stackful coroutines: "enables programs to implement asynchronous logic in a synchronous manner" [3 http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/spawn.html]
proposed librados interfaces for asio [4 https://github.com/ceph/ceph/pull/19054]
--header-only wrapper over librados c++ api
--conform to the Extensible Asynchronous Model, so support the same primitives - see unit tests for examples
--deeper Objecter integration work in progress by Adam Emerson [5]
--gives radosgw a unified interface for async operations over http and rados
async process_request() [6]
-add optional yield_context* argument to process_request()
-beast frontend passes one, civetweb passes nullptr
-any librados calls use new interface when given a yield_context
-requires the yield_context* to be passed everywhere in between
--but we can stash it in req_state to make it available to all ops
-getting started with the easy stuff
--rgw_get_system_obj()
--reading user objects for authentication
--reading bucket/bucket instance objects (common to most s3/swift ops)
-this process leaves a lot of gaps. for example, rgw_get_system_obj() is in tons of call paths without access to a yield_context
--(either outside process_request(), or just aren't hooked up yet)
--just passing 'nullptr' makes it impossible to differentiate the yield_context argument from its 4 other arguments that default to nullptr!
--that makes it impossible to reason about which call paths could run asynchronously
-measurable progress towards full asynchrony
--new vocabulary type 'optional_yield_context' with 'null_yield' for empty value
--null_yield designates a call site that is definitely synchronous
--makes it easy to audit the code and find the pieces that still need conversion
-fighting regression once we're close
--have librados calls log warnings when called synchronously from a beast frontend thread (using a thread_local flag)
--scan those logs in teuthology runs to flag failures
and then?
-vastly reduce the number of frontend threads for beast
-consolidate other background threads
remaining work:
- RGWGetObj waits on AioCompletions - use AioThrottle from PutObj instead
- replace librados IoCtx::operate() calls with rgw_rados_operate() and optional_yield_context
- thread optional_yield_context all the way from beast frontend to rgw_rados_operate() calls
- some cls client calls use IoCtx::operate() directly
- block_while_resharding() sleeps on a condition variable
- no async interface for pool object listings with IoCtx.nobjects_begin()
- libcurl http requests for auth (Keystone and OPA)
[1] "C++ Technical Specification - Extensions for Networking"
http://cplusplus.github.io/networking-ts/draft.pdf
[2] "Library Foundations for Asynchronous Operations"
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf
[3] Reference: boost::asio::spawn
http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/spawn.html
[4] "librados: add async interfaces for use with boost::asio"
https://github.com/ceph/ceph/pull/19054
[5] "osdc/Objecter: Boost.Asio (I object!)"
https://github.com/ceph/ceph/pull/16715
[6] work in progress branch:
https://github.com/cbodley/ceph/commits/wip-rgw-async-process-171120
Ceph 之 Background on http frontends的更多相关文章
- 虚拟机使用ceph-deploy安装ceph
参考: ceph官网介绍 使用ceph-deploy安装Ceph 12.x 安装虚拟机 首先安装虚拟机环境,虚拟机安装这里不做介绍,本实验使用的镜像为CentOS-7-x86_64-Everythin ...
- Ceph常用命令
目录 [1.环境准备] [2.部署管理] [3.集群扩容] [4.用户管理] [5.密钥环管理] [6.块设备管理] [7.快照管理] [8.参考链接] 简要说明: 最近心血来潮,对分布式存储感兴趣, ...
- ceph mimic版本 部署安装
ceph 寻址过程 1. file --- object映射, 把file分割成N个相同的对象 2. object - PG 映射, 利用静态hash得到objectID的伪随机值,在 "位 ...
- Ceph常规操作及常见问题梳理
Ceph集群管理 每次用命令启动.重启.停止Ceph守护进程(或整个集群)时,必须指定至少一个选项和一个命令,还可能要指定守护进程类型或具体例程. **命令格式如 {commandline} [opt ...
- ceph kubernetes中使用
1.在管理节点上,进入刚创建的放置配置文件的目录,用 ceph-deploy 执行如下步骤 mkdir /opt/cluster-ceph cd /opt/cluster-ceph ceph-depl ...
- 1、ceph-deploy之部署ceph集群
环境说明 server:3台虚拟机,挂载卷/dev/vdb 10G 系统:centos7.2 ceph版本:luminous repo: 公网-http://download.ceph.com,htt ...
- CEPH RGW多 ZONE的配置
相关的名称解释 Region :可以理解为区域,是基于地理位置的逻辑划分:如:华南,华北之类,包含多个region的Ceph集群必须指定一个master region,一个region可以包含一个或者 ...
- Ceph 管理和使用
ceph 管理 上次介绍了Ceph集群架构并且搭建了ceph集群,本节介绍ceph用户认证流程和挂载.cephFS.ceph RBD以及ceph mds高可用 1. ceph 授权流程和用户权限管理 ...
- 安装ceph (快速) 步骤二:存储集群
用 ceph-deploy 从管理节点建立一个 Ceph 存储集群,该集群包含三个节点,以此探索 Ceph 的功能. 创建一个 Ceph 存储集群,它有一个 Monitor 和两个 OSD 守护进程. ...
随机推荐
- 软件-浏览器-GoogleChrome:Google Chrome
ylbtech-软件-浏览器-GoogleChrome:Google Chrome Google Chrome是一款由Google公司开发的网页浏览器,该浏览器基于其他开源软件撰写,包括WebKit, ...
- 【arc075f】AtCoder Regular Contest 075 F - Mirrored
题意 给定一个数x,问有多少个正整数y,使得rev(y)-y==x 其中rev(x)表示x按位翻转之后得到的数. x<=1e9 做法 首先通过打表发现,这个答案不会很大. 这就说明解相当地松弛. ...
- container中的内容 垂直-水平居中
.container{ display: box; display: -webkit-box; -webkit-box-pack: center; -webkit-box-align: center; ...
- 可持久化线段树的学习(区间第k大和查询历史版本的数据)(杭电多校赛第二场1011)
以前我们学习了线段树可以知道,线段树的每一个节点都储存的是一段区间,所以线段树可以做简单的区间查询,更改等简单的操作. 而后面再做有些题目,就可能会碰到一种回退的操作.这里的回退是指回到未做各种操作之 ...
- DVWA 之low级别sql注入
将Security level设为low,在左侧列表中选择“SQL Injection”,然后在右侧的“User ID”文本框中输入不同的数字就会显示相应的用户信息. 我们首先需要判断这里所传输的参数 ...
- JDBC2 --- 获取数据库连接的方式二 --- 技术搬运工(尚硅谷)
/** * 方式二,对方式一的迭代 * 在如下的程序中,不出现第三方的api,使得程序具有更好的可移植性. * @throws Exception */ @Test public void testC ...
- nodejs package.json说明
{ "name": "test", //项目名称(必须),由小写英文字母.数字和下划线,不能含空格 "version": "1.0 ...
- TZ_13_微服务场景Eureka
1.搭建Eureka的注册中心 1.1Eureka几个时间间隔配置详解 1 >客户端信息上报到eureka服务的时间周期,配置的值越小,上报越频繁,eureka服务器应用状态管理一致性越高 #客 ...
- JavaScript多继承(转载)
js里是否有多继承,如何实现多继承.在这里可以看看java是如何处理多继承的问题,java里是没有多继承的,即一个子类不能同时继承多个父类,但可以实现多个接口,这也间接的实现了多继承.主要是因为多继承 ...
- [转]深入理解ajax系列——进度事件
一般地,使用readystatechange事件探测HTTP请求的完成.XHR2规范草案定义了进度事件Progress Events规范,XMLHttpRequest对象在请求的不同阶段触发不同类型的 ...