ceph-性能调优

Ceph 参数性能调优
https://blog.csdn.net/changtao381/article/details/49907115
这篇文章对我的环境有较大帮助

ceph优化记录 ceph.conf优化详解
https://blog.csdn.net/kingzdd/article/details/80103086

Ceph配置参数分析
https://blog.csdn.net/for_tech/article/details/72123223#t20

SSD固态盘应用于Ceph集群的四种典型使用场景
https://www.cnblogs.com/gzxbkk/p/7724851.html

手动部署 ceph osd (luminous 版)
https://blog.csdn.net/signmem/article/details/78579882

ceph常用运维技巧总结1
https://www.jianshu.com/p/5ab19b0c6149

CEPH集群操作入门--配置
https://www.cnblogs.com/luxiaodai/p/10006036.html

Monitor clock skew detected问题解决

mon clock drift allowed = 2 增加时间误差（不推荐）
mon clock drift warn backoff = 30

修改ceph.conf后
重启mon服务
/etc/init.d/ceph restart mon
或者直接重启monitor
systemctl restart ceph-mon

===============================================

查参数
root@cu-pve04:/etc/ceph# ceph-conf -D >a.txt
root@cu-pve04:/etc/ceph# cat a.txt |grep bluestore_cache

vi /etc/ceph/ceph.conf中改参数

查看并修改运行时参数的，好像也没用

ceph daemon osd.0 config show | less
ceph daemon osd.* help

ceph daemon osd.2 config get mon_osd_full_ratio
ceph tell osd.* injectargs

需要调整的参数，在ceph.conf中写入到global，但用空格代替_。

filestore_op_threads = 10
filestore_journal_writeahead = true
filestore_fiemap = true
filestore_wbthrottle_enable = false
filestore_fd_cache_size = 1024
filestore_commit_timeout = 30000
filestore_min_sync_interval = 5
filestore_max_sync_interval = 10
filestore_queue_max_ops = 25000
filestore_queue_max_bytes = 1048576000
journal_force_aio = true
journal_max_write_bytes = 1073714824
journal_max_write_entries = 10000
osd_pg_object_context_cache_count = 1024
osd_client_message_size_cap = 2147483648
osd_client_message_cap = 5000
rbd_cache_size = 2684354560
rbd_cache_max_dirty = 2684354560
rbd_cache_max_dirty_age = 5

===========================

空格方式

filestore op threads = 10
filestore journal parallel = false
filestore journal writeahead = true
journal dio = true
journal aio = true
journal force aio = true
filestore fiemap = true
filestore wbthrottle enable = false
filestore fd cache size = 1024
filestore omap header cache size = 1024
osd pg object context cache count = 1024
filestore commit timeout = 30000
filestore min sync interval = 5
filestore max sync interval = 10
osd client message size cap = 21474836480
osd client message cap = 5000
filestore queue max ops = 25000
filestore queue max bytes = 10485760000
journal max write bytes = 10737148240
journal max write entries = 10000
journal queue max ops = 50000
journal queue max bytes = 10485760000
rbd cache = true
rbd cache size = 26843545600
rbd cache max dirty = 13421772800
rbd cache max dirty age = 5

===========================

下面是默认值，有些就是，所以不用设置

需要调整的
filestore_op_threads = 2
filestore_journal_parallel=false
filestore_journal_writeahead=false
filestore_fiemap = false
filestore_wbthrottle_enable=true
filestore_fd_cache_size = 128
filestore_omap_header_cache_size = 1024
filestore_commit_timeout=600
filestore_min_sync_interval = 0.01
filestore_max_sync_interval = 5
filestore_queue_max_ops = 5
filestore_queue_max_bytes = 104857600

不需要调整的
journal_dio = true
journal_aio = true
journal_force_aio = false
journal_max_write_bytes = 10485760
journal_max_write_entries = 100
osd_pg_object_context_cache_count = 64
osd_client_message_size_cap = 524288000
osd_client_message_cap = 100
rbd_cache = true
rbd_cache_size = 33554432
rbd_cache_max_dirty = 25165824
rbd_cache_max_dirty_age = 1

没有这些参数
journal_queue_max_ops = 50000
journal_queue_max_bytes = 10485760000

===========================

这个有些用

osd client message size cap = 2147483648
rbd cache size = 2684354560
rbd cache max dirty = 2684354560
rbd cache max dirty age = 5