ceph-报错日志

由于时钟不一致问题，导致ceph存储有问题

clock skew
时钟偏移
overall
adj. 全部的；全体的；一切在内的
stamped
adj. 铭刻的；盖上邮戳的；顿足的

beacon
vt. 照亮，指引

2019-04-29 17:00:00.000223 mon.cu-pve04 mon.0 192.168.7.204:6789/0 1959 : cluster [WRN] overall HEALTH_WARN clock skew detected on mon.cu-pve05, mon.cu-pve06

2019-04-29 17:00:11.495180 mon.cu-pve04 mon.0 192.168.7.204:6789/0 1960 : cluster [WRN] mon.1 192.168.7.205:6789/0 clock skew 1.30379s > max 0.05s
2019-04-29 17:00:11.495343 mon.cu-pve04 mon.0 192.168.7.204:6789/0 1961 : cluster [WRN] mon.2 192.168.7.206:6789/0 clock skew 0.681995s > max 0.05s
2019-04-29 17:14:41.500133 mon.cu-pve04 mon.0 192.168.7.204:6789/0 2106 : cluster [WRN] mon.1 192.168.7.205:6789/0 clock skew 1.73357s > max 0.05s
2019-04-29 17:14:41.500307 mon.cu-pve04 mon.0 192.168.7.204:6789/0 2107 : cluster [WRN] mon.2 192.168.7.206:6789/0 clock skew 0.671272s > max 0.05s

2019-04-29 17:35:33.320667 mon.cu-pve04 mon.0 192.168.7.204:6789/0 2342 : cluster [WRN] message from mon.1 was stamped 2.355514s in the future, clocks not synchronized
2019-04-29 17:39:59.322154 mon.cu-pve04 mon.0 192.168.7.204:6789/0 2397 : cluster [DBG] osdmap e191: 24 total, 24 up, 24 in
2019-04-29 18:32:24.854130 mon.cu-pve04 mon.0 192.168.7.204:6789/0 3026 : cluster [DBG] osdmap e194: 24 total, 24 up, 24 in
2019-04-29 19:00:00.000221 mon.cu-pve04 mon.0 192.168.7.204:6789/0 3324 : cluster [WRN] overall HEALTH_WARN clock skew detected on mon.cu-pve05, mon.cu-pve06

2019-04-29 17:01:31.898307 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 649 : cluster [DBG] pgmap v676: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail
2019-04-29 17:01:33.927961 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 650 : cluster [DBG] pgmap v677: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.97KiB/s wr, 0op/s
2019-04-29 17:01:35.956276 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 651 : cluster [DBG] pgmap v678: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 588B/s rd, 2.71KiB/s wr, 1op/s
2019-04-29 17:01:37.981052 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 652 : cluster [DBG] pgmap v679: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 588B/s rd, 2.71KiB/s wr, 1op/s
2019-04-29 17:01:40.014386 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 653 : cluster [DBG] pgmap v680: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 589B/s rd, 4.03KiB/s wr, 1op/s
2019-04-29 17:01:42.042173 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 654 : cluster [DBG] pgmap v681: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 588B/s rd, 4.02KiB/s wr, 1op/s
2019-04-29 17:01:44.072142 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 655 : cluster [DBG] pgmap v682: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 588B/s rd, 5.01KiB/s wr, 1op/s
2019-04-29 17:01:46.100477 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 656 : cluster [DBG] pgmap v683: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.89KiB/s rd, 3.20KiB/s wr, 1op/s
2019-04-29 17:01:48.129701 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 657 : cluster [DBG] pgmap v684: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.31KiB/s rd, 2.46KiB/s wr, 0op/s
2019-04-29 17:01:50.161716 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 658 : cluster [DBG] pgmap v685: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.31KiB/s rd, 2.46KiB/s wr, 0op/s
2019-04-29 17:01:52.190373 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 659 : cluster [DBG] pgmap v686: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.31KiB/s rd, 1.15KiB/s wr, 0op/s
2019-04-29 17:01:54.220284 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 660 : cluster [DBG] pgmap v687: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.31KiB/s rd, 1.15KiB/s wr, 0op/s
2019-04-29 17:01:56.248956 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 661 : cluster [DBG] pgmap v688: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail; 1.31KiB/s rd, 168B/s wr, 0op/s
2019-04-29 17:01:58.273446 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 662 : cluster [DBG] pgmap v689: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail
2019-04-29 17:02:00.305394 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 663 : cluster [DBG] pgmap v690: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail
2019-04-29 17:02:02.334375 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 664 : cluster [DBG] pgmap v691: 1152 pgs: 1152 active+clean; 32.1GiB data, 121GiB used, 52.3TiB / 52.4TiB avail

2019-04-30 00:22:14.177176 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 13697 : cluster [DBG] pgmap v13716: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:22:16.203475 mgr.cu-pve05 client.64099 192.168.7.205:0/2045992877 13698 : cluster [DBG] pgmap v13717: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:22:28.348815 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6578 : cluster [WRN] daemon mds.cu-pve04 is not responding, replacing it as rank 0 with standby daemon mds.cu-pve06
2019-04-30 00:22:28.349010 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6579 : cluster [INF] Standby daemon mds.cu-pve05 is not responding, dropping it
2019-04-30 00:22:28.353359 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6580 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED)
2019-04-30 00:22:28.353476 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6581 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)
2019-04-30 00:22:28.364180 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6582 : cluster [DBG] osdmap e195: 24 total, 24 up, 24 in
2019-04-30 00:22:28.374585 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6583 : cluster [DBG] fsmap cephfs-1/1/1 up {0=cu-pve06=up:replay}
2019-04-30 00:22:29.413750 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6584 : cluster [INF] Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available)
2019-04-30 00:22:29.425556 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6585 : cluster [DBG] mds.0 192.168.7.206:6800/3970858648 up:reconnect
2019-04-30 00:22:29.425710 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6586 : cluster [DBG] mds.? 192.168.7.204:6800/2960873692 up:boot
2019-04-30 00:22:29.425883 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6587 : cluster [DBG] fsmap cephfs-1/1/1 up {0=cu-pve06=up:reconnect}, 1 up:standby
2019-04-30 00:22:30.435723 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6588 : cluster [DBG] mds.0 192.168.7.206:6800/3970858648 up:rejoin
2019-04-30 00:22:30.435868 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6589 : cluster [DBG] fsmap cephfs-1/1/1 up {0=cu-pve06=up:rejoin}, 1 up:standby
2019-04-30 00:22:30.449165 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6590 : cluster [INF] daemon mds.cu-pve06 is now active in filesystem cephfs as rank 0
2019-04-30 00:22:30.015869 mds.cu-pve06 mds.0 192.168.7.206:6800/3970858648 1 : cluster [DBG] reconnect by client.54450 192.168.7.205:0/1578906464 after 0
2019-04-30 00:22:30.019932 mds.cu-pve06 mds.0 192.168.7.206:6800/3970858648 2 : cluster [DBG] reconnect by client.64366 192.168.7.206:0/2722278656 after 0.00400001
2019-04-30 00:22:30.054313 mds.cu-pve06 mds.0 192.168.7.206:6800/3970858648 3 : cluster [DBG] reconnect by client.54120 192.168.7.204:0/254060409 after 0.0400001
2019-04-30 00:22:31.434592 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6591 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded)
2019-04-30 00:22:31.446526 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6592 : cluster [DBG] mds.0 192.168.7.206:6800/3970858648 up:active
2019-04-30 00:22:31.446675 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6593 : cluster [DBG] fsmap cephfs-1/1/1 up {0=cu-pve06=up:active}, 1 up:standby
2019-04-30 00:22:43.355044 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6595 : cluster [INF] Manager daemon cu-pve05 is unresponsive. No standby daemons available.
2019-04-30 00:22:43.355235 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6596 : cluster [WRN] Health check failed: no active mgr (MGR_DOWN)
2019-04-30 00:22:43.367182 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6597 : cluster [DBG] mgrmap e18: no daemons active
2019-04-30 00:22:53.658070 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6601 : cluster [INF] Activating manager daemon cu-pve05
2019-04-30 00:22:53.898363 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6602 : cluster [INF] Health check cleared: MGR_DOWN (was: no active mgr)
2019-04-30 00:22:53.917204 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6603 : cluster [DBG] mgrmap e19: cu-pve05(active, starting)
2019-04-30 00:22:53.979682 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6608 : cluster [INF] Manager daemon cu-pve05 is now available
2019-04-30 00:22:54.928868 mon.cu-pve04 mon.0 192.168.7.204:6789/0 6609 : cluster [DBG] mgrmap e20: cu-pve05(active)
2019-04-30 00:22:59.965578 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 1 : cluster [DBG] pgmap v2: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:23:00.677664 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 2 : cluster [DBG] pgmap v3: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:23:02.700917 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 3 : cluster [DBG] pgmap v4: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:23:04.707492 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 4 : cluster [DBG] pgmap v5: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:23:06.740218 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 5 : cluster [DBG] pgmap v6: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:23:08.746633 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 6 : cluster [DBG] pgmap v7: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:23:10.780395 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 7 : cluster [DBG] pgmap v8: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail

2019-04-30 00:32:18.562962 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 278 : cluster [DBG] pgmap v279: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:32:18.465670 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7327 : cluster [INF] osd.16 marked down after no beacon for 901.455814 seconds
2019-04-30 00:32:18.468437 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7328 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)
2019-04-30 00:32:18.483797 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7329 : cluster [DBG] osdmap e196: 24 total, 23 up, 24 in
2019-04-30 00:32:19.495106 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7331 : cluster [DBG] osdmap e197: 24 total, 23 up, 24 in
2019-04-30 00:32:21.501683 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7334 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive, 47 pgs peering (PG_AVAILABILITY)
2019-04-30 00:32:21.501774 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7335 : cluster [WRN] Health check failed: Degraded data redundancy: 794/38643 objects degraded (2.055%), 50 pgs degraded (PG_DEGRADED)
2019-04-30 00:32:20.596358 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 279 : cluster [DBG] pgmap v280: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:32:22.603039 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 280 : cluster [DBG] pgmap v281: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:32:24.628896 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 281 : cluster [DBG] pgmap v283: 1152 pgs: 41 stale+active+clean, 1111 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 00:32:26.642893 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 282 : cluster [DBG] pgmap v285: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:28.669528 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 283 : cluster [DBG] pgmap v286: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:30.683129 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 284 : cluster [DBG] pgmap v287: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:32.709629 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 285 : cluster [DBG] pgmap v288: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:34.717180 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 286 : cluster [DBG] pgmap v289: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:36.748749 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 287 : cluster [DBG] pgmap v290: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:38.756345 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 288 : cluster [DBG] pgmap v291: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:40.789378 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 289 : cluster [DBG] pgmap v292: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:42.796488 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 290 : cluster [DBG] pgmap v293: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:44.821576 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 291 : cluster [DBG] pgmap v294: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:46.835641 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 292 : cluster [DBG] pgmap v295: 1152 pgs: 25 active+undersized, 1030 active+clean, 47 peering, 50 active+undersized+degraded; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail; 794/38643 objects degraded (2.055%)
2019-04-30 00:32:48.475079 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7371 : cluster [INF] osd.17 marked down after no beacon for 903.631937 seconds
2019-04-30 00:32:48.475189 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7372 : cluster [INF] osd.20 marked down after no beacon for 901.611316 seconds
2019-04-30 00:32:48.483726 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7373 : cluster [WRN] Health check update: 3 osds down (OSD_DOWN)
2019-04-30 00:32:48.500282 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7374 : cluster [DBG] osdmap e198: 24 total, 21 up, 24 in
2019-04-30 00:32:49.510909 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7376 : cluster [DBG] osdmap e199: 24 total, 21 up, 24 in

2019-04-30 00:35:58.536182 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7645 : cluster [INF] osd.7 marked down after no beacon for 902.595536 seconds
2019-04-30 00:35:58.538784 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7646 : cluster [WRN] Health check update: 5 osds down (OSD_DOWN)
2019-04-30 00:35:58.554495 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7647 : cluster [DBG] osdmap e202: 24 total, 19 up, 24 in
2019-04-30 00:35:59.565253 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7649 : cluster [DBG] osdmap e203: 24 total, 19 up, 24 in
2019-04-30 00:36:01.657260 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7652 : cluster [WRN] Health check update: Reduced data availability: 202 pgs inactive, 206 pgs peering (PG_AVAILABILITY)
2019-04-30 00:36:01.657353 mon.cu-pve04 mon.0 192.168.7.204:6789/0 7653 : cluster [WRN] Health check update: Degraded data redundancy: 4903/38643 objects degraded (12.688%), 247 pgs degraded, 285 pgs undersized (PG_DEGRADED)

--------------------------------------

2019-04-30 05:31:46.580027 mon.cu-pve04 mon.0 192.168.7.204:6789/0 11871 : cluster [INF] Standby daemon mds.cu-pve05 is not responding, dropping it
2019-04-30 05:31:46.591494 mon.cu-pve04 mon.0 192.168.7.204:6789/0 11872 : cluster [DBG] fsmap cephfs-1/1/1 up {0=cu-pve06=up:active}, 1 up:standby
2019-04-30 05:31:50.842218 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 9143 : cluster [DBG] pgmap v9201: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 05:31:52.872419 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 9144 : cluster [DBG] pgmap v9202: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 05:31:54.899490 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 9145 : cluster [DBG] pgmap v9203: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 05:31:56.925830 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 9146 : cluster [DBG] pgmap v9204: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 05:31:58.957234 mgr.cu-pve05 client.84813 192.168.7.205:0/2429672320 9147 : cluster [DBG] pgmap v9205: 1152 pgs: 1152 active+clean; 50.1GiB data, 175GiB used, 52.2TiB / 52.4TiB avail
2019-04-30 05:32:01.596600 mon.cu-pve04 mon.0 192.168.7.204:6789/0 11890 : cluster [DBG] mgrmap e22: cu-pve05(active)

2019-04-30 05:43:16.717729 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12763 : cluster [INF] osd.18 marked down after no beacon for 902.818940 seconds
2019-04-30 05:43:16.717846 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12764 : cluster [INF] osd.19 marked down after no beacon for 902.818731 seconds
2019-04-30 05:43:16.717914 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12765 : cluster [INF] osd.23 marked down after no beacon for 900.786850 seconds
2019-04-30 05:43:16.726253 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12766 : cluster [WRN] Health check failed: 3 osds down (OSD_DOWN)
2019-04-30 05:43:16.742278 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12767 : cluster [DBG] osdmap e253: 24 total, 21 up, 24 in
2019-04-30 05:43:17.753181 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12771 : cluster [DBG] osdmap e254: 24 total, 21 up, 24 in
2019-04-30 05:43:19.209031 mon.cu-pve04 mon.0 192.168.7.204:6789/0 12774 : cluster [WRN] Health check failed: Reduced data availability: 51 pgs inactive, 293 pgs peering (PG_AVAILABILITY)

-----------------------------------------
2019-04-30 08:56:22.240506 mon.cu-pve04 mon.0 192.168.7.204:6789/0 19905 : cluster [DBG] Standby manager daemon cu-pve04 started
2019-04-30 05:43:17.030698 osd.18 osd.18 192.168.7.204:6811/5641 3 : cluster [WRN] Monitor daemon marked osd.18 down, but it is still running
2019-04-30 05:43:17.030714 osd.18 osd.18 192.168.7.204:6811/5641 4 : cluster [DBG] map e253 wrongly marked me down at e253
2019-04-30 05:43:18.450669 osd.19 osd.19 192.168.7.204:6807/5309 3 : cluster [WRN] Monitor daemon marked osd.19 down, but it is still running
2019-04-30 05:43:18.450689 osd.19 osd.19 192.168.7.204:6807/5309 4 : cluster [DBG] map e254 wrongly marked me down at e253
2019-04-30 05:43:18.652645 osd.23 osd.23 192.168.7.204:6801/4516 3 : cluster [WRN] Monitor daemon marked osd.23 down, but it is still running

2019-04-30 05:44:07.065692 osd.20 osd.20 192.168.7.204:6809/5441 4 : cluster [DBG] map e263 wrongly marked me down at e263
2019-04-30 08:56:22.458718 mon.cu-pve04 mon.0 192.168.7.204:6789/0 19906 : cluster [INF] daemon mds.cu-pve05 restarted
2019-04-30 08:56:26.088398 mon.cu-pve04 mon.0 192.168.7.204:6789/0 19910 : cluster [DBG] Standby manager daemon cu-pve06 started
2019-04-30 08:56:26.495852 mon.cu-pve04 mon.0 192.168.7.204:6789/0 19911 : cluster [DBG] mgrmap e23: cu-pve05(active), standbys: cu-pve04

ceph-报错日志的更多相关文章

[小菜随笔]关于monkey报错日志分析
今天小菜在一个测试群内看到群友发出一个monkey的报错信息,其实是一个很简单的报错个人觉得monkey虽然操作起来比较简易,但其实查看日志分析日志也是很重要的环节,如果对错误分析不够详细,就容易误 ...
linux系统报错日志学习
linux本身会自动记录系统报错日志:/var/log/messages 这个日志记录,我是在什么时候发现其强大的作用的呢?它有点像我们使用php脚本开发接口的时候技术员在重要地方打日志的效果,方便技 ...
Linux常用命令&定位生产报错日志
1. cd / 到根目录下 2. cd .. 返回上层目录 3.ls 显示当前目录有哪些文件 4. pwd 显示当前目录 5. ps -ef|grep tomcat7 查看当前运行进程 6. kill ...
php报错日志：PHP Deprecated:Automatically populating $HTTP_RAW_POST_DATA is deprecated
前几天将线上php服务升级到5.6.x版本后,php-error.log报出错误:PHP Deprecated: Automatically populating $HTTP_RAW_POST_DAT ...
k8s报错日志查看
看系统日志 cat /var/log/messages 用kubectl 查看日志 # 注意:使用Kubelet describe 查看日志,一定要带上命名空间,否则会报如下错误[root@node ...
nginx报错日志：see security.limit_extensions
访问出现部分.js.css等部分文件被拒绝错误日志如下: 19:20:13 [error] 1181#0: *287 FastCGI sent in stderr: "Access to t ...
gulp压缩js文件报错日志
输出 gulp-uglify 压缩js文件时报错信息 gulp.task('es6', function () { return gulp.src('src/main/webapp/bower_com ...
ceph报错
[ceph_deploy.mon][ERROR ] RuntimeError: config file /etc/ceph/ceph.conf exists with different conten ...
opennebula kvm 创建VM oned报错日志
Thu Jul :: [ReM][D]: Req: UID: VirtualMachineDeploy result SUCCESS, Thu Jul :: [TM][D]: Message rece ...
tomcat启动不起来，不知原因，没有报错日志，控制台一闪而过怎么办
在startup.bat文件中编辑,在最后一行回车加上一个词pause,暂停,然后再启动就看见控制台的错误信息啦,然后就自己解决吧

随机推荐

wsl和windows相互访问文件夹
How to access Windows folders from Bash on Ubuntu on Windows You'll find the Windows C:\ structure a ...
使用DataV制作实时销售数据可视化大屏（实验篇）
课时1:背景介绍任务说明 ABC是一家销售公司,其客户可以通过网站下单订购该公司经营范围内的商品,并使用信用卡.银行卡.转账等方式付费.付费成功后,ABC公司会根据客户地址依据就近原则选择自己的货仓 ...
sdb报告-10 错误问题定位
# sdb报告-10 错误问题定位在sdb 的集群环境中,如果面对的是一个高并发的操作场景,有时候会莫名其妙地报告 -10 错误. 在 sdb 的错误列表中,-10 错误代表:系统错误. 这是一个笼统 ...
python 中for与else搭配使用
先看一段程序: for i in range(10): if i == 5: print( 'found it! i = %s' % i) break else: print('not found i ...
ESET激活码，可用。
ESET Internet Security 12.1.31.0 Finalhttps://download.eset.com/com/eset/apps/home/eis/windows/v12/1 ...
vue 中router.go；router.push和router.replace的区别
vue 中router.go:router.push和router.replace的区别:https://blog.csdn.net/div_ma/article/details/79467165 t ...
Oracle学习笔记<3>
单值函数 1.函数的分类 Oracle数据库中函数分为两类: 1)单值函数 n条数据经过函数处理得到n条结果例如:查询所有员工last_name,并以全部大写形式输出 2)多值函数(组函数) n条数 ...
45-python基础-python3-字符串-常用字符串方法(三)-startswith()-endswith()
4-字符串方法 startswith()和 endswith() startswith()和 endswith()判断字符串是否以某个字符串开始或结尾,存在返回 True,否则,方法返回 False. ...
LeetCode Array Easy 219. Contains Duplicate II
---恢复内容开始--- Description Given an array of integers and an integer k, find out whether there are two ...
【记录】linux 命令拷贝文件到远程服务器，linux下载文件到本地
Linux scp命令用于Linux之间复制文件和目录 -1 强制scp命令使用协议ssh1 -2 强制scp命令使用协议ssh2 -4 强制scp命令只使用IPv4寻址 -6 强制scp命令只使用I ...

ceph-报错日志

ceph-报错日志的更多相关文章

随机推荐

热门专题