http://blog.csdn.net/liuyunfengheda/article/details/5260278  MFS总结

http://bbs.chinaunix.net/thread-1643863-1-1.html

http://bbs.linuxtone.org/thread-9190-1-1.html
http://tech.it168.com/a2012/0615/1360/000001360707_all.shtml
http://www.freeoa.net/osuport/storagebak/moosefs-use-summary_1623.html

chomd 755  /root/keepalived_notify.sh

可以用同网段的另一台服务器 arping 1 这个虚拟IP
看看对应该的MAC地址 可以知道虚拟IP对应的真实服务器

配置文件 不正确时 有时候日志会有以下提示
VRRP_Instance(VI_1) Now in FAULT state

advert_int 两台keepalived 这个参数 不一致时 日志会有以下提示
Keepalived_vrrp: advertissement interval mismatch mine=5000000 rcved=1

Metadata文件的大小取决于文件数(而不是他们的大小),Changelog的大小取决于每小时的操作次数。

endangered    有危险的,濒危
over goal    超出目标,达到目标
under goal    没达到目标
pending deletion    正在删除

bottleneck  瓶颈

[root@mfs-client mfs]# rpm -ql moosefs-ce-client
/etc/mfs/mfsmount.cfg.dist
/usr/bin/mfsappendchunks
/usr/bin/mfscheckfile
/usr/bin/mfsdeleattr
/usr/bin/mfsdelquota
/usr/bin/mfsdirinfo
/usr/bin/mfsfileinfo
/usr/bin/mfsfilepaths
/usr/bin/mfsfilerepair
/usr/bin/mfsgeteattr
/usr/bin/mfsgetgoal
/usr/bin/mfsgetquota
/usr/bin/mfsgettrashtime
/usr/bin/mfsmakesnapshot
/usr/bin/mfsmount
/usr/bin/mfsrgetgoal
/usr/bin/mfsrgettrashtime
/usr/bin/mfsrsetgoal
/usr/bin/mfsrsettrashtime
/usr/bin/mfsseteattr
/usr/bin/mfssetgoal
/usr/bin/mfssetquota
/usr/bin/mfssettrashtime
/usr/bin/mfssnapshot
/usr/bin/mfstools

测试1,一个读,一个写,非同时
[root@-shiyan mfs1]# time dd if=500M of=/dev/null
+ records in
+ records out
bytes ( MB) copied, 17.3807 s, 30.2 MB/s real 0m17.404s
user 0m0.094s
sys 0m0.474s [root@mfs-client mfs]# time dd if=/dev/zero of=500M bs=1024k count=
+ records in
+ records out
bytes ( MB) copied, 7.57029 s, 69.3 MB/s real 0m7.579s
user 0m0.002s
sys 0m1.440s 测试2,停掉一台chunkserver的网络,会看到/mnt/mfs容量减半,恢复网络后容量又回来了
[root@mfs-client mfs]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 15G 781M 13G % /
tmpfs 246M 246M % /dev/shm
/dev/sda1 485M 32M 428M % /boot
192.168.2.231: 100G 256K 100G % /mnt/mfs
[root@mfs-client mfs]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 15G 781M 13G % /
tmpfs 246M 246M % /dev/shm
/dev/sda1 485M 32M 428M % /boot
192.168.2.231: 199G 256K 199G % /mnt/mfs 测试3,停掉一台chunkserver的网络后,在客户端检查发现2 copies变为1,即chunkserver的数量
[root@mfs-chunk1 sh]# service network stop
[root@mfs-client mfs]# mfscheckfile rsyslog.conf
rsyslog.conf:
chunks with copies:
[root@mfs-client mfs]# mfscheckfile rsyslog.conf
rsyslog.conf:
chunks with copy: [root@mfs-client mfs]# mfscheckfile rsyslog.conf
rsyslog.conf:
chunks with copies:
[root@mfs-client mfs]# mfsfileinfo rsyslog.conf
rsyslog.conf:
chunk : 00000000000000E4_00000001 / (id: ver:)
copy : 192.168.2.232:
copy : 192.168.2.233:
[root@mfs-client mfs]# mfssetgoal rsyslog.conf
rsyslog.conf:
测试4,master的测试,当只有一个master时,停掉服务
[root@mfs-master ~]# service mfsmaster stop
然后监控如下图:
aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAABOMAAAF4CAIAAACZ46qtAAAgAElEQVR4nOzdPWjb3N/GcY3/4Rky3IOHDA10aKDDHcjQQJfbkKGCDBVkqKFDMR2K6VBMhyKyBJGhiAxFZCgmQ0EZCu5QUIaCuwSUIaAMBWUoKEMHDRk0ZNCQ4TyD/CLrzbLjxG7y/fGhOIp0dHRkO76qF0uCuscVhkJ+26rtOLVdt7br1T55dQAAAAC3aG1TM777N93U0pOa/SvMtPSkNpVeLT2pRQqmlN8QadZZiZpxGQe2/N6qfXRru27tk0deBQAAAG7TrSXVzq8w01SSaj8J9xuM1mjnr6J4Q0iq9718P6i+btV2nG5Y3fVqu17to6ts2cpW57q27dpHt/bJi0RPu/iDxOP0nOnH6X/rWS3X82dIR/Faqv34zPSf/tN/+k//6T/9p//0n/7fXP/7mfD6/V/b1PRevIymK9tO/8BmWj+pptcV9Wqs8Y+H4cTjcce/m1TDyzAy69BEzabqW6ay1YmHVWWrYx4Hnd/iWs5C8yhofHKUbbsbgKMn3+6w9JTE9E+piZ9SP2ZOL1hR/IWRnufT8Gy7OT/Sf/pP/+k//af/9J/+03/6f+3+R/FyKv3vNhWbofhQajypJlob6lXp8U+vbulJbYLxX9vUpPAy9H579pHt+z6R9X5W59itvjGVKKl+dGsfXeVDp30Stk+vy/olOmeiuecoO0720xEAAAC435KZcKpNLT1R8pOqkhsmr9GrxNHUiTdECi9D98z93//9r/ay5v32Oj86QRDMOjpdq8LTdsefdSd82zr9awJ/GIrqK0PZsvthVdm2ta9e+1Rcn/VLWKehstWpfZz9uwAAAAAwb1Y2Nf1bRiZceqIUKNlULI4Olk0l1YzW8no1Ujwb5/WzzJhI0UFU9YO6/Gi5+a5ZWax0fnQSSUaTJOm1FQghRGC9liRJa+1I8Wr8DIUInT1l6Z9ogmxmZ0Xf3Ogts5EzS5Sdzs3awwVJqlT33fFS17llHodC2Ok+2+M1lFt2uW0Pf7ba51Na5QR1Jdpf2/EJnZ/J3Rovfb8jv2sr246y49Y+usqOW9vuWL+mcFg1OrLa/Owo2xxWBQAAAJIyM2HBsdCx4mVmLr25pJru9mRhdWVTk9aerjXeNIw9o7JY+d///c85dTLO/j3WFhY1RwgRdpr/LGjHQghb60fNwGkd2OGpXt/3QiGECP1DrXWan4qOm83j4pjlt182rEmO7Prm+5aX2+dpVclt91rvi9L4jVb7a1t+Jusf9ehH/aNeXa8WhFXvPFiLDqtuO7UdV9lx5Q+d1g9/Kkm1fRqaPwP5Q6f20QMAAAAQt/Jc07/5iYlLq/HIFwzHv6DzK1xaVco01W9naVXpL9j5FXR+BfFflexVsX7LvbWEBe2PHBMpUcuPlu2j1NHHY7Pxtm6eC3Fqam/l4aTaaR34QghxqjcPy53veqyNyo2+uTNRxjtvNQ56y2X0eVpVdtv9g0brfIrrHa/ML6byXNF3dX1XV54r5hezeP7a+5b83lK2HWXHUXZcZctufHLap2H7ZAqsM1Hb7ig77szfCAAAAIC5kp9UgwLjJNXu/H2JKcqWXbJXBeIdjvqWnjLWmCSTamWx0nzfTIaYY9M8aDZ+ht5+q33QS6q9ReRuOAzdg2bttWGdjToYGkuq9o4kbTT1t2sL0sLSWysQQhzH+tQ7dOkdNqqLkrRYbRx6BQ2HPxuDRJrR54x20lP8aMo/S2uv290NOzXkRUn6Z603T+ltP9YaP2d5tap5YCqbirKpmAcjYqoQwvrpVl+3osOqyo6rbLvyW6t9EprT0D4Nta+e/MHu37Sp9yx0h6fEp8cfFC+Vni3R4FjT048z0X/6T//pP/2n//Sf/tN/+j+F/scy4dBv49kyLbP/6aa6cfG0F3FPB497jWT3s9fUGOM/3LehTRh3/AdJ9X//+5++qzfeNtwzN+vsX9M8bdd22uZ+xzvIOabarcA91GpvraIjokPHVG3tccPyQyG81kb/UtLhY6rnLaXboG+9VQqOUiaTaqLP6XYyWg6sHcMJhRBBZ8ewhRBhR9vpBEIIETp7mhWMs+2zTqqt/Vb9Vb3+qt7ab42cOQzDtRe6/KHTDavbrvy+ox24U0mq5knYPgnld5ay48bVdtxa70H334+DXympiUp6/uhBbIb4zImllI+DRZJNfUyt6+PwIh+H1kj/6T/9p//0n/7Tf/pP/+n/VPq/8lzTv/pT6X/UVHzi0qrSTacpS6tKQf+7vZrR+A+S6vLj5aLvpzk2Td9rbSzXD3w/mVQzyjuo6UXXqQ4n1W47vpmTVINvtVj+1Grf8o/ZDp/9m+hzup3slgPP2m/WX2vt6ADpeasaO+Y81rbP9uxffVdXNpX213b7a1vZVPRdfeQi2p5VfdOWP9jKlqNsO/KWrXyw2qfTSarWr7Cxa8tbdvplAAAAANxb1Vetxq4zlaYGobdH/mDnHZWVPxR9Mk83dXu2nLUXhrT0YKn6X7X+ql70NarHpnku7PfL+qnIS6rhz1brNMqQgfVWLkpoQ3dUiifVZi+peuZ7c3Ca75khd49qBp0d2TgriFpea6cd5PU53U5Gy7a550RHUN3oCGpgNd53hsNxyW2f5R2VootU+yf9mgem8lyxvlvFS7m//bUXhvy+o2w5UVitvm0bh/60TgA2fvjpw6oAAADAfSa/t3vfGXndpqYYL2eWVLectReG/MGWlOeKd+7Jz+SCpKpJkrRhOt9M+9yUpcG31Mjxc1+7v5Kkf5aUPSenreS31Ni9dvwDWZIkaccWfq+dWPslr1ONutHqf0vNcJ/tUtepeu2XayuLkiRVqr3zeINjXYm+Muet5cX6XLzts/2WmvAytA6Hcql9bIur0Qsqb43qW0v+YMtbjrLlyO87jV17WlerWr9C+b2lbHWvgx08HbeH/91OPd4efvqmZyizeHrOxDwjl433hP7Tf/pP/+k//af/9J/+0/9p9f+DvbaprTzXVja0lQ01Rcv5URt+nLeIlvo3sWxmC3l9SM+WXte1+i+/t+u7nmQf2eFlaOwZzknW99MIMYg3V2Io6lxlTSmoyeYsOUNsSnDc7vzJmqFgqalvmt9pnxYOZpmmMqdkzjC9TTMPnbWXLfl9Jwqr8pZTfW2ax8G0Dquq+678rpP9+gQAAADuq9qOW9/1mp/h1Xe92o4rRZenBheB+kENgiAZaRLxJi09Q3p+kZqefpyOUok5836bmcGKe8Km5W9acBmubGrVdx35vS1/cOQtp/rWqu90pnVY1TwOqq/bM38jAAAAADDPpCAIorDa+dHxfT8jHeFeEULdba+9NruHVaOw+sZSPlj6N888vm5SNX743aS65Spb7uBB/8dtd+i328O/LbNIYtmxpmylGs+cQv/pP/2n//Sf/tN/+k//6T/9v8n+SyKvZh6ZMCPOL29lUx8cVo3C6rvO2mtz7WXrul6b8gdH3nKvZft6i8981fSf/s+8EfpP/+k//af/9J/+0//57r8081yEOSS/0quv272w2sur7+2u6PGH2OP4rz448ntHft9/0Jv4ofd4Vi8SAAAAAH8JkioymN+dled69W2n+m44gn5wotssxZ5DjrzldvPnh/SvnOyJ0ePs1OokZ8udkresO9SZZAdc+k//6T/9p//0n/7Tf/pP/+n/nPefpIoM4ZWov2+tbGhrL8211+3qG2vgbQcAAAAAbhRJFdnCULQPHeWtsbIJAAAAALdKEldheBWGVyK8EuIqFFcivArFlRChEFdhVFFu6WWYaP5QhGEYivBKiHDw2+6yUTthKMJQhN2J4ZUIB4/D8CoUV2EYChEKEYpuB/rrihrs9qG7bNSa6K20v6L+qsN+y2F3c9i0a26auLubdof3GpvGprFpbBqbxqaxaWwam8am/e2bJoVR06nKmJSuq9GzxBsUYX/Iilco8n6RN01kzsWmZRab1muQTWPThn9m0/IazFy2bLFp6ZWzaTndY9Myly1bbFp65WxaTvfYtMxlyxabll75DW2alNNPiqIoiqIoiqIoippNkVQpiqIoiqIoiqKo+ao7mFSdE6f9rW0emAAAAACAG9L+1nZOnBuKdXctqTqnTudnx//jUxRFURRFURRFUTdYf/zOz477y72JZHfXkqp9ZPu+L64ERVEURVEURVEUdYN1JXzft4/sm0h2UngZDoRh8sdw+EHiV2HqV4nZCqaMbDmzP5k/xua3j+wgCGa9xyiKoiiKoiiKou5+BUHQTarl8lp2QsxaVsqY6e8VhvYxSZWiKIqiKIqiKOo2aiipThVJlaIoiqIoiqIoihqq9re2tq2ZX8ziKyuDILCP70FSdU4dbUervawpz5Xay5q2ozmnTv+3+kd9wqR6JcwvZlp4Gd7crqUoiqIoiqIoivoby//jqx/U4CIw9gz3l1sw5/wmVfvY7t6b+Hrt+L7ffNdce7JWXa/KG7LyXJE35Op6de3JWjRG+q6+8u/KZEk1DEPzi5mYYuwZhNW7X78s+2LWfbiTdek4v2fdB2q2deV3jrxZd+IaddP9/9vHh6Ioivrbqv2tbewZCe1v7cla6/zstD63ogftr0WNlEmqxp4hb8jFzANzyknVOrScY9s6tOxje+JGgotA2VSq69Xai1rtZa3+ut5816y/rtde1movalF2ra5X5WdymaSaDp/hZS+pXnVFSVUIUTKshr7VOvSHJznGy5WKJEmL1caBG2sisHeVpX8kSapU35puuSCc0b4Q4ZnZWF9akKSFpw3rfLDe1utqRZKkf5aqb9slPgeN25+c+fvrXVyp7TmJZjL7n12hZ+1p2paqbuvmkZ/RnT+W/smO/WdD4H4ztC1V3dL0L/ZggTNT3VIHvjhZK/M735Ndza4Lx/yoqVuq9tF0bjHZBkeGttsZGrgr3znobe9ny7u8vc6Ur+DIULdU82x2PbhwrJPh/5DKfV7lPH/GbV+I8I9t7mrqlqp+bA2eJKFnfdYH7Y++6/i4/cmZv7/ebc347iWbyRyfT5q6pbWOk9sV/GobO9GT3zA+W/6o6d3fHhnqlqoeFP0P6xSru7ot1TiazsUdyf5njI9rbqnqlun09+kfS99S1d6701yNz7Qqd5zznz+306vk++QtrDRz/w5e76oaezJQFEXNSaVjamTi1uwjWwgRXATatlZwAnB08LI4qUYnzBZ5Uau/rqeTahBchsFlEF4G4WUYBtEvgiDozxR0HwRBcBkEvdmiGdwz1zlx3F9u50fHPnbCyzCIfttfRxD9GARBEHTbjx4E/UbUD6r8TK69rDXfNe0jO8qc4WXY+dlpvmvWX9VrL2vRv/2edHsYtdnvYX5Sbe1nJ1UhRGu/MKz6pixJkiTJB/E/lH57c6URZbPQMzcrjZ9RC6GzU20cRp8nfev9yvJuZoIa2b4Qflt52rT8UAhh7yys7EWfeALrdUU5iD6YBp2dleUdu/Cz7rj9yZs/sF5XjdNotW5rs1I/7P2Bzut/dgX2nmZGn6Gv/M6eZp6GyRk+a/E//95XvTd/4HzRtG+9bH5mjz5Y+qs9SPhF5Vs7RvQ5yPumTeujcJkKT1raXvyzTujsG9Z5d0Jwamq7yU/Ac1LuwYyS6oVtZHySzn1e5T5/xmtfiAvb6P0vhnugat+j3RI6nzXjhy+EEFehe6Bpo5LJuP3JmT909vXuc/vSs/a01knvdZTT/+CoZf4KU/8NJMIzU9vp/e/MhdveNaJf500fqlPzlpNYcGRM8+UZ63/O+LjmltY+H6xd3TKjBeZzfKZV6XHOe/7cTqXeJ29+jdn7N7D3DOs8FEKIq9A/NbWdMv9ZTFEUdXsV5VL72PZ+e+6Zax/bEyfV8DJUt9T+mar6ru6e5f5R6ybVKC32Y2Asr4WXoXlgNt81G28bBaxDqx8/o5AoBderRFiNTgMeqwXnxKmuV+uv6o23Df+PH188vAxb+63663rU+yipFncmP6m2hBhOqp+MvtZ+q/hCYe9AHkpivilvmIOfz1tr3R8d7Z+mPZg+PFv59kXYeV/RjtMz2pqkDdoPrPqiXhg9x+1P3vx+++WSftqd7Cd7m+5/Tl3YRvyDTupzT3BsuadmLKl67d5HQyGEuOj05w+OOqM+A/qdryUPqA73aoYV2K2v8U8+QWfXuImzl90f193e6SbVcfsT/NCHPknnPq9ynz/jtS+E+0XL2l7XjLd/6bS2iz+5jtufvPkDe09r986+zsgV6f4fGPaFEBe2MfSJ32vvDNoRQogL2z4rmD5cZ2WT2PWfb1FNOanG+p8zPq71Udd/dCc4PwyjuzvmdHymVelxzhmfKdfcjEPO/r2wjeEd6h9af+V/RVAUdXcryqXmF7Pzo2Mf2+aBWT6pdn509I+6tqP1mQeDayedE0f/qOu7ur6razu6tqPFr6zsR7CimFamUi2MmVQvg8EB0iDsTonC6qnjnDrdsBqMEVb1j7qyqdRf1Ts/Oon+GZ8M5bnSeNMYJNXClkcm1f44BEHgx6q13wrDolCTzGbHmrQzCHQi7DS6AdI1Vhcah72mfFN+bZX505vKfrb2T6OT0SOvtV7VTwftK++z5hrUuP0ZPX8YOK23euJIbjq7CiGECOw9VduPxcXER7crJ/FB3zoJxVk8qfrWjto6jR0v+txtLTiyR3xEKHtAdZ6S6qXT2o5/Qgo6uy1n6icA+53rf9ycZlJN9ad/CmJc/KNz8pN07vMq9/lTXKlP6q4ZPws01nVrV2+fx9r/UvysHLc/JeYPPOtLO3FWf1Z2NdrnQvwyB0dfhRDn7eyDQnnTE1UyiZV+vo29369Zsf5nj49w7QNT715Z4FmHVvd5Na/jM60q+/yZbk3jfWmsyh3PvP176bQ+debiLwVFUVRO9U/3dX+5wUVQnG7iFQSBulV0fm+6tG3NO+++WZZJqlFWC4Nelhscbg26R2KzUt4YSdU5cdrf2mnWoWUf2e4vN8qrncOOc+qUb7bxtlF/Va+/qiemtz635A05OtYaqb9OzpOooqT6eSipxksI0frcKr5aNZnETvXll+3BX6wzY02S0wcr/QOl+bPUUyTZvm/KktY5bzfWK5K0sLSuDf48nrdriyu1vY4XuNaOUXzu78T9yZnf1iRJWq2bZ8lGyibV87a2F+vyH0vf6h8zDN0jJxRiOKkON3dkmL/6jzv2qalv510fGNjf+usN7E+qumu29zR1S28f28a2qn60PDE4VTLrY1/s+sBPRjt5lnK6Qv8o6o+qfbIGFw6Gvv2ldz3hgWV91ozoQq/edbYFHzSDU7P13Svqf/ziyS1N/+JkX9876H906d1ALG32+z+4Prb/Sc48E8GvtrGtqtvd5OweqOZpf9NasQO/g/Ua39xr9Keokp+ki55XQ0v1nz/jtX9hG1um69vdi5l3zcFAX9jGtmZ8d/1L3/lqlbwufdz+5MzvmluqutPqnKfSazrRnZnavmWfDF0YGxwZmRd4502Pftndj9uG9a3VT2I5z8Pc/Zv/vC2qrKSa/Xwb/boY7n/m+Ajh2kd2Owotfyzrd/cQ+ryNT8HrtOB1bZ5Gl56q2m7y4vySz5+h7U2+T2asd3h6/H0y/30g930yY7/3tqv3vhS/nnyc9/OC/RuctPSPLevUD8f5MEdRFHVr1T/7t/OzI67GuMFSEATqluqclj1TxPvtqVuq7/eu4i+RVI1Phrwhy8/ybcjmgZlYaoykGt02yTlx0qKYGnFOnM6PTvlmG28a9df1dArtXq06XMVN9S/nTYzmIKlmXeArJkiqwtEfr2g/AyGE8K3ma1lOJ9Xztpa681DZ9n1Tlpar77sfiL0DZWEQjD1r12jt1db+WVD2x/lgPE5/CuYPA8d8XdWG/9LnJNV0ee0dzfwVXe/qmPtGP1GEZ3b3U0VeUr2wzdidY7xvunkazRW6B6nr/c6soQOqF7axZTqXIvihq1+cMHFWbeYx1VOz1ftsFJ5b5qj7RQVHhrbX/UDmfR+ce+l/71/H6Hc+6daf4aVSZ2n2f+N8bdvxg3XZ/fetj61e10NnP/YJr6D/WdsbHLX718eKC7sVC37ugdo6aJuHXigC+5MWbYJ7MLj7lP9d74+/91XXv7nhlRBXoXdo6P2TmcfsT3GlPknnPq/iazHTdx4q2f6FbWxp+oEd7V//h6H2x+fKd75Z1veWvqUah2Vv2DRuf4rmD7zOZ709fB/m1PiE3pFpbJvuVT/V9BJX1kG/vOlCCO+bpkU3kLsK3QOtN1v+81Bk7t/C+fMrnaBynm+57ef0P3t8hHDtI9/5YtiBCE8s5zKWVOdvfDJfp3mv6/jrN/yVvB6+5PNHiNzXdd56894nc8ah15/U+2Te+4x7oKpZ70tjvZ8X7F8hhLj07ANDy7sjIEVR1EwrcRelsW6w5J652o7W/toeeSS286OjbWvOyeA/9fq3CiqIaSXvqJRYaoyk6p65zkl2UnVOnG5S/eXax7Z75pZvtvmu2XjTUDaV4MIvv1RmFSRVY68oqRp74ybV6Ma8FUlaWNo0nDNTjl8+KoQ4b+sHY8TIrKQaP/vXaz2tW4EQwm+/bHanB25rs6IclLuhw5j9GTF/0K7FDymPkVRF+Kd3YOqT5f2xu9d9hW6nf0vJzKR6Ybd/5H8suHRaQydrBfa34YPNvc9AvU9ggf1pVFI9bxtfbK/sNaJee8ews07THU6qWjKp5p3NeNoZaq2w/8GZbX0x9B1t6FhEQf+zPxknzoIbnJXtHqh66oPd0Nm//bMcL53Wx/hHXt/62Du2M15/RlR63LKfV7FVFD1/RrYf/U/B4ChKf7sCe8/s/nfOpWftlb4d15j9GfX8t429oSd8ov/hSat9JrxvmnEcCiH8w+4B+fGTqteOj8Pw2a3Zz0ORu39z58+v5H4peL5lt5/d/7zxEcK1j4Lgh27+Eu6PTiBGJtVZjk/W6zT3dT189n7yeviSzx8h8l7XeevNfZ8sGId0fwr2u3ugDs47iI//OO/nI5Jqtw+eva+n3xgpiqJmW/GDqCJ+MvCZ6565I28FHIaheWDqu7r/J/v9Lbqg0vhkBBdBYsGRSdU8MItvpxTdUSmxVCypXsT+zcmBebo3Ver9m+zmRepBrIw9o/6qrmwq7W/tjEVG1sVg5vAydE6cnKRqCCF650APiXbkuEl1qI6b0mY7doOlMY9eZiTVtjIUfX1zQzZ9Ic5ba29jATa06v9oo24uPLWjqbEavrHTOEl1qH6Z6p4dCBH+tq3DTudHp/Oj0zkw1J1W+0fHiR1RbB8WH30avrHNWdtKfNXnBElVCBG49oGh7RjmkTcqfwx3IF5Xbns3+vYUrZU65paXVJNX4eb2P3S/6Pq+5f4JwqvUtaN5/c/Y3vz+51ySmp1U/eQ3DNmf9NjXnpTtz3WvV+w9r/rtj3r+JCvZfpCIvr3x/2Pp8bMEQ6e1VeLyxXH7M+7zP9n/0Pls2BdCXNrGTtsTwjnqvbrPTHU367q7vOnJF1o/CRQ+DzOeb9nzj73fc59vef3J7H/++AjXPgrE77Z+aFuH3mDxuRyfrNdp7us6kVSH3g/LP3+6c6df13nrLXqfGSOp5r/PZL8v5fQzdzxz9296c7j3L0VRc11RLm3tt6Kk2tpvlbnBknPqaDtadoOfDOu7lb6cNQzDEXfVvcg+WzbjjkrDGVDqJr1+TM00HAgTU5wTp/Oz4/5y7eNeTC1uZHii+8uNvoRGea4MDsb2f3vmKpuKsqn4f/zsRmIzl0iqGSWum1RDe2e59i3o/xQ/vuEfWqOTZPbZxcv9e+0K4eiPG51QiFNtIZ5Uhd1MHMvN6t14/cme3zVWF7R+f4J27Wkr/hd6sqTqHnT/k36oEsdUQ9f6Pji6G5w4nhBC+Fb83owXtjH4L/bUAVUxaVKN6ir0j03ja/EHkuFjKfFffOt9q0FWlb1DTF7//1h67DLg7LscpfuffUw1eWZyv8ZIqoXHuMbpz4gqHreh51X282fc9hP3AvXaO6ZzJcTv9vD1bIUfxCfrT+7zXx305zL+/E/3P7D3ul+14h/q2hfT6t8U59JpbelDS146zu/86Yl7Efe/haX4eZjev2Wet1lV9thabvuZ/c8fnyipXjqtbc08FYP9O5fjkzVb7ut6eGbf+jj0LTtlnz/xGnpd5603931SiOkdU81Lqhn9zKm8/evb9tD/gZZ4vVMURd1uJVJo97LVI9s797xzzz4q9aU1wUWQl1S1Hc37nfEWOkiqeaGvfA0vKGWkx/KCwD62rUPLOXUGJ/2O34i+a0RJNbqU1vf9IAh83ze/mPIzubperf5Xtb5bI5sqSqqfCpPqpxFJ1d1bW3g7dBdcZ2chymb+YXNltf9VMYG1F/9PVq+1E12+GlivFyr5991Nt+9/UxZWuzdS8g8bK917/Dr66kqze2gucPeVSnd6Xvvj9idvftdYXet+n2roW+/XlG9DuTTd/6i15B2VhPC+dg8CBHlfRjeUVEPne+yzy5XfObC7FwfvtOxoGK4C54s2OAvrt5U8oCoGX+8R/Ii+cyKwP+mdi9hvU/+D7n43nf4R0Au71b3nauh8Tm5Rd7u+adoXJ4g+hF169vfeTVAuOq0dXf+oaTu6vmfaf4aPRhwZ/e/AiJXf2e1/Y2dh/8/b2m47uugr/OOYu6r5y+/8cPP7H3XPaW2ZzqUQV6Fz0v2g5R/qau/6MRF41o/BB7ChE+oyJ/6KHTs60I1DL7wSQoTeodE/h3zc/hSX9z26Xjc2Jft5lff8yd2Pee0HR4a6Y7rRDWlOTK17j1+vvaP1r5f2Do3e9Lz2x+1PwfO/932qV4HzJXUV33D//UNd3ev4V0Jcee0d1Tj0QhG6vzwhhPdNG+z3C7fdbT93unvQO5v90rP21O5+z38eCpG1f4vnz6/gyBg+3zLn+Vbwusjqf/74uJ0fQSzIDZLJHI5P5us073XtHqjdTY6et5+Tz/b4OBc8f/Je13nrzX2fzByHeH+G3ifz37KaY20AACAASURBVGey35cK33+yKnv/XjnmTss+T7/eKYqi5qLSJ/dGP/p//NZ+q7Xf8v/4ZZJq50fH+m5l/+pnJ/7lNP1KJtWsuFfqjkpfzKkm1YvAOrSck2vE1B71g1p7WVM2lbWna2tP1qrr1bUna9X/qsqmUntZa75v+r5fMqlmHJLuJ9WspURxUvVNWRpU/ztOw1NDebggSZWVl8bgj+3wzJIk9b7MJrBeLyxkJtWc9oUInf16dVGSpEr1bSzRBU7rdbUiSdI/S8pO/3qhnPbH7U/u/EIEtr65tCBJ0mK1Eb+ENbf/IjOphr8tY1tVt7T+fTXiv/ROOu19Xd81zMOOe5Fxb97B/5EHbntPS93zM7S/p2+HHNifVHVLNQ5MY0tVtwzzwFC3VPWTHfR+1RX7H/3gqGXs6dqWOnxvzND5rKrZn3IC95uh9e9p2d00r30Qmzn0rN3eruzd0zLrzpaJpFrQ/9D73ruh5RfbPWppW91Dfzn975Z3aKTujRl6h63oXp3aZ6v7fyHDZ8f1D1bE7zWaujln9r1Yx+9PTg0/JQb3Ss18XuU+f/L3Y0778fEZutf0pWd91rvbe+D0pue0P25/ip//nzR1S1UTd3bJ7n/g7OvdZ+aFZ+1p6rbR+dPbrt49WrW92D2r86aHvt1v6tTs7ffc52FUqf07Yv50JZ6Kmffojr8P5Laf3f/M8enekNY8E+53y++/V3TfJeZofPJep92mUq9rEd1+6bvd+pi4B2/eOOc+f/Jf19nrzXmfzBuHgvfJ3Hv/Zr4vFb//ZFXm/g06u/3OaPoXm1sqURQ1b9W/PLX/Y/k7KvVL/zi4TtU+sqOvTo0OBIaXobatBRepQHMlRiZVeUMue0elKSbVoVsoXa8p45NRf1WPOqpsKrUXteisYGPPiLbwOklV3y3aK/ruiGOq1N9RfzoZB1RnWJdO6+PgU1HoO+Yu1zVRFDXjyjwAS1EURf3tlYij5b+lpl/eby86vBeGofnF1Ha0zs+O8cnQd/XoO2ms71bGEdcSSbXUHZVSp9BeN6kGQfTNrdeNqVHOdM/caFDUD6q2o5kHpvfbK9rs0klV29G07Xw7OkmVuomKfy+i9il59i9FUdQtV//YY9lbVVMURVF/VY08alpQnZ+d9re27/v6rt7ab0WXSYorYR1a2rbmnDrRPYSTi41MqhfdOwoVm/Z1qjchthnjBuDcpBqG/h/fO/f8P35aNH3kdwdRFEVRFEVRFEXd1YruH6R/1O2j5D1bvXPP2DP0j7pzmro9a5mkOpFeUvUD/yLwfT/wo8e+7wf+RRBc+L7v+xe+7/vBhe9fBP6FH1wEgR89jn6MlvKDC9/3e611l+o15ft+NE/kIvC7awx6a+wtFc19EUTNdlfXXVe3P935uo0P1puXVMWViG58nCsMM5aiKIqiKIqiKIqiCqqXVMvktV7Q6yXBwigq+XeocpMqRVEURVEURVEUNfXqJ9Vp151KqkEQOqckVYqiKIqiKIqiqFupK+GcOuFlMDqtjVl3LKkG3m/PPXUJqxRFURRFURRFUTdbV8L55Xq/vSAgqY6qKKw6Jw4AAAAA4EbdUEz1715S9X2/zE2QAQAAAADXdEMx1b+TSZWiKIqiKIqiKIr6q0vyfd//4w/92+dnPfBjDxI//kn9qnzF+5BuNr3SgnYSm+OzaVlrYdNGdsln00a1w6axaX5qnnGLTSvTDpvGpvmpecYtNq1MO2wam+an5hm32LQy7ZTbNGloUnETf3LaTcyTucifrBbyfhzZTnHjmbOxaWwam8amsWlsGpvGprFpbBqbxqaxaX/Jpkntr20AAAAAAOaHtFCpAgAAAAAwP6aQVNf+q898MwAAAAAAd8a1kmrtpVp/o5lfrMqiXHkgr6zWZr49AAAAAIC/3eRJtfqsYX6xgiBwz7zlf5Xlx4qx1157Wq88lGe+VQAAAACAv9fkSXXpoWx+7QghhBC+H/h/fN8P7BN3+bEy860CAAAAAPy9rnX278qTmv/HF/G6ErVXWn+GykO58U5fqFQri/LC4uy3FgAAAAAw/66VVCsPZefUFcOlbre6MyxWW/uW+8trvNP1XbPyQObEYAAAAADASNe996/+yUwkVWOvXV1vLFSqS4+UMBRhKLxzX9lsqltGa79deUBYBQAAAAAUuW5SXXlS986HTgAOglB5oS5UqvLzZnyifeSsPK3XXqpLHFkFAAAAAOSbwvepys+b7i8vHkqXV2uVRdn43I4nWN8P3F+e+bXz9x1WXawuPVKWHimVxb+t5wAAAADwF5pCUl2oVJf/ranbLfeXJ66EEML6bnd+2GEY9mNqGIbOqad+MJYeDe4MXH3WaLzV+1ae1CdZ9WOl/kbrN1J/o0WrWFmtxRsvVnupplPo0mOl/kYzDzr2seueee6Z55y4nZ+O/smsv9Y4MgwAAAAAN2SMpFp5IFfXG7VXau2lWl1vxA+N1l5p1qFdf62pW0Z4OQio7i/PPLCa73V5oxHPqJHGO7393XbPusdj2986E9wfWN0y+sdyrR+2edBZWa0tVKrVjUb7u+2cuFF4FlfC++27v70E79wXVyK4CBLdUzabzqkbXobOiat/Mpvv9OZ7Q/9kdhscvsUxAAAAAGCKyibVlaf19rdOP4WGl6F1aEeZcKFSra43giAMgjBxzWrnpzPy2OPaf3XfD4QQwUWw9t94h1UrD+T+icfaTis9w9IjxTlxhRD2sbv8WIlO4o1bfqzou6a4EtVnjf5SymbT9wP/T1B/oyXOVa48lPVdM291AAAAAIDrG51Ulx8rzfeGl/jeVCGEENahvfZfffnfmvJCTWTU+K2Ai9tfeqS4v73oTGF91xyr98oLtR+ea6/UzHna3zvdA7Y5jVQeyv4fX/1gdPvzUHZO3TAM8xpceqS4Z575NbdBAAAAAMB1SCurtcjyv0r8qOPyak1+3tR3zf7ZuZkVBKH3x++eYZtVYSjqb7SCexFFSbVz5ISXoffbX36cPEm4QPtrxzl1gyC8TlJdqFTNA6ufPJXNZhgK59QtOBqs75rOqTfznQcAAAAAd5IUnbUbBIH3x/fOffeX55x67pkXXAQFAXW8uhLm185y71ThhCipGnvtzg9HXInGW71k11dWa8FF0HyvRycPXyep1l9r1qFdeSgvVKrND4YQovPTKZg/OoY8850HAAAAAHeSNLU4Oqq8cz/z4GqUVPVds/5GE0LYx07Jr7HRdlrumbf2tH79pFp5IC/1juVGSdU+dgu6UXkgr/1Xn+D+TwAAAACAkW4vqQohxJVo9i4H7esn1eXHinvmhZehstkc2e/KA9n97ekfzeXHyvWTalx09m8QhNX1RslFAAAAAABTdLtJVQjvt5/4Pph+Ul2oVKPb6pZJlcoLNQjClSf16yTVygM5feB06ZES3U+488NOf7PObe+eO1Ezf5YDAAAA+LvcdlIVV2L539ykuva0HgRBcBGsPMm+qLXP+m5b3+3KonydpGrstTPvNtx4q4dhKITo/HRWno73xTlT3j2StPy08VcjqQIAAAAY120kVd8PujcHvhpxTHVhsdr+1hFCqNtF31a6/G8tCMLaS3WhUr1OUm1/7Rifs79ER90yorAaXASNd3rJS2env3tIqgAAAADun9tIqrVXanW90dq3lBfqypPkIcqhpFqpKptNcSW8c7+S/yUx6nbL+92d4TpJ1Tlx85LqQqVaf61FLQshOkfO2n8zOLhKUgUAAABwD00/qfp+EB0X7daVWMn5fppIIqlWHsr2iSuuRP21lr3IYtX/4+sfu/OXTKrOqdd8r8e19ttCiIKkulCprqzWosWFEGEYGnvtW75ylaQKAAAA4B6aZlLt/HRqL9XKoiw/bxqf274fBEFYe5UTOHsSSXWhUm2+14UQ9nH2N5rWXqlhOEi/JZNqGArfD+LCy3BkUo0oL1T3zIu20Tv3o7OOb2n3kFQBAAAA3D9TS6rWD7t/MWf0daPVjcbyvyNujLSQlVSX/1W8cz+8DKvPMr4nxvphW9/t/neZjnH272I1bumR4v/xyyTVhUp16aGs7bSicCuuhLHXvp0vUyWpAgAAALiHppZU1a3kF6WWlE6qC5WqsdcWQphfrMTMy//WxJWIH9W8znWq1ne7ZFKNVNcbzonbDaufMm4aPP3dQ1IFAAAAcP9MLalqO0V36y2QmVTX/quHl2EQBMvD17jqu6Z37sfvxHudpGp+scZKqlFvrR+2ECIMQ3mjeeO7h6QKAAAA4P7pJtUwDK1Du/PTib6aZYKa7jHVygO588MWia+rWaz6fwLt49Cc10mqygtV2Rw7bS6v1vw/fl6bU949JFUAAAAA948khLB+2Gv/1SuLcuWBvLxaq7/W9F2z/a1jn7juL8/77QcXQRAUJVjv3F97OuGXuGQm1YVKtfZSFUK4v73+EdTaK1VcicRR1usk1Ynpu6YQIgiCG989JFUAAAAA949Ue6XGT6YdWKxWHspLj5SlR8ryY2X535q208o74tp8p0/cg7ykuvRIcc88cSX6dw/u/HSsQzsx20ySqrLZFEKEoVj+92a/tIakCgAAAOAeGiNFKJvNflLtfmnqVe/U3+0JL1JdyE+qC5Wqut0SQnSOnIVKdeVJ8l5Kkekm1eXHSvtrZ2XU8eHqs0aUVG/661VJqgAAAADuoTFShHVo9w+i6rtm5aEc3VtICNH5mf3dp2UUJNXlf2vBRRCGYXW9oX8y/T9+ZTF5+He6SbXyQHbPvOaHEdfc1t9oQgj/z4zP/k0c2S6fHidbiqQKAAAA4HZIJY8KrjypBRdBlG3sI2fpsbJQqTY/GN1DrH/85ccTHl0sSKoLi1XziyWEaH/teOd+5u2Fp372r3lg2Sfu0sOsM6Ir1YVKtbIoR222Ul+iM/3dU+KY6rhpMz7/dZJqyWVJqgAAAADGJXV+Osv/1kbOV3upRuf6+n6wtt49OXbtaT1KiWEYTnAT3UhRUu2dZyuuRBiGiXspRfpJtf5ay2yhm1S/l02qzQ+GuBLqlrGwmDMUr1RxJXw/mPgmUmPsnhtOqtdBUgUAAABwQyRxJZwTd+2/EaErumRUXInEmbHqditKsNrHCS9VrW40govA+m5n3tip8kC2j5yCg6LKZjO8DIUQxl473cLKas395Qkh3F9eybsfRZfjBkHY/GAkDjgvP1bULSMIgiAI8g7hTnn3kFQBAAAA3D9S64slhHDPRgS56MikfZw8LbbysHsqrHvmFZwxm0ndMuwT1/vjR3cnck4969BOHzhtvNUz76XUfG/Yx250QDU6ruucup2fzsqT+kKlKj9v2seu99sffJXOb98+dqP7MxVYeVKzj93wMhRXwj3z2t86+q6pfzLb3zrebz+8DO0jZ+IDyGPvnvGTav/q0/SVqCJVmb8iqQIAAACYLWnpkeKcuGEYys9z09fSo+4Ztpm3Glr7r+77QWaYLLb2tK5sNuPk5830cdHKA7nxTk/fS2nlSS2xeLeFh/JCpbr8WEn/VtlsyiVC5tIjpfqs0XinG3tt69C2T1z72G1/62g7LXkjo4c3uHsmOqaaTqEj5y8fQdNxt3h+kioAAACAcUkLi1Vjrx2GYXWjkTeT8kINQ+Gd+9lXtC5W21870fm3M9+eO2bipDpWEB0rqXJMFQAAAMBNk6JvEA0vw4IvEdU/mUII7WP2TY8WKlV5oxFehu6ZN9bxxsqirLxQ46f7ypvN6rPcwFxsebVWf6NFB06XHitLD+XKA1l5oWam66WHsvJCVV6oIy/QnfHuIakCAAAAuH8k98wLL0P/T1AQMp1TN7gIVp7k3iK4sii3v3XCUNReZd+AN4+y2ex/98zSI6X1xbrODXWjGwgvPVZWVmu1V+rSY6X+RsvMos33RvS1OgXnPC//q0z81TtT2z0kVQAAAAD3jxRdZ2h8zj1xd2W1Fl6G5tdO+krRuOp6IwhC57Tom0gzGn9a13Za0WHV6npD+2iuPK0vPVJqL9XaS3VltbawWFU2m8oLNYqU0YHQ6nqjsigrm83aSzV+KFj/ZFYeyCtP692jtb2kuvyvorxQa6+0/r185c1m450eXdFae6VGB3Jrr9S1p/XaS7X2Sl3+V1G3jOY7femRUl1vKC9UZbO5/FhpvtPl5836G21tvR715GZ3D0kVAAAAwP0jBUHQ+WGvZH1VaaT5wRBC5H3f6cBitfW5La5EFBdLrn7laV15odZfa5VFufZSbbzVV57WG+/05X+VygNZ3W6tPK1H3wfTeKsvP1aMvfbSI2XpsRJl16VHirpl9FfX2m9XNxqNd/pCpdp4p688rUeRUt1uLf+rrD2t198MDvkqL1T9k7myWlt7Wm+81Rcq1dpLNerM8mOl8kCuv9aq643lf5Xo62rqb7TqeiP6Lpz6ay26+VPBGdHT2T2FSTXztkYFPxbcBmmCe/+WRFIFAAAAMC6pIKNGjL22EML3g5Fz1t9o0Ze7tPYzvto008rT+vK/NXXLqD5rrP1Xj5JqPxXru6byQpU3GguVanRf37X1urbTUl6ozQ+Gstmsrjeqzxr9g73RgtHX7URN1d9o1WcNY68dzRl9gc1CpRotsvRI0XfNhcWqut2qPmtEB2Nrr1T9o7m8WouS6tp/dW2nFS2+vFqLomntpVp91qg8kPWZJtW/AkkVAAAAwLhGpwhjrx2GQlwJdbtVPOfKk5r3xw/DMAjC+utSF6xW1xsrqzV5o6l/MisP5cY7fe2/ev2NFp0DrG4ZK6u1xlu98kBufjCWV2srT2qVB7K204runFR5IK+s1qLYWVmUjb12P7VGTTXe6tX1RvODER0m7d9dSXmhVh7I0WHb6ERibadVeSgvr9air6ipvVLrr7Xqs8byY6X5wag8kJceK8v/1qJkW3ulyRvNykN59KHma+4ekioAAACA+2d0ilhZrcnPm7WXav+AZK7Fqvy8KW80ojw5evWLVfl5c+VJvXtx6QO59lJd+6/evTHvZjO6TlXeaCi9ewLXXqry82Z0baqy2ZQ3mv37PC3/q9TfaFEWje76G11K2m+wutHoX6daXW/UXnYvPV2oVJceytEJwMurteg7V5ceyiurtdpLdemREn1xa/VZY+mRUnulLj1S5OfN6npj+bFSf63d6NerklQBAAAA3EOkiGplUZY3mkuzvs1vJpIqAAAAgHuIFFGtvbrxW/hOjKQKAAAA4B6SFhart2TWm/o3IqkCAAAAuIdIEXONpAoAAADgHiJFzDWSKgAAAIB7iBQx10iqAAAAAO4hUsRcI6kCAAAAuIdIEXONpAoAAADgHpIENcclSewgiqIoiqIoiqLuXUm1NxoAAAAAAPODpAoAAAAAmC8kVQAAAADAfCGpAgAAAADmC0kVAAAAADBfSKoAAAAAgPlCUgUAAAAAzBeSKgAAAABgvpBUAQAAAADzhaQKAAAAAJgvJFUAAAAAwHwhqQIAAAAA5gtJFQAAAAAwX0iqAAAAAID5QlIFAAAAAMwXiaIoiqIoiqIoiqJmWBlJNbgIAAAAAACYCZIqAAAAAGC+kFQBAAAAAPOFpAoAuD1C4k8MAAAYjaQKAJiEkKS+sZa6/hrHamrcHk7c4F0N4entKjMFAIBrIqkCAMYWTyblU0renGVauGbEvbnNnxM31yWSKgBgJkiqAICxTf0o5Y2ukaQ63ZZJqgCAW0BSBQCMreDoaPzs3DKnB481T6LxghnKNB5Nj/+btxXpiYnGx+1P3vnDBUuNHI2CpfK2ouTgTzAFAIBrIqkCAMZWfKFmwYPyrY2cJx3Mxl17ZlLNfJC5rrwO3EJ/Ro5P+a24/siXbwoAgPJIqgCAsRUfU/1bkmq6wyUXzzuGWdyfICsuphfJO8Zb3M7Ig7cFR3GvOfIl9yAAAGMhqQIAxlYy2v0VSbX85hTMM1Z/ynRmrG4UrL3MIJMzAQBziKQKAJhE3nHFtP70ka2NXFfiXNbMHzOnlznhNrMzecczE+sq05+xOpO5xuJ2RvZz5HHXgl6NOwUAgGsiqQIAcBsKDoECAIAEkioAALekzDFMAAAQkFQBAAAAAPOGpArcU+2vbQAAgHkw889FmEMkVeCean9tC4qiKIqiqFkXSRWZSKrAPUVSpSiKoihqHoqkikwkVeCeIqlSFEVRFDUPRVJFJpIqcE+RVCmKoiiKmociqSITSRW4p5JJ9dyUN0x/8GO7sV6RJKmy3rDOo0mhe9CoPlyQJKmyWjNOw9w/OImmAsd4uVKRJGmx2jhwu4sFHW19aUGSFh4qWU2NXFf2DOGpUVtNdDunVxRFURRFzUeRVJGJpArcU0NJ9dyqr0pSP8iFneY/cus8FEKI0DVeak4oxKm+/LjZCYQQIjjWVv6pW0HWX5tEU8I3N9b042ixTvPxsn4qhPBa68vaqRBCiMCqp5saua7MGQKr3ut28LO5vGq4ub2iKIqiKGpeiqSKTCRVBNJwzbw/19+K+JT4rzJnu7f6STX4qa0trjXf1wZB7liTNkyv9/fD2V1u/Ewc1fTMDUk7Tv6lyWjqVF/+R7OTM9qaJPfm8DObGrmuUTN0mpLS9nN6RVEURVHU3BRJFZlIquiaVnKbYQKMZ9TMB/G+3UQ//670O0iqp7YbdNNpPKn2Q52zu7y27w3/TfFaTxe6B0VjlW7KP5Clt53Uybt+e3Ol6JhqiXUVzhB6B0plMwqqWRtIURRFUdTcFEkVmUiq6LqrSTWeUW+6b/OWVIv7k7xONR7kumf/CiGEODdriwvSzvBh0VN9ZVV38v7gxJpydhaWdy17V1n6R5IWq43DXuINrMbDaP+sNY+LcuqIdWXMYGurKxVppflzuFmSKkVRFEXNZZFUkYmkiq7E8cbMM2kLzq1NzFawloIF81rOXKRgE/KS6sgWMrc0c8GCIcpbqsxWlFl7yR6OVJRUhQhPDeXhgiRVqm8ta29tKKmGtra6ohXcUSnWlL0jLSzWWmeBEEKErrG+op8KETraai8Jh66xXu0+TtfIdeXNEHSaj4enk1QpiqIoai6LpIpMEkkVkXTsmeBBup2CFY37YOKWM0NdcSPpfFuwyWON4fXHp2QPRypOqvGy30vyQf83nrlZUQ68rBkzmvIO5OXdwfFOb39teddJn10cnydWI9dVNIOzu7ywE2uWpEpRFEVRc1kkVWQiqaLrlpNq5iLjJsO8TRiZVAsazMyB/UbSU4rHMLHUZFuRWHteUs080FqgdFL1zY2F7v2KAltbX6oVx9ThpsKfjYXY8djuJa9lkurIdY2aIXl5LUmVoiiKouaySKrIRFJF160l1SDrTNqxVjFyE4qPRhY3OO4Ry4I2y8fjCbYir4fljbpOtWqchUKI4LSlbLQ8IURg1RcXRhxNzWjK0Z4q5nn0uHeib+ho6w2r/y040SnB8Rq5rswZzlvVfxrdmzMFtva0e+/fjF5RFEVRFDU3RVJFJpIqhg7HxQ9CJg5IZh6yyzyUN/LIXmaEK2i5zNHC9GyJrcjrbWbfMpNt5hBldju9VN4A5m1Foqn0OBT0sGCc48pdp7qwtKnbgRBCePtriW7IB372d8wkMmFg65tLC5K08FDR+zdPOrca64mJg6ZGritnBhEc67XViiRJlfV6K3HxKkmVoiiKouaySKrIJJFUUVJx7JlVU5hYMqlSFEVRFEXNokiqyERSRSlljgeWb2fmm4OApEpRFEVR1HwUSRWZSKrAPUVSpSiKoihqHoqkikwkVeCeIqlSFEVRFDUPRVJFJpIqcE+RVCmKoiiKmociqSITSRW4p9pf2wAAAPNg5p+LMIdIqgAAYAwz/0TLh2MAuA9IqgAAYAztu3LtAEkVAOYZSRUAAIyBpAoAuAUkVQAAMAaSKgDgFpBUAQDAGEiqAIBbQFIFAABjGCTV83ZjvSJJUmW9YZ0nYmDYebsgH/jRY/egUX24IElSZbVmnIZDM56b8obp33gszSiSKgDMM5IqAAAYQzephp3mP3LrPBRCiNA1XmpOLIGGp9qKJHWT6qm+/LjZCYQQIjjWVv6pW0FvvnOrvipJJFUAQApJFQDmgjRcM+/P3ZA3nvdhhG/uidRNqseatGF6vdTn7C43fvajqtfaqNdf9pLqUHnmhqQdCyFE8FNbW1xrvq+RVAEAaSRVAJgj04oWf2MSu7k+z/+o3lDLt5BU+wnT2V1e2+/mVv+bohz49k52Um09XdBOhRAiOLXdINnObRZJFQDmGUkVAObI/Geq+d/2v3FU5y2pFi84fPavEEKIc7O2uCDt2NH0xlPdESI7qZ7qK6u6E59CUgUAZCGpAsAciSeE9MnAeVOC4VM9R55CHJ8hs7XMljMXyWs5MUNiSsGm5S2V1430gplTCjqTt66RzU6w7ZljOEHLxaNR3OeR/Rmpf0el8NRQHi5IUqX61rL21qKk6uxWm8ehyEyqoa2trmiJOyqRVAEAWSSSKgDMj3SomOBBup2CFY37oLjlfuwp39Wxtn2C7cpc3QTjPMVRHdmfCVoed+THGtWEzG+psd9L8oEvzlvy6+79klJJ1TM3K8qBl1ySpAoAyEJSBYA5cstJNXORcRNmXrPFB/cyj/UlfkwsVSZQlcnAY63r5vL/yG6P1XL5pDrBqCZkJVXf3FjQjoV/IEuJik4JDmxtfamWjqmCpAoAyEZSBYA5cmtJNUhlxZtLqgWLF/R5ZLwsM4AlmyqY4e4l1QlGNSF2nWrVOAuFEMFpS9loJWLo4JhqYNUXFzKOpkZFUgUAZCGpAsBcSByIik/pP05MzFw2MbF4jZl9KOhVyTYTeSndw7zWMjehYMOLRzJzVEeOYd7IlxnVkRtVsBUFLef1J7110vDIj7Wlxc+NuNR1qgtLm7odJHNgP6l6+2uJrg6dFUxSBQBkkUiqAPAXKY4Qs2oK92pIM69T/RuLpAoA84ykCgB/jYIjchO0M/PNuWOmtXfmH0kVAHALSKoAAGAMJFUAwC0gqQIA+7FjCQAAGxBJREFUgDGQVAEAt4CkCgAAxkBSBQDcApIqAAAYQ/tr+86Y+WACAPKQVAEAwCRmnjOJrABwh5FUAQDAJNqcBgwAuDEkVQAAMAmSKgDg5pBUAQDAJEiqAICbQ1IFAACTIKkCAG4OSRUAAExikFTP2431iiRJlfWGdZ6IgWHn7YJ84A9NOzflDXMwqWjx2yiSKgDMIZIqAACYRDephp3mP3LrPBRCiNA1XmpOOAiB4am2IklDSfXcqq9KUj+pFi5+O0VSBYA5RFIFgFmShmvm/UGxv3FP3VyHu0n1WJM2TK+X+pzd5cbPftb0Whv1+stBUg1+amuLa833tUFSLVr8loqkCgBziKQKALM3rSDxdyWoeejzZGu/uT7fUMu3kFT7x0yd3eW1/W7w9L8pyoFv78SS6qntBsOL5C9+a0VSBYA5RFIFgNkjqf5da78/SbV4weGzf4UQQpybtcUFaceOpjee6o4Q8aTarXg6zVv8FoukCgBziKQKALMXzwPpk4HzpgTDJ3aOPDE1PkNma5ktZy6S13JihsSUgk3LWyqvG+kF86YU9DCzM3ndzttfk41G5roylypuuXh8CganTH9G6t9RKTw1lIcLklSpvrWsvbUoajq71eZxKEYm1ZzFb7NIqgAwhySSKgDMXF4QGutBup2CFY37YGQ2S/w7sqtjbfsNbVfm2kf2Z4rjnG7n+i2Puy/GGueEzG+psd9L8oEvzlvyayuIpoxKqhmL326RVAFgDpFUAWD2bjmpZi4ybsLMa7b4UF5BFEwsUn7tmfNMnFRHtlN+QCYY1dtMqhOMc0JWUvXNjQXtWPgHspSo+JHS3KTaXfyWi6QKAHOIpAoAs3drSTVIZcWbS6oFi0+WFcsP4MQjdq+S6gTjnBC7TrVqnIVCiOC0pWy0EjdEKnGdatHit1AkVQCYQyRVAJilxGGn+JT+48TEzGUTE4vXmNmHgl6VbDORjtI9zGstcxMKNrx4JDObzRzVxGx568psp7hXIzezoIWClvO2It0raXhfFPRn5LYXjHnqOtWFpU3dDpI5sPR1qtmL30KRVAFgDkkkVQCYf2Wi2u03hXs+yJnXqf6NRVIFgDlEUgWAeTfyCN5Y7cx8c+68ae2v+UdSBQDcHJIqAACYBEkVAHBzSKoAAGASJFUAwM0hqQIAgEmQVAEAN4ekCgC4F3zfn3kf7liH21/bd8bMBxMAkEBSBQDcfd65Zx6Yfx3v3Jv50AEAMBMkVQDAHef7vnlghn9hmQfm/B9ZBQDgJpBUAQB3n/nl70yqX8yZDx0AADNBUgUA3H3dpHoZ/l3/klQBAPcWSRUAcPeZX8zwMvzrkFQBAPcWSRUAcPe19luZUbCzJWXUdme8SPm7JUtqp/uj71+k5/FazyR53yt8TFIFAGCApAoAuPta+62gsDpbkrzvFc9Tohz9yYL6Iz29m0gLH2dUa78186EDAGAmSKoAgLuvXFJ1J4+ovWZUScpKqm7rWb/9vMcZRVIFANxbJFUAwN1XKql+jiXGs5b8TNVfry1IC8rmivYzNv2FmTwGetaSJbUTuK1n3dOHU2HVbT3rt5/3mKQKAMAASRUAcPeNjHxWlFT7U85asrSgfHaCi8D/3ljZ6UTT3c9y7YuXXPysJUuqdREEF5YqSephuv1eIi16PEm3AQC4q0iqAIC7r/V5VFL9kE6qinkWBBdB8Kfd+FfrXATBhdt6ppi/U4v/urGkOqrbAADcVSRVAMDdN3ZSHYTP6LcV9TAIfrXkzZaXXpykCgDAtJFUAQB3X+tzyy8s64Mk77mDn38ZsqRa/R8P1coHy9mTlX0vY+HBzFFSTc/hGs/67ec9ziiSKgDg3iKpAgDuvjJJtTqcVKvxpOp3tH9leV1pnWUtPJi5oz2SGt+LVzVGkVQBAPcWSRUAcPcZe0ZxJrQ+SNU9Z/DzaSKp+p3tZemlmX30czCz19leW5Ck2pfEjI6x3m8/73FGGXvGzIcOAICZIKkCAO4+Y8/w//jX4e4rtS/uNRsZF0kVAHBvkVQBAHfftZOqbawrrV+3GlNJqgCA+4ykCgC4+66VVE+MqlRRPtu3HFNJqgCA+4ykCgC4+4w9wzv3/jokVQDAvUVSBQDccb7vG5/+1qTq+/7MBxAAgNtHUgUA3H3OqWN8Mv4ye4Zz6sx86AAAmAmSKgDgXvjrDk7+dR0GAGCKspMqRVEURVEURVEURc2wMpJqehIAAAAAADNEUgUAAAAAzBeSKgAAAABgvpBUAQAAAADzhaQKAAAAAJgvJFUAAAAAwHwhqQIAAAAA5gtJFQAAAAAwX0iqAAAAAID5QlIFAAAAAMwXkioAAAAAYL6QVAEAAAAA84WkCgAAAACYLyRVAAAAAMB8IakCAAAAAOYLSRUAAAAAMF9IqgAAAACA+SJRFEVRFEVRFEVR1AwrI6maX0wAAAAAAGaCpAoAAAAAmC8kVQAAAADAfCGpAgAAAADmC0kVAADT/KIq0d0bNhVlcCsHRR1v8fLzAwCAIiRVAAAiqiJJ0qZaInaSSwEAuFkkVQAAIt3DqsoWSRUAgBkjqQIAEMlLqr0Tg7tT1NTpwYkZEksVNAUAALKRVAEAiKiKJK28aWTlUkUdOo6aPqaaGW7zluKQLAAAI5BUAQCI3ERSTbYfK5IqAAC5SKoAAESipGqYow+EXiepElABABiNpAoAgBk/4DnqOtX0xOJLUkdOBAAASSRVAAAAAMB8IakCAAAAAOYLSRUAAAAAMF9IqgAAAACA+ZKdVCmKoiiKoiiKoihqhpWRVIOLAAAAAACAmSCpAgAAAADmC0kVAAAAADBfSKrAPSJJvLoZNwDAlKX/TNzVPxx3dbswn0iqtzrW0cu7/+A+618nfXPtT6WHwQ3sr8SV4umJZRbMa6fMmIy11Pzsi8w+J6aMNQ43uhUj+3w/3wTST/vyS82kqzMfsamMMxKDM9ZATfclPNk7SWY3rj8O6feokX8dbvnPxxS39KZXMXLKTW/g7axxWmu5nZ0ywSr+uqf3nSeRVG95uIPbff+aK7N9K5+gtzf0+SC+4emnRMHng7zFy4xk5pyzGv/J/njkLTvyU+Y8dHvmYz7dzbydz+jTHbep7L4b6tt0u3T7e+dvV/798/ZHbN7+EqWXun3Tei1P8f15hm/1t7wXZvgn9eb2+8xHFQVIqrc93ME9fgH8XZ/ab+HzwVh/7wvecMdafB7GfOK/NxP81SGp3sRmklSn27fpdomkOvVtv9tJddy/DjP/azLbpHrTofcWRmNOVkdSRRkk1RmMeHpK/H2/5N+kMkuV+cMm9SqznTIP8vqTOUOiP3kz5PU8c4b06vJWnV575vRxR7XM6kaORl43Cp45eaNavHjezGWGNHOQEw9KDle65eJByNzSvC6NO87FW1q8C0oO+8jGi0ejeOTz1hv/t+SWFndv5L4os1Te/sobnPJdKhifkY1kjliZLS3TvcTW5e3BqawrcxjHbTndw5J9zlz7BPurzDOqYBdntl/QbOaUkSM/cqniDpccrvLjU6bxkdtVftinuAcL9tfItU+81MgpZfZOwTNh5F6e7h4ct9vFLRf0ucy+K+5zYu3FS6W7NHJgx9rvZbYUsyWRVG9/xDN/LHhQ0Ej6DaJgnpLtlGl5sq1I92S6fZ7uOPffOssP1MjNTDwYOSAjf5XXTplxK/htmX9nst8zmx1rXWXWntefCf50jbWLy/fn5p69JTdzgj043X0xwVJl+nZDIzbZO/bN7Z0yLV+zz1Mfjcn2+7TekSYbscn2Tub8EzwzCzYz8w2/oJMjf3tze3AqY1hyD05rn2au5Yb2YEGfy3S4+CmRfnpM913imq/cyba0zN4p2Thuh0RSvf0Rz/xxgnfq+DtC3l+O9LtGQWdKdqPMuq7zV6Hkn4REy8VvYSWXyuvMBO+n6R+L91fBKBVML/9mWrzqkX+lrjkaU9zvI/dp+QXH2oqxRnus/mTOkLlRY71LZL78J3udjtyuMv2Z7r6YYKkyI59+5Wa2kzfbyMaLhyKz5cnWdf2Wy+yCgpbHGuqSLZdsp/g5n7e6kj0sHrGSfS4e58z5E5tW5kF6X6R7WNDVsX5VMKXMHpxgqXHXVWYPXrPl4kZuaA/mdWCsAUyvvbjPBeMz1jwlB2EqW5o3YplbWrzHccskkurtj3jmjyVfqOVnLvMyu+b7zljtlF9wgvedkesaa3jjM0z212XcMSzz/lj857zk5pRsp8y/5Ufjhvb7ddY1QctlBrl4r5VvJO+DyATvEhO0M8FbR8nn4RT3xQRLjfWSv4mWJ54yrb0zVsvX/Ctzm3+/EltRvAfLrG6y/V7y/WHc1868/SUqub/GeuWOtdQEY1hyD16z5eJGbmgPFvTtOiNf3OeCjo01z7hv+NfZ0rz9nrmlxW3ilkkk1Vse7sTf0f7EzNmKX37pKrO6kfMUt1xmXenH6Y0taL/8MKabyuxkZn8Kxi3eEynr/Usq3IMjx63gt8Wbn7ddIxdMjG3mIomWR/5b/Gwp2Ni8QSu5xwvGrcy6ymx7eivSPRw54Jl9ztyKgj018llX0JSU2lll9lfe6vI2sPwiBaMxcuRHDlrJ8RnZSHzExupPme4lWkvPn9fsddZ1nZalrCruc8GcZZ6HxaNR8kk11uDn9eGaS43scPFzLO9pmV5XuvHyz4SRW11mE0Y+n8fafWWWKtjYMn0euWDeDkrskbwf82a4iT2YOU/mlPL7saDPebu4eLgKtrTkhozcO3k9jP82b2Mz/023jBmSSKqYK7f2vpB4z5r5ht+x4b0zHZtJ9+Z8NDCu+/luAxSb/9fFyF5lprKZd/vWhuKubizmCkkV8+L2//uK/zObB+wCRuM+4N0GSLud14VEUdeomb9M7jmJpAoAAAAAmCskVQAAAADAfMlOqhRFURRFURRFURQ1w8pIqulJAAAAAADMEEkVAAAAADBfSKoAAAAAgPlCUgUAAAAAzBeSKgAAAABgvpBUAQD4//buGEduIwsAKI9iHcPARgoVCvAJBJ9C2MQQHAkOjMEeYAGFcmBAvsACuswGOoIdjGbEaVYVi8Ui61PzCi9ot9jkr8/PIv/0SAYAYtGpAgAAEItOFQAAgFh0qgAAAMSiUwUAACAWnSoAAACx6FQBAACIRacKAABALDpVAAAAYtGpAgAAEItOFQAAgFh0qgAAAMQyGYZhGIZhGIZhGMbAkehUP/z3AwAAAAyhUwUAACAWnSoAAACx6FQBIKa3r6dpml6/FQkAz49OFYCO7lua6fW/v77o0d503FVyt0fsvFd4caKKFg8A3zmdKgAdbepU65uf1S239lHz7QP2YL1C6ju1gIkC4LulUwWgoyt2qgHpVAF47nSqAHRU6FRv3pn/8m1um+RmNdskN0uH+nSDdJCz6bx8+XikJ3NMvF4kof73jR97wtxHyklYfjb5wWSchSB1qgCcZ9KpAtDP29cPnc9D/3PzG7aFX7tdblPz4uazuV2VAq4IcrHzn95WBNzc6T3ZzyKfNcmsTEuu400GqVMF4Dw6VQA6KneqN9/ULTufm68By61jub9KbpaNOf91YmbPP70uzi7X1FX+61DlTrWchEJaKuPMBalTBeA8OlUAOlr9TvV240Hfqa5+lZoLcv7mWje78ovHtb/9W/edanlqueNu+gZVpwrAeXSqAHSU61Q/1H0NWP7Llrl3Knd1G2dxJ8tjlXu2+Zbf/i5rfpvVlu9+45cva7+jbkhLMs7yfrSpAJxk0qkCQEhxmsM4kQDwXOhUAQAAiEWnCgAAQCw6VQAAAGJJd6qGYRiGYRiGYRiGMXAkOtXP//tMm2m6avbiRB4nEuacFwmUQOhOYROfKmUUnWr/hA6P4eqRx4mEOedFAiUQulPYxKdKGUWn2j+hw2O4euRxImHOeZFACYTuFDbxqVJG0an2T+jwGK4eeZxImHNeJFACoTuFTXyqlFEu0anevZmmaXpzNz6SqoReNDNxlqE4kTxkOHj5nXSBHHNeHjO8zPN8Xl83e/OfQ8/Isaf7sMJe5rDwzs1rCUymMfL1fuhMRyXhEivtUU5ZGS6R2OuWwdGL6viEHP8AMH6OWwK+RLTfiekKneqVHPwsFSfyytWzZZE9PoeboipvHOUucoKDH6ceO6jHZJ7fqX4+9IQeeaefZ+nmdeGFBHaM/ypLQU2cHedSv6vVLa+S4c1O+RHMwJJ+JjfcLrHFneCRVapEz3fgM3x3OtX+CR0ewymR61TbJ3hRgzrV+R/pVOsD1qmeHP9VlgKdajg61eqNI9eATvX8WSvRo6ccYjrxO9WbB9PC7wqGULyYvwX/9IG75om8/Ot8yXRty1U+8vLvFuaOfvPBZPvRkMOaUikfsTL45JbNu+pSBskJdi6D1treI1snD9Ov7FQf/uhfP/64K2m7snR6Asud6nLWhU71eSZwNZ/llWGZpfITzON49SqR7VwaN61FNSeuJobKiey/BdRs1nY3qanemgX52Kqbzu5Uax6rVkt6zw23cgHp80RRUQY7a2C+/+U19fWd/ENd/WW+nPJ5j8TTqZ1ql1X3Ks+Ee0q0bedtK3P9E0L/2gveqS6LePV5a6Rp5WK+ezNN0893Fd94JAv9ZrKrmdmQq3zkNYfOnZeWc7eWw1Ju76/hRYdTjqoydc276lUG5QajTxnsqO1md2++rX2lTnVxcnO7evuxNOXyeTxwVTkmgZVVUTnHZ5jAct4qr/rCiVg7I+lsl3Nbc2lvWS6exLA6l9zF2HYLqC+nhrvJlupdWZAPrLqDC7v+FKxmb7UMmk9c2w238jRV3/FbaiCzfeK6rol89UI+6v4+tEoPWnWbS7Qyz8276lWizTuvv5RW7xcnrZA61c4JXZ/Lz28WdfY4Vm8bazXUuoQVI1899M1muYt2OdmGHJaCzKxK5aiSwRfWoK276lgGyxPRuQx21Haz5dq3TMirV4mTu3VXuVQnk9Z/pscksP4ps/qW9rwSWJm3m+uucklM7vxp0tbv+uWQcqcp+c5yb4Uzvun5b7UwcvksPyFt2lVNzvN7XlmQD6y6adh3qoXiaSuDthr4nFlAetVATRk010DmPlt7XSejGnB/H1qlB626F3ombCvRrVVaU2nlzWqeEA6pPZ1q54TWTadQ1uvXXuazu5aw6shXw66JcGcOs+Hlf35WqKjK1NVMMJeHvmVQzupRd7LW81J14vLPyvcv9nSqNefljFXlmATmqrGtMp9hAgv5rJ91MuGrh9j0RNu2OBRmVDjjDc9/uU+1rU6b7iY1mV9dDFcX5AMdXNj1edt0v26+4Vae8e41sFoGbTWQ2WfLb6YkD3rS/X1olR606l7rmbChROurdDV1lQVZ84RwSO0F71QfT8zyVB14Te5JaN2Mlif4fjz8JYfSNvN6Slbt7QY1/w5NPvLloZfHKmyZXP1LwVTkMBvn2k/6NwVf8059HvaXQfnoudltK4N9td141hbhPU7/Sfyv1jvVZZbmmaxMWnuKRiQwOZ3cVVZTAM8wgcn4k5Od8uvAanpXr9/lzpO5TZ6CrYtV4c2bsZxLbqVdLYxN+dx6N8mGumXPuQX52CeN6aT/f1UuA8lrfLWk99xwG66F5WaVNbD1cA01sBp5cic1y8h59/dBVXroqnuhZ8LmEq2p0srUtW1WWD161l7wTjWX9HA96mNCh8dw9cjjRMKc8yKBEgjdKWziU6WMErpTvfv9bqoYd7/fDQ91ntDcH7375V3NdA4d7355tynyj398PDSej3983JTDgve/vT801O9pvP/tfd/a3uPFDy9G5+N2vPjhxcmLgwQOTOC9T399Gp02I+749Nen42pvOqaw3/06/pHDOH+8+zX7mBetSj/9adX93sanP/svlVPkTvWKpsv+2ClO5HEiYc55kUAJhO4UNvGpUkbRqfZP6PAYrh55nEiYc14kUAKhO4VNfKqUUdKdqmEYhmEYhmEYhmEMHIlO9cv/v9Bmmq6avTiRx4mEOedFAiUQulPYxKdKGUWn2j+hw2O4euRxImHOeZFACYTuFDbxqVJG0an2T+jwGK4eeZxImHNeJFACoTuFTXyqlFF0qv0TOjyGq0ceJxLmnBcJlEDoTmETnypllA6d6k35Pv7918I7x+2nbZvkp9ouy+ZPHTS1vpH/PU2Plu/8PdvDzevlBt1zSF/Jy2R4VFunUJhObsrlJPSK5xIkEE6gsIlPlTLK3k618Cx7/7rySTe3n+WLwn7atskd97RO9bipHRH5vEfNvbh5ndumYw7pq3wxHnesM+dVv0HwRksC4dIUNvGpUkbZ26l+yTdImzrV1S3Paef2f4P0bDvVZWuabFx1qpegUz0iQgm80EzhNAqb+FQpoxzeqeb+c3U/y18bq/9FsoZtlnvWqSYlO9VkX6pTva79nWryEv6yuNAqL+r5NvOPLN/PHSg3hZp4Vj/VkFUJPDqBtJmejuHxPAfy3DeZCvigxA6P4YoUZJccHvX3VG8eXzbtZ/m6fm9bj1UTQ3M2tn5kT2A7g6/8SM13qjrVq0vW0tbzsvxgclc1XVbDi+Secx3Upv3sT4gEnpNAuASFTXyqlFH6d6rL9yvru0unuqdNrT9K2873HDR+p3qzzfIfXspt3CWH9BWqU/2y+KZuuSYst0nuOfnOcs/lCKfZaM6qBJ6QQLgEhU18qpRR+neq5Yehyv20daptbWr5uA0J3fORS3eqm170zSF9hepUc59aDXK10WqLsLk+JfDkBMIlKGziU6WMsrdTTf4Y/uZH7MufoK/+YL7+neROCsfKbbN6rPqEtJ2G8tSat+kbec3/pabwp/4vNRdSU2+VH19ea8mNa/ZWs4bMD7q8xnNXetuntqZUAk9LIFyFwiY+Vcoo0/7vVLlJ6PAYrh55nEiYc14kUAKhO4VNfKqUUXSq/RM6PIarRx4nEuacFwmUQOhOYROfKmUUnWr/hA6P4eqRx4mEOedFAiUQulPYxKdKGUWn2j+hw2O4euRxImHOeZFACYTuFDbxqVJGSXeqhmEYhmEYhmEYhjFwJDrV5VsAAAAwkE4VAACAWHSqAAAAxKJTBQAAIBadKgAAALHoVAEAAIhFpwoAAEAsOlUAAABi0akCAAAQi04VAACAWHSqAAAAxKJTBQAAIBadKgAAALHoVAEAAIhFpwoAAEAsOlUAAABi0akCAAAQi04VAACAWHSqAAAAxKJTBQAAIBadKgAAALHoVAEAAIhFpwoAAEAsOlUAAABi0akCAAAQi04VAACAWP4B0kMbFHtwjvwAAAAASUVORK5CYII=" alt="" /> 客户端如下图:ls很长时间都不出来。
aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAAa0AAAAvCAIAAADfOwIHAAAIN0lEQVR4nO2dXbKrKhCFGVYYD+VsLAeTKqdxXm5VXhyM9wHktxtpYyLurK9unXOP240N4kqDsKLWdV3X9fXfP/un/R8AAPgdlP0LOggA+Fm61MGnUZbHeHUoIGIZ9TCvT6On19WhAHAmOzr4mrRSSik9Lt+KaBn1drl5UOYZfjI+nDwmz+EyauUxc1TSFrxSQ3xYgC0hjiFjHlRXojAPUVvwYccnF/HPpmxkyzLqYX5NerfkNWr8rtoHAJJ9HfxcPx4fptSn5IpR6hEpzmzih9wmKSVP49PJTE9FND72XbCMujmDfk1a8ckdfd+fRjXrYK0cADpDrIM2KdPTa/vAD1rm87UsLyOOJ0lckrzMg00G+awk0yZGB7dyaudkhDgf2gvKa9Jm8tGGetH5jquXu7Q7xxblB/vRr7h8c/A/Ij4Y8sIf2tgzXYHuV+JMMJtSIOvlaNbBSvkVSB2sxQPAFRzKB+0TuD3eVpLGh4rzOP+ccMdXJh+0+uUyOPopnU388DDj4lwH9563uKbzEJ7z16RjXcviIdonjTnU8TnHHxhex1+T9mHvZ0/LqJVrGftb+UdCUU2uXmS0tXptJycNu0e9xYh4ALiCgzqYHywSLve0cMfXda3q4PjQ48JcqDLz9TR+KpDRwdmoDB9D/KMk7wuXK+Jh8h2vfdH5aQoc62Cl/Bxfka2muzrI1at+RbJe9uCbOrgTDwBX0KkOBtWIHuw4NWMIqWIyJ9g2Lg5EeesBHfSnRRWcTRR5XObndZCuV/2KRb3yz4/GWb+dDLeMB4ArOEkH1y2DcwQ94o4nP9pGeewV+TFUPO52c/+WZTTR+Gt3aj+JM8orj+igVcAsGfTBpyntp3WQq1f9ilw+aJ70pxcHNb9cjQeAK5DqYJ4XJA+h6PgavzqIHq1l1Ns/wzwa/15lTebdExUIx0UvSaJ4/MuQbUrOp0KV8bWvGrOIR2kzuOU4fPkUvhGG2Z7s31aZZxZ/qDJZr3XvvQepg9uURYsOsu3DxQPAhVy5bobj/XV/13Pzxca8DqYvqQD4E/S3jvrmiFYyd0ltxRIAf5Iu99UBAMAXgQ4CAH4d6CAA4NeBDgIAfh3oIADg14EOAnAycNm5HawOwj+uG2ZTrHPuzfewH/L14RcsQZ0N1offjR7XUX8a0c6wPvjq6uUbtk/O930zT7z0We3/B+7j1ziig7x/3Gwkm6XCR/djTPcaqywV5Xz66v59It/DCmU540MppcfJXzf1tvGnJ+ZdzT6DRTnmGXSQy9NZX0gqHrb9G/cvFqaT+XUrPoxHkPWrtei3XPtz9zH2IU9cu/f7D5EMlu0mLf8DcRYw/YTrV1If0sr5PSDWwap/nKC/zoOKvQyClAh9+rjjUt9DjoqvYqo1vn8kvjKpFDb7DCb+NMS4WOILScfDtj/TPpX2JK/L+jAe4V0dXKv9hLyPFf+LSl1I32Ky3aTlnxtnCddPXNjl/WWOi/tJHxzIB0/xj+MHekKfPvq43O+LDYYpJ9boEEbxOawa+itBdkJhJCPwQ6PjqQ20ifaptidXHdqH8SuQOki2P30fD+pLkQzy7XZAB8+Ls4Dvt2Wxa+X4oX7SA+/NDx73j+OeQ7FP33U6mOy53sJg9eUaHay08+d1kPJh/BISHSTv4xF9IW7KmTp4WpwU1Qlo6KDQP042Lo5K3r56Se7T19a/930POXhfRUWOtbk3uTIdTDOL0nux3ReSi4duf1cXon0q7VmpTu7DuAPXf84ZF3P5IHkfQwnLqNPZWKb/0K+JuXaTln9enDS1FQjtOni0n1wOp4PH/ONk/TUuKp0HtOz79NX8+6S+hxxMOePDhZdclGkimc9gHqRSg9mmCLn7wvtC8reMbH+2feh2qF13K0qSldA3RTjvHLO51XLtz97HUF/3giKZ9irahx057ftyNpV/VpwVqH7C3V+pD+leP7karKM+SDZOATQdpwCWk+5jMp/zCdDfPgp08AicAzbw3MKH8S738S5x3hfoIADg14EOAgB+HeggOB+1KvK/q+MCgAY6CM4HOgjuBXQQnA90ENyLLnXQr3vC27GusLsFGpbCQAfBvejvezuXUW+Xmwd6PXC+5yxf/Ol4/3uQYz8Pkt58AEWrVRKHkugw+72dy6iHOdmYwQAdBPfiSv9Bcv9jcsUo9YgUJ9kEVm5p9L8be6scXsLW8tj3QrENucJr0opP7lg/G+gg+Isc2F/skoVG3zH6OO+Ptm3Mrn2bePIoMjqYbPDmtDKF9FV8TdpMPtpkHxWfnFK+exJfRRpb+EO7TWauwLCvOaGwkMvq5WjWwUr5JdBBcC8O5YMS3zGpD6DVL5fB0U9punmbGRfnOriXKHG+irE3X92N0deR9vwQ+irSeO+Jbd9u/pFQVLPqFynQQX9y0rAM0EFwLw7qYKvfjtz/yp7gdlMyhnrsuCzyv2F0kPWP4HwV6z4xjF835bsn9FWk8RXZarqrgzt+kRIdtAehg+Dv0akOBtWIHuzMNpkipIrJnGDbuDjQ5hu87rVPVEGxryLNER2k61W/YlGv/POjHid0ENyLk3Sw5tMn8wHkxobEmM4Vkk63eb1bRhONB3en9jlfxSM6WPruyX0VaeQ6WPWLFOeD5tnk7gkdBPdCqoNS3zG5D2DyXR/b+Y3fH5SqQDguekkSxcP71lXG175qzCKeBl9FOkTfCMNsT/Zvq8wziz9UmTMfrL/3IHVwm7KADoK/Ro/f2/n+ur/r6d53rw6vg03fIAodBPeiv3XUN+cWvntVaiuWGoEOgnvR5b46cHOgg+BeQAfB+UAHwb34Hyzfzcm1vw3qAAAAAElFTkSuQmCC" alt="" /> master再次启动后,客户端ls很快就出来了
[root@mfs-master ~]# service mfsmaster start
[root@250-shiyan mfs1]# ls
aa  logrotate.conf  passwd  rsyslog.conf  woei 停掉单点master后,会引起客户端df -h卡死,或ls时也卡死,终端都不能操作。
http://www.jsxubar.info/mfs%E5%8F%8C%E6%9C%BA%E5%88%87%E6%8D%A2%E6%B5%8B%E8%AF%95.html  双机切换测试
metalogger实际上是一个冷备的概念,非实时,还需要手动参与
环境介绍:
需要两台机器,都安装了master,metalogger,cgiserv
我们首先启动主服务器mfsmaster和备服务器mfsmetalogger,然后停止主的master,然后使用备服务器恢复元数据信息,再在备服务器启动mfsmaster,最后在mfschunkserver和mfsclient上更新hosts记录.之后进行了倒换,将mfsmaster服务器又倒回到了最开始的服务器,将整个系统恢复原来的拓扑结构,整个过程几乎都是手工操作,在出现故障的情况下完全不能实现自动切换.测试表明moosefs在高可用性方面还有很多事情要做,还有很大的改善空间.
主:mfsmaster stop
客:ls /mnt/mfs 备:mfsmaster -a 恢复并启动 chunk与客户都要重新指向新的masterIP地址
chunk:更新hosts文件
#主ip
备ip 客户:更新hosts文件
#主ip
备ip 修改/etc/hosts文件的目的就是为了方便操作,这样就可以不用重启服务了。
其实hosts文件只是起一个解析作用,是否可以使用DNS代替,这样不用手动更新/etc/hosts文件
yum install moosefs-ce-metalogger
rpm -ql moosefs-ce-metalogger
cd /etc/mfs/ 加入master的ip就可以了
vi mfsmetalogger.cfg
cd /var/lib/mfs/
[root@mfs-metalog mfs]# ll
total 16
-rw-r-----. 1 mfs mfs   54 Mar 13 09:37 changelog_ml.0.mfs
-rw-r-----. 1 mfs mfs  157 Mar 13 09:35 changelog_ml_back.0.mfs
-rw-r-----. 1 mfs mfs    0 Mar 13 09:35 changelog_ml_back.1.mfs
-rw-r-----. 1 mfs mfs 3147 Mar 13 09:35 metadata_ml.mfs.back
-rw-r-----. 1 mfs mfs 3147 Mar 13 09:30 metadata_ml.mfs.back.1 service moosefs-ce-metalogger start
日志里面有启动信息,如果发现日志时间记录不正确,重启rsyslog服务。
less /var/log/messages

目标(goal),是指文件被拷贝的份数,设定了拷贝的份数后是可以通过mfsgetgoal 命令来证实的,也可以通过mfsrsetgoal 来改变设定。
[root@250-shiyan mfs1]# mfsgetgoal /mnt/mfs1
/mnt/mfs1: 2
[root@250-shiyan mfs1]# mfsgetgoal -r /mnt/mfs1
/mnt/mfs1:
 files with goal        1 :          2
 files with goal        2 :          2
 directories with goal  2 :          4
[root@250-shiyan mfs1]# mfscheckfile passwd
passwd:
 chunks with 2 copies:            1
[root@250-shiyan mfs1]# mfsfileinfo passwd
passwd:
        chunk 0: 00000000000000E3_00000001 / (id:227 ver:1)
                copy 1: 192.168.2.232:9422
                copy 2: 192.168.2.233:9422
[root@250-shiyan mfs1]# mfsdirinfo /mnt/mfs1
/mnt/mfs1:
 inodes:                          8
  directories:                    4
  files:                          4
 chunks:                          4
 length:                       4527
 size:                       282624
 realsize:                   423936
一个不包含数据的零长度的文件,尽管没有设置为非零的目标(the non-zero “goal”),但用mfscheckfile 命令查询将返回一个空的结果;将文件填充内容后,其会根据设置的goal创建副本;这时再将文件清空,其副本依然作为空文件存在。
[root@250-shiyan mfs1]# touch a
[root@250-shiyan mfs1]# mfscheckfile a
a:

假如改变一个已经存在的文件的拷贝个数,那么文件的拷贝份数将会被扩大或者被删除,这个过程会有延时。可以通过mfscheckfile 命令来证实。
对一个目录设定“目标”,此目录下的新创建文件和子目录均会继承此目录的设定,但不会改变已经存在的文件及目录的拷贝份数。

用mfsgetgoal –r 和mfssetgoal –r 同样的操作可以对整个树形目录递归操作
[root@mfs-client ~]# mfssetgoal 2 -r  /mnt/mfs
/mnt/mfs:
 inodes with goal changed:               5
 inodes with goal not changed:           0
 inodes with permission denied:          0
一个删除文件能够存放在一个“ 垃圾箱”的时间就是一个隔离时间,这个时间可以用mfsgettrashtime 命令来验证,也可以用mfssettrashtime 命令来设置。

[root@mfs-client trash]# ll
total 23
-rw-r--r--. 1 root root     0 Mar  4 11:32 00000002|aa~
-rw-------. 1 root root     0 Mar  4 11:31 00000003|.aa.swp
-rw-------. 1 root root     0 Mar  4 11:31 00000004|.aa.swpx
-rw-r--r--. 1 root root 12288 Mar  4 11:32 00000005|.aa.swp
-rw-------. 1 root root     0 Mar  4 11:31 00000006|.aa.swpx
被删文件的文件名在 “垃圾箱”目录里还可见,文件名由一个八位十六进制的数i-node和被删文件的文件名组成,在文件名和i-node之间不是用“/”,而是用了“|”替 代。如果一个文件名的长度超过操作系统的限制(通常是255个字符),那么部分将被删除。通过从挂载点起全路径的文件名被删除的文件任然可以被读写。需要 注意的是被删除的文件在用全路径文件名(注意文件名是两部分)时一定要用单引号引起来。
[root@250-shiyan mfs1]# mfsrgettrashtime a
deprecated tool - use "mfsgettrashtime -r"
a:
 files with trashtime             86400 :          1
[root@250-shiyan mfs1]# mfssettrashtime 64800 a
时间的单位是秒(有用的值有:1小时3600秒,1天86400秒,1周604800秒)。就像文件被存储的份数一样, 为一个目录设定存放时间是要被新创建的文件和目录所继承的。数字0 意味着一个文件被删除后, 将立即被彻底删除,在想回收是不可能的。

在开始mfsmount进程时,用一个-m或-o mfsmeta的选项,这样可以挂接一个辅助的文件系统MFSMETA,这么做的目的是对于意外的从MooseFS卷上删除文件或者是为了释放磁盘空间而移动的文件而又此文件又过去了垃圾文件存放期的恢复,例如:
mfsmount -m /mnt/mfsmeta
需要注意的是,如果要决定挂载mfsmeta,那么一定要在mfsmaster的mfsexports.cfg文件中加入如下条目:
*                       .       rw
[root@mfs-client mnt]# mfsmount -m /mnt/mfsmeta/ -H 192.168.2.231
mfsmaster accepted connection with parameters: read-write,restricted_ip

[root@mfs-chunk1 ~]# mfschunkserver test
mfschunkserver pid: 2720
[root@mfs-chunk2 ~]# mfschunkserver test
mfschunkserver pid: 2614
[root@mfs-chunk2 ~]# mfschunkserver stop
sending SIGTERM to lock owner (pid:2614)
waiting for termination terminated
[root@mfs-master ~]# mfsmaster test
mfsmaster pid: 2201

[root@mfs-master ~]# mfsmaster start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
initializing mfsmaster modules ...
exports file has been loaded
topology file has been loaded
loading metadata ...
loading sessions data ... ok (0.0000)
loading objects (files,directories,etc.) ... ok (0.0557)
loading names ... ok (0.0663)
loading deletion timestamps ... ok (0.0000)
loading quota definitions ... ok (0.0000)
loading xattr data ... ok (0.0000)
loading posix_acl data ... ok (0.0000)
loading open files data ... ok (0.0000)
loading chunkservers data ... ok (0.0000)
loading chunks data ... ok (0.0645)
checking filesystem consistency ... ok
connecting files and chunks ... ok
all inodes:
directory inodes:
file inodes:
chunks:
metadata file has been loaded
stats file has been loaded
master <-> metaloggers module: listen on *:
master <-> chunkservers module: listen on *:
main master server module: listen on *:
mfsmaster daemon initialized properly [root@mfs-chunk1 ~]# mfschunkserver start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
setting glibc malloc arena max to
setting glibc malloc arena test to
initializing mfschunkserver modules ...
hdd space manager: path to scan: /sh/
hdd space manager: start background hdd scanning (searching for available chunks)
main server module: listen on *:
stats file has been loaded
mfschunkserver daemon initialized properly [root@mfs-master ~]# mfscgiserv start
lockfile created and locked
starting simple cgi server (host: any , port: , rootpath: /usr/share/mfscgi) [root@mfs-chunk1 ~]# mkfs.xfs /dev/sdb1
meta-data=/dev/sdb1 isize= agcount=, agsize= blks
= sectsz= attr=, projid32bit=
data = bsize= blocks=, imaxpct=
= sunit= swidth= blks
naming =version bsize= ascii-ci=
log =internal log bsize= blocks=, version=
= sectsz= sunit= blks, lazy-count=
realtime =none extsz= blocks=, rtextents=
客户端挂载点inode总是为1
[root@mfs-client mfs]# stat /mfs
File: `/mfs'
Size: Blocks: IO Block: directory
Device: 12h/18d Inode: Links:
Access: (/drwxrwxrwx) Uid: ( / root) Gid: ( / root)
Access: -- ::14.000000000 +
Modify: -- ::07.000000000 +
Change: -- ::07.000000000 + 客户端挂载点根目录有一个隐藏文件记录着信息
[root@mfs-client mfs]# cat .stats
fuse_ops:
fuse_ops.statfs:
fuse_ops.access:
fuse_ops.lookup:
fuse_ops.lookup.internal:
fuse_ops.lookup.master:
fuse_ops.lookup.master.error:
fuse_ops.lookup.master.negative:
fuse_ops.lookup.master.positive:
fuse_ops.lookup.cached:
fuse_ops.lookup.cached.negative:
fuse_ops.lookup.cached.readdir:
fuse_ops.getattr-cached:
fuse_ops.getattr:
fuse_ops.setattr:
fuse_ops.mknod:
fuse_ops.unlink:
fuse_ops.mkdir:
fuse_ops.rmdir:
fuse_ops.symlink:
fuse_ops.readlink:
fuse_ops.readlink.cached:
fuse_ops.readlink.master:
fuse_ops.rename:
fuse_ops.link:
fuse_ops.opendir:
fuse_ops.readdir:
fuse_ops.readdir.without_attrs:
fuse_ops.readdir.with_attrs:
fuse_ops.releasedir:
fuse_ops.create:
fuse_ops.open:
fuse_ops.release:
fuse_ops.read:
fuse_ops.write:
fuse_ops.flush:
fuse_ops.fsync:
fuse_ops.removexattr:
fuse_ops.listxattr:
fuse_ops.getxattr:
fuse_ops.setxattr:
negentry_cache.#entries: []
negentry_cache.search_misses:
negentry_cache.search_hits:
negentry_cache.removals:
negentry_cache.inserts:
symlink_cache.#links: []
symlink_cache.search_misses:
symlink_cache.search_hits:
symlink_cache.inserts:
master.reconnects:
master.bytes_sent:
master.bytes_received:
master.packets_sent:
master.packets_received:

测试环境
master 8核8G MEM
metalogger 8核8G MEM
chunkserver1 8核8G MEM 3disks
chunkserver2 8核8G MEM 3disks
chunkserver3 8核8G MEM 3disks
client1 8核8G MEM
client2 8核8G MEM
switch bandwidth 1G

1.读写测试
测试读写性能,2台客户机同时拷贝数据至MFS.
goal转换
Moose File System分布文件系统测试
2.冗余度测试
设置目录冗余度,在目录中创建文件,CRASH掉冗余度-1台节点,查看文件是否完整。
结论:完整
设置目录A冗余度,-r 2,设置目录冗余度B,-r 3,在目录A中创建文件a,查看文件a冗余度,拷贝a到B目录,查看a冗余度。将文件从A目录移动到B目录,查看移动后文件冗余度。
结论:移动不影响冗余度,复制冗余度受目录限制。
3.MASTER节点恢复测试
CRASH MASTER节点,恢复。查看数据是否丢失。
结论:无丢失。
4.chunkserver扩展测试
记录总容量A,增加节点,记录总容量B。比较A,B
结论:容量立即增加。
5.CHUNKSERVER节点crash测试
记录总容量A,CRASH某节点,记录总容量B,恢复某节点,记录总容量C。比较A,B,C
结论:CRASH后容量减去被CRASH的节点容量,恢复后容量恢复原状。
6.数据一致性测试
写A文件,CRASH某CHUNK,修改A文件,恢复某CRASH,查看A文件。
多个节点同时VI某个文件。
结论:文件一致。
7.回收站测试
超时测试:新建文件,设置trashtime,删除文件,查看剩余容量。比较超时设置与剩余容量。
回退测试:删除,回退。
结论:超时配置与预期一致,回退功能正常。
8.权限设置测试
测试目标uid,gid
结论:与目标UID,GID一致。
总结:
1.从MOOSEFS的架构可以看出最容易出现瓶颈的是MASTER节点部分,通信瓶颈或服务器的瓶颈
2.最好在MOUNT的时候取消文件和目录的CACHE,防止不同步
3.适当调整超时时间,必要时删除META立即回收空间
4.配置hotstandby提高可靠性,官方称可以将切换做到1分钟以内。
5.做好CHUNKSERVER的配额,防止溢出。

changelog_ml.*.mfs
MooseFS filesystem metadata change logs (backup of master change log files)
changelog_ml.*.mfs是MooseFS文件系统的元数据的changelog日志(备份的Master 的Master的changelog日志。) metadata.ml.mfs.back
Latest copy of complete metadata.mfs.back file from MooseFS master.
metadata.ml.mfs.back是从Master主机上下载的最新的完整metadata.mfs.back的拷贝 sessions.ml.mfs
Latest copy of sessions.mfs file from MooseFS master.
sessions.ml.mfs是从master下载的最新的sessions.mfs文件拷贝。 metadata.mfs.back #MFS元数据信息,延迟将内存数据写入文件,默认频率1h,默认1d同步一次此文件到metalogger server
changelog.*.mfs #文件操作日志记录,实时记录、同步到metalogger server
sessions.mfs #客户操作状态记录
stats.mfs #? 停止服务,删除所有文件后,但是保留metadata.mfs.empty,
[root@mfs-master mfs]# mfsmaster stop
sending SIGTERM to lock owner (pid:)
waiting for termination terminated
[root@mfs-master mfs]# ls
changelog..mfs changelog..mfs changelog_ml_back..mfs
changelog..mfs changelog..mfs changelog_ml_back..mfs
changelog..mfs changelog..mfs metadata.mfs
changelog..mfs changelog..mfs metadata.mfs.back.
changelog..mfs changelog..mfs metadata.mfs.empty
changelog..mfs changelog..mfs metadata_ml.mfs.back
changelog..mfs changelog..mfs metadata_ml.mfs.back.
changelog..mfs changelog_ml..mfs stats.mfs
[root@mfs-master mfs]# cp metadata.mfs.empty ../
[root@mfs-master mfs]# rm -rf *
[root@mfs-master mfs]# ls
[root@mfs-master mfs]# mv ../metadata.mfs.empty ./ master服务不能启动
[root@mfs-master mfs]# mfsmaster start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
initializing mfsmaster modules ...
mfsexports: unknown option 'passwd=123' in line: (ignored)
exports file has been loaded
topology file has been loaded
loading metadata ...
can't find metadata.mfs - try using option '-a'
init: metadata manager failed !!!
error occured during initialization - exiting 做如下操作便可启动
[root@mfs-master mfs]# ls
metadata.mfs.empty
[root@mfs-master mfs]# cp metadata.mfs.empty metadata.mfs
[root@mfs-master mfs]# ls
metadata.mfs metadata.mfs.empty
[root@mfs-master mfs]# mfsmaster start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
initializing mfsmaster modules ...
exports file has been loaded
topology file has been loaded
loading metadata ...
metadata file has been loaded
no charts data file - initializing empty charts
master <-> metaloggers module: listen on *:
master <-> chunkservers module: listen on *:
main master server module: listen on *:
mfsmaster daemon initialized properly
[root@mfs-master mfs]# ls
metadata.mfs.back metadata.mfs.empty
主要元数据文件metadata.mfs,当mfsmaster运行的时候会被命名为 metadata.mfs.back
master server可以单独启动(所谓单独启动就是在没有数据存储服务器(chunkserver)的时候也可以启动,但是不能存储,chunkserver启动后会自动的加入)。安装配置完MFS后,即可启动它。 metadata.mfs, metadata.mfs.back
MooseFS filesystem metadata image
metadata.mfs, metadata.mfs.back是MooseFS文件系统的元数据metadata的镜像 启动之后,当chunkserver未加入前,客户端可以挂载,可以创建目录,但不能创建有数据的文件,因为这时候没有存储chunkserver加入进来。
加入进来之后,在客户端之前所做的操作就立马体现出来了,并且生成了changelog..mfs文件, changelog.*.mfs
MooseFS filesystem metadata change logs (merged into metadata.mfs once per hour)
changelog.*.mfs 是MooseFS文件系统元数据的改变日志(每一个小时合并到metadata.mfs中一次) [root@mfs-master mfs]# ll
total
-rw-r----- mfs mfs Mar : changelog..mfs
-rwxr--r-- root root Mar : metadata.mfs.back
-rwxr--r-- root root Mar : metadata.mfs.empty
[root@mfs-master mfs]# ll
total
-rw-r----- mfs mfs Mar : changelog..mfs
-rw-r----- mfs mfs Mar : metadata.mfs.back
-rwxr--r-- root root Mar : metadata.mfs.back.
-rwxr--r-- root root Mar : metadata.mfs.empty
-rw-r----- mfs mfs Mar : stats.mfs
[root@mfs-master mfs]# date
Fri Mar :: EDT 下面说明先要停chunkserver服务,再删除/var/lib/mfs/下的所有文件,否则不能删除
[root@mfs-chunk1 mfs]# ls
chunkserverid.mfs csstats.mfs
[root@mfs-chunk1 mfs]# rm -rf *
[root@mfs-chunk1 mfs]# mfschunkserver stop
sending SIGTERM to lock owner (pid:)
waiting for termination terminated
[root@mfs-chunk1 mfs]# ls
csstats.mfs
[root@mfs-chunk1 mfs]# rm -rf *
[root@mfs-chunk1 mfs]# ls 删除以后,不能再启动,所以索性格式化磁盘,重来。
[root@mfs-chunk1 mfs]# mfschunkserver start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
setting glibc malloc arena max to
setting glibc malloc arena test to
initializing mfschunkserver modules ...
hdd space manager: chunkserver without meta id can't use drive with defined meta id (file: '/sh/.metaid')
hdd space manager: no hdd space defined in /etc/mfs/mfshdd.cfg file
init: hdd space manager failed !!!
error occured during initialization - exiting [root@mfs-chunk1 ~]# umount /sh
[root@mfs-chunk1 ~]# mkfs.xfs -f /dev/sdb1
[root@mfs-chunk1 ~]# mount /dev/sdb1 /sh/ 再次启动,还是不行,更改权限即可
[root@mfs-chunk1 mfs]# mfschunkserver start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
setting glibc malloc arena max to
setting glibc malloc arena test to
initializing mfschunkserver modules ...
hdd space manager: can't create lock file '/sh/.lock': EACCES (Permission denied)
hdd space manager: no hdd space defined in /etc/mfs/mfshdd.cfg file
init: hdd space manager failed !!!
error occured during initialization - exiting
[root@mfs-chunk1 mfs]# chown -R mfs:mfs /sh
[root@mfs-chunk1 mfs]# mfschunkserver start
open files limit has been set to:
working directory: /var/lib/mfs
lockfile created and locked
setting glibc malloc arena max to
setting glibc malloc arena test to
initializing mfschunkserver modules ...
hdd space manager: path to scan: /sh/
hdd space manager: start background hdd scanning (searching for available chunks)
main server module: listen on *:
no charts data file - initializing empty charts
mfschunkserver daemon initialized properly

mfs-用户的更多相关文章

  1. centos6.3配置MFS服务器

    一.简介 MooseFS(Moose File System,mfs)是一种分布式文件系统,它将数据分布在网络中的不同服务器上,支持FUSE(用户空间文件系统Filesystem in Userspa ...

  2. Centos下MooseFS(MFS)分布式存储共享环境部署记录

    分布式文件系统(Distributed File System)是指文件系统管理的物理存储资源不一定直接连接在本地节点上,而是通过计算机网络与节点相连,分布式文件系统的实际基于客户机/服务器模式.目前 ...

  3. mfs分布式系统从理论简介到实战部署

    文章前面想说的话:这篇博客写出来真是有点累到了,本来昨天就基本就写好了,放在草稿里面,今天打开就没有了!!唉,就尼玛离我而去了,只有重写,然后中间虚拟机还“爆炸”重启又搞了一会,不容易呀!!希望各位博 ...

  4. mfs客户端挂载

    1.安装fuse yum install fuse fuse-devel 2.加载fuse模块 modprobe fuse 3.创建mfs用户 useradd mfs -s /sbin/nologin ...

  5. mfs分布式文件系统,分布式存储,高可用(pacemaker+corosync+pcs),磁盘共享(iscsi),fence解决脑裂问题

    一.MFS概述 MooseFS是一个分布式存储的框架,其具有如下特性:(1)通用文件系统,不需要修改上层应用就可以使用(那些需要专门api的dfs很麻烦!).(2)可以在线扩容,体系架构可伸缩性极强. ...

  6. 学习MFS(六)

    一.文件系统选型 在一般的生产环境中,NFS共享存储算是比较常用的,简单.方便,但随着业务的不断扩展,数据量也是承爆发式的增长,因而对存储这些数据的文件系统要求也越来越高了,分存式.可扩展.大容量,这 ...

  7. moosefs的安装使用及遇到的问题

     一.获取源码安装包 到官网下载最新版本moosefs: https://moosefs.com/download/sources-archive-3-0.html到官网下载最新版本fuse源码 ht ...

  8. 分布式存储MooseFS

    MooseFS 简介 http://moosefs.org/ MFS特性 安装和配置简单方便   可靠性高(数据的多个拷贝被存储在多个不同的服务器上)   通过添加新的服务器或硬盘就可以实现容量的动态 ...

  9. MooseFS安装部署

    环境信息 Master服务器 dev04 chunkserver服务器 dev02.dev03.dev04 metalogger服务器 dev03 mount客户端 dev01.dev02 安装前下载 ...

  10. MFS文件系统

    一.MFS文件系统概论 MFS是linux下的开源存储系统,是由波兰人开发的.MFS文件系统能够实现RAID的功能,不但能够节约存储成本,而且不逊于专业的存储系统,能够实现在线扩展.MFS是一种半分布 ...

随机推荐

  1. 技术分享:如何用Solr搭建大数据查询平台

    0×00 开头照例扯淡 自从各种脱裤门事件开始层出不穷,在下就学乖了,各个地方的密码全都改成不一样的,重要帐号的密码定期更换,生怕被人社出祖宗十八代的我,甚至开始用起了假名字,我给自己起一新网名”兴才 ...

  2. UIkit框架之UIScrollView

    1.继承链:UIview:UIresponder:NSObject 引言 UIScrollView的是几个UIKit类包括的UITableView和UITextView中的超类. 一个UIScroll ...

  3. OD调试程序3

    条件跳转指令的图片,作为以后的参考. 载入了reverses.eve程序,F8下去,发现一个跳转,调用了一个函数,致使程序结束.于是我们绕过它,参考上面的 跳转指令图片. 然后继续F8 又会发现一个跳 ...

  4. 破解 crackme(“不可逆“函数)

    系统 : Windows xp 程序 : crackme 程序下载地址 :http://pan.baidu.com/s/1i41oh9r 要求 : 注册机编写 使用工具 : IDA Pro & ...

  5. iOS app的破解原理,就是去除已付费的账户信息的原理是什么?

    正规的应用程序(IPA 格式,官方软件店发布)在被 iTunes 同步到 iPhone 的时候会调用系统进程 INSTALLD 对应用程序进行证书校验(GPLv3 授权)而这个证书本身是由官方捆绑在应 ...

  6. URl中文转ASCII

    编码 System.Web.HttpUtility.UrlEncode("中国", System.Text.Encoding.GetEncoding("GB2312&qu ...

  7. 第二个Sprint冲刺第一天

    讨论地点:宿舍 讨论成员:邵家文.李新.朱浩龙.陈俊金 任务:第二次sprint前期把选择题的计时功能完成. 燃尽图: 开发感悟: 大三第一学期就快结束了,大家都为未来找工作而烦恼.班里的高手都去学校 ...

  8. 转载blog_Linux下Tomcat日志定期清理 及 logrotate 配置

    服务器上的tomcat的catalina.out文件越来越大,查看起来很不方便,以前每次都是想起来的时候手工清理一下(cat /dev/null > catalina.out),后来发现了log ...

  9. 利用curl计算网络文件大小

    ; CURL *handle = curl_easy_init(); curl_easy_setopt(handle, CURLOPT_URL, url); curl_easy_setopt(hand ...

  10. Alice and Bob

    类似于石子合并的游戏,在黑板上写下N个数,每次只能将其中的一个数减1(结果为0自动消去),或者将某两个数消去,将其和写在黑板上. Alice先手,彼此都采用最优策略,将最后一个数消去者获胜. 思路:设 ...