企业生产环境集群稳定性-HA就行吗？

在企业生产中，集群一旦运行，是要尽可能的将损失降到最低，现在所有的大数据技术都有HA，spark的、Hadoop的、HBase的等等，

HA分冷备和热备，热备是集群自带的，冷备就是硬件的。

这样一种情况：晚上了，人睡着了，突然的大数据量来了，nameNode、master崩了，HA启动了另一台备用节点，然而数据量依然很大，另一台也崩了，好吧集群废了？

咋办？

这就是冷备的需要，进程掉了，在服务器运行的情况下：

有一种机制可以启动进程服务，crontab+shell

五分钟之内检查进程情况，没有启动（时间自定义）

################################################################
#!/bin/bash
#该脚本用于监控Linux主机上的进程，由于某些情况进程丢失，而人员没有
#即时发觉得情况
#下面例子是jps->RunJar进程的监管
#在Linux服务器上创建一个crontab任务定时检查
#作者：舆
#时间：2016年
################################################################
ab=`jps | awk '{print $2=="RunJar"}'`
res=
OLD_IFS="$IFS"
IFS=" "
arr=($ab)
IFS="$OLD_IFS"
for s in ${arr[@]}
do
    if [ "$s" = "" ]
    then
       ((res++))
    fi
done
echo $res
if [ $res \>  ]
then
   echo "进程存在"
else
   echo "进程不存在"
   hive --service metastore &
fi

本脚本以hive进程测试成功

aaarticlea/png;base64,iVBORw0KGgoAAAANSUhEUgAAAnoAAAD5CAIAAAA/cw8EAAAaCklEQVR4nO3d2YGrIBSA4fRkOxZjL9ZiKdPHfYhRlrMgaCTe/3uaidtRkCNozOv1k8b572+ZhrvDAADgyUi3AABcjnQLAAAAAAAAAAAAAAAAAAAA9Gqc/1bzeHcsnsInk4dp+YW96VfDE+B7dZJqlD21Zc27O0r/hpPoF5/Sv7deBTPFswzT8pc458j+YhnhUuN8S2pyamJ0/mzzPTbddnZenhCOXQYtJeQte1vpmxse5/WIDtNySnSdVZkiN9arcNPpXJfVmF8sI1yqv3Q7ztH15/vacx7thX5bZzv20+n2NlZg21k2TMs5Jd1ZlSlyW71KP48DId3iW4x0+6mGW1czqjrREExWqdSpUb81H9yRqv722Vp991Wkm1XHi8QVZ2dDFHR6di7L5/PPVsK1mcNZ+kTnaJhRRbsll5HJjso8zl5U96Rbo/TtPUorQvJ/cMEn/u/v1H6Sbb3cMm1lZFJrjn00/O12WK+yRi76oDHdXlhGeBov3S7LEtTLaQwnRedg2h6pU7fPhMoXfrzV0nleP10/WVeVbCWJ228NkxDiKKP/tn/ebcEyDfFxG+c0OQf/561XHrN+GWxEFRwEsYxMZlTucfai6m4w2d4jJ92+ol1Uty9MEO4K6pkk01ZGHrXm+OnW2G6X9SofUog+SUvpSMW8tozwNF66FSfK7ZF2eVz2SfLp/uee46SsXjB0JO+qeNHuxRJHpZw70faLBpS0mfzjppdR1Qb3qcZxLinNHtOtukd+ut0WMbZu5uG9+hTvdlMZ+dSaU5Bui4/koZgvq1fbarY/ohmHaUmiKM6KF5cRnqZgMFmckFSyYF57arBZeQR6u8UVX+Z+0q3XLFpxJz3S5PaNNDK9jQBmSTZeIu/GBOeyf8Vs7Igelb2vNjsq+zgXRNVlunUHSNW5gxUbRWlVO/cSTVtfZRmVrr7kWvrIseqzXjnptnAt6qzXlRGepibdStVGSk7GVg6l28+J2Jpu9SCj+y9Z0tTSbTi8mN7rS1OiOVRlDK3rUTn76jGiKhpOtKJqSbdhXOW3CpzpZ6RbbyTBGmzZ6lHFgG9NGZWu++x02xLzdfXKGUwuXY0x80VlhKf5n3q3+95KY7L2oJCebrNl7XRvXn7nO+KcnmcMUGVRHe+FHIzrF3u3n3ud5aeLlj9qdv1gGRW4Kt1Wx3xhvRKv9vVCqKye55cRnqYm3Tp3WUruwcg1L/g0TnFnpdt1d+V41BPMTrdS6i4eXzJ3xGkVto013w8S+vrGcfajcmcqWkXdshXpNmnopWdatrXqB1yeUnnjNnesjHylF9PJ0Ti03V7qlX+X2Jq73NllhKepSrfSOZg+E6BOlWYJowmz9rsvED6Z3Jhut7Vmy2VR7kN/zmBytGR6h2/MnlrWIhZPRCMqf191dlTucfaicgO7sHtbkW6j/VkLUB+vkKqzHtdY/X6LxjLyOGMw2tFw+7591qtwU8lMcczquSiOTF1cRniaunT72vOKnLzsqekcaqbOY/UazZScVv2Q5CZGTLfpvkjX0uahUI+GGdU+sfpKXFlpSTNhRpXOoV1h2MdEJS5rln5ZQ/9ZJO/7CqPExq2QVzyrfMuhRGMZmbwxIOVoHDiSndWrIDDzDDPuQ2k3gq4qIzxN4xDXBZwnUoAunTOuD+Cx9ouzrlqKqK/CFSG61ulJBAAAAAAAAAAAAAAAAAAAAAAAHkX/6jcAADiD8WIzAABwhmOv7QYAABWytzf29zpHAAB+3NGfXAYAAIdtY8fbH9y+BQDgZKRbAAAux2AyAADX41EpAAAuxxeBAAD4Al5zAQDAN/ASRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHwBr7kAAOBavMQRAICL8RMFAABcjh/gAwDgavy8PAAAl9vGjrc/uH0LAMDJSLcAAFyOwWQAAK7Ho1IAAFyOLwIBAPAFvOYCAIBv4CWOAAAAAAAAAAAAAAAAAAAAAAAAAAAAwGsYx/w1UuO8LPP+9otlmbxv5F74joxhWrwVD9NS/bXhlmUBACgkJcpx3j8ZpkV6seM7S23evyw0junngX0bwZs1ilkvl1R/WaHgHdD8KgMA4EuydyUXpFt55vzfsnUocQiUdD5P0ufz+HqN87bKbNllmpzMT68XAHCygk6nlAzPSreV/cyWXzEapoWECgC4i9u7DfqJ8xxk6XnUknZxr/VYxh3nv2WehY5twWbjZDtMC7+HBAA4XTy0Wj2YnPQuxQ5jyYD03zyO8xpGWY91nPV8+hmXHgZts9vGrMABADhLnmgOpNu1M+t1C6Uh5nAV70etwgzo3siVZximabQ29p5H6MgmR0F8YhsAgGproil7XjjOY8O0LMvyN4/r883qOubxNc5SDgw7smmH0xjgXTcsD3PbuX//5o/+FLW4swAANMh6t1GH1urdjvM7xc7jK3oEOJhqbTgdM87S7Uv8Wuwno0qzp1sVhrqXaVQGvEmvAIDrpIkmTrB6ul3fP6Fm1WBClorF12LI+TNKmWHn9T273UV9BziP0aLa/WXSLQDgOtkTuukjRMpw7vreqSjdal+LHbfZX2paVT8PNhreUS3p3cprId0CAL7tfb/186yS9AiRmNS2D0t6t8WRHPkmDukWAPAb9nFa9dXBw2S/MFlPU32nW/E1VDwtBQA4m5CuCp5OTvPPlm7d53zdrwsdfEFUHH8QelXvFgCAjukvkrjaOAp98duiAQAAAAAAAAAAAAAAAAAAAAAAAC62f2O1/y+juj/L9ya+khnlCo+ztqj+FWh7asuad3eU/nUnUfxd+J4q9ZePs1AnpQhuiOpPeAFATwXVlX7r83ccfLvEmZs1WvSoVMI3Rj4z3TbktyucEI5dBi0l5C17W+mbG97eynLgLSvpL1COc95CXVdzvDWTbl/5D35/2q1j7+u5qQS/raQ+P11/6TYpheAFk73Vn9N0tmM/nW5vYwW2nWX+LzJHy+TvMJ96SbdfVphuv2yc//7meXvB3TAtyzwfi+p/LkH3JcGPY6TbT3XeuprpjwUJHVB3qvimyCS5alesa4HtqzBeQym9bjL5MCv/KOjkd3OXZfl8Ll3CmkOd+kTnaJhRRbsll5HJjso8zl5U96Rbo/TtPUorQvJ/+kZx5Q3jxk7tJ9mBN4PbrWVBzbGH39Wa4665+jgnS79/e7qwHrjp9qJz349q/9HvYVqEfbJalZYSbGpVClpvvVWpP1Ze3a9r65rLt6EVPchLt8uybJP3a5FkXEDqkKpTt8/knxjIT/t5Xj+Nb4ykYxNx3H5rmIQQRxn9t/3zLrPt95OC6elptP+ft+VyEySXsBFVcBDEMjKZUbnH2Yuqu8Fke4+cdPuKdlHdvjDBeJO4v/frwvaMas0x6+S2eqPmVA0mezUnWujQfc6mweSGc9+P6t1Mz+N6n0C4CDDXXFeCTa2K2z5bdaP6WLn1ub6tay3f2lb0OC/dihPl9ijMk/pU7ZPk0/3PPcdJWV0IT/s83tU4AjPC/c84KqU1ytoU91pJm8k/bnoZVW1wn2oc55LS7DHdqnvkp9ttEWPrZh7eq8+B3U6ydVkNMVaVNzFO41eXbotrzpHKK/bY8qVPP/f9qObxnRqn9135MIL6ljDXR6vScKyc+tzW1rWVb2UrWsFLt3b7Ic1rTw02q45hSONEn3TrNYtW3EmPNFhWSp773FKSjZfIuzGf7bT0Ueyo7H212VHZx7kgqi7Trb5HZfXqU8batq1q516imcJUU9o3suqkHa6/Zn35gzXnQD1o6t2+6s99P6p5fH2KaBsBU1vCfM1VJXhyq5IetSMt/9FryJdcn1vbuqbybWmPjqlJt1I5SsnJ2MqhdPs5YK3pVg/SvnbW0m04vJhetKeFqbWX1o4UXNHXVxQjKj85OVG1pNswrvJbBc70M9Ktdw1sDbZs9agm28ZbKOtbeHXylnSbrfKL6bb23PejShqsIIKiNdeWYFOrUpt+Wo5VLtzF5raupXz7Tre/2ruNBn+y0QR7cFVPtweu2fdrYG0D2Y447fMZFSWL6njv9mBcv9i7fbcNi77xbMPaeV5fYIXnUVGd/M96t7Xnvh+VHkHRmutLMFxDW6vS1LttcOxYObWloXz7TrfOaHjJ+L5cFYJP4xR3Vrpdd1eOx7xw0tPtoRsB7n4L0SpR7RtrrijC9aBxnItGj+yZjg9AFS/bngaETuS+Vv2Ay1P2dR/dZ+FxjdLzqKRO3pBu03+d4QIvnIPptu7c96MyIihZc30JeuuwWhW7FpWMERzm1Of2tq6+fDtPt2mLNM5RVbCnSrOE0YTH/90XCJ9Mbky321qz5bIo96E/ZzA5WvIzxhPcRzD6jfbRsKPy91VnR+UeZy8qN7ALu7e1aeCzzFqAes9Cqs56XGPF+y2C9aXFJB5m7T6EVieNcP01m8t7NSc/kF9Mt1Xnvh+VGYG/5qoSbGtV3PbZu2ipOlZufW5u66rLt/d0+9prgLyD9tR0DjVT57F6jWZKPvR+SPJgo5hu031Jj1sYl7Zr8tEwo9onVlUUI6qSyxozqnQOrS2wj4lKXNYs/bILiM8ied9XGCWWq7yUf+RbDgd3VKoZ2TzhdLtOltQcec1txzk+0mP5QXHS7WXnvh+Vl/DdNdeVYFurUtB6O1e0h49VtqAyFqSu2a+xteXbRbq9yZEhJqAX3ztnH8MbsAaeZb9I6qqliK5WOSPRtU5Pog6Fr2PwhqsBAEClcFSPXAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABcYJz/VvN4dyyecf77W6bBm22Yll/Ym34VHmdt0T+1RtlTW9a8u6P0f+kkAnCXcb6ljXBa9Kht3eZ7bLptyG9XOCEcuwxaSshb9rbSNzc8zusRHablt+omgLP0l27HOeolDNPy+b+zrHSeznbsp9PtbazAtrNsmJaeShrAFxnp9tN+bF3NqJ14p0Fpijk16rfmQ3BSm7V9tqaBfRXpZoOVp/skrDjLKlHQwbzj/Pe3LMvn889WwrWZQ536ROdomFFFuyWXkcmOyjzOXlT3pFuj9O09SitC8n9wwSf+7+/UfpJtvVwA/x0v3S7Lsk0epmkMJ23thtQhVadunwntTvjx1jbO8/rp+sm6qmQrSdx+a5iEEEcZ/bf9825ol2mIj9s4p8k5+D9vy/OY9e6kEVVwEMQyMplRucfZi6q7wWR7j5x0+4p2Ud2+MCG6JjGvqQA8n5duxYlyexTmSX2q9kny6f7nnuOkrC6Ep30e72ocgRnh/mccldJPibZfNDCrzeQfN72Mqja4TzWOc0lp9phu1T3y0+22iLF1Mw/v1Yc8C/yvCgaTxQlJcxTMa08NNiuPQG+3uOLu0yfdes2iFXfSIw2WlZJnNAKYJdl4ibwb89nOOsVuZI0d0aOy99VmR2Uf54Kouky3+h6V1atPGWvbtqqde4kG4Plq0q3UHEnJydjKoXT7aaZa060epHgP9c9Lt+HwYnqvL02J5iiiMbTuDETWJy4jKj85OVG1pNswrvJbBc70M9KtN5JgDbZw4xbAf9W73fdWGpO1B1f1dJst6z3Mk0860Lst39diWVTHe7cH4/rF3u3nLnn56aJdlzCgDPyfatLtnfdu29LturtyPGojaKfbQ7dT3f0WolWi2jfW3HoLfX3jOBfdgbRnarmJ6SxbkW6TCwjpGbxtrfoB15/Q48YtgFdlum1/Mll5rDjN2u++QPhkcmO63daaLZdFuQ/9OYPJ0ZLpHb4xe2pZi1jsMRpR+fuqs6Nyj7MXlRvYhd3binQb7c9agPp4hVSd9bhG3m8B4K0u3b72vCInL3tqOoeaqfNYvUYzJadVPyR5sFFMt+m+SE/ymodCPRpmVPvEqlbciKr4SV1zt4I51Buh5jFRicuapV92AfFZJO/7CqPExq2QVzyrfMsBwP+mvyEu54kUoEvnjOsDeKz9sr6rliLqq9ArQNc6PYkAAAAAAAAAAAAAAAAAAAAAAMDD/NJ3GLR3HSb4AmSjwuOsLaq9r8Ob2rLm3R2lf8NJ1FBGt7m3XgUzxbPkP+d10pH9xTLCpW56zYVTE+Wv3T423XZ2Xp4QTmc/UfAN3rtDz32XY2dVpsiN9SrctPiLXVfUmF8sI1yqv3QrvYBZ/jGAx+hsx3463d7G/2WM14nvcuysyhS5rV5Jr1VN3tJJusUXFLwzeetq5q9tVwdf1Kniq22T5Cq9NDj42bzo/bbZzihDSsKKs7MhCjo9O5fl8/lnK+qPrUkvzpUnOkfDjCraLbmMTHZU5nH2oron3Rqlb++R80bl9K2iyltGC373UfotB1NbGZnUmmMfDX+7HdarrJGLPmhMtxeWEZ7GS7fLsgT1cgrfyd/yi0DKhZ//i0D7qtRfM9B/CS3LkMqvwiT/bf/svwUYHbf893WC//PWq/wH+MyogoMglpHJjMo9zl5U3Q0m23vk/yRDsIvq9oUJwl1BPZNk2srIo9YcP90a2+2yXuVDCtEnaSkdqZjXlhGexku34kS5PdIuj8s+ST7d/4x/79bpCVufx7sqXrR7scRRKedO9nNt7jmmzeQft8rfc7Cjco5zSWn2mG7VPfLT7baI96OUWh7eq0/xbjeVkU+tOQXptvhIHor5snq1rWb7I/09r/RquTQrXlxGeJqCwWRxQlLJgnntqcFmjZ8wixf4zFzSLFpxJz3S5PaNNDK9jQBmSTZeIu/GBOeyf8Vs7Igelb2vNjsq+zgXRNVlunUHSNW5gxUbRWlVO/cSTVtfZRmVrr7kWvrIseqzXjnptnAt6qzXlRGepibdStVGSk7GVg6l28+J2Jpu9SCj+y9Z0tTSbTi8mN7rS1OiOVRlDK3rUTn76jGiKhpOtKJqSbdhXOW3CpzpZ6RbbyTBGmzZ6lHFgG9NGZWu++x02xLzdfXKGUwuXY0x80VlhKf5n3q3+95KY7L2oJCebrNl7XRvXn7nO+KcnmcMUGVRHe+FHIzrF3u3n3ud5aeLlj9qdv1gGRW4Kt1Wx3xhvRKv9vVCqKye55cRnqYm3Tp3WUruwcg1L/g0TnFnpdt1d+V41BPMTrdS6i4eXzJ3xGkVto013w8S+vrGcfajcmcqWkXdshXpNmnopWdatrXqB1yeUnnjNnesjHylF9PJ0Ti03V7qlX+X2Jq73NllhKepSrfSOZg+E6BOlWYJowmz9rsvED6Z3Jhut7Vmy2VR7kN/zmBytGR6h2/MnlrWIhZPRCMqf191dlTucfaicgO7sHtbkW6j/VkLUB+vkKqzHtdY/X6LxjLyOGMw2tFw+7591qtwU8lMcczquSiOTF1cRniaunT72vOKnLzsqekcaqbOY/UazZScVv2Q5CZGTLfpvkjX0uahUI+GGdU+sfpKXFlpSTNhRpXOoV1h2MdEJS5rln5ZQ/9ZJO/7CqPExq2QVzyrfMuhRGMZmbwxIOVoHDiSndWrIDDzDDPuQ2k3gq4qIzxN4xDXBZwnUoAunTOuD+Cx9ouzrlqKqK/CFSG61ulJBAAAAAAAAAAAAAAAAAAAAAAAHmX7gnfFFxj0r40DAICP9Q0XVV/PN16KBgAAMhXZ8tgrvwEAQEW6zd782N+rIAEA6MrxdHv055oBAPjvHU+329jx9ge3bwEAMJFuAQC4HIPJAABcjkelAAC4HF8EAgDgcrzmAgCA62xvcNwd6aLyEkcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANBumJb0Fcnp7/pE71QOJ6Qz8oNAAAAo7KSZpONxDn6JgHQLAECpOEta/yWfkG4BACgWpMmkMztMS5pAg1+1Jd0CAFBuy6rpL8RLCXScSbcAAFRY02yabendAgBwovcg8pJkW+febZKMhYecAQBAYP22T5YtrSeTo/+0FQAAgJ06FKx+73ZdaJvAYDIAADZyJQAAF8sekgIAACf6jAeTbAEAAAAAAAAAAAAAAAAAAAC02d5kwePJAABcYv2JH756CwDA5Ui3AABcjnQLAMDlSLcAAFyOdAsAwOVItwAAXI50CwDA5Ui3AABcjnQLAMDlSLcAAFxne4PjbpmGu6MCAAAAAAAAAAAAAAAAAAAAAAAAAAAAvm0Yx/x7tuO8LPP6/dthWpZl8l6B0f6ijPdXgOtW0bIsAABfICXKcd4/GaZFevNF8oKMZRper3EcpRdnrOxsOEyL/IaNcXYTqbosAAD9GOfkZVIF6VaeOf9XWoeSkudJ+nweX69x3hbPll2maVYSfFGeBwDgq0Ynbf3Jb3g8nm7FLdcmxWFaSKgAgF/k9m6DPuY8B1l6HrWkbaTbcf5b5lno2PqLJsl2mBZe+gwA6Eo8LFs9mJz0TMXOpr6Ocdbz6WeMexi0zY9zvCw9XQBAt/IkdSDdrp1Zr0spDTG/hLvG6yanUVkgCjpbNNkT8alrAABusSapglu3f+kt1mFalmX5m8f1+WZ1HfP4Guckf64Ly0PVdv7ev/mjPwktBgwAwE2y3m3UobV6t+P8TrHz+IoeHw6mGttcpiEbDpaWFIarl2lUBq1JrwCAPqVJKk6werodpuX9LR0lxQUTolQcdl7f6dbuor43Mo/Roto9YtItAKBP2dO96eNHYrodpvW9U1G61b5SO26zx3dUS3q3BUFbHwIA0IH3/db3a6GEJ5fkdBt0V0t6t9bGSbcAgIfbb4uqrx0eJvuFyXqKuz7diq+h4mkpAEBPhFRX8HRymru2dOs+Iyx95yeOIdh8Ve8WAICH0l9CUWIchf500xoBAAAAAAAAAAAAAAAAAAAAAACAh9q/7dr/F1nlX+zLrD9Q9I2InqnwOGuL6l+ftqe2rHl3R+nfeRI9prbfW+vUb/0LbxSojFLY4jlrQt/+AQJ5syq64NW2AAAAAElFTkSuQmCC" alt="" />

是想，生产环境下，上述情况是不是可以缓解，脚本写日志文件处理是不是也可以。