Hadoop 从节点的 NodeManager 无法启动
一、问题描述
日志文件信息如下:
-- ::, INFO nodemanager.NodeManager (LogAdapter.java:info()) - registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO recovery.NMLeveldbStateStoreService (NMLeveldbStateStoreService.java:openDatabase()) - Using state database at /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state for recovery
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:)
at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:)
at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.openDatabase(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.initStorage(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMStateStoreService.serviceInit(NMStateStoreService.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:)
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service NodeManager failed in state INITED; cause: org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:) .......
二、解决办法一
i. 删除 yarn-nm-state 目录的所有文件再次启动
[root@zwlbs3 ~]# cd /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/
[root@zwlbs3 yarn-nm-state]# ls
.log CURRENT LOCK MANIFEST-
[root@zwlbs3 yarn-nm-state]# rm -rf *
ii. 再次启动 NodeManager 组件
我这里使用的是 Ambari 来管理的,直接就Web界面操作,也可以使用命令操作启动。
iii. 查看是否启动成功,已启动成功
[root@zwlbs3 ~]# jps
DataNode
Jps
HRegionServer
NodeManager
三、解决办法二(如果办法一无效,试试办法二)
i. 如果过一会变成 stop 状态,报错日志跟上面相同,如下:
-- ::, INFO nodemanager.NodeManager (LogAdapter.java:info()) - registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO recovery.NMLeveldbStateStoreService (NMLeveldbStateStoreService.java:openDatabase()) - Using state database at /var/log/hadoop-yarn/nodema
nager/recovery-state/yarn-nm-state for recovery
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateSt
oreService failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/
MANIFEST-: No such file or directory
org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:)
at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:)
at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.openDatabase(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.initStorage(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMStateStoreService.serviceInit(NMStateStoreService.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:)
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service NodeManager failed in state INITED; cause: org.apache.hadoop.servic
e.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-:
No such file or directory
org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm
-state/MANIFEST-: No such file or directory
.......
ii. 删除 yarn-nm-state 目录的所有文件
[root@zwlbs3 ~]# cd /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/
[root@zwlbs3 yarn-nm-state]# ls
.log CURRENT LOCK MANIFEST-
[root@zwlbs3 yarn-nm-state]# rm -rf *
iii. 重启该服务器的整个Hadoop
命令行操作请自行到 Hadoop 目录下重启所有组件即可。
Hadoop 从节点的 NodeManager 无法启动的更多相关文章
- Hadoop 2.x 安装常见问题FAQ(一) NodeManager 无法启动问题解决
一.问题描述 在搭建 Hadoop hadoop-2.4.1 集群的最后一步启动集群,在命令窗口并没有报任何错误,但是Slave 节点的 NodeManager进程始终启动不起来.随后查看了后台启动日 ...
- Hadoop的多节点集群详细启动步骤(3或5节点)
版本1 利用自己写的脚本来启动,见如下博客 hadoop-2.6.0-cdh5.4.5.tar.gz(CDH)的3节点集群搭建 hadoop-2.6.0.tar.gz的集群搭建(3节点) hadoop ...
- Hadoop Datanode节点无法启动(All directories in dfs.data.dir are invalid)
Hadoop Datanode节点无法启动(All directories in dfs.data.dir are invalid) java.io.IOException: All director ...
- nodemanager 无法启动报错“doesn't satisfy minimum allocations”
在启动nodemanager节点nodemanager时候报错 2019-03-29 14:15:44,648 INFO org.apache.hadoop.yarn.server.nodemanag ...
- HBase的多节点集群详细启动步骤(3或5节点)(分为Zookeeper自带还是外装)
HBase的多节点集群详细启动步骤(3或5节点)分为: 1.HBASE_MANAGES_ZK的默认值是false(zookeeper外装)(推荐) 2.HBASE_MANAGES_ZK的默认值是tru ...
- Hadoop多节点Cluster
Hadoop多节点集群规划 服务起名称 内网IP HDFS YARN master 192.168.1.155 NameNode ResourceManager slave1 192.168.1.11 ...
- 实验-hadoop新增节点
关于hadoop新增节点网上的说法都有些差别,自己来实践一把 1.建立一个namenode一个datanode的集群 master:192.168.126.130 slave1:192.168.126 ...
- hadoop集群之Datenode无法启动解决办法
hadoop集群之Datenode无法启动解决办法 我们在启动hadoop集群的时候,通过jps查看进程,发现namenode RM和Secondary NameNode都有,但datanode没有启 ...
- Hadoop添加节点datanode(生产环境)
Hadoop添加节点datanode 博客分类: hadoop HadoopSSHJDKXML工作 1.部署hadoop 和普通的datanode一样.安装jdk,ssh 2.修改host ...
随机推荐
- docker镜像导入导出备份迁移
导出: docker save -o centos.tar centos:latest #将centos:latest镜像导出为centos.tar文件 导入: docker load -i cent ...
- C++踩坑——用memset对vector进行初始化
在一段程序中,使用memset对vector进行了初始化,然后得到了错误的结果.找这个bug花费了很长时间. vector中有其自身的结构,不能单纯的按字节进行初始化.使用memset对vector进 ...
- G1 垃圾收集器架构和如何做到可预测的停顿(阿里)
CMS垃圾回收机制 参考:图解 CMS 垃圾回收机制原理,-阿里面试题 CMS与G1的区别 参考:CMS收集器和G1收集器优缺点 写这篇文章是基于阿里面试官的一个问题:众所周期,G1跟其他的垃圾回收算 ...
- 炫酷的CSS3响应式表单
原创YouTube@ Online Tutorials css代码: * { margin: 0; padding: 0; box-sizing: border-box; font-family: ' ...
- thinkphp的运行
打开cmd切换到www目录下运行think E:\wamp64\www>php think run ThinkPHP Development server is started On <h ...
- nginx 一键安装
#!/bin/bash1. 关闭防火墙 和 selinux service iptables stopsetenforce 0 2. 判断是否有nginx服务netstat -ntlp |grep n ...
- ASP.NET开发实战——(十三)ASP.NET MVC 与数据库之EF实体类与数据库结构
大家都知道在关系型数据库中每张表的每个字段都会有自己的属性,如:数据类型.长度.是否为空.主外键.索引以及表与表之间的关系.但对于C#编写的类来说,它的属性只有一个数据类型和类与类之间的关系,但是在M ...
- 10.12 csp-s模拟测试70 木板+打扫卫生+骆驼
T1 木板 求$[\sqrt{n},n)$间有多少个数的平方是n的倍数 通过打表可以发现(我没带脑子我看不出来),符合条件的数构成一个等差数列,公差为首项 而首项就是将n质因数分解后每个质因数出现次数 ...
- 全国省市区 json sql 支付宝 2019年9月29日
数据来源: https://docs.alipay.com/isv/10327 链接:https://pan.baidu.com/s/14cQCRzU6QxAvknDrQfhyUA 提取码:baaf ...
- github上方便的小工具
目录 python中的fire模块 Install Reference python中的fire模块 它可以对所有Python 对象,包括functions, classes, modules, ob ...