Hadoop 从节点的 NodeManager 无法启动
一、问题描述
日志文件信息如下:
-- ::, INFO nodemanager.NodeManager (LogAdapter.java:info()) - registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO recovery.NMLeveldbStateStoreService (NMLeveldbStateStoreService.java:openDatabase()) - Using state database at /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state for recovery
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:)
at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:)
at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.openDatabase(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.initStorage(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMStateStoreService.serviceInit(NMStateStoreService.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:)
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service NodeManager failed in state INITED; cause: org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:) .......
二、解决办法一
i. 删除 yarn-nm-state 目录的所有文件再次启动
[root@zwlbs3 ~]# cd /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/
[root@zwlbs3 yarn-nm-state]# ls
.log CURRENT LOCK MANIFEST-
[root@zwlbs3 yarn-nm-state]# rm -rf *
ii. 再次启动 NodeManager 组件
我这里使用的是 Ambari 来管理的,直接就Web界面操作,也可以使用命令操作启动。
iii. 查看是否启动成功,已启动成功
[root@zwlbs3 ~]# jps
DataNode
Jps
HRegionServer
NodeManager
三、解决办法二(如果办法一无效,试试办法二)
i. 如果过一会变成 stop 状态,报错日志跟上面相同,如下:
-- ::, INFO nodemanager.NodeManager (LogAdapter.java:info()) - registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO recovery.NMLeveldbStateStoreService (NMLeveldbStateStoreService.java:openDatabase()) - Using state database at /var/log/hadoop-yarn/nodema
nager/recovery-state/yarn-nm-state for recovery
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateSt
oreService failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/
MANIFEST-: No such file or directory
org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-: No such file or directory
at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:)
at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:)
at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.openDatabase(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.initStorage(NMLeveldbStateStoreService.java:)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMStateStoreService.serviceInit(NMStateStoreService.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:)
-- ::, INFO service.AbstractService (AbstractService.java:noteFailure()) - Service NodeManager failed in state INITED; cause: org.apache.hadoop.servic
e.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/MANIFEST-:
No such file or directory
org.apache.hadoop.service.ServiceStateException: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm
-state/MANIFEST-: No such file or directory
.......
ii. 删除 yarn-nm-state 目录的所有文件
[root@zwlbs3 ~]# cd /var/log/hadoop-yarn/nodemanager/recovery-state/yarn-nm-state/
[root@zwlbs3 yarn-nm-state]# ls
.log CURRENT LOCK MANIFEST-
[root@zwlbs3 yarn-nm-state]# rm -rf *
iii. 重启该服务器的整个Hadoop
命令行操作请自行到 Hadoop 目录下重启所有组件即可。
Hadoop 从节点的 NodeManager 无法启动的更多相关文章
- Hadoop 2.x 安装常见问题FAQ(一) NodeManager 无法启动问题解决
一.问题描述 在搭建 Hadoop hadoop-2.4.1 集群的最后一步启动集群,在命令窗口并没有报任何错误,但是Slave 节点的 NodeManager进程始终启动不起来.随后查看了后台启动日 ...
- Hadoop的多节点集群详细启动步骤(3或5节点)
版本1 利用自己写的脚本来启动,见如下博客 hadoop-2.6.0-cdh5.4.5.tar.gz(CDH)的3节点集群搭建 hadoop-2.6.0.tar.gz的集群搭建(3节点) hadoop ...
- Hadoop Datanode节点无法启动(All directories in dfs.data.dir are invalid)
Hadoop Datanode节点无法启动(All directories in dfs.data.dir are invalid) java.io.IOException: All director ...
- nodemanager 无法启动报错“doesn't satisfy minimum allocations”
在启动nodemanager节点nodemanager时候报错 2019-03-29 14:15:44,648 INFO org.apache.hadoop.yarn.server.nodemanag ...
- HBase的多节点集群详细启动步骤(3或5节点)(分为Zookeeper自带还是外装)
HBase的多节点集群详细启动步骤(3或5节点)分为: 1.HBASE_MANAGES_ZK的默认值是false(zookeeper外装)(推荐) 2.HBASE_MANAGES_ZK的默认值是tru ...
- Hadoop多节点Cluster
Hadoop多节点集群规划 服务起名称 内网IP HDFS YARN master 192.168.1.155 NameNode ResourceManager slave1 192.168.1.11 ...
- 实验-hadoop新增节点
关于hadoop新增节点网上的说法都有些差别,自己来实践一把 1.建立一个namenode一个datanode的集群 master:192.168.126.130 slave1:192.168.126 ...
- hadoop集群之Datenode无法启动解决办法
hadoop集群之Datenode无法启动解决办法 我们在启动hadoop集群的时候,通过jps查看进程,发现namenode RM和Secondary NameNode都有,但datanode没有启 ...
- Hadoop添加节点datanode(生产环境)
Hadoop添加节点datanode 博客分类: hadoop HadoopSSHJDKXML工作 1.部署hadoop 和普通的datanode一样.安装jdk,ssh 2.修改host ...
随机推荐
- PyCharm颜色设置
选择主题和背景图片 选择字体.修改字体大小 新建颜色主题 修改背景颜色 修改注释颜色 File --> Setting (Ctrl + Shift + S) 1.选择不同的主题.选择背景图片 A ...
- [Linux] 使用mount来挂载设备到目录
一般情况下直接mount 设备路径 目录路径,就可以了.umount 设备名,就可以卸载这个设备了使用lsblk -f可以查看挂载的设备,以及这些设备的文件系统. root@tao-PC:/boot# ...
- 【bzoj2648】SJY摆棋子(kdtree)
传送门 题意: 二维平面上有若干个点. 现在要维护一种数据结构,支持插入一个点以及询问其余点到某个点的最小曼哈顿距离. 思路: 这是个\(kdtree\)模板题. \(kdtree\)是一种可以高效处 ...
- 4.web基础$_POST
- 解决Python开发中,Pycharm中无法使用中文输入法问题
Pycharm是开发Python程序的利器,但有时会遇到无法输入中文的情况.表现为:在Ubuntu系统可以正常输入中文,却在Pycharm内写注释的时候,切换不出中文.下面演示如何解决此问题. 1.在 ...
- 图像检索——VLAD
今天主要回顾一下关于图像检索中VLAD(Vector of Aggragate Locally Descriptor)算法,免得时间一长都忘记了.关于源码有时间就整理整理. 一.简介 虽然现在深度学习 ...
- Python爬取信息管理系统计算学分绩点
试手登录了下我们学校的研究生信息管理系统,自动计算学分绩点 # -*- coding:utf-8 -*- import urllib import urllib2 import re import c ...
- Leetcode练习题Remove Element
Leetcode练习题Remove Element Question: Given an array nums and a value val, remove all instances of tha ...
- jQuery 源码解析(六) $.each和$.map的区别
$.each主要是用来遍历数组或对象的,例如: var arr=[11,12,13,14]; $.each(arr,function(element,index){ //遍历arr数组 console ...
- Missing artifact com.lowagie:itextasian:jar:2.1.7
在maven中添加iTextAsian支持 导入maven项目 发现pom.xml文件 中这个依赖报错:Missing artifact com.lowagie:itextasian:jar:2. ...