转 http://blog.csdn.net/u012303571/article/details/46913471

 
查看 nodemanager 日志发下 如下信息
 
2015-07-16 15:28:58,643 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed: 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,645 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
2015-07-16 15:28:58,658 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2015-07-16 15:28:58,711 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
 
解决方法 
 删除部分没用的数据  让磁盘已用容量下降到90%一下
 
 
2015-07-16 15:29:06,808 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests
2015-07-16 15:38:58,817 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) turned good: 1/1 local-dirs are good: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are good: /opt/beh/logs/yarn/nmlogs

Hadoop YARN ERROR 1/1 local-dirs are bad *， 1/1 log-dirs are bad *的更多相关文章

  1. 传统应用迁移到kubernetes（Hadoop YARN）

    spark-on-yarn-with-kubernetes 该例子仅用来说明具体的步骤划分和复杂性,在生产环境应用还有待验证,请谨慎使用. 过程中可能用到的概念和术语初步整理如下: 整个迁移过程分为如 ...

  2. YARN分析系列之二 -- Hadoop YARN各个自模块说明

    先做如下声明,本代码版本是基于 3.1.2 版本. 其实,我们自己在写代码的时候,会有意识地将比较大的功能项独立成包,独立成module, 独立成项目,项目之间的关系既容易阅读理解,又便于管理. 如下 ...

  3. Hadoop YARN：调度性能优化实践(转)

    https://tech.meituan.com/2019/08/01/hadoop-yarn-scheduling-performance-optimization-practice.html 文章 ...

  4. File file:/data1/hadoop/yarn/local/usercache/hp/appcache/application_* does not exi

    AM Container for appattempt_1453292851883_0381_000002 exited with exitCode: -1000For more detailed o ...

  5. Hadoop YARN: 1/1 local-dirs are bad: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir; 1/1 log-dirs are bad: /var/log/hadoop-yarn/containers hdfs硬盘90% yarn unhealthy

    1/1 local-dirs are bad: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir; 1/1 log-dirs are bad: /var/log ...

  6. Hadoop Yarn配置项 yarn.nodemanager.resource.local-dirs探讨

    1. What is the recommended value for "yarn.nodemanager.resource.local-dirs"? We only have ...

  7. hadoop+yarn+hbase+storm+kafka+spark+zookeeper)高可用集群详细配置

    配置 hadoop+yarn+hbase+storm+kafka+spark+zookeeper 高可用集群,同时安装相关组建:JDK,MySQL,Hive,Flume 文章目录 环境介绍 节点介绍 ...

  8. Hadoop YARN配置参数剖析—RM与NM相关参数

    注意,配置这些参数前,应充分理解这几个参数的含义,以防止误配给集群带来的隐患.另外,这些参数均需要在yarn-site.xml中配置. 1.    ResourceManager相关配置参数 (1) ...

  9. hadoop错误org.apache.hadoop.yarn.exceptions.YarnException Unauthorized request to start container

    错误: 14/04/29 02:45:07 INFO mapreduce.Job: Job job_1398704073313_0021 failed with state FAILED due to ...

随机推荐

  1. oracle 行专列

    首先,做准备工作. 建表 -- Create table create table DEMO ( n_iden NUMBER, c_order_code NVARCHAR2(), c_order_na ...

  2. TortoiseGit HTTPS方式保存密码最简单的方法

    在TortoiseGit的设置 -> git 中选择 编辑本地 .git/config 在最后增加下面内容: [credential] helper = store

  3. jquery拖拽效果

    <!doctype html><html lang="en"><head> <meta charset="utf-8" ...

  4. Docker私有仓库1

    一.搭建私有仓库 #环境 Ip 192.168.0.34 X86云主机 [root@lh- /]# uname -a Linux lh- -.el7.x86_64 # SMP Thu Mar :: U ...

  5. [ An Ac a Day ^_^ ] hrbust 2291 Help C5 分形

    开博客这么久从来没写过自己学校oj的题解 今天写一篇吧 嘿嘿 原题链接:http://acm.hrbust.edu.cn/index.php?m=ProblemSet&a=showProble ...

  6. 浅谈SharePoint 2013 站点模板开发 转载自http://www.cnblogs.com/jianyus/p/3511550.html

    一直以来所接触的SharePoint开发,都是Designer配合Visual Studio,前者设计页面,后者开发功能,相互合作,完成SharePoint网站开发.直到SharePoint 2013 ...

  7. 安装mcrouter

    下载准备: mcrouter 下载地址 : https://github.com/facebook/mcrouter gflags 下载地址:http://download.csdn.net/deta ...

  8. 带宽 VS CDN （转载）

    并发的影响因素:带宽.web server(含php).static server.数据库 带宽价格: 静态IP企业宽带 5M 10M 50M 100M 200M 盛大云 华东双线 216/月 396 ...

  9. A - 小Y上学记——修学分

    A - 小Y上学记——修学分 Time Limit: 2000/1000MS (Java/Others)    Memory Limit: 128000/64000KB (Java/Others) S ...

  10. ios日期时间

    //1.获取当前时间 -(NSString*)getCurrentTime { NSDateFormatter *formatter = [[NSDateFormatter alloc]init]; ...