hadoop job -kill 与 yarn application -kii(作业卡了或作业重复提交或MapReduce任务运行到running job卡住)
问题详情


解决办法
[hadoop@master ~]$ hadoop job -kill job_1493782088693_0001
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0001 [hadoop@master ~]$ hadoop job -kill job_1493782088693_0002
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0002 [hadoop@master ~]$ hadoop job -kill job_1493782088693_0003
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
Killed job job_1493782088693_0003
有时候上述这样kill做下来,并不管用,得再来
[hadoop@master ~]$ yarn application -kill application_1493782088693_0001
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0001 has already finished
[hadoop@master ~]$ yarn application -kill application_1493782088693_0002
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0002 has already finished
[hadoop@master ~]$ yarn application -kill application_1493782088693_0003
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Application application_1493782088693_0003 has already finished
[hadoop@master ~]$

[hadoop@master ~]$ hadoop job -list
DEPRECATED: Use of this script to execute mapred command is deprecated.
Instead use the mapred command for it. // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:
Total jobs:
JobId State StartTime UserName Queue Priority UsedContainers RsvdContainers UsedMem RsvdMem NeededMem AM info
[hadoop@master ~]$
同时,大家要注意,是否是进程的消失?

也会是你的slave1 还是 slave2的进程自动消失了。注意 ,这是个很隐蔽的问题。
重新停止集群,再重新启动集群。
如果还是出现这个问题的话,则


关于这里参数配置的解释,见
Hadoop YARN配置参数剖析(2)—权限与日志聚集相关参数
注意,对master、slave1和slave2都要操作,然后,再[hadoop@master hadoop-2.6.0]$ sbin/stop-all.sh
再[hadoop@master hadoop-2.6.0]$ sbin/start-all.sh即可。



成功!
hadoop job -kill 与 yarn application -kii(作业卡了或作业重复提交或MapReduce任务运行到running job卡住)的更多相关文章
- hadoop job -kill 和 yarn application -kill 区别
hadoop job -kill 调用的是CLI.java里面的job.killJob(); 这里会分几种情况,如果是能查询到状态是RUNNING的话,是直接向AppMaster发送kill请求的.Y ...
- yarn application -kill application_id yarn kill 超时任务脚本
需求:kill 掉yarn上超时的任务,实现不同队列不同超时时间的kill机制,并带有任务名的白名单功能 此为python脚本,可配置crontab使用 # _*_ coding=utf-8 _*_ ...
- Hadoop(七)YARN的资源调度
一.YARN 概述 YARN 是一个资源调度平台,负责为运算程序提供服务器运算资源,相当于一个分布式的操 作系统平台,而 MapReduce 等运算程序则相当于运行于操作系统之上的应用程序 YARN ...
- spark-shell启动报错:Yarn application has already ended! It might have been killed or unable to launch application master
spark-shell不支持yarn cluster,以yarn client方式启动 spark-shell --master=yarn --deploy-mode=client 启动日志,错误信息 ...
- yarn application ID 增长达到10000后
Job, Task, and Task Attempt IDs In Hadoop 2, MapReduce job IDs are generated from YARN application I ...
- spark利用yarn提交任务报:YARN application has exited unexpectedly with state UNDEFINED
spark用yarn提交任务会报ERROR cluster.YarnClientSchedulerBackend: YARN application has exited unexpectedly w ...
- yarn application命令介绍
yarn application 1.-list 列出所有 application 信息 示例:yarn application -list 2.-appStates <Stat ...
- Apache hadoop namenode ha和yarn ha ---HDFS高可用性
HDFS高可用性Hadoop HDFS 的两大问题:NameNode单点:虽然有StandbyNameNode,但是冷备方案,达不到高可用--阶段性的合并edits和fsimage,以缩短集群启动的时 ...
- 【深入浅出 Yarn 架构与实现】3-1 Yarn Application 流程与编写方法
本篇学习 Yarn Application 编写方法,将带你更清楚的了解一个任务是如何提交到 Yarn ,在运行中的交互和任务停止的过程.通过了解整个任务的运行流程,帮你更好的理解 Yarn 运作方式 ...
随机推荐
- JavaScript 对时间日期格式化
JavaScript 对时间日期格式化 // 对Date的扩展,将 Date 转化为指定格式的String // 月(M).日(d).小时(h).分(m).秒(s).季度(q) 可以用 1-2 个占位 ...
- tensorflow笔记:流程,概念和简单代码注释
tensorflow是google在2015年开源的深度学习框架,可以很方便的检验算法效果.这两天看了看官方的tutorial,极客学院的文档,以及综合tensorflow的源码,把自己的心得整理了一 ...
- Mac上的抓包工具Charles[转载]
今天就来看一下Mac上如何进行抓包,之前有一篇文章介绍了使用Fidder进行抓包 http://blog.csdn.net/jiangwei0910410003/article/details/198 ...
- logz.io一个企业级的ELK日志分析器 内部集成了机器学习识别威胁——核心:利用用户对于特定日志事件的反馈处理动作来学习判断日志威胁 + 类似语音识别的专家系统从各方收集日志威胁信息
转自: 可看到它使用机器学习算法来识别DNS安全问题 http://logz.io/blog/machine-learning-log-analytics/ A Machine Learning Ap ...
- Python学习之路day3-文件操作
一.概述 作为IO操作的重要部分,文件操作需要经常用到,下面简述下python中操作文件的流程: 1. 打开文件,得到一个文件句柄并赋值给一个变量 f = open("test.txt&qu ...
- 13-THREE.JS 点光源
<!DOCTYPE html> <html> <head> <title>Example 03.02 - point Light</title&g ...
- RecycleView出现折叠效果--第三方开源--SectionedExpandableGridRecyclerView
下载地址:https://github.com/ddwhan0123/SectionedExpandableGridRecyclerView/archive/master.zip 具体见源码
- Spring_总结_03_装配Bean(四)_导入与混合配置
一.前言 本文承接上一节:Spring_总结_03_装配Bean(三)之XML配置 在典型的Spring应用中,我们可能会同时使用自动化和显示配置.同时,可能在某些场景下我们需要混合使用JavaCon ...
- hbase_异常_04_util.FSUtils: Waiting for dfs to exit safe mode...
一.异常现象 启动hbase的时,hbase的日志中可以发现: Waiting for dfs to exit safe mode... 然后就抛异常了 2018-03-22 17:00:28,994 ...
- HTTP协议状态代码和错误状态含义的解释
面试互联网公司经常被问的就是HTTP协议的知识,甚至比TCP/IP问的还多,其中HTTP代码的知识也是开发过程中经常会接触的,今天学习所有 HTTP 状态代码及其定义. 代码 指示 2xx ...