Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误
程序运行的过程中,报Failed to locate the winutils binary in the hadoop binary path Java.io.IOException: Could not locate executable
\bin\winutils.exe in the Hadoop binaries.
通过断点调试、查看源码发现程序需要根据HADOOP_HOME找到winutils.exe,由于win机器并没有配置该环境变量,所以程序报 null\bin\winutils.exe。
private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir"); // fall back to the system/user-global env variableif (home == null) {
home = System.getenv("HADOOP_HOME");
}
try {
// couldn't find either setting for hadoop's home directoryif (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}
if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(, home.length()-);
}
// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
thrownew IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}
home = homedir.getCanonicalPath();
} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}
return home;
} private static String HADOOP_HOME_DIR = checkHadoopHome(); public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable; File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
} /** a Windows utility to emulate Unix commands */publicstaticfinal String WINUTILS = getWinUtilsPath(); public static final String getWinUtilsPath() {
String winUtilsPath = null; try {
if (WINDOWS) {
winUtilsPath = getQualifiedBinPath("winutils.exe");
}
} catch (IOException ioe) {
LOG.error("Failed to locate the winutils binary in the hadoop binary path",
ioe);
} return winUtilsPath;
}
找到原因后就去网上问了度娘,找到了解决方案,很简单,如下:
1.下载winutils的windows版本
GitHub上,有人提供了winutils的windows的版本,项目地址是:https://github.com/srccodes/hadoop-common-2.2.0-bin,直接下载此项目的zip包,下载后是文件名是hadoop-common-2.2.0-bin-master.zip,随便解压到一个目录
2.配置环境变量
增加用户变量HADOOP_HOME,值是下载的zip包解压的目录,然后在系统变量path里增加$HADOOP_HOME\bin 即可。
再次运行程序,正常执行。
Windows7系统运行hadoop报Failed to locate the winutils binary in the hadoop binary path错误的更多相关文章
- WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报Failed to ...
- Spark报错:Failed to locate the winutils binary in the hadoop binary path
之前在mac上调试hadoop程序(mac之前配置过hadoop环境)一直都是正常的.因为工作需要,需要在windows上先调试该程序,然后再转到linux下.程序运行的过程中,报 Failed to ...
- Hadoop:开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
问题: windows开发机运行spark程序,抛出异常:ERROR Shell: Failed to locate the winutils binary in the hadoop binary ...
- Windows本地运行调试Spark或Hadoop程序失败:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
报错内容 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOExce ...
- windows本地调试安装hadoop(idea) : ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
1,本地安装hadoop https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/ 下载hadoop对应版本 (我本意是想下载hadoop ...
- Spark- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
运行 mport org.apache.log4j.{Level, Logger} import org.apache.spark.rdd.RDD import org.apache.spark.{S ...
- idea 提示:ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException解决方法
Windows系统中的IDEA链接Linux里面的Hadoop的api时出现的问题 提示:ERROR util.Shell: Failed to locate the winutils binary ...
- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
文章发自:http://www.cnblogs.com/hark0623/p/4170172.html 转发请注明 14/12/17 19:18:53 ERROR Shell: Failed to ...
- ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path
错误日志如下: -- ::, DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoo ...
随机推荐
- C# Smtp方式发送邮件
//简单邮件传输协议类 System.Net.Mail.SmtpClient client = new System.Net.Mail.SmtpClient(); ...
- JAVA语言基础内部测试题(50道选择题)
JAVA语言基础内部测试题 选择题(针对以下题目,请选择最符合题目要求的答案,针对每一道题目,所有答案都选对,则该题得分,所选答案错误或不能选出所有答案,则该题不得分.)(每题2分) 没有注明选择几项 ...
- Java精选笔记_面向对象(构造方法、this关键字、static关键字、内部类)
构造方法constructor构造器 构造方法的定义 [修饰符] 类名 (形式参数列表){ //语句 } 构造器用于构造该类的实例.作用:用来初始化对象!一般由系统在创建对象(即类的 ...
- 查看系统负载:uptime
uptime命令用于查看系统负载,跟 w 命令的输出内容一致 [root@localhost ~]$ uptime :: up days, :, users, load average: 0.03, ...
- 将display设置为inline-block之后产生间隙然后换行问题的解决方法
在我们会用display的时候,inline-block肯定不陌生吧,我今天在做项目的时候,用了inline-block,使a标签可以自定义宽度,但是随之而来的问题就是换行的BUG,如下图 特地加了一 ...
- js中hover事件时候的BUG以及解决方法
hover事件是我们在开发前段时候遇到的稀松平常的问题,但是有没有发现会出现有一个BUg,比如,你移动到一个元素上,让它执行一个方法,然后你快速的移入移出的时候,他会进行亮瞎你眼睛的频闪效果,而且跟得 ...
- 网络电话pjsip Getting Started: Building for Apple iPhone, iPad and iPod Touch
Getting Started: Building for Apple iPhone, iPad and iPod Touch ¶ Getting Started Preparation Get th ...
- Excel 2010 如何在Excel的单元格中加入下拉选项
http://jingyan.baidu.com/article/03b2f78c4ba8a05ea237ae95.html 第一步:打开excel文档,选中需加入下拉选项的单元格. 第二步:点击 ...
- JQuery事件e参数的方法preventDefault()取消默认行为
<!DOCTYPE html> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <m ...
- 使用servlet3.0提供的API来进行文件的上传操作
servlet 3.0针对文件上传做了一些优化,提供了一些更加人性化的API可以直接在request中的到文件的名称.文件size,MIME类型,以及用InputStream表示的文件流的信息 @Re ...