hadoop的单机部署很简单也不容易出错，但是对生产环境的价值和意义不大，但是可以快速用于开发。

部署hadoop的错误原因不少，并且很奇怪。

比如，用户名不同，造成客户端和服务器通讯产生认证失败的错误，客户端，服务器各节点的用户名应当是一致的，并且个节点应该建立ssh的无认证登陆。

一、出现下面错误：

13/07/09 13:57:07 INFO ipc.Client: Retrying connect to server: master/192.168.2.200:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

或

java.net.ConnectException: Call to master/192.168.2.200:9000 failed on connection exception: java.net.ConnectException: Connection refused

        at org.apache.hadoop.ipc.Client.wrapException(Client.java:1136)

        at org.apache.hadoop.ipc.Client.call(Client.java:1112)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)

        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.renewLease(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:379)

        at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:378)

        at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:400)

        at org.apache.hadoop.hdfs.LeaseRenewer.access$600(LeaseRenewer.java:69)

        at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:273)

        at java.lang.Thread.run(Thread.java:722)

Caused by: java.net.ConnectException: Connection refused

        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:719)

        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)

        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)

        at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:453)

        at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:579)

        at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:202)

        at org.apache.hadoop.ipc.Client.getConnection(Client.java:1243)

        at org.apache.hadoop.ipc.Client.call(Client.java:1087)

        ... 14 more

是客户端无法连接服务器造成的，可能是服务器没有启动或者启动了防火墙。

二、出现下面错误：

13/07/09 13:57:36 ERROR hdfs.DFSClient: Failed to close file /tmp/web304069331.log

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web304069331.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

原因很多，namenode节点和datanode节点不能正常通讯造成的。根据查询启动日志看到datanode节点没有办法解析机器名造成的，所以修改/etc/hostname和/etc/hosts文件。

三、出现一下错误：

13/07/09 13:59:01 INFO hdfs.DFSClient: Exception in createBlockOutputStream 192.168.2.201:50010 java.net.SocketTimeoutException: 63000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=/192.168.2.201:50010]

13/07/09 13:59:01 INFO hdfs.DFSClient: Abandoning blk_-6965665250189110825_1679

13/07/09 13:59:01 INFO hdfs.DFSClient: Excluding datanode 192.168.2.201:50010

13/07/09 13:59:01 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

13/07/09 13:59:01 WARN hdfs.DFSClient: Error Recovery for blk_-6965665250189110825_1679 bad datanode[0] nodes == null

13/07/09 13:59:01 WARN hdfs.DFSClient: Could not get block locations. Source file "/tmp/web465901718.log" - Aborting...

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

^C七月 09, 2013 2:00:46 下午 org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["http-bio-8080"]

七月 09, 2013 2:00:46 下午 org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["ajp-bio-8009"]

七月 09, 2013 2:00:46 下午 org.apache.catalina.core.StandardService stopInternal

INFO: Stopping service Catalina

13/07/09 14:00:46 ERROR hdfs.DFSClient: Failed to close file /tmp/web465901718.log

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/web465901718.log could only be replicated to 0 nodes, instead of 1

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)

        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)

        at org.apache.hadoop.ipc.Client.call(Client.java:1107)

        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:601)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3720)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3580)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2783)

        at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3023)

七月 09, 2013 2:00:46 下午 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads

是因为客户端与数据节点通讯失败造成的。客户端程序应该能够和所有的节点通讯才能保证数据的传输正常。

四、hadoop namenode -format时出现：

Format aborted in /home/cnsworder/hadoop/name

删除掉这个文件夹重新格式化。

hadoop部署错误的更多相关文章

  1. Hadoop部署方式-本地模式(Local (Standalone) Mode)

    Hadoop部署方式-本地模式(Local (Standalone) Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. Hadoop总共有三种运行方式.本地模式(Local ...

  2. hadoop常见错误总结三

    问题导读:1.... could only be replicated to 0 nodes, instead of 1 ...可能的原因是什么?2.Error: java.lang.NullPoin ...

  3. hadoop部署小结的命令

    hadoop部署总结的命令 学习笔记,转自:hadoop部署总结的命令http://www.aboutyun.com/thread-5385-1-1.html(出处: about云开发)

  4. Tomcat学习笔记 - 错误日志 - NetBeans配置tomcat出错情况总结 -- 部署错误: 启动 Tomcat 失败。-- '127.0.0.1' 不是内部或外部命令，也不是可运行的程序

    真的管用,不知道为啥管用.转载自:http://blog.sina.com.cn/s/blog_709548200102vgy4.html 问题描述: 新安装的NetBeans8.0.2,安装过程中还 ...

  5. SSIS 包部署错误 0xC0010014

    SSIS 包部署错误 0xC0010014 Reinhard 在部署 SSIS 包时,提示如下错误. 由于错误 0xC0010014"发生了一个或多个错误.在此消息之前应有更为具体的错误消息 ...

  6. Hadoop 部署文档

    Hadoop 部署文档 1 先决条件 2 下载二进制文件 3 修改配置文件 3.1 core-site.xml 3.2 hdfs-site.xml 3.3 mapred-site.xml 3.4 ya ...

  7. hadoop进阶----hadoop经验(一)-----生产环境hadoop部署在超大内存服务器的虚拟机集群上vs几个内存较小的物理机

    生产环境 hadoop部署在超大内存服务器的虚拟机集群上 好 还是  几个内存较小的物理机上好? 虚拟机集群优点 虚拟化会带来一些其他方面的功能. 资源隔离.有些集群是专用的,比如给你三台设备只跑一个 ...

  8. Hadoop部署方式-完全分布式(Fully-Distributed Mode)

    Hadoop部署方式-完全分布式(Fully-Distributed Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本博客搭建的虚拟机是伪分布式环境(https://w ...

  9. Hadoop部署方式-伪分布式(Pseudo-Distributed Mode)

    Hadoop部署方式-伪分布式(Pseudo-Distributed Mode) 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.下载相应的jdk和Hadoop安装包 JDK:h ...

随机推荐

  1. CentOS-6.3安装配置SVN

    安装说明 系统环境:CentOS-6.3 安装方式:yum install (源码安装容易产生版本兼容的问题) 安装软件:系统自动下载SVN软件 检查已安装版本 #检查是否安装了低版本的SVN [ro ...

  2. 写给初学前端工程师的一封信 (转于Kejun)

    大家好: 应波波的邀请写一写我对这个话题的想法.从去年开始不少朋友让我帮忙介绍前端工程师,绝大部分忙都没帮上,原因是真找不到人.我当时是这么跟他们分析的:过去的客户端以browser为主,所以HTML ...

  3. Asp.net TextBox只能输入数字

    原文:Asp.net TextBox只能输入数字 <asp:textbox id="TextBox1" onkeyup="if(isNaN(value))execC ...

  4. node.js + mongodb

    node.js + mongodb 这次内容是结合bootstrap把登陆注册做好,还有就是express的中间件等问题. 看这篇博客之前建议先看我上篇写的那篇博客http://www.cnblogs ...

  5. POJ 3380 最大流

    Paratroopers Time Limit:1000MS     Memory Limit:65536KB     64bit IO Format:%I64d & %I64u Submit ...

  6. 文件合并工具DiffMerge发布4.2版本

    DiffMerge一直是文件对比合并工具的佼佼者,其最大特点是多文件对比与合并,并提供可视化界面用于编辑. 此次DiffMerge v4.2发布,提高了文件差异对比,并提供了快速匹配功能,以及更好的用 ...

  7. 手机发送短信JS验证

    function tj() { var phone = jQuery('#phone').val(); var code = jQuery('#verificationcode').val(); va ...

  8. 通过js实现在页面中添加音乐

    代码如下!兼容IE // JavaScript Document function autoPlay(){//自动播放 var myAuto = document.getElementById('my ...

  9. Linux下PHP开发环境搭建

    平时写程序时都是在服务器已经搭建好的PHP环境进行的.出于对未知知识的好奇,这几天在自己的机器上搭建起了PHP开发环境.本想轻松顺利的看到phpinfo显示在我的页面上,没想到安装环境时一路的erro ...

  10. PHP中的表单提交和获取

    在php中表单提交的方式有两种: 1.post提交,这种安全性较高. 2.get提交,他提交的是一个url地址,因此在从地址上面就可以看到许多信息,因此不安全. 每个表单<form>后面都 ...