Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.

 "Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"

The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.

If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.

Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.

How to solve java.net.SocketTimeoutException:60000millis problem in HDFS的更多相关文章

  1. jsoup-提示java.net.SocketTimeoutException:Read timed out

    使用Jsoup.connect(url).get()连接某网站时偶尔会出现 java.net.SocketTimeoutException:Read timed out异常. 原因是默认的Socket ...

  2. java.net.SocketTimeoutException: Read timed out

    If you get java.net.SocketTimeoutException: Read timed out exception Try setting own timeout value w ...

  3. idea 连接redis 出现 Caused by: java.net.SocketTimeoutException: connect timed out

    Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.n ...

  4. 应用程序连接hbase报错:java.net.SocketTimeoutException: callTimeout=60000

    背景说明: 今天对生产环境hbase增加了节点,下午的时候一个同事反馈,应用程序后台报错,如下: Tue Feb 26 17:35:35 CST 2019, null, java.net.Socket ...

  5. Exception in thread "main" java.net.SocketTimeoutException: connect timed ou错误处理

    今天做了一个Jsoup解析网站的项目,使用Jsoup.connect(url).get()连接某网站时偶尔会出现java.net.SocketTimeoutException:Read timed o ...

  6. Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out

    问题: java连接不上redis. 异常信息: Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.ne ...

  7. phoenix连接hbase数据库,创建二级索引报错:Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions: Tue Mar 06 10:32:02 CST 2018, null, java.net.SocketTimeoutException: callTimeou

    v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VM ...

  8. java.net.SocketTimeoutException: Read timed out 错误解决

    这两天项目在测试环境下通过URLConnection 做数据传递时,出现了如下错误 java.net.SocketTimeoutException: Read timed out 经过查找研究,原因是 ...

  9. 报表使用hive数据源报java.net.SocketTimeoutException: Read timed out

    数据库表的数据量大概50W左右,在报表设计器下创建了hive的数据源,连接正常,由于数据量比较大,就用了润乾报表的大数据报表功能,报表设置好后,发布到页面中报错: 数据集ds1中,SQL语句SELEC ...

随机推荐

  1. [ios2] CABasicAnimation【转】

    caanimation 整理了解  http://geeklu.com/2012/09/animation-in-ios/ 几个可以用来实现热门APP应用PATH中menu效果的几个方法 +(CABa ...

  2. FineReport中以jws方式调用WebService数据源方案

    在使用WebService作为项目的数据源时,希望报表中也是直接调用这个WebService数据源,而不是定义数据连接调用对应的数据库表,这样要怎么实现呢? 在程序中访问WebService应用服务, ...

  3. Jquery下控制backgroundPosition位置的问题/jquery查找data id相等的元素

    1:想通过控制backgroundPosition 来实现动态效果: $(this).animate({"background-positionY":-13},1); //但是火狐 ...

  4. Linux环境快速搭建RocketMQ双Master模式

    RocketMQ的集群部署方式有多种,其中包括单个Master.多个Master.多Master多Slave模式(异步复制)以及多Master多Slave模式(同步双写).本次以多Master集群模式 ...

  5. Centos 7 拨号上网(PPPOE)

    rp-pppoe 注意事项 网卡名称要填对,用 ifconfig 查看 有人说要卸载 NetworkManager,其实没必要,不冲突 pppoe-stop && pppoe-star ...

  6. Xcode 注释工具的使用

    1:Xcode 8之后 Goodbye World In Xcode 8, Apple integrated a comment documentation generator plugin, whi ...

  7. 关于【IE兼容】的都在这

    滚动条 ie的滚动条覆盖了内容,为了触屏操作优化浏览器的内容显示,IE 浏览器提供了一种可以浮动显示,自动隐藏的滚动条样式,但是这个样式会在某些情况下造成一些困扰,比如下图... 其实默认情况下,桌面 ...

  8. react-gulp-browserify

    环境搭配参照 http://www.cnblogs.com/guolaomao/p/6276877.html 前半部分的内容. 首先安装browserify npm install --save-de ...

  9. 面向对象UML中类关系

    如果你确定两件对象之间是is-a的关系,那么此时你应该使用继承:比如菱形.圆形和方形都是形状的一种,那么他们都应该从形状类继承而不是聚合.如果你确定两件对象之间是has-a的关系,那么此时你应该使用聚 ...

  10. .net core 11