JedisConnectionException: Unexpected end of stream #932

  1. Repeatable exception and for the life of me, I cannot find something I'm doing wrong.
  2.  
  3. redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.
  4. at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:198)
  5. at redis.clients.util.RedisInputStream.read(RedisInputStream.java:180)
  6. at redis.clients.jedis.Protocol.processBulkReply(Protocol.java:158)
  7. at redis.clients.jedis.Protocol.process(Protocol.java:132)
  8. at redis.clients.jedis.Protocol.processMultiBulkReply(Protocol.java:183)
  9. at redis.clients.jedis.Protocol.process(Protocol.java:134)
  10. at redis.clients.jedis.Protocol.read(Protocol.java:192)
  11. at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:282)
  12. at redis.clients.jedis.Connection.getRawObjectMultiBulkReply(Connection.java:227)
  13. at redis.clients.jedis.JedisPubSub.process(JedisPubSub.java:108)
  14. at redis.clients.jedis.JedisPubSub.proceedWithPatterns(JedisPubSub.java:95)
  15. at redis.clients.jedis.Jedis.psubscribe(Jedis.java:2513)
  16. at BenchRedisConsumer$BenchRunner.run(BenchRedisConsumer.java:208)
  17. at java.lang.Thread.run(Thread.java:745)
  18. Running redis version 2.8.19 on Linux 3.16.0-33-generic #44-Ubuntu SMP Thu Mar 12 12:19:35 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
  19. Java version:
  20.  
  21. java version "1.7.0_76"
  22. Java(TM) SE Runtime Environment (build 1.7.0_76-b13)
  23. Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode)

Run the redis consumer followed by the producer of the project here:https://github.com/Climax777/message-queue-bench

client-output-buffer-limit was the cause. redis-server closed the connections, leading to the exceptions.

https://github.com/xetorthio/jedis/issues/932

http://stackoverflow.com/questions/2309561/how-to-fix-java-net-socketexception-broken-pipe

redis.clients.jedis.JedisPoolConfig

  1. package redis.clients.jedis;
  2.  
  3. import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
  4.  
  5. public class JedisPoolConfig extends GenericObjectPoolConfig {
  6. public JedisPoolConfig() {
  7. // defaults to make your life with connection pool easier :)
  8. setTestWhileIdle(true);
  9. setMinEvictableIdleTimeMillis(60000);
  10. setTimeBetweenEvictionRunsMillis(30000);
  11. setNumTestsPerEvictionRun(-1);
  12. }
  13. }

org.apache.commons.pool2.impl.GenericKeyedObjectPool

从Pool(也就是 LinkedBlockingDeque<PooledObject<T>>)中获取一个PooledObject<T> 需要等待的时间。

来自

  1. public GenericKeyedObjectPool(KeyedPooledObjectFactory<K,T> factory,
  2. GenericKeyedObjectPoolConfig config) {
  3.  
  4. super(config, ONAME_BASE, config.getJmxNamePrefix());
  5.  
  6. if (factory == null) {
  7. jmxUnregister(); // tidy up
  8. throw new IllegalArgumentException("factory may not be null");
  9. }
  10. this.factory = factory;
  11. this.fairness = config.getFairness();
  12.  
  13. setConfig(config);
  14.  
  15. startEvictor(getTimeBetweenEvictionRunsMillis());
  16. }
  1. public void setConfig(GenericKeyedObjectPoolConfig conf) {
  2. setLifo(conf.getLifo());
  3. setMaxIdlePerKey(conf.getMaxIdlePerKey());
  4. setMaxTotalPerKey(conf.getMaxTotalPerKey());
  5. setMaxTotal(conf.getMaxTotal());
  6. setMinIdlePerKey(conf.getMinIdlePerKey());
  7. setMaxWaitMillis(conf.getMaxWaitMillis());
  8. setBlockWhenExhausted(conf.getBlockWhenExhausted());
  9. setTestOnCreate(conf.getTestOnCreate());
  10. setTestOnBorrow(conf.getTestOnBorrow());
  11. setTestOnReturn(conf.getTestOnReturn());
  12. setTestWhileIdle(conf.getTestWhileIdle());
  13. setNumTestsPerEvictionRun(conf.getNumTestsPerEvictionRun());
  14. setMinEvictableIdleTimeMillis(conf.getMinEvictableIdleTimeMillis());
  15. setSoftMinEvictableIdleTimeMillis(
  16. conf.getSoftMinEvictableIdleTimeMillis());
  17. setTimeBetweenEvictionRunsMillis(
  18. conf.getTimeBetweenEvictionRunsMillis());
  19. setEvictionPolicyClassName(conf.getEvictionPolicyClassName());
  20. }

org.apache.commons.pool2.impl.GenericObjectPool

  1. /**
  2. * Borrow an object from the pool using the specific waiting time which only
  3. * applies if {@link #getBlockWhenExhausted()} is true.
  4. * <p>
  5. * If there is one or more idle instance available in the pool, then an
  6. * idle instance will be selected based on the value of {@link #getLifo()},
  7. * activated and returned. If activation fails, or {@link #getTestOnBorrow()
  8. * testOnBorrow} is set to <code>true</code> and validation fails, the
  9. * instance is destroyed and the next available instance is examined. This
  10. * continues until either a valid instance is returned or there are no more
  11. * idle instances available.
  12. * <p>
  13. * If there are no idle instances available in the pool, behavior depends on
  14. * the {@link #getMaxTotal() maxTotal}, (if applicable)
  15. * {@link #getBlockWhenExhausted()} and the value passed in to the
  16. * <code>borrowMaxWaitMillis</code> parameter. If the number of instances
  17. * checked out from the pool is less than <code>maxTotal,</code> a new
  18. * instance is created, activated and (if applicable) validated and returned
  19. * to the caller. If validation fails, a <code>NoSuchElementException</code>
  20. * is thrown.
  21. * <p>
  22. * If the pool is exhausted (no available idle instances and no capacity to
  23. * create new ones), this method will either block (if
  24. * {@link #getBlockWhenExhausted()} is true) or throw a
  25. * <code>NoSuchElementException</code> (if
  26. * {@link #getBlockWhenExhausted()} is false). The length of time that this
  27. * method will block when {@link #getBlockWhenExhausted()} is true is
  28. * determined by the value passed in to the <code>borrowMaxWaitMillis</code>
  29. * parameter.
  30. * <p>
  31. * When the pool is exhausted, multiple calling threads may be
  32. * simultaneously blocked waiting for instances to become available. A
  33. * "fairness" algorithm has been implemented to ensure that threads receive
  34. * available instances in request arrival order.
  35. *
  36. * @param borrowMaxWaitMillis The time to wait in milliseconds for an object
  37. * to become available
  38. *
  39. * @return object instance from the pool
  40. *
  41. * @throws NoSuchElementException if an instance cannot be returned
  42. *
  43. * @throws Exception if an object instance cannot be returned due to an
  44. * error
  45. */
  46. public T borrowObject(long borrowMaxWaitMillis) throws Exception {
  47. assertOpen();
  48.  
  49. AbandonedConfig ac = this.abandonedConfig;
  50. if (ac != null && ac.getRemoveAbandonedOnBorrow() &&
  51. (getNumIdle() < 2) &&
  52. (getNumActive() > getMaxTotal() - 3) ) {
  53. removeAbandoned(ac);
  54. }
  55.  
  56. PooledObject<T> p = null;
  57.  
  58. // Get local copy of current config so it is consistent for entire
  59. // method execution
  60. boolean blockWhenExhausted = getBlockWhenExhausted();
  61.  
  62. boolean create;
  63. long waitTime = System.currentTimeMillis();
  64.  
  65. while (p == null) {
  66. create = false;
  67. if (blockWhenExhausted) {
  68. p = idleObjects.pollFirst();
  69. if (p == null) {
  70. p = create();
  71. if (p != null) {
  72. create = true;
  73. }
  74. }
  75. if (p == null) {
  76. if (borrowMaxWaitMillis < 0) {
  77. p = idleObjects.takeFirst();
  78. } else {
  79. p = idleObjects.pollFirst(borrowMaxWaitMillis,
  80. TimeUnit.MILLISECONDS);
  81. }
  82. }
  83. if (p == null) {
  84. throw new NoSuchElementException(
  85. "Timeout waiting for idle object");
  86. }
  87. if (!p.allocate()) {
  88. p = null;
  89. }
  90. } else {
  91. p = idleObjects.pollFirst();
  92. if (p == null) {
  93. p = create();
  94. if (p != null) {
  95. create = true;
  96. }
  97. }
  98. if (p == null) {
  99. throw new NoSuchElementException("Pool exhausted");
  100. }
  101. if (!p.allocate()) {
  102. p = null;
  103. }
  104. }
  105.  
  106. if (p != null) {
  107. try {
  108. factory.activateObject(p);
  109. } catch (Exception e) {
  110. try {
  111. destroy(p);
  112. } catch (Exception e1) {
  113. // Ignore - activation failure is more important
  114. }
  115. p = null;
  116. if (create) {
  117. NoSuchElementException nsee = new NoSuchElementException(
  118. "Unable to activate object");
  119. nsee.initCause(e);
  120. throw nsee;
  121. }
  122. }
  123. if (p != null && (getTestOnBorrow() || create && getTestOnCreate())) {
  124. boolean validate = false;
  125. Throwable validationThrowable = null;
  126. try {
  127. validate = factory.validateObject(p);
  128. } catch (Throwable t) {
  129. PoolUtils.checkRethrow(t);
  130. validationThrowable = t;
  131. }
  132. if (!validate) {
  133. try {
  134. destroy(p);
  135. destroyedByBorrowValidationCount.incrementAndGet();
  136. } catch (Exception e) {
  137. // Ignore - validation failure is more important
  138. }
  139. p = null;
  140. if (create) {
  141. NoSuchElementException nsee = new NoSuchElementException(
  142. "Unable to validate object");
  143. nsee.initCause(validationThrowable);
  144. throw nsee;
  145. }
  146. }
  147. }
  148. }
  149. }
  150.  
  151. updateStatsBorrow(p, System.currentTimeMillis() - waitTime);
  152.  
  153. return p.getObject();
  154. }

socketTimeOut

redis.clients.jedis.JedisPool

  1. public JedisPool(final GenericObjectPoolConfig poolConfig, final String host, int port,
  2. int timeout, final String password) {
  3. this(poolConfig, host, port, timeout, password, Protocol.DEFAULT_DATABASE, null);
  4. }
  1. public JedisPool(final GenericObjectPoolConfig poolConfig, final String host, int port,
  2. int timeout, final String password, final int database, final String clientName) {
  3. super(poolConfig, new JedisFactory(host, port, timeout, password, database, clientName));
  4. }

redis.clients.jedis.JedisFactory

  1. @Override
  2. public PooledObject<Jedis> makeObject() throws Exception {
  3. final HostAndPort hostAndPort = this.hostAndPort.get();
  4. final Jedis jedis = new Jedis(hostAndPort.getHost(), hostAndPort.getPort(), this.timeout);
  5.  
  6. jedis.connect();
  7. if (null != this.password) {
  8. jedis.auth(this.password);
  9. }
  10. if (database != 0) {
  11. jedis.select(database);
  12. }
  13. if (clientName != null) {
  14. jedis.clientSetname(clientName);
  15. }
  16.  
  17. return new DefaultPooledObject<Jedis>(jedis);
  18. }

redis.clients.jedis.Jedis

  1. public Jedis(final String host, final int port, final int timeout) {
  2. super(host, port, timeout);
  3. }

redis.clients.jedis.BinaryJedis

  1. public BinaryJedis(final String host, final int port, final int timeout) {
  2. client = new Client(host, port);
  3. client.setConnectionTimeout(timeout);
  4. client.setSoTimeout(timeout);
  5. }

redis.clients.jedis.Connection

  1. public void connect() {
  2. if (!isConnected()) {
  3. try {
  4. socket = new Socket();
  5. // ->@wjw_add
  6. socket.setReuseAddress(true);
  7. socket.setKeepAlive(true); // Will monitor the TCP connection is
  8. // valid
  9. socket.setTcpNoDelay(true); // Socket buffer Whetherclosed, to
  10. // ensure timely delivery of data
  11. socket.setSoLinger(true, 0); // Control calls close () method,
  12. // the underlying socket is closed
  13. // immediately
  14. // <-@wjw_add
  15.  
  16. socket.connect(new InetSocketAddress(host, port), connectionTimeout);
  17. socket.setSoTimeout(soTimeout);
  18. if (ssl) {
  19. if (null == sslSocketFactory) {
  20. sslSocketFactory = (SSLSocketFactory)SSLSocketFactory.getDefault();
  21. }
  22. socket = (SSLSocket) sslSocketFactory.createSocket(socket, host, port, true);
  23. if (null != sslParameters) {
  24. ((SSLSocket) socket).setSSLParameters(sslParameters);
  25. }
  26. if ((null != hostnameVerifier) &&
  27. (!hostnameVerifier.verify(host, ((SSLSocket) socket).getSession()))) {
  28. String message = String.format(
  29. "The connection to '%s' failed ssl/tls hostname verification.", host);
  30. throw new JedisConnectionException(message);
  31. }
  32. }
  33.  
  34. outputStream = new RedisOutputStream(socket.getOutputStream());
  35. inputStream = new RedisInputStream(socket.getInputStream());
  36. } catch (IOException ex) {
  37. broken = true;
  38. throw new JedisConnectionException(ex);
  39. }
  40. }
  41. }
  1. java.net.Socket
  2. public void connect(@NotNull java.net.SocketAddress endpoint,
  3. int timeout)
  4. throws java.io.IOException
  5. Connects this socket to the server with a specified timeout value. A timeout of zero is interpreted as an infinite timeout. The connection will then block until established or an error occurs.
  6. Parameters:
  7. endpoint - the SocketAddress
  8. timeout - the timeout value to be used in milliseconds.
  9. Throws:
  10. java.io.IOException - if an error occurs during the connection
  11. java.net.SocketTimeoutException - if timeout expires before connecting
  12. java.nio.channels.IllegalBlockingModeException - if this socket has an associated channel, and the channel is in non-blocking mode
  13. IllegalArgumentException - if endpoint is null or is a SocketAddress subclass not supported by this socket
  14. Since:
  15. 1.4
  1. java.net.Socket
  2. public void setSoTimeout(int timeout)
  3. throws java.net.SocketException
  4. Enable/disable SO_TIMEOUT with the specified timeout, in milliseconds. With this option set to a non-zero timeout, a read() call on the InputStream associated with this Socket will block for only this amount of time. If the timeout expires, a java.net.SocketTimeoutException is raised, though the Socket is still valid. The option must be enabled prior to entering the blocking operation to have effect. The timeout must be > 0. A timeout of zero is interpreted as an infinite timeout.
  5. Parameters:
  6. timeout - the specified timeout, in milliseconds.
  7. Throws:
  8. java.net.SocketException - if there is an error in the underlying protocol, such as a TCP error.
  9. Since:
  10. JDK 1.1

Jedis超时时间设置梳理的更多相关文章

  1. session超时时间设置方法

    session超时时间设置方法 由于session值之前没有设置,以至于刚登录的网站,不到一分钟就超时了,总结了一下,原来是session过期的原因,以下是设置session时间的3个方法: 1. 在 ...

  2. unigui session超时时间设置

    unigui session超时时间设置 默认的SESSION超时时间是10分钟. 网络 SOCKET 程序,像 数据库,中间件,UNIGUI等...为了防止过多的僵死连接卡死服务端,服务端都会主动踢 ...

  3. GS 服务器超时时间设置

    工作中 遇到一个超时的问题 与徐庆同学沟通后 了解了下超时时间设置的地方 1.web.congfig问题: 常规路径 C:\Program Files\GenerSoft\bscw_local\web ...

  4. 接口调试工具ApiPost的发送超时时间设置方法

    有部分使用ApiPost的同学反应:发送接口调试时,响应超时时间设置的太短导致接口访问失败,怎么设置呢? 就连百度也有很多人在搜: 今天就来说一说. ApiPost简介: ApiPost是一个支持团队 ...

  5. MYSQL的数据连接超时时间设置

    大规模多线程操作事务的时候,有时候打开一个链接,会进行等待,这时候如果数据库的超时时间设置的过短,就可能会出现,数据链接自动被释放,当然设置过大也不好,慢SQL或其他因素引起的链接过长,导致整个系统被 ...

  6. 【Hadoop】Hadoop DataNode节点超时时间设置

    hadoop datanode节点超时时间设置 datanode进程死亡或者网络故障造成datanode无法与namenode通信,namenode不会立即把该节点判定为死亡,要经过一段时间,这段时间 ...

  7. (转)nginx限制上传大小和超时时间设置说明/php限制上传大小

    nginx限制上传大小和超时时间设置说明/php限制上传大小 原文:http://www.cnblogs.com/kevingrace/p/6093671.html 现象说明:在服务器上部署了一套后台 ...

  8. session的工作原理、django的超时时间设置及session过期判断

    1.session原理 cookie是保存在用户浏览器端的键值对 session是保存在服务器端的键值对 session服务端中存在的数据为: session = { 随机字符串1:{ 用户1的相关信 ...

  9. httpclient超时时间设置及代理设置

    超时时间 设置HttpClient的超时时间,非常有必要性,因为httpclient 默认超时时间很长,自己可以测试一下是多久,设置超时时间否则会影响自己系统的业务逻辑,例如阻塞系统,影响系统的吞吐量 ...

随机推荐

  1. java.lang.OutOfMemoryError: GC overhead limit exceeded 问题分析和解决(转)

    在项目历史数据导入过程中,出现了应用无法访问的情况.立刻对Weblogic进行分析,发现Weblogic的内存.线程等性能良好,Server也是Running的状态.随后查看了Weblogic日志,在 ...

  2. The end of other

    The end of other For language training our Robots want to learn about suffixes. In this task, you ar ...

  3. 精确覆盖DLX算法模板

    代码 struct DLX { int n,id; int L[maxn],R[maxn],U[maxn],D[maxn]; ]; ) //传列长 { n=nn; ;i<=n;i++) U[i] ...

  4. JAX-WS 学习一:基于java的最简单的WebService服务

    JAVA 1.6 之后,自带的JAX-WS API,这使得我们可以很方便的开发一个基于Java的WebService服务. 基于JAVA的WebService 服务 1.创建服务端WebService ...

  5. 【转】Alsa音频编程【精华】

    一.前序 这里了解一下各个参数的含义以及一些基本概念. 声音是连续模拟量,计算机将它离散化之后用数字表示,就有了以下几个名词术语. 样本长度(sample):样本是记录音频数据最基本的单位,计算机对每 ...

  6. python学习之路-6 冒泡算法、递归、反射、os/sys模块详解

    算法 冒泡算法 # 冒泡算法就是将需要排序的元素看作是一个个"气泡",最小的"气泡"最先浮出水面,排在最前面.从小到大依次排列. # 代码如下: li = [9 ...

  7. 分割视图控制器(UISplitViewController) 改_masterColumnWidth 导致在 IOS 10中出现闪退

    默认UISplitViewController的Master和Detail的宽度是固定的,可以通过下面的方式来改变 [splitViewController setValue:[NSNumber nu ...

  8. Java的IO以及线程练习

    文件的IO操作: 字节流: 输入字节流:  InputStream 所有输入字节流的基类,抽象类.  FileInputStream 读取文件的输入字节流.  BufferedInputStream ...

  9. 用CSS3实现对图片的放大效果

    用CSS3对图片放大效果 .right_div .topicons li a:hover img{     -webkit-transform:scale(1.5,1.5);     -moz-tra ...

  10. samba服务器概述

    一.samba服务器概述 Samba是一个能让Linux系统应用Microsoft网络通信协议的软件.而SMB是Server Message Block的缩写,即为服务器消息块.SMB主要作为Micr ...