1. Overview

This tutorial will show how to configure a timeout with the Apache HttpClient 4.

If you want to dig deeper and learn other cool things you can do with the HttpClient – head on over to the main HttpClient tutorial.

2. Configure Timeouts via raw String Parameters

The HttpClient comes with a lot of configuration parameters – and all of these can be set in a generic, map-like manner.

There are 3 timeout parameters to configure:

1
2
3
4
5
6
7
8
9
10
DefaultHttpClient httpClient = new DefaultHttpClient();
 
int timeout = 5; // seconds
HttpParams httpParams = httpClient.getParams();
httpParams.setParameter(
  CoreConnectionPNames.CONNECTION_TIMEOUT, timeout * 1000);
httpParams.setParameter(
  CoreConnectionPNames.SO_TIMEOUT, timeout * 1000);
// httpParams.setParameter(
//   ClientPNames.CONN_MANAGER_TIMEOUT, new Long(timeout * 1000));

A quick note is that the last parameter – the connection manager timeout – is commented out when using the 4.3.0 or 4.3.1 versions, because of this JIRA (due in 4.3.2).

3. Configure Timeouts via the API

The more important of these parameters – namely the first two – can also be set via a more type safe API:

1
2
3
4
5
6
7
8
DefaultHttpClient httpClient = new DefaultHttpClient();
 
int timeout = 5; // seconds
HttpParams httpParams = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(
  httpParams, timeout * 1000); // http.connection.timeout
HttpConnectionParams.setSoTimeout(
  httpParams, timeout * 1000); // http.socket.timeout

The third parameter doesn’t have a custom setter in HttpConnectionParams, and it will still need to be set manually via the setParameter method.

4. Configure Timeouts using the new 4.3. Builder

The fluent, builder API introduced in 4.3 provides the right way to set timeouts at a high level:

1
2
3
4
5
6
7
int timeout = 5;
RequestConfig config = RequestConfig.custom()
  .setConnectTimeout(timeout * 1000)
  .setConnectionRequestTimeout(timeout * 1000)
  .setSocketTimeout(timeout * 1000).build();
CloseableHttpClient client =
  HttpClientBuilder.create().setDefaultRequestConfig(config).build();

That is the recommended way of configuring all three timeouts in a type-safe and readable manner.

5. Timeout Properties Explained

Now, let’s explain what these various types of timeouts mean:

  • the Connection Timeout (http.connection.timeout) – the time to establish the connection with the remote host
  • the Socket Timeout (http.socket.timeout) – the time waiting for data – after the connection was established; maximum time of inactivity between two data packets
  • the Connection Manager Timeout (http.connection-manager.timeout) – the time to wait for a connection from the connection manager/pool

The first two parameters – the connection and socket timeouts – are the most important, but setting a timeout for obtaining a connection is definitely important in high load scenarios, which is why the third parameter shouldn’t be ignored.

6. Using the HttpClient

After being configured, the client can not be used to perform HTTP requests:

1
2
3
4
HttpGet getMethod = new HttpGet("http://host:8080/path");
HttpResponse response = httpClient.execute(getMethod);
System.out.println(
  "HTTP Status of response: " + response.getStatusLine().getStatusCode());

With the previously defined client, the connection to the host will time out in 5 seconds, and if the connection is established but no data is received, the timeout will also be 5 additional seconds.

Note that the connection timeout will result in an org.apache.http.conn.ConnectTimeoutException being thrown, while socket timeout will result in java.net.SocketTimeoutException.

7. Hard Timeout

While setting timeouts on establishing the HTTP connection and not receiving data is very useful, sometimes we need to set a hard timeout for the entire request.

For example, the download of a potentially large file fits into this category – in this case, the connection may be successfully established, data may be consistently coming through, but we still need to ensure that the operation doesn’t go over some specific time threshold.

HttpClient doesn’t have any configuration that allows us to set an overall timeout for a request; it does, however, provide abort functionality for requests, so we can leverage that mechanism to implement a simple timeout mechanism:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
HttpGet getMethod = new HttpGet(
 
int hardTimeout = 5; // seconds
TimerTask task = new TimerTask() {
    @Override
    public void run() {
        if (getMethod != null) {
            getMethod.abort();
        }
    }
};
new Timer(true).schedule(task, hardTimeout * 1000);
 
HttpResponse response = httpClient.execute(getMethod);
System.out.println(
  "HTTP Status of response: " + response.getStatusLine().getStatusCode());

We’re making use of the java.util.Timer and java.util.TimerTask to set up a simple delayed task which aborts the HTTP GET request after a 5 seconds hard timeout.

8. Timeout and DNS Round Robin – something to be aware of

It is quite common that some larger domains will be using a DNS round robin configuration – essentially having the same domain mapped to multiple IP addresses. This introduces a new challenge for a timeout against such a domain, simply because of the way HttpClient will try to connect to that domain that times out:

  • HttpClient gets the list of IP routes to that domain
  • it tries the first one – that times out (with the timeouts we configure)
  • it tries the second one – that also times out
  • and so on …

So, as you can see – the overall operation will not time out when we expect it to. Instead – it will time out when all the possible routes have timed out, and what it more – this will happen completely transparently for the client (unless you have your log configured at the DEBUG level). Here is a simple example you can run and replicate this issue:

1
2
3
4
5
6
7
8
9
10
int timeout = 3;
RequestConfig config = RequestConfig.custom().
  setConnectTimeout(timeout * 1000).
  setConnectionRequestTimeout(timeout * 1000).
  setSocketTimeout(timeout * 1000).build();
CloseableHttpClient client = HttpClientBuilder.create()
  .setDefaultRequestConfig(config).build();
 
HttpGet request = new HttpGet("http://www.google.com:81");
response = client.execute(request);

You will notice the retrying logic with a DEBUG log level:

1
2
3
4
5
6
7
8
9
10
11
12
DEBUG o.a.h.i.c.HttpClientConnectionOperator - Connecting to www.google.com/173.194.34.212:81
DEBUG o.a.h.i.c.HttpClientConnectionOperator -
 Connect to www.google.com/173.194.34.212:81 timed out. Connection will be retried using another IP address
 
DEBUG o.a.h.i.c.HttpClientConnectionOperator - Connecting to www.google.com/173.194.34.208:81
DEBUG o.a.h.i.c.HttpClientConnectionOperator -
 Connect to www.google.com/173.194.34.208:81 timed out. Connection will be retried using another IP address
 
DEBUG o.a.h.i.c.HttpClientConnectionOperator - Connecting to www.google.com/173.194.34.209:81
DEBUG o.a.h.i.c.HttpClientConnectionOperator -
 Connect to www.google.com/173.194.34.209:81 timed out. Connection will be retried using another IP address
//...

9. Conclusion

This tutorial discussed how to configure the various types of timeouts available for an HttpClient. It also illustrated a simple mechanism for hard timeout of an ongoing HTTP connection.

The implementation of these examples can be found in the GitHub project – this is a Maven-based project, so it should be easy to import and run as it is.

HttpClient Timeout的更多相关文章

  1. HttpClient Timeout waiting for connection from pool 问题解决方案

    错误:org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool 前言 ...

  2. HttpClient throws TaskCanceledException on timeout

    error msg: HttpClient throws TaskCanceledException on timeout HttpClient is throwing a TaskCanceledE ...

  3. TIMEOUT HANDLING WITH HTTPCLIENT

    https://www.thomaslevesque.com/2018/02/25/better-timeout-handling-with-httpclient/ The problem If yo ...

  4. HttpClient 使用

    Api支持 HttpClient 是基于Task的异步方法组,支持取消.超时异步特性,其可以分类为以下: Restful: GetAsync,PostAsync,DeleteAsync,PutAsyn ...

  5. android 请求网络 和 httpclient的使用上传下载

    访问网络最主要的也就是 http协议了. http协议很简单,但是很重要. 直接上代码了,里面都是1个代码块 代码块的,用哪一部分直接拷出去用就好了. 1.访问网络用 get 和 post  自己组拼 ...

  6. C# httpclient获取cookies实现模拟web登录

    目前在公司做一款平台化的产品,我主要负责PC端上的开发,在产品推荐过程中为了节省开发时间很多功能模块没来得及做原生,用CEF嵌入了很多带功能web页面,与客户端进行交互从而实现功能. 在二期开发中,产 ...

  7. Asp.Net Core 轻松学-HttpClient的演进和避坑

    前言     在 Asp.Net Core 1.0 时代,由于设计上的问题, HttpClient 给开发者带来了无尽的困扰,用 Asp.Net Core 开发团队的话来说就是:我们注意到,HttpC ...

  8. C# .net 中 Timeout 的处理及遇到的问题

    C# 中 Timeout 的处理 前言 最近在项目中要实现一个功能,是关于 Timeout 的,主要是要在要在 TCP 连接建立的时间 和 整个请求完成的时间,在这两个时间层面上,如果超出了设置的时间 ...

  9. .net core 中使用httpclient,HttpClientFactory的问题

    Microsoft 在.Net Framework 4.5中引入了HttpClient,并且是在.NET服务器端代码中使用Web API的最常用方法.但它有一些严重的问题,如释放HttpClient对 ...

随机推荐

  1. 去掉桌面SVN

    1新建记事本 2for /r . %%a in (.) do @if exist "%%a\.svn" rd /s /q "%%a\.svn" 3重命名 删除S ...

  2. emacs之配置speedbar

    安装sr-speedbar,这样的话,speedbar就内嵌到emacs里面了 emacsConfig/speedbar-setting.el (require 'sr-speedbar) (setq ...

  3. [转]Docker 为什么这么火

    本文来自:Docker为什么这么火 以及此文:http://cloud.51cto.com/art/201408/447966_1.htm Docker 我的理解是,相对于 VMware 的一个分支. ...

  4. py基础4--迭代器、装饰器、软件开发规范

    本节内容 迭代器&生成器 装饰器 Json & pickle 数据序列化 软件目录结构规范 作业:ATM项目开发 1. 列表生成式,迭代器&生成器 列表生成式 我现在有个需求, ...

  5. PY安装模块

    Python安装失败原因 0环境 , pip版本一般为 7.x , 所以一般需要先升级pip版本 , 也就是执行 ```shellpython -m pip install --upgrade pip ...

  6. 机器学习中的python常用函数

    glob模块 说明: 1.glob是python自己带的一个文件操作相关模块,用它可以查找符合自己目的的文件,就类似于Windows下的文件搜索, 支持通配符操作 *.?.[] 这三个通配符,*代表0 ...

  7. 《Linux内核精髓:精通Linux内核必会的75个绝技》一HACK #12 使用Memory Cgroup限制内存使用量

    HACK #12 使用Memory Cgroup限制内存使用量 Memory Cgroup是Cgroup的资源限制功能之一,可以控制特定进程可以使用的内存量.Memory CgroupMemory C ...

  8. django rest_framework 框架的使用03

    rest_framework的 数据解析器 首先需要知道前端发给后端的数据格式头有哪些: media_type = 'application/json' media_type = 'applicati ...

  9. All sentinels down, cannot determine where is mymaster master is running...

    修改配置的哨兵文件 vim /sentinel.conf 将保护模式关闭

  10. opencv:级联分类器训练(cascade classifier training)(两个分类器的区别)

    # 介绍 级联分类器包括两个工作阶段:训练(traning),检测(detection).检测阶段在文档<objdetect module of general OpenCV documenta ...