HttpClient(4.5.x)正确的使用姿势
前言:
httpclient(4.5.x)默认是启动连接池的, 其降低时耗(避免连接初3次握手, 以及关闭4次握手的消耗), 显著提升高并发处理能力(大量减少time_wait), 确实扮演了重要的角色. 但是封装httpclient, 需要了解不少细节, 还要根据业务合理配置参数.
这里结合这段时间深入httpclient(4.5.x)源码分析, 结合网上的代码案例, 以及线下测试的结果. 尝试写一个可用的的httpclient封装类, 兼顾性能, 接口友好. 感谢cctv, ^_^.
相关文章:
1. HttpClient官方sample代码的深入分析(连接池)
第一版本:
啥也不说了, 直接贴代码了.
public class PooledHttpClientAdaptor {
private static final int DEFAULT_POOL_MAX_TOTAL = 200;
private static final int DEFAULT_POOL_MAX_PER_ROUTE = 200;
private static final int DEFAULT_CONNECT_TIMEOUT = 500;
private static final int DEFAULT_CONNECT_REQUEST_TIMEOUT = 500;
private static final int DEFAULT_SOCKET_TIMEOUT = 2000;
private PoolingHttpClientConnectionManager gcm = null;
private CloseableHttpClient httpClient = null;
// 连接池的最大连接数
private final int maxTotal;
// 连接池按route配置的最大连接数
private final int maxPerRoute;
// tcp connect的超时时间
private final int connectTimeout;
// 从连接池获取连接的超时时间
private final int connectRequestTimeout;
// tcp io的读写超时时间
private final int socketTimeout;
public PooledHttpClientAdaptor() {
this(
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_TOTAL,
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_PER_ROUTE,
PooledHttpClientAdaptor.DEFAULT_CONNECT_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_CONNECT_REQUEST_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_SOCKET_TIMEOUT
);
}
public PooledHttpClientAdaptor(
int maxTotal,
int maxPerRoute,
int connectTimeout,
int connectRequestTimeout,
int socketTimeout
) {
this.maxTotal = maxTotal;
this.maxPerRoute = maxPerRoute;
this.connectTimeout = connectTimeout;
this.connectRequestTimeout = connectRequestTimeout;
this.socketTimeout = socketTimeout;
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
gcm = new PoolingHttpClientConnectionManager(registry);
gcm.setMaxTotal(this.maxTotal);
gcm.setDefaultMaxPerRoute(this.maxPerRoute);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectTimeout) // 设置连接超时
.setSocketTimeout(this.socketTimeout) // 设置读取超时
.setConnectionRequestTimeout(this.connectRequestTimeout) // 设置从连接池获取连接实例的超时
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClient = httpClientBuilder
.setConnectionManager(gcm)
.setDefaultRequestConfig(requestConfig)
.build();
}
public String doGet(String url) {
return this.doGet(url, Collections.EMPTY_MAP, Collections.EMPTY_MAP);
}
public String doGet(String url, Map<String, Object> params) {
return this.doGet(url, Collections.EMPTY_MAP, params);
}
public String doGet(String url,
Map<String, String> headers,
Map<String, Object> params
) {
// *) 构建GET请求头
String apiUrl = getUrlWithParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpGet);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
public String doPost(String apiUrl, Map<String, Object> params) {
return this.doPost(apiUrl, Collections.EMPTY_MAP, params);
}
public String doPost(String apiUrl,
Map<String, String> headers,
Map<String, Object> params
) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求headers
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpPost.addHeader(entry.getKey(), entry.getValue());
}
}
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpPost);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if ( entityRes != null ) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if (response != null) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
private HttpEntity getUrlEncodedFormEntity(Map<String, Object> params) {
List<NameValuePair> pairList = new ArrayList<NameValuePair>(params.size());
for (Map.Entry<String, Object> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue().toString());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
private String getUrlWithParams(String url, Map<String, Object> params) {
boolean first = true;
StringBuilder sb = new StringBuilder(url);
for (String key : params.keySet()) {
char ch = '&';
if (first == true) {
ch = '?';
first = false;
}
String value = params.get(key).toString();
try {
String sval = URLEncoder.encode(value, "UTF-8");
sb.append(ch).append(key).append("=").append(sval);
} catch (UnsupportedEncodingException e) {
}
}
return sb.toString();
}
}
存在问题&解决方案
这个版本基本没啥问题, 但是当流量为0时, 你会发现存在处于ClOSE_WAIT的连接. 究其原因是, httpclient清理过期/被动关闭的socket, 是采用懒惰清理的策略. 它是在连接从连接池取出使用的时候, 检测状态并做相应处理. 如果没有流量, 那这些socket将一直处于CLOSE_WAIT(半连接的状态), 系统资源被浪费.
不过解决方案也相当的简单, 官方的建议引入一个清理线程, 定期主动处理过期/空闲连接, 这样就OK了.
private class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean exitFlag = false;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
this.connMgr = connMgr;
setDaemon(true);
}
@Override
public void run() {
while (!this.exitFlag) {
synchronized (this) {
try {
this.wait(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 关闭失效的连接
connMgr.closeExpiredConnections();
// 可选的, 关闭30秒内不活动的连接
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
public void shutdown() {
this.exitFlag = true;
synchronized (this) {
notify();
}
}
}
最终版本
直接整合贴代码, ^_^.
public class PooledHttpClientAdaptor {
private static final int DEFAULT_POOL_MAX_TOTAL = 200;
private static final int DEFAULT_POOL_MAX_PER_ROUTE = 200;
private static final int DEFAULT_CONNECT_TIMEOUT = 500;
private static final int DEFAULT_CONNECT_REQUEST_TIMEOUT = 500;
private static final int DEFAULT_SOCKET_TIMEOUT = 2000;
private PoolingHttpClientConnectionManager gcm = null;
private CloseableHttpClient httpClient = null;
private IdleConnectionMonitorThread idleThread = null;
// 连接池的最大连接数
private final int maxTotal;
// 连接池按route配置的最大连接数
private final int maxPerRoute;
// tcp connect的超时时间
private final int connectTimeout;
// 从连接池获取连接的超时时间
private final int connectRequestTimeout;
// tcp io的读写超时时间
private final int socketTimeout;
public PooledHttpClientAdaptor() {
this(
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_TOTAL,
PooledHttpClientAdaptor.DEFAULT_POOL_MAX_PER_ROUTE,
PooledHttpClientAdaptor.DEFAULT_CONNECT_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_CONNECT_REQUEST_TIMEOUT,
PooledHttpClientAdaptor.DEFAULT_SOCKET_TIMEOUT
);
}
public PooledHttpClientAdaptor(
int maxTotal,
int maxPerRoute,
int connectTimeout,
int connectRequestTimeout,
int socketTimeout
) {
this.maxTotal = maxTotal;
this.maxPerRoute = maxPerRoute;
this.connectTimeout = connectTimeout;
this.connectRequestTimeout = connectRequestTimeout;
this.socketTimeout = socketTimeout;
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
this.gcm = new PoolingHttpClientConnectionManager(registry);
this.gcm.setMaxTotal(this.maxTotal);
this.gcm.setDefaultMaxPerRoute(this.maxPerRoute);
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(this.connectTimeout) // 设置连接超时
.setSocketTimeout(this.socketTimeout) // 设置读取超时
.setConnectionRequestTimeout(this.connectRequestTimeout) // 设置从连接池获取连接实例的超时
.build();
HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClient = httpClientBuilder
.setConnectionManager(this.gcm)
.setDefaultRequestConfig(requestConfig)
.build();
idleThread = new IdleConnectionMonitorThread(this.gcm);
idleThread.start();
}
public String doGet(String url) {
return this.doGet(url, Collections.EMPTY_MAP, Collections.EMPTY_MAP);
}
public String doGet(String url, Map<String, Object> params) {
return this.doGet(url, Collections.EMPTY_MAP, params);
}
public String doGet(String url,
Map<String, String> headers,
Map<String, Object> params
) {
// *) 构建GET请求头
String apiUrl = getUrlWithParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpGet);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
public String doPost(String apiUrl, Map<String, Object> params) {
return this.doPost(apiUrl, Collections.EMPTY_MAP, params);
}
public String doPost(String apiUrl,
Map<String, String> headers,
Map<String, Object> params
) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求headers
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
httpPost.addHeader(entry.getKey(), entry.getValue());
}
}
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
CloseableHttpResponse response = null;
try {
response = httpClient.execute(httpPost);
if (response == null || response.getStatusLine() == null) {
return null;
}
int statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if ( entityRes != null ) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
return null;
} catch (IOException e) {
} finally {
if (response != null) {
try {
response.close();
} catch (IOException e) {
}
}
}
return null;
}
private HttpEntity getUrlEncodedFormEntity(Map<String, Object> params) {
List<NameValuePair> pairList = new ArrayList<NameValuePair>(params.size());
for (Map.Entry<String, Object> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue().toString());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
private String getUrlWithParams(String url, Map<String, Object> params) {
boolean first = true;
StringBuilder sb = new StringBuilder(url);
for (String key : params.keySet()) {
char ch = '&';
if (first == true) {
ch = '?';
first = false;
}
String value = params.get(key).toString();
try {
String sval = URLEncoder.encode(value, "UTF-8");
sb.append(ch).append(key).append("=").append(sval);
} catch (UnsupportedEncodingException e) {
}
}
return sb.toString();
}
public void shutdown() {
idleThread.shutdown();
}
// 监控有异常的链接
private class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean exitFlag = false;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
this.connMgr = connMgr;
setDaemon(true);
}
@Override
public void run() {
while (!this.exitFlag) {
synchronized (this) {
try {
this.wait(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// 关闭失效的连接
connMgr.closeExpiredConnections();
// 可选的, 关闭30秒内不活动的连接
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
public void shutdown() {
this.exitFlag = true;
synchronized (this) {
notify();
}
}
}
}
总结:
其实没啥难道, 主要就是买个安心, 这样写是安全的, 是经得起线上考验的.
HttpClient(4.5.x)正确的使用姿势的更多相关文章
- 如约而至，Java 10 正式发布！ Spring+SpringMVC+MyBatis+easyUI整合进阶篇（十四）Redis缓存正确的使用姿势 努力的孩子运气不会太差，跌宕的人生定当更加精彩 优先队列详解(转载)
如约而至,Java 10 正式发布! 3 月 20 日,Oracle 宣布 Java 10 正式发布. 官方已提供下载:http://www.oracle.com/technetwork/java ...
- Servlet3.0+springmvc5+log4j2正确的开启姿势(WebLookUp)
前言 java社区占据市场份额比较大的日志组件由log4j 1.×,到logback,再到整合后的升级版 log4j 2.×,有网友测试后据说log4j2的性能最NB.于是开始往自己的springmv ...
- 论consul正确的关闭姿势
最近在工作中发现一个有意思的现象,我用 ctrl+c 关闭本地 consul 的时候,报警系统并没有发出告警,说我的 node 异常,自己看了一下代码,发现 consul 的关闭还是有点猫腻的,仔细来 ...
- Fragment全解析系列（二）：正确的使用姿势
作为一个稳定的app,从后台且回到前台,一定会在任何情况都能恢复到离开前的页面,并且保证数据的完整性. 如果你没看过本系列的第一篇,为了方便后面文章的介绍,先规定一个"术语",安卓 ...
- Log.isLoggable之一正确的使用姿势
DEBUG方法比较当我们在做APP开发时,需要设置调试开关打印Log,下面我列举出3种方法: 方法一:直接赋值public static final boolean DEBUG = true;//fa ...
- How Javascript works (Javascript工作原理) (五) 深入理解 WebSockets 和带有 SSE 机制的HTTP/2 以及正确的使用姿势
个人总结: 1.长连接机制——分清Websocket,http2,SSE: 1)HTTP/2 引进了 Server Push 技术用来让服务器主动向客户端缓存发送数据.然而,它并不允许直接向客户端程序 ...
- Spring Boot程序正确停止的姿势
Spring Boot提供了2种优雅关闭进程的方式: 基于管理端口关闭进程 基于系统服务方式关闭进程 基于管理端口关闭进程 基于管理端口方式实现进程关闭实际上是模块spring-boot-actuat ...
- Spring+SpringMVC+MyBatis+easyUI整合进阶篇（十四）Redis缓存正确的使用姿势
作者:13 GitHub:https://github.com/ZHENFENG13 版权声明:本文为原创文章,未经允许不得转载. 简介 这是一篇关于Redis使用的总结类型文章,会先简单的谈一下缓存 ...
- 正确的上网姿势:ubuntu18.04使用clash
本文为本人将CFW(Clash For Windows)上的配置文件应用到ubuntu上面的操作备忘,仅供个人使用 首先下载已经打包的clash压缩包:https://github.com/Dream ...
随机推荐
- 【转】你必须知道的EF知识和经验
注意:以下内容如果没有特别申明,默认使用的EF6.0版本,code first模式. 推荐MiniProfiler插件 工欲善其事,必先利其器. 我们使用EF和在很大程度提高了开发速度,不过随之带来的 ...
- ActiveMQ使用例子
网上收集的例子:有broker,producer,consumer public class MqApp { public static void main(String[] args) throws ...
- weblogic控制台用户名密码修改
1.记得用户名密码但想修改密码修改方法 保存后立即生效,即你退出后即能以新密码登录:但由于启动的用户名密码和登录的用户名密码是同一个,所以我们需要去修改DOMAIN_HOME/servers/serv ...
- 基于spring的PropertySource类实现配置的动态替换
public class ConfigPropertySource extends PropertySource<Properties> implements PriorityOrdere ...
- C#代码中如何比较两个日期的大小？
C#中比较两个日期大小的方法 /// <summary> /// 比较两个日期大小 /// </summary> /// <param name="dateSt ...
- Oracle.PL/SQL高级
一.匿名块 .使用returning ... INTO 保存增删改表数据时的一些列的值 ()增加数据时保存数据 DECLARE v_ename emp.ename%TYPE; v_sal emp.sa ...
- javascript 日期函数
获取当前日期的前一天的日期 var MyDate = new Date( ); //获取昨天的日期 var yesterday = myDate.getTime()-1000*60*60 ...
- Vue + Element UI 实现权限管理系统（第三方图标库）
使用第三方图标库 用过Elment的同鞋都知道,Element UI提供的字体图符少之又少,实在是不够用啊,幸好现在有不少丰富的第三方图标库可用,引入也不会很麻烦. Font Awesome Font ...
- day2编程语言的两大分类
编程的语言的发展经历了 机器语言 汇编语言 高级语言 高级语言更贴近人类的语言,但是必须被翻译成计算机能读懂的二进制后,才能够被执行,按照翻译方式分为 1 编译型(需要编译器,相当于用谷歌翻译); ...
- centos7 新装系统网络配置
[root@localhost ~]# cat /etc/sysconfig/grub GRUB_TIMEOUT= GRUB_DISTRIBUTOR="$(sed 's, release . ...