HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法
爬虫多次访问同一个网站一段时间后会出现错误 HTTPConnectionPool(host:XX)Max retries exceeded with url '<requests.packages.urllib3.connection.HTTPConnection object at XXXX>: Failed to establish a new connection: [Errno 99] Cannot assign requested address'
是因为在每次数据传输前客户端要和服务器建立TCP连接,为节省传输消耗,默认为keep-alive,即连接一次,传输多次,然而在多次访问后不能结束并回到连接池中,导致不能产生新的连接
headers中的Connection默认为keep-alive,
将header中的Connection一项置为close
headers = { 'Connection': 'close', } r = requests.get(url, headers=headers)
此时问题解决
HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法的更多相关文章
- HTTPConnectionPool(host='xx.xx.xx.xx', port=xx): Max retries exceeded with url:(Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000015A25025EB8>...))
HTTPConnectionPool(host='xx.xx.xx.xx', port=xx): Max retries exceeded with url:(Caused by ConnectTim ...
- scrapy::Max retries exceeded with url
运行scrapy时出错这个错误:Max retries exceeded with url解决方法: img1=requests.get(url=aa,headers=header1,timeout= ...
- HTTPConnectionPool(host:XX)Max retries exceeded with url
爬虫多次访问同一个网站一段时间后会出现错误 HTTPConnectionPool(host:XX)Max retries exceeded with url '<requests.package ...
- 关于python3.6上传文件时报错:HTTPSConnectionPool(host='***.org', port=443): Max retries exceeded with url: /post (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAIL解决办法
第一个报错: 最近在练习post请求中上传文件时遇到了一个奇葩事情,两台电脑上写了一模一样的代码,一个运行正常,另一个一片红. 最后了解了一下原因以及解决办法.先记录下关键代码: files = {& ...
- windows环境pip安装时一直报错Could not fetch URL https://pypi.org/simple/xrld/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url:
最近项目不忙了~~有开始专研的python大业,上来想用pip安装一个第三方的库,就一直报错: Could not fetch URL https://pypi.org/simple/xrld/: T ...
- HTTPSConnectionPool(host='xxxxx', port=443): Max retries exceeded with url:xxxxxxxx (Caused by NewConnectionError('<urllib3.connect,Max retries exceeded with ,(Caused by NewConnectionError
HTTPSConnectionPool(host='f6ws-sha8re-o88k.s3.ama66zaws.com', port=443): Max retries exceeded with u ...
- Max retries exceeded with url
78 Traceback (most recent call last): File "thread072413.py", line 163, in <module> ...
- 解决Max retries exceeded with url的问题
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='itunes.apple.com', port=443): Max ret ...
- requests.exceptions.SSLError……Max retries exceeded with url错误求助!!!
import requests head = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Appl ...
随机推荐
- Spring 中的接口知识整理
本想每个小知识一篇随笔,但是那样,看起来有些单薄,所以,就放在一片文章里了.而且,以后还会慢慢在最后不断的追加. 目录: FactoryBean BeanPostProcessor 1.Factory ...
- 【java编程】java中什么是bridge method(桥接方法)
https://blog.csdn.net/mhmyqn/article/details/47342577 https://www.cnblogs.com/strinkbug/p/5019453.ht ...
- oracle完全之dbf文件出现问题, ORA-01219
alter database datafile '/data/app/oradata/ora237/users01.dbf' offline drop; 强制删除该故障文件
- Gravitational Teleport简单使用
使用官方提供的二进制包进行快速启动测试,详细细节还需要在学习 下载软件包 mac 系统 https://gravitational.com/teleport/download/ wget https: ...
- KMPlayer速度步进设置及快捷键设置和同级视频自动加入播放列表
速度步进设置 快捷键设置 同级视频自动加入播放列表
- 解决jpgraph在php7.0版本下时,无法显示例子图表的问题
解决gpgraph4.02在php7.0显示空白框问题 Gpgraph类库强大的绘制图表的功能深受广大phper的喜爱,目前官方最新的版本是 jpgraph-4.0.2 ,适用于php5.0及7.0以 ...
- oracle-scn
在2012年第一季度的CPU补丁中,包含了一个关于SCN修正的重要变更,这个补丁提示,在异常情况下,Oracle的SCN可能出现异常增长,使得数据库的一切事务停止,由于SCN不能后退,所以数据库必须重 ...
- Windows git 初始设置
主要布署在 Linux 服务器上时,将全局设置 为提交自动转为 LF,签出不转换.git config --global core.autocrlf input(无效了,按默认即可) 设置全局用户名.
- Node。js 访问gmail
参考: https://developers.google.com/gmail/api/quickstart/nodejs step 1,在google网站上打开gmail api,下载JSOn st ...
- perventDefault, stopPropagation, stopImmediatePropagation 三者的区别
event有三种特别容易混淆的方法, 用来阻止默认事件的发生 1. e.preventDefault(); 2. e.stopPropagation(); 3. e.stopImmediatePro ...