scrapy RuntimeError: maximum recursion depth exceeded while calling a Python object 超出python最大递归数异常
2019-10-21 19:01:00 [scrapy.core.engine] INFO: Spider opened
2019-10-21 19:01:00 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2019-10-21 19:01:00 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2019-10-21 19:01:01 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://amp-api-search-edge.apps.apple.com/v1/catalog/cn/search?term=%E7%AE%A1%E8%B4%A6&platform=iphone&include=apps,top-apps&bubble[search]=apps,developers,groupings,editorial-items,app-bundles,in-apps&l=zh-Hans-CN&extend=editorialBadgeInfo,messagesScreenshots,minimumOSVersion,requiredCapabilities,screenshotsByType,supportsFunCamera,videoPreviewsByType> (referer: None)
2019-10-21 19:01:01 [scrapy.core.scraper] DEBUG: Scraped from <200 https://amp-api-search-edge.apps.apple.com/v1/catalog/cn/search?term=%E7%AE%A1%E8%B4%A6&platform=iphone&include=apps,top-apps&bubble[search]=apps,developers,groupings,editorial-items,app-bundles,in-apps&l=zh-Hans-CN&extend=editorialBadgeInfo,messagesScreenshots,minimumOSVersion,requiredCapabilities,screenshotsByType,supportsFunCamera,videoPreviewsByType>
None
2019-10-21 19:01:01 [scrapy.core.engine] INFO: Closing spider (finished)
Traceback (most recent call last):
File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
msg = self.format(record)
File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
return fmt.format(record)
File "/usr/lib/python2.7/logging/__init__.py", line 465, in format
record.message = record.getMessage()
File "/usr/lib/python2.7/logging/__init__.py", line 329, in getMessage
msg = msg % self.args
File "/home/project/release/venv2.7/local/lib/python2.7/site-packages/scrapy/spiders/__init__.py", line 107, in __str__
return "<%s %r at 0x%0x>" % (type(self).__name__, self.name, id(self))
...... 重复n行红色日志 .......
File "/home/project/release/venv2.7/local/lib/python2.7/site-packages/scrapy/spiders/__init__.py", line 107, in __str__
return "<%s %r at 0x%0x>" % (type(self).__name__, self.name, id(self))
RuntimeError: maximum recursion depth exceeded while calling a Python object
Logged from file signal.py, line 57
2019-10-21 19:01:01 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 812,
'downloader/request_count': 1,
'downloader/request_method_count/GET': 1,
'downloader/response_bytes': 19255,
'downloader/response_count': 1,
'downloader/response_status_count/200': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2019, 10, 21, 11, 1, 1, 765291),
'item_scraped_count': 1,
'log_count/DEBUG': 2,
'log_count/ERROR': 1,
'log_count/INFO': 9,
'memusage/max': 1424973824,
'memusage/startup': 1424973824,
'response_received_count': 1,
'scheduler/dequeued': 1,
'scheduler/dequeued/memory': 1,
'scheduler/enqueued': 1,
'scheduler/enqueued/memory': 1,
'start_time': datetime.datetime(2019, 10, 21, 11, 1, 0, 418993)}
2019-10-21 19:01:01 [scrapy.core.engine] INFO: Spider closed (finished)
Process finished with exit code 0
原因:spiders中的close函数中有异常!
scrapy RuntimeError: maximum recursion depth exceeded while calling a Python object 超出python最大递归数异常的更多相关文章
- Python递归报错:RuntimeError: maximum recursion depth exceeded in comparison
Python中默认的最大递归深度是989,当尝试递归第990时便出现递归深度超限的错误: RuntimeError: maximum recursion depth exceeded in compa ...
- 记 suds 模块循环依赖的坑-RuntimeError: maximum recursion depth exceeded
下面是soa接口调用的核心代码 #! /usr/bin/python # coding:utf-8 from suds.client import Clientdef SoaRequest(wsdl, ...
- Odoo8查询产品时提示"maximum recursion depth exceeded while calling a Python object"
今天在生产系统中查询产品时,莫名提示错误:maximum recursion depth exceeded while calling a Python object,根据错误日志提示,发现在查询产品 ...
- python递归深度报错--RuntimeError: maximum recursion depth exceeded
当你的程序递归的次数超过999次的时候,就会引发RuntimeError: maximum recursion depth exceeded. 解决方法两个: 1.增加系统的递归调用的次数: impo ...
- 函数递归时,递归次数到900多时,就是抛出异常exception RuntimeError('maximum recursion depth exceeded',)
import subprocess import multiprocessing import urllib import sys import os import pymongo import si ...
- python RecursionError: maximum recursion depth exceeded while calling
import copyimport sys # 导入sys模块sys.setrecursionlimit(8192) # 将默认的递归深度修改为r = sys.getrecursionlimit()_ ...
- python maximum recursion depth exceeded 处理办法
1.在执行命令 pyinstaller -F D:\py\programe\banksystem.py打包生成.exe文件时报错:python maximum recursion depth exce ...
- 爬豆瓣影评,记下解决maximum recursion depth exceeded in cmp
#主要是爬取后给别人做自然语言分析,没其他意思. #coding=utf8 import requests,re from lxml import etree import sys reload(sy ...
- python --RecursionError: maximum recursion depth exceeded in comparison
在学习汉娜塔的时候,遇到一个error RecursionError: maximum recursion depth exceeded in comparison 经过百度,百度的方法: 加上: i ...
随机推荐
- mariadb使用with子句重写SQL性能提升5倍
几个月前,我们有个产品的开发反馈了个问题,说有个组织结构的查询很慢,几千行的复杂关联需要1秒钟,表示太慢了,原语句如下: SELECT org.org_id, org.dimension, org.o ...
- 简易商城 [ html + css ] 练习
1. 前言 通过使用 HTML + CSS 编写一个简易商城首页. 如图: 2. 布局思路 通过页面分析,大致可以决定页面的布局分为 5 大板块. 接下来,可以先定义页面的布局: <!DOCTY ...
- javascript中this、new、apply和call详解
如果在javascript语言里没有通过new(包括对象字面量定义).call和apply改变函数的this指针,函数的this指针都是指向window的,重要的话要说三遍.... 讲解this指针的 ...
- [LeetCode] 502. IPO 上市
Suppose LeetCode will start its IPO soon. In order to sell a good price of its shares to Venture Cap ...
- 算法练习之杨辉三角,杨辉三角的第 k 行,买卖股票的最佳时机
1. 杨辉三角 给定一个非负整数 numRows,生成杨辉三角的前 numRows 行. 在杨辉三角中,每个数是它左上方和右上方的数的和. 示例: 输入: 输出: [ [], [,], [,,], [ ...
- PCL读取PCD文件的数据
1.pcd文件——rabbit.pcd 链接:https://pan.baidu.com/s/1v6mjPjwd7fIqUSjlIGTIGQ提取码:zspx 新建项目pcl rabbit.pcd 和p ...
- 用js实现promise
/* 自定义promise 1. 执行MyPromise构造函数,要立即执行executor 2. promise实例对象,内部有三种状态 ...
- TCP/IP学习笔记12-- IP协议基础
IP(internet protocol, 网际协议) IP相当于OSI参考模型中的第三层 -- 网络层. --网络层的下一层--数据链路层的主要作用是在 同一种数据链路 的节点之间进行包传递,而一旦 ...
- Python的线程、进程和协程
进程:一个进程就是一个正在运行的程序,它是计CPU分配资源的最小单位.每个进程都有自己独立的内存空间.能同时执行的进程数最多不超过内核数,也就是每个内核 同一时刻只能执行一个进程.那么多进程就是能[同 ...
- 常用Tables控件介绍(三)
向datagrid中添加临时记录: 代码: $(function(){ fun={ add:function(){ $.ajaxSettings.async=false; var rows=$('#d ...