RuntimeError: can't start new thread
明明我只是简单跑了一个数据清洗28W数据的python脚本,不知道怎么就报错如下:

too many threads running within your python process
The "can't start new thread" error almost certainly due to the fact that you have already have too many threads running within your python process,
and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you're creating; the maximum number you will be able to create will be determined by your environment,
but it should be in the order of hundreds at least. It would probably be a good idea to re-think your architecture here;
seeing as this is running asynchronously anyhow, perhaps you could use a pool of threads to fetch resources from another site instead of always starting up a thread for every request. Another improvement to consider is your use of Thread.join and Thread.stop; this would probably be better accomplished by providing a timeout value to the constructor of HTTPSConnection.
RuntimeError: can't start new thread的更多相关文章
- Python 3.5 RuntimeError: can't start new thread
/*********************************************************************** * Python 3.5 RuntimeError: ...
- tcp端口扫描(python多线程)
1 使用单线程扫描单台主机 首先实现的是对单台主机中0-1024端口的扫描,发现差不多每秒扫描一个端口,很慢. import socket def tcp_scanner(host,port): cl ...
- 我的Keras使用总结(3)——利用bottleneck features进行微调预训练模型VGG16
Keras的预训练模型地址:https://github.com/fchollet/deep-learning-models/releases 一个稍微讲究一点的办法是,利用在大规模数据集上预训练好的 ...
- [线程]Thead 中传参数RuntimeError: thread.__init__() not called
在写一个多线程类的时候调用报错 RuntimeError: thread.__init__() not called class NotifyTread(threading.Thread): def ...
- tensorflow_目标识别object_detection_api,RuntimeError: main thread is not in main loop,fig = plt.figure(frameon=False)_tkinter.TclError: no display name and no $DISPLAY environment variable
最近在使用目标识别api,但是报错了: File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/script_o ...
- 解决async 运行多线程时报错RuntimeError: There is no current event loop in thread 'Thread-2'
原来使用: loop = asyncio.get_event_loop()task = asyncio.ensure_future(do_work(checker))loop.run_until_co ...
- Python thread local
由于GIL的原因,笔者在日常开发中几乎没有用到python的多线程.如果需要并发,一般使用多进程,对于IO Bound这种情况,使用协程也是不错的注意.但是在python很多的网络库中,都支持多线程, ...
- [Python]Threading.Thread之Daemon线程
之前对Daemon线程理解有偏差,特记录说明: 一.什么是Daemon A thread can be flagged as a "daemon thread". The sign ...
- python threading.thread
Thread 是threading模块中最重要的类之一,可以使用它来创建线程.有两种方式来创建线程:一种是通过继承Thread类,重写它的run方法:另一种是创建一个threading.Thread对 ...
随机推荐
- bitmap位图原理和实现
引子 首先通过一道题来理解什么是bitmap. 题目:我有40亿个整数,再给一个新的整数,我需要判断新的整数是否在40亿个整数中,你会怎么做? 分析: 假设一个int占4个字节(32位),40个亿个整 ...
- [LuoguP2124]奶牛美容_bfs_floyd_曼哈顿距离
奶牛美容 题目链接:https://www.luogu.org/problem/P2124 数据范围:略. 题解: 发现数据范围只有$50$,显然可以直接$bfs$求出联通块,$floyd$求出相邻两 ...
- 1.3.3 并发容器类MAP/LIST/SET/QUEUE
HashMap 下标计算方法:hashCode & (length-1) ,&按位与操作,二进制相同位都是1,该位才为1 JDK1.7与JDK1.8中HashMap区别: JDK1.8 ...
- Win7 Eclipse 搭建spark java1.8环境:WordCount helloworld例子
[学习笔记] Win7 Eclipse 搭建spark java1.8环境:WordCount helloworld例子在eclipse oxygen上创建一个普通的java项目,然后把spark-a ...
- 记录一次hadoop2.8.4版本RM接入zk ha问题
背景: 公司将线上hadoop RM接入ZK 实现高可用 但ZK Znode 默认存储1M,当存储数据量大时候可能导致线上业务的崩溃 处理方案如下: 1,修改ZK配置 增加默认存储上限 2,修改RM数 ...
- js中实现输入框类似百度搜索的智能提示效果
说明:我这里显示的数据采用词典(词典在js中自定义的,看下面文字),主要显示key. 页面元素: <style type="text/css">.search { le ...
- 【AtCoder】AGC009
AGC009 A - Multiple Array 从后往前递推即可 #include <bits/stdc++.h> #define fi first #define se second ...
- window下 php7+apache2搭建
一.下载php7和apache2.4 首先下载php7的windows压缩包,到这里下载http://windows.php.net/download/ Php7 VC14 x86 Thread S ...
- Vue路由传参及传参后刷新导致参数消失处理
项目功能需要,要从列表页跳转到第三方提供的URL上(这里第三方页面我是通过iframe引入在详情页,目的是点击返回时可以通过keepAlive让列表页不刷新,如果不通过iframe直接跳第三方链接,那 ...
- windows phone 下拉刷新
在windows phone 中采用数据列表时为了保证用户体验常遇到加载数据的问题.这个问题普遍到只要你用到数据列表就要早晚面对这个问题. 很多人会说这个问题已经有解决方案. 其实真正问题并不在于如何 ...