明明我只是简单跑了一个数据清洗28W数据的python脚本,不知道怎么就报错如下:

too many threads running within your python process

The "can't start new thread" error almost certainly due to the fact that you have already have too many threads running within your python process,

and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you're creating; the maximum number you will be able to create will be determined by your environment,

but it should be in the order of hundreds at least. It would probably be a good idea to re-think your architecture here;

seeing as this is running asynchronously anyhow, perhaps you could use a pool of threads to fetch resources from another site instead of always starting up a thread for every request. Another improvement to consider is your use of Thread.join and Thread.stop; this would probably be better accomplished by providing a timeout value to the constructor of HTTPSConnection.

RuntimeError: can't start new thread的更多相关文章

  1. Python 3.5 RuntimeError: can't start new thread

    /*********************************************************************** * Python 3.5 RuntimeError: ...

  2. tcp端口扫描(python多线程)

    1 使用单线程扫描单台主机 首先实现的是对单台主机中0-1024端口的扫描,发现差不多每秒扫描一个端口,很慢. import socket def tcp_scanner(host,port): cl ...

  3. 我的Keras使用总结(3)——利用bottleneck features进行微调预训练模型VGG16

    Keras的预训练模型地址:https://github.com/fchollet/deep-learning-models/releases 一个稍微讲究一点的办法是,利用在大规模数据集上预训练好的 ...

  4. [线程]Thead 中传参数RuntimeError: thread.__init__() not called

    在写一个多线程类的时候调用报错 RuntimeError: thread.__init__() not called class NotifyTread(threading.Thread): def ...

  5. tensorflow_目标识别object_detection_api,RuntimeError: main thread is not in main loop,fig = plt.figure(frameon=False)_tkinter.TclError: no display name and no $DISPLAY environment variable

    最近在使用目标识别api,但是报错了: File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/script_o ...

  6. 解决async 运行多线程时报错RuntimeError: There is no current event loop in thread 'Thread-2'

    原来使用: loop = asyncio.get_event_loop()task = asyncio.ensure_future(do_work(checker))loop.run_until_co ...

  7. Python thread local

    由于GIL的原因,笔者在日常开发中几乎没有用到python的多线程.如果需要并发,一般使用多进程,对于IO Bound这种情况,使用协程也是不错的注意.但是在python很多的网络库中,都支持多线程, ...

  8. [Python]Threading.Thread之Daemon线程

    之前对Daemon线程理解有偏差,特记录说明: 一.什么是Daemon A thread can be flagged as a "daemon thread". The sign ...

  9. python threading.thread

    Thread 是threading模块中最重要的类之一,可以使用它来创建线程.有两种方式来创建线程:一种是通过继承Thread类,重写它的run方法:另一种是创建一个threading.Thread对 ...

随机推荐

  1. nginx 配置用户认证

    nginx 配置用户认证有两种方式: 1.auth_basic 本机认证,由ngx_http_auth_basic_module模块实现.配置段: http, server, location, li ...

  2. POJ3259 Wormholes 【spfa判负环】

    题目链接:http://poj.org/problem?id=3259 Wormholes Time Limit: 2000MS   Memory Limit: 65536K Total Submis ...

  3. Nginx05---负载均衡 upsteam

    参考   https://www.cnblogs.com/linjiqin/p/5494783.html

  4. 如何拿到美团offer的

    美团,我是在拉勾网上投的简历,之前也投过一次,简历都没通过删选,后来让学姐帮我改了一下简历,重新投另一个部门,获得了面试机会.10月23日,中午HR打电话过来预约了下午4点半面试,说会在线写代码,让我 ...

  5. c# base64及MD5工具类

    using System; using System.Collections.Generic; using System.Data; using System.IO; using System.Lin ...

  6. Linux系列(16)之系统资源的观察

    1.系统资源观察 1.观察内存使用情况:free 格式: free //默认显示的单位为KBytes,显示系统的内存容量 free  [-b | -k | -m | -g | -h]  [-t]  [ ...

  7. [转]Life of a binary

    发现一篇讲程序生命周期的文章,感觉蛮不错.

  8. mydumper,myloader原理及实战

    mydumper 特性 (1)多线程备份(和mysqlpump的多线程不同,mysqlpump多线程备份的粒度是表,mydumper多线程备份的粒度是行,这对于备份大表特别有用)(2)因为是多线程逻辑 ...

  9. 逆向知识第九讲,switch case语句在汇编中表达的方式

    一丶Switch Case语句在汇编中的第一种表达方式 (引导性跳转表) 第一种表达方式生成条件: case 个数偏少,那么汇编中将会生成引导性的跳转表,会做出 if else的情况(类似,但还是能分 ...

  10. Python脚本:Linux自动化执行Python脚本

    1.环境及其工具: ubuntu 16.04 python2.7(自带) pip2.7(安装) virtualenv(安装) crontab (自带) 2.pip2.7安装 (1)尝试使用 sudo ...