“无法启动新的线程错误"；在Python中 [英] "Can't start a new thread error" in Python
问题描述
我正在运行一个脚本，该脚本从debian软件包中提取信息并将其保存在数据库中.
I am running a script which extracts the information from debian packages and saves it in a database.
从大约100个程序包中提取信息后，会发生错误.错误是无法启动新线程" 为什么我会遇到此错误?有什么可能的解决方案?
After extracting information from about 100 packages an error occurs. The error is "can't start a new thread" Why am I facing this error? What can be the possible solution to it?
这是用于保存数据的代码:
This is the code used to save the data :
for i in list_pack:
if not i in oblist:
#Creating Packages
slu=slugify(i)
ob=Gbobject()
ob.title=i
ob.slug=slu
ob.content=''
ob.tags=tagname
#with reversion.create_revision():
ob.save()
gb=Gbobject.objects.get(title=i)
gb.objecttypes.add(Objecttype.objects.get(title='Packages'))
gb.sites.add(Site.objects.get_current())
#with reversion.create_revision():
gb.save()
gd=Gbobject.objects.get(title=i)
print 'The Object created was',gd
#Decsription
try:
atv=package_description_dict[i]
atvalue=unicode(atv,'utf-8')
except UnicodeDecodeError:
pass
try:
lo=Gbobject.objects.get(title=i)
loid=NID.objects.get(id=lo.id)
except Gbobject.DoesNotExist:
pass
if atvalue<>'':
slu=slugify(atvalue)
at=Attribute()
at.title=atvalue
at.slug=slu
at.svalue=atvalue
at.subject=loid
att=Attributetype.objects.get(title='Description')
at.attributetype=att
#with reversion.create_revision():
at.save()
print 'yeloow13'
就像Description一样，该软件包还有大约！2个以上的属性，以类似的方式保存.
Just like Description, there are around !2 more properties of the package which are saved in a similar way.
这是发生错误时得到的完整回溯:-
This is the full traceback that I get when the error occurs :-
error Traceback (most recent call last)
/home/radhika/Desktop/dev_75/gnowsys-studio/demo/<ipython console> in <module>()
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/gstudio/Harvest/debdata.py in <module>()
1086 # create_attribute_type()
1087 # create_relation_type()
-> 1088 create_objects()
1089 #create_sec_objects()
1090 #create_relations()
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/gstudio/Harvest/debdata.py in create_objects()
403 ob.sites.add(Site.objects.get_current())
404 #with reversion.create_revision():
--> 405 ob.save()
406 #time.sleep(10)
407 #gd=Gbobject.objects.get(title=i)
/usr/local/lib/python2.6/dist-packages/django_reversion-1.6.0-py2.6.egg/reversion/revisions.pyc in do_revision_context(*args, **kwargs)
298 try:
299 try:
--> 300 return func(*args, **kwargs)
301 except:
302 exception = True
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/objectapp/models.pyc in save(self, *args, **kwargs)
658 @reversion.create_revision()
659 def save(self, *args, **kwargs):
--> 660 super(Gbobject, self).save(*args, **kwargs) # Call the "real" save() method.
661
662
/usr/local/lib/python2.6/dist-packages/django_reversion-1.6.0-py2.6.egg/reversion/revisions.pyc in do_revision_context(*args, **kwargs)
298 try:
299 try:
--> 300 return func(*args, **kwargs)
301 except:
302 exception = True
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/gstudio/models.pyc in save(self, *args, **kwargs)
327 @reversion.create_revision()
328 def save(self, *args, **kwargs):
--> 329 super(Node, self).save(*args, **kwargs) # Call the "real" save() method.
330
331
/usr/local/lib/python2.6/dist-packages/django/db/models/base.pyc in save(self, force_insert, force_update, using)
458 if force_insert and force_update:
459 raise ValueError("Cannot force both insert and updating in model saving.")
--> 460 self.save_base(using=using, force_insert=force_insert, force_update=force_update)
461
462 save.alters_data = True
/usr/local/lib/python2.6/dist-packages/django/db/models/base.pyc in save_base(self, raw, cls, origin, force_insert, force_update, using)
568 if origin and not meta.auto_created:
569 signals.post_save.send(sender=origin, instance=self,
--> 570 created=(not record_exists), raw=raw, using=using)
571
572
/usr/local/lib/python2.6/dist-packages/django/dispatch/dispatcher.pyc in send(self, sender, **named)
170
171 for receiver in self._live_receivers(_make_id(sender)):
--> 172 response = receiver(signal=self, sender=sender, **named)
173 responses.append((receiver, response))
174 return responses
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/objectapp/signals.pyc in wrapper(*args, **kwargs)
65 if inspect.getmodulename(fr[1]) == 'loaddata':
66 return
---> 67 signal_handler(*args, **kwargs)
68
69 return wrapper
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/objectapp/signals.pyc in ping_external_urls_handler(sender, **kwargs)
90 from objectapp.ping import ExternalUrlsPinger
91
---> 92 ExternalUrlsPinger(gbobject)
93
94
/usr/local/lib/python2.6/dist-packages/django_gstudio-0.3.dev-py2.6.egg/objectapp/ping.pyc in __init__(self, gbobject, timeout, start_now)
153 threading.Thread.__init__(self)
154 if start_now:
--> 155 self.start()
156
157 def run(self):
/usr/lib/python2.6/threading.pyc in start(self)
472 _active_limbo_lock.release()
473 try:
--> 474 _start_new_thread(self.__bootstrap, ())
475 except Exception:
476 with _active_limbo_lock:
error: can't start new thread
我没有编写任何代码来处理线程.
I am not writing any code to handle the threads.
推荐答案
很抱歉，此处的解决方案不完整，我没有在评论部分中发布评分.
要检查的一件事是您正在运行的线程总数.我有一些代码(使用sys)检查内核数，然后启动线程并检查内核加载以测试OS如何处理线程分配，并且我了解到Windows 7(例如)似乎在32个线程之外抛出错误在8(逻辑)核心CPU上. [在Python 2.7上，Win 7 64位中的32位，等等，YMMMV].在其他计算机上，我可以获得超过1,000个线程.
One thing to check is the total number of threads you have running. I have some code that checks the count of cores (with sys) and then launches threads and checks core loading to test how the OS handles thread distribution and I've learned that windows 7 (for example) seems to throw an error beyond 32 threads on an 8 (logical) core CPU. [That's on Python 2.7, 32 bit within Win 7 64-bit, etc, YMMMV]. On other machines I can get past 1,000 Threads.
所以我想简短的版本是:当您收到该错误时，您已经拥有了多少个线程?您可以使用
So I guess the short version is: how many threads do you already have going when you get that error? You can check with
threading.active_count()
此外，您实际上并未在此处提供代码的线程部分，因此，我将引导您到这个出色的
Beyond that you don't really give the threading part of the code here so I'd direct you to this excellent Python Central page.
您也可以从前面的>堆栈溢出讨论中受益关于大线程任务计数及其处理方法.
You may also benefit from this previous Stack overflow discussion on large thread task counts and how to approach them.
再次，我很抱歉，这不是解决方案，更是一个方向，我认为可能需要更多信息来帮助我们了解您所遇到的问题.
这篇关于“无法启动新的线程错误"；在Python中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
