当超过32705个线程时，boost :: thread_resource_error [英] boost::thread_resource_error when more than 32705 threads

问题描述

我正在实现消息传递算法.消息通过图的节点传播，直到它们已从其他邻居接收到足够的信息以发送消息为止一直阻塞.

I'm implementing a message passing algorithm. The messages propagate through the nodes of the graph, blocking until they have have received enough information (from other neighbours) to send a message.

如果我将每条消息放在其自己的线程中并使用boost :: condition来暂停线程，直到所有必需的信息可用，该算法很容易编写.我创建了成千上万个线程，但是在任何时候几乎只有几个处于活动状态.这似乎工作得很好.

The algorithm is easy to write if I put each message in its own thread and use a boost::condition to pause the thread until all the required information is available. I create many thousands of threads, but mostly only a few are active at any time. This seems to work pretty well.

我的问题是，当进行单元测试时，我发现如果创建的线程数超过32705，我将得到

My problem is, when unit testing I find that if I create more than about 32705 threads, I get

未知位置(0):出现致命错误 "Tree_test":std :: exception: boost :: thread_resource_error

unknown location(0): fatal error in "Tree_test": std::exception: boost::thread_resource_error

，我不知道是什么原因导致的，或者如何解决.

似乎有足够的可用内存(每个线程仅包含两个指针-消息在其间传递的对象).

There seems to be pleanty of memory available (Each thread only holds two pointers - the objects that the message passes between).

从这个问题开始:在Linux中每个进程的最大线程数? 我认为以下信息是无关紧要的(尽管我真的不知道这意味着什么...)

From this question: Maximum number of threads per process in Linux? I think the following information is relevent (although I don't really know what any of it means...)

~> cat /proc/sys/kernel/threads-max
1000000

(我从60120开始增加-是否需要重新启动?)

(I increased this from 60120 - do I need to restart?)

 ~>ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

我尝试摆弄待处理信号(我的限制非常接近那个数字的2 *) 和ulimit -S -i 8191的堆栈大小-(我无法增加它)，但这些更改似乎根本没有效果)

I tried fiddling with the pending signals (my limit is very close to 2* that number) and stack size with ulimit -S -i 8191 - (I couldn't increase it) but these changes seemed to make no effect at all)

如果有帮助，我正在使用64位Ubuntu-10-10 ...

I'm on a 64 bit Ubuntu-10-10 if that helps...

推荐答案

我认为在系统上具有32K线程的情况下，您应该查看潜在的解决方案，而不是如何拥有更多的线程.例如，一个线程池(Boost为此提供了一些功能).

I think with 32K threads on the system, you should look at potential solutions other than how to have more threads. For example, a thread pool (Boost has some things for this).

无论如何，在您的系统上，PID是否不限于32768或类似的值?我认为，您迟早会用完，不妨将系统设计为允许处理比最大线程数更多的项目.

Anyway, on your system, aren't PIDs limited to 32768 or some such value? You're going to run out sooner or later, may as well design the system to allow processing more items than the max number of threads, I'd think.

也就是说，请查看/proc/sys/kernel/pid_max以查看最大PID，然后尝试增加它.这可能会使您超过32K(但对于不是为非正常大PID设计的程序，也可能导致意外行为，请谨慎行事.)

That said, look at /proc/sys/kernel/pid_max to see your max PID--and try increasing it. This may get you beyond 32K (but may also cause unexpected behavior with programs not designed for unusually large PIDs, so be cautious).

然后您可能会受到堆栈空间(与虚拟内存空间相对)的限制.如果愿意，可以尝试使用较小的堆栈创建线程.

And then you may be limited by the stack space (as opposed to virtual memory space). You could try creating threads with smaller stacks if you like.

这篇关于当超过32705个线程时，boost :: thread_resource_error的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！