使用数百个线程的成本是多少？ [英] What's the cost of using hundreds of threads?

问题描述

你好，


我写了一些代码，为每个连接创建了许多线程

（''main connection''）。此代码的目的是平衡多个连接（管道）之间的负载

。产生的线程数

取决于我创建的管道数量（= 2 * n + 2，其中n是

管道的数量）。


为了获得良好的结果，我可能会在10

管道之间共享主连接的负载 - 因此会产生22个线程。现在，如果转发大约50

连接，则线程数上升到数千个b $ b线程（如果建立更多连接，则为几千个）。


我的问题是：

- 创建这样数量的

线程的成本（内存/ CPU使用率）是多少？

- 是否存在限制线程数量的上限？ （是&b $ b $ python / OS相关）

- 这是笨拙编程的标志 - 即创造了这么多

线程坏习惯？ （我必须说它非常简化了我的

问题的解决方案。）


限制线程数是可能的，但会影响

数据流的独立性。 （好吧，我承认 - 创建棘手的算法

也许可以保证并发而不产生如此多的线程 -

但这是解决这个问题最简单的方法:)）。 br <>

Hello,

I have written some code, which creates many threads for each connection
(''main connection''). The purpose of this code is to balance the load
between several connections (''pipes''). The number of spawned threads
depends on how many pipes I create (= 2*n+2, where n is the number of
pipes).

For good results I''ll presumably share main connection''s load between 10
pipes - therefore 22 threads will be spawned. Now if about 50
connections are forwarded the number of threads rises to thousand of
threads (or several thousands if even more connections are established).

My questions are:
- What is the cost (in memory / CPU usage) of creating such amounts of
threads?
- Is there any ''upper boundary'' that limits the number of threads? (is
it python / OS related)
- Is that the sign of ''clumsy programming'' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

Limiting the number of threads is possible, but would affect the
independence of data flows. (ok I admit - creating tricky algorithm
could perhaps gurantee concurrency without spawning so many threads -
but it''s the simplest solution to this problem :) ).

推荐答案

Przemys3awRó？ycki写道：

Przemys3aw Ró?ycki wrote:

你好，

我写了一些代码，这为每个连接创建了许多线程
（''主要连接''）。此代码的目的是平衡多个连接（管道）之间的负载。产生的线程数
取决于我创建的管道数量（= 2 * n + 2，其中n是
管道的数量）。

为了获得良好的结果我'可能会在10个管道之间共享主要连接的负载 - 因此会生成22个线程。现在，如果转发大约50个连接，则线程数上升到数千个线程（如果建立更多连接，则为几千个）。

我的问题是：
- 创建这样数量的线程的成本（内存/ CPU使用率）是多少？
- 是否存在限制线程数量的上限？ （它与python / OS相关）
- 这是笨拙编程的标志 - 即创建这么多的线程是一个坏习惯吗？ （我必须说它非常简化了我的问题的解决方案。）

限制线程数是可能的，但会影响数据流的独立性。 （好吧，我承认 - 创建棘手的算法
也许可以保证并发而不产生如此多的线程 -
但它是解决这个问题的最简单方法:)）。

Hello,

I have written some code, which creates many threads for each connection
(''main connection''). The purpose of this code is to balance the load
between several connections (''pipes''). The number of spawned threads
depends on how many pipes I create (= 2*n+2, where n is the number of
pipes).

For good results I''ll presumably share main connection''s load between 10
pipes - therefore 22 threads will be spawned. Now if about 50
connections are forwarded the number of threads rises to thousand of
threads (or several thousands if even more connections are established).

My questions are:
- What is the cost (in memory / CPU usage) of creating such amounts of
threads?
- Is there any ''upper boundary'' that limits the number of threads? (is
it python / OS related)
- Is that the sign of ''clumsy programming'' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

Limiting the number of threads is possible, but would affect the
independence of data flows. (ok I admit - creating tricky algorithm
could perhaps gurantee concurrency without spawning so many threads -
but it''s the simplest solution to this problem :) ).

PR，

我注意到有一个资源模块带有一个

getrusage（谁）看起来会支持

a测试得到你需要的东西。

wes

PR,
I notice there''s a resource module with a
getrusage(who) that looks like it would support
a test to get what you need.
wes

Przemys3awRó？ycki napisa3（a）：

Przemys3aw Ró?ycki napisa3(a):

- 是否有限制线程数的上边界？ （它与python / OS相关）
- 这是笨拙编程的标志 - 即创建这么多的线程是一个坏习惯吗？ （我必须说它非常简化了我的
问题的解决方案。）

- Is there any ''upper boundary'' that limits the number of threads? (is
it python / OS related)
- Is that the sign of ''clumsy programming'' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

我在某处读过（我不记得哪里但是，它是MSDN吗？）

Windows不适合每个进程运行超过32个线程。我看到的大多数代码都不会产生超过一半的代码。


-

Jarek Zgoda
http://jpa.berlios.de/ | http://www.zgodowie.org/

I''ve read somewhere (I cann''t recall where, though, was it MSDN?) that
Windows is not well suited to run more than 32 threads per process. Most
of the code I saw doesn''t spawn more threads than a half of this.

--
Jarek Zgoda
http://jpa.berlios.de/ | http://www.zgodowie.org/

< br <>

Jarek Zgoda写道：

Jarek Zgoda wrote:

Przemys3awRó？ycki napisa3（a）：

Przemys3aw Ró?ycki napisa3(a):

- 是否有任何''上边界''限制线程数？ （它与python / OS相关）
- 这是笨拙编程的标志 - 即创建这么多的线程是一个坏习惯吗？ （我必须说它非常简化了我的
问题的解决方案。）

- Is there any ''upper boundary'' that limits the number of threads? (is
it python / OS related)
- Is that the sign of ''clumsy programming'' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).

我在某处读过（我不记得在哪里，但是，它是MSDN？）Windows不适合每个进程运行超过32个线程。我看到的大多数代码都没有产生超过一半的代码。

I''ve read somewhere (I cann''t recall where, though, was it MSDN?) that
Windows is not well suited to run more than 32 threads per process. Most
of the code I saw doesn''t spawn more threads than a half of this.

这是杜撰的。你有这个断言的确凿证据吗？


例如，Apache可以轻松地在Windows下生成更多线程，并且

我已经编写了代码它使用了200个具有优异性能的线程。

由于某些原因，事情似乎在2000左右的速度下慢下来我不熟悉。


问候

Steve

-

3月23日至25日与Python开发人员和你的clpy收藏会面>
来到PyCon DC 2005 http://www.pycon.org/

Steve Holden http://www.holdenweb.com/

这篇关于使用数百个线程的成本是多少？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！