WaitHandle限制的解决方法？ [英] Workaround for WaitHandle limitation?

问题描述

我正在进行一些分析，很容易在多核机器上分解成许多独立的

件。


我以为它会是最好在

ThreadPool中排队1000个这样的部分，并让该对象负责在机器的自由周期中在后台运行它们。


但我需要程序等待所有排队的部分才能完成

继续。我尝试给每个工作项一个ManualResetEvent并使用

WaitHandler来检测它们何时完成，但事实证明

WaitHandle.WaitAll（）赢了''需要超过64个WaitHandler。


当我们检测到进程的ThreadPool已经完成其

队列的正确方法是什么看看这个大小的队列？

I''m doing some analysis that is readily broken up into many independent
pieces, on a multicore machine.

I thought it would be best to just queue like 1000 of these pieces in the
ThreadPool, and let that object take care of running them in the background
on the machine''s free cycles.

But I need the program to wait for all the queued pieces to finish before it
continues. I tried giving each work item a ManualResetEvent and using
WaitHandler to detect when they''ve all finished, but it turns out that
WaitHandle.WaitAll() won''t take more than 64 WaitHandlers.

What is the right way to detect that a process''s ThreadPool has finished its
queue when we''re looking at queues of this size?

推荐答案

嗨Dave，

创建1000个线程是一个很大的开销，也是你不应该在线程池中长时间绑定

线程它们实际上只是用于b
用于快速任务。


我建议你创建一些你自己的线程，而不是1000但是b / b可能是10/20，这些线程将从你的数据消耗中获取

正在生成，每个线程将处理一些数据，然后将

处理后的数据放入列表或某个结构中，然后它可以返回
返回并处理另一部分数据。一旦所有的部分

都被处理完毕，你就可以在一个WaitHandle上发出信号，这将是
然后让你的其他程序继续前进。

Mark。

-
http：// www .markdawson.org

" Dave Booker"写道：

Hi Dave,
creating a 1000 threads is a lot of overhead, also you should not tie up
threads in the thread pool for long periods of time they are really only to
be used for quick tasks.

I would suggest that you create a number of your own threads, not 1000 but
maybe 10 / 20, these threads will then consume from where ever you data is
being produced, each thread will process some of the data and then place the
processed data into a list or some structure once it is done, it can then
return back and process another piece of the data. Once all of the pieces
have been processed you can they signal on a single WaitHandle which will
then allow the rest of your program to move forward.
Mark.
--
http://www.markdawson.org
"Dave Booker" wrote:

我正在进行一些分析，很容易在多核机器上分解成许多独立的

件。 />

我认为最好在

ThreadPool中排队1000个这样的部分，并让该对象负责在后台运行它们$>
机器的免费周期。


但我需要程序等待所有排队的部分才能完成

继续。我尝试给每个工作项一个ManualResetEvent并使用

WaitHandler来检测它们何时完成，但事实证明

WaitHandle.WaitAll（）赢了''需要超过64个WaitHandler。


当我们检测到进程的ThreadPool已经完成其

队列的正确方法是什么看着这么大的队列？

I''m doing some analysis that is readily broken up into many independent
pieces, on a multicore machine.

I thought it would be best to just queue like 1000 of these pieces in the
ThreadPool, and let that object take care of running them in the background
on the machine''s free cycles.

But I need the program to wait for all the queued pieces to finish before it
continues. I tried giving each work item a ManualResetEvent and using
WaitHandler to detect when they''ve all finished, but it turns out that
WaitHandle.WaitAll() won''t take more than 64 WaitHandlers.

What is the right way to detect that a process''s ThreadPool has finished its
queue when we''re looking at queues of this size?

谢谢，这是有道理的。


我剩下的问题是，我是否可以自动扩展在

计算机上的CPU空闲容量在任何给定时间使用的
线程数？我希望在服务器上以低优先级的形式运行它。过程，但是一个

当服务器没有做任何其他的时候会利用所有的备用CPU容量。


我正在使用

..BeginInvoke（）关闭异步执行的计算部分。将我的进程设置为低优先级似乎不会执行

技巧 - 如果我运行了太多线程，它仍然会在某些优先级更高的情况下蚕食
任务。


所以我认为一个理想的解决方案就是我可以检测当前CPU使用
的用法（并在我看到它下降时添加线程）阈）。

在.NET中可能出现这种情况吗？

Thanks, that makes sense.

My remaining question, then, is whether I can automatically scale the number
of threads being used at any given time by the CPU idle capacity on the
computer? I want to run this on servers as a "low-priority" process, but one
which will exploit all the spare CPU capacity when the server isn''t doing
anything else.

I''m spinning off the calculation pieces for asynchronous execution using
..BeginInvoke(). Setting my process as low-priority doesn''t seem to do the
trick -- if I have too many threads running it will still encroach somewhat
on higher-priority tasks.

So I think an ideal solution would be one in which I can detect current CPU
usage (and add threads when I see it dropping below some threshold). Is
something like that possible in .NET?

建议您查看Ami Bar的SmartThreadPool ;，您可以在codeproject.com找到

结束。他非常优雅地解决了64问题。

然而，马克斯的评论仍然合适。

彼得


-

联合创始人，Eggheadcafe.com开发者门户网站：
http://www.eggheadcafe.com

UnBlog：
http://petesbloggerama.blogspot.com


" Dave Booker"写道：

Suggest you have a look at Ami Bar''s "SmartThreadPool", which you can find
over at codeproject.com. He''s gotten around the 64 problem quite elegantly.
However, Marks'' comment would still be appropriate.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Dave Booker" wrote:

我正在进行一些分析，很容易在多核机器上分解成许多独立的

件。 />

我认为最好在

ThreadPool中排队1000个这样的部分，并让该对象负责在后台运行它们$>
机器的免费周期。


但我需要程序等待所有排队的部分才能完成

继续。我尝试给每个工作项一个ManualResetEvent并使用

WaitHandler来检测它们何时完成，但事实证明

WaitHandle.WaitAll（）赢了''需要超过64个WaitHandler。


当我们检测到进程的ThreadPool已经完成其

队列的正确方法是什么看着这么大的队列？

I''m doing some analysis that is readily broken up into many independent
pieces, on a multicore machine.

I thought it would be best to just queue like 1000 of these pieces in the
ThreadPool, and let that object take care of running them in the background
on the machine''s free cycles.

But I need the program to wait for all the queued pieces to finish before it
continues. I tried giving each work item a ManualResetEvent and using
WaitHandler to detect when they''ve all finished, but it turns out that
WaitHandle.WaitAll() won''t take more than 64 WaitHandlers.

What is the right way to detect that a process''s ThreadPool has finished its
queue when we''re looking at queues of this size?

这篇关于WaitHandle限制的解决方法？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！