使用System.Threading.Semaphore批量处理非常大的列表 [英] Processing a very large list in batches using System.Threading.Semaphore

问题描述

我已经在.NET 2.0中编写了以下代码

I have written the following code in .NET 2.0

foreach(Item i in millonItemsIterator) {
    ItemThreadManager m = new ItemThreadManager(i);
    System.Threading.Thread t = new System.Threading.Thread(m.Process);
    t.Start()
}
SendCompletionEmail();


public class ItemThreadManager {
    private static Semaphore s = new Semaphore(20, 20);
    private Item item = null;
    public ItemThreadManager(Item i) {
        item = i;
    } 

    public void Process() {
        s.WaitOne();
        // do something with item.
        s.Release();
    }
}    

我使用信号量方法的目的是要处理每批20个中的item列表中的项目。

My intention behind using the Semaphore approach was that I process the items in the millionItems list in batches of 20.

但是我不希望在处理完所有百万个项目之前发送电子邮件。上面的代码在处理完成之前发送电子邮件。

But I don't want the email to be sent until all million items have been processed. The above code is sending email before the processing is complete.

我的理解是，上面的代码一次只能处理20个项目。如果所有20个线程都在使用中，则会阻塞。...但是我认为对于最后20个线程....它不会等待它们完成，而是直接发送电子邮件。那么，对于最后20个我该如何实现Join？

My understanding is that above code will only process 20 items at a time. and will block if all 20 threads are in use.... but I think for the last 20 .... it will not wait for them to complete but straight go for sending email. So just for the last 20 how do I implement a Join?

我的理解正确吗？

推荐答案

要使该过程随着对SendEmail的调用而依次运行，您可以使用 lock 或使用 Tasks 链接通话。这是操作方法 http://msdn.microsoft.com/zh-CN上的链接us / library / dd537612.aspx

To make the process run sequentially with the call to SendEmail you can either just use a lock or use Tasks and chain the calls. Here's a link on How-to http://msdn.microsoft.com/en-us/library/dd537612.aspx

根据新信息，它是.net 2.0，并且您没有并行任务。使用 BackgroundWorker 运行批处理，然后挂钩到 RunWorkerCompleted 事件以了解工作何时完成以及可以发送电子邮件。此处的文档-> http：// msdn.microsoft.com/zh-CN/library/system.componentmodel.backgroundworker_events(v=vs.80).aspx

In light of the new info that it's .net 2.0 and you don't have Parallel Tasks. Use a BackgroundWorker to run the batch processing and then hook to it's RunWorkerCompleted event to know when the work is done and the email can be sent. Documentation here -> http://msdn.microsoft.com/en-US/library/system.componentmodel.backgroundworker_events(v=vs.80).aspx

这篇关于使用System.Threading.Semaphore批量处理非常大的列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！