如何工作的线程与ConcurrentQueue＆LT; T＆GT; [英] How to work threading with ConcurrentQueue<T>

问题描述

我试图找出一个队列工作的最佳方式是什么。我有一个返回DataTable的过程。每个DataTable，又是合并到previous数据表。有一个问题，太多的记录保持到最后BulkCopy（内存不足）。

I am trying to figure out what the best way of working with a queue will be. I have a process that returns a DataTable. Each DataTable, in turn, is merged with the previous DataTable. There is one problem, too many records to hold until the final BulkCopy (OutOfMemory).

所以，我已经确定，我应该立即处理每个传入的数据表。思考在 ConcurrentQueue＆LT; T＆GT; ...但我没有看到 WriteQueuedData（）方法怎么会知道出列表，并将其写入到数据库中。

So, I have determined that I should process each incoming DataTable immediately. Thinking about the ConcurrentQueue<T>...but I don't see how the WriteQueuedData() method would know to dequeue a table and write it to the database.

例如：

public class TableTransporter
{
    private ConcurrentQueue<DataTable> tableQueue = new ConcurrentQueue<DataTable>();

    public TableTransporter()
    {
        tableQueue.OnItemQueued += new EventHandler(WriteQueuedData);   // no events available
    }

    public void ExtractData()
    {
        DataTable table;

        // perform data extraction
        tableQueue.Enqueue(table);
    }

    private void WriteQueuedData(object sender, EventArgs e)
    {
        BulkCopy(e.Table);
    }
}

我的第一个问题是，除了事实上，我实际上没有任何事件订阅，如果我叫 ExtractData由（）将异步这是所有我需要？其次，是有什么我缺少有关的方式 ConcurrentQueue＆LT; T＆GT; 的功能和需要某种形式的触发与排队的对象异步工作

My first question is, aside from the fact that I don't actually have any events to subscribe to, if I call ExtractData() asynchronously will this be all that I need? Second, is there something I'm missing about the way ConcurrentQueue<T> functions and needing some form of trigger to work asynchronously with the queued objects?

更新
我刚才来源于 ConcurrentQueue℃的类; T＆GT; ，有一个OnItemQueued事件处理程序。然后：

Update I have just derived a class from ConcurrentQueue<T> that has an OnItemQueued event handler. Then:

new public void Enqueue (DataTable Table)
{
    base.Enqueue(Table);
    OnTableQueued(new TableQueuedEventArgs(Table));
}

public void OnTableQueued(TableQueuedEventArgs table)
{
    EventHandler<TableQueuedEventArgs> handler = TableQueued;

    if (handler != null)
    {
        handler(this, table);
    }
}

这个实现的任何问题？

Any concerns about this implementation?

推荐答案

从我对问题的理解，你缺少了一些东西。

From my understanding of the problem, you are missing a few things.

并发队列是一种数据结构设计为接受多个线程读，没有你需要明确地锁定数据结构写入到队列中。 （全爵士乐是照顾幕后，或集合被以这样一种方式，它并不需要采取锁来实现。）

The concurrent queue is a data structure designed to accept multiple threads reading and writing to the queue without you needing to explicitly lock the data structure. (All that jazz is taken care of behind the scenes, or the collection is implemented in such a way that it doesn't need to take a lock.)

考虑到这一点，它看起来像你要使用的模式是生产/消费。首先，你必须生产工作（和添加项目到队列）一些任务。其次，你有第二个任务从队列中消费的东西（和dequeing项目）。

With that in mind, it looks like the pattern you are trying to use is the "Produce/Consumer". First, you have some tasks producing work (and adding items to the queue). And second you have a second task Consuming things from the queue (and dequeing items).

所以，真的你希望两个线程：一个添加项和第二删除项目。由于您使用的是并发收集，你可以有多个线程添加项目和多线程删除项目。但显然更多争你对并发队列的速度越快，这将​​成为瓶颈。

So really you want two threads: one adding items and a second removing items. Because you are using a concurrent collection, you can have multiple threads adding items and multiple threads removing items. But obviously the more contention you have on the concurrent queue the quicker that will become the bottleneck.

这篇关于如何工作的线程与ConcurrentQueue＆LT; T＆GT;的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！