并发收集支持删除指定的项目? [英] Concurrent collection supporting removal of a specified item?

问题描述

非常简单:除了ConcurrentDictionary(如果需要，我将使用它，但它并不是真正的概念)，是否有任何Concurrent集合(IProducerConsumer实现)支持基于项的简单相等性删除特定项或定义删除条件的谓词?

Quite simple: Other than ConcurrentDictionary (which I'll use if I have to but it's not really the correct concept), is there any Concurrent collection (IProducerConsumer implementation) that supports removal of specific items based on simple equality of an item or a predicate defining a condition for removal?

说明:我有一个多线程，多阶段工作流算法，该算法从数据库中提取对象并将其粘贴在开始"队列中.从那里开始，它们被下一阶段抓住，进一步进行处理，并塞入其他队列.此过程将继续进行几个阶段.同时，第一阶段由其主管再次调用，并将对象从数据库中拉出，并且这些对象可以包括仍在处理中的对象(因为它们尚未完成处理，因此未用标记集说重新持久化)他们完成了.)

Explanation: I have a multi-threaded, multi-stage workflow algorithm, which pulls objects from the DB and sticks them in a "starting" queue. From there they are grabbed by the next stage, further worked on, and stuffed into other queues. This process continues through a few more stages. Meanwhile, the first stage is invoked again by its supervisor and pulls objects out of the DB, and those can include objects still in process (because they haven't finished being processed and so haven't been re-persisted with the flag set saying they're done).

我正在设计的解决方案是一个工作中"的大师系列；当对象在第一阶段被检索以进行处理时进入该队列，并在工作流的任何阶段完成必要的处理后将它们重新保存为已处理"到数据库后都将其删除.当该对象在该列表中时，如果在第一阶段重新检索该对象，则将忽略该对象.

The solution I am designing is a master "in work" collection; objects go in that queue when they are retrieved for processing by the first stage, and are removed after they have been re-saved to the DB as "processed" by whatever stage of the workflow completed the necessary processing. While the object is in that list, it will be ignored if it is re-retrieved by the first stage.

我原本打算使用ConcurrentBag，但是唯一的删除方法(TryTake)可以从袋子中删除任意项目，而不是指定的项目(并且.NET 4中ConcurrentBag是 slow ). ConcurrentQueue和ConcurrentStack也不允许删除它会给您的下一个项目，而留下ConcurrentDictionary，它可以工作，但超出了我的需要(我真正需要的是存储正在处理的记录的ID；它们在工作流程中不会更改.

I had planned to use a ConcurrentBag, but the only removal method (TryTake) removes an arbitrary item from the bag, not a specified one (and ConcurrentBag is slow in .NET 4). ConcurrentQueue and ConcurrentStack also do not allow removal of an item other than the next one it'll give you, leaving ConcurrentDictionary, which would work but is more than I need (all I really need is to store the Id of the records being processed; they don't change during the workflow).

推荐答案

之所以没有这样的数据结构，是因为所有集合的查找操作时间为O(n).这些是IndexOf，Remove(element)等.它们都枚举所有元素并检查它们是否相等.

The reason why there is no such a data structure is that all collections have lookup operation time of O(n). These are IndexOf, Remove(element) etc. They all enumerate through all elements and checking them for equality.

仅哈希表的查找时间为O(1).在并发场景O(n)中，查找时间将导致很长的集合锁定.在此期间，其他线程将无法添加元素.

Only hash tables have lookup time of O(1). In concurrent scenario O(n) lookup time would lead to very long lock of a collection. Other threads will not be able to add elements during this time.

在字典中，只有被哈希命中的单元格才会被锁定.当一个线程通过哈希单元中的元素检查是否相等时，其他线程可以继续添加.

In dictionary only the cell hit by hash will be locked. Other threads can continue adding while one is checking for equality through elements in hash cell.

我的建议是继续使用ConcurrentDictionary.

My advice is go on and use ConcurrentDictionary.

顺便说一句，对于您的解决方案来说，ConcurrentDictionary有点过大，这是正确的.您真正需要的是快速检查是否有物体在工作. HashSet将是一个完美的选择. Add(element)，Contains(element)，Remove(element)基本上不执行任何操作. java中有一个ConcurrentHeshSet实现.对于C#，我发现了这一点:如何在.Net中实现ConcurrentHashSet 不要知道这有多好.

By the way, you are right that ConcurrentDictionary is a bit oversized for your solution. What you really need is to check quickly weather an object is in work or not. A HashSet would be a perfect for that. It does basically nothing then Add(element), Contains(element), Remove(element). There is a ConcurrentHeshSet implementation in java. For c# I found this: How to implement ConcurrentHashSet in .Net don't know how good is it.

第一步，我仍将使用ConcurrentDictionary周围的HashSet接口编写包装程序，以使其启动并运行，然后尝试不同的实现并查看性能差异.

As a first step I would still write a wrapper with HashSet interface around ConcurrentDictionary bring it up and running and then try different implementations and see performance differences.

这篇关于并发收集支持删除指定的项目?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！