散列/分片ActionBlocks [英] Hashed/Sharded ActionBlocks

问题描述

我有我需要让我用 TPL数据流来并行处理某些项目的不断流动。美中不足的是，共享同一键（类似于字典）中的项目应在FIFO的顺序进行处理，而不是被彼此平行（它们可以平行于其他项目不同的值）。

I have a constant flow of certain items that I need to process in parallel so I'm using TPL Dataflow. The catch is that the items that share the same key (similar to a Dictionary) should be processed in a FIFO order and not be parallel to each other (they can be parallel to other items with different values).

正在做的工作是非常CPU绑定最小异步锁，使我的解决方案是创建 ActionBlock＆LT的数组; T＆GT; 大小S Environment.ProcessorCount 无并行性和后他们根据关键的 GetHash code 值。

The work being done is very CPU bound with minimal asynchronous locks so my solution was to create an array of ActionBlock<T>s the size of Environment.ProcessorCount with no parallelism and post to them according to the key's GetHashCode value.

创建：

_actionBlocks = new ActionBlock<Item>[Environment.ProcessorCount];
for (int i = 0; i < _actionBlocks.Length; i++)
{
    _actionBlocks[i] = new ActionBlock<Item>(_ => ProcessItemAsync(_));
}

用法：

bool ProcessItem(Key key, Item item)
{
    var actionBlock = _actionBlocks[(uint)key.GetHashCode() % _actionBlocks.Length];
    return actionBlock.Post(item);
}

所以，我的问题是，这是我的问题的最佳解决方案？我是不是伤害了性能/可扩展性？我失去了一些东西？

So, my question is, is this the best solution to my problem? Am I hurting performance/scalability? Am I missing something?

推荐答案

我觉得你的做法是合理的，假设你知道散列codeS将被分配好。

I think your approach is reasonable, assuming you know the hash codes will be distributed well.

如果你想拥有对不良分布的更好的保护，可以使用更大的 ActionBlock 取值数量，同时通过使用单个自定义<限制它们的总并发级别href="http://msdn.microsoft.com/en-us/library/system.threading.tasks.dataflow.dataflowblockoptions.taskscheduler"相对=nofollow> 的TaskScheduler 通过所有块共享。你可以找到这样的调度在ParallelExtensionsExtras 或< A HREF =htt​​p://msdn.microsoft.com/en-us/library/ee789351相对=nofollow> MSDN上。

If you want to have a better protection against bad distributions, you could use larger number of ActionBlocks while limiting their total concurrency level by using a single custom TaskScheduler shared by all blocks. You can find such scheduler in ParallelExtensionsExtras or on MSDN.

这篇关于散列/分片ActionBlocks的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！