tpl数据流:固定缓冲区大小，不丢掉任何项目 [英] tpl dataflow: fixed buffer size without throwing items away

问题描述

在处理数据流之后，我遇到了一个新问题.我想限制所有块的输入队列.我的Productionblock(ActionBlock)正在快速创建5000个元素，并将其发布到广播块.因此，如果我将广播块的BoundedCapacity设置为100，他将丢弃大量数据.但是我更希望Producerblock在我的bufferblock的输入队列中等待新的时隙.

After playing around with dataflow I encountered a new problem. I would like to limit the inputqueue of all blocks. My producingblock (ActionBlock) is creating 5000 elements really fast and posts them to an broadcastblock. So if i set the BoundedCapacity of the broadcastblock to 100 he throws a lot of data away. But I would prefer the producingblock to wait for new slots in the inputqueue of my bufferblock.

有什么办法可以解决这个问题?

Is there any way to get rid of this problem?

推荐答案

这正是 BufferBlock 的目的.如果您设置其 BoundedCapacity 且已满，它将延迟接收任何消息，直到有人使用它们.这意味着例如 Post()将被阻止，而 SendAsync()将返回未完成的 Task .

That's exactly what BufferBlock is for. If you set its BoundedCapacity and it gets full, it will postpone receiving any messages until someone consumes them. This means that for example Post() will block and SendAsync() will return an unfinished Task.

编辑:没有内置块可以发送到多个目标并且永远不会丢弃数据.但是您可以轻松地从 ActionBlock 构建自己的代码并发送循环:

There is no built-in block that sends to multiple targets and never throws data away. But you can easily build one yourself from ActionBlock and sending loop:

static ITargetBlock<T> CreateMultipleTargetsBlock<T>(
    IEnumerable<ITargetBlock<T>> targets, int boundedCapacity)
{
    var targetsList = targets.ToList();

    var block = new ActionBlock<T>(
        async item =>
        {
            foreach (var target in targetsList)
            {
                await target.SendAsync(item);
            }
        },
        new ExecutionDataflowBlockOptions { BoundedCapacity = boundedCapacity });

    // TODO: propagate completion from block to targets

    return block;
}

此代码假定您不需要为每个目标克隆数据，并且目标列表永不更改.为此，修改代码应该非常简单.

This code assumes that you don't need to clone the data for each target and that the list of targets never changes. Modifying the code for that should be fairly simple.

这篇关于tpl数据流:固定缓冲区大小，不丢掉任何项目的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！