Apache Beam:跳过已经构建好的管道中的步骤 [英] Apache Beam: Skipping steps in an already-constructed pipeline

问题描述

是否可以有条件地跳过已经构建的管道中的步骤?还是将管道建设设计为控制执行哪些步骤的唯一方法?

Is there a way to conditionally skip steps in an already-constructed pipeline? Or is pipeline construction designed to be the only way to control which steps are run?

推荐答案

通常，管道构造控制将在管道中执行哪些转换.

Normally, pipeline construction controls what transformations in a pipeline will be executed.

但是，您可以想象一个输入，多个输出ParDo，它将输入PCollection多路复用到输出PCollection之一.然后，通过选择将数据传递到哪个输出，可以动态地控制执行哪些步骤-可能不会执行没有任何输入的步骤和/或它们的执行可能无关紧要.

You can, however, imagine a single input, multiple output ParDo that multiplexes the input PCollection to one of the output PCollections. Then, by choosing which output to pass your data to, you can dynamically control which steps are executed -- steps without any input might not be executed and/or their execution might not matter.

一个相关的功能是参数化管道"或模板管道".这是我们非常感兴趣并且正在积极努力的事情.

A related feature is "parameterized pipelines" or "template pipelines". This is something we are very interested in and are actively working on.

这篇关于Apache Beam:跳过已经构建好的管道中的步骤的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！