如何在Google Cloud Dataflow中停止流式传输管道 [英] How to stop a streaming pipeline in google cloud dataflow

问题描述

我正在运行一个Streaming数据流来读取PUB/SUB订阅.

I have a Streaming dataflow running to read the PUB/SUB subscription.

经过一段时间或可能在处理了一定数量的数据之后，我希望管道自行停止.我不希望我的计算引擎实例无限期运行.

After a period of a time or may be after processing certain amount of data, i want the pipeline to stop by itself. I don't want my compute engine instance to be running indefinitely.

当我通过数据流控制台取消作业时，它显示为失败的作业.

When i cancel the job through dataflow console, it is shown as failed job.

有没有办法做到这一点?我想念什么吗?或者API中缺少该功能.

Is there a way to achieve this? am i missing something ? Or that feature is missing in the API.

推荐答案

您能做这样的事情吗?

Pipeline pipeline = ...;
... (construct the streaming pipeline) ...
final DataflowPipelineJob job =
    DataflowPipelineRunner.fromOptions(pipelineOptions)
                          .run(pipeline);
Thread.sleep(your timeout);
job.cancel();

这篇关于如何在Google Cloud Dataflow中停止流式传输管道的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！