Apache Beam Wait.on JdbcIO.write with unbounded PCollection 问题 [英] Apache Beam Wait.on JdbcIO.write with unbounded PCollection issue

问题描述

我正在尝试将以下场景与无界 pCollection 数据源(PubSub)一起使用.https://issues.apache.org/jira/browse/BEAM-6732

I am trying use the below scenario with unbounded pCollection datasource(PubSub). https://issues.apache.org/jira/browse/BEAM-6732

我能够写入 DB1.DB2 有一个 Wait.on DB1 (PCollection .withResults).但不幸的是，DB2 没有得到更新.

I am able to write to DB1. DB2 is having a Wait.on DB1 (PCollection .withResults). But unfortunately DB2 is not getting updated.

当我将源更改为有界虚拟 PCollection 时，它可以工作.任何输入表示赞赏.

When I change the source to a bounded dummy PCollection, it works. Any input is appreciated.

推荐答案

正如我在 Jira 上提到的，您的无限管道中是否使用了任何 Windows?只有在相应的窗口完全写入第一个数据库后，才会开始写入另一个数据库.否则，它将永远等待.

As I mentioned on Jira, do you have any Windows using in your unbounded pipeline? The writing to another database starts only after the respective window has been fully written to the first database. Otherwise, it will wait forever.

这篇关于Apache Beam Wait.on JdbcIO.write with unbounded PCollection 问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！