带有无限制PCollection问题的Apache Beam Wait.on JdbcIO.write [英] Apache Beam Wait.on JdbcIO.write with unbounded PCollection issue

问题描述

我正在尝试将以下情况与无限制的pCollection数据源(PubSub)结合使用. https://issues.apache.org/jira/browse/BEAM-6732

I am trying use the below scenario with unbounded pCollection datasource(PubSub). https://issues.apache.org/jira/browse/BEAM-6732

我能够写DB1.DB2正在使用一个Wait.on DB1(PCollection .withResults).但是不幸的是，DB2没有得到更新.

I am able to write to DB1. DB2 is having a Wait.on DB1 (PCollection .withResults). But unfortunately DB2 is not getting updated.

当我将源更改为有界虚拟PCollection时，它可以工作.任何输入表示赞赏.

When I change the source to a bounded dummy PCollection, it works. Any input is appreciated.

推荐答案

正如我在Jira上提到的，您在无限制的管道中使用任何Windows吗?仅在各个窗口已完全写入第一个数据库之后，才开始写入另一个数据库.否则，它将永远等待.

As I mentioned on Jira, do you have any Windows using in your unbounded pipeline? The writing to another database starts only after the respective window has been fully written to the first database. Otherwise, it will wait forever.

这篇关于带有无限制PCollection问题的Apache Beam Wait.on JdbcIO.write的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！