如何将现有的PubSub订阅与Google提供的PubSub一起使用到BigQuery Dataflow Template [英] How to use existing PubSub Subscription with Google-Provided PubSub to BigQuery Dataflow Template
问题描述
我正在尝试使用Google提供的模板PubSub to BigQuery设置数据流作业.我看到一个用于指定Cloud Pub/Sub输入订阅的选项，但是在GCP控制台UI中看不到用于指定Pub/Sub输入订阅的任何选项.
I am trying to setup a Dataflow job using the google provided template PubSub to BigQuery. I see an option to specify the Cloud Pub/Sub input topic but I don't see any option to specify Pub/Sub input subscription in GCP console UI.
如果我提供主题，则job将自动创建一个订阅以读取提供的主题中的消息.问题是，在Dataflow作业启动后，该作业将仅看到发布到该主题的消息.之前在同一主题上发表的任何内容都将被忽略.
If I provide the topic, job would automatically create a subscription to read the messages from the provided topic. Problem with this is, the job will see only messages published to the topic after the Dataflow job has started. Anything published before to the same topic would be ignored.
我的工作没有任何复杂的转变.因此，谷歌提供的模板将为我开箱即用.但是缺乏指定我自己的订阅的能力令我感到困扰.为此，我不想设置自定义作业管道.有人知道是否有解决方法吗?
I don't have any complex transformations to do in my job. So the google provided template would work for me out of the box. But the lack of ability to specify my own subscription is bothering me. I don't want to setup a custom job pipeline just for this reason. Anybody know if there is a workaround for this?
推荐答案
当前不支持.但是，它是一个很好的用例，并且受到了Google Cloud Team的关注.
That's not currently supported. However its a great use case and is on the Google Cloud Team's radar.
如果您可以通过bookman@google.com给我发送电子邮件，我一定会及时通知您有关开发的信息.
If you can email me at bookman@google.com I'll be sure to keep you posted on development to enable this.
赞赏反馈
科林
这篇关于如何将现有的PubSub订阅与Google提供的PubSub一起使用到BigQuery Dataflow Template的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
