数据流似乎卡住了3 [英] Dataflow appears to be stuck 3

问题描述

我在数据流日志中收到此错误:

I got this error on Dataflow log:

(11b55950300ec4ce):工作流失败.原因:(11b55950300ec459):数据流似乎卡住了.请通过以下方式与数据流小组联系 http://stackoverflow.com/questions/tagged/google-cloud-dataflow .

工作ID为: 2017-11-13_20_58_16-5812097262489045871

我尝试了完全相同的代码，它正在其他项目上工作...

I tried the exact same code and it is working on other project...

感谢您的帮助，

Yosua

推荐答案

由于默认网络区域(在本例中为 asia-southeast1 )与区域参数(默认: us-central1-f ).

It turns the dataflow cannot spawn workers because of mismatch between the default network's region (in asia-southeast1 in my case) and the zone parameters (default: us-central1-f).

因此，要解决此问题，我需要在管道参数中显式设置网络，子网和 zone .

So to solve this issue I need to explicitly set the network, subnet, and zone i the pipeline parameter.

注意:我发现这很不直观，但是可以将 region 设置为 us-central1 并将区域设置在 us-central1 之外，因为 region 参数是我们部署作业的位置，而 zone 是我们生成工作程序的位置.

Note: I found this unintuitive but it is okay to have region as us-central1 and zone outside us-central1 since the region parameter is where we deploy the jobs and zone is where we spawn the worker.

参考: https://cloud.google.com/dataflow/pipelines/specifying-exec-params

这篇关于数据流似乎卡住了3的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！