apache_beam.examples.wordcount 的输入 [英] Input of apache_beam.examples.wordcount

问题描述

我试图运行 Beam Python-SDK 示例，但在读取输入时遇到问题.

I was trying to run the beam Python-SDK example, but I had problem in reading the input.

https://cwiki.apache.org/confluence/display/BEAM/Usage+Guide#UsageGuide-RunaPython-SDKPipeline

当我使用 gs://dataflow-samples/shakespeare/kinglear.txt 作为输入时，错误是 apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://dataflow-samples/shakespeare/kinglear.txt': TypeError("__init__() 得到了一个意外的关键字参数 'response_encoding'",)}

when I used gs://dataflow-samples/shakespeare/kinglear.txt as the input, the error was apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://dataflow-samples/shakespeare/kinglear.txt': TypeError("__init__() got an unexpected keyword argument 'response_encoding'",)}

当我使用我的本地文件时，它似乎并没有真正读取文件，也没有输出任何内容.结果不包括has_job"，我不确定它的实际含义 https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount.py#L118.

when I used my local file, it seemed it didn't actually read the file, and output nothing. The result didn't include 'has_job', which I am not sure what it actually means https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount.py#L118.

推荐答案

此错误是因为您的 google-apitools 版本太旧.您需要 v0.5.23 或更高版本才能避免此错误.

This error is because your version of google-apitools is too old. You need v0.5.23 or newer to avoid this error.

Beam python 对依赖项有严格的版本要求.如果您运行的是 Beam 2.11，则需要使用 google-apitools v0.5.26.

Beam python has strict version requirements for dependencies. If you are running Beam 2.11, you need to be using google-apitools v0.5.26.

这篇关于apache_beam.examples.wordcount 的输入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！