无法在集群上运行python flink应用程序 [英] Unable to run a python flink application on cluster
问题描述
我正在尝试在独立的 Flink 集群上运行 Python Flink应用程序.该应用程序在单节点群集上可以正常运行,但在多节点群集上会引发以下错误. java.lang.Exception: The user defined 'open()' method caused an exception: An error occurred while copying the file
.请帮助我解决此问题.谢谢
I am trying to run a Python Flink Application on the standalone Flink cluster. The application works fine on a single node cluster but it throws the following error on a multi-node cluster. java.lang.Exception: The user defined 'open()' method caused an exception: An error occurred while copying the file
. Please help me resolve this problem. Thank you
我要执行的应用程序具有以下代码.
The application I am trying to execute has the following code.
from flink.plan.Environment import get_environment
from flink.plan.Constants import INT, STRING, WriteMode
env = get_environment()
data = env.from_elements("Hello")
data.map(lambda x: list(x)).output()
env.execute()
推荐答案
您必须在"flink-conf.yaml"中配置"python.dc.tmp.dir"以指向分布式文件系统(例如HDFS).该目录用于分发python脚本.
You have to configure "python.dc.tmp.dir" in "flink-conf.yaml" to point to a distributed filesystem (like HDFS). This directory is used to distributed the python scripts.
这篇关于无法在集群上运行python flink应用程序的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!