无法在集群上运行 python flink 应用程序 [英] Unable to run a python flink application on cluster

问题描述

我正在尝试在独立的 Flink 集群上运行 Python Flink 应用程序.该应用程序在单节点集群上运行良好，但在多节点集群上会引发以下错误.java.lang.Exception: 用户定义的open()"方法导致异常:复制文件时出错.请帮我解决这个问题.谢谢

I am trying to run a Python Flink Application on the standalone Flink cluster. The application works fine on a single node cluster but it throws the following error on a multi-node cluster. java.lang.Exception: The user defined 'open()' method caused an exception: An error occurred while copying the file. Please help me resolve this problem. Thank you

我尝试执行的应用程序具有以下代码.

The application I am trying to execute has the following code.

from flink.plan.Environment import get_environment
from flink.plan.Constants import INT, STRING, WriteMode

env = get_environment()

data = env.from_elements("Hello")

data.map(lambda x: list(x)).output()
env.execute()

推荐答案

您必须在flink-conf.yaml"中配置python.dc.tmp.dir"以指向分布式文件系统(如 HDFS).该目录用于分发python脚本.

You have to configure "python.dc.tmp.dir" in "flink-conf.yaml" to point to a distributed filesystem (like HDFS). This directory is used to distributed the python scripts.

这篇关于无法在集群上运行 python flink 应用程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！