如何在 Flink 中引用外部 Jar [英] How to Reference the External Jar in Flink

问题描述

大家.我尝试在 Flink 中引用我的公司 jar，以将其复制到所有任务管理器中的 $FLINK/lib 中，但失败了.而且我不想打包一个胖罐子，太重太浪费时间了.我认为第一种方法也不是一个好主意，因为我必须在整个集群中管理 jars.任何人都知道如何解决这个问题?任何建议将不胜感激.

everyone. I tried to reference my company jar in Flink in the way of copying it to $FLINK/lib in all of taskmanagers, but failed. And I don't want to package a fat jar, which is too heavy and waste of time. I think the first method is also not a good idea, because I have to manager jars in the whole cluster. Anyone kowns how to resolve this problem? Any suggestion would be appreciated.

推荐答案

一般来说，构建一个胖 jar 是最好的方法.不确定你的远罐有多大，你认为它太重了"?

In general, building a fat jar is the best way to go. Not sure how big your far jar gets, that you thinks it is "too heavy"?

将 jar 复制到 $FLINK/lib 应该可以工作.但是，您需要重新启动 Flink，以便将 jars 添加到 Flink 的类路径中.因此，这种方法不允许动态添加 jar——但是它应该适用于一堆稳定的 jar.

Copying jars to $FLINK/lib should work. However, you need to restart Flink such that the jars are added to Flink's classpath. Thus, this approach does not allow to dynamically add jars -- it should work for a bunch of stable jars however.

为了管理整个集群中的 jar，使用 NFS 文件夹作为 $FLINK/lib 来保持所有 TaskManager 同步可能会有所帮助.或者您只需编写一个 bash 脚本来分发您的 jar.

In order to manage jars in the whole cluster, it might be helpful to use a NFS folder as $FLINK/lib to keep all TaskManagers in sync. Or you simple write a bash script to distribute your jars.

这篇关于如何在 Flink 中引用外部 Jar的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！