如何在 Flink 中加载外部 jars [英] How to load external jars in Flink

问题描述

我在单机集群模式下向Flink提交作业时，发现每次taskManager都会从jobManager中取jar(即使是同一个jar)，耗时很长.我想知道是否可以将这些 jar 保存在每个工作节点中，以便它们每次运行时自动在本地加载 jar.

When I submit jobs to Flink in a standalone cluster mode, I find each time the taskManager will fetch the jar from the jobManager (even for the same jar), which takes a long time. I am wondering whether it is possible to keep these jars in each worker node such that they will automatically load the jars locally for each run.

推荐答案

你可以用你的内部代码/逻辑创建一个最小的 jar，并确保你依赖的所有 jar 在 /lib 中可用 文件夹(根据 Arthur 的建议).这将导致从 JobManager 获取 jar 的时间最短.此 Flink 文档 说应该可以(在某些情况下)避免加载任何动态 jar，但我从未尝试过.

You can create a minimal jar with just your internal code/logic, and ensure that all of the jars you depend on are available in the /lib folder (as per Arthur's suggestion). That will result in minimal time fetching the jar from the JobManager. This Flink documentation says it should be possible (in some situations) to avoid having to load any dynamic jars, but I've never tried that.

这篇关于如何在 Flink 中加载外部 jars的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！