如何覆盖 Apache Flink 中的配置值? [英] How could I override configuration value in Apache Flink?

问题描述

我正在尝试将指标从 Apache Flink 收集到 Prometheus 中.Flink 文档说我需要在 flink-conf.yaml 中添加以下几行:

I'm trying to gather metrics from Apache Flink into Prometheus. Flink documentation says that I need to add following lines to my flink-conf.yaml:

metrics.reporter.promgateway.class: org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter
metrics.reporter.promgateway.host: localhost
metrics.reporter.promgateway.port: 9091
metrics.reporter.promgateway.jobName: myJob

我想在 Prometheus 中用不同的名称标记不同的作业.如何在每个作业的基础上覆盖配置参数 metrics.reporter.promgateway.jobName(每个作业都在自己的 Flink 集群会话中运行)?

I want to mark different jobs with different names inside of Prometheus. How could I override configuration parameter metrics.reporter.promgateway.jobName on per-job basis (each job is running inside of its own Flink cluster session)?

有几个问题:

• 我无法覆盖 flink-conf.yaml.我发现只有 FLINK_CONF_DIR 参数可以覆盖整个配置目录.但是，为每个作业覆盖配置目录似乎不是一个正确的解决方案.
• 我无法覆盖 StreamExecutionEnvironment 的初始配置，因为它是在 StreamExecutionEnvironment.getExecutionEnvironment 方法内部构造的，并且在环境初始化后无法修改.

• I can't override flink-conf.yaml. I've found only FLINK_CONF_DIR parameter to override whole configuration directory. But it doesn't look like a right solution to override configuration directory for every single job.
• I can't override initial configuration of StreamExecutionEnvironment because it is being constructed inside of StreamExecutionEnvironment.getExecutionEnvironment method and can't be modified after environment's initialization.

推荐答案

可以通过在启动 Flink 作业集群时指定动态属性来修改有效配置.假设您要部署到 Yarn，命令将如下所示:

You can modify the effective configuration by specifying a dynamic property when starting a Flink job cluster. Assuming that you are deploying to Yarn the command would look like:

bin/flink run -m yarn-cluster -yD metrics.reporter.promgateway.jobName=myCustomJob <USER_CODE_JAR>

动态属性被发送到 Yarn 集群并覆盖现有的配置键值对.

The dynamic properties are sent to the Yarn cluster and overwrite existing configuration key value pairs.

这篇关于如何覆盖 Apache Flink 中的配置值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！