更改 Apache Storm 拓扑的 Log4J 配置 [英] Change Log4J Configuration for Apache Storm Topology

问题描述

我目前正在使用以下命令通过我的 Java 应用程序以编程方式提交 Storm 拓扑:

I'm currently submitting Storm topologies programatically via my Java application by using the following command:

Nimbus.Client client = NimbusClient.getConfiguredClient(stormConfigProvider.getStormConfig()).getClient();
client.submitTopology(
        this.topologyID.toString(),
        stormJarManager.getRemoteJarLocation(),
        JSONValue.toJSONString(stormConfigProvider.getStormConfig()),
        topology
);

在我的场景中，我有两种拓扑.测试拓扑和生产拓扑.对于这两种拓扑，我需要不同类型的日志记录.测试拓扑以 TRACE 级别运行，而生产拓扑将以 INFO 级别运行.此外，我要求生产拓扑配置了 SPLUNK Log4J2 appender，以集中记录我的生产应用程序.

In my scenario, I have two kinds of topologies. Testing topologies and production topologies. For both kind of topologies, I require different types of logging. While the testing topologies run with TRACE level, the production topologies will run with INFO level. In addition, I require that the production topologies have a SPLUNK Log4J2 appender configured, to centralize the logging of my production application.

为此，我在配置 SPLUNK 附加程序的拓扑 JAR 中包含了一个 log4j.xml 文件.但是，服务器不支持 log4j.xml 文件.相反，Storm Server 似乎使用了自己的配置.

For that, I included a log4j.xml file into my topology JAR which configures the SPLUNK appender. However, the log4j.xml file is not honored by the Server. Instead, the Storm Server seems to use its own configuration.

如何针对不同的拓扑更改我的 log4j 配置?(我不想修改每个 worker 上的 log4j.xml).

How can I change my log4j configuration for different topologies? (I don't want to modify the log4j.xml on each worker).

推荐答案

您可以使用 https://storm.apache.org/releases/current/dynamic-log-level-settings.html 为每个拓扑设置日志级别.

You can use https://storm.apache.org/releases/current/dynamic-log-level-settings.html to set log levels for each topology.

我不确定您将如何根据加载的拓扑添加/删除 splunk appender.您也许能够以编程方式配置 log4j https://logging.apache.org/log4j/2.x/manual/customconfig.html 并将您的工作人员的 log4j2.configurationFactory 系统属性设置为指向您的配置工厂(您可以通过将其添加到拓扑配置中的 topology.worker.childopts 属性).

I'm not sure how you'd add/remove the splunk appender based on the loaded topology. You might be able to configure log4j programatically https://logging.apache.org/log4j/2.x/manual/customconfig.html and set the log4j2.configurationFactory system property on your workers to point to your configuration factory (you can do this by adding it to the topology.worker.childopts property in your topology config).

仅作为上下文，这里是 Storm 设置导致 Log4j 加载 worker log4j 配置的系统属性 https://github.com/apache/storm/blob/4137328b75c06771f84414c3c2113e2d1c757c08/mainstorm-server/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L560.如果您想加载包含在拓扑 jar 中的 log4j2.xml，也许可以有条件地从为工作人员设置的系统属性中排除该设置.我认为这需要更改代码，因此您需要在 https://issues 上提出问题.apache.org/jira

Just for context, here's where Storm sets the system property that causes Log4j to load the worker log4j configuration https://github.com/apache/storm/blob/4137328b75c06771f84414c3c2113e2d1c757c08/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L560. If you wanted to load a log4j2.xml included in your topology jar, maybe it would be possible to conditionally exclude that setting from the system properties set for workers. I think it would require a code change though, so you'd need to raise an issue on https://issues.apache.org/jira

这篇关于更改 Apache Storm 拓扑的 Log4J 配置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！