更改Apache Storm拓扑的Log4J配置 [英] Change Log4J Configuration for Apache Storm Topology

问题描述

我目前正在使用以下命令通过Java应用程序以编程方式提交Storm拓扑:

I'm currently submitting Storm topologies programatically via my Java application by using the following command:

Nimbus.Client client = NimbusClient.getConfiguredClient(stormConfigProvider.getStormConfig()).getClient();
client.submitTopology(
        this.topologyID.toString(),
        stormJarManager.getRemoteJarLocation(),
        JSONValue.toJSONString(stormConfigProvider.getStormConfig()),
        topology
);

在我的场景中，我有两种拓扑.测试拓扑和生产拓扑.对于这两种拓扑，我都需要不同类型的日志记录.测试拓扑以TRACE级别运行，而生产拓扑将以INFO级别运行.另外，我要求生产拓扑具有配置的SPLUNK Log4J2附加程序，以集中生产应用程序的日志记录.

In my scenario, I have two kinds of topologies. Testing topologies and production topologies. For both kind of topologies, I require different types of logging. While the testing topologies run with TRACE level, the production topologies will run with INFO level. In addition, I require that the production topologies have a SPLUNK Log4J2 appender configured, to centralize the logging of my production application.

为此，我在配置SPLUNK附加程序的拓扑JAR中包含了一个log4j.xml文件.但是，服务器不支持log4j.xml文件.相反，Storm Server似乎使用了自己的配置.

For that, I included a log4j.xml file into my topology JAR which configures the SPLUNK appender. However, the log4j.xml file is not honored by the Server. Instead, the Storm Server seems to use its own configuration.

如何更改不同拓扑的log4j配置? (我不想在每个工作线程上修改log4j.xml).

How can I change my log4j configuration for different topologies? (I don't want to modify the log4j.xml on each worker).

推荐答案

您可以使用 https://storm.apache.org/releases/current/dynamic-log-level-settings.html 为每种拓扑设置日志级别.

You can use https://storm.apache.org/releases/current/dynamic-log-level-settings.html to set log levels for each topology.

我不确定您如何根据加载的拓扑添加/删除splunk附加程序.您可能可以通过程序 https://logging.apache来配置log4j. org/log4j/2.x/manual/customconfig.html 并在工作程序上将log4j2.configurationFactory系统属性设置为指向您的配置工厂(您可以通过将其添加到您的配置文件中的topology.worker.childopts属性来完成此操作拓扑配置).

I'm not sure how you'd add/remove the splunk appender based on the loaded topology. You might be able to configure log4j programatically https://logging.apache.org/log4j/2.x/manual/customconfig.html and set the log4j2.configurationFactory system property on your workers to point to your configuration factory (you can do this by adding it to the topology.worker.childopts property in your topology config).

仅出于上下文考虑，这里是Storm设置系统属性的地方，该属性导致Log4j加载工作人员log4j配置 https://issues上提出问题. apache.org/jira

Just for context, here's where Storm sets the system property that causes Log4j to load the worker log4j configuration https://github.com/apache/storm/blob/4137328b75c06771f84414c3c2113e2d1c757c08/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L560. If you wanted to load a log4j2.xml included in your topology jar, maybe it would be possible to conditionally exclude that setting from the system properties set for workers. I think it would require a code change though, so you'd need to raise an issue on https://issues.apache.org/jira

这篇关于更改Apache Storm拓扑的Log4J配置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！