Kafka Connect 分离日志 [英] Kafka Connect separated logging

问题描述

目前，我们正在为我们的融合 kafka 连接分布式工作者集群使用几个自定义连接器插件.困扰我很长时间的一件事是 kafka connect 将所有已部署连接器的所有日志写入一个文件/流.这使得调试成为绝对的噩梦.有没有办法让 kafka 连接在不同的文件/流中记录连接器?

Currently we are using a couple of custom connetor plugins for our confluent kafka connect distributed worker cluster. One thing that bothers me for a long time is that kafka connect writes all logs from all deployed connectors to one File/Stream. This makes debugging an absolute nightmare. Is there a way to let kafka connect log the connectors in different Files/Streams?

通过 connect-log4j.properties 我可以让特定的类记录到不同的文件/流.但这意味着对于每个额外的连接器，我都必须调整 connect-log4j.properties

Via the connect-log4j.properties I am able to let a specific class log to a different File/Stream. But this means that with every additional connector I have to adjust the connect-log4j.properties

谢谢

推荐答案

Kafka Connect 目前不支持这个.我同意这并不理想.

Kafka Connect does not currently support this. I agree that it is not ideal.

一种选择是拆分您的连接器，并为每个连接器设置一个专用的工作集群，从而分离日志文件.

One option would be to split out your connectors and have a dedicated worker cluster for each, and thus separate log files.

Kafka Connect 是 Apache Kafka 的一部分，因此您可以提出 JIRA 以进一步讨论这个，也许可以通过 PR 回馈?

Kafka Connect is part of Apache Kafka so you could raise a JIRA to discuss this further and maybe contribute it back via a PR?

编辑 2019 年 4 月 12 日:参见 https://cwiki.apache.org/confluence/display/KAFKA/KIP-449%3A+Add+connector+contexts+to+Connect+worker+logs

这篇关于Kafka Connect 分离日志的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！