Kafka Connect和Kafka Broker版本兼容性 [英] Kafka Connect and Kafka Broker version compatibility

问题描述

我们有一个运行代理版本为Apache Kafka 0.11的 Kerberized Kafka集群。该集群由

We have a "Kerberized Kafka cluster" running brokers version Apache Kafka 0.11.This cluster is managed by a different team and we dont have any control over this.

我们现在正在尝试在我们自己的K8S群集上安装Kafka Connect群集。

We are now trying to install Kafka Connect cluster on our own K8S cluster.

我们正在遵循以下兼容性列表 https：// docs .confluent.io / current / installation / versions-interoperability.html

We were following this compatibility matrix https://docs.confluent.io/current/installation/versions-interoperability.html

因此，我们必须坚持使用Confluent Platform 3.3.3图像作为Schema Registrys和Kafka Connect pods，因为Brokers版本是0.11

According to this, we had to stick with Confluent Platform 3.3.3 images for Schema Registry and Kafka Connect pods since the Brokers version is 0.11

我们还想启用在这里实现的Connect JMX指标： https://cwiki.apache.org/confluence/display/KAFKA/KIP-196%3A + Add + metrics + to + Kafka + Connect + framework

Also we want to enable Connect JMX metrics which was implemented here : https://cwiki.apache.org/confluence/display/KAFKA/KIP-196%3A+Add+metrics+to+Kafka+Connect+framework

根据文档，Kafka 1.0.0支持这些JMX度量标准

According to the documentation these JMX metrics are supported in Kafka 1.0.0

问题如下：

1）我们可以将最新的Confluent映像（例如：5.2.0版）与Kafka Brokers 0.11版一起使用吗？

1) Can we use latest Confluent images (eg: version 5.2.0) with Kafka Brokers version 0.11?

2）使用当前设置（Confluent 3.3.3），我已经启用了所有JMX指标，但没有看到任何基于Kafka Connector的指标。
我看到了一般的Kafka Connect指标，但没有看到每个连接器的指标（例如，源和接收器指标）。是因为我们使用的是旧版CP版本？

2) With current setup (Confluent 3.3.3) I have enabled all the JMX metrics but we don't see any Kafka Connector based metrics. I see general Kafka Connect metrics, but not metrics per each Connector(eg. Source and Sink metrics).Is this because of the old CP version we use ?

推荐答案

Confluent Platform 3.2和更高版本中包含的Kafka Connect Workers与Confluent Platform 3.0和更高版本中包含的任何Kafka代理兼容

Kafka Connect Workers that are included in Confluent Platform 3.2 and later are compatible with any Kafka broker that is included in Confluent Platform 3.0 and later

注意：Kafka Connect不是Confluent Platform功能，而是基于普通的Apache Kafka API构建的。因此，使用Kafka 1.0和2.0客户端库的Connect实例仍可与Kafka 0.11代理一起使用，以用于基本通信协议

Note: Kafka Connect is not a Confluent Platform feature and is built over plain Apache Kafka API. Therefore, Connect instances using Kafka 1.0 and 2.0 client libraries still work with Kafka 0.11 brokers for basic communication protocols

如果您遇到Kerberos问题，则这些问题可能无关

If you're having Kerberos issues, those are likely unrelated

我看到了一般的Kafka Connect指标，但没有看到每个连接器的指标

I see general Kafka Connect metrics, but not metrics per each Connector

通常，每个连接器指标都没有这样的指标，例如读/写到数据库的记录数，或者至少在很大程度上取决于连接器是否实际实现自己的指标，但是， 3.3.x版本全部基于Kafka 0.11，而不是1.0，其中添加了更多的Connect API指标

In general, there is no such per connector metric such as "number of records read/written to a database", or at least, it would largely depend if the connectors actual implement their own metrics, but yes, the 3.3.x releases are all based on Kafka 0.11, not 1.0, where more Connect API metrics were added

这篇关于Kafka Connect和Kafka Broker版本兼容性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！