Apache kafka 的监控 UI - kafka manager vs kafka monitor [英] Monitoring UI for Apache kafka - kafka manager vs kafka monitor

问题描述

我是 kafka 的新手.我们想要监控和管理 kafka 主题.我们尝试了不同的开源监控工具，比如

1. 解决方案

   镜头
   Lenses(前 Landoop)通过用户界面、流式 SQL 引擎和集群监控增强了 Kafka.它可以更快地监控 Kafka 数据管道.

   他们提供了一个免费的一体式 docker(Lenses Box)，可以为最多 2500 万条消息的单个代理.请注意，建议在开发环境中使用此方法.

   Cloudera SMM
   Streams Messaging Manager 是用于监控和管理运行 Cloudera 或 Hortonworks kafka 的集群的解决方案.它还具有复制功能.

   融合
   另一种选择是 Confluent Enterprise，它是用于生产环境的 Kafka 发行版.它还包括控制中心，它是Apache Kafka的管理系统，可以启用集群从用户界面进行监控和管理.

   Yahoo CMAK(Apache Kafka 集群管理器，以前称为 Kafka 管理器)
   Kafka Manager 或 CMAK 是一种用于监控 Kafka 的工具，与上述工具相比，功能较少.

   KafDrop
   KafDrop 是用于监控 Apache Kafka 集群的 UI.该工具显示代理、主题、分区等信息，甚至可以让您查看消息.它是一个轻量级的应用程序，运行在 Spring Boot 上，只需要很少的配置.

   LinkedIn Burrow
   Burrow 是 Apache Kafka 的监控伴侣，它提供消费者滞后检查服务，无需指定阈值.它监控所有消费者的承诺偏移量，并根据需要计算这些消费者的状态.提供了一个 HTTP 端点来按需请求状态，以及提供其他 Kafka 集群信息.还有一些可配置的通知程序，可以通过电子邮件或 HTTP 调用将状态发送到其他服务.

   Kafka 工具
   Kafka Tool 是一个用于管理和使用 Apache Kafka 集群的 GUI 应用程序.它提供了一个直观的 UI，允许快速查看 Kafka 集群中的对象以及存储在集群主题中的消息.它包含面向开发人员和管理员的功能.

   <小时>

   如果您负担不起许可证，请选择 Yahoo Kafka Manager、LinkedIn Burrow 或 KafDrop.Confluent 和 Landoop 的产品是最好的，但不幸的是，它们需要获得许可.

   更多细节可以参考我的博文Apache Kafka 集群的 UI 监控工具概述.

   I am new to kafka. We want to monitor and manage kafka topics. We tried different open source monitoring tools like

   1. kafka-monitor
   2. kafka-manager

   Both tools are good. But we are unable to make a decision which should be included in our deployment stack. Which one is better and why, and in which scenario?

   'kafka manager' from yahoo looks the older one and 'kafka monitor' from LinkedIn is newer one

   Kafka Monitor-

   解决方案

   Lenses
   Lenses (ex Landoop) enhances Kafka with User Interface, streaming SQL engine and cluster monitoring. It enables faster monitoring of Kafka data pipelines.

   They provide a free all-in-one docker (Lenses Box) which can serve a single broker for up to 25M messages. Note that this is recommended for development environments.

   Cloudera SMM
   Streams Messaging Manager is the solution for monitoring and managing clusters running Cloudera or Hortonworks kafka. It also comes with replication capability.

   Confluent
   Another option is Confluent Enterprise which is a Kafka distribution for production environments. It also includes Control Centre, which is a management system for Apache Kafka that enables cluster monitoring and management from a User Interface.

   Yahoo CMAK (Cluster Manager for Apache Kafka, previously known as Kafka Manager)
   Kafka Manager or CMAK is a tool for monitoring Kafka offering less functionality compared to the aforementioned tools.

   KafDrop
   KafDrop is a UI for monitoring Apache Kafka clusters. The tool displays information such as brokers, topics, partitions, and even lets you view messages. It is a lightweight application that runs on Spring Boot and requires very little configuration.

   LinkedIn Burrow
   Burrow is a monitoring companion for Apache Kafka that provides consumer lag checking as a service without the need for specifying thresholds. It monitors committed offsets for all consumers and calculates the status of those consumers on demand. An HTTP endpoint is provided to request status on demand, as well as provide other Kafka cluster information. There are also configurable notifiers that can send status out via email or HTTP calls to another service.

   Kafka Tool
   Kafka Tool is a GUI application for managing and using Apache Kafka clusters. It provides an intuitive UI that allows one to quickly view objects within a Kafka cluster as well as the messages stored in the topics of the cluster. It contains features geared towards both developers and administrators.

   ---

   If you cannot afford licenses, then go for Yahoo Kafka Manager, LinkedIn Burrow or KafDrop. Confluent's and Landoop's products are the best out there, but unfortunately, they require licensing.

   For more details, you can refer to my blog post Overview of UI Monitoring tools for Apache Kafka Clusters.

   这篇关于Apache kafka 的监控 UI - kafka manager vs kafka monitor的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！