如何在Nifi中查看Kafka的消费消息? [英] How do I view the consumed messages of Kafka in Nifi?

问题描述

我已经启动了一个 Nifi 进程(消费 Kafka)并将其连接到一个主题.它正在运行，但我无法(不知道)在哪里可以查看消息?

I have started a Nifi process(Consume Kafka) and connected it to a topic. It is running but I am not able to (don't know) where can I view the messages?

推荐答案

ConsumeKafka 处理器运行并为每条消息生成流文件.只有当您将处理器连接到其他组件(如另一个处理器或输出端口)时，您才能可视化正在传输的数据.

ConsumeKafka processor runs and generates flowfile for each message. Only when you connect a processor to other components like another processor or an output port, will you be able to visualize the data being moved through.

对于初学者，你可以试试这个:

For starters you can try this:

• 将 ConsumeKafka 与 LogAttribute 或任何其他处理器连接起来那件事.
• 停止或禁用 LogAttribute 处理器.
• 现在什么时候你启动 ConsumeKafka，所有从配置好的 Kafka 主题将以流文件的形式排队.
• 右键单击流文件排队的关系，然后点击List Queue就可以访问队列了.
• 点击任何项目队列，将出现一个上下文菜单.点击View按钮，你可以看到数据.

• Connect ConsumeKafka with LogAttribute or any other processor for that matter.
• Stop or disable the LogAttribute processor.
• Now when you start ConsumeKafka, all the received messages from the configured Kafka topic will be queued up in the form of flowfiles.
• Right click that relationship where the flowfiles are queued up and click List Queue and you can access the queue.
• Click any item on the queue, a context menu will come up. Click View button and you can see the data.

查看"Kafka 消息的整个解释只是为了帮助您进行调试并开始使用 NiFi.理想情况下，您会使用其他 NiFi 处理器来计算您的用例.

This whole explanation of "viewing" the Kafka message is just to help you in debugging and get started with NiFi. Ideally you would be using other NiFi processors to work out your usecase.

示例

您收到来自 Kafka 的消息并希望将其写入 MongoDB，因此您可以将流程设为:注意:有像 ConsumeKafkaRecord 和 PutMongoRecord 这样的基于记录的处理器，但它们基本上做同样的事情，但有更多的增强.由于您对此不熟悉，因此我建议了一个简单的流程.您可以找到有关基于 Record 的处理器的详细信息 此处 并尝试.

You receive messages from Kafka and wants to write it to MongoDB, so you can have the flow as: Note: There are record based processors like ConsumeKafkaRecord and PutMongoRecord but they are basically doing the same thing with more enhancements. Since you're new to this, I have suggested a simple flow. You can find details about the Record based processors here and try that.

这篇关于如何在Nifi中查看Kafka的消费消息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！