如何在Nifi中查看Kafka消耗的消息? [英] How do I view the consumed messages of Kafka in Nifi?

问题描述

我已经开始了一个Nifi进程(使用Kafka)并将其连接到一个主题.它正在运行，但是我无法(不知道)在哪里可以查看消息?

I have started a Nifi process(Consume Kafka) and connected it to a topic. It is running but I am not able to (don't know) where can I view the messages?

推荐答案

ConsumeKafka处理器运行并为每个消息生成流文件.只有将处理器连接到其他组件(例如另一个处理器或输出端口)时，您才能看到正在移动的数据.

ConsumeKafka processor runs and generates flowfile for each message. Only when you connect a processor to other components like another processor or an output port, will you be able to visualize the data being moved through.

对于初学者，您可以尝试以下方法:

For starters you can try this:

• 将ConsumeKafka与LogAttribute或任何其他处理器连接 那件事.
• 停止或禁用LogAttribute处理器.
• 现在什么时候 您启动ConsumeKafka，所有从 配置的Kafka主题将以流文件的形式排队.
• 右键单击流文件排队的关系，然后 单击List Queue，您可以访问队列.
• 单击上的任何项目 在队列中，将显示一个上下文菜单.单击View按钮，您 可以看到数据.

• Connect ConsumeKafka with LogAttribute or any other processor for that matter.
• Stop or disable the LogAttribute processor.
• Now when you start ConsumeKafka, all the received messages from the configured Kafka topic will be queued up in the form of flowfiles.
• Right click that relationship where the flowfiles are queued up and click List Queue and you can access the queue.
• Click any item on the queue, a context menu will come up. Click View button and you can see the data.

查看" Kafka消息的全部说明只是为了帮助您调试并开始使用NiFi.理想情况下，您将使用其他NiFi处理器来确定用例.

This whole explanation of "viewing" the Kafka message is just to help you in debugging and get started with NiFi. Ideally you would be using other NiFi processors to work out your usecase.

示例

您收到来自Kafka的消息，并希望将其写入MongoDB，因此您可以将其流程如下: 注意: 有基于记录的处理器，例如ConsumeKafkaRecord和PutMongoRecord，但是它们基本上在做同样的事情，但具有更多的增强功能.由于您是新手，所以我建议了一个简单的流程.您可以找到有关基于Record的处理器的详细信息

You receive messages from Kafka and wants to write it to MongoDB, so you can have the flow as: Note: There are record based processors like ConsumeKafkaRecord and PutMongoRecord but they are basically doing the same thing with more enhancements. Since you're new to this, I have suggested a simple flow. You can find details about the Record based processors here and try that.

这篇关于如何在Nifi中查看Kafka消耗的消息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！