将路由器数据推送到分布式消息系统 [英] Pushing Router Data to DIstributed Messaging System

问题描述

查询:将路由器作为kafka群集的生产者的接口.

问题:我的路由器接口正在尝试将数据推送到运行kafka的端口. (默认为9092).

问1但是kafka经纪人可以在不创建主题的情况下接受此数据吗?

问: 2 kafka使用者可以在不指定主题的情况下提取数据吗?

      If yes, How ?

      If not, What is work around this and how can i achieve this ?

第一次

我刚刚检查了Kafka经纪人配置是否具有"auto.create.topics.enable"字段. 如果我将其设置为true并在不使用"kafka-console-producer"命令的情况下在kafka的端口上发送一些随机数据，那么kafka经纪人会接受吗?

谢谢.

解决方案

kafka经纪人可以在不创建主题的情况下接受此数据

默认情况下，如果您尝试将数据推送到Kafka代理，但该代理尚不存在，则该代理会创建主题.此行为由auto.create.topics.enable参数(具有true/false值)控制.

但是经纪人完全不能在没有创建主题的情况下接受数据.是的，必须在生产者端指定主题名称.

kafka用户是否可以在不指定主题的情况下提取数据

不，不能.

如果没有，该如何解决?我该如何实现?

您必须以某种方式检索主题名称，例如，通过api列出所有主题并选择适当的主题，或者您可以直接深入Zookeeper内部，尽管我不建议您这样做.

我刚刚检查了Kafka经纪人配置是否具有"auto.create.topics.enable"字段.如果我将其设置为true并在不使用"kafka-console-producer"命令的情况下在kafka的端口上发送一些随机数据，那么kafka经纪人会接受吗?

当然不是. Kafka使用它自己的协议来建立生产者与经纪人，经纪人与消费者之间的联系，而且这些消息是经过专门编码的.

Query: Making an interface of router as the producer of kafka cluster.

Issue: My router's interface is trying to push the data to the port on which kafka is running. (by default 9092).

Q. 1 But can the kafka broker accept this data without a topic being created ?

Q. 2 Can a kafka consumer pull data without specifying a topic ?

      If yes, How ?

      If not, What is work around this and how can i achieve this ?

1st edit:

I just checked that Kafka broker configs have "auto.create.topics.enable" field. If i set this as true and send some random data on kafka's port without using "kafka-console-producer" command, will the kafka broker accept it?

Thank You.

解决方案

can the kafka broker accept this data without a topic being created

By default Kafka broker is set up in a way that it creates topic if you're trying to push data to it and it does not exists yet. This behaviour is controlled by auto.create.topics.enable parameter (with true/false values).

But broker cannot accept data without topic creation at all. And yes, topic name has to be specified on a producer side.

Can a kafka consumer pull data without specifying a topic

No, it cannot.

If not, What is work around this and how can i achieve this ?

You have to somehow retrieve topic name, for example, by listing all topics through the api and picking proper one, or you can dig directly into zookeeper internals, though I would not recommend you to do that.

I just checked that Kafka broker configs have "auto.create.topics.enable" field. If i set this as true and send some random data on kafka's port without using "kafka-console-producer" command, will the kafka broker accept it?

Of course not. Kafka uses it's own protocol to establish connection between producer and broker, broker and consumer, moreover, those messages are encoded specifically.

这篇关于将路由器数据推送到分布式消息系统的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！