Kafka Producer-默认情况下是否支持多线程? [英] Kafka Producer - By default supports Multithreading?

问题描述

我是kafka的新手.我已经使用kafka_2.9.2-0.8.1.1创建了示例kafka同步生产者和消费者组程序.所以我的问题是，我是否需要向生产者添加多线程代码(例如consumergroup类)以支持大量请求?我读过生产者发送方法是线程安全的.因此，kafka生产者将在内部处理多线程概念，还是开发者必须显式地编写代码?

I am newbie to kafka. I have created sample kafka sync producer and consumergroup programs using kafka_2.9.2-0.8.1.1. So My question is, do I need to add multithreading code to producer (like consumergroup class has) to support huge number of requests? I read producer send method is thread safe. So kafka producer will take care of multithreading concepts internally or developer has to code explicitly?

任何帮助将不胜感激.

谢谢，Cdhar

推荐答案

Kafka提供两种类型的生产者.(1)SyncProducer(2)AsyncProducer.如果将 producer.type 配置设置为 async ，它将使用 AsyncProducers .默认情况下，它使用同步生产器类.

There are two types of producers available with Kafka. (1) SyncProducer (2) AsyncProducer. If you set the producer.type configuration as async it will uses the AsyncProducers. By default it uses the Synchronous producer class.

一旦以异步模式运行，它将为每个代理创建一个单独的AsyncProducer实例，并且这些AsyncProducer实例中的每一个都维护自己的内部后台线程以发送消息.这些被称为 ProducerSendThread .

Once running in async mode it creates a separate AsyncProducer instance per broker.And each of these AsyncProducer instances maintains its own internal background thread for sending the messages. These are called ProducerSendThread.

因此，每个代理运行一个线程，并且您的并行性基于集群中可用的代理数量.因此，在集群中添加新的代理应该为您提供灵活性，以在使用Kafka生成数据时提高并行度.但是请记住，在集群中添加新的代理也应考虑其他参数.

So there is one thread running per broker and your parallelism is based on the number of brokers available in the cluster. So adding new brokers in the cluster should provide you the flexibilities to increase the level of parallelism while producing data using Kafka.But remember adding a new broker to your cluster should be considered taking other paramaters also into consideration.

这篇关于Kafka Producer-默认情况下是否支持多线程?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！