卡夫卡查询 [英] Kafka Queries
问题描述
我有以下问题关于Kafkka。如果您能解决这些问题，我们是否会感激？
I have the following questions with regards to Kafkka. Appreciate if you can address them ?
1.我们是否支持HDInsight中的故障转移 Kafka经纪人？
1.Do we have the support for failover in HDInsight Kafka brokers?
2.在Kafka经纪人重启期间是否会有任何数据丢失。
2.Will there be any loss of data during a Kafka broker reboot.
3.Will HDInsight Kafka <跨度>&NBSP; 自动更改领导者&如果代理发生故障，主题分区的副本配置将不会
是数据丢失？
3.Will HDInsight Kafka automatically change the leader & replica configuration for a topic partition if a broker goes down so that there won’t be data loss?
4.一旦领导者（经纪人节点）回来，它会自动恢复配置吗？
4.Will it automatically revive the configuration once the leader(broker node) is back?
推荐答案
嗨Raven，
Hi Raven,
HDInsight Kafka是100％开源的Kafka，所以打开的一切来源Kafka也支持我们的支持。
HDInsight Kafka is the 100% open source Kafka, so everything that open source Kafka supports we support as well.
在云环境中，代理可能由于各种原因（例如VM在主机上移动）或者由于内存不足而导致代理重启而失败（客户正在运行）一些占用资源的进程或者如果客户在小型虚拟机上运行太多分区
等等。
In a cloud environment brokers can fail due to various reasons such as VMs moving across host machines, or simply broker restarts due to out of memory (customer running some processes which eat up resources or if the customer is running too many partitions on small VMs etc).
在HDInsight中，我们为客户提供了选择运行Kafka群集的灵活性。
In HDInsight we give customers the flexibility to choose how they want to run their Kafka cluster.
为避免代理重启期间数据丢失是我们的建议：
In order to avoid data loss during broker restarts here are our recommendations:
- 将min.insync.replicas设置为3
- 在生产者中设置ACKS = all
- Unclean.leader.election.enabled = false
- 每个代理不超过4000个分区。
- 避免重新启动代理进程
对于Kafka消息丢失情况的必要阅读，我建议
For required reading on Kafka message loss scenarios I would recommend
https://jack-vanlightly.com/blog/2018/9/14/how-to-lose-messages- on-a-kafka-cluster-part1
https://jack-vanlightly.com/blog/2018/9/18/how-to-lose-messages-on-a-kafka-cluster-part -2
希望这会有所帮助。
这篇关于卡夫卡查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
