Partitions如何拆分成Kafka Broker? [英] How Partitions are split into Kafka Broker?

问题描述

我知道分区是在 Kafka Broker 之间拆分的.但分裂是基于什么?.例如，如果我有 3 个代理和 6 个分区，如何确保每个代理将有 2 个分区?Kafka 目前是如何进行这种拆分的?

I know that partitions are split across Kafka Broker. But the split is based on what ?. For instance, if I have 3 brokers and 6 partitions, how to ensure that each broker will have 2 partitions ? How this split is currently made in Kafka ?

推荐答案

分配策略是一个内部实现细节，没有记录，因为它可以随时更改.因此，您不应该依赖此算法保持不变.此外，您无法影响/配置此内部策略.

Assignment policy is an internal implementation detail and not documented as it can get changed at any point in time. Thus, you should not rely the this algorithms stay the same. Furthermore, there is nothing you can do to influent/configure this internal strategy.

基本策略是确保负载平衡，即将分区分配给分配的分区比其他代理少的代理.因此，对于您的示例，每个代理将分配 2 个分区.

The basic policy is to ensure load balancing, i.e., it assigns partitions to brokers that have less partitions assigned than other. Thus, for your example each broker will get 2 partitions assigned.

此外，对于分区副本，应用了机架感知"策略(从 Kafka 0.10 开始).这意味着如果可能的话，分区副本被分配到不同的机架，以在整个机架发生故障时再次保护离线分区.有关更多详细信息，请参阅此 KIP:https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment

Furthermore, for partitions replicas a "rack aware" policy is applied (as of Kafka 0.10). That means that partition replicas are assigned to different racks if possible to guard again offline partitions if a whole rack fails. See this KIP for more details about this: https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment

这篇关于Partitions如何拆分成Kafka Broker?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！