分区如何划分为Kafka Broker? [英] How Partitions are split into Kafka Broker?

问题描述

我知道分区是通过Kafka Broker划分的.但是拆分基于什么?例如，如果我有3个代理和6个分区，那么如何确保每个代理将有2个分区?目前如何在Kafka中进行此拆分?

I know that partitions are split across Kafka Broker. But the split is based on what ?. For instance, if I have 3 brokers and 6 partitions, how to ensure that each broker will have 2 partitions ? How this split is currently made in Kafka ?

推荐答案

分配策略是内部实施细节，没有记录在案，因为它可以随时更改.因此，您不应该依赖于此算法保持不变.此外，您无法采取任何措施来影响/配置这种内部策略.

Assignment policy is an internal implementation detail and not documented as it can get changed at any point in time. Thus, you should not rely the this algorithms stay the same. Furthermore, there is nothing you can do to influent/configure this internal strategy.

基本策略是确保负载平衡，即，将分区分配给分配的分区比其他分区少的代理.因此，在您的示例中，每个代理将分配2个分区.

The basic policy is to ensure load balancing, i.e., it assigns partitions to brokers that have less partitions assigned than other. Thus, for your example each broker will get 2 partitions assigned.

此外，对于分区副本，将应用机架感知"策略(从Kafka 0.10开始).这意味着，如果可能，则会将分区副本分配给不同的机架，以在整个机架出现故障时再次保护脱机分区.有关此详细信息，请参阅此KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment

Furthermore, for partitions replicas a "rack aware" policy is applied (as of Kafka 0.10). That means that partition replicas are assigned to different racks if possible to guard again offline partitions if a whole rack fails. See this KIP for more details about this: https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment

这篇关于分区如何划分为Kafka Broker?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！