为什么 kafka 中两个 Kstream 的共同分区需要两个流的分区数相同? [英] Why does co-partitioning of two Kstreams in kafka require same number of partitions for both the streams?

问题描述

我想知道为什么 kafka 中的两个 Kstream 的共同分区需要与以下 URL 中的文档中给出的相同数量的分区:在此处输入链接描述

I wanted to know why does co-partitioning of two Kstreams in kafka require same number of partitions for both the streams as is given in the documentation in below URL: enter link description here

推荐答案

正如名称co-partition"所示，您希望将来自不同主题但相同键的数据放入同一个 Kafka Streams 应用程序实例.如果您没有相同数量的分区，则不可能出现此行为.

As the name "co-partition" indicates, you want to put data from different topic but same key to the same Kafka Streams application instance. If you don't have the same number of partitions, it's not possible to get this behavior.

假设您的主题 A 有 2 个分区，主题 B 有 3 个分区.因此，可能会发生一个带有键 X 的记录被散列到分区 A-0 和 B-1(即，不同的分区号).但是，对于不同的键 Y，它可能会被散列到 A-0 而不是 B-2.

Assume you have topic A with 2 partitions and topic B with 3 partitions. Thus, it can happen that one record with key X is hashed to partitions A-0 and B-1 (ie, not same partition number). However, for a different key Y it might be hashed to A-0 but B-2.

仅当两个主题的分区数相同时，具有相同键的记录最终会出现在相同的分区中(当然是不同主题的)，这允许处理 A-0/B-0 和 A-1/B-1 等等.

Only if the number of partitions is the same for both topics, records with same key end up in the same partitions (of different topics of course), and this allows to process A-0/B-0 and A-1/B-1 etc together.

这篇关于为什么 kafka 中两个 Kstream 的共同分区需要两个流的分区数相同?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！