添加更多节点时，Storm如何处理字段分组? [英] How does Storm handle fields grouping when you add more nodes?

问题描述

只需阅读有关风暴的更多详细信息，就会发现它具有对字段进行分组的功能，因此，例如，如果您在计算每个用户的推文计数，并且您有两个任务都对用户ID进行了分组，则会将相同的用户ID发送到相同的任务.

Just reading more details on storm and came across it's ability to do fields grouping so for example if you where counting tweets per user and you had two tasks with a fields grouping of user-id the same user-id's would get sent to the same tasks.

因此任务1可能在内存中具有以下计数 鲍勃:10 爱丽丝:5

So task 1 could have the following counts in memory bob: 10 alice: 5

任务2可能在内存中具有以下计数 吉尔:10 乔:4

task 2 could have the following counts in memory jill:10 joe: 4

如果我将新计算机添加到群集中以增加容量并进行重新平衡，那么内存中的计数将如何处理?您会开始吸引不同人数的用户吗?

If I added a new machine to the cluster to increase capacity and ran rebalance, what happens to my counts in memory? Will you start to get users with different counts?

推荐答案

使用字段分组，我们可以指导特定字段执行特定任务.

Using fields grouping we can guide a specific field to go to a particular tasks.

字段分组:该流按分组中指定的字段进行分区.例如，如果流按"user-id"字段分组，则具有相同"user-id"的元组将始终执行相同的任务，但是具有不同"user-id"的元组可能会执行不同的任务

Fields grouping: The stream is partitioned by the fields specified in the grouping. For example, if the stream is grouped by the "user-id" field, tuples with the same "user-id" will always go to the same task, but tuples with different "user-id"'s may go to different tasks.

这些任务在风暴的生命周期中始终是静态的，您可以使用rebalance更改的是执行程序(线程)数.如果将新节点添加到群集，则可以重新配置要运行的执行程序的数量，而无需关闭拓扑，但是无论任务数量如何都保持不变.仅仅是增加一个新节点，便可以通过调整风暴的并行性来提高性能.

these task are always static in a storm's life cycle, what you can alter using the rebalance is number of executors(threads). in case of adding a new node to a cluster allows you to reconfigure the number of executors to run with out shutting down the topology but no matter what the number of tasks remains the same. its just that adding a new node gives you the advantage of increasing the performance by tuning the parallelism of storm.

这篇关于添加更多节点时，Storm如何处理字段分组?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！