从喷口杀死风暴拓扑 [英] Killing storm topology from spout

问题描述

我们有一个用例，我们不想连续运行风暴拓扑.相反，有一组输入(10K +)应该在指定的时间处理，Spout连续发出这些输入并由拓扑中的其余螺栓处理.处理完所有输入后，我的喷口中的nextTuple将不再发出任何内容.

We have an use case where we do not want to run storm topology continuously. Instead, there are set of inputs( 10K+) that should be processed at the specified time, Spout continuously emits these inputs and get processed by rest of the bolts in the topology. Once all the inputs are processed, there is nothing to emit from nextTuple in my spout.

此时，我们希望拓扑每天晚上12:00进入睡眠状态并重新启动该过程.

At this time we wanted our topology to go to sleep and restart the process everyday night 12:00 am.

是否在storm配置中设置了任何属性，以每天运行一次拓扑并在处理完成后休眠并在指定的时间开始运行?

Is there any property to set in the storm config to run the topology once a day and sleep after processing is done and start at the specified time?

推荐答案

我不知道您想要的功能. Storm不是批处理系统，而是要连续运行.考虑Storm是否适合此用例.

I'm not aware of a feature like what you're asking for. Storm isn't a batch processing system, it's meant to be running continuously. Consider if Storm is a great fit for this use case.

也就是说，您应该能够实现所需的功能.您可以在喷口输入的末尾输入我已完成"消息.当喷口击中该消息并且所有其他未决消息被确认时，它可以使用Nimbus客户端来终止或停用拓扑(取决于您要终止还是停用)，请参见

That said, you should be able to implement what you want. You could put in an "I'm done" message at the end of your spout input. When the spout hits that message and all other pending messages are acked, it could use the Nimbus client to kill or deactivate the topology (depending on whether you want to kill or deactivate), see https://stackoverflow.com/a/37134473/8845188. Then the final step would be using your favorite scheduling software to resubmit/reactivate the topology every day at midnight.

这篇关于从喷口杀死风暴拓扑的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！