消除风暴拓扑后清理资源 [英] Resource clean up after killing storm topology
问题描述
我们有一个与MariaDB数据库交互的风暴拓扑.我们的Bolts实现IRichBolt接口并覆盖生命周期方法.我们在prepare方法中打开一个数据库连接，然后在cleanup方法中关闭它.
cleanup方法文档说:
We have a storm topology which interacts with a MariaDB database. Our Bolts implement the IRichBolt interface and override the lifecycle methods. We open a db connection in our prepare method and close it in the cleanup method.
The cleanup method documentation says:
在将要关闭IBolt时调用.没有保证将被称为清理，因为主管在群集上杀死了-9的工作进程. 保证要调用清理的一种情况是在本地模式下运行Storm时拓扑被杀死
Called when an IBolt is going to be shutdown. There is no guarentee that cleanup will be called, because the supervisor kill -9's worker processes on the cluster. The one context where cleanup is guaranteed to be called is when a topology is killed when running Storm in local mode
kill -9命令可在不清理任何资源的情况下终止该进程.因此，我们得出的结论是，在终止拓扑时，不必调用cleanup方法并关闭数据库连接.
And the kill -9 command kills the process without cleaning up any resources. So we have come to this conclusion that on killing the topology it is not necessary that the cleanup method would be called and the db connection will be closed.
因此，前进到我的问题，我们有一个用于拓扑部署的shell脚本，该脚本在执行时会杀死超时为0的当前拓扑并部署新的拓扑.在数据库级别，我们面临着一个问题，那就是有许多打开的连接，这提示我们以前的连接没有关闭. (在上一个拓扑中打开的那个.)
So moving forward to my question, we have a shell script for topology deployment which when executed kills the current topology with a timeout of 0 and deploys a new topology. We are facing an issue at db level that there are many connections opened which gives us the hint that previous connections were not closed. (The one opened in the previous topology).
我们的假设正确吗?增加超时会清除所有资源吗?
Is our assumption correct? Will increasing the timeout clean up all the resources?
推荐答案
否.增加拓扑超时不会影响您的工作人员必须清理的时间.当您使用例如一个30秒的超时，它只是关闭了喷口，并给拓扑的其余部分30秒来完成处理.
No. Increasing the topology timeout won't affect how long your worker has to clean up. When you use e.g. a 30s timeout, that just shuts off the spout and gives the rest of the topology 30 secs to finish processing.
您想要的是增加Storm允许工人在强行杀死之前花费多长时间关闭. https://github.com/apache/storm/blob/b07413670fa62fec077c92cb78fc711c3bda820c/storm-server/src/main/java/org/apache/storm/DaemonConfig.java#L780 选项可让您指定Storm的持续时间将在发送kill -9之前等待.默认值为3秒.您需要在storm.yaml中设置此选项，在拓扑配置中进行设置将无效.
What you want is to increase how long Storm allows a worker to spend shutting down before force killing. The https://github.com/apache/storm/blob/b07413670fa62fec077c92cb78fc711c3bda820c/storm-server/src/main/java/org/apache/storm/DaemonConfig.java#L780 option allows you to specify how long Storm will wait before sending the kill -9. The default is 3 seconds. You need to set this option in storm.yaml, setting it in the topology configuration will have no effect.
这篇关于消除风暴拓扑后清理资源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
