apache Storm 可靠性超时配置 [英] apache storm reliablity timeout configuration

问题描述

我在 Linux Ubuntu 中部署了一个 nodejs->kafka>storm->Mongo.原本一切正常.然后我更改了 Storm worker 中的方法，这使得 Storm worker 处理消息的速度非常慢，每条消息大约 1 分钟，我注意到消息从 Storm 中一次又一次地发送.我恢复到原来的方法，一切都很好.(原始方法处理时间为每条消息 90ms).

I have a nodejs->kafka>storm->Mongo deployed in Linux Ubuntu. Everything is normal originally. Then I changed the method in storm worker which makes storm worker process message very slow, around 1 minute per message, I notice the message is sent again and again from storm. I revert back to original method, everything is fine. (original method process time is 90ms per message).

我想这是 Storm 的可靠性发挥作用.当消息未被确认或超时时，它再次发送消息.

I guess this is Storm reliability come into player. When message is not acknowledged, or time out, it sends message again.

如果我猜对了，如何配置这个超时时间?

If my guess is right, how to configure this timeout?

如果我的猜测是错误的，为什么同一条消息会被发送两次或三次?

If my guess is wrong, why same message is sent twice or three times?

推荐答案

您可以通过配置参数 Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS 设置超时时间.请参阅 https://storm.apache.org/javadoc/apidocs/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS

You can set the timeout via configuration parameter Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS. See https://storm.apache.org/javadoc/apidocs/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS

默认值为 30 秒，请参阅此处的 defaults.yaml:https://github.com/apache/storm/blob/master/conf/defaults.yaml

The default value is 30 seconds, see defaults.yaml here: https://github.com/apache/storm/blob/master/conf/defaults.yaml

# maximum amount of time a message has to complete before it's considered failed
topology.message.timeout.secs: 30

当一个元组失败时，它应该显示在 Storm UI 中并且也应该被记录(也许你需要调整日志级别).因此，您可以仔细检查元组是否超时.

When a tuple fails, it should show up in Storm UI and should be logged, too (maybe you need to adjust log level). So you can double check if a tuple times out or not.

这篇关于apache Storm 可靠性超时配置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！