Storm 和 Spring 4 集成 [英] Storm and Spring 4 integration

问题描述

我有一个原型 Storm 应用程序，它读取 STOMP 流并将输出存储在 HBase 上.它可以工作，但不是很灵活，我正在尝试以更一致的方式设置它与我们的其他应用程序，但没有多少运气弄清楚如何使用 Storm 的当前工作方式.我们使用 spring-jms 类，但不是以标准的 spring 方式使用它们，而是在运行时创建它们，并手动设置依赖项.

I have a prototype storm app that reads a STOMP stream and stores the output on HBase. It works, but is not very flexible and I'm trying to get it set up in a more consistent way with the rest of our apps, but not having much luck figuring out how the current way of working with Storm. We use spring-jms classes, but instead of using them in the standard spring way, they are being created at run time, and setting dependencies manually.

这个项目:https://github.com/granthenke/storm-spring 看起来很有前途，但它没有由于 Storm jars 已被放入 apache 孵化器并重新包装，因此在几年内没有接触过并且无法正常构建.

This project: https://github.com/granthenke/storm-spring looked promising, but it hasn't been touched in a couple years and doesn't build properly since the storm jars have been taken into apache incubator and repackaged.

是否有什么我遗漏的地方，或者我不值得花时间整合这些东西?

Is there something I'm missing, or is it not worth my while to get these things integrated?

推荐答案

事实上，storm-spring 似乎是你正在寻找的东西，但它没有更新并且有限制(例如不能在 bolts/spouts 上定义任务，等等).也许您应该推出自己的集成?

In fact, storm-spring seems to be what you are looking for but it is not updated and have limitations (cannot define tasks on bolts / spouts for instance, etc). Maybe you should roll your own integration?

不要忘记你的目标:一个有很多工人的集群.当您将使用storm api(例如重新平衡)部署拓扑时，spring 的行为如何?这是否意味着它必须在启动时在工作 JVM 上实例化一个新的 Spring 上下文，然后 Storm 才能部署目标螺栓/spout 并定义执行器?

Don't forget your target: a cluster with many workers. How does spring behave when you will deploy your topology with storm api (rebalance for instance) on one more worker? Does it mean it has to instanciate a new Spring context on the worker JVM at startup before Storm deploys the targeted bolts / spouts and defines the executors?

恕我直言，如果您在 Spring 配置中仅定义 Storm 组件，它应该可以工作(拓扑的启动配置然后 Storm 仅管理对象)但如果您依赖 Spring 来管理其他组件(使用 spring-jms 似乎如此)，那么它可能会在例如拓扑重新平衡时变得混乱(每个工作器/jvm 单个?还是整个拓扑?).

IMHO if you define only Storm components in a Spring configuration it should work (startup configuration for the topology then storm only manages the objects) but if you rely on Spring to manage other components (it seems so with spring-jms), then it could become messy on topology rebalances for instance (singleton per worker / jvm? Or the whole topology?).

是否值得麻烦由您来决定，我对 Spring 配置的担忧是您很容易忘记 Storm 拓扑(它似乎是一个 JVM，但可以有更多).就我个人而言，我为每个类加载器定义了自己的单例(例如静态 final，或者如果我需要延迟实例化，则使用双重检查锁定)，因为它不会隐藏(中高)复杂性.

It is up to you to decide if it is worth the trouble, my concern with a Spring configuration is that you easily forget the storm topology (it seems it is one JVM but can be many more). Personally I define my own singletons per class-loader (static final for instance or with double check locking if I need deferred instanciation), as it does not hide the (medium-high) complexity.

这篇关于Storm 和 Spring 4 集成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！