Apache Storm 与 Spring 框架的集成 [英] Apache Storm integration with Spring framework

问题描述

我是 Apache Storm 的新手.目前我正在处理涉及使用 Apache Storm 进行一些流处理的遗留项目.我想将当前项目与 Spring 集成.我发现了一些评论(Storm 和 Spring 4 集成，

I'm new to Apache Storm. Currently I'm working on legacy project that involves some streaming processing using Apache Storm. I want to integrate current project with Spring. I found couple comments (Storm and Spring 4 integration, http://mail-archives.apache.org/mod_mbox/storm-user/201605.mbox/[e-mail]%3E) saying that there are concerns doing that. Can someone explain me how to do such an integration or why it is impossible?

推荐答案

公平警告，我没有在 Storm 中使用 Spring，所以这完全基于我对 Storm 的了解，并且在非 Storm 项目中使用过 Spring，即这真的只是猜测.

Fair warning, I haven't used Spring in Storm, so this is based solely on my knowledge of Storm, and having used Spring on non-Storm projects, i.e. this is really just guesswork.

我认为您可以将 Spring 与 Storm 结合使用，但您应该注意一些注意事项.鉴于这些注意事项，Spring 是否仍然值得使用取决于您.

I think you can use Spring with Storm, but there are some caveats you should be aware of. Whether Spring is still worth using given these caveats is up to you.

不像例如一个 Spring MVC 应用程序，Spring 将不负责对象实例化或应用程序流程.Storm 不了解 Spring，当您运行拓扑时，Storm 将调用您的 bolt/spout 方法.这意味着您必须意识到应用程序的某些部分将在 Spring 上下文之外被调用.

Unlike e.g. a Spring MVC application, Spring will not be responsible for object instantiation or application flow. Storm doesn't know about Spring, and when you run your topology it will be Storm that calls your bolt/spout methods. This means you have to be aware that some parts of your application will be called outside the Spring context.

这是我对在拓扑部署的不同阶段可以在哪里使用 Spring 的猜测.

Here's my guess at where you could use Spring during different phases of a topology deployment.

• 当您设置拓扑并提交它(包括StormSubmitter.submitTopology在内的所有代码)时，您很可能可以像在任何独立的 Java 应用程序中一样使用 Spring.例如你可以像这个例子那样启动你的应用程序，然后把你所有的提交和布线代码放在Main.start.所有的 bolt/spout 构造函数都会在这个阶段运行，所以如果你愿意，你可以在这里使用自动装配.不过，您必须确保您的 spouts 和 bolts 是可序列化的.

• When you set up your topology and submit it (all your code up to and including StormSubmitter.submitTopology), you can most likely use Spring just like you would in any standalone Java application. e.g. you could start your application like in this example, and put all your submission and wiring code in Main.start. All bolt/spout constructors will run in this phase, so you can use autowiring here if you like. You have to ensure that your spouts and bolts are serializable though.

拓扑提交后，Storm 将序列化您的 spout 和 bolt(以及这些对象中的任何非瞬态字段)，并将它们发送到管理器机器，在那里它们将被反序列化.此时，如果您需要工作人员中可用的上下文，您可以在 worker hook(通过TopologyBuilder.addWorkerHook添加到topology中)，并通过hook上的一个静态方法暴露出来(有点丑陋，但我没有看到任何其他方法可以使其可用于代码的其他部分).

After topology submission, Storm will serialize your spouts and bolts (and any non-transient fields in these objects), and send them to the supervisor machines, where they will be deserialized. At this point if you need a context available in the worker, you could create one in a worker hook (added to the topology via TopologyBuilder.addWorkerHook), and expose it through a static method on the hook (which is a little ugly, but I don't see any other way to make it available to other parts of the code).

重申一下，如果你决定在你的 worker 中运行一个 Spring 上下文，你必须意识到 spout/bolt 方法将在 Spring 上下文之外被 Storm 调用.

Just to reiterate, if you decide to run a Spring context inside your workers, you have to be aware that the spout/bolt methods will be invoked by Storm outside the Spring context.

这篇关于Apache Storm 与 Spring 框架的集成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！