Apache Storm与Spring框架的集成 [英] Apache Storm integration with Spring framework

问题描述

我是Apache Storm的新手。目前，我正在处理旧项目，该项目涉及使用Apache Storm进行一些流处理。我想将当前项目与Spring集成。我发现了一些评论（ Storm和Spring 4集成， http：// archives.apache.org/mod_mbox/storm-user/201605.mbox/[e-mail]%3E ）说，这样做很令人担忧。有人可以向我解释如何进行这种集成或为什么无法实现这种集成吗？

I'm new to Apache Storm. Currently I'm working on legacy project that involves some streaming processing using Apache Storm. I want to integrate current project with Spring. I found couple comments (Storm and Spring 4 integration, http://mail-archives.apache.org/mod_mbox/storm-user/201605.mbox/[e-mail]%3E) saying that there are concerns doing that. Can someone explain me how to do such an integration or why it is impossible?

推荐答案

一般警告，我没有使用过Spring在Storm中，所以这完全基于我对Storm的了解，并且在非Storm项目中使用过Spring，也就是说，这实际上只是猜测。

Fair warning, I haven't used Spring in Storm, so this is based solely on my knowledge of Storm, and having used Spring on non-Storm projects, i.e. this is really just guesswork.

我认为您可以将Spring和Storm一起使用，但是需要注意一些警告。考虑到这些警告，Spring是否仍然值得使用。

I think you can use Spring with Storm, but there are some caveats you should be aware of. Whether Spring is still worth using given these caveats is up to you.

与其他方法不同一个Spring MVC应用程序，Spring将不负责对象实例化或应用程序流程。 Storm不了解Spring，在您运行拓扑时，将由Storm调用您的螺栓/喷口方法。这意味着您必须知道应用程序的某些部分将在Spring上下文之外被调用。

Unlike e.g. a Spring MVC application, Spring will not be responsible for object instantiation or application flow. Storm doesn't know about Spring, and when you run your topology it will be Storm that calls your bolt/spout methods. This means you have to be aware that some parts of your application will be called outside the Spring context.

这是我的猜测，您可以在a的不同阶段中使用Spring拓扑部署。

Here's my guess at where you could use Spring during different phases of a topology deployment.

• 设置拓扑并提交拓扑时（所有代码包括$code） > StormSubmitter.submitTopology ），您很有可能像在任何独立Java应用程序中一样使用Spring。例如您可以像在此示例中那样启动应用程序，然后将所有提交和连接代码放在 Main中.start 。所有螺栓/喷口构造函数都将在此阶段运行，因此您可以根据需要在此处使用自动装配。您必须确保喷嘴和螺栓可序列化。

• When you set up your topology and submit it (all your code up to and including StormSubmitter.submitTopology), you can most likely use Spring just like you would in any standalone Java application. e.g. you could start your application like in this example, and put all your submission and wiring code in Main.start. All bolt/spout constructors will run in this phase, so you can use autowiring here if you like. You have to ensure that your spouts and bolts are serializable though.

提交拓扑后，Storm将序列化您的喷嘴和螺栓（以及所有非瞬态字段）。这些对象），然后将它们发送到主管机器，然后在其中进行反序列化。此时，如果您需要工作线程中可用的上下文，则可以在工人钩子（通过 TopologyBuilder.addWorkerHook 添加到拓扑中），并通过静态方法公开钩子（这有点丑陋，但是我看不到任何其他方式可用于代码的其他部分）。

After topology submission, Storm will serialize your spouts and bolts (and any non-transient fields in these objects), and send them to the supervisor machines, where they will be deserialized. At this point if you need a context available in the worker, you could create one in a worker hook (added to the topology via TopologyBuilder.addWorkerHook), and expose it through a static method on the hook (which is a little ugly, but I don't see any other way to make it available to other parts of the code).

只需重申一下，如果您决定在工作人员中运行Spring上下文，则必须知道Spout / bolt方法将由Storm在Spring上下文之外调用。

Just to reiterate, if you decide to run a Spring context inside your workers, you have to be aware that the spout/bolt methods will be invoked by Storm outside the Spring context.

这篇关于Apache Storm与Spring框架的集成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！