EAI /中间件：巨大的信息内联或引用 [英] EAI/middleware: Huge messages inline or by reference

问题描述

是否有处理通过中间件或EAI系统中处理大量的数据（兆字节？GB的？）消息或请求/响应，assunming的中间件传输，但不处理海量数据的斑点良好做法？

Are there good practices for processing messages or requests/responses dealing with huge amounts of data (MBytes? GBytes?) via an middleware or an EAI system, assunming that the middleware transports but does not process the huge data blobs?

• 运输与消息中的数据，以保持一致性，但可能多次复制系统之间的海量数据

• Transport the data with the message in order to keep coherence but probably several times copy huge amounts of data between systems

请在商店或数据储​​存库中的数据（文件系统，数据库，...）和运输到的数据的参考，但是失去了消息和数据之间的紧密耦合

Keep the data in a store or data repository (file system, database,...) and transport a reference to the data, however loosing the close coupling between the message and the data

什么是解决这个问题的通用解决方案？这是如何通过EAI或中间件产品支持？

What is the common solution to this problem? How is this supported by EAI or middleware products?

推荐答案

在我的经验，这是很好的限制邮件的大小。在我知道的环境（的WebSphere MQ，SonicMQ的）我发现5-10 MB的大部分消息的很好的价值。在许多情况下，它不会伤害有大小很少邮件高达100 MB - >但是你要想想内存消耗，......在大多数消息传递API，尤其是JMS你将至少有消息完全在内存中一次，开始并行处理时，这可能会导致你有些头疼。

In my experience it is good to limit the size of the messages. In the environments i know (Websphere MQ, SonicMQ) i found 5-10 MB a good value for most of the messages. In many cases it will not hurt to have very few messages with sizes up to 100 MB -> but there you will have to think about memory consumption, .... In most messaging APIs, especially JMS you will at least have the message completely in memory one time, which may cause you some headaches when starting parallel processing.

有两件事情我们使用作为一种解决方法：

There are two things we use as a workaround:

• 使用消息序列模式添加了一些新：在大部分时间都是发送包含的参考一触发所有的序列号，以确保完整性


• 发送一个链接到数据

• use the Message sequence pattern with some additions: mostly we send a trigger containing a reference of all the sequence numbers to ensure completeness
• send a "link" to the data

通常我们使用第一溶液每当消息包含结构化数据，第二个是在媒体分发的情况下使用。

Normally we use the first solution whenever the messages contain structured data, the second one is used in cases of media distribution.

对于第一个解决方案，我们不也倾向于不使用提供了具体的解决方案，但在我们的自定义实现实现这个逻辑 - 为此我们不是绑定到特定的邮件系统

For the first solution we do also tend not to use provides specific solutions, but implement this logic in our custom implementation - therefor we are not bound to a specific messaging system.

这篇关于EAI /中间件：巨大的信息内联或引用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！