选择一个分布式共享存储解决方案 [英] Choosing a distributed shared memory solution

问题描述

我要建立一个原型为大规模可扩展分布式共享内存（DSM）的应用程序的任务。原型只能作为证明的概念，但我想通过挑选这将在以后的真正的解决方案中使用的组件最有效地把我的时间。

I have a task to build a prototype for a massively scalable distributed shared memory (DSM) app. The prototype would only serve as a proof-of-concept, but I want to spend my time most effectively by picking the components which would be used in the real solution later on.

这种解决方案的目的是采取从外部源输入数据，翻腾它，使可使用的一些前端的结果。那些前端只会从缓存中获取数据，并服务于它无需额外处理。前端的数量击中这个数据可以从字面上是每秒数百万。

The aim of this solution is to take data input from an external source, churn it and make the result available for a number of frontends. Those "frontends" would just take the data from the cache and serve it without extra processing. The amount of frontend hits on this data can literally be millions per second.

数据本身非常不稳定;它可以（和不）相当迅速改变。然而，前端应该看到旧的数据，直到最新的已被处理和缓存。处理和写入是通过一个单一的（冗余）节点完成，而其他节点只读取数据。换句话说：没有通读行为

The data itself is very volatile; it can (and does) change quite rapidly. However the frontends should see "old" data until the newest has been processed and cached. The processing and writing is done by a single (redundant) node while other nodes only read the data. In other words: no read-through behaviour.

我一直在寻找进入memcached的如解决方案然而这个特殊的不履行全部我们将在下面列出的要求

I was looking into solutions like memcached however this particular one doesn't fulfil all our requirements which are listed below:

1. 解决方案必须至少有 Java客户端API 其中作为应用程序的其余部分是用Java编写，我们经验丰富的Java开发者合理良好的维护;


2. 解决方案必须是完全的弹性：它应该是可能的，而不在集群中重新启动其他节点添加新节点;


3. 解决方案必须能够处理的故障。是的，我知道这意味着一些开销，但整体供应数据量并不大（最大1G），所以这不应该是一个问题。通过故障恢复我的意思是无缝的执行没有硬编码/更改服务器的IP地址（ES）像memcached的客户端时，一个节点出现故障;


4. 理想情况下它应该有可能以指定的数据的程度的重叠（例如，如何相同的数据的多个副本应存储在DSM簇中）;


5. 有没有必要永久地存储所有数据，但有可能是一个需要一些数据后处理（例如序列化到数据库）的。


6. 价格即可。很显然，我们preFER自由/开源，但我们很高兴，如果一个解决方案是值得付出的合理费用。以任何方式，支付24小时/天的支持合同是必须的。


7. 整个事情在我们的数据中心以托管这样的SaaS产品如Amazon的SimpleDB是超出范围。我们只会考虑这一点，如果没有其他的选项将是可用的。


8. 理想的解决办法是的严格一致的（如CAP）;然而，最终浓度的可以被认为是一种选择。

1. The solution must at least have Java client API which is reasonably well maintained as the rest of app is written in Java and we are seasoned Java developers;
2. The solution must be totally elastic: it should be possible to add new nodes without restarting other nodes in the cluster;
3. The solution must be able to handle failover. Yes, I realize this means some overhead, but the overall served data size isn't big (1G max) so this shouldn't be a problem. By "failover" I mean seamless execution without hardcoding/changing server IP address(es) like in memcached clients when a node goes down;
4. Ideally it should be possible to specify the degree of data overlapping (e.g. how many copies of the same data should be stored in the DSM cluster);
5. There is no need to permanently store all the data but there might be a need of post-processing of some of the data (e.g. serialization to the DB).
6. Price. Obviously we prefer free/open source but we're happy to pay a reasonable amount if a solution is worth it. In any way, paid 24hr/day support contract is a must.
7. The whole thing has to be hosted in our data centers so SaaS offerings like Amazon SimpleDB are out of scope. We would only consider this if no other options would be available.
8. Ideally the solution would be strictly consistent (as in CAP); however, eventual consistence can be considered as an option.

在此先感谢您的任何想法。

Thanks in advance for any ideas.

推荐答案

看一看 Hazelcast 。它是纯Java，开源（Apache许可）高度可扩展的内存数据网格产品。它提供7X24小时的支持。它并不能解决所有的问题，你我试着解释下面每个人：

Have a look at Hazelcast. It is pure Java, open source (Apache license) highly scalable in-memory data grid product. It does offer 7X24 support. And it does solve all of your problems I tried to explain each of them below:

1. 它有一个本地Java客户端。


2. 这是100％的动态。动态添加和删除节点。无需改变任何东西。


3. 一切又是动态的。


4. 您可以配置备份节点数量。


5. Hazelcast支持持久性。


6. 的Hazelcast提供一切都是免费的（开源），它确实提供了企业级的支持。


7. Hazelcast是一个jar文件。超级好用。只需添加罐子到类路径中。看一看在主屏幕页面投。


8. Hazelcast是严格一致。你永远不能读取陈旧数据。

这篇关于选择一个分布式共享存储解决方案的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！