如何创建一个用于在Java中运行物理模拟的Linux集群? [英] How to create a Linux cluster for running physics simulations in java?
问题描述
我正在开发用于执行物理模拟的科学应用程序.所使用的算法为O(n3)，因此对于大量数据而言，需要很长时间才能处理.该应用程序将在大约17分钟内运行一次仿真，而我必须运行大约25,000次仿真.那大约是一年的处理时间.
I am developing a scientific application used to perform physical simulations. The algorithms used are O(n3), so for a large set of data it takes a very long time to process. The application runs a simulation in around 17 minutes, and I have to run around 25,000 simulations. That is around one year of processing time.
好消息是，模拟彼此之间是完全独立的，因此我可以轻松地更改程序以在多台计算机之间分配工作.
The good news is that the simulations are completely independent from each other, so I can easily change the program to distribute the work among multiple computers.
我可以看到实现此目的的多种解决方案:
There are multiple solutions I can see to implement this:
- 获得一台多核计算机，并在所有核之间分配工作.我做不到足够的事情.
- 编写一个连接到多个处理"服务器的应用程序，并在其中分配负载.
- 获取廉价的Linux计算机群集，并使该程序将所有内容都视为一个单独的实体.
选项2相对容易实现，因此我对如何实现它的建议并不多(可以通过编写一个程序来完成，该程序在给定端口上等待参数，处理值和以序列化文件的形式返回结果).这将是网格计算的一个很好的例子.
Option number 2 is relatively easy to implement, so I don't look so much for suggestions for how to implement this (Can be done just by writing a program that waits on a given port for the parameters, processes the values and returns the result as a serialized file). That would be a good example of Grid Computing.
但是，我不知道最后一个选择的可能性，即传统集群.在linux网格中运行Java程序有多困难?是否将所有单独的计算机都视为具有多个内核的单台计算机，从而使程序易于调整?是否有任何好的资源指南可以使我开始使用?还是我使这个问题变得过于复杂了，所以我最好选择2号选项?
However, I wonder at the possibilities of the last option, a traditional cluster. How difficult is to run a Java program in a linux grid? Will all the separate computers be treated as a single computer with multiple cores, making it thus easy to adapt the program? Is there any good pointers to resources that would allow me to get started? Or I am making this over-complicated and I am better off with option number 2?
作为额外的信息，我对如何实现《连线杂志》这篇文章中所述的内容感兴趣:
As extra info, I am interested on how to implement something like described in this article from Wired Magazine: Scientific replaced a supercomputer with a Playstation 3 linux cluster. Definitively number two sounds like the way to go... but the coolness factor.
计算非常受CPU限制.基本上，在大型矩阵上有很多运算，例如逆和乘法.我试图为这些操作寻找更好的算法，但到目前为止，我发现我需要的操作为0(n3)(在通常可用的库中).数据集很大(用于此类操作)，但是它是根据输入参数在客户端上创建的.
EDIT 2: The calculation is very CPU-Bound. Basically there is a lot of operations on large matrixes, such as inverse and multiplication. I tried to look for better algorithms for these operations but so far I've found that the operations I need are 0(n3) (In libraries that are normally available). The data set is large (for such operations), but it is created on the client based on the input parameters.
我现在看到我对Linux下的计算机集群的工作方式有误解.我假设它会以某种方式工作，就好像您拥有所有可用计算机中的所有处理器一样，就像您有一台具有多个内核的计算机一样，但事实并非如此.似乎所有这些超级计算机都是通过执行由某个中央实体分发的任务的节点来工作的，并且有几种不同的库和软件包可以轻松地执行此分发.
I see now that I had a misunderstanding on how a computer cluster under linux worked. I had the assumption that it would work in such a way that it would just appear that you had all the processors in all computers available, just as if you had a computer with multiple cores, but that doesn't seem to be the case. It seems that all these supercomputers work by having nodes that execute tasks distributed by some central entity, and that there is several different libraries and software packages that allow to perform this distribution easily.
因此，真正的问题是，因为没有数字3这样的东西:创建集群Java应用程序的最佳方法是什么?
So the question really becomes, as there is no such thing as number 3, into: What is the best way to create a clustered java application?
推荐答案
我非常推荐Java并行处理框架，特别是因为您的计算已经独立.我和这个本科生做了很多工作，效果很好.实施工作已经为您完成，因此我认为这是实现第二个"目标的好方法.
I would very highly recommend the Java Parallel Processing Framework especially since your computations are already independant. I did a good bit of work with this undergraduate and it works very well. The work of doing the implementation is already done for you so I think this is a good way to achieve the goal in "number 2."
这篇关于如何创建一个用于在Java中运行物理模拟的Linux集群?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
