Java中的并行编程 [英] Parallel programming in Java

问题描述

我们如何在Java中进行并行编程？那有什么特别的框架吗？我们怎样才能使这些东西工作？

How can we do Parallel Programming in Java? Is there any special framework for that? How can we make the stuff work?

我会告诉你们我需要什么，认为我开发了一个网络爬虫，它从互联网上抓取了很多网站，一个系统爬行不会使事情正常工作，所以我们需要1000个系统才能工作，如果是这样的话我可以应用并行计算吗？你们能给我一个正确的例子吗？

I will tell you guys what I need, think that I am developed a web crawler, its crawl lot of website from the internet, one system crawling will not make things to work proper, so we need 1000 of system to work,if this is the case can I apply parallel computing?Can you guys gave me a proper example?

推荐答案

如果你问的是纯粹的并行编程，即不是并发编程那么你一定要尝试MPJExpress http://mpj-express.org/ 。它是mpiJava的线程安全实现，它支持分布式和共享内存模型。我试过了，发现非常可靠。

If you are asking about pure parallel programming i.e. not concurrent programming then you should definitely try MPJExpress http://mpj-express.org/. It is a thread-safe implementation of mpiJava and it supports both distributed and shared memory models. I have tried it and found very reliable.

1 import mpi.*;  
2  
3 
/**  
4  * Compile:impl specific.  
5  * Execute:impl specific.  
6  */  
7  
8 public class Send {  
9 
10     public static void main(String[] args) throws Exception { 
11 
12         MPI.Init(args); 
13 
14         int rank = MPI.COMM_WORLD.Rank() ; //The current process.
15         int size = MPI.COMM_WORLD.Size() ; //Total number of processes
16         int peer ; 
17 
18         int buffer [] = new int[10]; 
19         int len = 1 ;
20         int dataToBeSent = 99 ; 
21         int tag = 100 ; 
22 
23         if(rank == 0) { 
24 
25             buffer[0] = dataToBeSent ; 
26             peer = 1 ; 
27             MPI.COMM_WORLD.Send(buffer, 0, len, MPI.INT, peer, tag) ; 
28             System.out.println("process <"+rank+"> sent a msg to "+ 29                                "process <"+peer+">") ; 
30 
31         } else if(rank == 1) { 
32 
33             peer = 0 ; 
34             Status status = MPI.COMM_WORLD.Recv(buffer, 0, buffer.length, 35                                                 MPI.INT, peer, tag); 
36             System.out.println("process <"+rank+"> recv'ed a msg\n"+ 37                                "\tdata   <"+buffer[0]    +"> \n"+ 38                                "\tsource <"+status.source+"> \n"+ 39                                "\ttag    <"+status.tag   +"> \n"+ 40                                "\tcount  <"+status.count +">") ; 
41 
42         } 
43 
44         MPI.Finalize(); 
45 
46     }  
47 
48 }

MPJ Express等消息库提供的最常见功能之一是支持执行进程之间的点对点通信。在此上下文中，属于同一通信器的两个进程（例如MPI.COMM_WORLD通信器）可以通过发送和接收消息来彼此通信。 Send（）方法的变体用于从发送方进程发送消息。另一方面，接收器进程通过使用Recv（）方法的变体来接收所发送的消息。发送方和接收方都指定了一个标记，用于在接收方找到匹配的传入消息。

One of the most common functionalities provided by messaging libraries like MPJ Express is the support of point-to-point communication between executing processes. In this context, two processes belonging to the same communicator (for instance the MPI.COMM_WORLD communicator) may communicate with each other by sending and receiving messages. A variant of the Send() method is used to send the message from the sender process. On the other hand, the sent message is received by the receiver process by using a variant of the Recv() method. Both sender and receiver specify a tag that is used to find a matching incoming messages at the receiver side.

使用MPI.Init（args）初始化MPJ Express库之后在第12行的方法中，程序获得其等级和MPI.COMM_WORLD通信器的大小。两个进程初始化一个长度为10的整数数组，在第18行称为缓冲区。发送方进程级别0 - 在msg数组的第一个元素中存储值10。 Send（）方法的一种变体用于将msg数组的元素发送到接收进程。

After initializing the MPJ Express library using the MPI.Init(args) method on line 12, the program obtains its rank and the size of the MPI.COMM_WORLD communicator. Both processes initialize an integer array of length 10 called buffer on line 18. The sender process—rank 0—stores a value of 10 in the first element of the msg array. A variant of the Send() method is used to send an element of the msg array to the receiver process.

发送方进程调用第27行的Send（）方法前三个参数与发送的数据有关。发送缓冲区 - 缓冲区阵列 - 是第一个参数，后跟0（o！set）和1（count）。发送的数据是MPI.INT类型，目的地是1（对等变量）;数据类型和目标被指定为Send（）方法的第四个和第五个参数。最后一个和第六个参数是标记变量。标签用于识别接收方的消息。消息标记通常是特定通信器中特定消息的标识符。
另一方面，接收者进程（等级1）使用阻塞接收方法接收消息。

The sender process calls the Send() method on line 27. The first three arguments are related to the data being sent. The sending bu!er—the bu!er array—is the first argument followed by 0 (o!set) and 1 (count). The data being sent is of MPI.INT type and the destination is 1 (peer variable); the datatype and destination are specified as fourth and fifth argument to the Send() method. The last and the sixth argument is the tag variable. A tag is used to identify messages at the receiver side. A message tag is typically an identifier of a particular message in a specific communicator. On the other hand the receiver process (rank 1) receives the message using the blocking receive method.

这篇关于Java中的并行编程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！