我如何发送带有散点图的数组的一部分? [英] How can I send part of an array with scatter?

问题描述

我正在自学Python mpi4py模块，用于在多个进程中进行编程.我已经编写了以下代码来练习分散操作.

I am teaching myself the Python mpi4py module for programming in multiple processes. I have written the following piece of code to practice scatter.

from mpi4py import MPI

comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()

if rank == 0:
   data = [i for i in range(8)]
else:
   data = None
data = comm.scatter(data, root=0)
print str(rank) + ': ' + str(data)

使用8个进程运行上面的代码非常有用.但是，当我用4个进程运行它时，出现错误:

Running the above code with 8 processes works great. However, when I run it with 4 processes, I get an error:

Traceback (most recent call last):
  File "scatter.py", line 11, in <module>
    data = comm.scatter(data, root=0)
  File "Comm.pyx", line 874, in mpi4py.MPI.Comm.scatter (src/mpi4py.MPI.c:68023)
  File "pickled.pxi", line 656, in mpi4py.MPI.PyMPI_scatter (src/mpi4py.MPI.c:32402)
  File "pickled.pxi", line 127, in mpi4py.MPI._p_Pickle.dumpv (src/mpi4py.MPI.c:26813)
ValueError: expecting 4 items, got 8

此错误是什么意思?我的意图是将我的8个项目的大数组分解为8/4 = 2个项目的小数组，并向每个进程发送一个这样的子数组.我怎么做?我还想将可能的进程数归纳为3，例如3.

What does this error mean? My intention is to break up my large array of 8 items into small arrays of 8 / 4 = 2 items and send each process one such subarray. How do I do that? I would also like to generalize if possible to numbers of processes that do not divide evenly into 8 such as 3.

推荐答案

似乎comm.scatter不能将count作为参数，并且希望将精确地包含comm.size个元素的列表作为数据进行散布.因此，您需要自己在流程之间分配数据.这样的事情会做:

It seems that comm.scatter can not take count as an argument and expects a list of exactly comm.size elements as data to be scattered; so you need to distribute your data between processes yourself. Something like this will do:

if rank == 0:
    data = [i for i in range(8)]
# dividing data into chunks
    chunks = [[] for _ in range(size)]
    for i, chunk in enumerate(data):
        chunks[i % size].append(chunk)
else:
    data = None
    chunks = None
data = comm.scatter(chunks, root=0)
print str(rank) + ': ' + str(data)

[physics@tornado] ~/utils> mpirun -np 3 ./mpi.py 
2: [2, 5]
0: [0, 3, 6]
1: [1, 4, 7]

希望这会有所帮助.

这篇关于我如何发送带有散点图的数组的一部分?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！