沿着什么轴不mpi4py Scatterv功能拆分numpy的阵列？ [英] Along what axis does mpi4py Scatterv function split a numpy array?

问题描述

我用下面的MWE comm.Scatterv 和 comm.Gatherv 跨一个给定的分发四维阵列核心数量（尺寸）

I have the following MWE using comm.Scatterv and comm.Gatherv to distribute a 4D array across a given number of cores (size)

import numpy as np
from mpi4py import MPI
import matplotlib.pyplot as plt

comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()

if rank == 0:
    test = np.random.rand(411,48,52,40) #Create array of random numbers
    outputData = np.zeros(np.shape(test))
    split = np.array_split(test,size,axis = 0) #Split input array by the number of available cores

    split_sizes = []

    for i in range(0,len(split),1):
        split_sizes = np.append(split_sizes, len(split[i]))

    displacements = np.insert(np.cumsum(split_sizes),0,0)[0:-1]

    plt.imshow(test[0,0,:,:])
    plt.show()

else:
#Create variables on other cores
    split_sizes = None
    displacements = None
    split = None
    test = None
    outputData = None

#Broadcast variables to other cores
test = comm.bcast(test, root = 0)
split = comm.bcast(split, root=0) 
split_sizes = comm.bcast(split_sizes, root = 0)
displacements = comm.bcast(displacements, root = 0)

output_chunk = np.zeros(np.shape(split[rank])) #Create array to receive subset of data on each core, where rank specifies the core
print("Rank %d with output_chunk shape %s" %(rank,output_chunk.shape))

comm.Scatterv([test,split_sizes, displacements,MPI.DOUBLE],output_chunk,root=0) #Scatter data from test across cores and receive in output_chunk

output = output_chunk

plt.imshow(output_chunk[0,0,:,:])
plt.show()

print("Output shape %s for rank %d" %(output.shape,rank))

comm.Barrier()

comm.Gatherv(output,[outputData,split_sizes,displacements,MPI.DOUBLE], root=0) #Gather output data together

if rank == 0:
    print("Final data shape %s" %(outputData.shape,))
    plt.imshow(outputData[0,0,:,:])
    plt.show()

这产生随机数的四维阵列，并在原则上重组前，应把它横跨尺寸内核。我期望 Scatterv 0（长411）根据载体中的起始整数和位移沿轴线划分 split_sizes 和位移。不过，我与 Gatherv 重组时，得到一个错误（ mpi4py.MPI.Exception：MPI_ERR_TRUNCATE：消息截断）和在每个芯output_chunk的曲线图表明，大部分输入数据的已丢失，所以看来还没有沿着第一轴发生分裂。

This creates a 4D array of random numbers and in principle should divide it across size cores before recombining. I expected Scatterv to divide along axis 0 (length 411) according to the starting integers and displacements in the vectors split_sizes and displacements. However, I get an error when recombining with Gatherv (mpi4py.MPI.Exception: MPI_ERR_TRUNCATE: message truncated) and the plot of output_chunk on each core shows that most of the input data has been lost, so it appears that the split has not occurred along the first axis.

我的问题是：为什么不分裂沿着第一轴发生，我怎么知道哪个轴发生分裂一起，并有可能改变/指定轴发生这种情况沿着

My questions are: Why doesn't the split occur along the first axis, how do I know which axis the split occurs along, and is it possible to change/specify which axis this occurs along?

推荐答案

comm.Scatterv 和 comm.Gatherv 不知道的numpy的阵列尺寸东西。他们只是看到 sendbuf 作为一个内存块。因此，有必要指定 sendcounts 和位移时，考虑到这一点（见的 http://materials.jeremybejarano.com/MPIwithPython/collectiveCom.html 详情）。假设也是该数据在C语言风格（行大）在存储器布局

comm.Scatterv and comm.Gatherv do not know anything about the numpy array dimensions. They just see the sendbuf as a block of memory. Therefore it is necessary to take this into account when specifying the sendcounts and displacements (see http://materials.jeremybejarano.com/MPIwithPython/collectiveCom.html for details). The assumption is also that the data is laid out in C-style (row major) in memory.

有一个2D矩阵的一个例子如下。这code的关键部件设置 split_sizes_input / split_sizes_output 和 displacements_input / displacements_output 正确。在code需要第二个维度的大小考虑在内存块中指定正确的划分：

An example for a 2D matrix is given below. The key parts of this code are to set split_sizes_input/split_sizes_output and displacements_input/displacements_output correctly. The code takes the second dimension size into account to specify the correct divisions in the memory block:

split_sizes_input = split_sizes*512

对于更高层面，这条线将改为：

For higher dimensions, this line would be changed to:

split_sizes_input = split_sizes*indirect_dimension_sizes

其中，

indirect_dimension_sizes = npts2 * npts3 * npts4 * .... * nptsN

和同样为 split_sizes_output 。

的code创建具有数字1至512穿过一个尺寸递增的2D阵列。如果数据已被分割并正确重组很容易从图中看到。

The code creates a 2D array with the numbers 1 to 512 incrementing across one dimension. It is easy to see from the plots if the data has been split and recombined correctly.

import numpy as np
from mpi4py import MPI
import matplotlib.pyplot as plt

comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()

if rank == 0:
    test = np.arange(0,512,dtype='float64')
    test = np.tile(test,[256,1]) #Create 2D input array. Numbers 1 to 512 increment across dimension 2.
    outputData = np.zeros([256,512]) #Create output array of same size
    split = np.array_split(test,size,axis = 0) #Split input array by the number of available cores

    split_sizes = []

    for i in range(0,len(split),1):
        split_sizes = np.append(split_sizes, len(split[i]))

    split_sizes_input = split_sizes*512
    displacements_input = np.insert(np.cumsum(split_sizes_input),0,0)[0:-1]

    split_sizes_output = split_sizes*512
    displacements_output = np.insert(np.cumsum(split_sizes_output),0,0)[0:-1]


    print("Input data split into vectors of sizes %s" %split_sizes_input)
    print("Input data split with displacements of %s" %displacements_input)

    plt.imshow(test)
    plt.colorbar()
    plt.title('Input data')
    plt.show()

else:
#Create variables on other cores
    split_sizes_input = None
    displacements_input = None
    split_sizes_output = None
    displacements_output = None
    split = None
    test = None
    outputData = None

split = comm.bcast(split, root=0) #Broadcast split array to other cores
split_sizes = comm.bcast(split_sizes_input, root = 0)
displacements = comm.bcast(displacements_input, root = 0)
split_sizes_output = comm.bcast(split_sizes_output, root = 0)
displacements_output = comm.bcast(displacements_output, root = 0)

output_chunk = np.zeros(np.shape(split[rank])) #Create array to receive subset of data on each core, where rank specifies the core
print("Rank %d with output_chunk shape %s" %(rank,output_chunk.shape))
comm.Scatterv([test,split_sizes_input, displacements_input,MPI.DOUBLE],output_chunk,root=0)

output = np.zeros([len(output_chunk),512]) #Create output array on each core

for i in range(0,np.shape(output_chunk)[0],1):
    output[i,0:512] = output_chunk[i]

plt.imshow(output)
plt.title("Output shape %s for rank %d" %(output.shape,rank))
plt.colorbar()
plt.show()

print("Output shape %s for rank %d" %(output.shape,rank))

comm.Barrier()

comm.Gatherv(output,[outputData,split_sizes_output,displacements_output,MPI.DOUBLE], root=0) #Gather output data together



if rank == 0:
    outputData = outputData[0:len(test),:]
    print("Final data shape %s" %(outputData.shape,))
    plt.imshow(outputData)
    plt.colorbar()
    plt.show()
    print(outputData)

这篇关于沿着什么轴不mpi4py Scatterv功能拆分numpy的阵列？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！