派生数据类型与MPI [英] derived data types with MPI

问题描述

我正在学习Fortran中的BCASTing数据类型，并且有一个代码，它从终端获取两个值并在每个进程中显示它们。对于整数/整数和整数/实数类型的组合值1 /值2，这适用，但对于整数/实数* 8组合而言，它失败。

代码是：

 使用mpi 
隐式无
 
整型:: ierror，pid，ncpu，root = 0 
 
 integer :: counts，newtype，extent 
整数，维（2）:: oldtypes，blockcounts，偏移
 
类型值
整数： ：value1 = 0 
 real * 8 :: value2 
结束类型
 
类型（值）输入
 
调用MPI_INIT（ierror）
调用MPI_COMM_RANK（MPI_COMM_WORLD，pid，ierror）
调用MPI_COMM_SIZE（MPI_COMM_WORLD，ncpu，ierror）
 
！ （1）= MPI_INTEGER 
 blockcounts（1）= 1 
 
！$ 1 
 $设置1 MPI_REAL8字段：value2 
调用MPI_TYPE_EXTENT（MPI_INTEGER，extent，ierror）！确定MPI_INTEGER的偏移量
 offsetsets（2）= blockscounts（1）* extent！offset是1 MPI_INTEGER范围
 oldtypes（2）= MPI_REAL8 
 blockcounts（2）= 1 
 
！定义结构类型并提交
 counts = 2！for MPI_INTEGER + MPI_REAL8 
 call MPI_TYPE_STRUCT（counts，blockcounts，offset，& 
 oldtypes，newtype，ierror）
 call MPI_TYPE_COMMIT（newtype （输入％value1> = 0）
 if（pid == root）then 
 read（*，*）input 
 write （*，*）'输入为：'，输入
如果
调用MPI_BCAST（输入，1，newtype，& 
 root，MPI_COMM_WORLD，错误）
 write（* ，*），'process'，pid，'received：'，input 
 end do 
 
 call MPI_TYPE_FREE（newtype，ierror）
 call MPI_FINALIZE（ierror）
  

可以通过更改相应的声明和旧类型来检查整数/整数和整数/实数是否正常工作。整数/实数* 8组合失败，例如，输入-1 2.0产生：

 输入为：-1 2.0000000000000000 
过程0收到：-1 2.0000000000000000 
处理1收到：-1 0.0000000000000000 
处理2收到：-1 0.0000000000000000 
处理3收到：-1 0.0000000000000000 
  

这个带有类似问题的线程表明使用MPI_TYPE_EXTENT是不正确，因为可能会有额外的填充没有考虑到。不幸的是，我还没有能够解决这个问题，并希望有人在这里可以启发我。

thx提前

解决方案

您的基本思想是正确的 - 您创建了结构，但您认为双精度值紧跟在整数值后面存储，并且通常不正确。 Hristo的答案是你链接到C中给出了一个很好的答案。

问题是编译器通常会 align 你的数据结构字段。大多数系统可以读取/写入在内存中对齐的值比执行非对齐访问的速度快得多，如果它们完全可以执行的话。通常情况下，要求是按照元素大小进行对齐;即一个8字节的双精度数字必须与8字节的边界对齐（也就是说，它的第一个字节的地址是零模8），而整数只需要4字节对齐。这几乎肯定意味着在整数和double之间有4个字节的填充。

在很多情况下，您可以哄骗编译器放松这种行为 - 在fortran中，您也可以使用 序列 关键字要求连续存储数据。无论哪种方式，从性能的角度来看（这就是为什么你使用Fortran和MPI，一个假设），这几乎从来都不是正确的做法，但它可以用于与其他外部施加的字节到字节的兼容性数据类型或格式。

考虑到由于性能原因可能产生的填充效果，您可以将对齐和硬编码假设为您的程序;但是这可能不是正确的做法;如果添加其他字段，或将实数的类型更改为4个字节的单精度数字等，则代码将再次出错。最好是使用 MPI_Get_address 来显式地找到位置并自己计算正确的偏移量：

  integer（kind = MPI_Address_kind ）:: startloc，endloc 
 integer :: counts，newtype 
整数，维（2）:: oldtypes，blockcounts，偏移量
 
类型值
 integer :: value1 = 0 
 double precision :: value2 
结束类型
 
类型（值）::输入
 
！... 
 
！设置1个MPI_INTEGER字段：value1 
 call MPI_Get_address（input，startloc，ierror）
 oldtypes（1）= MPI_INTEGER 
 blockcounts（1）= 1 
 call MPI_Get_address（input％value1 ，endloc，ierror）
 offsetsets（1）= endloc  -  startloc 
 
 oldtypes（2）= MPI_DOUBLE_PRECISION 
 blockcounts（2）= 1 
 call MPI_Get_address（input％ value2，endloc，ierror）
 offset（2）= endloc  -  startloc 
 
 if（pid == 0）then 
 print *，'offset is：'，offsetsets 
 endif 
  

请注意，如果您有一组这样的派生类型，在一个项目的最后一个元素和下一个元素的开始之间，你也希望明确地测量它，并设置该类型的整体大小 - 该类型的一个成员的开始与开始下一个 - 与 MPI_Type_create_resized 。

I'm learning about BCASTing data types in Fortran and have a code which takes two values from the terminal and displays them on each process. For the combination value1/value2 of type integer/integer and integer/real this works, however for the combination integer/real*8 it fails.

The code is:

use mpi
implicit none

integer :: ierror, pid, ncpu, root = 0

integer :: counts, newtype, extent
integer, dimension(2) :: oldtypes, blockcounts, offsets

type value
    integer :: value1 = 0
    real*8 :: value2
end type

type (value) input

call MPI_INIT(ierror)
call MPI_COMM_RANK(MPI_COMM_WORLD, pid, ierror)
call MPI_COMM_SIZE(MPI_COMM_WORLD, ncpu, ierror)

! setup of 1 MPI_INTEGER field: value1
offsets(1) = 0
oldtypes(1) = MPI_INTEGER
blockcounts(1) = 1

! setup of 1 MPI_REAL8 field: value2
call MPI_TYPE_EXTENT(MPI_INTEGER, extent, ierror)  !determine offset of MPI_INTEGER
offsets(2) = blockcounts(1)*extent                 !offset is 1 MPI_INTEGER extents
oldtypes(2) = MPI_REAL8
blockcounts(2) = 1

! define struct type and commit
counts = 2 !for MPI_INTEGER + MPI_REAL8
call MPI_TYPE_STRUCT(counts, blockcounts, offsets, & 
                     oldtypes, newtype, ierror)
call MPI_TYPE_COMMIT(newtype, ierror)

do while (input%value1 >= 0)
    if (pid == root) then
        read(*,*) input
        write(*,*) 'input was: ', input
    end if
    call MPI_BCAST(input, 1, newtype, &
                   root, MPI_COMM_WORLD, ierror)
    write(*,*), 'process ', pid, 'received: ', input
end do

call MPI_TYPE_FREE(newtype, ierror)
call MPI_FINALIZE(ierror)

It can be checked that integer/integer and integer/real work fine by changing the corresponding declaration and oldtype. The combination integer/real*8 fails with e.g. inputs -1 2.0 generating:

input was:           -1   2.0000000000000000     
process            0 received:           -1   2.0000000000000000     
process            1 received:           -1   0.0000000000000000     
process            2 received:           -1   0.0000000000000000     
process            3 received:           -1   0.0000000000000000

This thread with a similar issue suggest that using MPI_TYPE_EXTENT is not correct as there might be additional padding which is not taken into account. unfortunately i havent been able to fix the problem and hope someone here can enlighten me.

thx in advance

解决方案

You have the basic idea right - you've created the structure, but you're assuming that the double precision value is stored immediately following the integer value, and that generally isn't correct. Hristo's answer that you link to gives a good answer in C.

The issue is that the compiler will normally align your data structure fields for you. Most systems can read/write values that are aligned in memory much faster than they can perform non-aligned accesses, if they can perform those at all. Typically, the requirement is that the alignment is on element sizes; that is a 8-byte double precision number will have to be aligned to 8-byte boundaries (that is, the address of it's first byte is zero modulo 8) whereas the integer only has to be 4-byte aligned. This almost certainly means that there are 4 bytes of padding between the integer and the double.

In many cases you can cajole the compiler into relaxing this behaviour - in fortran, you can also use the sequence keyword to require that the data be stored contiguously. Either way, from a performance point of view (which is why you're using Fortran and MPI, one assumes) this is almost never the right thing to do, but it can be useful for byte-to-byte compatibility with other externally imposed data types or formats.

Given the likely imposed padding for performance reasons, you could assume the alignment and hardcode that into your program; but that probably isn't the right thing to do, either; if you add other fields, or change the kind of the real number to be a 4-byte single precision number, etc, your code would be wrong again. Best is to use MPI_Get_address to explicitly find the locations and calculate the correct offsets yourself:

integer(kind=MPI_Address_kind) :: startloc, endloc    
integer :: counts, newtype
integer, dimension(2) :: oldtypes, blockcounts, offsets

type value
    integer :: value1 = 0
    double precision :: value2
end type

type (value) :: input

!...    

! setup of 1 MPI_INTEGER field: value1
call MPI_Get_address(input, startloc, ierror)
oldtypes(1) = MPI_INTEGER
blockcounts(1) = 1
call MPI_Get_address(input%value1, endloc, ierror)
offsets(1) = endloc - startloc

oldtypes(2) = MPI_DOUBLE_PRECISION
blockcounts(2) = 1
call MPI_Get_address(input%value2, endloc, ierror)
offsets(2) = endloc - startloc

if (pid == 0) then
    print *,'offsets are: ', offsets
endif

Note that if you had an array of such derived types, to cover the case of padding between the last element of one item and the start of the next, you'd want to explicitly measure that, as well, and set the overall size of the type - the offset between the start of one member of that type and the start of the next - with MPI_Type_create_resized.

这篇关于派生数据类型与MPI的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！