使用MPI-IO编写Fortran格式的文件 [英] Using MPI-IO to write Fortran-formatted files
问题描述
我正在尝试使用OVERFLOW-PLOT3D q文件格式(在此定义: http://overflow.larc.nasa.gov/files/2014/06/Appendix_A.pdf ).对于单个网格，基本上是
I am trying to save a solution using the OVERFLOW-PLOT3D q-file format (defined here: http://overflow.larc.nasa.gov/files/2014/06/Appendix_A.pdf). For a single grid, it is basically,
READ(1) NGRID
READ(1) JD,KD,LD,NQ,NQC
READ(1) REFMACH,ALPHA,REY,TIME,GAMINF,BETA,TINF, &
IGAM,HTINF,HT1,HT2,RGAS1,RGAS2, &
FSMACH,TVREF,DTVREF
READ(1) ((((Q(J,K,L,N),J=1,JD),K=1,KD),L=1,LD),N=1,NQ)
所有变量都是双精度数字，但NGRID，JD，KD，LD，NQ，NQC和IGAM是整数.我需要使用MPI-IO导出解决方案.如果我仅用一个处理器做一个非常简单的示例，以下代码将不起作用，但我不明白为什么.
All of the variables are double precision numbers, excepts for NGRID, JD, KD, LD, NQ, NQC and IGAM which are integers. I need to use MPI-IO to export the solution. If I take a very simple example with a single processor, the following code does not work, but I do not understand why.
call mpi_file_open( mpi_comm_world, fileOut, mpi_mode_wronly + mpi_mode_create, &
mpi_info_null, mpi_fh, ierr )
offset = 0
call mpi_file_seek( mpi_fh, offset, mpi_seek_set, ierr )
call mpi_file_write( mpi_fh, (/NGRID,JD,KD,LD,NQ,NQC/), 6, mpi_integer, mstat, ierr )
call mpi_file_write( mpi_fh, (/REFMACH,ALPHA,REY,TIME,GAMINF,BETA,TINF/), 7, mpi_double_precision, mstat, ierr )
call mpi_file_write( mpi_fh, IGAM, 1, mpi_integer, mstat, ierr )
call mpi_file_write( mpi_fh, (/HTINF,HT1,HT2,RGAS1,RGAS2,FSMACH,TVREF,DTVREF/), 8, mpi_double_precision, mstat, ierr )
call mpi_file_write( mpi_fh, Q, NQ*JD*KD*LD, mpi_double_precision, mstat, ierr )
Tecplot无法识别格式.但是，如果我编写这样的简单非MPI代码:
Tecplot does not recognize the format. However, if I write a simple non-MPI code such as this one:
open(2, file=fileOut, form='unformatted', convert='little_endian')
write(2) NGRID
write(2) JD, KD, LD, NQ, NQC
write(2) REFMACH,ALPHA,REY,TIME,GAMINF,BETA,TINF, &
IGAM,HTINF,HT1,HT2,RGAS1,RGAS2, &
FSMACH,TVREF,DTVREF
write(2) ((((Q(J,K,L,N),J=1,JD),K=1,KD),L=1,LD),N=1,NQ)
一切正常.我的MPI-IO代码有什么问题??非常感谢您的帮助！
everything works just fine. What is wrong with my MPI-IO code?? Thank you very much for your help!
Joachim
NB:我不知道这是否相关，但是如果我在最后的write语句之前添加一个mpi_file_seek(offset)，其偏移量为144.Tecplot同意加载该文件(但无法正确读取数据).这很奇怪，因为普通偏移量应该是7个整数+ 15个实数* 8 = 148个字节...
NB: I do not know if that is relevant, but if I add a mpi_file_seek(offset) just before the final write statement, with offset=144. Tecplot agrees to load the file (but the data is not read correctly). This is strange, because the normal offset should be 7 integers + 15 real*8 = 148 bytes...
由于某种原因，您的方法@Jonathan Dursi似乎不适用于Tecplot.以下代码有什么问题吗?(简化为单个处理器)
Your approach, @Jonathan Dursi, does not seem to work with Tecplot for some reason. Is there anything wrong with the following code? (simplified for a single processor)
call MPI_File_write(fileh, [4, ngrid, 4], 3, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [20, jd, kd, ld, nq, nqc, 20], 7, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [56], 1, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [refmach,alpha,rey,time,gaminf,beta,tinf], 7, MPI_double_precision, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [56], 1, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [4, IGAM, 4], 3, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [64], 1, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [HTINF,HT1,HT2,RGAS1,RGAS2,FSMACH,TVREF,DTVREF], 8, MPI_double_precision, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [64], 1, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [jd*kd*ld*nq*8], 1, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, q, jd*kd*ld*nq, MPI_double_precision, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [jd*kd*ld*nq*8], 1, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
推荐答案
@francescalus是正确的- Fortran顺序未格式化的数据是基于记录的-实际上对很多事情来说真的很不错，但是没有其他东西使用它(即使是Fortran中的MPI-IO，也更像C-该文件只是一大堆未区分的数据字节).
@francescalus is right - Fortran sequential unformatted data is record based - which is actually really nice for a lot of things, but nothing else uses it (even MPI-IO in Fortran, which is more C like - the file is just a big long stream of undifferentiated bytes).
让我们看一下您的写作程序的简化版本:
Let's take a look at a simplified version of your writing program in the question:
program testwrite
integer, parameter:: ngrid=2
integer, parameter:: jd=4, kd=3, ld=2, nq=1, nqc=-1
integer, parameter :: refmach=1, alpha=2, rey=3, time=4, gaminf=5
integer, parameter :: beta=6, tinf=7
integer, dimension(jd,kd,ld,nq) :: q
q = 0
open(2, file='ftest.dat', form='unformatted', convert='little_endian')
write(2) NGRID
write(2) JD, KD, LD, NQ, NQC
write(2) REFMACH,ALPHA,REY,TIME,GAMINF,BETA,TINF
write(2) ((((Q(J,K,L,N),J=1,JD),K=1,KD),L=1,LD),N=1,NQ)
close(2)
end program testwrite
运行此代码，并使用 od 查看生成的二进制文件(为了清楚起见，我将所有内容都设为整数):
Running this, and taking a look at the resulting binary file with od (I've made everything an integer for clarity in looking at the binary file):
$ gfortran -o fwrite fwrite.f90
$ ./fwrite
$ od --format "d" ftest.dat
0000000 4 2 4 20
0000020 4 3 2 1
0000040 -1 20 28 1
0000060 2 3 4 5
0000100 6 7 28 96
0000120 0 0 0 0
*
0000260 96
0000264
例如，我们在开头看到ngrid(2)整数，以4/4预定–记录的大小(以字节为单位).然后，以20/20预定，我们看到5个整数(5 * 4字节)4,3,2,1，-1-jd，kd，ld，nq，nqc.到最后，我们看到一串零被96表示(= 4字节/整数* 4 * 3 * 2 * 1)，代表q.(请注意，没有标准来定义这种行为，但是我不知道任何主要的Fortran编译器都无法做到这一点；但是，当记录变得比4字节整数所描述的大时，行为就开始了.不同.
We see, for instance, the ngrid (2) integer there at the start, bookended by 4/4 - the size of the record in bytes. Then, bookended by 20/20, we see the 5 integers (5*4 bytes) 4,3,2,1,-1 -- jd, kd, ld, nq, nqc. Towards the end, we see a bunch of zeros bookended by 96 (= 4bytes/integer *4*3*2*1) which represents q. (Note that there's no standard that defines this behaviour, but I'm unaware of any major Fortran compiler that doesn't do it this way; however, when records get larger than can be described by a 4-byte integer, behaviour starts to differ.
我们可以使用以下简单程序来测试数据文件:
We can use the following simple program to test the data file:
program testread
implicit none
integer :: ngrid
integer :: jd, kd, ld, nq, nqc
integer :: refmach, alpha, rey, time, gaminf
integer :: beta, tinf
integer :: j, k, l, n
integer, allocatable, dimension(:,:,:,:) :: q
character(len=64) :: filename
if (command_argument_count() < 1) then
print *,'Usage: read [filename]'
else
call get_command_argument(1, filename)
open(2, file=trim(filename), form='unformatted', convert='little_endian')
read(2) NGRID
read(2) JD, KD, LD, NQ, NQC
read(2) REFMACH,ALPHA,REY,TIME,GAMINF,BETA,TINF
allocate(q(jd, kd, ld, nq))
read(2) ((((Q(J,K,L,N),J=1,JD),K=1,KD),L=1,LD),N=1,NQ)
close(2)
print *, 'Ngrid = ', ngrid
print *, 'jd, kd, ld, nq, nqc = ', jd, kd, ld, nq, nqc
print *, 'q: min/mean/max = ', minval(q), sum(q)/size(q), maxval(q)
deallocate(q)
endif
end program testread
跑步并给予
$ ./fread ftest.dat
Ngrid = 2
jd, kd, ld, nq, nqc = 4 3 2 1 -1
q: min/mean/max = 0 0 0
很简单.
因此，这种行为在MPI-IO中非常容易模仿.这里实际上有三部分-标头Q(我假设要分发)(带有MPI子数组)和页脚(仅是数组的书挡).
So this behaviour is pretty easy to mimic in MPI-IO. There's really three parts here - the header, Q, which I assume to be distributed (with, say, MPI subarrays), and the footer (which is just the bookend to the array).
因此，让我们看一下Fortran中的MPI-IO程序，该程序将执行相同的操作:
So let's take a look at an MPI-IO program in Fortran that would do the same thing:
program mpiwrite
use mpi
implicit none
integer, parameter:: ngrid=2
integer, parameter:: jd=3, kd=3, ld=3, nlocq=3, nqc=-1
integer :: nq
integer, parameter :: refmach=1, alpha=2, rey=3, time=4, gaminf=5
integer, parameter :: beta=6, tinf=7
integer, dimension(jd,kd,ld,nlocq) :: q
integer :: intsize
integer :: subarray
integer :: fileh
integer(kind=MPI_Offset_kind) :: offset
integer :: comsize, rank, ierr
call MPI_Init(ierr)
call MPI_Comm_size(MPI_COMM_WORLD, comsize, ierr)
call MPI_Comm_rank(MPI_COMM_WORLD, rank, ierr)
nq = nlocq * comsize
q = rank
! create a subarray; each processor gets its own q-slice of the
! global array
call MPI_Type_create_subarray (4, [jd, kd, ld, nq], [jd, kd, ld, nlocq], &
[0, 0, 0, nlocq*rank], &
MPI_ORDER_FORTRAN, MPI_INTEGER, subarray, ierr)
call MPI_Type_commit(subarray, ierr)
call MPI_File_open(MPI_COMM_WORLD, 'mpi.dat', &
MPI_MODE_WRONLY + MPI_MODE_CREATE, &
MPI_INFO_NULL, fileh, ierr )
! the header size is:
! 1 field of 1 integer ( = 4*(1 + 1 + 1) = 12 bytes )
! +1 field of 5 integers( = 4*(1 + 5 + 1) = 28 bytes )
! +1 field of 7 integers( = 4*(1 + 7 + 1) = 36 bytes )
! +first bookend of array size = 4 bytes
offset = 12 + 28 + 36 + 4
! rank 1 writes the header and footer
if (rank == 0) then
call MPI_File_write(fileh, [4, ngrid, 4], 3, MPI_INTEGER, &
MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [20, jd, kd, ld, nq, nqc, 20], 7, MPI_INTEGER, &
MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, &
[28, refmach, alpha, rey, time, gaminf, beta, tinf, 28],&
9, MPI_INTEGER, MPI_STATUS_IGNORE, ierr)
call MPI_File_write(fileh, [jd*kd*ld*nq*4], 1, MPI_INTEGER, &
MPI_STATUS_IGNORE, ierr)
call MPI_File_seek(fileh, offset+jd*kd*ld*nq*4, MPI_SEEK_CUR, ierr)
call MPI_File_write(fileh, [jd*kd*ld*nq*4], 1, MPI_INTEGER, &
MPI_STATUS_IGNORE, ierr)
endif
! now everyone dumps their part of the array
call MPI_File_set_view(fileh, offset, MPI_INTEGER, subarray, &
'native', MPI_INFO_NULL, ierr)
call MPI_File_write_all(fileh, q, jd*kd*ld*nlocq, MPI_INTEGER, &
MPI_STATUS_IGNORE, ierr)
call MPI_File_close(fileh, ierr)
CALL MPI_Finalize(ierr)
end program mpiwrite
在此程序中，进程0负责写入标头和记录字段.首先，写入三个标题记录，每个记录由记录长度(以字节为单位)进行预订；然后为大Q数组编写两个书挡.
In this program, process 0 is responsible for writing the header and the record fields. It starts off by writing the three header records, each bookended by the record lengths in bytes; and then it writes the two bookends for the big Q array.
然后，每个等级将文件视图设置为首先跳过标头，然后仅描述其在全局数组中的部分(此处仅用其等级号填充)，并写出其本地数据.这些都是不重叠的数据.
Then, each rank sets the file view to first skip the header and then describe just its piece of the global array (filled here just with its rank number), and writes out its local data. These will all be non-overlapping pieces of data.
所以让我们尝试以下几种不同的大小:
So let's try this with a couple of different sizes:
$ mpif90 -o mpifwrite mpifwrite.f90
$ mpirun -np 1 ./mpifwrite
$ ./fread mpi.dat
Ngrid = 2
jd, kd, ld, nq, nqc = 3 3 3 3 -1
q: min/mean/max = 0 0 0
$ od --format="d" mpi.dat
0000000 4 2 4 20
0000020 3 3 3 3
0000040 -1 20 28 1
0000060 2 3 4 5
0000100 6 7 28 324
0000120 0 0 0 0
*
0000740 0 324
0000750
$ mpirun -np 3 ./mpifwrite
$ ./fread mpi.dat
Ngrid = 2
jd, kd, ld, nq, nqc = 3 3 3 9 -1
q: min/mean/max = 0 1 2
$ od --format="d" mpi.dat
0000000 4 2 4 20
0000020 3 3 3 9
0000040 -1 20 28 1
0000060 2 3 4 5
0000100 6 7 28 972
0000120 0 0 0 0
*
0000620 0 1 1 1
0000640 1 1 1 1
*
0001320 1 1 2 2
0001340 2 2 2 2
*
0002020 2 2 2 0
0002040 0 0 0 0
*
0002140 0 0 0 972
0002160
这是我们期望的输出.将事物扩展为多种数据类型或多种网格相对简单.
which is the output we expect. Extending things to multiple datatypes or multiple grids is relatively straightforward.
这篇关于使用MPI-IO编写Fortran格式的文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
