写入文件，MPI [英] Writing to files with MPI

问题描述

我在写一个文件，如下所示。顺序并不一定关系（虽然这将是很好，如果我能得到它被K下令，它可以在串行code固有）

I'm writing to a file as follows. The order does not necessarily matter (though it would be nice if I could get it ordered by K, as would be inherently in serial code)

            CALL MPI_BARRIER(MPI_COMM_WORLD, IERR)
            OPEN(EIGENVALUES_UP_IO, FILE=EIGENVALUES_UP_PATH, ACCESS='APPEND')
            WRITE(EIGENVALUES_UP_IO, *) K * 0.0001_DP * PI, (EIGENVALUES(J), J = 1, ATOM_COUNT)
            CLOSE(EIGENVALUES_UP_IO)

我知道这可能是最糟糕的选择。

I'm aware this is likely to be the worst option.

我已经采取了看MPI_FILE_WRITE_AT等等，但我不知道他们（直接）取表单数据，我有吗？

I've taken a look at MPI_FILE_WRITE_AT etc. but I'm not sure they (directly) take data in the form that I have?

该文件必须是相同的格式，因为这，它出来作为每K A线，ATOM_COUNT + 1列。的值是实（8）

The file must be in the same format as this, which comes out as a line per K, with ATOM_COUNT + 1 columns. The values are REAL(8)

我已经猎杀了个遍，并不能找到实现这一目标的任何简单引用。任何帮助吗？ ：）

I've hunted over and over, and can't find any simple references on achieving this. Any help? :)

类似code在C（假设它是基本相同的FORTRAN）是一样有用

Similar code in C (assuming it's basically the same as FORTRAN) is just as useful

谢谢！

推荐答案

所以，确定正确的IO策略取决于很多因素。如果你是刚刚发回的特征值的屈指可数，而你坚持写出ASCII，你可能是最好关闭只是将所有数据传回处理0写。这是的不的通常是一个成功的策略，因为它显然不能扩展;但是，如果数据量是非常小的，它很可能是比涉及试图写入到一个共享文件（其是，再次，更难用ASCII码）的争用更好。

So determining the right IO strategy depends on a lot of factors. If you are just sending back a handful of eigenvalues, and you're stuck writing out ASCII, you might be best off just sending all the data back to process 0 to write. This is not normally a winning strategy, as it obviously doesn't scale; but if the amount of data is very small, it could well be better than the contention involved in trying to write out to a shared file (which is, again, harder with ASCII).

有些code是低于它的数据量将拖带回到PROC 0，假设每个人都有相同的数据量。

Some code is below which will schlep the amount of data back to proc 0, assuming everyone has the same amount of data.

另一种方法也只是对大家有写出自己的KS和特征值，然后作为一个后处理步骤，一旦该计划完成后，猫它们放在一起。这避免了MPI一步，（用正确的文件系统）可以扩展相当的方式，容易;不管是更好是相当容易测试，并且将取决于数据量，处理器数量，和底层文件系统

Another approach would just be to have everyone write out their own ks and eigenvalues, and then as a postprocessing step once the program is finished, cat them all together. That avoids the MPI step, and (with the right filesystem) can scale up quite a ways, and is easy; whether that's better is fairly easily testable, and will depend on the amount of data, number of processors, and underlying file system.

   program testio
    use mpi
    implicit none

    integer, parameter :: atom_count = 5
    integer, parameter :: kpertask   = 2
    integer, parameter :: fileunit   = 7
    integer, parameter :: io_master  = 0
    double precision, parameter :: pi = 3.14159

    integer :: totalk
    integer :: ierr
    integer :: rank, nprocs

    integer :: handle
    integer(kind=MPI_OFFSET_KIND) :: offset
    integer :: filetype

    integer :: j,k
    double precision, dimension(atom_count, kpertask) :: eigenvalues
    double precision, dimension(kpertask) :: ks

    double precision, allocatable, dimension(:,:):: alleigenvals
    double precision, allocatable, dimension(:)  :: allks

    call MPI_INIT(ierr)
    call MPI_COMM_SIZE(MPI_COMM_WORLD, nprocs, ierr)
    call MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr)

    totalk   = nprocs*kpertask

    !! setup test data

    do k=1,kpertask
        ks(k) = (rank*kpertask+k)*1.d-4*PI
        do j=1,atom_count
            eigenvalues(j,k) = rank*100+j
        enddo
    enddo

    !! Everyone sends proc 0 their data

    if (rank == 0) then
        allocate(allks(totalk))
        allocate(alleigenvals(atom_count, totalk))
    endif

    call MPI_GATHER(ks, kpertask, MPI_DOUBLE_PRECISION,    &
                    allks, kpertask, MPI_DOUBLE_PRECISION, &
                    io_master, MPI_COMM_WORLD, ierr)

    call MPI_GATHER(eigenvalues, kpertask*atom_count, MPI_DOUBLE_PRECISION,  &
                    alleigenvals, kpertask*atom_count, MPI_DOUBLE_PRECISION, &
                    io_master, MPI_COMM_WORLD, ierr)

    if (rank == 0) then 
        open(unit=fileunit, file='output.txt')
        do k=1,totalk
            WRITE(fileunit, *) allks(k), (alleigenvals(j,k), j = 1, atom_count)
        enddo
        close(unit=fileunit)

        deallocate(allks)
        deallocate(alleigenvals)
    endif
    call MPI_FINALIZE(ierr)
end program testio

这篇关于写入文件，MPI的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！