在Fortran中写入磁盘的效率 [英] Efficiency in writing to disk in Fortran

问题描述

我正在尝试在fortran中将一堆大型矩阵写入磁盘。
矩阵为 V ， C 和 d 。都具有（2，n1，n2，n3，n4，n5）的大小。

I am trying to write a bunch of large matrices in fortran to disk. Matrices are V, C and d. All have size (2 , n1, n2, n3, n4, n5).

这些都是大矩阵。而且fortran大约需要3个小时来编写它们。

These are large matrices. And fortran is taking around 3 hours to write them.

do ind1=1,n1
do ind2=1,n2
do ind3=1,n3
do ind4=1,n4

     write(filename,'(a,i0,a,i0,a,i0,a,i0,a,i0,a)')'PF',t,'_',ind1,'_',ind2,'_',ind3,'_',ind4,'.txt'

     OPEN(UNIT=25,FILE=filename,STATUS='replace',ACTION='write')
     do ind5=1,n5
      WRITE(25,*) c(2,ind1,ind2,ind3, ind4,ind5)
     end do
     do ind5=1,n5
      WRITE(25,*) v(2,ind1,ind2,ind3, ind4,ind5)
     end do
     do ind5=1,n5
      WRITE(25,*) d(1,ind1,ind2,ind3, ind4,ind5)
     end do
     CLOSE(UNIT=25)

end do
end do
end do
end do

有什么聪明的方法可以更快地做到这一点？

Any smart way of doing this faster?

推荐答案

首先让我说我同意这些意见，最好的方法是将其转移到HDF5或NetCDF之类的工具上，这应该既给您带来数据的可移植性又给您带来了高性能

Let me start by saying that I agree with the comments, the best way would be to move to something like HDF5 or NetCDF which should give you both the portability of the data and also high performance.

这就是说，如果您要坚持使用简单的Fortran I / O，通常最好将I / O事务数保持在最低水平。我怀疑您的所有打开都不是一个好主意，我会将其全部粘贴在一个文件中。如果可以，那么您将拥有如下所示的3条路线，并可以在我的笔记本电脑（公认的是SSD）上运行。

That said if you want to stick with simple Fortran I/O it's usually a good idea to keep the number of I/O transactions to a minimum. I suspect all your opens are not a good idea, I would stick it all in one file. If that's OK you then have 3 routes as shown below and run on my laptop (which admittedly has a SSD). Anyway it should give you some ideas.

ian-admin@agon ~/test $ cat io.f90
Program test_io

  Implicit None

  Integer, Parameter :: wp = Selected_real_kind( 12, 70 )

  Integer, Parameter :: n1 = 201
  Integer, Parameter :: n2 = 50
  Integer, Parameter :: n3 = 2
  Integer, Parameter :: n4 = 2
  Integer, Parameter :: n5 = 21

  Real( wp ), Dimension( 1:2, 1:n1, 1:n2, 1:n3, 1:n4, 1:n5 ) :: stuff

  Integer :: start, finish, rate

  Integer :: i1, i2, i3, i4, i5

  Call Random_Number( stuff )

  Call System_Clock( start, rate )
  Do i1 = 1, n1
     Do i2 = 1, n2
        Do i3 = 1, n3
           Do i4 = 1, n4
              Do i5 = 1, n5
                 Write( 10, * ) stuff( 1, i1, i2, i3, i4, i5 )
              End Do
           End Do
        End Do
     End Do
  End Do
  Call System_Clock( finish, rate )
  Write( *, * ) 'Loops : ', Real( finish - start ) / Real( rate )

  Call System_Clock( start, rate )
  Write( 11, * ) stuff( 1, :, :, :, :, : )
  Call System_Clock( finish, rate )
  Write( *, * ) 'Array : ', Real( finish - start ) / Real( rate )

  Call System_Clock( start, rate )
  Write( 12 ) stuff( 1, :, :, :, :, : )
  Call System_Clock( finish, rate )
  Write( *, * ) 'Unform: ', Real( finish - start ) / Real( rate )

End Program test_io
ian-admin@agon ~/test $ gfortran -O -Wall -Wextra -std=f2003 io.f90
ian-admin@agon ~/test $ ./a.out
 Loops :    2.28500009    
 Array :    1.80200005    
 Unform:    5.79999983E-02
ian-admin@agon ~/test $ 

这篇关于在Fortran中写入磁盘的效率的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！